Abstract
Objectives: Effects of calibration on interrater agreement in evaluating magnetic resonance (MR) images of the temporomandibular joint (TMJ) have already been examined. The objectives of the present study were to assess to what extent the quality of MR images of the TMJ influences interrater agreement and to evaluate interrater agreement with respect to image quality assessment.
Methods: Two non-calibrated medical radiologists and two general dentists evaluated sagittal images of 100 TMJs for both a rating of the image quality and the performance of five diagnostic tasks. The agreement between these raters with respect to the diagnoses was evaluated. Additionally, two additional raters, calibrated during a 5 h training including the evaluation of 70 MR images, also evaluated the diagnostic aspects and the image quality, on the basis of objective criteria. The agreement between the subjective diagnoses of the non-calibrated raters and the objective diagnoses of the calibrated raters was evaluated. Afterwards, the subjective and the objective quality assessments were compared using kappa statistics.
Results: When good quality images were evaluated, higher kappa values were obtained for all diagnostic categories by the non-calibrated raters (mean Δk for making diagnoses >0.1). This finding was confirmed by the value obtained for the agreement between the non-calibrated and the calibrated raters. The non-calibrated raters were in good agreement (k=0.67, standard error ±0.09) with the calibrated raters for assessment of image quality.
Conclusion: The present study shows that it is possible even without calibration to obtain a better interrater agreement when higher quality MR images of the TMJ are evaluated.
References
1. Liedberg J, Panmekiate A, Petersson A, Rohlin M. Evidence-based evaluation of three imaging methods for the temporomandibular disc. Dentomaxillofac Radiol 1996; 25: 234–241.
2. Nakasato T, Ehara S, Tamakawa Y, Kobayakawa T. MRI and arthrography in the evaluation of TMJ disorders. Nippon Igaku Hoshasen Gakki Zasshi 1991; 51: 912–922.
3. Donlon WC, Moon KL. Comparison of magnetic resonance imaging, arthrotomography and clinical and surgical findings in temporomandibular joint internal derangements. Oral Surg Oral Med Oral Pathol 1987; 64: 2–5.
4. Katzberg RW. Temporomandibular joint imaging. Radiology 1989; 170: 297–307.
5. Crowley C, Wilkinson T, Piehslingher E, Wilson D, Czerny C. Correlations between anatomic and MRI sections of human cadaver temporomandibular joint in the coronal and sagittal planes. J Orofac Pain 1996; 10: 199–216.
6. Watson PJ, Hall LD, Malcolm A, Tyler JA. Degenerative joint disease in the guinea pig. Use of magnetic resonance imaging to monitor progession of bone pathology. Arthritis Rheum 1996; 39: 1327–1337.
7. Nebbe B, Brooks SL, Hatcher D, Hollender LG, Prasad NG, Major PW. Magnetic resonance imaging of the temporomandibular joint: interobserver agreement in subjective classification of disk status. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2000; 90: 102–107.
8. Tasaki MM, Westesson PL, Raubertas RF. Observer variation in interpretation of magnetic resonance images of the temporomandibular joint. Oral Surg Oral Med Oral Pathol 1993; 76: 231–234.
9. Roditi GH, Duncan KA, Needham G, Redpath TW. Temporomandibular joint MRI: a 2-D gradient-echo technique. Clin Radiol 1997; 52: 441–444.
10. Orsini MG, Kuboki T, Terada S, Matsuka Y, Yamashita A, Clark GT. Diagnostic value of 4 criteria to interpret temporomandibular joint normal disk position on magnetic resonance images. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1998; 86: 489–497.
11. Rammelsberg P, Pospiech P, Jäger L, Pho Duc JM, Böhm AO, Gernet W. Variability of disk position in asymptomatic volunteers and patients with internal derangements of the TMJ. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1997; 83: 393–399.
12. Tasaki MM, Westesson PL, Isberg AM, Ren YF, Tallents RH. Classification and prevalence of temporomandibular joint disk displacement in patients and symptom-free volunteers. Am J Orthod Dentofac Orthop 1996; 109: 249–262.
13. Murakami A, Takahashi A, Nishiyama M, Fujishita M, Fuchihata H. Magnetic resonance evaluation of the temporomandibular joint disc position and configuration. Dentomaxillofac Radiol 1993; 22: 205–207.
14. Heffez L, Jordan S. A classification of temporomandibular joint disk morphology. Oral Surg Oral Med Oral Pathol 1989; 67: 11–19.
15. Laskin DM. Diagnosis of pathology of the temporomandibular joint. Clinical and imaging perspectives. Radiol Clin North Am 1993; 31: 135–147.
16. DeLeeuw R, Boering G, Stegenga B, deBont LGM. Radiographic signs of temporomandibular joint osteoarthrosis and internal derangement 30 years after nonsurgical treatment. Oral Surg Oral Med Oral Pathol 1995; 79: 382–392.
17. Kundel HL, Polansky M. Measurement of observer agreement. Radiology 2003; 228: 303–308.
18. Landis JR. The measurement of observer agreement for categorial data. Biometrics 1977; 33: 159–174.
19. Barclay P, Hollender LG, Maravilla KR, Truelove EL. Comparison of clinical and magnetic resonance imaging diagnosis in patients with disk displacement in the temporomandibular joint. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1999; 88: 37–43.
20. Paesani D, Westesson PL, Hatala MP, Tallents RH, Brooks SL. Accuracy of clinical diagnosis for TMJ internal derangement and arthrosis. Oral Surg Oral Med Oral Pathol 1992; 73: 360–363.
21. Orsini MG, Terada S, Kuboki T, Matsuka Y, Yamashita A. The influence of observer calibration in temporomadibular joint magnetic resonance imaging diagnosis. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1997; 84: 82–87.
22. Dempsey J, Brooks J, King M. Quality Assurance in MRI. Radiol Management 1991; 13: 38–42.
23. Drace JE, Enzmann DR. Defining the normal temporomandibular joint: closed-, parially open-, and open-mouth MR imaging of asymptomatic subjects. Radiology 1990; 177: 67–71.
