Inter-observer agreement and inter-imaging accuracy in sinus computed tomography scans and panoramic radiography on signs of local dentoalveolar bone loss and periapical radiolucency

Anna Julkunen-Iivari; Satu Apajalahti; Riste Saat; Anna Maria Heikkinen; Jukka H Meurman; Sanna Toppila-Salmi

doi:10.4103/jomr.jomr_2_21

ORIGINAL ARTICLE

Year : 2021 | Volume : 9 | Issue : 1 | Page : 1-5

Inter-observer agreement and inter-imaging accuracy in sinus computed tomography scans and panoramic radiography on signs of local dentoalveolar bone loss and periapical radiolucency

Anna Julkunen-Iivari¹, Satu Apajalahti², Riste Saat², Anna Maria Heikkinen³, Jukka H Meurman³, Sanna Toppila-Salmi⁴
¹ Transplantation laboratory, Haartman Institute, University of Helsinki; Department of Oral and Maxillofacial Diseases, Helsinki, Finland
² Medical Imaging Center, Department of Radiology, Helsinki, Finland
³ Department of Oral and Maxillofacial Diseases, University of Helsinki, Helsinki, Finland
⁴ Transplantation laboratory, Haartman Institute; Department of Allergy, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

Date of Submission	13-Jan-2021
Date of Decision	19-Feb-2021
Date of Acceptance	20-Feb-2021
Date of Web Publication	20-May-2021

Correspondence Address:
Anna Julkunen-Iivari
Haartman Institute, P. O. Box: 21 (Haartmaninkatu 3), University of Helsinki, Helsinki 00014
Finland

Source of Support: None, Conflict of Interest: None

Check

DOI: 10.4103/jomr.jomr_2_21

Abstract

Background: There can be variability between observer's opinions when reading radiographic images. Aims: The aim of this prospective study was to evaluate inter-observer agreement and inter-imaging accuracy regarding radiological signs of local dentoalveolar bone loss and apical radiolucency on sinonasal computed tomography (CT) images and Panoramic tomography (PTG) images. Materials and Methods: Imaging data of 59 patients with oral maxillofacial or sinonasal diseases were used. Radiological signs of local dentoalveolar bone loss and apical radiolucency were analyzed blinded. Using Cohen's kappa statistic the reproducibility of the findings between the oral radiologist and three other observers were compared. Inter-imaging accuracy was calculated, based on the records by the oral radiologist. Results: Inter-observer agreement between the oral radiologist and the three different professionals was from poor (kappa - 0.054) to moderate (kappa 0.455) with PTG and CT scans. Inter-imaging accuracy was moderate (0.565) to very good (0.908). Conclusions: This study showed a large diagnostic variation between the oral radiologist and the three professional observers in recording signs of local dentoalveolar bone loss and periapical radiolucency. This finding indicates that before final treatment plan of the patient, multi-professional consultation is recommended. PTG may not be as reliable imaging method as CT.

Keywords: Alveolar bone loss, panoramic radiography, periapical periodontitis, periodontitis, X-rays

How to cite this article:
Julkunen-Iivari A, Apajalahti S, Saat R, Heikkinen AM, Meurman JH, Toppila-Salmi S. Inter-observer agreement and inter-imaging accuracy in sinus computed tomography scans and panoramic radiography on signs of local dentoalveolar bone loss and periapical radiolucency. J Oral Maxillofac Radiol 2021;9:1-5

How to cite this URL:
Julkunen-Iivari A, Apajalahti S, Saat R, Heikkinen AM, Meurman JH, Toppila-Salmi S. Inter-observer agreement and inter-imaging accuracy in sinus computed tomography scans and panoramic radiography on signs of local dentoalveolar bone loss and periapical radiolucency. J Oral Maxillofac Radiol [serial online] 2021 [cited 2023 Mar 25];9:1-5. Available from: https://www.joomr.org/text.asp?2021/9/1/1/316482

Introduction

Periodontitis and apical periodontitis may be asymptomatic clinical problems need to be treated. They both associate with general health problems such as cardiovascular diseases.^[1] Periodontitis manifests on radiological images as diffuse bone margin, horizontal, or vertical alveolar bone loss.^[2],[3] Apical periodontitis appears on a radiograph as periapical bone loss, radiolucency around the dental root.^[4]

In upper jaw, apical periodontitis can lead to chronic rhinosinusitis (CRS).^[5] Odontogenic origin should be considered especially when CRS is unilateral or mucosal thickening in maxillary sinus is associated with an infected tooth.^[5],[6] Odontogenic sinusitis is most often caused by apical periodontitis of the first or second upper molar.^[6] Approximately 10% to 12% of maxillary sinusitis has dental etiology.^[6],[7] Obayashi et al. found that 65.4% of the maxillary sinusitis patients had changes in their alveolar cortical plates around the infected teeth. They also discovered that 71.3% of the patients with dental infection had changes in maxillary sinuses.^[8]

There can be variability between observer's opinions when reading radiographic images especially if the observers have different background and experience.^[9],[10] Different diagnoses can then be made from same scans, which may lead to different and even erroneous treatment plans.

The aim of this study was to see how the opinions of four observers with different expertise and experience differences with an oral radiologist and is PTG scans as reliable as computed tomography (CT) scans for diagnosing local horizontal bone loss, vertical bone loss, and periapical radiolucency. Our hypothesis was that the inter-observer agreement is mostly good and PTG scans are as reliable for diagnosing sings of local alveolar bone loss and periapical radiolucency as CT scans. Intra-observer agreement was recorded for the validation of reliability.

Materials and Methods

This study was carried out at the Departments of Oral and Maxillofacial Diseases and Otorhinolaryngology, Helsinki University Hospital (HUH). We collected retrospectively a random sample of 70 patients from the HUH database. Data included patients with oral maxillofacial or sinonasal diseases. Inclusion criterion was that the patient had undergone both the sinus CT scan and PTG scan within 6 months. Patients with head and neck tumors, acute traumas, toothless jaws, and age under 17 years were excluded. The final sample included 59 patients, 24 (40.7%) males and 35 (59.3%) females. Patients' mean age was 35 years (range 17–75 years) [Table 1]. They had undergone sinus CT and PTG scans within 6 months due to medical reasons.

Table 1: Background factors from 59 subjects

Click here to view

CT-scans were performed with a 16-row CT scanner Bright Speed^TM (GE Healthcare Systems, U.S.) or Somatom Definition Edge^TM (Siemens, Erlangen, Germany). The data was reconstructed into 1.0 mm images in axial, sagittal and coronal plains using bone kernel. In addition, radiological examination included PTG images with OP-300^TM or OP-200^TM (Instrumentarium Imaging, Finland) or Promax^TM (Planmeca, Finland).

The Ethics Committee of the Hospital District of Helsinki and Uusimaa provided an approval that there was no need for written informed consent of the patients of this retrospective study (Rhinosinusitis Risk Prediction Study; no. 31/13/03/00/15).

The data that support the findings of this study is available from the corresponding author on reasonable request.

Image analysis

Four independent observers, blinded to the patient history, recorded the CT and PTG scans. The focus was on comparing the agreement between oral radiologist and three different observers in evaluating radiological pathology of dental alveolar bone and periapical radiolucency. The observers were a board-certified oral radiologist, a head and neck radiologist, an ear-, nose-, throat- (ENT) and rhino specialist, and a 3^rd-year dental student.

All four observers evaluated the scans in similar conditions at the HUH Radiology Department diagnostic monitors and filled out the predesigned forms. Time interval between the evaluation of CT and PTG scans was at least 7 days for each observer. Before starting recording the scans, all potential diagnostic choices were discussed by the observers. A pilot study with 10 CT and PTG scans ensured that all observers were familiar with filling out the forms used for evaluation.

The dentoalveolar bone of the upper jaw in premolar and molar areas was studied from the radiographs, including three different signs for local dentoalveolar bone loss (vertical bone loss, horizontal bone loss >3 mm) and periapical radiolucency on both the CT and PTG scans.[Figure 1]. On PTG scans additional sing of local diffuse bone margin was estimated. Right and left sides of the scans were separately evaluated, altogether 118 regions from the 59 patients. Findings were recorded as “Yes–No–Not Detectable.” The ENT -and rhino specialist also evaluated Lund-MacKay scores from the CT scans [Table 1].^[11] “Not detectable” was not an option in the items on Lund-MacKay scoring.

Figure 1: Radiological signs of dentoalveolar bone loss and periapical radiolucency. *On PTG scans additional sing of local diffuse bone margin was estimated

Click here to view

Statistical analysis

Statistical analyses were carried out with the SPSS Base 15.0 Statistical Software Package (SPSS Inc., Chicago, IL, USA). Cohen's kappa was used to compare the degree of inter-and intra-observer, and inter-imaging agreements on the scans, respectively. The established interpretation of kappa-value is classified into six subgroups: Poor <0.2, Fair 0.21–0.4, Moderate 0.41–0.6, Good 0.61–0.8, and Very Good 0.81–1.0.^[12] Associations between observers (inter-observer agreement), and between CT and PTG scans (inter-imaging agreement) and intra-observer agreement were assessed by the Fisher's exact test, respectively. Two-tailed P < 0.05 was considered statistically significant.

Results

Inter-observer agreement

The estimation by the oral radiologist served as a reference standard. Results of the inter-observer agreement are given in [Table 2].

Table 2: Inter-observer agreement between the oral radiologist and the three other observers

Click here to view

The inter-observer agreement between the oral radiologist and the head and neck radiologist in PTG scans was mostly moderate. Agreement in the CT scans was mostly fair. The greatest disagreements in PTG and CT scans were in recording the vertical bone loss [Table 2].

When evaluating the agreement between the oral radiologist and the dental student in PTG scans, the inter-observer agreement was either poor or fair. The greatest disagreement was in recording the periapical lesion. Agreement in the CT scans was mostly fair. The greatest disagreement was in recording vertical bone loss [Table 2].

When evaluating the agreement between the oral radiologist and the ENT -and rhino specialist in PTG and CT scans, the inter-observer agreement was mostly poor. The greatest disagreements in both scans were on recording vertical bone loss [Table 2].

Inter-imaging accuracy

We studied inter-imaging accuracy between sinus CT and PTG scans based on the records by the oral radiologist. Correlation in vertical bone loss was good, for horizontal bone loss very good and for periapical radiolucency moderate [Table 3].

Table 3: Inter-imaging accuracy between panoramic radiography and sinonasal computed tomography scans

Click here to view

Intra-observer agreement

Intra observer agreement was studied from the records by the oral radiologist for the validation of reliability. Intra observer agreement was from good (0.770) to very good (1.000) [Table 4].

Table 4: Intra-observer agreement by the oral radiologist

Click here to view

Discussion

The aim of this study was to see how the opinions of four observers with different expertise and experience differences with an oral radiologist and is PTG scans as reliable as CT scans for diagnosing local horizontal bone loss, vertical bone loss and periapical radiolucency. Our study demonstrated considerable variation in the evaluation of radiological periodontal findings between the observers.

Our study also showed that PTG is as reliable as CT for diagnosing local horizontal bone loss but not for vertical bone loss or periapical radiolucency. PTG scan is used for overview radiological periodontal situation especially to evaluate level of alveolar bone margin in whole jaw.

Diagnosing periodontal diseases is based on patients' symptoms and clinical signs, and supported by the evidence from radiographs. The color, shape, texture, and bleeding on probing of the gingival tissues are evaluated when making the diagnosis of periodontal diseases. Occasionally, histopathology, microbiology or serology are also needed for the diagnosis.^[13],[14] Periodontitis is mostly asymptomatic. Periapical periodontitis, however, may cause painful symptoms to a patient like in cases with acute periapical periodontitis but it may also be asymptomatic like chronic apical periodontitis.^[4] Treatment plan is made based on the clinical and radiological findings.^[4],[13]

There can be variation in inter-observer agreements. Shahzad et al. examined inter-observer agreement between four viewers; dental implantologist, oral surgeon and two dental radiologists. The aim of their research was to study radiological interpretation of the variable views of the mental foramen in panoramic radiography. Observes analyzed 130 panoramic radiographs of adult patients using the classification of mental foramins. The agreement varied from 0.34 to 0.49 according to the Kappa statistics. Their conclusion was that there were significant differences in analyzes among four examiners.^[10]

Fuhrmann et al. compared objective measurements between high-resolution CT scans and intraoral radiographs regarding the evaluation of vertical and horizontal alveolar bone loss. When horizontal bone loss was evaluated, they found no significant difference. While vertical bone loss was evaluated, 60% of the infra-alveolar bony defects could be identified from intraoral radiographs and vertical depth was underestimated by an average of 2.2 mm. In comparison, 100% infra-alveolar defects was identified from the CT scans and vertical depth was underestimated by a mean of 0.2 mm.^[15] Tang et al. suggested that panoramic radiographs are as reliable as cone beam computed tomography when evaluating horizontal and vertical distances in alveolar bone, but not as reliable to evaluate alveolar bone volumes in patients with periodontal diseases.^[16]

In Finland university studies in dental sciences last 5½ years, a 3^rd year dental student can practice dentistry under supervision and make dental treatment plans with clinical and radiological diagnoses. University trains bachelors to evaluate PTG scans well but evaluation of CT scans is trained in higher grades. An ENT-and rhino specialization takes 5 years, of which about 3½ years focus on ENT. During specialization and clinical careers, ENT doctors are trained to recognize sinonasal anatomical variations and signs of sinonasal diseases of the sinus CT scans, magnetic resonance imaging images and X-rays. There is a little training for detection of radiologic periodontal signs, but the training usually focuses on detection of periapical radiolucency.

To the knowledge of the authors, this study is the first one to evaluate inter-observer agreement and inter-imaging accuracy from sinus CT and PT scans.

Conclusions

This study showed surprisingly great variation in diagnostic accuracy between the assessors with different professional background when recording radiological signs of periodontitis and periapical periodontitis. The results were in contrast to what we had expected and indicate subjectivity in diagnostic radiology. The findings emphasize the importance of consulting and need for clinical dental examination before deciding a treatment plan. The results also show that PTG scans are not as reliable as CT scans for diagnosing local pathology of dentoalveolar bone loss and periapical radiolucency.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1.	Virtanen E, Nurmi T, Söder PÁ, Airila-Månsson S, Söder B, Meurman JH. Apical periodontitis associates with cardiovascular diseases: A cross-sectional study from Sweden. BMC Oral Health 2017;17:107.
2.	Papapanou PN, Sanz M, Buduneli N, Dietrich T, Feres M, Fine DH, et al. Periodontitis: Consensus report of workgroup 2 of the 2017 World Workshop on the Classification of Periodontal and Peri-Implant Diseases and Conditions. J Clin Periodontol 2018;45 Suppl 20:S162-70.
3.	Corbet EF, Ho DK, Lai SM. Radiographs in periodontal disease diagnosis and management. Aust Dent J 2009;54 Suppl 1:S27-43.
4.	Abbott PV. Classification, diagnosis and clinical manifestations of apical periodontitis. Endod Top 2004;8:36-54.
5.	Lechien JR, Filleul O, Costa de Araujo P, Hsieh JW, Chantrain G, Saussez S. Chronic maxillary rhinosinusitis of dental origin: a systematic review of 674 patient cases. Int J Otolaryngol. 2014;2014:465173. doi: 10.1155/2014/465173. Epub 2014 Apr 8. PMID: 24817890; PMCID: PMC4000986. https://pubmed.ncbi.nlm.nih.gov/24817890/.
6.	Maillet M, Bowles WR, McClanahan SL, John MT, Ahmad M. Cone-beam computed tomography evaluation of maxillary sinusitis. J Endod 2011;37:753-7.
7.	Phothikhun S, Suphanantachat S, Chuenchompoonut V, Nisapakultorn K. Cone-beam computed tomographic evidence of the association between periodontal bone loss and mucosal thickening of the maxillary sinus. J Periodontol 2012;83:557-64.
8.	Obayashi N, Ariji Y, Goto M, Izumi M, Naitoh M, Kurita K, et al. Spread of odontogenic infection originating in the maxillary teeth: Computerized tomographic assessment. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2004;98:223-31.
9.	Julkunen A, Terna E, Numminen J, Markkola A, Dastidar P, Karjalainen M, et al. Inter-observer agreement of paranasal sinus computed tomography scans. Acta Otolaryngol 2017;137:611-7.
10.	Shahzad A, Nauman Q, Rubab UE. In panoramic radiographs variability in inter observer assessment of mental foramen radiographically and its appearance. Indo Am J Pharm Sci 2018;5:11466-70.
11.	Hopkins C, Browne JP, Slack R, Lund V, Brown P. The lund-mackay staging system for chronic rhinosinusitis: How is it used and what does it predict? Otolaryng Head Neck Surg 2007;137:555-61.
12.	Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
13.	Highfield J. Diagnosis and classification of periodontal disease. Aust Dent J 2009;54 Suppl 1:S11-26.
14.	Armitage GC. The complete periodontal examination. Periodontol 2000 2004;34:22-33.
15.	Fuhrmann RA, Bücker A, Diedrich PR. Assessment of alveolar bone loss with high resolution computed tomography. J Periodontal Res 1995;30:258-63.
16.	Tang Z, Liu X, Chen K. Comparison of digital panoramic radiography versus cone beam computerized tomography for measuring alveolar bone. Head Face Med 2017;13:2.