Skip to main content
Free AccessResearch Article

A semi-automatic approach for longitudinal 3D upper airway analysis using voxel-based registration

Published Online:



To propose and validate a reliable semi-automatic approach for three-dimensional (3D) analysis of the upper airway (UA) based on voxel-based registration (VBR).


Post-operative cone beam computed tomography (CBCT) scans of 10 orthognathic surgery patients were superimposed to the pre-operative CBCT scans by VBR using the anterior cranial base as reference. Anatomic landmarks were used to automatically cut the UA and calculate volumes and cross-sectional areas (CSA). The 3D analysis was performed by two observers twice, at an interval of two weeks. Intraclass correlations and Bland-Altman plots were used to quantify the measurement error and reliability of the method. The relative Dahlberg error was calculated and compared with a similar method based on landmark re-identification and manual measurements.


Intraclass correlation coefficient (ICC) showed excellent intra- and inter-observer reliability (ICC ≥ 0.995). Bland-Altman plots showed good observer agreement, low bias and no systematic errors. The relative Dahlberg error ranged between 0.51 and 4.30% for volume and 0.24 and 2.90% for CSA. This was lower when compared with a similar, manual method. Voxel-based registration introduced 0.05–1.44% method error.


The proposed method was shown to have excellent reliability and high observer agreement. The method is feasible for longitudinal clinical trials on large cohorts due to being semi-automatic.


Assessment of the upper airway (UA) using computed tomography (CT), cone beam computed tomography (CBCT), and magnetic resonance imaging (MRI) has developed from two-dimensional (2D) linear measurements and lateral cephalometry to three-dimensional (3D) UA analysis.1–7 However, although 3D analysis provides additional information, for example, surface and volume measures, the analysis is often time-consuming, and the reliability is compromised by the uncertainty introduced by re-identification of landmarks.8–10 To encounter these shortcomings of the 3D analysis, superimposition techniques have been introduced for 3D analysis of the postoperative outcome of corrective jaw surgery, and orthognathic surgery.11–13 These techniques facilitate registration of two consecutive uni- or multimodal scans based on a reference structure. Using mutual image information [voxel-based registration (VBR)], surface similarity [surface-based registration (SBR)], or landmark similarity [landmark-based registration (LBR)], two scans or 3D objects can be superimposed.14–17 Alignment with VBR has shown higher accuracy,18 consistency and efficiency than SBR and LBR.19–21 Hence, VBR is the preferred method of choice for image registration.19,21,22 For the analysis of the surgical outcome of orthognathic surgery the anterior cranial base, including the zygomatic arches and forehead, structures unaffected by surgery have shown to be reliable references for superimposition18 and are therefore often used.22

Although superimposition has shown to be efficient and reliable for skeletal assessment of orthognathic surgery,11–13 this technique has only been sparsely adopted for UA analysis.23–32 The majority of these studies used CBCT as the imaging technique, dividing the UA into three segments using cutting planes and measuring the UA volume and CSA. Voxel-based registration was frequently used for superimposition, with the stable region of choice being the cranial base.27–29,31,32 Another common superimposition technique was combining LBR with SBR23–26,30 or with VBR.27

Although the studies showed promising results regarding UA analysis based on superimposition, questions remain regarding the reliability and reproducibility of the methods. Only three studies conducted a reliability study on the methods.25,26,30 The reliable methods, all similar, from the same research group, use LBR. However, LBR requires manual re-identification of points and has been shown to be less reliable than VBR.20 Furthermore, it is unclear whether cephalometric landmarks were re-identified and whether superimposition solely was used to visualize morphological UA CSA changes in these three studies. None of these studies contain both a reproducible UA analysis method using VBR and a reliability study of the method. Hence, there is a need for a reliable method for integration of VBR in automated UA analysis, avoiding re-identification of landmarks and reducing manual input.

The aim of the present study is to propose and validate a semi-automatic approach for 3D UA analysis based on superimposition. Its reliability is evaluated and directly compared to a similar method not applying superimposition. It is hypothesized that a semi-automatic method for 3D longitudinal UA analysis using VBR will produce reliable and more exact results than a similar method that relies on re-identification of landmarks and manual measurements.

Methods and materials

Permission was granted by the Danish Data Protection Agency (2008-58-0035, 31 August 2015, 18/40631).

The method developed in this study is based on and compared directly to the method developed by Di Carlo et al5. The patient dataset from Di Carlo et al5, including CBCT scans, 3D segmentations of the airways and anatomical landmarks, has been used during the development of the presented novel method. Consequently, additional descriptions of patient selection, image acquisition, and segmentation can be found in the original protocol.5

The automated analysis part of the approach was implemented in Python 3.7 (Python Software Foundation, Fredericksburg, Virginia) and incorporated in Mimics Innovation Suite (Materialise NV, Leuven, Belgium) through its scripting module. Mimics Innovation Suite is comprised of Mimics 23.0 and 3-matic 15.0. An overview of the workflow can be seen in Supplementary Material 1 available online.

Image acquisition and segmentation

The subjects from the Di Carlo et al5 cohort are 10 randomly selected patients (seven females) diagnosed with maxillomandibular growth disturbances that underwent orthognathic surgery in 2012 at the University Hospital of Southern Denmark, Department of Oral and Maxillofacial Surgery, Esbjerg, Denmark. Nine patients underwent a combined bilateral mandibular sagittal split osteotomy/maxillary Le Fort I osteotomy (eight with segmental maxillary osteotomies) and one patient underwent a maxillary Le Fort I osteotomy exclusively (with segmental maxillary osteotomies). The pre- and post-surgical CBCT scans, stored in Digital Imaging and Communications in Medicine (DICOM) format, from the dataset have been used. In order to avoid introducing variability from a different segmentation procedure, the segmentation of the UA space used in the Di Carlo et al5 UA analysis was maintained.

The selected sample size was calculated by Di Carlo et al5 in order to obtain a power equal to 0.99 with an effect size of 1 mm for linear measurements, 1 mm2 for surface and 1 mm3 for volumetric and α = 0.5. The exclusion criteria were: syndromes or detectable pathologies involving UA and CBCT scans not including all the craniofacial structures required for the cephalometric analysis. Particular attention was paid to patients’ positioning during CBCT scan: subjects without the mandible at maximum intercuspation were excluded as well as patients wearing a bite during scan acquisition. Patients with inappropriate head positioning, including major head extension, head flexion or head rotation, were excluded as well.5

Voxel-based registration

Superimposition of the post-surgical to the pre-surgical scan of each patient was performed using the “Automatic Image Registration” function in Mimics. This function is used to align two different images to the same anatomical coordinate system: a fixed image and a moving image. For this study, the fixed image was the pre-operative CBCT scan and the moving image was the post-operative CBCT scan. After the images were selected, they were manually pre-aligned into approximative positions. Then, a region of interest was specified under the form of a bounding box around the stable anatomical structures of the cranial base, including the zygomatic arches and forehead.18 The superimposition method was VBR. The registration function generated a transformation matrix that describes the transformation of the moving image toward the fixed image. This transformation matrix was applied to the post-operative CBCT scan, resulting in the post-operative CBCT scan being aligned with the pre-operative CBCT scan at the cranial base (Figure 1). To ensure a suitable registration had been achieved, the result was visually inspected in all views.

Figure 1.
Figure 1.

Upper airway analysis. Before voxel-based registration (VBR): pre- and post-surgical CBCTs (upper left), skulls (middle left), airways (lower left); after VBR: CBCTs (middle top), skulls (centre), airways (middle bottom); illustration of relationship between cutting planes, CBCT, and airway objects (upper right); analysed airways overlay (lower right).

Analysis of the upper airway

The landmarks selected for airway analysis followed the description in Di Carlo et al5, Table 1. Based on the same criteria, the cephalometric landmarks were placed on the preoperative scan. The axial, coronal, and sagittal views were used to place the landmarks, and additionally the 3D view was used to verify the placement. The airway analysis script was run on the pre- and post-operative UA spaces. The script used the anatomic landmarks to create several planes, as defined by Di Carlo et al5: the Frankfurt Horizontal (FH) plane, mid-sagittal plane, sella-nasion (SN) horizontal plane, and the five planes used to cut the airway into five partial volumes (Table 2 and Figure 1). The pre- and superimposed post-operative airways are automatically sectioned at the level of the cutting planes, and the volumes and cross-sectional area (CSA) are automatically calculated and displayed. These measurements are performed according to Di Carlo et al5 (Table 3). Running the script also provided an overlay between the pre- and post-operative airway segments to easily be visually inspected (Figure 1). All the measurements were exported to Excel (Microsoft, Redmond, WA). A video summary of the UA analysis procedure is available (Supplementary Video 1). For a more visual understanding of the process, the readers are encouraged to consult the video summary.

Table 1. Landmarks used in airway analysis

BaBasion, the anterior margin of the foramen magnum
NNasion, the intersection of the internasal and frontonasal sutures in the midsagittal plane
Or LOrbitale left, the most inferior anterior point on left orbit’s margin
Or ROrbitale right, the most inferior anterior point on right orbit’s margin
PoLPorion left, the most upper point on left bony external auditory meatus
PoRPorion left, the most upper point on right bony external auditory meatus
SSella turcica, the centroid of Sella turcica
SoaMidpoint of Sella-Basion line
HHyoid bone, upper most point of the hyoid bone
EEpiglottis, tip of epiglottis

aAutomatically calculated in analysis script.

Table 2. Planes used in airway analysis

References planes
 Frankfurt planeA plane passing through the inferior borders of the bony orbits, encompassed by OrR and OrL, and the upper margin of the auditory meatus encompassed by PoR and PoL.
 Sagittal SNPlane perpendicular to Frankfurt plane passing through S and N points
 SN horizontalPlane through S and N points, perpendicular to Sagittal (SN)
 S Ba coronalPlane through S and Ba points perpendicular to Sagittal (SN)
Retropalatal region
 Airway superior border 1Plane passing through So and perpendicular to S-Ba coronal and Sagittal (SN) planes
 Airway superior border 2Plane passing through Ba and parallel to Airway Superior Border 1
 Airway superior border 3Plane passing through Ba and parallel to Frankfurt
Oropharyngeal region
 Airway inferior border 1Plane passing through E and parallel to Frankfurt
 Airway inferior border 2Plane passing through 90% of the distance from Airway Superior Border 1 to Airway Inferior Border 1 and parallel to Frankfurt

Table 3. Upper airway volumes and cross-sectional measurements

 RetropalatalThe airway volume encompassed superiorly by airway superior border and inferiorly by airway superior border 2
 U-RetropalatalThe airway volume encompassed superiorly by airway superior border and inferiorly by airway superior border 3
 L-RetropalatalThe airway volume encompassed superiorly by airway superior border 3 and inferiorly by airway superior border 2
 Oropharynx D100The airway volume encompassed superiorly by airway superior border 2 and inferiorly by airway inferior border 1
 Oropharynx D90The airway volume encompassed superiorly by airway superior border 2 and, inferiorly by airway inferior border 2
 CSA1Cross-sections at the boundary between upper retropalatal volume and lower retropalatal volume
 CSA2Cross-sections at the boundary between lower retropalatal volume and oropharynx
 CSA3Cross-sections at the boundary between oropharynx D90 vol and oropharynx D100 vol
 CSA4Cross-section located at the bottom of oropharynx D100 vol


Two observers (MBH and AD) independently conducted the analysis twice (landmark placement, VBR and semi-automatic analysis), with a two-week interval, in order to evaluate intra- and inter-observer. The second time, the observers were blinded to the first round of analysis. To isolate VBR from the potential error introduced by landmark identification, an additional semi-automatic analysis was conducted where the same landmarks were used across measures, i.e. in the inter-observer case, both observers used the same landmarks, and in the intra-observer case, the same landmarks were used in the first and second measurements. This allowed to isolate error introduced by landmark re-identification from error introduced by the novel aspects of the method (VBR and semi-automatic analysis).

Statistical analysis

Statistical analysis was conducted using STATA v.16.1 (StataCorp, College Station, TX). The data were tested for normality using the Shapiro-Wilk test. For normally distributed data, a paired t-test was used to compare inter- and intra-observer measurements and to report confidence intervals. The intraclass correlation coefficient (ICC) was used to report the intra- and inter-observer reliability. Bland-Altman plots were used to determine the degree of agreement, outliers, bias, and systematic errors. For a direct comparison with the original method by Di Carlo et al5, Dahlberg’s formula was used to compare both inter- and intra-observer measurements and the isolated VBR measurements.33


The Shapiro-Wilk test indicated all the data were normally distributed. Consequently, a paired t-test was performed. The results of the t-test, used to compare inter- and intra-observer measurements, showed narrow confidence intervals and the repeated inter- and intra-observer measurements were not significantly different. Tables 4–7 show these results. Tables 4–7 also contain the mean and standard deviation for the measures, as well as summarize the results from the measures of reliability. ICC showed excellent intra- and inter-observer reliability (ICC ≥ 0.995).

Table 4. Intra-observer analysis for observer 1 (MBH)

first measurementsecond measurementICCDahlberg’s dp95% CIfirst measurementsecond measurementICCDahlberg’s dp95% CI
Retropalatal (mm3)9649.553147.709634.623173.040.999849.370.51%0.53−36.5166.3610310.883541.2910273.113561.050.999764.450.63%0.21−24.78100.32
U-Retropalatal (mm3)4012.562168.494021.742151.290.999739.050.97%0.63−50.2431.883949.422061.143926.432049.660.999354.881.39%0.38−32.9178.88
L-Retropalatal (mm3)5636.991725.315612.881769.240.999537.910.67%0.17−12.0060.226361.462523.666346.682573.040.998695.331.50%0.75−86.26115.83
Oropharynx D90 (mm3)7817.542547.327839.282550.760.999366.280.85%0.49−90.4947.029822.404585.159854.784559.190.9995106.951.09%0.53−143.7879.02
Oropharynx D100 (mm3)9385.793446.109410.633441.200.999672.710.77%0.47−100.0950.4011348.185433.4711383.565402.670.9996110.470.97%0.50−150.1379.36
CSA1 (mm2)556.40168.79555.27168.970.99982.070.37%0.24−0.903.16528.47180.20525.45178.380.99954.120.78%0.10−0.746.79
CSA2 (mm2)340.0393.34338.6593.830.99952.060.61%0.14−0.553.31422.59176.44423.07177.291.00001.030.24%0.32−1.520.56
CSA3 (mm2)241.73137.13242.22136.480.99991.000.42%0.30−1.490.52262.05150.09262.75150.890.99981.940.74%0.44−2.701.29
CSA4 (mm2)265.82141.22267.16142.300.99953.301.24%0.39−4.712.02225.40107.36226.95110.490.99893.691.64%0.38−5.312.22

Table 5. Intra-observer analysis for observer 2 (AD)

first measurementsecond measurementICCDahlberg’s dp95% CIfirst measurementsecond measurementICCDahlberg’s dp95% CI
Retropalatal (mm3)9655.323203.209562.833097.240.9978148.231.54%0.17−49.38234.3410324.683585.4110277.123587.700.999764.120.62%0.10−10.66105.78
U-Retropalatal (mm3)3986.852265.793941.842131.020.9950155.483.90%0.55−117.29207.303871.602098.383882.262072.800.9974107.182.77%0.84−124.67103.35
L-Retropalatal (mm3)5668.461758.935620.991715.520.9966100.811.78%0.32−53.89148.846453.082665.466394.862596.700.9978122.161.89%0.31−64.43180.88
Oropharynx D90 (mm3)7753.772514.427784.692593.020.9982106.891.38%0.55−142.5080.659729.144507.829758.714583.350.9993119.221.23%0.61−154.7395.59
Oropharynx D100 (mm3)9315.553406.129347.093495.170.9989116.471.25%0.57−153.4590.3511259.395372.7711280.795440.580.9995121.401.08%0.72−149.85107.06
CSA1 (mm2)547.93173.89554.96175.190.994612.782.33%0.24−19.585.53520.72182.70524.52187.840.996710.692.05%0.46−14.837.23
CSA2 (mm2)340.9993.14340.9595.090.99952.040.60%0.97−2.142.22420.83173.00421.09175.280.99973.210.76%0.87−3.683.17
CSA3 (mm2)243.08138.17242.80138.820.99982.010.83%0.78−1.862.40260.95148.96262.11149.120.99895.031.93%0.63−6.454.13
CSA4 (mm2)269.17144.54266.69141.480.99875.221.94%0.31−2.787.72230.08112.62226.93108.180.99706.092.65%0.27−2.909.19

Table 6. Inter-observer analysis, pre-surgical

Observer 1 vs observer 2 at the first measurementObserver 1 vs observer 2 at the second measurement
t-testDahlberg’s dICCt-testDahlberg’s dICC
p95% CIp95% CI
Retropalatal (mm3)0.84−68.5156.9958.990.61%0.99970.60−71.5643.96123.331.28%0.9985
U-Retropalatal (mm3)0.68−109.79161.20128.353.20%0.99660.23−58.67214.32172.854.30%0.9935
L-Retropalatal (mm3)0.47−126.1063.1691.491.62%0.99720.17−230.9947.73126.972.26%0.9947
Oropharynx D90 (mm3)0.17−32.61160.16101.011.29%0.99840.10−21.30207.81114.341.46%0.9980
Oropharynx D100 (mm3)0.15−29.64170.12106.021.13%0.99900.11−25.14202.71127.061.35%0.9987
CSA1 (mm2)0.11−2.2219.1611.682.10%0.99540.19−4.6220.1315.262.75%0.9921
CSA2 (mm2)0.46−3.771.862.720.80%0.99910.42−2.916.423.200.94%0.9989
CSA3 (mm2)0.04−2.62−0.071.530.63%0.99990.63−3.916.092.380.98%0.9997
CSA4 (mm2)0.14−8.071.365.021.89%0.99880.11−10.691.343.731.40%0.9993

Table 7. Inter-observer analysis, post-surgical

Observer 1 vs observer 2 at the first measurementObserver 1 vs observer 2 at the second measurement
t-testDahlberg’s dICCt-testDahlberg’s dICC
p95% CIp95% CI
Retropalatal (mm3)0.21−48.06191.6555.040.53%0.99980.94−112.57104.55101.840.99%0.9992
U-Retropalatal (mm3)0.33−94.31254.10139.333.53%0.99550.57−127.52215.86164.004.15%0.9937
L-Retropalatal (mm3)0.90−143.37127.16145.862.29%0.99680.63−269.89173.51210.673.31%0.9934
Oropharynx D90 (mm3)0.31−60.18169.36126.051.28%0.99920.32−111.76303.89206.392.10%0.9980
Oropharynx D100 (mm3)0.29−63.20190.28123.911.09%0.99950.31−111.10316.65213.321.88%0.9985
CSA1 (mm2)0.97−15.9616.5812.832.43%0.99500.88−12.7814.6612.882.44%0.9951
CSA2 (mm2)0.11−5.230.644.551.08%0.99930.48−
CSA3 (mm2)0.61−3.091.924.751.81%0.99900.62−2.173.462.681.02%0.9997
CSA4 (mm2)0.80−3.504.436.542.90%0.99651.00−4.344.364.081.81%0.9986

For comparison with the manual approach by Di Carlo et al5, the Dahlberg error was calculated. Figure 2 shows the pre-surgical measurement errors for the intra- and inter-observer measurements. Figure 3 shows the post-surgical measurement errors for the intra- and inter-observer measurements. The measurement error of the observers from the present study (AD and MBH) is lower across the board when compared to the measurement error of the observers from the Di Carlo et al5 study (GDC and SFG). The method error from the present study ranged between 0.51 and 4.30% for volumetric measurements and between 0.24 and 2.90% for CSA measurements. In comparison, the error from the Di Carlo et al5 study ranges between 1.6 and 10.2% for volume and between 2 and 12.2% for CSA. Figure 3 also shows the part of the measurement error which is introduced by the user input required for VBR (pre-alignment and selection of volume of interest), with the error for the intra- and inter-observer measurements ranging between 0.05% for pre-operative measurements and up to 1.44% for post-operative measurements.

Figure 2.
Figure 2.

Pre-surgical measurement error, intra-observer (top), and inter-observer (bottom). GDC & SFG (observers from Di Carlo et al. 2017), MBH & AD (observers from present study).

Figure 3.
Figure 3.

Post-surgical measurement error, intra-observer (top) and inter-observer (bottom). GDC & SFG (observers from Di Carlo et al. 2017), MBH & AD (observers from present study); a Isolated VBR: only VBR and automatic analysis were performed for repeated measures (the first set of landmarks was also used in the second measurement). b Isolated VBR: only VBR performed by different observers.


The purpose of the present study was to propose and validate a semi-automatic approach for 3D UA analysis based on VBR. The semi-automatic method for 3D UA analysis using VBR produced more reliable results than a similar method relying on re-identification of landmarks and manual measurements.

The ICC values obtained for the present method were 0.995 or higher, showing excellent to near perfect reliability. The Bland-Altman plots indicated no systematic error and low bias. Method error calculated with the Dahlberg formula was used for comparison with Di Carlo et al5. The Relative Dahlberg error was calculated because it allowed for comparison of measurements with different units and means.33 The measurements performed were volume (mm3) and CSA (mm2). The novel method with VBR and semi-automatic analysis has lower relative Dahlberg error (0.24–4.30%) than the method by Di Carlo et al5 (1.6–12.2%), which re-identifies landmarks and performs measurements manually (Figures 2 and 3). Paired t-tests resulted in narrow confidence intervals when compared with Di Carlo et al5 indicating greater reliability for the novel method.

The method error of the VBR and semi-automatic analysis was isolated using the same set of landmarks for all inter- and intra-observer measurements. The results indicate VBR introduces a very small portion of variability. There is 0.0% variability (relative Dahlberg error) in the pre-surgical measurements of the isolated VBR procedure. That is to be expected, as the same landmarks were used for both intra- and inter-observer measurement, and superimposition does not play a role in the pre-surgical measurements. Therefore, the analysis script does not introduce any variability. This is the intended purpose of automation: to avoid error from manual user input. The post-surgical measurement error ranged between 0.05 and 1.44% (Figure 3). Since the landmarks were identical, and it has been established that the analysis script does not introduce variability, this error range is therefore attributed to VBR. Voxel-based registration requires user input in pre-alignment and defining the region of interest.

The method can be used with any other set of cephalometric landmarks and cutting planes. Hence, other protocols for UA analysis than the one proposed by Di Carlo et al5 can be integrated in the semi-automatic analysis using voxel-based registration. This provides the flexibility to include other airway measurements, for example, airway length and shape.

In a systematic review by Zimmerman et al34, 42 studies were reviewed to evaluate the reliability of upper pharyngeal airway assessment using CBCT. The UA analysis methods demonstrated moderate to excellent intra- and inter-observer reliability. However, only five studies were deemed high quality, and only three of these studies assessed both intra- and inter-observer reliability. From the high-quality studies, airway volume measurements demonstrated greater intra- and inter-observer reliability (>0.880) than CSA measurements (>0.696). In comparison, the proposed semi-automatic method demonstrated near to perfect intra- and inter-observer reliability (>0.995) for both volume- and CSA measurements. The three high-quality studies which assessed intra- and inter-observer reliability all applied the Dolphin Imaging® software (Dolphin Imaging and Management Systems, Chatsworth, CA, USA) for UA analysis. Of these studies, the study by De Souza et al35 reported good reliability (>0.88); The study by Guijarro-Martinez,36 reported excellent reliability (>0.981) and observer error up to 4% for volume measurements, but only moderate reliability (>0.780) and observer error up to 20% for CSA measurements. The study by Mattos et al37 showed that the transversal width- and CSA measurements at the level of the vallecula had only moderate reliability (>0.63) with a mean observer difference of 16.1 and 13.3%, respectively. The proposed method, which adopts the UA definition by Di Carlo et al5, inferiorly limits the UA at Epiglottis, which may be more reliable, and automatically transfers the cutting planes to the post-operative CBCT, resulting in near to perfect reliability and low observer error for CSA measurements.

Recently, Chen et al38 assessed the reliability and accuracy of three different imaging software packages, Amira® (Visage Imaging Inc., Carlsbad, CA), 3Diagnosys® (3diemme, Cantu, Italy) and OnDemand3D® (CyberMed, Seoul, Republic of Korea) for three-dimensional analysis of the upper airway using CBCT images. The intra- and inter-observer reliability of the measurements using the different software packages were moderate to excellent (>0.78). All three software packages underestimated the upper airway volume by −8.8 to −12.3%, the minimum CSA by −6.2 to− 14.6% and the length by −1.6 to −2.9%.

In a recent study, Zimmerman et al39 presented the first study to evaluate the reliability of UA assessment using CBCT, where the observers performed each step of the analysis manually. The authors argue that this is the way UA assessment would be conducted in a clinical setting today. Intra- and inter-observer reliability of each step in the UA assessment were evaluated by six observers of varying levels of education and clinical experience. Threshold selection in the UA segmentation process showed poor intra- (0.473) and poor inter-observer reliability (0.100); CSA showed moderate intra- (0.591) and poor inter-observer reliability (0.223); Intra-observer reliability of volume measurements varied based on the region assessed (0.747–0.976) and was worst for hypopharynx and best for the oropharynx; inter-observer reliability of volume measurements was generally lower (0.175–0.945) and was worst for nasopharynx and best for the oropharynx. These results are of special interest for comparison with the present study, which in contrast to a fully manual UA analysis evaluated by Zimmerman et al39, is optimised to reduce observer error by automation of sub-processes. The great improvement in the intra- and inter-observer reliability clearly demonstrates the advantages of automation, which should be more generally adopted in clinical practice.

Although the proposed approach minimizes user input for the UA analysis and omits landmark re-identification, manual precursory steps still have to be taken, i.e. the segmentation of the UA and the initial placement of landmarks. Recent developments in the field automatic segmentation40 and landmark placement have been made using artificial intelligence, achieving results within clinically acceptable margins for landmark placement (<2 mm).41–43 These state of the art methods may complement the proposed UA analysis approach, making a fully automatic UA analysis workflow possible.

A noteworthy mention is that no measures have been taken to address the variability in head posture, which is known to affect the UA.44 This issue is inherent to the image acquisition process, and a general challenge of measuring soft tissue response before and after surgery. The logical goal would be to correlate these cross-sectional and volumetric measures with polysomnography findings. The airway changes associated with orthognathic surgery could provide functional justification for this procedure provided the demonstrable postoperative airway changes correlate with improved function are confirmed by polysomnography.


The proposed method was shown to be reliable. A lack of reliable methods for UA analysis that capitalize on the benefits of VBR in the literature is emphasized. When compared to a similar method that involves manual input and landmark re-identification, the semi-automatic approach using VBR shows less inter- and intra-observer measurement error. The proposed approach has great applicability for UA analysis in a clinical trial, due to high reliability. This is further made feasible by the automated part of the analysis, making the novel method efficient and suitable for longitudinal studies on large cohorts.

Alexandru Diaconu and Michael Boelstoft Holte have contributed equally to this study and should be considered as co-first authors.


  • 1. Abramson Z, Susarla SM, Lawler M, Bouchard C, Troulis M, Kaban LB. Three-Dimensional computed tomographic airway analysis of patients with obstructive sleep apnea treated by maxillomandibular advancement. J Oral Maxillofac Surg 2011; 69: 677–86. doi: Crossref Medline ISIGoogle Scholar

  • 2. Bianchi A, Betti E, Tarsitano A, Morselli-Labate AM, Lancellotti L, Marchetti C. Volumetric three-dimensional computed tomographic evaluation of the upper airway in patients with obstructive sleep apnoea syndrome treated by maxillomandibular advancement. Br J Oral Maxillofac Surg 2014; 52: 831–7. doi: Crossref Medline ISIGoogle Scholar

  • 3. Butterfield KJ, Marks PLG, McLean L, Newton J. Pharyngeal airway morphology in healthy individuals and in obstructive sleep apnea patients treated with maxillomandibular advancement: a comparative study. Oral Surg Oral Med Oral Pathol Oral Radiol 2015; 119: 285–92. doi: Crossref Medline ISIGoogle Scholar

  • 4. Christovam IO, Lisboa CO, Ferreira DMTP, Cury-Saramago AA, Mattos CT. Upper airway dimensions in patients undergoing orthognathic surgery: a systematic review and meta-analysis. Int J Oral Maxillofac Surg 2016; 45: 460–71. doi: Crossref Medline ISIGoogle Scholar

  • 5. Di Carlo G, Gurani SF, Pinholt EM, Cattaneo PM. A new simple three-dimensional method to characterize upper airway in orthognathic surgery patient. Dentomaxillofac Radiol 2017; 46: 20170042. doi: Link ISIGoogle Scholar

  • 6. Fernández-Ferrer L, Montiel-Company JM, Pinho T, Almerich-Silla JM, Bellot-Arcís C. Effects of mandibular setback surgery on upper airway dimensions and their influence on obstructive sleep apnoea - a systematic review. J Craniomaxillofac Surg 2015; 43: 248–53. doi: Crossref Medline ISIGoogle Scholar

  • 7. Gurani SF, Di Carlo G, Thorn JJ, Ingerslev J, Cattaneo PM, Pinholt EM. Two-Year postoperative upper airway cone-beam computed tomographic outcomes based on a verified upper airway analysis following bimaxillary Orthognathic surgery. J Oral Maxillofac Surg 2019; 77: 1435–45. doi: Crossref Medline ISIGoogle Scholar

  • 8. Katkar RA, Kummet C, Dawson D, Moreno Uribe L, Allareddy V, Finkelstein M, et al.. Comparison of observer reliability of three-dimensional cephalometric landmark identification on subject images from Galileos and i-CAT cone beam CT. Dentomaxillofac Radiol 2013; 42: 20130059. doi: Link ISIGoogle Scholar

  • 9. Titiz I, Laubinger M, Keller T, Hertrich K, Hirschfelder U. Repeatability and reproducibility of landmarks--a three-dimensional computed tomography study. Eur J Orthod 2012; 34: 276–86. doi: Crossref Medline ISIGoogle Scholar

  • 10. Xi T, van Luijn R, Baan F, Schreurs R, de Koning M, Bergé S, et al.. Landmark-Based versus voxel-based 3-dimensional quantitative analysis of bimaxillary osteotomies: a comparative study. J Oral Maxillofac Surg 2020; 78: 468.e1–468.e10. doi: Crossref ISIGoogle Scholar

  • 11. Baan F, Sabelis JF, Schreurs R, van de Steeg G, Xi T, van Riet TCT, et al.. Validation of the OrthoGnathicAnalyser 2.0-3D accuracy assessment tool for bimaxillary surgery and genioplasty. PLoS One 2021; 16: e0246196. doi: Crossref ISIGoogle Scholar

  • 12. Holte MB, Diaconu A, Ingerslev J, Thorn JJ, Pinholt EM, et al.. Virtual analysis of segmental bimaxillary surgery: a validation study. J Oral Maxillofac Surg 2021; 10:S0278-2391(21)00540-1. doi: 34245697 MedlineGoogle Scholar

  • 13. Stokbro K, Thygesen T. A 3-dimensional approach for analysis in Orthognathic Surgery-Using free software for voxel-based alignment and Semiautomatic measurement. J Oral Maxillofac Surg 2018; 76: 1316–26. doi: Crossref Medline ISIGoogle Scholar

  • 14. Cevidanes LHS, Styner MA, Proffit WR. Image analysis and superimposition of 3-dimensional cone-beam computed tomography models. Am J Orthod Dentofacial Orthop 2006; 129: 611–8. doi: Crossref Medline ISIGoogle Scholar

  • 15. Koerich L, Burns D, Weissheimer A, Claus JDP. Three-Dimensional maxillary and mandibular regional superimposition using cone beam computed tomography: a validation study. Int J Oral Maxillofac Surg 2016; 45: 662–9. doi: Crossref Medline ISIGoogle Scholar

  • 16. Lee J-H, Kim M-J, Kim S-M, Kwon O-H, Kim Y-K. The 3D CT superimposition method using image fusion based on the maximum mutual information algorithm for the assessment of oral and maxillofacial surgery treatment results. Oral Surg Oral Med Oral Pathol Oral Radiol 2012; 114: 167–74. doi: Crossref Medline ISIGoogle Scholar

  • 17. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P. Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 1997; 16: 187–98. doi: Crossref Medline ISIGoogle Scholar

  • 18. Nada RM, Maal TJJ, Breuning KH, Bergé SJ, Mostafa YA, Kuijpers-Jagtman AM. Accuracy and reproducibility of voxel based superimposition of cone beam computed tomography models on the anterior cranial base and the Zygomatic arches. PLoS One 2011; 6: e16520. doi: Crossref ISIGoogle Scholar

  • 19. Almukhtar A, Ju X, Khambay B, McDonald J, Ayoub A. Comparison of the accuracy of voxel based registration and surface based registration for 3D assessment of surgical change following orthognathic surgery. PLoS One 2014; 9: e93402. doi: Crossref Medline ISIGoogle Scholar

  • 20. Ghoneima A, Cho H, Farouk K, Kula K. Accuracy and reliability of landmark-based, surface-based and voxel-based 3D cone-beam computed tomography superimposition methods. Orthod Craniofac Res 2017; 20: 227–36. doi: Crossref Medline ISIGoogle Scholar

  • 21. Han G, Li J, Wang S, Wang L, Zhou Y, Liu Y. A comparison of voxel- and surface-based cone-beam computed tomography mandibular superimposition in adult orthodontic patients. J Int Med Res 2021; 49: 030006052098270. doi: Crossref ISIGoogle Scholar

  • 22. Gaber RM, Shaheen E, Falter B, Araya S, Politis C, Swennen GRJ, et al.. A systematic review to uncover a universal protocol for accuracy assessment of 3-dimensional virtually planned Orthognathic surgery. J Oral Maxillofac Surg 2017; 75: 2430–40. doi: Crossref Medline ISIGoogle Scholar

  • 23. Alsufyani NA, Dietrich NH, Lagravère MO, Carey JP, Major PW. Cone beam computed tomography registration for 3-D airway analysis based on anatomic landmarks. Oral Surg Oral Med Oral Pathol Oral Radiol 2014; 118: 371–83. doi: Crossref Medline ISIGoogle Scholar

  • 24. Alsufyani NA, Noga ML, Witmans M, Cheng I, El-Hakim H, Major PW. Using cone beam CT to assess the upper airway after surgery in children with sleep disordered breathing symptoms and maxillary-mandibular disproportions: a clinical pilot. J Otolaryngol Head Neck Surg 2017; 46: 31. doi: Crossref Medline ISIGoogle Scholar

  • 25. Chen X, Liu D, Liu J, Wu Z, Xie Y, Li L, et al.. Three-Dimensional evaluation of the upper airway morphological changes in growing patients with skeletal class III malocclusion treated by protraction headgear and rapid palatal expansion: a comparative research. PLoS One 2015; 10: e0135273. doi: ISIGoogle Scholar

  • 26. Chen Y, Hong L, Wang C-ling, Zhang S-jie, Cao C, Wei F, et al.. Effect of large incisor retraction on upper airway morphology in adult bimaxillary protrusion patients. Angle Orthod 2012; 82: 964–70. doi: Crossref Medline ISIGoogle Scholar

  • 27. Giralt-Hernando M, Valls-Ontañón A, Haas Junior OL, Masià-Gridilla J, Hernández-Alfaro F. What are the surgical movements in Orthognathic surgery that most affect the upper airways? a three-dimensional analysis. J Oral Maxillofac Surg 2021; 79: 450–62. doi: Crossref Medline ISIGoogle Scholar

  • 28. Jang S-I, Ahn J, Paeng JY, Hong J. Three-Dimensional analysis of changes in airway space after bimaxillary orthognathic surgery with maxillomandibular setback and their association with obstructive sleep apnea. Maxillofac Plast Reconstr Surg 2018; 40: 33. doi: Crossref MedlineGoogle Scholar

  • 29. Jeong S, Sung J, Kim S, Kim Y, Shin S, Kim SS. Upper airway morphologic changes after mandibular setback surgery in skeletal class III malocclusion patients measured using cone beam computed tomography superimposition. Int J Oral Maxillofac Surg 2018; 47: 1405–10. doi: Crossref Medline ISIGoogle Scholar

  • 30. Li L, Liu H, Cheng H, Han Y, Wang C, Chen Y, et al.. Cbct evaluation of the upper airway morphological changes in growing patients of class II division 1 malocclusion with mandibular retrusion using twin block appliance: a comparative research. PLoS One 2014; 9: e94378. doi: ISIGoogle Scholar

  • 31. Rückschloß T, Ristow O, Berger M, Engel M, Freudlsperger C, Hoffmann J, et al.. Relations between mandible-only advancement surgery, the extent of the posterior airway space, and the position of the hyoid bone in class II patients: a three-dimensional analysis. Br J Oral Maxillofac Surg 2019; 57: 1032–8. doi: Crossref Medline ISIGoogle Scholar

  • 32. Rückschloß T, Ristow O, Jung A, Roser C, Pilz M, Engel M, et al.. The relationship between bimaxillary orthognathic surgery and the extent of posterior airway space in class II and III patients – a retrospective three-dimensional cohort analysis. J Oral Maxillofac Surg Med Pathol 2021; 33: 30–8. doi: Crossref ISIGoogle Scholar

  • 33. Kim H-Y. Statistical notes for clinical researchers: evaluation of measurement error 2: Dahlberg's error, Bland-Altman method, and kappa coefficient. Restor Dent Endod 2013; 38: 182–5. doi: Crossref MedlineGoogle Scholar

  • 34. Zimmerman JN, Lee J, Pliska BT. Reliability of upper pharyngeal airway assessment using dental CBCT: a systematic review. Eur J Orthod 2017; 39: 489–96. doi: Medline ISIGoogle Scholar

  • 35. Souza KRSde, Oltramari-Navarro PVP, Navarro RdeL, Conti ACdeCF, Almeida MRde. Reliability of a method to conduct upper airway analysis in cone-beam computed tomography. Braz Oral Res 2013; 27: 48–54. doi: Crossref Medline ISIGoogle Scholar

  • 36. Guijarro-Martínez R, Swennen GRJ. Three-Dimensional cone beam computed tomography definition of the anatomical subregions of the upper airway: a validation study. Int J Oral Maxillofac Surg 2013; 42: 1140–9. doi: Crossref Medline ISIGoogle Scholar

  • 37. Mattos CT, Cruz CV, da Matta TCS, Pereira LdeA, Solon-de-Mello PdeA, Ruellas ACdeO, et al.. Reliability of upper airway linear, area, and volumetric measurements in cone-beam computed tomography. Am J Orthod Dentofacial Orthop 2014; 145: 188–97. doi: Crossref Medline ISIGoogle Scholar

  • 38. Chen H, van Eijnatten M, Wolff J, de Lange J, van der Stelt PF, Lobbezoo F, et al.. Reliability and accuracy of three imaging software packages used for 3D analysis of the upper airway on cone beam computed tomography images. Dentomaxillofac Radiol 2017; 46: 20170043. doi: Link ISIGoogle Scholar

  • 39. Zimmerman JN, Vora SR, Pliska BT. Reliability of upper airway assessment using CBCT. Eur J Orthod 2019; 41: 101–8. doi: Crossref Medline ISIGoogle Scholar

  • 40. Shujaat S, Jazil O, Willems H, Van Gerven A, Shaheen E, Politis C, et al.. Automatic segmentation of the pharyngeal airway space with convolutional neural network. J Dent 2021; 111: 103705. doi: Crossref Medline ISIGoogle Scholar

  • 41. Dot G, Rafflenbeul F, Arbotto M, Gajny L, Rouch P, Schouman T. Accuracy and reliability of automatic three-dimensional cephalometric landmarking. Int J Oral Maxillofac Surg 2020; 49: 1367–78. doi: Crossref Medline ISIGoogle Scholar

  • 42. Torosdagli N, Liberton DK, Verma P, Sincan M, Lee JS, Bagci U. Deep Geodesic learning for segmentation and anatomical Landmarking. IEEE Trans Med Imaging 2019; 38: 919–31. doi: Crossref Medline ISIGoogle Scholar

  • 43. Zhang J, Liu M, Wang L, Chen S, Yuan P, Li J, et al.. Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization. Med Image Anal 2020; 60: 101621. doi: Crossref ISIGoogle Scholar

  • 44. Gurani SF, Cattaneo PM, Rafaelsen SR, Pedersen MR, Thorn JJ, Pinholt EM. The effect of altered head and tongue posture on upper airway volume based on a validated upper airway analysis-An MRI pilot study. Orthod Craniofac Res 2020; 23: 102–9. doi: Crossref Medline ISIGoogle Scholar

Volume 51, Issue 3March 2022
Supplemental Materials

© 2022 The Authors. Published by the British Institute of Radiology


  • ReceivedMay 25,2021
  • RevisedSeptember 27,2021
  • AcceptedOctober 04,2021
  • Published onlineNovember 08,2021