Free AccessResearch Article

A semi-automatic approach for longitudinal 3D upper airway analysis using voxel-based registration

3D Lab Denmark, Department of Oral and Maxillofacial Surgery, University Hospital of Southern Denmark, Esbjerg, Denmark

Melbourne Dental School, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Victoria, Australia

Search for more papers by this author

and

Else Marie Pinholt

Search for more papers by this author

Published Online:8 Nov 2021https://doi.org/10.1259/dmfr.20210253

Abstract

Objectives:

To propose and validate a reliable semi-automatic approach for three-dimensional (3D) analysis of the upper airway (UA) based on voxel-based registration (VBR).

Methods:

Post-operative cone beam computed tomography (CBCT) scans of 10 orthognathic surgery patients were superimposed to the pre-operative CBCT scans by VBR using the anterior cranial base as reference. Anatomic landmarks were used to automatically cut the UA and calculate volumes and cross-sectional areas (CSA). The 3D analysis was performed by two observers twice, at an interval of two weeks. Intraclass correlations and Bland-Altman plots were used to quantify the measurement error and reliability of the method. The relative Dahlberg error was calculated and compared with a similar method based on landmark re-identification and manual measurements.

Results:

Intraclass correlation coefficient (ICC) showed excellent intra- and inter-observer reliability (ICC ≥ 0.995). Bland-Altman plots showed good observer agreement, low bias and no systematic errors. The relative Dahlberg error ranged between 0.51 and 4.30% for volume and 0.24 and 2.90% for CSA. This was lower when compared with a similar, manual method. Voxel-based registration introduced 0.05–1.44% method error.

Conclusions:

The proposed method was shown to have excellent reliability and high observer agreement. The method is feasible for longitudinal clinical trials on large cohorts due to being semi-automatic.

Introduction

Assessment of the upper airway (UA) using computed tomography (CT), cone beam computed tomography (CBCT), and magnetic resonance imaging (MRI) has developed from two-dimensional (2D) linear measurements and lateral cephalometry to three-dimensional (3D) UA analysis.^1–7 However, although 3D analysis provides additional information, for example, surface and volume measures, the analysis is often time-consuming, and the reliability is compromised by the uncertainty introduced by re-identification of landmarks.^8–10 To encounter these shortcomings of the 3D analysis, superimposition techniques have been introduced for 3D analysis of the postoperative outcome of corrective jaw surgery, and orthognathic surgery.^11–13 These techniques facilitate registration of two consecutive uni- or multimodal scans based on a reference structure. Using mutual image information [voxel-based registration (VBR)], surface similarity [surface-based registration (SBR)], or landmark similarity [landmark-based registration (LBR)], two scans or 3D objects can be superimposed.^14–17 Alignment with VBR has shown higher accuracy,¹⁸ consistency and efficiency than SBR and LBR.^19–21 Hence, VBR is the preferred method of choice for image registration.^19,21,22 For the analysis of the surgical outcome of orthognathic surgery the anterior cranial base, including the zygomatic arches and forehead, structures unaffected by surgery have shown to be reliable references for superimposition¹⁸ and are therefore often used.²²

Although superimposition has shown to be efficient and reliable for skeletal assessment of orthognathic surgery,^11–13 this technique has only been sparsely adopted for UA analysis.^23–32 The majority of these studies used CBCT as the imaging technique, dividing the UA into three segments using cutting planes and measuring the UA volume and CSA. Voxel-based registration was frequently used for superimposition, with the stable region of choice being the cranial base.^{27–29,31,32} Another common superimposition technique was combining LBR with SBR^23–26,30 or with VBR.²⁷

Although the studies showed promising results regarding UA analysis based on superimposition, questions remain regarding the reliability and reproducibility of the methods. Only three studies conducted a reliability study on the methods.^25,26,30 The reliable methods, all similar, from the same research group, use LBR. However, LBR requires manual re-identification of points and has been shown to be less reliable than VBR.²⁰ Furthermore, it is unclear whether cephalometric landmarks were re-identified and whether superimposition solely was used to visualize morphological UA CSA changes in these three studies. None of these studies contain both a reproducible UA analysis method using VBR and a reliability study of the method. Hence, there is a need for a reliable method for integration of VBR in automated UA analysis, avoiding re-identification of landmarks and reducing manual input.

The aim of the present study is to propose and validate a semi-automatic approach for 3D UA analysis based on superimposition. Its reliability is evaluated and directly compared to a similar method not applying superimposition. It is hypothesized that a semi-automatic method for 3D longitudinal UA analysis using VBR will produce reliable and more exact results than a similar method that relies on re-identification of landmarks and manual measurements.

Methods and materials

Permission was granted by the Danish Data Protection Agency (2008-58-0035, 31 August 2015, 18/40631).

The method developed in this study is based on and compared directly to the method developed by Di Carlo et al⁵. The patient dataset from Di Carlo et al⁵, including CBCT scans, 3D segmentations of the airways and anatomical landmarks, has been used during the development of the presented novel method. Consequently, additional descriptions of patient selection, image acquisition, and segmentation can be found in the original protocol.⁵

The automated analysis part of the approach was implemented in Python 3.7 (Python Software Foundation, Fredericksburg, Virginia) and incorporated in Mimics Innovation Suite (Materialise NV, Leuven, Belgium) through its scripting module. Mimics Innovation Suite is comprised of Mimics 23.0 and 3-matic 15.0. An overview of the workflow can be seen in Supplementary Material 1 available online.

Image acquisition and segmentation

The subjects from the Di Carlo et al⁵ cohort are 10 randomly selected patients (seven females) diagnosed with maxillomandibular growth disturbances that underwent orthognathic surgery in 2012 at the University Hospital of Southern Denmark, Department of Oral and Maxillofacial Surgery, Esbjerg, Denmark. Nine patients underwent a combined bilateral mandibular sagittal split osteotomy/maxillary Le Fort I osteotomy (eight with segmental maxillary osteotomies) and one patient underwent a maxillary Le Fort I osteotomy exclusively (with segmental maxillary osteotomies). The pre- and post-surgical CBCT scans, stored in Digital Imaging and Communications in Medicine (DICOM) format, from the dataset have been used. In order to avoid introducing variability from a different segmentation procedure, the segmentation of the UA space used in the Di Carlo et al⁵ UA analysis was maintained.

The selected sample size was calculated by Di Carlo et al⁵ in order to obtain a power equal to 0.99 with an effect size of 1 mm for linear measurements, 1 mm² for surface and 1 mm³ for volumetric and α = 0.5. The exclusion criteria were: syndromes or detectable pathologies involving UA and CBCT scans not including all the craniofacial structures required for the cephalometric analysis. Particular attention was paid to patients’ positioning during CBCT scan: subjects without the mandible at maximum intercuspation were excluded as well as patients wearing a bite during scan acquisition. Patients with inappropriate head positioning, including major head extension, head flexion or head rotation, were excluded as well.⁵

Voxel-based registration

Superimposition of the post-surgical to the pre-surgical scan of each patient was performed using the “Automatic Image Registration” function in Mimics. This function is used to align two different images to the same anatomical coordinate system: a fixed image and a moving image. For this study, the fixed image was the pre-operative CBCT scan and the moving image was the post-operative CBCT scan. After the images were selected, they were manually pre-aligned into approximative positions. Then, a region of interest was specified under the form of a bounding box around the stable anatomical structures of the cranial base, including the zygomatic arches and forehead.¹⁸ The superimposition method was VBR. The registration function generated a transformation matrix that describes the transformation of the moving image toward the fixed image. This transformation matrix was applied to the post-operative CBCT scan, resulting in the post-operative CBCT scan being aligned with the pre-operative CBCT scan at the cranial base (Figure 1). To ensure a suitable registration had been achieved, the result was visually inspected in all views.

Figure 1.
Upper airway analysis. Before voxel-based registration (VBR): pre- and post-surgical CBCTs (upper left), skulls (middle left), airways (lower left); after VBR: CBCTs (middle top), skulls (centre), airways (middle bottom); illustration of relationship between cutting planes, CBCT, and airway objects (upper right); analysed airways overlay (lower right).

Analysis of the upper airway

The landmarks selected for airway analysis followed the description in Di Carlo et al⁵, Table 1. Based on the same criteria, the cephalometric landmarks were placed on the preoperative scan. The axial, coronal, and sagittal views were used to place the landmarks, and additionally the 3D view was used to verify the placement. The airway analysis script was run on the pre- and post-operative UA spaces. The script used the anatomic landmarks to create several planes, as defined by Di Carlo et al⁵: the Frankfurt Horizontal (FH) plane, mid-sagittal plane, sella-nasion (SN) horizontal plane, and the five planes used to cut the airway into five partial volumes (Table 2 and Figure 1). The pre- and superimposed post-operative airways are automatically sectioned at the level of the cutting planes, and the volumes and cross-sectional area (CSA) are automatically calculated and displayed. These measurements are performed according to Di Carlo et al⁵ (Table 3). Running the script also provided an overlay between the pre- and post-operative airway segments to easily be visually inspected (Figure 1). All the measurements were exported to Excel (Microsoft, Redmond, WA). A video summary of the UA analysis procedure is available (Supplementary Video 1). For a more visual understanding of the process, the readers are encouraged to consult the video summary.

**Table 1. Landmarks used in airway analysis**
Landmark	Description
Ba	Basion, the anterior margin of the foramen magnum
N	Nasion, the intersection of the internasal and frontonasal sutures in the midsagittal plane
Or L	Orbitale left, the most inferior anterior point on left orbit’s margin
Or R	Orbitale right, the most inferior anterior point on right orbit’s margin
PoL	Porion left, the most upper point on left bony external auditory meatus
PoR	Porion left, the most upper point on right bony external auditory meatus
S	Sella turcica, the centroid of Sella turcica
So^a	Midpoint of Sella-Basion line
H	Hyoid bone, upper most point of the hyoid bone
E	Epiglottis, tip of epiglottis

aAutomatically calculated in analysis script.

**Table 2. Planes used in airway analysis**
Planes	Description
References planes
Frankfurt plane	A plane passing through the inferior borders of the bony orbits, encompassed by OrR and OrL, and the upper margin of the auditory meatus encompassed by PoR and PoL.
Sagittal SN	Plane perpendicular to Frankfurt plane passing through S and N points
SN horizontal	Plane through S and N points, perpendicular to Sagittal (SN)
S Ba coronal	Plane through S and Ba points perpendicular to Sagittal (SN)
Retropalatal region
Airway superior border 1	Plane passing through So and perpendicular to S-Ba coronal and Sagittal (SN) planes
Airway superior border 2	Plane passing through Ba and parallel to Airway Superior Border 1
Airway superior border 3	Plane passing through Ba and parallel to Frankfurt
Oropharyngeal region
Airway inferior border 1	Plane passing through E and parallel to Frankfurt
Airway inferior border 2	Plane passing through 90% of the distance from Airway Superior Border 1 to Airway Inferior Border 1 and parallel to Frankfurt

**Table 3. Upper airway volumes and cross-sectional measurements**
Measurement	Description
Volumes
Retropalatal	The airway volume encompassed superiorly by airway superior border and inferiorly by airway superior border 2
U-Retropalatal	The airway volume encompassed superiorly by airway superior border and inferiorly by airway superior border 3
L-Retropalatal	The airway volume encompassed superiorly by airway superior border 3 and inferiorly by airway superior border 2
Oropharynx D100	The airway volume encompassed superiorly by airway superior border 2 and inferiorly by airway inferior border 1
Oropharynx D90	The airway volume encompassed superiorly by airway superior border 2 and, inferiorly by airway inferior border 2
Cross-sections
CSA1	Cross-sections at the boundary between upper retropalatal volume and lower retropalatal volume
CSA2	Cross-sections at the boundary between lower retropalatal volume and oropharynx
CSA3	Cross-sections at the boundary between oropharynx D90 vol and oropharynx D100 vol
CSA4	Cross-section located at the bottom of oropharynx D100 vol

Reliability

Two observers (MBH and AD) independently conducted the analysis twice (landmark placement, VBR and semi-automatic analysis), with a two-week interval, in order to evaluate intra- and inter-observer. The second time, the observers were blinded to the first round of analysis. To isolate VBR from the potential error introduced by landmark identification, an additional semi-automatic analysis was conducted where the same landmarks were used across measures, i.e. in the inter-observer case, both observers used the same landmarks, and in the intra-observer case, the same landmarks were used in the first and second measurements. This allowed to isolate error introduced by landmark re-identification from error introduced by the novel aspects of the method (VBR and semi-automatic analysis).

Statistical analysis

Statistical analysis was conducted using STATA v.16.1 (StataCorp, College Station, TX). The data were tested for normality using the Shapiro-Wilk test. For normally distributed data, a paired t-test was used to compare inter- and intra-observer measurements and to report confidence intervals. The intraclass correlation coefficient (ICC) was used to report the intra- and inter-observer reliability. Bland-Altman plots were used to determine the degree of agreement, outliers, bias, and systematic errors. For a direct comparison with the original method by Di Carlo et al⁵, Dahlberg’s formula was used to compare both inter- and intra-observer measurements and the isolated VBR measurements.³³

Results

The Shapiro-Wilk test indicated all the data were normally distributed. Consequently, a paired t-test was performed. The results of the t-test, used to compare inter- and intra-observer measurements, showed narrow confidence intervals and the repeated inter- and intra-observer measurements were not significantly different. Tables 4–7 show these results. Tables 4–7 also contain the mean and standard deviation for the measures, as well as summarize the results from the measures of reliability. ICC showed excellent intra- and inter-observer reliability (ICC ≥ 0.995).

**Table 4. Intra-observer analysis for observer 1 (MBH)**
	Pre-surgical										Post-surgical
								t-test										t-test
	first measurement		second measurement		ICC	Dahlberg’s d		p	95% CI		first measurement		second measurement		ICC	Dahlberg’s d		p	95% CI
	Mean	SD	Mean	SD	ICC	d	%	p	Lower	Upper	Mean	SD	Mean	SD	ICC	d	%	p	Lower	Upper
Retropalatal (mm³)	9649.55	3147.70	9634.62	3173.04	0.9998	49.37	0.51%	0.53	−36.51	66.36	10310.88	3541.29	10273.11	3561.05	0.9997	64.45	0.63%	0.21	−24.78	100.32
U-Retropalatal (mm³)	4012.56	2168.49	4021.74	2151.29	0.9997	39.05	0.97%	0.63	−50.24	31.88	3949.42	2061.14	3926.43	2049.66	0.9993	54.88	1.39%	0.38	−32.91	78.88
L-Retropalatal (mm³)	5636.99	1725.31	5612.88	1769.24	0.9995	37.91	0.67%	0.17	−12.00	60.22	6361.46	2523.66	6346.68	2573.04	0.9986	95.33	1.50%	0.75	−86.26	115.83
Oropharynx D90 (mm³)	7817.54	2547.32	7839.28	2550.76	0.9993	66.28	0.85%	0.49	−90.49	47.02	9822.40	4585.15	9854.78	4559.19	0.9995	106.95	1.09%	0.53	−143.78	79.02
Oropharynx D100 (mm³)	9385.79	3446.10	9410.63	3441.20	0.9996	72.71	0.77%	0.47	−100.09	50.40	11348.18	5433.47	11383.56	5402.67	0.9996	110.47	0.97%	0.50	−150.13	79.36
CSA1 (mm²)	556.40	168.79	555.27	168.97	0.9998	2.07	0.37%	0.24	−0.90	3.16	528.47	180.20	525.45	178.38	0.9995	4.12	0.78%	0.10	−0.74	6.79
CSA2 (mm²)	340.03	93.34	338.65	93.83	0.9995	2.06	0.61%	0.14	−0.55	3.31	422.59	176.44	423.07	177.29	1.0000	1.03	0.24%	0.32	−1.52	0.56
CSA3 (mm²)	241.73	137.13	242.22	136.48	0.9999	1.00	0.42%	0.30	−1.49	0.52	262.05	150.09	262.75	150.89	0.9998	1.94	0.74%	0.44	−2.70	1.29
CSA4 (mm²)	265.82	141.22	267.16	142.30	0.9995	3.30	1.24%	0.39	−4.71	2.02	225.40	107.36	226.95	110.49	0.9989	3.69	1.64%	0.38	−5.31	2.22

**Table 5. Intra-observer analysis for observer 2 (AD)**
	Pre-surgical										Post-surgical
								t-test										t-test
	first measurement		second measurement		ICC	Dahlberg’s d		p	95% CI		first measurement		second measurement		ICC	Dahlberg’s d		p	95% CI
	Mean	SD	Mean	SD	ICC	d	%	p	Lower	Upper	Mean	SD	Mean	SD	ICC	d	%	p	Lower	Upper
Retropalatal (mm³)	9655.32	3203.20	9562.83	3097.24	0.9978	148.23	1.54%	0.17	−49.38	234.34	10324.68	3585.41	10277.12	3587.70	0.9997	64.12	0.62%	0.10	−10.66	105.78
U-Retropalatal (mm³)	3986.85	2265.79	3941.84	2131.02	0.9950	155.48	3.90%	0.55	−117.29	207.30	3871.60	2098.38	3882.26	2072.80	0.9974	107.18	2.77%	0.84	−124.67	103.35
L-Retropalatal (mm³)	5668.46	1758.93	5620.99	1715.52	0.9966	100.81	1.78%	0.32	−53.89	148.84	6453.08	2665.46	6394.86	2596.70	0.9978	122.16	1.89%	0.31	−64.43	180.88
Oropharynx D90 (mm³)	7753.77	2514.42	7784.69	2593.02	0.9982	106.89	1.38%	0.55	−142.50	80.65	9729.14	4507.82	9758.71	4583.35	0.9993	119.22	1.23%	0.61	−154.73	95.59
Oropharynx D100 (mm³)	9315.55	3406.12	9347.09	3495.17	0.9989	116.47	1.25%	0.57	−153.45	90.35	11259.39	5372.77	11280.79	5440.58	0.9995	121.40	1.08%	0.72	−149.85	107.06
CSA1 (mm²)	547.93	173.89	554.96	175.19	0.9946	12.78	2.33%	0.24	−19.58	5.53	520.72	182.70	524.52	187.84	0.9967	10.69	2.05%	0.46	−14.83	7.23
CSA2 (mm²)	340.99	93.14	340.95	95.09	0.9995	2.04	0.60%	0.97	−2.14	2.22	420.83	173.00	421.09	175.28	0.9997	3.21	0.76%	0.87	−3.68	3.17
CSA3 (mm²)	243.08	138.17	242.80	138.82	0.9998	2.01	0.83%	0.78	−1.86	2.40	260.95	148.96	262.11	149.12	0.9989	5.03	1.93%	0.63	−6.45	4.13
CSA4 (mm²)	269.17	144.54	266.69	141.48	0.9987	5.22	1.94%	0.31	−2.78	7.72	230.08	112.62	226.93	108.18	0.9970	6.09	2.65%	0.27	−2.90	9.19

**Table 6. Inter-observer analysis, pre-surgical**
	Pre-surgical
	Observer 1 vs observer 2 at the first measurement						Observer 1 vs observer 2 at the second measurement
	t-test			Dahlberg’s d		ICC	t-test			Dahlberg’s d		ICC
	p	95% CI		Dahlberg’s d			p	95% CI		Dahlberg’s d
	p	Lower	Upper	d	%		p	Lower	Upper	d	%
Retropalatal (mm³)	0.84	−68.51	56.99	58.99	0.61%	0.9997	0.60	−71.56	43.96	123.33	1.28%	0.9985
U-Retropalatal (mm³)	0.68	−109.79	161.20	128.35	3.20%	0.9966	0.23	−58.67	214.32	172.85	4.30%	0.9935
L-Retropalatal (mm³)	0.47	−126.10	63.16	91.49	1.62%	0.9972	0.17	−230.99	47.73	126.97	2.26%	0.9947
Oropharynx D90 (mm³)	0.17	−32.61	160.16	101.01	1.29%	0.9984	0.10	−21.30	207.81	114.34	1.46%	0.9980
Oropharynx D100 (mm³)	0.15	−29.64	170.12	106.02	1.13%	0.9990	0.11	−25.14	202.71	127.06	1.35%	0.9987
CSA1 (mm²)	0.11	−2.22	19.16	11.68	2.10%	0.9954	0.19	−4.62	20.13	15.26	2.75%	0.9921
CSA2 (mm²)	0.46	−3.77	1.86	2.72	0.80%	0.9991	0.42	−2.91	6.42	3.20	0.94%	0.9989
CSA3 (mm²)	0.04	−2.62	−0.07	1.53	0.63%	0.9999	0.63	−3.91	6.09	2.38	0.98%	0.9997
CSA4 (mm²)	0.14	−8.07	1.36	5.02	1.89%	0.9988	0.11	−10.69	1.34	3.73	1.40%	0.9993

**Table 7. Inter-observer analysis, post-surgical**
	Post-surgical
	Observer 1 vs observer 2 at the first measurement						Observer 1 vs observer 2 at the second measurement
	t-test			Dahlberg’s d		ICC	t-test			Dahlberg’s d		ICC
	p	95% CI		Dahlberg’s d			p	95% CI		Dahlberg’s d
	p	Lower	Upper	d	%		p	Lower	Upper	d	%
Retropalatal (mm³)	0.21	−48.06	191.65	55.04	0.53%	0.9998	0.94	−112.57	104.55	101.84	0.99%	0.9992
U-Retropalatal (mm³)	0.33	−94.31	254.10	139.33	3.53%	0.9955	0.57	−127.52	215.86	164.00	4.15%	0.9937
L-Retropalatal (mm³)	0.90	−143.37	127.16	145.86	2.29%	0.9968	0.63	−269.89	173.51	210.67	3.31%	0.9934
Oropharynx D90 (mm³)	0.31	−60.18	169.36	126.05	1.28%	0.9992	0.32	−111.76	303.89	206.39	2.10%	0.9980
Oropharynx D100 (mm³)	0.29	−63.20	190.28	123.91	1.09%	0.9995	0.31	−111.10	316.65	213.32	1.88%	0.9985
CSA1 (mm²)	0.97	−15.96	16.58	12.83	2.43%	0.9950	0.88	−12.78	14.66	12.88	2.44%	0.9951
CSA2 (mm²)	0.11	−5.23	0.64	4.55	1.08%	0.9993	0.48	−4.04	8.01	5.82	1.38%	0.9989
CSA3 (mm²)	0.61	−3.09	1.92	4.75	1.81%	0.9990	0.62	−2.17	3.46	2.68	1.02%	0.9997
CSA4 (mm²)	0.80	−3.50	4.43	6.54	2.90%	0.9965	1.00	−4.34	4.36	4.08	1.81%	0.9986

For comparison with the manual approach by Di Carlo et al⁵, the Dahlberg error was calculated. Figure 2 shows the pre-surgical measurement errors for the intra- and inter-observer measurements. Figure 3 shows the post-surgical measurement errors for the intra- and inter-observer measurements. The measurement error of the observers from the present study (AD and MBH) is lower across the board when compared to the measurement error of the observers from the Di Carlo et al⁵ study (GDC and SFG). The method error from the present study ranged between 0.51 and 4.30% for volumetric measurements and between 0.24 and 2.90% for CSA measurements. In comparison, the error from the Di Carlo et al⁵ study ranges between 1.6 and 10.2% for volume and between 2 and 12.2% for CSA. Figure 3 also shows the part of the measurement error which is introduced by the user input required for VBR (pre-alignment and selection of volume of interest), with the error for the intra- and inter-observer measurements ranging between 0.05% for pre-operative measurements and up to 1.44% for post-operative measurements.

Figure 3.
Post-surgical measurement error, intra-observer (top) and inter-observer (bottom). GDC & SFG (observers from Di Carlo et al. 2017), MBH & AD (observers from present study); ^a Isolated VBR: only VBR and automatic analysis were performed for repeated measures (the first set of landmarks was also used in the second measurement). ^b Isolated VBR: only VBR performed by different observers.

Discussion

The purpose of the present study was to propose and validate a semi-automatic approach for 3D UA analysis based on VBR. The semi-automatic method for 3D UA analysis using VBR produced more reliable results than a similar method relying on re-identification of landmarks and manual measurements.

The ICC values obtained for the present method were 0.995 or higher, showing excellent to near perfect reliability. The Bland-Altman plots indicated no systematic error and low bias. Method error calculated with the Dahlberg formula was used for comparison with Di Carlo et al⁵. The Relative Dahlberg error was calculated because it allowed for comparison of measurements with different units and means.³³ The measurements performed were volume (mm³) and CSA (mm²). The novel method with VBR and semi-automatic analysis has lower relative Dahlberg error (0.24–4.30%) than the method by Di Carlo et al⁵ (1.6–12.2%), which re-identifies landmarks and performs measurements manually (Figures 2 and 3). Paired t-tests resulted in narrow confidence intervals when compared with Di Carlo et al⁵ indicating greater reliability for the novel method.

The method error of the VBR and semi-automatic analysis was isolated using the same set of landmarks for all inter- and intra-observer measurements. The results indicate VBR introduces a very small portion of variability. There is 0.0% variability (relative Dahlberg error) in the pre-surgical measurements of the isolated VBR procedure. That is to be expected, as the same landmarks were used for both intra- and inter-observer measurement, and superimposition does not play a role in the pre-surgical measurements. Therefore, the analysis script does not introduce any variability. This is the intended purpose of automation: to avoid error from manual user input. The post-surgical measurement error ranged between 0.05 and 1.44% (Figure 3). Since the landmarks were identical, and it has been established that the analysis script does not introduce variability, this error range is therefore attributed to VBR. Voxel-based registration requires user input in pre-alignment and defining the region of interest.

The method can be used with any other set of cephalometric landmarks and cutting planes. Hence, other protocols for UA analysis than the one proposed by Di Carlo et al⁵ can be integrated in the semi-automatic analysis using voxel-based registration. This provides the flexibility to include other airway measurements, for example, airway length and shape.

In a systematic review by Zimmerman et al³⁴, 42 studies were reviewed to evaluate the reliability of upper pharyngeal airway assessment using CBCT. The UA analysis methods demonstrated moderate to excellent intra- and inter-observer reliability. However, only five studies were deemed high quality, and only three of these studies assessed both intra- and inter-observer reliability. From the high-quality studies, airway volume measurements demonstrated greater intra- and inter-observer reliability (>0.880) than CSA measurements (>0.696). In comparison, the proposed semi-automatic method demonstrated near to perfect intra- and inter-observer reliability (>0.995) for both volume- and CSA measurements. The three high-quality studies which assessed intra- and inter-observer reliability all applied the Dolphin Imaging^® software (Dolphin Imaging and Management Systems, Chatsworth, CA, USA) for UA analysis. Of these studies, the study by De Souza et al³⁵ reported good reliability (>0.88); The study by Guijarro-Martinez,³⁶ reported excellent reliability (>0.981) and observer error up to 4% for volume measurements, but only moderate reliability (>0.780) and observer error up to 20% for CSA measurements. The study by Mattos et al³⁷ showed that the transversal width- and CSA measurements at the level of the vallecula had only moderate reliability (>0.63) with a mean observer difference of 16.1 and 13.3%, respectively. The proposed method, which adopts the UA definition by Di Carlo et al⁵, inferiorly limits the UA at Epiglottis, which may be more reliable, and automatically transfers the cutting planes to the post-operative CBCT, resulting in near to perfect reliability and low observer error for CSA measurements.

Recently, Chen et al³⁸ assessed the reliability and accuracy of three different imaging software packages, Amira^® (Visage Imaging Inc., Carlsbad, CA), 3Diagnosys^® (3diemme, Cantu, Italy) and OnDemand3D^® (CyberMed, Seoul, Republic of Korea) for three-dimensional analysis of the upper airway using CBCT images. The intra- and inter-observer reliability of the measurements using the different software packages were moderate to excellent (>0.78). All three software packages underestimated the upper airway volume by −8.8 to −12.3%, the minimum CSA by −6.2 to− 14.6% and the length by −1.6 to −2.9%.

In a recent study, Zimmerman et al³⁹ presented the first study to evaluate the reliability of UA assessment using CBCT, where the observers performed each step of the analysis manually. The authors argue that this is the way UA assessment would be conducted in a clinical setting today. Intra- and inter-observer reliability of each step in the UA assessment were evaluated by six observers of varying levels of education and clinical experience. Threshold selection in the UA segmentation process showed poor intra- (0.473) and poor inter-observer reliability (0.100); CSA showed moderate intra- (0.591) and poor inter-observer reliability (0.223); Intra-observer reliability of volume measurements varied based on the region assessed (0.747–0.976) and was worst for hypopharynx and best for the oropharynx; inter-observer reliability of volume measurements was generally lower (0.175–0.945) and was worst for nasopharynx and best for the oropharynx. These results are of special interest for comparison with the present study, which in contrast to a fully manual UA analysis evaluated by Zimmerman et al³⁹, is optimised to reduce observer error by automation of sub-processes. The great improvement in the intra- and inter-observer reliability clearly demonstrates the advantages of automation, which should be more generally adopted in clinical practice.

Although the proposed approach minimizes user input for the UA analysis and omits landmark re-identification, manual precursory steps still have to be taken, i.e. the segmentation of the UA and the initial placement of landmarks. Recent developments in the field automatic segmentation⁴⁰ and landmark placement have been made using artificial intelligence, achieving results within clinically acceptable margins for landmark placement (<2 mm).^41–43 These state of the art methods may complement the proposed UA analysis approach, making a fully automatic UA analysis workflow possible.

A noteworthy mention is that no measures have been taken to address the variability in head posture, which is known to affect the UA.⁴⁴ This issue is inherent to the image acquisition process, and a general challenge of measuring soft tissue response before and after surgery. The logical goal would be to correlate these cross-sectional and volumetric measures with polysomnography findings. The airway changes associated with orthognathic surgery could provide functional justification for this procedure provided the demonstrable postoperative airway changes correlate with improved function are confirmed by polysomnography.

Conclusion

The proposed method was shown to be reliable. A lack of reliable methods for UA analysis that capitalize on the benefits of VBR in the literature is emphasized. When compared to a similar method that involves manual input and landmark re-identification, the semi-automatic approach using VBR shows less inter- and intra-observer measurement error. The proposed approach has great applicability for UA analysis in a clinical trial, due to high reliability. This is further made feasible by the automated part of the analysis, making the novel method efficient and suitable for longitudinal studies on large cohorts.

Alexandru Diaconu and Michael Boelstoft Holte have contributed equally to this study and should be considered as co-first authors.

REFERENCES

1. Abramson Z, Susarla SM, Lawler M, Bouchard C, Troulis M, Kaban LB. Three-Dimensional computed tomographic airway analysis of patients with obstructive sleep apnea treated by maxillomandibular advancement. J Oral Maxillofac Surg 2011; 69: 677–86. doi: https://doi.org/10.1016/j.joms.2010.11.037 http://www.ncbi.nlm.nih.gov/pubmed/21353929 Crossref Medline ISI, Google Scholar
2. Bianchi A, Betti E, Tarsitano A, Morselli-Labate AM, Lancellotti L, Marchetti C. Volumetric three-dimensional computed tomographic evaluation of the upper airway in patients with obstructive sleep apnoea syndrome treated by maxillomandibular advancement. Br J Oral Maxillofac Surg 2014; 52: 831–7. doi: https://doi.org/10.1016/j.bjoms.2014.07.101 http://www.ncbi.nlm.nih.gov/pubmed/25129655 Crossref Medline ISI, Google Scholar
3. Butterfield KJ, Marks PLG, McLean L, Newton J. Pharyngeal airway morphology in healthy individuals and in obstructive sleep apnea patients treated with maxillomandibular advancement: a comparative study. Oral Surg Oral Med Oral Pathol Oral Radiol 2015; 119: 285–92. doi: https://doi.org/10.1016/j.oooo.2014.11.016 http://www.ncbi.nlm.nih.gov/pubmed/25592868 Crossref Medline ISI, Google Scholar
4. Christovam IO, Lisboa CO, Ferreira DMTP, Cury-Saramago AA, Mattos CT. Upper airway dimensions in patients undergoing orthognathic surgery: a systematic review and meta-analysis. Int J Oral Maxillofac Surg 2016; 45: 460–71. doi: https://doi.org/10.1016/j.ijom.2015.10.018 http://www.ncbi.nlm.nih.gov/pubmed/26616028 Crossref Medline ISI, Google Scholar
5. Di Carlo G, Gurani SF, Pinholt EM, Cattaneo PM. A new simple three-dimensional method to characterize upper airway in orthognathic surgery patient. Dentomaxillofac Radiol 2017; 46: 20170042. doi: https://doi.org/10.1259/dmfr.20170042 http://www.ncbi.nlm.nih.gov/pubmed/28749703 Link ISI, Google Scholar
6. Fernández-Ferrer L, Montiel-Company JM, Pinho T, Almerich-Silla JM, Bellot-Arcís C. Effects of mandibular setback surgery on upper airway dimensions and their influence on obstructive sleep apnoea - a systematic review. J Craniomaxillofac Surg 2015; 43: 248–53. doi: https://doi.org/10.1016/j.jcms.2014.11.017 http://www.ncbi.nlm.nih.gov/pubmed/25547214 Crossref Medline ISI, Google Scholar
7. Gurani SF, Di Carlo G, Thorn JJ, Ingerslev J, Cattaneo PM, Pinholt EM. Two-Year postoperative upper airway cone-beam computed tomographic outcomes based on a verified upper airway analysis following bimaxillary Orthognathic surgery. J Oral Maxillofac Surg 2019; 77: 1435–45. doi: https://doi.org/10.1016/j.joms.2019.02.038 http://www.ncbi.nlm.nih.gov/pubmed/30926544 Crossref Medline ISI, Google Scholar
8. Katkar RA, Kummet C, Dawson D, Moreno Uribe L, Allareddy V, Finkelstein M, et al.. Comparison of observer reliability of three-dimensional cephalometric landmark identification on subject images from Galileos and i-CAT cone beam CT. Dentomaxillofac Radiol 2013; 42: 20130059. doi: https://doi.org/10.1259/dmfr.20130059 http://www.ncbi.nlm.nih.gov/pubmed/23833319 Link ISI, Google Scholar
9. Titiz I, Laubinger M, Keller T, Hertrich K, Hirschfelder U. Repeatability and reproducibility of landmarks--a three-dimensional computed tomography study. Eur J Orthod 2012; 34: 276–86. doi: https://doi.org/10.1093/ejo/cjq190 http://www.ncbi.nlm.nih.gov/pubmed/21566086 Crossref Medline ISI, Google Scholar
10. Xi T, van Luijn R, Baan F, Schreurs R, de Koning M, Bergé S, et al.. Landmark-Based versus voxel-based 3-dimensional quantitative analysis of bimaxillary osteotomies: a comparative study. J Oral Maxillofac Surg 2020; 78: 468.e1–468.e10. doi: https://doi.org/10.1016/j.joms.2019.10.019 http://www.ncbi.nlm.nih.gov/pubmed/31785251 Crossref ISI, Google Scholar
11. Baan F, Sabelis JF, Schreurs R, van de Steeg G, Xi T, van Riet TCT, et al.. Validation of the OrthoGnathicAnalyser 2.0-3D accuracy assessment tool for bimaxillary surgery and genioplasty. PLoS One 2021; 16: e0246196. doi: https://doi.org/10.1371/journal.pone.0246196 http://www.ncbi.nlm.nih.gov/pubmed/33497422 Crossref ISI, Google Scholar
12. Holte MB, Diaconu A, Ingerslev J, Thorn JJ, Pinholt EM, et al.. Virtual analysis of segmental bimaxillary surgery: a validation study. J Oral Maxillofac Surg 2021; 10:S0278-2391(21)00540-1. doi: https://doi.org/10.1016/j.joms.2021.06.003 34245697 Medline, Google Scholar
13. Stokbro K, Thygesen T. A 3-dimensional approach for analysis in Orthognathic Surgery-Using free software for voxel-based alignment and Semiautomatic measurement. J Oral Maxillofac Surg 2018; 76: 1316–26. doi: https://doi.org/10.1016/j.joms.2017.11.010 http://www.ncbi.nlm.nih.gov/pubmed/29241643 Crossref Medline ISI, Google Scholar
14. Cevidanes LHS, Styner MA, Proffit WR. Image analysis and superimposition of 3-dimensional cone-beam computed tomography models. Am J Orthod Dentofacial Orthop 2006; 129: 611–8. doi: https://doi.org/10.1016/j.ajodo.2005.12.008 http://www.ncbi.nlm.nih.gov/pubmed/16679201 Crossref Medline ISI, Google Scholar
15. Koerich L, Burns D, Weissheimer A, Claus JDP. Three-Dimensional maxillary and mandibular regional superimposition using cone beam computed tomography: a validation study. Int J Oral Maxillofac Surg 2016; 45: 662–9. doi: https://doi.org/10.1016/j.ijom.2015.12.006 http://www.ncbi.nlm.nih.gov/pubmed/26794399 Crossref Medline ISI, Google Scholar
16. Lee J-H, Kim M-J, Kim S-M, Kwon O-H, Kim Y-K. The 3D CT superimposition method using image fusion based on the maximum mutual information algorithm for the assessment of oral and maxillofacial surgery treatment results. Oral Surg Oral Med Oral Pathol Oral Radiol 2012; 114: 167–74. doi: https://doi.org/10.1016/j.tripleo.2011.06.003 http://www.ncbi.nlm.nih.gov/pubmed/22776729 Crossref Medline ISI, Google Scholar
17. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P. Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 1997; 16: 187–98. doi: https://doi.org/10.1109/42.563664 http://www.ncbi.nlm.nih.gov/pubmed/9101328 Crossref Medline ISI, Google Scholar
18. Nada RM, Maal TJJ, Breuning KH, Bergé SJ, Mostafa YA, Kuijpers-Jagtman AM. Accuracy and reproducibility of voxel based superimposition of cone beam computed tomography models on the anterior cranial base and the Zygomatic arches. PLoS One 2011; 6: e16520. doi: https://doi.org/10.1371/journal.pone.0016520 http://www.ncbi.nlm.nih.gov/pubmed/21347419 Crossref ISI, Google Scholar
19. Almukhtar A, Ju X, Khambay B, McDonald J, Ayoub A. Comparison of the accuracy of voxel based registration and surface based registration for 3D assessment of surgical change following orthognathic surgery. PLoS One 2014; 9: e93402. doi: https://doi.org/10.1371/journal.pone.0093402 http://www.ncbi.nlm.nih.gov/pubmed/24695577 Crossref Medline ISI, Google Scholar
20. Ghoneima A, Cho H, Farouk K, Kula K. Accuracy and reliability of landmark-based, surface-based and voxel-based 3D cone-beam computed tomography superimposition methods. Orthod Craniofac Res 2017; 20: 227–36. doi: https://doi.org/10.1111/ocr.12205 http://www.ncbi.nlm.nih.gov/pubmed/28960842 Crossref Medline ISI, Google Scholar
21. Han G, Li J, Wang S, Wang L, Zhou Y, Liu Y. A comparison of voxel- and surface-based cone-beam computed tomography mandibular superimposition in adult orthodontic patients. J Int Med Res 2021; 49: 030006052098270. doi: https://doi.org/10.1177/0300060520982708 http://www.ncbi.nlm.nih.gov/pubmed/33459090 Crossref ISI, Google Scholar
22. Gaber RM, Shaheen E, Falter B, Araya S, Politis C, Swennen GRJ, et al.. A systematic review to uncover a universal protocol for accuracy assessment of 3-dimensional virtually planned Orthognathic surgery. J Oral Maxillofac Surg 2017; 75: 2430–40. doi: https://doi.org/10.1016/j.joms.2017.05.025 http://www.ncbi.nlm.nih.gov/pubmed/28646644 Crossref Medline ISI, Google Scholar
23. Alsufyani NA, Dietrich NH, Lagravère MO, Carey JP, Major PW. Cone beam computed tomography registration for 3-D airway analysis based on anatomic landmarks. Oral Surg Oral Med Oral Pathol Oral Radiol 2014; 118: 371–83. doi: https://doi.org/10.1016/j.oooo.2014.05.027 http://www.ncbi.nlm.nih.gov/pubmed/25151592 Crossref Medline ISI, Google Scholar
24. Alsufyani NA, Noga ML, Witmans M, Cheng I, El-Hakim H, Major PW. Using cone beam CT to assess the upper airway after surgery in children with sleep disordered breathing symptoms and maxillary-mandibular disproportions: a clinical pilot. J Otolaryngol Head Neck Surg 2017; 46: 31. doi: https://doi.org/10.1186/s40463-017-0204-4 http://www.ncbi.nlm.nih.gov/pubmed/28399908 Crossref Medline ISI, Google Scholar
25. Chen X, Liu D, Liu J, Wu Z, Xie Y, Li L, et al.. Three-Dimensional evaluation of the upper airway morphological changes in growing patients with skeletal class III malocclusion treated by protraction headgear and rapid palatal expansion: a comparative research. PLoS One 2015; 10: e0135273. doi: https://doi.org/10.1371/journal.pone.0135273 http://www.ncbi.nlm.nih.gov/pubmed/26252015 ISI, Google Scholar
26. Chen Y, Hong L, Wang C-ling, Zhang S-jie, Cao C, Wei F, et al.. Effect of large incisor retraction on upper airway morphology in adult bimaxillary protrusion patients. Angle Orthod 2012; 82: 964–70. doi: https://doi.org/10.2319/110211-675.1 http://www.ncbi.nlm.nih.gov/pubmed/22462464 Crossref Medline ISI, Google Scholar
27. Giralt-Hernando M, Valls-Ontañón A, Haas Junior OL, Masià-Gridilla J, Hernández-Alfaro F. What are the surgical movements in Orthognathic surgery that most affect the upper airways? a three-dimensional analysis. J Oral Maxillofac Surg 2021; 79: 450–62. doi: https://doi.org/10.1016/j.joms.2020.10.017 http://www.ncbi.nlm.nih.gov/pubmed/33171114 Crossref Medline ISI, Google Scholar
28. Jang S-I, Ahn J, Paeng JY, Hong J. Three-Dimensional analysis of changes in airway space after bimaxillary orthognathic surgery with maxillomandibular setback and their association with obstructive sleep apnea. Maxillofac Plast Reconstr Surg 2018; 40: 33. doi: https://doi.org/10.1186/s40902-018-0171-3 http://www.ncbi.nlm.nih.gov/pubmed/30474021 Crossref Medline, Google Scholar
29. Jeong S, Sung J, Kim S, Kim Y, Shin S, Kim SS. Upper airway morphologic changes after mandibular setback surgery in skeletal class III malocclusion patients measured using cone beam computed tomography superimposition. Int J Oral Maxillofac Surg 2018; 47: 1405–10. doi: https://doi.org/10.1016/j.ijom.2018.05.002 http://www.ncbi.nlm.nih.gov/pubmed/29859651 Crossref Medline ISI, Google Scholar
30. Li L, Liu H, Cheng H, Han Y, Wang C, Chen Y, et al.. Cbct evaluation of the upper airway morphological changes in growing patients of class II division 1 malocclusion with mandibular retrusion using twin block appliance: a comparative research. PLoS One 2014; 9: e94378. doi: https://doi.org/10.1371/journal.pone.0094378 http://www.ncbi.nlm.nih.gov/pubmed/24705466 ISI, Google Scholar
31. Rückschloß T, Ristow O, Berger M, Engel M, Freudlsperger C, Hoffmann J, et al.. Relations between mandible-only advancement surgery, the extent of the posterior airway space, and the position of the hyoid bone in class II patients: a three-dimensional analysis. Br J Oral Maxillofac Surg 2019; 57: 1032–8. doi: https://doi.org/10.1016/j.bjoms.2019.09.001 http://www.ncbi.nlm.nih.gov/pubmed/31563482 Crossref Medline ISI, Google Scholar
32. Rückschloß T, Ristow O, Jung A, Roser C, Pilz M, Engel M, et al.. The relationship between bimaxillary orthognathic surgery and the extent of posterior airway space in class II and III patients – a retrospective three-dimensional cohort analysis. J Oral Maxillofac Surg Med Pathol 2021; 33: 30–8. doi: https://doi.org/10.1016/j.ajoms.2020.07.009 Crossref ISI, Google Scholar
33. Kim H-Y. Statistical notes for clinical researchers: evaluation of measurement error 2: Dahlberg's error, Bland-Altman method, and kappa coefficient. Restor Dent Endod 2013; 38: 182–5. doi: https://doi.org/10.5395/rde.2013.38.3.182 http://www.ncbi.nlm.nih.gov/pubmed/24010087 Crossref Medline, Google Scholar
34. Zimmerman JN, Lee J, Pliska BT. Reliability of upper pharyngeal airway assessment using dental CBCT: a systematic review. Eur J Orthod 2017; 39: 489–96. doi: https://doi.org/10.1093/ejo/cjw079 http://www.ncbi.nlm.nih.gov/pubmed/27999121 Medline ISI, Google Scholar
35. Souza KRSde, Oltramari-Navarro PVP, Navarro RdeL, Conti ACdeCF, Almeida MRde. Reliability of a method to conduct upper airway analysis in cone-beam computed tomography. Braz Oral Res 2013; 27: 48–54. doi: https://doi.org/10.1590/S1806-83242013000100009 http://www.ncbi.nlm.nih.gov/pubmed/23306626 Crossref Medline ISI, Google Scholar
36. Guijarro-Martínez R, Swennen GRJ. Three-Dimensional cone beam computed tomography definition of the anatomical subregions of the upper airway: a validation study. Int J Oral Maxillofac Surg 2013; 42: 1140–9. doi: https://doi.org/10.1016/j.ijom.2013.03.007 http://www.ncbi.nlm.nih.gov/pubmed/23623785 Crossref Medline ISI, Google Scholar
37. Mattos CT, Cruz CV, da Matta TCS, Pereira LdeA, Solon-de-Mello PdeA, Ruellas ACdeO, et al.. Reliability of upper airway linear, area, and volumetric measurements in cone-beam computed tomography. Am J Orthod Dentofacial Orthop 2014; 145: 188–97. doi: https://doi.org/10.1016/j.ajodo.2013.10.013 http://www.ncbi.nlm.nih.gov/pubmed/24485733 Crossref Medline ISI, Google Scholar
38. Chen H, van Eijnatten M, Wolff J, de Lange J, van der Stelt PF, Lobbezoo F, et al.. Reliability and accuracy of three imaging software packages used for 3D analysis of the upper airway on cone beam computed tomography images. Dentomaxillofac Radiol 2017; 46: 20170043. doi: https://doi.org/10.1259/dmfr.20170043 http://www.ncbi.nlm.nih.gov/pubmed/28467118 Link ISI, Google Scholar
39. Zimmerman JN, Vora SR, Pliska BT. Reliability of upper airway assessment using CBCT. Eur J Orthod 2019; 41: 101–8. doi: https://doi.org/10.1093/ejo/cjy058 http://www.ncbi.nlm.nih.gov/pubmed/30184085 Crossref Medline ISI, Google Scholar
40. Shujaat S, Jazil O, Willems H, Van Gerven A, Shaheen E, Politis C, et al.. Automatic segmentation of the pharyngeal airway space with convolutional neural network. J Dent 2021; 111: 103705. doi: https://doi.org/10.1016/j.jdent.2021.103705 Crossref Medline ISI, Google Scholar
41. Dot G, Rafflenbeul F, Arbotto M, Gajny L, Rouch P, Schouman T. Accuracy and reliability of automatic three-dimensional cephalometric landmarking. Int J Oral Maxillofac Surg 2020; 49: 1367–78. doi: https://doi.org/10.1016/j.ijom.2020.02.015 http://www.ncbi.nlm.nih.gov/pubmed/32169306 Crossref Medline ISI, Google Scholar
42. Torosdagli N, Liberton DK, Verma P, Sincan M, Lee JS, Bagci U. Deep Geodesic learning for segmentation and anatomical Landmarking. IEEE Trans Med Imaging 2019; 38: 919–31. doi: https://doi.org/10.1109/TMI.2018.2875814 http://www.ncbi.nlm.nih.gov/pubmed/30334750 Crossref Medline ISI, Google Scholar
43. Zhang J, Liu M, Wang L, Chen S, Yuan P, Li J, et al.. Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization. Med Image Anal 2020; 60: 101621. doi: https://doi.org/10.1016/j.media.2019.101621 http://www.ncbi.nlm.nih.gov/pubmed/31816592 Crossref ISI, Google Scholar
44. Gurani SF, Cattaneo PM, Rafaelsen SR, Pedersen MR, Thorn JJ, Pinholt EM. The effect of altered head and tongue posture on upper airway volume based on a validated upper airway analysis-An MRI pilot study. Orthod Craniofac Res 2020; 23: 102–9. doi: https://doi.org/10.1111/ocr.12348 http://www.ncbi.nlm.nih.gov/pubmed/31550076 Crossref Medline ISI, Google Scholar

Volume 51, Issue 3March 2022

Supplemental Materials

History

ReceivedMay 25,2021
RevisedSeptember 27,2021
AcceptedOctober 04,2021
Published onlineNovember 08,2021

Metrics

Keywords

Download PDF

Verify Phone