A semi-automatic approach for longitudinal 3D upper airway analysis using voxel-based registration
To propose and validate a reliable semi-automatic approach for three-dimensional (3D) analysis of the upper airway (UA) based on voxel-based registration (VBR).
Post-operative cone beam computed tomography (CBCT) scans of 10 orthognathic surgery patients were superimposed to the pre-operative CBCT scans by VBR using the anterior cranial base as reference. Anatomic landmarks were used to automatically cut the UA and calculate volumes and cross-sectional areas (CSA). The 3D analysis was performed by two observers twice, at an interval of two weeks. Intraclass correlations and Bland-Altman plots were used to quantify the measurement error and reliability of the method. The relative Dahlberg error was calculated and compared with a similar method based on landmark re-identification and manual measurements.
Intraclass correlation coefficient (ICC) showed excellent intra- and inter-observer reliability (ICC ≥ 0.995). Bland-Altman plots showed good observer agreement, low bias and no systematic errors. The relative Dahlberg error ranged between 0.51 and 4.30% for volume and 0.24 and 2.90% for CSA. This was lower when compared with a similar, manual method. Voxel-based registration introduced 0.05–1.44% method error.
The proposed method was shown to have excellent reliability and high observer agreement. The method is feasible for longitudinal clinical trials on large cohorts due to being semi-automatic.
Assessment of the upper airway (UA) using computed tomography (CT), cone beam computed tomography (CBCT), and magnetic resonance imaging (MRI) has developed from two-dimensional (2D) linear measurements and lateral cephalometry to three-dimensional (3D) UA analysis.1–7 However, although 3D analysis provides additional information, for example, surface and volume measures, the analysis is often time-consuming, and the reliability is compromised by the uncertainty introduced by re-identification of landmarks.8–10 To encounter these shortcomings of the 3D analysis, superimposition techniques have been introduced for 3D analysis of the postoperative outcome of corrective jaw surgery, and orthognathic surgery.11–13 These techniques facilitate registration of two consecutive uni- or multimodal scans based on a reference structure. Using mutual image information [voxel-based registration (VBR)], surface similarity [surface-based registration (SBR)], or landmark similarity [landmark-based registration (LBR)], two scans or 3D objects can be superimposed.14–17 Alignment with VBR has shown higher accuracy,18 consistency and efficiency than SBR and LBR.19–21 Hence, VBR is the preferred method of choice for image registration.19,21,22 For the analysis of the surgical outcome of orthognathic surgery the anterior cranial base, including the zygomatic arches and forehead, structures unaffected by surgery have shown to be reliable references for superimposition18 and are therefore often used.22
Although superimposition has shown to be efficient and reliable for skeletal assessment of orthognathic surgery,11–13 this technique has only been sparsely adopted for UA analysis.23–32 The majority of these studies used CBCT as the imaging technique, dividing the UA into three segments using cutting planes and measuring the UA volume and CSA. Voxel-based registration was frequently used for superimposition, with the stable region of choice being the cranial base.27–29,31,32 Another common superimposition technique was combining LBR with SBR23–26,30 or with VBR.27
Although the studies showed promising results regarding UA analysis based on superimposition, questions remain regarding the reliability and reproducibility of the methods. Only three studies conducted a reliability study on the methods.25,26,30 The reliable methods, all similar, from the same research group, use LBR. However, LBR requires manual re-identification of points and has been shown to be less reliable than VBR.20 Furthermore, it is unclear whether cephalometric landmarks were re-identified and whether superimposition solely was used to visualize morphological UA CSA changes in these three studies. None of these studies contain both a reproducible UA analysis method using VBR and a reliability study of the method. Hence, there is a need for a reliable method for integration of VBR in automated UA analysis, avoiding re-identification of landmarks and reducing manual input.
The aim of the present study is to propose and validate a semi-automatic approach for 3D UA analysis based on superimposition. Its reliability is evaluated and directly compared to a similar method not applying superimposition. It is hypothesized that a semi-automatic method for 3D longitudinal UA analysis using VBR will produce reliable and more exact results than a similar method that relies on re-identification of landmarks and manual measurements.
Methods and materials
Permission was granted by the Danish Data Protection Agency (2008-58-0035, 31 August 2015, 18/40631).
The method developed in this study is based on and compared directly to the method developed by Di Carlo et al5. The patient dataset from Di Carlo et al5, including CBCT scans, 3D segmentations of the airways and anatomical landmarks, has been used during the development of the presented novel method. Consequently, additional descriptions of patient selection, image acquisition, and segmentation can be found in the original protocol.5
The automated analysis part of the approach was implemented in Python 3.7 (Python Software Foundation, Fredericksburg, Virginia) and incorporated in Mimics Innovation Suite (Materialise NV, Leuven, Belgium) through its scripting module. Mimics Innovation Suite is comprised of Mimics 23.0 and 3-matic 15.0. An overview of the workflow can be seen in Supplementary Material 1 available online.
Image acquisition and segmentation
The subjects from the Di Carlo et al5 cohort are 10 randomly selected patients (seven females) diagnosed with maxillomandibular growth disturbances that underwent orthognathic surgery in 2012 at the University Hospital of Southern Denmark, Department of Oral and Maxillofacial Surgery, Esbjerg, Denmark. Nine patients underwent a combined bilateral mandibular sagittal split osteotomy/maxillary Le Fort I osteotomy (eight with segmental maxillary osteotomies) and one patient underwent a maxillary Le Fort I osteotomy exclusively (with segmental maxillary osteotomies). The pre- and post-surgical CBCT scans, stored in Digital Imaging and Communications in Medicine (DICOM) format, from the dataset have been used. In order to avoid introducing variability from a different segmentation procedure, the segmentation of the UA space used in the Di Carlo et al5 UA analysis was maintained.
The selected sample size was calculated by Di Carlo et al5 in order to obtain a power equal to 0.99 with an effect size of 1 mm for linear measurements, 1 mm2 for surface and 1 mm3 for volumetric and α = 0.5. The exclusion criteria were: syndromes or detectable pathologies involving UA and CBCT scans not including all the craniofacial structures required for the cephalometric analysis. Particular attention was paid to patients’ positioning during CBCT scan: subjects without the mandible at maximum intercuspation were excluded as well as patients wearing a bite during scan acquisition. Patients with inappropriate head positioning, including major head extension, head flexion or head rotation, were excluded as well.5
Superimposition of the post-surgical to the pre-surgical scan of each patient was performed using the “Automatic Image Registration” function in Mimics. This function is used to align two different images to the same anatomical coordinate system: a fixed image and a moving image. For this study, the fixed image was the pre-operative CBCT scan and the moving image was the post-operative CBCT scan. After the images were selected, they were manually pre-aligned into approximative positions. Then, a region of interest was specified under the form of a bounding box around the stable anatomical structures of the cranial base, including the zygomatic arches and forehead.18 The superimposition method was VBR. The registration function generated a transformation matrix that describes the transformation of the moving image toward the fixed image. This transformation matrix was applied to the post-operative CBCT scan, resulting in the post-operative CBCT scan being aligned with the pre-operative CBCT scan at the cranial base (Figure 1). To ensure a suitable registration had been achieved, the result was visually inspected in all views.
Analysis of the upper airway
The landmarks selected for airway analysis followed the description in Di Carlo et al5, Table 1. Based on the same criteria, the cephalometric landmarks were placed on the preoperative scan. The axial, coronal, and sagittal views were used to place the landmarks, and additionally the 3D view was used to verify the placement. The airway analysis script was run on the pre- and post-operative UA spaces. The script used the anatomic landmarks to create several planes, as defined by Di Carlo et al5: the Frankfurt Horizontal (FH) plane, mid-sagittal plane, sella-nasion (SN) horizontal plane, and the five planes used to cut the airway into five partial volumes (Table 2 and Figure 1). The pre- and superimposed post-operative airways are automatically sectioned at the level of the cutting planes, and the volumes and cross-sectional area (CSA) are automatically calculated and displayed. These measurements are performed according to Di Carlo et al5 (Table 3). Running the script also provided an overlay between the pre- and post-operative airway segments to easily be visually inspected (Figure 1). All the measurements were exported to Excel (Microsoft, Redmond, WA). A video summary of the UA analysis procedure is available (Supplementary Video 1). For a more visual understanding of the process, the readers are encouraged to consult the video summary.
|Ba||Basion, the anterior margin of the foramen magnum|
|N||Nasion, the intersection of the internasal and frontonasal sutures in the midsagittal plane|
|Or L||Orbitale left, the most inferior anterior point on left orbit’s margin|
|Or R||Orbitale right, the most inferior anterior point on right orbit’s margin|
|PoL||Porion left, the most upper point on left bony external auditory meatus|
|PoR||Porion left, the most upper point on right bony external auditory meatus|
|S||Sella turcica, the centroid of Sella turcica|
|Soa||Midpoint of Sella-Basion line|
|H||Hyoid bone, upper most point of the hyoid bone|
|E||Epiglottis, tip of epiglottis|
|Frankfurt plane||A plane passing through the inferior borders of the bony orbits, encompassed by OrR and OrL, and the upper margin of the auditory meatus encompassed by PoR and PoL.|
|Sagittal SN||Plane perpendicular to Frankfurt plane passing through S and N points|
|SN horizontal||Plane through S and N points, perpendicular to Sagittal (SN)|
|S Ba coronal||Plane through S and Ba points perpendicular to Sagittal (SN)|
|Airway superior border 1||Plane passing through So and perpendicular to S-Ba coronal and Sagittal (SN) planes|
|Airway superior border 2||Plane passing through Ba and parallel to Airway Superior Border 1|
|Airway superior border 3||Plane passing through Ba and parallel to Frankfurt|
|Airway inferior border 1||Plane passing through E and parallel to Frankfurt|
|Airway inferior border 2||Plane passing through 90% of the distance from Airway Superior Border 1 to Airway Inferior Border 1 and parallel to Frankfurt|
|Retropalatal||The airway volume encompassed superiorly by airway superior border and inferiorly by airway superior border 2|
|U-Retropalatal||The airway volume encompassed superiorly by airway superior border and inferiorly by airway superior border 3|
|L-Retropalatal||The airway volume encompassed superiorly by airway superior border 3 and inferiorly by airway superior border 2|
|Oropharynx D100||The airway volume encompassed superiorly by airway superior border 2 and inferiorly by airway inferior border 1|
|Oropharynx D90||The airway volume encompassed superiorly by airway superior border 2 and, inferiorly by airway inferior border 2|
|CSA1||Cross-sections at the boundary between upper retropalatal volume and lower retropalatal volume|
|CSA2||Cross-sections at the boundary between lower retropalatal volume and oropharynx|
|CSA3||Cross-sections at the boundary between oropharynx D90 vol and oropharynx D100 vol|
|CSA4||Cross-section located at the bottom of oropharynx D100 vol|
Two observers (MBH and AD) independently conducted the analysis twice (landmark placement, VBR and semi-automatic analysis), with a two-week interval, in order to evaluate intra- and inter-observer. The second time, the observers were blinded to the first round of analysis. To isolate VBR from the potential error introduced by landmark identification, an additional semi-automatic analysis was conducted where the same landmarks were used across measures, i.e. in the inter-observer case, both observers used the same landmarks, and in the intra-observer case, the same landmarks were used in the first and second measurements. This allowed to isolate error introduced by landmark re-identification from error introduced by the novel aspects of the method (VBR and semi-automatic analysis).
Statistical analysis was conducted using STATA v.16.1 (StataCorp, College Station, TX). The data were tested for normality using the Shapiro-Wilk test. For normally distributed data, a paired t-test was used to compare inter- and intra-observer measurements and to report confidence intervals. The intraclass correlation coefficient (ICC) was used to report the intra- and inter-observer reliability. Bland-Altman plots were used to determine the degree of agreement, outliers, bias, and systematic errors. For a direct comparison with the original method by Di Carlo et al5, Dahlberg’s formula was used to compare both inter- and intra-observer measurements and the isolated VBR measurements.33
The Shapiro-Wilk test indicated all the data were normally distributed. Consequently, a paired t-test was performed. The results of the t-test, used to compare inter- and intra-observer measurements, showed narrow confidence intervals and the repeated inter- and intra-observer measurements were not significantly different. Tables 4–7 show these results. Tables 4–7 also contain the mean and standard deviation for the measures, as well as summarize the results from the measures of reliability. ICC showed excellent intra- and inter-observer reliability (ICC ≥ 0.995).
|first measurement||second measurement||ICC||Dahlberg’s d||p||95% CI||first measurement||second measurement||ICC||Dahlberg’s d||p||95% CI|
|Oropharynx D90 (mm3)||7817.54||2547.32||7839.28||2550.76||0.9993||66.28||0.85%||0.49||−90.49||47.02||9822.40||4585.15||9854.78||4559.19||0.9995||106.95||1.09%||0.53||−143.78||79.02|
|Oropharynx D100 (mm3)||9385.79||3446.10||9410.63||3441.20||0.9996||72.71||0.77%||0.47||−100.09||50.40||11348.18||5433.47||11383.56||5402.67||0.9996||110.47||0.97%||0.50||−150.13||79.36|
|first measurement||second measurement||ICC||Dahlberg’s d||p||95% CI||first measurement||second measurement||ICC||Dahlberg’s d||p||95% CI|
|Oropharynx D90 (mm3)||7753.77||2514.42||7784.69||2593.02||0.9982||106.89||1.38%||0.55||−142.50||80.65||9729.14||4507.82||9758.71||4583.35||0.9993||119.22||1.23%||0.61||−154.73||95.59|
|Oropharynx D100 (mm3)||9315.55||3406.12||9347.09||3495.17||0.9989||116.47||1.25%||0.57||−153.45||90.35||11259.39||5372.77||11280.79||5440.58||0.9995||121.40||1.08%||0.72||−149.85||107.06|
|Observer 1 vs observer 2 at the first measurement||Observer 1 vs observer 2 at the second measurement|
|t-test||Dahlberg’s d||ICC||t-test||Dahlberg’s d||ICC|
|p||95% CI||p||95% CI|
|Oropharynx D90 (mm3)||0.17||−32.61||160.16||101.01||1.29%||0.9984||0.10||−21.30||207.81||114.34||1.46%||0.9980|
|Oropharynx D100 (mm3)||0.15||−29.64||170.12||106.02||1.13%||0.9990||0.11||−25.14||202.71||127.06||1.35%||0.9987|
|Observer 1 vs observer 2 at the first measurement||Observer 1 vs observer 2 at the second measurement|
|t-test||Dahlberg’s d||ICC||t-test||Dahlberg’s d||ICC|
|p||95% CI||p||95% CI|
|Oropharynx D90 (mm3)||0.31||−60.18||169.36||126.05||1.28%||0.9992||0.32||−111.76||303.89||206.39||2.10%||0.9980|
|Oropharynx D100 (mm3)||0.29||−63.20||190.28||123.91||1.09%||0.9995||0.31||−111.10||316.65||213.32||1.88%||0.9985|
For comparison with the manual approach by Di Carlo et al5, the Dahlberg error was calculated. Figure 2 shows the pre-surgical measurement errors for the intra- and inter-observer measurements. Figure 3 shows the post-surgical measurement errors for the intra- and inter-observer measurements. The measurement error of the observers from the present study (AD and MBH) is lower across the board when compared to the measurement error of the observers from the Di Carlo et al5 study (GDC and SFG). The method error from the present study ranged between 0.51 and 4.30% for volumetric measurements and between 0.24 and 2.90% for CSA measurements. In comparison, the error from the Di Carlo et al5 study ranges between 1.6 and 10.2% for volume and between 2 and 12.2% for CSA. Figure 3 also shows the part of the measurement error which is introduced by the user input required for VBR (pre-alignment and selection of volume of interest), with the error for the intra- and inter-observer measurements ranging between 0.05% for pre-operative measurements and up to 1.44% for post-operative measurements.
The purpose of the present study was to propose and validate a semi-automatic approach for 3D UA analysis based on VBR. The semi-automatic method for 3D UA analysis using VBR produced more reliable results than a similar method relying on re-identification of landmarks and manual measurements.
The ICC values obtained for the present method were 0.995 or higher, showing excellent to near perfect reliability. The Bland-Altman plots indicated no systematic error and low bias. Method error calculated with the Dahlberg formula was used for comparison with Di Carlo et al5. The Relative Dahlberg error was calculated because it allowed for comparison of measurements with different units and means.33 The measurements performed were volume (mm3) and CSA (mm2). The novel method with VBR and semi-automatic analysis has lower relative Dahlberg error (0.24–4.30%) than the method by Di Carlo et al5 (1.6–12.2%), which re-identifies landmarks and performs measurements manually (Figures 2 and 3). Paired t-tests resulted in narrow confidence intervals when compared with Di Carlo et al5 indicating greater reliability for the novel method.
The method error of the VBR and semi-automatic analysis was isolated using the same set of landmarks for all inter- and intra-observer measurements. The results indicate VBR introduces a very small portion of variability. There is 0.0% variability (relative Dahlberg error) in the pre-surgical measurements of the isolated VBR procedure. That is to be expected, as the same landmarks were used for both intra- and inter-observer measurement, and superimposition does not play a role in the pre-surgical measurements. Therefore, the analysis script does not introduce any variability. This is the intended purpose of automation: to avoid error from manual user input. The post-surgical measurement error ranged between 0.05 and 1.44% (Figure 3). Since the landmarks were identical, and it has been established that the analysis script does not introduce variability, this error range is therefore attributed to VBR. Voxel-based registration requires user input in pre-alignment and defining the region of interest.
The method can be used with any other set of cephalometric landmarks and cutting planes. Hence, other protocols for UA analysis than the one proposed by Di Carlo et al5 can be integrated in the semi-automatic analysis using voxel-based registration. This provides the flexibility to include other airway measurements, for example, airway length and shape.
In a systematic review by Zimmerman et al34, 42 studies were reviewed to evaluate the reliability of upper pharyngeal airway assessment using CBCT. The UA analysis methods demonstrated moderate to excellent intra- and inter-observer reliability. However, only five studies were deemed high quality, and only three of these studies assessed both intra- and inter-observer reliability. From the high-quality studies, airway volume measurements demonstrated greater intra- and inter-observer reliability (>0.880) than CSA measurements (>0.696). In comparison, the proposed semi-automatic method demonstrated near to perfect intra- and inter-observer reliability (>0.995) for both volume- and CSA measurements. The three high-quality studies which assessed intra- and inter-observer reliability all applied the Dolphin Imaging® software (Dolphin Imaging and Management Systems, Chatsworth, CA, USA) for UA analysis. Of these studies, the study by De Souza et al35 reported good reliability (>0.88); The study by Guijarro-Martinez,36 reported excellent reliability (>0.981) and observer error up to 4% for volume measurements, but only moderate reliability (>0.780) and observer error up to 20% for CSA measurements. The study by Mattos et al37 showed that the transversal width- and CSA measurements at the level of the vallecula had only moderate reliability (>0.63) with a mean observer difference of 16.1 and 13.3%, respectively. The proposed method, which adopts the UA definition by Di Carlo et al5, inferiorly limits the UA at Epiglottis, which may be more reliable, and automatically transfers the cutting planes to the post-operative CBCT, resulting in near to perfect reliability and low observer error for CSA measurements.
Recently, Chen et al38 assessed the reliability and accuracy of three different imaging software packages, Amira® (Visage Imaging Inc., Carlsbad, CA), 3Diagnosys® (3diemme, Cantu, Italy) and OnDemand3D® (CyberMed, Seoul, Republic of Korea) for three-dimensional analysis of the upper airway using CBCT images. The intra- and inter-observer reliability of the measurements using the different software packages were moderate to excellent (>0.78). All three software packages underestimated the upper airway volume by −8.8 to −12.3%, the minimum CSA by −6.2 to− 14.6% and the length by −1.6 to −2.9%.
In a recent study, Zimmerman et al39 presented the first study to evaluate the reliability of UA assessment using CBCT, where the observers performed each step of the analysis manually. The authors argue that this is the way UA assessment would be conducted in a clinical setting today. Intra- and inter-observer reliability of each step in the UA assessment were evaluated by six observers of varying levels of education and clinical experience. Threshold selection in the UA segmentation process showed poor intra- (0.473) and poor inter-observer reliability (0.100); CSA showed moderate intra- (0.591) and poor inter-observer reliability (0.223); Intra-observer reliability of volume measurements varied based on the region assessed (0.747–0.976) and was worst for hypopharynx and best for the oropharynx; inter-observer reliability of volume measurements was generally lower (0.175–0.945) and was worst for nasopharynx and best for the oropharynx. These results are of special interest for comparison with the present study, which in contrast to a fully manual UA analysis evaluated by Zimmerman et al39, is optimised to reduce observer error by automation of sub-processes. The great improvement in the intra- and inter-observer reliability clearly demonstrates the advantages of automation, which should be more generally adopted in clinical practice.
Although the proposed approach minimizes user input for the UA analysis and omits landmark re-identification, manual precursory steps still have to be taken, i.e. the segmentation of the UA and the initial placement of landmarks. Recent developments in the field automatic segmentation40 and landmark placement have been made using artificial intelligence, achieving results within clinically acceptable margins for landmark placement (<2 mm).41–43 These state of the art methods may complement the proposed UA analysis approach, making a fully automatic UA analysis workflow possible.
A noteworthy mention is that no measures have been taken to address the variability in head posture, which is known to affect the UA.44 This issue is inherent to the image acquisition process, and a general challenge of measuring soft tissue response before and after surgery. The logical goal would be to correlate these cross-sectional and volumetric measures with polysomnography findings. The airway changes associated with orthognathic surgery could provide functional justification for this procedure provided the demonstrable postoperative airway changes correlate with improved function are confirmed by polysomnography.
The proposed method was shown to be reliable. A lack of reliable methods for UA analysis that capitalize on the benefits of VBR in the literature is emphasized. When compared to a similar method that involves manual input and landmark re-identification, the semi-automatic approach using VBR shows less inter- and intra-observer measurement error. The proposed approach has great applicability for UA analysis in a clinical trial, due to high reliability. This is further made feasible by the automated part of the analysis, making the novel method efficient and suitable for longitudinal studies on large cohorts.
1. . Three-Dimensional computed tomographic airway analysis of patients with obstructive sleep apnea treated by maxillomandibular advancement. J Oral Maxillofac Surg 2011; 69: 677–86. doi: https://doi.org/10.1016/j.joms.2010.11.037 http://www.ncbi.nlm.nih.gov/pubmed/21353929
2. . Volumetric three-dimensional computed tomographic evaluation of the upper airway in patients with obstructive sleep apnoea syndrome treated by maxillomandibular advancement. Br J Oral Maxillofac Surg 2014; 52: 831–7. doi: https://doi.org/10.1016/j.bjoms.2014.07.101 http://www.ncbi.nlm.nih.gov/pubmed/25129655
3. . Pharyngeal airway morphology in healthy individuals and in obstructive sleep apnea patients treated with maxillomandibular advancement: a comparative study. Oral Surg Oral Med Oral Pathol Oral Radiol 2015; 119: 285–92. doi: https://doi.org/10.1016/j.oooo.2014.11.016 http://www.ncbi.nlm.nih.gov/pubmed/25592868
4. . Upper airway dimensions in patients undergoing orthognathic surgery: a systematic review and meta-analysis. Int J Oral Maxillofac Surg 2016; 45: 460–71. doi: https://doi.org/10.1016/j.ijom.2015.10.018 http://www.ncbi.nlm.nih.gov/pubmed/26616028
5. . A new simple three-dimensional method to characterize upper airway in orthognathic surgery patient. Dentomaxillofac Radiol 2017; 46:
20170042. doi: https://doi.org/10.1259/dmfr.20170042 http://www.ncbi.nlm.nih.gov/pubmed/28749703
6. . Effects of mandibular setback surgery on upper airway dimensions and their influence on obstructive sleep apnoea - a systematic review. J Craniomaxillofac Surg 2015; 43: 248–53. doi: https://doi.org/10.1016/j.jcms.2014.11.017 http://www.ncbi.nlm.nih.gov/pubmed/25547214
7. . Two-Year postoperative upper airway cone-beam computed tomographic outcomes based on a verified upper airway analysis following bimaxillary Orthognathic surgery. J Oral Maxillofac Surg 2019; 77: 1435–45. doi: https://doi.org/10.1016/j.joms.2019.02.038 http://www.ncbi.nlm.nih.gov/pubmed/30926544
8. . Comparison of observer reliability of three-dimensional cephalometric landmark identification on subject images from Galileos and i-CAT cone beam CT. Dentomaxillofac Radiol 2013; 42:
20130059. doi: https://doi.org/10.1259/dmfr.20130059 http://www.ncbi.nlm.nih.gov/pubmed/23833319
9. . Repeatability and reproducibility of landmarks--a three-dimensional computed tomography study. Eur J Orthod 2012; 34: 276–86. doi: https://doi.org/10.1093/ejo/cjq190 http://www.ncbi.nlm.nih.gov/pubmed/21566086
10. . Landmark-Based versus voxel-based 3-dimensional quantitative analysis of bimaxillary osteotomies: a comparative study. J Oral Maxillofac Surg 2020; 78: 468.e1–468.e10. doi: https://doi.org/10.1016/j.joms.2019.10.019 http://www.ncbi.nlm.nih.gov/pubmed/31785251
11. . Validation of the OrthoGnathicAnalyser 2.0-3D accuracy assessment tool for bimaxillary surgery and genioplasty. PLoS One 2021; 16:
e0246196. doi: https://doi.org/10.1371/journal.pone.0246196 http://www.ncbi.nlm.nih.gov/pubmed/33497422
12. . Virtual analysis of segmental bimaxillary surgery: a validation study. J Oral Maxillofac Surg 2021; 10:S0278-2391(21)00540-1. doi: https://doi.org/10.1016/j.joms.2021.06.003 34245697
13. . A 3-dimensional approach for analysis in Orthognathic Surgery-Using free software for voxel-based alignment and Semiautomatic measurement. J Oral Maxillofac Surg 2018; 76: 1316–26. doi: https://doi.org/10.1016/j.joms.2017.11.010 http://www.ncbi.nlm.nih.gov/pubmed/29241643
14. . Image analysis and superimposition of 3-dimensional cone-beam computed tomography models. Am J Orthod Dentofacial Orthop 2006; 129: 611–8. doi: https://doi.org/10.1016/j.ajodo.2005.12.008 http://www.ncbi.nlm.nih.gov/pubmed/16679201
15. . Three-Dimensional maxillary and mandibular regional superimposition using cone beam computed tomography: a validation study. Int J Oral Maxillofac Surg 2016; 45: 662–9. doi: https://doi.org/10.1016/j.ijom.2015.12.006 http://www.ncbi.nlm.nih.gov/pubmed/26794399
16. . The 3D CT superimposition method using image fusion based on the maximum mutual information algorithm for the assessment of oral and maxillofacial surgery treatment results. Oral Surg Oral Med Oral Pathol Oral Radiol 2012; 114: 167–74. doi: https://doi.org/10.1016/j.tripleo.2011.06.003 http://www.ncbi.nlm.nih.gov/pubmed/22776729
17. . Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 1997; 16: 187–98. doi: https://doi.org/10.1109/42.563664 http://www.ncbi.nlm.nih.gov/pubmed/9101328
18. . Accuracy and reproducibility of voxel based superimposition of cone beam computed tomography models on the anterior cranial base and the Zygomatic arches. PLoS One 2011; 6:
e16520. doi: https://doi.org/10.1371/journal.pone.0016520 http://www.ncbi.nlm.nih.gov/pubmed/21347419
19. . Comparison of the accuracy of voxel based registration and surface based registration for 3D assessment of surgical change following orthognathic surgery. PLoS One 2014; 9:
e93402. doi: https://doi.org/10.1371/journal.pone.0093402 http://www.ncbi.nlm.nih.gov/pubmed/24695577
20. . Accuracy and reliability of landmark-based, surface-based and voxel-based 3D cone-beam computed tomography superimposition methods. Orthod Craniofac Res 2017; 20: 227–36. doi: https://doi.org/10.1111/ocr.12205 http://www.ncbi.nlm.nih.gov/pubmed/28960842
21. . A comparison of voxel- and surface-based cone-beam computed tomography mandibular superimposition in adult orthodontic patients. J Int Med Res 2021; 49: 030006052098270. doi: https://doi.org/10.1177/0300060520982708 http://www.ncbi.nlm.nih.gov/pubmed/33459090
22. . A systematic review to uncover a universal protocol for accuracy assessment of 3-dimensional virtually planned Orthognathic surgery. J Oral Maxillofac Surg 2017; 75: 2430–40. doi: https://doi.org/10.1016/j.joms.2017.05.025 http://www.ncbi.nlm.nih.gov/pubmed/28646644
23. . Cone beam computed tomography registration for 3-D airway analysis based on anatomic landmarks. Oral Surg Oral Med Oral Pathol Oral Radiol 2014; 118: 371–83. doi: https://doi.org/10.1016/j.oooo.2014.05.027 http://www.ncbi.nlm.nih.gov/pubmed/25151592
24. . Using cone beam CT to assess the upper airway after surgery in children with sleep disordered breathing symptoms and maxillary-mandibular disproportions: a clinical pilot. J Otolaryngol Head Neck Surg 2017; 46: 31. doi: https://doi.org/10.1186/s40463-017-0204-4 http://www.ncbi.nlm.nih.gov/pubmed/28399908
25. . Three-Dimensional evaluation of the upper airway morphological changes in growing patients with skeletal class III malocclusion treated by protraction headgear and rapid palatal expansion: a comparative research. PLoS One 2015; 10:
e0135273. doi: https://doi.org/10.1371/journal.pone.0135273 http://www.ncbi.nlm.nih.gov/pubmed/26252015
26. . Effect of large incisor retraction on upper airway morphology in adult bimaxillary protrusion patients. Angle Orthod 2012; 82: 964–70. doi: https://doi.org/10.2319/110211-675.1 http://www.ncbi.nlm.nih.gov/pubmed/22462464
27. . What are the surgical movements in Orthognathic surgery that most affect the upper airways? a three-dimensional analysis. J Oral Maxillofac Surg 2021; 79: 450–62. doi: https://doi.org/10.1016/j.joms.2020.10.017 http://www.ncbi.nlm.nih.gov/pubmed/33171114
28. . Three-Dimensional analysis of changes in airway space after bimaxillary orthognathic surgery with maxillomandibular setback and their association with obstructive sleep apnea. Maxillofac Plast Reconstr Surg 2018; 40: 33. doi: https://doi.org/10.1186/s40902-018-0171-3 http://www.ncbi.nlm.nih.gov/pubmed/30474021
29. . Upper airway morphologic changes after mandibular setback surgery in skeletal class III malocclusion patients measured using cone beam computed tomography superimposition. Int J Oral Maxillofac Surg 2018; 47: 1405–10. doi: https://doi.org/10.1016/j.ijom.2018.05.002 http://www.ncbi.nlm.nih.gov/pubmed/29859651
30. . Cbct evaluation of the upper airway morphological changes in growing patients of class II division 1 malocclusion with mandibular retrusion using twin block appliance: a comparative research. PLoS One 2014; 9:
e94378. doi: https://doi.org/10.1371/journal.pone.0094378 http://www.ncbi.nlm.nih.gov/pubmed/24705466
31. . Relations between mandible-only advancement surgery, the extent of the posterior airway space, and the position of the hyoid bone in class II patients: a three-dimensional analysis. Br J Oral Maxillofac Surg 2019; 57: 1032–8. doi: https://doi.org/10.1016/j.bjoms.2019.09.001 http://www.ncbi.nlm.nih.gov/pubmed/31563482
32. . The relationship between bimaxillary orthognathic surgery and the extent of posterior airway space in class II and III patients – a retrospective three-dimensional cohort analysis. J Oral Maxillofac Surg Med Pathol 2021; 33: 30–8. doi: https://doi.org/10.1016/j.ajoms.2020.07.009
33. . Statistical notes for clinical researchers: evaluation of measurement error 2: Dahlberg's error, Bland-Altman method, and kappa coefficient. Restor Dent Endod 2013; 38: 182–5. doi: https://doi.org/10.5395/rde.2013.38.3.182 http://www.ncbi.nlm.nih.gov/pubmed/24010087
34. . Reliability of upper pharyngeal airway assessment using dental CBCT: a systematic review. Eur J Orthod 2017; 39: 489–96. doi: https://doi.org/10.1093/ejo/cjw079 http://www.ncbi.nlm.nih.gov/pubmed/27999121
35. . Reliability of a method to conduct upper airway analysis in cone-beam computed tomography. Braz Oral Res 2013; 27: 48–54. doi: https://doi.org/10.1590/S1806-83242013000100009 http://www.ncbi.nlm.nih.gov/pubmed/23306626
36. . Three-Dimensional cone beam computed tomography definition of the anatomical subregions of the upper airway: a validation study. Int J Oral Maxillofac Surg 2013; 42: 1140–9. doi: https://doi.org/10.1016/j.ijom.2013.03.007 http://www.ncbi.nlm.nih.gov/pubmed/23623785
37. . Reliability of upper airway linear, area, and volumetric measurements in cone-beam computed tomography. Am J Orthod Dentofacial Orthop 2014; 145: 188–97. doi: https://doi.org/10.1016/j.ajodo.2013.10.013 http://www.ncbi.nlm.nih.gov/pubmed/24485733
38. . Reliability and accuracy of three imaging software packages used for 3D analysis of the upper airway on cone beam computed tomography images. Dentomaxillofac Radiol 2017; 46:
20170043. doi: https://doi.org/10.1259/dmfr.20170043 http://www.ncbi.nlm.nih.gov/pubmed/28467118
39. . Reliability of upper airway assessment using CBCT. Eur J Orthod 2019; 41: 101–8. doi: https://doi.org/10.1093/ejo/cjy058 http://www.ncbi.nlm.nih.gov/pubmed/30184085
40. . Automatic segmentation of the pharyngeal airway space with convolutional neural network. J Dent 2021; 111:
103705. doi: https://doi.org/10.1016/j.jdent.2021.103705
41. . Accuracy and reliability of automatic three-dimensional cephalometric landmarking. Int J Oral Maxillofac Surg 2020; 49: 1367–78. doi: https://doi.org/10.1016/j.ijom.2020.02.015 http://www.ncbi.nlm.nih.gov/pubmed/32169306
42. . Deep Geodesic learning for segmentation and anatomical Landmarking. IEEE Trans Med Imaging 2019; 38: 919–31. doi: https://doi.org/10.1109/TMI.2018.2875814 http://www.ncbi.nlm.nih.gov/pubmed/30334750
43. . Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization. Med Image Anal 2020; 60:
101621. doi: https://doi.org/10.1016/j.media.2019.101621 http://www.ncbi.nlm.nih.gov/pubmed/31816592
44. . The effect of altered head and tongue posture on upper airway volume based on a validated upper airway analysis-An MRI pilot study. Orthod Craniofac Res 2020; 23: 102–9. doi: https://doi.org/10.1111/ocr.12348 http://www.ncbi.nlm.nih.gov/pubmed/31550076