Original Article
Evaluation of an automated superimposition method based on multiple
landmarks for growing patients
ABSTRACT
Objectives: To determine if an automated superimposition method using six landmarks (Sella,
Nasion, Porion, Orbitale, Basion, and Pterygoid) would be more suitable than the traditional SellaNasion (SN) method to evaluate growth changes.
Materials and Methods: Serial lateral cephalograms at an average interval of 2.7 years were taken
on 268 growing children who had not undergone orthodontic treatment. The T1 and T2 lateral
images were manually traced. Three different superimposition methods: Björk’s structural method,
conventional SN, and the multiple landmark (ML) superimposition methods were applied. Bjork’s
structural method was used as the gold standard. Comparisons among the superimposition
methods were carried out by measuring the linear distances between Anterior Nasal Spine, point A,
point B, and Pogonion using each superimposition method. Multiple linear regression analysis was
performed to identify factors that could affect the accuracy of the superimpositions.
Results: The ML superimposition method demonstrated smaller differences from Björk’s method
than the conventional SN method did. Greater differences among the cephalometric landmarks
tested resulted when: the designated point was farther from the cranial base, the T1 age was older,
and the more time elapsed between T1 and T2.
Conclusions: From the results of this study in growing patients, the ML superimposition method
seems to be more similar to Björk’s structural method than the SN superimposition method. A major
advantage of the ML method is likely to be that it can be applied automatically and may be just as
reliable as manual superimposition methods. (Angle Orthod. 0000;00:000–000.)
KEY WORDS: Growth evaluation; Automatic superimposition method; Artificial intelligence
comes of orthodontic treatment.1–3 Various superimposition methods using different reference planes have
been developed.4–6 Depending on which references
are used for superimpositions, assessment of treatment outcomes or growth change analysis can be
different. Therefore, in comparing serial lateral cephalometric radiographs, the selection of references is
fundamental. Especially in superimpositions of growing
patients, stable structures that are not affected by
growth should be selected as references.7 Meanwhile,
owing to rapid development of artificial intelligence (AI),
numerous research groups have been applying machine learning methods.8–14 The latest AI demonstrated
a level of accuracy equivalent to humans when
identifying cephalometric landmarks.10 Recently, an
automatic superimposition method that can be implemented immediately after automatic landmark identification was proposed.15
Björk’s structural method has conventionally been
recognized as the gold standard for superimposition in
a growing patient. It depends on relatively stable
INTRODUCTION
Cephalometric superimposition has been used to
evaluate changes associated with growth and outa
Postgraduate Student, Department of Orthodontics, Graduate School, Seoul National University, Seoul, Korea.
b
Clinical Lecturer, Department of Orthodontics, Seoul National University Dental Hospital, Seoul, Korea.
c
Assistant Professor, Graduate Program Director, Department of Orthodontics, University of Florida College of Dentistry,
Gainesville, FL, USA.
d
Professor, Department of Orthodontics and Dental Research
Institute, Seoul National University School of Dentistry, Seoul,
Korea.
Corresponding author: Dr Shin-Jae Lee, Professor, Department of Orthodontics and Dental Research Institute, Seoul
National University School of Dentistry, 101 Daehakro, JongroGu, Seoul 03080, Korea
(e-mail: nonext@snu.ac.kr)
Accepted: July 2021. Submitted: January 2021.
Published Online: October 4, 2021
Ó 0000 by The EH Angle Education and Research Foundation,
Inc.
DOI: 10.2319/010121-1.1
1
Angle Orthodontist, Vol 00, No 00, 0000
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
Min-Gyu Kima; Jun-Ho Moona; Hye-Won Hwangb; Sung Joo Choa; Richard E. Donatellic; Shin-Jae Leed
2
MATERIALS AND METHODS
The subjects of this study consisted of 268 children
who visited the Department of Orthodontics, Seoul
National University Dental Hospital, Seoul, Korea.
Although they had sought orthodontic treatment, some
of them hesitated to begin treatment immediately and
deferred their treatment for timing or monetary reasons. Others were diagnosed to be enlisted as
orthognathic surgery patients, and their treatment
plans were to observe growth without any orthodontic/orthognathic intervention at the first visit. For all
subjects of this study, at least two serial lateral
cephalometric images existed: the one taken at the
first visit (T1) and another after the observation period
(T2). The mean ages at T1 and T2 were 11.1 and 13.7
years, respectively. The mean time elapsed between
the two visits was 2.7 years. The characteristics of the
Angle Orthodontist, Vol 00, No 00, 0000
Table 1. Descriptive Summary of Study Dataa
Study Variables
N (%)
Total
268 (100)
Female
145 (54.0)
Male
123 (46.0)
Cephalometric images taken at T1 and T2 536 (100)
Age at T1 (years)
268 (50.0)
Age at T2 (years)
268 (50.0)
Time elapsed between the two serial
images (T2 – T1, years)
Skeletal Classification
268 (100)
Class I
70 (26.0)
Class II
107 (39.9)
Class III
91 (34.1)
a
Mean SD
11.1
13.7
2.7
3.0
3.8
2.0
SD indicates standard deviation.
subjects included in the present study are listed in
Table 1. The institutional review board for the
protection of human subjects of the Seoul National
University Dental Hospital reviewed and approved the
research protocol (ERI 19007).
For each T1 and T2 image, cephalometric tracing,
landmark identification, and labeling were manually
performed by examiner 1 (SJL). The T2 image was
superimposed on the T1 image by three superimposition methods: Björk’s structural method, the conventional SN superimposition method, and the
automatic superimposition method based on multiple
landmarks (ML). The experimental design is summarized in Figure 1.
Three Superimposition Methods
Superimposition by Björk’s structural method was
assisted by a custom made user-interface implemented by the Python programming language (Python Software Foundation, Wilmington, Delaware,
USA). On the user interface, the transparency of
each image was freely adjustable to facilitate the
superimposition of cranial base structures. Examiner
2 (MGK) manually translated and rotated the second
image at T2 on the first image at T1 so that the stable
cranial base structures overlapped as much as
possible. To evaluate the reproducibility of manual
superimposition by Björk’s structural method using
the user-interface, 30 subjects were randomly selected and the superimposition procedure was
repeated by the same examiner at a 3-month
interval. As a result, the mean intra-examiner
differences for the selected landmarks were 0.84 6
0.66 mm at Anterior Nasal Spine (ANS), 0.87 6 0.67
mm at Point A, 1.32 6 0.99 mm at Point B, and 1.44
6 1.97 mm at Pogonion.
Both of the SN and ML methods were implemented
based on the x, y coordinates of the cranial base
landmarks. Unlike the conventional SN superimposi-
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
cranial base structures and has been adopted and
advocated by the American Board of Orthodontics.
However, this superimposition method not only
requires high-quality radiographs, but also demands
time and effort in accurately identifying cranial base
contours.16 Since Björk’s structural method requires
line drawings of curved contours instead of just
pinpointing certain landmarks, it is not compatible
with the current computer environment. Therefore, for
computer-aided superimpositions, the Sella-Nasion
(SN) superimposition method has widely been applied
for use in clinical practice.12,14,17–20 However, since
Nasion is affected by growth and remodeling, the SN
superimposition method that exclusively depends on
entirely these two landmarks is likely to result in
errors, especially when evaluating growth.21–23 To
overcome the disadvantages of the traditional SN
superimposition method, an automatic superimposition method that depends collectively upon six
landmarks (Sella, Nasion, Porion, Orbitale, Basion,
and Pterygoid) was proposed.15 This method is highly
compatible with computer-aided cephalometrics and
demonstrates more accurate results than the conventional SN superimposition method. However, the
automatic method was tested on adult patients.
Although it might be suitable for comparing posttreatment outcomes in non-growing patients, its
application to growing patients has not yet been
determined.15
With Björk’s structural method as a reference, the
purpose of this study was to compare the superimposition results of the conventional SN superimposition
method to the automated superimposition method on
growing patients. The null hypothesis was that there
would be no difference between the two superimposition methods in growing patients.
KIM, MOON, HWANG, CHO, DONATELLI, LEE
AUTOMATIC GROWTH SUPERIMPOSITION
3
tion method that is based only on two cranial base
landmarks, Sella and Nasion, the ML superimposition
method included four additional landmarks: Porion,
Orbitale, Basion, and Pterygoid. For each lateral
cephalometric radiograph and its associated cranial
base landmarks, the second tracing was location
shifted and rotation transformed until the sum of
squared Euclidean distance measures in the cranial
base landmarks between the first and second images
were minimized, while Sella remained in an identical
location.15
Comparisons Among Superimposition Methods
In the present study, Björk’s structural method, which
is dependent upon stable cranial base structures, was
considered the reference for comparing the superimposition accuracy between the SN superimposition and
the ML superimposition methods (Figure 2). To
compare the accuracy between the SN and ML
superimposition methods, four cephalometric landmarks (ANS, Point A, Point B, and Pogonion) were
tested. These landmarks were selected since they
Angle Orthodontist, Vol 00, No 0, 0000
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
Figure 1. Study flow chart.
4
KIM, MOON, HWANG, CHO, DONATELLI, LEE
were known to best reflect the subjects’ characteristics
and best recognize the variability of the maxilla and the
mandible.14 Using each superimposition method, T2
image points were designated as P2Björk, P2SN, and
P2ML. To evaluate the accuracy of the two superimposition methods, the Euclidean distances between
P2Björk and P2SN, and P2Björk and P2ML, were calculated
in millimeters. Whichever value was closer to zero
indicates which method (SN or ML) could be regarded
as a more accurate superimposition method for
growing patients (Figure 2).
Statistical Analysis
Paired t-test was applied to compare the differences
between the conventional SN and ML superimposition
methods. Multiple linear regression analysis was
performed to identify factors that could affect the
accuracy of the superimpositions. In the regression
model, the distance from Björk’s method of each
reference point was set as the dependent variable.
Angle Orthodontist, Vol 00, No 00, 0000
The factors that were assumed to affect the accuracy
of superimposition in growing patients were chosen as
independent variables: SN vs ML superimposition
methods, cephalometric landmarks, age at T1, and
the time elapsed between the two consecutive images
(T1 to T2). Data preparation and all the statistical
analyses were performed using Language R.24 The
significance level was set at P , .05.
RESULTS
The difference between the SN and ML superimposition methods for ANS, Point A, Point B, and Pogonion
are summarized in Table 2. For every landmark, there
was a statistically significant difference between the
SN and ML superimposition methods. The ML superimposition method demonstrated smaller differences
from Björk’s method than did the conventional SN
method. The differences increased in order from ANS,
Point A, Point B, to Pogonion.
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
Figure 2. T1 (at the first visit) and T2 (at the second visit) tracings superimposed by three different superimposition methods. The figure illustrates
focusing on the specific landmark of Pogonion (Pog). For a given cephalometric landmark P, P2Björk signifies the position of P in the second tracing
oriented by Björk’s structural method; P2SN by the SN superimposition method; and P2ML by the ML superimposition method. P2ML - P2Björk indicates
the difference between the ML and Björk’s structural method; P2SN - P2Björk indicates the difference between the SN and Björk’s structural method.
AUTOMATIC GROWTH SUPERIMPOSITION
5
Table 2. Comparisons of the Differences Between the
Superimposition Methodsa,b
ML method,
P2Björk - P2ML
Mean
Mean
1.53
1.57
2.19
2.39
SD
1.45
1.47
1.97
2.15
1.36
1.40
1.94
2.12
SD
1.29
1.31
1.79
1.94
Mean
Difference
0.17
0.17
0.24
0.27
P Valuec
.0034
.0033
.0053
.0062
a
For each landmark P (ANS, Point A, Point B, Pogonion), the
position of P in the T2 image oriented by each superimposition
method was denoted by P2Björk, P2SN, and P2ML, for Björk’s, the SN,
and the ML superimposition methods, respectively.
b
SD indicates standard deviation.
c
Results from the paired t-test.
Results of the multiple linear regression analysis
indicated that the superimposition method, the cephalometric landmarks, the age at T1, and the time
elapsed between the two serial images were statistically significant factors when determining the differences among the superimpositions. For example, the
SN method had a greater difference than the ML
method. Pogonion, the farthest designated point from
the cranial base, had the greatest difference among the
cephalometric landmarks tested. The older the T1 age,
the more the superimposition difference decreased.
The greater elapsed time between T2 and T1 also
resulted in the greater difference among the cephalometric landmarks tested (Table 3).
Figure 3 demonstrates an example of the results of
all the superimposition methods being applied to
subject images. Just from casual observation, it is
evident that the ML method more closely followed
Björk’s structural method.
DISCUSSION
The purpose of this study was to compare the widely
used conventional SN superimposition method with the
ML superimposition method on growing patients. The
reference superimposition method was Björk’s structural method. The null hypothesis was that there would
be no difference in the accuracy of the two superimposition methods. To identify variables that may affect
the accuracy of a superimposition method, multiple
linear regression analysis was used. The results of this
study indicated that there was a statistically significant
difference between the two superimposition methods.
The ML method was found to be 0.17 mm, 0.17 mm,
0.24 mm, and 0.27 mm more accurate than the SN
method at ANS, Point A, Point B, and Pogonion,
respectively. The difference between the two methods
was an average of 0.21 mm, indicating that the ML
method was 0.21 mm closer to Björk’s structural
superimposition method used as a reference. Howev-
Study Variables
b
SN vs ML superimposition method
0.21
(reference, ML)
Cephalometric landmarks (reference, ANS)
Point A
0.04
Point B
0.62
Pogonion
0.81
Age at T1 (1-year increments)
-0.08
T2 – T1 (1-year increments)
0.22
SE(b) P Value
0.07
.0025
0.10
0.10
0.10
0.01
0.03
.7028
,.0001
,.0001
,.0001
,.0001
a
ANS indicates anterior nasal spine; b, regression coefficients;
ML, automated superimposition method based on multiple
landmarks; SE, standard error; SN, Sella-Nasion superimposition
method.
er, this might not be a clinically significant result.
Several studies claimed that a difference of more than
1mm should be considered clinically significant.2,25 The
major advantage of the ML method was that it could be
applied automatically and be just as, if not more,
reliable than the laborious, manual Björk’s structural
superimposition method.
The location of each anatomical landmark, the age
at the first visit, and the time elapsed between the two
consecutive images were found to affect the differences between the superimposition methods. The difference between superimposition methods tended to
increase as the landmarks were positioned lower in
the face (Tables 2 and 3). Since cranial base structures
are used as a reference when superimposing lateral
cephalometric images, differences between superimposition methods would increase as the landmarks
were more distant from the cranial base. Therefore,
ANS and Point A, which are relatively close to the
cranial base, produced more accurate superimposition
results than Point B and Pogonion. In other words,
when analyzing growth, superimposition is more likely
to differ from the reference Björk’s structural method in
the mandible compared to the maxilla. In addition,
regression coefficients obtained were -0.08 for age at
T1 and 0.22 for the time elapsed between the two
serial images (T2-T1), which indicated that the younger
the first image was taken and the longer the time
interval between the first and second images, the
greater the difference between the two superimposition
methods.
The increase in difference between superimposition
methods may have been related to growth of the
cranial base. As growth occurs, there naturally would
be changes in the cranial base structures used as
references in superimpositions. The conventional SN
method might not properly reflect the changes caused
by growth over time, resulting in greater differences
Angle Orthodontist, Vol 00, No 0, 0000
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
ANS
Point A
Point B
Pogonion
SN method,
P2Björk - P2SN
Table 3. Summary of Multiple Linear Regression Analysis of the
Differences in the Superimpositions. Model Included Factors Such as
SN vs ML, Cephalometric Landmarks, Age at T1, and Time Elapsed
Between the Two Serial Images (T2 – T1)a
6
KIM, MOON, HWANG, CHO, DONATELLI, LEE
from Björk’s structural method. The anterior cranial
base consists of structures that complete their growth
at different times. Presphenoidal regions were reported
to nearly complete growth at seven years of age;
ethmoid regions were reported nearly complete at four
years of age. However, the frontal segment and Sella
Turcica have been reported to continue to remodel until
early adulthood. Sella is constantly remodeling and
moves backward and downward; Nasion moves
forward due to the expansion of the frontal sinus.21
Even anteroposterior length of the maxillary complex
might be associated with the anterior cranial base.26
Therefore, the SN superimposition method using Sella
and Nasion as landmarks that move during the growth
phase may be less reliable and result in faulty
judgment.
The ML superimposition method may have several
advantages over the conventional SN superimposition
method and even Björk’s structural method. First, it is
capable of reducing the overdependence on Sella and
Nasion points. From previous studies, superimposition
errors could occur from rotation and translation
Angle Orthodontist, Vol 00, No 00, 0000
mistakes. Typical superimposition errors are mainly
caused by rotation errors rather than translation
errors.27 The ML method fixed Sella at the same
location and rotated the T2 image until the sum of the
distances of the remaining five landmarks were
minimized. Therefore, superimposition errors were
reduced because of a greater degree of freedom to
rotate around Sella was permitted, unlike the SN
superimposition method. Second, the ML superimposition method is compatible with computer-aided
cephalometrics, allowing full automation. By automatically identifying landmarks as well as, or better than, a
human, then using multiple landmarks to create
accurate lateral cephalograph tracings,10,13 then automatically superimposing these lateral cephalographs
with comparable or better accuracy of growth changes,
the ML method demonstrated its value to orthodontics.
CONCLUSIONS
The null hypothesis that there would be no difference
in the accuracy between the SN and ML superimposition methods on growing subjects is rejected.
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
Figure 3. Real patient application. The growth observation periods for these boys were 3 years, 2 months; and 3 years, 6 months, respectively.
AUTOMATIC GROWTH SUPERIMPOSITION
ACKNOWLEDGMENT
This study was partly supported by grant 05-2020-0021 from
the Seoul National University Dental Hospital Research Fund.
REFERENCES
1. Gu Y, McNamara Jr JA. Cephalometric superimpositions: a
comparison of anatomical and metallic implant methods.
Angle Orthod. 2008;78:967–976.
2. Huja S, Grubaugh E, Rummel A, Fields H, Beck F.
Comparison of hand-traced and computer-based cephalometric superimpositions. Angle Orthod. 2009;79:428–435.
3. Koerich L, Weissheimer A, de Menezes LM, Lindauer SJ.
Rapid 3D mandibular superimposition for growing patients.
Angle Orthod. 2017;87:473–479.
4. Bjork A, Skieller V. Normal and abnormal growth of the
mandible. A synthesis of longitudinal cephalometric implant
studies over a period of 25 years. Eur J Orthod. 1983;5:1–
46.
5. Goel S, Bansal M, Kalra A. A preliminary assessment of
cephalometric orthodontic superimposition. Eur J Orthod.
2004;26:217–222.
6. Ricketts RM. A four-step method to distinguish orthodontic
changes from natural growth. J Clin Orthod. 1975;9:208–
228.
7. You Q, Hägg U. A comparison of three superimposition
methods. Eur J Orthod. 1999;21:717–725.
8. Hwang HW, Moon JH, Kim MG, Donatelli RE, Lee SJ.
Evaluation of automated cephalometric analysis based on
the latest deep learning method. Angle Orthod. 2021;91:
329–335.
9. Moon JH, Hwang HW, Yu Y, Kim MG, Donatelli RE, Lee SJ.
How much deep learning is enough for automatic identification to be reliable? A cephalometric example. Angle Orthod.
2020;90:823–830.
10. Hwang HW, Park JH, Moon JH, et al. Automated identification of cephalometric landmarks: Part 2- Might it be better
than human? Angle Orthod. 2020;90:69–76.
11. Chen S, Wang L, Li G, et al. Machine learning in
orthodontics: Introducing a 3D auto-segmentation and
auto-landmark finder of CBCT images to assess maxillary
constriction in unilateral impacted canine patients. Angle
Orthod. 2020;90:77–84.
12. Suh HY, Lee HJ, Lee YS, Eo SH, Donatelli RE, Lee SJ.
Predicting soft tissue changes after orthognathic surgery: the
sparse partial least squares method. Angle Orthod. 2019;89:
910–916.
13. Park JH, Hwang HW, Moon JH, et al. Automated identification of cephalometric landmarks: Part 1-Comparisons
between the latest deep-learning methods YOLOV3 and
SSD. Angle Orthod. 2019;89:903–909.
14. Kang TJ, Eo SH, Cho H, Donatelli RE, Lee SJ. A sparse
principal component analysis of Class III malocclusions.
Angle Orthod. 2019;89:768–774.
15. Moon JH, Hwang HW, Lee SJ. Evaluation of an automated
superimposition method for computer-aided cephalometrics.
Angle Orthod. 2020;90:390–396.
16. Arat ZM, Türkkahraman H, English JD, Gallerano RL, Boley
JC. Longitudinal growth changes of the cranial base from
puberty to adulthood: a comparison of different superimposition methods. Angle Orthod. 2010;80:725–732.
17. Lee HJ, Suh HY, Lee YS, et al. A better statistical method of
predicting postsurgery soft tissue response in Class II
patients. Angle Orthod. 2014;84:322–328.
18. Lee YS, Suh HY, Lee SJ, Donatelli RE. A more accurate
soft-tissue prediction model for Class III 2-jaw surgeries. Am
J Orthod Dentofacial Orthop. 2014;146:724–733.
19. Suh HY, Lee SJ, Lee YS, et al. A more accurate method of
predicting soft tissue changes after mandibular setback
surgery. J Oral Maxillofac Surg. 2012;70:e553–562.
20. Yoon KS, Lee HJ, Lee SJ, Donatelli RE. Testing a better
method of predicting postsurgery soft tissue response in
Class II patients: a prospective study and validity assessment. Angle Orthod. 2015;85:597–603.
21. Afrand M, Ling CP, Khosrotehrani S, Flores-Mir C,
Lagravère-Vich MO. Anterior cranial-base time-related
changes: a systematic review. Am J Orthod Dentofacial
Orthop. 2014;146:21–32.e6.
22. Arat M, Köklü A, Özdiler E, Rübendüz M, Erdogan B.
Craniofacial growth and skeletal maturation: a mixed
longitudinal study. Eur J Orthod. 2001;23:355–361.
23. Arat ZM, Rübendüz M, Arman Akgül A. The displacement of
craniofacial reference landmarks during puberty: a comparison of three superimposition methods. Angle Orthod. 2003;
73:374–380.
24. R Core Team. R: A language and environment for statistical
computing. Vienna, Austria: R Foundation for Statistical
Computing; 2021.
25. Ghafari J, Engel FE, Laster LL. Cephalometric superimposition on the cranial base: a review and a comparison of four
methods. Am J Orthod Dentofacial Orthop. 1987;91:403–
413.
26. Savoldi F, Massetti F, Tsoi JKH, et al. Anteroposterior length
of the maxillary complex and its relationship with the anterior
cranial base. Angle Orthod. 2021;91:88–97.
27. Baumrind S, Miller D, Molthen R. The reliability of head film
measurements: 3. Tracing superimposition. Am J Orthod.
1976;70:617–644.
Angle Orthodontist, Vol 00, No 0, 0000
Downloaded from http://meridian.allenpress.com/angle-orthodontist/article-pdf/doi/10.2319/010121-1.1/2916964/10.2319_010121-1.1.pdf by Lebanon user on 10 October 2021
From the results of this study in growing patients, the
automated superimposition method based on the
multiple landmarks seems to be more similar to
Björk’s structural method than the SN superimposition method, but only marginally so.
Despite the small differences, the major advantage of
the ML method is likely to be that it can be applied
automatically and may be just as reliable as manual
superimposition methods.
7