REVIEW 4 major objections 6 minor 11 references
Geometric Deep Learning for Automated Landmarking of Maxillary Arches on 3D Oral Scans from Newborns with Cleft Lip and Palate
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A geometric deep learning pipeline can automatically place 16 clinically meaningful landmarks on 3D maxillary arch scans of infants with cleft lip and palate, reporting 94.44% accuracy and a mean absolute error of $1.676 \pm 0.959$ mm.
desk verdict First automated landmarking results on infant cleft-arch scans, with a useful clinical payoff, but the human-comparison claim is unsupported and the accuracy reporting is sloppy; worth peer review after major revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the multi-view consensus landmarking pipeline adapted for 3D dental meshes: each maxillary arch mesh is rendered into multiple 2D views, the 16 ground-truth landmarks are projected onto those views, and a two-stack hourglass network (residual blocks with 3x3 convolutions, batch normalization, and dropout) predicts one heatmap per landmark per view. A 3D landmark is then estimated from the consensus of the 2D view estimates, treating each feature point as the intersection of rays. This lets a non-Euclidean 3D mesh be handled by a Euclidean CNN without losing the 3D spatial relationships that define the landmarks.
What would settle it
Have two or more experienced clinicians independently landmark the same set of 10 test models and compute pairwise inter-rater distances per landmark; if the mean inter-rater distance is comparable to or greater than the model's 1.676 mm mean absolute error, the reported accuracy is measuring agreement with one rater rather than clinical truth.
Extended reading notes
Core claim
The central claim, stated in the paper's own terms, is that a two-stack hourglass convolutional network, trained on 90 models, each rendered into multiple 2D views, and applied to 10 previously unseen models, can place 16 clinically significant landmarks with an accuracy of 94.44% (errors below 0.01% of model volume, equal to 2.56 mm on this sample) and an absolute mean error of $1.676 \pm 0.959$ mm. The model produces 16 heatmaps per 2D view, treats each feature point as a ray, and estimates each 3D landmark by multi-view consensus. The errors concentrate on smooth regions that lack distinct anatomical features, such as anterior cleft points and widest points, and are larger for unusually small or large arches. The paper further claims that the method has higher accuracy and reproducibility than human operators and reduces landmarking time from roughly 45 minutes per model to minutes.
Load-bearing premise
All training and evaluation treat the 16 landmarks placed by a single experienced clinician as error-free ground truth, and the paper reports no intra-rater or inter-rater reliability; if a second expert's labels differ from the first by an amount close to or larger than the reported 1.676 mm mean error, the accuracy figure and the claim of matching human operators lose their clinical meaning.
Editorial extensions
If this is right
- Because applying the trained model to 10 test models took 6 minutes, a non-specialist can generate the full 16-landmark set in minutes instead of the roughly 45 minutes a specialist spends per model.
- Automated landmarking removes operator drift, so longitudinal studies comparing pre- and post-treatment arches can be measured with the same internal standard.
- The pipeline works with fewer than 100 training examples, which matters for rare or hard-to-collect pediatric populations like newborns with cleft lip and palate.
- The labeled landmarks unlock downstream morphometric measures such as palatal width, depth, surface area, and volume, feeding treatment planning and outcome prediction.
- The same training recipe can be pointed at other landmark sets and populations, such as adolescents or elderly dentitions, wherever 3D meshes are available.
Reading between the lines
- Because the ground-truth labels come from one clinician, the 94.44% figure should probably be read as closeness to that clinician's convention; a fair test of 'as accurate as a human' would compare model-to-clinician distance with clinician-to-clinician distance on the same models.
- The volume-normalized tolerance means the headline accuracy depends on the size of the arches in the sample; reporting the absolute millimeter threshold alongside each arch volume would make the metric more comparable across studies.
- The same 2D-render-plus-ray-consensus architecture is a natural candidate for other smooth, feature-poor pediatric anatomical surfaces, but its error rates would need to be re-benchmarked against multi-rater variability on those surfaces before being trusted clinically.
- A direct practical extension is to measure the model's repeatability on repeated inference runs and its agreement with a panel of clinicians; if the agreement with the panel's consensus matches the panel's internal agreement, the pipeline has met the standard for replacing manual landmarking in routine workflow.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a geometric deep learning pipeline for automatically placing 16 clinically relevant landmarks on 3D maxillary arch scans from newborn infants with cleft lip and palate. The approach renders 3D meshes into multiple 2D views, projects manual landmarks onto these views, and trains a two-stack hourglass CNN to predict landmark heatmaps. The model is trained on 90 models and tested on 10 models from the same dataset. The authors report 94.44% accuracy, a mean absolute error of 1.676 ± 0.959 mm, and claim that the pipeline achieves higher accuracy and reproducibility than human operators while requiring less than 100 training samples and reducing manual landmarking time from about 45 minutes per model to minutes.
Significance. If the results were fully supported, this would be a useful contribution to automated morphometric analysis in cleft lip and palate research, with potential to enable larger longitudinal studies and reduce specialist workload. The work addresses an under-served infant population, uses a relatively small training set, and leverages a multi-view consensus strategy that is appropriate for 3D surface landmarking. However, the strongest claims—matching or exceeding human operators and delivering clinically usable accuracy—rest on an evaluation against a single clinician's labels, with no human reliability study, no comparison to previous automated methods, and only a 10-model test set. As presented, the evidence is suggestive but not sufficient to establish the headline claims.
major comments (4)
- [Abstract; Section 1 (Contributions); Section 4] The abstract and the contributions list claim that the pipeline achieves 'higher accuracy and reproducibility than with human operators,' but no experiment compares the model to a human operator. Section 2.1 describes labels from a single clinician, and Section 4 reports only a timing anecdote (45 minutes per model). Without a direct comparison of repeated human landmarkings or inter-operator agreement, the claim that the model is more accurate and reproducible than humans is not supported by the presented data.
- [Section 2.1] The ground truth for both training and evaluation consists of landmarks placed by one experienced clinician, with no intra-rater or inter-rater reliability study. If human labeling variability is comparable to or larger than the reported 1.676 mm mean error, then the accuracy figure does not establish clinical equivalence or superiority. Please report repeated annotation variability (e.g., the same clinician re-landmarking a subset, or a second annotator) and discuss its relation to the model error.
- [Section 2.2] The evaluation uses 10 test models from a single 90/10 split, and the abstract states accuracy was demonstrated on '100 models,' which conflicts with the reported test set size. Moreover, because each patient contributes both a pre-treatment and a post-treatment model, the paper should state whether the split was performed at the patient level; otherwise, models from the same infant may appear in both training and test sets, inflating apparent performance. Cross-validation or multiple splits would help establish stability of the reported accuracy.
- [Section 2.2, Eq. (1)] Equation (1) is garbled in the manuscript ('Accuracy(%)=+∑(ei≤T)Ni=1 N 3∗100'), and the threshold T=0.01% of model volume is introduced without clinical justification or sensitivity analysis. Since the 94.44% accuracy figure depends entirely on the choice of T, please provide the correct formula, justify the threshold, and report accuracy across a range of thresholds.
minor comments (6)
- [Abstract] The phrase 'on a set of 100 models' is misleading because only 10 models were used for testing; please clarify that 90 models were used for training and 10 for independent testing.
- [Table 1] The column header 'Root Mean Squared Error ± SD' is inconsistent with the text's 'absolute mean error'; the values appear to be reported as mean ± SD. Harmonize the terminology and equations.
- [Section 3] The one-sample t-test against a mean error of 0 tests whether the mean error is biased, not the magnitude of error; consider presenting confidence intervals for the mean error and the proportion of landmarks within clinically relevant thresholds.
- [Section 1] The Introduction contains repeated sentences, including 'These 3D meshes are much easier and faster to annotate than traditional physical cast methods' and 'Such analysis can further aid in tailoring specialized and personalized treatment plans...', which should be removed.
- [Section 1] The prior work of Croquet et al. [8] is mentioned but no quantitative comparison is provided; reporting the performance of that method on the same or similar data would contextualize the claimed improvements.
- [Section 2.2] The manuscript should specify the number of 2D views, camera angles, and rendering settings used to generate the multi-view inputs, as this is central to reproducibility.
Circularity Check
No material circularity; the model is trained and evaluated against independent manual landmark labels, and the self-citations are not load-bearing. The 'better than human operators' claim is unsupported by reliability data, but that is an evidence gap, not a circular derivation.
full rationale
The paper's pipeline is a standard supervised geometric-deep-learning landmarking system: 90 models with manually placed landmarks are used for training, and performance is measured on 10 unseen models by comparing predicted landmarks to the same kind of manual labels (Section 2.2, Section 3). This held-out comparison is independent of the training signal, so the central accuracy result does not reduce to a fitted input. The accuracy threshold T = 0.01% of model volume is a hand-chosen success criterion, not a learned parameter and not a prediction. The contributions claim of 'higher accuracy and reproducibility than with human operators' is not supported because no inter-rater or intra-rater reliability of the single clinician's ground truth is reported; however, this is a validity/evidence problem, not a circularity. The only self-citations ([9] on BioAMP therapy, [13] on outcome prediction) are background/related-work references and are not used to justify the landmarking derivation. An internal inconsistency between the abstract's 'set of 100 models' and Section 2.2's 10-model test set is noted but is also not circularity. Overall, the derivation chain is self-contained.
Assumptions & free parameters
free parameters (1)
- Accuracy threshold T =
0.01% of model volume, 2.56 mm on this sample
assumptions (4)
- domain assumption Manual landmarks from one clinician are an accurate gold standard
- domain assumption A test set of 10 models is representative of the infant CLP population
- domain assumption Multi-view 2D projections with consensus preserve 3D landmark positions
- ad hoc to paper The chosen threshold (0.01% of model volume) is clinically meaningful
Cite this review
Pith. "Pith review of Geometric Deep Learning for Automated Landmarking of Maxillary Arches on 3D Oral Scans from Newborns with Cleft Lip and Palate." pith.science (2026). https://pith.science/paper/SN77LRWN
@misc{pith2026250115737,
author = {Pith},
title = {Pith review of: Geometric Deep Learning for Automated Landmarking of Maxillary Arches on 3D Oral Scans from Newborns with Cleft Lip and Palate},
year = {2026},
howpublished = {\url{https://pith.science/paper/SN77LRWN}},
note = {Machine review of arXiv:2501.15737}
}
read the original abstract
Rapid advances in 3D model scanning have enabled the mass digitization of dental clay models. However, most clinicians and researchers continue to use manual morphometric analysis methods on these models such as landmarking. This is a significant step in treatment planning for craniomaxillofacial conditions. We aimed to develop and test a geometric deep learning model that would accurately and reliably label landmarks on a complicated and specialized patient population -- infants, as accurately as a human specialist without a large amount of training data. Our developed pipeline demonstrated an accuracy of 94.44% with an absolute mean error of 1.676 +/- 0.959 mm on a set of 100 models acquired from newborn babies with cleft lip and palate. Our proposed pipeline has the potential to serve as a fast, accurate, and reliable quantifier of maxillary arch morphometric features, as well as an integral step towards a future fully automated dental treatment pipeline.
Figures
Reference graph
Works this paper leans on
-
[1]
N. K. Tanna, A. A. A. Y. AlMuzaini, and M. Mupparapu, “Imaging in Orthodontics,” Dent. Clin. North Am., vol. 65, no. 3, pp. 623–641, Jul. 2021, doi: 10.1016/j.cden.2021.02.008
-
[2]
Imaging modalities to inform the detection and diagnosis of early caries,
T. Walsh et al., “Imaging modalities to inform the detection and diagnosis of early caries,” Cochrane Database Syst. Rev., vol. 2021, no. 3, p. CD014545, Mar. 2021, doi: 10.1002/14651858.CD014545
-
[3]
C.-C. Chen et al., “Automatic recognition of teeth and periodontal bone loss measurement in digital radiographs using deep-learning artificial intelligence,” J. Dent. Sci., vol. 18, no. 3, p. 1301, Apr. 2023, doi: 10.1016/j.jds.2023.03.020
-
[4]
Personalized and precision orthodontic therapy,
L. R. Iwasaki, D. A. Covell, S. A. Frazier-Bowers, S. Kapila, S. S. Huja, and J. C. Nickel, “Personalized and precision orthodontic therapy,” Orthod. Craniofac. Res., vol. 18 Suppl 1, pp. 1–7, Apr. 2015, doi: 10.1111/ocr.12089
-
[6]
Multi-view consensus CNN for 3D facial landmark placement
R. R. Paulsen, K. A. Juhl, T. M. Haspang, T. Hansen, M. Ganz, and G. Einarsson, “Multi-view consensus CNN for 3D facial landmark placement,” Oct. 14, 2019, arXiv: arXiv:1910.06007. doi: 10.48550/arXiv.1910.06007
work page Pith review arXiv doi:10.48550/arxiv.1910.06007 2019
-
[7]
Estimation of Apnea-Hypopnea Index Using Deep Learning On 3-D Craniofacial Scans,
U. Hanif et al., “Estimation of Apnea-Hypopnea Index Using Deep Learning On 3-D Craniofacial Scans,” IEEE J. Biomed. Health Inform., vol. 25, no. 11, pp. 4185–4194, Nov. 2021, doi: 10.1109/JBHI.2021.3078127
-
[8]
Automated landmarking for palatal shape analysis using geometric deep learning,
B. Croquet et al., “Automated landmarking for palatal shape analysis using geometric deep learning,” Orthod. Craniofac. Res., vol. 24, no. Suppl 2, p. 144, Jul. 2021, doi: 10.1111/ocr.12513
-
[9]
H. Choo and H. Ahn, “Biocreative Alveolar Molding Plate Treatment (BioAMP) for neonatal unilateral cleft lip and palate with excessively wide alveolar cleft and maxillary arch width,” Korean J. Orthod., vol. 54, no. 1, pp. 69–73, Jan. 2024, doi: 10.4041/kjod23.163
Show all 11 references
-
[10]
3D Slicer as an image computing platform for the Quantitative Imaging Network,
A. Fedorov et al., “3D Slicer as an image computing platform for the Quantitative Imaging Network,” Magn. Reson. Imaging, vol. 30, no. 9, pp. 1323–1341, Nov. 2012, doi: 10.1016/j.mri.2012.05.001
2012 doi
-
[11]
Three-Dimensional Morphological Changes of the True Cleft under Passive Presurgical Orthopaedics in Unilateral Cleft Lip and Palate: A Retrospective Cohort Study,
P. Nalabothu, B. K. Benitez, M. Dalstra, C. Verna, and A. A. Mueller, “Three-Dimensional Morphological Changes of the True Cleft under Passive Presurgical Orthopaedics in Unilateral Cleft Lip and Palate: A Retrospective Cohort Study,” J. Clin. Med., vol. 9, no. 4, p. 962, Mar....
2020 doi
-
[12]
User Experience of Intraoral Scanners in Dentistry: Transnational Questionnaire Study,
A. Al-Hassiny et al., “User Experience of Intraoral Scanners in Dentistry: Transnational Questionnaire Study,” Int. Dent. J., vol. 73, no. 5, pp. 754–759, Oct. 2023, doi: 10.1016/j.identj.2023.04.002
2023 doi
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.