{"id":"35d939d4-fe12-4294-809d-5e77cfd6f5d5","arxiv_id":"2501.10984","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A two-stage HRNetV2-style network with Self-ONN bottlenecks reports state-of-the-art cephalometric landmark detection on ISBI 2015 (82.25% within 2 mm) and external PKU (75.95%).","lead":"This paper proposes a two-stage deep learning system for automatically finding 19 dental landmarks in side-view head X-rays, using a Self-Operational Neural Network block inside an HRNetV2 backbone. The authors report 82.25% of landmarks identified within 2 mm on the ISBI 2015 benchmark and 75.95% on an external PKU dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 82.25% SDR rests on using Test1 and Test2 as each other's validation sets (Section 5.4), so neither is an independent test; a clean retraining with a proper held-out validation split is required to confirm the benchmark claim.","rationale":"Read in good faith, the paper is an empirical benchmark submission: it proposes a two-stage HRNetV2+Self-ONN pipeline and reports SDR on ISBI 2015 and PKU. The central claim is that the pipeline reaches 82.25% within 2mm on Test1+Test2, at or slightly above prior art. For this to be true, the evaluation must be an honest out-of-sample assessment. The paper itself, in Section 5.4, reveals that Test1 and Test2 were used as each other's validation sets. That makes the reported numbers selected, not merely measured. I do not accuse the authors of fabrication; the per-landmark tables are largely internally consistent (70.95 = mean of 75.05 and 66.84; 82.25 = mean of 87.82 and 76.68), and the external PKU result is a genuinely useful sanity check. But the PKU result is also reported without any validation-split detail, and the comparison to Zeng et al. uses the same potentially selected stage-2 model. The cleanest way to settle the concern is to retrain with a held-out validation split from the 150 training images and re-evaluate on both test sets. If the SDR holds, the paper is credible; if it drops, the headline overclaim is exposed. The Table 1 landmark-definition errors are a separate correctness problem for the classification section, but the detection benchmark is the main claim. Because the core issue is methodological and fixable, CONDITIONAL remains the right verdict; it does not need to be escalated to REJECT.","tokens_in":28795,"tokens_out":4750,"duration_ms":51830,"concrete_test":"Retrain the full two-stage pipeline using only the 150-image training split, holding out a random 20% of training images for validation (or 5-fold cross-validation) to select checkpoints and hyperparameters; with the chosen models frozen, evaluate once on Test1 and Test2 and compute the 2mm SDR average. If the clean average is not above Oh et al.'s 82.08 by a margin larger than checkpoint-selection noise (e.g., >0.5 points), the paper's benchmark-superiority claim fails. Also report the same retraining without any test-set-based validation to verify the 70.94% stage-1 number.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.4 states: \"During the evaluation of the Test1 dataset, we utilized the Test2 dataset as a validation set, while for evaluating the Test2 dataset, we employed the Test1 dataset as the validation set.\" This is the load-bearing step for the headline claim. Because Test2 was used to guide model selection (early stopping, learning-rate schedule, checkpoint choice) for the model evaluated on Test1, and vice versa, neither number is an independent test-set evaluation. The same protocol applies to the 19 second-stage models, so the reported 82.25% average over Test1+Test2 (Table 6) inherits the bias. The concern is concrete: the margin over Oh et al. is only 82.25 vs 82.08, 0.17 percentage points, well within the range of validation-selection noise. A secondary but real issue is Table 1: the anatomical definitions for ANB/SNB/SNA etc. are incorrect (e.g., ANB is described using landmarks 5, 2, and 6), which invalidates the Section 5.2 classification comparisons even though it does not directly affect the detection SDR. The central claim should be treated as unverified until the evaluation protocol is repaired.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Self-CephaloNet, a two-stage deep learning framework for cephalometric landmark detection on lateral cephalograms. Stage 1 uses an HRNetV2 backbone with a novel Self-ONN bottleneck to predict 19 landmarks jointly; stage 2 trains 19 separate patch-based refinement models. The authors report a stage-1 success detection rate (SDR) of 70.94% and a stage-2 SDR of 82.25% within 2 mm on the ISBI 2015 Test1 and Test2 datasets, plus 75.95% on the external PKU dataset, together with anatomical classification accuracy. The central claim is that the architecture achieves state-of-the-art or near-state-of-the-art performance on this benchmark.","tokens_in":29090,"tokens_out":5431,"duration_ms":50508,"significance":"If the reported numbers were obtained under a clean evaluation protocol, the work would be of interest because it combines a high-resolution backbone with a Self-ONN bottleneck and demonstrates strong performance on a public benchmark, including external validation. The paper includes per-landmark error tables, multiple tolerance thresholds, a complexity analysis, and interpretability visualizations. The main limitation is that the reported test numbers are not independent evaluations because of the validation protocol in Section 5.4; the anatomical classification experiments are also compromised by incorrect definitions in Table 1.","major_comments":[{"comment":"The stated protocol invalidates the reported Test1 and Test2 scores as independent test evaluations. The text says: \"During the evaluation of the Test1 dataset, we utilized the Test2 dataset as a validation set, while for evaluating the Test2 dataset, we employed the Test1 dataset as the validation set.\" Because validation curves typically guide early stopping, learning-rate schedule, and checkpoint selection, the model evaluated on Test1 is selected using Test2 and vice versa. The headline stage-2 average of 82.25% (Table 6) is only 0.17 percentage points above Oh et al. (82.08%), so the claimed superiority is well within the range of validation-selection noise. The authors should re-run the experiments with a proper held-out validation split (for example, a subset of the 150 training images) and report Test1 and Test2 results only after that split is fixed.","section":"Section 5.4"},{"comment":"The definitions of the eight clinical measurements in Table 1 are anatomically incorrect. For example, ANB is the angle between A-point, Nasion, and B-point, yet the table describes it as the angle between Landmark 5, Landmark 2, and Landmark 6, which are dental landmarks. Similar errors affect SNB, SNA, and other measures. Because Section 5.2 and Tables 8-11 report classification success rates computed from these measurements using the predicted landmark positions, those results cannot be interpreted as standard cephalometric classification and are not comparable to the cited methods. The authors should correct the definitions and recompute the classification evaluation.","section":"Table 1 and Section 5.2"},{"comment":"The statement that \"Our first-stage results surpassed previous studies\" is contradicted by the paper's own Table 4, in which SCN Payer et al. (73.33%) outperforms the proposed stage-1 (70.94%) at the 2.0 mm threshold. Similarly, Table 6 shows that Oh et al. achieve higher SDR at 3.0 mm and 4.0 mm (92.34% and 96.92%) than the proposed method (92.00% and 96.25%). The claims of superiority should be limited to the specific metrics and thresholds where the comparison is actually favorable.","section":"Abstract and Section 5.1"},{"comment":"The causal claim that the Self-ONN bottleneck is responsible for the improvement over HRNet is not established. Table 7 compares Self-CepahloNet with an HRNet baseline, but no details are given about the baseline's training schedule, data augmentation, patch size, or second-stage setup. A controlled ablation under identical training conditions, replacing only the Self-ONN bottleneck with a standard bottleneck, is required to attribute the performance difference to the Self-ONN component. Without such an ablation, the novelty claim is not supported.","section":"Section 3.2 and Table 7"}],"minor_comments":[{"comment":"The name is spelled \"Self-CepahloNet\" in the title and abstract but \"Self-CephaloNet\" elsewhere; please standardize the spelling throughout.","section":"Title/Abstract"},{"comment":"The text refers to \"Align-Net\" without defining it, which appears to be an unintended reference or typo; please replace it with the correct model name or define the term.","section":"Section 5.7"},{"comment":"The text states \"Mean Error (MRE) of 2.79 ± 1.87\" but Table 14 reports \"1.87 ± 2.79\"; the order of MRE and SD is inconsistent and should be corrected.","section":"Section 5.7 and Table 14"},{"comment":"The term \"Radical Error\" should be \"Radial Error\" throughout the paper and in Figure 5.","section":"Section 4.1"},{"comment":"The table caption says \"compared across the first and second stages of some method\"; this is unclear and should be rewritten to indicate that the comparison is between the proposed first-stage results and other methods' final-stage results.","section":"Table 4"},{"comment":"The text uses \"Gram Cam\" and \"Grad-CAM\" interchangeably; please use the standard \"Grad-CAM\" consistently.","section":"Section 5.6"},{"comment":"The learning-rate schedule lists a reduction to 0.000001 at epoch 30 and again at epoch 50; if the 40th-epoch value is different, it should be stated explicitly.","section":"Section 4.4"},{"comment":"The abbreviation IPE is used inconsistently: \"Image-specific Radical Error\" in Section 5.5 and \"Inverse Perspective Error\" in Section 5.7; please define and use one term consistently.","section":"Section 5.5 and 5.7"}],"recommendation":"major_revision","confidential_remarks":"The evaluation-protocol issue in Section 5.4 is the main barrier to acceptance; the authors should be asked to redo the benchmark evaluation with a clean held-out validation split. The incorrect anatomical definitions in Table 1 are also a serious issue and should be checked by a domain expert before the classification claims are taken seriously. The paper is otherwise a standard empirical benchmark study with useful per-landmark tables and external validation, but these two issues prevent me from recommending acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this one. The architecture is a reasonable incremental combination—HRNetV2 with a Self-ONN bottleneck plus per-landmark second-stage refinement—and the paper reports 82.25% SDR at 2mm on ISBI 2015, which is 0.17 points above Oh et al. But that number is not trustworthy as reported, because Section 5.4 says they used Test1 and Test2 as each other's validation sets. Neither test set is an independent evaluation, and a 0.17-point edge is well inside the range of model-selection noise. The headline benchmark claim therefore needs to be re-established with a clean validation split.\n\nWhat is actually good: the paper gives full per-landmark MRE/SD/SDR tables, quantifies the two-stage gain cleanly (70.94 to 82.25), and includes a genuinely useful external validation on the PKU dataset (75.95% vs Zeng's 64.88%). That PKU result is the most convincing evidence in the paper—it is an independent dataset from a different institution, and the comparison to a published method is favorable. The Grad-CAM analysis and parameter counts are reasonable additions, and the hyperparameter table is complete enough to reproduce with effort.\n\nThe soft spots are not minor. First, the test-as-validation protocol is load-bearing. The margin over Oh et al. is tiny, so the claim of state-of-the-art is unverified until they retrain with a proper held-out split or honestly report both test results without using them for model selection. Second, Table 1's anatomical definitions are wrong—ANB is not the angle between landmarks 5, 2, and 6—so the classification results in Section 5.2 and the comparisons in Tables 10–11 rest on incorrect measurements. Third, there is no same-training ablation isolating the Self-ONN contribution; Table 7 compares against a different HRNet implementation, not an ablation. Fourth, the text overclaims by saying they outperform prior methods at 2mm and 2.5mm, but Table 6 shows Oh et al. is better at 3mm and 4mm. Finally, no code is provided.\n\nWho is this for: people working on cephalometric landmark detection or two-stage heatmap refinement. The architecture and PKU numbers are worth knowing about, but only after the evaluation protocol is fixed. A serious referee should see this because the flaw is fixable and the external validation is valuable, but the paper should not be accepted without a clean re-evaluation, corrected anatomy definitions, and an ablation.","headline":"A reasonable incremental architecture for cephalometric landmarking whose headline 82.25% SDR is undercut by using Test1 and Test2 as mutual validation sets; the PKU external validation is the most solid contribution.","tokens_in":29678,"tokens_out":3721,"would_cite":false,"duration_ms":38036,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-stage AI net hits 82.25% on 2 mm dental landmark test","keywords":["cephalometric landmark detection","Self-ONN","HRNetV2","two-stage framework","ISBI 2015 dataset","success detection rate","anatomical classification","orthodontic treatment planning"],"falsifier":"Retrain the identical two-stage pipeline with the self-bottleneck replaced by a conventional convolutional bottleneck of the same width and parameter count, evaluating on an untouched test split rather than the cross-validation scheme; if the 2-mm success rate falls well below the reported 82.25% or the gap over plain HRNet disappears, the adaptive-node mechanism is not doing the claimed work.","tokens_in":28617,"feed_emoji":"🦷","tokens_out":11144,"duration_ms":104119,"temperature":0.7,"pith_summary":"Self-CephaloNet is a two-stage deep learning framework for detecting 19 cephalometric landmarks in lateral skull X-rays, and the paper claims it reaches near the top of the ISBI 2015 challenge benchmark. In the first stage, a single network predicts all landmarks at once, achieving a 70.95% success rate within 2 mm on the Test1 and Test2 sets combined; the second stage refines each landmark on its own 512x512 patch, raising the success rate to 82.25%. The same second-stage model reaches 75.95% within 2 mm on an external cephalogram dataset [65] without retraining. The authors attribute the gain to replacing the standard bottleneck in an HRNetV2 backbone with a self-operational neural network block whose neurons learn their own nonlinear operators. If these results hold, orthodontists would have an end-to-end tool that automates landmarking at acceptable precision while reducing the need for multiple independently trained CNNs.","feed_headline":"Two-stage AI net hits 82.25% on 2 mm dental landmark test","feed_subtitle":"Single end-to-end model with per-landmark refinement matches the best ISBI 2015 cephalogram scores.","key_machinery":"The load-bearing component is the Self Bottleneck, a replacement for the standard convolutional bottleneck in the HRNetV2 backbone. It is built from Self-ONN layers, where each neuron's operator is a $q$-th order Taylor polynomial $\\sum_{k=0}^{q} w_k x^k$; the weights $w_k$ are learned by backpropagation, so the network adapts the nonlinear operation per connection instead of using a fixed linear convolution. The framework also relies on a two-stage cascade: whole-image heatmap regression in stage one, then patch-based refinement with 19 per-landmark models in stage two. The paper attributes the accuracy gain to the adaptive nodal operators of Self-ONN, rather than to the high-resolution backbone alone.","core_discovery":"The central claim is that a two-stage cascade built on HRNetV2 with a novel 'self-bottleneck' based on Self-ONN (self-operational neural networks) matches or exceeds prior state-of-the-art on the ISBI 2015 cephalometric landmark detection benchmark. Stage one predicts heatmaps for all 19 landmarks simultaneously from the whole image, giving a mean radial error of 1.53 mm on Test1 and 1.80 mm on Test2 and a combined 2-mm success detection rate of 70.95%. Stage two crops a 512x512 patch around each first-stage prediction and runs 19 per-landmark refinement models, improving the combined 2-mm SDR to 82.25% and the mean radial error to 1.08 mm on Test1 and 1.44 mm on Test2. On the external dataset [65], the stage-two model reports a 75.95% SDR at 2 mm, outperforming the cascaded CNN baseline [16] used for comparison. The paper further claims that the end-to-end first stage alone beats several published second-stage results, so the self-bottleneck's adaptive nodes are carrying real weight.","pith_inferences":["A controlled ablation replacing only the Self-ONN layers with standard convolutions of the same width and parameter count would isolate how much of the gain comes from adaptive nodal operators; the comparison to plain HRNet in Table 7 suggests the attributed difference but does not control for training schedule.","Because Test2 served as the validation set while reporting Test1 (and Test1 for Test2), checkpoint selection may have leaked information; a re-evaluation on an untouched split, or using the external set [65] as the primary evidence, would give a cleaner measure of generalisation.","The landmark-to-measurement definitions in Table 1 appear to mix up anatomical points such as SNA and SNB angles; practitioners should verify those mappings before using the clinical classification numbers."],"forward_implications":["A single end-to-end first-stage network reaches a 70.95% 2-mm SDR across Test1 and Test2, so landmark detection does not require a multi-network cascade to be useful.","Adding per-landmark patch refinement lifts the 2-mm SDR to 82.25%, confirming that targeted local refinement remains a reliable source of accuracy in heatmap-based landmark localization.","The 75.95% 2-mm SDR on the external dataset [65] suggests the model transfers across scanner types and populations without additional training.","The reported anatomical classification averages (86.75% on Test1, 83.87% on Test2) imply that skeletal classification computed from predicted landmarks retains usable accuracy."],"supporting_citations":[{"why":"Provides the ISBI 2015 challenge dataset, ground-truth labels, and the success-detection-rate evaluation protocol reported throughout the paper.","marker":"[14]"},{"why":"Defines Self-ONN with generative neurons, the adaptive-operator mechanism the self-bottleneck is built on.","marker":"[21]"},{"why":"Supplies the original high-resolution network architecture that the backbone descends from.","marker":"[17]"},{"why":"Describes HRNetV2's high-resolution representation fusion, the backbone the paper modifies with the self-bottleneck.","marker":"[53]"},{"why":"Sets the main second-stage baseline with an 82.08% 2-mm success rate against which the paper compares its 82.25%.","marker":"[15]"},{"why":"Provides the cascaded-CNN baseline and the external validation dataset used for the generalisation test.","marker":"[16]"},{"why":"Reports the HRNet-based cascade-refine model used in the head-to-head comparison in Table 7.","marker":"[42]"}],"fun_headline_variants":["Two-stage self-ONN framework hits 82.25% on 2 mm landmark test","Self-CephaloNet achieves 82.25% on dental landmarks","Cascaded self-ONN net improves cephalometric detection to 82%","HRNetV2 with self-bottleneck lands 82.25% cephalometric SDR","End-to-end two-stage AI hits 82% on dental landmark test"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy stands on the assumption that using Test2 as the validation set while reporting Test1 results (and Test1 as validation for Test2) did not bias model selection, so the Test1 and Test2 scores are genuine out-of-sample results.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage self-ONN framework hits 82.25% on 2 mm landmark test","Self-CephaloNet achieves 82.25% on dental landmarks","Cascaded self-ONN net improves cephalometric detection to 82%","HRNetV2 with self-bottleneck lands 82.25% cephalometric SDR","End-to-end two-stage AI hits 82% on dental landmark test"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000646,"raw_usage":{"total_tokens":3034,"prompt_tokens":1077,"completion_tokens":1957,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":1849}},"tokens_in":693,"tokens_out":1957,"duration_ms":15539,"temperature":1.0,"reasoning_tokens":1849,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:45:25.079414+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the identical two-stage pipeline with the self-bottleneck replaced by a conventional convolutional bottleneck of the same width and parameter count, evaluating on an untouched test split rather than the cross-validation scheme; if the 2-mm success rate falls well below the reported 82.25% or the gap over plain HRNet disappears, the adaptive-node mechanism is not doing the claimed work.","supporting_citations":[{"cited_title":"Knowledge-Based Systems, 2023","cited_arxiv_id":null,"evidence_quote":"Reports the HRNet-based cascade-refine model used in the head-to-head comparison in Table 7."}],"review_version":1}