{"id":"cb8e0bf4-e749-47b0-b71d-bcd8c3aa055a","arxiv_id":"2412.01182","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MeasureNet detects three-point polylines along villi and crypts in duodenal biopsies, using length-aware losses and segmentation-mask guidance, and derives celiac disease grade from the measured villi-to-crypt ratio.","lead":"MeasureNet measures the length of villi and crypts in intestinal biopsy images by detecting polylines, and uses those lengths to grade celiac disease. The authors also introduce a new annotated dataset of 750 duodenal images, reporting about 82% binary and 81% multi-class grading accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ground truth grade labels are never defined; if they are derived by thresholding manual polyline ratios, the reported classification accuracy is not an independent diagnostic result.","rationale":"The reader's weakest assumption identifies exactly this issue: the classification accuracy is a threshold transform of the measurement, and the thresholds come from a self-cited study without external validation. My stress-test confirms that this is the most load-bearing concern because it determines the interpretation of the central claim. I considered alternative technical issues, such as the length loss in Equations 5-7 using squared L2 distances rather than true Euclidean lengths, which is a real internal inconsistency but would only affect the training objective, not the validity of the clinical claim. Similarly, the conversion of 2-point annotations to 3 points by duplicating endpoints is a simplification but is consistently applied to baselines and method. The missing ground truth label definition, however, is a fundamental omission: without knowing how the Marsh grades were generated, the classification metrics in Table 2 cannot be interpreted. If the labels are derived from the same thresholds, the reported accuracy is not an independent diagnostic outcome; if they are pathologist-assigned, the paper fails to describe a key part of the dataset. Either way, the evidence is insufficient to support the diagnostic claim as stated. The reader's conditional verdict appropriately requires this validation, and my read does not change that verdict, so I mark it UNCHANGED.","tokens_in":13658,"tokens_out":5188,"duration_ms":49276,"concrete_test":"Ask the authors to disclose the provenance of the ground truth grade labels for the CeDeM test set. If the labels were assigned by expert pathologists independently of the measurements, recompute Table 2 and compare to those labels. If the labels were instead generated by thresholding the manual polyline ratios with the Section 5.1 cutoffs, then obtain independent pathologist Marsh grades for the same 70 test images and recompute the classification accuracy against that reference; report Cohen's kappa between the threshold-derived labels and the pathologist grades. A significant drop or low agreement would show that the current accuracy figures are not clinical diagnostic results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central diagnostic claim rests on classification accuracy numbers that are never tied to a defined ground truth. Section 4 (Dataset Details) describes CeDeM as containing polyline, villi shoulder, and crypt border annotations, but says nothing about how the Marsh grade for each image was obtained. Section 5.1 defines grade thresholds from self-cited reference [7] but does not state whether these thresholds were used to generate the test labels. If the ground truth grades were created by applying these thresholds to the manual polyline ratios, then Table 2's accuracy is a deterministic function of the measurement error, not an independent clinical validation. The reported 81.42% grade accuracy would merely restate how well predicted ratios cross the fixed cutoffs, and the claimed 11-point improvement over LETR would be a byproduct of improved ratio MAE. No inter-observer agreement for the manual polyline annotations is reported, and no external validation of the thresholds from [7] on this dataset is provided. This is load-bearing because it determines what the headline claim actually asserts: a diagnostic improvement or a measurement improvement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MeasureNet, a transformer-based polyline detection framework for measuring villi and crypt lengths in duodenal biopsy images, with the goal of grading celiac disease from the villi-to-crypt length ratio. The method augments a DINO-DETR detector with chamfer-distance, length, and part-length losses, and uses segmentation masks of the villi shoulder and crypt border as auxiliary guidance, with a feature mixup to reduce over-reliance on those masks. The authors introduce CeDeM, a new dataset of 750 H&E-stained duodenum biopsy images with 6,800 polyline annotations, and report measurement, localization, and classification results against segmentation, lane-detection, and line-segment baselines, claiming consistent improvements (e.g., ratio MAE from 0.70 to 0.47, mAP from 40.3 to 50.2, and multi-class grading accuracy from 70% to 81.4%).","tokens_in":13853,"tokens_out":3458,"duration_ms":31316,"significance":"If the reported results are valid, the paper makes a useful contribution by formulating villi/crypt measurement as a polyline detection task rather than a post-hoc segmentation measurement, and by providing a new annotated dataset in a domain where public data are scarce. The ablation study gives a clear account of the contribution of each loss term and of the mask-guidance mechanism, and the code/dataset release plan is a strength for reproducibility. However, the central evaluation claim about celiac grade classification is only meaningful if the ground-truth grade labels are defined independently of the manual polyline measurements; the manuscript does not state how those labels were obtained. Since the reported classification accuracy is computed by thresholding the predicted ratio, it may simply restate the measurement error rather than validate a diagnostic hypothesis. The evaluation also rests on a single 70-image test split with no error bars or significance testing, which weakens the confidence in the headline 11-point accuracy gain.","major_comments":[{"comment":"The manuscript never specifies how the ground-truth Marsh grade label for each test image was obtained. Section 4 describes only polyline, villi-shoulder, and crypt-border annotations, while Section 5.1 defines grade thresholds from self-cited reference [7] and states that the average villi-to-crypt ratio determines the grade. If those thresholds were applied to the manual polyline ratios to generate the test labels, then Table 2's accuracy is a deterministic function of the predicted ratio crossing the same cutoffs; the 11-point improvement over LETR would then be a restatement of the ratio MAE improvement in Table 1 (0.704 to 0.472) rather than an independent diagnostic evaluation. Please clarify the label generation procedure, report inter-observer agreement for the manual polyline annotations, and, if possible, evaluate against independently assigned clinical grades.","section":"§4 and §5.1"},{"comment":"All localization, measurement, and classification numbers are reported on a single fixed 70-image test split, with no repeated runs, cross-validation, confidence intervals, or significance tests. With only 70 test images, the reported accuracy differences (e.g., 70.10% to 81.42% for multi-class grading) are not shown to be statistically reliable. The authors should report variance across multiple seeds or bootstrap confidence intervals and provide per-class confusion matrices for the grade classification task.","section":"§5.1, Table 1 and Table 2"},{"comment":"The true-positive criterion for localization is stated as 'normalized chamfer distance below 0.05,' but the normalization is not defined. It is unclear whether the distance is normalized by image dimensions, polyline length, or another quantity, making precision, recall, and mAP numbers not reproducible. Additionally, Eq. (2) defines chamfer distance with squared L2 distances, so the relationship between the loss used in training and the threshold used for evaluation should be clarified.","section":"§5.1, Eq. (2)"},{"comment":"The robustness claim for mask mixup is only weakly supported by the ablation: Table 4 shows that adding mask mixup changes ratio MAE from 0.5134 to 0.4722 and crypt MAE from 7.01 to 6.83, while villi MAE actually degrades from 16.98 to 14.39 only after also including other components. The sensitivity to the mixup beta range delta and to the weak segmentor capacity (trained to 50%) is not analyzed, and hyperparameters were selected on the validation set without a final held-out evaluation protocol that accounts for this selection. Please report the variance of these ablation results and the sensitivity of the main gains to the chosen delta range.","section":"§3.3 and Table 4"}],"minor_comments":[{"comment":"There are typographical and spacing inconsistencies in the method name ('M EASURE NET' vs 'MeasureNet'), and the abstract reports 81% multi-class accuracy while the conclusion reports 81.4%; please standardize all numbers.","section":"Abstract and Introduction"},{"comment":"The grade thresholds (Vd:Cd > 3, 1.05-3, 0.95-1.05, <0.95) are taken from a self-cited reference [7] without independent validation on this dataset; even if the label generation is clarified, the paper should discuss how threshold choices affect the reported accuracy, since small ratio errors near the boundaries can change the grade.","section":"§5.1"},{"comment":"Table 3 reports results for 2-, 3-, and 4-point polylines, but it is unclear whether the 2- and 4-point models were trained with the same total number of queries, loss weights, and training schedule as the 3-point model; the comparison should state the training protocol for each configuration.","section":"Table 3"},{"comment":"The dynamic time warping loss L_DTW is described as computed between 'predicted contour of VS and CB,' but the contour extraction from segmentation masks is not described; please specify how the contour is obtained and whether the DTW loss is differentiable with respect to the segmentation output.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The core measurement contribution is interesting and the dataset is potentially valuable, but the classification claim needs substantial clarification: the grade-label provenance is the single most important issue. If the grades are thresholded manual ratios, the paper should be reframed as a measurement paper with classification as a derived metric, rather than as an independent diagnostic validation. I would also encourage the editor to consider whether the clinical claim requires external validation or an inter-observer study, which is currently missing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the core idea is actually good: framing villi and crypt length measurement as polyline detection, with length and part-length losses aimed directly at the measurement objective, is a sensible departure from the usual segmentation-then-post-process pipeline. Second, the paper's diagnostic claims are shakier than the measurement claims, because the reported classification accuracy is just a deterministic function of the predicted villi-to-crypt ratio passed through thresholds from the authors' own prior work [7]. The stress-test note is right: Section 4 never defines how ground truth Marsh grades were obtained, and if they were generated by thresholding the manual polyline ratios, then Table 2's accuracy restates measurement error rather than testing a clinical hypothesis. What is genuinely new: the CeDeM dataset (750 duodenal biopsy images with 6,800 polyline annotations, villi shoulder, and crypt border) is a real contribution, and the object-driven losses - length loss and part-length loss plus chamfer distance - are a reasonable way to make a DETR-style detector optimize for measurement rather than box overlap. The ablation table is clear and shows each component pulling its weight, especially the crypt MAE improvement when mask guidance is added and the robustness gain from mask mixup. The qualitative figures support the claimed failure modes: mis-identified villi shoulder/crypt border, false positives on denuded villi, and curvature underfitting. That is honest error analysis. The soft spots are real but proportionate. Evaluation sits on a single 70-image test split with no error bars or repeated runs, so the 0.47 vs 0.70 ratio MAE and the 10-point mAP gains come with unknown variance. The classification thresholds themselves are taken from a self-cited study without external validation on this dataset or any inter-observer agreement on the manual annotations. That matters for the headline 'grading accuracy' but not for the measurement improvement, which is the paper's actual novel contribution. Also, the code and dataset are promised but not yet released; for this kind of work, the dataset release is part of the contribution and should be a condition of acceptance. Who this is for: anyone working on quantitative histopathology or measurement-oriented detection. It deserves a serious referee. I would send it out with a request for external validation of the grading thresholds, error bars or repeated runs, and a clear statement of how ground truth grades were constructed. The method is solid enough that those are fixable, not fatal.","headline":"A genuinely useful measurement-first framing for celiac disease grading with a new dataset, but the headline classification numbers are thresholded versions of the same ratio measurements and need independent validation before they carry diagnostic weight.","tokens_in":750,"tokens_out":739,"would_cite":true,"duration_ms":16070,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that celiac disease can be graded from duodenal biopsy images by predicting villi and crypt lengths as three-point polylines, and that the proposed MeasureNet system cuts villi-to-crypt ratio error from 0.70 to 0.47…","keywords":["celiac disease","villi-to-crypt ratio","polyline detection","histopathology","Marsh grading","biopsy image analysis","deep learning"],"falsifier":"A concrete test: take the 70 test images, have two additional pathologists redraw the villi and crypt polylines, and recompute the grade labels from their measurements; if MeasureNet's predicted grades agree with the original labels much more often than with the new pathologists' labels, the headline accuracy is an artifact of one annotation set rather than a robust measurement of disease severity.","tokens_in":13471,"feed_emoji":"🔬","tokens_out":5831,"duration_ms":47384,"temperature":0.7,"pith_summary":"MeasureNet is built on the idea that the lengths of the villi and crypts in a duodenal biopsy—the ratio of which is the clinical marker for celiac disease severity—can be estimated directly by predicting each structure as a three-point polyline, rather than by segmenting pixels or detecting straight lines. The paper introduces this polyline detection framework together with the CeDeM dataset of 750 annotated biopsy images, and reports that the system lowers the mean absolute error of the villi-to-crypt length ratio from about 0.70 to 0.47, raises detection mAP from 40.3 to 50.2, and improves multi-class Marsh grading accuracy from about 70% to 81.4% relative to the closest baseline. A sympathetic reader would care because accurate, less subjective measurement of villi and crypts could make celiac disease grading faster and more reproducible.","feed_headline":"Villi-crypt error drops from 0.70 to 0.47 with polyline detection","feed_subtitle":"MeasureNet grades celiac disease from biopsy images by predicting villi and crypt lengths as three-point polylines.","key_machinery":"The load-bearing object is a three-point polyline per villus or crypt, defined by start, middle, and end coordinates; the model is a transformer-based object detector adapted to output six coordinates per instance plus a class label. The argument runs through four interacting mechanisms: a localization loss on point coordinates, a Chamfer-distance loss that matches the predicted polyline to the ground-truth curve, and two object-driven losses that force the total length and the two segment lengths of the predicted polyline to match the ground truth; in addition, a frozen segmentation model supplies a mask of the villi shoulder and crypt border, and its features are mixed with features from a deliberately weaker segmentation model during training so that the detector does not depend too heavily on mask accuracy.","core_discovery":"The central claim is that villi and crypts in H&E-stained duodenal biopsy images can be represented and measured as polylines with a start, middle, and end point, and that a detection model trained to localize these polylines, with losses that directly compare predicted and ground-truth segment lengths, yields clinically usable villi-to-crypt ratios. On the newly collected CeDeM dataset, the system achieves a villi-crypt length-ratio MAE of 0.472, a mAP of 50.29, 82.66% accuracy for binary celiac detection, and 81.42% accuracy for multi-class Marsh grading, improving on segmentation-based, lane-detection, and line-segment-detection baselines. The paper attributes the gains to three ingredients acting together: object-driven length and part-length losses, auxiliary segmentation guidance that locates the villi shoulder and crypt border when crypts are only partially visible, and a mixup over strong and weak segmentation features that prevents over-reliance on the auxiliary mask.","pith_inferences":["One could replace the discrete Marsh thresholds with a continuous severity score, since the model already outputs a continuous ratio; this might preserve more information for monitoring treatment response.","The same polyline-plus-auxiliary-segmentation recipe could transfer to other histopathology measurements that rely on linear structures, such as glandular architecture or tumor dimensions.","A direct test of clinical usefulness would be measuring inter-observer agreement when multiple pathologists annotate the same CeDeM images, since the paper's ground-truth grades are tied to one annotation set.","If the dataset were expanded across centers and scanners, the robustness benefits of mask mixup would predictably matter more, because segmentation quality varies more across sites."],"forward_implications":["If the reported numbers hold, automated grading of celiac disease from duodenal biopsy images becomes feasible at an accuracy close to the manual ratio-based workflow.","A three-point polyline appears sufficient to capture villi and crypt curvature, since going to four points gives negligible measurement gain.","Auxiliary segmentation of landmark structures such as the villi shoulder and crypt border is beneficial for measurement, and exposing the detector to noisy masks during training makes that benefit robust.","The object-driven length losses, not just point localization, are what drive the ratio error down in the ablations."],"supporting_citations":[{"why":"Supplies the clinical thresholds that convert the measured villi-to-crypt ratio into normal, Marsh 1, Marsh 2, and Marsh 3 grades.","marker":"[7]"},{"why":"Supplies the base transformer detection model that MeasureNet adapts from bounding-box detection to polyline prediction.","marker":"[38]"},{"why":"Supplies the segmentation model used to predict villi-shoulder and crypt-border masks that guide crypt polyline prediction.","marker":"[34]"},{"why":"Defines the closest baseline, a transformer line-segment detector adapted to three-point prediction, against which the main gains are measured.","marker":"[35]"},{"why":"Provides the mixup technique used to blend strong and weak segmentation mask features to reduce exposure bias.","marker":"[37]"},{"why":"Focal loss is used as the classification loss to handle the class imbalance between villi and crypts.","marker":"[13]"},{"why":"Chamfer distance is used both as a loss to match polyline curvature and as the metric defining true-positive localization.","marker":"[9]"}],"fun_headline_variants":["Polyline detection cuts villi-crypt error by 33%","MeasureNet hits 82.66% accuracy in celiac grading","Celiac grading: polyline model cuts measurement error 33%","New AI model measures celiac severity from biopsy polylines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The weakest assumption is that the clinical grade follows deterministically from the villi-to-crypt ratio thresholds (normal, Marsh 1, Marsh 2, Marsh 3) and that the manually drawn polylines used as ground truth are reliable; if either the threshold mapping or the annotations are off, the reported grading accuracy does not reflect real-world diagnosis.","fun_headline_variants_meta":{"raw":{"variants":["Polyline detection cuts villi-crypt error by 33%","MeasureNet hits 82.66% accuracy in celiac grading","Celiac grading: polyline model cuts measurement error 33%","New AI model measures celiac severity from biopsy polylines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001163,"raw_usage":{"total_tokens":4859,"prompt_tokens":1035,"completion_tokens":3824,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":3749}},"tokens_in":651,"tokens_out":3824,"duration_ms":23162,"temperature":1.0,"reasoning_tokens":3749,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:36:30.551014+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: take the 70 test images, have two additional pathologists redraw the villi and crypt polylines, and recompute the grade labels from their measurements; if MeasureNet's predicted grades agree with the original labels much more often than with the new pathologists' labels, the headline accuracy is an artifact of one annotation set rather than a robust measurement of disease severity.","supporting_citations":[{"cited_title":"Quanti- tative histology-based classification system for assessment of the intestinal mucosal histological changes in patients with celiac disease","cited_arxiv_id":null,"evidence_quote":"Supplies the clinical thresholds that convert the measured villi-to-crypt ratio into normal, Marsh 1, Marsh 2, and Marsh 3 grades."},{"cited_title":"Segformer: Simple and efficient design for semantic segmentation with transform- ers","cited_arxiv_id":null,"evidence_quote":"Supplies the segmentation model used to predict villi-shoulder and crypt-border masks that guide crypt polyline prediction."},{"cited_title":"Line segment detection using transformers without edges","cited_arxiv_id":null,"evidence_quote":"Defines the closest baseline, a transformer line-segment detector adapted to three-point prediction, against which the main gains are measured."},{"cited_title":"Hierarchical recurrent attention networks for structured online maps","cited_arxiv_id":null,"evidence_quote":"Chamfer distance is used both as a loss to match polyline curvature and as the metric defining true-positive localization."}],"review_version":1}