{"id":"0327cc38-09c7-40ed-bde7-ea7174dc582f","arxiv_id":"2507.06643","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new 'Crag and Tail' loss improves sparse-label training for dense carcinosis keypoint localization in ovarian cancer laparoscopy.","lead":"This paper proposes a loss function that lets a neural network learn to spot all cancer deposits in laparoscopic videos of advanced ovarian cancer from only a few labeled points per frame. It reports better localization recall than standard MSE and Hill losses on a new private dataset of 30 surgical videos.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed state-of-the-art gain of Crag and Tail over Hill at λ=1 (F1 54.59 vs 52.06) comes from a single training run on 6 test videos, with no error bars or significance test; the difference is within plausible seed/video noise.","rationale":"Read in good faith, the paper introduces a clinically motivated task, formulates a plausible loss, and reports an extensive ablation. The qualitative figures and the fact that recall (46.76) far exceeds the 5-point annotation density suggest the model does find unlabeled carcinosis instances. However, the strongest claim in the abstract and contributions is about state-of-the-art performance, and that claim rests entirely on Table 1. With 6 test videos, one run per condition, and the best λ selected on validation, a 2.5-point F1 gap over Hill is not distinguishable from noise; this is a correctness risk rather than a disagreement with community conventions. The reader's concern about appearance consistency is legitimate but longer-term; if the within-dataset gain is not statistically reliable, the generalization question becomes moot. Therefore the conditional verdict remains appropriate: the paper is promising, but the headline comparison needs variance estimates, multi-seed runs, and ideally code release before the claim can be accepted as established.","tokens_in":8418,"tokens_out":7307,"duration_ms":84269,"concrete_test":"Re-run Hill(λ=1) and Crag-and-Tail(λ=1) with the same training protocol for at least 5 random seeds, then compute per-video F1 on the 6 test videos and a paired bootstrap (resampling videos with replacement) 95% confidence interval for the mean F1 difference. If the interval contains 0, or if the sign of the difference flips across seeds, the state-of-the-art claim is not supported. A leave-one-video-out analysis would additionally show whether the ranking depends on a single test video. Releasing trained checkpoints or code would make this check independently reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central claim of state-of-the-art dense carcinosis localization, Table 1 must show that the Crag-and-Tail advantage over the strongest baseline is a real effect, not sampling noise. The evidence is currently a single point estimate per loss on a test set of 6 videos (279 frames with strong within-video correlation). At the tuned λ=1, Crag and Tail reaches F1 54.59, recall 46.76, precision 73.79, versus Hill with F1 52.06, recall 44.01, precision 73.89: the F1 gain is 2.53 points and precision is actually slightly lower. At the default λ=1.5 the gain is larger in relative terms (39.84 vs 35.23), but both numbers are much lower and no variance is reported. λ was tuned on the validation split for both methods, so the comparison is partially selected; without nested resampling or multiple seeds, the reported ranking may not reproduce. The inference also uses different thresholds (t=0.2 for MSE, t=0.3 for all other losses), which is defensible only if threshold selection is part of a fixed protocol and its effect is reported. Since the central claim is an empirical ranking, the missing error bars, single seed, and effective sample size of 6 test videos are the most load-bearing weakness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a new task of dense carcinosis keypoint localization in laparoscopic video frames for advanced ovarian cancer assessment, where only a few sparse point annotations are available per training image. To address the sparse-label setting, the authors adapt the Hill loss, originally designed for multi-label classification with missing labels, to dense heatmap regression, and propose a new loss function named Crag and Tail loss that adds a positive reinforcement term (H - H_hat)^2 to the Hill loss. The method is evaluated on a newly collected dataset of 30 videos, comparing against MSE and Hill baselines, with metrics for point localization and multilabel station classification. The paper reports that Crag and Tail loss achieves the highest recall and F1 for point localization, and supports the claim with an ablation study of loss components.","tokens_in":8709,"tokens_out":6359,"duration_ms":63280,"significance":"If the reported results hold, the work is practically significant because it reduces annotation effort for dense localization tasks in surgical imaging, and it introduces a clinically relevant new task. The adaptation of Hill loss to dense heatmap regression is a reasonable idea, and the ablation study provides some insight into loss design. However, the empirical support is currently limited: the evaluation is based on a single training run, a test set of only 6 videos, and comparisons that involve different inference thresholds and tuned hyperparameters, so the claimed state-of-the-art improvement is not yet established. No code or trained models are released, which limits reproducibility.","major_comments":[{"comment":"The central claim of state-of-the-art performance rests on a single training run per loss evaluated on 6 test videos (279 frames) with no confidence intervals, error bars, or significance tests. At the tuned lambda=1 setting, Crag and Tail improves point localization F1 over Hill by 2.53 points (54.59 vs 52.06) but has slightly lower precision (73.79 vs 73.89) and lower multilabel F1 (77.88 vs 78.03). Given the strong within-video correlation of the test frames, these differences are within plausible sampling noise. Please provide results across multiple training seeds or cross-validation folds, report variance, and use a paired statistical test across videos (e.g., Wilcoxon signed-rank) to support the superiority claim.","section":"Table 1, Section 3"},{"comment":"The inference procedure selects the top k=30 predicted points per image, while Section 3 states that test images contain on average 36 carcinosis instances. This imposes an upper bound on recall of roughly 30/36 = 83% even with perfect localization, and the cap may affect different losses unevenly. The authors should either increase k to a value above the maximum number of instances in any test image (e.g., 50 or 100) and report results, or explicitly justify the choice of k=30 and analyze its influence on the reported recall and F1.","section":"Section 3, Implementation Details"},{"comment":"The proposed Crag and Tail loss mixes unnormalized raw network outputs H_hat in the reinforcement term (H - H_hat)^2 with sigmoid-transformed values H_hat+ and H_hat- in the other terms. HRNet outputs are unbounded, so the squared term may have a different scale and could dominate the loss or lead to training instability. Please clarify whether this mixed use is intentional, report the typical range of raw outputs during training, or use a consistent transformation (e.g., (H - H_hat+)^2) so that all terms operate on comparable scales.","section":"Section 2.2, Eq. (2)"},{"comment":"The comparison between losses is confounded by method-specific inference thresholds (t=0.2 for MSE, t=0.3 for all others) and by hyperparameter tuning: Crag and Tail uses lambda=1 while Hill uses lambda=1.5 in the main comparison, and the paper states that tuned lambda=1 results for both are provided without describing the tuning grid, the validation metric, or the selection procedure. This makes it difficult to attribute the observed differences to the loss function rather than to threshold or hyperparameter choices. Please report the tuning protocol and show results across a range of thresholds (e.g., precision-recall curves) or use a common threshold selection procedure for all methods.","section":"Section 4, Table 1; Section 3, Implementation Details"}],"minor_comments":[{"comment":"The log( H_hat+ ) and log( H_hat ) terms should include a small epsilon inside the logarithm to avoid numerical instability when the predicted probability is zero.","section":"Eq. (1), Eq. (2)"},{"comment":"The notation Y* in R^{2 x n} uses n for the number of keypoints, but later in Section 2.2 the text uses n=0 and n>0 to indicate whether any carcinosis points exist; please use distinct symbols or clarify the overloading.","section":"Section 2.1"},{"comment":"The sentence 'For a frame-level indication of whether the station contains carcinosis or not, 2D keypoint annotations are used for 1-5 prominent cancerous keypoints inside the masks' is unclear about how the keypoints relate to the frame-level labels; please rephrase to explain the annotation protocol.","section":"Section 3"},{"comment":"The ablation rows such as 'Hlog(H_hat+) = 0' and 'Only reinforce term in pos' are not self-explanatory; please include the exact modified loss equations or reference the components of Eq. (2) that are removed or changed.","section":"Table 2"},{"comment":"The caption for the qualitative results does not describe the content; it should specify which methods are compared, what the markers indicate, and what the reader should observe.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript states it has been accepted for publication in MICCAI 2025. If this is being submitted as a journal version, the editor may wish to consider the overlap with the conference publication. The novelty is moderate, but the main weakness is the lack of statistical rigor in the evaluation: a single seed, six test videos, and method-specific thresholds. These issues are fixable with additional experiments and analysis, so I do not recommend rejection, but the current evidence does not support the strong claim of state-of-the-art performance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-scoped, clearly written paper with a genuinely new loss variant (Crag and Tail) and a new sparse-label dense-localization task for ovarian cancer laparoscopy. The 'state-of-the-art' wording, though, is ahead of the evidence as reported.\n\nThe new thing is small but real: adapting Hill loss to dense heatmap regression and adding a squared-error reinforcement term for true positives. The paper also introduces a new clinical task and a private dataset, with dense masks on test frames for evaluation. I credit them for that; it is a useful contribution to surgical AI.\n\nThe soft spots are mostly about statistical evidence. The main comparison is a single run over six test videos. At the tuned lambda=1, Crag and Tail beats Hill on point F1 by 2.5 points (54.59 vs 52.06), but precision is slightly lower (73.79 vs 73.89) and multilabel F1 is slightly worse (77.88 vs 78.03). At lambda=1.5, the gain is larger (39.84 vs 35.23) but both numbers are far lower. No error bars, no multiple seeds, no significance test, and with strong within-video correlation the effective sample size is closer to 6 than 279. The threshold difference (t=0.2 for MSE, 0.3 for all others) is another tunable degree of freedom that is not fully explored. None of this is fatal, but it means the SOTA claim is not established.\n\nThe dataset is small and not released. The gap between about 5 sparse points per training image and 36 true instances per test image is large, and the paper does not analyze whether missed instances look systematically different from annotated ones. That is the main risk to generalization. The loss design itself is not circular; hyperparameters are tuned on validation and evaluation is independent. The ablation is useful, though it shows lambda is fragile: F1 drops from 54.59 to 39.84 between lambda=1 and lambda=1.5.\n\nWho is this for? Researchers working on sparse-point supervision in medical imaging, and anyone building intraoperative decision support for ovarian cancer. The dataset/task alone justifies submission. It deserves a serious referee. For review, I would ask for multiple seeds with variance estimates, a per-video breakdown, a fixed threshold protocol, and code or a detailed annotation protocol. If the 2.5-point F1 advantage disappears across seeds, the paper still stands as a new task plus a reasonable baseline, but not as a SOTA method.","headline":"A genuinely new sparse-label loss and clinical task, but the SOTA claim rests on one seed and six test videos.","tokens_in":9271,"tokens_out":3255,"would_cite":false,"duration_ms":37016,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that a heatmap-regression loss called Crag and Tail, trained on only about five sparse carcinosis point labels per frame, densely localizes carcinosis nodules in laparoscopic video and outperforms the MSE and Hill…","keywords":["sparse annotations","keypoint localization","heatmap regression","carcinosis","ovarian cancer","laparoscopy","noisy labels","surgical AI"],"falsifier":"One concrete experiment would be to compute per-station recall on the 279 test frames for Crag and Tail($\\lambda=1$). If stations that supplied few or no point labels during training show disproportionately low recall relative to the 46.76% average, the assumption that five sparse points generalize to all ~36 carcinosis instances would be disproved for those stations.","tokens_in":8238,"feed_emoji":"📍","tokens_out":13566,"duration_ms":121436,"temperature":0.7,"pith_summary":"The paper seeks to establish that dense localization of carcinosis nodules in laparoscopic video can be learned from only a few sparse point annotations per frame, without dense pixel-level labels. To do this it introduces a heatmap-regression task and a loss function, Crag and Tail, that adapts the multi-label Hill loss by down-weighting possible missed annotations while adding a term that reinforces true positives. On the authors' dataset, training on about five points per image with this loss reaches point-localization F1 of 54.59 and recall 46.76, beating the Hill and MSE baselines, and it also improves station-level classification used in Fagotti-score assessment. The result matters because full pixel-level annotation of a surgical image can take up to 90 minutes, whereas sparse points are quick for clinicians to provide; if the claim holds, sparse annotation could support automated ovarian-cancer assessment.","feed_headline":"Five point labels per frame enable dense carcinosis localization","feed_subtitle":"A new loss, Crag and Tail, raises recall to 46.76 and F1 to 54.59 on the point-localization task.","key_machinery":"The central object is the Crag and Tail loss, a dense heatmap-regression loss that combines a false-negative-suppressing negative term inherited from Hill loss with a new positive-reinforcement term. The loss is $$L_{\\mathrm{CragAndTail}}=\\sum_p \\big(H\\log(\\hat{H}_+)+(H-\\hat{H})^2\\big)(1-\\hat{H}_+)^\\gamma + (1-H)\\big(-(\\$\\lambda$-\\hat{H}_-)\\hat{H}_-^2\\big),$$ with $\\hat{H}_+=\\sigma(\\hat{H}-m)$ and $\\hat{H}_-=\\sigma(\\hat{H})$, where $H$ is the sparse Gaussian-peak ground-truth heatmap and $\\hat{H}$ the predicted heatmap. The first factor and the negative term jointly down-weight loss on pixels that are probably missed annotations, while the $(H-\\hat{H})^2$ term prevents the focal positive term from under-weighting easy true positives. At inference, the predicted heatmap is processed by 5x5 non-maximum suppression followed by a threshold $t$ to obtain point predictions.","core_discovery":"The central discovery is that a model trained on only a few sparse point annotations per frame can densely localize all carcinosis instances if the loss is designed to treat unlabeled positives as false negatives and to keep true positives strongly weighted. The paper adapts the Hill loss, originally for multi-label classification with missing labels, to pixel-level heatmap regression and adds a positive-reinforcement term $(H-\\hat{H})^2$ to obtain the Crag and Tail loss. On the authors' test set, at the tuned false-negative weight $\\lambda=1$, this loss reaches point-localization F1 54.59 and recall 46.76, versus 52.06 and 44.01 for Hill and 18.99 and 15.83 for MSE, with similar or better station-level multilabel classification at default settings.","pith_inferences":["A natural extension is per-station recall evaluation: because the training set averages five points over six anatomical stations, stations that contributed few or no labeled points may be under-recovered, and the paper's reported recall gain may not be uniform across stations.","The same loss could transfer to other sparse-instance detection tasks where annotators mark only a subset of instances, such as counting cell nuclei or detecting polyps, but the optimal $\\lambda$ would need retuning to the missing-label rate.","Comparing Crag and Tail against strategic point placement, such as one point on each of five stations rather than five points clustered on one station, would show whether the loss or the annotation distribution drives the observed gain."],"forward_implications":["At $\\lambda=1$, Crag and Tail raises point-localization recall by 2.75 points over Hill (46.76 vs. 44.01) while keeping precision nearly equal (73.79 vs. 73.89).","The ablation that sets the positive-reinforcement term to zero retains only the negative loss and drops F1 to 11.89, and removing the original positive Hill term drops F1 to 11.98, so both positive terms are necessary for the reported performance.","Setting $\\lambda=1$ rather than the default 1.5 improves F1 from 39.84 to 54.59, showing that the false-negative suppression strength is a key tunable for this sparse-label regime.","On the station-level multilabel task, Crag and Tail at default $\\lambda$ reaches F1 81.84 versus Hill's 80.06 and MSE's 48.22, suggesting the dense localization gains carry over to the clinically relevant Fagotti-score-assessment proxy.","The 0.5 Masked MSE baseline, which naively halves the loss on assumed false negatives, improves over MSE from F1 18.99 to 29.70, confirming that false-negative handling is the main driver of the sparse-label difficulty."],"supporting_citations":[{"why":"Hill loss for multi-label learning with missing labels; its false-negative down-weighting is the starting point that Crag and Tail adapts and extends.","marker":"[26]"},{"why":"HRNet backbone used for heatmap regression in all experiments.","marker":"[24]"},{"why":"Survey of learning from noisy labels; supports the choice of loss reweighting over clean-label requirements or geometric priors.","marker":"[22]"},{"why":"Documents the up-to-90-minute cost of pixel-level annotation in Cityscapes, motivating the sparse-label setting.","marker":"[5]"},{"why":"Define the Fagotti score and its six anatomical stations, the clinical scoring system the localization task is intended to support.","marker":"[6–8]"}],"fun_headline_variants":["Sparse point labels drive dense ovarian cancer mapping","Crag and Tail loss maps all carcinosis from few points","Few point labels enough for dense carcinosis localization","Sparse annotation to dense localization in ovarian cancer","Crag and Tail loss gives dense localization from sparse points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the roughly five carcinosis points a surgeon marks per frame are representative of all 36 carcinosis instances actually present, so the model can learn to detect the unlabeled ones from these examples.","fun_headline_variants_meta":{"raw":{"variants":["Sparse point labels drive dense ovarian cancer mapping","Crag and Tail loss maps all carcinosis from few points","Few point labels enough for dense carcinosis localization","Sparse annotation to dense localization in ovarian cancer","Crag and Tail loss gives dense localization from sparse points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00055,"raw_usage":{"total_tokens":2617,"prompt_tokens":928,"completion_tokens":1689,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":1613}},"tokens_in":544,"tokens_out":1689,"duration_ms":13511,"temperature":1.0,"reasoning_tokens":1613,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:57:19.491135+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete experiment would be to compute per-station recall on the 279 test frames for Crag and Tail($\\lambda=1$). If stations that supplied few or no point labels during training show disproportionately low recall relative to the 46.76% average, the assumption that five sparse points generalize to all ~36 carcinosis instances would be disproved for those stations.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Hill loss for multi-label learning with missing labels; its false-negative down-weighting is the starting point that Crag and Tail adapts and extends."},{"cited_title":"IEEE transactions on neural networks and learning systems 34(11), 8135–8153 (2022)","cited_arxiv_id":null,"evidence_quote":"Survey of learning from noisy labels; supports the choice of loss reweighting over clean-label requirements or geometric priors."}],"review_version":1}