{"id":"1681afeb-9f82-4a7c-8158-dd344d55f213","arxiv_id":"2501.05933","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A weakly supervised HRF segmentation framework using LRP-prompted SAM2 with iterative inference, where a Compact Convolutional Transformer backbone outperforms attention-based MIL.","lead":"Researchers combine a weakly supervised classifier with relevance maps to prompt SAM2 and segment tiny hyper-reflective foci in retinal OCT images, achieving Dice of 0.33. The framework is a step toward reducing annotation effort for small-structure medical segmentation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported Dice gains depend on the unmeasured point-level accuracy of the max-LRP-pixel prompt; if that pixel frequently misses HRFs, CCT-vs-MIL and SAM-vs-threshold conclusions both weaken.","rationale":"Both the reader and I identify the same weak point: unvalidated point-level localization of the LRP maximum. I considered alternative concerns (missing significance testing, confounded architecture comparison, no external baselines) but they affect the strength of the evidence, not the mechanism of the claim. The mechanism claim, that a relevance maximum can serve as a SAM 2 prompt, is load-bearing for every reported segmentation improvement. The paper's own oracle experiment (0.67 Dice with GT centers) shows how much depends on prompt placement, making the absence of a localization check conspicuous. Existing independent support is limited: no released code, no machine-checked proofs, and the quantitative tables lack error bars; the qualitative figure is illustrative only. A direct hit-rate measurement would settle whether the premise holds. Because the reader's verdict is already conditional on this concern, my assessment leaves the verdict unchanged.","tokens_in":5486,"tokens_out":4825,"duration_ms":50705,"concrete_test":"On the held-out test set, for CCT-2 and MIL-2, compute for each iteration: (1) the fraction of max-relevance prompts whose pixel lies inside any HRF mask (hit@0), and inside a radius of 3 or 5 pixels from an HRF; (2) the median distance from the prompt pixel to the nearest HRF mask; (3) the same Dice numbers when the prompt is replaced by a randomly chosen pixel in the retinal region. If hit@0 is high (more than 80%) for CCT and comparable for MIL, the assumption holds. If hit@0 is low, or if random-pixel prompts yield similar Dice, the claimed advantage of CCT/LRP prompting is not supported and the central comparison is confounded.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The framework's central mechanism (Sec. 2.3) is to prompt SAM 2 with a bounding box centered on the most relevant pixel from the LRP map. The paper never measures how often that pixel lies on an HRF. The reported results are consistent with the assumption, but not evidence for it. This matters because the same section reports a mean Dice of about 0.67 when prompting with ground-truth HRF centers, while the full pipeline gets 0.33 (Tab. 1): the 0.34 gap is likely dominated by prompt-localization error. If the max-relevance pixel frequently falls on background or on a non-HRF bright structure, then (i) the iterative occlusion loop (Sec. 2.3) cannot recover from the first miss, and (ii) the headline comparison of CCT (0.33) against MIL (0.13) and thresholding (0.16) may reflect differences in relevance-map noise rather than a genuine 'further and substantial increase' in segmentation ability. The grid search over crop/box sizes used all training HRFs, but it tunes the prompt geometry, not the probability that the chosen pixel is on an HRF. No point-localization metric (hit rate, distance to nearest HRF) is reported for either CCT or MIL. This is the load-bearing assumption: if it fails, the entire prompting strategy has no grounded mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a weakly supervised segmentation framework for hyper-reflective foci (HRF) in OCT images. The framework first trains a multiple instance learning (MIL) model or a compact convolutional transformer (CCT) on image-level labels (binary, multi-class, or count regression). Layer-wise relevance propagation (LRP) then produces pixel-level relevance maps, from which the most relevant pixel is used to define a bounding-box prompt for SAM 2; images are iteratively re-run after occluding detected HRFs. On a dataset of 962 HRF-positive B-scans, the authors report that the CCT with binary labels achieves a test Dice of 0.33 after six iterations, compared with 0.13 for MIL and 0.16 for simple relevance-map thresholding. They conclude that replacing MIL with CCT yields a substantial improvement in segmentation accuracy.","tokens_in":5727,"tokens_out":6149,"duration_ms":54416,"significance":"The proposed pipeline is methodologically transparent: prompt geometry is tuned on training data only, the threshold baseline is calibrated on validation data, and an oracle threshold is reported as an upper bound. The comparison of SAM 2 prompting against relevance-map thresholding is a useful sanity baseline, and the study covers three weak-label types. However, the paper's central mechanism -- prompting SAM 2 at the maximum-relevance pixel -- is not validated by any point-localization metric. The reported Dice gain of CCT over MIL and of SAM 2 over thresholding is therefore conditional on an untested assumption about the relevance localization quality. If the assumption holds, the framework is a practical step toward full-resolution weakly supervised segmentation of small structures; the paper provides evidence for the relative benefit of CCT over MIL, but not yet for the reliability of the prompting strategy.","major_comments":[{"comment":"The prompting strategy relies on the most relevant pixel from the LRP relevance map lying on an HRF, yet the paper never measures this point-level localization accuracy. The authors report a mean Dice of about 0.67 when prompting with ground-truth HRF centers (Sec. 2.3), while the full pipeline achieves 0.33 (Table 1), a 0.34 gap that is likely dominated by prompt-localization error. If the max-relevance pixel frequently falls on background or on non-HRF bright structures, the iterative occlusion loop cannot recover from the first miss, and the headline comparison between CCT (0.33) and MIL (0.13) may reflect differences in relevance-map noise rather than a genuine segmentation improvement. Please add a quantitative analysis of prompt localization (e.g., hit rate within a few pixels of an HRF, or distance from the selected pixel to the nearest HRF) for both CCT and MIL, and condition the segmentation results on whether the prompt hit.","section":"Sec. 2.3 and Table 1"},{"comment":"All classification and segmentation metrics are point estimates from a single 80/20 split. The paper's central claim that CCT 'consistently outperforms' MIL and that SAM 2 postprocessing yields a 'substantial increase' in Dice is made without confidence intervals or multiple-seed training. Given the small test set and the high variance typical of Dice on small structures, the reported differences (e.g., CCT 0.33 vs. MIL 0.13 in Table 1) need at least bootstrap confidence intervals over B-scans or a multi-seed analysis to assess stability.","section":"Section 3 and Table 1"},{"comment":"The paper cites TSSK-Net and MS-CAM as earlier weakly supervised segmentation methods, but it does not include any of these as a baseline. Without a comparison to at least one existing method, the contribution is positioned only against relevance-map thresholding and the MIL variant; the reader cannot judge whether the proposed framework advances the state of the art in weakly supervised HRF segmentation. If a direct comparison is infeasible, the authors should state this explicitly and justify the omission.","section":"Introduction and Section 3"}],"minor_comments":[{"comment":"In the Segmentation paragraph, 'even more pronounced than for the segmentation task' should presumably read 'than for the classification task'; the sentence is comparing segmentation differences to classification differences.","section":"Section 3"},{"comment":"The phrase 'manually annoted' is a typo for 'manually annotated'.","section":"Section 2.1"},{"comment":"The stopping threshold of 0.05 and the maximum of six iterations are introduced without explaining how these values were chosen; please state whether they were fixed a priori or tuned on validation data.","section":"Section 2.3"},{"comment":"The sentence 'Comparing Dice scores to those from an oracle-based thresholding ... suggests that we cannot expect to outperform SAM 2' confuses the two postprocessing approaches; it should read '...cannot expect to outperform the oracle thresholding'.","section":"Section 4"},{"comment":"The columns 'Segm. 1 Iteration' and 'Segm. 6 Iterations' should clarify that the latter is subject to the early-stopping threshold; reporting the average number of iterations would help interpret the recall/precision trade-off.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"This is a short conference paper, and the missing point-localization analysis could be added with a modest set of experiments. The lack of comparison to existing weakly supervised baselines is a concern for a methods paper, but the central comparison (CCT vs. MIL) is internally consistent. I recommend major revision to require the localization analysis and statistical uncertainty reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing here is the crop-and-upsample prompting strategy: SAM2 prompted with a box around the max-LRP pixel, on an upsampled crop, with iterative occlusion. That is a genuine adaptation to SAM2's weakness on small objects, and the paper reports it clearly. The comparison across three label types (2-class, 3-class, regression) is also a nice empirical sweep, and the oracle-threshold upper bound in Table 2 is an honest reference point.\n\nThe soft spot is exactly the one the stress test flags. The pipeline assumes the single most relevant pixel from LRP actually lies on an HRF. The paper never measures that. The internal evidence says it matters: prompting with ground-truth HRF centers gives mean Dice around 0.67, while the full pipeline gets 0.33. The 0.34 gap is almost certainly dominated by prompt-localization error. If the max-relevance pixel frequently lands on background or a bright non-HRF structure, then the iterative loop starts from a miss and cannot recover, and the headline comparison (CCT 0.33 vs MIL 0.13) may just be comparing levels of relevance-map noise. The paper needs a point-localization metric, say hit rate or distance to nearest HRF, for both CCT and MIL.\n\nOther concerns are more ordinary. Single 80/20 split with no confidence intervals or multiple seeds; absolute Dice values are modest; no comparison to existing weakly supervised OCT segmentation methods like TSSK-Net at comparable resolution. The grid search for crop/box sizes was on training HRFs, which is legitimate for tuning prompt geometry, but it does not validate the localization assumption.\n\nNone of this kills the paper. The comparative claims are internally consistent, and the framework is plausible. It is a conference-level contribution for the OCT/weakly-supervised crowd, not a field-shifting result. A serious referee should ask for statistical rigor and the missing localization measurement, but the work deserves that referee time.\n\nRecommendation: engage with it; send it to review, with the expectation of heavy revision.","headline":"The crop-and-upsample SAM2 prompting is a genuine technical adaptation, but the paper never measures whether the max-LRP pixel actually lands on an HRF, and that unmeasured assumption likely explains most of the reported gains.","tokens_in":6384,"tokens_out":2257,"would_cite":true,"duration_ms":21879,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims hyper-reflective foci in OCT can be segmented from image-level labels alone by prompting SAM2 with the most relevant pixel from a relevance map, and that a compact convolutional transformer more than doubles Dice over…","keywords":["weakly supervised segmentation","hyper-reflective foci","optical coherence tomography","layer-wise relevance propagation","Segment Anything Model 2","compact convolutional transformer","multiple instance learning","iterative inference"],"falsifier":"For each test B-scan, compare the maximally relevant pixel against the annotated HRF masks and compute the fraction of prompts that fall inside an HRF; if that fraction is low yet SAM2 still outputs useful masks, the reported Dice gain is not coming from the claimed mechanism. A complementary ceiling check is to prompt SAM2 with the true centers of all annotated HRFs and compare the resulting masks with the weakly supervised ones.","tokens_in":5278,"feed_emoji":"👁","tokens_out":8390,"duration_ms":75467,"temperature":0.7,"pith_summary":"The paper asks whether hyper-reflective foci (HRF) — tiny bright spots in retinal OCT scans that flag progression of age-related macular degeneration — can be segmented without any pixel-level training labels. It proposes a pipeline that trains a full-resolution image-level classifier, uses layer-wise relevance propagation to build a per-pixel relevance map, and feeds the most relevant pixel as a prompt to SAM2, which returns the final mask; found foci are then occluded and the process repeats to find additional foci. The authors report that the best variant (a Compact Convolutional Transformer trained on binary HRF-present/absent labels) reaches a test Dice of 0.33, versus 0.13 for attention-based MIL and 0.16 for simple relevance-map thresholding, and close to the 0.35 an oracle threshold with ground-truth guidance achieves. The practical stake is that clinicians could monitor HRF burden in large OCT datasets with image-level annotations only, which are far cheaper than the pixel-level masks currently required.","feed_headline":"Tiny retinal spots hit 0.33 Dice with image-level labels only","feed_subtitle":"A relevance-map-prompted SAM2 beats attention-MIL by 0.20 Dice and nears ground-truth oracle.","key_machinery":"The load-bearing object is the relevance map produced by layer-wise relevance propagation (LRP), a backpropagation-style rule that assigns each input pixel a score measuring how much it drove the image-level prediction. LRP turns the classifier's global decision into per-pixel evidence, and the pixel with the highest relevance becomes the SAM2 prompt; without this map there is no spatially precise cue for the foundation model. Around the prompt, a fixed-size box (4 pixels) and a cropped, upsampled region (50–100 pixels) make the HRF large enough for SAM2, which is otherwise poor at small objects. The supporting machinery is the Compact Convolutional Transformer (CCT), which forms tokens through convolutional layers and includes positional encoding, allowing attention to exchange information between patches so that small structures on patch borders are less likely to be missed.","core_discovery":"The central discovery, in the authors' framing, is that a coarse image-level decision can be converted into a precise pixel-level segmentation by treating the classifier's explanation as a prompt for a foundation model. They implement this with layer-wise relevance propagation on either an attention-based MIL classifier or a Compact Convolutional Transformer, take the pixel of maximum relevance, crop and upsample a local region so that the tiny HRF becomes large enough for SAM2, and use a small box around that pixel as the prompt. Iterative inference occludes already-detected HRFs and re-runs the pipeline to find additional foci, which mainly helps images containing more than one HRF (Dice 0.22 after the first iteration, 0.28 after three). Replacing MIL with CCT, which adds positional encoding and lets different image regions exchange information through attention, raises Dice from 0.13 to 0.33 on the 2-class task; the authors interpret this as the CCT catching HRFs that fall on patch borders. They also report that binary presence/absence labels are sufficient, with multi-class or count-based labels offering no additional segmentation benefit.","pith_inferences":["The paper does not test whether the maximally relevant pixel usually lands inside an HRF; computing that hit rate on the test set would isolate whether the Dice gain comes from accurate localization or from SAM2 segmenting plausible bright spots even when prompted off-target.","The closeness of the SAM-based Dice (0.33) to the oracle-threshold ceiling (0.35) suggests that with a perfect prompt the remaining gap is set by SAM2's mask quality, pointing to fine-tuning SAM2 on small objects or using multiple prompt points as unexplored next steps.","The CCT was trained from scratch while the MIL encoder used ImageNet-pretrained weights, so the reported architecture advantage may partly reflect training protocol; an ablation with a from-scratch full-resolution MIL would separate the two."],"forward_implications":["Binary image-level labels are enough: adding extra classes or an HRF-count regression target does not improve segmentation Dice.","Prompting SAM2 with the most relevant LRP pixel substantially outperforms thresholding the relevance map directly (0.33 vs 0.16 Dice), and reaches near the 0.35 ceiling of an oracle threshold tuned per sample with ground truth.","Iterative inference mostly helps difficult multi-HRF images, raising their Dice from 0.22 after one iteration to 0.28 after three, while on the average B-scan recall gains are offset by precision losses.","Full-resolution processing matters: downsampling to 256×256 or 224×224 would discard 16% or 22% of annotated HRFs, so a full-resolution classifier like CCT is a prerequisite for this approach.","The authors expect the same pipeline to transfer to other small-structure segmentation problems in OCT, not just HRF."],"supporting_citations":[{"why":"Supplies the attention-weighted multiple instance learning baseline that the framework augments with LRP prompts.","marker":"[7]"},{"why":"Provides layer-wise relevance propagation, which converts the classifier's image-level decision into per-pixel relevance maps.","marker":"[8]"},{"why":"Is the SAM2 foundation model that turns the relevance-derived prompt into the final segmentation mask.","marker":"[9]"},{"why":"Introduces the Compact Convolutional Transformer architecture whose full-resolution tokenization and positional encoding drive the reported accuracy gain.","marker":"[10]"},{"why":"Extends LRP to transformer architectures, which the paper needs to compute relevance maps for the CCT.","marker":"[13]"},{"why":"Supplies the LEAD-study OCT volumes used for training and testing.","marker":"[11]"},{"why":"Provides the manual HRF annotations and motivates the exclusion of spots smaller than five pixels.","marker":"[12]"}],"fun_headline_variants":["CCT boosts weakly-supervised retinal foci Dice from 0.13 to 0.33","Image-level labels only? LRP-prompted SAM2 hits 0.33 Dice on HRF","Weakly supervised fine segmentation: LRP+SAM2 on OCT foci","Iterative SAM2 with LRP finds more retinal spots, raises Dice to 0.28"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the most influential pixel identified by the classifier's pixel-level explanation actually lies inside a hyper-reflective focus whenever the classifier says one is present, and the paper never measures that point-level hit rate.","fun_headline_variants_meta":{"raw":{"variants":["CCT boosts weakly-supervised retinal foci Dice from 0.13 to 0.33","Image-level labels only? LRP-prompted SAM2 hits 0.33 Dice on HRF","Weakly supervised fine segmentation: LRP+SAM2 on OCT foci","Iterative SAM2 with LRP finds more retinal spots, raises Dice to 0.28"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001444,"raw_usage":{"total_tokens":5811,"prompt_tokens":934,"completion_tokens":4877,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":4781}},"tokens_in":550,"tokens_out":4877,"duration_ms":28614,"temperature":1.0,"reasoning_tokens":4781,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:05:40.472963+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For each test B-scan, compare the maximally relevant pixel against the annotated HRF masks and compute the fraction of prompts that fall inside an HRF; if that fraction is low yet SAM2 still outputs useful masks, the reported Dice gain is not coming from the claimed mechanism. A complementary ceiling check is to prompt SAM2 with the true centers of all annotated HRFs and compare the resulting masks with the weakly supervised ones.","supporting_citations":[{"cited_title":"Attention-Based Deep Multiple Instance Learning","cited_arxiv_id":null,"evidence_quote":"Supplies the attention-weighted multiple instance learning baseline that the framework augments with LRP prompts."},{"cited_title":"On Pixel-Wise Ex- planations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation","cited_arxiv_id":null,"evidence_quote":"Provides layer-wise relevance propagation, which converts the classifier's image-level decision into per-pixel relevance maps."},{"cited_title":"XAI for Transformers: Better Explanations through Conservative Propagation","cited_arxiv_id":null,"evidence_quote":"Extends LRP to transformer architectures, which the paper needs to compute relevance maps for the CCT."},{"cited_title":"Ophthalmology","cited_arxiv_id":null,"evidence_quote":"Supplies the LEAD-study OCT volumes used for training and testing."},{"cited_title":"Hyper- reflective Foci not Seen as Hyperpigmentary Abnormalities on Color Fundus Photographs in Age-Related Macular Degeneration","cited_arxiv_id":null,"evidence_quote":"Provides the manual HRF annotations and motivates the exclusion of spots smaller than five pixels."}],"review_version":1}