{"id":"9d047556-d5f8-4b68-81ce-8a170aa8c716","arxiv_id":"2608.07299","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"EliSeg constructs segmentation targets directly from unfiltered radiology reports and verifies them with a text-only checker before generating masks.","lead":"EliSeg is a system that reads an unfiltered radiology report and a chest X-ray, decides which findings actually deserve a segmentation mask, and then draws the masks without being told what to look for. It outperformed current methods on a chest X-ray benchmark and rejected most mentions that should not be segmented.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The k-th slot is assumed to be the k-th positive finding under a fixed canonical order, but the model never predicts or verifies slot identity; the consistency gate cannot catch count-preserving swaps, so mask-to-finding correspondence is the central unverified claim.","rationale":"The paper is unusually transparent and internally consistent; I do not see a separate failure that outweighs the reader's concern. The strongest unresolved technical risk is the slot-to-finding correspondence, and the reader identified it correctly. My independent reading agrees: the canonical-order binding is the only mechanism connecting masks to findings, yet the system never predicts or verifies slot identity, and the consistency gate only checks action and cardinality. The paper's own Sec. 5 admits slot-level verification is missing. The proposed check directly tests the binding in the exact regime where the gate is silent: multi-finding sentences with agreeing Actor and Verifier. If the canonical binding holds, the concern is resolved. If not, the central claim that EliSeg accurately localizes 'the corresponding abnormalities' is overstated, because aggregate and even per-finding IoU can be robust while individual masks are attached to the wrong findings. I also credit the paper's independent support: the Actor is parameter-identical to ROSALIA (Sec. A.6), the ablation in Table 4 is clean, per-finding results in Table 2 are consistently strong, and the statistical limitations in Sec. B.3 are disclosed honestly. None of these removes the identity-assumption risk, so the conditional verdict stands without change.","tokens_in":19928,"tokens_out":6857,"duration_ms":78368,"concrete_test":"On the MIMIC-CXR-ILS test split, take all multi-finding sentences for which the consistency gate does not fire (Actor and Verifier agree on action and cardinality). For each such sentence, decode all slot masks and compute the IoU of every slot against every reference mask for that sentence's positive findings. Run maximum-weight bipartite matching between predicted slots and reference findings, and compare the matched assignment with the canonical-order assignment. If the assignments differ in more than 5% of sentences, or if pooled IoU under the best matching exceeds the reported pooled IoU by more than 1 point, the learned binding is not the assumed canonical order and the correspondence claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"EliSeg's headline result requires that each generated mask be attached to the correct finding. The only mechanism for this is the canonical-order binding in Sec. 3.3/A.2: slot k is supervised and later evaluated as the k-th positive finding under the fixed vocabulary order (cardiomegaly, edema, effusion, atelectasis, lung opacity, pneumonia, consolidation). The Verifier (Sec. 3.4) produces an inventory in the same order, and Revision (Sec. 3.5) supplies only the revised cardinality, explicitly without injecting finding names, and re-executes the Actor teacher-forced. No component ever predicts a slot's semantic identity, and the consistency gate compares only action and cardinality. A swap or substitution that preserves count is therefore invisible to the system, exactly as the paper concedes in Sec. 5 ('slot-level verification could provide finer correction'). The assumption is load-bearing: Table 2's per-finding IoU values are assigned to findings using this same canonical order, so those numbers cannot independently validate the binding. The risk is concrete in multi-finding or negated sentences where mention order and canonical order disagree, and it is sharpest at slot 3, where the autoregressive decode rate is only 36.4% (Sec. A.4/B.2) and Revision must force the slot without any identity check.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EliSeg, a propose-verify-revise framework for report-grounded abnormality segmentation on chest radiographs. Given an image and an unfiltered report sentence, a grammar-constrained Actor proposes a sentence-level action, a target cardinality, and provisional masks through a fixed set of control tokens; a frozen text-only Verifier independently reconstructs the eligible finding inventory; a deterministic gate compares action and cardinality, and when they disagree, Revision teacher-forces a corrected control sequence through the same Actor. The method is evaluated on MIMIC-CXR-ILS under report-inferred, gold-inventory, and native-prompt settings, with additional zero-shot given-target transfer on CheXlocalize. The reported results show large improvements over direct text-based segmentation and extract-then-segment cascades, with detailed appendices on annotation, metrics, inference cost, and parameter accounting.","tokens_in":20268,"tokens_out":5960,"duration_ms":64272,"significance":"If the results hold, EliSeg addresses a genuine and under-studied problem: constructing executable segmentation targets from radiology reports without a hidden target oracle. The paper is unusually transparent in several respects: it reports a shared metric implementation, retains missing predictions as empty masks, accounts for parameter counts and inference cost, releases code, and includes an explicit limitations section. These strengths make the empirical comparisons reproducible within the chosen evaluation universe. However, the central claim of correct finding-to-mask correspondence rests on an unverified canonical-ordering assumption, and the empirical evidence is based on single deterministic runs without variability estimates or a fully specified balanced sampling procedure. The contribution is potentially significant, but the current manuscript needs revision before the central claim is fully supported.","major_comments":[{"comment":"The finding-to-mask correspondence is never explicitly predicted or verified. Slot k is assumed to correspond to the k-th positive finding under the fixed canonical vocabulary order, the Verifier supplies only cardinality and action, and Section 3.5 states that verified finding names are not injected into the Actor. Consequently, any count-preserving permutation of mask identities is invisible to the consistency gate and to Revision. The paper itself concedes in Section 5 that such semantic errors may remain undetected. This assumption is load-bearing because Table 2 assigns per-finding IoU values to slots using the same canonical order, so those numbers cannot independently validate the binding. I recommend adding an explicit slot-level identity check or a human-audited correspondence analysis, or otherwise demonstrating that slot identity errors are rare in practice.","section":"Sec. 3.3, 3.5, A.2"},{"comment":"All comparisons are single deterministic runs with no confidence intervals, and Appendix B.3 explains that significance tests are deliberately omitted. This is understandable given target correlation within studies, but it leaves several claims undersupported. For example, Table 2 shows EliSeg's cardiomegaly IoU advantage over CheXbert to ROSALIA is 2.8 points, and the NSD gains for several findings are 1-2 points; without seed variance or a study-level bootstrap, it is not possible to assess whether these differences are stable. I request at least a small number of training seeds for EliSeg and the strongest baseline, or a bootstrap over studies, reported as confidence intervals on the headline metrics.","section":"Sec. 4.2, B.3"},{"comment":"The evaluation universe is a balanced subsample of the test split (1,008 targets and 600 ineligible mentions), but the sampling protocol is not specified. Because IoU and Dice are pooled over targets, the prevalence of each finding in the evaluation set directly affects the aggregate numbers. A balanced subsample gives rare findings such as pneumonia and consolidation much larger weight than in the natural MIMIC-CXR-ILS distribution, which may change the relative ranking between systems if their per-finding error profiles differ. Please describe how the balanced subsample was constructed, whether studies or targets were sampled, and also report results on the full test split or on both the natural and balanced universes.","section":"Sec. A.1, A.8"}],"minor_comments":[{"comment":"The abstract and conclusion state that EliSeg transfers to CheXlocalize, but Section 4.5 evaluates only the Actor without the Verifier or Revision because paired reports are unavailable. Please soften the wording to 'the EliSeg Actor transfers' or explicitly state that full-framework transfer remains untested.","section":"Abstract and Sec. 4.5"},{"comment":"The parameter accounting is inconsistent as written: Table 9 lists ROSALIA-7B as 7.402B while the text says 'Under a common accounting both total 7.705B.' Clarify whether the 7.402B figure excludes the bundled CLIP encoder and state which accounting is used for the comparison.","section":"Sec. A.6"},{"comment":"The Report-Inferred setting for direct text-based baselines is not fully specified. For models whose native interface is a finding phrase (e.g., ROSALIA, MedCLIP-SAMv2), it is unclear whether the entire unfiltered report is used as the text prompt and how long reports are truncated. Please document the exact input construction for each baseline under the R setting.","section":"Sec. 4.1 and Table 1"},{"comment":"There is a typo in the abstract: 'atcor-verify-revise' should read 'actor-verify-revise'.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a careful and well-documented empirical study. My main concern is that the central finding-to-mask correspondence is guaranteed only by a canonical-ordering assumption that is not independently verified, and the empirical claims lack any variability estimates. These issues are addressable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: EliSeg is a legitimate next step for report-grounded chest X-ray segmentation. It removes the hidden target oracle and makes target eligibility, cardinality, and finding-to-mask correspondence explicit problems, then attacks them with a propose-verify-revise loop. The paper is unusually transparent for this area: detailed appendices on annotation, metrics, inference cost, and an honest statistical-considerations section that refuses to run significance tests it doesn't support. That transparency earns real credit.\n\nWhat's new: the problem formulation itself—report-grounded abnormality segmentation—where the model sees an unfiltered report and must decide what to segment, how many masks, and which mask goes with which finding. No prior work does this without a target oracle. The architecture is simple and effective: a grammar-constrained Actor proposes slots and masks, a frozen text-only Qwen2.5-VL verifier reconstructs the eligible finding inventory, and a consistency gate triggers re-execution when action or cardinality disagree. The ablation shows the gate fires on only 22% of sentences and is a no-op when the Actor already got the structure right. That is a clean result. The CheXlocalize transfer and the per-target cost analysis are also solidly done.\n\nThe soft spots, in order. First, all results are single deterministic runs. The authors argue, correctly, that a significance test over the 1,008 targets would be pseudoreplication. Fine. But the main claim—59.4 vs 53.9 IoU over CheXbert→ROSALIA—has no variance estimate, so we cannot know if it is robust to seed or training noise. Second, the ILS annotations are not public, and the evaluation universe is a balanced subsample with UNSUPPORTED sentences excluded. Defensible, but it makes independent replication harder. Third, and this is the one that matters most: the k-th slot is assumed to be the k-th positive finding under a fixed canonical order. No component ever predicts slot identity. The consistency gate compares only action and cardinality, so a count-preserving swap between findings is invisible. The paper acknowledges this in Section 5, and the appendix shows the third slot decodes only 36.4% of the time. This is a real limitation on the claim of finding-to-mask correspondence. It does not sink the paper—the overall target-construction result still stands—but readers should not take the per-finding numbers in Table 2 as proof that the binding is correct in hard cases.\n\nWho this is for: anyone working on medical vision-language grounding or report-guided segmentation. It deserves a serious referee; the formulation is new, the evaluation is careful, and the limitations are spelled out rather than buried. I would accept it for review and push the authors to address slot-level verification, or at least report swap rates on multi-finding sentences.","headline":"EliSeg is a genuinely new formulation with an honest evaluation; the unverified slot-to-finding binding is a real but disclosed limitation.","tokens_in":20734,"tokens_out":3065,"would_cite":true,"duration_ms":28634,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Report-grounded segmentation without a target oracle: EliSeg reaches 59.4 IoU on MIMIC-CXR-ILS, surpassing direct text-based and extract-then-segment baselines while rejecting most ineligible mentions.","keywords":["report-grounded abnormality segmentation","chest X-ray","target construction","actor-verify-revise","radiology report","MIMIC-CXR-ILS","finding eligibility verification","zero-shot transfer"],"falsifier":"Read a test set of sentences that assert two eligible findings in reverse canonical order (for instance 'pneumonia and cardiomegaly' instead of 'cardiomegaly and pneumonia'), decode both masks, then record whether the first-slot mask matches the cardiomegaly reference or the pneumonia reference; if, despite a correct count, the first-slot mask more often matches the second-listed finding than chance, the canonical slot-to-finding identity is broken and pooled overlap metrics would hide the swap.","tokens_in":19685,"feed_emoji":"🩻","tokens_out":9686,"duration_ms":85699,"temperature":0.7,"pith_summary":"Radiology reports describe abnormalities in words, not as segmentation targets, and a single report may mention findings that are present, absent, historical, or uncertain. This paper asks whether a model can read the raw report, decide which findings deserve a mask, and then produce those masks, without ever being given the finding name, a point, or a box. It proposes EliSeg, which first has an image-grounded Actor propose a number of mask slots, then has an independent text-only Verifier reconstruct the eligible finding list, and re-runs the Actor when the two disagree on how many findings to segment. On the MIMIC-CXR-ILS benchmark this report-inferred pipeline reaches 59.4 IoU, beating the best direct text-to-mask method (40.8 IoU) and the best extract-then-segment cascade (53.9 IoU), while keeping false segmentation of ineligible mentions at 14.2%. The paper's central claim is that target construction—choosing which reported findings are segmentable—is a distinct stage that can be verified and corrected on text evidence alone, without needing a target oracle.","feed_headline":"One pass, no prompts: 59.4 IoU from reports to masks","feed_subtitle":"On MIMIC-CXR-ILS, EliSeg reaches 59.4 IoU without any target prompt, beating the best cascade by 5.5 points.","key_machinery":"The load-bearing mechanism is the canonical control-sequence grammar. The Actor emits up to three segmentation tokens [SEG1]–[SEG3] under a fixed finding ordering (cardiomegaly, edema, effusion, atelectasis, lung opacity, pneumonia, consolidation); the k-th token is supervised against the k-th positive finding in that order, so the token's contextual representation carries the finding's identity without an explicit label. The Verifier independently produces a verified inventory in the same canonical order, and Revision constructs a teacher-forced sequence of the verified length but does not inject finding names; the correspondence is recovered from the Actor's report-conditioned representations under the same ordering. This lets the framework correct eligibility and cardinality errors—false targets, omissions, wrong counts—while still using a single shared mask decoder, with no explicit slot-level identity verification.","core_discovery":"EliSeg establishes that report-grounded abnormality segmentation can be performed without any predefined target identity or spatial prompt: given a chest radiograph and its unfiltered report, the model predicts a sentence-level action (SEG/REJ/SILENT), a target cardinality, and one mask per eligible finding. The framework is an actor–verify–revise loop: a grammar-constrained Actor proposes mask slots through control tokens [SEG1]...[SEG_K]; an independent text-only Verifier (a frozen Qwen2.5-VL-7B) reconstructs the eligible finding inventory; a consistency gate compares action and cardinality; and when they disagree, Revision builds a corrected canonical control sequence and re-executes the shared Actor in a teacher-forced pass. On MIMIC-CXR-ILS, EliSeg attains 59.4 IoU, 74.5 Dice, 23.2 NSD, and 213.9 HD95 under report-inferred input, surpassing the best direct text-based method (ROSALIA, 40.8 IoU) and the best extract-then-segment cascade (CheXbert→ROSALIA, 53.9 IoU), while achieving a false segmentation rate of 14.2% and an IoU+ of 60.8 on eligible targets. Ablations show that neither verification alone nor revision alone improves much over the Actor (circa 49–50 IoU), whereas the closed loop reaches 54.1 IoU on a shared Actor, with Revision primarily rescuing cases where the Actor declined to segment or dropped a slot (22% of sentences) and leaving intact the 78% with full slot structure. The paper further reports zero-shot transfer of the Actor to CheXlocalize (31.9 IoU in a given-target setting), and shows the framework adds no trainable parameters over its ROSALIA-7B initialization, with the extra inference cost coming almost entirely from the frozen Verifier.","pith_inferences":["The framework effectively treats the fixed vocabulary ordering as an implicit language of slot identity; a natural testable extension is to make finding identity explicit in the control tokens (for instance, emitting a 'cardiomegaly' token) so that per-slot verification can catch count-preserving swaps, which Section 5 flags as missing.","Because the Verifier dominates serial latency (roughly two thirds of per-sentence cost) and is frozen, batching its fixed prompt prefix or distilling it into a smaller labeler is the obvious lever to make the gate practical at scale; the paper already measures batching amortization (1,522 ms vs 2,868 ms per sentence at batch 8).","The residual rejection error is concentrated on 'prior' mentions (29% false-segmentation rate vs 2% for negated and 11.5% for uncertain evidence), which suggests temporal assertion reasoning, not lexical negation, is the next bottleneck for report-grounded segmentation; a verifier that reads across the full report section rather than the current sentence is the direct test.","The paper reports that over-planning materializes a mask for non-existent targets in 5.6% of sentences, with lung opacity most affected; since reference masks in MIMIC-CXR-ILS are sometimes pixel-identical across labels, some of this 'false target' rate is annotation arbitrariness, and an external adjudicated set would separate verifier errors from label noise."],"forward_implications":["Report-grounded segmentation becomes a one-pass structured prediction: a report goes in and all eligible masks come out, without one target-conditioned invocation per finding.","Eligibility and cardinality errors that are invisible to mask geometry—false targets, omitted findings, wrong counts—can be repaired at the target-construction stage, and the ablation attributes a roughly 200-pixel HD95 reduction to this structural correction.","The frozen text-only Verifier acts as a plug-in consistency signal: any future Actor can be gated against it without retraining the Verifier, and the gate fires only on disagreement, which occurs in 22% of sentences.","The gain over extract-then-segment cascades (59.4 vs 53.9 IoU) comes precisely from recovering omitted targets and suppressing spurious ones, errors a cascade cannot fix after extraction, and the Actor alone transfers to CheXlocalize (31.9 vs 29.6 IoU for ROSALIA).","The framework adds no trainable parameters over the segmenter it initializes from, so its improvements are attributable to the verify–revise loop rather than added capacity."],"supporting_citations":[{"why":"Supplies the ROSALIA-7B initialization of the Actor as well as the strongest direct text-based baseline (40.8 IoU) that EliSeg must beat.","marker":"(Choi et al. 2025)"},{"why":"Provides MIMIC-CXR, the paired radiograph–report corpus from which the MIMIC-CXR-ILS evaluation universe (1,008 targets, 600 ineligible mentions) is derived.","marker":"(Johnson et al. 2019)"},{"why":"CheXbert is the report-labeling front end used to build the extract-then-segment cascade baselines, defining the comparison that the verify–revise loop is designed to outperform.","marker":"(Smit et al. 2020)"},{"why":"The frozen Qwen2.5-VL-7B model serves as the report-only Verifier that reconstructs the eligible finding inventory.","marker":"(Bai et al. 2025)"},{"why":"MAIRA-2 supplies a grounded report-generation baseline and an extract-then-segment cascade member, providing a target-conditioned comparison point.","marker":"(Bannur et al. 2024)"},{"why":"CheXagent acts as a foundation-model baseline for report understanding and text-driven segmentation in the comparison tables.","marker":"(Chen et al. 2024)"},{"why":"MedSAM provides the gold-derived spatial-prompt baseline (tight box) that quantifies the gap to target-with-spatial-oracle performance.","marker":"(Ma et al. 2024)"},{"why":"CheXlocalize supplies the external expert-mask dataset used to test zero-shot transfer of the Actor in a given-target setting.","marker":"(Saporta et al. 2022)"}],"fun_headline_variants":["No prompts, just reports: EliSeg reaches 59.4 IoU","Report-grounded masks: EliSeg beats best cascade by 5.5","EliSeg: verify-revise loop for prompt-free segmentation","From unfiltered reports to masks: EliSeg hits 74.5 Dice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The k-th mask slot always refers to the k-th eligible finding under the fixed vocabulary order, so a correct count is assumed to mean correct finding-to-mask pairing; the consistency gate never checks which finding each mask represents, and count-preserving swaps go undetected.","fun_headline_variants_meta":{"raw":{"variants":["No prompts, just reports: EliSeg reaches 59.4 IoU","Report-grounded masks: EliSeg beats best cascade by 5.5","EliSeg: verify-revise loop for prompt-free segmentation","From unfiltered reports to masks: EliSeg hits 74.5 Dice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000415,"raw_usage":{"total_tokens":2275,"prompt_tokens":1211,"completion_tokens":1064,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":827,"completion_tokens_details":{"reasoning_tokens":982}},"tokens_in":827,"tokens_out":1064,"duration_ms":8910,"temperature":1.0,"reasoning_tokens":982,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:49:21.849628+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Read a test set of sentences that assert two eligible findings in reverse canonical order (for instance 'pneumonia and cardiomegaly' instead of 'cardiomegaly and pneumonia'), decode both masks, then record whether the first-slot mask matches the cardiomegaly reference or the pneumonia reference; if, despite a correct count, the first-slot mask more often matches the second-listed finding than chance, the canonical slot-to-finding identity is broken and pooled overlap metrics would hide the swap.","supporting_citations":[],"review_version":2}