{"id":"8852bd28-b902-4514-919d-e7dd7abf81af","arxiv_id":"2607.17341","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"EgoMed-Agent reports 71.34% average Dice on a new egocentric medical segmentation benchmark (523 videos, 5 modalities), versus 11.70% for zero-shot text-prompted baselines, using detector + LLM clarification + SAM2 propagation with detection-based re-initialization.","lead":"A multi-agent system combining a trained detector, an LLM-based confirmation step, and SAM2 propagation segments organs from smart-glasses videos of on-screen medical images, reporting 71.34% average Dice on a new 523-video benchmark. The performance gap over text-prompted baselines is real in this setup, but the comparison is not apples-to-apples because the system's detectors were trained on the same data while the baselines were used zero-shot.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 71.34% vs 11.70% headline gap is confounded by unequal training access: EgoMed-Agent's detectors are trained on the IEMIS dataset while baselines are zero-shot.","rationale":"The reader's weakest_assumption identifies the exact load-bearing concern: EgoMed-Agent's detectors are trained on the IEMIS dataset while text-prompted baselines are evaluated zero-shot. This is not a minor methodological detail; it directly undermines the headline quantitative comparison. The paper's own numbers demonstrate the issue: Frame-wise Det. (trained detector only, no propagation) reaches 59.27% average Dice, more than five times the best baseline (11.70%), and the nnU-Net upper bound (81.69%) shows how much supervised target-specific information contributes. Therefore the claimed 'far above' and 'approaching nnU-Net' are largely a reflection of training access, not of the proposed multi-agent workflows. The internal ablations (Init-Only Prop. vs Frame-wise Det. vs full system) are coherent and show that propagation and localization do contribute, but they do not validate the headline comparison to zero-shot baselines. The paper's own statement that the detectors are 'trained on our data' (Sec IV-A.2) contradicts the 'training-free generality' bullet in the introduction. The consistency-threshold tuning on the test set and the 100% confirmation scores on a 200-sample set are additional concerns, but they are secondary; the training-access asymmetry is the most load-bearing because it affects the central claim. A fine-tuning experiment on the strongest baseline would settle whether the gap is real. Therefore I agree with the reader's conditional verdict and recommend no change.","tokens_in":17037,"tokens_out":3671,"duration_ms":37846,"concrete_test":"Fine-tune the strongest baseline (MedSAM3) on the IEMIS training split using the same mask annotations used to train the Detection Agent, then evaluate on the same test set. If its average Dice approaches or exceeds 71.34%, the reported superiority is not attributable to the multi-agent design but to training access; if it remains near 11.70%, the comparison is fair and the claim holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that EgoMed-Agent 'far above' text-prompted baselines (71.34 vs 11.70) and approaches nnU-Net 'without training the segmentation model on mask annotations' is not supported by the reported comparison. In Sec IV-A.2, the image-type classifier and all five modality-specific detectors are trained on the IEMIS data; the propagation agent (SAM2) uses box-prompt initialization and per-frame re-initialization from these trained detectors. The baselines (Grounded SAM2, LangSAM, LISA, MedSAM3, SAM3) are used zero-shot with text only, no access to the training set or box prompts. Thus the comparison conflates architecture with training information. The paper's own ablation shows Frame-wise Det., which uses only the trained detector per frame, already reaches 59.27% (Table II), so the detector alone accounts for most of the gap over the 11.70% baselines. The 'training-free generality' claim (Sec I contributions, Sec III-A) contradicts Sec IV-A.2. This is a load-bearing fairness issue: if a text-prompted baseline were fine-tuned or box-prompted with the same supervised information, the gap could shrink substantially or invert.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces interactive egocentric medical image segmentation (IEMIS), a task in which a clinician wearing smart glasses issues an instruction referring to a target in a medical image displayed on screen, and the system must segment that target across the captured egocentric video. The authors propose EgoMed-Agent, a multi-agent pipeline with a Detection Agent (image-type classifier plus modality-specific detectors), a Confirmation Agent (an LLM that scores grounding reliability and optionally asks for clarification), and a Propagation Agent (SAM2 with box initialization and re-initialization). They construct a dataset of 523 egocentric videos from five public medical datasets, train the detection toolbox on it, and report a mean Dice of 71.34% versus 11.70% for the best text-prompted baseline, approaching a fully supervised nnU-Net at 81.69%. They also report ablations showing the benefit of coupling propagation with detection, a threshold analysis for the consistency parameter, and a 200-instruction confirmation study.","tokens_in":17363,"tokens_out":6905,"duration_ms":67551,"significance":"The strengths are the task formulation, the multi-agent decomposition, and the release of a new benchmark with code. The Target Confirmation workflow (reliability-scored grounding with clarification) and Localization-Guided Propagation workflow (detector-based re-initialization when propagation drifts) are sensible, and the ablations in Table II suggest each component contributes. The dataset, spanning five modalities and real-world capture scenes, is a useful resource. However, the central quantitative claim — that EgoMed-Agent is 'far above' text-prompted baselines — is not supported by the current protocol because EgoMed-Agent's detectors are trained on the test benchmark's training split while the baselines are zero-shot, and because the main configuration's consistency threshold is selected on the test set. These issues are fixable, but the headline result as presented is confounded by the comparison setup.","major_comments":[{"comment":"The headline result is confounded by unequal training access. The Detection Agent's classifier and all five detectors are trained on the IEMIS data (Sec. IV-A.2), whereas every text-prompted baseline is used zero-shot. Table II shows that Frame-wise Det., using only the trained detector, already reaches 59.27% average Dice, so most of the 71.34 vs 11.70 gap comes from supervised training rather than the proposed workflows. This contradicts the claimed 'training-free generality' (Sec. I, contribution c) and makes 'approaches nnU-Net without training the segmentation model on mask annotations' (Sec. IV-B.1) misleading, since the detectors are trained on the same data. Please re-run with matched training access (e.g., fine-tuned or box-prompted baselines) or with a zero-shot EgoMed-Agent, or at least report an ablation that removes the trained-detector advantage.","section":"Sec. IV-A.2, Tables I-II"},{"comment":"The consistency threshold τ2 is selected by sweeping on the test set that is also used for Table I: 'we adopt τ2 = 0.6 as the consistency threshold in all of our main experiments' after reporting that τ2 = 0.6 achieves the highest average Dice on the twelve target test results. This is test-set tuning of a free parameter of the reported configuration. The 71.34% number and the paired t-tests in Table I are therefore optimistically biased and do not have the claimed frequentist interpretation. Threshold selection should be performed on a held-out validation split (or via nested cross-validation), with the test set used only once. The same issue applies to the choice of τ1 = 0.8, whose selection is not documented.","section":"Sec. IV-B.4, Fig. 6"},{"comment":"The confirmation workflow is validated on a 200-sample evaluation set with 50 samples per instruction type, but the paper gives no details on how the instructions were sampled, how the ground-truth grounding state and target were labeled, or whether multiple annotators agreed. Since the same instruction taxonomy is used to create the interactions evaluated in Table I, the 100% GSA/TCA in Table IV may reflect the authors' own instruction-generation process rather than a realistic user distribution. Please report the annotation protocol, inter-annotator agreement, and ideally evaluate on instructions produced by independent clinicians. This is important because the confirmation workflow is one of the two central contributions.","section":"Sec. IV-B.7, Table IV"}],"minor_comments":[{"comment":"The notation M_{1:t} = A_pro(V_{1:t}, b_loc_1) does not reflect that re-initialization occurs at multiple frames with new boxes; please make the temporal dependence of the box prompts explicit.","section":"Eq. (7), Sec. III-C"},{"comment":"The dataset is described as containing 523 videos, while Sec. IV-B.5 mentions '533 video–target instances' on the test set. Clarify the distinction between videos and video–target instances.","section":"Sec. IV-A.1 vs Sec. IV-B.5"},{"comment":"The Confirmation Agent uses DeepSeek-V4-Flash, a proprietary API model, without specifying the model version or the prompts used. Please include the exact prompts and model version for reproducibility.","section":"Sec. IV-A.2"},{"comment":"The claim of being the 'first multi-agent system for IEMIS' is tautological because IEMIS is the authors' own task definition; please phrase the novelty claim more carefully, e.g., 'first under the proposed formulation'.","section":"Sec. I"},{"comment":"The y-axis is labeled 'DSC (%)' while the text uses 'Dice'; please standardize the terminology and ensure axis labels match the metric definitions.","section":"Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper introduces a useful benchmark and a plausible multi-agent system, but the main empirical claim is not yet established. The revision must include genuinely matched baselines and validation-based threshold selection; if the authors decline to re-run the comparison under fair conditions, the paper should not be accepted. I see the issues as fixable within the scope of a major revision, not as a fundamental flaw in the proposed approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: this paper introduces IEMIS, a new task, and a new 523-video egocentric medical segmentation dataset. Both are genuinely useful. The multi-agent design—reliability-based clarification plus detection-corrected mask propagation—is coherent, and the ablations suggest each component contributes. But the headline result, 71.34% vs 11.70% Dice, is not a fair comparison. EgoMed-Agent's detectors and image-type classifier are trained on the IEMIS training split (Sec IV-A.2); the text-prompted baselines are zero-shot. The paper's own ablation shows a frame-wise detector alone reaches 59.27%, so much of the gap comes from the supervised detectors, not from the agent architecture. A fair baseline given the same training access, or EgoMed-Agent run zero-shot, could close most of that gap. This is load-bearing.\n\nThere are two more soft spots, both real. First, tau2=0.6 is chosen by sweeping on the test set (Fig. 6), making the headline number a post-hoc selection. Second, the paper's \"training-free generality\" claim (Sec I, Sec III-A) directly contradicts Sec IV-A.2, where the detectors are trained on the collected data. The confirmation-agent evaluation is also too clean: 200 self-constructed samples with 100% GSA/TCA is implausible and needs external validation.\n\nWhat the paper does well: the task formulation is sensible and fills a real gap; the dataset spans five modalities and five capture scenes and is public; the qualitative and ablation evidence is internally consistent. The propagation/localization coupling is a reasonable engineering contribution, and the system runs near real-time. These are real positives.\n\nBottom line: the paper deserves a serious referee, but it is not acceptable as is. The authors need to redo the comparison with matched training access, select tau2 on validation, and fix the training-free language. If the gap survives those changes, the result is interesting; if not, the dataset and task still have value. I'd bring it to a reading group and would point students to the dataset, but I would not trust the headline number until the comparison is fair.","headline":"Useful new task and dataset, but the headline 71.34 vs 11.70 gap is confounded by trained detectors against zero-shot baselines.","tokens_in":17815,"tokens_out":3065,"would_cite":true,"duration_ms":31163,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EgoMed-Agent claims 71.34% average Dice by first confirming which organ the user means and then correcting propagation drift with per-frame localization.","keywords":["interactive egocentric medical image segmentation","multi-agent system","smart glasses","target confirmation","mask propagation","consistency evaluation","egocentric video","Dice similarity"],"falsifier":"Run the strongest text-prompted baseline after giving it the same supervised detector training on the IEMIS training split, or run EgoMed-Agent with its detectors untrained on the test distribution; if the average Dice gap drops to near zero, or the baseline exceeds 71.34%, the claimed advantage is training access rather than the multi-agent design. A second check: disable only the re-initialization trigger and verify that average Dice falls back toward the reported 50.46% propagation-only level rather than staying near 71.34%.","tokens_in":16908,"feed_emoji":"🥽","tokens_out":7450,"duration_ms":75727,"temperature":0.7,"pith_summary":"Interactive egocentric medical image segmentation—drawing the outline of the organ or lesion a clinician refers to while wearing smart glasses—is the perceptual step that could let a glasses copilot ground its advice in visible evidence. The paper argues that this task fails when instructions are matched directly to pixels, because phrases like 'the kidney' are ambiguous and because head motion makes the target shift between frames. It proposes EgoMed-Agent, described as the first multi-agent system for this task: a Detection Agent proposes candidate targets, a Confirmation Agent scores how reliably the instruction picks one out and asks for clarification when uncertain, and a Propagation Agent carries the mask forward while detection re-localizes the target and resets propagation on disagreement. On a new 523-video benchmark spanning five imaging modalities, the system reports 71.34% average Dice, far above the best text-prompted baseline's 11.70% and within ten points of a fully supervised upper bound, without training the segmentation model on mask annotations. If these numbers hold, the paper's central suggestion is that decomposing the interaction into confirm-then-track is the key to practical smart-glasses medical copilots.","feed_headline":"Multi-agent system hits 71.34% on egocentric medical segmentation","feed_subtitle":"Confirming the target first and correcting drift beats text prompts by 60 Dice points, near fully supervised results.","key_machinery":"The load-bearing object is the two-workflow decomposition itself. Target Confirmation couples a candidate-generation toolbox with a reliability-scored grounding step: candidate targets are extracted by specialized detectors selected by an image-type classifier, and a language model scores how reliably the instruction picks out one of them, triggering clarification below a threshold. Localization-Guided Propagation couples a temporal mask propagator with per-frame detection: the propagated mask is converted to a box, its IoU with the detector's box is computed each frame, and whenever this IoU falls below a consistency threshold the propagator is re-initialized from the detected box. This con","core_discovery":"The paper's central claim is that the bottleneck in egocentric medical image segmentation is not pixel-level segmentation ability but knowing which target the user means and keeping it locked across frames. It therefore builds the task as a loop: candidates are extracted from the first frame by modality-specific detectors; a language model grounds the user's instruction against those candidates, returns a reliability score, and requests clarification when the score falls below a threshold; once the target is fixed, a video segmentation model propagates its mask forward while the detector re-localizes the target in every frame, and a consistency check re-initializes propagation whenever the p","pith_inferences":["We infer that the fairest control experiment is not the one reported: giving the strongest text-prompted baseline the same supervised detector training on the IEMIS split would isolate how much of the 71.34% versus 11.70% gap is architecture versus training access; the paper's own 81.69% supervised upper bound shows how much target-specific supervision contributes.","If the confirmation step is as important as the results suggest, the clarification mechanism could be reused for other interactive clinical tasks—measurement, annotation, report generation—wherever user instructions are incomplete; that would be a natural extension beyond segmentation.","A testable extension is to replace the fixed five-detector toolbox with open-vocabulary detection, which the paper names as future work; the consistency-evaluation loop would remain unchanged, so the hypothesis that coverage, not the loop, limits generality can be tested directly.","The dataset's construction—recording screens of public medical images through smart glasses in everyday rooms—means the claim should be read as egocentric review of displayed images, not full clinical deployment; the paper says the same, and real workstation capture remains the open test."],"forward_implications":["If the reported numbers hold, direct text-to-mask grounding is not a viable route for egocentric medical video; an explicit target-confirmation step is required before segmentation.","A pipeline assembled from off-the-shelf components can approach a fully supervised upper bound on a newly defined task without mask-supervised training of the segmenter, implying that the task's difficulty lies in interaction design rather than segmentation model capacity.","Mask propagation drift is a frequent, measurable failure mode—over 27 re-initializations per target on average—and localization-guided correction removes most of that error.","The system's 10.7 frames-per-second operation on a single GPU suggests interactive use is feasible, so the bottleneck for deployment is not runtime but detector coverage and data realism."],"fun_headline_variants":["EgoMed-Agent: confirm target first, then propagate mask for 71% Dice","Multi-agent system asks for clarity, then segments egocentric video at 71%","Egocentric medical segmentation: grounding instruction improves Dice to 71%","Interactively confirm target to hit 71% Dice, up from 11.7% text-prompt baseline"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The result depends on comparing EgoMed-Agent—whose image-type classifier and all five modality detectors were trained on the IEMIS dataset—with text-prompted baselines used zero-shot; if the baselines received the same supervised detection training, or EgoMed-Agent were evaluated zero-shot, the 71.34% versus 11.70% gap could shrink or reverse.","fun_headline_variants_meta":{"raw":{"variants":["EgoMed-Agent: confirm target first, then propagate mask for 71% Dice","Multi-agent system asks for clarity, then segments egocentric video at 71%","Egocentric medical segmentation: grounding instruction improves Dice to 71%","Interactively confirm target to hit 71% Dice, up from 11.7% text-prompt baseline"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1560,"prompt_tokens":791,"completion_tokens":769,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":677}},"tokens_in":535,"tokens_out":769,"duration_ms":8520,"temperature":1.0,"reasoning_tokens":677,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T18:15:26.812893+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the strongest text-prompted baseline after giving it the same supervised detector training on the IEMIS training split, or run EgoMed-Agent with its detectors untrained on the test distribution; if the average Dice gap drops to near zero, or the baseline exceeds 71.34%, the claimed advantage is training access rather than the multi-agent design. A second check: disable only the re-initialization trigger and verify that average Dice falls back toward the reported 50.46% propagation-only level rather than staying near 71.34%.","supporting_citations":[],"review_version":1}