Pith. sign in

REVIEW 3 major objections 5 minor 58 references

EgoMed-Agent claims 71.34% average Dice by first confirming which organ the user means and then correcting propagation drift with per-frame localization.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 18:15 UTC pith:HNNQMPKQ

load-bearing objection Useful new task and dataset, but the headline 71.34 vs 11.70 gap is confounded by trained detectors against zero-shot baselines. the 3 major comments →

arxiv 2607.17341 v1 pith:HNNQMPKQ submitted 2026-07-19 cs.CV

Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentation

Rongjun Ge , Dongyang Wang , Heng Zhu , Zhirui Li , Yang Chen , Yuting He This is my paper
classification cs.CV
keywords interactive egocentric medical image segmentationmulti-agent systemsmart glassestarget confirmationmask propagationconsistency evaluationegocentric videoDice similarity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Interactive egocentric medical image segmentation—drawing the outline of the organ or lesion a clinician refers to while wearing smart glasses—is the perceptual step that could let a glasses copilot ground its advice in visible evidence. The paper argues that this task fails when instructions are matched directly to pixels, because phrases like 'the kidney' are ambiguous and because head motion makes the target shift between frames. It proposes EgoMed-Agent, described as the first multi-agent system for this task: a Detection Agent proposes candidate targets, a Confirmation Agent scores how reliably the instruction picks one out and asks for clarification when uncertain, and a Propagation Agent carries the mask forward while detection re-localizes the target and resets propagation on disagreement. On a new 523-video benchmark spanning five imaging modalities, the system reports 71.34% average Dice, far above the best text-prompted baseline's 11.70% and within ten points of a fully supervised upper bound, without training the segmentation model on mask annotations. If these numbers hold, the paper's central suggestion is that decomposing the interaction into confirm-then-track is the key to practical smart-glasses medical copilots.

Core claim

The paper's central claim is that the bottleneck in egocentric medical image segmentation is not pixel-level segmentation ability but knowing which target the user means and keeping it locked across frames. It therefore builds the task as a loop: candidates are extracted from the first frame by modality-specific detectors; a language model grounds the user's instruction against those candidates, returns a reliability score, and requests clarification when the score falls below a threshold; once the target is fixed, a video segmentation model propagates its mask forward while the detector re-localizes the target in every frame, and a consistency check re-initializes propagation whenever the p

What carries the argument

The load-bearing object is the two-workflow decomposition itself. Target Confirmation couples a candidate-generation toolbox with a reliability-scored grounding step: candidate targets are extracted by specialized detectors selected by an image-type classifier, and a language model scores how reliably the instruction picks out one of them, triggering clarification below a threshold. Localization-Guided Propagation couples a temporal mask propagator with per-frame detection: the propagated mask is converted to a box, its IoU with the detector's box is computed each frame, and whenever this IoU falls below a consistency threshold the propagator is re-initialized from the detected box. This con

Load-bearing premise

The result depends on comparing EgoMed-Agent—whose image-type classifier and all five modality detectors were trained on the IEMIS dataset—with text-prompted baselines used zero-shot; if the baselines received the same supervised detection training, or EgoMed-Agent were evaluated zero-shot, the 71.34% versus 11.70% gap could shrink or reverse.

What would settle it

Run the strongest text-prompted baseline after giving it the same supervised detector training on the IEMIS training split, or run EgoMed-Agent with its detectors untrained on the test distribution; if the average Dice gap drops to near zero, or the baseline exceeds 71.34%, the claimed advantage is training access rather than the multi-agent design. A second check: disable only the re-initialization trigger and verify that average Dice falls back toward the reported 50.46% propagation-only level rather than staying near 71.34%.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the reported numbers hold, direct text-to-mask grounding is not a viable route for egocentric medical video; an explicit target-confirmation step is required before segmentation.
  • A pipeline assembled from off-the-shelf components can approach a fully supervised upper bound on a newly defined task without mask-supervised training of the segmenter, implying that the task's difficulty lies in interaction design rather than segmentation model capacity.
  • Mask propagation drift is a frequent, measurable failure mode—over 27 re-initializations per target on average—and localization-guided correction removes most of that error.
  • The system's 10.7 frames-per-second operation on a single GPU suggests interactive use is feasible, so the bottleneck for deployment is not runtime but detector coverage and data realism.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that the fairest control experiment is not the one reported: giving the strongest text-prompted baseline the same supervised detector training on the IEMIS split would isolate how much of the 71.34% versus 11.70% gap is architecture versus training access; the paper's own 81.69% supervised upper bound shows how much target-specific supervision contributes.
  • If the confirmation step is as important as the results suggest, the clarification mechanism could be reused for other interactive clinical tasks—measurement, annotation, report generation—wherever user instructions are incomplete; that would be a natural extension beyond segmentation.
  • A testable extension is to replace the fixed five-detector toolbox with open-vocabulary detection, which the paper names as future work; the consistency-evaluation loop would remain unchanged, so the hypothesis that coverage, not the loop, limits generality can be tested directly.
  • The dataset's construction—recording screens of public medical images through smart glasses in everyday rooms—means the claim should be read as egocentric review of displayed images, not full clinical deployment; the paper says the same, and real workstation capture remains the open test.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces interactive egocentric medical image segmentation (IEMIS), a task in which a clinician wearing smart glasses issues an instruction referring to a target in a medical image displayed on screen, and the system must segment that target across the captured egocentric video. The authors propose EgoMed-Agent, a multi-agent pipeline with a Detection Agent (image-type classifier plus modality-specific detectors), a Confirmation Agent (an LLM that scores grounding reliability and optionally asks for clarification), and a Propagation Agent (SAM2 with box initialization and re-initialization). They construct a dataset of 523 egocentric videos from five public medical datasets, train the detection toolbox on it, and report a mean Dice of 71.34% versus 11.70% for the best text-prompted baseline, approaching a fully supervised nnU-Net at 81.69%. They also report ablations showing the benefit of coupling propagation with detection, a threshold analysis for the consistency parameter, and a 200-instruction confirmation study.

Significance. The strengths are the task formulation, the multi-agent decomposition, and the release of a new benchmark with code. The Target Confirmation workflow (reliability-scored grounding with clarification) and Localization-Guided Propagation workflow (detector-based re-initialization when propagation drifts) are sensible, and the ablations in Table II suggest each component contributes. The dataset, spanning five modalities and real-world capture scenes, is a useful resource. However, the central quantitative claim — that EgoMed-Agent is 'far above' text-prompted baselines — is not supported by the current protocol because EgoMed-Agent's detectors are trained on the test benchmark's training split while the baselines are zero-shot, and because the main configuration's consistency threshold is selected on the test set. These issues are fixable, but the headline result as presented is confounded by the comparison setup.

major comments (3)
  1. [Sec. IV-A.2, Tables I-II] The headline result is confounded by unequal training access. The Detection Agent's classifier and all five detectors are trained on the IEMIS data (Sec. IV-A.2), whereas every text-prompted baseline is used zero-shot. Table II shows that Frame-wise Det., using only the trained detector, already reaches 59.27% average Dice, so most of the 71.34 vs 11.70 gap comes from supervised training rather than the proposed workflows. This contradicts the claimed 'training-free generality' (Sec. I, contribution c) and makes 'approaches nnU-Net without training the segmentation model on mask annotations' (Sec. IV-B.1) misleading, since the detectors are trained on the same data. Please re-run with matched training access (e.g., fine-tuned or box-prompted baselines) or with a zero-shot EgoMed-Agent, or at least report an ablation that removes the trained-detector advantage.
  2. [Sec. IV-B.4, Fig. 6] The consistency threshold τ2 is selected by sweeping on the test set that is also used for Table I: 'we adopt τ2 = 0.6 as the consistency threshold in all of our main experiments' after reporting that τ2 = 0.6 achieves the highest average Dice on the twelve target test results. This is test-set tuning of a free parameter of the reported configuration. The 71.34% number and the paired t-tests in Table I are therefore optimistically biased and do not have the claimed frequentist interpretation. Threshold selection should be performed on a held-out validation split (or via nested cross-validation), with the test set used only once. The same issue applies to the choice of τ1 = 0.8, whose selection is not documented.
  3. [Sec. IV-B.7, Table IV] The confirmation workflow is validated on a 200-sample evaluation set with 50 samples per instruction type, but the paper gives no details on how the instructions were sampled, how the ground-truth grounding state and target were labeled, or whether multiple annotators agreed. Since the same instruction taxonomy is used to create the interactions evaluated in Table I, the 100% GSA/TCA in Table IV may reflect the authors' own instruction-generation process rather than a realistic user distribution. Please report the annotation protocol, inter-annotator agreement, and ideally evaluate on instructions produced by independent clinicians. This is important because the confirmation workflow is one of the two central contributions.
minor comments (5)
  1. [Eq. (7), Sec. III-C] The notation M_{1:t} = A_pro(V_{1:t}, b_loc_1) does not reflect that re-initialization occurs at multiple frames with new boxes; please make the temporal dependence of the box prompts explicit.
  2. [Sec. IV-A.1 vs Sec. IV-B.5] The dataset is described as containing 523 videos, while Sec. IV-B.5 mentions '533 video–target instances' on the test set. Clarify the distinction between videos and video–target instances.
  3. [Sec. IV-A.2] The Confirmation Agent uses DeepSeek-V4-Flash, a proprietary API model, without specifying the model version or the prompts used. Please include the exact prompts and model version for reproducibility.
  4. [Sec. I] The claim of being the 'first multi-agent system for IEMIS' is tautological because IEMIS is the authors' own task definition; please phrase the novelty claim more carefully, e.g., 'first under the proposed formulation'.
  5. [Fig. 6] The y-axis is labeled 'DSC (%)' while the text uses 'Dice'; please standardize the terminology and ensure axis labels match the metric definitions.

Circularity Check

1 steps flagged

The headline 71.34% is chosen on the test set via τ2=0.6, so the main result is partly fitted; the trained-detector vs zero-shot baseline gap is a confound but not circular.

specific steps
  1. fitted input called prediction [Sec. IV-B.4 (Ablation of Hyperparameters, Fig. 6); result reported in Sec. IV-B.1 / Table I]
    "Specifically, τ2 = 0, τ2 = 0.3, τ2 = 0.6, and τ2 = 0.9 achieve average DSC scores of 56.18%, 69.61%, 71.34%, and 70.17% across all twelve targets, respectively. Among them, τ2 = 0.6 achieves the highest average DSC. When the threshold further increases to τ2 = 0.9, the performance does not continue to improve, suggesting that overly frequent re-initialization does not necessarily further benefit segmentation. Thus, we adopt τ2 = 0.6 as the consistency threshold in all of our main experiments."

    The reported headline result (71.34%) is the average Dice at τ2=0.6. The paper selects τ2 by comparing four values on the test set and keeping the one with the highest average DSC. Hence the headline number is a post-hoc maximized configuration, not an a-priori prediction of the method's performance; the evaluation metric is used to choose the very threshold at which the metric is then reported, making the reported value statistically forced upward.

full rationale

The paper's core scientific content is an empirical system evaluation rather than a formal derivation, so most of the claimed chain is not circular. The main circularity-like defect is in Sec. IV-B.4: the consistency threshold τ2 is tuned on the test set (0, 0.3, 0.6, 0.9) and the value with the highest average Dice (71.34%) is then adopted in all main experiments. This is a fitted input presented as the system's achieved performance, so the exact headline number is not an independent prediction. I do not count as circularity the separate concern that EgoMed-Agent's image-type classifier and modality-specific detectors are trained on the IEMIS data while text-prompted baselines are used zero-shot (Sec. IV-A.2 vs. Sec. I contribution c), because that is a comparison-fairness / correctness issue rather than a reduction of the result to its inputs by construction; it should be weighed as a confound but does not make the derivation circular. No load-bearing self-citation, no uniqueness theorem, and no ansatz-smuggling were found. Score 4 reflects a partially fitted headline configuration while the central architecture and the large gap over baselines retain independent content.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on trained detectors and a hand-set/test-fitted pair of thresholds, plus a self-constructed benchmark and a small in-house confirmation-evaluation set. No new physical entities are introduced.

free parameters (2)
  • grounding reliability threshold τ1 = 0.8
    Hand-set threshold in Sec III-B2 for deciding whether the LLM's grounding is reliable enough to confirm the target or trigger clarification. No ablation or robustness study is reported.
  • consistency threshold τ2 = 0.6
    Selected in Sec IV-B.4 by sweeping {0, 0.3, 0.6, 0.9} and choosing the value with the highest average Dice on the test set. The headline 71.34% result uses this test-fitted value.
axioms (4)
  • domain assumption The modality-specific detectors and image-type classifier, trained on the IEMIS training split, provide reliable candidate targets and per-frame localization for all 12 target types.
    Sec IV-A.2 trains YOLO26-det and YOLO26-cls on the authors' data. If these detectors fail, both Target Confirmation and Localization-Guided Propagation degrade. This is an empirical foundation, not proven in the paper.
  • domain assumption Zero-shot text-prompted segmentation is the appropriate baseline, and the gain from supervised detection on the same data is not the source of the improvement.
    Table I compares against Grounded SAM2, LangSAM, LISA, MedSAM3, and SAM3 used zero-shot, while EgoMed-Agent's detectors are trained on the dataset. The paper's 'far above' claim requires this assumption to hold.
  • ad hoc to paper The 200-instruction confirmation evaluation set, balanced across four instruction types, is representative of real user ambiguity and has reliable ground-truth labels.
    Sec IV-B.7 reports 100% GSA/TCA on this in-house set. No external benchmark, no inter-annotator agreement, and no independent validation are provided.
  • ad hoc to paper An LLM-produced reliability score with threshold τ1=0.8 correctly separates unambiguous from ambiguous instructions.
    Sec III-B2 defines the mechanism but the threshold is not ablated or justified beyond the small in-house evaluation.

pith-pipeline@v1.3.0-alltime-deepseek · 16733 in / 11752 out tokens · 107620 ms · 2026-08-01T18:15:26.812893+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentation." pith.science (2026). https://pith.science/paper/HNNQMPKQ

@misc{pith2026260717341,
  author       = {Pith},
  title        = {Pith review of: Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HNNQMPKQ}},
  note         = {Machine review of arXiv:2607.17341}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Interactive egocentric medical image segmentation (IEMIS) plays an important role in smart-glasses-assisted medical image review, segmenting the medical targets a clinician refers to from their egocentric view. Once it succeeds, the object-level visual evidence it provides strengthens the review and underpins fine-grained analysis and clinical decision-making. However, the instruction and the video both come from the user's egocentric perspective, which poses two challenges. (1) Semantic ambiguity leaves the model unable to confirm the user-intended target. (2) Visual variability makes the segmentation jump from frame to frame. In this paper, we propose EgoMed-Agent, a multi-agent system that understands the target from the human perspective through two workflows. (1) The \textit{Target Confirmation Workflow} grounds the instruction against candidate targets with a reliability score, confirming the target when the grounding is reliable and asking the user to clarify when it is not, thereby confirming the segmentation target. (2) The \textit{Localization-Guided Propagation Workflow} couples mask propagation with per-frame target localization, using the localized target to correct the propagated mask whenever the two diverge, so the segmentation stays on the target across the egocentric video. Extensive experiments show that EgoMed-Agent reaches 71.34\% average Dice, far above the best text-prompted baseline (11.70\%). Our code is available at \href{https://github.com/wdyyyyyy/EgoMed-Agent}{our project page}.

Figures

Figures reproduced from arXiv: 2607.17341 by Dongyang Wang, Heng Zhu, Rongjun Ge, Yang Chen, Yuting He, Zhirui Li.

Figure 1
Figure 1. Figure 1: (a) Medical image review leaves the doctor without support, so the clinical decision is experience-dependent. (b) Smart-glasses￾assisted medical image review gives the doctor support, so the clinical decision is reliable. decisions [3]. During this process, the reviewer faces confusion over where a lesion lies and what its characteristics indicate, relying on their own expertise [4], so clinical decisions … view at source ↗
Figure 2
Figure 2. Figure 2: IEMIS is formulated as an interactive loop among the doctor, smart glasses, and the medical smart glasses copilot. In each interac￾tion cycle, the doctor issues an instruction, the smart glasses capture the egocentric video stream and forward it to the copilot, and the copilot returns the corresponding segmentation results (top). This egocentric interaction poses two challenges (bottom). (1) Semantic ambig… view at source ↗
Figure 3
Figure 3. Figure 3: EgoMed-Agent decomposes IEMIS into two cooperating agent workflows. The Target Confirmation Workflow, run by the Detection and Confirmation Agents, confirms the user-intended target, while the Localization-Guided Propagation Workflow, run by the Detection and Propagation Agents, keeps the segmentation on that target across the egocentric video through Consistency Evaluation. segmentation on the user-intend… view at source ↗
Figure 4
Figure 4. Figure 4: Our dataset covers five imaging modalities and five capture scenes, recorded as egocentric video of on-screen medical images. The two pies give the number of videos per modality and per capture scene, and the panels on the right show one egocentric example per modality. determines whether propagation remains aligned with the user-intended target. When inconsistency occurs, the target￾aware localization res… view at source ↗
Figure 5
Figure 5. Figure 5: On representative targets from different imaging scenarios, EgoMed-Agent produces masks concentrated on the user-intended target and close to the ground truth, while the text-prompted baselines produce over-expanded masks or segment irrelevant screen and background regions. regions, these methods struggle to distinguish the target from nearby structures or visually similar regions. In contrast, EgoMed-Agen… view at source ↗
Figure 6
Figure 6. Figure 6: A positive consistency threshold τ2 improves segmentation over τ2 = 0, and τ2 = 0.6 gives the best average Dice. Each subplot reports the Dice score under different τ2 settings for one medical target. the highest average DSC. When the threshold further increases to τ2 = 0.9, the performance does not continue to improve, suggesting that overly frequent re-initialization does not nec￾essarily further benefit… view at source ↗
Figure 7
Figure 7. Figure 7: Consistency Evaluation keeps EgoMed-Agent on the user-intended target where naive propagation drifts off it. The green contour denotes the ground-truth target and the red region denotes the predicted mask. From t = 0 to t = 2, both Init-Only Prop. and EgoMed-Agent follow the user-intended target. At t = 3, Init-Only Prop. drifts toward a nearby non-target region and keeps propagating this erroneous mask, l… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 10 linked inside Pith

  1. [1]

    The use of smart glasses in the healthcare metaverse: A systematic review,

    L. Malungana and B. Chimbo, “The use of smart glasses in the healthcare metaverse: A systematic review,”Suid-Afrikaanse Tydskrif vir Natuurwetenskap en Tegnologie, vol. 44, no. 1, pp. 8–16, 2025

    2025

  2. [2]

    Designing and implementing smart glass technology for emergency medical services: a sociotechnical perspective,

    Z. Zhang, N. A. Ramiya Ramesh Babu, K. Adelgais, and M. Ozkaynak, “Designing and implementing smart glass technology for emergency medical services: a sociotechnical perspective,”JAMIA open, vol. 5, no. 4, p. ooac113, 2022

    2022

  3. [3]

    Interpre- tive error in radiology,

    S. Waite, J. Scott, B. Gale, T. Fuchs, S. Kolla, and D. Reede, “Interpre- tive error in radiology,”American Journal of Roentgenology, vol. 208, no. 4, pp. 739–749, 2017

    2017

  4. [4]

    The radiologist’s workflow environment: evaluation of disruptors and potential implications,

    J.-P. J. Yu, A. P. Kansagra, and J. Mongan, “The radiologist’s workflow environment: evaluation of disruptors and potential implications,”Jour- nal of the American College of Radiology, vol. 11, no. 6, pp. 589–593, 2014

    2014

  5. [5]

    The invisible gorilla strikes again: sustained inattentional blindness in expert observers,

    T. Drew, M. L.-H. V ˜o, and J. M. Wolfe, “The invisible gorilla strikes again: sustained inattentional blindness in expert observers,”Psycholog- ical Science, vol. 24, no. 9, pp. 1848–1853, 2013

    2013

  6. [6]

    Ova-fields: Weakly supervised open-vocabulary affordance fields for robot operational part detection,

    H. Su, M. Xie, N. Cao, Y . Ding, B. Shao, X. Long, F. Gu, and C. Chen, “Ova-fields: Weakly supervised open-vocabulary affordance fields for robot operational part detection,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 6385–6395

    2025

  7. [7]

    Clamber: A benchmark of identifying and clarifying ambiguous information needs in large language models,

    T. Zhang, P. Qin, Y . Deng, C. Huang, W. Lei, J. Liu, D. Jin, H. Liang, and T.-S. Chua, “Clamber: A benchmark of identifying and clarifying ambiguous information needs in large language models,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 10 746–10 766

    2024

  8. [8]

    Open-vocabulary 3d instruction ambiguity detection,

    J. Ding, H. Tang, and G. Li, “Open-vocabulary 3d instruction ambiguity detection,”arXiv preprint arXiv:2601.05991, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    Challenges and trends in egocentric vision: A survey,

    X. Li, H. Qiu, L. Wang, H. Zhang, C. Qi, L. Han, H. Xiong, and H. Li, “Challenges and trends in egocentric vision: A survey,”Machine Intelligence Research, vol. 23, no. 1, pp. 1–33, 2026

    2026

  10. [10]

    Temporal segmentation of egocentric videos,

    Y . Poleg, C. Arora, and S. Peleg, “Temporal segmentation of egocentric videos,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 2537–2544

    2014

  11. [11]

    Eventego3d: 3d human motion capture from egocentric event streams,

    C. Millerdurai, H. Akada, J. Wang, D. Luvizon, C. Theobalt, and V . Golyanik, “Eventego3d: 3d human motion capture from egocentric event streams,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024, pp. 1186–1195

    2024

  12. [12]

    Caldiff: calibrating uncertainty and accessing reliability of diffusion models for trustworthy lesion segmentation,

    X. Wang, M. Yang, S. Tosun, K. Nakamura, S. Li, and X. Li, “Caldiff: calibrating uncertainty and accessing reliability of diffusion models for trustworthy lesion segmentation,”IEEE journal of biomedical and health informatics, 2025

    2025

  13. [13]

    Agent-oriented planning in multi-agent systems,

    A. Li, Y . Xie, S. Li, F. Tsung, B. Ding, and Y . Li, “Agent-oriented planning in multi-agent systems,”arXiv preprint arXiv:2410.02189, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Agentorchestra: A hierarchical multi-agent framework for general- purpose task solving,

    W. Zhang, C. Cui, Y . Zhao, R. Hu, Y . Liu, Y . Zhou, and B. An, “Agentorchestra: A hierarchical multi-agent framework for general- purpose task solving,”arXiv e-prints, pp. arXiv–2506, 2025

    2025

  15. [15]

    Evolutionary generation of multi-agent systems,

    Y . Hu, M. Trager, Y . Zhang, Y . Zhang, S. Yang, W. Xia, and S. Soatto, “Evolutionary generation of multi-agent systems,”arXiv preprint arXiv:2602.06511, 2026

    Pith/arXiv arXiv 2026

  16. [16]

    Scaling egocentric vision: The epic-kitchens dataset,

    D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kaza- kos, D. Moltisanti, J. Munro, T. Perrett, W. Priceet al., “Scaling egocentric vision: The epic-kitchens dataset,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 720–736

    2018

  17. [17]

    Ego4d: Around the world in 3,000 hours of egocentric video,

    K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liuet al., “Ego4d: Around the world in 3,000 hours of egocentric video,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, 10 IEEE TRANSACTIONS AND JOURNALS TEMPLATE pp. 18 995–19 012

    2022

  18. [18]

    Omnia de egotempo: Benchmarking temporal understanding of multi-modal llms in egocentric videos,

    C. Plizzari, A. Tonioni, Y . Xian, A. Kulshrestha, and F. Tombari, “Omnia de egotempo: Benchmarking temporal understanding of multi-modal llms in egocentric videos,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 24 129–24 138

    2025

  19. [19]

    Egotextvqa: Towards egocentric scene-text aware video question answering,

    S. Zhou, J. Xiao, Q. Li, Y . Li, X. Yang, D. Guo, M. Wang, T.-S. Chua, and A. Yao, “Egotextvqa: Towards egocentric scene-text aware video question answering,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 3363–3373

    2025

  20. [20]

    Robust egocentric referring video object segmentation via dual-modal causal intervention,

    H. Liu, Z. Song, H. Wu, T. Pu, K. Wang, and L. Lin, “Robust egocentric referring video object segmentation via dual-modal causal intervention,” inAdvances in Neural Information Processing Systems, 2025

    2025

  21. [21]

    Cognition transferring and decoupling for text-supervised egocentric semantic segmentation,

    Z. Shi, H. Qiu, L. Wang, F. Meng, Q. Wu, and H. Li, “Cognition transferring and decoupling for text-supervised egocentric semantic segmentation,”arXiv preprint arXiv:2410.01341, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Egosurgery-phase: A dataset of surgical phase recognition from egocentric open surgery videos,

    R. Fujii, M. Hatano, H. Saito, and H. Kajita, “Egosurgery-phase: A dataset of surgical phase recognition from egocentric open surgery videos,” inInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2024, pp. 187–196

    2024

  23. [23]

    An egocentric life-saving interventional procedure dataset of actions, medical ques- tions, maneuvers and tools,

    Y . Zhuo, E. Zhang, X. Yu, A. Pachpande, W. Fang, X. Chen, A. W. Kirkpatrick, K. Couperus, C. Colombo, O. Tranet al., “An egocentric life-saving interventional procedure dataset of actions, medical ques- tions, maneuvers and tools,”Scientific Data, 2025

    2025

  24. [24]

    Egosurgery-hts: A dataset for egocentric hand–tool segmentation in open surgery videos,

    N. Darjana, R. Fujii, H. Saito, and H. Kajita, “Egosurgery-hts: A dataset for egocentric hand–tool segmentation in open surgery videos,” Healthcare Technology Letters, vol. 12, no. 1, p. e70049, 2025

    2025

  25. [25]

    An outlook into the future of egocentric vision: C. plizzari et al

    C. Plizzari, G. Goletto, A. Furnari, S. Bansal, F. Ragusa, G. M. Farinella, D. Damen, and T. Tommasi, “An outlook into the future of egocentric vision: C. plizzari et al.”International Journal of Computer Vision, vol. 132, no. 11, pp. 4880–4936, 2024

    2024

  26. [26]

    Egocentric co-pilot: Web- native smart-glasses agents for assistive egocentric ai,

    S. Yang, Y . Huang, W. Cai, S. Sun, F. Fang, Y . He, Y . Xie, J. Deng, H. Zhang, J. Song, and Z. Zhang, “Egocentric co-pilot: Web- native smart-glasses agents for assistive egocentric ai,”arXiv preprint arXiv:2603.01104, 2026

    arXiv 2026

  27. [27]

    Smart glasses in surgery: the theatre and beyond,

    S. Gollapalli, V . Sharma, A. Al Ghazwi, and L. Heskin, “Smart glasses in surgery: the theatre and beyond,”Surgical Innovation, vol. 31, no. 5, pp. 502–508, 2024

    2024

  28. [28]

    Clinical and surgical applications of smart glasses,

    S. Mitrasinovic, E. Camacho, N. Trivedi, J. Logan, C. Campbell, R. Zilinyi, B. Lieber, E. Bruce, B. Taylor, D. Martineauet al., “Clinical and surgical applications of smart glasses,”Technology and health care, vol. 23, no. 4, pp. 381–401, 2015

    2015

  29. [29]

    A multimodal generative ai copilot for human pathology,

    M. Y . Lu, B. Chen, D. F. Williamson, R. J. Chen, M. Zhao, A. K. Chow, K. Ikemura, A. Kim, D. Pouli, A. Patelet al., “A multimodal generative ai copilot for human pathology,”Nature, vol. 634, no. 8033, pp. 466–473, 2024

    2024

  30. [30]

    Advances in medical image segmentation: A comprehensive review of traditional, deep learning and hybrid approaches,

    Y . Xu, R. Quan, W. Xu, Y . Huang, X. Chen, and F. Liu, “Advances in medical image segmentation: A comprehensive review of traditional, deep learning and hybrid approaches,”Bioengineering, vol. 11, no. 10, p. 1034, 2024

    2024

  31. [31]

    Segment anything in medical images,

    J. Ma, Y . He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,”Nature communications, vol. 15, no. 1, p. 654, 2024

    2024

  32. [32]

    Deepigeos: a deep interactive geodesic framework for medical image segmentation,

    G. Wang, M. A. Zuluaga, W. Li, R. Pratt, P. A. Patel, M. Aertsen, T. Doel, A. L. David, J. Deprest, S. Ourselinet al., “Deepigeos: a deep interactive geodesic framework for medical image segmentation,”IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 7, pp. 1559–1572, 2018

    2018

  33. [33]

    Limis: Towards language-based interactive medi- cal image segmentation,

    L. Heinemann, A. Jaus, Z. Marinov, M. Kim, M. F. Spadea, J. Kleesiek, and R. Stiefelhagen, “Limis: Towards language-based interactive medi- cal image segmentation,” in2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI). IEEE, 2025, pp. 1–5

    2025

  34. [34]

    Tgsam-2: Text-guided medical image segmentation using segment anything model 2,

    R. Yuan, L. Zhou, J. Xu, Q. Li, M. Chen, Y . Zhang, R. Feng, T. Zhang, and S. Gao, “Tgsam-2: Text-guided medical image segmentation using segment anything model 2,” inInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2025, pp. 565–574

    2025

  35. [35]

    Medsam3: Delving into segment anything with medical concepts,

    A. Liu, R. Xue, X. R. Cao, Y . Shen, Y . Lu, X. Li, Q. Chen, and J. Chen, “Medsam3: Delving into segment anything with medical concepts,” arXiv preprint arXiv:2511.19046, 2025

    arXiv 2025

  36. [36]

    Lisa: Reasoning segmentation via large language model,

    X. Lai, Z. Tian, Y . Chen, Y . Li, Y . Yuan, S. Liu, and J. Jia, “Lisa: Reasoning segmentation via large language model,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 9579–9589

    2024

  37. [37]

    Prism: A promptable and robust interactive segmentation model with visual prompts,

    H. Li, H. Liu, D. Hu, J. Wang, and I. Oguz, “Prism: A promptable and robust interactive segmentation model with visual prompts,” in International Conference on Medical Image Computing and Computer- Assisted Intervention. Springer, 2024, pp. 389–399

    2024

  38. [38]

    Simtxtseg: Weakly-supervised medical image segmentation with simple text cues,

    Y . Xie, T. Zhou, Y . Zhou, and G. Chen, “Simtxtseg: Weakly-supervised medical image segmentation with simple text cues,” inInternational Conference on Medical Image Computing and Computer-Assisted Inter- vention. Springer, 2024, pp. 634–644

    2024

  39. [39]

    Temporally-extended prompts optimization for sam in interactive medical image segmenta- tion,

    C. Shen, W. Li, Y . Zhang, Y . Wang, and X. Wang, “Temporally-extended prompts optimization for sam in interactive medical image segmenta- tion,” in2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2023, pp. 3550–3557

    2023

  40. [40]

    Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments,

    S. Schmidgall, R. Ziaei, C. Harris, E. Reis, J. Jopling, and M. Moor, “Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments,”arXiv preprint arXiv:2405.07960, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    Medagentsim: Self- evolving multi-agent simulations for realistic clinical interactions,

    M. Almansoori, K. Kumar, and H. Cholakkal, “Medagentsim: Self- evolving multi-agent simulations for realistic clinical interactions,” in International Conference on Medical Image Computing and Computer- Assisted Intervention. Springer, 2025, pp. 362–372

    2025

  42. [42]

    Medical ai consensus: A multi-agent framework for radiology report generation and evaluation,

    A. T. Elboardy, G. Khoriba, and E. A. Rashed, “Medical ai consensus: A multi-agent framework for radiology report generation and evaluation,” arXiv preprint arXiv:2509.17353, 2025

    arXiv 2025

  43. [43]

    A multimodal multi-agent framework for radiology report generation,

    Z. Yi, T. Xiao, and M. V . Albert, “A multimodal multi-agent framework for radiology report generation,”arXiv preprint arXiv:2505.09787, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Reflectool: Towards reflection- aware tool-augmented clinical agents,

    Y . Liao, S. Jiang, Y . Wang, and Y . Wang, “Reflectool: Towards reflection- aware tool-augmented clinical agents,” inProceedings of the 63rd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 13 507–13 531

    2025

  45. [45]

    Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation,

    Y . Ji, H. Bai, C. Ge, J. Yang, Y . Zhu, R. Zhang, Z. Li, L. Zhanng, W. Ma, X. Wanet al., “Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation,”Advances in neural information processing systems, vol. 35, pp. 36 722–36 732, 2022

    2022

  46. [46]

    Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved?

    O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, X. Yang, P.-A. Heng, I. Cetin, K. Lekadir, O. Camara, M. A. G. Ballesteret al., “Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved?”IEEE transactions on medical imaging, vol. 37, no. 11, pp. 2514–2525, 2018

    2018

  47. [47]

    Deep learning for segmentation using an open large-scale dataset in 2d echocardiography,

    S. Leclerc, E. Smistad, J. Pedrosa, A. Østvik, F. Cervenansky, F. Es- pinosa, T. Espeland, E. A. R. Berg, P.-M. Jodoin, T. Grenieret al., “Deep learning for segmentation using an open large-scale dataset in 2d echocardiography,”IEEE transactions on medical imaging, vol. 38, no. 9, pp. 2198–2210, 2019

    2019

  48. [48]

    Two public chest x-ray datasets for computer-aided screen- ing of pulmonary diseases,

    S. Jaeger, S. Candemir, S. Antani, Y .-X. J. W ´ang, P.-X. Lu, and G. Thoma, “Two public chest x-ray datasets for computer-aided screen- ing of pulmonary diseases,”Quantitative imaging in medicine and surgery, vol. 4, no. 6, p. 475, 2014

    2014

  49. [49]

    A multi- centre polyp detection and segmentation dataset for generalisability assessment,

    S. Ali, D. Jha, N. Ghatwary, S. Realdon, R. Cannizzaro, O. E. Salem, D. Lamarque, C. Daul, M. A. Riegler, K. V . Anonsenet al., “A multi- centre polyp detection and segmentation dataset for generalisability assessment,”Scientific Data, vol. 10, no. 1, p. 75, 2023

    2023

  50. [50]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer, “Sam 2: Segment anything in images and videos,” 2024. [Online]. Available: https://arxiv.org/abs/2408.00714

    Pith/arXiv arXiv 2024

  51. [51]

    Grounding dino: Marrying dino with grounded pre- training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhuet al., “Grounding dino: Marrying dino with grounded pre- training for open-set object detection,”arXiv preprint arXiv:2303.05499, 2023

    Pith/arXiv arXiv 2023

  52. [52]

    Grounding dino 1.5: Advance the

    T. Ren, Q. Jiang, S. Liu, Z. Zeng, W. Liu, H. Gao, H. Huang, Z. Ma, X. Jiang, Y . Chen, Y . Xiong, H. Zhang, F. Li, P. Tang, K. Yu, and L. Zhang, “Grounding dino 1.5: Advance the ”edge” of open-set object detection,” 2024

    2024

  53. [53]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang, “Grounded sam: Assembling open-world models for diverse visual tasks,” 2024

    2024

  54. [54]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll´ar, and R. Girshick, “Segment anything,”arXiv:2304.02643, 2023

    Pith/arXiv arXiv 2023

  55. [55]

    Language segment-anything,

    L. Medeiros and contributors, “Language segment-anything,” https:// github.com/luca-medeiros/lang-segment-anything, 2023, gitHub repos- itory

    2023

  56. [56]

    Sam 3: Segment anything with concepts,

    N. Carion, L. Gustafson, Y .-T. Hu, S. Debnath, R. Hu, D. Suris, C. Ryali, K. V . Alwala, H. Khedr, A. Huanget al., “Sam 3: Segment anything with concepts,”arXiv preprint arXiv:2511.16719, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,

    F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,”Nature methods, vol. 18, no. 2, pp. 203–211, 2021

    2021

  58. [58]

    Measures of the amount of ecologic association between species,

    L. R. Dice, “Measures of the amount of ecologic association between species,”Ecology, vol. 26, no. 3, pp. 297–302, 1945

    1945