pith. sign in

arxiv: 2606.09249 · v1 · pith:D7HXJYYOnew · submitted 2026-06-08 · 💻 cs.CV

MAGIS: Evidence-Based Multi-Agent Reasoning for Interpretable Strabismus Clinical Decision-Making

Xikai Tang , Yifan Wang , Jiafan Zhuang , Li Luo , Jinming Guo , Xiaoling Xie , Jiacheng Liu , Peiwei Wei
show 7 more authors
Lihao Zhong Xiaoli Kang Jie Cen Guangqiang Yin Kunliang Qiu Ce Zheng Zhun Fan
This is my paper

Pith reviewed 2026-06-27 17:17 UTC · model grok-4.3

classification 💻 cs.CV
keywords strabismus diagnosismulti-agent reasoninginterpretable medical AIvision-language modelsevidence-based verificationclinical decision supportocular imaging
0
0 comments X

The pith

MAGIS structures strabismus diagnosis as a multi-agent process with evidence verification, lifting weighted F1 from 72.0% to 91.3%.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces MAGIS to convert black-box image-to-report generation into a sequence of hypothesis generation, dual-evidence constrained context, corrective verification, and final reporting for fine-grained strabismus subtype diagnosis. It tackles hallucination in large vision-language models by requiring every diagnostic step to align with both nine-gaze-position photographs and explicit clinical rules. The Dual-Evidence Constrained Context mechanism assembles visual cues and rule constraints into a shared context, while Evidence-Based Corrective Verification compares the current hypothesis against those cues and rules and triggers refinement on detected inconsistency. Experiments on a dedicated benchmark show gains in accuracy and in clinical report metrics of consistency, alignment, and completeness. A sympathetic reader would care because reliable, auditable reasoning matters when treatment planning depends on precise subtype identification.

Core claim

MAGIS transforms black-box end-to-end generation into a structured diagnostic process consisting of candidate hypothesis generation, dual-evidence constrained context, evidence-based corrective verification, and report generation. The Dual-Evidence Constrained Context jointly organizes visual evidence from the photograph of the nine cardinal positions of gaze and evidence-based clinical diagnostic rules into a constrained context. The Evidence-Based Corrective Verification mechanism verifies whether the current diagnostic hypothesis is supported by visual evidence, heatmap-based visual cues, and evidence-based clinical diagnostic rules, triggering hypothesis refinement when inconsistency is

What carries the argument

Dual-Evidence Constrained Context (DECC) that assembles gaze photographs and clinical rules into a shared constrained context, paired with Evidence-Based Corrective Verification (EBCV) that checks hypotheses against visual evidence, heatmaps, and rules before allowing report generation.

If this is right

  • Diagnostic accuracy improves while the generated reports become auditable against the same visual and rule evidence used by clinicians.
  • Clinical reliability metrics (consistency, alignment, completeness) rise because every output step is required to pass explicit verification.
  • The method replaces end-to-end black-box prediction with an explicit sequence that can be inspected or interrupted at any stage.
  • The same evidence-constraint pattern can be applied to other image-plus-rule medical tasks that currently suffer from hallucination.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The multi-agent loop could be extended to accept clinician overrides as additional evidence sources during verification.
  • If the nine-gaze photographs prove sufficient, the approach might reduce the need for more invasive or costly diagnostic tests in borderline cases.
  • Report generation could be made interactive so that a clinician can query which specific rule or image region supported each sentence.

Load-bearing premise

The verification steps will catch and fix hallucinations or rule violations on new unseen photographs without introducing their own systematic biases.

What would settle it

Run MAGIS on a fresh collection of clinical photographs containing known hallucinations produced by standard vision-language models and measure whether the corrective loop removes those errors without creating new rule violations or inconsistencies.

Figures

Figures reproduced from arXiv: 2606.09249 by Ce Zheng, Guangqiang Yin, Jiacheng Liu, Jiafan Zhuang, Jie Cen, Jinming Guo, Kunliang Qiu, Lihao Zhong, Li Luo, Peiwei Wei, Xiaoli Kang, Xiaoling Xie, Xikai Tang, Yifan Wang, Zhun Fan.

Figure 1
Figure 1. Figure 1: Visualization of strabismus diagnostic results generated by Gemini-3-Flash-Preview, GPT-5.2, and Qwen3-VL-Plus. These examples show hallucinated responses, including incorrect subtype predictions and clinically unsupported outputs. Text highlighted in red denotes hallucinated responses. Our MAGIS Review of the photograph of the nine cardinal positions of gaze suggests A-pattern exotropia. The patient’right… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of MAGIS for strabismus subtype diagnosis. The framework uses three interacting agents and integrates Dual￾Evidence Constrained Context (DECC) with Evidence-Based Cor￾rective Verification (EBCV) to verify the classifier’s predicted subtype against visual evidence and diagnostic rules, enabling iterative refinement before report generation. Large vision-language models (LVLMs) (Bai et al., 2025; Du… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the MAGIS framework for hallucination-mitigated strabismus subtype diagnosis. The upper panel presents the overall workflow of MAGIS, which consists of a cyclic three-agent architecture integrating Dual-Evidence Constrained Context (DECC) and Evidence-Based Corrective Verification (EBCV). The Classifier first generates a confidence-ranked list of candidate subtype predictions from the input ima… view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of the Complementary Decoupled Attention (CDA) module. CDA separately models intra-gaze and inter-gaze dependencies from the photograph of the nine cardinal positions of gaze, generating attention heatmaps as visual evidence and a confidence-ranked list of candidate subtype predictions Together with the confidence-ranked candidate subtype pre￾dictions {(𝑘𝑖 (𝐗), 𝑝𝑖 (𝐗))}𝑛 𝑖=1, these elements fo… view at source ↗
Figure 5
Figure 5. Figure 5: Doctor-in-the-loop evidence-based clinical diagnostic rules formalization. Unstructured clinical guidelines are first converted into candidate diagnostic rules by a large language model (LLM), and then reviewed and refined by strabismus specialists to ensure clinical validity. The bottom panel shows a representative example of this process: (i) clinical knowledge is collected from strabismus guidelines; (i… view at source ↗
Figure 6
Figure 6. Figure 6: Overview of Evidence-Based Corrective Verification (EBCV). EBCV validates each candidate subtype prediction through two complementary checks: rule-guided CV and heatmap-guided CV. In this example, the current diagnostic hypothesis (Exotropia) fails the rule=-guided CV. Consequently, EBCV rejects this prediction and selects the next-best candidate. The refined prediction, A-pattern Esotropia, passes both CV… view at source ↗
Figure 7
Figure 7. Figure 7: Representative diagnostic reports generated by MAGIS using Gemini-3-Flash-Preview as the LVLM backbone. Each report com￾prises several components: prediction, key abnormalities, pattern and angle analysis, corrective verification, treatment recommendations, and rules used. 4.4. Ablation Study We conduct ablation studies using Gemini-3-Flash￾Preview as the default LVLM to evaluate the contributions of the p… view at source ↗
read the original abstract

Strabismus is a common ocular disorder that requires fine-grained subtype diagnosis for individualized treatment planning. However, existing deep learning methods mainly provide diagnostic predictions without transparent reasoning, while recent large vision-language models (LVLMs), although promising for joint image understanding and report generation, remain highly prone to hallucination in this evidence-sensitive and rule-driven medical task. To address these challenges, we propose MAGIS, an evidence-based Multi-AGent reasoning for Interpretable Strabismus diagnosis framework. MAGIS transforms black-box end-to-end generation into a structured diagnostic process consisting of candidate hypothesis generation, dual-evidence constrained context, evidence-based corrective verification, and report generation. Specifically, we introduce a Dual-Evidence Constrained Context (DECC) mechanism that jointly organizes visual evidence from the photograph of the nine cardinal positions of gaze and evidence-based clinical diagnostic rules into a constrained context for reliable diagnostic reasoning. We further develop an Evidence-Based Corrective Verification (EBCV) mechanism that verifies whether the current diagnostic hypothesis is supported by visual evidence, heatmap-based visual cues, and evidence-based clinical diagnostic rules. Hypothesis refinement is triggered when inconsistency is detected. Experiments on a fine-grained strabismus benchmark demonstrate that MAGIS not only significantly outperforms other state-of-the-art diagnostic systems, improving the weighted F1 score from 72.0% to 91.3%, but also substantially improves the clinical reliability (consistency, alignment, and completeness) of generated diagnostic reports. These results demonstrate that MAGIS provides an effective solution for building accurate, evidence-based, and clinically interpretable strabismus diagnosis systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes MAGIS, a multi-agent framework for interpretable strabismus diagnosis that converts black-box LVLM generation into a structured process of hypothesis generation, Dual-Evidence Constrained Context (DECC) organization of visual evidence from nine-gaze photographs plus clinical rules, Evidence-Based Corrective Verification (EBCV) for inconsistency detection and refinement, and final report generation. On a fine-grained strabismus benchmark it reports raising weighted F1 from 72.0% to 91.3% over prior SOTA while also improving clinical reliability metrics (consistency, alignment, completeness) of the generated reports.

Significance. If the reported gains and reliability improvements are robustly supported, the work would constitute a meaningful step toward evidence-constrained, hallucination-resistant multi-agent systems for fine-grained medical image diagnosis. The explicit separation of visual evidence, rule-based constraints, and corrective verification offers a concrete template that could generalize to other rule-driven clinical tasks where pure end-to-end LVLMs currently fail.

major comments (2)
  1. [§4 Experiments] §4 Experiments (and associated tables): the central performance claim (weighted F1 rising from 72.0% to 91.3%) is presented without any description of baseline LVLM implementations, training regimes, dataset size/diversity, or statistical testing; this information is load-bearing for assessing whether the reported margin reflects genuine generalization rather than implementation differences.
  2. [§3.3 EBCV] §3.3 EBCV: the mechanism is asserted to detect and correct rule violations and hallucinations on unseen photographs, yet no ablation or failure-mode analysis is supplied that tests whether EBCV itself introduces systematic biases when the visual evidence or rule set is out-of-distribution; this directly bears on the weakest assumption identified in the review.
minor comments (1)
  1. [Abstract] Abstract: the phrase 'nine cardinal positions of gaze' is used without a brief parenthetical definition or citation; adding one sentence would improve accessibility for non-ophthalmology readers.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive review. The comments highlight important aspects of reproducibility and robustness that we address below. We provide point-by-point responses to the major comments.

read point-by-point responses

  1. Referee: [§4 Experiments] §4 Experiments (and associated tables): the central performance claim (weighted F1 rising from 72.0% to 91.3%) is presented without any description of baseline LVLM implementations, training regimes, dataset size/diversity, or statistical testing; this information is load-bearing for assessing whether the reported margin reflects genuine generalization rather than implementation differences.

    Authors: We agree that these details are necessary for proper evaluation of the results. While Section 4 describes the benchmark and reports the main metrics, it does not provide exhaustive specifications of the baseline LVLM setups, training details, full dataset statistics, or statistical tests. In the revised manuscript we will expand §4 with: (i) precise descriptions of each baseline LVLM (model variants, prompting strategies, and any fine-tuning), (ii) dataset characteristics including total images, subtype distribution, and acquisition variability, and (iii) statistical significance results (e.g., McNemar’s test or bootstrap confidence intervals) for the F1 improvements. These additions will be made without changing the reported numbers. revision: yes

  2. Referee: [§3.3 EBCV] §3.3 EBCV: the mechanism is asserted to detect and correct rule violations and hallucinations on unseen photographs, yet no ablation or failure-mode analysis is supplied that tests whether EBCV itself introduces systematic biases when the visual evidence or rule set is out-of-distribution; this directly bears on the weakest assumption identified in the review.

    Authors: We acknowledge that the manuscript lacks an explicit ablation or failure-mode study of EBCV under out-of-distribution conditions. The current evaluation is confined to the in-distribution benchmark, and while the design of EBCV relies on explicit evidence checking, we cannot rule out potential biases without additional analysis. In revision we will add a dedicated limitations paragraph in §3.3 (and §5) that discusses possible failure modes, provides qualitative examples of edge cases, and explicitly flags OOD robustness as an open question for future work. New quantitative OOD experiments are not feasible within the scope of this revision because they would require new data collection. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper proposes MAGIS as a multi-agent framework with DECC and EBCV mechanisms that organize visual evidence and clinical rules into constrained contexts for hypothesis generation, verification, and report generation. Performance is reported via direct empirical comparison on a fine-grained strabismus benchmark (weighted F1 rising from 72.0% to 91.3%). No equations, fitted parameters renamed as predictions, self-definitional loops, or load-bearing self-citations appear in the text; the derivation chain consists of architectural description followed by external benchmark evaluation rather than any reduction of outputs to inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no equations, no explicit parameter counts, and no statements of background assumptions; therefore the ledger cannot be populated beyond the generic observation that any deployed LVLM carries implicit training-data and prompting assumptions.

pith-pipeline@v0.9.1-grok · 5872 in / 1321 out tokens · 22125 ms · 2026-06-27T17:17:24.717224+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 7 canonical work pages · 6 internal anchors

  1. [1]

    Self-rag:Learning to retrieve, generate, and critique through self-reflection, in: ICLR

    Asai,A.,Wu,Z.,Wang,Y.,Sil,A.,Hajishirzi,H.,2023. Self-rag:Learning to retrieve, generate, and critique through self-reflection, in: ICLR. Bai, S., et al.,

    2023

  2. [2]

    Qwen3-VL Technical Report

    Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 . Bai,Z.,etal.,2024. Hallucinationofmultimodallargelanguagemodels:A survey. arXiv preprint arXiv:2404.18930 . Chan, C.M., et al.,

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate

    Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201 . Chen, J., et al.,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    7346–7370

    Towards injecting medical visual knowledge into multimodal llms at scale, in: Proceedings of the 2024 conference on empirical methods in natural language processing, pp. 7346–7370. Chen, W.F., Zhao, Z., Karimi, A., Flek, L., 2025a. Explainable halluci- nation through natural language inference mapping, in: Findings of the Association for Computational Lin...

    2024

  5. [5]

    3563–3578

    Chain-of-verification reduces hallucination in largelanguagemodels,in:Findingsoftheassociationforcomputational linguistics: ACL 2024, pp. 3563–3578. Dosovitskiy, et al.,

    2024

  6. [6]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 . Du,Y.,Li,S.,Torralba,A.,Tenenbaum,J.B.,Mordatch,I.,2024.Improving factuality and reasoning in language models through multiagent debate, in: Forty-first international conference on machine learning. Dubey,A.,etal.,2024. Thellama3herdofmodels. arXi...

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  7. [7]

    Ophthalmology

    Prevalence of amblyopia and strabismus in white and african american children aged 6 through 71 months: the baltimore pediatric eye disease study. Ophthalmology . Google,2026. Gemini3flashpreview.https://ai.google.dev/gemini-api/ docs/models/gemini-3-flash-preview. Accessed: June 2,

    2026

  8. [8]

    Deepresiduallearningforimage recognition,in:ProceedingsoftheIEEEconferenceoncomputervision and pattern recognition, pp

    He,K.,Zhang,X.,Ren,S.,Sun,J.,2016. Deepresiduallearningforimage recognition,in:ProceedingsoftheIEEEconferenceoncomputervision and pattern recognition, pp. 770–778. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.,

    2016

  9. [9]

    Comt: Chain-of-medical-thought reduces hallu- cination in medical report generation, in: ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 1–5. Jin,H.,Che,H.,He,S.,Chen,H.,2025. Achainofdiagnosisframeworkfor accurateandexplainableradiologyreportgeneration.IEEETransactions on Medical Imaging . Jin...

    2025

  10. [10]

    24841–24877

    Multiple llm agents debate for equitable cultural alignment, in: Proceedings of the 63rd An- nualMeetingoftheAssociationforComputationalLinguistics(Volume 1: Long Papers), pp. 24841–24877. Lee,C.,Park,S.,Shin,C.I.,Choi,W.H.,Park,H.J.,Lee,J.E.,Ye,J.C.,2026. Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation. Medi...

    2026

  11. [11]

    NeurIPS 33, 9459–9474

    Retrieval-augmented generation for knowledge- intensive nlp tasks. NeurIPS 33, 9459–9474. Li,J.,Su,T.,Zhao,B.,Lv,F.,Wang,Q.,Navab,N.,Hu,Y.,Jiang,Z.,2024. Ultrasound report generation with cross-modality feature alignment via unsupervisedguidance. IEEETransactionsonMedicalImaging44,19–

    2024

  12. [12]

    4048–4080

    Visual evidence prompting mitigates hallucinations in large vision-language models, in: ACL, pp. 4048–4080. doi:10.18653/ v1/2025.acl-long.205. Liang, T., et al.,

    2025

  13. [13]

    17889– 17904

    Encouraging divergent thinking in large language models through multi-agent debate, in: Proceedings of the 2024 confer- ence on empirical methods in natural language processing, pp. 17889– 17904. Lin,T.,etal.,2025. HealthGPT:Amedicallargevision-languagemodelfor unifying comprehension and generation via heterogeneous knowledge adaptation, in: Proceedings o...

    2024

  14. [14]

    9004–9017

    Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models, in: Proceedings of the 2023 conference on empirical methods in natural language processing, pp. 9004–9017. McKean-Cowdin,R.,etal.,2013. Prevalenceofamblyopiaorstrabismusin asian and non-hispanic white preschool children: multi-ethnic pediatric eye disease s...

    2023

  15. [15]

    arXiv preprint arXiv:2406.07155 , year=

    Scaling large language model-based multi-agent collaboration. arXiv preprint arXiv:2406.07155 . Repka, M.X., et al.,

    work page arXiv

  16. [16]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    Re- flexion:Languageagentswithverbalreinforcementlearning. Advances in neural information processing systems 36, 8634–8652. Simonyan,K.,etal.,2014.Verydeepconvolutionalnetworksforlarge-scale image recognition. arXiv preprint arXiv:1409.1556 . Sun, Z., et al.,

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  17. [17]

    Prevalence and course of strabismus throughage6yearsinparticipantsoftheearlytreatmentforretinopathy of prematurity randomized trial. J. AAPOS 15, 536–540. Wang, X., et al., 2022a. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 . Wang, Z., Han, H., Wang, L., Li, X., Zhou, L., 2022b. Automated radiog...

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [18]

    2550–2575

    Large language models are better reasoners with self-verification, in: Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 2550–2575. Yang, Y., et al.,

    2023

  19. [19]

    13807–13816

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback, in: CVPR, pp. 13807–13816. Zhang, J., Wang, G., Kalra, M.K., Yan, P., 2024a. Disease-informed adaptation of vision-language models. IEEE Transactions on Medical Imaging . Zhang, X., et al., 2024b. Self-alignment for factuality: Mitigating halluci- natio...

    1946

  20. [20]

    Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

    Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 . 15

    work page internal anchor Pith review Pith/arXiv arXiv