pith. sign in

arxiv: 2604.26283 · v2 · pith:IUBQOMLSnew · submitted 2026-04-29 · 💻 cs.CV · cs.AI

MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution

Chunzheng Zhu , Jiaqi Zeng , Junyu Jiang , Jianxin Lin , Yijun Wang This is my paper

Pith reviewed 2026-05-21 00:46 UTC · model grok-4.3

classification 💻 cs.CV cs.AI
keywords medical vision language modelslatent memory evolutiondiagnostic accuracycausal counterfactual refinementreinforcement learningclinical diagnosis
0
0 comments X

The pith

MedSynapse-V improves diagnostic accuracy in medical vision-language models by evolving latent diagnostic memories to capture clinical intuition.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current medical vision-language models suffer from information loss due to tokenization and lack the ability to adaptively invoke expert knowledge for each case. The paper proposes a method to dynamically synthesize and refine implicit diagnostic memories within the model's internal states to bridge this gap between visual perception and clinical reasoning. If this approach works, it would enable AI diagnostic tools to internalize and apply expert patterns more effectively than methods relying on external prompts or step-by-step reasoning chains.

Core claim

The paper claims that through a process of latent diagnostic memory evolution using meta queries for prior memorization, causal counterfactual refinement with reinforcement learning on masked image regions, and intrinsic memory transition to align student and teacher branches, external diagnostic expertise can be transferred into the model's endogenous parameters, leading to better performance than state-of-the-art methods including chain-of-thought paradigms.

What carries the argument

The latent diagnostic memory evolution framework that simulates experiential invocation of clinicians by synthesizing implicit memories in the hidden stream.

If this is right

  • Diagnostic accuracy increases across multiple medical imaging datasets.
  • Performance exceeds that of chain-of-thought approaches in particular.
  • Latent representations become more aligned with actual diagnostic logic through causal pruning of redundant memories.
  • Expertise is internalized rather than relying on external sources during inference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This could suggest that similar memory evolution techniques might help in non-medical vision tasks where long-range context is important.
  • Future work might test whether the dual-branch transition generalizes to other model architectures.
  • If the causal alignment holds, it may reduce hallucinations in medical AI by grounding memories in verifiable image regions.

Load-bearing premise

The assumption that counterfactual rewards derived from region-level feature masking can accurately quantify and align the causal contribution of each latent memory with actual diagnostic logic.

What would settle it

Observing no significant accuracy gains on a new medical dataset when comparing the full model against a version without the causal counterfactual refinement step.

Figures

Figures reproduced from arXiv: 2604.26283 by Chunzheng Zhu, Jianxin Lin, Jiaqi Zeng, Junyu Jiang, Yijun Wang.

Figure 1
Figure 1. Figure 1: Existing medical VLMs suffer from coarse symbolic granularity and long-range information dissipation in discrete reasoning. MedSynapse-V addresses this by evolving diagnostic implicit memory in latent space via anatomical prior condensation, causal counterfactual refinement, and autonomous latent memory internalization. that enables near-instantaneous pattern recognition against accumulated case knowledge … view at source ↗
Figure 2
Figure 2. Figure 2: Stages I and II of MedSynapse-V. The hook features from an encoder are con￾densed into diagnostic implicit memory via learnable meta-query probes and injected into the VLM hidden stream. The memory is then refined through RL with composite rewards, ensuring causal alignment between memory and clinical decision logic. chain of diagnostic memory: Fana Meta Query −−−−−−−−−→ M CCR −−−−→ M⋆ IMT −−−−→ Mauto, whe… view at source ↗
Figure 3
Figure 3. Figure 3: Intrinsic Memory Transition (IMT) is achieved via Jensen–Shannon divergence alignment between the teacher (π +, conditioned on encoder-derived Mpri) and student (π −, conditioned on Mauto) branches. Gradients propagate solely to Aψ, enabling complete removal of the anatomical encoder at inference with negligible overhead. Privileged Branch and Autonomous Branch. The teacher branch (priv￾ileged) retains the… view at source ↗
Figure 4
Figure 4. Figure 4: Effect of diagnostic probe count N. Performance peaks around N=16 across benchmarks; further increasing N dilutes diagnostically relevant signals view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison across CT, MRI, and Ultrasound cases. MedSynapse￾V produces concise, correct diagnoses, while Med-R1 and MMedExpert-R1 generate verbose CoT with hallucinated findings (red) leading to misdiagnoses. full pipeline (Avg 67.7) confirms non-redundant contributions: MQPM grounds semantics, CCR refines via exploration, IMT compresses into an autonomous pathway. (ii) Reward design. rcausal i… view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy–latency trade-off across compared VLM categories. 0 250 500 750 1000 1250 1500 1750 2000 Training Steps 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Average Reward exploration dip Full (w/ rcausal) w/o rcausal view at source ↗
Figure 7
Figure 7. Figure 7: The RL training reward dy￾namics with and without rcausal. Performance–efficiency trade-off. As shown in view at source ↗
Figure 8
Figure 8. Figure 8: t-SNE visualization of implicit memory Mauto after CCR. (a) Eight imaging modalities form well-separated clusters with clinically coherent proximity. (b, c) Within CT and Pathology, disease subtypes further segregate into distinct regions. tinguish memory-dependent from shortcut trajectories; without causal pressure the model bypasses M entirely, treating injected memory as inert padding. Latent space structure view at source ↗
Figure 9
Figure 9. Figure 9: Detailed architecture of the Diagnos￾tic Memory Sampler Pϕ. The frozen anatomi￾cal encoder Eana extracts spatial features F ∈ R Hf ×Wf ×df , which are flattened into a token se￾quence and used as key–value pairs for the learnable meta-query probes Q0. Through L layers of self￾attention, feed-forward processing, cross-attention, and a final linear projection (df → dh), the module produces N compact implicit… view at source ↗
Figure 10
Figure 10. Figure 10: Training dynamics across three stages: (a-c) Stage II reward optimization and gradient stabilization via causal refinement; (d) Stage I NTP loss convergence; (e) Stage II policy-KL evolution; (f) Stage III distillation fidelity and output agreement. 6 Training Dynamics Analysis view at source ↗
Figure 11
Figure 11. Figure 11: Causal intervention visualization on fundus (left group) and dermoscopy (right group). Each group: original image, MedSAM3 region mask B, and post-CCR memory attention map. After refinement, memory attention concentrates on diagnos￾tically critical structures while suppressing background. 8.2 Visualization of Causal Counterfactual Intervention view at source ↗
Figure 12
Figure 12. Figure 12: Memory evolution across training stages. view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative comparison across Chest X-ray, Pathology, and Head CT cases. MedSynapse-V produces concise, correct diagnoses (∼38–43 tokens), while other meth￾ods generate verbose CoT (∼195–215 tokens) with hallucinated findings (red). 9.2 Failure Case Analysis CT MRI X-ray Dermoscopy Fundus OCT Pathology Utrasound 0 20 40 60 80 100 Training sample (%) 70 60 40 20 0 78% ACC 52% ACC Single Lesion Multi-lesion… view at source ↗
Figure 14
Figure 14. Figure 14: Three representative challenging modes view at source ↗
Figure 15
Figure 15. Figure 15: Prompt template for closed-ended multi-choice VQA (VQA-RAD, SLAKE, PathVQA, PMC-VQA, MMMU*, MedXpertQA-MM, GMAI-MMBench). The number of options varies by dataset (2–5); the template adapts accordingly. System: You are a helpful medical assistant. Provide a concise answer to the question. User: <image> {question} Answer the question using a single word or phrase. Assistant view at source ↗
Figure 16
Figure 16. Figure 16: Prompt template. Notably, Mauto is autonomously generated and injected in the hidden stream without altering the text prompt. the y-axis is binary diagnostic correctness (1=correct, 0=incorrect; vertical jit￾ter applied for visibility). While high-confidence predictions are predominantly correct, a notable cluster at conf < 0.3 with correctness= 0 reveals that border￾line cases (e.g., benign vs. dysplasti… view at source ↗
read the original abstract

High-precision medical diagnosis relies not only on static imaging features but also on the implicit diagnostic memory experts instantly invoke during image interpretation. We pinpoint a fundamental cognitive misalignment in medical VLMs caused by discrete tokenization, leading to quantization loss, long-range information dissipation, and missing case-adaptive expertise. To bridge this gap, we propose ours, a framework for latent diagnostic memory evolution that simulates the experiential invocation of clinicians by dynamically synthesizing implicit diagnostic memories within the model's hidden stream. Specifically, it begins with a Meta Query for Prior Memorization mechanism, where learnable probes retrieve structured priors from an anatomical prior encoder to generate condensed implicit memories. To ensure clinical fidelity, we introduce Causal Counterfactual Refinement (CCR), which leverages reinforcement learning and counterfactual rewards derived from region-level feature masking to quantify the causal contribution of each memory, thereby pruning redundancies and aligning latent representations with diagnostic logic. This evolutionary process culminates in Intrinsic Memory Transition (IMT), a privileged-autonomous dual-branch paradigm that internalizes teacher-branch diagnostic patterns into the student-branch via full-vocabulary divergence alignment. Comprehensive empirical evaluations across multiple datasets demonstrate that ours, by transferring external expertise into endogenous parameters, significantly outperforms existing state-of-the-art methods, particularly chain-of-thought paradigms, in diagnostic accuracy. The code is available at https://github.com/zhcz328/MedSynapse-V.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces MedSynapse-V, a framework for evolving latent diagnostic memories in medical vision-language models to address cognitive misalignment from discrete tokenization. It proposes three components: (1) Meta Query for Prior Memorization, which uses learnable probes to retrieve structured priors from an anatomical prior encoder and synthesize condensed implicit memories; (2) Causal Counterfactual Refinement (CCR), which applies reinforcement learning with counterfactual rewards obtained via region-level feature masking to quantify each memory's causal contribution, prune redundancies, and align representations with diagnostic logic; and (3) Intrinsic Memory Transition (IMT), a dual-branch paradigm that internalizes teacher-branch patterns into the student branch through full-vocabulary divergence alignment. The central claim is that transferring external expertise into endogenous parameters yields significant outperformance over state-of-the-art methods, especially chain-of-thought paradigms, in diagnostic accuracy across multiple datasets, with code released at https://github.com/zhcz328/MedSynapse-V.

Significance. If the empirical gains and the causal fidelity of the CCR mechanism are substantiated, the work could advance medical VLMs by moving beyond static feature extraction toward dynamic, case-adaptive memory evolution that more closely mimics expert clinical intuition. The public release of code is a clear strength supporting reproducibility. However, the significance is tempered by the load-bearing dependence on the validity of region-masking counterfactuals for isolating diagnostic causality.

major comments (2)
  1. [§3.2] §3.2 (Causal Counterfactual Refinement): The manuscript asserts that RL rewards derived from region-level feature masking quantify the causal contribution of each latent memory and enable alignment with diagnostic logic. This step is load-bearing for the headline claim of clinical fidelity and outperformance. Yet the description provides no validation that the resulting counterfactuals isolate true diagnostic evidence rather than spatially correlated artifacts common in medical images; without such checks (e.g., expert agreement studies or controlled interventions), the subsequent pruning and IMT alignment risk misrepresenting clinician reasoning.
  2. [§4] §4 (Empirical Evaluations): The claim of significant outperformance over SOTA methods, particularly chain-of-thought, is presented without reported ablations isolating the contribution of CCR versus the prior-memorization or IMT components, nor any error bars, statistical tests, or dataset-specific breakdowns. This makes it impossible to assess whether the gains are robust or attributable to the proposed causal mechanism.
minor comments (2)
  1. [Abstract] The abstract refers to 'multiple datasets' and 'comprehensive empirical evaluations' but does not name the datasets or primary metrics in the opening summary; these should be stated explicitly in the abstract and introduction for immediate clarity.
  2. [§3.1 and §3.3] Notation for the Meta Query probes and the full-vocabulary divergence loss in IMT could be formalized with explicit equations to aid reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. The comments highlight important aspects of validation and empirical rigor that we will address in the revision. Below we respond point-by-point to the major comments.

read point-by-point responses

  1. Referee: [§3.2] §3.2 (Causal Counterfactual Refinement): The manuscript asserts that RL rewards derived from region-level feature masking quantify the causal contribution of each latent memory and enable alignment with diagnostic logic. This step is load-bearing for the headline claim of clinical fidelity and outperformance. Yet the description provides no validation that the resulting counterfactuals isolate true diagnostic evidence rather than spatially correlated artifacts common in medical images; without such checks (e.g., expert agreement studies or controlled interventions), the subsequent pruning and IMT alignment risk misrepresenting clinician reasoning.

    Authors: We agree that explicit validation of the counterfactual mechanism is necessary to support claims of clinical fidelity. Region-level masking is motivated by the anatomical prior encoder to target spatially localized diagnostic features, but we acknowledge the risk of capturing correlated artifacts. In the revised manuscript we will expand §3.2 with (i) qualitative examples overlaying masked regions on expert-annotated diagnostic findings from a held-out subset and (ii) a controlled sensitivity study measuring accuracy degradation when masking is applied to clinically relevant versus irrelevant areas. Full-scale multi-radiologist agreement studies exceed the scope and resources of the current work and are noted as future work; the added analyses will nevertheless strengthen the causal interpretation. revision: partial

  2. Referee: [§4] §4 (Empirical Evaluations): The claim of significant outperformance over SOTA methods, particularly chain-of-thought, is presented without reported ablations isolating the contribution of CCR versus the prior-memorization or IMT components, nor any error bars, statistical tests, or dataset-specific breakdowns. This makes it impossible to assess whether the gains are robust or attributable to the proposed causal mechanism.

    Authors: We accept that the empirical presentation would be strengthened by component-wise analysis and statistical reporting. The revised §4 will include: (1) ablation tables removing Meta Query, CCR, and IMT individually while keeping the other modules fixed; (2) mean and standard deviation over five random seeds for all main results; (3) paired statistical significance tests (Wilcoxon signed-rank) against the strongest chain-of-thought baseline; and (4) per-dataset breakdowns of accuracy, sensitivity, and specificity. These additions will make the contribution of the causal refinement mechanism transparent. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation remains self-contained

full rationale

The provided abstract and description outline a multi-stage framework (Meta Query for Prior Memorization, CCR via RL on region-masked counterfactuals, and IMT alignment) leading to claimed empirical outperformance on diagnostic accuracy. No equations, self-citations, or fitted-parameter renamings are present that reduce any load-bearing prediction or uniqueness claim to its own inputs by construction. The central performance claim rests on external evaluations across datasets rather than tautological fitting, satisfying the criteria for an independent derivation chain. Potential concerns about reward validity belong to correctness or assumption risk, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so specific free parameters, axioms, and invented entities cannot be extracted or audited. The framework introduces concepts such as implicit diagnostic memories and causal counterfactual rewards, but their grounding is not detailed.

pith-pipeline@v0.9.0 · 5784 in / 1113 out tokens · 37029 ms · 2026-05-21T00:46:06.658400+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Deformba: Vision State Space Model with Adaptive State Fusion

    cs.CV 2026-05 unverdicted novelty 6.0

    Deformba introduces context-adaptive state fusion to vision SSMs for better spatial augmentation and cross-stream interactions, showing strong results on 2D classification/detection/segmentation and 3D BEV perception ...

Reference graph

Works this paper leans on

69 extracted references · 69 canonical work pages · cited by 1 Pith paper · 11 internal anchors

  1. [1]

    arXiv preprint arXiv.2407.15621

    Arasteh, S.T., Lotfinia, M., Bressem, K., Siepmann, R., Adams, L., Ferber, D., Kuhl, C., Kather, J.N., Nebelung, S., Truhn, D.: Radiorag: factual large language models for enhanced diagnostics in radiology using online retrieval augmented gen- eration 2024. arXiv preprint arXiv.2407.15621

    work page arXiv 2024

  2. [2]

    Qwen3-VL Technical Report

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    arXiv preprint arXiv:2512.16201 (2025)

    Bose, S., Rajendran, R.K., Debnath, B., Karydis, K., Roy-Chowdhury, A.K., Chakradhar, S.: Visual alignment of medical vision-language models for grounded radiology report generation. arXiv preprint arXiv:2512.16201 (2025)

    work page arXiv 2025

  4. [4]

    Cognitive research: prin- ciples and implications4(1), 7 (2019)

    Brunyé, T.T., Drew, T., Weaver, D.L., Elmore, J.G.: A review of eye tracking for understanding and improving diagnostic interpretation. Cognitive research: prin- ciples and implications4(1), 7 (2019)

    work page 2019

  5. [5]

    arXiv preprint arXiv:2510.12603 (2025)

    Chen, C., Ma, Z., Li, Y., Hu, Y., Wei, Y., Li, W., Nie, L.: Reasoning in the dark: Interleaved vision-text reasoning in latent space. arXiv preprint arXiv:2510.12603 (2025)

    work page arXiv 2025

  6. [6]

    Zhu et al

    Chen, J., Gui, C., Ouyang, R., Gao, A., Chen, S., Chen, G.H., Wang, X., Cai, Z., Ji, K., Wan, X., et al.: Towards injecting medical visual knowledge into multimodal 28 C. Zhu et al. llms at scale. In: Proceedings of the 2024 conference on empirical methods in natural language processing. pp. 7346–7370 (2024)

    work page 2024

  7. [7]

    arXiv preprint arXiv:2510.10052 (2025)

    Chen, K., Rui, S., Jiang, Y., Wu, J., Zheng, Q., Song, C., Wang, X., Zhou, M., Liu, M.: Think twice to see more: Iterative visual reasoning in medical vlms. arXiv preprint arXiv:2510.10052 (2025)

    work page arXiv 2025

  8. [8]

    Sam-med2d

    Cheng, J., Ye, J., Deng, Z., Chen, J., Li, T., Wang, H., Su, Y., Huang, Z., Chen, J., Jiang, L., et al.: Sam-med2d. arXiv preprint arXiv:2308.16184 (2023)

    work page arXiv 2023

  9. [9]

    arXiv preprint arXiv:2506.08356 (2025)

    Chopra, S., Sanchez-Rodriguez, G., Mao, L., Feola, A.J., Li, J., Kira, Z.: Medmoe: modality-specialized mixture of experts for medical vision-language understanding. arXiv preprint arXiv:2506.08356 (2025)

    work page arXiv 2025

  10. [10]

    From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step

    Deng, Y., Choi, Y., Shieber, S.: From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint arXiv:2405.14838 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [11]

    arXiv preprint arXiv:2601.10949 (2026)

    Ding, M., Zhang, J., Wang, W., Zhong, H., Luo, X., Chen, W., Shen, L.: Mmedexpert-r1: Strengthening multimodal medical reasoning via domain-specific adaptation and clinical guideline reinforcement. arXiv preprint arXiv:2601.10949 (2026)

    work page arXiv 2026

  12. [12]

    Gai, X., Zhou, C., Liu, J., Feng, Y., Wu, J., Liu, Z.: Medthink: Explaining medical visualquestionansweringviamultimodaldecision-makingrationale.arXivpreprint arXiv:2404.12372 (2024)

    work page arXiv 2024

  13. [13]

    Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

    Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B.R., Kailkhura, B., Bhatele, A., Goldstein, T.: Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint arXiv:2502.05171 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  14. [14]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Gu, T., Yang, K., Liu, D., Cai, W.: Lapa: Latent prompt assist model for med- ical visual question answering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4971–4980 (2024)

    work page 2024

  15. [15]

    Training Large Language Models to Reason in a Continuous Latent Space

    Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., Tian, Y.: Train- ing large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    PathVQA: 30000+ Questions for Medical Visual Question Answering

    He, X., Zhang, Y., Mou, L., Xing, E., Xie, P.: Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2003

  17. [17]

    Iclr1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Iclr1(2), 3 (2022)

    work page 2022

  18. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Hu, Y., Li, T., Lu, Q., Shao, W., He, J., Qiao, Y., Luo, P.: Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22170–22183 (2024)

    work page 2024

  19. [19]

    IEEE Transactions on Medical Imaging (2026)

    Lai, Y., Zhong, J., Li, M., Zhao, S., Li, Y., Psounis, K., Yang, X.: Med-r1: Rein- forcement learning for generalizable medical reasoning in vision-language models. IEEE Transactions on Medical Imaging (2026)

    work page 2026

  20. [20]

    Scientific data 5(1), 1–10 (2018)

    Lau, J.J., Gayen, S., Ben Abacha, A., Demner-Fushman, D.: A dataset of clinically generated visual questions and answers about radiology images. Scientific data 5(1), 1–10 (2018)

    work page 2018

  21. [21]

    arXiv preprint arXiv:2510.22728 (2025)

    Le-Duc, K., Nguyen, D.M., Trinh, P.T., Nguyen, T.P., Diep, N.T., Ngo, A., Vu, T., Vuong, T., Nguyen, A.T., Nguyen, M., et al.: S-chain: Structured visual chain- of-thought for medicine. arXiv preprint arXiv:2510.22728 (2025)

    work page arXiv 2025

  22. [22]

    Advances in neural information processing systems 33, 9459–9474 (2020) Medical Latent Memory Evolution 29

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020) Medical Latent Memory Evolution 29

    work page 2020

  23. [23]

    In: Findings of the Association for Computational Linguistics: EMNLP 2024

    Li, B., Yan, T., Pan, Y., Luo, J., Ji, R., Ding, J., Xu, Z., Liu, S., Dong, H., Lin, Z., et al.: Mmedagent: Learning to use medical tools with multi-modal agent. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 8745–8760 (2024)

    work page 2024

  24. [24]

    Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    work page 2023

  25. [25]

    Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space, 2025a

    Li, H., Li, C., Wu, T., Zhu, X., Wang, Y., Yu, Z., Jiang, E.H., Zhu, S.C., Jia, Z., Wu, Y.N., et al.: Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space. arXiv preprint arXiv:2505.13308 (2025)

    work page arXiv 2025

  26. [26]

    arXiv preprint arXiv:2411.14522 (2024)

    Li, T., Su, Y., Li, W., Fu, B., Chen, Z., Huang, Z., Wang, G., Ma, C., Chen, Y., Hu, M., et al.: Gmai-vl & gmai-vl-5.5 m: A large vision-language model and a comprehensive multimodal dataset towards general medical ai. arXiv preprint arXiv:2411.14522 (2024)

    work page arXiv 2024

  27. [27]

    IEEE Transactions on Information theory37(1), 145–151 (2002)

    Lin, J.: Divergence measures based on the shannon entropy. IEEE Transactions on Information theory37(1), 145–151 (2002)

    work page 2002

  28. [28]

    arXiv preprint arXiv:2511.19046 (2025)

    Liu, A., Xue, R., Cao, X.R., Shen, Y., Lu, Y., Li, X., Chen, Q., Chen, J.: Medsam3: Delving into segment anything with medical concepts. arXiv preprint arXiv:2511.19046 (2025)

    work page arXiv 2025

  29. [29]

    Liu, B., Zhan, L.M., Xu, L., Ma, L., Yang, Y., Wu, X.M.: Slake: A semantically- labeledknowledge-enhanceddatasetformedicalvisualquestionanswering.In:2021 IEEE 18th international symposium on biomedical imaging (ISBI). pp. 1650–1654. IEEE (2021)

    work page 2021

  30. [30]

    arXiv preprint arXiv:2412.17747 (2024)

    Liu, L., Pfeiffer, J., Wu, J., Xie, J., Szlam, A.: Deliberation in latent space via differentiable cache augmentation. arXiv preprint arXiv:2412.17747 (2024)

    work page arXiv 2024

  31. [31]

    In: Machine learning for health (ML4H)

    Moor, M., Huang, Q., Wu, S., Yasunaga, M., Dalmia, Y., Leskovec, J., Zakka, C., Reis, E.P., Rajpurkar, P.: Med-flamingo: a multimodal medical few-shot learner. In: Machine learning for health (ML4H). pp. 353–367. PMLR (2023)

    work page 2023

  32. [32]

    arXiv preprint arXiv:2602.23363 (2026)

    Mullappilly, S.S., Kurpath, M.I., Mohamed, O., Zidan, M., Khan, F., Khan, S., Anwer, R., Cholakkal, H.: Medix-r1: Open ended medical reinforcement learning. arXiv preprint arXiv:2602.23363 (2026)

    work page arXiv 2026

  33. [33]

    arXiv preprint arXiv:2412.07769 (2024)

    Mullappilly, S.S., Kurpath, M.I., Pieri, S., Alseiari, S.Y., Cholakkal, S., Aldahmani, K., Khan, F., Anwer, R., Khan, S., Baldwin, T., et al.: Bimedix2: Bio-medical expert lmm for diverse medical modalities. arXiv preprint arXiv:2412.07769 (2024)

    work page arXiv 2024

  34. [34]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Nath, V., Li, W., Yang, D., Myronenko, A., Zheng, M., Lu, Y., Liu, Z., Yin, H., Law, Y.M., Tang, Y., et al.: Vila-m3: Enhancing vision-language models with medical expert knowledge. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14788–14798 (2025)

    work page 2025

  35. [35]

    Advances in Health Sciences Education14(Suppl 1), 37–49 (2009)

    Norman, G.: Dual processing and diagnostic errors. Advances in Health Sciences Education14(Suppl 1), 37–49 (2009)

    work page 2009

  36. [36]

    Advances in neural information processing sys- tems35, 27730–27744 (2022)

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. Advances in neural information processing sys- tems35, 27730–27744 (2022)

    work page 2022

  37. [37]

    In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention

    Pan, J., Liu, C., Wu, J., Liu, F., Zhu, J., Li, H.B., Chen, C., Ouyang, C., Rueck- ert, D.: Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention. pp. 337–347. Springer (2025) 30 C. Zhu et al

    work page 2025

  38. [38]

    Multimodal chain of continuous thought for latent-space reasoning in vision- language models,

    Pham, T.H., Ngo, C.: Multimodal chain of continuous thought for latent-space reasoning in vision-language models. arXiv preprint arXiv:2508.12587 (2025)

    work page arXiv 2025

  39. [39]

    Advances in neural information processing systems36, 53728–53741 (2023)

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems36, 53728–53741 (2023)

    work page 2023

  40. [40]

    Proximal Policy Optimization Algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  41. [41]

    MedGemma Technical Report

    Sellergren, A., Kazemzadeh, S., Jaroensri, T., Kiraly, A., Traverse, M., Kohlberger, T., Xu, S., Jamil, F., Hughes, C., Lau, C., et al.: Medgemma technical report. arXiv preprint arXiv:2507.05201 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  42. [42]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  43. [43]

    In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

    Shen, Z., Yan, H., Zhang, L., Hu, Z., Du, Y., He, Y.: Codi: Compressing chain- of-thought into continuous space via self-distillation. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 677–693 (2025)

    work page 2025

  44. [44]

    arXiv preprint arXiv:2504.01886 (2025)

    Su, Y., Li, T., Liu, J., Ma, C., Ning, J., Tang, C., Ju, S., Ye, J., Chen, P., Hu, M., et al.: Gmai-vl-r1: Harnessing reinforcement learning for multimodal medical reasoning. arXiv preprint arXiv:2504.01886 (2025)

    work page arXiv 2025

  45. [45]

    arXiv preprint arXiv:2506.16962 (2025)

    Sun, H., Jiang, Y., Lou, W., Zhang, Y., Li, W., Wang, L., Liu, M., Liu, L., Wang, X.: Chiron-o1: Igniting multimodal large language models towards gen- eralizable medical reasoning via mentor-intern collaborative search. arXiv preprint arXiv:2506.16962 (2025)

    work page arXiv 2025

  46. [46]

    arXiv preprint arXiv:2505.16552 (2025)

    Tan, W., Li, J., Ju, J., Luo, Z., Song, R., Luan, J.: Think silently, think fast: Dy- namic latent compression of llm reasoning chains. arXiv preprint arXiv:2505.16552 (2025)

    work page arXiv 2025

  47. [47]

    In: International Conference on Medical Image Computing and Computer- Assisted Intervention

    Van Sonsbeek, T., Derakhshani, M.M., Najdenkoska, I., Snoek, C.G., Worring, M.: Open-ended medical visual question answering through prefix tuning of language models. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 726–736. Springer (2023)

    work page 2023

  48. [48]

    American Journal of Roentgenology208(4), 739–749 (2017)

    Waite, S., Scott, J., Gale, B., Fuchs, T., Kolla, S., Reede, D.: Interpretive error in radiology. American Journal of Roentgenology208(4), 739–749 (2017)

    work page 2017

  49. [49]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Wang, Y., Liu, J., Gao, S., Feng, B., Tang, Z., Gai, X., Wu, J., Liu, Z.: V2t- cot: From vision to text chain-of-thought for medical reasoning and diagnosis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 658–668. Springer (2025)

    work page 2025

  50. [50]

    Advances in neural information processing systems35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)

    work page 2022

  51. [51]

    Nature Communications16(1), 7866 (2025)

    Wu, C., Zhang, X., Zhang, Y., Hui, H., Wang, Y., Xie, W.: Towards generalist foun- dation model for radiology by leveraging web-scale 2d&3d medical data. Nature Communications16(1), 7866 (2025)

    work page 2025

  52. [52]

    arXiv preprint arXiv:2504.00993 (2025)

    Wu, J., Deng, W., Li, X., Liu, S., Mi, T., Peng, Y., Xu, Z., Liu, Y., Cho, H., Choi, C.I., et al.: Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs. arXiv preprint arXiv:2504.00993 (2025)

    work page arXiv 2025

  53. [53]

    arXiv preprint arXiv:2408.04187 (2024) Medical Latent Memory Evolution 31

    Wu, J., Zhu, J., Qi, Y., Chen, J., Xu, M., Menolascina, F., Grau, V.: Medical graph rag: Towards safe medical large language model via graph retrieval-augmented generation. arXiv preprint arXiv:2408.04187 (2024) Medical Latent Memory Evolution 31

    work page arXiv 2024

  54. [54]

    In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Xu, Y., Guo, X., Zeng, Z., Miao, C.: Softcot: Soft chain-of-thought for efficient reasoning with llms. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 23336–23351 (2025)

    work page 2025

  55. [55]

    arXiv preprint arXiv:2505.11484 (2025)

    Xu, Y., Guo, X., Zeng, Z., Miao, C.: Softcot++: Test-time scaling with soft chain- of-thought reasoning. arXiv preprint arXiv:2505.11484 (2025)

    work page arXiv 2025

  56. [56]

    In: Findings of the Association for Computa- tional Linguistics: EMNLP 2024

    Xu, Z., Wang, H., Bespalov, D., Wu, X., Stone, P., Qi, Y.: Lars: Latent reasoning skills for chain-of-thought reasoning. In: Findings of the Association for Computa- tional Linguistics: EMNLP 2024. pp. 3624–3643 (2024)

    work page 2024

  57. [57]

    Advances in Neural Information Processing Sys- tems37, 94327–94427 (2024)

    Ye, J., Wang, G., Li, Y., Deng, Z., Li, W., Li, T., Duan, H., Huang, Z., Su, Y., Wang, B., et al.: Gmai-mmbench: A comprehensive multimodal evaluation bench- mark towards general medical ai. Advances in Neural Information Processing Sys- tems37, 94327–94427 (2024)

    work page 2024

  58. [58]

    arXiv e-prints pp

    Yu, H., Cheng, T., Cheng, Y., Feng, R.: Finemedlm-o1: Enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training. arXiv e-prints pp. arXiv–2501 (2025)

    work page 2025

  59. [59]

    Vismem: Latent vision memory unlocks potential of vision-language models,

    Yu, X., Xu, C., Zhang, G., Chen, Z., Zhang, Y., He, Y., Jiang, P.T., Zhang, J., Hu, X., Yan, S.: Vismem: Latent vision memory unlocks potential of vision-language models. arXiv preprint arXiv:2511.11007 (2025)

    work page arXiv 2025

  60. [60]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9556–9567 (2024)

    work page 2024

  61. [61]

    Nejm ai1(2), AIoa2300068 (2024)

    Zakka, C., Shad, R., Chaurasia, A., Dalal, A.R., Kim, J.L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., et al.: Almanac—retrieval-augmented lan- guage models for clinical medicine. Nejm ai1(2), AIoa2300068 (2024)

    work page 2024

  62. [62]

    Appagent: Multimodal agents as smartphone users

    Zhang, G., Fu, M., Yan, S.: Memgen: Weaving generative latent memory for self- evolving agents. arXiv preprint arXiv:2509.24704 (2025)

    work page arXiv 2025

  63. [63]

    arXiv preprint arXiv:2508.02258 (2025)

    Zhang, W., Guo, J., Zhang, H., Zhang, P., Chen, J., Zhang, S., Zhang, Z., Yi, Y., Bu, H.: Patho-agenticrag: towards multimodal agentic retrieval- augmented generation for pathology vlms via reinforcement learning. arXiv preprint arXiv:2508.02258 (2025)

    work page arXiv 2025

  64. [64]

    PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering

    Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., Xie, W.: Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  65. [65]

    arXiv preprint arXiv:2505.19092 (2025)

    Zhang, Y., Xu, W., Zhao, X., Wang, W., Feng, F., He, X., Chua, T.S.: Reinforced latent reasoning for llm-based recommendation. arXiv preprint arXiv:2505.19092 (2025)

    work page arXiv 2025

  66. [66]

    In: Pro- ceedings of the ACM on Web Conference 2025

    Zhao, X., Liu, S., Yang, S.Y., Miao, C.: Medrag: Enhancing retrieval-augmented generation with knowledge graph-elicited reasoning for healthcare copilot. In: Pro- ceedings of the ACM on Web Conference 2025. pp. 4442–4457 (2025)

    work page 2025

  67. [67]

    InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

    Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al.: Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  68. [68]

    arXiv preprint arXiv:2412.06141 (2024)

    Zhu, K., Xia, P., Li, Y., Zhu, H., Wang, S., Yao, H.: Mmedpo: Aligning medical vision-language models with clinical-aware multimodal preference optimization. arXiv preprint arXiv:2412.06141 (2024)

    work page arXiv 2024

  69. [69]

    MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding

    Zuo, Y., Qu, S., Li, Y., Chen, Z., Zhu, X., Hua, E., Zhang, K., Ding, N., Zhou, B.: Medxpertqa: Benchmarking expert-level medical reasoning and understanding. arXiv preprint arXiv:2501.18362 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025