Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge

Bo Liu; Huazhu Fu; Huiqi Li; Jia Guo; Jiawei Du; Meng Wang; Shengzhu Yang; Shuai Lu; Weihang Zhang

REVIEW 5 major objections 5 minor 21 references

Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge

T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B retinal VLM claims to top massive proprietary systems on ophthalmic VQA

desk verdict The architecture is sensible and the story is coherent, but the central empirical claim is unverifiable as reported because the private training set and the private TM4K evaluation set may overlap, and no artifacts are released. read the letter →

arxiv 2603.07131 v4 pith:ZSFMHEXN submitted 2026-03-07 cs.CV cs.AI

Shuai Lu , Meng Wang , Jia Guo , Jiawei Du , Bo Liu , Shengzhu Yang , Weihang Zhang , Huazhu Fu

show 1 more author

Huiqi Li

This is my paper

classification cs.CVcs.AI

keywords retinalfundusimagesophthalmicvision-languagemodelvisualquestionansweringexpertinjectiondual-streamencodinggatedfusionhallucinationgroundinglow-datamedicalAI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that general-purpose vision-language models fail at retinal diagnosis for two structural reasons: coarse visual encoders miss fine lesions, and language priors overwhelm weak visual evidence in deeper layers. It proposes EyExIn, a framework that couples a frozen general encoder with a trainable expert fundus encoder, fuses the two streams with a gated module, and injects the fused visual features directly into intermediate LLM layers as persistent 'vision anchors.' Trained on 150K fundus images, the 7-billion-parameter model reports state-of-the-art macro-F1 on four retinal benchmarks, including 78.07% on a private 21-class dataset, outperforming far larger proprietary systems. The architectural claim is that grounding deep reasoning in raw visual features, rather than letting prompts decay, is what makes the difference.

What carries the argument

The load-bearing mechanism is Adaptive Deep Expert Injection. A dual-stream encoder produces two token sets: a frozen general stream for anatomical context and a trainable expert stream for fine pathology. A Semantic-Adaptive Gated Fusion computes a token-wise convex blend, amplifying lesion regions while suppressing background. Then, at each selected intermediate LLM layer, a per-token routing gate and a zero-initialized layer scale inject the fused features as a residual bias. These 'vision anchors' create a shortcut that refreshes visual evidence in deep layers without erasing pre-trained language capabilities.

What would settle it

Check whether any image in TM4K, or in the other test sets, appears in the 150K training set, or comes from the same acquisition protocol: if overlap or near-duplicate distributions exist, the reported gap would not stand. A cleaner falsifier is to train the same EyExIn pipeline and a baseline on disjoint splits of the same dataset and compare marginal improvements on never-seen distributions.

Watch

Extended reading notes

Core claim

EyExIn's central discovery is that injecting fused expert visual features as residual biases directly into intermediate LLM layers—'Vision Anchors'—prevents visual signal decay and keeps clinical reasoning grounded in image evidence. The paper identifies a Perception Gap (general encoders miss microaneurysms and other fine pathology) and a Reasoning Gap (language priors dominate when visual evidence is weak), and it addresses both with a dual-stream encoder plus semantic-adaptive gated fusion followed by adaptive deep injection. In the paper's experiments, the 7B model achieves state-of-the-art diagnostic precision across four retinal benchmarks, with closed-VQA macro-F1 of 78.07% on TM4K an

Load-bearing premise

The evaluation assumes the private TM4K test set is disjoint from and statistically different from the 150K training images, so the reported 78.07% F1 reflects generalization rather than memorization of the training distribution.

Editorial extensions

If this is right

If correct, small open-weight models in a low-data medical setting can match or exceed much larger proprietary systems without massive instruction tuning or RLHF.
The reported mechanism gives a concrete recipe: pair a generalist encoder with a domain encoder, fuse selectively, and inject at depth—rather than relying on prompt-level integration.
The paper's ablation ties each component to a measurable effect: the expert stream raises recall, gated fusion raises precision, and adaptive injection prevents visual signal decay while preserving language fluency.
On open-ended VQA, the claimed precision numbers suggest fewer hallucinated lesions, a property central to clinical trust.
The framework is data-efficient: only the LLM is LoRA-fine-tuned and the expert stream is trainable, keeping the approach feasible in privacy-sensitive settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the separation of anatomical and pathological streams is what carries the benefit, a testable extension is to swap the expert encoder for other specialized medical encoders and see whether the same gated deep injection lifts diagnostic VQA performance.
The claim that language priors override visual evidence in deep layers suggests that the same injection mechanism could reduce hallucinations in general vision-language models on fine-grained visual tasks beyond medicine.
The paper leaves open how the layer injection schedule is chosen; a natural follow-up is to learn which layers need anchoring per task rather than applying the same set everywhere.
Because the evaluation emphasizes macro-F1 on rare classes, a direct next step is to measure per-class recall on the smallest classes to see if gated fusion genuinely improves rare-pathology detection or only the average.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper has a clear and plausible architecture: a frozen general visual encoder paired with a trainable fundus expert encoder, token-wise gated fusion, and adaptive residual injection into intermediate LLM layers. The framing of a Perception Gap and a Reasoning Gap is useful, and the ablation on TM4K shows each component contributes. If the reported numbers are real, this would be a meaningful within-subfield result for low-data ophthalmic VQA. I agree with the reader that the novelty is in the engineering combination, not in a new scientific principle, and that is fine.

The soft spots are real and load-bearing. Most importantly, the paper never states that the 150K training set and the TM4K evaluation set are disjoint. Both are private clinical datasets, and TM4K is also used as the validation set for ablations. That makes the reported 78.07% F1 on TM4K genuinely suspect until overlap or near-duplicates are ruled out. The same problem does not afflict the public benchmarks (JSIEC, Retina, ODIR), but the headline numbers come from TM4K. Additionally, there are no error bars, no statistical tests, and no code or data release, so the proprietary baseline comparisons are not reproducible. The "strictly grounded" claim is supported only by selected qualitative examples and text-similarity metrics that do not measure grounding at all. These are exactly the kind of evidence gaps that matter for an empirical SOTA claim.

The stress-test note holds up: the ambiguity around the TM4K split is the central issue, and the paper does not address it. I would not go so far as to say the architecture is flawed; the design choices are reasonable and the writing is honest about what it does.

Bottom line: this is a paper for researchers building medical VLMs, and it deserves engagement in principle. But as it stands, the evidence quality does not justify acceptance. If the authors clarify the data split, release code and data, and add basic statistics, this could become a solid contribution. I would send it to peer review with the explicit request to probe the dataset overlap and reproducibility, but I would expect major revision before publication.

Referee Report

5 major / 5 minor

Summary. The paper proposes EyExIn, a framework for retinal VQA that combines a frozen general vision encoder with a trainable expert fundus encoder, fuses their outputs with a learned gating module, and injects the fused visual features into intermediate LLM transformer layers as residual biases ('Vision Anchors'). The authors identify a 'Perception Gap' and a 'Reasoning Gap' in standard LVLMs and claim that EyExIn addresses both. Using Qwen2.5-VL (7B) as backbone and a pre-trained fundus foundation model, they fine-tune on 150K private fundus images and evaluate on TM4K, JSIEC, Retina, and ODIR. They report state-of-the-art macro F1-scores, e.g., 78.07% on TM4K and 80.66% on JSIEC closed VQA, outperforming proprietary models such as GPT-5.2 and Gemini3-Pro. An ablation study on TM4K supports the contribution of each component.

Significance. If the reported numbers are reproducible, the contribution is significant: a 7B open-weights model outperforming much larger proprietary LVLMs in a low-data medical domain would be practically valuable and would lend support to the idea that domain-specific expert encoders and deep visual injection mitigate hallucination. The architecture is clean, the ablation is logically structured, and the authors are explicit about several implementation details (LoRA rank, learning rate, batch size). However, the empirical claims are not currently verifiable: no code or data is released, no error bars or significance tests are provided, and the evaluation metrics do not measure the 'grounding' that is central to the paper's thesis. The strengths of the paper are the design and the plausibility of the mechanism; the weakness is the evidentiary support.

major comments (5)

[§3 (Experimental Datasets) and Table 3] The manuscript never states whether the 150K-image training set and TM4K (4,022 images) are disjoint, nor whether patient-level de-duplication was performed. The ablation is conducted on 'the closed validation set of TM4K' (Section 3, Ablation Study), indicating that TM4K is used for model development; if the same images or near-duplicates appear in the training set, the Table 1 result of 78.07% F1 on TM4K closed VQA may reflect memorization rather than generalization. This is load-bearing because the central claim is that EyExIn 'achieves state-of-the-art diagnostic precision' on held-out benchmarks. Please provide an explicit statement of dataset splits, overlap statistics, and patient-level de-duplication, and make the evaluation protocol reproducible.
[§3 (Main Results, Tables 1 and 2)] Differences between methods are reported as point estimates without variance, confidence intervals, or significance tests. For a clinical diagnostic claim, this is insufficient. Moreover, the computation of Macro F1/Recall/Precision for open-ended VQA is not defined; if these are derived from text-similarity thresholds, the reported 'precision' (e.g., 96.15% on Retina) is not a clinical precision. Table 2's BLEU/ROUGE/METEOR/BERT-F1 measure lexical or embedding similarity and do not measure whether outputs are grounded in the image.
[§2.3 and §4] The paper repeatedly claims that Deep Expert Injection 'forces the reasoning stack to remain strictly grounded in visual evidence' and suppresses hallucination, but no evaluation quantifies grounding or hallucination (e.g., expert clinician rating, evidence-attribution metric, or object-level grounding). The text-similarity metrics in Table 2 cannot distinguish a correct, grounded reasoning chain from a fluent, internally consistent but ungrounded output. Please add a direct grounding/hallucination evaluation or temper the claims accordingly.
[§1 and Figure 1] The claim that 'sparse visual evidence is progressively overridden by massive language priors in deeper transformer layers' is not empirically established in this manuscript. No analysis of layer-wise visual signal decay is provided; the qualitative example does not quantify the effect. If the paper's motivation is this structural deficiency, the architecture's success on benchmarks is only indirect evidence. An additional analysis (e.g., layer-wise attention or intervention) would substantiate the mechanism.
[§2.3, Eq. (3)] The injection is specified only at a high level. It is not stated how 'corresponding visual token positions' are identified, how many intermediate layers receive injection, whether all visual tokens are injected, or how the router g_l is trained. Without these details, the method is not reproducible. Please report the layer range, the number of injected layers, and the exact position mapping.

minor comments (5)

[Tables 1 and 2] Formatting issues: 'T able 1', 'LLaV A', and 'BER T-F1' appear broken in several places. Please fix the table rendering.
[§1 and §3] The term 'data-efficient' is misleading because the method is trained on 150K images. Clarify that efficiency refers to parameter efficiency (LoRA) or the low-data regime relative to proprietary models, not the absolute training set size.
[§3 (Datasets)] TM4K is not described with a citation or a data card. Please provide details on collection, annotation protocol, class distribution, and any preprocessing.
[§3 (Implementation Details)] The proprietary models (Qwen3-VL-Max, ChatGPT-5.2, Gemini3-Pro) are compared without specifying their exact versions, API access dates, or inference settings (e.g., temperature, decoding strategy). This affects comparability.
[§2.3, Eq. (3)] Please state explicitly that γ_l is initialized to zero and describe whether it is learned layer-wise. The notation tanh(γ_l) suggests a learned scalar, but the text only says 'zero-initialized.'

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-citation in the expert stream; central VQA claims rest on external benchmarks, so no substantial circularity.

self citation load bearing [Section 2.1 (Expert-Aware Dual-Stream Encoding); also Implementation Details]
"Expert Stream (Pathological Semantics): A contrastively pre-trained fundus foundation encoder [9] extracts fine-grained features Fexp, providing high sensitivity to subtle lesions like microaneurysms. To align the feature spaces, a linear projection ϕ maps the expert features to the general dimension..."

The paper's central 'expert knowledge' source is the authors' own prior retinal foundation model (reference [9] includes multiple co-authors of this paper). The claim that this encoder resolves fine-grained pathologies is taken from that self-citation rather than derived or independently validated here. This makes the Perception-Gap component somewhat load-bearing on a self-citation. However, the main empirical claim—state-of-the-art VQA on four benchmarks—is evaluated against external/public datasets and ablations compare with and without the expert stream, so the circularity is limited.

full rationale

The derivation chain is architectural: the paper identifies Perception and Reasoning gaps, then introduces a dual-stream encoder, gated fusion, and deep injection (Eqs. 1-3). No equation is defined in terms of the target result, and no fitted parameter is renamed as a prediction. The only circularity-adjacent element is the self-cited expert encoder [9] used as the source of domain knowledge; its authors overlap with this paper, and its pathology sensitivity is assumed from that citation. This is a real but minor load-bearing self-citation, not an equation-level reduction. The final VQA claims are benchmarked against external public datasets (JSIEC, Retina, ODIR) and a private TM4K set, with ablations isolating component contributions. The potential TM4K training/evaluation overlap is a data-hygiene and verifiability risk, not a circularity reduction under the rubric. Overall, the paper's central performance claim has independent empirical content beyond the self-citation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The paper's central claim rests on several unmeasured domain assumptions: the existence of the two gaps, the reliability of the self-cited fundus encoder, and the safety of deep residual injection. These are not derived and not independently probed. The only broadly self-contained part is the model evaluation, which is itself compromised by private data and missing statistics.

free parameters (4)

LoRA rank = 16
Chosen by hand; governs the capacity of LLM adaptation and is load-bearing for the parameter-efficient fine-tuning claim, but no sweep or sensitivity analysis is reported.
LoRA scaling factor α = 32
Chosen by hand; interacts with rank and learning rate; no sensitivity analysis is provided.
Peak learning rate = 2e-5
A standard fine-tuning hyperparameter; no evidence that results are robust to this choice.
Maximum sequence length = 2048 tokens
Caps generated report length; no analysis of its effect on text-similarity metrics.

assumptions (4)

domain assumption The 'Perception Gap' and 'Reasoning Gap' are real structural deficiencies causing medical hallucination.
Introduced in §1 and §2 as 'structural deficiencies'; no direct measurement of visual signal decay or prior domination is provided, and the architecture is designed entirely around them.
domain assumption The contrastively pre-trained fundus foundation encoder [9] provides reliable complementary pathological semantics.
§2.1 relies on this encoder to supply fine-grained pathological features; no analysis of its failure modes is included, and it is the authors' own prior model.
domain assumption Injecting F_fused into visual token positions with zero-initialized tanh layer scaling preserves pretrained LLM behavior while adding visual grounding.
§2.3, Eq. (3) assumes this; it is only evaluated indirectly via final macro metrics, not via tests of catastrophic forgetting or syntactic disruption.
domain assumption Text-similarity metrics (BLEU-1, ROUGE-L, METEOR, BERT-F1) approximate clinical fidelity.
Table 2 uses these as evidence of 'clinical fidelity', but no clinician study, expert adjudication, or hallucination-specific metric is reported.

invented entities (1)

Vision Anchors
purpose: Persistent residual visual biases injected into intermediate LLM layers that supposedly 'strictly ground' reasoning in visual evidence.
Introduced in §2.3 and Figure 1; evidence is only internal ablated metrics and two qualitative cases. No external probe of grounding exists, so there is no falsifiable handle outside this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge." pith.science (2026). https://pith.science/paper/ZSFMHEXN

@misc{pith2026260307131,
  author       = {Pith},
  title        = {Pith review of: Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZSFMHEXN}},
  note         = {Machine review of arXiv:2603.07131}
}

read the original abstract

Large Vision Language Models (LVLMs) show immense potential for automated ophthalmic diagnosis. However, their clinical deployment is severely hindered by lacking domain-specific knowledge. In this work, we identify two structural deficiencies hindering reliable medical reasoning: 1) the Perception Gap, where general-purpose visual encoders fail to resolve fine-grained pathological cues (e.g., microaneurysms); and 2) the Reasoning Gap, where sparse visual evidence is progressively overridden by massive language priors in deeper transformer layers, leading to ungrounded hallucinations. To bridge these gaps, we propose EyExIn, a data-efficient framework designed to anchor retinal VLMs with expert knowledge via a Deep Expert Injection mechanism. Our architecture employs an Expert-Aware Dual-Stream encoding strategy that decouples visual representation into a general stream for anatomical context and a specialized expert stream for pathological semantics. To ensure high-fidelity integration, we design a Semantic-Adaptive Gated Fusion module, which dynamically amplifies subtle lesion signals while filtering irrelevant background noise. Furthermore, we introduce Adaptive Deep Expert Injection to embed persistent "Vision Anchors" by integrating fused visual features as residual biases directly into intermediate LLM layers. This mechanism creates a visual shortcut that forces the reasoning stack to remain strictly grounded in visual evidence. Extensive experiments across four benchmarks demonstrate that our model consistently outperforms massive proprietary systems. EyExIn significantly enhances domain-specific knowledge embedding and achieves state-of-the-art precision in ophthalmic visual question answering, advancing the development of trustworthy ophthalmic AI.

Figures

Figures reproduced from arXiv: 2603.07131 by the authors.

**Figure 1.** Architectural comparison between a standard VLM and the proposed EyExIn framework. (Top) Standard VLMs suffer from visual signal decay across deep LLM layers, leading to ungrounded reasoning. (Bottom) EyExIn introduces an expertaware dual-stream encoder and Semantic-Adaptive Gated Fusion (GF). Crucially, Deep Expert Injection establishes persistent “Vision Anchors” in intermediate layers, ensuring evidence-grounded… view at source ↗

**Figure 2.** Qualitative VQA comparison on two real-world clinical cases. Red text indicates severe missed diagnoses and hallucinated artifacts by Gemini, while green text highlights accurate, expert-aligned lesion identification and clinical metrics by EyExIn. Real-World Clinical Validation [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 7 linked inside Pith

[1]

Retina_dataset (2016), https://github.com/yiweichen04/retina_dataset

2016
[2]

(2021), https://www.kaggle.com/datasets/andrewmvd/ocular-disease-recognition- odir5k/data

International competition on ocular disease intelligent recognition. (2021), https://www.kaggle.com/datasets/andrewmvd/ocular-disease-recognition- odir5k/data

2021
[3]

arXiv preprint arXiv:2511.21631 (2025)

Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025)

arXiv 2025
[4]

arXiv preprint arXiv:2502.13923 (2025)

Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., et al.: Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

arXiv 2025
[5]

Nature communications12(1), 4828 (2021)

Cen, L.P., Ji, J., Lin, J.W., Ju, S.T., Lin, H.J., Li, T.P., Wang, Y., Yang, J.F., Liu, Y.F., Tan, S., et al.: Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nature communications12(1), 4828 (2021)

2021
[6]

In: Proceedings of the 2024 conference on empirical methods in natural language processing

Chen, J., Gui, C., Ouyang, R., Gao, A., Chen, S., Chen, G.H., Wang, X., Cai, Z., Ji, K., Wan, X., et al.: Towards injecting medical visual knowledge into multimodal llms at scale. In: Proceedings of the 2024 conference on empirical methods in natural language processing. pp. 7346–7370 (2024)

2024
[7]

Advances in neural information processing systems36, 49250–49267 (2023)

Dai, W., Li, J., Li, D., Tiong, A., Zhao, J., Wang, W., Li, B., Fung, P.N., Hoi, S.: Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems36, 49250–49267 (2023)

2023
[8]

arXiv preprint arXiv:2501.12948 (2025)

Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

arXiv 2025

Show all 21 references

[9]

arXiv preprint arXiv:2512.14499 (2025)

Guo, J., Du, J., Yang, S., Lu, S., Cheng, W., Zhang, K., Sun, Y., Yang, C., Zhang, W., Chen, F., et al.: Native intelligence emerges from large-scale clinical practice: A retinal foundation model with deployment efficiency. arXiv preprint arXiv:2512.14499 (2025)

2025
[10]

In: Advances in Neural Information Processing Systems

Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: LLaVA-Med: Training a large language-and-vision assistant for biomedicine in one day. In: Advances in Neural Information Processing Systems. vol. 36, pp. 28541–28564 (2023)

2023
[11]

Li, J., Guan, Z., Wang, J., Cheung, C.Y., Zheng, Y., Lim, L.L., Lim, C.C., Ruamvi- boonsuk, P., Raman, R., Corsino, L., et al.: Integrated image-based deep learning andlanguagemodelsforprimarydiabetescare.Naturemedicine30(10),2886–2896 (2024)

2024
[12]

In: Proceedings of the 33rd ACM International Conference on Multimedia

Li, S., Lin, T., Lin, L., Zhang, W., Liu, J., Yang, X., Li, J., He, Y., Song, X., Xiao, J., et al.: Eyecaregpt: Boosting comprehensive ophthalmology understanding with tailored dataset, benchmark and model. In: Proceedings of the 33rd ACM International Conference on Multimedia...

2025
[13]

arXiv preprint arXiv:2502.09838 (2025)

Lin, T., Zhang, W., Li, S., Yuan, Y., Yu, B., Li, H., He, W., Jiang, H., Li, M., Song, X., et al.: Healthgpt: A medical large vision-language model for unifying compre- hension and generation via heterogeneous knowledge adaptation. arXiv preprint arXiv:2502.09838 (2025)

2025 arXiv
[14]

In: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision

Liu,B.,Zou,K.,Zhan,L.M.,Lu,Z.,Dong,X.,Chen,Y.,Xie,C.,Cao,J.,Wu,X.M., Fu, H.: Gemex: A large-scale, groundable, and explainable medical vqa benchmark for chest x-ray diagnosis. In: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision. pp. 21310–21320 (2025)

2025
[15]

arXiv preprint arXiv:2402.00253 (2024)

Liu, H., Xue, W., Chen, Y., Chen, D., Zhao, X., Wang, K., Hou, L., Li, R., Peng, W.: A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253 (2024)

2024 arXiv
[16]

Advances in neural information processing systems36, 34892–34916 (2023)

Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems36, 34892–34916 (2023)

2023
[17]

Advances in neural information processing sys- tems35, 27730–27744 (2022)

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. Advances in neural information processing sys- tems35, 27730–27744 (2022)

2022
[18]

arXiv preprint arXiv:2409.12191 (2024)

Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)

2024 arXiv
[19]

arXiv preprint arXiv:2503.158921(2025)

Yu, H., Yi, S., Niu, K., Zhuo, M., Umit, B.L.: Unifying medical imaging tasks via vision-language models. arXiv preprint arXiv:2503.158921(2025)

2025 arXiv
[20]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.T., Sun, M., et al.: Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

2024
[21]

Nature Communications16(1), 3108 (2025)

Zambrano Chaves, J.M., Huang, S.C., Xu, Y., Xu, H., Usuyama, N., Zhang, S., Wang, F., Xie, Y., Khademi, M., Yang, Z., et al.: A clinically accessible small multimodal radiology model and evaluation metric for chest x-ray findings. Nature Communications16(1), 3108 (2025)

2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Retina_dataset (2016), https://github.com/yiweichen04/retina_dataset

2016

[2] [2]

(2021), https://www.kaggle.com/datasets/andrewmvd/ocular-disease-recognition- odir5k/data

International competition on ocular disease intelligent recognition. (2021), https://www.kaggle.com/datasets/andrewmvd/ocular-disease-recognition- odir5k/data

2021

[3] [3]

arXiv preprint arXiv:2511.21631 (2025)

Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025)

arXiv 2025

[4] [4]

arXiv preprint arXiv:2502.13923 (2025)

Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., et al.: Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

arXiv 2025

[5] [5]

Nature communications12(1), 4828 (2021)

Cen, L.P., Ji, J., Lin, J.W., Ju, S.T., Lin, H.J., Li, T.P., Wang, Y., Yang, J.F., Liu, Y.F., Tan, S., et al.: Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nature communications12(1), 4828 (2021)

2021

[6] [6]

In: Proceedings of the 2024 conference on empirical methods in natural language processing

Chen, J., Gui, C., Ouyang, R., Gao, A., Chen, S., Chen, G.H., Wang, X., Cai, Z., Ji, K., Wan, X., et al.: Towards injecting medical visual knowledge into multimodal llms at scale. In: Proceedings of the 2024 conference on empirical methods in natural language processing. pp. 7346–7370 (2024)

2024

[7] [7]

Advances in neural information processing systems36, 49250–49267 (2023)

Dai, W., Li, J., Li, D., Tiong, A., Zhao, J., Wang, W., Li, B., Fung, P.N., Hoi, S.: Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems36, 49250–49267 (2023)

2023

[8] [8]

arXiv preprint arXiv:2501.12948 (2025)

Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

arXiv 2025