Pith. sign in

REVIEW 5 major objections 7 minor 86 references

Frozen specialist models can be coordinated at inference by an offline routing policy and light test-time adaptation, improving biomedical image analysis without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 00:58 UTC pith:4DWRQTWH

load-bearing objection Solid multi-task ensemble recipe with real same-pool gains, but the shift-deployment story is mostly untested and the offline-RL framing is oversold. the 5 major comments →

arxiv 2607.25108 v1 pith:4DWRQTWH submitted 2026-07-27 cs.CV cs.AIcs.LGeess.IV

OPERA: Offline Policy-guided Expert Routing and Adaptation for Universal Biomedical Image Analysis

Zihan Li , Feiyang Liu , Dandan Shan , Ruibo Wang , Qingqi Hong This is my paper
classification cs.CV cs.AIcs.LGeess.IV
keywords biomedical image analysismulti-agent ensembleoffline policy learningtest-time adaptationexpert routingconfidence calibrationdistribution shiftmedical AI deployment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Clinical AI models often fail when scanners, protocols, or patient populations change, and fixing them usually means expensive fine-tuning that hospitals cannot always afford or share data for. This paper argues there is a third path between brittle specialists and heavy foundation models: keep several pretrained experts frozen, learn once from a small labeled validation set which experts to trust for which classes, then adapt their fusion at test time using only unlabeled batches. The resulting system, OPERA, profiles experts by validation AUC, calibrates each model’s confidence with temperature scaling, gently updates class weights from test-batch statistics, and routes hard samples by inter-model agreement and entropy. Across nine datasets spanning fundus photos, chest X-rays, CT, MRI, and multimodal diagnosis, the coordinated ensemble beats strong single models and other ensembles on classification, segmentation, and multiple-choice diagnosis while improving calibration—without any gradient updates to the experts at deployment. A sympathetic reader cares because the method turns existing, already-trained models into a deployable universal analyzer under the privacy and label constraints that actually govern clinics.

Core claim

Treating ensemble weight assignment as offline policy learning—profiling frozen experts on a small validation set, then combining per-expert temperature calibration, distribution-aware class-weight adaptation, and instance-level agreement/entropy routing—yields consistent gains in accuracy and calibration across diverse biomedical imaging tasks without any retraining of the experts at deployment.

What carries the argument

OPERA’s hierarchical inference-time fusion: Expert Profiling Module (EPM) learns model- and class-level weights from validation AUC via temperature-scaled softmax; Per-Expert Confidence Calibration (PECC), Distribution-Aware Adaptation (DAA), and Instance-level Expert Routing (IER) then adjust those weights on unlabeled test data using sharpness, batch confidence, agreement, and entropy.

Load-bearing premise

A small labeled validation set that is representative enough of the target domain must exist so that the offline AUC-derived expert weights transfer, and the hand-chosen routing thresholds and blend factors must generalize beyond the reported datasets.

What would settle it

On a new scanner or protocol where the same three frozen experts are available but the validation cohort is either absent or badly mismatched, measure whether OPERA (and its uniform-weight OPERA* variant) still beats strong same-pool baselines on held-out test AUC/Dice; a clear loss of the reported margins would falsify the zero-retraining deployment claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Clinics can compose already-deployed specialist models into a stronger universal analyzer without joint retraining or data sharing.
  • When even a small labeled validation set is unavailable, the fully unsupervised OPERA* path still recovers most of the gain via test-time modules alone.
  • The same offline-policy-plus-test-time-routing pattern applies unchanged to classification, pixel-wise segmentation, and multimodal multiple-choice diagnosis.
  • Calibration quality improves alongside accuracy because each expert’s probabilities are temperature-adjusted before fusion.
  • Cost-aware or cascaded variants become natural next steps: invoke the full ensemble only when agreement is low.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method effectively turns diversity of inductive bias (transformer vs residual vs dense nets, or weaker MLLMs) into a free robustness resource once a lightweight routing policy exists.
  • If the hand-tuned thresholds prove brittle across sites, replacing them with a small learned gating head that still freezes the experts would be a direct, testable extension.
  • Privacy-preserving multi-hospital deployment is a natural fit: each site keeps its own experts and only shares the tiny offline weight tensors.
  • The same coordination logic could be tried on non-imaging clinical time series or lab panels where multiple legacy models already exist.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes OPERA, an inference-time ensemble framework that fuses three frozen expert models (e.g., ViT/ResNet50/DenseNet121, or TransUNet/U-Net/V-Net) via four mechanisms: EPM (validation-AUC-derived model- and class-level softmax weights, Eqs. 1–2), PECC (sharpness-thresholded per-expert temperature scaling, Eqs. 3–5), DAA (unlabeled test-batch confidence reweighting of class weights, Eqs. 6–8), and IER (instance-level routing by inter-model agreement and entropy, Eqs. 9–13). The framing is "offline policy learning" for deployment under distribution shift without retraining. Experiments cover 9 datasets (fundus, CXR, CT, MRI classification/segmentation, plus a multimodal MLLM multiple-choice benchmark) against 30+ baselines, with component ablations (Tables 7–10), an unsupervised variant (OPERA*), and a same-expert-pool control (Table 13) showing +5.17 AUC / +1.09 Dice over the strongest same-pool baseline. All evaluations are on i.i.d. test splits of the datasets used to derive the routing weights.

Significance. If the in-distribution results hold, this is a useful, practical contribution: cheap, gradient-free fusion of off-the-shelf experts with consistent gains across three task families and nine datasets. Strengths worth naming: the same-expert-pool comparison (Table 13) properly localizes gains to the routing/adaptation rather than to "using more models"; OPERA* provides a genuine label-free ablation; per-component and expert-composition ablations are thorough; Appendix A.5 is unusually honest that inverse-entropy weighting is a heuristic rather than optimal; and code is released. However, the paper's stated purpose — robustness under scanner/protocol/population shift and deployability without retraining — is never evaluated: every result is an i.i.d. split. As written, the evidence supports "a strong in-distribution ensemble router," not the deployment-under-shift claim that motivates the work. Substantiating that claim would materially raise the paper's significance.

major comments (5)
  1. [§1, §4, Tables 1–6; Appendix F] The paper is motivated throughout by 'severe distribution shifts across scanners, protocols, and patient populations' and claims to be 'a practical path to deployable biomedical AI without retraining,' yet no experiment evaluates distribution shift. In all nine benchmarks, experts are fine-tuned on the dataset's train split, EPM/PECC/IER hyperparameters are derived from its validation split, and evaluation is on its i.i.d. test split. Appendix F asserts that source-domain EPM weights 'transfer effectively to related target domains' with no supporting experiment. This is the central claim and it is untested. A feasible fix within scope: at least one cross-domain transfer experiment, e.g., EPM weights computed on RFMiD validation deployed on OIA-DDR (or vice versa), or CXR14-derived weights deployed on an external CXR dataset, compared against re-derived in-domain weights and OPERA*.
  2. [§3.3, Eqs. (6)–(8); Prop. A.9] DAA reweights experts toward those with higher test-batch confidence |p−0.5| using only unlabeled data, and is explicitly marketed as the shift-handling mechanism ('This online policy correction under shift...'). Prop. A.9 concedes this works only 'if the confidence scores are monotonically related to model accuracy' — precisely the condition that severe covariate shift violates, since models are frequently confidently wrong under shift. In that regime DAA would amplify the worst expert, and no experiment rules this out. Please add a stress test: e.g., evaluate on a corrupted/shifted variant of one benchmark, or include one deliberately degraded expert in the pool, and show DAA does not fall below the W_orig-only (β=0) baseline. Table 10 shows aggressive blending (α=β=0.2) already degrades in-distribution, which heightens this concern.
  3. [§3.1 vs. Appendix C.3, Table 12; Fig. 2; §G] The description of EPM is internally inconsistent. The main text, Fig. 2 ('One-Time Analytical Calibration | No Gradient Optimization'), and §G ('computes AUC scores analytically without iterative optimization') describe EPM as a closed-form softmax over validation AUCs (Eqs. 1–2). Appendix C.3 instead describes gradient-based tuning of the combination weights with Adam (lr 0.01, 20 epochs, early stopping), and Table 12 lists an 'initial confidence temperature 2.0' distinct from τ=10. Which procedure produced the reported numbers? This matters for reproducibility and for the 'analytical, one-time, five-minute' characterization. Relatedly, the 'offline policy learning' framing is not substantiated: there is no policy, reward signal, or environment in the RL sense — the 'policy' is an AUC softmax. Either tone down the RL nomenclature or make the correspondence precise.
  4. [§3.2–3.4; Appendix A.2; Table 10] The method has many hand-chosen thresholds: IER gates (agreement 0.90/0.60, variance 0.03/0.12), PECC sharpness cutoffs (0.4/0.1), τ=10, γ=2.0, μ=3.0, ζ=0.7. Appendix A.2 states these were 'selected based on empirical validation across our benchmark datasets' — i.e., tuned on the same benchmark family used for evaluation, a mild form of circularity. Only the DAA blend factors (α, β) are ablated, and only on two segmentation datasets (Table 10). Please add sensitivity analyses for the IER gates and τ (and ideally PECC cutoffs) showing the gains in Tables 7–8 are not threshold-specific, ideally on a dataset not used for threshold selection.
  5. [§4, Tables 1–6; Table 11] No variance or significance is reported anywhere, and several headline margins are small: Chest X-Ray14 +0.63 AUC over Ark+ (83.05 vs 82.42), OrganSMNIST +0.42 AUC over ELF* (98.83 vs 98.41), and Table 6's multimodal average +1.96 over InternVL2 on only 845 images with classes as small as n=7 (CSR: 71.43% = 5/7), n=9 (retinitis), n=12 (tessellation). Single-run numbers at these margins cannot support 'consistently outperforms.' Please report std over seeds or bootstrap CIs at least where margins are under ~2 points, and per-class counts or CIs for Table 6.
minor comments (7)
  1. [§2.2] This section appears to describe a different method: 'EPM... encoding expert radiotherapy trajectories' (no radiotherapy appears in this paper), DAA expanded as 'Dynamic Anchor Adaptation' and IER as 'Intelligent Experience Replay' (vs. 'Distribution-Aware Adaptation' and 'Instance-level Expert Routing' in §3), and 'routing incoming deployment trajectories to modality-specific replay buffers,' a mechanism that does not exist in the paper. Please rewrite to describe the actual method.
  2. [§2.1 vs. §A.7/Appendix E] §2.1 claims OPERA 'eliminat[es] the substantial computational overhead with ensemble methods,' but OPERA is itself an ensemble requiring M forward passes, and Appendix E states its cost 'equals a standard ensemble' in memory and latency. The claim should be about eliminating retraining cost, not inference cost.
  3. [Definition A.2] 'where M=2 denotes the number of models in each pair (distinct from the M=3 ensemble expert agents)' — reusing M for a different quantity in the same formula (Eq. 31) is confusing; use a separate symbol.
  4. [§1, contributions] 'The first agentic vision ensemble framework with offline policy learning' is a strong priority claim; given that the 'policy learning' is an AUC softmax, this should be softened or better justified against prior validation-weighted/stacked ensemble work.
  5. [§4, Tables 1–5] Please clarify which baseline numbers are re-run under the paper's protocol versus quoted from prior papers, particularly for the fundus and segmentation tables where baselines cluster tightly.
  6. [Appendix C.2] RandAugment is 'optionally' employed for validation-based weight tuning — state precisely when it is used, since it affects the EPM weights.
  7. [Abstract/§1] Minor language issues: 'To this end, We propose' (capitalization), 'expert agents coordination' (abstract), and garbled text rendering in Figs. 1–2 of the preprint.

Circularity Check

0 steps flagged

No derivation circularity: OPERA is an empirical ensemble-routing system whose weights are fit on validation and scored on held-out tests, not forced by definitional identity.

full rationale

The paper’s load-bearing claims are empirical performance gains from a fixed pipeline (EPM validation-AUC weights, PECC sharpness temperatures, DAA confidence blending, IER agreement/entropy gates) applied to frozen experts and evaluated on held-out test splits and same-expert-pool baselines (Tables 1–10, 13). Fitting combination weights on a validation cohort and reporting test AUC/Dice is standard supervised evaluation; test metrics are not algebraically determined by the validation AUCs or the hand-chosen thresholds (τ=10, α=β=0.1, agreement 0.90/0.60, etc.). The appendix theory (bias–variance, ambiguity decomposition, temperature scaling, Banach contraction for DAA) is post-hoc justification of design choices using classical external results, not a chain that defines the reported numbers into existence. Framing ensemble weighting as “offline policy learning” is methodological vocabulary, not a renaming that substitutes for evidence. Concerns about i.i.d. splits versus real scanner shift, or about DAA under miscalibrated confidence (Prop. A.9’s monotonicity caveat), are external-validity / correctness issues, not circular derivation. No self-definitional loop, fitted-input-as-prediction identity, or load-bearing self-cited uniqueness theorem is present.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 2 invented entities

The central claim rests on standard ensemble/TTA mathematics plus many hand-set routing constants and the operational premise that a small labeled validation set (or uniform init) plus unlabeled test batches suffice to route frozen heterogeneous experts under biomedical distribution shift. No new physical entities; invented ‘modules’ are algorithmic components. Load-bearing free parameters are the numerous thresholds and blend coefficients chosen empirically.

free parameters (7)
  • EPM softmax temperature τ = 10.0
    Controls concentration of model- and class-level weights from validation AUC; set to 10.0 by authors.
  • DAA blend factors α, β = α=0.1, β=0.1
    Control how much class weights move from validation prior toward test confidence EMA; chosen and ablated around 0.1.
  • PECC sharpness thresholds and temperatures = T=1.5 if s>0.4; T=0.7 if s<0.1; else 1.0
    Maps batch sharpness s_m to discrete temperatures T_m ∈ {1.5, 1.0, 0.7} at cutoffs 0.4 and 0.1.
  • IER agreement/variance gates = a_i>0.90 & var<0.03; a_i<0.60 or var>0.12
    Decide among base weights, inverse-entropy weights, and extremity mixture; selected on benchmarks.
  • IER entropy/extremity coefficients γ, μ, ζ = γ=2.0, μ=3.0, ζ=0.7
    Shape instance-level reweighting strength in disagreement and intermediate regimes.
  • Number of experts M and expert identities = M=3 task-specific pools
    Ensemble size fixed to 3 per task family (e.g., CLIP-ViT/ResNet/DenseNet or TransUNet/U-Net/V-Net); composition is a design choice that strongly affects results (Table 9).
  • DAA update interval (every 100 batches) and EMA accumulation = update every 100 batches
    Controls how often class attention is refreshed from running confidence statistics.
axioms (6)
  • standard math Weighted ensembles of diverse predictors reduce error when errors are imperfectly correlated (bias–variance / ambiguity decomposition).
    Invoked in Appendix A.1–A.2 to justify multi-architecture experts.
  • domain assumption Temperature scaling preserves rank order while adjusting confidence calibration.
    PECC §3.2 and Appendix A.3 treat temperature as sufficient calibration for fusion.
  • domain assumption Validation AUC (or uniform weights) is a sufficient offline signal for class- and model-level routing under subsequent target shift.
    EPM §3.1; weakened but still used as the preferred path versus OPERA*.
  • domain assumption Unlabeled test-batch confidence and inter-model agreement/entropy are monotonically informative about which expert to trust under covariate shift.
    DAA §3.3, IER §3.4, Proposition A.9; authors note entropy weighting is heuristic.
  • ad hoc to paper Hand-chosen discrete routing thresholds transfer across the nine evaluation datasets.
    Thresholds stated as empirically selected (Appendix A.2 practical interpretation).
  • ad hoc to paper Framing analytical AUC softmax and test-time reweighting as offline RL policy learning without environment interaction.
    Introduction/Methods equate forward-pass weight computation with offline policy optimization; no Bellman backup or constrained offline RL algorithm is solved.
invented entities (2)
  • OPERA coordinator (EPM + PECC + DAA + IER hierarchy) no independent evidence
    purpose: Name the full inference-time routing/adaptation stack that produces final fused probabilities without expert gradient updates.
    Algorithmic system composition; not a physical entity. Independent evidence is the empirical tables, not an external measurable object.
  • Expert Profiling Module routing policy tensors (w_model, W_class, T_m) no independent evidence
    purpose: Store offline-computed ensemble policy deployed at test time.
    Standard weight tensors rebranded as a ‘routing policy’; falsifiable only via reported benchmarks.

pith-pipeline@v1.2.0-grok45-kimik3 · 32676 in / 4207 out tokens · 76114 ms · 2026-07-31T00:58:32.677434+00:00 · methodology

0 comments
read the original abstract

Biomedical image analysis spans diverse modalities and tasks, yet real-world deployment is hindered by severe distribution shifts across scanners, protocols, and patient populations. High-performing models consequently require repeated domain-specific fine-tuning, which is a costly cycle that becomes impractical when labels are scarce or privacy constraints limit data sharing. We propose OPERA (Offline Policy-guided Expert Routing and Adaptation), a multi-agent ensemble framework that addresses this deployment bottleneck by treating expert weight assignment as an offline policy learning problem: a routing policy is learned from a small validation set without gradient updates to any expert agent, then deployed with test-time adaptation to handle distribution shift. OPERA coordinates heterogeneous specialist agents through complementary mechanisms. The expert profiling module learns selection policies offline, enabling informed allocation of expertise. Each agent undergoes confidence calibration through temperature adjustment, ensuring more reliable probabilistic outputs. OPERA also incorporates distribution aware adaptation, where class weights are dynamically adjusted at the batch level using statistics derived from unlabeled test data. Instance level routing assigns each sample to the most suitable expert by leveraging inter model agreement and predictive entropy. We evaluate OPERA on 9 datasets covering fundus photography, chest X-ray, CT, MRI, and multimodal diagnostic benchmarks, comparing against 30+ baselines across classification, segmentation, and multimodal settings. OPERA consistently improves performance and calibration quality, demonstrating that offline policy-guided expert agents coordination is a practical path to deployable biomedical AI without retraining. Code is on \href{https://github.com/HUANGLIZI/OPERA}{GitHub}.

Figures

Figures reproduced from arXiv: 2607.25108 by Dandan Shan, Feiyang Liu, Qingqi Hong, Ruibo Wang, Zihan Li.

Figure 1
Figure 1. Figure 1: Motivation of OPERA. Top: a baseline model yields [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the OPERA framework. In the Offline Stage, the Expert Profiling Module (EPM) extracts initial model [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative results on COVID-19 datasets. Top: COVID-Xray (QaTa-COV19). Bottom: COVID-CT (MosMedData+). [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative segmentation results on the LA-MRI dataset with 20% labeled data. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative segmentation results on the Pancreas-CT dataset with 20% labeled data. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Representative case studies of multimodal LLM [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Additional qualitative segmentation results on the COVID-Xray (QaTa-COV19) dataset. From left to right: original [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Additional qualitative segmentation results on the COVID-CT (MosMedData+) dataset. From left to right: original [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative results under limited annotations (20% labeled data). Top: Left atrial MRI (LA-MRI). Bottom: Pancreas CT [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

86 extracted references · 11 linked inside Pith

  1. [1]

    Sukesh Adiga, Jose Dolz, and Herve Lombaert. 2024. Anatomically-aware un- certainty for semi-supervised image segmentation.Medical Image Analysis91 (2024), 103011

    2024

  2. [2]

    Tomohiro Ando. 2011. Predictive Bayesian model selection.American Journal of Mathematical and Management Sciences31, 1-2 (2011), 13–38

    2011

  3. [3]

    Jinze Bai et al. 2023. Qwen-VL: A Versatile Vision-Language Model for Under- standing, Localization, Text Reading, and Beyond.arXiv preprint arXiv:2308.12966 (2023)

    Pith/arXiv arXiv 2023

  4. [4]

    Hritam Basak and Zhaozheng Yin. 2023. Pseudo-label guided contrastive learning for semi-supervised medical image segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 19786–19797

    2023

  5. [5]

    Bing Cao, Yinan Xia, Yi Ding, Changqing Zhang, and Qinghua Hu. 2024. Predic- tive Dynamic Fusion.ICML(2024)

    2024

  6. [6]

    Jieneng Chen et al. 2024. TransUNet: Rethinking the U-Net architecture design for medical image segmentation through the lens of transformers.Medical Image Analysis97 (2024), 103280

    2024

  7. [7]

    Xiaokang Chen et al. 2025. Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling.arXiv preprint arXiv:2501.17811(2025)

    Pith/arXiv arXiv 2025

  8. [8]

    Zhe Chen et al. 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InCVPR. 24185–24198

    2024

  9. [9]

    Kenneth Clark et al. 2013. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository.Journal of digital imaging26 (2013), 1045–1057

    2013

  10. [10]

    Noel CF Codella, Q-B Nguyen, Sharath Pankanti, David A Gutman, Brian Helba, Allan C Halpern, and John R Smith. 2017. Deep learning ensembles for melanoma recognition in dermoscopy images.IBM Journal of Research and Development61, 4/5 (2017), 5–1

    2017

  11. [11]

    Wenliang Dai et al. 2023. Instructblip: Towards general-purpose vision-language models with instruction tuning.NIPS36 (2023), 49250–49267

    2023

  12. [12]

    Aysen Degerli, Serkan Kiranyaz, Muhammad EH Chowdhury, and Moncef Gab- bouj. 2022. Osegnet: Operational segmentation network for covid-19 detection using chest x-ray images. In2022 IEEE International Conference on Image Process- ing (ICIP). IEEE, 2306–2310

    2022

  13. [13]

    Alexey Dosovitskiy et al. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. InICLR

    2021

  14. [14]

    Jiawei Du, Jia Guo, Weihang Zhang, Shengzhu Yang, Hanruo Liu, Huiqi Li, and Ningli Wang. 2024. Ret-clip: A retinal image foundation model pre-trained with clinical diagnostic reports. InInternational conference on medical image computing and computer-assisted intervention. Springer, 709–719

    2024

  15. [15]

    Sasan Farhadi, Samuele Tatullo, and Francesco Ferrian. 2025. Comparative analy- sis of ensemble learning techniques for enhanced fatigue life prediction.Scientific Reports15, 1 (2025), 11136

    2025

  16. [16]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research23, 120 (2022), 1–39

    2022

  17. [17]

    M Eshaghi Gordji, M Ramezani, Manuel De La Sen, and Yeol Je Cho. 2017. On orthogonal sets and Banach fixed point theorem.Fixed point theory18, 2 (2017), 569–578

    2017

  18. [18]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

    2016

  19. [19]

    Johannes Hofmanninger, Forian Prayer, Jeanny Pan, Sebastian Röhrich, Helmut Prosch, and Georg Langs. 2020. Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem.European radiology experimental4, 1 (2020), 50

    2020

  20. [20]

    Neil Houlsby et al. 2019. Parameter-efficient transfer learning for NLP. InInter- national conference on machine learning. PMLR, 2790–2799

    2019

  21. [21]

    Edward J Hu et al. 2022. Lora: Low-rank adaptation of large language models. ICLR1, 2 (2022), 3

    2022

  22. [22]

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger

  23. [23]

    Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier- Hein. 2021. nnU-Net: a self-configuring method for deep learning-based biomed- ical image segmentation.Nature methods18, 2 (2021), 203–211

    2021

  24. [24]

    Cheng Ju, Aurélien Bibaut, and Mark van der Laan. 2018. The relative perfor- mance of ensemble methods with deep convolutional neural networks for image classification.Journal of applied statistics45, 15 (2018), 2800–2818

    2018

  25. [25]

    Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer, Salman Khan, and Fahad Shahbaz Khan. 2024. Unimed-clip: Towards a unified image-text pretraining paradigm for diverse medical imaging modalities.arXiv preprint arXiv:2412.10372(2024)

    Pith/arXiv arXiv 2024

  26. [26]

    Alexander Kirillov et al. 2023. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision. 4015–4026

    2023

  27. [27]

    Yilun Kong, Guozheng Ma, Qi Zhao, Haoyu Wang, Li Shen, Xueqian Wang, and Dacheng Tao. 2025. Mastering massive multi-task reinforcement learning via mixture-of-expert decision transformer.arXiv preprint arXiv:2505.24378(2025)

    Pith/arXiv arXiv 2025

  28. [28]

    Ludmila Kuncheva and Chris Whitaker. 2003. Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy.Machine Learning 51 (05 2003), 181–207. doi:10.1023/A:1022859003006

    doi:10.1023/a:1022859003006 2003

  29. [29]

    Seunghyun Lee, Younggyo Seo, Kimin Lee, Pieter Abbeel, and Jinwoo Shin. 2022. Offline-to-online reinforcement learning via balanced replay and pessimistic q-ensemble. InConference on Robot Learning. PMLR, 1702–1712

    2022

  30. [30]

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. 2020. Offline rein- forcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643(2020)

    Pith/arXiv arXiv 2020

  31. [31]

    Chunyuan Li et al. 2023. Llava-med: Training a large language-and-vision assis- tant for biomedicine in one day.NIPS36 (2023)

    2023

  32. [32]

    Tao Li, Yingqi Gao, Kai Wang, Song Guo, Hanruo Liu, and Hong Kang. 2019. Diag- nostic assessment of deep learning algorithms for diabetic retinopathy screening. Information Sciences501 (2019), 511–522

    2019

  33. [33]

    Yanwei Li et al. 2024. Mini-gemini: Mining the potential of multi-modality vision language models.arXiv preprint arXiv:2403.18814(2024)

    Pith/arXiv arXiv 2024

  34. [34]

    Zihan Li et al. 2023. Lvit: language meets vision transformer in medical image segmentation.IEEE transactions on medical imaging43, 1 (2023), 96–107

    2023

  35. [35]

    Zihan Li, Wentao Chen, Zhiqing Wei, Xingqi Luo, and Bing Su. 2022. Semi- wtc: A practical semi-supervised framework for attack categorization through weight-task consistency.arXiv preprint arXiv:2205.09669(2022)

    Pith/arXiv arXiv 2022

  36. [36]

    Kinahan, and Qingqi Hong

    Zihan Li, Dandan Shan, Yunxiang Li, Paul E. Kinahan, and Qingqi Hong. 2026. Scale-aware adaptive supervised network with limited medical annotations.Pat- tern Recognition172 (2026), 112743. doi:10.1016/j.patcog.2025.112743

    arXiv 2026

  37. [37]

    Zihan Li, Diping Song, Zefeng Yang, Deming Wang, Fei Li, Xiulan Zhang, Paul E Kinahan, and Yu Qiao. 2025. Visionunite: A vision-language foundation model for ophthalmology enhanced with clinical knowledge.IEEE Transactions on Pattern Analysis and Machine Intelligence(2025)

    2025

  38. [38]

    Zihan Li, Yiqing Wang, Sina Farsiu, and Paul Kinahan. 2025. Boosting Medical Visual Understanding From Multi-Granular Language Learning.arXiv preprint arXiv:2511.15943(2025)

    arXiv 2025

  39. [39]

    Zihan Li, Yiqing Wang, Sina Farsiu, and Paul Kinahan. 2026. Large-scale data harmonization of radiology studies via multigranular vision-language alignment. InMedical Imaging 2026: Imaging Informatics, Vol. 13930. SPIE, 171–179

    2026

  40. [40]

    Jian Liang, Ran He, and Tieniu Tan. 2025. A comprehensive survey on test-time adaptation under distribution shifts.International Journal of Computer Vision 133, 1 (2025), 31–64

    2025

  41. [41]

    Haotian Liu et al. 2023. Visual instruction tuning.NIPS36 (2023)

    2023

  42. [42]

    Jeremiah Liu, John Paisley, Marianthi-Anna Kioumourtzoglou, and Brent Coull

  43. [43]

    Xiangde Luo et al. 2025. Ensemble learning of foundation models for precision oncology.arXiv preprint arXiv:2508.16085(2025)

    Pith/arXiv arXiv 2025

  44. [44]

    DongAo Ma, Jiaxuan Pang, Michael B Gotway, and Jianming Liang. 2025. A fully open AI foundation model applied to chest radiography.Nature(2025), 1–11

    2025

  45. [45]

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. 2024. Segment anything in medical images.Nature Communications15, 1 (2024), 654

    2024

  46. [46]

    Qinghe Ma, Jian Zhang, Lei Qi, Qian Yu, Yinghuan Shi, and Yang Gao. 2024. Con- structing and exploring intermediate domains in mixed domain semi-supervised medical image segmentation. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition. 11642–11651

    2024

  47. [47]

    Maciej A Mazurowski, Haoyu Dong, Hanxue Gu, Jichen Yang, Nicholas Konz, and Yixin Zhang. 2023. Segment anything model for medical image analysis: an experimental study.Medical Image Analysis89 (2023), 102918

    2023

  48. [48]

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV). Ieee, 565–571

    2016

  49. [49]

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. 2023. Foundation models for generalist medical artificial intelligence.Nature616, 7956 (2023), 259–265

    2023

  50. [50]

    Sergey P Morozov et al. 2020. Mosmeddata: Chest ct scans with covid-19 related findings dataset.arXiv preprint arXiv:2005.06465(2020)

    Pith/arXiv arXiv 2020

  51. [51]

    Chenggong Ni, Fan Lyu, Jiayao Tan, Fuyuan Hu, Rui Yao, and Tao Zhou. 2025. Maintaining consistent inter-class topology in continual test-time adaptation. In Proceedings of the Computer Vision and Pattern Recognition Conference. 15319– 15328

    2025

  52. [52]

    Julia MH Noothout et al. 2022. Knowledge distillation with ensembles of con- volutional neural networks for medical image segmentation.Journal of Medical Imaging9, 5 (2022), 052407–052407

    2022

  53. [53]

    Samiksha Pachade, Prasanna Porwal, Dhanshree Thulkar, et al . 2021. Retinal fundus multi-disease image dataset (RFMiD): a dataset for multi-disease detection research.Data6, 2 (2021), 14

    2021

  54. [54]

    Fernando Perez-Garcia et al. 2025. Exploring scalable medical image encoders beyond text supervision.Nature Machine Intelligence7, 1 (2025), 119–130

    2025

  55. [55]

    Rafael Figueiredo Prudencio, Marcos ROA Maximo, and Esther Luna Colombini

  56. [56]

    Wenbo Qi, Jiafei Wu, and SC Chan. 2024. Gradient-Aware for Class-Imbalanced Semi-supervised Medical Image Segmentation. InEuropean Conference on Com- puter Vision. Springer, 473–490

    2024

  57. [57]

    Jianing Qiu et al. 2024. Development and validation of a multimodal multitask vision foundation model for generalist ophthalmic artificial intelligence.NEJM AI1, 12 (2024), AIoa2300221

    2024

  58. [58]

    Alec Radford et al . 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learning. PmLR, 8748–8763

    2021

  59. [59]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. InInternational Conference on Medical image computing and computer-assisted intervention. Springer, 234–241

    2015

  60. [60]

    Dandan Shan, Zihan Li, Yunxiang Li, Qingde Li, Jie Tian, and Qingqi Hong. 2025. STPNet: Scale-aware Text Prompt Network for Medical Image Segmentation. IEEE Transactions on Image Processing(2025)

    2025

  61. [61]

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538(2017)

    Pith/arXiv arXiv 2017

  62. [62]

    Peilun Shi, Jianing Qiu, Sai Mu Dalike Abaxi, Hao Wei, Frank P-W Lo, and Wu Yuan. 2023. Generalist vision foundation models for medical imaging: A case study of segment anything model on zero-shot medical segmentation.Diagnostics 13, 11 (2023), 1947

    2023

  63. [63]

    Xiaojin Shi and Roberto Manduchi. 2003. A study on Bayes feature fusion for im- age classification. In2003 Conference on Computer Vision and Pattern Recognition Workshop, Vol. 8. IEEE, 95–95

    2003

  64. [64]

    Julio Silva-Rodriguez, Hadi Chakor, Riadh Kobbi, Jose Dolz, and Ismail Ben Ayed

  65. [65]

    Fuying Wang, Yuyin Zhou, Shujun Wang, Varut Vardhanabhuti, and Lequan Yu

  66. [66]

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. 2024. Mixture-of-agents enhances large language model capabilities.arXiv preprint arXiv:2406.04692(2024)

    Pith/arXiv arXiv 2024

  67. [67]

    Xiaosong Wang, Yifan Peng, Le Lu, et al . 2017. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. InProceedings of the IEEE conference on computer vision and pattern recognition. 2097–2106

    2017

  68. [68]

    Yuhan Wang, Zihan Li, Han Liu, Simon Arberet, Martin Kraus, Yuyin Zhou, Florin-Cristian Ghesu, Dorin Comaniciu, Ali Kamen, and Riqiang Gao. 2026. Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Plan- ning Study. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16520–16530

    2026

  69. [69]

    Linshan Wu, Jiaxin Zhuang, and Hao Chen. 2025. Large-scale 3d medical image pre-training with geometric context priors.IEEE Transactions on Pattern Analysis and Machine Intelligence(2025)

    2025

  70. [70]

    Ruiqi Wu, Chenran Zhang, Jianle Zhang, Yi Zhou, Tao Zhou, and Huazhu Fu

  71. [71]

    Yicheng Wu, Zongyuan Ge, Donghao Zhang, Minfeng Xu, Lei Zhang, Yong Xia, and Jianfei Cai. 2022. Mutual consistency learning for semi-supervised medical image segmentation.Medical Image Analysis81 (2022), 102530

    2022

  72. [72]

    Junfei Xiao, Yutong Bai, Alan Yuille, and Zongwei Zhou. 2023. Delving into masked autoencoders for multi-label thorax disease classification. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3588–3600

    2023

  73. [73]

    Yutong Xie, Jianpeng Zhang, Yong Xia, and Qi Wu. 2024. UniMiSS+: Universal medical self-supervised learning from cross-dimensional unpaired data.IEEE Transactions on Pattern Analysis and Machine Intelligence(2024)

    2024

  74. [74]

    Zhaohan Xiong et al. 2021. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Medical image analysis67 (2021), 101832

    2021

  75. [75]

    Jiancheng Yang et al. 2023. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification.Scientific Data10, 1 (2023), 41

    2023

  76. [76]

    Zefan Yang, Xuanang Xu, Jiajin Zhang, Ge Wang, Mannudeep K Kalra, and Pingkun Yan. 2025. Chest X-ray Foundation Model with Global and Local Repre- sentations Integration.IEEE Transactions on Medical Imaging(2025)

    2025

  77. [77]

    Hantao Yao, Rui Zhang, and Changsheng Xu. 2023. Visual-language prompt tun- ing with knowledge-guided context optimization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6757–6767

    2023

  78. [78]

    Jingfeng Yao et al . 2025. Eva-x: A foundation model for general chest x-ray analysis with self-supervised learning.npj Digital Medicine8, 1 (2025), 678

    2025

  79. [79]

    simply using more models

    Yukun Zhou et al. 2023. A foundation model for generalizable disease detection from retinal images.Nature622, 7981 (2023), 156–163. OPERA: Offline Policy-guided Expert Routing and Adaptation for Universal Biomedical Image Analysis MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil A Theoretical Analysis of OPERA In this section, we provide theoretical j...

    2023

  80. [86]

    zero- retraining

    and MM-Retinal [70]. These results suggest that fusing het- erogeneous expert representations can yield stronger and more transferable features than any individual foundation model, even within highly specialized domains. Beyond ophthalmology, OPERA generalizes effectively to chest and abdominal imaging tasks. On MM ’26, November 10–14, 2026, Rio de Janei...

    2026

Showing first 80 references.