{"id":"589bad8b-ae56-474a-9f91-e431ae29ab65","arxiv_id":"2608.06613","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A controlled benchmark of five 3D medical encoders shows that MRI artifacts distort representation geometry differently per model and artifact, with 3DINO most stable and BrainIAC most sensitive.","lead":"This paper tests how five pretrained 3D medical AI models respond to seven simulated MRI artifacts at five severity levels. It finds that robustness depends strongly on the model and the artifact, and that a model can lose geometric similarity without losing dimensionality.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Preprocessing confound: 3DINO's 112^3 resizing and other pipelines are applied after artifacts, so the robustness ranking may reflect preprocessing attenuation rather than learned representations.","rationale":"The paper's central claim is comparative: robustness is strongly model- and artifact-dependent, with 3DINO most stable, and the practical conclusion is that pretraining scale or domain specificity does not guarantee artifact invariance. The reader's weakest assumption identifies exactly the load-bearing soft spot: the evaluation uses each model's official preprocessing pipeline, and corruptions are applied before that preprocessing. The Appendix confirms large differences in P_m (resizing, cropping, z-scoring, quantization, skull-stripping), so the model ranking could be inherited from preprocessing rather than from the learned representations. For example, 3DINO's 112^3 resampling is a strong low-pass filter that would attenuate the very frequency-domain artifacts (spikes, ghosting, Gibbs) on which 3DINO appears most robust, making the headline result potentially artifactual. The paper is otherwise careful: equations for artifacts are explicit, matched clean/corrupted pairs are used, and the authors correctly note severity levels are not calibrated. A preprocessing-only control is a tractable and decisive check, and if the ranking survives it, the central claim is substantially strengthened. I therefore keep the reader's CONDITIONAL verdict: the work is valuable and reproducible, but the central attribution to pretraining is not yet secure.","tokens_in":16199,"tokens_out":3994,"duration_ms":36914,"concrete_test":"Run a preprocessing-only control on T1: for each artifact and severity level, compute linear CKA (or normalized cross-correlation) between the vectorized preprocessed volumes P_m(I_clean) and P_m(I_corrupted) for each model m, before any encoder is applied. If the ordering of these input-level CKA values across models matches the ordering of the representation-level CKA curves in Figs. 1-2 (e.g., 3DINO highest because 112^3 resampling suppresses high-frequency artifacts), then the robustness ranking is confounded by preprocessing. The paper would then need either a matched-preprocessing comparison (e.g., feeding all encoders volumes on a common grid with identical normalization, where the architectures permit) or a revised conclusion framed as pipeline-level robustness rather than representation-level invariance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central ranking (3DINO most robust, BrainIAC least) is computed as z = g_m(f_m(P_m(I))) with corruptions applied before P_m (Eq. 1; Sections 3.1-3.2). The model-specific preprocessors differ drastically: 3DINO resizes to 112^3 (A.1), BrainIAC to 96^3 and z-scores nonzero voxels (A.2), BrainFM skull-strips and resamples to 1-mm isotropic (A.3), NeuroVFM quantizes (A.4), and Neuro-SimCLR crops and z-scores (A.5). Because artifacts are injected before these operators, a 2x downsampling in 3DINO acts as a low-pass filter that attenuates k-space spikes, ghosting, and Gibbs ringing before the encoder sees them, potentially explaining 3DINO's high CKA on exactly those frequency-domain artifacts. Similarly, BrainIAC's 96^3 resize and z-scoring, NeuroVFM's quantization, and Neuro-SimCLR's cropping each alter the effective artifact amplitude differently. The paper states that identical preprocessing is applied to clean and corrupted volumes, but that only holds the artifact realization fixed within a model; it does not control for differences across models. Without isolating P_m, the claim that 'larger-scale or domain-specific pretraining alone does not guarantee artifact invariance' is not supported, because the observed model ordering could be driven by preprocessing rather than by the learned representations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical benchmark of five frozen 3D medical image encoders (3DINO, BrainIAC, BrainFM, NeuroVFM, Neuro-SimCLR) under seven simulated MRI artifacts at five severity levels, using 95 BraTS-Africa cases across four modalities. Robustness is measured by linear CKA between clean and corrupted embedding matrices, RankMe spectral effective rank, UMAP visualization, and the Dice consistency of an independent segmentation model (TumorSynth). The authors report that robustness varies strongly by model and artifact, with 3DINO as the most consistently stable encoder, BrainIAC as the most sensitive, and CKA declining while RankMe remains comparatively stable, which they interpret as geometric drift without dimensional collapse.","tokens_in":16462,"tokens_out":5138,"duration_ms":49436,"significance":"The benchmark addresses a real and under-explored gap: how pretrained 3D medical foundation models behave under clinically relevant MRI artifacts at the representation level rather than through a single downstream task. The design is largely non-circular: artifact parameters are fixed in Table 2, no constants are fitted to the reported metrics, and the model ranking is a falsifiable empirical claim. The inclusion of an independent segmentation-consistency reference and four MRI modalities strengthens the study. However, the headline conclusions about model ordering are, as analyzed below, threatened by a preprocessing confound and by the absence of uncertainty quantification. If these issues are addressed, the paper would be a useful reference for robustness evaluation and model selection in heterogeneous MRI settings.","major_comments":[{"comment":"The central model ranking may be driven by differences in model-specific preprocessing rather than by the learned representations. In Eq. (1) and Eq. (2), artifacts are applied to the common normalized volume I before the model-specific preprocessing operator P_m: z = g_m(f_m(P_m(I))). The preprocessing operators differ drastically across models: 3DINO resizes to 112^3 (A.1), BrainIAC resizes to 96^3 and z-scores non-zero voxels (A.2), BrainFM skull-strips and resamples to 1-mm isotropic (A.3), NeuroVFM applies its own quantization (A.4), and Neuro-SimCLR crops and z-scores (A.5). In particular, 3DINO's large downsampling acts as a low-pass filter that can attenuate k-space spikes, ghosting, and Gibbs ringing before the encoder sees them, which may explain its high CKA on exactly those frequency-domain artifacts. The statement in Section 3.1 that identical preprocessing is applied to clean and corrupted volumes only controls for within-model consistency; it does not isolate the contribution of P_m from that of f_m. The abstract and conclusion claim that 'larger-scale or domain-specific pretraining alone does not guarantee artifact invariance' is therefore not supported unless the preprocessing confound is addressed. I request a control experiment, for example applying a common preprocessing to all models, corrupting after preprocessing, or quantifying the effective artifact amplitude after each P_m, to show that the observed model ordering survives.","section":"Sections 3.1–3.2 and Appendix A"},{"comment":"The normalized singular values are not correctly normalized. In Eq. (15), p_j is defined as sigma_j / sum_k sigma_k + epsilon, so the p_j sum to 1 + r*epsilon rather than 1, and for zero singular values p_j = epsilon enters the entropy as epsilon log epsilon. The text calls these 'normalized singular values' and treats them as a probability distribution, which is inconsistent with the displayed formula. Since RankMe values and the conclusion that RankMe remains stable across corruptions are based on this formula, the definition should be corrected (for example, p_j = sigma_j / (sum_k sigma_k + r*epsilon), or using log(sigma_j + epsilon) inside the entropy), and the value of epsilon should be stated. Without this fix, the reported RankMe numbers are not reproducible from the paper as written.","section":"Equation (15)"},{"comment":"All quantitative claims are based on single point estimates computed from N = 95 volumes per modality, with no confidence intervals, bootstrap resampling, or significance tests. The differences between CKA curves that are described as 'strongly' model- and artifact-dependent may be within sampling variability for this sample size, and the ordering of models in the severity curves is presented without an uncertainty measure. The segmentation-consistency result in Figure 4 similarly reports mean Dice without error bars or paired tests. The authors should add, at minimum, bootstrap confidence intervals over cases (or equivalently a paired significance test for the main model-ranking comparisons), and ideally report variability over the random artifact parameters xi_a in Eq. (2), which currently appear to be fixed but are not described as such.","section":"Section 3.3 and Figures 1–2"}],"minor_comments":[{"comment":"For periodic k-space ghosting, the period R = 3 and axis q = x are fixed, and only alpha_gh is varied; since real ghosting depends on the phase-encoding direction and period, a sentence acknowledging this limitation would be useful, and an additional analysis varying R and q would strengthen the generality of the ghosting results.","section":"Section 3.2"},{"comment":"The UMAP description says 'UMAP is run with the default setting' without reporting n_neighbors, min_dist, metric, or random seed; for reproducibility, these parameters should be listed in the appendix.","section":"Section 3.3"},{"comment":"The sentence after Eq. (15), 'RankMe is defined as', is grammatically incomplete and should be rewritten to describe the formula properly.","section":"Section 3.3"},{"comment":"The caption states that Dice is 'averaged across labels 1–18 and cases'; please clarify whether the whole-tumor label 18 is included in this average or treated separately, and whether Dice is computed per label and then averaged or computed on the union of labels.","section":"Figure 4 caption"},{"comment":"There are formatting errors in the reference list, for example 'PMlR' in reference [12] and 'PmLR' in reference [17]; these should be corrected to 'PMLR'.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward empirical benchmark with a clear scope, but the preprocessing confound is the main correctness risk and should be handled prominently in the revision. I would also encourage the authors to add a data/code availability statement; the manuscript currently does not mention whether the artifact simulation code and evaluation scripts will be released, which would materially improve reproducibility. The benchmark nature of the work seems appropriate for the venue, and the topic is timely."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you're choosing a 3D medical encoder or working on robustness evaluation. It's the first controlled comparison of 3DINO, BrainIAC, BrainFM, NeuroVFM, and Neuro-SimCLR under seven MRI artifacts, using CKA, RankMe, and an independent segmentation-consistency check. The artifact equations are concrete and the benchmark is cleanly scoped: BraTS-Africa T1/T1ce/T2/FLAIR, five severity levels, 13,300 corrupted volumes. The headline pattern—3DINO quite stable, BrainIAC very sensitive, CKA dropping while RankMe stays flat—is visible in the curves and not oversold in the main text. The geometric-drift-without-collapse observation is genuinely useful; it shows the two metrics measure different failure modes. Running an independent segmentation model as a task-level reference is the right call.\n\nNow the soft spots, in proportion. All reported metrics are point estimates. CKA and RankMe are computed once per condition on the 95-case set, with no bootstrap or significance testing, so the model ordering could shift with sampling. The preprocessing confound is real and unresolved. Artifacts are applied before each model's own preprocessing (Eq. 1 and Appendix A), and those pipelines differ drastically: 3DINO resizes to 112^3, BrainIAC to 96^3, NeuroVFM quantizes, Neuro-SimCLR crops. Since resizing is a low-pass operation, 3DINO's apparent immunity to k-space spikes and ghosting may partly be preprocessing attenuating high frequencies before the encoder sees them. The paper admits architecture, preprocessing, objective, and scale all vary together, and the abstract's 'larger-scale pretraining alone does not guarantee invariance' is hedged enough to survive. Still, the title says 'foundation models' and the framing is about learned representations, so a reviewer should insist on a control: corrupt after preprocessing, or quantify how much each preprocessing operator attenuates an injected artifact. Third, severity levels are chosen by visual inspection rather than calibrated, which makes cross-artifact strength comparisons approximate; the paper discloses this, and within-artifact model comparisons remain valid. The self-citations for the artifact simulator are appropriate; the cited code exists.\n\nBottom line: a solid descriptive benchmark, not a causal study. Don't treat the model ranking as definitive until uncertainty is quantified and preprocessing is controlled. Worth serious refereeing; with those additions it becomes a citable resource.","headline":"A useful first controlled benchmark of 3D medical encoder robustness to MRI artifacts, with honest caveats but unresolved preprocessing confounds and no uncertainty quantification.","tokens_in":17016,"tokens_out":3613,"would_cite":true,"duration_ms":31443,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MRI artifacts disturb 3D medical encoders in model- and artifact-specific ways, usually by geometrically drifting the feature space rather than collapsing its dimensionality.","keywords":["MRI artifacts","representation robustness","3D medical foundation models","self-supervised learning","centered kernel alignment","RankMe","artifact simulation","BraTS-Africa"],"falsifier":"Run the same CKA and RankMe benchmark with a single shared preprocessing step for all five encoders; if the model ranking changes, then preprocessing, not the learned representations, produced the reported robustness ordering.","tokens_in":15966,"feed_emoji":"🧠","tokens_out":7004,"duration_ms":61125,"temperature":0.7,"pith_summary":"Pretrained 3D medical encoders are increasingly used as general-purpose feature extractors, and this paper asks whether their internal representations stay stable when MRI scans are corrupted by realistic artifacts. By perturbing BraTS-Africa volumes with seven frequency- and image-domain artifacts at five severity levels and comparing clean versus corrupted embeddings with linear CKA and RankMe, the authors find that robustness is strongly model- and artifact-dependent: 3DINO is the most consistently stable, BrainIAC is highly sensitive, and the other encoders sit in between with distinct profiles. They also find that CKA often drops substantially while RankMe stays flat, so artifacts reorganize representation geometry without collapsing the effective rank. A separate segmentation-consistency check shows that task-level degradation only partially matches representation-level sensitivity. The practical point is that pretraining scale and domain specificity are not enough to guarantee artifact invariance, so explicit robustness evaluation belongs in model selection for heterogeneous MRI settings.","feed_headline":"MRI artifacts warp 3D medical model features unevenly","feed_subtitle":"A five-encoder test shows MRI corruption effects depend on model and artifact, so pretraining scale is no shield.","key_machinery":"Controlled artifact simulation is the load-bearing mechanism: a clean volume $I$ is passed through an artifact operator $T_a(I;\\theta_{a,\\ell},\\xi_a)$ implemented either in the frequency domain (k-space spikes, periodic ghosting, Gibbs truncation) or in the image domain (Rician noise, Gaussian blur, bias field, gamma contrast), and only then through each encoder's official preprocessing. Robustness is read off two complementary statistics on matched clean and corrupted embedding matrices. Linear CKA, $\\mathrm{CKA}(X,Y) = \\| X^\\top Y \\|_F^2 / (\\|X^\\top X\\|_F \\|Y^\\top Y\\|_F)$ on centered embeddings, measures whether the pairwise geometry of the clean set survives the corruption; RankMe, computed from normalized singular values as $\\exp(-\\sum_j p_j \\log p_j)$ with $p_j = \\sigma_j/\\sum_k \\sigma_k + \\varepsilon$, measures whether the effective number of occupied directions survives. Interpreting the two together is what separates geometric drift from dimensional collapse.","core_discovery":"On the paper's own terms, the central discovery is that artifact robustness in 3D medical encoders is neither guaranteed by scale nor by domain specialization: across 95 BraTS-Africa cases, four MRI sequences, seven artifact types, and five severity settings, the five frozen encoders show strongly model- and artifact-dependent behavior. 3DINO, pretrained on roughly 100,000 multi-organ volumes, is the most consistently stable, while BrainIAC, a brain-MRI-specific contrastive ViT, is the most sensitive, with CKA approaching zero under frequency-domain corruptions. The most informative pattern is a dissociation between the two metrics: linear CKA drops substantially for many artifact–model combinations while RankMe stays nearly flat, which the authors read as geometric drift in the embedding space without dimensional collapse. A separate segmentation-consistency analysis with TumorSynth shows Dice declining most under ghosting and Rician noise, but the task-level ordering matches representation-level ordering only partially. The paper's conclusion is therefore that explicit representation-robustness evaluation is needed before deploying these models in heterogeneous MRI settings.","pith_inferences":["A direct implication the authors do not pursue is that the CKA-versus-RankMe dissociation predicts stronger degradation for tasks relying on local neighborhood structure, such as retrieval or few-shot classification, than for tasks using global subspace information; a direct test would run the same corruptions through linear-probe and retrieval benchmarks.","Because architecture, objective, pretraining domain, and scale differ simultaneously across the five models, the paper cannot identify which design choice drives 3DINO's stability; a controlled ablation varying only the pretraining domain would isolate that factor.","Because corruptions pass through each model's official preprocessing, the reported numbers describe end-to-end pipeline robustness rather than pure encoder robustness; a shared-preprocessing variant would split the two contributions.","If the pattern generalizes, a hospital using a single foundation model across scanners should expect artifact-induced feature-space shifts even when anatomy is unchanged, making quality-based sample exclusion or artifact-robust fine-tuning a practical next step."],"forward_implications":["A model trained on broad multi-organ medical data can be robust to MRI artifacts even when a brain-specific model trained for the same modality is not.","Deploying these encoders in heterogeneous clinical settings should include an explicit artifact-robustness check, since neither pretraining scale nor domain specificity predicts stability.","Reporting only a spectral metric like RankMe can miss large geometric changes measured by CKA, so robust-representation evaluation should report both.","Representation-level sensitivity and task-level segmentation consistency do not move together, so downstream validation cannot substitute for representation analysis."],"supporting_citations":[{"why":"Supplies the linear centered kernel alignment metric used to compare clean and corrupted embeddings.","marker":"[12]"},{"why":"Supplies the RankMe effective-rank metric used to detect spectral collapse.","marker":"[13]"},{"why":"Provides 3DINO, the multi-organ encoder that is the most robust model in the comparison.","marker":"[1]"},{"why":"Provides NeuroVFM, the large-scale clinical encoder whose artifact profile is compared.","marker":"[5]"},{"why":"Supplies the 95 BraTS-Africa glioma cases and four MRI sequences used as the evaluation data.","marker":"[27]"},{"why":"Supplies TumorSynth, the independent segmentation model used for the task-level consistency check.","marker":"[26]"},{"why":"Grounds the seven simulated MRI artifact types in the radiology artifact literature.","marker":"[6]"},{"why":"Supply the augmentation framework adapted for simulating Gibbs ringing and the bias field.","marker":"[21,22]"}],"fun_headline_variants":["Scale won't shield 3D medical models from MRI artifacts","MRI artifacts expose model-specific weaknesses in 3D encoders","3D medical AI: scale not shield against MRI artifacts","Model-specific fragility dominates 3D MRI robustness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the way each model rescales, crops, or normalizes the image before analysis does not decide the results, even though the corruptions are added before that step.","fun_headline_variants_meta":{"raw":{"variants":["Scale won't shield 3D medical models from MRI artifacts","MRI artifacts expose model-specific weaknesses in 3D encoders","3D medical AI: scale not shield against MRI artifacts","Model-specific fragility dominates 3D MRI robustness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000694,"raw_usage":{"total_tokens":3170,"prompt_tokens":1004,"completion_tokens":2166,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":2099}},"tokens_in":620,"tokens_out":2166,"duration_ms":14560,"temperature":1.0,"reasoning_tokens":2099,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:12:00.946005+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same CKA and RankMe benchmark with a single shared preprocessing step for all five encoders; if the model ranking changes, then preprocessing, not the learned representations, produced the reported robustness ordering.","supporting_citations":[{"cited_title":"In: International Conference on Machine Learning, PMlR (2019) 3519–3529 2, 9","cited_arxiv_id":null,"evidence_quote":"Supplies the linear centered kernel alignment metric used to compare clean and corrupted embeddings."},{"cited_title":"In: International Conference on Machine Learning, PMLR (2023) 10929–10974 2, 9","cited_arxiv_id":null,"evidence_quote":"Supplies the RankMe effective-rank metric used to detect spectral collapse."},{"cited_title":"npj Digital Medicine8(1) (2025) 639 1, 4, 5","cited_arxiv_id":null,"evidence_quote":"Provides 3DINO, the multi-organ encoder that is the most robust model in the comparison."},{"cited_title":"Nature Medicine (2026) 1–7 1, 4, 5","cited_arxiv_id":null,"evidence_quote":"Provides NeuroVFM, the large-scale clinical encoder whose artifact profile is compared."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 95 BraTS-Africa glioma cases and four MRI sequences used as the evaluation data."},{"cited_title":"Radiology: Imaging Cancer8(2) (2026) e250222 10","cited_arxiv_id":null,"evidence_quote":"Supplies TumorSynth, the independent segmentation model used for the task-level consistency check."},{"cited_title":"Polish Journal of Radiology80(2015) 93 2, 5, 6","cited_arxiv_id":null,"evidence_quote":"Grounds the seven simulated MRI artifact types in the radiology artifact literature."}],"review_version":1}