Pith. sign in

REVIEW 3 major objections 5 minor 76 references

CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that representing chest radiographs through a 'world model'—an encoder that predicts unseen anatomy and known appearance changes in latent space—sets a new state of the art for medical image representation learning.

desk verdict Solid SSL method with an over-stated SOTA claim; the same-compute gains are real, but the foundation-model comparison is not. read the letter →

arxiv 2504.13820 v1 pith:DSYQ426W submitted 2025-04-18 cs.CV

Yang Yue , Yulin Wang , Chenxin Tao , Pan Liu , Shiji Song , Gao Huang This is my paper · ORCID
classification cs.CV
keywords chestradiographrepresentationlearningself-supervisedworldmodelanatomicalstructuremodelingdomainvariationjoint-embeddingpredictivearchitecturemedicalfoundationtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that chest X-ray representation learning benefits from a 'world model': an encoder that, when shown part of a radiograph, predicts the unseen anatomy in latent space rather than reconstructing pixels. CheXWorld unifies three world-modeling tasks—predicting fine-grained local tissue structure from masked regions, predicting a second crop from the first given their relative geometry, and predicting how features change under known appearance transformations. The authors report that this single pre-trained encoder outperforms previous self-supervised methods and medical foundation models on eight classification, segmentation, and few-shot benchmarks, and that fine-tuning with only 10% of the VinDr-CXR training data beats baselines trained on the full dataset. If correct, the framework would let medical teams build accurate chest X-ray models with much smaller annotation budgets.

What carries the argument

The central mechanism is a joint-embedding predictive architecture in which a context encoder processes unmasked image patches, a target encoder (the exponential moving average of the context encoder) supplies prediction targets, and a masked-token predictor forecasts target features in latent space. The paper's contribution is to express all three knowledge dimensions through this single predictor, conditioned on two latent variables: the relative spatial displacement $\Delta_{x\to y}$ between image crops and an augmentation parameter vector $a$ describing the appearance transformation. The unified objective $L=L_{1\to1}+L_{2\to2}+L_{1\to2}+L_{2\to1}$ reuses context and target features across four supervisions, so one forward pass supervises local structure, global layout, and domain variation at the same time.

What would settle it

Train a CheXWorld variant without the domain-variation task, then fine-tune both versions on a chest X-ray dataset acquired with a different device or protocol than any in pre-training (for example, portable radiographs from a different institution), and compare performance. If the domain-variation component gives no measurable benefit on this genuinely out-of-distribution data—or if the model's ability to pick the true target among synthetic candidates collapses on real domain pairs—the proxy assumption behind the domain-variation claim is refuted.

Watch

Extended reading notes

Core claim

CheXWorld is presented as the first self-supervised world model for radiographic images. It encodes three dimensions of radiographic knowledge—local anatomical structures, global anatomical layout, and cross-domain appearance variation—into one latent-space predictor. The training objective runs four pairwise supervisions on two sampled crops: each context predicts its own target under masking and augmentation conditions, and each predicts the other crop given their relative displacement $\Delta_{x\to y}$ and the augmentation parameter vector $a$. The paper's empirical claim is that this model, pre-trained from scratch on roughly 0.5 million frontal chest X-rays with a ViT-Base backbone, outperforms strong self-supervised baselines and the medical foundation models tested on all eight benchmarks, and that it performs comparably to a much larger radiology encoder trained with 20 times more computational resources and private data.

Load-bearing premise

The load-bearing premise is that the synthetic brightness, contrast, gamma, and blur augmentations used during pre-training faithfully stand in for the real appearance differences between hospitals, devices, and patients; the paper evaluates domain sensitivity only on augmented images drawn from that same synthetic distribution.

Editorial extensions

If this is right

  • If the reported results hold, a single pre-trained CheXWorld encoder can be fine-tuned for multi-label disease classification, dense segmentation, and few-shot adaptation, replacing task-specific training pipelines.
  • The VinDr-CXR 10%-data result implies that annotation budgets for chest X-ray tasks could be reduced by more than 90 percent while still surpassing baselines that see the full dataset.
  • Building an equivariant representation, where a known appearance change produces a predictable change in features, preserves information that invariant contrastive methods discard.
  • The comparison with a much larger encoder trained on private data suggests that carefully chosen self-supervised objectives can substitute for model scale in medical imaging.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper tests domain sensitivity only on synthetic augmentations from the pre-training distribution; a genuinely out-of-distribution dataset from an unseen hospital or device would directly probe whether the modeled domain transitions capture real acquisition shifts.
  • The unified single-forward-pass coupling of the three objectives suggests the same design could transfer to other imaging modalities, such as mammography or CT, but that extension is untested here.
  • Because the domain-variation task is conditioned on known augmentation parameters, the framework could be extended to actively controlled acquisition settings, where the exact device settings are known and could serve as supervised domain labels.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces CheXWorld, a self-supervised representation learning framework for chest X-rays built around a joint-embedding predictive architecture (JEPA). It unifies three prediction tasks: local anatomical structure modeling (reconstructing features of masked regions), global anatomical layout modeling (predicting the features of a randomly cropped region from another crop conditioned on their relative position), and domain variation modeling (predicting how features change under photometric augmentations conditioned on the augmentation parameters). The model is pre-trained on about 0.5M frontal chest X-rays with a ViT-Base encoder, and evaluated by full fine-tuning on five classification benchmarks, two segmentation benchmarks, and a few-shot benchmark, plus sample-efficiency experiments on VinDr-CXR. The paper also provides analytical visualizations and ablations to support the claim that the model captures the three intended knowledge dimensions.

Significance. The framework is technically clean and the transfer results, if reproduced, would be a strong addition to medical image SSL. The unification of three tasks in a single pass is elegant, the equivariant formulation is a sensible alternative to invariant contrastive learning, and the 10%-data VinDr-CXR result (94.71 AUROC vs best full-data baseline 92.81) is a credible and practically valuable outcome. The release of code and pre-trained models, the explicit leakage control in Appendix B.1, and the RCDM-based visualizations are commendable. The main weakness is that the strongest headline claim—significant outperformance over large-scale medical foundation models—is not supported by the paper's own comparison with Rad-DINO and by the heterogeneous baseline protocols.

major comments (3)
  1. [Abstract, Section 5.2, Table 1] The unqualified claim that CheXWorld 'significantly outperforms existing SSL methods and large-scale medical foundation models' is not supported by the reported data. Section 5.2 states that CheXWorld 'performs comparably to Rad-DINO across all benchmarks,' and Table 1 shows Rad-DINO ahead on ChestX-ray14 (83.61 vs 83.58) and within 0.08–0.69 AUROC/accuracy on the other four benchmarks. Rad-DINO is dismissed only in a footnote about compute and private data, which does not turn a comparable result into a significant win. Moreover, several baseline entries are taken from prior papers with different fine-tuning protocols (daggered entries; refs [32], [33], [46]) and no paired significance tests are reported. Please either limit the claim to 'comparable same-compute SSL baselines' or add a head-to-head evaluation with Rad-DINO and other foundation models under an identical protocol with significance testing.
  2. [Section 4.3 and Section 5.1 (Table 4)] The domain variation modeling task is described as modeling transitions across appearance domains caused by different hospitals, devices, or patients, and is claimed to enable 'cross-domain adaptability.' The only quantitative evidence for this is the domain-sensitivity test of Table 4, which uses synthetic photometric augmentations (brightness, contrast, gamma, Gaussian blur) sampled from the same distribution as pre-training; it is a self-consistency check rather than an out-of-distribution evaluation. While the downstream benchmarks span multiple real datasets, the ablation in Table 3 does not isolate the cross-domain benefit of the domain-variation component. Please either add an evaluation under genuine hospital/device shift (e.g., training on one acquisition protocol and testing on another) or soften the domain-shift claims to 'photometric appearance variations.'
  3. [Section 1 and Section 5.2 (Table 1)] The paper asserts that comparisons are made with 'comparable backbone capacity, pre-training data, and pre-training computational cost,' but Table 1 shows that SimMIM (Swin-B) and Adam-v2 (ConvNeXt-B) are initialized from ImageNet weights and pre-trained on 0.9M X-rays, while CheXWorld uses a ViT-B trained from scratch on 0.5M X-rays. The backbones are all 'base' scale, but the pre-training data and initialization differ substantially. Please state this asymmetry explicitly in the main text and discuss why CheXWorld's gains are not attributable to the comparison setup; the current 'comparable' wording overstates the fairness of the comparison.
minor comments (5)
  1. [Table 1] Several entries (MAE on ChestX-ray14, Adam-v2 on ShenZhen and ChestX-ray14) are reported without standard deviations because they are adopted from the original papers; please add a clear table note and avoid visual comparisons where the margin is smaller than the missing uncertainty.
  2. [Table 3] The rendering makes it hard to tell which tasks and latent variables are active in each row; please add explicit row labels or use a legend for the checkmarks.
  3. [Section 4.2, Eqs. (4)-(5)] The symbol Δx→y is used for both the displacement vector and the coordinate mapping; consider renaming one of them to avoid confusion.
  4. [Figures 4 and 5] The red bounding boxes mentioned in the captions are difficult to see in the inline reproduction; please enlarge or add insets so the qualitative claims are verifiable.
  5. [Section 4.3, references] The reference [15] (Cui et al., confidence-based reliable learning) appears unrelated to the sentence citing it in Section 4.3; please verify the citation.

Circularity Check

1 steps flagged · score 2.0 of 10

No load-bearing circularity in the transfer-learning claims; one internal self-consistency check (domain sensitivity) is presented as evidence for domain-variation knowledge.

  1. fitted input called prediction [Section 4.3 (Eq. 8), Section 5.1 / Table 4, Appendix B.2]
    "We simulate domain shifts using data augmentation and construct context-target pairs to model these transitions. ... The parameter a contains the strength and other configurations of the augmentation. ... For each predicted output ˆyi, we calculate the top-k recall rate of the true target yi over the entire candidate set using L2 distance. As shown in Table 4, our model achieves an average top-5 recall of 77.67 (10 times higher than random choice), demonstrating the model's strong discriminative ability across domains."

    The pre-training objective L_domain (Eq. 8) trains the predictor to map (f_theta(x), a) to f'_theta'(y), where x = T_a(y) and T is the brightness/contrast/gamma/Gaussian-blur augmentation family. The Section 5.1 'domain sensitivity test' builds (x_i, y_i, a_i) from exactly the same T and a, and ranks candidate targets by the same L2 embedding distance used in training. High top-5 recall therefore shows that the model can invert the synthetic augmentation family it was explicitly fitted to invert; it does not measure generalization to real hospital/device/patient appearance shifts. The paper's evidence that CheXWorld 'captures domain variations' is thus a self-consistency check rather than an independent external validation.

full rationale

CheXWorld's main transfer-learning results (Tables 1 and 2, Figure 7, and Table 5) are measured on external benchmarks with fixed fine-tuning protocols, so the state-of-the-art claim does not reduce to the pre-training objective. The only element approaching circularity is the domain-variation analysis: both the pre-training task L_domain (Eq. 8) and the Section 5.1 'domain sensitivity test' use the same synthetic augmentation family (brightness, contrast, gamma, Gaussian blur) and the same conditioning variable a. High recall on that test confirms that the model learned the fitted augmentation-to-embedding mapping, not that it has robust knowledge of genuine hospital/device/patient domain shifts. The paper would need out-of-distribution domain transfer experiments to support the broader domain-variation claim. No load-bearing self-citation chain or imported uniqueness argument appears; the authors' self-citations [15, 39, 66] are background citations only. The Rad-DINO comparison concern raised in the skeptic analysis is an evidence-strength and protocol-matching issue, not a circularity issue. Overall, the central derivation is self-contained against external benchmarks, so the score is low.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

CheXWorld introduces no new physical or mathematical entities. The framework relies on standard SSL machinery (EMA encoders, ViT, sinusoidal position encodings) and on three domain assumptions about what the prediction objectives teach. The main hand-set choices are augmentation ranges, crop and mask configurations, and loss weights; these are not fitted to downstream data but are free in the sense that other values could change the learned representation.

free parameters (5)
  • Masking configuration (local task) = 4 rectangular blocks, scale 0.15 to 0.2, visible area shrunk up to 0.25
    Hand-chosen; controls the difficulty and nature of the local structure prediction task. Not fitted to downstream data.
  • Crop configuration (global task) = scale 0.3 to 1.0, aspect ratio 0.75 to 1.33
    Hand-chosen; defines the relative-position prediction window sizes.
  • Domain augmentation ranges = brightness 0.6 to 1.4, contrast 0.6 to 1.4, gamma 0.5 to 2.0, blur kernel 0.05 to 2.0
    Hand-chosen; defines the simulated domain transitions. Central to the domain variation task's coverage.
  • Loss weights = all four losses weighted equally at 1.0
    No tuning reported; may affect the balance of the three knowledge dimensions.
  • Target encoder EMA schedule = initial 0.996 increasing to 1.0 cosine
    Standard JEPA practice; hand-chosen, affects target representation stability.
assumptions (4)
  • standard math The joint-embedding predictive architecture with EMA target encoder and L2 prediction loss produces useful representations for downstream tasks.
    Inherited from I-JEPA [2]; not re-derived in this paper.
  • domain assumption Synthetic photometric augmentations (brightness, contrast, gamma, blur) are a sufficient proxy for real-world radiograph domain variations across hospitals, devices, and patients.
    Stated as a hypothesis in Section 4.3; the paper provides no real-domain shift experiment to support it.
  • domain assumption Predicting the features of a spatially distant crop conditioned on relative displacement teaches global anatomical layout.
    Section 4.2; the assumption that the objective induces layout knowledge is supported only indirectly by downstream transfer.
  • domain assumption Predicting features of masked local regions teaches fine-grained anatomical structure.
    Section 4.1; similar to masked autoencoding assumptions; supported by transfer gains but not directly verified as anatomy knowledge.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning." pith.science (2026). https://pith.science/paper/DSYQ426W

@misc{pith2026250413820,
  author       = {Pith},
  title        = {Pith review of: CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DSYQ426W}},
  note         = {Machine review of arXiv:2504.13820}
}
read the original abstract

Humans can develop internal world models that encode common sense knowledge, telling them how the world works and predicting the consequences of their actions. This concept has emerged as a promising direction for establishing general-purpose machine-learning models in recent preliminary works, e.g., for visual representation learning. In this paper, we present CheXWorld, the first effort towards a self-supervised world model for radiographic images. Specifically, our work develops a unified framework that simultaneously models three aspects of medical knowledge essential for qualified radiologists, including 1) local anatomical structures describing the fine-grained characteristics of local tissues (e.g., architectures, shapes, and textures); 2) global anatomical layouts describing the global organization of the human body (e.g., layouts of organs and skeletons); and 3) domain variations that encourage CheXWorld to model the transitions across different appearance domains of radiographs (e.g., varying clarity, contrast, and exposure caused by collecting radiographs from different hospitals, devices, or patients). Empirically, we design tailored qualitative and quantitative analyses, revealing that CheXWorld successfully captures these three dimensions of medical knowledge. Furthermore, transfer learning experiments across eight medical image classification and segmentation benchmarks showcase that CheXWorld significantly outperforms existing SSL methods and large-scale medical foundation models. Code & pre-trained models are available at https://github.com/LeapLabTHU/CheXWorld.

Figures

Figures reproduced from arXiv: 2504.13820 by the authors.

Figure 1
Figure 1. Overview of the CheXWorld framework1 . The upper part of the figure depicts three dimensions of medical knowledge that are formulated in our framework, including (a) local anatomical structures describing the fine-grained characteristics of local tissues, (b) global anatomical layouts describing the global organization of the human body and (c) domain variations that encourage CheXWorld to model the transitions acro… view at source ↗
Figure 2
Figure 2. A basic framework of world modeling. A basic framework. Here, we introduce our basic framework of in￾stantiating world mod￾eling, upon which we will further discuss how to establish a radiology world model by formu￾lating various medical knowledge in Section 4. In specific, our basic framework mainly follows the joint-embedding predictive architecture (JEPA) [2, 41], as shown in [PITH_FULL_IMAGE:figures/full_fig_p0… view at source ↗
Figure 3
Figure 3. Formulation of global anatomical layout modeling. As illustrated in Fig￾ure 1(b), we randomly crop two anatomical ar￾eas from the original radiograph, serving as context x and target y, respectively. The world model learns to predict y from x given their rel￾ative position informa￾tion ∆x→y. In particu￾lar, we assume the con￾text and target image crop share the same spatial size of h pixels in height and w pixels in… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization of the CheXWorld predictor outputs (zooming in for details). The images presented in this figure were not included in the pre-training of CheXWorld or the training of the diffusion model. Regions in red bounding boxes denote the predictor outputs that are…
Figure 6
Figure 6. Figure 6: CheXWorld learns anatomical correspondence. We compute pixel-level embeddings using RoI-pooling [23] and calculate the embedding similarity between four anatomical landmarks in a reference image and each pixel in the test image to create feature similarity heatmaps. at…
Figure 7
Figure 7. Figure 7: Fine-tuning with 1%, 10%, and 100% training data on VinDr-CXR. MAE Adam-v2 Ours GT [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Visualization of segmentation masks on SIIM-ACR pneumothorax dataset. GT stands for the ground truth masks [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 48 canonical work pages

  1. [32]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 3, 4, 7, 8, 1, 2

    work page 2022

  2. [33]

    Representing part-whole hierarchies in foundation models by learning localizability, composability, and decomposability from anatomy via self-supervision

    Mohammad Reza Hosseinzadeh Taher, Michael Gotway, and Jianming Liang. Representing part-whole hierarchies in foundation models by learning localizability, composability, and decomposability from anatomy via self-supervision. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, 2024. 3, 7, 8, 1, 2

    work page 2024

  3. [46]

    Benchmarking and boosting transformers for medical image classification

    DongAo Ma, Mohammad Reza Hosseinzadeh Taher, Jiax- uan Pang, Nahid UI Islam, Fatemeh Haghighi, Michael B Gotway, and Jianming Liang. Benchmarking and boosting transformers for medical image classification. In MICCAI Workshop on Domain Adaptation and Representation Trans- fer, pages 12–22. Springer, 2022. 7, 8, 1, 2

    work page 2022

  4. [1]

    Siim-acr pneumothorax seg- mentation

    Zawacki Anna, Wu Carol, Shih George, Elliott Julia, Fomitchev Mikhail, Hussain Mohannad, Lakhani Paras, Cul- liton Phil, and Bao Shunxing. Siim-acr pneumothorax seg- mentation. 2019. 7, 1

    work page 2019

  5. [2]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bo- janowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619–15629, 2023. 1, 3, 4, 2

    work page 2023

  6. [3]

    Big self-supervised models advance medical image classifica- tion

    Shekoofeh Azizi, Basil Mustafa, Fiona Ryan, Zachary Beaver, Jan Freyberg, Jonathan Deaton, Aaron Loh, Alan Karthikesalingam, Simon Kornblith, Ting Chen, et al. Big self-supervised models advance medical image classifica- tion. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 3478–3488, 2021. 3

    work page 2021

  7. [4]

    Data2vec: A general frame- work for self-supervised learning in speech, vision and lan- guage

    Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. Data2vec: A general frame- work for self-supervised learning in speech, vision and lan- guage. In International Conference on Machine Learning , pages 1298–1312. PMLR, 2022. 3

    work page 2022

  8. [5]

    Efficient self-supervised learning with contextualized target representations for vision, speech and language

    Alexei Baevski, Arun Babu, Wei-Ning Hsu, and Michael Auli. Efficient self-supervised learning with contextualized target representations for vision, speech and language. In In- ternational Conference on Machine Learning , pages 1416–

Show all 76 references
  1. [6]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. 7, 1

    2021 arXiv

  2. [7]

    V-jepa: Latent video prediction for visual represen- tation learning

    Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mido Assran, and Nicolas Ballas. V-jepa: Latent video prediction for visual represen- tation learning. 2023. 3

    2023

  3. [8]

    High fidelity visualization of what your self-supervised rep- resentation knows about

    Florian Bordes, Randall Balestriero, and Pascal Vincent. High fidelity visualization of what your self-supervised rep- resentation knows about. arXiv preprint arXiv:2112.09164,

    arXiv

  4. [9]

    Applied optimal control: optimization, estimation and control

    Arthur Earl Bryson. Applied optimal control: optimization, estimation and control. Routledge, 2018. 2

    2018

  5. [10]

    Constrained model predictive control

    Eduardo F Camacho, Carlos Bordons, Eduardo F Camacho, and Carlos Bordons. Constrained model predictive control. Springer, 2007. 2

    2007

  6. [11]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 7, 1

    2021

  7. [12]

    Contrastive learning of global and local fea- tures for medical image segmentation with limited annota- tions

    Krishna Chaitanya, Ertunc Erdil, Neerav Karani, and Ender Konukoglu. Contrastive learning of global and local fea- tures for medical image segmentation with limited annota- tions. Advances in neural information processing systems , 33:12546–12558, 2020. 3

    2020

  8. [13]

    Towards a general-purpose foundation model for computational pathology

    Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Andrew H Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30(3):850–862,

  9. [14]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9640–9649, 2021. 7, 1

    2021

  10. [15]

    Confidence- based reliable learning under dual noises

    Peng Cui, Yang Yue, Zhijie Deng, and Jun Zhu. Confidence- based reliable learning under dual noises. Advances in Neu- ral Information Processing Systems, 35:35116–35129, 2022. 5

    2022

  11. [16]

    Equivariant contrastive learning

    Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljaˇci´c. Equivariant contrastive learning. arXiv preprint arXiv:2111.00899, 2021. 3, 5

    2021 arXiv

  12. [17]

    Equimod: An equivariance module to improve visual instance discrimina- tion

    Alexandre Devillers and Mathieu Lefort. Equimod: An equivariance module to improve visual instance discrimina- tion. In International Conference on Learning Representa- tions, 2023. 3, 5

    2023

  13. [18]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  14. [19]

    Prob- abilistic contrastive learning for long-tailed visual recogni- tion

    Chaoqun Du, Yulin Wang, Shiji Song, and Gao Huang. Prob- abilistic contrastive learning for long-tailed visual recogni- tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 3

    2024

  15. [20]

    A learned representation for artistic style

    Vincent Dumoulin, Jonathon Shlens, and Manjunath Kud- lur. A learned representation for artistic style. arXiv preprint arXiv:1610.07629, 2016. 2

    2016 arXiv

  16. [21]

    Synthetic data accelerates the development of gener- alizable learning-based algorithms for x-ray image analysis

    Cong Gao, Benjamin D Killeen, Yicheng Hu, Robert B Grupp, Russell H Taylor, Mehran Armand, and Mathias Un- berath. Synthetic data accelerates the development of gener- alizable learning-based algorithms for x-ray image analysis. Nature Machine Intelligence, 5(3):294–308, 2023. 1

    2023

  17. [22]

    Learning and leveraging world models in visual representation learning

    Quentin Garrido, Mahmoud Assran, Nicolas Ballas, Adrien Bardes, Laurent Najman, and Yann LeCun. Learning and leveraging world models in visual representation learning. arXiv preprint arXiv:2403.00504, 2024. 1, 3

    2024 arXiv

  18. [23]

    Rich feature hierarchies for accurate object detection and semantic segmentation

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 580–587, 2014. 6

    2014

  19. [24]

    Domain adaptation for medical image analysis: a survey

    Hao Guan and Mingxia Liu. Domain adaptation for medical image analysis: a survey. IEEE Transactions on Biomedical Engineering, 69(3):1173–1185, 2021. 2, 5

    2021

  20. [25]

    Recurrent world models facilitate policy evolution

    David Ha and J ¨urgen Schmidhuber. Recurrent world models facilitate policy evolution. Advances in neural information processing systems, 31, 2018. 2

    2018

  21. [26]

    World models

    David Ha and J ¨urgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2018. 3

    2018 arXiv

  22. [27]

    Dream to control: Learning behaviors by la- tent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Moham- mad Norouzi. Dream to control: Learning behaviors by la- tent imagination. arXiv preprint arXiv:1912.01603, 2019. 9

    1912 arXiv

  23. [28]

    Mastering atari with discrete world models.arXiv preprint arXiv:2010.02193, 2020

    Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models.arXiv preprint arXiv:2010.02193, 2020. 2

    2010 arXiv

  24. [29]

    Mastering diverse domains through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. 3

    2023 arXiv

  25. [30]

    Transferable visual words: Exploiting the semantics of anatomical patterns for self-supervised learning.IEEE trans- actions on medical imaging, 40(10):2857–2868, 2021

    Fatemeh Haghighi, Mohammad Reza Hosseinzadeh Taher, Zongwei Zhou, Michael B Gotway, and Jianming Liang. Transferable visual words: Exploiting the semantics of anatomical patterns for self-supervised learning.IEEE trans- actions on medical imaging, 40(10):2857–2868, 2021. 3

    2021

  26. [31]

    Dira: Discrimina- tive, restorative, and adversarial learning for self-supervised medical image analysis

    Fatemeh Haghighi, Mohammad Reza Hosseinzadeh Taher, Michael B Gotway, and Jianming Liang. Dira: Discrimina- tive, restorative, and adversarial learning for self-supervised medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2022

  27. [34]

    Gaia-1: A generative world model for au- tonomous driving

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gian- luca Corrado. Gaia-1: A generative world model for au- tonomous driving. arXiv preprint arXiv:2309.17080, 2023. 3

    2023 arXiv

  28. [35]

    A visual–language foundation model for pathology image analysis using medical twitter

    Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou. A visual–language foundation model for pathology image analysis using medical twitter. Nature medicine, 29(9):2307–2316, 2023. 1

    2023

  29. [36]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Sil- viana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAA...

    2019

  30. [37]

    Two public chest x- ray datasets for computer-aided screening of pulmonary dis- eases

    Stefan Jaeger, Sema Candemir, Sameer Antani, Y `ı-Xi´ang J W´ang, Pu-Xuan Lu, and George Thoma. Two public chest x- ray datasets for computer-aided screening of pulmonary dis- eases. Quantitative imaging in medicine and surgery , 4(6): 475, 2014. 7, 1

    2014

  31. [38]

    Mimic-cxr, a de- identified publicly available database of chest radiographs with free-text reports

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de- identified publicly available database of chest radiographs with free-text reports. Scientific data , 6(1):317, 2019. 1, 2

    2019

  32. [39]

    How far is video generation from world model: A physical law perspective

    Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: A physical law perspective. arXiv preprint arXiv:2411.02385, 2024. 3

    2024 arXiv

  33. [40]

    Videopoet: A large language model for zero-shot video generation

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos ´e Lezama, Jonathan Huang, Rachel Hornung, Hartwig Adam, Hassan Akbari, Yair Alon, Vighnesh Birodkar, et al. Videopoet: A large language model for zero-shot video generation. arXiv preprint arXiv:2312.14125, 2023. 3

    2023 arXiv

  34. [41]

    A path towards autonomous machine intelli- gence version 0.9

    Yann LeCun. A path towards autonomous machine intelli- gence version 0.9. 2, 2022-06-27.Open Review, 62(1), 2022. 1, 2, 3

    2022

  35. [42]

    Exploring plain vision transformer backbones for object de- tection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. In European conference on computer vision , pages 280–296. Springer, 2022. 3

    2022

  36. [43]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 7, 1, 2

    2021

  37. [44]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 11976–11986,

  38. [45]

    Fixing weight decay reg- ularization in adam

    Ilya Loshchilov and Frank Hutter. Fixing weight decay reg- ularization in adam. 2018. 2

    2018

  39. [47]

    Case courtesy of phillip marsh, radiopae- dia.org

    Phillip Marsh. Case courtesy of phillip marsh, radiopae- dia.org. https : / / radiopaedia . org / cases / 58938, 2023. Case ID: rID: 58938. 2

    2023

  40. [48]

    Lvm-med: Learning large- scale self-supervised vision models for medical imaging via second-order graph matching

    Duy MH Nguyen, Hoang Nguyen, Nghiem Diep, Tan Ngoc Pham, Tri Cao, Binh Nguyen, Paul Swoboda, Nhat Ho, Shadi Albarqouni, Pengtao Xie, et al. Lvm-med: Learning large- scale self-supervised vision models for medical imaging via second-order graph matching. Advances in Neural Info...

    2024

  41. [49]

    Foundation models for generalist medi- cal artificial intelligence

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. Foundation models for generalist medi- cal artificial intelligence. Nature, 616(7956):259–265, 2023. 1

    2023

  42. [50]

    Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations

    Ha Q Nguyen, Khanh Lam, Linh T Le, Hieu H Pham, Dat Q Tran, Dung B Nguyen, Dung D Le, Chi M Pham, Hang TT Tong, Diep H Dinh, et al. Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations. Scientific Data, 9(1):429, 2022. 7, 1

    2022

  43. [51]

    Video generation models as world simula- tors

    OpenAI. Video generation models as world simula- tors. https : / / openai . com / index / video - generation - models - as - world - simulators/,

  44. [52]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. 10 Transactions on Machine Learning Research ...

    2024

  45. [53]

    Causality-inspired single- source domain generalization for medical image segmenta- tion

    Cheng Ouyang, Chen Chen, Surui Li, Zeju Li, Chen Qin, Wenjia Bai, and Daniel Rueckert. Causality-inspired single- source domain generalization for medical image segmenta- tion. IEEE Transactions on Medical Imaging , 42(4):1095– 1106, 2022. 2, 5

    2022

  46. [54]

    Popar: Patch order prediction and appearance recovery for self-supervised medical image anal- ysis

    Jiaxuan Pang, Fatemeh Haghighi, DongAo Ma, Nahid Ul Is- lam, Mohammad Reza Hosseinzadeh Taher, Michael B Got- way, and Jianming Liang. Popar: Patch order prediction and appearance recovery for self-supervised medical image anal- ysis. In MICCAI Workshop on Domain Adaptation an...

    2022

  47. [55]

    Rad-dino: Exploring scalable medical image encoders beyond text supervision

    Fernando P ´erez-Garc´ıa, Harshita Sharma, Sam Bond-Taylor, Kenza Bouzid, Valentina Salvatelli, Maximilian Ilse, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Matthew P Lungren, et al. Rad-dino: Exploring scalable medical image encoders beyond text supervision. arXiv pr...

    2024 arXiv

  48. [56]

    Why don’t radiology textbooks have imperfect images? https://www.diagnosticimaging.com/ view/why- dont- radiology- textbooks- have- imperfect-images, 2019

    Eric Postal. Why don’t radiology textbooks have imperfect images? https://www.diagnosticimaging.com/ view/why- dont- radiology- textbooks- have- imperfect-images, 2019. Accessed: Feb, 2024. 2, 5

    2019

  49. [57]

    Introducing general world models

    RunwayML. Introducing general world models. https: / / research . runwayml . com / introducing - general-world-models, 2023. Accessed: May, 2023. 2

    2023

  50. [58]

    Augmenting the national institutes of health chest radiograph dataset with expert annotations of possi- ble pneumonia

    George Shih, Carol C Wu, Safwan S Halabi, Marc D Kohli, Luciano M Prevedello, Tessa S Cook, Arjun Sharma, Judith K Amorosa, Veronica Arteaga, Maya Galperin- Aizenberg, et al. Augmenting the national institutes of health chest radiograph dataset with expert annotations of possi...

    2019

  51. [59]

    Neu- ral representational geometry underlies few-shot concept learning

    Ben Sorscher, Surya Ganguli, and Haim Sompolinsky. Neu- ral representational geometry underlies few-shot concept learning. Proceedings of the National Academy of Sciences, 119(43):e2200800119, 2022. 1

    2022

  52. [60]

    Moco pretraining improves representation and transferability of chest x-ray models

    Hari Sowrirajan, Jingbo Yang, Andrew Y Ng, and Pranav Rajpurkar. Moco pretraining improves representation and transferability of chest x-ray models. In Medical Imaging with Deep Learning, pages 728–744. PMLR, 2021. 3

    2021

  53. [61]

    Caid: a self- supervised learning framework for empowering instance dis- crimination in medical imaging

    Mohammad Reza Hosseinzadeh Taher, Fatemeh Haghighi, Michael B Gotway, and Jianming Liang. Caid: a self- supervised learning framework for empowering instance dis- crimination in medical imaging. Proceedings of Machine Learning Research, 3, 2022. 3

    2022

  54. [62]

    Revisiting rubik’s cube: self-supervised learning with volume-wise transformation for 3d medical image seg- mentation

    Xing Tao, Yuexiang Li, Wenhui Zhou, Kai Ma, and Yefeng Zheng. Revisiting rubik’s cube: self-supervised learning with volume-wise transformation for 3d medical image seg- mentation. In Medical Image Computing and Computer As- sisted Intervention–MICCAI 2020: 23rd International ...

    2020

  55. [63]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 4, 2

    2017

  56. [64]

    A real-world dataset and benchmark for foundation model adaptation in medical image classification

    Dequan Wang, Xiaosong Wang, Lilong Wang, Mengzhang Li, Qian Da, Xiaoqiang Liu, Xiangyu Gao, Jun Shen, Junjun He, Tian Shen, et al. A real-world dataset and benchmark for foundation model adaptation in medical image classification. Scientific Data, 10(1):574, 2023. 7, 1

    2023

  57. [65]

    Chestx- ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mo- hammadhadi Bagheri, and Ronald M Summers. Chestx- ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on...

  58. [66]

    Efficienttrain: Exploring gener- alized curriculum learning for training visual backbones

    Yulin Wang, Yang Yue, Rui Lu, Tianjiao Liu, Zhao Zhong, Shiji Song, and Gao Huang. Efficienttrain: Exploring gener- alized curriculum learning for training visual backbones. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5852–5864, 2023. 3

    2023

  59. [67]

    Delving into masked autoencoders for multi-label thorax dis- ease classification

    Junfei Xiao, Yutong Bai, Alan Yuille, and Zongwei Zhou. Delving into masked autoencoders for multi-label thorax dis- ease classification. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 3588–3600, 2023. 3, 7, 8, 1, 2

    2023

  60. [68]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9653–9663, 2022. 7, 8, 1, 2

    2022

  61. [69]

    Sam: Self-supervised learning of pixel-wise anatom- ical embeddings in radiological images

    Ke Yan, Jinzheng Cai, Dakai Jin, Shun Miao, Dazhou Guo, Adam P Harrison, Youbao Tang, Jing Xiao, Jingjing Lu, and Le Lu. Sam: Self-supervised learning of pixel-wise anatom- ical embeddings in radiological images. IEEE Transactions on Medical Imaging, 41(10):2658–2669, 2022. 3

    2022

  62. [70]

    Comparing to learn: Surpassing ima- genet pretraining on radiographs by comparing image repre- sentations

    Hong-Yu Zhou, Shuang Yu, Cheng Bian, Yifan Hu, Kai Ma, and Yefeng Zheng. Comparing to learn: Surpassing ima- genet pretraining on radiographs by comparing image repre- sentations. In Medical Image Computing and Computer As- sisted Intervention–MICCAI 2020: 23rd International C...

    2020

  63. [71]

    Self pre-training with masked autoencoders for medical image classification and segmentation

    Lei Zhou, Huidong Liu, Joseph Bae, Junjun He, Dim- itris Samaras, and Prateek Prasanna. Self pre-training with masked autoencoders for medical image classification and segmentation. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pages 1–6. IEEE, 2023. 3

    2023

  64. [72]

    A foundation model for generalizable disease detection from retinal images

    Yukun Zhou, Mark A Chia, Siegfried K Wagner, Murat S Ayhan, Dominic J Williamson, Robbert R Struyven, Tim- ing Liu, Moucheng Xu, Mateo G Lozano, Peter Woodward- Court, et al. A foundation model for generalizable disease detection from retinal images. Nature, 622(7981):156–163,

  65. [73]

    Models genesis

    Zongwei Zhou, Vatsal Sodha, Jiaxuan Pang, Michael B Got- way, and Jianming Liang. Models genesis. Medical image analysis, 67:101840, 2021. 2, 3

    2021

  66. [74]

    Learning anatom- ically consistent embedding for chest radiography

    Ziyu Zhou, Haozhe Luo, Jiaxuan Pang, Xiaowei Ding, Michael Gotway, and Jianming Liang. Learning anatom- ically consistent embedding for chest radiography. arXiv preprint arXiv:2312.00335, 2023. 2, 3 11 CheXWorld: Exploring Image World Modeling for Radiograph Representation Lea...

    2023 arXiv

  67. [76]

    Detailed information on these datasets is provided below

    and NIH ChestX-ray14 [65] for both pre-training and fine-tuning; VinDr-CXR [50], ShenZhen-CXR [37], RSNA Pneumonia [58], MedFMC-ChestDR [64], and SIIM-ACR Pneumothorax [1] for fine-tuning. Detailed information on these datasets is provided below. • MIMIC-CXR [38] is one of the...

  68. [2024]

    Accessed: Feb, 2024. 3

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.