Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training

Haiming Li; H. Peter Soyer; Jiajun Sun; Jingmin Zhu; Monika Janda; Victoria Mar; Yingsheng Liu; Zhen Yu; Zongyuan Ge

REVIEW 3 major objections 4 minor 25 references

Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A pre-training method that models medical tabular data's two-dimensional structure—feature importance and interval-based value semantics—achieves state-of-the-art results on three large clinical benchmarks.

desk verdict Adaptive importance masking plus soft-label binning is a solid, reproducible advance in image-tabular pretraining with consistent gains, but the semantic-awareness story needs direct validation of its PCA/TabPFN importance proxy. read the letter →

arxiv 2608.10522 v1 pith:65SERJAJ submitted 2026-08-11 cs.CV cs.AI

Yingsheng Liu , Haiming Li , Jingmin Zhu , Jiajun Sun , Victoria Mar , Monika Janda , H. Peter Soyer , Zongyuan Ge

show 1 more author

Zhen Yu

This is my paper · ORCID

classification cs.CVcs.AI

keywords medicaltabulardatamultimodalpre-trainingself-supervisedlearningfeatureimportancemaskingsoft-labeldiscretizationdermatologyophthalmologyimage-tabular

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces AID, a self-supervised pre-training framework for pairs of medical images and structured clinical tables. It claims that modeling two structural properties of tabular data—some features matter more than others, and exact continuous values should be reconstructed as soft distributions over ordinal intervals—produces representations that transfer better than treating tables as flat vectors. On a geographically separated out-of-domain test set of skin-lesion images, the method reaches a full fine-tuning ROC-AUC of 0.944 and a partial AUC of 0.162, versus 0.911 and 0.141 for the strongest previous method, and it reports similar gains on a second dermatology dataset and on retinal fundus images. The practical stake is that large collections of unlabeled image-plus-table clinical data could be converted into robust diagnostic models without expert annotations.

What carries the argument

The machinery is the two-dimensional model of tabular structure that AID encodes, with two components. For inter-feature structure, an offline importance vector $s$ is extracted by fitting a frozen tabular foundation model to balanced binary pseudo-labels from the median split of the first principal component of the standardized feature matrix; each feature $j$ then gets masking rate $r_j = \min(r_{\mathrm{base}} + \alpha s_j, r_{\max})$. For intra-feature structure, the soft-label discretized module builds $B$ quantile bins per feature and maps a value $v$ falling in bin $c$ with boundaries $[b_{c-1}, b_c)$ to a probability vector over adjacent bins using the triangle kernel and intra-bin position $p = (v - b_{c-1})/(b_c - b_{c-1})$, which preserves ordinal relationships. Pre-training minimizes $\mathcal{L}_{\mathrm{AID}} = (\mathcal{L}_{\mathrm{ITC}} + \mathcal{L}_{\mathrm{ITM}} + \mathcal{L}_{\mathrm{DR}})/3$, combining image-tabular contrastive learning, image-tabular matching with hard negatives, and discretized reconstruction whose continuous part is a KL divergence between predicted and soft-target bin distributions.

What would settle it

Swap the learned importance vector for a random permutation of itself while holding the total masking budget fixed and re-run pre-training; if the out-of-domain AUC stays near 0.944, the importance mechanism is not the cause of the gain, whereas a drop toward the uniform-masking result of about 0.930 would show the importance prior is doing real work.

Watch

Extended reading notes

Core claim

The central discovery is that the two-dimensional hierarchy of medical tabular data can be exploited during pre-training without using diagnostic labels. Inter-feature, the model estimates importance by fitting a frozen general-purpose tabular model to pseudo-labels derived from the first principal component of the standardized features, then masks high-importance features more often so reconstruction must draw on image evidence. Intra-feature, it converts each continuous value into a triangle-kernel soft label over quantile-based bins, so reconstruction becomes a stable distribution-matching problem rather than exact numerical regression. The paper reports state-of-the-art results on the SLICE-3D, HOP, and EyePACS benchmarks, with the out-of-domain fine-tuning AUC of 0.944 against 0.911 for the best semantic-agnostic baseline, and the in-domain linear-probe AUC of 0.984 against the baseline's full fine-tuning 0.971.

Load-bearing premise

The method's core bet is that feature-importance scores derived without medical labels—by fitting a general pretrained tabular model to pseudo-labels from the largest axis of variation in the data—point to the features clinicians would actually find diagnostic; if that bet fails, the adaptive masking corrupts the wrong features and the reported gains may be a regularization effect rather than semantic awareness.

Editorial extensions

If this is right

Clinical archives that contain images paired with structured electronic health records, but no expert labels, can be pre-trained with this method and then fine-tuned for diagnostic tasks.
The out-of-domain improvement is the strongest support for the claim that the learned representations are robust to population shift, which is a major barrier to deploying medical models across institutions.
Linear probing reaches 0.984 AUC on in-domain dermatology data, close to full fine-tuning performance, suggesting downstream deployment can use lightweight classifiers.
The retinal-imaging result supports the claim that the two-dimensional structural hypothesis generalizes beyond dermatology to other clinical specialties.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The importance proxy is never validated against diagnostic labels, so a natural next step is to compare the learned importance vector with clinician rankings or with leave-one-feature-out ablations on labeled data; divergence would weaken the semantic reading of the gains.
Because the soft-label module is mathematically an ordinal regression with triangular targets, using clinically predefined ordinal categories rather than quantile bins could improve calibration at clinically critical thresholds without changing the method.
An end-to-end learned importance estimator trained jointly with the objective could replace the frozen offline prior, making the masking curriculum adaptive during pre-training; the paper leaves this unexplored.
The same treatment of continuity and discreteness could be applied to other structured inputs, such as continuous imaging biomarkers or longitudinal lab values paired with images, which would test whether the mechanism is general.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, the core idea is genuinely useful: a self-supervised image-tabular pretraining objective that corrupts features according to a data-driven importance score and reconstructs them as soft distributions over quantile bins instead of continuous regression. The ablations across SLICE-3D, HOP, and EyePACS back this up, and the gains are consistent. Second, the "semantic-aware" interpretation is the weak joint. The importance score comes from attention weights of a frozen TabPFN fitted to pseudo-labels from the first principal component of the tabular features. There is no validation that this score tracks clinical importance; it might just track variance. That leaves the central claim about prioritising salient features under-supported, and the OOD improvement over uniform masking (0.930 vs 0.944 AUC) could be a curriculum/regularisation effect rather than evidence of semantic awareness.

What is actually new: the combination of adaptive importance masking with soft-label discretized reconstruction is not in TIP, CITab, or MMCL. Both pieces build on known machinery, but the integration is sensible and the design choices are motivated. The paper also does the right things on evaluation: all baselines reproduced under identical settings, a genuine geographical OOD split, a second private dataset, and a cross-specialty dataset. Code link is provided, though no commit hash; EyePACS tabular data is promised. The math is straightforward and the core objectives are clearly defined. Citation pattern is reasonable; self-citations point to datasets and prior vision-language work, not to the central objective.

Soft spots, in proportion. Minor-to-moderate: no error bars, confidence intervals, or significance tests; several ablation margins are small (0.930 vs 0.944, 0.938 vs 0.942). The claim that triangle soft labels "mathematically preserve ordinal relations" is overstated—soft-label mass proportional to distance does encode monotonicity, but no proof is given and "mathematically preserves" is too strong. Moderate: the HOP dataset is private, so those numbers cannot be independently checked. Load-bearing: the feature importance proxy. The stress-test note is right that a PCA/TabPFN-derived score has no established link to clinical diagnostic importance. The authors acknowledge the lack of theoretical guarantees and lean on TabPFN's prior, which is an appeal, not evidence. I would not call this fatal: even if the score is a variance-based proxy, adaptive masking can still help by creating a structured curriculum, and the gains are real on held-out data. But the semantic-awareness narrative needs a direct check—e.g., compare estimated importance with a clinician-defined ranking or with a supervised importance from gradient-based attribution on downstream labels, if only as validation.

Who this is for: anyone working on multimodal medical learning with tabular plus imaging. The paper deserves serious refereeing. I would send it out, asking for error bars, code/data release, and a validation or softening of the importance-proxy and ordinality claims. It is a conditional accept, not a reject.

Referee Report

3 major / 4 minor

Summary. The paper proposes AID, a self-supervised pre-training framework for paired medical images and tabular data. It introduces two components: an Importance-Aware Adaptive Masking scheme that estimates per-feature importance offline by fitting a frozen TabPFN v2 model to pseudo-labels derived from a median split of the first principal component and then modulates masking rates via Eq. (1), and a Soft-Label Discretized Module that replaces continuous regression with a triangle-kernel-smoothed distribution-matching objective for masked tabular reconstruction. The model is pre-trained with a composite loss of ITC, ITM, and DR, and is evaluated on SLICE-3D (in-domain and geographically isolated out-of-domain splits), the private HOP dataset, and EyePACS. The authors report state-of-the-art results, including a SLICE-3D OOD fine-tuning AUC of 0.944 versus 0.911 for the strongest prior method TIP, and provide ablations that isolate the contribution of each proposed module.

Significance. If the results hold, the paper makes a useful and credible contribution to medical image-tabular pre-training. Its strengths are the scale and diversity of the evaluation (a 400,000-image dermatology dataset with a geographically isolated OOD test set, a private external dermatology dataset, and a cross-specialty ophthalmology benchmark), the release of code, and the consistent ablation evidence that both proposed modules contribute to the reported gains. The claimed pre-training objective does not use downstream diagnostic labels, which is a genuine practical advantage. However, the significance is moderated by two issues: all results are reported without error bars or significance tests, and the 'semantic awareness' claim rests on an unvalidated feature-importance proxy. The paper is therefore a strong candidate for major revision rather than acceptance in its current form.

major comments (3)

[Section 2.2, Eq. (1)] The feature importance vector s_j is the sole input to the adaptive masking curriculum, yet it is never validated. It is derived from median-split pseudo-labels on the first principal component and attention weights of a frozen TabPFN v2 model, and the paper provides no evidence that this ranking corresponds to diagnostic salience. A control experiment using a deliberately arbitrary or reversed feature ranking would be needed to rule out the alternative explanation that any non-uniform masking schedule acts as a regularizer and produces the observed OOD gain (0.944 vs 0.930 in Table 2). This is the most load-bearing assumption in the paper, and it is not tested.
[Tables 1 and 2] All results are reported as single numbers with no error bars, confidence intervals, or significance tests. Several comparisons that the ablation narrative depends on are small (e.g., OOD FT AUC 0.944 vs 0.940 for the Gaussian kernel, and ID pAUC 0.192 vs 0.187 in Table 2). Without repeated runs with different seeds or statistical testing, the claimed SOTA margins over TIP and the module-level contributions are not fully established.
[Section 2.3] The statement that the triangle kernel 'mathematically preserves the ordinal relationships' of clinical measurements is not supported by any formal argument. The construction guarantees that the target distribution has support on at most two adjacent bins, but it does not by itself establish that the KL objective or the learned representations preserve ordinality. A theorem or a precise statement of the preserved property is needed; otherwise this claim should be softened to an empirical one.

minor comments (4)

[Figure 2] Figure 2 is extremely dense and the text is very small; it should be redrawn with larger fonts and clearer separation of the loss modules for readability.
[Table 1] The header 'LP/FT' is ambiguous for the supervised methods, which appear with a single value per metric; please clarify whether linear probing is not applicable to these methods or whether the single value is the fine-tuning result.
[Section 3.1] The statement that 'all hyperparameter settings were empirically established through extensive validation experiments' should be accompanied by the search ranges and the selected values for r_base, alpha, r_max, and B; currently only the final values are reported.
[Abstract and Section 1] The term 'semantic-aware' is used to describe a data-driven, label-free importance estimate and a binning scheme; consider defining the intended scope of 'semantic' more explicitly to avoid overclaiming in light of the unvalidated importance proxy.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AID's importance scores, masking rates, and soft-label targets are all constructed without downstream diagnostic labels, and the reported gains are evaluated on external holdouts.

full rationale

The derivation chain is self-contained and not circular. The importance vector s_j is derived offline in Sec. 2.2 from PCA median-split pseudo-labels plus frozen TabPFN v2 attention, with no downstream labels used ('without accessing actual diagnostic targets' and 'labels-free'). The masking rate r_j is then a deterministic function of s_j (Eq. 1), and the DR loss (Eq. 2) reconstructs masked values against quantile-derived soft-label targets. None of these quantities are defined in terms of the downstream AUC/pAUC metrics; the target task labels enter only at fine-tuning time, which is standard practice. The paper's central claims are empirically tested on externally held-out SLICE-3D OOD, HOP, and EyePACS sets against reproduced baselines, and the ablations (Table 2) show that removing components changes results, so the outcome is not forced by construction. The self-citations (e.g., SLICE-3D dataset papers, prior dermatology VLP papers, and the authors' own baselines) supply datasets and background, not the load-bearing pre-training objective. The main weakness—that TabPFN attention fitted to PCA pseudo-labels may not reflect clinical importance—is a correctness/validity risk about an unvalidated proxy, not a circularity, because the proxy is independent of the target result and could in principle be wrong.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central method depends on two modeling assumptions (the PCA-plus-TabPFN importance proxy and the two-dimensional structure hypothesis) and four tuned hyperparameters. It introduces no new physical or domain entities. The assumptions are flagged because they are not validated against diagnostic ground truth within the paper.

free parameters (4)

r_base = 0.1
Base masking probability in Equation (1); tuned on validation and controls the overall corruption level.
alpha = 0.7
Sensitivity of the masking rate to the feature importance score in Equation (1); tuned on validation.
r_max = 0.7
Upper bound on the masking rate in Equation (1); tuned on validation and set equal to alpha.
B = 50
Number of quantile bins per continuous feature in the soft-label discretized module; tuned on validation.

assumptions (4)

ad hoc to paper PCA first-principal-component median split is a valid label-free target for eliciting feature importance.
Section 2.2 uses this split to fit TabPFN v2; the link between PCA structure and diagnostic importance is not established.
domain assumption Attention weights of a frozen TabPFN v2 model, fit to pseudo-labels, provide a reliable proxy for true clinical feature importance.
Section 2.2 relies on this proxy to set masking rates; no validation against diagnostic labels is shown.
domain assumption Medical tabular data has a two-dimensional structure: inter-feature importance hierarchy and intra-feature continuity-discreteness duality.
Section 2.1 posits this as the design principle that motivates both modules.
domain assumption Quantile bin boundaries from the training distribution remain meaningful across distribution shifts.
Section 2.3 defines bins on training data, then applies the same bins during OOD evaluation and HOP/EyePACS fine-tuning.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training." pith.science (2026). https://pith.science/paper/65SERJAJ

@misc{pith2026260810522,
  author       = {Pith},
  title        = {Pith review of: Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/65SERJAJ}},
  note         = {Machine review of arXiv:2608.10522}
}

read the original abstract

While vision-language models dominate medical representation learning, unstructured text lacks the dense, quantitative diagnostic phenotypes inherent in structured clinical tables. However, existing multimodal pre-training methods underutilize this potential due to semantic-agnostic designs that treat tabular inputs as flat vectors and employ unstable continuous regression objectives. To overcome this, we propose a novel semantic-aware framework explicitly modeling the intrinsic two-dimensional structure of tabular data. First, addressing the inter-feature hierarchy of varying diagnostic importance, we introduce Importance-Aware Adaptive Masking to construct a label-free curriculum prioritizing salient features. Second, addressing the intra-feature continuity-discreteness duality, we propose a Soft-Label Discretized Module that replaces unstable numerical regression with stable distribution matching, thereby mathematically preserving ordinal relationships. Extensive experiments across large-scale dermatology (SLICE-3D, HOP) and ophthalmology (EyePACS) datasets establish a new state-of-the-art (SOTA), demonstrating exceptional robustness and cross-domain generalizability.

Figures

Figures reproduced from arXiv: 2608.10522 by the authors.

**Figure 1.** Limitations of Current Semantic-Agnostic Designs and Motivation for the TwoDimensional Hierarchical Structure. need for self-supervised learning methods capable of harnessing unlabeled imagetabular data to acquire robust representations for downstream clinical tasks. Despite this urgent need, existing multimodal pre-training methods are limited by a semantic-agnostic design[7, 11]. As in [PITH_FULL_IMAGE:figures… view at source ↗

**Figure 2.** Overall Architecture of the Proposed AID Framework. targets, this module constructs a label leakage-free curriculum prioritizing diagnostically salient features. To address the intra-feature duality, an end-to-end soft-label discretized module replaces unstable numerical regression with a stable distribution matching objective. Crucially, by utilizing a triangle kernel to allocate probability mass across adjacent … view at source ↗

**Figure 3.** Qualitative comparison between the SOTA baseline (top row) and the proposed framework (bottom row). (a) and (b) visualize t-SNE projections for color feature (tbp_lv_H) reconstructions, while (c) and (d) show t-SNE projections for shape feature (tbp_lv_L) reconstructions. (e) and (f) present self-attention weight heatmaps from the tabular pathway of the multimodal encoder. creates heavily skewed semantic targets. … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 13 canonical work pages

[1]

arXiv preprint arXiv:2106.15147 (2021)

Bahri,D.,Jiang,H.,Tay,Y.,Metzler,D.:Scarf:Self-supervisedcontrastivelearning using random feature corruption. arXiv preprint arXiv:2106.15147 (2021)

arXiv 2021
[2]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Bhat, S.F., Alhashim, I., Wonka, P.: Adabins: Depth estimation using adaptive bins. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4009–4018 (2021)

2021
[3]

Nature562(7726), 203–209 (2018)

Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L.T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al.: The uk biobank resource with deep phenotyping and genomic data. Nature562(7726), 203–209 (2018)

work page 2018
[4]

In: International conference on machine learning

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PmLR (2020)

2020
[5]

In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers)

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidi- rectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). pp. 4171–4186 (2019)

2019
[6]

arXiv preprint arXiv:2010.11929 (2020)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

arXiv 2020
[7]

In: European Con- ference on Computer Vision

Du, S., Zheng, S., Wang, Y., Bai, W., O’Regan, D.P., Qin, C.: Tip: Tabular-image pre-training for multimodal classification with incomplete data. In: European Con- ference on Computer Vision. pp. 478–496. Springer (2024)

work page 2024
[8]

https://kaggle.com/competitions/diabetic-retinopathy-detection (2015), kaggle

Dugas, E., Jared, Jorge, Cukierski, W.: Diabetic retinopathy detection. https://kaggle.com/competitions/diabetic-retinopathy-detection (2015), kaggle

work page 2015

Show all 25 references

[9]

arXiv preprint arXiv:2512.14026 (2025)

Fu, Y., Zhao, Y., Zeng, Z., Chen, C., Jin, Y.: Unleashing the power of image- tabular self-supervised learning via breaking cross-tabular barriers. arXiv preprint arXiv:2512.14026 (2025)

2025
[10]

Advances in neural information processing systems34, 18932–18943 (2021)

Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. Advances in neural information processing systems34, 18932–18943 (2021)

2021
[11]

In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition

Hager, P., Menten, M., Rueckert, D.: Best of both worlds: Multimodal contrastive learning with tabular and imaging data. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition. pp. 23924–23935 (2023) 10 Y.Liu et al

2023
[12]

Nature637(8045), 319–326 (2025)

Hollmann, N., Müller, S., Purucker, L., Krishnakumar, A., Körfer, M., Hoo, S.B., Schirrmeister, R.T., Hutter, F.: Accurate predictions on small data with a tabular foundation model. Nature637(8045), 319–326 (2025)

2025
[13]

NPJ digital medicine3(1), 136 (2020)

Huang, S.C., Pareek, A., Seyyedi, S., Banerjee, I., Lungren, M.P.: Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ digital medicine3(1), 136 (2020)

2020
[14]

Scientific Data11(1), 884 (2024)

Kurtansky, N.R., D’Alessandro, B.M., Gillis, M.C., Betz-Stablein, B., Cerminara, S.E., Garcia, R., Girundi, M.A., Goessinger, E.V., Gottfrois, P., Guitera, P., et al.: The slice-3d dataset: 400,000 skin lesion image crops extracted from 3d tbp for skin cancer detection. Scient...

2024
[15]

npj Digital Medicine8(1), 708 (2025)

Kurtansky, N.R., Gillis, M.C., Codella, N.C., D’Alessandro, B.M., Ge, Z., Gui- tera, P., Halpern, A.C., Kittler, H., Malvehy, J., Liopyris, K., et al.: Automated triage of cancer-suspicious skin lesions with 3d total-body photography. npj Digital Medicine8(1), 708 (2025)

2025
[16]

Bioinformatics36(4), 1234–1240 (2020)

Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C.H., Kang, J.: Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics36(4), 1234–1240 (2020)

2020
[17]

arXiv preprint arXiv:2512.03445 (2025)

Li, X., Yan, S., Liu, Y., Soyer, H.P., Janda, M., Mar, V., Ge, Z.: Multi-aspect knowledge-enhanced medical vision-language pretraining with multi-agent data generation. arXiv preprint arXiv:2512.03445 (2025)

2025
[18]

Advances in neural information pro- cessing systems31(2018)

Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. Advances in neural information pro- cessing systems31(2018)

2018
[19]

arXiv preprint arXiv:2508.10104 (2025)

Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khali- dov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al.: Dinov3. arXiv preprint arXiv:2508.10104 (2025)

2025 arXiv
[20]

arXiv preprint arXiv:2106.01342 (2021)

Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., Goldstein, T.: Saint: Improved neural networks for tabular data via row attention and contrastive pre- training. arXiv preprint arXiv:2106.01342 (2021)

2021 arXiv
[21]

NeuroImage260, 119505 (2022)

Wolf, T.N., Pölsterl, S., Wachinger, C., Initiative, A.D.N., et al.: Daft: A universal module to interweave tabular data and 3d images in cnns. NeuroImage260, 119505 (2022)

2022
[22]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Yan,S.,Hu,M.,Jiang,Y.,Li,X.,Fei,H.,Tschandl,P.,Kittler,H.,Ge,Z.:Derm1m: A million-scale vision-language dataset aligned with clinical ontology knowledge for dermatology. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 12681–12690 (2025)

2025
[23]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Yan, S., Li, X., Hu, M., Jiang, Y., Yu, Z., Ge, Z.: Make: Multi-aspect knowledge- enhanced vision-language pretraining for zero-shot dermatological assessment. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 369–379. Springer (2025)

2025
[24]

Yan, S., Li, X., Mo, D., Tschandl, P., Jiang, Y., Wang, Z., Hu, M., Ju, L., Alonso, C., Zheng, Y., et al.: A vision-language foundation model for zero-shot clinical collaboration and automated concept discovery in dermatology (2026)

2026
[25]

Translational vision science & technol- ogy11(7), 12–12 (2022)

Zhou, Y., Wagner, S.K., Chia, M.A., Zhao, A., Xu, M., Struyven, R., Alexan- der, D.C., Keane, P.A., et al.: Automorph: automated retinal vascular morphology quantification via a deep learning pipeline. Translational vision science & technol- ogy11(7), 12–12 (2022)

2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

arXiv preprint arXiv:2106.15147 (2021)

Bahri,D.,Jiang,H.,Tay,Y.,Metzler,D.:Scarf:Self-supervisedcontrastivelearning using random feature corruption. arXiv preprint arXiv:2106.15147 (2021)

arXiv 2021

[2] [2]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Bhat, S.F., Alhashim, I., Wonka, P.: Adabins: Depth estimation using adaptive bins. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4009–4018 (2021)

2021

[3] [3]

Nature562(7726), 203–209 (2018)

Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L.T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al.: The uk biobank resource with deep phenotyping and genomic data. Nature562(7726), 203–209 (2018)

work page 2018

[4] [4]

In: International conference on machine learning

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PmLR (2020)

2020

[5] [5]

In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers)

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidi- rectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). pp. 4171–4186 (2019)

2019

[6] [6]

arXiv preprint arXiv:2010.11929 (2020)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

arXiv 2020

[7] [7]

In: European Con- ference on Computer Vision

Du, S., Zheng, S., Wang, Y., Bai, W., O’Regan, D.P., Qin, C.: Tip: Tabular-image pre-training for multimodal classification with incomplete data. In: European Con- ference on Computer Vision. pp. 478–496. Springer (2024)

work page 2024

[8] [8]

https://kaggle.com/competitions/diabetic-retinopathy-detection (2015), kaggle

Dugas, E., Jared, Jorge, Cukierski, W.: Diabetic retinopathy detection. https://kaggle.com/competitions/diabetic-retinopathy-detection (2015), kaggle

work page 2015