{"id":"7475180d-d1ed-44fb-8d8c-e996cb9f7d31","arxiv_id":"2608.10522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A semantic-aware image-tabular pre-training method, using importance-guided masking and soft-label discretized reconstruction, sets new state-of-the-art results on dermatology and ophthalmology benchmarks.","lead":"Medical AI models usually read images or text, not the organized tables of patient measurements that doctors actually use. This paper introduces a pre-training method that uses those tables more intelligently and reports better results on skin and eye disease datasets, especially on data from new patient groups.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unvalidated feature-importance proxy: AID's masking curriculum may reflect PCA/TabPFN variance structure rather than clinical salience, so OOD gains could be regularization, not semantic awareness.","rationale":"The reader's weakest assumption is the same one I would flag. The importance vector s_j is the only component that makes the masking curriculum 'semantic-aware'; without it, Equation (1) reduces to a generic corruption-rate schedule. The paper's own wording concedes that attention-based importance lacks theoretical guarantees, and no experiment separates the content of s_j from the fact of adaptive masking. The shuffled-importance control is the minimal experiment that would do so. I also considered the 'mathematically preserves ordinality' claim in Section 2.3; it is overstated for a KL/CE objective because categorical cross-entropy does not impose a distance metric on bins, but this is a presentation issue rather than the load-bearing mechanism, because the DR loss could still improve learning even if the ordinality claim is softened. Missing error bars are a reproducibility concern, but with test sets of 80k to 200k samples the reported gaps are likely real, so this does not threaten the central claim as much as the unvalidated proxy. The ablations in Table 2 are consistent and the architecture is coherent, so the paper deserves conditional acceptance, not rejection. The condition should be to validate s_j against a label-based importance measure or add a shuffled-importance control; if the control matches AID, the contribution should be reframed as adaptive masking without the clinical-semantics interpretation.","tokens_in":8572,"tokens_out":8667,"duration_ms":85351,"concrete_test":"Train a CatBoost (or logistic regression) classifier on the SLICE-3D ID training labels using the same tabular features, and compute a supervised permutation-importance ranking. Compare this ranking to AID's s_j via Spearman rank correlation. Independently, rerun the full AID pre-training with s_j randomly permuted across features, preserving the marginal distribution of mask rates, and compare OOD AUC/pAUC to the reported 0.944/0.162 and to the uniform-masking ablation 0.930. If shuffled s_j matches AID within 0.002, the ordering is irrelevant and the semantic-awareness claim fails; if it drops to near 0.930, the ordering is doing work, and the supervised correlation then determines whether that work is clinical or statistical.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 estimates feature importance s_j by fitting frozen TabPFN v2 attention to pseudo-labels from a median split of the first principal component of the tabular matrix. Equation (1) then sets the masking rate r_j = min(r_base + alpha * s_j, r_max). The entire semantic-aware claim rests on this s_j being a faithful proxy for diagnostic importance. The paper provides no validation of s_j against clinical labels, domain knowledge, or even a supervised importance ranking; the only evidence is the downstream gain of adaptive masking over uniform masking (Table 2, OOD AUC 0.930 vs 0.944). That gain could equally arise from a generic curriculum effect: masking some features more than others, regardless of which features, acts as a regularizer. The authors themselves note that attention-based importance lacks theoretical guarantees and rely on TabPFN v2's prior to 'mitigate' this, which is an appeal to an unexamined assumption. If s_j is dominated by the dominant variance direction, such as lesion size or color statistics rather than border or asymmetry morphology, the model preferentially corrupts clinically uninformative features while leaving diagnostically critical ones intact, making the reported robustness a property of the corruption schedule rather than of semantic awareness. Because the paper's central novelty is the importance-aware curriculum, this untested proxy is the most load-bearing assumption in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AID, a self-supervised pre-training framework for paired medical images and tabular data. It introduces two components: an Importance-Aware Adaptive Masking scheme that estimates per-feature importance offline by fitting a frozen TabPFN v2 model to pseudo-labels derived from a median split of the first principal component and then modulates masking rates via Eq. (1), and a Soft-Label Discretized Module that replaces continuous regression with a triangle-kernel-smoothed distribution-matching objective for masked tabular reconstruction. The model is pre-trained with a composite loss of ITC, ITM, and DR, and is evaluated on SLICE-3D (in-domain and geographically isolated out-of-domain splits), the private HOP dataset, and EyePACS. The authors report state-of-the-art results, including a SLICE-3D OOD fine-tuning AUC of 0.944 versus 0.911 for the strongest prior method TIP, and provide ablations that isolate the contribution of each proposed module.","tokens_in":8845,"tokens_out":4086,"duration_ms":40112,"significance":"If the results hold, the paper makes a useful and credible contribution to medical image-tabular pre-training. Its strengths are the scale and diversity of the evaluation (a 400,000-image dermatology dataset with a geographically isolated OOD test set, a private external dermatology dataset, and a cross-specialty ophthalmology benchmark), the release of code, and the consistent ablation evidence that both proposed modules contribute to the reported gains. The claimed pre-training objective does not use downstream diagnostic labels, which is a genuine practical advantage. However, the significance is moderated by two issues: all results are reported without error bars or significance tests, and the 'semantic awareness' claim rests on an unvalidated feature-importance proxy. The paper is therefore a strong candidate for major revision rather than acceptance in its current form.","major_comments":[{"comment":"The feature importance vector s_j is the sole input to the adaptive masking curriculum, yet it is never validated. It is derived from median-split pseudo-labels on the first principal component and attention weights of a frozen TabPFN v2 model, and the paper provides no evidence that this ranking corresponds to diagnostic salience. A control experiment using a deliberately arbitrary or reversed feature ranking would be needed to rule out the alternative explanation that any non-uniform masking schedule acts as a regularizer and produces the observed OOD gain (0.944 vs 0.930 in Table 2). This is the most load-bearing assumption in the paper, and it is not tested.","section":"Section 2.2, Eq. (1)"},{"comment":"All results are reported as single numbers with no error bars, confidence intervals, or significance tests. Several comparisons that the ablation narrative depends on are small (e.g., OOD FT AUC 0.944 vs 0.940 for the Gaussian kernel, and ID pAUC 0.192 vs 0.187 in Table 2). Without repeated runs with different seeds or statistical testing, the claimed SOTA margins over TIP and the module-level contributions are not fully established.","section":"Tables 1 and 2"},{"comment":"The statement that the triangle kernel 'mathematically preserves the ordinal relationships' of clinical measurements is not supported by any formal argument. The construction guarantees that the target distribution has support on at most two adjacent bins, but it does not by itself establish that the KL objective or the learned representations preserve ordinality. A theorem or a precise statement of the preserved property is needed; otherwise this claim should be softened to an empirical one.","section":"Section 2.3"}],"minor_comments":[{"comment":"Figure 2 is extremely dense and the text is very small; it should be redrawn with larger fonts and clearer separation of the loss modules for readability.","section":"Figure 2"},{"comment":"The header 'LP/FT' is ambiguous for the supervised methods, which appear with a single value per metric; please clarify whether linear probing is not applicable to these methods or whether the single value is the fine-tuning result.","section":"Table 1"},{"comment":"The statement that 'all hyperparameter settings were empirically established through extensive validation experiments' should be accompanied by the search ranges and the selected values for r_base, alpha, r_max, and B; currently only the final values are reported.","section":"Section 3.1"},{"comment":"The term 'semantic-aware' is used to describe a data-driven, label-free importance estimate and a binning scheme; consider defining the intended scope of 'semantic' more explicitly to avoid overclaiming in light of the unvalidated importance proxy.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's citation pattern is concentrated on the authors' group's earlier vision-language work (refs 17, 22, 23, 24), which is understandable given the topic and does not appear problematic. The main risk is that the central 'semantic-aware' claim is stronger than the current evidence: the importance proxy is unvalidated and the results lack uncertainty quantification. I would encourage requiring the authors to add a validation of the feature-importance ranking or a control experiment with a non-informative ranking, and to report repeated-run statistics, before the paper is accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the core idea is genuinely useful: a self-supervised image-tabular pretraining objective that corrupts features according to a data-driven importance score and reconstructs them as soft distributions over quantile bins instead of continuous regression. The ablations across SLICE-3D, HOP, and EyePACS back this up, and the gains are consistent. Second, the \"semantic-aware\" interpretation is the weak joint. The importance score comes from attention weights of a frozen TabPFN fitted to pseudo-labels from the first principal component of the tabular features. There is no validation that this score tracks clinical importance; it might just track variance. That leaves the central claim about prioritising salient features under-supported, and the OOD improvement over uniform masking (0.930 vs 0.944 AUC) could be a curriculum/regularisation effect rather than evidence of semantic awareness.\n\nWhat is actually new: the combination of adaptive importance masking with soft-label discretized reconstruction is not in TIP, CITab, or MMCL. Both pieces build on known machinery, but the integration is sensible and the design choices are motivated. The paper also does the right things on evaluation: all baselines reproduced under identical settings, a genuine geographical OOD split, a second private dataset, and a cross-specialty dataset. Code link is provided, though no commit hash; EyePACS tabular data is promised. The math is straightforward and the core objectives are clearly defined. Citation pattern is reasonable; self-citations point to datasets and prior vision-language work, not to the central objective.\n\nSoft spots, in proportion. Minor-to-moderate: no error bars, confidence intervals, or significance tests; several ablation margins are small (0.930 vs 0.944, 0.938 vs 0.942). The claim that triangle soft labels \"mathematically preserve ordinal relations\" is overstated—soft-label mass proportional to distance does encode monotonicity, but no proof is given and \"mathematically preserves\" is too strong. Moderate: the HOP dataset is private, so those numbers cannot be independently checked. Load-bearing: the feature importance proxy. The stress-test note is right that a PCA/TabPFN-derived score has no established link to clinical diagnostic importance. The authors acknowledge the lack of theoretical guarantees and lean on TabPFN's prior, which is an appeal, not evidence. I would not call this fatal: even if the score is a variance-based proxy, adaptive masking can still help by creating a structured curriculum, and the gains are real on held-out data. But the semantic-awareness narrative needs a direct check—e.g., compare estimated importance with a clinician-defined ranking or with a supervised importance from gradient-based attribution on downstream labels, if only as validation.\n\nWho this is for: anyone working on multimodal medical learning with tabular plus imaging. The paper deserves serious refereeing. I would send it out, asking for error bars, code/data release, and a validation or softening of the importance-proxy and ordinality claims. It is a conditional accept, not a reject.","headline":"Adaptive importance masking plus soft-label binning is a solid, reproducible advance in image-tabular pretraining with consistent gains, but the semantic-awareness story needs direct validation of its PCA/TabPFN importance proxy.","tokens_in":9383,"tokens_out":2683,"would_cite":true,"duration_ms":24554,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pre-training method that models medical tabular data's two-dimensional structure—feature importance and interval-based value semantics—achieves state-of-the-art results on three large clinical benchmarks.","keywords":["medical tabular data","multimodal pre-training","self-supervised learning","feature importance masking","soft-label discretization","dermatology","ophthalmology","image-tabular learning"],"falsifier":"Swap the learned importance vector for a random permutation of itself while holding the total masking budget fixed and re-run pre-training; if the out-of-domain AUC stays near 0.944, the importance mechanism is not the cause of the gain, whereas a drop toward the uniform-masking result of about 0.930 would show the importance prior is doing real work.","tokens_in":8386,"feed_emoji":"🩺","tokens_out":10073,"duration_ms":84906,"temperature":0.7,"pith_summary":"This paper introduces AID, a self-supervised pre-training framework for pairs of medical images and structured clinical tables. It claims that modeling two structural properties of tabular data—some features matter more than others, and exact continuous values should be reconstructed as soft distributions over ordinal intervals—produces representations that transfer better than treating tables as flat vectors. On a geographically separated out-of-domain test set of skin-lesion images, the method reaches a full fine-tuning ROC-AUC of 0.944 and a partial AUC of 0.162, versus 0.911 and 0.141 for the strongest previous method, and it reports similar gains on a second dermatology dataset and on retinal fundus images. The practical stake is that large collections of unlabeled image-plus-table clinical data could be converted into robust diagnostic models without expert annotations.","feed_headline":"Semantic-aware pre-training tops medical image-table benchmarks","feed_subtitle":"On unseen skin-lesion data, AUC rises to 0.944 from 0.911; gains carry to retinal imaging","key_machinery":"The machinery is the two-dimensional model of tabular structure that AID encodes, with two components. For inter-feature structure, an offline importance vector $s$ is extracted by fitting a frozen tabular foundation model to balanced binary pseudo-labels from the median split of the first principal component of the standardized feature matrix; each feature $j$ then gets masking rate $r_j = \\min(r_{\\mathrm{base}} + \\alpha s_j, r_{\\max})$. For intra-feature structure, the soft-label discretized module builds $B$ quantile bins per feature and maps a value $v$ falling in bin $c$ with boundaries $[b_{c-1}, b_c)$ to a probability vector over adjacent bins using the triangle kernel and intra-bin position $p = (v - b_{c-1})/(b_c - b_{c-1})$, which preserves ordinal relationships. Pre-training minimizes $\\mathcal{L}_{\\mathrm{AID}} = (\\mathcal{L}_{\\mathrm{ITC}} + \\mathcal{L}_{\\mathrm{ITM}} + \\mathcal{L}_{\\mathrm{DR}})/3$, combining image-tabular contrastive learning, image-tabular matching with hard negatives, and discretized reconstruction whose continuous part is a KL divergence between predicted and soft-target bin distributions.","core_discovery":"The central discovery is that the two-dimensional hierarchy of medical tabular data can be exploited during pre-training without using diagnostic labels. Inter-feature, the model estimates importance by fitting a frozen general-purpose tabular model to pseudo-labels derived from the first principal component of the standardized features, then masks high-importance features more often so reconstruction must draw on image evidence. Intra-feature, it converts each continuous value into a triangle-kernel soft label over quantile-based bins, so reconstruction becomes a stable distribution-matching problem rather than exact numerical regression. The paper reports state-of-the-art results on the SLICE-3D, HOP, and EyePACS benchmarks, with the out-of-domain fine-tuning AUC of 0.944 against 0.911 for the best semantic-agnostic baseline, and the in-domain linear-probe AUC of 0.984 against the baseline's full fine-tuning 0.971.","pith_inferences":["The importance proxy is never validated against diagnostic labels, so a natural next step is to compare the learned importance vector with clinician rankings or with leave-one-feature-out ablations on labeled data; divergence would weaken the semantic reading of the gains.","Because the soft-label module is mathematically an ordinal regression with triangular targets, using clinically predefined ordinal categories rather than quantile bins could improve calibration at clinically critical thresholds without changing the method.","An end-to-end learned importance estimator trained jointly with the objective could replace the frozen offline prior, making the masking curriculum adaptive during pre-training; the paper leaves this unexplored.","The same treatment of continuity and discreteness could be applied to other structured inputs, such as continuous imaging biomarkers or longitudinal lab values paired with images, which would test whether the mechanism is general."],"forward_implications":["Clinical archives that contain images paired with structured electronic health records, but no expert labels, can be pre-trained with this method and then fine-tuned for diagnostic tasks.","The out-of-domain improvement is the strongest support for the claim that the learned representations are robust to population shift, which is a major barrier to deploying medical models across institutions.","Linear probing reaches 0.984 AUC on in-domain dermatology data, close to full fine-tuning performance, suggesting downstream deployment can use lightweight classifiers.","The retinal-imaging result supports the claim that the two-dimensional structural hypothesis generalizes beyond dermatology to other clinical specialties."],"supporting_citations":[{"why":"Supplies the strongest semantic-agnostic pre-training baseline, the image-tabular contrastive and matching formulation that AID builds on, and the previous best out-of-domain numbers that AID surpasses.","marker":"[7]"},{"why":"Provides the frozen tabular foundation model whose self-attention weights, after fitting to principal-component pseudo-labels, generate the importance vector that drives adaptive masking.","marker":"[12]"},{"why":"Defines the primary dermatology dataset, including the geographically isolated out-of-domain test split responsible for the headline AUC of 0.944.","marker":"[14]"},{"why":"Defines the ophthalmology benchmark used to demonstrate cross-specialty transfer of the pre-training strategy.","marker":"[8]"},{"why":"Supplies a recent image-tabular self-supervised baseline that AID compares against on all three datasets.","marker":"[9]"},{"why":"Supplies a multimodal contrastive baseline and the hard-negative mining approach that the image-tabular matching objective adapts.","marker":"[11]"}],"fun_headline_variants":["Semantic-aware pretraining beats baselines on medical tables","Tabular-aware pre-training sets new SOTA in medical imaging","Exploiting table hierarchy lifts medical pre-training AUC","No labels needed: semantic-aware pretraining wins on medical data","Two-dimensional table modeling tops medical multimodal benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's core bet is that feature-importance scores derived without medical labels—by fitting a general pretrained tabular model to pseudo-labels from the largest axis of variation in the data—point to the features clinicians would actually find diagnostic; if that bet fails, the adaptive masking corrupts the wrong features and the reported gains may be a regularization effect rather than semantic awareness.","fun_headline_variants_meta":{"raw":{"variants":["Semantic-aware pretraining beats baselines on medical tables","Tabular-aware pre-training sets new SOTA in medical imaging","Exploiting table hierarchy lifts medical pre-training AUC","No labels needed: semantic-aware pretraining wins on medical data","Two-dimensional table modeling tops medical multimodal benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000475,"raw_usage":{"total_tokens":2334,"prompt_tokens":902,"completion_tokens":1432,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1353}},"tokens_in":518,"tokens_out":1432,"duration_ms":8543,"temperature":1.0,"reasoning_tokens":1353,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:18:29.285663+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Swap the learned importance vector for a random permutation of itself while holding the total masking budget fixed and re-run pre-training; if the out-of-domain AUC stays near 0.944, the importance mechanism is not the cause of the gain, whereas a drop toward the uniform-masking result of about 0.930 would show the importance prior is doing real work.","supporting_citations":[{"cited_title":"In: European Con- ference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Supplies the strongest semantic-agnostic pre-training baseline, the image-tabular contrastive and matching formulation that AID builds on, and the previous best out-of-domain numbers that AID surpasses."},{"cited_title":"Scientific Data11(1), 884 (2024)","cited_arxiv_id":null,"evidence_quote":"Defines the primary dermatology dataset, including the geographically isolated out-of-domain test split responsible for the headline AUC of 0.944."},{"cited_title":"https://kaggle.com/competitions/diabetic-retinopathy-detection (2015), kaggle","cited_arxiv_id":null,"evidence_quote":"Defines the ophthalmology benchmark used to demonstrate cross-specialty transfer of the pre-training strategy."},{"cited_title":"In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies a multimodal contrastive baseline and the hard-negative mining approach that the image-tabular matching objective adapts."}],"review_version":1}