{"id":"9db55164-c90f-4eff-ba95-477b0cc071d9","arxiv_id":"2506.00813","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"TIME fuses frozen TabPFN tabular embeddings with image features and beats MLP and NCART baselines on five tabular-image datasets, including incomplete medical data.","lead":"This paper presents TIME, a multimodal framework that pairs a frozen TabPFN tabular encoder with a ResNet image encoder for tasks like medical classification and price prediction. It reports consistent gains over MLP and NCART baselines, and shows the approach handles missing tabular values without imputation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) lets training-set embeddings see the query's own label inside Dtrain, so target leakage into the frozen TabPFN features may inflate the reported wins over NCART/MLP; a leave-one-out embedding protocol is needed.","rationale":"The reader's weakest assumption correctly identifies the missingness-mechanism issue: the MCAR-only sensitivity analysis on Adoption does not establish that TabPFN's advantage over median imputation transfers to informative missingness in medical data. I agree that is a real external-validity concern, and the paper's Table A.3 gives only counts, not patterns. However, I find a more load-bearing concern in the primary comparison itself: Eq. (3) permits target-label leakage into training embeddings because Dtrain contains the query's own label, and TabPFN is an in-context transformer. This affects the headline 'TIME outperforms baselines' claim on complete/imputed data, before any missingness question arises. I credit the paper for clear tables, multiple fusion strategies, explicit dataset statistics, and for being transparent that code is only promised upon acceptance; I am not alleging misconduct, only an unstated covariate shift in the feature-generation protocol. The proposed leave-one-out test cleanly settles whether the leakage is exploitable. If it passes, the Table 1 claims stand and the remaining issues are the TIP baseline omission, the missingness-mechanism analysis, and the Painting frozen-setting overstatement. If it fails, the paper's central empirical comparison would need to be redone. Since the test has not yet been run, the CONDITIONAL verdict remains appropriate; the condition should explicitly include the embedding-protocol check.","tokens_in":12797,"tokens_out":10836,"duration_ms":120346,"concrete_test":"Reproduce the TIME rows of Table 1 under two embedding protocols: (a) the current Eq. (3), where each training row's own (X_i, y_i) remains in Dtrain; and (b) leave-one-out, where each training embedding is computed with Dtrain excluding that row's own label, while validation/test embeddings use the full Dtrain. If protocol (b) lowers TIME's mean accuracy/raises MSE by more than about one point on any dataset, or reverses the ordering against NCART/MLP, then the reported advantage is substantially leakage-driven and all comparisons must be rerun with matched train/test feature distributions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim in Section 4.2, Table 1, is that TIME beats NCART and MLP because frozen TabPFN provides better tabular embeddings. Eq. (3) defines E = f(X*, Dtrain) and the text explicitly says X* may belong to the training set. Since Dtrain contains (X_i, y_i), a training row's own label is present in the in-context set when its embedding is produced. TabPFN is a transformer trained for in-context prediction, so the resulting training embedding can encode the target label through attention to its support-set copy; at test time the query label is absent. The linear head in Eqs. (5)-(6) is then trained on feature vectors whose distribution differs from the test distribution, while MLP/NCART embeddings never contain the label. This covariate shift can inflate TIME's apparent advantage even if its representation is not genuinely better. The manuscript does not state that training embeddings are computed leave-one-out, so the comparison in Table 1 does not yet isolate the contribution of TabPFN's pretrained representation. A secondary internal inconsistency also weakens the universal phrasing: Section 4.2 claims superiority in both best and average scores, but Table 1 shows NCART-Max (16.69 MSE) beating TIME-DAFT (17.01 MSE) on Painting in the frozen setting. The leakage issue, however, is the load-bearing one because it potentially contaminates every dataset.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TIME, a multimodal framework that uses a frozen TabPFN as a tabular encoder and combines its embeddings with pretrained ResNet image features through several fusion strategies (concatenation, sum, max, DAFT). The method is evaluated on five datasets (Adoption, Breast Cancer, Covid-19, Skin Cancer, Painting) under fully tuned and frozen image-encoder settings, and on three medical datasets with real missing values. The main empirical claims are that TIME outperforms MLP and NCART baselines on complete data, and that training directly on raw incomplete data gives better results than median imputation. The paper also reports a random-masking sensitivity analysis on the Adoption dataset.","tokens_in":13085,"tokens_out":5260,"duration_ms":48993,"significance":"If the results hold, TIME would be a useful demonstration that a pretrained tabular foundation model can serve as a drop-in tabular encoder for multimodal tabular-image learning, while also providing native handling of missing values. The paper has concrete strengths: experiments are repeated over five seeds with standard deviations, multiple fusion strategies and both frozen/fine-tuned settings are considered, the raw-vs-imputed comparison directly tests a practical claim, and the code is promised for release upon acceptance. However, a potentially load-bearing target-leakage issue in the embedding protocol, the absence of the most relevant missingness-aware baseline (TIP), and the narrow sensitivity analysis make the current evidence conditional rather than conclusive.","major_comments":[{"comment":"The manuscript defines the tabular embedding as E = f(X*, Dtrain) and explicitly states that X* may belong to training, validation, or test sets. When X* is a training row, its own label y_i is present in Dtrain. Since TabPFN is a transformer trained for in-context prediction, the embedding for that row can attend to its own label in the support set and thereby encode the target value. At test time the query label is absent, so the linear head in Eqs. (5)–(6) is trained on feature vectors whose distribution differs from the test distribution, while the MLP and NCART baselines never expose the label in their tabular features. This covariate shift can inflate TIME's apparent advantage in Table 1 even if TabPFN's representation is not genuinely better. The paper does not state that training embeddings are computed leave-one-out; please specify the exact protocol (e.g., leave-one-out support construction or a held-out support set) and, if leakage is present, rerun the comparison.","section":"§3.1–3.2, Eq. (3)"},{"comment":"TIP [12] is cited in the related work as addressing missingness in downstream tabular data in a tabular-image setting, yet it is not included as a baseline in Table 2, and Section 2.2 claims to be 'the first to address tabular-image multimodal learning with missing values in a supervised setting.' Because the robustness evaluation in Table 2 compares TIME on raw data only against TIME on median-imputed data, the broader conclusion that TIME 'consistently outperforms competitive baselines' under missingness is not supported by the experiments. Please compare against TIP (or, if its code/data are unavailable, state this explicitly and justify the scope) and reconcile the novelty claim with the cited work.","section":"§2.2 and §4.3, Table 2"},{"comment":"The sensitivity analysis randomly masks tabular features on the Adoption dataset only, and the manuscript does not characterize the missingness mechanisms of the three medical datasets beyond the sample counts in Table A.3. The claim that TabPFN's native missingness handling is robust therefore rests on the untested assumption that random masking resembles real clinical missingness. Please report per-feature missing rates and, if feasible, evaluate under informative (e.g., outcome-dependent) missingness mechanisms or otherwise restrict the claim.","section":"§4.4 and Table A.3"}],"minor_comments":[{"comment":"The sentence 'TIME models outperform both NCART and MLP baselines, achieving superior results in terms of both individual best scores and average performance' is too strong: on Painting in the frozen setting, NCART-Max (16.69) beats the best TIME variant (TIME-DAFT, 17.01). Please qualify the claim to reflect the few settings where a baseline wins.","section":"§4.2, Table 1"},{"comment":"The labels F- and R- are used in the figures for imputed versus raw data, but the text reuses them in a way that is confusing (e.g., 'F-Cat under the frozen setting' refers to a value that in Figure 5 is actually the raw R-Cat result, 73.81%). Consider renaming the conditions (e.g., Imputed vs Raw) and aligning the text with the figure labels.","section":"§4.3 and Figures 4–5"},{"comment":"The caption says 'the bold denotes the best average performance' but the bold formatting appears to mark the best result for each dataset; please clarify the formatting convention.","section":"Table 2 caption"},{"comment":"The MLP and NCART baselines are trained only after median imputation; please state explicitly whether the same imputation was applied at test time and whether any hyperparameter tuning was performed for these baselines beyond the shared training schedule.","section":"§4.1"},{"comment":"The caption refers to 'the three dotted lines' for baselines, but the baseline labels in the figure legend are not clearly visible in the printed version; add explicit line labels or a legend key so the comparison is readable.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The main concern is whether the target-leakage issue can be resolved within a revision; because it affects all Table 1 results, the paper should not be accepted without the authors specifying the embedding protocol and, if leakage is confirmed, rerunning the experiments. The omission of TIP and the 'first' claim also need attention in revision. The comparison with NCART, the authors' own prior method, appears fair on its face; I do not see a circularity problem there. The paper's scope is appropriate for a submission in tabular-image multimodal learning, but the central claim currently rests on an unverified assumption about TabPFN's embedding behavior."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is simple and worth testing: use frozen TabPFN as a tabular encoder in an image-tabular pipeline, and let its native missing-value handling replace imputation. The paper runs a serious set of experiments, five seeds, standard deviations, four fusion strategies, on natural and medical datasets, and the results mostly support the claim that TabPFN embeddings help. That is a genuinely new application-level contribution, even if the fusion modules themselves are off-the-shelf.\n\nThe soft spots are real, though. The load-bearing one is Eq. (3). You compute embeddings as f(X*, Dtrain), and the text explicitly says X* may be a training row. TabPFN is an in-context predictor, so a training row's embedding can attend to its own label inside Dtrain. At test time the query label is absent. That means the linear head is trained on features that have a different distribution from the features it sees at test time, while the MLP and NCART baselines never see labels in their tabular features. The direction of the bias is not clear—it could plausibly hurt test accuracy by teaching the head to rely on a shortcut—but it certainly breaks the clean comparison. The authors need to compute training embeddings leave-one-out with respect to Dtrain to isolate what TabPFN's pretrained representation actually contributes.\n\nAlso missing is TIP, which is cited in the related work and is the most relevant baseline for incomplete tabular-image learning. Leaving it out substantially weakens the claim of robustness against prior art. The sensitivity analysis is MCAR masking on one natural-image dataset; it does not address the informative missingness that medical data often has, and the paper never analyzes the actual missingness patterns in its three medical datasets. There is also an internal overclaim: Section 4.2 says TIME is superior in best and average scores, but Table 1 shows NCART-Max beating TIME-DAFT on Painting in the frozen setting. Minor, but the phrasing needs adjusting.\n\nCredit where due: the paper is honest about TabPFN's size limits, reports code will be released only upon acceptance, and the empirical effort is well above the subfield average. The limitation section is candid.\n\nThis deserves a serious referee, but not acceptance as is. The leakage issue needs a leave-one-out protocol, TIP needs to be run, and the missingness analysis needs to face real missingness mechanisms. I would send it back for major revision.","headline":"Useful recipe for injecting frozen TabPFN into multimodal fusion, but the training-set embedding protocol leaks labels and the evaluation misses the most relevant baseline.","tokens_in":13580,"tokens_out":2621,"would_cite":false,"duration_ms":27401,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a frozen TabPFN tabular encoder, fused with image features, beats task-trained tabular encoders in multimodal learning and that raw incomplete tables beat median-imputed ones.","keywords":["multimodal learning","tabular-image fusion","TabPFN","missing values","tabular foundation model","medical imaging","robustness","pretrained encoder"],"falsifier":"A direct test is to take a medical dataset where missingness is known to be informative, such as lab results absent more often for severe cases, and compare TIME trained on raw incomplete tables against the same model trained on median-imputed tables. If the imputed version matches or beats the raw version, or if a strong imputation-based baseline does, then the claim that TabPFN's native missing-value handling is the source of robustness fails.","tokens_in":12597,"feed_emoji":"🩻","tokens_out":6637,"duration_ms":60208,"temperature":0.7,"pith_summary":"This paper tries to establish that tabular-image multimodal learning can be made more robust by borrowing a pretrained tabular foundation model, TabPFN, as a frozen encoder. It argues that this approach removes the need for task-specific tabular training and avoids median imputation, which can distort data distributions. The proposed TIME framework fuses TabPFN embeddings with pretrained image features and, in experiments on five datasets, outperforms MLP and NCART baselines under both complete and missing tabular inputs. A sympathetic reader would care because medical datasets are small, heterogeneous, and full of missing values, which is exactly the regime where pretrained tabular representations could help.","feed_headline":"Frozen TabPFN encoder lifts tabular-image models on missing data","feed_subtitle":"Fusing frozen TabPFN embeddings with image features beats MLP and NCART baselines, especially on incomplete medical tables.","key_machinery":"The load-bearing object is the frozen TabPFN encoder $f_{\\mathrm{TabPFN}}(X, D_{\\mathrm{train}})$: a transformer-based tabular foundation model pretrained on synthetic tasks with in-context learning, which maps each tabular row to a 192-dimensional embedding while conditioning on the whole training set and natively tolerating missing entries. This embedding is fused with a ResNet-50 image embedding by concatenation, element-wise sum, element-wise maximum, or a DAFT transform, and the fused vector feeds a linear classifier or regressor. Keeping TabPFN frozen is what preserves its zero-shot generalization and missing-value behavior; only the projection layers and, optionally, the image encoder are trained.","core_discovery":"On its own terms, the paper's central discovery is that a pretrained tabular foundation model can serve as a drop-in, frozen tabular encoder for multimodal learning: TIME, built by fusing TabPFN's 192-dimensional row embeddings with ResNet-50 image features, beats both a task-trained MLP and the NCART tree-based encoder on five datasets, and does so by larger margins on medical imaging data. The paper further reports that training TIME on raw, incomplete tabular data yields higher accuracy than training on median-imputed data across all three medical datasets, and that performance degrades only mildly when up to half the tabular entries are randomly masked. The claimed mechanism is that TabPFN's native missing-value handling and pretrained in-context representations transfer to the multimodal setting, so imputation, which can distort distributions, becomes unnecessary.","pith_inferences":["A natural extension is to swap TabPFN for any future tabular foundation model with native missing-value support; the paper's architecture does not depend on TabPFN-specific details beyond the embedding output, so its core claim would be testable as better tabular pretraining arrives.","Because TabPFN's embedding for one row depends on the entire training set via in-context conditioning, TIME's advantage may shrink as datasets approach TabPFN's stated limits of 10,000 samples, 500 features, and 10 classes; the paper acknowledges this constraint but does not test it.","The sensitivity analysis masks only the Adoption dataset, so the paper's inference about medical missingness would be stronger if random masking were applied to the medical datasets and compared with missingness-correlated masking, which would test whether native missing-value handling survives informative missingness."],"forward_implications":["Adding tabular information through TIME improves over a vision-only ResNet on all five datasets, so the paper's recipe benefits natural images, paintings, and medical images alike.","TIME's gains over MLP and NCART are largest on the three medical datasets, suggesting pretrained tabular embeddings matter most where data are small and heterogeneous.","Because raw incomplete inputs outperform median-imputed inputs in every medical dataset and fusion configuration, the paper implies that imputation is not just unnecessary but can be harmful in this pipeline.","Under random missingness up to 50%, TIME's accuracy stays near its complete-data level, with concatenation and max fusion the most resilient strategies.","The best fusion strategy depends on encoder and setting, with concatenation and DAFT broadly strong and sum fusion excelling with TabPFN under full fine-tuning."],"supporting_citations":[{"why":"It supplies the frozen TabPFN encoder with native missing-value handling and the 192-dimensional embeddings that TIME builds on.","marker":"[26]"},{"why":"It introduces the original TabPFN architecture and in-context learning paradigm that the later version extends and that motivates freezing.","marker":"[35]"},{"why":"It provides the pretrained ResNet-50 image backbone used to extract image embeddings in TIME.","marker":"[7]"},{"why":"It defines the NCART baseline, a tree-based tabular encoder that TIME is compared against and outperforms.","marker":"[20]"},{"why":"It supplies the MLP tabular encoder baseline and the contrastive multimodal setting that TIME contrasts with.","marker":"[13]"},{"why":"It contributes the DAFT fusion module, one of the four fusion strategies TIME evaluates.","marker":"[33]"},{"why":"It is the prior work on tabular-image pre-training with incomplete data that the paper extends by handling missingness in a supervised multimodal setting.","marker":"[12]"}],"fun_headline_variants":["TabPFN frozen encoder lifts multimodal models on incomplete data","Frozen TabPFN embeddings beat imputation in tabular-image fusion","TabPFN tabular encoder improves medical imaging models with missing data","TIME framework uses TabPFN to handle missing data in multimodal fusion","TabPFN powers robust tabular-image learning despite missing entries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The robustness conclusion relies on random missingness being a fair stand-in for the real missingness in medical tables, yet the paper never examines why values are missing in its medical datasets; if missingness carries information about the outcome, TabPFN's advantage over imputation may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["TabPFN frozen encoder lifts multimodal models on incomplete data","Frozen TabPFN embeddings beat imputation in tabular-image fusion","TabPFN tabular encoder improves medical imaging models with missing data","TIME framework uses TabPFN to handle missing data in multimodal fusion","TabPFN powers robust tabular-image learning despite missing entries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000517,"raw_usage":{"total_tokens":2493,"prompt_tokens":916,"completion_tokens":1577,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":1488}},"tokens_in":532,"tokens_out":1577,"duration_ms":10872,"temperature":1.0,"reasoning_tokens":1488,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:56:57.356155+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to take a medical dataset where missingness is known to be informative, such as lab results absent more often for severe cases, and compare TIME trained on raw incomplete tables against the same model trained on median-imputed tables. If the imputed version matches or beats the raw version, or if a strong imputation-based baseline does, then the claim that TabPFN's native missing-value handling is the source of robustness fails.","supporting_citations":[{"cited_title":"Hollmann, S","cited_arxiv_id":null,"evidence_quote":"It supplies the frozen TabPFN encoder with native missing-value handling and the 192-dimensional embeddings that TIME builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It defines the NCART baseline, a tree-based tabular encoder that TIME is compared against and outperforms."},{"cited_title":"Hager, M","cited_arxiv_id":null,"evidence_quote":"It supplies the MLP tabular encoder baseline and the contrastive multimodal setting that TIME contrasts with."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It contributes the DAFT fusion module, one of the four fusion strategies TIME evaluates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the prior work on tabular-image pre-training with incomplete data that the paper extends by handling missingness in a supervised multimodal setting."}],"review_version":1}