{"id":"396fd260-24cd-4932-a7ea-e81b6a1229bc","arxiv_id":"2501.00057","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A pre-trained image ViT encoder, fed with learned projections of tabular rows, beats tree ensembles and tabular deep learning baselines on average across 23 small datasets.","lead":"VisTabNet adapts a vision transformer pre-trained on images to classify small tabular datasets by projecting table rows into image-patch-like embeddings. On 23 small UCI benchmarks it reports the best average Matthews correlation coefficient, with the largest gains in few-shot settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own transferability test is missing: Table 2 compares pre-trained ViT against 'No ViT encoder,' not against a from-scratch ViT, so the central cross-modal transfer claim is not established.","rationale":"The paper's strongest claim is that an ImageNet-pretrained ViT can be reused for tabular data and beats baselines. The load-bearing condition is that the pre-trained weights, not just the transformer architecture, cause the gains. The authors' own formal definition in Section 3.2 makes this condition testable: same architecture, pre-trained versus from-scratch. The evidence in Table 2 does not implement this comparison. The 'No ViT encoder' condition is an architecture ablation, not an initialization control, so it cannot tell us whether pretraining helps. This is an internal gap in the argument, not a disagreement with external consensus. The benchmark's small margins and lack of significance tests are secondary; even if the mean MCC advantage were robust, the mechanism would remain untested without the from-scratch baseline. Because the missing experiment is clearly specified and feasible, I keep the conditional verdict rather than reject. This agrees with the reader's weakest_assumption.","tokens_in":14083,"tokens_out":5312,"duration_ms":54162,"concrete_test":"Rerun the Table 2 protocol on the 5 analysis datasets (and ideally all 23 datasets in Table 1) with a fourth condition: identical VisTabNet architecture and hyperparameter search, but ViT encoder weights randomly initialized instead of ImageNet pre-trained, using the same train/validation/test splits, seeds, and optimization budget. Compute per-dataset MCC and mean rank for pre-trained versus from-scratch. If from-scratch ViT is within one standard deviation of the pre-trained version (or not significantly worse by a paired test), the central transferability claim is unsupported; if pre-trained clearly wins, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines transferability of a pre-trained network g_theta to task B as h'_phi composed with g_theta performing at least as well as h' composed with g trained from scratch using the same architecture. Section 3.3 explicitly says this property is shown for ViT in Table 2. However, Table 2's last column is 'No ViT encoder,' i.e. an adapter and classification head with the encoder removed entirely. That is a different architecture: removing the transformer body changes capacity, depth, and the attention mechanism, so any MCC difference cannot be attributed to the pre-trained initialization. The main benchmark (Table 1) and few-shot experiments (Figure 2) also compare only against other algorithms, never against a randomly initialized ViT with the same adapter. The paper's central claim that 'pre-trained image models can be transferred to solve tabular problems' requires exactly the pre-trained-versus-from-scratch comparison, and that comparison is absent. The proposed mechanism, 'forcing similarity between transformed tabular inputs and image patch embeddings' (Section 3.3, Conclusion), is also not implemented by any auxiliary loss; the adapter is trained with classification error only. Thus the observed gains could be due to the transformer adapter architecture rather than to cross-modal pretraining.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VisTabNet, a cross-modal transfer learning method that replaces the patch embedding layer of a pre-trained Vision Transformer (ViT) with an adaptation network of feed-forward projections. Each projection maps a tabular row into a vector in the ViT patch-embedding space, and the resulting sequence of views is processed by the frozen ViT encoder before a classification head. The method is benchmarked on 23 small tabular datasets (the text says 20) against random forests, gradient boosting, XGBoost, LightGBM, ResNet, Feature Transformer, and NODE, using double cross-validation, identical splits, and three repeats. The paper also reports few-shot experiments and ablations over backbone choice, fine-tuning strategy, depth of adapter/head, and the selection of ViT encoder layers. The central claim is that pre-trained image models can be transferred to tabular data and outperform both tree ensembles and tabular deep learning models on small datasets.","tokens_in":14423,"tokens_out":5217,"duration_ms":50471,"significance":"The idea of adapting a pre-trained vision transformer to tabular data is original and practically appealing, since it promises to reuse large-scale image representations without designing task-specific architectures or training from scratch. The experimental protocol has clear strengths: double cross-validation, identical train/validation/test splits across methods, hyperparameter optimization with PyHopper, three repeated runs, the use of MCC for class-imbalanced data, and a public code repository. If the central transferability claim were convincingly established, the result would extend transfer learning beyond same-domain feature reuse and would be of broad interest. However, the paper's own transferability definition is not tested by the provided experiments, because the pre-trained ViT is never compared against the same architecture trained from scratch; and the headline improvements over random forests are not shown to be statistically significant. The work therefore currently establishes a promising architecture and benchmark but not the cross-modal transfer phenomenon as defined in the paper.","major_comments":[{"comment":"The transferability definition in Section 3.2 requires that a pre-trained network g_theta with adapter/head performs at least as well as the same architecture trained from scratch. Table 2 instead compares VisTabNet with a 'No ViT encoder' variant, which removes the transformer body entirely, thereby changing the architecture (depth, attention mechanism, parameter count). Any performance difference between these two configurations cannot be attributed to the pre-trained initialization. Therefore the statement in Section 3.3 that 'this property holds in most cases for ViT (Table 2)' is not supported by the presented experiment. A baseline with a randomly initialized ViT encoder of the same architecture (same adapter and head, same training protocol) is needed to test the actual cross-modal transfer claim.","section":"Section 3.3, Table 2"},{"comment":"The paper repeatedly states that VisTabNet works by 'forcing similarity between transformed tabular inputs and the embeddings of image patches' (Section 3.3 and Conclusion). However, no similarity loss, distribution-alignment term, or any auxiliary objective is defined or trained; the adapter and head are trained exclusively with classification error, as described in Section 3.3 and the hyperparameter details in Appendix A. Consequently, the proposed mechanism is not implemented or tested. Either add an explicit alignment objective (and demonstrate its effect) or rephrase these statements to say that the adapter is designed to produce inputs in patch-embedding space, not that similarity is actively enforced.","section":"Section 3.3, Conclusion"},{"comment":"The headline claim of superiority rests on a mean MCC difference of 1.62 points over Random Forests (67.43 vs. 65.81) and a mean rank difference of 0.11 (3.93 vs. 4.04), with standard deviations that overlap across datasets. No significance test (e.g., Wilcoxon signed-rank test or paired permutation test over dataset means) is reported. Given the small magnitude of the difference and the variability visible in Table 1, the claim of overall superiority is not statistically supported. Add paired significance tests across the datasets and report p-values or bootstrap confidence intervals for both MCC differences and ranks.","section":"Section 4.1, Table 1"},{"comment":"The few-shot experiment claims that VisTabNet 'achieves significantly better scores' and 'consistently outperforms' other methods between 2 and 10 shots. The evidence is based on only 5 datasets, no error bars are shown, and no significance test is reported; moreover, for the 1-shot case RF and GB outperform VisTabNet. Report per-shot variability (e.g., across the three repeats or multiple seeds) and apply a paired test, or temper the claim to state an average improvement over the 2-10 shot range without the word 'significantly'.","section":"Section 4.1, Figure 2"}],"minor_comments":[{"comment":"The text states that 'The experiments were conducted on 20 datasets', but Table 4 lists 23 datasets. Please correct the count.","section":"Section 4.1, Appendix A"},{"comment":"The abstract says datasets with 'less than 1k samples', yet Statlog German has exactly 1000 samples. Use 'at most 1k' or adjust the dataset description.","section":"Abstract"},{"comment":"References [9] and [28] contain bracketed editorial annotations such as '99 citations (Semantic Scholar/arXiv) [2023-02-06]' and '561 citations (Semantic Scholar/arXiv) [2024-01-16]'. These appear to be leftover notes and should be removed.","section":"References"},{"comment":"There are several typographical errors, including 'paramterized', 'learnig rate', and 'V it' in the text around Section 4.2; a careful copyediting pass is needed.","section":"Section 4.2"},{"comment":"Unlike Table 1, Table 2 does not report standard deviations for the MCC values. Adding them would help assess whether the differences between fine-tuning strategies and backbones are meaningful.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a topic within the scope of the journal and the experimental setup is generally careful. The main concern is that the paper's central claim — cross-modal transferability of a pre-trained ViT — is not directly tested, since the ablation compares against removing the encoder rather than training the same architecture from scratch. This is fixable with an additional set of experiments, but it is load-bearing. I also recommend that the editor ask for statistical significance testing for the headline benchmark before considering acceptance. The 'forcing similarity' wording should be corrected to match what is actually implemented, otherwise readers may be misled about the mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the idea is genuinely new, and the benchmark is a cut above the usual tabular DL paper. But the paper's central claim—that a pre-trained ViT encoder transfers across modalities—is not actually tested by the experiments as run. The missing control is a from-scratch ViT with the same adapter; Table 2's \"No ViT encoder\" column removes the encoder entirely, which changes the architecture. That tells you the transformer backbone helps, not that pre-training helps. Section 3.2 defines transferability exactly as pre-trained vs from-scratch, and Section 3.3 says it is shown in Table 2, but Table 2 does not include that comparison. So the paper does not establish its own stated result.\n\nWhat is new and worth credit: the idea of projecting tabular rows into ViT patch-embedding space and running a frozen pre-trained ViT encoder over them is a fresh idea in the tabular transfer literature. The experimental protocol is careful: double cross-validation, identical splits across methods, hyperparameter optimization, three repeats, and code released. The ablations on fine-tuning, backbone choice, and layer ranges are useful practical information.\n\nThe soft spots are real but mostly addressable. The mean rank advantage over random forest (3.93 vs 4.04) is small, standard deviations overlap, and no significance test is reported. CatBoost, a standard strong baseline named in the intro, is not in the benchmark. The few-shot plot in Figure 2 has no error bars. And the \"forcing similarity\" mechanism described as the source of transferability is not implemented; the adapter is trained with plain classification error. That is a conceptual gap, but the empirical question—does a pre-trained ViT help at all on tabular data—remains interesting.\n\nWho this is for: researchers working on tabular deep learning or empirical transfer learning will get value, especially the ablation results. But the paper should not be cited as evidence that cross-modal image-to-table transfer works, because that comparison is missing. It deserves a serious referee: the idea and benchmark warrant engagement, but it needs a from-scratch ViT control, significance testing, and a more honest framing of what is shown. My recommendation: send it to review with a request for major revision.","headline":"The cross-modal transfer idea is fresh and the benchmark is solid, but the paper never runs the pre-trained-vs-from-scratch control its own definition requires, so the headline transferability claim is unsupported.","tokens_in":14913,"tokens_out":3215,"would_cite":false,"duration_ms":29896,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VisTabNet claims that a Vision Transformer encoder pre-trained on images, fed with learned projections of tabular rows, beats tree ensembles and deep tabular models on small tabular datasets.","keywords":["cross-modal transfer learning","tabular data","Vision Transformer","small tabular datasets","adapter network","few-shot learning","Matthews correlation coefficient","transferability"],"falsifier":"Train VisTabNet with the identical adapter and head but a randomly initialized ViT encoder, holding layer choice, learning rate, and tuning budget fixed; if the random-encoder version matches the pre-trained version's MCC across the 20 benchmark datasets, the pre-trained image weights are not what carries the result.","tokens_in":13916,"feed_emoji":"📊","tokens_out":9008,"duration_ms":81902,"temperature":0.7,"pith_summary":"VisTabNet tries to establish that a Vision Transformer encoder pre-trained on images can be transplanted across modalities to tabular data. The method replaces ViT's patch embedding layer with a small adapter network that projects each tabular row into several views in ViT's embedding space, then feeds those views through the frozen image-trained encoder to a classification head. On 20 small tabular datasets with fewer than 1,000 samples each, the paper reports the highest mean Matthews correlation coefficient, 67.43, edging out random forests by 1.62 points and the best deep tabular baseline, NODE, by 2.5 points. If the claim holds, pre-trained image models become reusable for tabular problems without designing a tabular architecture or paying the cost of training a large network from scratch.","feed_headline":"Image-pretrained Transformer tops tabular benchmarks","feed_subtitle":"VisTabNet adapts a frozen ViT encoder to tables, averaging 67.43 MCC on 20 small datasets.","key_machinery":"The carrying mechanism is the adaptation network $\\pi$, a set of $n$ small feed-forward projections that maps a tabular row into $n$ vectors in the same $D$-dimensional space ViT uses for image patch embeddings. These projected views play the role of patch embeddings, with a CLS token prepended, so the frozen image-pretrained Transformer Encoder can process tabular data unchanged. The mechanism is supposed to make tabular inputs and image patch inputs similar enough in representation space that the pre-trained attention layers transfer useful structure; conceptually the paper frames this as forcing similarity, while in practice the adapter is trained by classification error alone. The choice of which ViT layers to keep, the backbone size, and whether to fine-tune the encoder are analyzed as secondary controls on the same mechanism.","core_discovery":"On the paper's own terms, the central discovery is that transferability of a pre-trained network is not limited to early feature extractors and is not confined to the original data modality. The paper defines transferability as follows: a pre-trained encoder $g_\\theta$ is transferable to a downstream task if a new head $h'_\\phi$ composed with it performs at least as well as the same architecture trained from scratch. VisTabNet realizes this by replacing ViT's patch embedding layer $f$ with an adaptation network $\\pi = (\\pi_1,\\ldots,\\pi_n)$, where each $\\pi_i$ is a feed-forward map from the tabular input $x \\in \\mathbb{R}^M$ to a view $v_i \\in \\mathbb{R}^D$ analogous to a patch embedding. The sequence $[\\mathrm{CLS}, v_1, \\ldots, v_n]$ is then processed by the frozen pre-trained ViT encoder, and an MLP head produces the prediction. The experiments report the best mean MCC and best mean rank against random forests, gradient boosting, XGBoost, LightGBM, ResNet, Feature Transformer, and NODE across 20 datasets, and improved few-shot performance from 2 to 10 examples per class. The paper also reports that starting the encoder at an intermediate layer (around layer 5) improves average MCC over using all 12 layers, and that fully training all parameters at once degrades performance.","pith_inferences":["Beyond the paper, a decisive comparison would pit VisTabNet with pre-trained ViT weights against VisTabNet with a randomly initialized ViT encoder of the same architecture; the paper's ablations remove the encoder entirely but do not isolate pre-training as the cause.","Beyond the paper, if the gain comes from generic attention over learned views rather than image-specific features, then other sequence encoders, such as time-series or graph transformers, should transfer with the same adapter trick, which is a testable prediction.","Beyond the paper, the benchmark covers only small datasets; extending VisTabNet to larger tabular benchmarks with high-cardinality categorical features would show where tree ensembles regain the edge.","Beyond the paper, the paper's similarity-forcing language suggests an explicit distribution-matching loss between projected views and real patch embeddings; adding such a loss and measuring MCC changes would directly test the mechanism the paper invokes."],"forward_implications":["Cross-modal transfer is viable: an encoder trained on images can serve tabular tasks, so pre-trained model libraries are not locked to their original modality.","Middle layers of a pre-trained transformer are transferable, not just early feature extractors; the paper reports that starting from layer 5 of ViT outperforms using the full encoder.","Small tabular datasets can benefit from large pre-trained models with few trainable parameters, reducing both overfitting and training cost relative to deep models trained from scratch.","The same adapter idea extends in part to NLP encoders: BERT-based versions are promising, while the ViT-based version achieves the best mean rank.","The stable default is to freeze the encoder and train only the adapter and head; training all parameters at once clearly hurts performance."],"supporting_citations":[{"why":"Supplies the Vision Transformer architecture and the pre-trained image weights that VisTabNet adapts.","marker":"[7]"},{"why":"Provides the transferability criterion that Section 3.2 formalizes and the paper tests cross-modally.","marker":"[33]"},{"why":"Random Forests is the strongest shallow baseline that VisTabNet surpasses in average MCC.","marker":"[2]"},{"why":"XGBoost is one of the gradient-boosting baselines VisTabNet is benchmarked against.","marker":"[3]"},{"why":"Supplies the ResNet and Feature Transformer deep tabular baselines used in the comparison.","marker":"[9]"},{"why":"NODE is the best deep-learning baseline and the one VisTabNet beats by 2.5 mean MCC points.","marker":"[23]"}],"fun_headline_variants":["Image-pretrained ViT beats tabular baselines","Cross-modal transfer: ViT excels on tabular data","Frozen ViT encoder adapts to tabular wins","VisTabNet: vision transformer for tabular tasks","Transfer learning across modalities: ViT on tables"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a Transformer encoder trained on images keeps supplying useful representations when its inputs are learned projections of tabular rows, rather than the adapter and classification head doing all of the work.","fun_headline_variants_meta":{"raw":{"variants":["Image-pretrained ViT beats tabular baselines","Cross-modal transfer: ViT excels on tabular data","Frozen ViT encoder adapts to tabular wins","VisTabNet: vision transformer for tabular tasks","Transfer learning across modalities: ViT on tables"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000189,"raw_usage":{"total_tokens":1397,"prompt_tokens":1069,"completion_tokens":328,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":685,"completion_tokens_details":{"reasoning_tokens":250}},"tokens_in":685,"tokens_out":328,"duration_ms":4104,"temperature":1.0,"reasoning_tokens":250,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:29:58.916758+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train VisTabNet with the identical adapter and head but a randomly initialized ViT encoder, holding layer choice, learning rate, and tuning budget fixed; if the random-encoder version matches the pre-trained version's MCC across the 20 benchmark datasets, the pre-trained image weights are not what carries the result.","supporting_citations":[{"cited_title":"Yosinski, J","cited_arxiv_id":null,"evidence_quote":"Provides the transferability criterion that Section 3.2 formalizes and the paper tests cross-modally."},{"cited_title":"Breiman , Random forests, Machine Learning, 45 (2001), pp","cited_arxiv_id":null,"evidence_quote":"Random Forests is the strongest shallow baseline that VisTabNet surpasses in average MCC."},{"cited_title":"Chen and C","cited_arxiv_id":null,"evidence_quote":"XGBoost is one of the gradient-boosting baselines VisTabNet is benchmarked against."}],"review_version":1}