{"id":"24e078f3-d02c-449b-9c90-fc6754ab068e","arxiv_id":"2502.10095","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"TCL, a CPU-trained contrastive encoder for tabular data, reports higher F1 than FT-Transformer and ResNet on most OOD classification datasets, but the result is not statistically supported.","lead":"The paper introduces TCL, a contrastive learning method for tabular data that runs on CPU, and claims it outperforms larger models such as FT-Transformer and ResNet on out-of-distribution classification tasks. It also presents a workflow for detecting OOD examples using OpenMax and temperature scaling, but the evaluation relies on manually chosen thresholds and lacks error bars.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The OOD test splits underpinning Table III are built from hand-picked OpenMax/TemperatureScaling thresholds (Section VI.B) and validated only by the expected performance drop, so the reported TCL advantage may be an artifact of the arbitrary split rather than a property of the model.","rationale":"The reader identified the same load-bearing weakness I would flag: the OOD test sets are constructed by hand-picked thresholds and validated only by the expected performance drop. This makes the central comparison in Table III conditional on an arbitrary and potentially circular split definition. I considered other possible objections—missing error bars, the invalid efficiency metric, and the SCARF baseline that reports 0.00 F1 on several datasets—but these are secondary or would only matter after the OOD split is shown to be meaningful. The TCL method itself is not internally inconsistent; the problem is the evaluation protocol. Because the abstract's superiority claim depends on these splits, the REJECT verdict should stand; I use UNCHANGED to indicate no change from the reader's assessment.","tokens_in":9609,"tokens_out":6193,"duration_ms":62580,"concrete_test":"Sweep each threshold in Section VI.B around the reported value (e.g., ±10% and ±25% of the score range) and recompute Table III for TCL, FT-Transformer, and ResNet with at least five seeds; also compare against an external OOD definition such as held-out class groups or covariate corruption. If TCL's rank reverses in either test, the headline claim is an artifact of the manually chosen split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim that TCL outperforms FT-Transformer and ResNet on OOD tabular classification rests entirely on the OOD test sets used in Table III. Section VI.B says the OOD thresholds were 'manually assigned by observing the graphs' and 'selecting a single point on a tail', and that the split was validated only by 'expecting to find that the performance decreased'. This is circular: the OOD set is defined as a low-confidence tail of a model-based score, so any model calibrated to the bulk is expected to do worse on it; a model may look better merely by handling low-confidence samples rather than by generalizing to a genuinely different distribution. No external ground truth for pood (Definition 2) is used, and no sensitivity analysis shows how Table III changes with small threshold perturbations. The absence of error bars and the likely broken SCARF baseline (0.00 F1 on Helena and Aloi) further weaken the comparison, but the split construction is the load-bearing issue: if the thresholds do not isolate true distribution shift, the headline comparison is not a valid test of OOD performance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TCL, a contrastive-learning representation method for tabular data, and claims that it outperforms existing deep models such as FT-Transformer and ResNet on out-of-distribution (OOD) classification while remaining competitive on regression, and that it does so with much lower training cost on CPU hardware. The experiments cover ten tabular datasets, use OpenMax and TemperatureScaling to split data into in-distribution and OOD sets, compare TCL with twelve baselines, and report F1/RMSE, training time, and a speed/accuracy trade-off metric. The paper also presents practical guidance for detecting and visualizing OOD data.","tokens_in":9993,"tokens_out":2736,"duration_ms":28604,"significance":"If the central claim were established, the contribution would be practically valuable: a CPU-trainable contrastive method that beats GPU-trained deep models on OOD tabular classification would lower the hardware barrier for many practitioners. The paper also makes a potentially useful effort to expose OOD detection and evaluation as an accessible workflow, and it provides a public code link. However, the evidence as presented does not support the headline claims. The OOD test sets are constructed from manually selected thresholds on model-based confidence scores without external ground truth for out-of-distribution samples, the main comparison table has no error bars or repeated runs, several baseline entries are missing, and at least one baseline (SCARF) reports F1=0.00 on multiple datasets, which suggests a broken implementation rather than a meaningful comparison. The significance is therefore conditional on a substantially stronger evaluation.","major_comments":[{"comment":"The OOD test splits are load-bearing for the paper's central claim, but their validity is not established. The thresholds are 'manually assigned by observing the graphs' and validated only by the expectation that performance decreases on the separated tail. This is circular: any model with calibrated confidence will perform worse on low-confidence samples, so TCL's better Table III scores may reflect handling of low-confidence points rather than generalization to a genuinely different distribution pood. The paper needs an independent definition of OOD, such as a principled covariate-shift construction or an external OOD benchmark, plus a sensitivity analysis showing that small threshold perturbations do not change the ranking.","section":"Section VI.B, Tables II and III"},{"comment":"The main performance comparison is reported without error bars, standard deviations, or information about the number of seeds or runs. Many entries are missing (for example, FT-Transformer is absent for CoverType, Yahoo, and Microsoft, and several models are missing on multiple datasets), while SCARF obtains F1=0.00 on Helena and Aloi and near-zero values on other datasets. A zero F1 on a multi-class dataset is not a plausible trained-model result and suggests a broken or misconfigured baseline. Without complete baselines and repeated trials, the claim that 'TCL outperforms other models' in Table III is not supported.","section":"Table III"},{"comment":"The training-time comparison is not apples-to-apples. TCL was trained on a CPU while the deep baselines were trained on an H100 GPU, and the time recorded for TCL is the time to 'stabilise their loss' (around 15 epochs), whereas the times for other models appear to include extensive hyperparameter tuning, with Yahoo and Microsoft requiring five days of tuning. The subsequent speed/accuracy trade-off in Table V therefore mixes tuning time, final training time, and different hardware, so the efficiency advantage is not cleanly quantified. The paper should report a fixed training protocol with matched epochs/budgets and separate tuning time from final training time.","section":"Section VII.C and Table IV"},{"comment":"The efficiency-accuracy objective is not precisely defined. The text first writes T = P/t, then defines T as 'max P_ood' over 'min t', and the regression adjustment uses 1/P without a clear derivation. Because Table V is used to support a major contribution, this metric needs a formal definition that is applied consistently across classification and regression, including how the reported trade-off values in Table V are computed from the F1/RMSE and time values in Tables III and IV.","section":"Section III.A, Eq. (5) and Definition 3"}],"minor_comments":[{"comment":"The notation is inconsistent: 'RestNet' and 'Resnet' appear alongside 'ResNet', 'GDBT' is used for gradient boosting, and 'Scarfc' should be 'SCARF'. The paper would benefit from a careful proofreading pass.","section":"Throughout"},{"comment":"The threshold column contains entries such as '1.46eE-03' and '-8.30eE-03', which appear to be typographical errors in scientific notation. The detection method used for each dataset is also only given as a one-letter code without an explanation in the table caption.","section":"Table II"},{"comment":"The simplified contrastive loss formula 'MSE([0], dot(xe1 · xe2))/T' is malformed as written. It should be stated in standard notation with the batch dimension and the temperature parameter clearly defined.","section":"Section IV.A, Eq. (9)"},{"comment":"The validation of OOD separation using linear regression is not described. The reader is not told what regression is performed, which variables are used, or what result would confirm that the separated set is genuinely out-of-distribution.","section":"Section VII.A, Table II"},{"comment":"The paper does not provide the actual threshold values or the sample sizes M and N for each dataset. Since the thresholds are a free parameter of the evaluation, they should be reported in full for reproducibility.","section":"Section VI.B"},{"comment":"The description of the augmentation and training procedure is incomplete: no noise distribution or magnitude is given, and the encoder/decoder architectures are not specified beyond 'one hidden layer and one normalisation layer'. This makes it difficult to reproduce TCL from the text alone.","section":"Section V"},{"comment":"The conclusion acknowledges limitations such as the need for continual learning and interpretability, but it does not acknowledge the more immediate issue that the OOD evaluation protocol itself is not validated. A discussion of the limitations of the manual thresholding approach would be appropriate.","section":"Section VIII"}],"recommendation":"reject","confidential_remarks":"The paper has a useful practical motivation and a simple, potentially efficient method, but the evaluation does not currently support the published claims. The OOD split construction is the central weakness: because the test set is defined by manually selected low-confidence thresholds, the reported 'OOD' advantage is not a valid demonstration of out-of-distribution generalization. This is compounded by missing baseline entries, a likely broken SCARF baseline, and incomparable training-time measurements. These issues are not localized presentation problems; they require redoing the experimental evaluation with a principled OOD setup, multiple seeds, and complete baselines. I would be open to reconsidering a substantially revised version that addresses these points, but in its current form the manuscript is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the method is a modest but reasonable extension of contrastive learning for tabular data, and the practical OOD-detection walkthrough is useful. But the headline claim — TCL beats FT-Transformer and ResNet on OOD classification while training on CPU — is not supported by the experiments as written, mainly because the OOD test sets are hand-picked tails and the SCARF baseline looks broken.\n\nWhat's actually new: TCL adapts the authors' own CFL method with full-matrix augmentation (as opposed to SubTab's slicing or SCARF's feature corruption) and a simplified dot-product contrastive loss. That is a small, plausible change. The paper also gives a step-by-step recipe for using OpenMax and temperature scaling to flag OOD samples, with visualizations, which genuinely helps practitioners who want to try this on their own data. Comparing across 10 datasets and a dozen baselines is a lot of legwork, and the CPU-training angle is worth taking seriously.\n\nNow the soft spots, in proportion. The OOD split construction is the load-bearing flaw. Section VI.B says thresholds were 'manually assigned by observing the graphs' and 'selecting a single point on a tail,' and the split was validated only by expecting a performance drop. That is close to circular: pick a low-confidence tail, and any calibrated model will do worse on it. The reported TCL advantage could simply mean TCL is less sensitive to low-confidence inputs, not that it generalizes to a genuinely different distribution. There is no external ground truth for pood, no threshold sensitivity analysis, and no error bars or seeds. With single runs, a 0.831 vs 0.782 F1 gap is not something I'd bet on. SCARF scoring 0.00 F1 on Helena and Aloi is a red flag that the baseline is misconfigured — you can't claim to beat a model that isn't running. FT-Transformer is missing on Covertype, Yahoo, and Microsoft, which further wobbles the headline comparison. And comparing CPU-trained TCL against GPU-trained baselines makes the speedup numbers hard to interpret, though the point that TCL is trainable on CPU is still valid. Regression results are honestly reported as competitive, not better; that part is fine.\n\nThe self-citation to CFL is just method provenance, not evidence of correctness, so I don't hold that against them.\n\nBottom line: this paper is for practitioners who want a lightweight tabular representation method and a working recipe for spotting OOD data. It deserves a serious referee, but not acceptance in its current form. The OOD evaluation needs to be redone with validated splits (or external OOD data), multiple seeds with variance, a fixed SCARF implementation, and ideally equal-hardware timing. That is a heavy but well-scoped revision. I wouldn't cite it in the current state.","headline":"TCL is a plausible lightweight tabular contrastive learner, but the OOD comparison is built on hand-picked splits and a broken baseline, so the headline claim is not supported.","tokens_in":10363,"tokens_out":3407,"would_cite":false,"duration_ms":34094,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CPU-trained contrastive model, TCL, is reported to beat GPU-heavy deep networks on out-of-distribution tabular classification while staying competitive on regression.","keywords":["out-of-distribution prediction","tabular data","contrastive learning","representation learning","CPU training","efficiency-accuracy trade-off","OpenMax","temperature scaling"],"falsifier":"Re-run the comparison with OOD test sets defined by a fixed, non-manual rule, for example holding out whole classes or taking the lowest-scoring 10% of rows by OpenMax score without inspecting model performance, and check whether TCL still leads on classification; if the margin shrinks, reverses, or the regression gap widens, the central claim is an artifact of the hand-picked thresholds.","tokens_in":9431,"feed_emoji":"⚡","tokens_out":11579,"duration_ms":86782,"temperature":0.7,"pith_summary":"The paper argues that a lightweight contrastive-learning model for tabular data, called TCL, can handle out-of-distribution (OOD) prediction better than much heavier deep models while running entirely on ordinary CPUs. The authors build OOD test sets by applying OpenMax and TemperatureScaling to separate low-confidence rows from each of ten tabular datasets, then compare twelve baselines including FT-Transformer and ResNet. They report that TCL achieves the best F1 on most classification datasets, competitive RMSE on regression datasets, and the highest speed/accuracy trade-off because its training time is far shorter. If true, this means strong OOD tabular performance is achievable without GPU access.","feed_headline":"CPU-trained TCL beats GPU deep nets on out-of-distribution data","feed_subtitle":"Training on CPU instead of GPU, TCL still tops FT-Transformer and ResNet on out-of-distribution tabular tasks.","key_machinery":"TCL (Tabular Contrastive Learning) is the central mechanism. The key design choice is full matrix augmentation: both augmented views contain the complete row of features instead of sliced subsets, and the simplified contrastive loss is a dot product between the two encoded views divided by a temperature, so the encoder is forced to make the representation of a row invariant to added noise. A reconstruction loss and a distance loss between encodings ensure that the representation is not degenerate.","core_discovery":"On the paper's own terms, the discovery is that a contrastive representation learner operating on entire tabular rows, not feature subsets, can learn features that transfer to out-of-distribution test data better than the features learned by heavier architectures. TCL's training creates two noisy copies of each row, pushes their encoded forms together with a temperature-scaled dot-product loss, and reconstructs the input, while at inference only the encoder remains. Across ten datasets the authors report that TCL has the best F1 on most classification benchmarks, is close to the best on regression, and achieves the best speed/accuracy trade-off because its CPU training time is a small fraction of the GPU training time of FT-Transformer and ResNet.","pith_inferences":["A natural next test is to feed TCL's learned representations into a tree-based classifier, since the paper only tests the encoder with its own supervised head.","The manual OOD threshold selection could be automated by choosing thresholds on a held-out calibration set; this would turn the paper's recipe into a standardised benchmark and reveal how sensitive the ranking is to the split.","Because the contrastive loss uses only dot-product similarity between two noisy views, adding corruptions beyond Gaussian noise, such as feature masking or swapping, might further improve OOD robustness."],"forward_implications":["If TCL's reported results hold, users with only CPU hardware can train a tabular model that outperforms GPU-trained FT-Transformer and ResNet on OOD classification tasks.","The short training time makes extensive hyperparameter search feasible on consumer hardware, which usually favours the model that can be tuned more thoroughly.","TCL's unsupervised contrastive step can use unlabeled rows ahead of the supervised head, so adding unlabeled data could strengthen the learned representation.","The paper's OOD detection recipe, OpenMax plus TemperatureScaling with manual thresholds, gives practitioners a direct way to construct OOD test sets from a single tabular dataset."],"supporting_citations":[{"why":"It supplies the contrastive-learning approach for tabular data that TCL adapts, and the paper positions TCL as a local, non-federated version of it.","marker":"[11]"},{"why":"It defines the contrastive-learning paradigm and the use of augmented views that TCL transfers from images to tabular rows.","marker":"[12]"},{"why":"It is a feature-subsetting contrastive baseline whose approach TCL contrasts with full matrix augmentation.","marker":"[13]"},{"why":"It is a random-feature-corruption contrastive baseline that TCL is compared against.","marker":"[29]"},{"why":"It is the reference for FT-Transformer, the deep transformer baseline that TCL claims to outperform on classification.","marker":"[22]"},{"why":"It is the reference for ResNet, the deep residual baseline that TCL claims to beat on classification.","marker":"[23]"},{"why":"It is one of the two OOD detectors used to split each dataset into in-distribution and out-of-distribution sets.","marker":"[7]"},{"why":"It is the other OOD detector, and together with OpenMax it provides the score plots from which the OOD thresholds are manually chosen.","marker":"[15]"},{"why":"It defines the speed/accuracy trade-off metric that underlies the efficiency comparison and TCL's reported efficiency advantage.","marker":"[14]"},{"why":"It provides the implementation of the OOD detection algorithms used in the experiments.","marker":"[17]"}],"fun_headline_variants":["CPU-only TCL tops GPU-trained models on out-of-distribution tabular tasks","TCL beats GPU heavyweights on OOD tabular data using only CPU","Lightweight TCL: CPU-trained model rivals GPU transformers on OOD","CPU-only contrastive learner beats GPU deep nets for OOD tabular data","TCL: one CPU model that outperforms GPU benchmarks on OOD tabular"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire comparison depends on the manually chosen thresholds from OpenMax and TemperatureScaling score plots; if those thresholds do not genuinely isolate out-of-distribution rows, the reported gap between TCL and the other models reflects the split rather than the model.","fun_headline_variants_meta":{"raw":{"variants":["CPU-only TCL tops GPU-trained models on out-of-distribution tabular tasks","TCL beats GPU heavyweights on OOD tabular data using only CPU","Lightweight TCL: CPU-trained model rivals GPU transformers on OOD","CPU-only contrastive learner beats GPU deep nets for OOD tabular data","TCL: one CPU model that outperforms GPU benchmarks on OOD tabular"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001063,"raw_usage":{"total_tokens":4426,"prompt_tokens":882,"completion_tokens":3544,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":3442}},"tokens_in":498,"tokens_out":3544,"duration_ms":20612,"temperature":1.0,"reasoning_tokens":3442,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:26:12.648343+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison with OOD test sets defined by a fixed, non-manual rule, for example holding out whole classes or taking the lowest-scoring 10% of rows by OpenMax score without inspecting model performance, and check whether TCL still leads on classification; if the margin shrinks, reverses, or the regression gap widens, the central claim is an artifact of the hand-picked thresholds.","supporting_citations":[{"cited_title":"Contrastive federated learn- ing with tabular data silos,","cited_arxiv_id":null,"evidence_quote":"It supplies the contrastive-learning approach for tabular data that TCL adapts, and the paper positions TCL as a local, non-federated version of it."},{"cited_title":"A simple framework for contrastive learning of visual representations,","cited_arxiv_id":null,"evidence_quote":"It defines the contrastive-learning paradigm and the use of augmented views that TCL transfers from images to tabular rows."},{"cited_title":"Subtab: Subset- ting features of tabular data for self-supervised representation learning,","cited_arxiv_id":null,"evidence_quote":"It is a feature-subsetting contrastive baseline whose approach TCL contrasts with full matrix augmentation."},{"cited_title":"SCARF: SELF- SUPERVISED CONTRASTIVE LEARNING USING RAN- DOM FEATURE CORRUPTION,","cited_arxiv_id":null,"evidence_quote":"It is a random-feature-corruption contrastive baseline that TCL is compared against."},{"cited_title":"Rest-net: Diverse activation modules and parallel subnets-based cnn for spatial image steganalysis,","cited_arxiv_id":null,"evidence_quote":"It is the reference for ResNet, the deep residual baseline that TCL claims to beat on classification."},{"cited_title":"Towards open set deep net- works,","cited_arxiv_id":null,"evidence_quote":"It is one of the two OOD detectors used to split each dataset into in-distribution and out-of-distribution sets."},{"cited_title":"Probabilistic outputs for support vector ma- chines and comparisons to regularized likelihood methods,","cited_arxiv_id":null,"evidence_quote":"It is the other OOD detector, and together with OpenMax it provides the score plots from which the OOD thresholds are manually chosen."},{"cited_title":"Speed/accuracy trade-offs for modern convolutional object detectors,","cited_arxiv_id":null,"evidence_quote":"It defines the speed/accuracy trade-off metric that underlies the efficiency comparison and TCL's reported efficiency advantage."},{"cited_title":"Pytorch-ood: A library for out-of-distribution detection based on pytorch,","cited_arxiv_id":null,"evidence_quote":"It provides the implementation of the OOD detection algorithms used in the experiments."}],"review_version":1}