REVIEW 4 major objections 6 minor 43 references
Fully Test-time Adaptation for Tabular Data
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read FTAT, a fully test-time adaptation method for tabular data, reweights predictions to match the shifted label distribution and consistently outperforms non-adaptation and existing test-time adaptation baselines on six benchmarks.
desk verdict A real first benchmark for tabular fully test-time adaptation with consistent reported gains, but the core label-distribution estimator is asserted rather than derived and the update rule as written moves the wrong way. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Confident Distribution Optimizer is the load-bearing mechanism: it estimates the shifted label distribution $\hat{P}_t$ from predictions whose entropy is below a threshold, corrects that estimate with the inverse of the model's per-batch confusion matrix $\hat{C}_t^{-1}$, and updates it with a temporal-ensemble step. The corrected estimate is used to reweight model predictions as $\hat{f}(x) = f(x) \circ \hat{P}_t / P_0$ (elementwise ratio), aligning outputs to the current label distribution. The Local Consistent Weighter and Dynamic Model Ensembler support it by suppressing unstable samples and averaging models across learning rates.
What would settle it
Run FTAT on a synthetic tabular stream with a known label distribution shift and compare the Confident Distribution Optimizer's estimate to the true distribution at every batch using KL divergence; if the corrected estimate stays far from the truth while low-entropy predictions are selected, the central mechanism fails.
Extended reading notes
Core claim
The central claim is that a fully test-time adaptation method can work for tabular data if it does three things at once: track the shifted label distribution from confident predictions, replace augmentation-based consistency with local nearest-neighbor consistency, and remove learning-rate sensitivity by ensembling several adapted models. The Confident Distribution Optimizer estimates the current label distribution from low-entropy predictions, corrects the estimate using the model's batch-level confusion matrix, and smooths it over time; predictions are then reweighted by the ratio of estimated shift to the original label distribution. The authors report that FTAT consistently outperforms the non-adaptation baseline and all six compared FTTA methods across MLP, TabTransformer, and FT-Transformer backbones on six benchmark datasets, and that the ablation without the distribution optimizer loses almost all of the F1 gain on DIABETE.
Load-bearing premise
The method's gains depend on the assumption that the label-distribution estimate built from confident predictions and corrected by the batch-level confusion matrix is unbiased; if it is biased, the prediction re-weighting mis-calibrates the model and the reported gains shrink or reverse.
Editorial extensions
If this is right
- FTAT outperforms the non-adaptation baseline and all compared test-time adaptation methods on six TableShift datasets across all three backbones.
- On the DIABETE dataset, removing the Confident Distribution Optimizer drops F1 from 59.27 to 51.26, so the label-shift correction is the main source of the gain.
- The Dynamic Model Ensembler matches or beats the best single learning rate without needing to know that rate in advance.
Reading between the lines
- The inverse-confusion correction is a generic way to debias low-entropy label estimates; it could be transferred to any test-time method that tracks label shift, including non-tabular settings.
- One testable extension would apply the Local Consistent Weighter to time-series or graph data, where augmentation is also hard and nearest-neighbor consistency is well defined.
- Because FTAT adapts without source data, it could be combined with models that output calibrated probabilities to push the label-distribution estimate closer to unbiased.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces FTAT, a fully test-time adaptation method for tabular classification, consisting of three modules: a Confident Distribution Optimizer, a Local Consistent Weighter, and a Dynamic Model Ensembler. The authors motivate the method by three observations about tabular distribution shifts, ineffective augmentations, and sensitivity to learning rates, and they evaluate on six TableShift datasets with three backbone models, reporting average accuracy/balanced accuracy/F1 improvements over non-adaptation and existing FTTA baselines such as TENT, CoTTA, LAME, and ODS. The claimed contribution is a tuning-light, augmentation-free FTTA method specifically designed for tabular data.
Significance. If the empirical claims were supported, FTAT would be a useful contribution to test-time adaptation for tabular data, an important but underexplored area. The paper provides a broad comparison of existing FTTA methods on tabular benchmarks, releases code and datasets, and includes ablations that isolate the effect of each module. However, the validity of the central Confident Distribution Optimizer rests on an asserted but unproven unbiasedness claim, and the reported headlining F1 gains depend on that module. As written, the contribution is not established at the level required for a journal publication.
major comments (4)
- [§3.1, Eq. (4)] The claim that the quantity \hat C_t^{-1} \hat P_t is an unbiased estimate of the shifted label distribution is asserted without derivation and is not generally correct. The matrix in Eq. (4) has rows indexed by the model's predicted class, so it represents a conditional distribution given predicted labels, while a BBSE-style label-shift correction requires the confusion matrix conditional on true labels. In a fully test-time setting with no labels, the required true-label confusion matrix is not estimable from Eq. (3). This is load-bearing because the ablation in Table 5 shows that removing CDO drops DIABETE F1 from 59.27 to 51.26, meaning the headline F1 gains are carried by this unsubstantiated estimator.
- [§3.1, Eq. (5)] The update rule \hat P_t = Norm(\hat P_{t-1} - α \hat C_t^{-1} \hat P_t) is not a plausible tracking rule: it subtracts the newly estimated distribution from the running estimate rather than moving toward it, and Softmax is not a projection onto the probability simplex. The paper provides no derivation or justification for this particular form, and the minus sign is especially concerning because it does not match an exponential moving average. The authors need to provide a correct update or derive the rule from an explicit objective.
- [§4.2, Table 4] The claim that FTAT 'consistently outperforms non-adaptation baseline' is overstated. On ANES with MLP, FTAT's F1 (83.42) is lower than the non-adaptation baseline (84.24), and several other dataset-metric combinations show differences within one standard deviation. The average improvements in Table 3 are driven by large F1 gains on a subset of datasets, notably those where CDO is responsible for most of the gain. The authors should report per-dataset statistical significance or confidence intervals and temper the 'consistent outperformance' language.
- [§6, Limitation] The paper's own limitation statement concedes that the method 'lacks deep theoretical understanding,' but the method section simultaneously makes a specific mathematical assertion of unbiasedness for the CDO estimator. Since the estimator is neither derived nor an obvious consequence of standard label-shift theory, the empirical results do not compensate for the missing correctness argument. The authors should either supply a rigorous derivation or clearly frame CDO as a heuristic and validate it with additional controlled experiments, such as label-shift-only and covariate-shift-only synthetic settings.
minor comments (6)
- [§3.1] The term 'covariate matrix' is used for what appears to be a confusion matrix; please clarify the terminology and define the matrix entry indices explicitly.
- [§3.1, Eq. (2)] The symbol ∘ is not defined; if it denotes elementwise multiplication or a composition operation, please state the intended meaning.
- [§3.1, Eq. (5)] The initialization of \hat P_0 is not specified; the recurrence requires a starting distribution, presumably the source label distribution P0, but this should be stated.
- [§5] The paper states that AdapTable and TabLog require training data, which undercuts the comparison scope; this claim should be substantiated with specific details or softened, as AdapTable is presented as a test-time adaptation method in the cited work.
- [§2.2, Fig. 2] Figure 2 shows the estimated label distribution without error bars or repeated-run information, making it hard to assess the reliability of the low-entropy estimation claim.
- [§4.3, Fig. 6] The hyperparameter robustness plot shows F1 variation of roughly 0.07–0.08 across the tested α values; the text claims that 'minor perturbations' do not significantly affect performance, but this magnitude is not negligible and should be acknowledged.
Circularity Check
No significant circularity: the central adaptation claim is evaluated empirically against external baselines, and the CDO estimator is presented as a heuristic rather than derived from a self-citation.
full rationale
The paper's central claim is an empirical one: FTAT outperforms non-adaptation and existing FTTA methods on six TableShift datasets across three backbones. This claim is supported by direct benchmark comparisons (Tables 3, 4, 7, 8), not by a derivation that presupposes the conclusion. The Confident Distribution Optimizer is a heuristic module: Equations (3)-(5) define an estimator from the model's own low-entropy predictions, a confusion-matrix correction, and a temporal update. Whether this estimator is unbiased is a correctness question, not a circularity question; the paper explicitly concedes in the Conclusion that the design 'lacks deep theoretical understanding,' which reinforces that the method is not presented as a theorem deduced from prior results. The self-citation to ODS (Zhou et al. 2023) is used as a baseline and as prior work that the paper claims 'fails' on tabular data; FTAT is not justified by that citation, and the performance comparison is external. No equation in the paper reduces a predicted quantity to a fitted input by construction, and no load-bearing premise is imported from the authors' own prior work. The odd minus sign in Eq. (5) and the conditioning of Eq. (4) on predicted classes are potential technical flaws, but they are not instances of circular reasoning under the definitions used here. Therefore the appropriate finding is a low circularity score.
Assumptions & free parameters
free parameters (6)
- alpha (temporal ensembling factor) =
0.1
- epsilon (entropy threshold) =
0.611 (Entropy([0.7, 0.3]))
- beta (consistency threshold) =
0.3
- base learning rates for the ensemble =
1e-5, 5e-4, 1e-4
- neighborhood radius factor =
average pairwise L2 distance in the batch (Dist_t)
- P0 (source label distribution) =
not stated in the paper
assumptions (4)
- domain assumption Low-entropy model predictions suffice to estimate the test label distribution (Eq. 3).
- domain assumption The confusion-matrix correction bC_t^{-1} makes the low-entropy estimate unbiased (Eq. 4).
- ad hoc to paper The temporal update bPt = Norm(bPt-1 - alpha bC_t^{-1} ePt) tracks the shifting label distribution (Eq. 5).
- domain assumption Neighborhood consistency in raw feature space is a valid surrogate for augmentation-based consistency under covariate shift (Eqs. 6-8).
Cite this review
Pith. "Pith review of Fully Test-time Adaptation for Tabular Data." pith.science (2026). https://pith.science/paper/U3X75PSH
@misc{pith2026241210871,
author = {Pith},
title = {Pith review of: Fully Test-time Adaptation for Tabular Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/U3X75PSH}},
note = {Machine review of arXiv:2412.10871}
}
read the original abstract
Tabular data plays a vital role in various real-world scenarios and finds extensive applications. Although recent deep tabular models have shown remarkable success, they still struggle to handle data distribution shifts, leading to performance degradation when testing distributions change. To remedy this, a robust tabular model must adapt to generalize to unknown distributions during testing. In this paper, we investigate the problem of fully test-time adaptation (FTTA) for tabular data, where the model is adapted using only the testing data. We identify three key challenges: the existence of label and covariate distribution shifts, the lack of effective data augmentation, and the sensitivity of adaptation, which render existing FTTA methods ineffective for tabular data. To this end, we propose the Fully Test-time Adaptation for Tabular data, namely FTAT, which enables FTTA methods to robustly optimize the label distribution of predictions, adapt to shifted covariate distributions, and suit a variety of tasks and models effectively. We conduct comprehensive experiments on six benchmark datasets, which are evaluated using three metrics. The experimental results demonstrate that FTAT outperforms state-of-the-art methods by a margin.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Altman, N.; and Krzywinski, M. 2017. Tabular data. Nature Methods, 14(4): 329--330
work page 2017
-
[4]
Alvarez-Melis, D.; and Fusi, N. 2020. Geometric dataset distances via optimal transport. In Advances in Neural Information Processing Systems, 21428--21439
work page 2020
-
[5]
Arik, S. \" O .; and Pfister, T. 2021. TabNet: Attentive Interpretable Tabular Learning. In Proceedings of the 35th AAAI Conference on Artificial Intelligence , 6679--6687
work page 2021
-
[6]
Badirli, S.; Liu, X.; Xing, Z.; Bhowmik, A.; and Keerthi, S. S. 2020. Gradient Boosting Neural Networks: GrowNet. CoRR, abs/2002.07971
arXiv 2020
-
[7]
Bai, Y.; Zhang, Y.-J.; Zhao, P.; Sugiyama, M.; and Zhou, Z.-H. 2022. Adapting to Online Label Shift with Provable Guarantees. In Advances in Neural Information Processing Systems
work page 2022
-
[8]
Boudiaf, M.; Mueller, R.; Ben Ayed, I.; and Bertinetto, L. 2022. Parameter-free Online Test-time Adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 8344--8353
work page 2022
Show all 43 references
-
[9]
Chen, D.; Wang, D.; Darrell, T.; and Ebrahimi, S. 2022. Contrastive Test-Time Adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 295--305
2022
-
[10]
S.; Beaulieu-Jones, B
Ching, T.; Himmelstein, D. S.; Beaulieu-Jones, B. K.; Kalinin, A. A.; Do, B. T.; Way, G. P.; Ferrero, E.; Agapow, P.-M.; Zietz, M.; Hoffman, M. M.; et al. 2018. Opportunities and obstacles for deep learning in biology and medicine. Journal of the Royal Society Interface, 15(14...
2018
-
[11]
Fang, J.; Tang, C.; Cui, Q.; Zhu, F.; Li, L.; Zhou, J.; and Zhu, W. 2022. Semi-Supervised Learning with Data Augmentation for Tabular Data. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management , 3928--3932
2022
-
[12]
D.; and Whelton, P
Fuchs, F. D.; and Whelton, P. K. 2020. High Blood Pressure and Cardiovascular Disease. Hypertension, 75(2): 285--292
2020
-
[13]
Gardner, J.; Popovic, Z.; and Schmidt, L. 2023. Benchmarking Distribution Shift in Tabular Data with TableShift. In Advances in Neural Information Processing Systems
2023
-
[14]
Gong, T.; Jeong, J.; Kim, T.; Kim, Y.; Shin, J.; and Lee, S.-J. 2022. NOTE : Robust Continual Test-time Adaptation Against Temporal Correlation. In Advances in Neural Information Processing Systems, 27253--27266
2022
-
[15]
Gorishniy, Y.; Rubachev, I.; and Babenko, A. 2022. On Embeddings for Numerical Features in Tabular Deep Learning. In Advances in Neural Information Processing Systems
2022
-
[16]
Gorishniy, Y.; Rubachev, I.; Kartashev, N.; Shlenskii, D.; Kotelnikov, A.; and Babenko, A. 2024. TabR: Tabular Deep Learning Meets Nearest Neighbors. In Proceedings of th 12th International Conference on Learning Representations
2024
-
[17]
Gorishniy, Y.; Rubachev, I.; Khrulkov, V.; and Babenko, A. 2021. Revisiting Deep Learning Models for Tabular Data. In Advances in Neural Information Processing Systems, 18932--18943
2021
-
[18]
Goyal, S.; Sun, M.; Raghunathan, A.; and Kolter, J. Z. 2022. Test Time Adaptation via Conjugate Pseudo-labels. In Advances in Neural Information Processing Systems, 6204--6218
2022
-
[19]
Grinsztajn, L.; Oyallon, E.; and Varoquaux, G. 2022. Why do tree-based models still outperform deep learning on typical tabular data? In Advances in Neural Information Processing Systems
2022
-
[20]
Guo, Y.; Hu, C.; and Yang, Y. 2023. Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 1029--1038
2023
-
[21]
B.; and Doll \' a r, P
He, K.; Girshick, R. B.; and Doll \' a r, P. 2019. Rethinking ImageNet Pre-Training. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision , 4917--4926
2019
-
[22]
A.; and Sterzing, V
Hein, D.; Depeweg, S.; Tokic, M.; Udluft, S.; Hentschel, A.; Runkler, T. A.; and Sterzing, V. 2017. A benchmark environment motivated by industrial control problems. In Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence , 1--8
2017
-
[23]
C.; and Johnson, J
Helton, J. C.; and Johnson, J. D. 2011. Quantification of margins and uncertainties: Alternative representations of epistemic uncertainty. Reliability Engineering & System Safety, 96(9): 1034--1052
2011
-
[24]
Huang, X.; Khetan, A.; Cvitkovic, M.; and Karnin, Z. S. 2020. TabTransformer: Tabular Data Modeling Using Contextual Embeddings. CoRR, abs/2012.06678
2020 arXiv
-
[25]
Y.; and Yang, E
Kim, C.; Kim, T.; Woo, S.; Yang, J. Y.; and Yang, E. 2024. AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler
2024
-
[26]
Klambauer, G.; Unterthiner, T.; Mayr, A.; and Hochreiter, S. 2017. Self-Normalizing Neural Networks. In Advances in Neural Information Processing Systems, 971--980
2017
-
[27]
Kolesnikov, S. 2023. Wild-Tab: A Benchmark for Out-Of-Distribution Generalization in Tabular Regression. CoRR, abs/2312.01792
2023 arXiv
-
[28]
Kritzman, M.; Page, S.; and Turkington, D. 2012. Regime shifts: Implications for dynamic strategies. Financial Analysts Journal, 68(3): 22--39
2012
-
[29]
Laine, S.; and Aila, T. 2017. Temporal Ensembling for Semi-Supervised Learning. In Proceeding of the 5th International Conference on Learning Representations
2017
-
[30]
Liu, Y.; Kothari, P.; van Delft, B.; Bellot - Gurlet, B.; Mordan, T.; and Alahi, A. 2021. TTT++: When Does Self-Supervised Test-Time Training Fail or Thrive? In Advances in Neural Information Processing Systems, 21808--21820
2021
-
[31]
Niu, S.; Wu, J.; Zhang, Y.; Chen, Y.; Zheng, S.; Zhao, P.; and Tan, M. 2022. Efficient Test-Time Model Adaptation without Forgetting. In Proceedings of the 39th International Conference on Machine Learning, 16888--16905
2022
-
[32]
Niu, S.; Wu, J.; Zhang, Y.; Wen, Z.; Chen, Y.; Zhao, P.; and Tan, M. 2023. Towards Stable Test-time Adaptation in Dynamic Wild World. In Proceedings of the 11th International Conference on Learning Representations
2023
-
[33]
M.; Gudelek, M
Ozbayoglu, A. M.; Gudelek, M. U.; and Sezer, O. B. 2020. Deep learning for financial applications: A survey. Applied Soft Computing, 93: 106384
2020
-
[34]
Ren, W.; Li, X.; Chen, H.; Rakesh, V.; Wang, Z.; Das, M.; and Honavar, V. G. 2024. TabLog: Test-Time Adaptation for Tabular Data Using Logic Rules. In Proceedings of the 41st International Conference on Machine Learning
2024
-
[35]
Salehpour, A.; and Samadzamini, K. 2024. A bibliometric analysis on the application of deep learning in economics, econometrics, and finance. International Journal of Computer Sciences and Engineering, 27(2): 167--181
2024
-
[36]
P.; Gennings, C.; Olmo, J
Strack, B.; DeShazo, J. P.; Gennings, C.; Olmo, J. L.; Ventura, S.; Cios, K. J.; and Clore, J. N. 2014. Impact of HbA1c Measurement on Hospital Readmission Rates: Analysis of 70,000 Clinical Database Patient Records. BioMed Research International, 2014(1): 781670
2014
-
[37]
A.; and Hardt, M
Sun, Y.; Wang, X.; Liu, Z.; Miller, J.; Efros, A. A.; and Hardt, M. 2020. Test-Time Training with Self-Supervision for Generalization under Distribution Shifts. In Proceedings of the 37th International Conference on Machine Learning, 9229--9248
2020
-
[38]
Torralba, A.; and Oliva, A. 2003. Statistics of natural image categories. Network: Computation in Neural Systems, 14(3): 391
2003
-
[39]
E.; Isaacs, S
Umpierrez, G. E.; Isaacs, S. D.; Bazargan, N.; You, X.; Thaler, L. M.; and Kitabchi, A. E. 2002. Hyperglycemia: An Independent Marker of In-Hospital Mortality in Patients with Undiagnosed Diabetes. The Journal of Clinical Endocrinology & Metabolism, 87(3): 978--982
2002
-
[40]
A.; and Darrell, T
Wang, D.; Shelhamer, E.; Liu, S.; Olshausen, B. A.; and Darrell, T. 2021. Tent: Fully Test-Time Adaptation by Entropy Minimization. In Proceedings of the 9th International Conference on Learning Representations
2021
-
[41]
V.; and Dai, D
Wang, Q.; Fink, O.; Gool, L. V.; and Dai, D. 2022. Continual Test-Time Domain Adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 7191--7201
2022
-
[42]
Wu, R.; Guo, C.; Su, Y.; and Weinberger, K. Q. 2021. Online Adaptation to Label Distribution Shift. In Advances in Neural Information Processing Systems, 11340--11351
2021
-
[43]
Zhou, Z.; Guo, L.; Jia, L.; Zhang, D.; and Li, Y. 2023. ODS: Test-Time Adaptation in the Presence of Open-World Data Shift. In Proceedings of the 40th International Conference on Machine Learning, 42574--42588
2023
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.