CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation

Aditya Gorla; Ryan Wang; Sriram Sankararaman; Ulzee An; Zhengtong Liu

REVIEW 3 major objections 5 minor 45 references

CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CACTI claims that a transformer imputer trained on a dataset's own missingness patterns, truncated at the batch median and primed with column-name text embeddings, beats all thirteen baselines on every metric across MCAR, MAR, and MNAR.

desk verdict CACTI's MT-CM masking trick is a genuinely useful idea with a broad benchmark, but the headline SOTA numbers are undercut by hyperparameters tuned on test splits of four of the ten datasets. read the letter →

arxiv 2506.02306 v1 pith:25SLZ7J7 submitted 2025-06-02 cs.LG stat.ML

Aditya Gorla , Ryan Wang , Zhengtong Liu , Ulzee An , Sriram Sankararaman This is my paper

classification cs.LGstat.ML MSC 62D1068T07

keywords tabulardataimputationmaskedautoencodercopymaskingmediantruncationmissingnessmechanismsmissingnotatrandomcontext-awareembeddingslanguagemodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a tabular imputer learns better when it trains on the missingness patterns that actually occur in a dataset rather than on random masks, and when it is told what the columns mean. It introduces CACTI, a transformer masked autoencoder whose training masks are recycled from the dataset's own missing-value patterns and truncated at the batch median so the encoder never confronts more than 50% null tokens. Column names and descriptions, embedded by a language model, are concatenated into the feature embeddings of both encoder and decoder. Across ten datasets and the three canonical missingness mechanisms (MCAR, MAR, MNAR), the paper reports that CACTI beats all thirteen baselines on every metric, with an average $R^2$ gain of 7.8% over the next-best method and gains of 13.4%, 6.1%, and 5.3% under MNAR, MAR, and MCAR respectively. The payoff would be materially better imputation in settings such as biomedical surveys, where missingness is structured and feature semantics carry information.

What carries the argument

The central mechanism is median truncated copy masking (MT-CM). Naive copy masking takes the observed missingness mask matrix and, with probability $p_{\mathrm{cm}}$, replaces a sample's training mask with another sample's real missingness pattern, so the model is trained to reconstruct under the same kinds of gaps it will face at test time; MT-CM then truncates each sample's sequence to at most the batch median number of observed features, padding the remainder with null tokens, which caps the null-token fraction at 50% regardless of masking rate. This truncation is what makes high masking rates (the paper finds 90\text{--}99% best, with performance rising as the rate increases) trainable in a transformer. The second mechanism is context-aware embedding: each scalar value is projected to 75% of the embedding dimension, a language-model embedding of the column name and description (default: GTE-en-MLM-large) is projected to the remaining 25%, the two are concatenated, and positional encoding is added, in both the encoder and the decoder stage. The reconstruction loss is the MSE over both masked cells and still-observed cells, which forces the latent space to preserve observed relationships while inferring missing ones.

What would settle it

Train CACTI and ReMasker on a synthetic dataset engineered to be purely MCAR with column names replaced by random strings, so both claimed inductive biases are neutralized; a material $R^2$ gap in that setting would show the gains come from architecture or tuning, while a tie would confirm the biases do the work.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that two cheap sources of inductive bias, when wired into a masked autoencoder, dominate all current tabular imputation methods in the authors' benchmarks. The first source is empirical missingness: instead of hiding random cells during training, CACTI recycles the actual missingness patterns observed in the dataset (row-wise permuted copy masking) and truncates each sample to the batch-median number of observed features (median truncation), which keeps null-token density bounded while allowing masking rates near $p_{\mathrm{cm}}=0.90$\text{--}0.99$. The second source is column semantics: language-model embeddings of column names and descriptions are projected into 25% of each feature's embedding and concatenated with the value embedding in both encoder and decoder. In the paper's comparisons, the copy-masked model alone (CMAE) improves on the random-masking baseline ReMasker with $p<0.05$, adding context yields statistically significant further gains in all settings except MAR at 30% missingness, and the full CACTI model reports the best scores on all three metrics ($$R^{2}$$, RMSE, Wasserstein distance) under all three missingness mechanisms.

Load-bearing premise

The method assumes that the missingness patterns visible in the training data faithfully represent how missingness actually forms, so that training with recycled real patterns transfers to test time; the paper proves this equivalence only for fully observed samples and asserts it for the partially observed case.

Editorial extensions

If this is right

If the central claim holds, CACTI is the current best tabular imputation method on the benchmarked datasets, ahead of the strongest masked-autoencoder, diffusion, and iterative baselines (ReMasker, DiffPuter, HyperImpute) on all three metrics.
Both components are modular: MT-CM can replace random masking in any masked learning pipeline, and the context-embedding trick can be attached to any deep imputation backbone, so the claimed gains are not tied to the specific CACTI architecture.
Because performance rises as the copy masking rate increases under MT-CM and falls under naive copy masking, the paper's results imply that fidelity to empirical missingness patterns, not blanket masking, is what drives the improvement.
The largest relative gains occur under MNAR (13.4% in $R^2$), implying the method is most valuable precisely where missingness depends on unobserved values, the regime the paper's theory motivates most directly.
The combined observed-plus-masked loss is essential: training on observed cells alone collapses ($R^2$ near 0.03\text{--}0.04), so the paper's recipe requires reconstruction over hidden cells to learn feature dependence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same recipe should transfer to non-tabular structured data with genuine missingness, such as electronic health records, survey panels, and clinical registries, because the paper only benchmarks fully observed datasets with simulated gaps; a curated test on real missing data would show whether the advantage survives outside simulation.
A controlled experiment that varies the missingness model's structuredness, from pure MCAR to strongly correlated, hierarchical MNAR, should trace out a monotone relationship between MT-CM's advantage over random masking and the amount of exploitable structure; the paper's mechanism-level claim predicts this, but the benchmarks do not measure it directly.
The context component's value plausibly grows when column names are informative and the feature count is large relative to sample size; a cheap ablation with scrambled or meaningless column names would quantify how much of the context gain is semantic rather than merely dimensional.
A rigorous theory of copy masking under partial observation, since the paper's risk-equivalence argument is proven only for fully observed samples, would tell whether the empirical-mask bias has a provable foundation and would predict exactly which missingness structures yield gains.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: CACTI is a masked autoencoder for tabular imputation that swaps random masking for median-truncated copy masking (MT-CM) and adds LM-derived column embeddings. The MT-CM fix is real and portable: naive copy masking fills transformer batches with null tokens, and truncating each sample to the batch-median observed count solves that. The ablation shows MT-CM is the main driver; the context embeddings give a smaller, mostly significant boost. The benchmark is broad (13 baselines, 10 datasets, three missingness mechanisms, four missingness rates, standard errors, paired t-tests), and the code is public.

The soft spot is the one the stress-test flags, and it lands: the default hyperparameters (pcm=0.90, encoder depth 10, embedding 64) were chosen from sensitivity runs on four of the ten datasets (bike, default, spam, students), and those runs report test-split performance. There is no validation split. So for those four datasets, test labels likely influenced model selection. Baselines got default settings. That makes the "outperforms all baselines" claim in Table 1 partly self-dealing for those datasets. The fix is easy—validate on a held-out split or per-dataset CV, then re-report, or at minimum show a fixed off-the-shelf config still beats ReMasker and DiffPuter. Until then, the exact gain sizes are uncertain, but the direction is probably right: their internal CMAE-versus-ReMasker comparison across all ten datasets also favors MT-CM, and that comparison uses the same architecture, so it's less affected by the tuning leak.

The theory in Section 3.1.1 is a motivation, not a proof—it covers fully observed samples and then asserts the general case. That's fine for what it is. The citation pattern is normal; copy masking comes from the same group's earlier work, but it's clearly cited and the extension is substantive.

Who this is for: anyone doing tabular imputation will want the MT-CM idea. Don't cite it for the absolute SOTA numbers; cite it for the method and the evaluation design (with the caveat above). If this landed on my desk as a new submission, I'd send it to review—the idea is sound and the benchmark effort is above average—but I'd ask for a validation-split re-run before accepting the headline claims.

Referee Report

3 major / 5 minor

Summary. The paper introduces CACTI, a transformer-based masked autoencoder for tabular data imputation. Its two claimed contributions are median truncated copy masking (MT-CM), which trains on masks sampled from empirical missingness patterns, and context-aware embeddings derived from language models applied to column names and descriptions. The authors report that CACTI outperforms 13 baselines on 10 datasets under MCAR, MAR, and MNAR at several missingness rates, with headline average relative R2 gains of 13.4%, 6.1%, and 5.3% over the next-best method, and they support this with ablations, paired t-tests, sensitivity analyses, and a public code repository.

Significance. If the comparison is controlled, the paper would be a useful contribution: it presents a modular training strategy and a simple way to inject textual feature context, both of which are broadly applicable. The evaluation is extensive, covering 10 datasets, 13 baselines, three missingness mechanisms, four missingness rates, ablations, and statistical tests, and the code is publicly available. The main uncertainty is whether the headline gains reflect an apples-to-apples comparison, since CACTI's defaults appear to have been selected on the test splits of four datasets that are themselves part of the benchmark.

major comments (3)

[Section 4.2, Table A8, Section C.2] As written, the experimental protocol permits model selection on test labels. The sensitivity analyses in Section 4.2 and Figures A13-A18 were run on the four datasets bike, default, spam, and students, and the figures report train/test metrics with no separate validation split (Section 4, 'Datasets', describes only an 80-20 train-test split). CACTI's defaults in Table A8 (300 epochs, batch 128, pcm=0.90, Ne=10, Nd=4, E=64) are justified by those analyses, and those same four datasets contribute to Table 1. Please specify exactly whether any test labels were used for hyperparameter selection, and if so, provide a corrected protocol with a validation split or nested selection, or report results under a clearly pre-specified default configuration.
[Table 1 and Abstract] The headline relative gains are not directly reproducible from the aggregate results. Table 1 shows that the second-best method is often the authors' own CMAE (CACTI without context), e.g., test R2 under MNAR is 0.46 for CACTI and 0.44 for CMAE, which is roughly a 4.5% relative gain rather than the stated 13.4%. Please state explicitly how the 13.4%, 6.1%, and 5.3% figures are computed, whether they are per-dataset relative improvements averaged over datasets, and whether CMAE is counted as a baseline for the 'next-best method' comparison. This is needed to substantiate the central state-of-the-art claim.
[Section 3.1.1, Equations (4)-(5)] The theoretical motivation for copy masking is only established for fully observed samples. Equation (5) shows that when M=1, choosing Q to approximate M|X makes the surrogate risk RQ close to the target risk R, but for partially observed samples (M not equal to 1), the authors assert rather than prove the corresponding equivalence. Please either provide a proof for the general case or explicitly label this argument as heuristic motivation, since the current wording claims a theoretical result that the derivation does not deliver.

minor comments (5)

[Section 2.1] There is a typo: 'copy maksing' should be 'copy masking'.
[Table 1] In the WD row for CACTI under MAR, the entry '1.87|1.944.45 |4.57' appears to have a missing separator between '1.94' and '4.45'; please fix the formatting.
[Section E, Table A15] 'statically significant improvement' should be 'statistically significant improvement'.
[Section 4.2.4] The text recommends 'monitor overfitting on validation data,' but no validation split is described in the experimental setup; please clarify whether a validation split exists or rephrase the recommendation.
[Figure A17] The axis label 'Split train val' is inconsistent with the rest of the appendix, which reports train/test splits; please make the label consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation-level circularity: the SOTA claim is an external-benchmark comparison; same-group copy-masking citation is real evidence and the test-set tuning concern is a validity risk, not a circular derivation.

full rationale

The paper's central claim is empirical: CACTI's reported R2 gains are computed against 13 external baselines (ReMasker, DiffPuter, HyperImpute, MICE, GAIN, etc.) on simulated missingness from 10 datasets, so the superiority claim is not entailed by the method's own definitions. MT-CM is a constructive training-mask algorithm (Algorithm 2) that samples and median-truncates observed missingness masks; the context embeddings are precomputed LM embeddings of column names and descriptions; neither component defines the evaluation metric in terms of itself. The theoretical motivation (Eqs. 3-5) is a standard masked-reconstruction risk identity used only to motivate choosing a masking distribution close to the missingness mechanism, and its restriction to fully observed samples is visible rather than a hidden circular premise. The only same-group citation (An et al. 2023) supplies the naive copy-masking primitive, but the paper also benchmarks AutoComplete as an external baseline and evaluates MT-CM against ReMasker, so the citation is independently falsifiable evidence, not a load-bearing self-referential premise. A benchmark-validity concern remains: the sensitivity analysis (Section 4.2) that informed CACTI's defaults was run on four datasets that also appear in Table 1, with no explicit validation split, which could affect comparison fairness; but this is a data-contamination risk, not a reduction of the prediction to its inputs by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method rests on the stated domain assumptions that empirical missingness patterns and semantic column descriptions are useful inductive biases, and on standard transformer/MAE machinery. Model hyperparameters are tuned on a subset of the benchmark datasets. No new physical or conceptual entities are introduced.

free parameters (5)

MT-CM masking ratio pcm = 0.90 default; 0.95 (MNAR) and 0.99 (MAR) optimal in sensitivity analysis
Controls the strength of the empirical-missingness inductive bias; chosen by sensitivity analysis on a 4-dataset subset.
Embedding dimension E = 64
Chosen by sensitivity analysis; larger sizes degrade performance (Table 4c).
Encoder depth Ne and decoder depth Nd = 10 and 4
Slightly favored in sensitivity analysis (Table 4a, 4b).
Context embedding proportion C/E = 0.25
Default from Table A8; Table A17 shows 0.25 or 0.50 as optimal.
Loss weighting (LO plus LM) = Equal weights (1:1)
The combined reconstruction loss is chosen over observed-only or masked-only losses (Table A16).

assumptions (5)

domain assumption The empirical missingness patterns in observed data approximate the true missingness mechanism (Q approximately equals M|X), so copy masking provides a useful inductive bias.
Invoked in Section 3.1 and in the theoretical motivation of Section 3.1.1; not proven for partially observed data.
domain assumption Column names and descriptions embedded by a general-purpose language model capture feature relationships relevant to imputation.
The context-awareness contribution relies on this semantic transfer; only empirically validated.
domain assumption Simulated missingness (MCAR, MAR, MNAR via the HyperImpute package) reflects real missingness mechanisms.
Evaluation design assumption; standard in the field.
standard math Transformer architecture with self-attention can represent tabular feature dependence, and positional encoding is meaningful for tabular features.
Backbone design; standard result.
domain assumption R2 is a valid concordance metric for binary and categorical features, and min-max scaling followed by inverse transformation preserves comparability.
Evaluation choice; not justified for categorical data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation." pith.science (2026). https://pith.science/paper/25SLZ7J7

@misc{pith2026250602306,
  author       = {Pith},
  title        = {Pith review of: CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/25SLZ7J7}},
  note         = {Machine review of arXiv:2506.02306}
}

abstract

We present CACTI, a masked autoencoding approach for imputing tabular data that leverages the structure in missingness patterns and contextual information. Our approach employs a novel median truncated copy masking training strategy that encourages the model to learn from empirical patterns of missingness while incorporating semantic relationships between features - captured by column names and text descriptions - to better represent feature dependence. These dual sources of inductive bias enable CACTI to outperform state-of-the-art methods - an average $R^2$ gain of 7.8% over the next best method (13.4%, 6.1%, and 5.3% under missing not at random, at random and completely at random, respectively) - across a diverse range of datasets and missingness conditions. Our results highlight the value of leveraging dataset-specific contextual information and missingness patterns to enhance imputation performance.

Figures

Figures reproduced from arXiv: 2506.02306 by the authors.

**Figure 1.** Median Truncated Copy Masking overview. In contrast to random masking, where some subset of features are masked uniformly at random, copy masking recycles missing value patterns actually present in the dataset. This approach simulates realistic missingness patterns that provide a source of useful inductive bias during training. Median Truncated Copy Masking extends this strategy for MAE training by truncating the n… view at source ↗

**Figure 2.** CACTI model overview. CACTI samples observed missingness patterns to generate masks via Median Truncated Copy Masking (MT-CM) to guide the learning. Features’ context are also embedded with a language model. The MT-CM strategy masks out some portion of the observed features from sample n using observed missingness patterns from other samples (j) in the same dataset. This is followed by concatenating context informat… view at source ↗

**Figure 3.** Top-5 methods benchmark. Violin plots display the distribution of R 2 metrics for each of the top five methods across all missing percentages and conditions over all datasets. we conduct a comprehensive sensitivity analysis to identify key aspects and hyperparameter configurations that significantly impact the performance and usability of our method. Baseline methods We benchmark CACTI against 13 top methods from t… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: CACTI Training profile. Evaluated across training epochs under MAR and MNAR on the letter dataset with 30% missingness. Points are mean ± 95% CI. 300-600 epochs and monitor overfitting on validation data. 5. Conclusion This work introduces a conceptual framework for le…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 16 canonical work pages

[1]

R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., and McDermott, M

Alsentzer, E., Murphy, J. R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., and McDermott, M. B. A. Publicly available clinical bert embeddings, 2019. URL https://arxiv.org/abs/1904.03323

arXiv 2019
[2]

J., Kendler, K., Pajukanta, P., Flint, J., Zaitlen, N., and et al

An, U., Pazokitoroudi, A., Alvarez, M., Huang, L., Bacanu, S., Schork, A. J., Kendler, K., Pajukanta, P., Flint, J., Zaitlen, N., and et al. Deep learning-based phenotype imputation on population-scale biobank data increases genetic discoveries. Nature Genetics, 55 0 (12): 0 2269–2276, Nov 2023. doi:10.1038/s41588-023-01558-w

work page doi:10.1038/s41588-023-01558-w 2023
[3]

A., Jeong, M., Gorla, A., Chiang, J

An, U., Lee, S. A., Jeong, M., Gorla, A., Chiang, J. N., and Sankararaman, S. Dk-behrt: Teaching language models international classification of disease (icd) codes using known disease descriptions. In Proceedings of The First AAAI Bridge Program on AI for Medicine and Healthcare, volume 281 of Proceedings of Machine Learning Research, pp.\ 133--143. PMLR...

work page 2025
[4]

Arik, S. O. and Pfister, T. Tabnet: Attentive interpretable tabular learning, 2020. URL https://arxiv.org/abs/1908.07442

arXiv 2020
[5]

Multiple Imputation via Generative Adversarial Network for High-dimensional Blockwise Missing Value Problems

Dai, Z., Bu, Z., and Long, Q. Multiple imputation via generative adversarial network for high-dimensional blockwise missing value problems, 2021. URL https://arxiv.org/abs/2112.11507

work page Pith review arXiv 2021
[6]

Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805

arXiv 2019
[7]

Remasker: Imputing tabular data with masked autoencoding

Du, T., Melis, L., and Wang, T. Remasker: Imputing tabular data with masked autoencoding. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=KI9NqjLVDT

work page 2024
[8]

Position information in transformers: An overview, 2021

Dufter, P., Schmitt, M., and Schütze, H. Position information in transformers: An overview, 2021. URL https://arxiv.org/abs/2102.11090

arXiv 2021

Show all 45 references

[9]

C., and Schmidt, L

Gardner, J., Perdomo, J. C., and Schmidt, L. Large scale transfer learning for tabular data via language modeling, 2024. URL https://arxiv.org/abs/2406.12031

2024 arXiv
[10]

Matrix completion and low-rank svd via fast alternating least squares, 2014

Hastie, T., Mazumder, R., Lee, J., and Zadeh, R. Matrix completion and low-rank svd via fast alternating least squares, 2014. URL https://arxiv.org/abs/1410.2596

2014 arXiv
[11]

and Elliott, P

Hawthorne, G. and Elliott, P. Imputing cross-sectional missing data: Comparison of common techniques. Australian and New Zealand Journal of Psychiatry, 39 0 (7): 0 583--590, Jul 2005. doi:10.1080/j.1440-1614.2005.01630.x

2005
[12]

Masked autoencoders are scalable vision learners, 2021

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. Masked autoencoders are scalable vision learners, 2021. URL https://arxiv.org/abs/2111.06377

2021 arXiv
[13]

Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing, 2023

He, P., Gao, J., and Chen, W. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing, 2023. URL https://arxiv.org/abs/2111.09543

2023 arXiv
[14]

B., Schirrmeister, R

Hollmann, N., Müller, S., Purucker, L., Krishnakumar, A., Körfer, M., Hoo, S. B., Schirrmeister, R. T., and Hutter, F. Accurate predictions on small data with a tabular foundation model. Nature, 637 0 (8045): 0 319–326, Jan 2025. doi:10.1038/s41586-024-08328-6

2025 doi
[15]

Tabtransformer: Tabular data modeling using contextual embeddings, 2020

Huang, X., Khetan, A., Cvitkovic, M., and Karnin, Z. Tabtransformer: Tabular data modeling using contextual embeddings, 2020. URL https://arxiv.org/abs/2012.06678

2020 arXiv
[16]

B., Mattei, P.-A., and Frellsen, J

Ipsen, N. B., Mattei, P.-A., and Frellsen, J. not-miwae: Deep generative modelling with missing not at random data, 2021. URL https://arxiv.org/abs/2006.12871

2021 arXiv
[17]

A complete characterisation of structured missingness, 2023

Jackson, J., Mitra, R., Hagenbuch, N., McGough, S., and Harbron, C. A complete characterisation of structured missingness, 2023. URL https://arxiv.org/abs/2307.02650

2023 arXiv
[18]

C., Liu, T., Curth, A., and van der Schaar, M

Jarrett, D., Cebere, B. C., Liu, T., Curth, A., and van der Schaar, M. Hyperimpute: Generalized iterative imputation with automatic model selection. In International Conference on Machine Learning, pp.\ 9916--9937. PMLR, 2022

2022
[19]

The uci machine learning repository

Kelly, M., Longjohn, R., and Nottingham, K. The uci machine learning repository. URL https://archive.ics.uci.edu
[20]

M., Kosec, M., Perez, S

Krell, M. M., Kosec, M., Perez, S. P., and Fitzgibbon, A. Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance, 2022. URL https://arxiv.org/abs/2107.02027

2022 arXiv
[21]

Nv-embed: Improved techniques for training llms as generalist embedding models, 2025

Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catanzaro, B., and Ping, W. Nv-embed: Improved techniques for training llms as generalist embedding models, 2025. URL https://arxiv.org/abs/2405.17428

2025 arXiv
[22]

J., Ziegler, Z., Nadler, D., Szolovits, P., Johnson, A., and Alsentzer, E

Lehman, E., Hernandez, E., Mahajan, D., Wulff, J., Smith, M. J., Ziegler, Z., Nadler, D., Szolovits, P., Johnson, A., and Alsentzer, E. Do we still need clinical language models?, 2023. URL https://arxiv.org/abs/2302.08091

2023 arXiv
[23]

Ctsyn: A foundational model for cross tabular data generation, 2024

Lin, X., Xu, C., Yang, M., and Cheng, G. Ctsyn: A foundational model for cross tabular data generation, 2024. URL https://arxiv.org/abs/2406.04619

2024
[24]

Little, R. J. A. and Rubin, D. B. Statistical analysis with missing data. Wiley, 1987

1987
[25]

Met: Masked encoding for tabular data, 2022

Majmundar, K., Goyal, S., Netrapalli, P., and Jain, P. Met: Masked encoding for tabular data, 2022. URL https://arxiv.org/abs/2206.08564

2022 arXiv
[26]

and Frellsen, J

Mattei, P.-A. and Frellsen, J. Miwae: Deep generative modelling and imputation of incomplete data, 2019. URL https://arxiv.org/abs/1812.02633

2019 arXiv
[27]

F., Chakraborti, T., Holmes, C., Copping, R., Hagenbuch, N., Biedermann, S., Noonan, J., Lehmann, B., Shenvi, A., and et al

Mitra, R., McGough, S. F., Chakraborti, T., Holmes, C., Copping, R., Hagenbuch, N., Biedermann, S., Noonan, J., Lehmann, B., Shenvi, A., and et al. Learning from data with structured missingness. Nature Machine Intelligence, 5 0 (1): 0 13–23, Jan 2023. doi:10.1038/s42256-022-00596-z

2023 doi
[28]

Missing data imputation using optimal transport, 2020

Muzellec, B., Josse, J., Boyer, C., and Cuturi, M. Missing data imputation using optimal transport, 2020. URL https://arxiv.org/abs/2002.03860

2020 arXiv
[29]

M., Ghahramani, Z., and Valera, I

Nazabal, A., Olmos, P. M., Ghahramani, Z., and Valera, I. Handling incomplete heterogeneous data using vaes, 2020. URL https://arxiv.org/abs/1807.03653

2020 arXiv
[30]

W., Wu, W., Lin, L., Xu, B., and Bernal, E

Richardson, T. W., Wu, W., Lin, L., Xu, B., and Bernal, E. A. Mcflow: Monte carlo flow models for data imputation, 2020. URL https://arxiv.org/abs/2003.12628

2020 arXiv
[31]

and White, I

Royston, P. and White, I. R. Multiple imputation by chained equations (mice): Implementation in stata. Journal of Statistical Software, 45 0 (4): 0 1–20, 2011. doi:10.18637/jss.v045.i04. URL https://www.jstatsoft.org/index.php/jss/article/view/v045i04

2011 doi
[32]

Rubin, D. B. Inference and missing data. Biometrika, 63 0 (3): 0 581, Dec 1976. doi:10.2307/2335739

1976 doi
[33]

Rubin, D. B. Multiple imputation for nonresponse in surveys. Wiley Series in Probability and Statistics, Jun 1987. doi:10.1002/9780470316696

1987 doi
[34]

Schafer, J. L. and Graham, J. W. Missing data: our view of the state of the art. Psychological methods, 7 2: 0 147--77, 2002. URL https://api.semanticscholar.org/CorpusID:7745507

2002
[35]

Stekhoven, D. J. and Bühlmann, P. Missforest—non-parametric missing value imputation for mixed-type data. Bioinformatics, 28 0 (1): 0 112–118, Oct 2011. doi:10.1093/bioinformatics/btr597

2011 doi
[36]

and Groothuis-Oudshoorn, K

van Buuren, S. and Groothuis-Oudshoorn, K. mice: Multivariate imputation by chained equations in r. Journal of Statistical Software, 45 0 (3): 0 1–67, 2011. doi:10.18637/jss.v045.i03. URL https://www.jstatsoft.org/index.php/jss/article/view/v045i03

2011 doi
[37]

Unitabe: A universal pretraining protocol for tabular foundation model in data science, 2024

Yang, Y., Wang, Y., Liu, G., Wu, L., and Liu, Q. Unitabe: A universal pretraining protocol for tabular foundation model in data science, 2024. URL https://arxiv.org/abs/2307.09249

2024 arXiv
[38]

Tabert: Pretraining for joint understanding of textual and tabular data, 2020

Yin, P., Neubig, G., tau Yih, W., and Riedel, S. Tabert: Pretraining for joint understanding of textual and tabular data, 2020. URL https://arxiv.org/abs/2005.08314

2020 arXiv
[39]

GAIN : Missing data imputation using generative adversarial nets

Yoon, J., Jordon, J., and van der Schaar, M. GAIN : Missing data imputation using generative adversarial nets. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.\ 568...

2018
[40]

Vime: Extending the success of self- and semi-supervised learning to tabular domain

Yoon, J., Zhang, Y., Jordon, J., and van der Schaar, M. Vime: Extending the success of self- and semi-supervised learning to tabular domain. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 3...

2020
[41]

and Sull, S

Yoon, S. and Sull, S. Gamin: Generative adversarial multiple imputation network for highly missing data. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 8453--8461, 2020. doi:10.1109/CVPR42600.2020.00848

2020
[42]

Zhang, H., Fang, L., and Yu, P. S. Unleashing the potential of diffusion models for incomplete data imputation, 2024 a . URL https://arxiv.org/abs/2405.20690

2024 arXiv
[43]

mgte: Generalized long-context text representation and reranking models for multilingual text retrieval, 2024 b

Zhang, X., Zhang, Y., Long, D., Xie, W., Dai, Z., Tang, J., Lin, H., Yang, B., Xie, P., Huang, F., Zhang, M., Li, W., and Zhang, M. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval, 2024 b . URL https://arxiv.org/abs/2407.19669

2024 arXiv
[44]

and Charoenphakdee, N

Zheng, S. and Charoenphakdee, N. Diffusion models for missing value imputation in tabular data, 2023. URL https://arxiv.org/abs/2210.17128

2023 arXiv
[45]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., and McDermott, M

Alsentzer, E., Murphy, J. R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., and McDermott, M. B. A. Publicly available clinical bert embeddings, 2019. URL https://arxiv.org/abs/1904.03323

arXiv 2019

[2] [2]

J., Kendler, K., Pajukanta, P., Flint, J., Zaitlen, N., and et al

An, U., Pazokitoroudi, A., Alvarez, M., Huang, L., Bacanu, S., Schork, A. J., Kendler, K., Pajukanta, P., Flint, J., Zaitlen, N., and et al. Deep learning-based phenotype imputation on population-scale biobank data increases genetic discoveries. Nature Genetics, 55 0 (12): 0 2269–2276, Nov 2023. doi:10.1038/s41588-023-01558-w

work page doi:10.1038/s41588-023-01558-w 2023

[3] [3]

A., Jeong, M., Gorla, A., Chiang, J

An, U., Lee, S. A., Jeong, M., Gorla, A., Chiang, J. N., and Sankararaman, S. Dk-behrt: Teaching language models international classification of disease (icd) codes using known disease descriptions. In Proceedings of The First AAAI Bridge Program on AI for Medicine and Healthcare, volume 281 of Proceedings of Machine Learning Research, pp.\ 133--143. PMLR...

work page 2025

[4] [4]

Arik, S. O. and Pfister, T. Tabnet: Attentive interpretable tabular learning, 2020. URL https://arxiv.org/abs/1908.07442

arXiv 2020

[5] [5]

Multiple Imputation via Generative Adversarial Network for High-dimensional Blockwise Missing Value Problems

Dai, Z., Bu, Z., and Long, Q. Multiple imputation via generative adversarial network for high-dimensional blockwise missing value problems, 2021. URL https://arxiv.org/abs/2112.11507

work page Pith review arXiv 2021

[6] [6]

Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805

arXiv 2019

[7] [7]

Remasker: Imputing tabular data with masked autoencoding

Du, T., Melis, L., and Wang, T. Remasker: Imputing tabular data with masked autoencoding. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=KI9NqjLVDT

work page 2024

[8] [8]

Position information in transformers: An overview, 2021

Dufter, P., Schmitt, M., and Schütze, H. Position information in transformers: An overview, 2021. URL https://arxiv.org/abs/2102.11090

arXiv 2021