Empowering Tabular Data Preparation with Language Models: Why and How?

Jianwei Wang; Kai Wang; Mengshi Chen; Tengchao Li; Wenjie Zhang; Xuemin Lin; Ying Zhang; Yuxiang Sun

REVIEW 4 major objections 7 minor 13 references

Empowering Tabular Data Preparation with Language Models: Why and How?

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Language models match what tabular data preparation needs—semantic understanding, generation, instruction following—and this survey maps the field into four phases and two strategies, showing which pattern fits where.

desk verdict A useful, well-organized survey of LM-based tabular data preparation whose main soft spot is a qualitative 'why' narrative; worth peer review and a place in the reading group. read the letter →

arxiv 2508.01556 v1 pith:EPT6QWEP submitted 2025-08-03 cs.AI

Mengshi Chen , Yuxiang Sun , Tengchao Li , Jianwei Wang , Kai Wang , Xuemin Lin , Ying Zhang , Wenjie Zhang This is my paper

classification cs.AI

keywords tabulardatapreparationlanguagemodelslargeacquisitionintegrationcleaningtransformationpromptengineering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most tabular data preparation is still done with rules and task-specific models that miss the semantic relationships inside tables. This survey argues that language models—particularly large ones—bring exactly the semantic understanding, generative ability, and instruction-following skill that preparation tasks need, and it organizes the recent literature around that claim. It works through four phases (data acquisition, integration, cleaning, transformation) and, for each, explains which of two enabling strategies fits: LM-centric use through prompting or fine-tuning, or LM-in-the-loop use where the model acts as an encoder or decoder inside a larger pipeline. The practical payoff is a decision map that ties task nature to LM usage, along with an account of where cost, hallucination, and robustness still block full automation.

What carries the argument

The organizing device is a four-phase pipeline—(1) data acquisition (table discovery, joinable and unionable table search), (2) data integration (schema and entity matching), (3) data cleaning (error detection, repair, imputation), and (4) data transformation (format and semantic transformation)—crossed with two enabling strategies, LM-centric (prompt engineering, fine-tuning) and LM-in-the-loop (LM-as-encoder, LM-as-decoder). Within each phase the paper identifies the task's inherent prior—for example, acquisition is query-driven similarity search, integration is pairwise and quadratic, cleaning is instance-level and partly generative, transformation is instruction-driven—and uses that prior to explain which LM strategy dominates. This task-demand matching, not any single model or benchmark, carries the survey's argument.

What would settle it

A systematic benchmark that applies the best available LM-based pipeline and the best non-LM baseline (rule-based or classical machine learning) to the same tables across all four phases, measuring prepared-data quality with the paper's own metrics (accuracy, F1, RMSE, downstream-task gain), would settle the claim. If non-LM baselines matched or beat LM pipelines once cost is equalized, the claimed capability-to-task match would be refuted; a consistent LM advantage under controlled comparison would confirm it.

Watch

Extended reading notes

Core claim

The paper's central claim is that the reason LMs fit tabular data preparation is a capability-to-task match: preparation tasks vary in whether they are query-driven or pairwise, discriminative or generative, schema-level or instance-level, and LMs' strengths—semantic understanding, contextual encoding, generative repair, instruction following—line up with those demands. On the 'how' side, the survey organizes recent methods into two enabling strategies. LM-centric strategies use the model directly, through prompt engineering or fine-tuning; LM-in-the-loop strategies use LMs as encoders, producing semantic representations for retrieval or classification, or as decoders, generating repairs, imputations, or transformed records. Across the four phases this yields recognizable patterns: data acquisition is framed as similarity search with LM encoders; data integration uses LLMs as rerankers or matchers in block-and-match pipelines; data cleaning exploits LMs as generative correctors and imputers; data transformation leans on instruction-driven code and formula generation. The survey's contribution is this unifying organization plus a phase-by-phase assessment of what works and what remains open.

Load-bearing premise

The survey's account stands on the assumption that the four phases and the two strategies (LM-centric versus LM-in-the-loop) capture the real design space of tabular data preparation, and that the qualitative reasons given for why LMs help—semantic understanding, generative ability, instruction following—are representative of what actually drives the surveyed systems' performance, without a systematic empirical comparison against non-LM baselines.

Editorial extensions

If this is right

If the taxonomy is right, the choice of LM strategy is predictable from task type: retrieval-shaped acquisition tasks call for encoder-based representation-index-search, while generative cleaning tasks call for prompted or fine-tuned decoders.
Hybrid pipelines—cheap filters or small-language-model encoders pruning candidates before an LLM reranks or repairs—become the standard way to keep LLM-powered preparation affordable.
Distilling LLM knowledge into smaller local models (as in instruction-tuned preprocessors) is a viable route to practical deployment without losing most of the accuracy gain.
The field should move toward multi-agent and full-pipeline orchestration, where planning, execution, and verification are handled by cooperating LM agents, because phase interactions matter (cleaning improves integration, and integration exposes cleaning targets).
Hallucination and inconsistency remain the main reliability barriers; retrieval-augmented and knowledge-grounded variants are the current mitigation, not a solved problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable prediction follows from the paper's logic: on a generative task (repair, imputation, transformation), decoder-style prompting should beat encoder-only representation learning at equal model size, while the reverse should hold on retrieval-style tasks; the survey does not run this comparison, but its taxonomy sets it up.
Because the survey defines preparation as an operator sequence on dataset states, it implicitly treats LM-based preparation as a planning problem; the next step would be to benchmark whole-pipeline LM agents against the best per-phase methods and measure whether orchestration adds value beyond the sum of phase-level gains.
The absence of a unified whole-pipeline benchmark, which the paper itself flags, leaves the qualitative 'why' claims underdetermined; a benchmark spanning all four phases with shared metrics would be the natural test.
The 'why' argument suggests LM gains will concentrate where tables carry rich natural-language-like semantics (headers, descriptions, categorical values) and shrink for purely numeric tables; this boundary condition is untested in the survey.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a solid survey with one real soft spot, and the soft spot is not fatal. What the paper does well: it gives the LM-for-tabular-data-preparation literature a genuinely useful shape, organizing around four phases (acquisition, integration, cleaning, transformation) and two strategies (LM-centric and LM-in-the-loop). The per-phase breakdown of tasks, the pros-and-cons summaries, and the taxonomy table make it easy to find what has been tried, where the approaches cluster, and where gaps are. The citation list is current and broad, covering 2024–2025 work, and the limitations section is honest about the tabular-only, preparation-only scope. For a survey, that is a real contribution: it is a map, not a new measurement.

The soft spot is exactly what the stress-test note flags. The repeated "why" claims—that LMs fit these tasks because of semantic understanding, flexibility, and generative competence—are asserted from selected cited works rather than backed by a systematic comparison against non-LM baselines. The reader's weakest-assumption note is also right: the four phases and two strategies are a deliberate framing, not an exhaustive partition. Both are acceptable for a survey, but readers should treat the "why" as a set of research hypotheses, not established findings. I would like the paper to say that explicitly in the introduction or limitations, so a busy reader does not walk away with more confidence than the evidence supports.

There are also minor internal inconsistencies that should be cleaned up: Jellyfish appears as prompt engineering in Table 1 but is described as fine-tuning in Section 3.6; the table uses MELT while the text uses MELD; and the imputation method is called CRILM in the text but CLAIM in the table. None of these undermine the survey's value, but they will confuse someone using the table as an index.

Bottom line: this is a paper for researchers entering or navigating this subfield, and for program committees looking for a readable synthesis. It deserves a serious referee. I would accept it with minor revisions—fix the table/text mismatches, and add a sentence framing the "why" arguments as hypotheses based on selected evidence rather than a meta-analytic verdict. The reader's accept vote is right; the stress-test correctly identifies the one caveat to put on the record.

Referee Report

4 major / 7 minor

Summary. This manuscript is a survey of language-model-enabled tabular data preparation. It identifies four core phases—data acquisition (table discovery, joinable and unionable table search), data integration (schema and entity matching), data cleaning (error detection, repair, imputation), and data transformation (format and semantic transformation)—and two enabling strategies: LM-centric methods (prompt engineering, fine-tuning) and LM-in-the-loop methods (LM-as-encoder, LM-as-decoder). For each phase, the survey provides task definitions, a qualitative 'why' rationale based on LM capabilities, a 'how' survey of recent methods with pros and cons, and pointers to benchmarks, challenges, and future directions. Its central claim is that this taxonomy and qualitative synthesis give a systematic and reliable overview of the state of the art.

Significance. If the synthesis is reliable, the survey would be a valuable roadmap for a fast-growing literature: it reduces many ad hoc methods to a small number of recurring patterns (representation–index–search, retrieval–rerank, blocker–matcher, code generation, agent orchestration) and maps them onto concrete preparation tasks. The explicit task definitions in Appendix B and the consolidated taxonomy table are useful assets. The paper ships no code or new experiments; its contribution is organizational, and the 'why' arguments should be read as grounded hypotheses rather than established comparative findings. Those hypotheses are plausible and align with the cited works, but the survey does not provide a systematic evidence audit against non-LM baselines.

major comments (4)

[Table 1 (Appendix D); §3.6; Figure 1] Several entries in the central taxonomy table contradict the main text. Jellyfish is listed as 'Prompt Engineering' in Table 1, but §3.6 describes it as training a local LLM on a multi-task corpus via instruction tuning, and Figure 1 labels it [F] for fine-tuning. GIDCL is listed under Data Repair as 'Prompt Engineering', but §3.4.2 explicitly places it under 'Fine-tuning' and describes a fine-tuned local LLM as the corrector. These mismatches undermine confidence in the survey's organizing artifact; every row of Table 1 should be reconciled with its corresponding section and figure label.
[Table 1; §4.1] The taxonomy and benchmark discussion include several non-tabular items. Table 1 lists SIGLLM, LLMAD, and LogPrompt under Error Detection, yet the cited works target time-series and log anomaly detection, not tabular data preparation; §4.1 cites MEDEC, a clinical-note error-detection benchmark, as an error-detection benchmark without clarifying that it is not tabular. Unless the authors deliberately broaden 'tabular data preparation' to include these modalities, these entries should be removed or explicitly justified, because they blur the survey's stated scope of tabular data.
[§3.2–§3.5 Touchpoint paragraphs; Limitations] The 'why' thread of the paper is asserted rather than systematically evidenced. The touchpoint paragraphs repeatedly state that LMs' semantic understanding, generative ability, or instruction-following makes them suitable for each phase, and the Introduction describes traditional methods as limited, but no systematic comparison against non-LM baselines is provided and the Limitations section does not acknowledge this absence. I do not regard this as a soundness error—surveys may synthesize without meta-analysis—but because the title promises 'Why', the paper should either add a compact evidence summary of reported gains over non-LM baselines or explicitly label the 'why' arguments as research hypotheses.
[§3.2.1] The treatment of OpenWikiTable appears to misattribute. The text says OpenWikiTable (Kweon et al., 2023) 'provides several encoder options such as BERT and TAPAS', but the cited paper is a question-answering dataset, not a table-discovery method with multiple encoder backbones. The authors should verify the citation or rephrase the sentence to describe the actual role of this resource in the discussion.

minor comments (7)

[Table 1; §3.6; Figure 1] The name 'MELT' in Table 1 refers to the model called 'MELD' in §3.6 and Figure 1; the name should be unified.
[§3.3.2; Table 1; References] ComEM is cited as (Wang et al., 2024b) in Table 1 but as (Wang et al., 2025b) in §3.3.2; the citation year should be aligned with the reference list entry actually used.
[§3.2.1] The sentence on Birdie ends with 'that directly determines.' and is incomplete; the missing object should be supplied.
[Figure 1] The CRILM entry misspells the author list as 'Hasanet al.'; it should be corrected.
[Table 1; §3.3; Appendix B] The table uses 'Schema Mapping' and 'Entity Linkage' while the section headings and Appendix B use 'Schema Matching' and 'Entity Matching'; the terminology should be consistent.
[§3.4.3; §4.1] The same work is called 'CRILM' in the text and 'CLAIM' in Table 1; one canonical name should be chosen and used throughout.
[References] The reference list duplicates Hameed and Naumann (2020a/2020b) and Devlin et al. (2019a/2019b); these duplicate entries should be merged or replaced by a single canonical citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey organizes external literature and makes no derived predictions that reduce to its inputs.

full rationale

This paper is a survey, not a derivation. Its central contribution is a taxonomy (four phases: data acquisition, integration, cleaning, transformation; two enabling strategies: LM-centric and LM-in-the-loop) and qualitative syntheses of how language models are applied to tabular data preparation. There is no fitted parameter that is then relabeled as a prediction, no claimed first-principles result, and no uniqueness theorem invoked to force a choice. The 'why' arguments (e.g., LMs have strong semantic understanding, or that data repair is generative and thus suited to LMs) are qualitative assessments supported by citation to external work such as Ditto, Table-GPT, MatchGPT, and Jellyfish; they are not derived from the survey's own definitions or equations. The paper does cite works with overlapping authorship, notably UnIMP (Wang et al., 2025a) and the uncertainty-driven imputation network (Wang et al., 2024a), but these are presented as surveyed methods among many and are not load-bearing for the survey's organizing claims. The Limitations section appropriately narrows the scope to tabular data and preparation, and explicitly notes that conclusions may not generalize beyond this setting; this is a scope limitation, not circularity. No step in the survey reduces by construction to its inputs, so the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

This survey makes no new predictions or derivations. It relies on the accuracy and representativeness of the cited literature, and on the chosen organizational framework. No free parameters or invented entities are introduced.

assumptions (2)

domain assumption The selected references accurately represent the state of the art in each phase.
The survey's taxonomy and conclusions are derived from a curated set of papers. If the selection is biased or incomplete, the characterizations could be misleading.
domain assumption The four phases (acquisition, integration, cleaning, transformation) cover the main scope of tabular data preparation.
The survey explicitly limits scope to these phases in Section 1. Other aspects like data versioning or labeling are excluded, which could make the taxonomy incomplete.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Empowering Tabular Data Preparation with Language Models: Why and How?." pith.science (2026). https://pith.science/paper/EPT6QWEP

@misc{pith2026250801556,
  author       = {Pith},
  title        = {Pith review of: Empowering Tabular Data Preparation with Language Models: Why and How?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EPT6QWEP}},
  note         = {Machine review of arXiv:2508.01556}
}

read the original abstract

Data preparation is a critical step in enhancing the usability of tabular data and thus boosts downstream data-driven tasks. Traditional methods often face challenges in capturing the intricate relationships within tables and adapting to the tasks involved. Recent advances in Language Models (LMs), especially in Large Language Models (LLMs), offer new opportunities to automate and support tabular data preparation. However, why LMs suit tabular data preparation (i.e., how their capabilities match task demands) and how to use them effectively across phases still remain to be systematically explored. In this survey, we systematically analyze the role of LMs in enhancing tabular data preparation processes, focusing on four core phases: data acquisition, integration, cleaning, and transformation. For each phase, we present an integrated analysis of how LMs can be combined with other components for different preparation tasks, highlight key advancements, and outline prospective pipelines.

Figures

Figures reproduced from arXiv: 2508.01556 by the authors.

**Figure 2.** Framework of applying LMs for tabular data [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

[4]

Prompt-Matcher: Leveraging Large Models to Reduce Uncertainty in Schema Matching Results

Cost-aware uncertainty reduction in schema matching with GPT-4: the prompt-matcher frame- work. CoRR, abs/2408.14507. Jerome H Friedman. 2001. Greedy function approx- imation: a gradient boosting machine. Annals of statistics, pages 1189–1232. Yuhang Ge, Yachuan Liu, Yuren Mao, and Yunjun Gao. 2025. Text-to-pipeline: Bridging natural language and data pre...

work page Pith review arXiv 2001
[5]

A Context-Aware Approach for Enhancing Data Imputation with Pre-trained Language Models

CLAIM your data: Enhancing imputation ac- curacy with contextual large language models.CoRR, abs/2405.17712. Xinrui He, Yikun Ban, Jiaru Zou, Tianxin Wei, Curtiss B. Cook, and Jingrui He. 2024. Llm- forest for health tabular data imputation. CoRR, abs/2410.21520. Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun...

work page Pith review arXiv 2024
[9]

IOP Confer- ence Series: Materials Science and Engineering , 1090(1):012053

Data mining preparation: Process, techniques and major issues in data analysis. IOP Confer- ence Series: Materials Science and Engineering , 1090(1):012053. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with gpus. IEEE Trans. Big Data, 7(3):535–547. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Ch...

arXiv 2021
[10]

Anthropic

IEEE. Anthropic. 2023. Anthropic claude. https://www. anthropic.com/index/claude. Gustavo EAPA Batista and Maria Carolina Monard

work page 2023
[11]

CoRR, abs/2309.07864

The rise and potential of large language model based agents: A survey. CoRR, abs/2309.07864. Yongqin Xu, Huan Li, Ke Chen, and Lidan Shou

arXiv
[12]

CoRR, abs/2410.12480

Kcmf: A knowledge-compliant framework for schema and entity matching with fine-tuning-free llms. CoRR, abs/2410.12480. Mengyi Yan, Yaoshu Wang, Kehan Pang, Min Xie, and Jianxin Li. 2024a. Efficient mixture of experts based on large language models for low-resource data preprocessing. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery ...

arXiv
[13]

erroneous

and Claude (Anthropic, 2023), are emerging. B Task Definitions B.1 Data Acquisition Definition 1 (Table Discovery): Given a natu- ral language query q and a table repository T = {T1, T2, . . . , TN }, the goal of table discovery is to retrieve the top-k tables Tt ∈ Tthat are most rel- evant to the query. The relevance is measured by a ranking score Rdisc(...

work page 2023
[309]

Sebastian Jäger, Arndt Allhorn, and Felix Bießmann

IEEE. Sebastian Jäger, Arndt Allhorn, and Felix Bießmann

work page

Show all 13 references

[2003]

Applied artificial intelligence, 17(5-6):519–533

An analysis of four missing data treatment methods for supervised learning. Applied artificial intelligence, 17(5-6):519–533. Jason Brownlee. 2020. Data preparation for machine learning: data cleaning, feature selection, and data transforms in Python. Machine Learning Mastery....

2020 arXiv
[2020]

arXiv preprint arXiv:2010.14701

Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701. Jonathan Herzig, Thomas Müller, Syrine Krichene, and Julian Martin Eisenschlos. 2021. Open domain ques- tion answering over tables via dense retrieval. In Proceedings of the 2021 Conference o...

2010 arXiv
[2021]

Frontiers Big Data, 4:693674

A benchmark for data imputation methods. Frontiers Big Data, 4:693674. Abhinav Jain, Hima Patel, Lokesh Nagalapatti, Nitin Gupta, Sameep Mehta, Shanmukha Guttula, Shashank Mujumdar, Shazia Afzal, Ruhi Sharma Mit- tal, and Vitobha Munigala. 2020. Overview and im- portance of da...

2020
[2023]

In IEEE International Conference on Big Data, BigData 2023, Sorrento, Italy, December 15-18, 2023, pages 1824–1834

Automatic data transformation using large language model - an experimental study on building energy data. In IEEE International Conference on Big Data, BigData 2023, Sorrento, Italy, December 15-18, 2023, pages 1824–1834. IEEE. Ankita Sharma, Jaykumar Tandel, Xuanmao Li, Lan- ...

2023 arXiv
[2024]

CoRR, abs/2412.19260

MEDEC: A benchmark for medical error detection and correction in clinical notes. CoRR, abs/2412.19260. Sarah Alnegheimish, Linh Nguyen, Laure Berti-Équille, and Kalyan Veeramachaneni. 2024. Can large lan- guage models be anomaly detectors for time se- ries? In 11th IEEE Intern...

2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [4]

Prompt-Matcher: Leveraging Large Models to Reduce Uncertainty in Schema Matching Results

Cost-aware uncertainty reduction in schema matching with GPT-4: the prompt-matcher frame- work. CoRR, abs/2408.14507. Jerome H Friedman. 2001. Greedy function approx- imation: a gradient boosting machine. Annals of statistics, pages 1189–1232. Yuhang Ge, Yachuan Liu, Yuren Mao, and Yunjun Gao. 2025. Text-to-pipeline: Bridging natural language and data pre...

work page Pith review arXiv 2001

[2] [5]

A Context-Aware Approach for Enhancing Data Imputation with Pre-trained Language Models

CLAIM your data: Enhancing imputation ac- curacy with contextual large language models.CoRR, abs/2405.17712. Xinrui He, Yikun Ban, Jiaru Zou, Tianxin Wei, Curtiss B. Cook, and Jingrui He. 2024. Llm- forest for health tabular data imputation. CoRR, abs/2410.21520. Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun...

work page Pith review arXiv 2024

[3] [9]

IOP Confer- ence Series: Materials Science and Engineering , 1090(1):012053

Data mining preparation: Process, techniques and major issues in data analysis. IOP Confer- ence Series: Materials Science and Engineering , 1090(1):012053. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with gpus. IEEE Trans. Big Data, 7(3):535–547. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Ch...

arXiv 2021

[4] [10]

Anthropic

IEEE. Anthropic. 2023. Anthropic claude. https://www. anthropic.com/index/claude. Gustavo EAPA Batista and Maria Carolina Monard

work page 2023

[5] [11]

CoRR, abs/2309.07864

The rise and potential of large language model based agents: A survey. CoRR, abs/2309.07864. Yongqin Xu, Huan Li, Ke Chen, and Lidan Shou

arXiv

[6] [12]

CoRR, abs/2410.12480

Kcmf: A knowledge-compliant framework for schema and entity matching with fine-tuning-free llms. CoRR, abs/2410.12480. Mengyi Yan, Yaoshu Wang, Kehan Pang, Min Xie, and Jianxin Li. 2024a. Efficient mixture of experts based on large language models for low-resource data preprocessing. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery ...

arXiv

[7] [13]

erroneous

and Claude (Anthropic, 2023), are emerging. B Task Definitions B.1 Data Acquisition Definition 1 (Table Discovery): Given a natu- ral language query q and a table repository T = {T1, T2, . . . , TN }, the goal of table discovery is to retrieve the top-k tables Tt ∈ Tthat are most rel- evant to the query. The relevance is measured by a ranking score Rdisc(...

work page 2023

[8] [309]

Sebastian Jäger, Arndt Allhorn, and Felix Bießmann

IEEE. Sebastian Jäger, Arndt Allhorn, and Felix Bießmann

work page