Pith. sign in

REVIEW 14 cited by

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2005.08314 v1 pith:H5LYIHJV submitted 2020-05-17 cs.CL cs.LG

Pengcheng Yin , Graham Neubig , Wen-tau Yih , Sebastian Riedel This is my paper
classification cs.CLcs.LG
keywords tabertdatasemanticstructuredtablesfree-formlanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks. Such models are typically trained on free-form NL text, hence may not be suitable for tasks like semantic parsing over structured data, which require reasoning over both free-form NL questions and structured tabular data (e.g., database tables). In this paper we present TaBERT, a pretrained LM that jointly learns representations for NL sentences and (semi-)structured tables. TaBERT is trained on a large corpus of 26 million tables and their English contexts. In experiments, neural semantic parsers using TaBERT as feature representation layers achieve new best results on the challenging weakly-supervised semantic parsing benchmark WikiTableQuestions, while performing competitively on the text-to-SQL dataset Spider. Implementation of the model will be available at http://fburl.com/TaBERT .

Discussion (0). Sign in to comment.

Forward citations

Cited by 14 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PIPER: Content-Based Table Search via profiling and LLM-Generated Pseudoqueries

    cs.IR 2026-05 unverdicted novelty 6.0 of 10

    PIPER retrieves and ranks tabular datasets by profiling their content and using LLM-generated queries for dense vector search, outperforming metadata baselines and TableQA methods in low-metadata settings.

  2. Structure-Aware Chunking for Tabular Data in Retrieval-Augmented Generation

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    STC reduces tabular chunk counts by up to 56% versus baselines and raises hybrid MRR to 0.5945 and BM25 Recall@1 to 0.754 by preserving row structure during chunking.

  3. Generalizing Numerical Reasoning in Table Data through Operation Sketches and Self-Supervised Learning

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    TaNOS improves cross-domain numerical reasoning over tables by combining header anonymization, operation sketches, and self-supervised pretraining, achieving 80.13% accuracy on FinQA with 10% of training data.

  4. TEN: Table Explicitization, Neurosymbolically

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    A neurosymbolic system with structural decomposition prompting and a checker-driven self-debug loop improves table extraction from semistructured text over purely neural baselines.

  5. Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation

    cs.CL 2025-02 unverdicted novelty 6.0 of 10

    KaSLA applies knapsack optimization hierarchically to schema linking for LLM text-to-SQL, claiming better results than large models and improved SQL generation on Spider and BIRD.

  6. TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Shared-backbone low-rank ensemble adapters let neural tabular models match much of full-ensemble accuracy on large data without linear parameter growth or frequent OOMs.

  7. Generalizing Numerical Reasoning in Table Data through Operation Sketches and Self-Supervised Learning

    cs.LG 2026-04 unverdicted novelty 5.0 of 10

    TaNOS decouples table semantics from numerical structure via anonymization, sketches, and program-first self-supervision, yielding 80.13% FinQA accuracy with 10% data and near-zero cross-domain gap versus over 10pp fo...

  8. TabEmb: Joint Semantic-Structure Embedding for Table Annotation

    cs.LG 2026-04 unverdicted novelty 5.0 of 10

    TabEmb decouples LLM-based semantic column embeddings from graph-based structural modeling to produce joint representations that improve table annotation tasks.

  9. ExpliCIT-QA: Explainable Code-Based Image Table Question Answering

    cs.CL 2025-07 conditional novelty 5.0 of 10

    ExpliCIT-QA turns table-image question answering into a traceable sequence of table extraction, natural-language reasoning, code generation, execution, and explanation.

  10. XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQL

    cs.CL 2025-07 unverdicted novelty 5.0 of 10

    XiYan-SQL achieves SOTA Text-to-SQL accuracy by combining schema filtering, a multi-generator ensemble fine-tuned on varied SQL formats, and a selection model.

  11. Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretraining

    cs.LG 2024-03 unverdicted novelty 5.0 of 10

    Table-specific pretraining of Llama-2 yields significant gains on zero-shot, few-shot, and in-context tabular prediction tasks over prior benchmarks.

  12. When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tables

    cs.CL 2025-09 unverdicted novelty 4.0 of 10

    EnoTab is a dual denoising framework for TableQA that performs evidence-based question denoising via semantic unit decomposition and evidence tree-guided table pruning with post-order rollback to improve performance o...

  13. Ta-G-T: Subjectivity Capture in Table to Text Generation via RDF Graphs

    cs.CL 2025-07 conditional novelty 4.0 of 10

    A modular T5-based pipeline using RDF triples, sentence aggregation, and style transfer generates factual text with subjective interpretations from tables, achieving moderate gains over several LLM baselines.

  14. Interactive Text-to-SQL via Expected Information Gain for Disambiguation

    cs.DB 2025-07 reject novelty 4.0 of 10

    An interactive text-to-SQL framework selects clarification questions by expected information gain over a distribution of candidate SQL queries.

Pith tools