Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Rethinking Table Instruction Tuning

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Smaller learning rate and 2,600 examples produce a table LLM that beats GPT-3.5 and often GPT-4.

desk verdict The data-efficiency recipe is credible and useful; the 'surpasses GPT' headline rests on uncontrolled cited baselines and a 500-point subsample ambiguity. read the letter →

arxiv 2501.14693 v4 pith:H3HEDH5V submitted 2025-01-24 cs.CL cs.AI

Naihao Deng , Rada Mihalcea This is my paper · ORCID
classification cs.CLcs.AI
keywords tableunderstandinginstructiontuninglearningratedataefficiencyLLMfine-tuningout-of-domaingeneralizationgeneralcapabilitiesTAMA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that table instruction tuning has been tuned with the wrong dials: prior open table LLMs train on hundreds of thousands to millions of examples at a large learning rate, and as a result they lose out-of-domain table skill and general ability. The paper claims that a much smaller learning rate near $10^{-6}$, two epochs, and only 200 randomly sampled instruction pairs per dataset (2,600 pairs total) produce stronger table understanding while preserving the base model's general capabilities. Applied to LLaMA 3.1 8B Instruct, that recipe yields TAMA, which the paper reports matches or beats GPT-3.5 on most table benchmarks and exceeds GPT-4 on several, while keeping MMLU and IFEval near base levels. The payoff the authors aim at is lower annotation cost and more efficient model development through careful hyperparameter selection.

What carries the argument

The central object is a training recipe rather than a new architecture: full-parameter instruction tuning of LLaMA 3.1 8B Instruct at learning rate $10^{-6}$ for two epochs on 200 randomly sampled instruction pairs from each of thirteen table datasets, 2,600 pairs in total. The machinery that justifies it is a systematic sweep over learning rate ($10^{-7}$ to $10^{-5}$), number of examples (30 to 1,500), and epochs (1 to 6), evaluated on table tasks and on general benchmarks, plus a hindsight grid over the final setup confirming that the chosen hyperparameters sit on the best-performing ridge. The paper's interpretation is that the base model already holds most of the needed table reasoning, so a few hundred examples at a small learning rate sharpen that ability without erasing general knowledge.

What would settle it

Train TAMA ten times, each with a different random seed for the 200-example draw, and compare the spread on HybridQA (reported 60.86 versus 32.83 for the base model). If some draws fall well below that figure or the variance exceeds roughly five accuracy points, the data-efficiency claim rests on the particular sample rather than the recipe.

Watch

Extended reading notes

Core claim

Existing open table LLMs—TableLLaMA, TableLLM, and TableBenchLLM—all use a learning rate of $2 \times 10^{-5}$ and training sets of 20K to 2M examples, and the paper measures that every one of them degrades on held-out table tasks such as Table-Syn and on general benchmarks such as MMLU, AI2ARC, and IFEval compared with its base model. Through full-parameter tuning experiments on LLaMA 3.1 8B Instruct with data from FeTaQA, HiTab, and TabFact, the paper finds a learning-rate sweet spot near $10^{-6}$ to $5 \times 10^{-7}$: larger rates near $10^{-5}$ damage general capabilities, while smaller rates near $10^{-7}$ leave table gains on the table. Performance rises sharply over the first 200 examples and then plateaus, and extra epochs add little. Combining these findings, the paper trains TAMA for two epochs at learning rate $10^{-6}$ on 200 randomly selected instruction pairs from each of thirteen datasets, and reports that this 8B model surpasses GPT-3.5 on nearly every table benchmark, beats GPT-4 on WikiSQL, HybridQA, InfoTabs, and FEVEROUS, and keeps MMLU, AI2ARC, and most of IFEval at or above base-model levels.

Load-bearing premise

The recipe assumes that 200 randomly drawn instruction pairs from each dataset are representative enough that the large reported gains are not a favorable draw; the paper reports no repeated sampling, seeds, or variance estimates.

Editorial extensions

If this is right

  • Table instruction tuning can be done with roughly 2,600 instruction pairs rather than 20K to 2M, cutting annotation cost by orders of magnitude.
  • A learning rate near $10^{-6}$, not $2 \times 10^{-5}$, should become the default starting point for full-parameter table tuning of instruct-tuned 7B/8B models; the same qualitative pattern holds across Llama 2, Llama 3.1, Qwen 2.5, Mistral, and Phi in the paper's appendix.
  • Specializing a model on tables need not trade away general knowledge: TAMA keeps MMLU and AI2ARC at or above base level and retains most IFEval instruction-following, so table-specialized assistants remain usable for general requests.
  • Out-of-domain table generalization is achievable with this recipe, as TAMA outperforms GPT-3.5 on two synthesized table benchmarks whose distributions differ from all training data.
  • Existing table LLMs' reported drops on general benchmarks are largely attributable to an overly large learning rate combined with long training, not to table tuning itself.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This inference is mine: the 200-example-per-dataset recipe should transfer to adjacent structured-data tasks such as text-to-SQL and document question answering, but only if the base model already has strong instruction following; for weaker base models the required sample size is likely larger.
  • This inference is mine: because the paper reports a single random draw of 200 examples per dataset without seeds or variance estimates, the headline numbers should be read as point estimates; repeating the draw across seeds is the direct test of whether the recipe or the particular sample carries the result.
  • This inference is mine: the learning-rate sweet spot probably shifts with model scale and with LoRA/QLoRA, so a practical recipe should re-sweep the learning rate whenever the tuning method changes.
  • This inference is mine: the paper's observation that MMLU STEM scores improve after table tuning suggests table instruction data may double as targeted math-reasoning training; a testable extension is to measure STEM gains on a separate math benchmark.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies instruction tuning of LLMs for table understanding, focusing on hyperparameter choices rather than data scale. It first evaluates three open-source table LLMs (TableLLaMA, TableLLM, TableBenchLLM) on out-of-domain table understanding (Table-Syn) and general benchmarks (IFEval, MMLU, MMLU Pro, AI2ARC, GPQA), finding large drops relative to their base models. It then performs a systematic exploration on LLaMA 3.1 8B Instruct over learning rate, number of training examples, epochs, multi-task training, and additional setups (LoRA/QLoRA, other base models), concluding that a small learning rate (around 1e-6), two epochs, and 200 random examples per dataset (2,600 total) yield strong table understanding while preserving general capabilities. The resulting model, TAMA, is reported to match or exceed GPT-3.5 and GPT-4 on several table benchmarks while retaining MMLU, IFEval, and AI2ARC performance. The paper also releases code, models, and data.

Significance. If the central recipe is robust, the paper makes a practically valuable contribution: it suggests that table instruction tuning can be far more data-efficient than the multi-million-example corpora used by prior systems (TableLLaMA, TableBenchLLM), and that careful hyperparameter selection can avoid the general-capability degradation reported for existing table LLMs. The exploration is unusually broad for this area, spanning full-parameter tuning across five model families (Table 14), LoRA/QLoRA (Tables 16-17), varying training sizes, and epoch counts, with consistent diminishing-returns trends across settings. The open release of code, model weights, and the 2.6K instruction dataset increases the reproducibility value of the work. However, the two headline claims—that TAMA surpasses GPT-3.5/GPT-4 and that the 200-example recipe is reliably optimal—are weakened by uncontrolled baseline comparisons and by selection on test benchmarks, as detailed in the major comments.

major comments (3)
  1. [Section 4.2, Table 7] The abstract's claim that TAMA is 'on par with, or surpassing GPT-3.5 and GPT-4 on table tasks' is supported only by Table 7, whose GPT columns are a mixture of cited scores: the dagger footnote assigns 11 of the 13 GPT-3.5 and GPT-4 values to Gou et al. (2023), Srivastava et al. (2024), and Zhang et al. (2024a). These scores come from different papers with different prompts, decoding settings, and possibly different test splits. Footnote 2 states that 'Due to budget limit for prompting GPT models, we uniformly sample 500 data points from the original test set as our test set,' but the paper does not state that TAMA's numbers are computed on the same 500-instance subset, and for the cited datasets no matched subsampling is described. The win/loss summary in Section 4.2 is therefore not a controlled comparison. To support the advertised comparison to GPT-3.5/GPT-4, the authors should re-run the proprietary baselines under identical evaluation conditions (same instances, prompts, decoding, and scoring) for all datasets, or explicitly restrict the claim to the two datasets (AIT-QA and InfoTabs) for which they appear to have run GPT themselves.
  2. [Section 4.3, Figure 3] The 'hindsight analysis' validates the selected hyperparameters by training with learning rates from 1.0e-7 to 1.0e-5 and one to six epochs, and then aggregating performance on 'all the datasets listed in Table 6' (Section 4.3, Figure 3a). These are the same test and development sets whose scores are later reported as TAMA's final results in Table 7. Selecting the configuration that maximizes performance on these benchmarks means the reported TAMA numbers are not clean held-out estimates; the 'hindsight' framing does not remove the selection. Additionally, Figure 3a sums raw scores across tasks with incompatible metrics (accuracy, BLEU, ROUGE-L, F1), which is not a meaningful aggregate. The paper should reserve a held-out set of table benchmarks for final evaluation, or present Section 4.3 as an exploratory diagnostic and avoid using it as evidence that the selected hyperparameters are optimal.
  3. [Section 3.2, Section 4.1 (Table 6)] The data-efficiency claim rests on 200 randomly selected instruction pairs per dataset, but no random seeds or repeated draws are reported. Section 3.2's learning curves (Figure 2) and the final TAMA results (Section 4.1, Table 7) are based on single realizations of the random subset, so the large gains on some test sets (e.g., HybridQA 60.86 vs. base 32.83) could partly reflect a favorable draw rather than a robust property of the recipe. Since the paper's central recommendation is that 200 examples per dataset suffice, the authors should report mean and variance over at least three seeds, or provide a bootstrap over the training pool, to establish that the recipe transfers to new random subsets and new datasets.
minor comments (5)
  1. [Abstract] The sentence 'We post our updates in Table 1.' is out of place in an abstract; it should be removed or moved to the appendix.
  2. [Section 4.2] The text states 'TAMA achieves 52.81' for WikiTQ, but Table 7 reports 52.88; the numbers should be made consistent.
  3. [Section 3.2] The phrase 'the learning rate is around 1.0e-6 and 5.0e-7' should read 'around 1.0e-6 or 5.0e-7' to avoid ambiguity about whether both values are recommended simultaneously.
  4. [Figure 2] The learning-curve plots do not show numeric values or error bars, which makes it difficult to verify the claim that performance improvement beyond 200 examples is 'marginal'; a companion table with the plotted values would strengthen the exposition.
  5. [Section 4.3, Table 10] The explanation that table training improves STEM performance in MMLU because table tasks involve math reasoning is plausible but speculative; it should be clearly framed as a hypothesis rather than a demonstrated conclusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: hyperparameters are selected on separate dev sweeps, and the final test evaluations are out-of-sample; the Section 4.3 hindsight sweep is a test-set reuse concern, not a derivation loop.

full rationale

This is an empirical paper with no equation-level derivation, and I find no circular step that reduces a claimed prediction to its own inputs by construction. The hyperparameters (LR 1e-6, two epochs, 200 examples per dataset) are chosen in Section 3 from sweeps on FeTaQA, HiTab, and TabFact, with FEVEROUS, MMLU, and IFEval as auxiliary checks. TAMA is then evaluated on nine additional test datasets plus the out-of-domain Table-Syn sets (S1, S2) and on general benchmarks (MMLUPro, AI2ARC, GPQA) that were not part of those sweeps, so the main claims about improved table understanding and preserved general ability have independent, out-of-sample content. The closest concern is Section 4.3, where the authors 'in hindsight' sweep learning rates and epochs and aggregate performance over 'all the datasets listed in Table 6'; this is a genuine test-set reuse / model-selection caveat, not a circular derivation, because the final model configuration was fixed from the Section 3 dev analysis and the test results are measurements rather than quantities forced by the selection procedure. The GPT-3.5/GPT-4 comparisons in Table 7 are mostly cited from other papers and involve a 500-point subsample for prompting GPT models, but this is a comparability and reproducibility risk, not circularity. The paper also contains one self-citation (Deng et al. 2024) supporting the text-representation choice, but it is not load-bearing for the central findings. The Limitations section explicitly acknowledges manual hyperparameter tuning and random data sampling, which further supports classifying the concerns as methodological risk rather than circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No derivation is present, so the paper's central recipe rests on hand-selected hyperparameters and benchmark assumptions rather than postulated entities. The hyperparameters (LR, epochs, data size, batch size) act as fitted values selected by experiments, and the benchmark validity and GPT comparability are assumed.

free parameters (5)
  • learning_rate = 1e-6
    Chosen from a grid search over 1e-7 to 1e-5 in Section 3; affects both table and general performance.
  • num_epochs = 2
    Chosen because no significant gains beyond 2 epochs (Section 3 and Appendix D.2).
  • train_instances_per_dataset = 200
    Chosen based on diminishing returns beyond 200 examples (Section 3.2); central to the data-efficiency claim.
  • batch_size = 16
    Selected after a preliminary batch size study (Appendix B.3); avoids the BLEU collapse seen at batch size 64.
  • random_subset_seed = not reported
    Random subsets of 200 examples per dataset are used with no seed reported, so the exact training data is not reproducible and sampling variance is unknown.
assumptions (4)
  • domain assumption Table-Syn, IFEval, MMLU, MMLU Pro, AI2ARC, and GPQA collectively measure out-of-domain table understanding and general capabilities.
    Used to support claims of OOD table ability and general capability preservation (Table 3).
  • domain assumption Reported GPT-3.5/GPT-4 numbers from prior work are comparable despite different prompts, sampling, and evaluation scripts.
    GPT results for several datasets are taken from Gou et al. 2023, Srivastava et al. 2024, Zhang et al. 2024a, with a 500-sample subset for some GPT evals (footnote 2).
  • domain assumption 200 randomly selected instruction pairs from each training dataset are representative of that dataset's task distribution.
    Basis of the data-efficiency claim; no seed or variance analysis is provided.
  • domain assumption Continuing instruction tuning from an already instruction-tuned model preserves general abilities better than tuning the base model.
    Supported by Figure 2f vs Figure 4c, but still a modeling assumption about the right starting point.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking Table Instruction Tuning." pith.science (2026). https://pith.science/paper/H3HEDH5V

@misc{pith2026250114693,
  author       = {Pith},
  title        = {Pith review of: Rethinking Table Instruction Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3HEDH5V}},
  note         = {Machine review of arXiv:2501.14693}
}
read the original abstract

Recent advances in table understanding have focused on instruction-tuning large language models (LLMs) for table-related tasks. However, existing research has overlooked the impact of hyperparameter choices, and also lacks a comprehensive evaluation of the out-of-domain table understanding ability and the general capabilities of these table LLMs. In this paper, we evaluate these abilities in existing table LLMs, and find significant declines in both out-of-domain table understanding and general capabilities as compared to their base models. Through systematic analysis, we show that hyperparameters, such as learning rate, can significantly influence both table-specific and general capabilities. Contrary to the previous table instruction-tuning work, we demonstrate that smaller learning rates and fewer training instances can enhance table understanding while preserving general capabilities. Based on our findings, we introduce TAMA, a TAble LLM instruction-tuned from LLaMA 3.1 8B Instruct, which achieves performance on par with, or surpassing GPT-3.5 and GPT-4 on table tasks, while maintaining strong out-of-domain generalization and general capabilities. Our findings highlight the potential for reduced data annotation costs and more efficient model development through careful hyperparameter selection. We open-source the project and our models.

Figures

Figures reproduced from arXiv: 2501.14693 by the authors.

Figure 1
Figure 1. Performance comparison between our model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. LLaMA 3.1 8B Instruct’s performance (y-axis) with respect to the number of training instances (x-axis). [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Performance scores (y-axis) with respect to the number of epochs (x-axis) and learning rates. In Figure [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: LLaMA 3.1 8B’s accuracy scores (y-axis) on TabFact, MMLU, and IFEval with respect to the number of [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: LLaMA 3.1 8B Instruct model’s performance [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Heatmap when we fine-tune the LLaMA 3.1 8B Instruct model on a single dataset (y-axis) and test against the others (x-axis). In this plot, we fine-tune the model for two epochs at a learning rate of 1.0e-6 with 500 instruction pairs. marks. Interestingly, for models su…
Figure 7
Figure 7. Figure 7: Heatmap when we fine-tune LLaMA 3.1 8B Instruct model on a single dataset (y-axis) and test against the [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Heatmap when we fine-tune LLaMA 3.1 8B Instruct model on a single dataset (y-axis) and test against the [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Performance scores (y-axis) with respect to the number of epochs (x-axis) and learning rates. In Figure [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Table-r1 combines a layout-transformation self-supervised task and a mix-paradigm GRPO stage so 7B/8B models outperform other small-model table reasoners and approach GPT-4o-level accuracy.

Reference graph

Works this paper leans on

60 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  4. [4]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

    arXiv 2024

  5. [5]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  6. [6]

    The fact extraction and VER ification over unstructured and structured information ( FEVEROUS ) shared task

    Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. The fact extraction and VER ification over unstructured and structured information ( FEVEROUS ) shared task. In Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlicht...

    2021

  7. [7]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  8. [8]

    How to prompt llms for text-to-sql: A study in zero-shot, single-domain, and cross-domain settings

    Shuaichen Chang and Eric Fosler-Lussier. How to prompt llms for text-to-sql: A study in zero-shot, single-domain, and cross-domain settings. arXiv preprint arXiv:2305.11853, 2023

    arXiv 2023

Show all 60 references
  1. [9]

    Tabfact: A large-scale dataset for table-based fact verification

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv:1909.02164, 2019

    1909 arXiv

  2. [10]

    H ybrid QA : A dataset of multi-hop question answering over tabular and textual data

    Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang. H ybrid QA : A dataset of multi-hop question answering over tabular and textual data. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguistics: E...

    2020 doi

  3. [11]

    H i T ab: A hierarchical table dataset for question answering and natural language generation

    Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. H i T ab: A hierarchical table dataset for question answering and natural language generation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), P...

    2022 doi

  4. [12]

    Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2 0 (3): 0 6, 2023

    2023

  5. [13]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25 0 (70): 0 1--53, 2024

    2024

  6. [14]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018

    2018 arXiv

  7. [15]

    Tables as texts or images: Evaluating the table reasoning ability of LLM s and MLLM s

    Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. Tables as texts or images: Evaluating the table reasoning ability of LLM s and MLLM s. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Co...

    2024

  8. [16]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024

    2024

  9. [17]

    BERT : Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT : Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North A merican Chapter of...

    2019 doi

  10. [18]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  11. [19]

    Key-value retrieval networks for task-oriented dialogue

    Mihail Eric and Christopher D Manning. Key-value retrieval networks for task-oriented dialogue. arXiv preprint arXiv:1705.05414, 2017

    2017 arXiv

  12. [20]

    Critic: Large language models can self-correct with tool-interactive critiquing

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738, 2023

    2023 arXiv

  13. [21]

    INFOTABS : Inference on tables as semi-structured data

    Vivek Gupta, Maitrey Mehta, Pegah Nokhiz, and Vivek Srikumar. INFOTABS : Inference on tables as semi-structured data. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguisti...

    2020 doi

  14. [22]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  15. [23]

    T a P as: Weakly supervised table parsing via pre-training

    Jonathan Herzig, Pawel Krzysztof Nowak, Thomas M \"u ller, Francesco Piccinno, and Julian Eisenschlos. T a P as: Weakly supervised table parsing via pre-training. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting o...

    2020 doi

  16. [24]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  17. [25]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  18. [26]

    AIT-QA : Q uestion answering dataset over complex tables in the airline industry

    Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan, and Soumen Chakrabarti. AIT-QA : Q uestion answering dataset over complex tables in the airline industry. In ...

    2022

  19. [27]

    Neural text generation from structured data with application to the biography domain

    R \'e mi Lebret, David Grangier, and Michael Auli. Neural text generation from structured data with application to the biography domain. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, p...

    2016 doi

  20. [28]

    Table-gpt: Table-tuned gpt for diverse table tasks

    Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. Table-gpt: Table-tuned gpt for diverse table tasks. arXiv preprint arXiv:2310.09263, 2023

    2023 arXiv

  21. [29]

    Table-to-text generation by structure-aware seq2seq learning

    Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. Table-to-text generation by structure-aware seq2seq learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  22. [30]

    Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning

    Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In International Conference on Learning Representations (ICLR), 2023

    2023

  23. [31]

    F e T a QA : Free-form table question answering

    Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry \'s ci \'n ski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, and Drago...

    2022 doi

  24. [32]

    ToTTo : A controlled table-to-text generation dataset

    Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. ToTTo : A controlled table-to-text generation dataset. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Meth...

    2020 doi

  25. [33]

    Compositional semantic parsing on semi-structured tables

    Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In Chengqing Zong and Michael Strube (eds.), Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natura...

    2015 doi

  26. [34]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  27. [35]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022, 2023

    2023 arXiv

  28. [36]

    Assessing llms' mathematical reasoning in financial document question answering

    Pragya Srivastava, Manuj Malik, and Tanuja Ganu. Assessing llms' mathematical reasoning in financial document question answering. arXiv preprint arXiv:2402.11194, 2024

    2024

  29. [37]

    Tablegpt2: A large multimodal model with tabular data integration

    Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, Haoze Li, et al. Tablegpt2: A large multimodal model with tabular data integration. arXiv preprint arXiv:2411.02059, 2024

    2024 arXiv

  30. [38]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  31. [39]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574, 2024

    2024 arXiv

  32. [40]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  33. [41]

    Tablebench: A comprehensive and complex benchmark for table question answering

    Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xinrun Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, et al. Tablebench: A comprehensive and complex benchmark for table question answering. arXiv preprint arXiv:2408.09174, 2024

    2024 arXiv

  34. [42]

    Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A

    Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, N...

    2022

  35. [43]

    Mmtu: A massive multi-task table understanding and reasoning benchmark

    Junjie Xing, Yeye He, Mengyu Zhou, Haoyu Dong, Shi Han, Lingjiao Chen, Dongmei Zhang, Surajit Chaudhuri, and HV Jagadish. Mmtu: A massive multi-task table understanding and reasoning benchmark. arXiv preprint arXiv:2506.05587, 2025

    2025

  36. [44]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025 a

    2025 arXiv

  37. [45]

    T able F ormer: Robust transformer modeling for table-text encoding

    Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, and Shachi Paul. T able F ormer: Robust transformer modeling for table-text encoding. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Associati...

    2022 doi

  38. [46]

    Table-r1: Inference-time scaling for table reasoning

    Zheyuan Yang, Lyuhao Chen, Arman Cohan, and Yilun Zhao. Table-r1: Inference-time scaling for table reasoning. arXiv preprint arXiv:2505.23621, 2025 b

    2025

  39. [47]

    T a BERT : Pretraining for joint understanding of textual and tabular data

    Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. T a BERT : Pretraining for joint understanding of textual and tabular data. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for ...

    2020 doi

  40. [48]

    Tablegpt: Towards unifying tables, nature language and commands into one gpt

    Liangyu Zha, Junlin Zhou, Liyao Li, Rui Wang, Qingyi Huang, Saisai Yang, Jing Yuan, Changbao Su, Xiang Li, Aofeng Su, et al. Tablegpt: Towards unifying tables, nature language and commands into one gpt. arXiv preprint arXiv:2307.08674, 2023

    2023 arXiv

  41. [49]

    T able L lama: Towards open large generalist models for tables

    Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. T able L lama: Towards open large generalist models for tables. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lingui...

    2024 doi

  42. [50]

    Tablellm: Enabling tabular data manipulation by llms in real office usage scenarios

    Xiaokang Zhang, Jing Zhang, Zeyao Ma, Yang Li, Bohan Zhang, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, et al. Tablellm: Enabling tabular data manipulation by llms in real office usage scenarios. arXiv preprint arXiv:2403.19318, 2024 b

    2024 arXiv

  43. [51]

    Multimodal table understanding

    Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. Multimodal table understanding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  44. [52]

    L lama F actory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. L lama F actory: Unified efficient fine-tuning of 100+ language models. In Yixin Cao, Yang Feng, and Deyi Xiong (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics...

    2024 doi

  45. [53]

    Seq2sql: Generating structured queries from natural language using reinforcement learning

    Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103, 2017

    2017 arXiv

  46. [54]

    Lima: Less is more for alignment

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024

    2024

  47. [55]

    Instruction-following evaluation for large language models

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023

    2023 arXiv

  48. [56]

    TAT - QA : A question answering benchmark on a hybrid of tabular and textual content in finance

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. TAT - QA : A question answering benchmark on a hybrid of tabular and textual content in finance. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Pro...

    2021

  49. [57]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  50. [58]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  51. [59]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  52. [60]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.