Pith. sign in

REVIEW 4 major objections 5 minor 54 references

What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Base model choice, not training data, dominates table LLM instruction-tuning performance.

desk verdict Solid controlled grid, but the 'base-model dominance' claim is not backed by the paper's own tables. read the letter →

arxiv 2501.14717 v2 pith:XD4UXFVD submitted 2025-01-24 cs.CL

Naihao Deng , Sheng Zhang , Henghui Zhu , Shuaichen Chang , Jiani Zhang , Alexander Hanbo Li , Chung-Wei Hang , Hideo Kobayashi
show 2 more authors
Yiqun Hu Patrick Ng
This is my paper · ORCID
classification cs.CL
keywords tableinstructiontuningbasemodelselectionout-of-domainevaluationquestionansweringLLMmeta-evaluationgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper isolates two factors that earlier table-LLM work conflates: which base model you start from and which table instruction data you fine-tune on. The authors fine-tune three 7B instruction-tuned models on four public table datasets, producing twelve models, and evaluate them on sixteen table benchmarks plus general reasoning and instruction-following benchmarks. The central finding is that the base model matters more than the training data: switching base models changes out-of-domain performance more than switching training sets. The paper also reports that the untuned base models are already strong table learners, that a training set that works best for one base model is not necessarily best for another, and that fine-tuning for tables does not always compromise general abilities. If this finding holds, practitioners building table LLMs should rank base-model selection above dataset construction.

What carries the argument

The load-bearing mechanism is a crossed design: three 7B instruction-tuned base models (Mistral v0.3, OLMo, and Phi 3 Small) are each fine-tuned on the four public training sets from TableLlama, TableLLM, TableBench, and Table-GPT, forming a 3 × 4 grid of twelve models. Every model is evaluated on sixteen out-of-domain table benchmarks and on general reasoning and instruction-following benchmarks. The grid separates data effects (same base model, different data) from model effects (same data, different base model), and the out-of-domain evaluation prevents comparisons from reducing to memorized training distributions.

What would settle it

Train a known-weak base model on the training set the paper finds best and a known-strong base model on the training set it finds weakest, each with its own separately tuned learning rate and epoch count. If the weak-base/best-data model outperforms the strong-base/weak-data model on the out-of-domain suite, or if a variance decomposition over per-model hyperparameter sweeps shows a larger data component than model component, the claim that base model dominates training data would be undercut.

Watch

Extended reading notes

Core claim

The paper claims that in table instruction tuning, the choice of base model exerts a more dominant effect on downstream performance than the choice of training data. The claim is supported by a controlled grid in which the same three 7B instruction-tuned base models are each fine-tuned on four existing table instruction datasets, so every comparison differs in exactly one factor. Out-of-domain evaluation across table question answering, fact verification, table-to-text generation, and schema reasoning shows that untuned base models frequently match or beat their tuned counterparts, that the best training data differ across base models, and that the base model with the strongest pretrained abilities produces the best tuned models. The paper further shows that table fine-tuning can preserve or even improve scores on general benchmarks such as MMLU, ARC, GPQA, and IFEval for some base models, so specialization and general capability are not strictly opposed.

Load-bearing premise

The analysis rests on the assumption that one shared learning rate (5e-7) and three training epochs give comparably good fits for all twelve model–data combinations, so performance differences are caused by base model and data rather than by uneven tuning quality.

Editorial extensions

If this is right

  • Practitioners should choose the strongest available base model before investing in table-specific training data, because the same data produces very different out-of-domain results across bases.
  • Untuned base models should become routine baselines in table-LLM work, since they already match or exceed tuned 7B models on several benchmarks.
  • Dataset quality rankings are not portable: a training set that is best for one base model can be below-average for another.
  • Strong performance on one benchmark within a task category, such as TabFact for fact verification, does not predict performance on a sibling dataset such as InfoTabs.
  • Table-specific fine-tuning does not necessarily degrade general reasoning or instruction-following ability, so specialization and general utility can coexist.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper uses one shared learning rate and epoch count for all twelve runs; a natural follow-up is to repeat the grid with per-combination hyperparameter search and compare the variance attributed to data versus models.
  • A cheap screening rule follows if the claim transfers: probe candidate base models on a few representative table tasks before fine-tuning any of them, because the untuned ranking largely predicts the tuned ranking.
  • The data-by-model interaction suggests future table instruction datasets should be designed against a target base model rather than assumed to transfer, and the same crossed design could test whether base-model dominance is a general property of instruction tuning in other structured-data domains such as text-to-SQL.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a controlled replication of four table instruction-tuned LLMs by fine-tuning three open 7B base models (Mistral v0.3, OLMo, and Phi-3 Small Instruct) on four public training recipes (TableLlama, TableLLM, TableBenchLLM, and Table-GPT), yielding 12 models. These models are evaluated on 16 table benchmarks spanning table QA, table fact verification, table-to-text generation, schema reasoning, and a miscellaneous task, plus five general benchmarks (MMLU, MMLUPro, AI2ARC, GPQA, IFEval). The headline finding is that base model choice plays a more dominant role than training data for out-of-domain table performance; secondary findings include strong untuned baselines, data effects that depend on the base model, transferability across table tasks, and limited degradation of general capabilities after table instruction tuning.

Significance. The controlled 3x4 grid is a genuine contribution: it is the first apples-to-apples replication of four existing table LLMs and provides reusable results, including a new SOTA on HiTab with only 5% of TableLlama's training data. If the dominance claim held, it would give practitioners actionable guidance to prioritize base-model selection over dataset choice. The paper also offers falsifiable observations, such as TableLLM's training data improving table QA and the untuned base models being competitive. However, the headline quantitative claim is not actually quantified, and several supporting statements are contradicted by the paper's own tables; these issues must be resolved before the paper can be accepted.

major comments (4)
  1. [Abstract; §5.3; Table 10] The central claim that base model choice plays a 'more dominant role' than training data is never quantified. Table 10 reports 16 benchmarks in heterogeneous metrics, and direct inspection shows large effects on both axes: for example, on WikiTQ, varying the training data for Mistral moves accuracy from 23.8 to 37.4, while varying the base model on TableLLM data moves it from 26.7 to 37.7; on TabFact, varying the training data for Mistral moves accuracy from 11.9 to 86.8. 'Dominant' requires a per-benchmark standardized effect-size analysis, such as a variance partition, ANOVA on ranks, or a regression with base model, training data, and interaction terms, together with a statement of uncertainty. The current tables provide neither, so the abstract's claim of 'quantitatively disentangl[ing]' overstates the evidence.
  2. [§3 Experimental Setups] The hyperparameter choice is a potential confound for the main comparison. The paper states that models were trained with learning rate 5e-7 for three epochs after a sweep over eight rates, but it does not say whether the sweep was performed separately for each of the 12 base-model/data combinations or only once. If 5e-7 is optimal for only some cells, the measured differences between base models and between datasets are partly tuning-quality differences, not pure base/data effects. Please add per-cell validation curves or an explicit sensitivity analysis (for example, re-running at neighboring learning rates on a subset of cells) to show that the conclusion is robust.
  3. [§5.2; Table 7 and Table 10] The claim that TableLLM's training data 'consistently achieves the best ... on table QA tasks across all three base models' is not supported by the displayed results. In Table 7, on HiTab the best Mistral-based model is TableLlama (70.6), not TableLLM (44.1), and TableLlama's training data includes HiTab (gray cell), so this is an in-domain comparison rather than an out-of-domain one. In Table 10, the best WikiTQ result for Mistral is TableBench (37.4), not TableLLM (32.3), and the best TabFact result for OLMo is TableLlama (83.8), with TableLLM data at 40.6. Please either revise the claim to name the specific datasets where it holds and exclude overlapping gray cells, or present a formal aggregation that supports the stated conclusion.
  4. [§5; Table 10 note] The out-of-domain analysis includes gray cells whose training data overlaps with the evaluation benchmark, and these cells appear to contribute to the 'most number of top performance' markers used in the analysis. Because the paper's main claims are about generalization to unseen tasks, overlapping cells should be removed from the comparison or analyzed separately; otherwise the out-of-domain finding is partly circular. Please report the counts with and without gray cells and make clear which of the paper's claims each analysis supports.
minor comments (5)
  1. [Abstract] The abstract writes 'Hitab' where the rest of the paper uses 'HiTab'; please correct the spelling.
  2. [Table 7 caption] The caption says 'Out-of-domain evaluation' but the same table marks train-data overlap in gray; please clarify in the caption how gray cells should be interpreted relative to the out-of-domain framing.
  3. [§4.1] The phrase 'we rule out the table operation datasets and only maintain the training data' is awkward; 'we exclude the table operation datasets and keep only the training data' would be clearer.
  4. [§6.2] The suggestion that the AI2ARC improvement is 'likely due to' table tasks involving reasoning is not tested by any analysis; please soften the causal language or provide supporting evidence.
  5. [Appendix B] For the multiple-choice benchmarks, the paper selects the completion with highest likelihood without length normalization; this is a defensible choice, but a sentence of justification or a citation would help readers interpret the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's findings are direct empirical observations from a fully crossed training grid, not a derived quantity that reduces to its inputs.

full rationale

This paper is an empirical meta-evaluation rather than a derivation, so the usual circularity mechanisms are absent. The central claim that base model choice plays a more dominant role than training data comes from a controlled 3x4 design: three base models are each fine-tuned on four public training datasets and then scored on external benchmarks. No fitted parameter is later relabeled as a prediction; the reported numbers are raw evaluation results, and the paper does not define any quantity in terms of another and then present that identity as a finding. The only self-citation, Deng et al. (2024), is used to justify focusing on text rather than image table representations; this is a scope decision and is not load-bearing for the data-versus-model conclusion, which depends on the benchmark comparisons themselves. The limitation statements in the paper concede only that not all datasets could be covered, not that any result is assumed. The gray-shaded in-domain cells in the out-of-domain tables are disclosed in the captions, so the overlap is visible to the reader rather than concealed; even if those cells inflate some per-dataset observations, the paper's core comparison across base models and training data remains an empirical report, not a construction. The single learning rate and epoch count across all cells is a potential confound in the strength of the headline claim, but a confound is not circularity. No step in the paper's reasoning reduces to its own input, so a score of 0 is appropriate.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or fitted scientific constants. Its free parameters are standard fine-tuning hyperparameters. The main assumptions are about representativeness of the model and dataset sample, parity of training conditions, and validity of the evaluation pipeline, including the Haiku extraction step.

free parameters (2)
  • learning rate = 5e-7
    Selected from eight candidate values as the empirically best across all models; applied uniformly, so differences in optimal learning rate per base model or dataset are not controlled.
  • number of epochs = 3
    Applied uniformly to all 12 models, chosen for consistency rather than per-task convergence, and this choice could affect the comparison.
assumptions (4)
  • domain assumption The three base models (Mistral v0.3 7B Instruct, OLMo 7B Instruct, Phi 3 Small Instruct) and four training datasets are representative of the table instruction-tuning landscape.
    Section 3 introduces these as the foundation families; conclusions about base models generally depend on this selection.
  • domain assumption Training each model for three epochs with learning rate 5e-7 yields comparably near-optimal fits across all base model and dataset combinations.
    Section 3 'Experimental Setups' states that 5e-7 was best and that three epochs were used for consistency; unequal optimality would confound comparisons.
  • domain assumption Using Haiku 3.5 to extract answer entities from free-form generations does not systematically favor any base model.
    Appendix A describes this extraction for table QA metrics; bias in extraction would distort benchmark scores.
  • domain assumption The selected evaluation benchmarks are valid proxies for table understanding and general reasoning abilities.
    Sections 5.1 and 6.1 select benchmarks as proxies for table QA, fact verification, table-to-text, and general reasoning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects." pith.science (2026). https://pith.science/paper/XD4UXFVD

@misc{pith2026250114717,
  author       = {Pith},
  title        = {Pith review of: What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XD4UXFVD}},
  note         = {Machine review of arXiv:2501.14717}
}
read the original abstract

Table modeling has progressed for decades. In this work, we revisit this trajectory and highlight emerging challenges in the LLM era, particularly the paradox of choice: the difficulty of attributing performance gains amid diverse base models and training sets in the context of table instruction tuning. We replicate four table LLMs by instruction-tuning three foundation models on four existing datasets, yielding 12 models. We then evaluate these models across 16 table benchmarks. Our study is the first to quantitatively disentangle the effects of training data and base model selection, revealing that base model choice plays a more dominant role than the training data itself. Generalization and reasoning remain challenging, inviting future effort on table modeling. Based on our findings, we share our thoughts on the future directions for table modeling.

Figures

Figures reproduced from arXiv: 2501.14717 by the authors.

Figure 1
Figure 1. Performance of fine-tuned models trained on different data (e.g. TableLlama) on general benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Performance of Phi 3 Mini Instruct (4B) versus Phi 3 Small Instruct (7B) model on different table tasks [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Performance of Phi 3 Mini Instruct (4B) versus Phi 3 Small Instruct (7B) model on different table tasks [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance difference between Phi 3 Mini Instruct (4B) versus Phi 3 Small Instruct (7B) model. On [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 5 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  4. [4]

    Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. https://doi.org/10.18653/v1/2021.fever-1.1 The fact extraction and VER ification over unstructured and structured information ( FEVEROUS ) shared task . In Proceedings of the Fourth Workshop on Fact Extrac...

    doi:10.18653/v1/2021.fever-1.1 2021

  5. [5]

    Shuaichen Chang and Eric Fosler-Lussier. 2023. How to prompt llms for text-to-sql: A study in zero-shot, single-domain, and cross-domain settings. arXiv preprint arXiv:2305.11853

    arXiv 2023

  6. [6]

    Hsin-Hsi Chen, Shih-Chung Tsai, and Jin-He Tsai. 2000. https://aclanthology.org/C00-1025 Mining tables from large scale HTML texts . In COLING 2000 Volume 1: The 18th International Conference on Computational Linguistics

    work page 2000

  7. [7]

    Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Wang, and William W Cohen. 2020 a . Open question answering over tables and text. arXiv preprint arXiv:2010.10439

    arXiv 2020

  8. [8]

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv:1909.02164

    arXiv 2019

Show all 54 references
  1. [9]

    Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang. 2020 b . https://doi.org/10.18653/v1/2020.findings-emnlp.91 H ybrid QA : A dataset of multi-hop question answering over tabular and textual data . In Findings of the Association for Computation...

    2020 doi

  2. [10]

    Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2024. Longlora: Efficient fine-tuning of long-context large language models. In The International Conference on Learning Representations (ICLR)

    2024

  3. [11]

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.300 F in QA : A dataset of numerical reasoning over financial d...

    2021 doi

  4. [12]

    Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2022. https://doi.org/10.18653/v1/2022.acl-long.78 H i T ab: A hierarchical table dataset for question answering and natural language generation . In Proceedings of...

    2022 doi

  5. [13]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    2018 arXiv

  6. [14]

    Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. 2024. https://doi.org/10.18653/v1/2024.findings-acl.23 Tables as texts or images: Evaluating the table reasoning ability of LLM s and MLLM s . In Findings of the Association for ...

    2024 doi

  7. [15]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  8. [16]

    Mihail Eric and Christopher D Manning. 2017. Key-value retrieval networks for task-oriented dialogue. arXiv preprint arXiv:1705.05414

    2017 arXiv

  9. [17]

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. 2024. Olmo: Accelerating the science of language models. arXiv preprint arXiv:2402.00838

    2024 arXiv

  10. [18]

    Vivek Gupta, Maitrey Mehta, Pegah Nokhiz, and Vivek Srikumar. 2020. https://doi.org/10.18653/v1/2020.acl-main.210 INFOTABS : Inference on tables as semi-structured data . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2309--23...

    2020 doi

  11. [19]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  12. [20]

    Jonathan Herzig, Pawel Krzysztof Nowak, Thomas M \"u ller, Francesco Piccinno, and Julian Eisenschlos. 2020. https://doi.org/10.18653/v1/2020.acl-main.398 T a P as: Weakly supervised table parsing via pre-training . In Proceedings of the 58th Annual Meeting of the Association ...

    2020 doi

  13. [21]

    Sujay Kumar Jauhar, Peter Turney, and Eduard Hovy. 2016. Tabmcq: A dataset of general knowledge tables and multiple-choice questions. arXiv preprint arXiv:1602.03960

    2016 arXiv

  14. [22]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  15. [23]

    Pei Ke, Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. 2024. https://doi.org/10.18653/v1/2024.acl-long.704 C ritique LLM : Towards an informative critique generation model for evalua...

    2024 doi

  16. [24]

    R \'e mi Lebret, David Grangier, and Michael Auli. 2016. https://doi.org/10.18653/v1/D16-1128 Neural text generation from structured data with application to the biography domain . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages...

    2016 doi

  17. [25]

    Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. 2023. Table-gpt: Table-tuned gpt for diverse table tasks. arXiv preprint arXiv:2310.09263

    2023 arXiv

  18. [26]

    Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2018. Table-to-text generation by structure-aware seq2seq learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32

    2018

  19. [27]

    Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. 2022. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. arXiv preprint arXiv:2209.14610

    2022 arXiv

  20. [28]

    Nafise Sadat Moosavi, Andreas R \"u ckl \'e , Dan Roth, and Iryna Gurevych. 2021. Scigen: a dataset for reasoning-aware text generation from scientific tables. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)

    2021

  21. [29]

    Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry \'s ci \'n ski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, and Drago...

    2022 doi

  22. [30]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  23. [31]

    Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.89 ToTTo : A controlled table-to-text generation dataset . In Proceedings of the 2020 Conference on Empirical Methods i...

    2020 doi

  24. [32]

    Panupong Pasupat and Percy Liang. 2015. https://doi.org/10.3115/v1/P15-1142 Compositional semantic parsing on semi-structured tables . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natur...

    2015 doi

  25. [33]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022

    2023 arXiv

  26. [34]

    Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, Haoze Li, et al. 2024. Tablegpt2: A large multimodal model with tabular data integration. arXiv preprint arXiv:2411.02059

    2024 arXiv

  27. [35]

    Ashwin Tengli, Yiming Yang, and Nian Li Ma. 2004. https://aclanthology.org/C04-1142 Learning table extraction from examples . In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics , pages 987--993, Geneva, Switzerland. COLING

    2004

  28. [36]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  29. [37]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. https://arxiv.org/abs/2406.01574 Mmlu-pro: A more robust and...

    2024 arXiv

  30. [38]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36

    2024

  31. [39]

    Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xinrun Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, Guanglin Niu, Tongliang Li, and Zhoujun Li. 2024. https://arxiv.org/abs/2408.09174 Tablebench: A comprehensive and complex benchmark for table question ans...

    2024 arXiv

  32. [40]

    Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A

    Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, N...

    2022 doi

  33. [41]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  34. [42]

    Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, and Shachi Paul. 2022. https://doi.org/10.18653/v1/2022.acl-long.40 T able F ormer: Robust transformer modeling for table-text encoding . In Proceedings of the 60th Annual Meeting of the Association for Comput...

    2022 doi

  35. [43]

    Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. 2020. https://doi.org/10.18653/v1/2020.acl-main.745 T a BERT : Pretraining for joint understanding of textual and tabular data . In Proceedings of the 58th Annual Meeting of the Association for Computational Ling...

    2020 doi

  36. [44]

    Liangyu Zha, Junlin Zhou, Liyao Li, Rui Wang, Qingyi Huang, Saisai Yang, Jing Yuan, Changbao Su, Xiang Li, Aofeng Su, et al. 2023. Tablegpt: Towards unifying tables, nature language and commands into one gpt. arXiv preprint arXiv:2307.08674

    2023 arXiv

  37. [45]

    Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. 2024 a . https://doi.org/10.18653/v1/2024.naacl-long.335 T able L lama: Towards open large generalist models for tables . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational ...

    2024 doi

  38. [46]

    Xiaokang Zhang, Jing Zhang, Zeyao Ma, Yang Li, Bohan Zhang, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, et al. 2024 b . Tablellm: Enabling tabular data manipulation by llms in real office usage scenarios. arXiv preprint arXiv:2403.19318

    2024 arXiv

  39. [47]

    Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. 2024. Multimodal table understanding. arXiv preprint arXiv:2406.08100

    2024 arXiv

  40. [48]

    Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103

    2017 arXiv

  41. [49]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2024. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36

    2024

  42. [50]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911

    2023 arXiv

  43. [51]

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. https://doi.org/10.18653/v1/2021.acl-long.254 TAT - QA : A question answering benchmark on a hybrid of tabular and textual content in finance . In Proceedings of...

    2021 doi

  44. [52]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  45. [53]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  46. [54]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.