Pith. sign in

REVIEW 3 major objections 4 minor 75 references

Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current language models hold steady across table formats but drop up to 34% on scientific tables.

desk verdict TableEval is a genuinely useful benchmark and the format-robustness result is solid, but the headline claim about scientific tables being inherently harder is contradicted by the paper's own matched QA comparison. read the letter →

arxiv 2507.00152 v1 pith:3BFJ44QZ submitted 2025-06-30 cs.CL

Ekaterina Borisova , Fabio Barth , Nils Feldhus , Raia Abu Ahmad , Malte Ostendorff , Pedro Ortiz Suarez , Georg Rehm , Sebastian Möller This is my paper
classification cs.CL
keywords tableunderstandingmultimodallargelanguagemodelsscientifictablescross-domainevaluationrepresentationformatsinterpretabilitybenchmarktable-to-textgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TableEval is the paper's new benchmark: 3,017 tables from scholarly articles, Wikipedia, and financial reports, each rendered as an image, a Python dictionary, HTML, XML, and LaTeX, giving 11,312 test instances over question answering and table-to-text tasks. The authors run nine open and closed text-based and multimodal language models zero-shot and ask whether the source domain or the representation format matters more for table understanding. Their central finding is that models are largely indifferent to whether a table arrives as an image or as one of four text encodings, but they score markedly lower on scientific tables; the non-scientific split outperforms the scientific split by up to 34% on aggregated metrics. A small gradient-based interpretability study on selected instances suggests the gap shows up in model confidence and number tokenization, with better models attending to the relevant rows and columns while weaker models anchor on prompt edges.

What carries the argument

The load-bearing object is TableEval, a benchmark of 3,017 tables and 11,312 instances assembled from PubMed Central articles, ACL Anthology and arXiv papers, Wikipedia, and S&P 500 earnings reports, with every table rendered in five formats: PNG image, Python dictionary, HTML, XML, and LaTeX. This design is what allows the authors to separate two questions that earlier benchmarks conflate: does the representation format matter, and does the scientific source matter? The evaluation protocol keeps prompts fixed per task, runs all models zero-shot, and averages eight automatic metrics; a secondary mechanism is the interpretability pass, which applies Input x Gradient attributions to generated tokens so that model confidence and input usage can be compared across instances.

What would settle it

Run the same question set with the same output format on scientific and non-scientific tables matched for row and column count, cell types, answer length, and reasoning type; if the gap collapses to the 1-3% range seen in the paper's ComTQA comparison, the claim of a science-specific weakness is not supported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that current text-based and multimodal large language models hold steady across table modalities but not across domains: averaged over nine models and five table formats, the non-scientific subsets of TableEval beat the scientific subsets by up to 34%, with the largest gaps on n-gram overlap metrics like ROUGE and BLEU. The authors trace this gap to two causes: higher intrinsic complexity and reasoning demands in the scientific table-to-text subsets (SciGen and numericNLG) and the scarcity of scholarly tables in pretraining data. They also show that the format effect is small, at most about 4% among the four text encodings, with images performing best overall by roughly 1-13%. The interpretability case studies add a mechanism: gradient attributions and generation log-probabilities reveal that the stronger model focuses on the answer row and column while the weaker model attends to prompt boundaries and truncates four-digit numbers into three-digit tokens.

Load-bearing premise

The comparison assumes that the scientific and non-scientific tables in TableEval are equally hard, so the up-to-34% score gap can be blamed on the scientific content rather than on the harder summarization tasks and longer gold outputs in the scientific subsets; the paper's own matched ComTQA comparison shows only a 1-3% gap on most metrics.

Editorial extensions

If this is right

  • If the central claim is right, format conversion is not the lever: image and text encodings stay within a few points of each other, so improving table understanding means working on content and training data, not on serialization.
  • Scientific tables should be treated as a distinct target for domain adaptation or pretraining, because the largest and most consistent score drops appear on scholarly subsets across every model and format.
  • Automatic scholarly-document pipelines that rely on table QA or table-to-text will inherit this weakness, and switching a table from LaTeX to HTML or from text to image will not close the gap.
  • The interpretability results imply that attribution and generation confidence can flag unreliable table answers, which could serve as a practical filter before outputs are used downstream.
  • Model size does not predict table performance in this study, so model selection for table-heavy applications should be empirical rather than based on parameter count.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 34% domain gap may shrink under a stricter controlled comparison: the paper's own ComTQA subsets, which share the same QA task, differ by only 1-3% on most metrics, so task difficulty is confounded with domain in the headline number.
  • Number tokenization is a plausible concrete mechanism worth isolating: the interpretability examples show one model reading years digit-by-digit and another cutting four-digit years to three digits, which would directly hurt dense scientific tables regardless of domain.
  • The release of each table in five formats invites a format-mixing pretraining recipe, where the same table seen as image, LaTeX, HTML, XML, and dictionary could teach format-agnostic table structure before any scientific-domain training.
  • The observed saliency pattern of strong attention to prompt edges and question words suggests a cheap testable intervention: repositioning or re-weighting the answer-relevant row and column labels in the prompt may improve accuracy without any model change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces TableEval, a benchmark of 3017 tables from scientific (PubTables-1M/PMC, numericNLG/ACL Anthology, SciGen/arXiv) and non-scientific (FinTabNet, LogicNLG, Logic2Text/Wikipedia) sources, each provided in five formats (Image, Dictionary, HTML, XML, LaTeX). The authors evaluate ten text-based and multimodal LLMs on QA and table-to-text tasks, and add a small interpretability analysis with Inseq. The headline claims are that models are robust across table modalities but face significant challenges on scientific tables, with a reported score boost of up to 34% for the non-scientific split.

Significance. The paper contributes a publicly released, multi-format benchmark (TableEval), a reproducible evaluation pipeline, and results across several models including a closed-source baseline. If the central domain claim were solid, it would identify scientific tables as a distinct weakness for current (M)LLMs and would be a useful signal for the community. However, the core finding is currently confounded by task differences between the scientific and non-scientific splits, and the paper's own controlled QA comparison shows only a small domain gap. The benchmark itself and the evaluation infrastructure are nevertheless valuable, and the interpretability case study, though small, is a useful initial exploration.

major comments (3)
  1. [§3.2, Figure 2 (right), Table 17] The headline claim that LLMs 'face significant challenges when processing scientific tables' (abstract) is confounded. The scientific split consists of ComTQA PubTables-1M (short-answer QA), numericNLG, and SciGen (multi-sentence generation), whereas the non-scientific split consists of ComTQA FinTabNet (short-answer QA), LogicNLG, and Logic2Text (single-statement generation). Output lengths differ substantially across subsets (Table 3 and Figure 15), and BLEU, ROUGE, and BLEURT are sensitive to output length and reference overlap. The paper's own controlled QA comparison between ComTQA PubTables-1M and ComTQA FinTabNet (Tables 6-7, 18) shows only 1-3% gaps on most metrics, with a 17% BLEURT outlier. Thus the 34% aggregated 'score boost' in Table 17 cannot be attributed to domain alone; it also reflects task complexity and output structure. The authors should either provide a task-matched cross-domain comparison (e.g., re-annotating all subsets for a common task, which they suggest in Limitations) or explicitly limit the claim to 'scientific tables in long-form generation tasks' rather than scientific tables in general.
  2. [§3.2, 'score boost of up to 34%'] The numerical claim of a 'score boost of up to 34%' is not traceable to the reported results. Table 17 shows relative differences between the non-scientific and scientific averages that range from about 5% (BertScore.F1) to over 100% (ROUGE-L.F1, ROUGE-2.F1), and no listed absolute or relative difference equals 34%. Please identify the exact metric, model, and data subset that produce this figure, or replace it with a precisely defined aggregate (e.g., mean relative improvement across metrics with a confidence interval).
  3. [§3.2, Figure 2 (left), Figures 13-14] The modality-robustness conclusion is also based on an uncontrolled comparison. For ComTQA (PubTables-1M), the image-condition prompt includes explicit 'Table caption' and 'Table footnote' fields (Figure 13), while the text-condition prompt for the same subset provides only 'Table: {table}' (Figure 14), which may or may not include the caption/footnote depending on the text format. This extra context could contribute to the reported image advantage for that subset. Furthermore, the aggregate 1-13% image-versus-text differences are averaged over heterogeneous tasks and models with no significance testing; per-subset, per-prompt controlled comparisons would be needed to support the statement that models are 'robust across table modalities.'
minor comments (4)
  1. [Table 5 (Appendix F)] The column header 'LaT eX' is a typo and should read 'LaTeX'.
  2. [§2.1, criterion 4] Criterion 4 promises that target tasks are 'identical or very similar across datasets to maintain consistency and ensure comparability,' but the final selection includes QA, long-form T2T, and single-statement T2T; this criterion is not satisfied by the chosen datasets and should be revised to reflect the actual task diversity.
  3. [Abstract, Introduction] The paper uses 'efficiency' to describe model performance, but the experiments report only quality/accuracy metrics and do not measure computational cost (runtime, FLOPs, API cost); 'effectiveness' would be a more accurate term.
  4. [Appendix G, Figure 16] Figure 16 (prediction examples) is not referenced in the main text; consider adding a pointer in Section 3.2 where prediction length differences are discussed.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the paper is an empirical benchmark evaluation with external gold labels and standard metrics; the domain-gap claim is confounded but not derived from its own inputs by construction.

full rationale

This is an empirical evaluation study, not a derivation. The TableEval benchmark is assembled from existing datasets with gold annotations, and all reported scores use standard external metrics (BLEU, ROUGE, METEOR, BERTScore, BLEURT, MoverScore). No parameter is fitted to a subset and then renamed as a prediction, and no theoretical claim is justified by an equation that reduces to its own input. The only self-citation of note is Inseq (Sarti et al., 2023), which includes co-author Nils Feldhus, but it is used merely as an interpretability software tool and is not load-bearing for the central benchmark findings. The headline claim that scientific tables pose "significant challenges" is weakened by a task-complexity confound: the scientific split contains long-form generation subsets (numericNLG, SciGen) while the non-scientific split contains single-sentence or masked-entity subsets (LogicNLG, Logic2Text), so the up-to-34% gap in Section 3.2 may reflect task and output complexity rather than domain alone. However, this is an experimental-design and interpretation concern, not circular reasoning: the result is not forced by definition, by fitted inputs, or by a self-citation chain. The paper itself acknowledges the complexity hypothesis and notes in the Limitations that annotating all subsets for a common task would be beneficial, which further confirms that the claim is empirically contingent rather than circular. Accordingly, the appropriate circularity score is low.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no fitted parameters or theoretical entities. It relies on assumptions about format fidelity, correctness of inherited annotations, and comparability of tasks across the scientific/non-scientific split. The third assumption is the most consequential and is only partially supported by the data.

assumptions (3)
  • domain assumption Format conversions preserve table content
    Fidelity is only manually validated on ~100 random instances per format and subset (Section 2.2), so the remaining corpus may contain conversion errors.
  • domain assumption Gold annotations from source datasets are correct
    The benchmark uses existing annotations from ComTQA, numericNLG, SciGen, LogicNLG, and Logic2Text without re-annotation, assuming their labels are ground truth.
  • ad hoc to paper Scientific vs non-scientific split isolates domain effect
    The interpretation of the 34% gap as domain-related assumes task difficulty and output complexity are comparable across splits, which the controlled ComTQA comparison contradicts (Section 3.2, Tables 6-7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data." pith.science (2026). https://pith.science/paper/3BFJ44QZ

@misc{pith2026250700152,
  author       = {Pith},
  title        = {Pith review of: Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3BFJ44QZ}},
  note         = {Machine review of arXiv:2507.00152}
}
read the original abstract

Tables are among the most widely used tools for representing structured data in research, business, medicine, and education. Although LLMs demonstrate strong performance in downstream tasks, their efficiency in processing tabular data remains underexplored. In this paper, we investigate the effectiveness of both text-based and multimodal LLMs on table understanding tasks through a cross-domain and cross-modality evaluation. Specifically, we compare their performance on tables from scientific vs. non-scientific contexts and examine their robustness on tables represented as images vs. text. Additionally, we conduct an interpretability analysis to measure context usage and input relevance. We also introduce the TableEval benchmark, comprising 3017 tables from scholarly publications, Wikipedia, and financial reports, where each table is provided in five different formats: Image, Dictionary, HTML, XML, and LaTeX. Our findings indicate that while LLMs maintain robustness across table modalities, they face significant challenges when processing scientific tables.

Figures

Figures reproduced from arXiv: 2507.00152 by the authors.

Figure 1
Figure 1. Schematic representation of the main phases in our experiments: 1. Develop TableEval dataset, 2. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. BertScore.F1, MoverScore, ROUGE-L.F1, and METEOR for the table formats averaged over data subsets [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Values of BertScore.F1, MoverScore, ROUGE-L.F1, and METEOR [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: BertScore.F1, MoverScore, ROUGE-L.F1, and METEOR for each data subset averaged over table formats [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 2
Figure 2. Figure 2: Extraction precision at increasing yield by genre. points, making parsing errors responsible for about 28% of the incorrect extractions. Since the mate￾tools parser is trained on the full TIGER treebank, including our experimental data, its error contribu￾tion on unsee…
Figure 11
Figure 11. Figure 11: An example from Logic2Text, illustrating a table, a logical form, and a corresponding gold statement. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Distribution of image aspect ratios (width/height) across subsets in the TableEval benchmark. Each box [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Prompts used for experiments based on images of tables. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Prompts used for experiments based on textual representations of tables. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Mean prediction lengths (in characters) for each model and data subset. Here “_llm” and “_mm” are [PITH_FULL_IMAGE:figures/full_fig_p027_15.png]
Figure 16
Figure 16. Figure 16: An example illustrating differences in prediction length across Idefics3, Gemini-2.0-Flash, and Qwen2.0- [PITH_FULL_IMAGE:figures/full_fig_p028_16.png]
Figure 17
Figure 17. Figure 17: Interpretability analysis for the ComTQA (FinTabNet) instance with a table represented in a Dict format. [PITH_FULL_IMAGE:figures/full_fig_p030_17.png]
Figure 18
Figure 18. Figure 18: Interpretability analysis for the ComTQA (PubTables-1M) instance with a table represented in a Dict [PITH_FULL_IMAGE:figures/full_fig_p031_18.png]
Figure 19
Figure 19. Figure 19: Interpretability analysis for ComTQA (PubTables-1M) instance with the Dict format. The ground truth [PITH_FULL_IMAGE:figures/full_fig_p032_19.png]
Figure 20
Figure 20. Figure 20: Interpretability analysis the LogicNLG instance comparing the Dict (left) with the LA [PITH_FULL_IMAGE:figures/full_fig_p033_20.png]
Figure 21
Figure 21. Figure 21: Interpretability analysis for the LogicNLG instance comparing the Dict (left) with the LA results of operations, which is incorporated herein by reference. [PITH_FULL_IMAGE:figures/full_fig_p034_21.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 26 canonical work pages

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. https://arxiv.org/abs/25...

    arXiv 2025

  2. [2]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909/ METEOR : An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan. Association for Compu...

    2005

  3. [3]

    Chandra Sekhar Bhagavatula, Thanapon Noraset, and Doug Downey. 2015. Ta bEL : Entity linking in web tables. In The Semantic Web - ISWC 2015, pages 425--441, Cham. Springer International Publishing

    work page 2015

  4. [4]

    Vadim Borisov, Tobias Leemann, Kathrin Sessler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. 2022. https://doi.org/10.1109/tnnls.2022.3229161 Deep neural networks and tabular data: A survey . IEEE Transactions on Neural Networks and Learning Systems , pages 1--21

    arXiv 2022

  5. [5]

    Lutz Bornmann, Robin Haunschild, and R \"u diger Mutz. 2021. Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanities and Social Sciences Communications, 8(1):1--15

    work page 2021

  6. [6]

    Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, and Rita Cucchiara. 2024. https://doi.org/10.18653/v1/2024.findings-acl.807 The revolution of multimodal large language models: A survey . In Findings of the Association for Computational Linguistics: ACL 2024, pages 13590...

    doi:10.18653/v1/2024.findings-acl.807 2024

  7. [7]

    Yu, Qiang Yang, and Xing Xie

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. https://doi.org/10.1145/3641289 A survey on evaluation of large language models . ACM Trans. Intell. Syst. Technol., 15(3)

    doi:10.1145/3641289 2024

  8. [8]

    Wenhu Chen. 2023. https://doi.org/10.18653/v1/2023.findings-eacl.83 Large language models are few(1)-shot table reasoners . In Findings of the Association for Computational Linguistics: EACL 2023, pages 1120--1130, Dubrovnik, Croatia. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-eacl.83 2023

Show all 75 references
  1. [9]

    Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Wang, and William W. Cohen. 2021. https://arxiv.org/abs/2010.10439 Open question answering over tables and text . Preprint, arXiv:2010.10439

    2021 arXiv

  2. [10]

    Wenhu Chen, Jianshu Chen, Yu Su, Zhiyu Chen, and William Yang Wang. 2020 a . https://doi.org/10.18653/v1/2020.acl-main.708 Logical natural language generation from open-domain tables . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ...

    2020 doi

  3. [11]

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020 b . Tab F act: A large-scale dataset for table-based fact verification. In International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia

    2020

  4. [12]

    Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan, and William Yang Wang. 2020 c . https://doi.org/10.18653/v1/2020.findings-emnlp.190 L ogic2 T ext: High-fidelity natural language generation from logical forms . In Findings of the Association for...

    2020 doi

  5. [13]

    Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2022. https://doi.org/10.18653/v1/2022.acl-long.78 H i T ab: A hierarchical table dataset for question answering and natural language generation . In Proceedings of...

    2022 doi

  6. [14]

    Christopher Clark and Santosh Divvala. 2016. https://doi.org/10.1145/2910896.2910904 PDFF igures 2.0: Mining figures from research papers . In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries, JCDL '16, page 143–152, New York, NY, USA. Association f...

    2016

  7. [15]

    Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. 2024. https://doi.org/10.18653/v1/2024.findings-acl.23 Tables as texts or images: Evaluating the table reasoning ability of LLM s and MLLM s . In Findings of the Association for ...

    2024 doi

  8. [16]

    James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan Arik, Yan Liu, and Tomas Pfister. 2024. https://doi.org/10.18653/v1/2024.findings-acl.832 T ext G en SHAP : Scalable post-hoc explanations in text generation with long documents . In Findings of the Association for Computati...

    2024 doi

  9. [17]

    Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Scott Nickleach, Diego Socolinsky, Srinivasan Sengamedu, and Christos Faloutsos. 2024. https://arxiv.org/abs/2402.17944 Large language models ( LLMs ) on tabular data: Prediction, generation, and understa...

    2024 arXiv

  10. [18]

    Costa-jussà

    Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R. Costa-jussà. 2024. https://arxiv.org/abs/2405.00208 A primer on the inner workings of transformer-based language models . arXiv, abs/2405.00208

    2024 arXiv

  11. [19]

    Santo Fortunato, Carl T Bergstrom, Katy B \"o rner, James A Evans, Dirk Helbing, Sta s a Milojevi \'c , Alexander M Petersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, et al. 2018. Science of science. Science, 359(6379):eaao0185

    2018

  12. [20]

    Sebastian Gehrmann, Elizabeth Clark, and Thibault Sellam. 2023. https://doi.org/10.1613/jair.1.13715 Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text . J. Artif. Int. Res., 77

    2023 doi

  13. [21]

    Heng Gong, Yawei Sun, Xiaocheng Feng, Bing Qin, Wei Bi, Xiaojiang Liu, and Ting Liu. 2020. https://doi.org/10.18653/v1/2020.coling-main.179 T able GPT : Few-shot table-to-text generation with table structure reconstruction and content matching . In Proceedings of the 28th Inte...

    2020 doi

  14. [22]

    Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. 2021. Revisiting deep learning models for tabular data. In Advances in Neural Information Processing Systems, volume 34, pages 18932--18943. Curran Associates, Inc

    2021

  15. [23]

    Aaron Grattafiori, Abhimanyu Dubey, and Abhinav Jauhri et. al. 2024. https://arxiv.org/abs/2407.21783 The L lama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  16. [24]

    Jonathan Herzig, Pawel Krzysztof Nowak, Thomas M \"u ller, Francesco Piccinno, and Julian Eisenschlos. 2020. https://doi.org/10.18653/v1/2020.acl-main.398 T a P as: Weakly supervised table parsing via pre-training . In Proceedings of the 58th Annual Meeting of the Association ...

    2020 doi

  17. [25]

    Xanh Ho, Anh Khoa Duong Nguyen, An Tuan Dao, Junfeng Jiang, Yuki Chida, Kaito Sugimoto, Huy Quoc To, Florian Boudin, and Akiko Aizawa. 2024. https://arxiv.org/abs/2401.17824 A survey of pre-trained language models for processing scientific text . Preprint, arXiv:2401.17824

    2024 arXiv

  18. [26]

    Zhi Hong, Logan Ward, Kyle Chard, Ben Blaiszik, and Ian Foster. 2021. https://doi.org/10.1007/s11837-021-04902-9 Challenges and advances in information extraction from scientific literature: a review . JOM, 73:1543--1851

    2021 doi

  19. [27]

    Hiroshi Iida, Dung Thai, Varun Manjunatha, and Mohit Iyyer. 2021. https://doi.org/10.18653/v1/2021.naacl-main.270 TABBIE : Pretrained representations of tabular data . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling...

    2021 doi

  20. [28]

    Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. 2024. https://arxiv.org/abs/2408.12637 Building and better understanding vision-language models: insights and future directions . Preprint, arXiv:2408.12637

    2024 arXiv

  21. [29]

    Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. 2024. https://llava-vl.github.io/blog/2024-05-10-llava-next-stronger-llms/ LLaVA-NeXT : Stronger LLM s supercharge multimodal capabilities in the wild

    2024

  22. [30]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  23. [31]

    Xinyuan Lu, Liangming Pan, Yubo Ma, Preslav Nakov, and Min-Yen Kan. 2024. https://openreview.net/forum?id=UF8RE1rkfU TART : An open-source tool-augmented framework for explainable table-based reasoning . In NeurIPS 2024 Third Table Representation Learning Workshop

    2024

  24. [32]

    Mattia Marzocchi, Marco Cremaschi, Riccardo Pozzi, Roberto Avogadro, and Matteo Palmonari. 2022. https://ceur-ws.org/Vol-3320/paper3.pdf Mammo T ab: A giant and comprehensive dataset for semantic table interpretation . In Proceedings of the Semantic Web Challenge on Tabular Da...

    2022

  25. [33]

    Nafise Sadat Moosavi, Andreas R \"u ckl \'e , Dan Roth, and Iryna Gurevych. 2021. https://openreview.net/forum?id=Jul-uX7EV_I Sci G en: a dataset for reasoning-aware text generation from scientific tables . In Thirty-fifth Conference on Neural Information Processing Systems Da...

    2021

  26. [34]

    Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry \'s ci \'n ski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, and Drago...

    2022 doi

  27. [35]

    Giang Nguyen, Ivan Brugere, Shubham Sharma, Sanjay Kariyappa, Anh Totti Nguyen, and Freddy Lecue. 2025. https://arxiv.org/abs/2412.12386 Interpretable LLM -based table question answering . arXiv, abs/2412.12386

    2025 arXiv

  28. [36]

    OpenAI, Josh Achiam, and Steven Adler et. al. 2024. https://arxiv.org/abs/2303.08774 GPT -4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  29. [37]

    Alfonso Ure \ n a-L \'o pez, Eugenio Mart \'i nez C \'a mara, and Jose Camacho-Collados

    Jorge Os \'e s Grijalba, L. Alfonso Ure \ n a-L \'o pez, Eugenio Mart \'i nez C \'a mara, and Jose Camacho-Collados. 2024. https://aclanthology.org/2024.lrec-main.1179/ Question answering over tabular data with D ata B ench: A large-scale empirical evaluation of LLM s . In Pro...

    2024

  30. [38]

    Chaoxu Pang, Yixuan Cao, Chunhao Yang, and Ping Luo. 2024. https://doi.org/10.18653/v1/2024.findings-acl.82 Uncovering limitations of large language models in information seeking from tables . In Findings of the Association for Computational Linguistics: ACL 2024, pages 1388--...

    2024 doi

  31. [39]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  32. [40]

    Letitia Parcalabescu and Anette Frank. 2025. https://openreview.net/forum?id=lCasyP21Bf Do vision & language decoders use images and text equally? H ow self-consistent are their explanations? In The Thirteenth International Conference on Learning Representations

    2025

  33. [41]

    Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.89 ToTTo : A controlled table-to-text generation dataset . In Proceedings of the 2020 Conference on Empirical Methods i...

    2020 doi

  34. [42]

    Matt Post. 2018. https://doi.org/10.18653/v1/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191, Brussels, Belgium. Association for Computational Linguistics

    2018 doi

  35. [43]

    Jirui Qi, Gabriele Sarti, Raquel Fern \'a ndez, and Arianna Bisazza. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.347 Model internals-based answer attribution for trustworthy retrieval-augmented generation . In Proceedings of the 2024 Conference on Empirical Methods in Na...

    2024 doi

  36. [44]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  37. [45]

    Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam

    Mohaimenul Azam Khan Raiaan, Md. Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam. 2024. https://doi.org/10.1109/ACCESS.2024.3365742 A review on large language models: Architectures...

    2024

  38. [46]

    o nnqvist, Aki-Juhani Kyr \

    Samuel R \"o nnqvist, Aki-Juhani Kyr \"o l \"a inen, Amanda Myntti, Filip Ginter, and Veronika Laippala. 2022. https://doi.org/10.18653/v1/2022.findings-acl.85 Explaining classes through stable word attributions . In Findings of the Association for Computational Linguistics: A...

    2022 doi

  39. [47]

    Maria Sahakyan, Zeyar Aung, and Talal Rahwan. 2021. https://doi.org/10.1109/ACCESS.2021.3116481 Explainable artificial intelligence for tabular data: A survey . IEEE Access, 9:135392--135422

    2021

  40. [48]

    Gabriele Sarti, Nils Feldhus, Ludwig Sickert, and Oskar van der Wal. 2023. https://aclanthology.org/2023.acl-demo.40 Inseq: An interpretability toolkit for sequence generation models . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (...

    2023

  41. [49]

    Howcroft, Ondrej Platek, and Adarsa Sivaprasad

    Patricia Schmidtova, Saad Mahamood, Simone Balloccu, Ondrej Dusek, Albert Gatt, Dimitra Gkatzia, David M. Howcroft, Ondrej Platek, and Adarsa Sivaprasad. 2024. https://aclanthology.org/2024.inlg-main.44/ Automatic metrics in natural language generation: A survey of current eva...

    2024

  42. [50]

    Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. https://doi.org/10.18653/v1/2020.acl-main.704 BLEURT : Learning robust metrics for text generation . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881--7892, Online. Ass...

    2020 doi

  43. [51]

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 3145–3153. JMLR.org

    2017

  44. [52]

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. https://arxiv.org/abs/1312.6034 Deep inside convolutional networks: Visualising image classification models and saliency maps . In Workshop at International Conference on Learning Representations

    2014 arXiv

  45. [53]

    Ananya Singha, José Cambronero, Sumit Gulwani, Vu Le, and Chris Parnin. 2023. https://arxiv.org/abs/2310.10358 Tabular representation, noisy operators, and impacts on table structure understanding tasks in LLM s . Preprint, arXiv:2310.10358

    2023 arXiv

  46. [54]

    Brandon Smock, Rohith Pesala, and Robin Abraham. 2022. https://doi.org/10.1109/CVPR52688.2022.00459 Pub T ables-1 M : Towards comprehensive table extraction from unstructured documents . In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4624--4632

    2022

  47. [55]

    Lya Hulliyyatus Suadaa, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura, and Hiroya Takamura. 2021. https://doi.org/10.18653/v1/2021.acl-long.115 Towards table-to-text generation with numerical reasoning . In Proceedings of the 59th Annual Meeting of the Association for C...

    2021 doi

  48. [56]

    Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. 2024. https://doi.org/10.1145/3616855.3635752 Table meets LLM : Can large language models understand structured table data? A benchmark and empirical study . In Proceedings of the 17th ACM International Conferenc...

    2024

  49. [57]

    Gemini Team, Rohan Anil, and Sebastian Borgeaud et. al. 2024. https://arxiv.org/abs/2312.11805 Gemini: A family of highly capable multimodal models . Preprint, arXiv:2312.11805

    2024 arXiv

  50. [58]

    Ian Tenney, Ryan Mullins, Bin Du, Shree Pandya, Minsuk Kahng, and Lucas Dixon. 2024. https://arxiv.org/abs/2404.07498 Interactive prompt debugging with sequence salience . arXiv, abs/2404.07498

    2024 arXiv

  51. [59]

    Gomez, ukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 6000–6010, ...

    2017

  52. [60]

    Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. 2024. https://arxiv.org/abs/2409.18839 Miner U : An open-source solution for...

    2024 arXiv

  53. [61]

    Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, and Chenghua Lin. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.746 S ci MMIR : Benchmarking scientific mult...

    2024 doi

  54. [62]

    Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xinrun Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, Guanglin Niu, Tongliang Li, and Zhoujun Li. 2024 b . https://arxiv.org/abs/2408.09174 Table B ench: A comprehensive and complex benchmark for table questio...

    2024 arXiv

  55. [63]

    Bohao Yang, Yingji Zhang, Dong Liu, André Freitas, and Chenghua Lin. 2025. https://arxiv.org/abs/2501.13042 Does table source matter? B enchmarking and improving multimodal scientific table understanding and reasoning . arXiv, abs/2501.13042

    2025 arXiv

  56. [64]

    Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. 2020. https://doi.org/10.18653/v1/2020.acl-main.745 T a BERT : Pretraining for joint understanding of textual and tabular data . In Proceedings of the 58th Annual Meeting of the Association for Computational Ling...

    2020 doi

  57. [65]

    Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.738 MM - LLM s: Recent advances in M ulti M odal large language models . In Findings of the Association for Computational Linguistics: ACL...

    2024 doi

  58. [66]

    Shuo Zhang and Krisztian Balog. 2020. https://doi.org/10.1145/3372117 Web table extraction, retrieval, and augmentation: A survey . ACM Trans. Intell. Syst. Technol., 11(2)

    2020 doi

  59. [67]

    Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.335 T able L lama: Towards open large generalist models for tables . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational ...

    2024 doi

  60. [68]

    Weinberger, and Yoav Artzi

    Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr BERTS core: Evaluating text generation with BERT . In International Conference on Learning Representations

    2020

  61. [69]

    Xiaokang Zhang, Jing Zhang, Zeyao Ma, Yang Li, Bohan Zhang, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, Jifan Yu, Shu Zhao, Juanzi Li, and Jie Tang. 2024 c . https://arxiv.org/abs/2403.19318 Table LLM : Enabling tabular data manipulation by LLM s in real ...

    2024 arXiv

  62. [70]

    Xuanliang Zhang, Dingzirui Wang, Longxu Dou, Baoxin Wang, Dayong Wu, Qingfu Zhu, and Wanxiang Che. 2024 d . https://arxiv.org/abs/2408.08841 FLEXTAF : Enhancing table reasoning with flexible tabular formats . arXiv, abs/2408.08841

    2024 arXiv

  63. [71]

    Bowen Zhao, Changkai Ji, Yuejie Zhang, Wen He, Yingwen Wang, Qing Wang, Rui Feng, and Xiaobo Zhang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.914 Large language models are complex table parsers . In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan...

    2023 doi

  64. [72]

    Meyer, and Steffen Eger

    Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019. https://doi.org/10.18653/v1/D19-1053 M over S core: Text generation evaluating with contextualized embeddings and earth mover distance . In Proceedings of the 2019 Conference on Empirical ...

    2019 doi

  65. [73]

    Weichao Zhao, Hao Feng, Qi Liu, Jingqun Tang, Shu Wei, Binghong Wu, Lei Liao, Yongjie Ye, Hao Liu, Wengang Zhou, Houqiang Li, and Can Huang. 2024. https://arxiv.org/abs/2406.01326 Tab P edia: Towards comprehensive visual table understanding with concept synergy . Preprint, arX...

    2024 arXiv

  66. [74]

    Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. 2024. https://doi.org/10.18653/v1/2024.acl-long.493 Multimodal table understanding . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum...

    2024 doi

  67. [75]

    Xinyi Zheng, Doug Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. 2020. https://arxiv.org/abs/2005.00589 Global table extractor ( GTE ): A framework for joint table identification and cell structure recognition using visual context . Preprint, arXiv:2005.00589

    2020 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.