REVIEW 5 major objections 6 minor 1 cited by
Multimodal Tabular Reasoning with Privileged Structured Information
T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read TURBO distills structured-table reasoning traces into an 8B multimodal model that then answers from table images alone, reporting 76.42% average accuracy across seven benchmarks, 7.2 points above prior open-source state of the art.
desk verdict The core claim isn't actually tested, but the paper is a plausible, well-motivated empirical pipeline that deserves peer review with a demand for a proper control ablation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the modality-bridged reasoning trace: a textual chain of thought that walks from the question through the relevant cells of a structured table to the gold answer, wrapped in <think>...</think><answer>...</answer> tags. It is generated by DeepSeek-R1 from the triple of structured table, question, and answer, making it answer-conditioned, and then cleaned by reject sampling to about 9k examples. The second mechanism is GRPO: for each question the model samples 16 candidate responses, computes a group-relative advantage from format and accuracy rewards, and updates the policy with a clipped objective plus a KL penalty to stay near the reference model.
What would settle it
Render the training tables as images with a controlled visual perturbation, such as blurring cell text or removing grid lines, retrain TURBO from the same 9k traces, and compare accuracy on the seven benchmarks; if accuracy stays near the reported 76.42 average, the reasoning is not depending on visual table reading, whereas a large drop would confirm the traces are genuinely image-grounded.
Extended reading notes
Core claim
The paper's central claim is that the logical path from a tabular question to its answer is modality-invariant: the same chain of thought that is correct over a clean markdown table is correct over a rendered image of that table, so it can serve as supervision for a model that never sees the markdown version at test time. TURBO implements this by generating such traces with DeepSeek-R1, filtering them with reject sampling, fine-tuning an 8B model to reproduce the trace-and-answer format, and then applying GRPO with a format reward and a rule-based accuracy reward to reinforce better reasoning paths. On the seven benchmarks the paper reports 76.42 average accuracy, presented as a 7.2-point improvement over the strongest prior open multimodal baseline.
Load-bearing premise
The load-bearing premise is that a reasoning trace written over a clean markdown table teaches the same reasoning the model needs when it sees only the rendered table image; if the image hides the structure the trace relies on, the supervision can teach answer-memorizing behavior rather than visual reasoning.
Editorial extensions
If this is right
- At inference time, TURBO needs only the table image; OCR and markdown or HTML table extraction are unnecessary, so the recipe fits deployments where only screenshots are available.
- The reported gains hold across question answering and fact verification, including 57.32 accuracy on MMMU's table questions, suggesting the learned reasoning generalizes to unfamiliar table formats.
- The text-only variant of TURBO remains competitive at 79.58 average accuracy, indicating the training also strengthens underlying language-model reasoning rather than only vision-specific skills.
- The full pipeline uses about 9k filtered examples and roughly a day of GPU time, so the recipe is reproducible with modest compute.
Reading between the lines
- If the gains depend on answer-conditioned traces, a model trained on traces written without seeing the gold answer would be the clean test; the paper's current pipeline gives the generator the answer as input along with the table and question.
- The same privileged-information transfer could be tried for other structured sources that are rendered visually at deployment, such as databases, knowledge graphs, forms, and charts.
- Scaling the GRPO group size or replacing the binary accuracy reward with a process reward on intermediate steps would show whether the reinforcement stage is truly learning step-level reasoning or just answer selection.
- The paper's limitation section notes that clean regular tables dominate the benchmarks; merged cells, nested headers, and noisy screenshots remain a plausible failure boundary for the distilled traces.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces TURBO, a framework for multimodal tabular reasoning from table images, using clean markdown tables as privileged information at training time. A teacher LLM (DeepSeek-R1) is given (table, question, answer) to generate reasoning traces, which are reject-sampled and then used to fine-tune an 8B MLLM (Ovis2) via supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO). At inference, only the table image is available. The paper reports average accuracy improvements over several open-source baselines on TABMWP, WTQ, HiTab, TAT-QA, TabFact, InfoTabs, and the MMMU table subset, claiming a +7.2% relative improvement over the strongest baseline.
Significance. The potential significance of the paper lies in showing that expensive but clean structural representations can be leveraged offline to improve reasoning on visual tables, using only about 9k training examples. If the core attribution is correct, the result is practically relevant and novel: it requires no test-time structure extraction and transfers reasoning from a large text-only LLM into a much smaller multimodal model. The paper provides a complete recipe with clear engineering details, including a two-stage training pipeline and comparisons across multiple benchmarks. The main weakness is that the mechanism advertised as the key contribution—the privileged structured information—is not directly isolated in the experiments; the reported gains could, in principle, arise from generic chain-of-thought fine-tuning. The paper also contains a misattributed MMMU comparison and lacks statistical support for its headline numbers.
major comments (5)
- [Section 4.1 / Table 1 / Figure 4] The central attribution of the gains to privileged structured tables is untested. The training traces are generated from (T, Q, A), but no ablation removes the structured table T from the trace-generation step, nor does any control compare against a generic CoT supervision source (e.g., traces from Q and A alone, or traces generated from table images with a strong multimodal teacher). Since all SFT/RL arms in Figure 4 use the same (T, Q, A)-conditioned traces, the reported improvements over the Ovis2 baseline could be explained by any reasoning-trace fine-tuning recipe. Please add at least one control arm that removes the privileged structured table from the trace-generation process, and adjust the abstract's causal language accordingly.
- [Section 5.2 / Table 1] The claim of 'surpassing HIPPO by 59.3% relative improvement' on MMMU is inaccurate: the HIPPO row in Table 1 has no MMMU entry, and the 57.32 vs 35.98 comparison is against HIPPO w/o ST, not HIPPO itself. The comparison should be reported correctly, or the absence of an MMMU result for HIPPO should be stated explicitly.
- [Table 2 / Section 5.2] The MMMU subset consists of only 165 test questions. For a multi-choice accuracy near 57%, the standard error is approximately 3.8 percentage points, so the observed gap over Ovis2-CoT (50.61%) is roughly 1.2 standard errors. No confidence intervals, significance tests, or seed variation are reported for any benchmark. Please provide at least bootstrap confidence intervals for the main results, and temper the robustness claim on MMMU.
- [Section 5.3] The ablation results are reported only as relative improvements ('10.1%', '11.8%', '13.8%') without per-dataset scores or raw accuracy values anywhere in the text, and Figure 4 is not legible in the provided manuscript. Since the two-stage contribution claim depends on these numbers, please provide a full numeric table with per-dataset accuracies and variances for the baseline, +SFT, +RL, and full TURBO.
- [Section 4.1] Because the teacher receives the ground-truth answer A while generating the reasoning trace, the model is trained on answer-conditioned rationales. While this is a common practice, the paper does not analyze whether the resulting SFT model truly grounds its reasoning in image content or simply learns to emit the answer in a formatted way; the case studies are only qualitative. To support the visually-grounded reasoning claim, please include a quantitative analysis, e.g., by perturbing table images (adding noise, altering cell order, or masking rows) and measuring accuracy drop, or by reporting the model's ability to cite specific cells from the image.
minor comments (6)
- [Section 5.1 / Appendix A.1] The text states 'we use five representative datasets: TABMWP, WTQ, HiTab, and TAT-QA'—only four are listed; also 'We export FeTaQA' should be 'We exclude FeTaQA'.
- [Section 3.1, Eq. (1)] The notation 'Y a' is used inconsistently; the 'a' should be a subscript, likely 'Y_a'.
- [Table 1] The 'Average' column does not include a value for HIPPO because its MMMU score is missing; consider reporting the average over the six datasets with a note, or explicitly marking the missing entry.
- [Section 5.2] The phrase 'surpasses the strong tabular understanding baseline HIPPO up to a relative improvement of 21.7%' is ambiguous; 'up to' should be 'by'.
- [Figure 4] As noted, Figure 4 is not legible in the manuscript; presenting the ablation numbers in a table would help.
- [General] The paper does not mention code or data release; given the importance of reproducibility, please state whether the code and generated reasoning data will be made available.
Circularity Check
No significant circularity: TURBO's reported gains are measured on held-out external benchmarks, and answer-conditioned trace generation is a training-data construction, not a validation reduction.
full rationale
The paper's derivation chain is empirical rather than definitional: structure-aware traces are generated by DeepSeek-R1 from (T,Q,A) in Section 4.1, then SFT and GRPO train an Ovis2-based MLLM to answer from table images alone (Section 4.2, Eqs. 2-4), with final results reported on held-out test splits of TABMWP, WTQ, HiTab, TAT-QA, TabFact, InfoTabs, and MMMU (Table 1). The teacher traces contain the ground-truth answer, but the trained model does not see A at test time, so the test accuracy is not forced by construction; this is standard answer-conditioned data generation, not circular validation. The base model Ovis2 has overlapping authors with the present paper, but it is used as a pretrained initialization and is also included as an external baseline; no load-bearing claim reduces to a self-cited uniqueness or equivalence theorem. The lack of an ablation that removes the structured table T or the answer A from trace generation is a possible attribution concern, but it does not make the reported external-benchmark results circular. Appendix C's explicit limitations—clean table types and a remaining gap to DeepSeek—further confirm that the claims are not definitionally secured. Accordingly, no circular step is identified.
Assumptions & free parameters
assumptions (3)
- domain assumption Reasoning trajectories are modality-invariant: the same logical path from question to answer holds for a markdown table and its rendered image.
- domain assumption DeepSeek-R1, prompted with the ground-truth answer, produces coherent and faithful traces after reject sampling.
- domain assumption Qwen2.5-72B-Instruct is a reliable judge of open-ended tabular answers.
Cite this review
Pith. "Pith review of Multimodal Tabular Reasoning with Privileged Structured Information." pith.science (2026). https://pith.science/paper/2TY67GXU
@misc{pith2026250604088,
author = {Pith},
title = {Pith review of: Multimodal Tabular Reasoning with Privileged Structured Information},
year = {2026},
howpublished = {\url{https://pith.science/paper/2TY67GXU}},
note = {Machine review of arXiv:2506.04088}
}
abstract
Tabular reasoning involves multi-step information extraction and logical inference over tabular data. While recent advances have leveraged large language models (LLMs) for reasoning over structured tables, such high-quality textual representations are often unavailable in real-world settings, where tables typically appear as images. In this paper, we tackle the task of tabular reasoning from table images, leveraging privileged structured information available during training to enhance multimodal large language models (MLLMs). The key challenges lie in the complexity of accurately aligning structured information with visual representations, and in effectively transferring structured reasoning skills to MLLMs despite the input modality gap. To address these, we introduce TabUlar Reasoning with Bridged infOrmation ({\sc Turbo}), a new framework for multimodal tabular reasoning with privileged structured tables. {\sc Turbo} benefits from a structure-aware reasoning trace generator based on DeepSeek-R1, contributing to high-quality modality-bridged data. On this basis, {\sc Turbo} repeatedly generates and selects the advantageous reasoning paths, further enhancing the model's tabular reasoning ability. Experimental results demonstrate that, with limited ($9$k) data, {\sc Turbo} achieves state-of-the-art performance ($+7.2\%$ vs. previous SOTA) across multiple datasets.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance
Decoupling structure from content during alignment plus global-to-local structured inference improves LVLM table reasoning with only 10K alignment images and no external tools.
Reference graph
Works this paper leans on
-
[1]
Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023
arXiv 2023
-
[2]
Doc- former: End-to-end transformer for document understanding
Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R Manmatha. Doc- former: End-to-end transformer for document understanding. InProceedings of the IEEE/CVF international conference on computer vision, pages 993–1003, 2021
2021
-
[3]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025
arXiv 2025
-
[4]
Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020
1901
-
[5]
Hao-Run Cai and Han-Jia Ye. Understanding the limits of deep tabular methods with temporal shift.arXiv preprint arXiv:2502.20260, 2025
arXiv 2025
-
[6]
Bigtable: A distributed storage system for structured data.ACM Transactions on Computer Systems (TOCS), 26(2):1–26, 2008
Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C Hsieh, Deborah A Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E Gruber. Bigtable: A distributed storage system for structured data.ACM Transactions on Computer Systems (TOCS), 26(2):1–26, 2008
2008
-
[7]
A survey on evaluation of large language models.ACM transactions on intelligent systems and technology, 15(3):1–45, 2024
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models.ACM transactions on intelligent systems and technology, 15(3):1–45, 2024
2024
-
[8]
Open question answering over tables and text.arXiv preprint arXiv:2010.10439, 2020
Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Wang, and William W Cohen. Open question answering over tables and text.arXiv preprint arXiv:2010.10439, 2020
arXiv 2010
Show all 90 references
-
[9]
Tabfact: A large-scale dataset for table-based fact verification
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv:1909.02164, 2019
1909 arXiv
-
[10]
Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024
2024 arXiv
-
[11]
Hitab: A hierarchical table dataset for question answering and natural language generation.arXiv preprint arXiv:2108.06712, 2021
Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. Hitab: A hierarchical table dataset for question answering and natural language generation.arXiv preprint arXiv:2108.06712, 2021
2021 arXiv
-
[12]
Multi-granularity prediction with learnable fusion for scene text recognition.arXiv preprint arXiv:2307.13244, 2023
Cheng Da, Peng Wang, and Cong Yao. Multi-granularity prediction with learnable fusion for scene text recognition.arXiv preprint arXiv:2307.13244, 2023
2023 arXiv
-
[13]
Instructblip: Towards general-purpose vision-language models with instruction tuning
Wenliang Dai, Junnan Li, D Li, AMH Tiong, J Zhao, W Wang, B Li, P Fung, and S Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. arxiv 2023.arXiv preprint arXiv:2305.06500, 2, 2023
2023 arXiv
-
[14]
Tables as texts or images: Evaluating the table reasoning ability of llms and mllms
Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. Tables as texts or images: Evaluating the table reasoning ability of llms and mllms. InFindings of the Association for Computational Linguistics ACL 2024, pages 407–426, 2024
2024
-
[15]
Turl: Table understanding through representation learning.ACM Special Interest Group on Management of Data Record, 51(1):33–40, 2022
Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. Turl: Table understanding through representation learning.ACM Special Interest Group on Management of Data Record, 51(1):33–40, 2022
2022
-
[16]
Unidoc: A universal large multimodal model for simultaneous text detection, recognition, spotting and understanding.arXiv preprint arXiv:2308.11592, 2023
Hao Feng, Zijian Wang, Jingqun Tang, Jinghui Lu, Wengang Zhou, Houqiang Li, and Can Huang. Unidoc: A universal large multimodal model for simultaneous text detection, recognition, spotting and understanding.arXiv preprint arXiv:2308.11592, 2023
2023 arXiv
-
[17]
Speculative ensemble: Fast large language model ensemble via speculation.arXiv preprint arXiv:2502.01662, 2025
Jiale Fu, Yuchu Jiang, Junkai Chen, Jiaming Fan, Xin Geng, and Xu Yang. Speculative ensemble: Fast large language model ensemble via speculation.arXiv preprint arXiv:2502.01662, 2025
2025 arXiv
-
[18]
Table detection using deep learning
Azka Gilani, Shah Rukh Qasim, Imran Malik, and Faisal Shafait. Table detection using deep learning. In2017 14th IAPR international conference on document analysis and recognition, volume 1, pages 771–776. IEEE, 2017
2017
-
[19]
Xylayoutlm: Towards layout-aware multimodal networks for visually-rich document understanding
Zhangxuan Gu, Changhua Meng, Ke Wang, Jun Lan, Weiqiang Wang, Ming Gu, and Liqing Zhang. Xylayoutlm: Towards layout-aware multimodal networks for visually-rich document understanding. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 45...
2022
-
[20]
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 10
2025 arXiv
-
[21]
Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale.arXiv preprint arXiv:2412.05237, 2024
Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale.arXiv preprint arXiv:2412.05237, 2024
2024 arXiv
-
[22]
Infotabs: Inference on tables as semi-structured data.arXiv preprint arXiv:2005.06117, 2020
Vivek Gupta, Maitrey Mehta, Pegah Nokhiz, and Vivek Srikumar. Infotabs: Inference on tables as semi-structured data.arXiv preprint arXiv:2005.06117, 2020
2005 arXiv
-
[23]
Right for the right reason: Evidence extraction for trustworthy tabular reasoning
Vivek Gupta, Shuo Zhang, Alakananda Vempala, Yujie He, Temma Choji, and Vivek Srikumar. Right for the right reason: Evidence extraction for trustworthy tabular reasoning. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...
2022
-
[24]
A survey on vision transformer.IEEE transactions on pattern analysis and machine intelligence, 45(1):87–110, 2022
Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on vision transformer.IEEE transactions on pattern analysis and machine intelligence, 45(1):87–110, 2022
2022
-
[25]
Tapas: Weakly supervised table parsing via pre-training.arXiv preprint arXiv:2004.02349, 2020
Jonathan Herzig, Paweł Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Mar- tin Eisenschlos. Tapas: Weakly supervised table parsing via pre-training.arXiv preprint arXiv:2004.02349, 2020
2004 arXiv
-
[26]
Accurate predictions on small data with a tabular foundation model.Nature, 637(8045):319–326, 2025
Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model.Nature, 637(8045):319–326, 2025
2025
-
[27]
Representation learning for tabular data: A comprehensive survey.arXiv preprint arXiv:2504.16109, 2025
Jun-Peng Jiang, Si-Yang Liu, Hao-Run Cai, Qile Zhou, and Han-Jia Ye. Representation learning for tabular data: A comprehensive survey.arXiv preprint arXiv:2504.16109, 2025
2025 arXiv
-
[28]
Tabular insights, visual impacts: transferring expertise from tables to images
Jun-Peng Jiang, Han-Jia Ye, Leye Wang, Yang Yang, Yuan Jiang, and De-Chuan Zhan. Tabular insights, visual impacts: transferring expertise from tables to images. InForty-first International Conference on Machine Learning, 2024
2024
-
[29]
A survey on table question an- swering: recent advances
Nengzheng Jin, Joanna Siebert, Dongfang Li, and Qingcai Chen. A survey on table question an- swering: recent advances. InChina Conference on Knowledge Graph and Semantic Computing, pages 174–186. Springer, 2022
2022
-
[30]
Donut: Document understanding trans- former without ocr.arXiv preprint arXiv:2111.15664, 7(15):2, 2021
Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Donut: Document understanding trans- former without ocr.arXiv preprint arXiv:2111.15664, 7(15):2, 2021
2021 arXiv
-
[31]
Tablevqa-bench: A visual question answering benchmark on multiple table domains.arXiv preprint arXiv:2404.19205, 2024
Yoonsik Kim, Moonbin Yim, and Ka Yeon Song. Tablevqa-bench: A visual question answering benchmark on multiple table domains.arXiv preprint arXiv:2404.19205, 2024
2024 arXiv
-
[32]
Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023
2023
-
[33]
Tablebank: Table benchmark for image-based table detection and recognition
Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, and Zhoujun Li. Tablebank: Table benchmark for image-based table detection and recognition. InProceedings of the Twelfth Language Resources and Evaluation Conference, pages 1918–1925, 2020
1918
-
[34]
Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning.arXiv preprint arXiv:2503.16188, 2025
Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, and Kaipeng Zhang. Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning.arXiv preprint arXiv:2503.16188, 2025
2025
-
[35]
Visual instruction tuning.Advances in neural information processing systems, 36, 2024
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36, 2024
2024
-
[36]
Tabpfn unleashed: A scalable and effective solution to tabular classification problems.arXiv preprint arXiv:2502.02527, 2025
Si-Yang Liu and Han-Jia Ye. Tabpfn unleashed: A scalable and effective solution to tabular classification problems.arXiv preprint arXiv:2502.02527, 2025
2025 arXiv
-
[37]
Hippo: Enhancing the table understanding capability of large language models through hybrid-modal preference optimization.arXiv preprint arXiv:2502.17315, 2025
Zhenghao Liu, Haolan Wang, Xinze Li, Qiushi Xiong, Xiaocui Yang, Yu Gu, Yukun Yan, Qi Shi, Fangfang Li, Ge Yu, et al. Hippo: Enhancing the table understanding capability of large language models through hybrid-modal preference optimization.arXiv preprint arXiv:2502.17315, 2025
2025
-
[38]
Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning.arXiv preprint arXiv:2209.14610, 2022
Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning.arXiv preprint arXiv:2209.14610, 2022
2022 arXiv
-
[39]
Ovis: Structural embedding alignment for multimodal large language model.arXiv preprint arXiv:2405.20797, 2024
Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural embedding alignment for multimodal large language model.arXiv preprint arXiv:2405.20797, 2024
2024 arXiv
-
[40]
Ocr-vqa: Visual question answering by reading text in images
Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In2019 international conference on document analysis and recognition, pages 947–952. IEEE, 2019
2019
-
[41]
Fetaqa: Free-form table question answering.Transactions of the Association for Computational Linguistics, 10:35–49, 2022
Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry´sci´nski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, et al. Fetaqa: Free-form table question answering.Transactions of the Association for Computational Linguistics, 10:35–49, 2022. 11
2022
-
[42]
Tableformer: Table structure understanding with transformers
Ahmed Nassar, Nikolaos Livathinos, Maksym Lysak, and Peter Staar. Tableformer: Table structure understanding with transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4614–4623, 2022
2022
-
[43]
Gpt-4o: Hello gpt-4o
OpenAI. Gpt-4o: Hello gpt-4o. Technical report, OpenAI, 2024
2024
-
[44]
Learning to reason with llms
OpenAI. Learning to reason with llms. Technical report, OpenAI, 2024
2024
-
[45]
Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...
2022
-
[46]
Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images
Shubham Singh Paliwal, D Vishwanath, Rohit Rahul, Monika Sharma, and Lovekesh Vig. Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images. In2019 international conference on document analysis and recognition (ICDAR...
2019
-
[47]
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002
2002
-
[48]
Compositional semantic parsing on semi-structured tables
Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305, 2015
2015 arXiv
-
[49]
Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl.arXiv preprint arXiv:2503.07536, 2025
Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl.arXiv preprint arXiv:2503.07536, 2025
2025 arXiv
-
[50]
Qvq: To see the world with wisdom
QwenTeam. Qvq: To see the world with wisdom. Technical report, QwenTeam, Alibaba, 2024
2024
-
[51]
Qwq: Reflect deeply on the boundaries of the unknown
QwenTeam. Qwq: Reflect deeply on the boundaries of the unknown. Technical report, QwenTeam, Alibaba, 2024
2024
-
[52]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...
2021
-
[53]
Direct preference optimization: Your language model is secretly a reward model
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023
2023
-
[54]
Deep learning for table detection and structure recognition: A survey.ACM Computing Surveys, 56(12):1–41, 2024
Mahmoud Salaheldin Kasem, Abdelrahman Abdallah, Alexander Berendeyev, Ebrahem Elkady, Mohamed Mahmoud, Mahmoud Abdalla, Mohamed Hamada, Sebastiano Vascon, Daniyar Nurseitov, and Islam Taj-Eddin. Deep learning for table detection and structure recognition: A survey.ACM Computin...
2024
-
[55]
Deepdesrt: Deep learning for detection and structure recognition of tables in document images
Sebastian Schreiber, Stefan Agne, Ivo Wolf, Andreas Dengel, and Sheraz Ahmed. Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In2017 14th IAPR international conference on document analysis and recognition, volume 1, pages 1162–116...
2017
-
[56]
Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017
2017 arXiv
-
[57]
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024
2024 arXiv
-
[58]
Math-llava: Bootstrapping mathematical reasoning for multimodal large language models.arXiv preprint arXiv:2406.17294, 2024
Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng, Lidong Bing, and Roy Ka-Wei Lee. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models.arXiv preprint arXiv:2406.17294, 2024
2024 arXiv
-
[59]
Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records.arXiv preprint arXiv:2401.07128, 2024
Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl Yang, and May D Wang. Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records.arXiv preprint arXiv:2401.07128, 2024
2024 arXiv
-
[60]
Table understanding: Problem overview.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13(1):e1482, 2023
Alexey Shigarov. Table understanding: Problem overview.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13(1):e1482, 2023
2023
-
[61]
Tablegpt2: A large multimodal model with tabular data integration.arXiv preprint arXiv:2411.02059, 2024
Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Gang Chen, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, et al. Tablegpt2: A large multimodal model with tabular data integration.arXiv preprint arXiv:2411.02059, 2024
2024 arXiv
-
[62]
Mitigating visual forgetting via take-along visual conditioning for multi-modal long cot reasoning
Hai-Long Sun, Zhun Sun, Houwen Peng, and Han-Jia Ye. Mitigating visual forgetting via take-along visual conditioning for multi-modal long cot reasoning. InACL, 2025
2025
-
[63]
Parrot: Multilingual visual instruction tuning
Hai-Long Sun, Da-Wei Zhou, Yang Li, Shiyin Lu, Chao Yi, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, et al. Parrot: Multilingual visual instruction tuning. In International conference on machine learning, 2025. 12
2025
-
[64]
Eva-clip: Improved training techniques for clip at scale.arXiv preprint arXiv:2303.15389, 2023
Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale.arXiv preprint arXiv:2303.15389, 2023
2023 arXiv
-
[65]
Multimodalqa: Complex question answering over text, tables and images.arXiv preprint arXiv:2104.06039, 2021
Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. Multimodalqa: Complex question answering over text, tables and images.arXiv preprint arXiv:2104.06039, 2021
2021 arXiv
-
[66]
Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023
2023 arXiv
-
[67]
Learning using privileged information: similarity control and knowledge transfer.J
Vladimir Vapnik, Rauf Izmailov, et al. Learning using privileged information: similarity control and knowledge transfer.J. Mach. Learn. Res., 16(1):2023–2049, 2015
2023
-
[68]
A new learning paradigm: Learning using privileged information.Neural networks, 22(5-6):544–557, 2009
Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using privileged information.Neural networks, 22(5-6):544–557, 2009
2009
-
[69]
Omniparser: A unified framework for text spotting key information extraction and table recognition
Jianqiang Wan, Sibo Song, Wenwen Yu, Yuliang Liu, Wenqing Cheng, Fei Huang, Xiang Bai, Cong Yao, and Zhibo Yang. Omniparser: A unified framework for text spotting key information extraction and table recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and ...
2024
-
[70]
Tuta: Tree-based transformers for generally structured table pre-training
Zhiruo Wang, Haoyu Dong, Ran Jia, Jia Li, Zhiyi Fu, Shi Han, and Dongmei Zhang. Tuta: Tree-based transformers for generally structured table pre-training. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1780–1790, 2021
2021
-
[71]
Chain-of-table: Evolving tables in the reasoning chain for table understanding.arXiv preprint arXiv:2401.04398, 2024
Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al. Chain-of-table: Evolving tables in the reasoning chain for table understanding.arXiv preprint arXiv:2401.04398, 2024
2024 arXiv
-
[72]
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022
2022
-
[73]
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024
2024 arXiv
-
[74]
Does table source matter? benchmarking and improving multimodal scientific table understanding and reasoning
Bohao Yang, Yingji Zhang, Dong Liu, André Freitas, and Chenghua Lin. Does table source matter? benchmarking and improving multimodal scientific table understanding and reasoning. arXiv preprint arXiv:2501.13042, 2025
2025 arXiv
-
[75]
Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024
2024 arXiv
-
[76]
A closer look at deep learning on tabular data.arXiv preprint arXiv:2407.00956, 2024
Han-Jia Ye, Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, and De-Chuan Zhan. A closer look at deep learning on tabular data.arXiv preprint arXiv:2407.00956, 2024
2024
-
[77]
A closer look at tabpfn v2: Strength, limitation, and extension.arXiv preprint arXiv:2502.17361, 2025
Han-Jia Ye, Si-Yang Liu, and Wei-Lun Chao. A closer look at tabpfn v2: Strength, limitation, and extension.arXiv preprint arXiv:2502.17361, 2025
2025 arXiv
-
[78]
Revisiting nearest neighbor for tabular data: A deep tabular baseline two decades later
Han-Jia Ye, Huai-Hong Yin, De-Chuan Zhan, and Wei-Lun Chao. Revisiting nearest neighbor for tabular data: A deep tabular baseline two decades later. InInternational Conference on Learning Representations, 2025
2025
-
[79]
Large language models are versatile decomposers: Decomposing evidence and questions for table-based rea- soning
Yunhu Ye, Binyuan Hui, Min Yang, Binhua Li, Fei Huang, and Yongbin Li. Large language models are versatile decomposers: Decomposing evidence and questions for table-based rea- soning. InProceedings of the 46th international ACM SIGIR conference on research and development in i...
2023
-
[80]
Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on Co...
2024
-
[81]
Sigmoid loss for language image pre-training
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023
2023
-
[82]
Mavis: Mathematical visual instruction tuning.arXiv e-prints, pages arXiv–2407, 2024
Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Yichi Zhang, Ziyu Guo, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, et al. Mavis: Mathematical visual instruction tuning.arXiv e-prints, pages arXiv–2407, 2024
2024
-
[83]
A survey of table reasoning with large language models.Frontiers of Computer Science, 19(9):199348, 2025
Xuanliang Zhang, Dingzirui Wang, Longxu Dou, Qingfu Zhu, and Wanxiang Che. A survey of table reasoning with large language models.Frontiers of Computer Science, 19(9):199348, 2025
2025
-
[84]
Tabpedia: Towards comprehensive visual table understanding 13 with concept synergy
Weichao Zhao, Hao Feng, Qi Liu, Jingqun Tang, Binghong Wu, Lei Liao, Shu Wei, Yongjie Ye, Hao Liu, Wengang Zhou, et al. Tabpedia: Towards comprehensive visual table understanding 13 with concept synergy. InAdvances in Neural Information Processing Systems, pages 7185–7212, 2024
2024
-
[85]
Openrt: an open-source framework for reasoning over tabular data
Yilun Zhao, Boyu Mi, Zhenting Qi, Linyong Nan, Minghao Guo, Arman Cohan, and Dragomir Radev. Openrt: an open-source framework for reasoning over tabular data. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrat...
2023
-
[86]
Multimodal table understanding
Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. Multimodal table understanding. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9102–9124, 2024
2024
-
[87]
Minigpt-4: En- hancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023
2023 arXiv
-
[88]
Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance.arXiv preprint arXiv:2105.07624, 2021
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance.arXiv preprint arXiv:2105.07624, 2021
2021 arXiv
-
[89]
Tat-llm: A specialized language model for discrete reasoning over financial tabular and textual data
Fengbin Zhu, Ziyang Liu, Fuli Feng, Chao Wang, Moxin Li, and Tat Seng Chua. Tat-llm: A specialized language model for discrete reasoning over financial tabular and textual data. In Proceedings of the 5th ACM International Conference on AI in Finance, pages 310–318, 2024
2024
-
[90]
optimum moisture content\
Wenwen Zhuang, Xin Huang, Xiantao Zhang, and Jin Zeng. Math-puma: Progressive up- ward multimodal alignment to enhance mathematical reasoning. InProceedings of the AAAI Conference on Artificial Intelligence, pages 26183–26191, 2025. 14 A Training Details In this section, we wi...
2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.