Pith. sign in

REVIEW 30 cited by

TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2105.07624 v2 pith:YU4ATALK submitted 2021-05-17 cs.CL cs.AI

Fengbin Zhu , Wenqiang Lei , Youcheng Huang , Chao Wang , Shuo Zhang , Jiancheng Lv , Fuli Feng , Tat-Seng Chua This is my paper · ORCID
classification cs.CLcs.AI
keywords datahybridtat-qareasoningtabulartextualansweranswering
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Hybrid data combining both tabular and textual content (e.g., financial reports) are quite pervasive in the real world. However, Question Answering (QA) over such hybrid data is largely neglected in existing research. In this work, we extract samples from real financial reports to build a new large-scale QA dataset containing both Tabular And Textual data, named TAT-QA, where numerical reasoning is usually required to infer the answer, such as addition, subtraction, multiplication, division, counting, comparison/sorting, and the compositions. We further propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text. It adopts sequence tagging to extract relevant cells from the table along with relevant spans from the text to infer their semantics, and then applies symbolic reasoning over them with a set of aggregation operators to arrive at the final answer. TAGOPachieves 58.0% inF1, which is an 11.1% absolute increase over the previous best baseline model, according to our experiments on TAT-QA. But this result still lags far behind performance of expert human, i.e.90.8% in F1. It is demonstrated that our TAT-QA is very challenging and can serve as a benchmark for training and testing powerful QA models that address hybrid form data.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 30 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. V-FiLLM: Verified Financial LLM Reasoning Benchmark

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A deterministic pipeline generates financial tabular QA benchmarks with answers correct by construction and controllable difficulty.

  2. FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use

    cs.AI 2026-03 conditional novelty 6.0 of 10

    FinToolBench couples 760 executable financial APIs with 295 tool-required queries and scores agents on execution success plus timeliness, intent, and domain compliance, with a finance-aware retrieval baseline (FATR).

  3. AlphaForgeBench: Benchmarking End-to-End Trading Strategy Design with Large Language Models

    q-fin.TR 2026-02 conditional novelty 6.0 of 10

    LLMs are unreliable when asked to emit buy/sell/hold actions, so this paper benchmarks them as code-writing quantitative researchers whose generated strategies are backtested deterministically.

  4. Efficacy of AI RAG Tools for Complex Information Extraction and Data Annotation Tasks: A Case Study Using Banks Public Disclosures

    cs.AI 2025-07 conditional novelty 6.0 of 10

    In a small three-person case study, a RAG tool cut bank-document annotation time by up to 90 percent, while interactive AI use showed higher agreement with an expert benchmark, though the interactive condition reused ...

  5. Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A grammar-guided evolutionary search that composes prompt edits outperformed PromptWizard, OPRO, and RL-Prompt on small LLMs across four domain-specific tasks.

  6. FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation

    cs.IR 2025-04 conditional novelty 6.0 of 10

    FinDER offers 5,703 expert-annotated ambiguous financial queries grounded in 10-K filings and shows current retrieval and generation models still struggle with them.

  7. Understanding Financial Reasoning in AI: A Multimodal Benchmark and Error Learning Approach

    cs.AI 2025-04 conditional novelty 6.0 of 10

    A new multimodal financial reasoning benchmark and a retrieval-based error feedback prompting method that improves model accuracy, with the improvement partly confounded by information leakage.

  8. TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments

    cs.LG 2025-01 conditional novelty 6.0 of 10

    TabFSBench simulates feature removal in tabular data across four scenarios and 24 models, reporting that most models degrade, tabular LLMs are most robust, and that closed-environment performance correlates with featu...

  9. ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding

    cs.CV 2025-01 conditional novelty 6.0 of 10

    Visual editing of input images as a chain of thought improves multimodal LLM accuracy on structured image tasks by 3 to 12 points.

  10. TQA-Bench: Evaluating LLMs for Multi-Table Question Answering

    cs.AI 2024-11 conditional novelty 6.0 of 10

    TQA-Bench is a multi-table QA benchmark with 8K-64K contexts and symbolic question extensions; tests on 22 LLMs show accuracy declines with context length.

  11. MATATA: Weakly Supervised End-to-End MAthematical Tool-Augmented Reasoning for Tabular Applications

    cs.LG 2024-11 conditional novelty 6.0 of 10

    MATATA uses final-answer-only weak supervision with instruction tuning and KTO preference optimization to train tool-augmented SLM agents that beat or match much larger models on FinQA, TAT-QA, and TabMWP.

  12. CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Multimodal context-learning benchmark CLBench-V separates grounding, information application, and knowledge acquisition; the best evaluated model scores 0.2847.

  13. When Tables Go Crazy: Evaluating Multimodal Models on French Financial Documents

    cs.CL 2026-02 conditional novelty 5.0 of 10

    A new French financial document benchmark shows vision-language models are strong at text/table extraction but brittle on charts and multi-turn dialogue, with accuracy converging near 50% in conversational settings.

  14. CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation

    cs.CL 2025-08 conditional novelty 5.0 of 10

    A fine-tuned Llama 3 model with a trained document critic and program-based reasoning beats GPT-4o and other baselines on a new carbon footprint QA benchmark.

  15. FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Fine-tuning small language models on synthetic financial errors yields high detection and editing scores, but the evaluation is limited to synthetic data from the same pipeline.

  16. Unlocking Speech Instruction Data Potential with Query Rewriting

    cs.AI 2025-07 conditional novelty 5.0 of 10

    A multi-LLM rewriting and multi-agent validation pipeline makes text-to-speech synthesized speech instruction data far more usable and improves downstream speech instruction following.

  17. What to Keep and What to Drop: Adaptive Table Filtering Framework

    cs.CL 2025-06 conditional novelty 5.0 of 10

    ATF prunes table columns and rows with LLM scoring plus retrieval, cutting cells by about 70% and improving out-of-domain TableQA accuracy, while hurting in-domain QA and fact verification.

  18. Multimodal Tabular Reasoning with Privileged Structured Information

    cs.LG 2025-06 conditional novelty 5.0 of 10

    An 8B multimodal LLM trained on 9k reasoning traces distilled from structured tables reaches state-of-the-art open-source accuracy on table-image question answering and fact verification.

  19. MMTABREAL: Real-World Benchmark for Multimodal Table Understanding

    cs.CV 2025-05 conditional novelty 5.0 of 10

    The paper releases a 500-table, 4,021-question benchmark of real-world multimodal tables and shows that leading vision-language models drop 20-40% in accuracy relative to earlier synthetic benchmarks.

  20. Realistic Evaluation of TabPFN v2 in Open Environments

    cs.LG 2025-05 conditional novelty 5.0 of 10

    TabPFN v2 underperforms tree-based models on most open-environment tabular tasks and is only preferable on small, covariate-shifted, class-balanced data.

  21. KFinEval-Pilot: A Comprehensive Benchmark Suite for Korean Financial Language Understanding

    cs.CL 2025-04 conditional novelty 5.0 of 10

    KFinEval-Pilot is a new Korean financial benchmark combining knowledge, legal reasoning, and toxicity tasks, and its model evaluations show clear performance and safety differences.

  22. Knowledge Boundary of Large Language Models: A Survey

    cs.CL 2024-12 conditional novelty 5.0 of 10

    A survey that formalizes the knowledge boundary of LLMs into a four-type taxonomy and reviews detection and mitigation methods.

  23. SusGen-GPT: A Data-Centric LLM for Financial NLP and Sustainability Report Generation

    cs.CL 2024-12 reject novelty 5.0 of 10

    Small fine-tuned models on SusGen-30K are reported to nearly match GPT-4 on financial and ESG tasks, with a new TCFD-Bench benchmark, though the comparison is biased.

  24. Hierarchical Reranking for Scalable Financial RAG System

    cs.IR 2026-07 reject novelty 4.0 of 10

    A finance-specific RAG pipeline combining table-to-JSON conversion, two-stage reranking, and long-context split-fusion reports NDCG@20=0.7918 and second place in the ICAIF '24 FinanceRAG challenge.

  25. On the Comprehensibility of Multi-structured Financial Documents using LLMs and Pre-processing Tools

    cs.IR 2025-06 conditional novelty 4.0 of 10

    Preprocessing financial PDFs into text, tables, and chart data with existing tools improves LLM question-answering accuracy over direct GPT-4o image input in a small private evaluation.

  26. Learning Sparsity for Effective and Efficient Music Performance Question Answering

    cs.SD 2025-06 conditional novelty 4.0 of 10

    Sparsify reports state-of-the-art accuracy on Music AVQA benchmarks by borrowing three existing sparsification techniques, cutting training time by 28% and retaining 70-80% of accuracy on a 25% data subset.

  27. Exploring Performance Contrasts in TableQA: Step-by-Step Reasoning Boosts Bigger Language Models, Limits Smaller Language Models

    cs.CL 2024-11 conditional novelty 4.0 of 10

    A sequential prompting flow for TableQA improves accuracy in 70B-class LLMs but lowers it in most 7B-class LLMs, with the largest diagnosed gap in table-structure understanding.

  28. PHLoRA: data-free Post-hoc Low-Rank Adapter extraction from full-rank checkpoint

    cs.LG 2025-09 conditional novelty 3.0 of 10

    PHLoRA extracts LoRA-compatible adapters from full-rank fine-tuned models via truncated SVD of the weight delta, matching full-rank performance on several benchmarks with no gradients or training data.

  29. Open FinLLM Leaderboard: Towards Financial AI Readiness

    cs.CE 2025-01 conditional novelty 3.0 of 10

    The paper presents an open, continuously updated FinLLM leaderboard that aggregates existing financial benchmarks and demos for comparing models.

  30. Text2Insight: Transform natural language text into insights seamlessly using multi-model architecture

    cs.AI 2024-12 reject novelty 3.0 of 10

    Text2Insight combines an LLM text-to-SQL step with a rule-based chart predictor and BERT-based question answering and prediction, but its end-to-end performance claims rest on circular or missing evaluation.

Pith tools