T-REX: Table -- Refute or Entail eXplainer

Baptiste Geisenberger; Mehwish Alam; Tim Luka Horstmann

REVIEW 4 major objections 5 minor 8 references

T-REX: Table -- Refute or Entail eXplainer

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read T-REX is a live web tool that verifies claims against uploaded tables, and its best configuration scores 89% on TabFact.

desk verdict A genuinely useful open demo for table fact-checking, undermined by a headline accuracy number with no evaluation protocol attached. read the letter →

arxiv 2508.14055 v1 pith:3E2JFZKX submitted 2025-08-08 cs.CL cs.AI

Tim Luka Horstmann , Baptiste Geisenberger , Mehwish Alam This is my paper

classification cs.CLcs.AI

keywords tablefact-checkinglargelanguagemodelsreal-timefactverificationnaturalprocessingOCRmultilingualinterpretabilityTab

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

T-REX is a web-based tool that takes a table—as a CSV file, pasted text, or OCR-extracted image—and a claim, and returns a true/false verdict with streaming reasoning and highlighted cells. The paper's core empirical finding is that its best configuration, Phi-4 with the table turned into naturalized prose and chain-of-thought prompting, reaches 89% accuracy on the TabFact benchmark, ahead of the RePanda baseline. The authors present T-REX as the first live, interactive, domain-agnostic table fact-checker aimed at non-experts, with input and output in eight languages and no user data stored externally. If the tool works in deployment as it does on TabFact, it would let journalists, analysts, and ordinary readers check numerical claims against tables without writing code.

What carries the argument

The key mechanism is the serialisation-and-prompt pipeline: tables are cleaned, stamped with a synthetic row_index so the model can point at cells, and converted into one of four textual formats (Markdown, HTML, JSON, naturalized text). A chain-of-thought instruction then asks the reasoning LLM to output its deliberation followed by a JSON object containing the verdict and the relevant cells; post-processing recovers the JSON if parsing fails. In the winning setup, the naturalized-text format is what lets Phi-4 reach 89% while keeping the reasoning visible.

What would settle it

Take 100 table/claim pairs newly written from French and Spanish Wikipedia tables and 100 photos of printed tables with the same claims; run them through the public T-REX demo with Phi-4, naturalized formatting, and chain-of-thought; compare verdicts to human labels. If accuracy on this held-out, deployment-like set falls below 75%, the claim that the tool generalizes to arbitrary user-provided tables would be falsified.

Watch

Extended reading notes

Core claim

On its own terms, T-REX is a working system, not a new method: users upload a CSV, paste a table, or submit an image (OCR via IBM Granite 3.2 or Tesseract), type a claim, and the backend serialises the table with a synthetic row-index column, prompts an instruction-tuned reasoning LLM, and streams the reasoning plus a structured JSON verdict with self-identified relevant cells. The paper's empirical claim is that the best configuration—Phi-4, with the table rewritten as naturalized text and chain-of-thought prompting—achieves 89% accuracy on the TabFact test set, above the RePanda baseline. The novelty claim is that this is the first live, interactive, domain-agnostic table fact-checker for

Load-bearing premise

The assumption is that the prompt pipeline tuned on TabFact's English, cleanly formatted tables transfers unchanged to arbitrary user-provided tables—including OCR-extracted images and eight non-English languages—without additional tuning or validation.

Editorial extensions

If this is right

Non-experts can verify a claim against their own table—CSV upload, OCR photograph, or a TabFact example—and get a true/false verdict with visible reasoning in real time.
Eight-language support and highlighted cells make the reasoning transparent enough for a user to check the model's work.
Because inference runs on open-source models and all data stays in memory, the tool avoids storing user data externally.
The modular design means the same prompt pipeline can be swapped onto larger or future instruction-tuned models as hardware allows.
The 89% TabFact result positions the best configuration ahead of the RePanda baseline, so a strong open-source setup exists for table verification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 89% figure is an offline TabFact result; whether it transfers to OCR-extracted or multilingual tables is a testable prediction, not yet a measured one.
The row-indexed, naturalized-table prompt format is generic enough that the same pipeline could be pointed at the future tasks the paper names—table completion, semantic table retrieval, and cell-level evidence justification.
The choice of smaller models plus a 'Deep Thinking' mode suggests a natural accuracy-versus-latency dial for a live service.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Bottom line: T-REX is a real, working demo with an open repo, and that's the paper's actual contribution. The empirical headline—89% on TabFact—is presented without the protocol needed to trust it, and the authors appear to have selected the best configuration on the same benchmark they report. That needs fixing before the number is cited as a comparison.

What's genuinely new: it's a live end-to-end system that takes arbitrary user-provided tables (CSV or OCR'd images), supports eight languages, streams reasoning, and highlights the cells the model says it used. Prior tools like OpenTFV and Aletheia assume predefined datasets; T-REX doesn't. The modular FastAPI backend with Ollama, in-memory processing, and privacy posture is sensible. For a system/demo paper, this is a solid engineering contribution, and making the code and demo public is exactly right.

The soft spot is the evaluation in Section 2.1. We get 'Phi-4 with naturalized table formatting and CoT prompting achieved 89% accuracy, surpassing prior methods like RePanda.' There are no comparison numbers for RePanda, no error bars, no statement of how many TabFact claims were used or whether the full test set was used, no info on multiple runs, and no description of how the final setup was selected among three strategies, five LLMs, four formats, and two prompting techniques. Since that search happened on TabFact itself, the 89% is a selected maximum and likely inflated. The paper points to the GitHub and the demo for detailed results; that's better than nothing, but the protocol belongs in the paper because the number is load-bearing for the 'accurate verification' claim. It's also worth noting the 'first live interactive tool' phrasing in the abstract is a product-type claim; not enough to call it out beyond a raised eyebrow.

The transfer from curated TabFact tables to OCR'd, multilingual, user-submitted inputs is asserted rather than measured. That's a minor concern because demo papers aren't expected to prove real-world generalization, but a small case study would help.

Who's this for: people building or evaluating table fact-checking interfaces, and anyone who wants a concrete open baseline for LLM-based claim verification over tables. The paper deserves peer review—in a demo/system track, not a main-research one—with the expectation that the authors add a real evaluation section: exact protocol, baseline numbers, variance, and at least a few qualitative live examples. As it stands, I'd read it as a demo announcement with a useful open resource, not as a benchmark claim.

Referee Report

4 major / 5 minor

Summary. The paper introduces T-REX, a web-based tool for verifying natural-language claims against user-supplied tables, including CSV uploads and OCR-extracted images. The system uses open-source instruction-tuned LLMs served through Ollama, supports streaming chain-of-thought reasoning, highlights relevant table cells, and offers multilingual input/output. The central empirical claim is made in Section 2.1: the best configuration, Phi-4 with naturalized table formatting and CoT prompting, achieves 89% accuracy on the TabFact test set and 'surpasses prior methods like RePanda.' The paper also claims T-REX is the first live, interactive tool for table claim verification.

Significance. If the accuracy and usability claims are substantiated, T-REX would be a useful contribution: it addresses a real accessibility gap, is openly available as a demo, and its design choices (interpretability via cell highlighting and streaming reasoning, no user-data storage) are commendable. The manuscript additionally ships a live demo and GitHub repository, which is a concrete strength. However, the paper's value proposition is anchored to the 89% TabFact number and the claim of surpassing RePanda, and the manuscript provides no reproducible protocol, no comparison number, no variance estimate, and no evaluation of the live tool's actual inputs (OCR, multilingual, arbitrary user tables). The contribution is therefore currently contingent on evidence that is not in the paper.

major comments (4)

[Section 2.1] The headline result, 'Phi-4 with naturalized table formatting and CoT prompting, achieved 89% accuracy, surpassing prior methods like RePanda on TabFact,' is not supported by the manuscript. The paper does not state whether the full TabFact test set or a subset was used, how many claims were evaluated, whether multiple runs were averaged, which exact prompts/templates were used, what decoding parameters were set, or what RePanda's reported accuracy is and whether the evaluation protocol matches. Without these details, the 89% figure cannot be reproduced or compared. Please add a complete evaluation protocol and a comparison table with RePanda and, ideally, DATER/ARTEMIS-DA.
[Section 2.1, selection of the best configuration] The best configuration was selected after 'extensive offline experiments on the TabFact test set' and 'iteratively refined prompt design,' and only the top result is reported. Since the same benchmark is used for both configuration selection and final evaluation, the reported 89% is likely a selected maximum over a large grid (3 strategies × 5 LLMs × 4 formats × 2 prompting styles) and may overstate expected performance. The manuscript should either report results for all configurations (a table is sufficient) or apply a selection-bias-aware analysis, such as a held-out validation set not used during prompt iteration. This is load-bearing because the 89% number is the main evidence for the tool's accuracy.
[Section 2.1 / 2.2, external validity of the live tool] The paper claims T-REX supports OCR-extracted images, multilingual inputs, and arbitrary user-provided tables, but no evaluation is reported on any of these actual use cases. The TabFact benchmark consists of clean, structured text tables, not OCR output or user-uploaded images. A small evaluation on the live tool—e.g., a set of OCR'd tables or multilingual claims—would be needed to support the 'accessible to non-experts' claim. Without it, the accuracy result does not transfer to the system as deployed.
[Section 2.1 and Conclusion] The manuscript defers actual results to external links: 'Detailed results for (1) are available in the online demo; final results for (1-3) on our GitHub.' This makes the paper non-self-contained for its central empirical claim. The 'surpassing RePanda' assertion must be backed by numbers in the paper itself, not only in a repository. Please include the key results in the manuscript, even as an appendix or short table.

minor comments (5)

[Figure 1] The label 'TRUEFALSEor' is garbled; it should read 'TRUE/FALSE' or 'TRUE or FALSE'.
[Throughout] Typographical issues: 'T-REXsupportstableinput' in Section 2.1 lacks a space; 'entailed or refuted' should be consistent with 'entail/refute' terminology used elsewhere.
[Section 1] The claim of being 'the first live, interactive tool' should be phrased carefully relative to OpenTFV and Aletheia, which the paper itself describes as partially addressing interactive verification; clarify the precise novelty (e.g., arbitrary user-provided tables and open-source SOTA LLMs).
[Section 2.3] The hardware description ('two NVIDIA GTX 1080 GPUs') is useful, but the manuscript should note whether inference latency is acceptable for interactive use; even approximate latency numbers would strengthen the 'real-time' claim.
[References] Several references are arXiv preprints; that is acceptable for a fast-moving area, but the comparison with RePanda should cite the specific accuracy number reported in RePanda and state the source.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: T-REX reports measured benchmark accuracy, not a derivation from its own inputs.

full rationale

The paper's load-bearing empirical claim is the reported 89% TabFact accuracy (Section 2.1) and the 'first live tool' positioning. Neither is derived from a premise that includes the conclusion. The 89% figure is an observed accuracy from running Phi-4 with naturalized formatting and CoT on the TabFact test set, not a quantity obtained by renaming an input or fitting a parameter and then calling the result a prediction. No model parameters are fit; the configuration is manually selected among LLMs, formats, and prompts, and the accuracy is then read off the same benchmark. This creates a risk of selection-induced inflation (the same TabFact test set was used for iterative prompt design and final reporting), and the comparison to RePanda would require a matched protocol, but this is an evaluation-validity concern, not circularity. The paper cites only external prior work (RePanda, DATER, ARTEMIS-DA, TabFact, OpenTFV, Aletheia); there are no self-citations or imported uniqueness theorems. Therefore no step in the paper reduces by construction to its own inputs.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities or adjustable numeric parameters beyond the choice of the best-performing configuration. The key assumptions are about benchmark validity, faithfulness of model reasoning, OCR reliability, and prompt generalization.

free parameters (1)

Best configuration (model, table format, prompting) = Phi-4, naturalized table formatting, chain-of-thought
This configuration was selected based on comparative experiments on the TabFact test set and is the only result reported. It is a hand-chosen setup that may overfit the benchmark.

assumptions (4)

domain assumption TabFact is a valid and representative benchmark for table fact-checking.
The paper uses TabFact to guide design and report accuracy; its validity as ground truth is assumed.
domain assumption The LLM's self-identified relevant cells and reasoning text faithfully reflect the basis for the verdict.
The interpretability claim relies on the model's reasoning output being faithful, which is not independently verified.
domain assumption OCR and preprocessing accurately preserve table structure from images.
The multimodal input mode depends on OCR fidelity, but the paper provides no OCR accuracy evaluation.
ad hoc to paper The manually refined prompt design generalizes beyond TabFact.
The prompts were iteratively tuned using manual evaluation on TabFact; no evidence shows they transfer to other domains or user-provided tables.

how reviews work

0 comments

Cite this review

Pith. "Pith review of T-REX: Table -- Refute or Entail eXplainer." pith.science (2026). https://pith.science/paper/3E2JFZKX

@misc{pith2026250814055,
  author       = {Pith},
  title        = {Pith review of: T-REX: Table -- Refute or Entail eXplainer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3E2JFZKX}},
  note         = {Machine review of arXiv:2508.14055}
}

read the original abstract

Verifying textual claims against structured tabular data is a critical yet challenging task in Natural Language Processing with broad real-world impact. While recent advances in Large Language Models (LLMs) have enabled significant progress in table fact-checking, current solutions remain inaccessible to non-experts. We introduce T-REX (T-REX: Table -- Refute or Entail eXplainer), the first live, interactive tool for claim verification over multimodal, multilingual tables using state-of-the-art instruction-tuned reasoning LLMs. Designed for accuracy and transparency, T-REX empowers non-experts by providing access to advanced fact-checking technology. The system is openly available online.

Figures

Figures reproduced from arXiv: 2508.14055 by the authors.

**Figure 1.** T-REX system overview. Users provide tables and claims via the frontend, while the backend handles OCR, preprocessing, and LLM-based verification. 2.1 Table Processing and Verification Pipeline T-REX supports table input via CSV files or OCR-extracted images, using IBM’s Granite 3.2 vision model for high accuracy or Tesseract as a faster alternative. Tables are preprocessed for robustness (unifying delimiters, clea… view at source ↗

**Figure 2.** T-REX interface after claim verification. Relevant cells are highlighted for interpretability and reflect the model’s self-identified reasoning steps. The model’s reasoning is accessible via dropdown menus. Ollama serves both LLM inference and OCR tasks, while SlowAPI enforces request rate limiting to maintain robustness under load. T-REX currently runs on two NVIDIA GTX 1080 GPUs, and its modular design makes it … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

8 extracted references · 4 canonical work pages

[1]

RePanda: Pandas-powered Tabular Verification and Reasoning

Chegini, A.M., Rezaei, K., Eghbalzadeh, H., Feizi, S.: RePanda : Pandas-powered Tabular Verification and Reasoning . In: arXiv :2503.11921 (2025)

work page Pith review arXiv 2025
[2]

In: arXiv :2301.13808

Ye, Y., Hui, B., Yang, M., Li, B., Huang, F., Li, Y.: Large language models are versatile decomposers: Decompose evidence and questions for table-based reasoning. In: arXiv :2301.13808. arXiv (2023)

arXiv 2023
[3]

In: arXiv:2412.14146 (2024)

Hussain, A.S.: ARTEMIS - DA : An Advanced Reasoning and Transformation Engine for Multi-Step Insight Synthesis in Data Analytics with Large Language Models . In: arXiv:2412.14146 (2024)

arXiv 2024
[4]

In: International Conference on Learning Representations (2020)

Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., Wang, W.Y.: TabFact : A Large-scale Dataset for Table-based Fact Verification . In: International Conference on Learning Representations (2020)

work page 2020
[5]

In: International Conference on Management of Data (2022)

Gu, Z., Fan, R., Zhao, X., Zhang, M., Fan, J., Du, X.: OpenTFV: An Open Domain Table-Based Fact Verification System . In: International Conference on Management of Data (2022)

work page 2022
[6]

The Data Says Otherwise

Fu, Y., Guo, S., Hoffswell, J., Bursztyn, V.S., Rossi, R., Stasko, J.: " The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims . In: Annual ACM Symposium on User Interface Software and Technology (2024)

work page 2024
[7]

, " * write output.state after.block = add.period write

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...
[8]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

RePanda: Pandas-powered Tabular Verification and Reasoning

Chegini, A.M., Rezaei, K., Eghbalzadeh, H., Feizi, S.: RePanda : Pandas-powered Tabular Verification and Reasoning . In: arXiv :2503.11921 (2025)

work page Pith review arXiv 2025

[2] [2]

In: arXiv :2301.13808

Ye, Y., Hui, B., Yang, M., Li, B., Huang, F., Li, Y.: Large language models are versatile decomposers: Decompose evidence and questions for table-based reasoning. In: arXiv :2301.13808. arXiv (2023)

arXiv 2023

[3] [3]

In: arXiv:2412.14146 (2024)

Hussain, A.S.: ARTEMIS - DA : An Advanced Reasoning and Transformation Engine for Multi-Step Insight Synthesis in Data Analytics with Large Language Models . In: arXiv:2412.14146 (2024)

arXiv 2024

[4] [4]

In: International Conference on Learning Representations (2020)

Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., Wang, W.Y.: TabFact : A Large-scale Dataset for Table-based Fact Verification . In: International Conference on Learning Representations (2020)

work page 2020

[5] [5]

In: International Conference on Management of Data (2022)

Gu, Z., Fan, R., Zhao, X., Zhang, M., Fan, J., Du, X.: OpenTFV: An Open Domain Table-Based Fact Verification System . In: International Conference on Management of Data (2022)

work page 2022

[6] [6]

The Data Says Otherwise

Fu, Y., Guo, S., Hoffswell, J., Bursztyn, V.S., Rossi, R., Stasko, J.: " The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims . In: Annual ACM Symposium on User Interface Software and Technology (2024)

work page 2024

[7] [7]

, " * write output.state after.block = add.period write

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

[8] [8]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...