Pith. sign in

REVIEW 9 cited by

DePlot: One-shot visual language reasoning by plot-to-table translation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2212.10505 v2 pith:MKZN3CNF submitted 2022-12-20 cs.CL cs.AIcs.CV

Fangyu Liu , Julian Martin Eisenschlos , Francesco Piccinno , Syrine Krichene , Chenxi Pang , Kenton Lee , Mandar Joshi , Wenhu Chen
show 2 more authors
Nigel Collier Yasemin Altun
This is my paper
classification cs.CLcs.AIcs.CV
keywords deplotreasoninglanguagetaskvisualone-shotsotacapabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Visual language such as charts and plots is ubiquitous in the human world. Comprehending plots and charts requires strong reasoning skills. Prior state-of-the-art (SOTA) models require at least tens of thousands of training examples and their reasoning capabilities are still much limited, especially on complex human-written queries. This paper presents the first one-shot solution to visual language reasoning. We decompose the challenge of visual language reasoning into two steps: (1) plot-to-text translation, and (2) reasoning over the translated text. The key in this method is a modality conversion module, named as DePlot, which translates the image of a plot or chart to a linearized table. The output of DePlot can then be directly used to prompt a pretrained large language model (LLM), exploiting the few-shot reasoning capabilities of LLMs. To obtain DePlot, we standardize the plot-to-table task by establishing unified task formats and metrics, and train DePlot end-to-end on this task. DePlot can then be used off-the-shelf together with LLMs in a plug-and-play fashion. Compared with a SOTA model finetuned on more than >28k data points, DePlot+LLM with just one-shot prompting achieves a 24.0% improvement over finetuned SOTA on human-written queries from the task of chart QA.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A large-scale benchmark shows that leading multimodal language models still underperform expert humans at verifying climate claims from scientific charts.

  2. Mixture of Cognitive Experts in Large Vision-Language Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Routing CV experts into atomic evidence then Bloom-staged verbalization improves LVLM benchmarks and yields measurable query-conditioned reasoning traces.

  3. Visual Programmability: A Guide for Code-as-Thought in Chart Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A vision-language model learns to dynamically switch between code-based and visual reasoning for chart questions, improving average accuracy by about one point over fixed strategies.

  4. In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding

    cs.CL 2025-07 conditional novelty 6.0 of 10

    ChartScope, using a template-based synthetic data pipeline and dual-path reasoning training, outperforms prior chart-reading models on several advanced chart benchmarks.

  5. VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection

    cs.CV 2025-05 conditional novelty 6.0 of 10

    VisTA uses GRPO reinforcement learning to train a vision-language agent to select external visual tools for a frozen reasoning model, improving accuracy on ChartQA, Geometry3K, BlindTest, and MathVerse.

  6. ChartLens: Fine-grained Visual Attribution in Charts

    cs.CL 2025-05 conditional novelty 6.0 of 10

    ChartLens uses segmentation and set-of-marks prompting to attribute chart-based answers to specific visual elements, and the authors release a new benchmark for evaluating such attribution.

  7. Chart-to-Experience: Benchmarking Multimodal LLMs for Predicting Experiential Impact of Charts

    cs.HC 2025-05 conditional novelty 6.0 of 10

    Multimodal LLMs underperform humans at directly rating charts' experiential impact, but they are substantially better at pairwise comparisons, especially when the human ratings differ clearly.

  8. Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence

    cs.CV 2025-02 conditional novelty 6.0 of 10

    Granite Vision is a ~3B parameter open-weights vision-language model that reaches state-of-the-art scores on document understanding benchmarks despite its small size.

  9. A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools

    cs.LG 2025-06 unverdicted novelty 4.0 of 10

    This survey organizes foundation models, LLM agents, datasets, and tools in materials science into six task areas.

Pith tools