ChartArena unifies eight chart families across three real-world visual scenarios and two languages under a format-agnostic triple/graph evaluation protocol, revealing clear gaps among 26 MLLMs.
hub
Chartllama: A mul- timodal llm for chart understanding and generation
19 Pith papers cite this work. Polarity classification is still indexing.
19
Pith papers citing it
abstract
Multi-modal large language models have demonstrated impressive performances on most vision-language tasks. However, the model generally lacks the understanding capabilities for specific domain data, particularly when it comes to interpreting chart figures. This is mainly due to the lack of relevant multi-modal instruction tuning datasets. In this article, we create a high-quality instruction-tuning dataset leveraging GPT-4. We develop a multi-step data generation process in which different steps are responsible for generating tabular data, creating chart figures, and designing instruction tuning data separately. Our method's flexibility enables us to generate diverse, high-quality instruction-tuning data consistently and efficiently while maintaining a low resource expenditure. Additionally, it allows us to incorporate a wider variety of chart and task types not yet featured in existing datasets. Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset. ChartLlama outperforms all prior methods in ChartQA, Chart-to-text, and Chart-extraction evaluation benchmarks. Additionally, ChartLlama significantly improves upon the baseline in our specially compiled chart dataset, which includes new chart and task types. The results of ChartLlama confirm the value and huge potential of our proposed data generation method in enhancing chart comprehension.
hub tools
citation-role summary
background 3
dataset 1
citation-polarity summary
representative citing papers
Introduces QCalEval benchmark showing best zero-shot VLM score of 72.3 on quantum calibration plots, with fine-tuning and in-context learning effects varying by model type.
InterChart is a new benchmark that reveals steep drops in VLM accuracy when moving from single-chart facts to integrative reasoning over 2-3 related charts, with better performance after decomposing complex charts.
Introduces a benchmark for MLLM-based chart data extraction from unlabeled images and a human-centered training framework that reaches SOTA numerical accuracy with a 7B model.
TRON supplies 520 rule-verifiable online visual reasoning environments across five ability buckets that generate unlimited training instances for RL post-training, yielding consistent gains on ten external multimodal benchmarks for three vision-language models.
ChartAct is a benchmark for dynamic chart understanding that shows most multimodal models struggle with interactive charts, with top performance at 84.5%.
Chart-FR1 uses Focus-CoT for linking reasoning to visual cues and Focus-GRPO reinforcement learning with efficiency rewards to outperform prior MLLMs on dense chart reasoning tasks.
A 7B/8B model trained with decoupled tri-perspective SFT and QA-verified RL matches GPT-4o and approaches GPT-5 on chart-to-code generation benchmarks.
ChartVerse uses Rollout Posterior Entropy and truth-anchored inverse QA synthesis to produce 640K high-quality chart reasoning samples, training an 8B model that surpasses its 30B teacher.
CycleChart is a consistency-based framework that organizes chart generation, schema parsing, data parsing, and QA around single data instances to enforce bidirectional semantic alignment and improve cross-task generalization.
ChatImage renders LLM answers as images, then uses visual grounding to place clickable hotspots on rendered regions for interactive follow-up.
chart-plot is an agentic harness using style-aware code generation from venue figures, a LaTeX-aware render-and-revise loop, and structured edit handles to produce top-venue-ready academic charts.
The work introduces a chart-to-dictionary auxiliary task paired with Program-of-Thoughts prompting to enable zero-shot chart summarization that matches existing methods on semantic and factual metrics.
A structured survey of multimodal code intelligence that formulates the field by code roles and organizes work into four domains while proposing verification-centered research directions.
Survey proposing a taxonomy for document parsing into pipeline-based systems and VLM-driven unified models, reviewing components, metrics, benchmarks, and challenges.
citing papers explorer
-
ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats
ChartArena unifies eight chart families across three real-world visual scenarios and two languages under a format-agnostic triple/graph evaluation protocol, revealing clear gaps among 26 MLLMs.
-
QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding
Introduces QCalEval benchmark showing best zero-shot VLM score of 72.3 on quantum calibration plots, with fine-tuning and in-context learning effects varying by model type.
-
InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information
InterChart is a new benchmark that reveals steep drops in VLM accuracy when moving from single-chart facts to integrative reasoning over 2-3 related charts, with better performance after decomposing complex charts.
-
Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework
Introduces a benchmark for MLLM-based chart data extraction from unlabeled images and a human-centered training framework that reaches SOTA numerical accuracy with a 7B model.
-
TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RL
TRON supplies 520 rule-verifiable online visual reasoning environments across five ability buckets that generate unlimited training instances for RL post-training, yielding consistent gains on ten external multimodal benchmarks for three vision-language models.
-
ChartAct: A Benchmark for Dynamic Chart Understanding
ChartAct is a benchmark for dynamic chart understanding that shows most multimodal models struggle with interactive charts, with top performance at 84.5%.
-
Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts
Chart-FR1 uses Focus-CoT for linking reasoning to visual cues and Focus-GRPO reinforcement learning with efficiency rewards to outperform prior MLLMs on dense chart reasoning tasks.
-
CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution
A 7B/8B model trained with decoupled tri-perspective SFT and QA-verified RL matches GPT-4o and approaches GPT-5 on chart-to-code generation benchmarks.
-
ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch
ChartVerse uses Rollout Posterior Entropy and truth-anchored inverse QA synthesis to produce 640K high-quality chart reasoning samples, training an 8B model that surpasses its 30B teacher.
-
CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generation
CycleChart is a consistency-based framework that organizes chart generation, schema parsing, data parsing, and QA around single data instances to enforce bidirectional semantic alignment and improve cross-task generalization.
-
ChatImage: Navigating Long-Form LLM Answers through Interactive Images
ChatImage renders LLM answers as images, then uses visual grounding to place clickable hotspots on rendered regions for interactive follow-up.
-
Demonstrating chart-plot: Closing the Last Mile of Academic Chart Generation
chart-plot is an agentic harness using style-aware code generation from venue figures, a LaTeX-aware render-and-revise loop, and structured edit handles to produce top-venue-ready academic charts.
-
From Data to Insights: Exploring Program-of-Thoughts Prompting for Chart Summarization
The work introduces a chart-to-dictionary auxiliary task paired with Program-of-Thoughts prompting to enable zero-shot chart summarization that matches existing methods on semantic and factual metrics.
-
Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence
A structured survey of multimodal code intelligence that formulates the field by code roles and organizes work into four domains while proposing verification-centered research directions.
-
Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction
Survey proposing a taxonomy for document parsing into pipeline-based systems and VLM-driven unified models, reviewing components, metrics, benchmarks, and challenges.
- DataComp-VLM: Improved Open Datasets for Vision-Language Models
- ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models
- CharTool: Tool-Integrated Visual Reasoning for Chart Understanding
- AppAgent: Multimodal Agents as Smartphone Users