Pith. sign in

REVIEW 5 major objections 5 minor 52 references

LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LongChart, a multi-chart VQA benchmark of 557 images and 2,876 close-ended questions synthesized from one consistent latent-graph data space, shows MLLM accuracy declines and spreads as the computation hops needed to answer grow.

desk verdict A useful benchmark idea with a plausible hop-accuracy trend, but the formal guarantee that makes it valid is unproven and the paper needs major cleanup before it is citable. read the letter →

arxiv 2608.01328 v1 pith:GEGCNF4A submitted 2026-08-02 cs.CL cs.AIcs.CV

Ziyan Xiao , Yinghao Zhu , Wenting Zhang , Heaju Kim , Lequan Yu This is my paper
classification cs.CLcs.AIcs.CV
keywords LongChartmulti-chartVQAmultimodallargelanguagemodelsreasoningbenchmarklatentgraphhopsMLLMevaluationvisualgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces LongChart, a multi-chart visual question answering benchmark in which each question set averages 6.5 chart images and 31.2 finite-answer questions, all generated from a single latent dependency graph so every chart in a set lives in one consistent data space. The authors claim this design makes difficulty a controlled function of hops, the number of deterministic computation steps an answer requires, rather than answer length or external knowledge. Evaluating ten multimodal large language models, they report that accuracy falls and varies more as hop count grows, that supplying intermediate visualizations (extended context) beats reasoning from a minimal chart set, and that grounding and pointing outputs are unreliable and hurt accuracy. If the construction holds, the benchmark separates accuracy losses due to reasoning depth from losses due to answer ambiguity, providing a controlled testbed for multi-chart and long-context reasoning.

What carries the argument

The latent graph: a DAG $G=(V,E)$ over data nodes, computation nodes (deterministic transforms such as sum, ratio, difference, time-shift), and image nodes. Edges carry an information-preservation attribute — bidirected $u \leftrightarrow v$ means the transformation is reversible ($H(u|v)=H(v|u)=0$), directed $u \to v$ means $v$ cannot reconstruct $u$. Graph traversal finds a minimal generative set $S$ (complete, independent, conflict-free), the only nodes synthesized by an LLM under human-specified trends; all other nodes are propagated deterministically, guaranteeing uniqueness and consistency. Questions are then constructed with a hop count, the number of computation steps from the displa

What would settle it

Pick one benchmark chart and have independent annotators (or an automated chart-extraction tool) reconstruct every data value from the pixels alone. Any value that cannot be recovered — axis truncation, overlapping labels, rounded ticks, hidden series — shows $H(V|S)>0$, so answer uniqueness fails and the accuracy drop measured on such questions becomes a benchmark artifact. A complementary check: compute inter-annotator agreement on whether each of the 2,876 questions has exactly one defensible answer.

Watch

Extended reading notes

Core claim

LongChart is a multi-chart VQA benchmark built so question difficulty is controlled by reasoning depth, not answer format. Each question set is generated from a directed acyclic latent graph with data, computation, and image nodes; edges are bidirected (reversible, lossless) or directed (one-way). Only a minimal generative set $S$ — complete ($H(V|S)=0$), independent, conflict-free — is synthesized; the rest is computed deterministically, so all charts share one data space and each question has a unique finite answer. Difficulty is counted in hops. Ten MLLMs lose accuracy and diverge as hops grow: under In-Depth input, average accuracy falls to about 33.8% at five hops and 15.3% beyond six,

Load-bearing premise

The entire benchmark rests on the claim that the curation pipeline makes every question uniquely answerable from the supplied images alone — an unproven minimal-set guarantee ($H(V|S)=0$) plus the assumption that rendered charts lose no data — so any lossy or ambiguous chart would make the measured accuracy drops an artifact of the benchmark rather than a property of the models.

Editorial extensions

If this is right

  • Reasoning depth, not context length, is the dominant limiter: accuracy collapses at six or more hops even though models see the same images, so future work should target multi-step computation rather than longer windows.
  • Visualizing intermediate computation nodes consistently helps, so models or agent pipelines that externalize intermediate values (charts, tables, derived numbers) should outperform pure latent reasoning.
  • Cross-chart computation is a universal bottleneck: every evaluated model scores lower on compute questions than on extract or compare, making it the sharpest discriminator of multi-chart reasoning ability.
  • Grounding and pointing cannot yet serve as reliable agentic intermediate tools: forcing bounding boxes or points roughly halves the best model's accuracy, so agent designs should not depend on them.
  • Perturbation robustness is model-specific and sometimes positive, so robustness claims need per-model perturbation profiles rather than a single aggregate number.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the minimal-set guarantee holds, the hop axis enables an error-accumulation test the paper does not run: whether failure at hop $k$ predicts failure at hop $k+1$ would distinguish compounding-error from independent-failure accounts of MLLM reasoning.
  • The same graph-based consistency design could build multi-document or multi-figure reasoning benchmarks (several figures sharing one underlying dataset), where answer ambiguity is the usual failure mode.
  • The positive perturbation effects hint that some models overfit rendering artifacts; a direct probe would compare confidence distributions on original versus perturbed charts to test whether perturbations act as a regularizer.
  • The finite-answer, hop-labelled question bank suits test-time compute scaling studies — whether more inference tokens flatten the hop-accuracy curve — which the paper's cost-performance frontier raises but does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces LongChart, a synthetic multi-chart VQA benchmark consisting of 557 images and 2,876 questions organized into sets averaging 6.5 images and 31.2 questions. The authors propose a latent-graph pipeline with data, computation, and image nodes to guarantee internal consistency, uniqueness, and answer sufficiency. They evaluate 10 MLLMs under two reasoning patterns (Extended Context and In-Depth), grounding/pointing auxiliary outputs, and six types of image perturbation/transformation. The central empirical claim is that MLLM accuracy decreases substantially as the number of computational hops increases, and that this drop is attributable to reasoning depth rather than answer ambiguity.

Significance. If the benchmark validity claim held, LongChart would be a useful controlled testbed for multi-chart, multi-hop reasoning: the use of a shared latent data space and per-set multi-question design goes beyond single-chart benchmarks, and the evaluation covers a broad set of proprietary and open models with several analysis dimensions (reasoning pattern, grounding, robustness, cost-performance). The paper should be credited for clearly identifying a gap in multi-chart evaluation and for attempting a principled synthesis pipeline. However, the central validity guarantee is not established: the formal information-theoretic argument in Appendix C is internally inconsistent, the minimal generative set theorem is asserted rather than proven, and the human verification lacks inter-annotator agreement or an In-Depth human baseline. The robustness results in Table 7 contain large positive accuracy changes that are not explained and that undermine the attribution of the hop-size effect to reasoning complexity. At present, the accuracy-vs-hops trend may be an artifact of ambiguous or lossy benchmark items.

major comments (5)
  1. [§C.2, directed edges] The entropy characterization of directed edges is backwards. The paper states that for u→v (one-way, u not reconstructible from v), 'Equivalently, H(u|v)=0.' But H(u|v)=0 means u is fully determined by v, i.e., u IS reconstructible from v. This is the opposite of the intended one-way information flow. Since §C.3 (equivalence classes), §C.4 (completeness, conflict-freeness), and the claimed minimal-set guarantee all use the same entropy semantics, the formal basis for the 'charts alone are sufficient' guarantee collapses. This is not a typo in a single line; it is load-bearing for the In-Depth evaluation mode, which supplies only the minimal set of images.
  2. [§C.4, §3.1, Algorithm 1] The key theorem that a minimal generative set S with H(V|S)=0 exists for the constructed graphs is never proved. The text asserts that graph traversal 'yields' such a set, and that the algorithm 'automatically compensates' until sufficiency is reached, but no traversal algorithm, termination condition, or proof of the three required properties (completeness, independence, conflict-freeness) is given. Algorithm 1 line 6 merely calls 'Verify images can sufficiently support each question' without specifying the verification procedure. Consequently, the paper does not establish that all 2,876 questions are uniquely answerable from the provided images alone, which is the premise for interpreting hop-count accuracy drops as reasoning-depth effects.
  3. [Table 7, Fig. 6, Appendix B] The robustness results contradict the paper's narrative that accuracy changes can be 'directly attributed to the perturbation' (Appendix B). Under In-Depth evaluation, perturbations cause large positive accuracy changes, e.g., doubao-seed-2.0-lite +24.59% under data noise and +25.41% under text noise, qwen3.5-flash +23.00% under blurring, and gpt-5.4-mini +14.63% under compression. If perturbations are semantically neutral as claimed, such large improvements are implausible and suggest either unstable model behavior, ambiguous items, or a mismatched baseline. The paper does not explain these positive effects, and no error bars or significance tests are provided, so the robustness conclusions and, by extension, the main accuracy-vs-hops trend are not statistically supported.
  4. [§3.2, Appendix D, Table 2] Human validation is presented as the backstop for question uniqueness, but no inter-annotator agreement is reported, and the human baseline in Table 2 is only under Extended Context ('Entended Context'), not under In-Depth. The human baseline therefore does not test whether a human can answer from the minimal image set alone. Without per-item verification or IAA, the possibility remains that some questions have multiple defensible readings or require information not present in the minimal set, confounding the hop-count result.
  5. [Fig. 4, Table 2] The central empirical claim—accuracy decreases sharply with hop count—is presented without confidence intervals, significance tests, or multiple-comparison corrections. The high-hop bins are small (e.g., 41, 39, and 25 questions at hops 4, 5, and 6+ in Figure 4), so the steep declines beyond hop 3 could be within sampling noise. Please report per-bin confidence intervals and, ideally, bootstrap significance tests for the monotonic-decrease claim.
minor comments (5)
  1. [Table 2 caption] The caption refers to 'M2MChartBench question-type accuracy', but the paper introduces LongChart; this appears to be a leftover from another project and should be corrected.
  2. [§3.3] The text says 'Figure 2 summarizes the question and image characteristics' but the dataset statistics appear in Figure 3; please fix the cross-reference.
  3. [Abstract and §1] Several sentences are ungrammatical or incomplete, e.g., 'connections remain insufficient for evaluating MLLMs might be insufficient in measuring MLLM’s capability' and 'radidly improves' in §2.1. A careful language edit is needed.
  4. [Appendix C.6] The GraphSpec tuple (V,E,T,E) uses E twice (edges and operator/flow attributes), which is confusing. Please use distinct symbols.
  5. [Table 1] The 'Ours' row reports 3–18 images and 'Finite Answer', but the table header and other rows use inconsistent formatting; consider clarifying what '#', 'G', and blank cells mean in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation chain: the benchmark's internal validity gaps are correctness concerns, not circular reductions.

full rationale

The paper's central claim is that MLLM accuracy drops as reasoning hops increase, measured on the LongChart benchmark. This is an empirical measurement of model outputs against fixed synthetic labels, not a quantity that is forced by construction. The curation pipeline defines a latent graph, synthesizes data, generates images and questions, and then verifies sufficiency; no parameter is fitted to a subset of the data and then renamed as a prediction. There is no load-bearing self-citation: the related-work citations to MultiChartQA and others are contextual comparisons, not foundations for the paper's unique claims, and no uniqueness theorem from the authors' prior work is invoked. The asserted minimal-generative-set guarantee and the entropy characterization in Appendix C are internal consistency and validity issues: if the directed-edge characterization H(u|v)=0 is inverted, or if the sufficiency guarantee is unproven, then the benchmark may contain ambiguous or unanswerable items, which would confound the accuracy-vs-hops finding with benchmark noise. But that is a flaw in the benchmark's warrant, not a circular step in the derivation. The paper itself acknowledges limitations of dataset size and model coverage, which do not constitute circularity. Accordingly, no step reduces by definition or by self-citation to its own inputs, and the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The claimed guarantees of the benchmark rest on three unverified assumptions: minimal-set sufficiency, human verification reliability, and lossless image rendering. No free parameters are fit, but the numerical tolerance is a hand-set evaluation choice.

free parameters (1)
  • Numerical answer tolerance = 1% relative error
    Authors choose 1% relative error as the exact-match criterion for numerical questions in Appendix F.1; no justification is given, and this threshold directly affects all reported accuracy numbers.
assumptions (3)
  • ad hoc to paper A minimal generative set S with H(V|S)=0 guarantees all questions are answerable from the images alone
    Asserted in Section 3.1 and Appendix C.3; no proof or released implementation is provided, and the information-lossless bidirected edge assumption underpins it.
  • domain assumption Human reviewers can reliably verify that every QA set is readable, uniquely answerable, and accurate
    Section 3.2 reports about 30 minutes of human verification per set and no inter-annotator agreement or error audit; the human baseline itself is 81.78%, not 100%.
  • ad hoc to paper Chart images preserve all information of their data nodes (bidirected edges are reversible)
    Section 3.1 defines bidirected edges as information-equivalent; real chart rendering is lossy (axis scaling, color, labels), so this is a strong modeling assumption.
invented entities (1)
  • Latent dependency graph DAG (data nodes, computation nodes, image nodes)
    purpose: Backbone for consistent multi-chart data synthesis and minimal-set selection
    A methodological construct; its claimed guarantees are asserted and not independently verified, and no code, dataset, or formal proof is shipped.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning." pith.science (2026). https://pith.science/paper/GEGCNF4A

@misc{pith2026260801328,
  author       = {Pith},
  title        = {Pith review of: LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GEGCNF4A}},
  note         = {Machine review of arXiv:2608.01328}
}
read the original abstract

Multimodal large language models (MLLMs) are rapidly evolving with expanded context windows and stronger reasoning capabilities, enabling multi-chart understanding and multi-step inference. These abilities are increasingly important as MLLMs are adopted in complex agentic tasks. However, existing benchmarks largely emphasize single-chart perception, while simple chart-to-chart connections are insufficient to evaluate these capabilities. To capture multi-chart complexity while ensuring consistency and validity, we design a synthesis pipeline supported by latent graphs. Building on this pipeline, we introduce LongChart, a benchmark whose VQA sets contain an average of 6.5 images and 31.2 questions. We evaluate 10 state-of-the-art MLLMs and examine three factors that influence performance: reasoning patterns, auxiliary tools, and robustness to image perturbations. Our results show that MLLM accuracy decreases and varies substantially as computational complexity increases, highlighting directions for future research in multi-chart reasoning.

Figures

Figures reproduced from arXiv: 2608.01328 by the authors.

Figure 1
Figure 1. Motivation of this work 2023; Kil et al., 2024; Huang et al., 2026). In these scenarios, maintaining precision in understanding across multiple images with intricate relationships among them become central challenges. Multi-chart, as a common medium for rich information, remains an underexplored area for MLLM’s performance evaluation. Pioneering works of chart-related benchmarks (Masry et al., 2022; Methani et al., … view at source ↗
Figure 2
Figure 2. Overview of the VQA set curation process and evaluation. The pipeline comprises graph design, data synthesis, and VQA generation with human-in-the-loop quality control. reasoning (Zhu et al., 2025), and the visual in￾sights (Tang et al., 2025). Multi-image benchmarks has become prevalent in other domains. Exist￾ing works in this area includes, in-depth reasoning questions (Fan et al., 2025). Despite the progress, tw… view at source ↗
Figure 3
Figure 3. Dataset characteristics of LongChart VQA. (a) Question distribution across curation categories. (b) A sample question; (c) Image and question count. (d) Distribution of number of images, questions and data points per QA set; (e) Statistics in the VQA. 3.1 Latent Space That Supports VQA Curation Graph-as-backbone. To ensure consistency and meaningfulness across charts, we design the graph as the backbone of the VQA c… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Model performance as a function of reasoning hops. (a) Accuracy degradation across increasing reasoning complexity under Extended Context (top) and In-Depth (bottom).(b) Performance ranks across model under different difficulty level of questions source state-of-the-ar…
Figure 5
Figure 5. Figure 5: Performance with grounding and pointing. (a) mDIoU and Chamfer distance indicate low quality in generated grounding box and point, with significant decrease in question answer accuracy. (b) a case study showing the prevalent patterns of low-quality grounding VQA catego…
Figure 6
Figure 6. Figure 6: Impact of robustness perturbations. The value shows equals the accuracy in the robustness set minus the accuracy for the corresponding original syn￾thesized charts. Trans=Image Transformation. 5.2 Analysis Grounding and pointing are inaccurate. Fig￾ure 5(a) illustrates…
Figure 7
Figure 7. Figure 7: Cost-performance distribution of evaluated models. Two seperate efficient frontier curves of Ex￾tended Context and In-Depth presents, with both token count (top) and runtime (bottom) versus accuracy. 6 Conclusion This paper introduces LONG-CHART BENCH, a new pipeline a…
Figure 8
Figure 8. Figure 8: Performance comparison across figure types on the two evaluation splits. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Comparison of model accuracy before and after reasoning across question types. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Annotation interface developed to annotate [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Examples of VQA sets covering mixed chart types, statistical views, style transfer, and text-noise [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 22 canonical work pages

  1. [1]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  2. [2]

    Proceedings of the IEEE/cvf conference on computer vision and pattern recognition , pages=

    Ok-vqa: A visual question answering benchmark requiring external knowledge , author=. Proceedings of the IEEE/cvf conference on computer vision and pattern recognition , pages=

  3. [3]

    Advances in neural information processing systems , volume=

    Learn to explain: Multimodal reasoning via thought chains for science question answering , author=. Advances in neural information processing systems , volume=

  4. [4]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  5. [5]

    arXiv preprint arXiv:2310.02255 , year=

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts , author=. arXiv preprint arXiv:2310.02255 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2302.00923 , year=

    Multimodal chain-of-thought reasoning in language models , author=. arXiv preprint arXiv:2302.00923 , year=

    arXiv

  7. [7]

    European Conference on Computer Vision , pages=

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  8. [8]

    Advances in Neural Information Processing Systems , volume=

    Measuring multimodal mathematical reasoning with math-vision dataset , author=. Advances in Neural Information Processing Systems , volume=

Show all 52 references
  1. [9]

    arXiv preprint arXiv:2306.13394 , year=

    Mme: A comprehensive evaluation benchmark for multimodal large language models , author=. arXiv preprint arXiv:2306.13394 , year=

    arXiv

  2. [10]

    Advances in Neural Information Processing Systems , volume=

    Mllm-compbench: A comparative reasoning benchmark for multimodal llms , author=. Advances in Neural Information Processing Systems , volume=

  3. [11]

    5: Scaling reinforcement learning with llms , author=

    Kimi k1. 5: Scaling reinforcement learning with llms , author=. arXiv preprint arXiv:2501.12599 , year=

    arXiv

  4. [12]

    Nature , volume=

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , author=. Nature , volume=. 2025 , publisher=

    2025

  5. [13]

    arXiv preprint arXiv:2504.21277 , year=

    Reinforced mllm: A survey on rl-based reasoning in multimodal large language models , author=. arXiv preprint arXiv:2504.21277 , year=

    arXiv

  6. [14]

    arXiv preprint arXiv:2601.22060 , year=

    Vision-deepresearch: Incentivizing deepresearch capability in multimodal large language models , author=. arXiv preprint arXiv:2601.22060 , year=

  7. [15]

    arXiv preprint arXiv:2311.16483 , year=

    Chartllama: A multimodal llm for chart understanding and generation , author=. arXiv preprint arXiv:2311.16483 , year=

    arXiv

  8. [16]

    IEEE Transactions on Visualization and Computer Graphics , volume=

    Advancing multimodal large language models in chart question answering with visualization-referenced instruction tuning , author=. IEEE Transactions on Visualization and Computer Graphics , volume=. 2024 , publisher=

    2024

  9. [17]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Matcha: Enhancing visual language pretraining with math reasoning and chart derendering , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  10. [18]

    arXiv preprint arXiv:2409.03277 , year=

    Chartmoe: Mixture of diversely aligned expert connector for chart understanding , author=. arXiv preprint arXiv:2409.03277 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:2602.16455 , year=

    Visual self-refine: A pixel-guided paradigm for accurate chart parsing , author=. arXiv preprint arXiv:2602.16455 , year=

  12. [20]

    arXiv preprint arXiv:2505.13444 , year=

    Chartmuseum: Testing visual reasoning capabilities of large vision-language models , author=. arXiv preprint arXiv:2505.13444 , year=

  13. [21]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    ChartAssistant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  14. [22]

    arXiv preprint arXiv:2510.17932 , year=

    From charts to code: A hierarchical benchmark for multimodal models , author=. arXiv preprint arXiv:2510.17932 , year=

    arXiv

  15. [23]

    The Fourteenth International Conference on Learning Representations , year=

    DaVinci: Reinforcing Visual-Structural Syntax in MLLMs for Generalized Scientific Diagram Parsing , author=. The Fourteenth International Conference on Learning Representations , year=

  16. [24]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Chartpoint: Guiding mllms with grounding reflection for chart reasoning , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  17. [25]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Chartedit: How far are mllms from automating chart analysis? evaluating mllms’ capability via chart editing , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  18. [26]

    Findings of the association for computational linguistics: ACL 2022 , pages=

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning , author=. Findings of the association for computational linguistics: ACL 2022 , pages=

    2022

  19. [27]

    Proceedings of the ieee/cvf winter conference on applications of computer vision , pages=

    Plotqa: Reasoning over scientific plots , author=. Proceedings of the ieee/cvf winter conference on applications of computer vision , pages=

  20. [28]

    arXiv preprint arXiv:2312.15915 , year=

    Chartbench: A benchmark for complex visual reasoning in charts , author=. arXiv preprint arXiv:2312.15915 , year=

    arXiv

  21. [29]

    Advances in Neural Information Processing Systems , volume=

    Charxiv: Charting gaps in realistic chart understanding in multimodal llms , author=. Advances in Neural Information Processing Systems , volume=

  22. [30]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Chartinsights: Evaluating multimodal large language models for low-level chart question answering , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  23. [31]

    arXiv preprint arXiv:2410.22492 , year=

    RealCQA-V2: Visual Premise Proving A Manual COT Dataset for Charts , author=. arXiv preprint arXiv:2410.22492 , year=

  24. [32]

    IEEE Transactions on Image Processing , year=

    Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning , author=. IEEE Transactions on Image Processing , year=

  25. [33]

    MultiChartQA: Benchmarking vision-language models on multi-chart problems , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  26. [34]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Chartmind: A comprehensive benchmark for complex real-world multimodal chart question answering , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  27. [35]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Wikimixqa: a multimodal benchmark for question answering over tables and charts , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  28. [36]

    arXiv preprint arXiv:2507.11939 , year=

    POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering , author=. arXiv preprint arXiv:2507.11939 , year=

  29. [37]

    arXiv preprint arXiv:2510.26781 , year=

    ChartAB: A Benchmark for Chart Grounding & Dense Alignment , author=. arXiv preprint arXiv:2510.26781 , year=

  30. [38]

    arXiv preprint arXiv:2507.14298 , year=

    In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding , author=. arXiv preprint arXiv:2507.14298 , year=

    arXiv

  31. [39]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Unmasking deceptive visuals: Benchmarking multimodal large language models on misleading chart question answering , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  32. [40]

    arXiv preprint arXiv:2412.12150 , year=

    Rethinking comprehensive benchmark for chart understanding: A perspective from scientific literature , author=. arXiv preprint arXiv:2412.12150 , year=

    arXiv

  33. [41]

    2026 , note =

    Seed2.0 , howpublished =. 2026 , note =

    2026

  34. [42]

    2026 , note =

    Google DeepMind , title =. 2026 , note =

    2026

  35. [43]

    INTERCHART: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information , author=. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computationa...

  36. [44]

    2026 , note =

    Qwen3.5: Towards Native Multimodal Agents , howpublished =. 2026 , note =

    2026

  37. [45]

    2026 , note =

    OpenAI , title =. 2026 , note =

    2026

  38. [46]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  39. [47]

    Publications Manual , year = "1983", publisher =

    1983

  40. [48]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    1981

  41. [49]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  42. [50]

    Dan Gusfield , title =. 1997

    1997

  43. [51]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  44. [52]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.