Pith. sign in

REVIEW 3 major objections 7 minor 46 references

Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A lightweight draft-and-repair loop using a cheap off-the-shelf LLM cuts text-to-chart execution errors to 4.5% on Text2Chart31 and 4.6% on ChartX, beating fine-tuned baselines by nearly 5 points.

desk verdict A cheap draft-and-repair loop genuinely cuts execution errors in text-to-chart, but the headline comparison to fine-tuned baselines is softer than it looks. read the letter →

arxiv 2506.06175 v1 pith:ZQ53ZETL submitted 2025-06-06 cs.CL

James Ford , Anthony Rios This is my paper
classification cs.CL
keywords text-to-chartgenerationmulti-agentpipelinecoderepairexecutionerrorrateGPT-4o-miniText2Chart31ChartXaccessibility
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the residual execution failures in text-to-chart generation are a model limitation or a limitation of single-prompt design. It answers: largely the latter. A lightweight loop that drafts Python/Matplotlib code, runs it, and feeds tracebacks to a rewriter for up to three rounds, using a cheap off-the-shelf LLM, cut execution errors on Text2Chart31 from about 14.8% to 4.5% in the few-shot setting, and reached 4.6% on ChartX. These numbers beat published fine-tuned baselines by nearly 5 percentage points while using far less compute, and visual-quality scores stayed roughly unchanged. A manual review of 100 sampled charts found 83% accurate, but only 33.3% of Text2Chart31 charts and 7.2% of ChartX charts passed a colorblindness check, so the paper argues the field's next target should be aesthetics, semantic fidelity, and accessibility.

What carries the argument

The load-bearing mechanism is a two-agent repair loop. A drafting agent converts the natural-language description and CSV data into a complete Python/Matplotlib script; the script is executed; if it fails, the traceback plus source code is passed to a rewriter agent that edits only the flagged lines and returns complete code; the loop repeats up to three times. The separation of drafting from rewriting, and the constraint that the rewriter changes only what the error suggests, is what converts a roughly 15% failure rate into about 4.5%, at a total reported experiment cost near $150.

What would settle it

Run the exact pipeline and the published baselines through the same evaluation harness on the Text2Chart31 test split: if the few-shot agentic error rate does not stay near 4.5% while the SFT+RL baseline stays near 9.2%, the claimed margin is an evaluation artifact. A cheaper check is to re-run the non-agentic GPT-4o-mini baseline under the same environment, which should reproduce about 14.1% error before repair if the comparison is clean.

Watch

Extended reading notes

Core claim

The central claim is that a simple draft-execute-repair-judge pipeline, with no fine-tuning, solves execution reliability on current text-to-chart benchmarks: 4.5% error on Text2Chart31 (few-shot agentic) and 4.6% on ChartX, versus about 9.2% for the strongest fine-tuned baseline on Text2Chart31 and about 9.4% for the few-shot non-agentic baseline on ChartX. The paper also claims this reliability gain does not degrade visual quality: SSIM and multimodal-LLM perceptual scores are nearly identical between the baseline and agentic outputs. It concludes that execution success is largely solved by current benchmark standards, and that the remaining deficits—6 hallucinations in 100 sampled charts, low colorblindness compliance, and stylistic mismatches—define the next problem.

Load-bearing premise

The headline comparison assumes the published fine-tuned baseline numbers were measured under the same evaluation conditions, including data splits, interpreter versions, and metric definitions, as the new pipeline; the paper itself notes that comparisons to published results might suffer from inconsistent alignment with their methodologies.

Editorial extensions

If this is right

  • If the claim holds, execution error rate no longer discriminates among strong systems, so current benchmarks should be retired or extended with harder scenarios such as noisy data or multi-step analytic sessions.
  • Fine-tuning for execution may be unnecessary for this task: a cheap off-the-shelf model plus a repair loop can match or exceed trained baselines.
  • Evaluation should shift toward semantic fidelity, style, and accessibility, with code-similarity metrics like METEOR and CodeBLEU treated as weak proxies.
  • Accessibility becomes a measurable target: only 7–33% of generated charts pass basic colorblindness criteria, so future systems should be judged partly on that.
  • Agentic repair should be tested on more complex charting scenarios, including multi-step analytic tasks, to see whether the near-solved execution status generalizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but the same runtime-traceback repair loop could be applied to other code-generation domains where executability is the bottleneck, such as data-cleaning scripts or API-call generation.
  • A cheap next experiment the paper leaves implicit is adding a colorblindness-aware instruction to the rewriter's prompt—for example requiring redundant encodings or lightness contrast—and measuring whether the accessibility pass rate rises without hurting execution reliability.
  • The authors' own caveat that comparisons to published results might suffer from inconsistent alignment means the 5-point margin is the least certain part of the paper; re-running published baselines in the same harness would settle it.
  • The 83% manual-accuracy figure comes from only 100 sampled charts; a larger human or automated semantic-fidelity study would clarify how much of the remaining error is truly harmless stylistic variation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes a lightweight multi-agent pipeline for text-to-chart generation using an off-the-shelf GPT-4o-mini model. A drafting agent generates Matplotlib code, the code is executed, and a rewriter agent repairs failing scripts using the runtime traceback for up to three iterations. On Text2Chart31 the system reduces execution errors from roughly 14-15% to 4.5%, and on ChartX to 4.6%; the authors compare these numbers with published fine-tuned baselines and additionally report code-similarity scores, SSIM and LLM-as-judge image-quality scores, a colorblindness audit, an iteration ablation, and a manual review of 100 sampled outputs. They conclude that execution success on current benchmarks is largely solved and that future work should focus on semantic fidelity, aesthetics, and accessibility.

Significance. If the execution-error reduction is verified in a controlled comparison, the paper offers valuable evidence that a simple draft-repair loop with a cheap proprietary LLM can match or beat expensive fine-tuning on execution reliability, at a fraction of the cost. The execution metric itself is objective and directly measured, and the paper is transparent about total API cost (~$150), includes an iteration ablation, and acknowledges limitations such as synthetic datasets, use of a proprietary model, and a simple two-agent design. The manual review and the accessibility analysis are useful secondary contributions, although they are less rigorously controlled than the execution experiments. The central claim, however, depends on an apples-to-apples comparison with published baselines, and that comparison is not fully established in the current manuscript.

major comments (3)
  1. [Section 5, Table 2] The headline comparison to published fine-tuned baselines is not controlled for evaluation environment. The text itself notes that 'comparisons to the published results might suffer from inconsistent alignment with their methodologies,' and Table 5 shows environment-sensitive failures (matplotlib's 'use_line_collection' argument, missing modules 'mplfinance' and 'squarify'). Because the baseline numbers come from the original Text2Chart31 paper and this paper uses its own Python environment and metric pipeline, the claimed 4.71 percentage-point advantage over SFT+RLpref CLI-13B could be an artifact of environment mismatch rather than evidence that agentic repair beats fine-tuning. Please rerun at least one strong baseline (e.g., GPT-4o, Claude3Opus, or the released SFT+RLpref checkpoints if available) in the same environment and with the same metric script, and report the resulting error rates. If the fine-tuned checkpoints are not accessible, the abstract should be reframed to present the comparison as indicative rather than a direct benchmark result.
  2. [Section 5, Experimental Details; Abstract] The claim that the approach 'requires significantly less compute' is not substantiated. The only evidence given is the total API cost of ~$150, with no comparison of training or inference compute for the fine-tuned baselines (e.g., GPU-hours, parameter updates, or API tokens). Since this is a stated advantage in the abstract, please provide a quantified compute comparison or remove the claim.
  3. [Section 5, Image Quality Analysis; Table 4] The baseline and agentic image-quality averages are computed over different sets of successfully generated images. The paper notes that the agentic calculations have more data points than baseline calculations, meaning the baseline SSIM and MM-LLM scores reflect only the subset of cases that ran without repair, while the agentic scores include many additional repaired cases. This confounds the 'no drop in visual quality' comparison. Please report SSIM and MM-LLM scores on the same subset of test cases (e.g., cases where both baseline and agentic succeed) or use a designated handling for failures so that the comparison is not affected by selection bias.
minor comments (7)
  1. [Section 5, ChartX Results; Table 3] The sentence stating that 'the final zero-shot agentic results were almost identical between both datasets at 4.5 and 4.6 percent' is inconsistent with the tables: Text2Chart31 zero-shot agentic is 6.75% and ChartX zero-shot agentic is 3.13%, whereas the few-shot agentic results are 4.50% and 4.60%. Please correct the wording to refer to the few-shot agentic results.
  2. [Appendix, Color Blindness Analysis] The colorblindness criteria are grounded in an uncited source ('recommendations from ?'); the missing reference should be supplied. In addition, the prompt text contains garbled tokens ('Äppropriateör') and conflicting output instructions ('Just return Appropriate or Not appropriate, do NOT return anything else' followed by 'Return the result as a JSON object').
  3. [Section 5, Iteration Ablation; Figures 3 and 4] The per-iteration counts in Figure 3 appear to sum to more than the number of baseline execution failures implied by Table 2, so it is unclear whether the bars count successful repairs, re-executions, or scripts that failed at each round. Please clarify the definition and ensure the figure and text are consistent.
  4. [Section 5, Implications] The phrase 'structured prompting alone' understates the method, since the pipeline includes execution feedback and iterative repair; consider rephrasing to 'structured prompting with execution feedback and self-correction.'
  5. [Section 5, Error Analysis] The manual review of 100 sampled charts should specify how the sample was drawn, who the annotator(s) were, whether inter-annotator agreement was measured, and how 'hallucination' relates to the 'Error Data/Other' category. This is relevant because the 83% success figure is used to support the claim that execution is largely solved.
  6. [Section 3, Data; Abstract] The ChartX results are compared only to the authors' own baselines, so the abstract's phrase 'demonstrating strong generalization' is stronger than what the evidence supports; please soften it unless an external point of comparison is added.
  7. [Reproducibility] The manuscript does not state whether code or prompts will be released, nor the exact GPT-4o-mini API snapshot/date used; given the proprietary model and the sensitivity of the repair loop to API versions, please provide a reproducibility statement with model version, prompt versions, and package versions.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the execution metric is objective and external, and the repair loop is not fitted to the benchmark; minor self-evaluation and baseline-alignment caveats do not make the derivation circular.

full rationale

The central claim—that a draft-execute-repair loop using GPT-4o-mini lowers Text2Chart31 execution errors to 4.5% versus 9.21% for the SFT+RLpref CLI-13B baseline—is not circular. Execution success is an external, objective property of generated Python code: a script either runs or raises an exception, and the pipeline does not fit a parameter to the benchmark labels. The comparison to published fine-tuned baselines depends on evaluation alignment, and the authors themselves caution that "comparisons to the published results might suffer from inconsistent alignment with their methodologies." That is a validity risk, not a circularity, because the 4.5% figure does not reduce by construction to the baseline numbers. The secondary image-quality and colorblindness evaluations use GPT-4o-mini as the multimodal judge, the same model that generates the charts; this is a self-evaluation burden, though it is partly mitigated by the manual check of 50 color-blindness annotations (46 correctly labeled) and by the fact that the headline execution result is independent of these judgments. The color-blindness section says the prompt "drew on practical recommendations from ?"—a missing-reference placeholder that weakens provenance but does not make the assessment definitional. The only self-citation (Ford et al. 2025) appears in related work as motivation for why execution failures matter; it is not load-bearing for any derived result. No equation in the paper defines the output in terms of the input, and no fitted parameter is renamed as a prediction. The main execution claim is self-contained against an external benchmark, so the appropriate finding is no significant circularity, with a low score reflecting only minor self-evaluation and alignment caveats.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the two benchmarks, the execution environment, the comparability of prior baselines, and to a lesser extent the reliability of LLM-based evaluation. No fitted numerical parameters are required, but the number of repair iterations and few-shot examples are hand-set design choices.

free parameters (2)
  • Maximum repair iterations = 3
    The pipeline stops after three repair attempts; the choice is an author decision, not optimized on a validation set. The paper reports diminishing returns after the first pass.
  • In-context examples for few-shot = 2 examples
    Two curated examples from the training set are used for the few-shot prompts; their selection is a design choice.
assumptions (4)
  • domain assumption The Text2Chart31 and ChartX benchmarks are valid and representative proxies for text-to-chart generation.
    The paper's generalization claims rest on these synthetic datasets; the authors acknowledge in Limitations that they are synthetic and may not reflect real-world complexity.
  • domain assumption The Python execution environment and the interpreter errors accurately reflect the failure modes of chart code.
    The repair loop is driven by runtime exceptions; undetected semantic errors or environment-specific issues are not captured.
  • domain assumption Prior published baseline numbers were produced under comparable evaluation conditions.
    The paper itself notes 'comparisons to the published results might suffer from inconsistent alignment with their methodologies', so the improvement claim depends on this alignment.
  • domain assumption LLM-based judgment (GPT-4o-mini) is a reliable evaluator of image quality and colourblindness compliance.
    The image-quality and accessibility analyses use the same model that generates the charts, validated only on 46/50 charts for colourblindness labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach." pith.science (2026). https://pith.science/paper/ZQ53ZETL

@misc{pith2026250606175,
  author       = {Pith},
  title        = {Pith review of: Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZQ53ZETL}},
  note         = {Machine review of arXiv:2506.06175}
}
read the original abstract

Large language models can translate natural-language chart descriptions into runnable code, yet approximately 15\% of the generated scripts still fail to execute, even after supervised fine-tuning and reinforcement learning. We investigate whether this persistent error rate stems from model limitations or from reliance on a single-prompt design. To explore this, we propose a lightweight multi-agent pipeline that separates drafting, execution, repair, and judgment, using only an off-the-shelf GPT-4o-mini model. On the \textsc{Text2Chart31} benchmark, our system reduces execution errors to 4.5\% within three repair iterations, outperforming the strongest fine-tuned baseline by nearly 5 percentage points while requiring significantly less compute. Similar performance is observed on the \textsc{ChartX} benchmark, with an error rate of 4.6\%, demonstrating strong generalization. Under current benchmarks, execution success appears largely solved. However, manual review reveals that 6 out of 100 sampled charts contain hallucinations, and an LLM-based accessibility audit shows that only 33.3\% (\textsc{Text2Chart31}) and 7.2\% (\textsc{ChartX}) of generated charts satisfy basic colorblindness guidelines. These findings suggest that future work should shift focus from execution reliability toward improving chart aesthetics, semantic fidelity, and accessibility.

Figures

Figures reproduced from arXiv: 2506.06175 by the authors.

Figure 1
Figure 1. Multi-agent text-to-chart pipeline. A natural-language description is first passed to the Drafting Agent, which synthesises runnable Python/Matplotlib code. The script is executed; if a runtime exception occurs, the full traceback and source are forwarded to the Re-writer Agent, which edits the code and resubmits it. This repair loop repeats (up to three iterations in our experiments) until the code runs successfull… view at source ↗
Figure 2
Figure 2. Iteration Human Study Review (n=100) weakest result being 13.2 percent error for 3D and Volumetric charts. Pairwise plots had an error rate of only 1.5 percent. However, the metrics involving code similarity did not show similar improvements. While com￾parisons to the published results might suffer from inconsistent alignment with their methodologies, the fact that our agentic scores are no better than the baseline … view at source ↗
Figure 4
Figure 4. Iteration Analysis for the ChartX dataset [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Examples of visual similarity and variation in generated charts compared to ground truth. (a) and (b) show [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Example of the agent generating a successful [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 8
Figure 8. Figure 8: Example of the agent generating the wrong [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Color Blindness assessment correct [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Color Blindness assessment error [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 18 canonical work pages

  1. [1]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909/ METEOR : An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan. Association for Compu...

    2005

  2. [2]

    Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. 2024. https://arxiv.org/abs/2403.17134 Repairagent: An autonomous, llm-based agent for program repair . Preprint, arXiv:2403.17134

    arXiv 2024

  3. [3]

    Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. 2024. https://doi.org/10.1109/TVCG.2024.3456320 Viseval: A benchmark for data visualization in the era of large language models . IEEE Transactions on Visualization and Computer Graphics, pages 1--11

    arXiv 2024

  4. [4]

    Xiaoxue Cheng, Junyi Li, Xin Zhao, Hongzhi Zhang, Fuzheng Zhang, Di Zhang, Kun Gai, and Ji-Rong Wen. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.809 Small agent can also rock! empowering small language models as hallucination detector . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14600--14615, Mia...

    doi:10.18653/v1/2024.emnlp-main.809 2024

  5. [5]

    https://github.com/crewaiinc/crewai

    CrewAI. https://github.com/crewaiinc/crewai

    work page

  6. [6]

    David de Fitero-Dominguez, Eva Garcia-Lopez, Antonio Garcia-Cabot, and Jose-Javier Martinez-Herraiz. 2024. https://doi.org/10.1016/j.engappai.2024.109291 Enhanced automated code vulnerability repair using large language models . Engineering Applications of Artificial Intelligence, 138:109291

    work page arXiv 2024

  7. [7]

    Zhiyu Fan, Xiang Gao, Martin Mirchev, Abhik Roychoudhury, and Shin Hwei Tan. 2023. https://doi.org/10.1109/ICSE48619.2023.00128 Automated repair of programs from large language models . In Proceedings of the 45th International Conference on Software Engineering, ICSE '23, page 1469–1481. IEEE Press

    arXiv 2023

  8. [8]

    James Ford, Xingmeng Zhao, Dan Schumacher, and Anthony Rios. 2025. https://aclanthology.org/2025.coling-main.501/ Charting the future: Using chart question-answering for scalable evaluation of LLM -driven data visualizations . In Proceedings of the 31st International Conference on Computational Linguistics, pages 7497--7510, Abu Dhabi, UAE. Association fo...

    work page 2025

Show all 46 references
  1. [9]

    Joshua Gorniak, Yoon Kim, Donglai Wei, and Nam Wook Kim. 2024. https://doi.org/10.1145/3654777.3676414 Vizability: Enhancing chart accessibility with llm-based conversational interaction . In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technolog...

    2024

  2. [10]

    Kanika Goswami, Puneet Mathur, Ryan Rossi, and Franck Dernoncourt. 2025. Plotedit: Natural language-driven accessible chart editing in pdfs via multimodal llm agents. In Advances in Information Retrieval, pages 130--134, Cham. Springer Nature Switzerland

    2025

  3. [11]

    Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. https://arxiv.org/abs/2311.16483 Chartllama: A multimodal llm for chart understanding and generation . Preprint, arXiv:2311.16483

    2023 arXiv

  4. [12]

    Linmei Hu, Duokang Wang, Yiming Pan, Jifan Yu, Yingxia Shao, Chong Feng, and Liqiang Nie. 2024. https://doi.org/10.1145/3664647.3680790 Novachart: A large-scale dataset towards chart understanding and generation of multimodal large language models . In Proceedings of the 32nd ...

    2024

  5. [13]

    Yuyang Ji and Haohan Wang. 2025. https://arxiv.org/abs/2504.09764 Socratic chart: Cooperating multiple agents for robust svg chart understanding . Preprint, arXiv:2504.09764

    2025 arXiv

  6. [14]

    Wonjoong Kim, Sangwu Park, Yeonjun In, Seokwon Han, and Chanyoung Park. 2025. https://arxiv.org/abs/2405.00021 Simplot: Enhancing chart question answering by distilling essentials . Preprint, arXiv:2405.00021

    2025 arXiv

  7. [15]

    Woosung Koh, Jang Han Yoon, MinHyung Lee, Youngjin Song, Jaegwan Cho, Jaehyun Kang, Taehyeon Kim, Se-Young Yun, Youngjae Yu, and Bongshin Lee. 2025. https://arxiv.org/abs/2410.18652 c^2 : Scalable auto-feedback for llm-based chart generation . Preprint, arXiv:2410.18652

    2025 arXiv

  8. [16]

    https://python.langchain.com/docs/introduction/

    Langchain. https://python.langchain.com/docs/introduction/

  9. [17]

    Guozheng Li, Xinyu Wang, Gerile Aodeng, Shunyuan Zheng, Yu Zhang, Chuangxin Ou, Song Wang, and Chi Harold Liu. 2024 a . https://arxiv.org/abs/2401.11255 Visualization generation with large language models: An evaluation . Preprint, arXiv:2401.11255

    2024

  10. [18]

    Xinhang Li, Jingbo Zhou, Wei Chen, Derong Xu, Tong Xu, and Enhong Chen. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.716 Visualization recommendation with prompt-based reprogramming of large language models . In Proceedings of the 62nd Annual Meeting of the Association f...

    2024 doi

  11. [19]

    En Low, Carmen Cheh, and Binbin Chen. 2024. https://doi.org/10.1109/SecDev61143.2024.00008 Repairing infrastructure-as-code using large language models . In 2024 IEEE Secure Development Conference (SecDev), pages 20--27

    2024

  12. [20]

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. https://doi.org/10.18653/v1/2022.findings-acl.177 C hart QA : A benchmark for question answering about charts with visual and logical reasoning . In Findings of the Association for Computational Ling...

    2022 doi

  13. [21]

    Ahmed Masry, Megh Thakkar, Aayush Bajaj, Aaryaman Kartha, Enamul Hoque, and Shafiq Joty. 2024. https://arxiv.org/abs/2407.04172 Chartgemma: Visual instruction-tuning for chart reasoning in the wild . Preprint, arXiv:2407.04172

    2024 arXiv

  14. [22]

    Khapra, and Pratyush Kumar

    Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    2020

  15. [23]

    Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta, and Dan Roth. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.973 Unraveling the truth: Do VLM s really understand charts? a deep dive into consistency and robustness . In Findings of the Asso...

    2024 doi

  16. [24]

    Fatemeh Pesaran Zadeh, Juyeon Kim, Jin-Hwa Kim, and Gunhee Kim. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.640 T ext2 C hart31: Instruction tuning for chart generation with automatic feedback . In Proceedings of the 2024 Conference on Empirical Methods in Natural Langua...

    2024 doi

  17. [25]

    Luca Podo, Marco Angelini, and Paola Velardi. 2024 a . https://arxiv.org/abs/2406.15259 V-recs, a low-cost llm4vis recommender with explanations, captioning and suggestions . Preprint, arXiv:2406.15259

    2024 arXiv

  18. [26]

    Luca Podo, Muhammad Ishmal, and Marco Angelini. 2024 b . https://arxiv.org/abs/2402.02167 Vi(e)va llm! a conceptual stack for evaluating and interpreting generative ai-based visualizations . Preprint, arXiv:2402.02167

    2024 arXiv

  19. [27]

    Meghana Puvvadi, Sai Kumar Arava, Adarsh Santoria, Sesha Sai Prasanna Chennupati, and Harsha Vardhan Puvvadi. 2025. https://doi.org/10.1109/CSNT64827.2025.10968728 Coding agents: A comprehensive survey of automated bug fixing systems and benchmarks . In 2025 IEEE 14th Internat...

    2025

  20. [28]

    Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020. https://arxiv.org/abs/2009.10297 Codebleu: a method for automatic evaluation of code synthesis . Preprint, arXiv:2009.10297

    2020 arXiv

  21. [29]

    Kamath, Aziz Zeidieh, Saairam Venkatesh, and Sean McCurry

    JooYoung Seo, Sanchita S. Kamath, Aziz Zeidieh, Saairam Venkatesh, and Sean McCurry. 2024. https://doi.org/10.1145/3663548.3675660 Maidr meets ai: Exploring multimodal llm-based data visualization interpretation by and with blind and low-vision users . In Proceedings of the 26...

    2024

  22. [30]

    Chuyi Shang, Amos You, Sanjay Subramanian, Trevor Darrell, and Roei Herzig. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.544 T rave LER : A modular multi- LMM agent framework for video question-answering . In Proceedings of the 2024 Conference on Empirical Methods in Natu...

    2024 doi

  23. [31]

    Leixian Shen, Haotian Li, Yun Wang, and Huamin Qu. 2024 a . https://doi.org/10.1109/GEN4DS63889.2024.00008 From data to story: Towards automatic animated data video creation with llm-based multi-agent systems . In 2024 IEEE VIS Workshop on Data Storytelling in an Era of Genera...

    2024

  24. [32]

    Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024 b . https://doi.org/10.18653/v1/2024.emnlp-main.929 Small LLM s are weak tool learners: A multi- LLM agent . In Proceedings of the 2024 Conference on Empirical Methods...

    2024 doi

  25. [33]

    Yuan Tian, Weiwei Cui, Dazhen Deng, Xinjing Yi, Yurun Yang, Haidong Zhang, and Yingcai Wu. 2025. https://doi.org/10.1109/TVCG.2024.3368621 Chartgpt: Leveraging llms to generate charts from abstract natural language . IEEE Transactions on Visualization and Computer Graphics, 31...

    2025

  26. [34]

    Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. 2024 a . https://doi.org/10.1109/TSE.2024.3368208 Software testing with large language models: Survey, landscape, and vision . IEEE Transactions on Software Engineering, 50(4):911--936

    2024

  27. [35]

    Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, and Danqi Chen. 2024 b . https://arxiv.org/abs/2406.18521 Charxiv: Charting gaps in realistic chart understanding in mu...

    2024 arXiv

  28. [36]

    Tianhe Wu, Kede Ma, Jie Liang, Yujiu Yang, and Lei Zhang. 2025. A comprehensive study of multimodal large language models for image quality assessment. In Computer Vision -- ECCV 2024, pages 143--160, Cham. Springer Nature Switzerland

    2025

  29. [37]

    Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Peng Ye, Min Dou, Botian Shi, Junchi Yan, and Yu Qiao. 2025. https://arxiv.org/abs/2402.12185 Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning . Pre...

    2025 arXiv

  30. [38]

    Pengyu Yan, Mahesh Bhosale, Jay Lal, Bikhyat Adhikari, and David Doermann. 2024. Chartreformer: Natural language-driven chart image editing. In Document Analysis and Recognition - ICDAR 2024, pages 453--469, Cham. Springer Nature Switzerland

    2024

  31. [39]

    Yilin Ye, Jianing Hao, Yihan Hou, Zhan Wang, Shishi Xiao, Yuyu Luo, and Wei Zeng. 2024. https://doi.org/10.1016/j.visinf.2024.04.003 Generative ai for visualization: State of the art and future directions . Visual Informatics, 8(2):43--66

    2024 doi

  32. [40]

    Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. 2025. Depicting beyond scores: Advancing image quality assessment through multi-modal language models. In Computer Vision -- ECCV 2024, pages 259--276, Cham. Springer Nature Switzerland

    2025

  33. [41]

    Lu Zhang, Tiancheng Zhao, Heting Ying, Yibo Ma, and Kyusong Lee. 2024 a . https://doi.org/10.18653/v1/2024.emnlp-main.559 O m A gent: A multi-modal agent framework for complex video understanding with task divide-and-conquer . In Proceedings of the 2024 Conference on Empirical...

    2024 doi

  34. [42]

    Zhehao Zhang, Weicheng Ma, and Soroush Vosoughi. 2024 b . https://doi.org/10.18653/v1/2024.findings-emnlp.485 Is GPT -4 V (ision) all you need for automating academic data visualization? exploring vision-language models' capability in reproducing academic charts . In Findings ...

    2024 doi

  35. [43]

    Yuheng Zhao, Junjie Wang, Linbin Xiang, Xiaowen Zhang, Zifei Guo, Cagatay Turkay, Yu Zhang, and Siming Chen. 2024. https://doi.org/10.1109/TVCG.2024.3496112 Lightva: Lightweight visual analytics with llm agent-based task planning and execution . IEEE Transactions on Visualizat...

    2024

  36. [44]

    Chang Zong, Yuchen Yan, Weiming Lu, Jian Shao, Yongfeng Huang, Heng Chang, and Yueting Zhuang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.101 Triad: A framework leveraging a multi-role LLM -based agent to solve knowledge base question answering . In Proceedings of the 2...

    2024 doi

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.