Pith. sign in

REVIEW 4 major objections 9 minor 67 references

Even advanced vision-language models systematically change their answers when charts are redrawn with misleading designs—truncated or inverted axes, distorted ratios, deceptive colors—while the data stays identical.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:27 UTC pith:6ZGYYRNF

load-bearing objection A useful paired benchmark and a sensible metric; the vulnerability result holds up broadly, but the controlled-pair premise needs verification for inverted-axis charts and the mitigation claims are under-controlled. the 4 major comments →

arxiv 2607.22600 v1 pith:6ZGYYRNF submitted 2026-06-12 cs.AI

Chart Deception in Vision-Language Models: From Vulnerability to Mitigation

Ridwan Mahbub , Mohammed Saidul Islam , Md Tahmid Rahman Laskar , Mizanur Rahman , Mir Tafseer Nayeem , Enamul Hoque This is my paper
classification cs.AI
keywords VisDeceptionchart deceptionvision-language modelsmisleading visualizationsDeception Scorepaired benchmarkmulti-agent reasoningchart question answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to prove that misleading chart designs—truncated or inverted axes, distorted aspect ratios, disproportionate encodings, deceptive color scales—change what vision-language models conclude from a chart even when the underlying data values are untouched. The authors built VisDeception, a controlled benchmark of 1,600 chart pairs in which every misleading chart shares its data with a faithful twin, and the Deception Score, a paired metric that subtracts a model's error on the faithful chart from its error on the misleading chart to isolate the deception's causal effect. On 32,000 responses from 10 current models, they find that even the strongest systems shift toward wrong answers, with inverted axes the most consistently damaging manipulation. They then show that an inference-time pipeline—a Data Extraction Agent that rewrites the chart as structured metadata plus a Solver Agent that answers from both image and metadata—cuts deception-induced error for most models, and by 76% for one leading model. The larger point: chart-understanding accuracy and robustness to deception are different properties, and both deserve measurement.

Core claim

On the paper's own terms, the central discovery is that deceptive visual encodings causally shift VLM reasoning: when a chart's only change is a misleading design choice, model answers move systematically away from the data's correct interpretation, and this effect can be separated from ordinary chart-reading error and then reduced. The evidence is a paired corpus of 800 faithful/misleading chart pairs built from real-world data and eight deception categories, a gold answer computed from the data table rather than the image, and a Deception Score defined as the mean over pairs of (error on misleading chart minus error on faithful chart). The mitigation result is that explicit structured extr

What carries the argument

VisDeception, the paired benchmark, is the instrument: each of 800 underlying datasets yields a faithful chart and a misleading chart that differ by one designed deception, so response shifts are attributable to that single intervention. The Deception Score is the metric: it measures the average increase in ordinal error (on a 7-point Likert scale) from the faithful to the misleading chart, isolating deception-induced failure from baseline errors. The mitigation machinery is a two-agent pipeline: a Data Extraction Agent converts the chart image into structured JSON metadata describing marks, scales, axis directions, data points, and optional visual warnings, and a Solver Agent answers the qu

Load-bearing premise

The load-bearing premise is that each misleading chart in the corpus differs from its faithful twin by exactly one intentional deceptive intervention, so any response shift can be blamed on that intervention—an assumption the paper only partially verifies, since some variants (middle truncation, certain inverted axes) required manual post-processing and no pixel-level diff is supplied.

What would settle it

Render the full corpus with fully automated, seed-controlled chart generation so each misleading chart differs from the faithful twin by exactly the intended axis or scale change, then re-measure Deception Scores across the same models; if the scores largely disappear, the reported vulnerability is an artifact of unintended rendering differences instead of the deceptive designs themselves.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, any VLM deployed for chart-based analysis should be evaluated on paired faithful/misleading sets, not just answer accuracy on single charts.
  • Deception Scores give a per-category diagnostic: inverted axes and inappropriate encodings are the weakest points, so targeted training or prompt interventions can prioritize those.
  • Inference-time structured extraction is a practical mitigation that works without telling the model the chart may be deceptive, so it transfers to neutral user queries.
  • The Deception Score's separation of baseline error from deception error means benchmark numbers can be compared across models without confusing general chart skill with deception robustness.
  • Models strong at chart understanding are not automatically strong against deception, so scaling alone is insufficient.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same paired methodology could be extended to other multimodal tasks—diagrams, maps, medical images—where data-preserving visual manipulations can alter interpretation.
  • Beyond the paper: the two failure modes the authors identify (extraction errors and the solver ignoring metadata) suggest future work could focus on self-consistent verification loops between agents rather than one-shot extraction.
  • Beyond the paper: the mitigation's explicit intermediate metadata positions the framework as a human-audit layer; a user could inspect the extracted JSON to see whether a chart's axis is inverted before trusting an answer, a possibility the paper gestures at but does not develop.
  • Beyond the paper: the benchmark's reliance on programmatically generated charts means real-world deceptive charts with mixed cues (captions, annotations) remain untested; a hybrid benchmark adding contextual deception would test whether the current findings hold.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 9 minor

Summary. The paper introduces VisDeception, a benchmark of 800 faithful/misleading chart pairs (1,600 chart images) spanning eight categories of deceptive visualization design, and a paired Deception Score metric that anchors both faithful- and misleading-chart responses to a common gold answer. Ten VLMs are evaluated in single-agent and multi-agent settings, where the multi-agent system first extracts structured Vega-Lite-style metadata and then answers with that metadata as additional context. The paper reports that misleading designs produce statistically significant response shifts, that Inverted Axis is the most consistently deceptive category, and that the proposed mitigation reduces Deception Scores for most models, with the largest gains on stronger VLMs. The claims are accompanied by public benchmark, prompts, and code.

Significance. If the controlled-pair premise is upheld, this is a valuable contribution: VisDeception addresses a real gap by isolating deception-induced reasoning errors from baseline chart-understanding errors, and the Deception Score is a sensible paired metric. The systematic evaluation across 10 models and the inference-time mitigation framework are useful steps toward robust chart understanding, and the release of benchmark and code will benefit the community. The internal consistency of the GPT-4o category-level Deception Scores with the aggregate (Tables II and III) is a positive sign. However, the strength of the central causal claim depends on demonstrating that each misleading chart differs from its faithful twin only by the intended intervention; this is not yet fully established for the categories that drive the headline results.

major comments (4)
  1. [Section III-B, Principle P3 (Section III-A)] The controlled-pair premise is load-bearing for the entire Deception Score attribution, but it is not established for manually post-processed pairs. The text admits that middle-axis truncation and "certain inverted-axis variants" required manual editing after rendering, and Inverted Axis is the category with the highest Deception Scores in Table II (e.g., GPT-4o DS=1.97, Qwen3-VL-8B DS=2.09). Manual editing can change tick spacing, plot-area cropping, line interpolation, or anti-aliasing, all of which would inflate DS. No pixel-level diffs, rendering-seed controls, or human validation are provided. Please supply per-pair quantitative evidence that faithful and misleading rendered images differ only in the intended manipulation, or regenerate these categories with a fully programmatic pipeline that supports the transformation.
  2. [Section IV-C and Section V-C] The mitigation claim is weakened by the absence of a no-metadata control and by the possibly cueing role of the extraction schema. The Data Extraction Agent prompt asks for axis direction, scale ranges, and an optional "visual_warnings" field; the field name itself can implicitly instruct the model to look for chart anomalies, especially since examples in Figure 5 show the extractor flagging "the y-axis is inverted, which can be misleading." Thus the improvement may come from implicit deception-aware prompting rather than from structured grounding. Add an ablation that removes the visual_warnings field or replaces it with a neutral "chart_notes" field, and a control that gives the Solver Agent only a data table without any structural metadata. Report significance tests or confidence intervals for the Table III changes, since two models worsen and all differences are point estimates.
  3. [Section V-A, Table II] The Wilcoxon signed-rank tests are not corrected for multiple comparisons. The paper reports p<0.05 for 80 tests (8 categories x 10 models), and several p-values are close to the threshold (e.g., GPT-4o Aspect Ratio p=0.03, Gemma3-12B-IT Distorted Projection p=0.03, Qwen3-VL-4B Distorted Projection p=0.04). Without FDR or family-wise correction, individual category-level significance claims may be spurious. Please report corrected p-values or pre-specified hypotheses, and include an effect size such as the rank-biserial correlation. The overall qualitative pattern may survive correction, but the current table overstates the number of statistically robust findings.
  4. [Section IV-C / Table III] The multi-agent setup is not reproducible as described. For each model row in Table III, it is unclear whether the same model is used as both the Data Extraction Agent and the Solver Agent, or whether a stronger fixed extractor is used. This matters because the stated failure mode for weaker models is extraction error propagation. Please specify the exact agent assignment per model, decoding parameters (temperature, top-p, max tokens), number of runs, and how ties or JSON parse failures were handled. Also report per-category variance for the Deception Score reductions, since Table IV shows that some categories increase under mitigation (e.g., Qwen3-VL-32B on Truncated Axis and Data-Visual Disproportion).
minor comments (9)
  1. [Abstract / Section III-B] The abstract says "1,600 paired faithful and misleading charts," but the benchmark contains 800 pairs / 1,600 chart images. Please rephrase to avoid implying 1,600 pairs.
  2. [Abstract / Introduction / Section V / Conclusion] The number of responses is inconsistent: Abstract and Section V say 32,000, while Introduction and Conclusion say "48,000 responses." Since 10 models x 1,600 chart images x 2 settings = 32,000, the larger number is unexplained and should be corrected.
  3. [Section IV-A / Table II / Section V-A] Model naming is inconsistent: "Claude Haiku 4.5" in the text, "Claude-4.5-Haiku" in Table II, and "Claude-3.5-Haiku" in Section V-A. Standardize to the version actually evaluated.
  4. [Figure 1] The Multi-Agent response in the right panel reads "Option 6 - Moderate Decrease: A noticeable downward trend," but the ground truth is Option 6 (Moderate Increase). This looks like a typo and should be fixed.
  5. [Section V-D] The sentence "Inappropriate Encoding, Data Visual Disproportion, Inappropriate Encoding, and Dual Axis" repeats "Inappropriate Encoding." Remove the duplicate.
  6. [Figure 6(d)] The Solver Agent reasoning contains "Modereate Increase"; correct the spelling.
  7. [Equation (2)] Using absolute difference in option indices treats the seven ordered categories as equally spaced. Since the scale is ordinal, this is an interval-level assumption. Please state it explicitly as a design choice or justify it, and note that the Deception Score is not invariant under monotone transforms of the options.
  8. [Table II] The dashes for Qwen3-VL-2B on Dual Axis and Truncated Axis are unexplained. If Wilcoxon could not be computed because all responses were identical, say so; otherwise the missing statistics are confusing.
  9. [Section III-B] The sentence "each major deception categories with100 paired instances" has a grammar error; also clarify how the 100 instances per major category are distributed across subcategories (Table I already gives counts, but the text should be consistent).

Circularity Check

0 steps flagged

No significant circularity: the Deception Score is a defined evaluation metric, and the vulnerability results rest on new paired evaluations rather than on the paper's own assumptions.

full rationale

The paper's load-bearing empirical claim is that misleading chart designs shift VLM responses. This is supported by a paired benchmark whose construction is described independently (Sec. III-B: ChartQA data, programmatic rendering in Matplotlib/Vega-Lite/Highcharts, gold answers computed from data tables), and by paired Wilcoxon tests plus the Deception Score (Eq. 1). The Deception Score is an explicit operationalization — the mean increase in ordinal error on the misleading chart relative to the faithful chart — not a fitted parameter renamed as a prediction; no quantity is estimated from the data and then reported as if derived. The mitigation framework is an inference-time pipeline whose improvements (e.g., Gemini 2.5 Pro 0.099 to 0.024) are measured, not derived from the framework's design. Self-citations to the authors' earlier VIS 2025 short paper [20] occur for the taxonomy, question format, and benchmark-construction principles, but the central vulnerability findings do not reduce to those citations; they are independently evaluated here. The manual post-processing of middle-truncated and some inverted-axis charts (Sec. III-B) and the optional visual_warnings field in the extraction prompt are legitimate threats to construct validity and to the 'cue-free' framing, but they are confounds or design choices, not cases where an equation or conclusion is equivalent to its input by construction. Therefore no circular step can be exhibited, and the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

No numeric parameters are fitted to data. The central measurement depends on hand-chosen response bins and the causal-attribution premise; the mitigation depends on the extraction agent producing reliable chart metadata.

free parameters (1)
  • 7-point Likert bin boundaries = 10 / 25 / 50 / 75 / 100 percent thresholds
    Hand-chosen thresholds define the ordinal options used to compute the Deception Score (Eq. 1-2); changing bins changes ordinal distances and could change model rankings. Not fitted to data.
axioms (4)
  • domain assumption Each misleading chart differs from its faithful counterpart by exactly one deceptive intervention, making response shifts causally attributable.
    Section III-A P3 and Section III-B assert controlled pairing; Section III-B admits manual post-processing for middle truncation and some inverted axes without pixel-level validation.
  • domain assumption The eight deception categories from prior visualization literature are genuinely misleading and transfer to VLM perception.
    Section III-A grounds taxonomy in prior work [5,47,50]; no human validation of misleadingness is reported.
  • domain assumption Absolute difference between seven-point option indices is a meaningful measure of error severity.
    Eq. (2) uses |O-G| and |M-G|; a trend reversal (Option 6 vs 2) is treated as four steps, the same as any four-step move, with no perceptual justification.
  • domain assumption Gold answers computed programmatically from data tables are unambiguous under the Likert definitions.
    Section III-C computes answers via Python from data; relies on bins such as '10% to less than 25%' being clear.

pith-pipeline@v1.3.0-alltime-deepseek · 22643 in / 19928 out tokens · 200739 ms · 2026-08-02T11:27:24.130081+00:00 · methodology

0 comments
read the original abstract

Information visualizations are widely used to communicate patterns, trends, and outliers, yet deceptive design choices-such as truncated or inverted axes, distorted aspect ratios, inappropriate encodings, and misleading color mappings-can systematically alter interpretation while preserving the underlying data. As Vision-Language Models (VLMs) are increasingly used for chart understanding and analytical reasoning, assessing their robustness to such deceptive visualizations has become critical for trustworthy data analysis. We introduce VisDeception, the first controlled paired benchmark for evaluating the robustness of VLMs to misleading chart designs. The benchmark contains 1,600 paired faithful and misleading charts spanning eight major categories of deceptive visualization tactics, where each misleading chart is paired with a faithful counterpart generated from the same underlying data. To isolate deception-induced reasoning errors from baseline chart-understanding errors, we introduce the Deception Score, a paired evaluation metric that quantifies how misleading visualizations shift model responses away from the faithful interpretation of the data. Across 32,000 responses from 10 state-of-the-art VLMs, we find that even advanced models remain highly vulnerable to deceptive visual manipulations. To improve robustness, we further propose an inference-time multi-agent mitigation framework that grounds reasoning in structured chart metadata extracted from the visualization before answer generation, enabling models to reduce the influence of deceptive visual cues without requiring explicit user instructions. Together, our findings reveal important reliability gaps in current chart-understanding systems and establish benchmark-driven evaluation, deception-aware metrics, and structured reasoning as promising directions for developing more trustworthy VLMs for visual analytics.

Figures

Figures reproduced from arXiv: 2607.22600 by Enamul Hoque, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Mizanur Rahman, Mohammed Saidul Islam, Ridwan Mahbub.

Figure 1
Figure 1. Figure 1: Misleading visual design can alter VLM reasoning [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our benchmark construction process: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the eight misleading chart designs studied, along with the subtypes of each of the major types. For each [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example question instance for the truncated-axis de [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Inference time multi-agent mitigation framework. A Data Extraction Agent first converts the chart into a structured [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A qualitative analysis showing cases where mitigation approach has been success for the best model, Gemini 2.5 pro, [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 13 linked inside Pith

  1. [1]

    Narrative visualization: Telling stories with data,

    E. Segel and J. Heer, “Narrative visualization: Telling stories with data,” IEEE Transactions on Visualization and Computer Graphics, vol. 16, no. 6, pp. 1139–1148, 2010

    2010

  2. [2]

    Huff,How to lie with statistics

    D. Huff,How to lie with statistics. Penguin UK, 2023

    2023

  3. [3]

    Cairo,How charts lie: Getting smarter about visual information

    A. Cairo,How charts lie: Getting smarter about visual information. WW Norton & Company, 2019

    2019

  4. [4]

    ”yeah, this graph doesn’t show that

    M. Lisnic, A. Lex, and M. Kogan, “”yeah, this graph doesn’t show that”: Analysis of online engagement with misleading data visualizations,” inProceedings of the 2024 CHI, 2024. [Online]. Available: https://doi.org/10.1145/3613904.3642448

    arXiv 2024

  5. [5]

    How deceptive are deceptive visualizations? an empirical analysis of common distortion techniques,

    A. V . Pandey, K. Rall, M. L. Satterthwaite, O. Nov, and E. Bertini, “How deceptive are deceptive visualizations? an empirical analysis of common distortion techniques,” inProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2015, pp. 1469–1478

    2015

  6. [6]

    Improving the efficiency and ease of healthcare analysis through use of data visualization dashboards,

    J. G. Stadler, K. Donlon, J. D. Siewert, T. Franken, and N. E. Lewis, “Improving the efficiency and ease of healthcare analysis through use of data visualization dashboards,”Big data, vol. 4, no. 2, pp. 129–135, 2016

    2016

  7. [7]

    Data visualization in annual reports–impacting investment decisions,

    M. M. Uddin, R. Ullah, and M. Moniruzzaman, “Data visualization in annual reports–impacting investment decisions,”International Journal for Multidisciplinary Research, vol. 6, no. 5, 2024

    2024

  8. [8]

    Social media visual analytics,

    S. Chenet al., “Social media visual analytics,” inComputer Graphics Forum, vol. 36, no. 3. Wiley Online Library, 2017, pp. 563–587

    2017

  9. [9]

    From charts to fair narratives: Uncovering and mitigating geo-economic biases in chart-to-text,

    R. Mahbub, M. S. Islam, M. T. Nayeem, M. T. R. Laskar, M. Rahman, S. Joty, and E. Hoque, “From charts to fair narratives: Uncovering and mitigating geo-economic biases in chart-to-text,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 28 917–28 935

    2025

  10. [10]

    Biased llms can influence political decision-making,

    J. Fisher, S. Feng, R. Aron, T. Richardson, Y . Choi, D. W. Fisher, J. Pan, Y . Tsvetkov, and K. Reinecke, “Biased llms can influence political decision-making,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 6559–6607

    2025

  11. [11]

    Chart-to-text: A large-scale benchmark for chart summa- rization,

    S. Kantharaj, R. T. K. Leong, X. Lin, A. Masry, M. Thakkar, E. Hoque, and S. Joty, “Chart-to-text: A large-scale benchmark for chart summa- rization,”arXiv preprint arXiv:2203.06486, 2022

    Pith/arXiv arXiv 2022

  12. [12]

    Chart question answering: State of the art and future directions,

    E. Hoque, P. Kavehzadeh, and A. Masry, “Chart question answering: State of the art and future directions,” inComputer Graphics Forum, vol. 41, no. 3. Wiley Online Library, 2022, pp. 555–572

    2022

  13. [13]

    Enhancing data literacy on-demand: Llms as guides for novices in chart interpretation,

    K. Choe, C. Lee, S. Lee, J. Song, A. Cho, N. W. Kim, and J. Seo, “Enhancing data literacy on-demand: Llms as guides for novices in chart interpretation,”IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  14. [14]

    Vizability: Enhancing chart accessibility with llm-based conversational interaction,

    J. Gorniak, Y . Kim, D. Wei, and N. W. Kim, “Vizability: Enhancing chart accessibility with llm-based conversational interaction,” inProc. of the 37th Annual ACM Symposium on UIST, 2024, pp. 1–19

    2024

  15. [15]

    Can gpt-4 models detect misleading visualizations?

    J. Alexander, P. Nanda, K.-C. Yang, and A. Sarvghad, “Can gpt-4 models detect misleading visualizations?” in2024 IEEE VIS, 2024, pp. 106–110

    2024

  16. [16]

    How good (or bad) are llms at detecting misleading visualizations?

    L. Y .-H. Lo and H. Qu, “How good (or bad) are llms at detecting misleading visualizations?”IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  17. [17]

    Chartqapro: A more diverse and challenging benchmark for chart question answering,

    A. Masry, M. S. Islam, M. Ahmed, A. Bajaj, F. Kabir, A. Kartha, M. T. R. Laskar, M. Rahman, S. Rahman, M. Shahmohammadiet al., “Chartqapro: A more diverse and challenging benchmark for chart question answering,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 19 123–19 151

    2025

  18. [18]

    Misvisfix: An interactive dashboard for de- tecting, explaining, and correcting misleading visualizations using large language models,

    A. K. Das and K. Mueller, “Misvisfix: An interactive dashboard for de- tecting, explaining, and correcting misleading visualizations using large language models,”IEEE Transactions on Visualization and Computer Graphics, 2025

    2025

  19. [19]

    Unmasking deceptive visuals: Benchmarking multimodal large lan- guage models on misleading chart question answering,

    Z. Chen, S. Song, K. Shum, Y . Lin, R. Sheng, W. Wang, and H. Qu, “Unmasking deceptive visuals: Benchmarking multimodal large lan- guage models on misleading chart question answering,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 13 767–13 800. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    2025

  20. [20]

    The perils of chart deception: How misleading visual- izations affect vision-language models,

    R. Mahbub, M. S. Islam, M. T. R. Laskar, M. Rahman, M. T. Nayeem, and E. Hoque, “The perils of chart deception: How misleading visual- izations affect vision-language models,” in2025 IEEE Visualization and Visual Analytics (VIS). IEEE, 2025, pp. 6–10

    2025

  21. [21]

    Are large vision language models up to the challenge of chart comprehension and reasoning? an extensive investigation into the capabilities and limitations of lvlms,

    M. S. Islam, R. Rahman, A. Masry, M. T. R. Laskar, M. T. Nayeem, and E. Hoque, “Are large vision language models up to the challenge of chart comprehension and reasoning? an extensive investigation into the capabilities and limitations of lvlms,”arXiv preprint arXiv:2406.00257, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Chart-to-text: A large-scale benchmark for chart summarization,

    K. Shankar, L. Rixie Tiffany Ko, L. Xiang, M. Ahmed, T. Megh, H. Enamul, and J. Shafiq, “Chart-to-text: A large-scale benchmark for chart summarization,” inIn Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2022, 2022

    2022

  23. [23]

    Vistext: A benchmark for semantically rich chart captioning,

    B. Tang, A. Boggust, and A. Satyanarayan, “Vistext: A benchmark for semantically rich chart captioning,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 7268–7298

    2023

  24. [24]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning,

    A. Masry, X. L. Do, J. Q. Tan, S. Joty, and E. Hoque, “ChartQA: A benchmark for question answering about charts with visual and logical reasoning,” inFindings of the Association for Computational Linguistics: ACL 2022. Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 2263–2279. [Online]. Available: https://aclanthology.org/2022.fi...

    2022

  25. [25]

    Opencqa: Open-ended question answering with charts,

    S. Kantharaj, X. L. Do, R. T. K. Leong, J. Q. Tan, E. Hoque, and S. Joty, “Opencqa: Open-ended question answering with charts,” inProceedings of EMNLP (to appear), 2022

    2022

  26. [26]

    Pix2struct: Screenshot parsing as pretraining for visual language understanding,

    K. Lee, M. Joshi, I. Turc, H. Hu, F. Liu, J. Eisenschlos, U. Khandelwal, P. Shaw, M.-W. Chang, and K. Toutanova, “Pix2struct: Screenshot parsing as pretraining for visual language understanding,”arXiv preprint arXiv:2210.03347, 2022

    Pith/arXiv arXiv 2022

  27. [27]

    Reading and reasoning over chart images for evidence-based automated fact-checking,

    M. Akhtar, O. Cocarascu, and E. Simperl, “Reading and reasoning over chart images for evidence-based automated fact-checking,” in Findings of the Association for Computational Linguistics: EACL 2023, A. Vlachos and I. Augenstein, Eds. Dubrovnik, Croatia: Association for Computational Linguistics, May 2023, pp. 399–414. [Online]. Available: https://aclanth...

    2023

  28. [28]

    Chartcheck: An evidence-based fact-checking dataset over real-world chart images,

    M. Akhtar, N. Subedi, V . Gupta, S. Tahmasebi, O. Cocarascu, and E. Simperl, “Chartcheck: An evidence-based fact-checking dataset over real-world chart images,” 2023

    2023

  29. [29]

    From data to story: Towards automatic animated data video creation with llm-based multi-agent systems,

    L. Shen, H. Li, Y . Wang, and H. Qu, “From data to story: Towards automatic animated data video creation with llm-based multi-agent systems,” in2024 IEEE VIS Workshop on Data Storytelling in an Era of Generative AI (GEN4DS). IEEE, 2024, pp. 20–27

    2024

  30. [30]

    DataNarrative: Automated data-driven storytelling with visualizations and texts,

    M. S. Islam, M. T. R. Laskar, M. R. Parvez, E. Hoque, and S. Joty, “DataNarrative: Automated data-driven storytelling with visualizations and texts,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov....

    2024

  31. [31]

    Dataweaver: Authoring data-driven narratives through the integrated composition of visualization and text,

    Y . Fu, D. Bromley, and V . Setlur, “Dataweaver: Authoring data-driven narratives through the integrated composition of visualization and text,” inComputer Graphics Forum, vol. 44, no. 3, 2025, p. e70098

    2025

  32. [32]

    Text2Vis: A challenging and diverse benchmark for generating multimodal visualizations from text,

    M. Rahman, M. T. R. Laskar, S. Joty, and E. Hoque, “Text2Vis: A challenging and diverse benchmark for generating multimodal visualizations from text,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Peng, Eds. Suzhou, China: Association for Computational Ling...

    2025

  33. [33]

    Deepvis: Bridging natural language and data visualization through step-wise reasoning,

    Z. Shuai, B. Li, S. Yan, Y . Luo, and W. Yang, “Deepvis: Bridging natural language and data visualization through step-wise reasoning,” IEEE Transactions on Visualization and Computer Graphics, 2025

    2025

  34. [34]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    Introducing the next generation of claude,

    Anthropic, “Introducing the next generation of claude,” 2024. [Online]. Available: https://www.anthropic.com/news/claude-3-family

    2024

  36. [36]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

    G. Comanici, E. Bieberet al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,”arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Charxiv: Charting gaps in realistic chart understanding in multimodal llms,

    Z. Wang, M. Xia, L. He, H. Chen, Y . Liu, R. Zhu, K. Liang, X. Wu, H. Liu, S. Malladiet al., “Charxiv: Charting gaps in realistic chart understanding in multimodal llms,”Advances in Neural Information Processing Systems, vol. 37, pp. 113 569–113 697, 2024

    2024

  38. [38]

    Llava-onevision: Easy visual task transfer,

    B. Li, Y . Zhang, Guoet al., “Llava-onevision: Easy visual task transfer,” arXiv:2408.03326, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling,

    Z. Chenet al., “Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling,” 2025. [Online]. Available: https://arxiv.org/abs/2412.05271

    Pith/arXiv arXiv 2025

  40. [40]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    Unichart: A universal vision-language pretrained model for chart comprehension and reasoning,

    A. Masry, P. Kavehzadeh, X. L. Do, E. Hoque, and S. Joty, “Unichart: A universal vision-language pretrained model for chart comprehension and reasoning,” inProc. of EMNLP 2023, 2023, pp. 14 662–14 684

    2023

  42. [42]

    ChartInstruct: Instruction tuning for chart comprehension and reasoning,

    A. Masry, M. Shahmohammadi, M. R. Parvez, E. Hoque, and S. Joty, “ChartInstruct: Instruction tuning for chart comprehension and reasoning,” inFindings of ACL 2024, Aug. 2024, pp. 10 387–10 409. [Online]. Available: https://aclanthology.org/2024.findings-acl.619/

    2024

  43. [43]

    ChartGemma: Visual instruction-tuning for chart reasoning in the wild,

    A. Masry, M. Thakkar, A. Bajaj, A. Kartha, E. Hoque, and S. Joty, “ChartGemma: Visual instruction-tuning for chart reasoning in the wild,” inProceedings of the 31st International Conference on Computational Linguistics: Industry Track, Jan. 2025, pp. 625–643. [Online]. Available: https://aclanthology.org/2025.coling-industry.54/

    2025

  44. [44]

    Bigcharts-r1: Enhanced chart reasoning with visual reinforcement fine- tuning,

    A. Masry, A. Puri, M. Hashemi, J. A. Rodriguez, M. Thakkar, K. Ma- hajan, V . Yadav, S. T. Madhusudhan, A. Pich ´e, D. Bahdanauet al., “Bigcharts-r1: Enhanced chart reasoning with visual reinforcement fine- tuning,”arXiv preprint arXiv:2508.09804, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning,

    A. Masry, X. L. Do, J. Q. Tan, S. Joty, and E. Hoque, “Chartqa: A benchmark for question answering about charts with visual and logical reasoning,” inFindings of ACL 2022, 2022, pp. 2263–2279

    2022

  46. [46]

    From pixels to insights: A survey on automatic chart understanding in the era of large foundation models,

    K.-H. Huang, H. P. Chan, Y . R. Fung, H. Qiu, M. Zhou, S. Joty, S.-F. Chang, and H. Ji, “From pixels to insights: A survey on automatic chart understanding in the era of large foundation models,” 2024. [Online]. Available: https://arxiv.org/abs/2403.12027

    Pith/arXiv arXiv 2024

  47. [47]

    The deceptive potential of com- mon design tactics used in data visualizations,

    C. Lauer, S. O’Brien, and S. Witte, “The deceptive potential of com- mon design tactics used in data visualizations,”IEEE Transactions on Professional Communication, vol. 63, no. 4, pp. 353–364, 2020

    2020

  48. [48]

    “i came across a junk

    X. Lan and Y . Liu, ““i came across a junk”: Understanding design flaws of data visualization from the public’s perspective,”IEEE TVCG, 2024

    2024

  49. [49]

    Calvi: Critical thinking assessment for literacy in visualizations,

    L. W. Ge, Y . Cui, and M. Kay, “Calvi: Critical thinking assessment for literacy in visualizations,” inProc. of the 2023 CHI, 2023, pp. 1–18

    2023

  50. [50]

    Misleading beyond visual tricks: How people actually lie with charts,

    M. Lisnic, C. Polychronis, A. Lex, and M. Kogan, “Misleading beyond visual tricks: How people actually lie with charts,” inProceedings of the 2023 CHI Conference on Human Factors in Computing Systems, ser. CHI ’23. New York, NY , USA: Association for Computing Machinery,

    2023

  51. [51]

    Truncating the y-axis: Threat or menace?

    M. Correll, E. Bertini, and S. Franconeri, “Truncating the y-axis: Threat or menace?” inProc. of the 2020 CHI, 2020, pp. 1–12

    2020

  52. [52]

    To cut or not to cut? a systematic exploration of y-axis truncation,

    S. Long and M. Kay, “To cut or not to cut? a systematic exploration of y-axis truncation,” inProc. of the 2024 CHI, 2024, pp. 1–12

    2024

  53. [53]

    Annotating line charts for addressing deception,

    A. Fan, Y . Ma, M. Mancenido, and R. Maciejewski, “Annotating line charts for addressing deception,” inProc. of the CHI, 2022, pp. 1–12

    2022

  54. [54]

    Misinformed by visualization: What do we learn from misinformative visualizations?

    L. Y .-H. Lo, A. Gupta, K. Shigyo, A. Wu, E. Bertini, and H. Qu, “Misinformed by visualization: What do we learn from misinformative visualizations?” inComputer Graphics Forum, vol. 41, no. 3. Wiley Online Library, 2022, pp. 515–525

    2022

  55. [55]

    The good, the bad, and the biased: Five ways visualizations can mislead (and how to fix them),

    D. A. Szafir, “The good, the bad, and the biased: Five ways visualizations can mislead (and how to fix them),”interactions, vol. 25, no. 4, pp. 26– 33, 2018

    2018

  56. [56]

    Multi-scale banking to 45 degrees,

    J. Heer and M. Agrawala, “Multi-scale banking to 45 degrees,”IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 5, pp. 701–708, 2006

    2006

  57. [57]

    W. S. Cleveland,Visualizing Data. Summit, NJ: Hobart Press, 1993

    1993

  58. [58]

    Rainbow color map (still) considered harmful,

    D. Borland and R. M. Taylor Ii, “Rainbow color map (still) considered harmful,”IEEE Computer Graphics and Applications, vol. 27, no. 2, pp. 14–17, 2007

    2007

  59. [59]

    Colorbrewer.org: An online tool for selecting colour schemes for maps,

    M. Harrower and C. A. Brewer, “Colorbrewer.org: An online tool for selecting colour schemes for maps,”The Cartographic Journal, vol. 40, no. 1, pp. 27–37, 2003

    2003

  60. [60]

    Matplotlib: A 2d graphics environment,

    J. D. Hunter, “Matplotlib: A 2d graphics environment,”Computing in Science & Engineering, vol. 9, no. 3, pp. 90–95, 2007

    2007

  61. [61]

    Vega-lite: A grammar of interactive graphics,

    A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer, “Vega-lite: A grammar of interactive graphics,”IEEE Transactions on Visualization & Computer Graphics (Proc. InfoVis), 2017. [Online]. Available: http://idl.cs.washington.edu/papers/vega-lite

    2017

  62. [62]

    Highcharts: Highcharts JS, the JavaScript charting framework,

    “Highcharts: Highcharts JS, the JavaScript charting framework,” https: //github.com/highcharts/highcharts, 2026, accessed: 2026-05-19

    2026

  63. [63]

    Gemma: Open models based on gemini research and technology,

    G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivi `ere, M. S. Kale, J. Loveet al., “Gemma: Open models based on gemini research and technology,”arXiv:2403.08295, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 14

    Pith/arXiv arXiv 2024

  64. [64]

    Pixtral 12b,

    P. Agrawal, S. Antoniak, E. B. Hanna, B. Bout, D. Chaplot, J. Chud- novsky, D. Costa, B. De Monicault, S. Garg, T. Gervetet al., “Pixtral 12b,”arXiv preprint arXiv:2410.07073, 2024

    Pith/arXiv arXiv 2024

  65. [65]

    Datareel: Automated data-driven video story generation with animations,

    R. Mahbub, S. Aziz, M. Ahmed, S. Rahman, M. Rahman, S. Joty, and E. Hoque, “Datareel: Automated data-driven video story generation with animations,”arXiv preprint arXiv:2604.25220, 2026

    Pith/arXiv arXiv 2026

  66. [66]

    Mitigating social bias in large language models: A multi- objective approach within a multi-agent framework,

    Z. Xu, W. Chen, Y . Tang, X. Li, C. Hu, Z. Chu, K. Ren, Z. Zheng, and Z. Lu, “Mitigating social bias in large language models: A multi- objective approach within a multi-agent framework,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 24, 2025, pp. 25 579–25 587. Ridwan Mahbubis an MSc student at York Uni- versity and a Grad...

    2025

  67. [2023]

    Available: https://doi.org/10.1145/3544548.3580910

    [Online]. Available: https://doi.org/10.1145/3544548.3580910

    arXiv