Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Dataset

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that ChartInsighter, a pipeline pairing large language models with external data-analysis modules and a self-consistency check, reduces hallucinations in time-series chart summaries to 0.14 per sentence versus 0.48 for…

desk verdict Useful benchmark and pipeline; the headline hallucination-rate advantage may be mostly about omissions, not factual errors. read the letter →

arxiv 2501.09349 v1 pith:V4ZW4D5T submitted 2025-01-16 cs.CL cs.HC

Fen Wang , Bomiao Wang , Xueli Shu , Zhen Liu , Zekai Shao , Chao Liu , Siming Chen This is my paper · ORCID
classification cs.CLcs.HC
keywords time-serieschartsummarizationhallucinationmitigationlargelanguagemodelsmulti-agentcollaborationself-consistencysummarybenchmarkdatavisualization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that chart-summary generation for time-series data becomes substantially more reliable when an LLM is paired with external data-analysis modules and iterative multi-agent drafting, instead of being asked to compute and reason about the data from text alone. It defines a taxonomy of ten hallucination types and a set of L1-L3 summary elements, then builds ChartInsighter, a pipeline that segments the data into trend-consistent patches, computes key statistics externally, drafts and refines a summary through collaborating agents, and checks the final text with a self-consistency test. To measure the result, the authors contribute a benchmark of 75 real-world time-series line charts with sentence-level hallucination annotations, on which they report a hallucination rate of 0.14 for ChartInsighter versus 0.48 for GPT-4 and 1.63 for VL2NL, and a human quality rating of 3.79 versus 2.86 and 1.70. If these numbers hold, automatic chart summaries could be used for decision support and data journalism without checking every number by hand.

What carries the argument

The load-bearing object is the patch-based representation produced by the Numerical Pattern Analysis Module: the module splits a long time series at significant extrema, merges consecutive low-variance patches using a threshold derived from the median of patch variances, and outputs each patch's time span, maximum, minimum, trend, and volatility. This turns the LLM's weakest tasks, arithmetic and fine-grained trend recognition, into statements of precomputed facts. The Multi-dimensional Relation Analysis Module uses the same patches to resolve coarse temporal phrases to precise time ranges, and the self-consistency test rechecks flagged sentences against those facts. The hallucination taxonomy from Section 3.3 is the guide that tells each module which errors to look for.

What would settle it

A blinded re-annotation study: have annotators who have not seen the authors' hallucination taxonomy label the factual errors in the same 75 chart summaries, and have human raters score summaries without knowing which system produced them. If independent labels do not show ChartInsighter's hallucination rate below GPT-4's and VL2NL's, or if blinded quality scores do not rank ChartInsighter highest, the central claim is not supported.

Watch

Extended reading notes

Core claim

The authors' central claim is that hallucinations in time-series chart summaries are not one undifferentiated failure but a set of distinct, nameable error types, and that each type can be reduced by a different mechanism. ChartInsighter has a Uni-Insighter agent call a numerical pattern analysis module that cuts the time series into patches and outputs extrema, volatility, and growth statistics; a Multi-Insighter agent generates multidimensional relationship descriptions three times and keeps the majority vote; a Writer refines the draft iteratively; and a self-consistency test re-examines sentences that may contain extremum or proportion-perception errors, correcting them when reanalysis disagrees. The authors report that on their benchmark this pipeline produces the lowest hallucination rate among GPT-4, VL2NL, and ChartInsighter, and the highest human ratings for accuracy, fluency, and matching between chart and summary.

Load-bearing premise

The load-bearing premise is that the benchmark's sentence-level hallucination labels, produced by annotators trained on a taxonomy the same authors designed, are an unbiased measure of how factually correct each summary is.

Editorial extensions

If this is right

  • On the reported numbers, ChartInsighter's hallucination rate of 0.14 per sentence versus 0.48 for GPT-4 and 1.63 for VL2NL means chart summaries can be produced with far fewer factual errors per sentence.
  • The ten-type hallucination taxonomy gives future systems a common checklist for targeting specific failures such as extremum errors, trend-direction errors, and detail omission.
  • The benchmark's sentence-level annotations let new summarization methods be compared on hallucination reduction rather than only on semantic similarity or human preference.
  • The interactive text-to-chart linking in ChartInsighter lets readers hover over a sentence and see the referenced chart region, making residual errors easier to spot and correct.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the taxonomy generalizes beyond line charts, it could be adapted into a quality checklist for captions of bar charts, maps, and other visualization types, where similar numerical and relational errors appear.
  • A testable extension is to make the trend segmentation fully deterministic and statistical, removing the LLM from the patch-splitting step, and to measure whether hallucination rates drop further; the paper's variance-based merging already moves in that direction.
  • The sentence-level labels could support hallucination detection as a separate task: a model fine-tuned on them might flag suspect claims in any chart summary, not only in summaries produced by this pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ChartInsighter, a multi-agent LLM pipeline for generating time-series chart summaries, which uses external data analysis modules, iterative refinement between agents, and a self-consistency test to reduce hallucinations. The authors also define a taxonomy of summary elements and hallucination types, and release a benchmark of 75 charts with 2,693 sentence-level annotated summaries. The central claim, stated in the abstract, is that ChartInsighter surpasses state-of-the-art models and achieves the lowest summary hallucination rate, supported by Table 1 with a hallucination rate of 0.14 versus 0.48 for GPT-4 and 1.63 for VL2NL, and a human quality score of 3.79 versus 2.86 and 1.70.

Significance. If validated, the work would make a useful contribution: it provides a publicly released benchmark, a sentence-level hallucination taxonomy for time-series chart summaries, an interactive system with text-to-chart linking, and a pipeline that combines tool-based computation with multi-agent LLM collaboration. The benchmark resource and the design rationale are valuable for future work on chart summarization. However, the current evidence for the central claim is weakened by the fact that the evaluation uses a benchmark and taxonomy created by the same group, reports no reliability statistics or significance tests, and counts omissions and vague statements in the headline hallucination metric. The proposed system and benchmark are promising, but the central comparative claim needs stronger validation before publication.

major comments (3)
  1. [Section 3.3, Section 6.2, Table 1] The headline hallucination rate conflates factual hallucinations with three categories that the paper itself labels 'Limitations of Chart Summaries Generated': Detail Omission, Junk Description, and Proportion Perception Error. The Hallucination Rate in Table 1 appears to count all ten categories, because Section 6.2 instructs annotators to follow the Section 3.3 taxonomy and the qualitative analysis treats Detail Omission and Junk Description as hallucinations. Detail Omission is an absence of content rather than a false statement, Junk Description is vacuous but not necessarily fabricated, and Proportion Perception Error is a subjective judgment about words such as 'significant.' Since ChartInsighter is explicitly designed to extract more detailed insights through external modules and iterative refinement, the reported advantage (0.14 vs. 0.48) may reflect greater recall and specificity rather than fewer factual errors. Please report per-category rates separately, and make the seven true hallucination types the primary metric.
  2. [Section 5, Section 6.1] The evaluation lacks the reliability information needed to support the comparative claim. The benchmark annotations and the human quality ratings were produced by small numbers of participants trained on the authors' own taxonomy, and the paper does not state whether raters were blinded to which system generated each summary. Table 1 reports point estimates only, with no confidence intervals, error bars, or significance tests across the 75 charts. Because the benchmark and the system were developed by the same group, this is a real circularity risk. Please report inter-annotator agreement (e.g., Cohen's kappa or Krippendorff's alpha), describe the blinding protocol, and provide error bars or significance tests for the hallucination-rate and human-score differences.
  3. [Section 6.1, Section 6.2] The abstract's claim that the method 'surpasses state-of-the-art models' is not supported by the evidence. The evaluation compares only GPT-4 and VL2NL. Other LLMs used to derive the taxonomy in Section 3.3 (Claude-3, GPT-4o, LLaMA-3.1-70B) and existing chart-summarization systems such as ChartThinker and VisText are not included as baselines. There is also no ablation isolating the contributions of the external data-analysis modules, the iterative multi-agent refinement, or the self-consistency test. Please add stronger baselines and an ablation study, or temper the claim to cover the two compared systems.
minor comments (5)
  1. [Section 4.1] The text and the prompt template in Figure 4 contain the typo 'time patchs'; this should be 'time patches.'
  2. [Figure 3] The label 'Sta bility Error' in Figure 3 has an erroneous space; it should read 'Stability Error.'
  3. [Section 4.2, Section 6.3] The stopping criterion for the Refining step is described in Section 4.2 as continuing until Multi-Insighter finds no new insights, but Section 6.3 states that a maximum of five iterations is set. Please clarify which rule is used and whether the limit is reached in practice.
  4. [Section 6.2] The definition of Hallucination Rate should explicitly state that the numerator is the total number of annotated hallucinations, which can be greater than the number of sentences, so the 'rate' can exceed 1; calling it a density or average count per sentence would be clearer.
  5. [Section 6.1] The human evaluation reports only the aggregate score; reporting the per-criterion scores for Accuracy, Fluency, and Matching Degree would help readers understand what drives the overall 3.79 rating.

Circularity Check

2 steps flagged · score 5.0 of 10

The headline hallucination-rate comparison is measured on a benchmark whose labels come from the same author-defined taxonomy that ChartInsighter was explicitly built to satisfy; counting omissions as hallucinations makes the reported advantage partly a construct artifact.

  1. self definitional [Sec. 3.3 (Detail Omission), Sec. 4.2 (Refining), Sec. 6.2 (Quality Evaluation)]
    "Detail Omission... This error refers to when LLMs tend to generalize data within a specific range, focusing on overall trends while overlooking key fluctuations and turning points in time-series data. ... The Hallucination Rate is determined by the ratio of the number of hallucinations to the total number of sentences."

    The paper defines Hallucination Rate with Detail Omission counted as a hallucination, and Detail Omission is an absence of content rather than a false claim. ChartInsighter's Refining loop is explicitly designed to keep adding insights 'until Multi-Insighter determines that no new insights remain uncovered,' so a larger, more detailed summary automatically lowers the omission component of the rate. Since no per-category breakdown is reported, the headline 0.14 vs 0.48 advantage could be driven entirely by these non-factual 'limitations' categories rather than by reduced factual errors.

  2. other [Sec. 3.3 (Taxonomy construction), Sec. 5 (Benchmark annotation), Sec. 6.2 (Quality Evaluation)]
    "Then four authors, all with visualization backgrounds, reviewed the L1-L3 parts of the generated summary and independently created initial classifications of hallucination types. Then, we integrated each person's classifications and collectively discussed the different findings to collaboratively establish a unified, final taxonomy. ... We explained the types of hallucinations and their definitions to 6 participants, who then performed a sentence-by-sentence review of each generated summary, annotating instances of hallucinations."

    The hallucination categories used as evaluation labels were invented by the same authors who designed ChartInsighter to reduce exactly those categories, and the benchmark annotators were trained on that same taxonomy. The evaluation therefore measures alignment with the authors' own construct rather than against an independent, externally grounded notion of factual error. The human quality ratings in Sec. 6.1 also rely on the same definitions, so the closed loop between taxonomy, system design, and evaluation inflates the apparent strength of the central claim.

full rationale

ChartInsighter is not circular in the formal sense of fitting a parameter and renaming it a prediction: the system does not train on the benchmark labels, and the evaluation includes real human annotation and real-world charts. However, the central quantitative claim—lowest hallucination rate—is partially circular by construction. The taxonomy in Sec. 3.3 was produced by the same authors, then used both as generation guidelines and as the annotation scheme for the benchmark. More specifically, the Hallucination Rate metric counts Detail Omission and Junk Description as hallucinations even though these are categories of incompleteness or vagueness, not false statements. ChartInsighter's iterative refinement is explicitly designed to maximize coverage and eliminate omissions, so a lower rate on this metric is partly the intended effect of the design rather than evidence of fewer factual errors. The absence of a per-category breakdown prevents the reader from separating the seven true hallucination types from the three 'limitations' categories. The minor self-citations in Related Work (LEVA, LightVA) are not load-bearing. Overall, this is a real but partial circularity, warranting a score of 5 rather than a clean 0-2.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central evaluation rests on the authors' domain assumptions and hand-set hyperparameters; there are no invented physical entities. The main burden is that the hallucination taxonomy is both the design guide and the measurement instrument.

free parameters (3)
  • k (patch merge variance threshold multiplier) = 0
    In the Numerical Pattern Analysis Module, consecutive patches with variance below the median plus k times the standard deviation are merged; the authors state that after multiple attempts k=0 worked best, so this is a hand-tuned hyperparameter (Section 4.1).
  • repeat_times (multi-dimensional insight generations) = 3
    Multi-Insighter repeats generation of multi-dimensional trend descriptions three times to obtain diverse candidates for majority voting; the paper says this was set empirically (Section 4.1).
  • max_iterations (refining rounds) = 5
    The refining step is capped at five iterations to limit computation time (Section 6.3).
assumptions (5)
  • domain assumption Pandas and the external modules produce numerically correct statistical summaries from the data table.
    The pipeline relies on code execution (e.g., max, min) to compute exact values; the paper does not verify the code's outputs against the data (Section 4.1).
  • ad hoc to paper The hallucination taxonomy in Section 3.3 is a valid and complete account of hallucination types for time-series chart summaries.
    The taxonomy was created by the authors from 80 summaries of 20 charts; it is used both to guide the system and to define the evaluation labels, so the benchmark inherits this assumption (Sections 3.3, 5).
  • domain assumption Human annotators can reliably and consistently classify hallucinations at the sentence level using the taxonomy.
    Six participants annotated 2693 sentences; no inter-annotator agreement or adjudication procedure is reported (Section 5).
  • domain assumption The gold summaries are accurate ground truths.
    Gold summaries were produced by participants, sometimes by editing LLM drafts; the paper states a manual review was done but provides no independent verification (Section 5).
  • domain assumption LLM output in the self-consistency test is reliable enough to detect and correct errors.
    The final correction step asks the LLM itself to compare original and revised sentences, so correctness depends on the LLM's ability to identify its own errors (Section 4.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Dataset." pith.science (2026). https://pith.science/paper/V4ZW4D5T

@misc{pith2026250109349,
  author       = {Pith},
  title        = {Pith review of: ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V4ZW4D5T}},
  note         = {Machine review of arXiv:2501.09349}
}
read the original abstract

Effective chart summary can significantly reduce the time and effort decision makers spend interpreting charts, enabling precise and efficient communication of data insights. Previous studies have faced challenges in generating accurate and semantically rich summaries of time-series data charts. In this paper, we identify summary elements and common hallucination types in the generation of time-series chart summaries, which serve as our guidelines for automatic generation. We introduce ChartInsighter, which automatically generates chart summaries of time-series data, effectively reducing hallucinations in chart summary generation. Specifically, we assign multiple agents to generate the initial chart summary and collaborate iteratively, during which they invoke external data analysis modules to extract insights and compile them into a coherent summary. Additionally, we implement a self-consistency test method to validate and correct our summary. We create a high-quality benchmark of charts and summaries, with hallucination types annotated on a sentence-by-sentence basis, facilitating the evaluation of the effectiveness of reducing hallucinations. Our evaluations using our benchmark show that our method surpasses state-of-the-art models, and that our summary hallucination rate is the lowest, which effectively reduces various hallucinations and improves summary quality. The benchmark is available at https://github.com/wangfen01/ChartInsighter.

Figures

Figures reproduced from arXiv: 2501.09349 by the authors.

Figure 1
Figure 1. Examples of time-series chart summaries generated with GPT-4, VL2NL [28], and ChartInsighter. Errors are indicated in red text, while correct points are highlighted in green text. GPT-4 makes an “Extremum Error”, misidentifying 2008 as the peak year instead of the correct year, 2007, and a “Trend Direction Error”, incorrectly describing a downward trend as an upward trend. VL2NL makes a “Numerical Value Error”, inco… view at source ↗
Figure 2
Figure 2. Examples of time-series chart summary elements. We classify them into L1-L3, employ simple line diagrams to visually illustrate the meaning of these elements, and present example sentences containing specific elements. 3 PRELIMINARIES In this section, we derive the requirements for generating an accu￾rate and comprehensive summary of the time-series data chart. We summarize the key summary elements according to L1-L… view at source ↗
Figure 3
Figure 3. The frequency of different types of hallucinations in LLM [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The pipeline of ChartInsighter includes three steps: Brainstorming, Refining, and Self-consistency Test. In ChartInsighter, we input visualization specification and data table to initiate the analysis process. This is first handled by both Uni-Insighter and Multi-Insig…
Figure 5
Figure 5. Figure 5: The overview of ChartInsighter. Users can input a Vega-Lite specification and data table to generate a summary. By hovering over sentences containing data references, the corresponding portions in the chart are highlighted (a). Additionally, users can interact with the…
Figure 6
Figure 6. Figure 6: An example of our benchmark dataset. We carefully crafted gold summaries and labeled the hallucinations at sentence granularity for the summaries generated by each of the three models GPT-4, VL2NL, and ChartInsighter for each chart. To bridge this gap, we have introduc…
Figure 7
Figure 7. Figure 7: Evaluation results of algorithm performance. The boxplot displays the time spent on each step processing charts of different complex￾ity—Brainstorming, Refining, and Self-consistency Test—showing the range, median, and outliers for each phase. VL2NL is the highest. The…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SceneLoom: Communicating Data with Scene Context

    cs.HC 2025-07 conditional novelty 6.0 of 10

    SceneLoom guides a vision-language model through a design space derived from 54 data videos to generate chart-in-image designs aligned with user narrative intent.

  2. Data-to-Dashboard: Multi-Agent LLM Framework for Insightful Visualization in Enterprise Analytics

    cs.AI 2025-05 conditional novelty 5.0 of 10

    A multi-agent LLM system that detects the business domain of a raw dataset, generates domain-grounded insights, and renders them as charts, claims to beat single-prompt GPT-4o in insight quality.

Reference graph

Works this paper leans on

78 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [1]

    Federal reserve bank of st. louis. https://www.stlouisfed.org/,

    work page

  2. [2]

    https://www.statista.com/, 2024

    Statista. https://www.statista.com/, 2024. Accessed: 2024-12-31. 7

    work page 2024

  3. [3]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 3

    arXiv 2023

  4. [4]

    Anthropic. Claude. https://www.anthropic.com/claude, 2023. Ac- cessed: 2024-12-31. 3

    work page 2023

  5. [5]

    Y . Bai, H. Zhou, K. Zhao, J. Chen, J. Yu, and K. Wang. Transformer-opu: An fpga-based overlay processor for transformer networks. In 2023 IEEE 31st Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 221–221. IEEE, 2023. doi: 10.1109/ FCCM57271.2023.00049 9

    arXiv 2023

  6. [6]

    Battle, P

    L. Battle, P. Duan, Z. Miranda, D. Mukusheva, R. Chang, and M. Stone- braker. Beagle: Automated extraction and interpretation of visualizations from the web. CHI ’18, 8 pages, p. 1–8. Association for Computing Machinery, New York, NY , USA, 2018. doi:10.1145/3173574.3174168 1

    arXiv 2018

  7. [7]

    Burns, S

    R. Burns, S. Carberry, and S. Elzer. Modeling relative task effort for grouped bar charts. In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 31, 2009. 3

    work page 2009

  8. [8]

    W. Chen, X. Ma, X. Wang, and W. W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022. 2

    arXiv 2022

Show all 78 references
  1. [9]

    X. Chen, R. Aksitov, U. Alon, J. Ren, K. Xiao, P. Yin, S. Prakash, C. Sut- ton, X. Wang, and D. Zhou. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311, 2023. 5

    2023 arXiv

  2. [10]

    Cohen, M

    R. Cohen, M. Hamri, M. Geva, and A. Globerson. Lm vs lm: Detecting factual errors via cross examination. arXiv preprint arXiv:2305.13281,

    arXiv

  3. [11]

    V . Dibia. LIDA: A tool for automatic generation of grammar-agnostic visualizations and infographics using large language models. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pp. 113–126. Associ...

    2023

  4. [12]

    Dubey, A

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 3

    2024 arXiv

  5. [13]

    Frieder, L

    S. Frieder, L. Pinchetti, A. Chevalier, R.-R. Griffiths, T. Salvatori, T. Lukasiewicz, P. Petersen, and J. Berner. Mathematical capabilities of chatgpt. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, article no. 1205, 46...

    2024 doi

  6. [14]

    T.-c. Fu. A review on time series data mining.Engineering Applications of Artificial Intelligence, 24(1):164–181, 2011. doi: 10.1016/j.engappai.2010. 09.007 1

    2011 doi

  7. [15]

    J. Gao, X. Ding, B. Qin, and T. Liu. Is ChatGPT a good causal reasoner? a comprehensive evaluation. In H. Bouamor, J. Pino, and K. Bali, eds., Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 11111–11126. Association for Computational Linguistics, Sin...

    2023 doi

  8. [16]

    Greenbacker, P

    C. Greenbacker, P. Wu, S. Carberry, K. F. McCoy, and S. Elzer. Abstractive summarization of line graphs from popular media. In Proceedings of the Workshop on Automatic Summarization for Different Genres, Media, and Languages, pp. 41–48, 2011. doi: doi/10.5555/2018987.2018993 3

    2011

  9. [17]

    M. U. Hadi, Q. Al Tashi, A. Shah, R. Qureshi, A. Muneer, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, et al. Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. Authorea Preprints, 2024. 1

    2024

  10. [18]

    Y . Han, C. Zhang, X. Chen, X. Yang, Z. Wang, G. Yu, B. Fu, and H. Zhang. Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483, 2023. 9

    2023 arXiv

  11. [19]

    Y . He, S. Cao, Y . Shi, Q. Chen, K. Xu, and N. Cao. Leveraging large models for crafting narrative visualization: a survey. arXiv preprint arXiv:2401.14010, 2024. 2

    2024 arXiv

  12. [20]

    Hegarty and M.-A

    M. Hegarty and M.-A. Just. Constructing mental models of machines from text and diagrams. Journal of memory and language, 32(6):717–742,

  13. [21]

    Huang and K

    J. Huang and K. C.-C. Chang. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403, 2022. 2

    2022 arXiv

  14. [22]

    Huang, H

    K.-H. Huang, H. P. Chan, Y . R. Fung, H. Qiu, M. Zhou, S. Joty, S.- F. Chang, and H. Ji. From pixels to insights: A survey on automatic chart understanding in the era of large foundation models. arXiv preprint arXiv:2403.12027, 2024. 9

    2024 arXiv

  15. [23]

    S. O. U. Islam, I. Škrjanec, O. Dušek, and V . Demberg. Tackling halluci- nations in neural chart summarization. pp. 414–423, Sept. 2023. doi: 10. 18653/v1/2023.inlg-main.30 3

    2023

  16. [24]

    Kantharaj, R

    S. Kantharaj, R. T. Leong, X. Lin, A. Masry, M. Thakkar, E. Hoque, and S. Joty. Chart-to-text: A large-scale benchmark for chart summarization. In S. Muresan, P. Nakov, and A. Villavicencio, eds.,Proceedings of the 60th Annual Meeting of the Association for Computational Lingu...

    2022 doi

  17. [25]

    D. H. Kim, S. Choi, J. Kim, V . Setlur, and M. Agrawala. Emphasischecker: A tool for guiding chart and caption emphasis. IEEE Transactions on Visu- alization and Computer Graphics, 2023. doi: 10.1109/TVCG.2023.3327150 3, 6

    2023

  18. [26]

    D. H. Kim, V . Setlur, and M. Agrawala. Towards understanding how readers integrate charts and captions: A case study with line charts. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–11, 2021. doi: 10.1145/3411764.3445443 1, 3

    2021

  19. [27]

    Kim and K

    E. Kim and K. F. McCoy. Multimodal deep learning using images and text for information graphic classification. In Proceedings of the 20th Interna- tional ACM SIGACCESS Conference on Computers and Accessibility, pp. 143–148, 2018. doi: 10.1145/3234695.3236357 3

    2018

  20. [28]

    H.-K. Ko, H. Jeon, G. Park, D. H. Kim, N. W. Kim, J. Kim, and J. Seo. Natural language dataset generation framework for visualizations powered by large language models. In Proceedings of the CHI Conference on Human Factors in Computing Systems , pp. 1–22, 2024. doi: 10.1145/ 3...

    2024

  21. [29]

    Large, J

    A. Large, J. Beheshti, A. Breuleux, and A. Renaud. Multimedia and comprehension: The relationship among text, animation, and captions. Journal of the American society for information science, 46(5):340–347,

  22. [30]

    P.-M. Law, A. Endert, and J. Stasko. Characterizing automated data insights. In 2020 IEEE Visualization Conference (VIS) , pp. 171–175. IEEE, 2020. doi: 10.1109/VIS47514.2020.00041 3

    2020

  23. [31]

    H. Li, Y . Wang, S. Zhang, Y . Song, and H. Qu. Kg4vis: A knowledge graph-based approach for visualization recommendation. IEEE Transac- tions on Visualization and Computer Graphics, 28(1):195–205, 2021. doi: 10.1109/TVCG.2021.3114863 2

    2021

  24. [32]

    Z. Lin, C. Liu, R. Zhang, P. Gao, L. Qiu, H. Xiao, H. Qiu, C. Lin, W. Shao, K. Chen, et al. Sphinx: The joint mixing of weights, tasks, and vi- sual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575, 2023. 9

    2023 arXiv

  25. [33]

    M. Liu, D. Chen, Y . Li, G. Fang, and Y . Shen. Chartthinker: A contex- tual chain-of-thought approach to optimized chart summarization. arXiv preprint arXiv:2403.11236, 2024. 1, 2

    2024 arXiv

  26. [34]

    Lundgard and A

    A. Lundgard and A. Satyanarayan. Accessible visualization via natural language descriptions: A four-level model of semantic content. IEEE Transactions on Visualization and Computer Graphics, 28(1):1073–1083,

  27. [35]

    P. Ma, R. Ding, S. Han, and D. Zhang. Metainsight: Automatic discovery of structured knowledge for exploratory data analysis. In Proceedings of the 2021 international conference on management of data, pp. 1262–1274,

    2021

  28. [36]

    Masry, P

    A. Masry, P. Kavehzadeh, X. L. Do, E. Hoque, and S. Joty. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. arXiv preprint arXiv:2305.14761, 2023. 3

    2023 arXiv

  29. [37]

    F. Meng, W. Shao, Q. Lu, P. Gao, K. Zhang, Y . Qiao, and P. Luo. Chartassis- stant: A universal chart multimodal language model via chart-to-table pre- training and multitask instruction tuning.arXiv preprint arXiv:2401.02384,

    arXiv

  30. [38]

    Menghani

    G. Menghani. Efficient deep learning: A survey on making deep learning models smaller, faster, and better. ACM Computing Surveys, 55(12):1–37,

  31. [39]

    Narechania, A

    A. Narechania, A. Srinivasan, and J. Stasko. Nl4dv: A toolkit for gener- ating analytic specifications for data visualization from natural language queries. IEEE Transactions on Visualization and Computer Graphics , 27(2):369–379, 2021. doi: 10.1109/TVCG.2020.3030378 2

    2021

  32. [40]

    Obeid and E

    J. Obeid and E. Hoque. Chart-to-text: Generating natural language de- scriptions for charts by adapting the transformer model. arXiv preprint arXiv:2010.09142, 2020. 1, 3

    2010 arXiv

  33. [41]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. https://openai.com/index/ hello-gpt-4o/ , 2024. Accessed: 2024-12-31. 3

    2024

  34. [42]

    Peifeng, L

    L. Peifeng, L. Qian, X. Zhao, and B. Tao. Joint knowledge graph and large language model for fault diagnosis and its application in aviation assembly. IEEE Transactions on Industrial Informatics, 2024. doi: 10.1109/TII.2024. 3366977 9

    2024 doi

  35. [43]

    Pew research center.https://www.pewresearch

    Pew Research Center. Pew research center.https://www.pewresearch. org/, 2024. Accessed: 2024-12-31. 3

    2024

  36. [44]

    G. C. Project. Global carbon budget. https://globalcarbonbudget. org/, 2024. Accessed: 2024-12-31. 9

    2024

  37. [45]

    Roser, H

    M. Roser, H. Ritchie, and E. Ortiz-Ospina. Our world in data. https: //ourworldindata.org/, 2024. Accessed: 2024-12-31. 3, 7

    2024

  38. [46]

    Satyanarayan, D

    A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer. Vega-lite: A grammar of interactive graphics. IEEE Transactions on Visualization and Computer Graphics, 23(1):341–350, 2016. doi: 10.1109/TVCG.2016. 2599030 3

    2016 doi

  39. [47]

    Schmidt, N

    L. Schmidt, N. Goodman, D. Barner, B. Edu, and J. Tenenbaum. How tall is tall? compositionality, statistics, and gradable adjectives. 09 2010. doi: uc/item/8ft3x4c7 4, 9

    2010

  40. [48]

    Setlur, M

    V . Setlur, M. Tory, and A. Djalali. Inferencing underspecified natural language utterances in visual analysis. In Proceedings of the 24th interna- tional conference on intelligent user interfaces, pp. 40–51, 2019. doi: 10. 1145/3301275.3302270 2

    2019

  41. [49]

    Y . Shi, B. Chen, Y . Chen, Z. Jin, K. Xu, X. Jiao, T. Gao, and N. Cao. Supporting guided exploratory visual analysis on time series data with reinforcement learning. IEEE Transactions on Visualization and Computer Graphics, 2023. doi: 10.1109/TVCG.2023.3327200 3

    2023

  42. [50]

    Singh and A

    S. Singh and A. Yassine. Big data mining of energy time series for behav- ioral analytics and energy consumption forecasting. Energies, 11(2):452,

  43. [51]

    Sultanum and A

    N. Sultanum and A. Srinivasan. Datatales: Investigating the use of large language models for authoring data-driven articles. In 2023 IEEE Visu- alization and Visual Analytics (VIS), pp. 231–235. IEEE, 2023. doi: 10. 1109/VIS54172.2023.00055 1, 2, 6

    2023

  44. [52]

    B. J. Tang, A. Boggust, and A. Satyanarayan. VisText: A Benchmark for Semantically Rich Chart Captioning. In The Annual Meeting of the Association for Computational Linguistics (ACL), 2023. 1, 2, 3, 6, 9

    2023

  45. [53]

    Y . Tian, W. Cui, D. Deng, X. Yi, Y . Yang, H. Zhang, and Y . Wu. Chartgpt: Leveraging llms to generate charts from abstract natural language. IEEE Transactions on Visualization and Computer Graphics , 2024. doi: 10. 1109/TVCG.2024.3368621 2

    2024

  46. [54]

    Vega-altair: Declarative visualization in python

    Vega-Altair Developers. Vega-altair: Declarative visualization in python. https://altair-viz.github.io/, 2024. Accessed: 2024-12-31. 7

    2024

  47. [55]

    L. Wang, S. Zhang, Y . Wang, E.-P. Lim, and Y . Wang. LLM4Vis: Ex- plainable visualization recommendation using ChatGPT. In M. Wang and I. Zitouni, eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 675–692. Associ...

    2023

  48. [56]

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. 2

    2022 arXiv

  49. [57]

    Y . Wang, M. Perry, D. Whitlock, and J. W. Sutherland. Detecting anoma- lies in time series data from a manufacturing system using recurrent neural networks. Journal of Manufacturing Systems, 62:823–834, 2022. doi: 10. 1016/j.jmsy.2020.12.007 1

    2022

  50. [58]

    Z. Wang, S. Mao, W. Wu, T. Ge, F. Wei, and H. Ji. Unleashing the emer- gent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. In K. Duh, H. Gomez, and S. Bethard, eds., Proceedings of the 2024 Conference of the North Ame...

    2024

  51. [59]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824–24837, 2022. doi: doi/10.5555/3600270.3602070 2

    2022

  52. [60]

    R. Xia, B. Zhang, H. Ye, X. Yan, Q. Liu, H. Zhou, Z. Chen, M. Dou, B. Shi, J. Yan, et al. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12185,

    arXiv

  53. [61]

    Xu and E

    Z. Xu and E. Wall. Exploring the Capability of LLMs in Performing Low- Level Visual Analytic Tasks on SVG Data Visualizations . In2024 IEEE Visualization and Visual Analytics (VIS), pp. 126–130. IEEE Computer Society, Los Alamitos, CA, USA, Oct. 2024. doi: 10.1109/VIS55277.202...

    2024

  54. [62]

    W. Yang, M. Liu, Z. Wang, and S. Liu. Foundation models meet visual- izations: Challenges and opportunities. Computational Visual Media, pp. 1–26, 2024. doi: 10.1007/s41095-023-0393-x 2

    2024 doi

  55. [63]

    Y . Ye, J. Hao, Y . Hou, Z. Wang, S. Xiao, Y . Luo, and W. Zeng. Gener- ative ai for visualization: State of the art and future directions. Visual Informatics, 8(2):43–66, 2024. doi: 10.1016/j.visinf.2024.04.003 2

    2024 doi

  56. [64]

    Z. Yuan, H. Yuan, C. Tan, W. Wang, and S. Huang. How well do large language models perform in arithmetic tasks? arXiv preprint arXiv:2304.02015, 2023. 2

    2023 arXiv

  57. [65]

    Zhang, A

    L. Zhang, A. Hu, H. Xu, M. Yan, Y . Xu, Q. Jin, J. Zhang, and F. Huang. TinyChart: Efficient chart understanding with program-of-thoughts learn- ing and visual token merging. In Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, eds., Proceedings of the 2024 Conference on Empirical M...

    2024 doi

  58. [66]

    Zhang, H

    S. Zhang, H. Li, H. Qu, and Y . Wang. Adavis: Adaptive and explainable visualization recommendation for tabular data. IEEE Transactions on Visualization and Computer Graphics, 30(9):5923–5938, 2023. doi: 10. 1109/TVCG.2023.3316469 2

    2023

  59. [67]

    Zhang, Y

    W. Zhang, Y . Shen, L. Wu, Q. Peng, J. Wang, Y . Zhuang, and W. Lu. Self-contrast: Better reflection through inconsistent solving perspectives. In L.-W. Ku, A. Martins, and V . Srikumar, eds.,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisti...

    2024 doi

  60. [68]

    Y . Zhao, J. Wang, L. Xiang, X. Zhang, Z. Guo, C. Turkay, Y . Zhang, and S. Chen. Lightva: Lightweight visual analytics with llm agent-based task planning and execution. IEEE Transactions on Visualization and Computer Graphics, pp. 1–13, 2024. doi: 10.1109/TVCG.2024.3496112 2

    2024

  61. [69]

    Y . Zhao, Y . Zhang, Y . Zhang, X. Zhao, J. Wang, Z. Shao, C. Turkay, and S. Chen. Leva: Using large language models to enhance visual analytics. IEEE Transactions on Visualization and Computer Graphics, pp. 1–17,

  62. [70]

    Zheng, J

    S. Zheng, J. Huang, and K. C.-C. Chang. Why does chatgpt fall short in answering questions faithfully. arXiv preprint arXiv:2304.10513, 2023. 2

    2023 arXiv

  63. [77]

    doi: 10.1109/TVCG.2024.3368060 2

    2024

  64. [1993]

    doi: 10.1006/jmla.1993.1036 1

    1993

  65. [1995]

    doi: 10.1002/(SICI)1097-4571(199506)46:5<340::AID-ASI5>3.0.CO;2-S 1

    doi

  66. [2018]

    doi: 10.3390/en11020452 1

    doi

  67. [2021]

    doi: 10.1145/3448016.3457267 3

  68. [2022]

    doi: 10.1109/TVCG.2021.3114770 2, 3

    2021

  69. [2023]

    doi: 10.1145/3578938 9

    doi

  70. [2024]

    Accessed: 2024-12-31. 3, 7

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.