Pith. sign in

REVIEW 3 major objections 5 minor 57 references

ChartLens: Fine-grained Visual Attribution in Charts

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces ChartLens, a post-hoc visual attribution method that grounds chart-related answers in the specific bars, lines, and pie slices that support them, and claims a 26-66% improvement over existing baselines on a new…

desk verdict A genuinely new task and a useful benchmark, but the headline numbers don't match the tables and the benchmark has a same-model bias risk. read the letter →

arxiv 2505.19360 v1 pith:43EOY5UR submitted 2025-05-25 cs.CL

Manan Suri , Puneet Mathur , Nedim Lipka , Franck Dernoncourt , Ryan A. Rossi , Dinesh Manocha This is my paper
classification cs.CL
keywords chartattributionfine-grainedvisualset-of-markspromptingmultimodallargelanguagemodelshallucinationmitigationsegmentationgroundingbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that chart answers from multimodal language models can be verified by attributing each answer to the exact chart elements supporting it, and that a segmentation-plus-prompting pipeline can do this reliably. It introduces ChartLens, which detects and labels bars, pie sectors, and line segments, then asks a multimodal LLM to point to the labels that justify the answer. To measure this, it builds ChartVA-Eval, a benchmark of over 1,200 chart-question-answer triples with fine-grained attribution annotations drawn from synthetic and real-world sources. The paper reports that ChartLens improves fine-grained attribution by 26-66% over zero-shot GPT-4o, Kosmos-2, and LISA. If true, this offers a plug-and-play verification layer for chart-VQA systems that does not require retraining the underlying model.

What carries the argument

The mechanism that carries the argument is set-of-marks prompting coupled with segmentation-based mark generation. Segmentation turns a chart into discrete candidate objects: bars and pie sectors emerge from heuristic preprocessing (thresholding, contour extraction, radial unrolling) refined by the Segment Anything Model, while lines are extracted by LineFormer and divided into segments. Each candidate receives an alphanumeric mark overlaid on the image. The multimodal LLM is then given the marked chart plus a prompt instructing it to validate the answer and identify the specific marks that substantiate it. This makes attribution a selection problem over labeled visual anchors rather than a free-form localization problem.

What would settle it

Take a random sample of ChartVA-Eval and have new human annotators mark the ground truth from scratch, enforcing precision by deleting any element that does not directly support the answer; recompute ChartLens's F1 against these revised sets. If the 26-66% margin over baselines shrinks or disappears, the headline improvement is partly an artifact of annotation bias.

Watch

Extended reading notes

Core claim

The central claim is that post-hoc fine-grained visual attribution for charts can be solved by a two-stage pipeline: segment the chart into candidate elements, then use set-of-marks prompting, where candidate regions are overlaid with alphanumeric labels, so that a multimodal LLM selects the elements that validate the answer. ChartLens instantiates this pipeline: bars are found by Otsu thresholding and contour analysis refined with SAM; pie sectors by radial unrolling and edge detection refined with SAM; and lines by LineFormer, split into equally spaced segments. The MLLM is prompted with a marked chart, a chain-of-thought instruction to first validate the answer and then name the marked elements that support it. Against zero-shot GPT-4o bounding boxes, Kosmos-2, and LISA, the paper reports F1 of 69.28, 34.65, and 64.14 on bar-chart splits of AITQA, PlotQA, and ChartQA respectively, and 48.56 on pie charts, with line-chart detection rates of 52-78% at drastically reduced chart-area coverage.

Load-bearing premise

The ground-truth attributions in ChartVA-Eval are reliable: they were drafted by GPT-4o from data tables and human-checked only for relevance and completeness, not for precision, so the benchmark may contain extra elements and may favor whatever GPT-4o treats as supporting evidence.

Editorial extensions

If this is right

  • ChartLens can serve as a plug-and-play verification layer for any chart question-answering MLLM: responses are checkable by looking at which marked elements support the answer.
  • The ChartVA-Eval benchmark provides a standardized way to measure attribution quality, so future methods can be compared on grounding rather than only answer accuracy.
  • If an answer has no supporting marks, the pipeline flags it as likely inconsistent with the chart, enabling hallucination detection.
  • The line-chart metrics (detection rate and area covered) make precision explicit for continuous chart elements, where bounding-box overlap is not well-defined.
  • Because segmentation is modular, improving the segmentation stage should improve attribution without changing the prompting stage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's dependence on GPT-4o both as annotator and as attribution engine means the reported gains could partly reflect self-consistency; a re-annotation study with humans enforcing precision would measure that bias.
  • A natural extension is to use attribution sets as corrective feedback: when the selected marks do not support the answer, feed that signal back to the MLLM to revise its response, extending post-hoc attribution to hallucination repair.
  • The benchmark currently ignores textual chart components such as titles, axis labels, and legends; extending attribution to these textual marks could improve real-world applicability and is explicitly left for future work.
  • The segmentation bottleneck suggests that attribution quality will track advances in chart element segmentation; replacing SAM and LineFormer with stronger segmenters is a direct way to push the reported numbers higher.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Post-Hoc Fine-grained Visual Attribution for Charts, a task that grounds a chart-associated textual response to the specific visual elements (bars, lines, pie sectors) that support it. To address the task, the authors propose ChartLens, a pipeline that segments chart elements using heuristic methods, SAM, and LineFormer, then applies set-of-marks prompting to ChatGPT-4o to generate attributions. They also introduce ChartVA-Eval, a benchmark of 1,244 queries built from MATSA-AITQA, PlotQA, and ChartQA, with human-verified attribution annotations. Experiments compare ChartLens against zero-shot ChatGPT-4o, Kosmos-2, and LISA, reporting substantially higher F1 scores on bar and pie charts and better precision-recall tradeoffs on line charts. The abstract and conclusion claim a 26–66% improvement over baselines.

Significance. If the results are valid, the paper makes a useful contribution: it formalizes a practically important task (chart attribution for hallucination mitigation), releases a new benchmark with real-world and synthetic charts, and demonstrates a modular method that combines segmentation with set-of-marks prompting. The qualitative examples in Figure 3 show clear localization improvements over the baselines, and the paper explicitly limits the method's scope to visual elements, which is honest. The main significance is conditional, however, on resolving the mismatch between the headline improvement figure and the reported tables, and on addressing the risk that the benchmark's ground truth is biased by the same model family used in the proposed method. The benchmark itself, if properly validated, would be a valuable resource for evaluating chart grounding.

major comments (3)
  1. [Abstract, §1, §7] The headline claim of a 26–66% improvement is not supported by the reported tables. Relative F1 improvements of ChartLens over the best baseline are approximately 204% on ChartVA-AITQA bars (69.28 vs 22.77), 950% on ChartVA-PlotQA bars (34.65 vs 3.30), 728% on ChartVA-ChartQA bars (64.14 vs 7.75), and 315% on ChartVA-ChartQA pies (48.56 vs 11.70). Absolute F1 point gains range from 31 to 56 points. No subset of Tables 2–4 yields the claimed 26–66% range. Please specify how this number was computed or correct it, since it is the central quantitative claim of the paper.
  2. [§4.2, §6.3, §3] The annotation protocol in §4.2 states that GPT-4o generated initial attribution annotations for PlotQA and ChartQA, and human annotators reviewed only Relevance and Completeness, not Precision, even though Precision is the third criterion in the problem statement (§3). ChartLens's attribution MLLM is ChatGPT-4o (§6.3). This creates a same-model bias risk: ground truth may reflect GPT-4o's notion of which chart elements support an answer, rewarding ChartLens for agreeing with the annotation model while disadvantaging non-GPT-4o baselines (Kosmos-2, LISA) for reasons unrelated to attribution quality. This affects 943 of the 1,244 benchmark queries (§4.1, Table 1). Please add a precision-oriented human verification step, report inter-annotator agreement on precision, or provide evidence that this bias does not affect the ranking.
  3. [§7, Table 3] The statement in §7 that ChartLens reduces Chart% area by approximately 3–50 times is contradicted by Table 3. On PlotQA, ChartLens has a higher Chart Ar (9.98%) than zero-shot ChatGPT-4o (8.63%), and on AITQA the reduction is only about 1.55× (1.25% vs 1.94%). Only the ChartQA row (5.34% vs 1.15%) approaches the claimed reduction. Please correct this statement with the actual ratios or clarify which baselines and datasets support the 3–50× claim.
minor comments (5)
  1. [Table 3] The LISA rows for ChartVA-PlotQA and ChartVA-ChartQA are identical (Detection 50.21 and Chart Ar 40.92 for both), which appears to be a copy-paste error. Please verify the entries.
  2. [§4.2] The sentence 'This process ensured high-quality and precise attribution annotations' overstates what was checked, since the protocol assessed only Relevance and Completeness; the word 'precise' should be removed unless Precision is added to the verification criteria.
  3. [Abstract footnote] The abstract contains a superscript '1' after the '26–66%' claim but no corresponding footnote or link in the main text; if this refers to code/data, please provide the repository URL.
  4. [Throughout] Several typos and formatting issues should be corrected: 'methods that leverages' in §1 should be 'method that leverages'; 'LLms' in §5.2 should be 'LLMs'; 'O(104)' in §4.1 should be '10^4'; and the dataset name 'ChartV A-Eval' is inconsistently spaced.
  5. [§6.2] For line charts, Detection Rate and Average Area are reported separately without a combined metric, making it difficult to compare line-chart performance with the P/R/F1 results for bars and pies; consider reporting a single precision-recall composite or an F1-style score.

Circularity Check

1 steps flagged · score 4.0 of 10

Partially self-referential benchmark: GPT-4o generated the PlotQA/ChartQA target attributions while ChatGPT-4o powers ChartLens, so the headline 26-66% gain is partly a measure of agreement with the annotation model.

  1. other [Section 4.2 (Attribution Annotation) and Section 6.3 (Implementation Details)]
    "We utilized GPT-4o to generate initial annotations by leveraging the underlying data tables, questions, and answers. ... annotators reviewed the rendered bounding boxes on the charts and assessed the annotations based on two criteria: (1) Relevance ... and (2) Completeness ... The base multimodal language model (MLLM) for ChartLens is ChatGPT-4o, which is used for zero-shot bounding box detection and attribution tasks."

    The target sets G in the F1 evaluation are produced by GPT-4o from data tables, while the evaluated attribution engine is ChatGPT-4o. Any systematic preference of GPT-4o about which chart elements support a given answer is therefore baked into both the ground truth and the predictor; the benchmark measures agreement with GPT-4o's attribution choices rather than an independently fixed visual-evidence set. The human pass checked only Relevance and Completeness, not the third stated criterion of Precision, so off-target labels generated by GPT-4o remain in G and count favorably when ChartLens reproduces them.

full rationale

The core method chain is otherwise self-contained: mark generation uses classical heuristics, SAM, and LineFormer; attribution uses set-of-marks prompting with ChatGPT-4o; and evaluation uses IoU-matched F1 plus line-chart area metrics. None of these components is fitted to the benchmark, and no parameter is fitted and then renamed as a prediction. The load-bearing circularity risk sits in benchmark construction: Section 4.2's GPT-4o-generated labels for PlotQA and ChartQA become the ground truth for a method whose attribution MLLM is ChatGPT-4o (Section 6.3), and the human validation omitted Precision, the third criterion in the problem statement. This makes the claimed '26-66% improvement' partly a self-consistency check with GPT-4o rather than a fully independent attribution measurement. The issue is mitigated, though not eliminated, by the fact that the zero-shot GPT-4o baseline also uses GPT-4o and still scores far below ChartLens, so the segmentation-plus-prompting pipeline contributes real behavior; still, non-GPT-4o baselines (Kosmos-2, LISA) are disadvantaged by the label-generation bias. Separately, and as a correctness concern rather than circularity, the reported tables do not reproduce the abstract's '26-66%' range: Table 2 relative F1 gains are roughly 204% to 950%, Table 4 gives about 315%, and Table 3 line-detection gains over the GPT-4o baseline are 223% to 2195%, with an apparent copy error in LISA's PlotQA and ChartQA rows. The MATSA citation is an author self-citation, but it is used as data provenance for the AITQA subset, not as a forced uniqueness theorem or a derivational premise, so it does not independently raise the score. Overall, there is one genuine partial circularity in the benchmark loop, giving a score of 4 rather than 0 or 2.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the validity of the human-verified annotations, the reliability of SAM and LineFormer for chart element extraction, and the assumption that GPT-4o can ground responses using set-of-marks. No new physical or conceptual entities are introduced, but several free evaluation thresholds and segmentation parameters are left unspecified.

free parameters (4)
  • IoU matching threshold = 0.9
    In Section 6.2, detected regions are matched to ground truth only if IoU is at least 0.9. This threshold is chosen by hand, and no sensitivity analysis is provided, so it directly affects all reported precision, recall, and F1 numbers for bars and pies.
  • SAM point sample count = not specified
    Section 5.1 says n points are sampled from each detected element to prompt SAM, but n is never given. The choice affects mask quality and the resulting attribution marks.
  • Line segment spacing = not specified
    Section 5.1 states lines are divided into equally spaced segments along their domain extent, but the segment length is unspecified. This controls the granularity of line chart marks and the line chart evaluation.
  • Contour filtering thresholds = not specified
    Section 5.1 mentions filtering contours by solidity and area thresholds to retain well-defined bars, but the threshold values are not reported, making the preprocessing hard to reproduce exactly.
assumptions (5)
  • domain assumption Ground-truth attributions in ChartVA-Eval are correct and unbiased.
    Section 4.2 uses GPT-4o to generate initial annotations and humans to verify relevance and completeness only, not precision. The evaluation scores against these annotations, so any bias or error in them directly affects the conclusions.
  • domain assumption SAM produces masks that accurately enclose chart elements.
    Section 5.1 relies on SAM to refine heuristic detections and to suppress background elements such as grid lines. If SAM masks are imperfect, the attribution marks may be misaligned with actual chart elements.
  • domain assumption LineFormer reliably extracts lines from chart images.
    Section 5.1 uses LineFormer for line segmentation and then splits lines into segments. If LineFormer misses or adds lines, the set-of-marks candidates for line charts will be incomplete or noisy.
  • domain assumption GPT-4o with set-of-marks prompting can perform accurate visual grounding on charts.
    Section 5.2 depends on the multimodal model correctly interpreting labeled marks and selecting the right ones. The entire method is built around this capability, which is taken as given rather than demonstrated independently.
  • domain assumption The ChartVA-Eval benchmark is representative of real-world chart attribution scenarios.
    Section 4.1 draws on three datasets, including synthetic charts from MATSA-AITQA and PlotQA and real charts from ChartQA. The conclusions are framed broadly, but the benchmark may not capture all chart types, styles, and question complexities that occur in practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ChartLens: Fine-grained Visual Attribution in Charts." pith.science (2026). https://pith.science/paper/43EOY5UR

@misc{pith2026250519360,
  author       = {Pith},
  title        = {Pith review of: ChartLens: Fine-grained Visual Attribution in Charts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/43EOY5UR}},
  note         = {Machine review of arXiv:2505.19360}
}
read the original abstract

The growing capabilities of multimodal large language models (MLLMs) have advanced tasks like chart understanding. However, these models often suffer from hallucinations, where generated text sequences conflict with the provided visual data. To address this, we introduce Post-Hoc Visual Attribution for Charts, which identifies fine-grained chart elements that validate a given chart-associated response. We propose ChartLens, a novel chart attribution algorithm that uses segmentation-based techniques to identify chart objects and employs set-of-marks prompting with MLLMs for fine-grained visual attribution. Additionally, we present ChartVA-Eval, a benchmark with synthetic and real-world charts from diverse domains like finance, policy, and economics, featuring fine-grained attribution annotations. Our evaluations show that ChartLens improves fine-grained attributions by 26-66%.

Figures

Figures reproduced from arXiv: 2505.19360 by the authors.

Figure 1
Figure 1. We introduce the task of visual attribution for [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. ChartLens: ➊ Chart elements, such as bars and pie sectors, are extracted through heuristic-guided methods and refined using SAM, while lines are segmented using Lineformer. ➋ The segmented elements are then marked, labeled, and used to prompt multimodal LLMs, enabling fine-grained attribution by grounding textual responses to visual regions. 5 ChartLens ChartLens ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison of our ChartLens with the baselines. ChartLens is able to effectively localize [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of annotation guidelines provided to annotators for ensuring accurate and consistent visual [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: The design decision option space for MATSA synthetic charts, illustrating the various configurable [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 15 canonical work pages

  1. [1]

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930

    arXiv 2024

  2. [2]

    Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. 2023. Complex claim verification with evidence retrieved in the wild. arXiv preprint arXiv:2305.11859

    arXiv 2023

  3. [3]

    Tianyu Cui, Yanling Wang, Chuanpu Fu, Yong Xiao, Sijia Li, Xinhao Deng, Yunpeng Liu, Qinglin Zhang, Ziyi Qiu, Peiyang Li, Zhixing Tan, Junwu Xiong, Xinyu Kong, Zujie Wen, Ke Xu, and Qi Li. 2024. https://arxiv.org/abs/2401.05778 Risk taxonomy, mitigation, and assessment benchmarks of large language model systems . Preprint, arXiv:2401.05778

    arXiv 2024

  4. [4]

    Dr Ossama Embarak and Ossama Embarak. 2018. The importance of data visualization in business intelligence. Data analysis and visualization using python: analyze data to create visualizations for BI systems, pages 85--124

    work page 2018

  5. [5]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling large language models to generate text with citations. arXiv preprint arXiv:2305.14627

    arXiv 2023

  6. [6]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. 2024. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    work page 2024

  7. [7]

    Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483

    arXiv 2023

  8. [8]

    Kieran Healy. 2024. Data visualization: a practical introduction. Princeton University Press

    work page 2024

Show all 57 references
  1. [9]

    Ting-Yao Hsu, C Lee Giles, and Ting-Hao'Kenneth' Huang. 2021. Scicap: Generating captions for scientific figures. arXiv preprint arXiv:2110.11624

    2021 arXiv

  2. [10]

    Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403

    2022 arXiv

  3. [11]

    Wen Huang, Hongbin Liu, Minxin Guo, and Neil Zhenqiang Gong. 2024. Visual hallucinations of multi-modal large language models. arXiv preprint arXiv:2402.14683

    2024 arXiv

  4. [12]

    Siqing Huo, Negar Arabzadeh, and Charles Clarke. 2023. Retrieving supporting evidence for generative question answering. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, pages 11--20

    2023

  5. [13]

    Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018. Dvqa: Understanding data visualizations via question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5648--5656

    2018

  6. [14]

    Kushal Kafle, Robik Shrestha, Scott Cohen, Brian Price, and Christopher Kanan. 2020. Answering questions about data visualizations using efficient bimodal fusion. In Proceedings of the IEEE/CVF Winter conference on applications of computer vision, pages 1498--1507

    2020

  7. [15]

    Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, \'A kos K \'a d \'a r, Adam Trischler, and Yoshua Bengio. 2017. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300

    2017 arXiv

  8. [16]

    Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022. Chart-to-text: A large-scale benchmark for chart summarization. arXiv preprint arXiv:2203.06486

    2022 arXiv

  9. [17]

    Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan, and Soumen Chakrabarti. 2021. https://arxiv.org/abs/2106.12944 Ait-qa: Question answering dataset over comple...

    2021 arXiv

  10. [18]

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. Ocr-free document understanding transformer. In European Conference on Computer Vision, pages 498--517. Springer

    2022

  11. [19]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015--4026

    2023

  12. [20]

    Ashutosh Kumar, Sagarika Singh, Shiv Vignesh Murty, and Swathy Ragupathy. 2024. https://arxiv.org/abs/2401.12273 The ethics of interaction: Mitigating security threats in llms . Preprint, arXiv:2401.12273

    2024 arXiv

  13. [21]

    Jay Lal, Aditya Mitkari, Mahesh Bhosale, and David Doermann. 2023. Lineformer: Line chart data extraction using instance segmentation. In International Conference on Document Analysis and Recognition, pages 387--400. Springer

    2023

  14. [22]

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023 a . Pix2struct: Screenshot parsing as pretraining for visual language understanding. In International Co...

    2023

  15. [23]

    Seongyun Lee, Sue Hyun Park, Yongrae Jo, and Minjoon Seo. 2023 b . Volcano: mitigating multimodal hallucination through self-feedback guided revision. arXiv preprint arXiv:2311.07362

    2023 arXiv

  16. [24]

    Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, and Min Zhang. 2023 a . A survey of large language models attribution. arXiv preprint arXiv:2311.03731

    2023 arXiv

  17. [25]

    Xiaonan Li, Changtai Zhu, Linyang Li, Zhangyue Yin, Tianxiang Sun, and Xipeng Qiu. 2023 b . Llatrieval: Llm-verified retrieval for verifiable generation. arXiv preprint arXiv:2311.07838

    2023 arXiv

  18. [26]

    Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. 2022 a . Deplot: One-shot visual language reasoning by plot-to-table translation. arXiv preprint arXiv:2212.10505

    2022 arXiv

  19. [27]

    Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Martin Eisenschlos. 2022 b . Matcha: Enhancing visual language pretraining with math reasoning and chart derendering. arXiv preprint arXiv:2212.09662

    2022 arXiv

  20. [28]

    Quan Khanh Luu, Xiyu Deng, Anh Van Ho, and Yorie Nakahira. 2024. https://arxiv.org/abs/2403.11863 Context-aware llm-based safe control against latent risks . Preprint, arXiv:2403.11863

    2024 arXiv

  21. [29]

    Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. 2023. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. arXiv preprint arXiv:2305.14761

    2023 arXiv

  22. [30]

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244

    2022 arXiv

  23. [31]

    Puneet Mathur, Alexa Siu, Nedim Lipka, and Tong Sun. 2024. Matsa: Multi-agent table structure attribution. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 250--258

    2024

  24. [32]

    Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. 2024. Chartassisstant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. arXiv preprint arXiv:2401.02384

    2024 arXiv

  25. [33]

    Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. 2022. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147

    2022 arXiv

  26. [34]

    Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1527--1536

    2020

  27. [35]

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332

    2021 arXiv

  28. [36]

    Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry \'s ci \'n ski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, et al. 2022. Fetaqa: Free-form table question answering. Transactions of the Association for Computational Linguistics, 10:35--49

    2022

  29. [37]

    OpenAI . 2024. Hello, gpt-4o! https://openai.com/index/hello-gpt-4o/

    2024

  30. [38]

    Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. Totto: A controlled table-to-text generation dataset. arXiv preprint arXiv:2004.14373

    2020 arXiv

  31. [39]

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2023. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824

    2023 arXiv

  32. [40]

    Denis Peskoff and Brandon M Stewart. 2023. Credible without credit: Domain experts assess generative language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 427--438

    2023

  33. [41]

    Vipula Rawte, Amit Sheth, and Amitava Das. 2023. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922

    2023 arXiv

  34. [42]

    Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. 2017. A simple neural network module for relational reasoning. Advances in neural information processing systems, 30

    2017

  35. [43]

    Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pages 393--402

    2011

  36. [44]

    Hrituraj Singh and Sumit Shekhar. 2020. Stl-cqa: Structure-based transformers with localization and encoding for chart question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3275--3284

    2020

  37. [45]

    Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2022. Recitation-augmented language models. arXiv preprint arXiv:2210.01296

    2022 arXiv

  38. [46]

    Benny J Tang, Angie Boggust, and Arvind Satyanarayan. 2023. Vistext: A benchmark for semantically rich chart captioning. arXiv preprint arXiv:2307.05356

    2023 arXiv

  39. [47]

    Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. 2024. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. arXiv preprint arXiv:2406.18521

    2024 arXiv

  40. [48]

    Renqiu Xia, Bo Zhang, Haoyang Peng, Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Yu Qiao, and Junchi Yan. 2023. Structchart: Perception, structuring, reasoning for visual chart understanding. arXiv preprint arXiv:2309.11268

    2023 arXiv

  41. [49]

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441

    2023 arXiv

  42. [50]

    Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. 2024. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data, 18(6):1--32

    2024

  43. [51]

    Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia. 2023 a . Cognitive mirage: A review of hallucinations in large language models. arXiv preprint arXiv:2309.06794

    2023 arXiv

  44. [52]

    Xi Ye, Ruoxi Sun, Sercan \"O Arik, and Tomas Pfister. 2023 b . Effective large language model adaptation for improved grounding. arXiv preprint arXiv:2311.09533

    2023 arXiv

  45. [53]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045

    2023 arXiv

  46. [54]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. Siren's song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219

    2023 arXiv

  47. [55]

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754

    2023 arXiv

  48. [56]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.