Infogen: Generating Complex Statistical Infographics from Documents

Akash Ghosh; Aparna Garimella; Pritika Ramu; Sambaran Bandyopadhyay; Sriparna Saha

REVIEW 4 major objections 5 minor 28 references

Infogen: Generating Complex Statistical Infographics from Documents

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Infogen turns documents into multi-chart infographics, beating GPT-4o

desk verdict A genuinely new task and a first benchmark, but the benchmark's synthetic documents are chart paraphrases, so the SOTA numbers should not be interpreted as document-to-infographic conversion. read the letter →

arxiv 2507.20046 v1 pith:W5YTN4XI submitted 2025-07-26 cs.CL

Akash Ghosh , Aparna Garimella , Pritika Ramu , Sambaran Bandyopadhyay , Sriparna Saha This is my paper

classification cs.CL

keywords infographicgenerationtext-to-visualizationmetadatalargelanguagemodelsdirectpreferenceoptimizationdatasetcurationcodebenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Infogen, a two-stage system for generating complex statistical infographics — multi-chart visuals such as paired bar, line, and pie charts — directly from text-heavy documents. To train and test it, the authors build Infodat, the first benchmark dataset for this task, derived from 3,463 Pew infographics with machine-written input texts and human-verified structured metadata. The paper claims Infogen (large) outperforms both closed and open-source LLMs, reporting 74.69% sub-chart accuracy and 89.56% statistical accuracy. A sympathetic reader would care because prior work handled only single charts from structured data, while Infogen targets the harder planning problem of inferring chart layouts, data, and insights from unstructured prose.

What carries the argument

The central object is the infographic metadata: a structured JSON-like description that specifies the overall title, a summary insight, and, for every sub-chart, its type, axes, statistics, associated text, position relative to other sub-charts, text position, background, dimensions in pixels, fonts, and alignment. The paper's central hypothesis is that producing this intermediate representation first makes generation tractable: it turns the open-ended planning problem (how many charts, of what kind, saying what) into a structured prediction problem, and leaves the rendering to a code generator. The ranker LLM is the load-bearing piece of the metadata stage, because it decides which of three DPO-aligned models' outputs best matches the document, and the feedback-judge loop is the load-bearing piece of the code stage, because it catches misalignments such as overlapping text or wrong sub-chart counts.

What would settle it

Run Infogen on a held-out set of authentic text-heavy documents (for example, full Pew Research articles with the infographic images removed) and check whether the predicted sub-chart count, types, and statistics match the infographics that the original authors actually published; if accuracy drops toward the prompting baselines, the claimed state of the art does not transfer from machine-written chart descriptions to real documents.

Watch

Extended reading notes

Core claim

The central claim is that the text-to-infographic problem is best solved by decomposing it into metadata generation followed by code generation, rather than generating visuals directly from text. Infogen's metadata module fine-tunes Qwen-2, Llama-3, and Phi-3 with QLoRA, aligns them with DPO using synthetic preferences, and then uses a fine-tuned ranker LLM to select the most accurate metadata among the three model outputs. The code module employs a coder LLM and an iterative feedback-judge loop that refines Python code until it matches the metadata's sub-chart count, types, statistics, positions, and layout. On Infodat, Infogen (large) achieves the highest sub-chart accuracy (74.69%), statistical accuracy (89.56%), and lowest RSE (1.80) among all compared models, and human evaluation rates its infographics higher on readability, visual appeal, and data accuracy than GPT-4o and Phi-3+DPO baselines.

Load-bearing premise

The benchmark assumes that a GPT-4o-written description of an existing infographic is a faithful and leakage-free stand-in for a real text-heavy document, so the reported gains are measured against reconstructing chart metadata from machine-written chart paraphrases.

Editorial extensions

If this is right

Fine-tuned LLMs beat prompting on text-to-metadata: GPT-4o with BM25 clustering few-shot reaches 57.69% sub-chart accuracy, while Infogen (large) reaches 74.69%.
DPO with synthetic preferences reliably improves metadata quality across Llama-3, Phi-3, and Qwen-2; Phi-3 large with DPO jumps from 63.46% to 72.11% sub-chart accuracy.
A fine-tuned ranker outperforms in-context few-shot merging of multiple LLM outputs, so the framework's performance depends strongly on the ranker module.
The feedback-judge code loop corrects visual layout errors like overlapping text, producing higher human-evaluated readability, visual appeal, and data alignment scores.
Even the small version of Infogen (59.2% sub-chart accuracy) beats individual small models, suggesting the multi-model plus ranker design is robust to model size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported state of the art is measured on a benchmark whose input text is a GPT-4o paraphrase of an existing chart, not an authentic document; on real text-heavy reports the advantage may shrink unless metadata quality holds.
Because the ranker is trained on heuristic-verified outputs (sub-chart count and type checks), a richer preference dataset with human pairwise judgments could further reduce the roughly 25% sub-chart errors that remain.
A testable extension beyond the paper is to apply the same two-stage pipeline to healthcare or finance documents where the input text contains statistics that are not already laid out in an infographic, and measure whether sub-chart planning still succeeds.
The metadata schema itself could serve as an evaluation interface for other text-to-visualization systems, since it separates planning accuracy (chart count, types, statistics) from rendering quality.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing you should know: this paper defines a genuinely new task — turning a text document into a multi-chart infographic — and builds a first benchmark, Infodat, with 3,463 samples. That part is real and useful. The pipeline, metadata generation followed by code synthesis, is a sensible engineering combination of QLoRA fine-tuning, DPO, a learned ranker, and an LLM code-feedback loop. The human evaluation, though on a small sample, shows a preference for Infogen over GPT-4o few-shot.

The soft spots are in the benchmark construction, and the stress-test note is correct. The input 'documents' are not independent articles. They are GPT-4o descriptions of the very infographic that also produced the ground-truth metadata. The prompt explicitly asks the model to include all statistics and associated text so a human could reconstruct the chart. So the test measures reverse-engineering a clean paraphrase, not planning an infographic from a realistic text-heavy document. The average input is 185 words versus 341 words of metadata, which confirms the input is a dense, complete specification. That weakens the core claim that Infogen converts messy text into infographics.

A second issue is the ranker fairness. The ranker is trained with heuristics like 'verify the number of subcharts and correct types' — exactly the metrics used for Subchart Accuracy and Type Accuracy. That is a to some degree circular optimization loop. The ranker also selects among the three fine-tuned models' outputs, so performance gains could partly come from choosing the best oracle, not from true reasoning.

Third, no error bars or significance tests are reported even though the FAQ says the scores are means of three runs. Fourth, neither code nor data is released. For a new benchmark this is a real impediment.

None of these are fatal to the paper as a first step. The task is worth having, the dataset is non-trivial, and the pipeline shows a reasonable way to approach it. But the current version overstates what the benchmark actually measures. A serious referee should ask for a holdout of authentic articles, deconfounded ranker training, and at least standard deviations.

I'd send it to review — this deserves a full referee cycle, not a desk reject. But I'd expect heavy revision before acceptance.

Referee Report

4 major / 5 minor

Summary. The paper introduces Infodat, a benchmark of 3,463 text-to-infographic metadata pairs built from Pew infographic images, and Infogen, a two-stage system that first generates structured metadata with fine-tuned LLMs and a learned ranker, then converts the metadata to Python code through a coder-feedback loop. On Infodat, Infogen (large) reports Subchart Accuracy of 74.69% and Statistical Accuracy of 89.56%, outperforming prompting and fine-tuning baselines, and the paper claims state-of-the-art performance for text-to-statistical-infographic generation.

Significance. If the reported results are accepted, the paper would provide the first benchmark and a reasonably complete pipeline for a useful and underexplored task. The metadata schema is detailed, the metadata is human-verified, several automatic and human evaluation dimensions are introduced, and the baseline coverage is broad. However, the benchmark's input texts are synthetic, machine-written descriptions generated from the very infographics that define the ground truth, and the headline statistical metric sorts numbers before comparing them, discarding the contextual placement that matters in an infographic. The current evidence therefore does not yet establish the claimed capability of generating infographics from text-heavy documents. These issues are addressable with additional evaluation on authentic documents and with metric improvements, so the work is worth revising rather than rejecting.

major comments (4)

[Section 3 and Appendix A.8] The central 'document-to-infographic' claim is not supported by the benchmark as constructed. The input texts are synthesized by prompting GPT-4o with the 'Prompt for Detailed Chart Description', which asks for a description 'such that a human is able to reconstruct the chart based on the generated description' and explicitly instructs the model to mention all statistics and associated text while omitting chart type, section count, and terms like 'chart' or 'image'. The same infographic images are then used, with human verification, to produce the ground-truth metadata. Each test item is therefore a paraphrase-to-metadata reconstruction problem, not a document-to-infographic task. Table 1 confirms that the inputs are short (185.46 words on average) relative to the metadata (341.01 words) and to realistic text-heavy articles. The Abstract and Section 6.1 R1 should either be re-scoped to synthetic chart descriptions, or the authors should add an evaluation on authentic documents, such as original Pew article text or comparable sources, and report those results separately.
[Section 5.4 and Appendix A.5, Eq. (6)] Statistical Accuracy sorts all numerical values in the generated metadata and in the ground-truth metadata and then compares them sequentially. This discards the association between a number and its subchart, axis label, or surrounding text. A generated metadata that transposes statistics between subcharts, or places correct numbers under the wrong headings, can still receive a high Statistical Accuracy despite producing a misleading infographic. Since Statistical Accuracy is one of the two headline metrics in Table 2, the metric should be replaced or supplemented by a context-aware matching procedure that compares statistics within matched subcharts and under matched labels.
[Section 4.2(c) and Appendix A.4] The ranker is trained on a dataset whose labels are produced by heuristics that verify 'the number of sub-charts and ensuring the correct sub-chart types', which are exactly the criteria measured by Subchart Accuracy and Subchart Type Accuracy. Because the same ranker is then used to select among the DPO models on the test set, part of Infogen's reported gain may reflect optimization toward these evaluation heuristics rather than improvement in semantic correctness or visual quality. The authors should report ranker agreement with human preference judgments on a sample, or retrain the ranker with human labels, to show that the selection step improves genuinely meaningful quality and not just the automatic scores.
[Appendix A.1, FAQ-4 and Table 2] The FAQ states that the Infogen scores are the mean of three runs, but Table 2 reports single numbers without standard deviations, confidence intervals, or significance tests. The margins over the strongest baselines are often small, for example Infogen (large) versus phi3_qlora_large_dpo is 74.69 versus 72.11 for Subchart Accuracy and 89.56 versus 89.44 for Statistical Accuracy. Without variance estimates or significance tests, the claim that Infogen 'outperforms both closed and open-source LLMs' is not statistically established. Run-level results and significance tests, or at least bootstrap confidence intervals, should be reported for the main comparisons.

minor comments (5)

[Table 2 and text] The header 'GTP4o' contains a typo and should read 'GPT4o'; in Section 5.3 the text refers to 'Table 1' when the baseline results are actually shown in Table 2.
[Appendix A.6] In the metadata examples, Example 2's second subchart repeats the same statistics and associated text as the first subchart, and Example 3 appears to contain the same statistics as a single chart; this raises concerns about duplicate or noisy metadata in the benchmark and should be clarified or corrected.
[Section 5.1] The sentence 'We used LLAMA-3 (80GB) calls' likely should say 'LLaMA-3 70B on an A100 80GB GPU', since LLaMA-3 70B is the model used elsewhere; also 'unsloath' should be 'Unsloth'.
[Appendix A.1, FAQ-11] The answer to FAQ-11 contains garbled wording: 'of metadata is must to check if its satisfy the expected quality' should be rephrased as a complete sentence.
[Section 6.3 and Table 3] The human evaluation reports scores on a 1-5 scale for '35% of test samples' but does not state the number of annotators, the number of samples, or the variance of the scores; these details should be added so the reader can judge the reliability of the differences in Table 3.

Circularity Check

1 steps flagged · score 6.0 of 10

The Infodat input 'documents' are GPT-4o-written descriptions of the very infographics used to build the ground-truth metadata, so the reported SOTA largely measures reconstructing chart metadata from a chart-derived paraphrase rather than generating infographics from real documents.

self definitional [Section 3 (Input Text Document Synthesis; MetaData Synthesis) and Appendix A.8 (Prompt for Detailed Chart Description)]
"Can you give a detailed description of the attached chart in plain English, such that a human is able to reconstruct the chart based on the generated description? For each section, mention all the statistics and the associated text attached to it in the image, like the title or heading for that section. ... Write everything in a single passage, without sub-passages, so it feels like all information is provided in one continuous context."

The benchmark input and output are both derived from the same infographic image: GPT-4o produces the input text from the chart image with instructions to include every statistic and associated title/heading, while the ground-truth metadata is produced from the same image (with human verification). Thus each test sample's 'document' is a reconstruction-oriented paraphrase of the target infographic itself. A model trained and evaluated on Infodat is therefore learning to re-extract chart metadata from a complete chart description, not to plan an infographic from an independent text-heavy document. The central state-of-the-art claim (Table 2, R1) reduces to a self-referential reconstruction task by construction, despite the paper's assertion that leakage was prevented.

full rationale

The single load-bearing circularity is in dataset construction. The paper's own Appendix prompt defines the input text as a detailed, all-statistics-inclusive description of the target chart image, and the same image is the source of the ground-truth metadata. This makes the benchmark's input-output pairing self-referential: the input is generated from the output's visual source. Consequently, the abstract's and Section 6.1's claim of state-of-the-art 'text-to-statistical infographic generation' is not an independent test of document understanding; it is substantially a chart-paraprase-to-chart-metadata reconstruction. I do not score higher than 6 because the model must still infer chart types and counts (the prompt withholds those), so the numbers are not fully forced by definition, and because the paper contains additional components (code generation and human evaluation) that are not themselves circular. The ranker's heuristic training labels (subchart count and type) are supervised signals on training data rather than a fitted parameter renamed as a prediction, so I do not list that as a separate circular step. No load-bearing self-citations or imported uniqueness theorems are present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the synthetic benchmark (GPT-4o-written inputs matched with human-verified metadata from the same images), on the assumption that metadata quality transfers to infographic quality, and on two metric/algorithmic choices: sorted numeric comparison for statistical accuracy, and GPT-3.5 Turbo as the DPO reference model. None of these are independently validated outside the paper. Hyperparameters for DPO and the feedback loop are also chosen by hand.

free parameters (3)

DPO scaling factor beta = not reported
Appears in Eq. 2 (Section 4.2b) and controls the strength of preference optimization; its value is not given, so the reported gains from DPO cannot be reproduced or checked for sensitivity.
Maximum feedback iterations = 5
FAQ 6 (Appendix A.1) sets a cap of 5 coder-feedback cycles; chosen by hand and affects final infographic quality and inference cost.
Temperature for code and feedback LLM calls = 0.5
Section 5.1 sets temperature 0.5 for code and feedback module; a sampling choice that affects output variability and the reported qualitative results.

assumptions (4)

ad hoc to paper GPT-4o-generated input texts are faithful, leakage-free proxies for real documents
Section 3 'Input Text Document Synthesis' relies on GPT-4o chain-of-thought to rewrite infographic images as text without revealing chart counts; the validity of the whole benchmark depends on this generation being accurate and unbiased.
domain assumption Metadata quality is a valid proxy for final infographic quality
FAQ 14 (Appendix A.1) and Section 6.4 state the hypothesis that better metadata/code leads to better infographics; code generation is never evaluated on its own, so the headline SOTA claim implicitly assumes this transfer.
domain assumption Sorted numeric comparison is a meaningful measure of statistical accuracy
Appendix A.5 defines Statistical Accuracy by sorting numbers from generated and ground-truth metadata and comparing sequentially; this discards which chart or axis each number belongs to.
ad hoc to paper GPT-3.5 Turbo's token probabilities can serve as the reference policy in DPO
Eq. 2 (Section 4.2b) uses pi_GPT in the denominator as the reference model, but GPT-3.5 Turbo only produced preference rankings; standard DPO uses the SFT model as reference, and using a proprietary model's probabilities is an unvalidated modification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Infogen: Generating Complex Statistical Infographics from Documents." pith.science (2026). https://pith.science/paper/W5YTN4XI

@misc{pith2026250720046,
  author       = {Pith},
  title        = {Pith review of: Infogen: Generating Complex Statistical Infographics from Documents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W5YTN4XI}},
  note         = {Machine review of arXiv:2507.20046}
}

read the original abstract

Statistical infographics are powerful tools that simplify complex data into visually engaging and easy-to-understand formats. Despite advancements in AI, particularly with LLMs, existing efforts have been limited to generating simple charts, with no prior work addressing the creation of complex infographics from text-heavy documents that demand a deep understanding of the content. We address this gap by introducing the task of generating statistical infographics composed of multiple sub-charts (e.g., line, bar, pie) that are contextually accurate, insightful, and visually aligned. To achieve this, we define infographic metadata that includes its title and textual insights, along with sub-chart-specific details such as their corresponding data and alignment. We also present Infodat, the first benchmark dataset for text-to-infographic metadata generation, where each sample links a document to its metadata. We propose Infogen, a two-stage framework where fine-tuned LLMs first generate metadata, which is then converted into infographic code. Extensive evaluations on Infodat demonstrate that Infogen achieves state-of-the-art performance, outperforming both closed and open-source LLMs in text-to-statistical infographic generation.

Figures

Figures reproduced from arXiv: 2507.20046 by the authors.

**Figure 1.** Sample instance of text to complext statistical infographic task. The input is a textual document as shown in the left and the output is the corresponding statistical infographic in right. mainly focus on basic charts like bar or line graphs based on structured instructions. Also, they take data in the form of CSV and tabular formats. However, if the user provides just the information in the form of textual documen… view at source ↗

**Figure 2.** The various stages of generating the Infodat dataset are outlined as follows. We generated the metadata from the selected infographics using the pipeline illustrated in (a). Additionally, we identified the complex infographics by following the pipeline shown in (b). work in detail. 4.1 Problem Statement In our approach, we tackle the task of text-tocomplex statistical infographic generation using Infogen. The frame… view at source ↗

**Figure 3.** The architecture of our proposed framework, Infogen, consists of two main stages: the MetaData Generation Module and the Code Generation Module. The MetaData Generation Module is broken down into three steps: first, fine-tuning LLMs; second, post-training with synthetic data generated using DPO; and finally, using a fine-tuned LLM as a ranker to produce the final metadata. In the Code Generation Module, there are tw… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: The qualitative analysis of the generated infographics from GPT-4o, Phi3 (DPO), and Infogen reveals a clear advantage of Infogen. Notably, Infogen accurately identifies the correct number of sub-charts from the text and captures the alignment of the sub-charts perfectl…

**Figure 5.** Figure 5: This figure demonstrates the impact of the feedback module on improving infographic generation. On the [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 7.** Figure 7: Word Cloud Generated from InputText [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

**Figure 8.** Figure 8: Distribution of number of subcharts in Infodat A.8 Different Prompts Used for Infodat and Infogen are shown below . Infodat prompts are shown in blue and Infogen in green . Prompt for Detailed Chart Description You are an intelligent AI assistant that can thoroughly re…

**Figure 6.** Figure 6: Word Cloud Generated from Metadata [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 25 canonical work pages

[1]

top-left

What are the human annotation guidelines for synthetic metadata verification ? Ans: We instruct the human annotators to fol- low the below guidelines to verify the accuracy of metadata generated from images containing multi- ple sub-charts : a. Count the Total Number of Subcharts: Ensure the metadata correctly identifies the total number of subcharts in t...

work page
[2]

Matthew Sadiku, Adebowale E Shadare, Sarhan M Musa, Cajetan M Akujuobi, and Roy Perry

Direct preference optimization: Your language model is secretly a reward model.Advances in Neu- ral Information Processing Systems, 36. Matthew Sadiku, Adebowale E Shadare, Sarhan M Musa, Cajetan M Akujuobi, and Roy Perry. 2016. Data visualization.International Journal of En- gineering Research And Advanced Technology (IJERAT), 2(12):11–16. Pranab Sahoo, ...

arXiv 2016
[3]

LIDA explicitly transforms a CSV file into an infographic, convert- ing it into text internally, while ChartGPT uses a query and a CSV file to generate infographics step by step

Why ChartGPT and Lida are not taken into baselines ? Ans: ChartGPT and LIDA operate in a setup where both text and accompanying data tables (e.g., CSV files) are provided as inputs. LIDA explicitly transforms a CSV file into an infographic, convert- ing it into text internally, while ChartGPT uses a query and a CSV file to generate infographics step by st...

work page
[4]

Are the results statistically significant ? Ans: The results of Infogen both small and large are statistically significant The scores of them are the mean of three runs conducted

work page
[5]

For the final evaluation of the generated infographics, the inter- annotator agreement was measured using Cohen’s Kappa, yielding a score of 0.78

What are the annotator demographics and what was the inter-annotator agreement? Ans: The annotators comprised graduate interns with expertise in statistical infographics and prior experience with Pew/statistical data. For the final evaluation of the generated infographics, the inter- annotator agreement was measured using Cohen’s Kappa, yielding a score of 0.78

work page
[6]

yes," indicating that the coder module has successfully generated the final code. If the judge module returns

How does the loop between the coder mod- ule and feedback module end? Ans: The loop ends when the judge module returns a "yes," indicating that the coder module has successfully generated the final code. If the judge module returns "no," the process repeats, with a maximum of 5 iterations. 7.Can I see more examples of Dataset ? Ans: Please go through this...

work page
[8]

On avgInfogentakes 1.5 times more infer- ence time

Is inference time a bottleneck forInfogen? Ans:Yes, asInfogenis using both bunch of LLM agents in both metageneration and codegeneration phase . On avgInfogentakes 1.5 times more infer- ence time . So, performance versus inference trade- off is there. We believe we can reduce the inference time by using recent techniques like Speculative decoding but the ...

work page
[9]

Will the same framework work for other domains like Healthcare and Finance? Ans: We are confident that the same framework can be applied to data from other domains. How- ever, we believe that the training dataset should ideally include some domain-specific data before deploying it in new areas, as the metageneration phase relies on fine-tuned LLMs. Additi...

work page

Show all 28 references

[10]

What is the significance of this study com- pare to previous research works in this direc- tion? Ans: This work represents the first attempt at generating complex statistical infographics that consist of multiple subcharts, focusing specifically on testing the capability of la...
[11]

Ans: They are compensated according to our agreement, which adheres to the Government’s Minimum Wage guidelines

How the annotators and human verifiers compensated for their work ?of metadata is must to check if its satisfy the expected quality . Ans: They are compensated according to our agreement, which adheres to the Government’s Minimum Wage guidelines. 12 . Why were different models...
[12]

To facilitate broader usability, we provide detailed steps for extending the dataset, making it adaptable to various domains

Do you not feel the size ofInfodatdataset is rather small? Ans: In this work, we focus on incorporating infographics with multiple subcharts, deliberately excluding simpler ones to preserve the dataset’s complexity and ensure meaningful evaluations. To facilitate broader usabi...
[13]

Does the definition of infographics in the paper align with the past literature? Ans: In this study, we focus exclusively on the domain of statistical infographics, specifically tar- geting complex statistical infographics that consist of multiple subcharts, each with a distin...
[14]

However, we extensively assessed the quality of the infograph- ics generated from the code through human and qualitative evaluations

Is code generation evaluated indepen- dently? Ans: We acknowledge that code generation was not explicitly evaluated on its own. However, we extensively assessed the quality of the infograph- ics generated from the code through human and qualitative evaluations. Our hypothesis,...

2023
[15]

It or- ganizes subcharts within the infographic, specify- ing their relationships and context, while ensuring clarity and coherence

What aspects are covered by the meta- data, and what are excluded? Ans: The metadata captures structured details about infographics, including chart types (e.g., line charts), axis labels (e.g., months, percentages), sta- tistical data points, text annotations, layout infor- m...
[17]

• Examples are selected from cluster cen- troids, ensuring better alignment with the input text

GPT-4o_BM25_clustering_10shot , GPT- 4o_BM25_clustering_20shot: • These configurations enhance the few- shot prompting method by using BM25 embeddings to cluster similar examples. • Examples are selected from cluster cen- troids, ensuring better alignment with the input text
[18]

REFT_LLAMA,REFT_Phi3, REFT_Qwen2: • These are fine-tuned versions of popular LLMs (e.g., LLAMA, Phi3, and Qwen2) using the REFT (Representation Fine- Tuning) technique
[19]

• Thesmallversion has fewer parameters, while thelargeversion delivers better performance due to its larger capacity

LLAMA3_qlora_small , LLAMA3_qlora_large: • These baselines involve fine-tuning LLAMA3 models using the QLoRA technique, which allows efficient fine- tuning of large language models by quan- tizing their weights. • Thesmallversion has fewer parameters, while thelargeversion del...
[20]

(a) Example-1 (b) Example-2 Figure 5: This figure demonstrates the impact of the feedback module on improving infographic generation

Qwen_qlora_small , Qwen_qlora72B_large: • These baselines use the QLoRA fine- tuning method for the Qwen model. (a) Example-1 (b) Example-2 Figure 5: This figure demonstrates the impact of the feedback module on improving infographic generation. On the left, text overlap issue...
[21]

It evaluates how accurately the system de- tects the number and types of sub charts

Subchart AccuracyThis metric measures the percentage of correctly identified sub charts in the generated metadata compared to the ground truth. It evaluates how accurately the system de- tects the number and types of sub charts. Subchart Accuracy= Number of Correctly Predicted...
[22]

RSE= vuut 1 N NX i=1 (ˆyi −y i)2 (4) Where: •ˆyi: Number of subcharts in the generated metadata for thei-th sample

Root Square Error (RSE) for Subcharts This metric computes the error between the number of subcharts in the ground truth and the generated metadata. RSE= vuut 1 N NX i=1 (ˆyi −y i)2 (4) Where: •ˆyi: Number of subcharts in the generated metadata for thei-th sample. •y i: Number...
[23]

Subchart Type Accuracy= Number of Correct Subchart Types Total Subcharts in Ground Truth ×100(5)

Subchart Type AccuracyThis metric evalu- ates the percentage of correctly classified subchart types in the generated metadata. Subchart Type Accuracy= Number of Correct Subchart Types Total Subcharts in Ground Truth ×100(5)
[24]

Nu- merical values from both the generated and ground truth metadata are sorted and compared sequen- tially for correctness

Statistical AccuracyThis metric evaluates the accuracy of numerical values extracted from the text and included in the generated metadata. Nu- merical values from both the generated and ground truth metadata are sorted and compared sequen- tially for correctness. Statistical A...
[25]

U.S. adults

Textual Information MetricsThese metrics evaluate the quality of textual components in the generated metadata. a. Title Rouge-LThis metric compares the generated title with the ground truth title using the Rouge-L metric, which considers the longest common subsequence (LCS) be...
[26]

title": ,

Title of the context? 2. Summary of the context (what is being inferred)? 3. How many subcharts are there? For each sub- chart, identify: - Type of subchart - Axes (including associated text) - Statistics - Text (title or heading) - Position relative to other subcharts - Posit...
[27]

Axes (with text) 3) Stats of the subchart
[28]

ti- tle": {},

Text (title or heading) 5) Subchart posi- tion relative to others 6) Position of text 7) Background type 8) Dimensions (px) 9) Fonts (one-word) 10) Alignment (horizontal or ver- tical) 11) Subchart summary (include text not in title or heading) Return the response in JSON form...
[2023]

Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atti- cus Geiger, Dan Jurafsky, Christopher D Manning, and Christopher Potts

Socrates: Data story generation via adaptive machine-guided elicitation of user feedback.IEEE Transactions on Visualization and Computer Graph- ics, 30(1):131–141. Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atti- cus Geiger, Dan Jurafsky, Christopher D Manning, and Christopher P...

2024 arXiv
[2024]

Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty

Datanarrative: Automated data-driven story- telling with visualizations and texts.arXiv preprint arXiv:2408.05346. Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022. Chart-to-text: A large-scale benchmark for c...

2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

top-left

What are the human annotation guidelines for synthetic metadata verification ? Ans: We instruct the human annotators to fol- low the below guidelines to verify the accuracy of metadata generated from images containing multi- ple sub-charts : a. Count the Total Number of Subcharts: Ensure the metadata correctly identifies the total number of subcharts in t...

work page

[2] [2]

Matthew Sadiku, Adebowale E Shadare, Sarhan M Musa, Cajetan M Akujuobi, and Roy Perry

Direct preference optimization: Your language model is secretly a reward model.Advances in Neu- ral Information Processing Systems, 36. Matthew Sadiku, Adebowale E Shadare, Sarhan M Musa, Cajetan M Akujuobi, and Roy Perry. 2016. Data visualization.International Journal of En- gineering Research And Advanced Technology (IJERAT), 2(12):11–16. Pranab Sahoo, ...

arXiv 2016

[3] [3]

LIDA explicitly transforms a CSV file into an infographic, convert- ing it into text internally, while ChartGPT uses a query and a CSV file to generate infographics step by step

Why ChartGPT and Lida are not taken into baselines ? Ans: ChartGPT and LIDA operate in a setup where both text and accompanying data tables (e.g., CSV files) are provided as inputs. LIDA explicitly transforms a CSV file into an infographic, convert- ing it into text internally, while ChartGPT uses a query and a CSV file to generate infographics step by st...

work page

[4] [4]

Are the results statistically significant ? Ans: The results of Infogen both small and large are statistically significant The scores of them are the mean of three runs conducted

work page

[5] [5]

For the final evaluation of the generated infographics, the inter- annotator agreement was measured using Cohen’s Kappa, yielding a score of 0.78

What are the annotator demographics and what was the inter-annotator agreement? Ans: The annotators comprised graduate interns with expertise in statistical infographics and prior experience with Pew/statistical data. For the final evaluation of the generated infographics, the inter- annotator agreement was measured using Cohen’s Kappa, yielding a score of 0.78

work page

[6] [6]

yes," indicating that the coder module has successfully generated the final code. If the judge module returns

How does the loop between the coder mod- ule and feedback module end? Ans: The loop ends when the judge module returns a "yes," indicating that the coder module has successfully generated the final code. If the judge module returns "no," the process repeats, with a maximum of 5 iterations. 7.Can I see more examples of Dataset ? Ans: Please go through this...

work page

[7] [8]

On avgInfogentakes 1.5 times more infer- ence time

Is inference time a bottleneck forInfogen? Ans:Yes, asInfogenis using both bunch of LLM agents in both metageneration and codegeneration phase . On avgInfogentakes 1.5 times more infer- ence time . So, performance versus inference trade- off is there. We believe we can reduce the inference time by using recent techniques like Speculative decoding but the ...

work page

[8] [9]

Will the same framework work for other domains like Healthcare and Finance? Ans: We are confident that the same framework can be applied to data from other domains. How- ever, we believe that the training dataset should ideally include some domain-specific data before deploying it in new areas, as the metageneration phase relies on fine-tuned LLMs. Additi...

work page