pith. machine review for the scientific record. sign in

arxiv: 2604.25914 · v1 · submitted 2026-04-28 · 💻 cs.CL

Recognition: unknown

DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios

Jinxiang Meng , Shaoping Huang , Fangyu Lei , Jingyu Guo , Haoxiang Liu , Jiahao Su , Sihan Wang , Yao Wang
show 12 more authors
Enrui Wang Ye Yang Hongze Chai Jinming Lv Anbang Yu Huangjing Zhang Yitong Zhang Yiming Huang Zeyao Ma Shizhu He Jun Zhao Kang Liu
Authors on Pith no claims yet

Pith reviewed 2026-05-07 16:05 UTC · model grok-4.3

classification 💻 cs.CL
keywords data visualizationAI agentsbenchmarkingreal-world scenariosperformance evaluationintent alignmentspreadsheet manipulationDV-World
0
0 comments X

The pith

State-of-the-art models achieve less than 50% performance on a benchmark for real-world data visualization agents.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces DV-World as a benchmark to test data visualization agents in conditions that mirror professional work environments. It addresses limitations in prior tests by including native tool use, adaptation to new data, and dealing with vague instructions through a user simulator. Results indicate that leading models complete fewer than half the tasks successfully, revealing weaknesses in key areas like precise data handling and intent clarification. The benchmark is positioned as a tool to direct future improvements in AI systems for practical data visualization workflows.

Core claim

The central claim is that real-world data visualization requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Existing benchmarks fall short by confining agents to code sandboxes, limiting to creation tasks in one language, and assuming perfect user intent. DV-World provides 260 tasks across DV-Sheet for spreadsheet chart and dashboard creation with repairs, DV-Evolution for adapting visuals to new data in diverse paradigms, and DV-Interact for aligning with ambiguous requirements via a simulator. A hybrid evaluation uses table-value alignment for numbers and MLLM-as-a-judge for visuals. Experiments demonstrate state-of-the-art models achieve less

What carries the argument

The DV-World benchmark, a collection of 260 tasks divided into three domains that test native spreadsheet manipulation, visual artifact evolution across programming languages, and proactive intent alignment with simulated users, assessed through a combination of numerical table matching and semantic visual judgment by multimodal models.

Load-bearing premise

The three domains and 260 tasks accurately represent the requirements of real-world professional data visualization lifecycles, including native environmental grounding and proactive intent alignment.

What would settle it

If state-of-the-art models were tested on a collection of real enterprise data visualization projects outside the benchmark and achieved significantly higher success rates, this would challenge the claim of critical deficits in handling complex challenges.

read the original abstract

Real-world data visualization (DV) requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Yet, existing benchmarks often suffer from code-sandbox confinement, single-language creation-only tasks, and assumption of perfect intent. To bridge these gaps, we introduce DV-World, a benchmark of 260 tasks designed to evaluate DV agents across real-world professional lifecycles. DV-World spans three domains: DV-Sheet for native spreadsheet manipulation including chart and dashboard creation as well as diagnostic repair; DV-Evolution for adapting and restructuring reference visual artifacts to fit new data across diverse programming paradigms and DV-Interact for proactive intent alignment with a user simulator that mimics real-world ambiguous requirements. Our hybrid evaluation framework integrates Table-value Alignment for numerical precision and MLLM-as-a-Judge with rubrics for semantic-visual assessment. Experiments reveal that state-of-the-art models achieve less than 50% overall performance, exposing critical deficits in handling the complex challenges of real-world data visualization. DV-World provides a realistic testbed to steer development toward the versatile expertise required in enterprise workflows. Our data and code are available at \href{https://github.com/DA-Open/DV-World}{this project page}.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces DV-World, a benchmark of 260 tasks spanning three domains (DV-Sheet for native spreadsheet manipulation and repair, DV-Evolution for cross-paradigm adaptation of visual artifacts, and DV-Interact for proactive alignment with ambiguous user intents) to evaluate data visualization agents under realistic professional conditions. It contrasts this with prior benchmarks limited to code sandboxes or perfect-intent assumptions, proposes a hybrid evaluation combining table-value alignment for numerical accuracy with MLLM-as-a-Judge for semantic-visual quality, and reports that state-of-the-art models achieve less than 50% overall performance, which the authors interpret as exposing critical deficits in real-world DV capabilities.

Significance. If the tasks prove representative and the evaluation reliable, the benchmark would fill a gap by testing environmental grounding, evolution across platforms, and intent handling that current DV agent evaluations largely omit. The open release of data and code supports reproducibility and could usefully steer research toward more robust enterprise-grade visualization agents.

major comments (3)
  1. [§3] §3 (Benchmark Construction): The high-level domain descriptions do not include task sourcing details (e.g., derivation from actual spreadsheet logs or user sessions), practitioner validation steps, or coverage metrics showing how the 260 tasks instantiate native environmental grounding and proactive intent alignment. This directly affects the load-bearing claim that <50% performance reveals intrinsic model deficits rather than benchmark construction artifacts.
  2. [§4] §4 (Evaluation Framework): No information is provided on MLLM judge rubric calibration, inter-rater agreement, or human validation of the semantic-visual scores. Without these, the hybrid performance numbers used to support the central <50% result cannot be independently verified.
  3. [§5] §5 (Experiments): The overall performance figure is reported without per-domain or per-challenge breakdowns, statistical significance tests, or error analysis that would isolate whether the deficits concentrate in evolution, interaction, or grounding subtasks.
minor comments (2)
  1. [Abstract] The abstract and introduction use the phrase 'critical deficits' without first establishing task representativeness; a more measured phrasing would better reflect the empirical nature of the contribution.
  2. [Appendix / Data Release] Ensure the released repository includes the full task specifications, rubrics, and any human validation data so that the hybrid evaluation can be reproduced exactly.

Simulated Author's Rebuttal

3 responses · 0 unresolved

Thank you for the opportunity to respond to the referee's report. We value the feedback provided, which has helped us identify areas where the manuscript can be improved for clarity and completeness. We address each of the major comments below, indicating the revisions made to the manuscript.

read point-by-point responses

  1. Referee: [§3] §3 (Benchmark Construction): The high-level domain descriptions do not include task sourcing details (e.g., derivation from actual spreadsheet logs or user sessions), practitioner validation steps, or coverage metrics showing how the 260 tasks instantiate native environmental grounding and proactive intent alignment. This directly affects the load-bearing claim that <50% performance reveals intrinsic model deficits rather than benchmark construction artifacts.

    Authors: We thank the referee for pointing this out. The original high-level descriptions in §3 have been expanded in the revision to include task sourcing details, explaining that tasks were sourced from real-world spreadsheet logs and user sessions in professional contexts. We describe the practitioner validation steps undertaken, including expert reviews for authenticity. Coverage metrics are now provided to show the distribution of tasks that test native environmental grounding and proactive intent alignment. These additions substantiate that the <50% performance indicates real deficits in current models. revision: yes

  2. Referee: [§4] §4 (Evaluation Framework): No information is provided on MLLM judge rubric calibration, inter-rater agreement, or human validation of the semantic-visual scores. Without these, the hybrid performance numbers used to support the central <50% result cannot be independently verified.

    Authors: We agree with the need for more information on the evaluation framework. In the revised §4, we have included details on the MLLM judge rubric calibration process, inter-rater agreement statistics, and results from human validation of the semantic-visual scores. This will allow independent verification of the hybrid evaluation approach and the associated performance results. revision: yes

  3. Referee: [§5] §5 (Experiments): The overall performance figure is reported without per-domain or per-challenge breakdowns, statistical significance tests, or error analysis that would isolate whether the deficits concentrate in evolution, interaction, or grounding subtasks.

    Authors: The revised §5 now presents per-domain and per-challenge breakdowns of the performance figures, includes statistical significance tests, and provides an error analysis to identify where the deficits are concentrated (e.g., in evolution, interaction, or grounding). This offers a more nuanced view of the results beyond the overall <50% figure. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical benchmark without derivations or self-referential predictions

full rationale

The paper presents an empirical benchmark (DV-World with 260 tasks across three domains) and reports model performance results. No derivation chain, equations, fitted parameters, or predictions exist that could reduce to inputs by construction. Task definitions, domains, and hybrid evaluation (Table-value Alignment + MLLM-as-Judge) are externally specified rather than self-defined. No self-citations are invoked as load-bearing support for any uniqueness claim or ansatz. The <50% performance finding is a direct empirical observation, not a tautology.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that the constructed tasks capture real-world DV requirements; no free parameters or invented entities are introduced.

axioms (1)
  • domain assumption The selected tasks across DV-Sheet, DV-Evolution, and DV-Interact faithfully represent real-world professional data visualization lifecycles including native grounding and ambiguous intent.
    Invoked in the abstract when stating the benchmark bridges gaps in existing evaluations.

pith-pipeline@v0.9.0 · 5575 in / 1262 out tokens · 55149 ms · 2026-05-07T16:05:14.390606+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    WebArena: A Realistic Web Environment for Building Autonomous Agents

    URLhttps://arxiv.org/abs/2307.13854. Hongda Zhu, Yiwen Zhang, Bing Zhao, Jingzhe Ding, Siyao Liu, Tong Liu, Dandan Wang, Yanan Liu, and Zhaojian Li. Frontendbench: A benchmark for evaluating llms on front-end development via automatic evaluation.arXiv preprint arXiv:2506.13832, 2025. 14 Appendix Contents A Task Construction Details 17 A.1 Data Sources . ....

    work page internal anchor Pith review arXiv 2025

  2. [2]

    No gridlines

    Technical Specifications (Project Requirements), focusing on visual and functional constraints akin to formal requirement documents (e.g., "No gridlines") to test strict adherence; and 3) Ad-hoc Queries (Instant Messaging), centering on intent inference where agents must deduce necessary metrics from vague requests without clear guidance. (3) Dynamic rubr...

  3. [3]

    You also do not know exact technical column names

    **Non-Technical:** You do not know code/SQL. You also do not know exact technical column names

  4. [4]

    Use ONLY what appears in <fact_source>

    **No Hallucination:** Do not invent information. Use ONLY what appears in <fact_source>

  5. [5]

    **Natural:** Speak casually like a normal person in a chat app

  6. [6]

    **Consistency:** Stay consistent with <trajectory>

  7. [7]

    Name-First, Type-Second

    **Output:** Output ONLY the user message (1-3 sentences). No analysis, no formatting, no extra tags. ### Input Context <initial_instruction> {instruction} </initial_instruction> <agent_message> {agent_message} </agent_message> <table_schema> {table_schema} </table_schema> <fact_source> {fact_source} </fact_source> <trajectory> {trajectory} </trajectory> #...

    1997

  8. [8]

    Itevaluateswhetherthecharttype,scales,andencodings are suitable for the requirements provided in the natural language instruction

    Appropriateness:This measures the degree to which the chosen visualization aligns with the user’s specificanalyticalintentandthecontextofthedata. Itevaluateswhetherthecharttype,scales,andencodings are suitable for the requirements provided in the natural language instruction. An appropriate visualization effectively bridges the gap between raw data and ac...

  9. [9]

    Dimension_1

    Aesthetics:This dimension focuses on the visual quality, professional layout, and design compliance of the output. It evaluates nuances such as visual semantics, readability, and aesthetic compliance to ensure the chart is suitable for enterprise-level reports or professional analytical dashboards. Rubrics example.As shown in Tab. 12, we provide a scoring...

  10. [10]

    1 1.1.2 Grouping Keys:Aggregation is done by (smoker×age_group), producing exactly 8 bubbles (2 statuses×4 bins)

    Data Fidelity (9 pts) 1.1.1 Binning Boundaries:Records are assigned into exactly 4 bins: 18-29, 30-39, 40-49, 50-64 (equivalent to [18,30), [30,40), [40,50), [50,65)). 1 1.1.2 Grouping Keys:Aggregation is done by (smoker×age_group), producing exactly 8 bubbles (2 statuses×4 bins). 1 1.1.3 X-Axis Value:Each bubble’s X equals themean(age)of its group (value...

  11. [11]

    Smoker” vs. “Non- Smoker

    Visual Logic (6 pts) 2.1.1 Chart Type:Uses a bubble chart (or XY scatter with bubble markers) that clearly supports size encoding. 1 2.1.2 Series Separation:Smoker and Non-Smoker are represented as two distinct series. 1 2.1.3 Color Distinction:Color mapping makes Smoker vs. Non-Smoker immediately distinguish- able. 1 2.1.4 Legend:A legend is present and ...

  12. [12]

    Age”, “Smoking

    Presentation Quality (5 pts) 3.1.1 Descriptive Title:Title mentions core context (e.g., includes “Age”, “Smoking”, “Expenses”).1 3.1.2 Axis Titles:X-axis indicates Average Age; Y-axis indicates Average Medical Expenses. 1 3.1.3 Currency Format:Y-axisvaluesareformattedascurrency(e.g.,$symbol,thousandsseparator). 1 3.1.4 Readability:Data labels (if present)...

  13. [13]

    It ensures that the charts and tables maintain correct data bindings and accurately reflect the underlying spreadsheet values

    Accuracy:This assesses the technical precision of the individual artifacts within the dashboard. It ensures that the charts and tables maintain correct data bindings and accurately reflect the underlying spreadsheet values

  14. [14]

    It reflects how well the agent integrates multiple visual elements into a cohesive, professional layout suitable for enterprise workflows

    Professionalism:This evaluates the holistic spatial planning and logical organization of the analytical view. It reflects how well the agent integrates multiple visual elements into a cohesive, professional layout suitable for enterprise workflows

  15. [15]

    Dimension_1

    Aesthetics:This focuses on the visual clarity, professional formatting, and design quality of the output. It captures the nuances of visual semantics to ensure the dashboard is readable and adheres to high-level design standards. Rubrics example.As shown in Tab. 13, we provide a scoring rubric that decomposes the task into requirements and sub-standards, ...

  16. [16]

    1 1.1.2 Comparison of at least 5 property/land-use types is explicitly shown

    Business Insight (9 pts) 1.1 1.1.1 All 5 boroughs (Manhattan, Brooklyn, Queens, Bronx, Staten Island) present in charts. 1 1.1.2 Comparison of at least 5 property/land-use types is explicitly shown. 1 1.1.3 Grouped chart compares median sale price by borough×property type. 1 1.2 1.2.1 Inclusion of log-scale price distribution (boxplot/density) for spread....

  17. [17]

    1 2.1.2 Missing Data: Observations missing price, landuse, or borough are excluded

    Data Consistency (6 pts) 2.1 2.1.1 Price Validity Filter: All charts exclude transactions with sale_price≤$1,000. 1 2.1.2 Missing Data: Observations missing price, landuse, or borough are excluded. 1 2.2 2.2.1 Confidence Interval (CI) Logic: Error bars represent 95% CI around medians. 1 2.2.2 Scale Logic: Boxplots correctly utilize a logarithmic Y-axis fo...

  18. [18]

    NYC Housing Market

    Visual Design (7 pts) 3.1 3.1.1 Distribution→Box/Strip: Dispersion shown via boxplot + jittered points. 1 3.1.2 Category Comparison→Bar: Borough×type uses grouped bar charts. 1 3.1.3 Ranking→Sorted Bar: Top-value categories are sorted descending by price. 1 3.2 3.2.1 Log Scale Justification: Log scale used only for spans>100×range. 1 3.2.2 Axis Formatting...

  19. [19]

    It assesses the agent’s ability to maintain the original visualization logic (such as chart styles, axis configurations, and color mappings) even as the underlying data evolves

    Consistency:This measures the preservation of design semantics and visual structure from the reference artifact. It assesses the agent’s ability to maintain the original visualization logic (such as chart styles, axis configurations, and color mappings) even as the underlying data evolves

  20. [20]

    Aesthetics:This focuses on the professional quality and design compliance of the final visual output. It evaluates the nuances of visual semantics and professional formatting to ensure that the evolved chart remains readable and visually consistent with enterprise-level standards across different languages like Python or D3.js. Rubric prompt.Below is the ...

  21. [21]

    **Image 1 (Ground Truth):** The correct chart with the perfect style and layout

  22. [22]

    **Image 2 (Candidate):** The chart generated by the AI agent

  23. [23]

    Dimension_1

    Task context: {task_context} # Evaluation Criteria You must verify the Candidate Chart against the following specific criteria. For each item, award **1 point** if fully satisfied, and **0 points** if failed. ## 1. Data Integrity * 1.1 [1pt] **Trend/Pattern:** Do the bars, lines, or points follow the same visual trend as the Ground Truth? The general dire...

  24. [24]

    It evaluates how effectively the agent seeks clarification to uncover the user’s latent intent

    Interaction:Thisdimensionassessestheagent’scapabilitytoactasacommunicativepartnerbyproactively identifying underspecified prompts and resolving data ambiguities through dialogue. It evaluates how effectively the agent seeks clarification to uncover the user’s latent intent

  25. [25]

    It evaluates the precision of the resulting visualization based on the data and requirements established during the multi-turn exchange

    Accuracy:This measures the alignment between the final visual artifact and the user’s true analytical objectives. It evaluates the precision of the resulting visualization based on the data and requirements established during the multi-turn exchange

  26. [26]

    It ensures the generated visualization adheres to expert-defined design standards and professional formatting required for enterprise workflows

    Aesthetics:This dimension focuses on the visual quality, professional layout, and semantic clarity of the output. It ensures the generated visualization adheres to expert-defined design standards and professional formatting required for enterprise workflows. Rubrics example.As shown in Tab. 14, we provide a scoring rubric that decomposes the task into req...

  27. [27]

    **Interaction Evaluation (Dimension 1):** Check the dialogue records in the **trajectory data **

  28. [28]

    **Logic Evaluation (Dimension 2):** Check the **last block of Python code** generated by the assistant in the **trajectory data**

  29. [29]

    Dimension_1

    **Presentation Evaluation (Dimension 3):** Check whether the **visualization chart** meets the requirements. # Final Scoring Logic Final score = sum of scores across all requirements. 36 Requirement score = sum of scores of its criteria. Each criterion score = direct binary scoring (0/1). Please strictly follow the rubric and verify whether the assistant’...

  30. [30]

    1 1.2 RefinementCompliance:Updatedcodeaddressesoverplottingvialayout/densityadjustments

    Interaction & Iteration (3 pts) 1.1 Default Disclosure:Agent locks in ambiguous specs (p90, Top-10 metric, unique IATA) as per Fact Source. 1 1.2 RefinementCompliance:Updatedcodeaddressesoverplottingvialayout/densityadjustments. 1 1.3Dialogue Discipline:No redundant questions regarding schema or non-existent time fields. 1

  31. [31]

    1 2.2Validation:Code contains explicit existence checks for all required columns

    Data Integrity & Business Logic (9 pts) 2.1Data Source:Script usespd.read_csv()and avoids hardcoding primary rows. 1 2.2Validation:Code contains explicit existence checks for all required columns. 1 2.3Mandatory Row Filter:Drops rows missing coordinates, elevation, or country. 1 2.4Threshold Definition:Cutoff computed as exactly the 90th percentile of ele...

  32. [32]

    1 3.2View A Axis Mapping:X/Y mapped to x/y coords with units labeled as “degrees”

    Visualization Semantics (10 pts) 3.1View A Type:Scatter plot (point marks) used for both high and non-high subsets. 1 3.2View A Axis Mapping:X/Y mapped to x/y coords with units labeled as “degrees”. 1 3.3View A Encodings:size∝elevation, color = country, shape = triangle vs. circle. 1 3.4View A Annotation:Top 5 airports labeled with IATA/ICAO and elevation...

    2024

  33. [33]

    **Tool Usage:** Use tool calls only; every step must be a tool call

  34. [34]

    - **NEVER** try to open the Excel app visually

    **NO GUI:** - You are running on a headless server. - **NEVER** try to open the Excel app visually

  35. [35]

    - **REQUIRED:** The output must be a real Excel Chart Object that users can click and edit

    **Native Objects Only:** - **FORBIDDEN:** Do NOT use ‘matplotlib‘, ‘seaborn‘, or ‘PIL‘ to generate static images. - **REQUIRED:** The output must be a real Excel Chart Object that users can click and edit

  36. [36]

    - Set explicit **Chart Titles** and **Axis Labels**

    **Code Quality:** - Ensure the chart references valid data ranges. - Set explicit **Chart Titles** and **Axis Labels**. - If data aggregation is needed (e.g., Sum of Sales by Region), write the aggregated data to the new sheet first, then plot based on that range

  37. [37]

    Style: Use classic style (no rounded corners) chart.style = 2 # NEVER use values >= 10 # 2

    **Chart Configuration (CRITICAL - Apply to ALL charts):** ‘‘‘python from openpyxl.chart.legend import Legend # 1. Style: Use classic style (no rounded corners) chart.style = 2 # NEVER use values >= 10 # 2. Legend: Always outside plot area chart.legend = Legend() chart.legend.overlay = False chart.legend.position = ’r’ # 3. Axes: Ensure all axes are visibl...

  38. [38]

    max"‘ - Link to primary X-axis: ‘x_axis.axId = 100‘, ‘x_axis.crosses =

    **Axis-Specific Rules:** **For Category Axes (BarChart, LineChart):** - Prevent tick label skipping: ‘chart.x_axis.tickLblSkip = 1‘ (if attribute exists) - Avoid modern 3D shapes: Do NOT set ‘chart.shape = 4‘ **For Secondary Axis (Dual Y-axis):** - Secondary chart must set: ‘y_axis.axId = 200‘, ‘y_axis.crosses = "max"‘ - Link to primary X-axis: ‘x_axis.ax...

  39. [39]

    - Always set series.tx with SeriesLabel

    **Series Naming:** - DO NOT set series.title to a raw string. - Always set series.tx with SeriesLabel. ‘‘‘python from openpyxl.chart.series import SeriesLabel from openpyxl.chart.data_source import StrRef # Preferred (static label, safest across openpyxl versions): series.tx = SeriesLabel(v="Name") ‘‘‘

  40. [40]

    --- 40 # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call

    **Finish:** Call the ‘finish‘ tool to save the workbook and exit. --- 40 # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call. - Do not mix narrative text with tool calls. --- ## TASK {task} DVSheet-Fix prompt.Below is the DVSheet-Fix system prompt. DVSheet-Fix System Prompt # Role Definition You are an exp...

  41. [41]

    **Diagnose:** Identify why the current chart fails (e.g., numbers stored as text, wrong axis range, incorrect data series reference)

  42. [42]

    --- ## Rules & Constraints

    **Fix In-Place (CRITICAL):** - **Do NOT create a new sheet.** - **Do NOT delete the existing sheet.** - You must modify the **existing data cells** or the **existing chart object** directly on the current sheet. --- ## Rules & Constraints

  43. [44]

    **NO GUI:** NEVER try to open the Excel app visually

  44. [45]

    **Native Objects Only:** - Output must be a real Excel Chart Object

  45. [46]

    - **Data Cleaning:** When converting text to numbers, write the clean values back to the ** original cell addresses**

    **In-Place Modification Rules:** - **Target:** Operate on the active sheet or the sheet specified in the task. - **Data Cleaning:** When converting text to numbers, write the clean values back to the ** original cell addresses**. Do not move the data unless asked. - **Chart Preservation:** Try to preserve the chart’s location and size

  46. [47]

    --- # RESPONSE FORMAT # 41 - Before a tool call, give a one-sentence English plan, then return the tool call

    **Finish:** Call the ‘finish‘ tool to save the workbook and exit. --- # RESPONSE FORMAT # 41 - Before a tool call, give a one-sentence English plan, then return the tool call. - Do not mix narrative text with tool calls. --- ## TASK {task} DVSheet-Dash prompt.Below is the DVSheet-Dash system prompt. DVSheet-Dash System Prompt # Role Definition You are an ...

  47. [48]

    **Analyze:** Identify key metrics (KPIs) and trends from the source data

  48. [49]

    DO NOT modify the raw data sheet

    **Setup Target:** Create a **NEW sheet named ’result’** to host the dashboard. DO NOT modify the raw data sheet

  49. [50]

    **Construct:** Build a grid-based dashboard on the ’result’ sheet combining **KPI Cards** ( Big Numbers), **Data Tables**, and **Charts**

  50. [51]

    --- ## Rules & Constraints

    **Format:** Apply professional styling (remove gridlines, specific fonts) to make it look like a BI application. --- ## Rules & Constraints

  51. [52]

    **Tool Usage:** Use tool calls only

  52. [53]

    - Do not leave the dashboard elements on the raw data sheet

    **Target Sheet:** - ALL visual elements (Charts, KPIs, Titles) MUST be on the **’result’** sheet. - Do not leave the dashboard elements on the raw data sheet

  53. [54]

    Total Sales

    **Dashboard Styling Rules (Professional Look):** - **Gridlines:** MUST turn off gridlines on ’result’ sheet: ‘ws.sheet_view.showGridLines = False‘. - **Title:** Add a clear, bold title at the top (Row 1). - **KPI Cards:** Display key numbers clearly with labels (e.g., "Total Sales" in B3, "$1.2 M" in B4)

  54. [55]

    --- # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call

    **Finish:** Call the ‘finish‘ tool to save the workbook and exit. --- # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call. - Do not mix narrative text with tool calls. --- 42 ## TASK {task} DV-Evol prompt.Below is the DV-Evol system prompt. DV-Evol System Prompt # Role Definition You are an expert **Data V...

  55. [56]

    **Reference Image:** Provides the visual style (colors, layout, aesthetics)

  56. [57]

    **New Data:** Provides the actual values to be plotted

  57. [58]

    You have access to tools ‘load_image‘ and ‘render_chart‘

    **New Requirements:** Specifies how to adapt or modify the chart. You have access to tools ‘load_image‘ and ‘render_chart‘. You start in the {work_dir} directory. The maximum number of steps allowed is {max_steps}. --- ## Objective

  58. [59]

    **Analyze Style (via Tool):** Use ‘load_image‘ to inspect the reference image

  59. [60]

    Based on the provided image, new data, and new modification requirements, create a visualization spec in **{viz_lang}**

  60. [61]

    The chart must be based entirely on this final table, and this table must be saved separately as ‘result.csv‘

    Before plotting, filter the tabular data to obtain the final dataset used for visualization. The chart must be based entirely on this final table, and this table must be saved separately as ‘result.csv‘

  61. [62]

    --- ## Rules & Constraints

    After the spec/snippet is written, call ‘render_chart‘ to render it to ‘result.png‘. --- ## Rules & Constraints

  62. [63]

    **Tool Usage (Mandatory):** - You **MUST** call ‘load_image(image_path="...")‘ first to capture the visual style

  63. [64]

    - **Prompt Data = USE.** Use the **New Data** provided in the text

    **Data Source of Truth:** - **Image Data = IGNORE.** Do NOT use the numbers seen in the reference image. - **Prompt Data = USE.** Use the **New Data** provided in the text. - **Consistency:** The data hardcoded in your spec must match the data saved to ‘result.csv ‘

  64. [65]

    result.json|result.js

    **File Saving Rules (CRITICAL):** - ECharts: save the option JSON to ‘result.json‘. - Vega-Lite: save the JSON spec to ‘result.json‘. - D3.js: save a runnable snippet to ‘result.js‘ (read ‘result.csv‘). - Plotly.js: save a runnable snippet to ‘result.js‘ (read ‘result.csv‘). - Then call ‘render_chart(file_path="result.json|result.js", tool_type="{viz_lang...

  65. [66]

    - **Logic:** Follow the user’s specific instructions (e.g., change chart type)

    **Style & Logic Adaptation:** - **Style:** Inherit the reference image’s aesthetic (colors, background, fonts). - **Logic:** Follow the user’s specific instructions (e.g., change chart type). 43

  66. [67]

    --- # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call

    **Finish:** Call the ‘finish‘ tool to save the workbook and exit. --- # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call. - Do not mix narrative text with tool calls. --- ## TASK {task} DV-Inter prompt.Below is the DV-Inter system prompt. DV-Inter System Prompt # Role Definition You are a dedicated Python...

  67. [68]

    Read Data: Load the specified data file (CSV or SQLite) from the current directory

  68. [69]

    Data Processing: Perform necessary cleaning and transformation to meet the plotting requirements (e.g., handle null values, aggregate data)

  69. [70]

    Plot Charts: Use Matplotlib or Seaborn to generate charts that meet the task requirements

  70. [71]

    --- ## Rules & Constraints

    Save Results: Save the charts as .png files, absolutely never display them or generate text reports. --- ## Rules & Constraints

  71. [72]

    Use tool calls only; every step must be a tool call

  72. [73]

    - NEVER call ‘plt.show()‘

    NO GUI / NO INTERACTIVITY: - You are running on a headless server. - NEVER call ‘plt.show()‘. It will crash the environment. - ALWAYS save figures using ‘plt.savefig(’filename.png’)‘. - After saving, call ‘plt.close()‘ to release memory

  73. [74]

    - Handle missing values (dropna or fillna) before plotting to avoid errors

    Code Quality: - Use ‘pandas‘ for data manipulation. - Handle missing values (dropna or fillna) before plotting to avoid errors. - Ensure charts have Titles, Axis Labels, Legends, and Gridlines (if necessary). 44 - Avoid Japanese/Chinese characters unless specific fonts are loaded; prefer English labels to prevent "tofu" boxes

  74. [75]

    **Do NOT** guess user intent, if anything is unclear, please use ‘ask_user‘ to ask the user

  75. [76]

    Select Data

    Finish: Call the ‘finish‘ tool. --- # RESPONSE FORMAT # - Before a tool call, give a one-sentence English plan, then return the tool call. - Do not mix narrative text with tool calls. ## TASK {task} D.4 Human Evaluation Human performance baseline.Toestablisharealisticperformanceceiling,10evaluatorscompletedatotalof 50sampledtasksacrossalldomains. Particip...

    2025

  76. [78]

    University Count by Country with Median Scores

    Layout & Aesthetics ❌ 3.1 Labels: 0/1 Legend and title positioning are misaligned compared to the standard layout. ✅ 3.2 Readability: 1/1 Excessive ridge overlap and small font sizes compromise the overall legibility. ❌ 3.3 Axis Alignment and Spacing: 0/1 The axes are poorly calibrated, causing data to shift away from intended baselines. ❌ 3.4 Element Ali...

    1990

  77. [79]

    ❌ 2.2 Chart Elements: 0/1 Specific ridgeline outlines and gridline styles were not properly reproduced

    Style Imitation ✅ 2.1 Color Palette: 1/1 The model correctly applied the designated HEX codes for all income categories. ❌ 2.2 Chart Elements: 0/1 Specific ridgeline outlines and gridline styles were not properly reproduced. ✅ 2.3 Background: 1/1 The background color and light/dark mode are consistent with the Ground Truth. ❌ 2.4 Font and Text Styles: 0/1...

  78. [80]

    X-axis: Log10(...)

    Layout & Aesthetics ❌ 3.1 Labels: 0/1 Legend and title positioning are misaligned compared to the standard layout. ✅ 3.2 Readability: 1/1 Excessive ridge overlap and small font sizes compromise the overall legibility. ❌ 3.3 Axis Alignment and Spacing: 0/1 The axes are poorly calibrated, causing data to shift away from intended baselines. ❌ 3.4 Element Ali...

    1990

  79. [82]

    Refined Outlier Correlation: Sorted by Impact

    Layout & Aesthetics ❌ 3.1 Labels: 0/1 Legend and title positioning are misaligned compared to the standard layout. ❌ 3.2 Readability: 0/1 Excessive ridge overlap and small font sizes compromise the overall legibility. ❌ 3.3 Axis Alignment and Spacing: 0/1 The axes are poorly calibrated, causing data to shift away from intended baselines. ❌ 3.4 Element Ali...

    1990

  80. [84]

    0–20” instead of “1–20

    Layout & Aesthetics ❌ 3.1 Labels: 0/1 Legend and title positioning are misaligned compared to the standard layout. ✅ 3.2 Readability: 1/1 Excessive ridge overlap and small font sizes compromise the overall legibility. ❌ 3.3 Axis Alignment and Spacing: 0/1 The axes are poorly calibrated, causing data to shift away from intended baselines. ❌ 3.4 Element Ali...

    1990

Showing first 80 references.