Pith. sign in

REVIEW 3 major objections 5 minor 48 references

From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces DiagramGenBenchmark, a dataset of nearly 7,000 diagram code–image pairs across eight diagram types, and DiagramAgent, a four-agent pipeline that generates, codes, and edits structured diagrams from text, reporting…

desk verdict The benchmark is a genuine contribution, but the headline outperformance claim is not supported by the experiments as run because of fine-tuning asymmetry and GPT-4o contamination; send to peer review, but expect a major revision. read the letter →

arxiv 2411.11916 v1 pith:47QNH2GZ submitted 2024-11-18 cs.DB

Jingxuan Wei , Cheng Tan , Qi Chen , Gaowei Wu , Siyuan Li , Zhangyang Gao , Linzhuang Sun , Bihui Yu
show 1 more author
Ruifeng Guo
This is my paper · ORCID
classification cs.DB
keywords text-to-diagramgenerationdiagrameditingbenchmarkdatasetmulti-agentframeworkstructureddiagramsTikZDOTcode
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that generating structured diagrams—flowcharts, model architectures, mind maps, and similar—should be treated as a code-generation problem with an explicit edit loop, not as a text-to-image problem. To support this, it contributes a new benchmark with human-like instructions, code, and images for eight diagram categories, plus a four-agent framework that plans, writes diagram code, checks it, and converts diagrams back into code. The authors claim that this framework outperforms 16 existing text-to-code and vision-language baselines on accuracy, structural coherence, and modifiability, with the largest gains in the editing task. If correct, the work gives the research community a standardized task definition and a strong baseline for building editable, logically coherent diagram generators.

What carries the argument

The load-bearing mechanism is the four-agent loop: Plan Agent (query expansion via LLM), Code Agent (a fine-tuned open-weight code model generating TikZ or DOT), Check Agent (compiler-based debugging plus LLM-based verification), and Diagram-to-Code Agent (a vision-language model turning diagram images back into code). The Check Agent's compiler feedback loop is what converts raw code generation into a self-correcting process, and the inverse Diagram-to-Code Agent is what makes editing possible by recovering editable source from a rendered diagram.

What would settle it

Collect a held-out set of diagram requests written independently by human users (not derived from the target code), run DiagramAgent on them, and have those same users rate whether the output matches their intended diagram. If Pass@1 and human scores drop substantially on human-authored requests while staying high on the benchmark's auto-generated queries, the central claim of superior general text-to-diagram ability would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that diagram generation reduces to two inverse code mappings: generation as $D_{\mathrm{gen}} = f_{\mathrm{img}}(f_{\mathrm{code}}(x_{\mathrm{ins}}))$ and editing as $D_{\mathrm{mod}} = f_{\mathrm{img}}(f_{\mathrm{mod}}(f_{\mathrm{code}}^{-1}(D_{\mathrm{ori}}), x_{\mathrm{edit}}))$. The paper's DiagramAgent implements these mappings with four cooperating modules: a Plan Agent that expands incomplete instructions, a Code Agent fine-tuned to emit compilable LaTeX/TikZ or DOT code, a Check Agent that compiles code and uses a large language model to verify completeness, and a Diagram-to-Code Agent that converts an existing diagram image back into code. On the introduced DiagramGenBenchmark, the authors report that this pipeline beats all compared baselines on the generation, coding, and editing tasks, with human evaluation aligning with the objective metrics.

Load-bearing premise

The benchmark's ground truth is valid: the natural-language queries, expanded complete queries, and edit instructions were auto-generated by a large language model, and the edited codes were produced by another model and then filtered for compilability, so if these references do not match real user intent, every reported score measures alignment with an automatic pipeline's output distribution rather than general diagram-generation ability.

Editorial extensions

If this is right

  • If the reported results hold, fine-tuned code-generation models with a compile-and-verify loop become the practical route to producing editable, logically coherent diagrams from text.
  • The benchmark provides a standardized set of 8 diagram types and three tasks (generation, coding, editing), enabling direct comparison of future text-to-diagram systems.
  • The editing results being far stronger than generation results suggests that interactive refinement over generated drafts is a more tractable path than one-shot perfect generation.
  • The inverse diagram-to-code capability opens the door to editing existing diagrams found in documents and slide decks, not only diagrams the system itself produced.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated consequence is that the benchmark's auto-generated queries may make the generation task easier than real user requests, since those queries were reverse-engineered from the target code; a human-requested test set would clarify whether the reported gains hold outside the benchmark distribution.
  • The same four-agent loop could plausibly transfer to other structured visual outputs such as SVG, UML, or circuit schematics, but the paper only demonstrates TikZ and DOT.
  • The reliance on a commercial LLM for data annotation and verification implies a reproducibility ceiling: the dataset and agent behavior are partly tied to a model the authors do not control.
  • A testable extension would be to use the Diagram-to-Code Agent as a self-training signal, letting the system generate new diagrams, convert them back to code, and use mismatches as synthetic training data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces DiagramGenBenchmark, a dataset and evaluation suite for text-to-diagram generation, diagram coding, and diagram editing across eight diagram types (flowchart, model architecture, mind map, bar chart, etc.), and presents DiagramAgent, a four-module framework (Plan Agent, Code Agent, Check Agent, Diagram-to-Code Agent) that generates, codes, and edits structured diagrams via LaTeX/TikZ or DOT code. The experiments compare DiagramAgent against 16 code-LLM baselines and 11 vision-language-model baselines, reporting large gains (e.g., Pass@1 58.15 for generation, 68.89 for coding, 98.00 for editing) together with ablations of the compiler-debugging and GPT-4o-verification components and a human evaluation.

Significance. If the reported gains are genuine, this is a valuable contribution: it defines a new task family, provides a reusable benchmark with three subtasks, and offers a modular agentic baseline that is reproducible from open models plus a closed verifier. The paper ships explicit prompts, data statistics, and ablations, and it makes falsifiable metric claims. However, the central comparative claim—that DiagramAgent significantly outperforms existing baselines—is not yet supported because the head-to-head comparisons are confounded by fine-tuning asymmetry and by GPT-4o involvement in both the benchmark construction and the evaluation pipeline. The benchmark itself may still be useful as a resource, but its validity as a measure of general user intent requires additional human-written references or a calibration study.

major comments (3)
  1. [Sec. 6 (Setup), Sec. 4.2, Eq. (6), Tables 2 and 6] The DiagramAgent Code Agent is fine-tuned for four epochs on the DiagramGenBenchmark training split, whereas all 16 baselines in Table 2 and all editing baselines in Table 6 are evaluated off-the-shelf. Consequently, the reported gaps (e.g., Pass@1 58.15 vs. 55.56 for DeepSeek-Coder in Table 2; 98.00 vs. 95.00 for DeepSeek V2.5 in Table 6) conflate in-distribution fine-tuning with the multi-agent framework. The paper must provide a controlled comparison—either fine-tuning the baselines on the same training data, or evaluating the fine-tuned Code Agent without the Plan/Check agent loop—before the claim of significant outperformance can be accepted.
  2. [Appendix A.2, Sec. 4.4, Figs. 8–13] The benchmark's natural-language instructions, expanded queries, and edit suggestions were all generated by GPT-4o, and the Check Agent uses GPT-4o for verification. This creates a systematic bias: the reference outputs and the verifier are drawn from the same model family, so the reported metrics may measure alignment with GPT-4o's output distribution rather than general diagram-generation ability. The paper does not discuss this risk. Please add a human-written test subset (or at least a manually verified subset) and report whether the relative rankings and absolute scores are stable when the references are not GPT-4o-generated.
  3. [Sec. 4.3–4.4, Table 4, Table 10 (Appendix E)] The Diagram-to-Code Agent is described as 'Qwen2-VL-7B in its default configuration,' yet the full DiagramAgent pipeline augments it with compiler debugging and GPT-4o verification, while the coding baselines are single-pass. Moreover, the ablation 'w/o GPT-4o & Compiler' in Table 10 still achieves Pass@1 of 52.59 versus 28.89 for the off-the-shelf Qwen2-VL baseline; this large unexplained gap indicates that the comparison isolates more than just the removed modules (e.g., prompt design, Plan Agent orchestration, or evaluation harness differences). The paper needs a clean ablation that varies only the framework components on the same base model and same inference-time budget.
minor comments (5)
  1. [Tables 2–7] All objective metrics are reported as point estimates without confidence intervals or significance tests; given the small test sets (270 generation/coding and 200 editing samples) and gaps of only a few points, please provide bootstrap intervals or significance tests.
  2. [Fig. 4 and Appendix D] The human evaluation reports mean scores from three raters but no inter-annotator agreement statistic (e.g., Krippendorff's alpha) and no per-model variance; please add these to support the claim of alignment between human and automated metrics.
  3. [Table 1] The 'Complete Query Length (tokens)' row for the editing column reports an average of 98 while the minimum is 115, which is impossible; the value likely belongs to a different column or needs correction.
  4. [Eq. (9)] The Check Agent operation is written as f_check(c) = f_debug(...) + f_verify(...), but f_debug and f_verify return errors and completeness judgments, not scalars; the '+' notation is formally unexplained and should be replaced with a clearer composition description.
  5. [References and typos] References [41] and [42] cite the same paper; please disambiguate. The Figure 3 caption contains 'inlcuding' for 'including,' and the Figure 4 caption mentions a 'Modify diagram generation' task that does not match the task names used elsewhere in the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation-to-input circularity: the paper's claims are empirical benchmark results, and the self-referential elements (GPT-4o-generated queries, GPT-4o verification) are validity concerns rather than circular reductions.

full rationale

The paper contains no derivation chain that reduces to its inputs. The framework is defined by Eqs. (1)-(9), which merely formalize generation, coding, and editing as compositions of functions; no result is derived from these equations. The headline claims are empirical comparisons in Tables 2, 4, and 6. The Code Agent is trained on the benchmark training split (Eq. 6) and evaluated on a held-out test split, so its Pass@1 and related metrics are genuine predictions, not fitted values renamed as predictions. The benchmark construction is self-referential in a way that creates contamination risk: queries and expanded queries were generated by GPT-4o (Appendix A.2, Figures 7 and 8), and the Check Agent also uses GPT-4o (Sec. 4.4, Figure 13). This could inflate DiagramAgent's scores by rewarding alignment with GPT-4o's own text-code mapping, and the comparison is further confounded because baselines are evaluated zero-shot while DiagramAgent's Code Agent is fine-tuned. However, these are experimental-control and benchmark-validity caveats, not cases where a claimed result is equivalent to its input by construction. There are no load-bearing self-citations: the authors' own prior work [8, 15] appears only as general surveys in the introduction and related work. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The load-bearing assumptions are about the validity of auto-generated ground truth and the fairness of the model comparison.

assumptions (4)
  • domain assumption Compiled LaTeX/TikZ and DOT code faithfully represents the logical structure and content of a diagram
    All dataset construction and evaluation assume that compiling code to an image preserves the intended structure; used throughout Sections 5 and 6.
  • ad hoc to paper GPT-4o-generated instructions, expanded queries, and edit suggestions are representative of real user requests
    Queries and edits in DiagramGenBenchmark were produced by GPT-4o (Appendix A.1 and A.2); no human validation of this assumption is reported.
  • ad hoc to paper Off-the-shelf baselines are valid comparators for a model fine-tuned on the benchmark training set
    Experiment Setup (Section 6) fine-tunes DiagramAgent's Code Agent while evaluating baselines without fine-tuning; the comparison implicitly assumes this is fair.
  • domain assumption Pass@1, CodeBLEU, ROUGE-L, and image fidelity metrics capture diagram quality
    The paper uses these metrics without validating that they correlate with human judgment of logical structure; human evaluation is only a mean score without agreement statistics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing." pith.science (2026). https://pith.science/paper/47QNH2GZ

@misc{pith2026241111916,
  author       = {Pith},
  title        = {Pith review of: From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/47QNH2GZ}},
  note         = {Machine review of arXiv:2411.11916}
}
read the original abstract

We introduce the task of text-to-diagram generation, which focuses on creating structured visual representations directly from textual descriptions. Existing approaches in text-to-image and text-to-code generation lack the logical organization and flexibility needed to produce accurate, editable diagrams, often resulting in outputs that are either unstructured or difficult to modify. To address this gap, we introduce DiagramGenBenchmark, a comprehensive evaluation framework encompassing eight distinct diagram categories, including flowcharts, model architecture diagrams, and mind maps. Additionally, we present DiagramAgent, an innovative framework with four core modules-Plan Agent, Code Agent, Check Agent, and Diagram-to-Code Agent-designed to facilitate both the generation and refinement of complex diagrams. Our extensive experiments, which combine objective metrics with human evaluations, demonstrate that DiagramAgent significantly outperforms existing baseline models in terms of accuracy, structural coherence, and modifiability. This work not only establishes a foundational benchmark for the text-to-diagram generation task but also introduces a powerful toolset to advance research and applications in this emerging area.

Figures

Figures reproduced from arXiv: 2411.11916 by the authors.

Figure 1
Figure 1. Challenges in Existing Text-to-Image and Text-to-Code Methods for Diagram Generation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Workflow of DiagramAgent. The DiagramAgent handles diagram generation, coding, and editing tasks, processing the user query [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. provides example queries and diagrams, inlcuding model architecture, flowcharts, line charts, directed/undi￾rected graphs, tables, bar charts, and mind maps. As de￾tailed in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Human evaluation results for different models on dia [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: DiagramGenBenchmark Data Curation Process [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Data Distribution of DiagramGenBenchmark. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Prompt for Plan Agent: Analyzing and expanding user [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Prompt for Expanding Drawing Query to Generate Com [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 10
Figure 10. Figure 10: Check Agent Prompt for Diagram Coding: Verifying [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Code Agent Prompt for Diagram Generation: Generat [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 13
Figure 13. Figure 13: Check Agent: Using GPT-4o to perform error detection [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Common Error Types in the Diagram Generation: (Left) Shape Understanding Error, (Center) Structure Understanding Error, [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Common Error Types in the Diagram Coding: (Left) Position Understanding Error, (Center) Layout Understanding Error, [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Common Error Types in the Diagram Editing: (Left) Color Understanding Error, (Center) Line Type Understanding Error, [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 18 canonical work pages

  1. [1]

    Meet yi-coder: A small but mighty llm for code,

    01.AI. Meet yi-coder: A small but mighty llm for code,

    work page

  2. [2]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024. 6, 7, 4

    arXiv 2024

  3. [3]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  4. [4]

    A survey of machine learning for big code and naturalness

    Miltiadis Allamanis, Earl T Barr, Premkumar Devanbu, and Charles Sutton. A survey of machine learning for big code and naturalness. ACM Computing Surveys , 51(4):1–37,

    work page

  5. [5]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. 3

    arXiv 2021

  6. [6]

    Au- tomatikz: Text-guided synthesis of scientific vector graphics with tikz

    Jonas Belouadi, Anne Lauscher, and Steffen Eger. Au- tomatikz: Text-guided synthesis of scientific vector graphics with tikz. ICLR, 2024. 5, 1

    work page 2024

  7. [7]

    Internlm2 technical report

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprint arXiv:2403.17297, 2024. 3, 6, 7, 8

    arXiv 2024

  8. [8]

    A sur- vey on generative diffusion models

    Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Z Li. A sur- vey on generative diffusion models. IEEE Transactions on Knowledge and Data Engineering, 2024. 1, 2

    work page 2024

Show all 48 references
  1. [9]

    Evalu- ating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Hen- rique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evalu- ating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. 3

    2021 arXiv

  2. [10]

    Openbias: Open-set bias detection in text-to-image generative models

    Moreno D’Inc `a, Elia Peruzzo, Massimiliano Mancini, Dejia Xu, Vidit Goel, Xingqian Xu, Zhangyang Wang, Humphrey Shi, and Nicu Sebe. Openbias: Open-set bias detection in text-to-image generative models. In CVPR, pages 12225– 12235, 2024. 2

    2024

  3. [11]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  4. [12]

    Fluid: Scaling autoregressive text-to-image generative models with continuous tokens

    Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint arXiv:2410.13863, 2024. 3

    arXiv

  5. [13]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024. 3, 6, 7, 8, 4

    2024 arXiv

  6. [14]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024. 3, 6, 7, 8

    2024 arXiv

  7. [15]

    A survey on advancements in image-text multimodal models: From general techniques to biomedical implementations

    Ruifeng Guo, Jingxuan Wei, Linzhuang Sun, and et.al. A survey on advancements in image-text multimodal models: From general techniques to biomedical implementations. Computers in Biology and Medicine, page 108709, 2024. 1

    2024

  8. [16]

    Cogvlm2: Visual language mod- els for image and video understanding

    Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Jun- hui Ji, Zhao Xue, et al. Cogvlm2: Visual language mod- els for image and video understanding. arXiv preprint arXiv:2408.16500, 2024. 6, 7, 4

    2024 arXiv

  9. [17]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024. 3, 4, 6, 7, 8

    2024 arXiv

  10. [18]

    A comprehensive review of the latest ad- vancements in large generative ai models

    Satyam Kumar, Dayima Musharaf, Seerat Musharaf, and Anil Kumar Sagar. A comprehensive review of the latest ad- vancements in large generative ai models. In ICACIS, pages 90–103. Springer, 2023. 1

    2023

  11. [19]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. 6, 7, 4

    2024 arXiv

  12. [20]

    Mmcode: Evaluating multi-modal code large language models with visually rich programming problems

    Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, and Jing Ma. Mmcode: Evaluating multi-modal code large language models with visually rich programming problems. arXiv preprint arXiv:2404.09486, 2024. 3

    2024 arXiv

  13. [21]

    Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161,

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muen- nighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161,

    arXiv

  14. [22]

    Evaluating text-to-visual generation with image-to-text gen- eration

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text gen- eration. In ECCV, pages 366–384. Springer, 2025. 1

    2025

  15. [23]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434, 2024. 3, 6, 7, 8

    2024 arXiv

  16. [24]

    Deepseek-vl: towards real-world vision- language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, et al. Deepseek-vl: towards real-world vision- language understanding. arXiv preprint arXiv:2403.05525,

    arXiv

  17. [25]

    Wizardcoder: Empowering code large language models with evol-instruct

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. In ICLR, 2024. 3, 6, 7, 8

    2024

  18. [26]

    Star: Scale-wise text-to- image generation via auto-regressive representations

    Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: Scale-wise text-to- image generation via auto-regressive representations. arXiv preprint arXiv:2406.10797, 2024. 3

    2024 arXiv

  19. [27]

    Text to diagram to symbol: Representational transformations in problem-solving

    W Michael McCracken and Wendy C Newstetter. Text to diagram to symbol: Representational transformations in problem-solving. In FIE, pages F2G–13. IEEE, 2001. 1

    2001

  20. [28]

    Precisecontrol: En- hancing text-to-image diffusion models with fine-grained at- tribute control

    Rishubh Parihar, VS Sachidanand, Sabariswaran Mani, Te- jan Karmali, and R Venkatesh Babu. Precisecontrol: En- hancing text-to-image diffusion models with fine-grained at- tribute control. In ECCV, pages 469–487. Springer, 2025. 1, 2

    2025

  21. [29]

    Autoact: Automatic agent learning from scratch for qa via self-planning

    Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Huajun Chen, et al. Autoact: Automatic agent learning from scratch for qa via self-planning. In ICLR, 2024. 3

    2024

  22. [30]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, pages 8821–

  23. [31]

    Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrit- twieser, et al. Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context. arXiv...

    2024 arXiv

  24. [32]

    Code llama: Open foun- dation models for code

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foun- dation models for code. arXiv preprint arXiv:2308.12950 ,

    arXiv

  25. [33]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 35:36479–36494, 2022. 1, 2

    2022

  26. [34]

    Design2code: How far are we from automating front-end engineering? arXiv preprint arXiv:2403.03163, 2024

    Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2code: How far are we from automating front-end engineering? arXiv preprint arXiv:2403.03163, 2024. 3

    2024 arXiv

  27. [35]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 6, 7, 4

    2024 arXiv

  28. [36]

    Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots

    Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhix- uan Liang, Zeyu Lu, Ying Shan, and Ping Luo. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. arXiv preprint arXiv:2405.07990, 2024. 3

    2024 arXiv

  29. [37]

    Attngan: Fine- grained text to image generation with attentional generative adversarial networks

    Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine- grained text to image generation with attentional generative adversarial networks. In CVPR, pages 1316–1324, 2018. 1, 2

    2018

  30. [38]

    Baichuan 2: Open large-scale language models

    Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305, 2023. 3, 6, 7, 8

    2023 arXiv

  31. [39]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024. 2, 3

    2024 arXiv

  32. [40]

    Matplotagent: Method and evaluation for llm-based agentic scientific data visualization.arXiv preprint arXiv:2402.11453, 2024

    Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, et al. Matplotagent: Method and evaluation for llm-based agentic scientific data visualization.arXiv preprint arXiv:2402.11453, 2024. 3

    2024 arXiv

  33. [42]

    Yi: Open foundation models by 01

    Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652, 2024. 6, 7, 8

    2024 arXiv

  34. [43]

    Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. InICCV, pages 5907–5915,

  35. [44]

    Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual in- put and output

    Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, et al. Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual in- put and output. arXiv preprint arXiv:2407.03320, 2024. 6, 7, 4

    2024 arXiv

  36. [45]

    Unifying the perspectives of NLP and software engineering: A survey on language models for code

    Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. Unifying the perspectives of NLP and software engineering: A survey on language models for code. TMLR, 2024. 4

    2024

  37. [46]

    Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, et al. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In ACM SIGKDD, pages 5673–5684, 2023. 6, 7, 8, 1

    2023

  38. [47]

    Vision+ language ap- plications: A survey

    Yutong Zhou and Nobutaka Shimada. Vision+ language ap- plications: A survey. In CVPR, pages 826–842, 2023. 1, 2

    2023

  39. [48]

    Drawing" pass the expanded instructions to the code agent. For

    Bocheng Zou, Mu Cai, Jianrui Zhang, and Yong Jae Lee. Vgbench: Evaluating large language models on vector graphics understanding and generation. arXiv preprint arXiv:2407.10972, 2024. 5, 1 From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generati...

    2024 arXiv

  40. [2023]

    2, 3, 5, 6, 7, 8, 1, 4

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.