Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper presents SVGenius, a 2,377-query benchmark covering SVG understanding, editing, and generation, and argues that all current LLMs lose performance as SVG complexity rises, with reasoning-trained models improving more from…

desk verdict A useful new SVG benchmark with real-world data, but the complexity labels and headline findings need a careful revision before I'd trust the rankings. read the letter →

arxiv 2506.03139 v1 pith:FVWRIGHW submitted 2025-06-03 cs.CV cs.AI

Siqi Chen , Xinyu Dong , Haolei Xu , Xingyu Wu , Fei Tang , Hang Zhang , Yuchen Yan , Linjuan Wu
show 5 more authors
Wenqi Zhang Guiyang Hou Yongliang Shen Weiming Lu Yueting Zhuang
This is my paper
classification cs.CVcs.AI
keywords SVGbenchmarkcomplexitystratificationlargelanguagemodelsmultimodalLLMsvectorgraphicsunderstandingeditinggenerationstyletransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SVGenius is built to answer a concrete question: how well do large language models really handle scalable vector graphics, and where do they hit their limits? The paper constructs a benchmark of 2,377 queries from real-world icons spanning 24 application domains, splits them into Easy, Medium, and Hard levels by structural complexity, and evaluates 22 models on eight task types that go from visual understanding through code repair and optimization to full generation and style transfer. The central result is that no model family escapes a systematic drop in performance as SVG complexity increases, and that style transfer is the hardest task for every model type. The motivation is practical: knowing where models fail on complex vector graphics is what would let developers target better training data and model designs for automated graphic design.

What carries the argument

The mechanism that carries the argument is a per-SVG complexity score. The paper counts paths, control points, and advanced path commands, weights command types by their computational demand (basic moves and lines weight 1, quadratic curves 2–3, cubic curves 3–4, elliptical arcs 5), and combines these with entropy and curvature features through principal component analysis to sort icons into Easy, Medium, and Hard levels. Everything else in the benchmark hangs off that axis: every model is scored separately at each complexity level, so the observed degradation curve is what the complexity score produces. The benchmark also adds code-aware metrics, notably a path-structure similarity score that blends visual overlap with matched SVG path structure, so that generation quality is judged on both rendered appearance and syntactic fidelity.

What would settle it

Recompute the complexity score for the same 927 icons under alternative, openly specified weightings (for example, weighing path count more heavily than curve commands, or vice versa) and check whether the 33/34/33 Easy/Medium/Hard partition changes substantially; if the re-sorted samples flatten or reverse the reported degradation curves on the 22 models, the universal-complexity claim would not survive.

Watch

Extended reading notes

Core claim

The paper's claim is that SVGenius is the first systematic, complexity-stratified evaluation framework spanning the three progressive dimensions of SVG processing — understanding, editing, and generation — rather than testing one isolated capability. On this benchmark, the paper reports that proprietary models lead on most tasks, but all 22 tested models, including specialized SVG systems, show universal degradation from Easy to Hard samples; that reasoning-enhanced open models narrow the gap to proprietary models more effectively than simply scaling up parameters; and that style transfer is the weakest capability across the board. The authors interpret this as evidence of a fundamental limitation in how current models represent and reason about structured vector graphics, and they offer SVGenius as the baseline against which future SVG-capable models can be measured.

Load-bearing premise

The difficulty levels depend on complexity weights the paper describes as 'empirically determined' but never discloses, so the Easy, Medium, and Hard labels — and with them the headline finding that performance falls as complexity rises — rest on those unpublished numbers.

Editorial extensions

If this is right

  • Benchmarking SVG models only on simple icons will overstate their ability, so future SVG evaluations should report complexity-stratified scores across easy, medium, and hard samples.
  • Training for explicit reasoning is a more promising route to complex SVG handling than model scaling alone, which gives a concrete target for open-source model development.
  • Style transfer should be evaluated as its own capability with dedicated metrics, since it is consistently harder than direct generation from text or image.
  • Code-aware metrics such as path-structure similarity should accompany raster-image metrics when judging generated SVG, because visual similarity alone misses structural errors.
  • The benchmark provides a baseline of 22 models across eight tasks, so future SVG-capable systems have a common yardstick for claiming progress.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper does not develop is to use the complexity-stratified scores per application domain to identify which icon categories each model family handles worst, turning the benchmark into a diagnostic tool for targeted data collection.
  • The finding that reasoning training outperforms scaling suggests a testable recipe for specialized SVG models: train on easy-to-hard SVG problem sequences with explicit step-by-step repair and planning, then measure whether the degradation curve flattens.
  • The path-structure similarity metric could transfer to other structured code-generation domains, such as UI layout code or 3D scene descriptions, where pixel-level metrics are similarly blind to syntax.
  • If the benchmark is meant to become a lasting standard, the unpublished complexity weights and PCA loadings will need to be released, since reproducibility of the difficulty axis is what the main conclusion depends on.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. SVGenius introduces a benchmark for evaluating LLMs on SVG understanding, editing, and generation using 2,377 queries built from 927 human-filtered, real-world SVG icons across 24 domains, stratified into Easy/Medium/Hard complexity levels. The paper evaluates 22 models with 8 task categories and 18 metrics, and reports four findings: proprietary models outperform open-source models, all models degrade with increasing complexity, reasoning-enhanced training helps more than scaling, and style transfer is the hardest capability. The dataset construction and many evaluation protocols are described in detail, with appendices containing full leaderboards and metric definitions.

Significance. If the complexity stratification and metric choices are sound, SVGenius would be a useful public resource: it combines understanding, editing, and generation in one benchmark, uses real-world data with human filtering, covers 24 domains, and provides code and data. The proposed PSS and rCLIP metrics are clearly motivated and could be reused by other SVG evaluations. The paper also ships an unusually broad model comparison (22 models), which is valuable for the community even if some headline conclusions need additional support. The main risks are the unverified complexity labels, the confounded reasoning-vs-scaling comparison, and the absence of uncertainty quantification in all reported rankings.

major comments (5)
  1. [Section 3 and Appendix B.2] The complexity stratification is not reproducible. Section 3 says path count, control points, and complex commands are 'normalized and combined using empirically determined weights,' but the weights are never given. Appendix B.2 instead describes a different procedure with per-command weights, entropy terms, curvature, and PCA, and the PCA loadings are not reported. This matters directly for the headline claim of 'universal performance degradation with complexity,' because the Easy/Medium/Hard labels are the independent variable. The validation in Tables 6-7 undercuts the claim that the stratification captures geometric sophistication: the fraction of complex commands per path is roughly constant across levels (12.03/2.14 ≈ 5.6, 56.01/9.87 ≈ 5.7, 79.94/16.02 ≈ 5.0), and Type Entropy is nearly flat (1.48, 1.67, 1.68). The ANOVA in B.2 is computed on the same features used to construct the levels, so it does not independently validate them. The authors should disclose the exact formula, report PCA component loadings, and provide a human-rated difficulty validation or an out-of-sample difficulty measure; otherwise the complexity-degradation results may reflect token length or path count rather than task difficulty.
  2. [Section 5.3] The claim that 'reasoning-enhanced training proves more effective than pure scaling' is not supported by the comparisons shown. DS-R1-Qwen-32B and QwQ-32B differ from Qwen2.5-72B and Qwen3-32B not only in reasoning training but also in base model generation, training recipe, instruction tuning, and release version. A proper test would compare the same base model with and without the reasoning RL stage, or at least control for base model and data. As written, the conclusion in the abstract and Section 5.3 is a confounded observation, and the paper should either add controlled comparisons or soften the claim.
  3. [Section 5.1 and all result tables] The paper states that three independent runs are used 'for statistical robustness,' but no standard deviations, confidence intervals, or significance tests are reported anywhere. Many of the headline differences are small (e.g., Table 4 Easy PSS 16.78 vs 16.60; HPS 19.19 vs 19.39), and without error bars these differences cannot be distinguished from noise. The authors should report per-run variance or confidence intervals for at least the main comparisons, and perform significance tests for the claims that proprietary models are better, degradation is universal, and reasoning models beat scaled baselines.
  4. [Appendix E.3] The style transfer evaluation relies entirely on LLM-as-a-judge scoring without validation. The local metric uses GPT-4o-mini scores on five rubrics, and the global ranking uses DeepSeek-R1 as the reference in an AlpacaEval-style win-rate computation. There is no human agreement study, no correlation with human judgments, and no analysis of judge bias (e.g., preference for longer or more verbose output). Since style transfer is one of the four central findings, the scoring procedure needs at least a small human-validation study or a calibration against existing human preference data.
  5. [Tables 4, 9, and 10] Several reported numbers are implausible and affect the results. In Table 4 (and repeated in Appendix Table 10), Claude-3.7-Sonnet's Medium HPS is 85.71 while its Easy and Hard HPS are 21.35 and 18.74, respectively; this is almost certainly a typo and must be corrected. In Table 9, the Hard rMSE for DeepSeek-R1-Distill-Qwen-1.5B is 6933.75, which is impossible under the definition in Eq. (3) since rMSE is capped at 1. Many other rMSE values in Table 9 exceed 1 (e.g., 28.29, 54.51, 57.04), suggesting a unit or scaling error. These anomalies should be fixed and the leaderboards recomputed before the rankings are interpreted.
minor comments (5)
  1. [CCS Concepts] The CCS Concepts block still contains the placeholder text 'Do Not Use This Code → Generate the Correct Terms for Your Paper'; this should be replaced with proper ACM classification terms.
  2. [Table 4 caption] The caption for Table 4 lists 'FSS' as a metric, but the table column is labeled 'PSS'; this typo should be corrected.
  3. [Abstract and Section 3] The paper reports 2,377 queries but also 300 sampled SVGs; the relationship between the 927 filtered SVGs, the 300-sample stratified set, and the 2,377 queries is never explicitly reconciled, and should be clarified.
  4. [Appendix B.3] The manual difficulty-stratification assessment is only mentioned, with a screenshot in Figure 4; no quantitative agreement rates or stratification-quality scores are reported, so the claim that the levels are 'validated' is unsupported.
  5. [References] Reference [21] cites a 2010 Mistral paper on cloud resource management, but the text uses 'Mistral-Small-3.1-24B-Instruct'; the reference should point to the actual Mistral model documentation.

Circularity Check

1 steps flagged · score 2.0 of 10

No central circularity; one minor self-referential artifact in the style-transfer ranking baseline.

  1. self definitional [Appendix E.3, Global Ranking Evaluation (Table 12)]
    "The outputs of each competing model are compared against the reference model’s results across all samples. As shown in Table 12, the final rankings are derived from aggregated win–loss statistics, sorted by win rate relative to the baseline."

    In this AlpacaEval-style design, DeepSeek-R1 serves as the reference model, so its own outputs are compared against themselves; its reported 50.55% win rate (labeled 'Reference Model') is forced to be near 50% by the pairwise-comparison construction. The number conveys no independent information about DeepSeek-R1's style-transfer quality. The central claims of the paper do not depend on this self-referential number, so the circularity is minor and localized.

full rationale

The paper's central claims are empirical benchmark observations, not derivations from fitted parameters. The Easy/Medium/Hard splits are assigned from hand-defined structural features (path count, control points, complex commands, PCA on command weights); model performances are measured afterward, so a universal degradation trend is not forced by construction — a model could in principle perform better on the Hard subset. The new metrics PSS and rCLIP are defined formulaically (weighted sums and CLIP ratios) and are used to score outputs; they do not encode the paper's conclusions (model rankings, reasoning-versus-scaling, style-transfer difficulty). The undisclosed 'empirically determined weights' in Section 3 and the switch to PCA in Appendix B.2 are reproducibility and validity concerns, not circularity. The only exhibitable reduction is the AlpacaEval style-transfer ranking, where the reference model's own win rate is self-comparison; that artifact is transparent and does not carry the headline findings. Score 2 reflects this lone minor self-referential element.

Assumptions & free parameters 3 free parameters · 4 assumptions · 2 invented entities

The central claims rest on the benchmark's construction choices: undisclosed complexity weights, hand-set metric weights, and unvalidated LLM-as-judge scoring. These are not derived from first principles, and none has independent external evidence. The empirical findings are observations over these choices, so they carry the assumptions forward.

free parameters (3)
  • Complexity normalization weights = not disclosed
    Section 3 says path count, control points, and complex commands are 'normalized and combined using empirically determined weights'; these weights are never given, making the stratification non-reproducible.
  • Command complexity weights = M/L/Z=1, Q/T=2-3, C/S=3-4, A=5
    Appendix B.2 assigns hand-chosen weights to SVG path commands for the structure-aware complexity model.
  • PSS combination weights = 0.6 visual alignment, 0.4 structural correspondence
    Appendix D defines the Path-Structure Similarity Score as a weighted sum with weights 0.6 and 0.4, chosen without external calibration.
assumptions (4)
  • domain assumption Iconfont icons are representative of real-world SVG usage.
    Section 3 and Appendix A build the dataset entirely from IconFont, a user-created icon library; if this source is not representative, benchmark conclusions may not generalize.
  • domain assumption Automated perceptual metrics (CLIP, HPS, SSIM, LPIPS, DINO, MSE) validly measure SVG quality.
    Section 4.3 and Appendix E use these raster-based metrics on rendered SVGs without validating them against human judgments for SVG-specific artifacts.
  • domain assumption LLM-as-judge (GPT-4o-mini and AlpacaEval with DeepSeek-R1) produces valid style-transfer quality scores.
    Appendix E.3 uses GPT-4o-mini to score style transfer on 1-5 rubrics and DeepSeek-R1 as a reference in AlpacaEval; no validation of these judge scores against human ratings is provided.
  • standard math PCA and ANOVA correctly separate complexity levels.
    Appendix B.2 uses principal component analysis and ANOVA F-tests (p<0.001) to justify stratification; standard statistical tools, assumed valid.
invented entities (2)
  • Path-Structure Similarity Score (PSS)
    purpose: New metric to evaluate structural and visual fidelity of generated SVGs by matching paths via Hungarian algorithm and weighted IoU.
    Defined in Appendix D; no external validation or comparison against human judgments, and weights are hand-set.
  • Relative CLIP Consistency (rCLIP)
    purpose: New metric measuring semantic degradation of generated SVG relative to ground truth using CLIP similarities.
    Defined in Appendix E.1 Eq. (1); it is a normalized transformation of CLIP scores with no independent validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation." pith.science (2026). https://pith.science/paper/FVWRIGHW

@misc{pith2026250603139,
  author       = {Pith},
  title        = {Pith review of: SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FVWRIGHW}},
  note         = {Machine review of arXiv:2506.03139}
}
read the original abstract

Large Language Models (LLMs) and Multimodal LLMs have shown promising capabilities for SVG processing, yet existing benchmarks suffer from limited real-world coverage, lack of complexity stratification, and fragmented evaluation paradigms. We introduce SVGenius, a comprehensive benchmark comprising 2,377 queries across three progressive dimensions: understanding, editing, and generation. Built on real-world data from 24 application domains with systematic complexity stratification, SVGenius evaluates models through 8 task categories and 18 metrics. We assess 22 mainstream models spanning different scales, architectures, training paradigms, and accessibility levels. Our analysis reveals that while proprietary models significantly outperform open-source counterparts, all models exhibit systematic performance degradation with increasing complexity, indicating fundamental limitations in current approaches; however, reasoning-enhanced training proves more effective than pure scaling for overcoming these limitations, though style transfer remains the most challenging capability across all model types. SVGenius establishes the first systematic evaluation framework for SVG processing, providing crucial insights for developing more capable vector graphics models and advancing automated graphic design applications. Appendix and supplementary materials (including all data and code) are available at https://zju-real.github.io/SVGenius.

Figures

Figures reproduced from arXiv: 2506.03139 by the authors.

Figure 1
Figure 1. Overview of SVGenius. SVGenius evaluates (M)LLMs capabilities across three progressive dimensions: Understanding (perceptual and semantic QA), Editing (bug fixing, code optimization, style editing), and Generation (text-to-SVG, image-to-SVG, style transfer). Built on real-world data from 24 domains with systematic complexity stratification, our benchmark enables comprehensive assessment of SVG processing capabilitie… view at source ↗
Figure 2
Figure 2. SVGenius dataset construction and complexity validation. Left: systematic pipeline from data collection, processing, human filtering to complexity stratification. Center: 24-domain coverage across diverse applications. Right: validation of complexity stratification showing clear hierarchical separation across Easy, Medium, and Hard levels through feature distributions and complexity scores. have demonstrated the fea… view at source ↗
Figure 3
Figure 3. Word cloud visualization of the 24 diverse application [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Screenshot of Our Questionnaires. Semantic Alignment. This dimension measures the degree to which the generated SVG semantically aligns with the input caption. We employ two complementary metrics: • CLIP Score: The generated SVG is rendered and embedded along￾side the …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    SVGEval benchmarks and trains an explainable multimodal scorer for perceptual quality of text-to-SVG generation, showing a consistent gap on spatial and structural judgments.

Reference graph

Works this paper leans on

81 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Anthropic. 2023. Claude 2. https://www.anthropic.com/index/claude-2. Accessed: 2025-05-29

    2023

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923(2025)

    arXiv 2025

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in neural information processing systems33 (2020), 1877–1901

    2020

  4. [4]

    Mu Cai, Zeyi Huang, Yuheng Li, Utkarsh Ojha, Haohan Wang, and Yong Jae Lee. 2023. Leveraging large language models for scalable vector graphics-driven image understanding.arXiv preprint arXiv:2306.06094(2023)

    arXiv 2023

  5. [5]

    Alexandre Carlier, Martin Danelljan, Alexandre Alahi, and Radu Timofte. 2020. Deepsvg: A hierarchical generative network for vector graphics animation.Ad- vances in Neural Information Processing Systems33 (2020), 16351–16361

    work page 2020

  6. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374(2021)

    arXiv 2021

  7. [7]

    Zehao Chen and Rong Pan. 2025. SVGBuilder: Component-Based Colored SVG Generation with Text-Guided Autoregressive Transformers. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 2358–2366

    work page 2025

  8. [8]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Se- bastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research24, 240 (2023), 1–113

    2023

Show all 81 references
  1. [9]

    Louis Clouâtre and Marc Demers. 2019. Figr: Few-shot image generation with reptile.arXiv preprint arXiv:1901.02199(2019)

    2019 arXiv

  2. [10]

    SVG Contributors. 2024. SVGO: Node.js tool for optimizing SVG files. https: //github.com/svg/svgo Accessed: 2024-12-XX

    2024

  3. [11]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  4. [12]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    2025 arXiv

  5. [13]

    David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477(2017)

    2017 arXiv

  6. [14]

    Teng Hu, Ran Yi, Baihong Qian, Jiangning Zhang, Paul L Rosin, and Yu-Kun Lai. 2024. Supersvg: Superpixel-based scalable vector graphics synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 24892–24901

    2024

  7. [15]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186(2024)

    2024 arXiv

  8. [16]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card.arXiv preprint arXiv:2410.21276(2024)

    2024 arXiv

  9. [17]

    IconFont. 2024. IconFont - Alibaba Vector Icon Library. https://www.iconfont.cn/. Accessed: 2025-05-29

    2024

  10. [18]

    Ghfran Jabour, Sergey Muravyov, and Valeria Efimova. 2025. Layerwise Image Vectorization via Bayesain-Optimized Contour.Proceedings Copyright831 (2025), 838

    2025

  11. [19]

    Ajay Jain, Amber Xie, and Pieter Abbeel. 2023. Vectorfusion: Text-to-svg by ab- stracting pixel-based diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1911–1920

    2023

  12. [20]

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues?arXiv preprint arXiv:2310.06770(2023)

    2023 arXiv

  13. [21]

    Gueyoung Jung, Matti A Hiltunen, Kaustubh R Joshi, Richard D Schlichting, and Calton Pu. 2010. Mistral: Dynamically managing power, performance, and adaptation cost in cloud infrastructures. In2010 IEEE 30th International Conference on Distributed Computing Systems. IEEE, 62–73

    2010

  14. [22]

    Claire Le Goues, Neal Holtschulte, Edward K Smith, Yuriy Brun, Premkumar Devanbu, Stephanie Forrest, and Westley Weimer. 2015. The ManyBugs and IntroClass benchmarks for automated repair of C programs.IEEE Transactions on Software Engineering41, 12 (2015), 1236–1256

    2015

  15. [23]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. InInternational conference on machine learning. PMLR, 12888–12900

    2022

  16. [24]

    Tzu-Mao Li, Michal Lukáč, Michaël Gharbi, and Jonathan Ragan-Kelley. 2020. Differentiable vector graphics rasterization for editing and learning.ACM Trans- actions on Graphics (TOG)39, 6 (2020), 1–15

    2020

  17. [25]

    Derrick Lin, James Koppel, Angela Chen, and Armando Solar-Lezama. 2017. QuixBugs: A multi-lingual program repair benchmark set based on the Quixey Challenge. InProceedings Companion of the 2017 ACM SIGPLAN international conference on systems, programming, languages, and applic...

    2017

  18. [26]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning.Advances in neural information processing systems36 (2023), 34892–34916

    2023

  19. [27]

    Raphael Gontijo Lopes, David Ha, Douglas Eck, and Jonathon Shlens. 2019. A learned representation for scalable vector graphics. InProceedings of the IEEE/CVF International Conference on Computer Vision. 7930–7939

    2019

  20. [28]

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambro- sio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al . 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation.arXiv preprint arXiv:2102.04664(2021)

    2021 arXiv

  21. [29]

    Xu Ma, Yuqian Zhou, Xingqian Xu, Bin Sun, Valerii Filev, Nikita Orlov, Yun Fu, and Humphrey Shi. 2022. Towards layer-wise image vectorization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16314– 16323

    2022

  22. [30]

    Kunato Nishina and Yusuke Matsui. 2024. SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM’s SVG Editing Capabilities. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8142–8147

    2024

  23. [31]

    Kunato Nishina and Yusuke Matsui. 2025. SVGEditBench V2: A Benchmark for Instruction-based SVG Editing.arXiv preprint arXiv:2502.19453(2025)

    2025 arXiv

  24. [32]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El- Nouby, et al. 2023. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193(2023)

    2023 arXiv

  25. [33]

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Zhen Liu, Tim Z Xiao, Katherine M Collins, Joshua B Tenenbaum, Adrian Weller, Michael J Black, and Bernhard Schölkopf

  26. [34]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  27. [35]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learnin...

    2021

  28. [36]

    Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J Mitra. 2021. Im2vec: Synthesizing vector graphics without vector supervision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7342–7351

    2021

  29. [37]

    Juan A Rodriguez, Shubham Agarwal, Issam H Laradji, Pau Rodriguez, David Vazquez, Christopher Pal, and Marco Pedersoli. 2023. Starvector: Generating scalable vector graphics code from images.arXiv preprint arXiv:2312.11556(2023)

    2023 arXiv

  30. [38]

    Christoph Schuhmann. 2022. Improved Aesthetic Predictor. https://github.com/ christophschuhmann/improved-aesthetic-predictor. Accessed: 2025-05-29

    2022

  31. [39]

    I-Chao Shen and Bing-Yu Chen. 2021. Clipgen: A deep generative model for clipart vectorization and synthesis.IEEE Transactions on Visualization and Computer Graphics28, 12 (2021), 4211–4224

    2021

  32. [40]

    Yiren Song, Xuning Shao, Kang Chen, Weidong Zhang, Zhongliang Jing, and Minzhe Li. 2023. Clipvg: Text-guided image manipulation using differentiable vector graphics. InProceedings of the AAAI conference on artificial intelligence, Vol. 37. 2312–2320

    2023

  33. [41]

    Hao Su, Xuefeng Liu, Jianwei Niu, Jiahe Cui, Ji Wan, Xinghao Wu, and Nana Wang. 2023. Marvel: Raster gray-level manga vectorization via primitive-wise deep reinforcement learning.IEEE Transactions on Circuits and Systems for Video Technology34, 4 (2023), 2677–2693

    2023

  34. [42]

    Zecheng Tang, Chenfei Wu, Zekai Zhang, Mingheng Ni, Shengming Yin, Yu Liu, Zhengyuan Yang, Lijuan Wang, Zicheng Liu, Juntao Li, et al. 2024. Strokenuwa: Tokenizing strokes for vector graphic synthesis.arXiv preprint arXiv:2401.17093 (2024)

    2024 arXiv

  35. [43]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530(2024)

    2024 arXiv

  36. [44]

    Vikas Thamizharasan, Difan Liu, Matthew Fisher, Nanxuan Zhao, Evangelos Kalogerakis, and Michal Lukac. 2024. Nivel: Neural implicit vector layers for text-to-vector generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4589–4597

    2024

  37. [45]

    Lucas Theis, Aäron van den Oord, and Matthias Bethge. 2015. A note on the evaluation of generative models.arXiv preprint arXiv:1511.01844(2015). Conference’17, July 2017, Washington, DC, USA Chen et al

    2015 arXiv

  38. [46]

    Yingtao Tian and David Ha. 2022. Modern evolution strategies for creativity: Fitting concrete images and abstract concepts. InInternational conference on computational intelligence in music, sound, art and design (part of evostar). Springer, 275–291

    2022

  39. [47]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

    2023 arXiv

  40. [48]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research9, 11 (2008)

    2008

  41. [49]

    Yael Vinker, Ehsan Pajouheshgar, Jessica Y Bo, Roman Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir. 2022. Clipasso: Semantically-aware object sketching.ACM Transactions on Graphics (TOG)41, 4 (2022), 1–11

    2022

  42. [50]

    Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, et al . 2023. Visionllm: Large language model is also an open-ended decoder for vision-centric tasks.Advances in Neural Information Processing Systems36 (2023), 61501–61513

    2023

  43. [51]

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing13, 4 (2004), 600–612

    2004

  44. [52]

    Ronghuan Wu, Wanchao Su, and Jing Liao. 2024. Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models.arXiv preprint arXiv:2411.16602(2024)

    2024

  45. [53]

    Ronghuan Wu, Wanchao Su, Kede Ma, and Jing Liao. 2023. Iconshop: Text-guided vector icon synthesis with autoregressive transformers.ACM Transactions on Graphics (TOG)42, 6 (2023), 1–14

    2023

  46. [54]

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. Human preference score: Better aligning text-to-image models with human preference. InProceedings of the IEEE/CVF International Conference on Computer Vision. 2096– 2105

    2023

  47. [55]

    Ximing Xing, Juncheng Hu, Guotao Liang, Jing Zhang, Dong Xu, and Qian Yu

  48. [56]

    Ximing Xing, Chuang Wang, Haitao Zhou, Jing Zhang, Qian Yu, and Dong Xu

  49. [57]

    arXiv preprint arXiv:2412.11102(2024)

    Empowering LLMs to Understand and Generate Complex Vector Graphics. arXiv preprint arXiv:2412.11102(2024)

    2024 arXiv

  50. [58]

    Zhongzheng Xu and Emily Wall. 2024. Exploring the capability of llms in per- forming low-level visual analytic tasks on svg data visualizations. In2024 IEEE Visualization and Visual Analytics (VIS). IEEE, 126–130

    2024

  51. [59]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    2025 arXiv

  52. [60]

    Ximing Xing, Haitao Zhou, Chuang Wang, Jing Zhang, Dong Xu, and Qian Yu. 2024. Svgdreamer: Text guided svg generation with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4546–4555

    2024

  53. [61]

    Peiying Zhang, Nanxuan Zhao, and Jing Liao. 2024. Text-to-vector generation with neural path representation.ACM Transactions on Graphics (TOG)43, 4 (2024), 1–13

    2024

  54. [62]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang

  55. [63]

    Yiying Yang, Wei Cheng, Sijin Chen, Xianfang Zeng, Jiaxu Zhang, Liao Wang, Gang Yu, Xingjun Ma, and Yu-Gang Jiang. 2025. OmniSVG: A Unified Scalable Vector Graphics Generation Model.arXiv preprint arXiv:2504.06263(2025)

    2025

  56. [64]

    Describe this image in brief

    Bocheng Zou, Mu Cai, Jianrui Zhang, and Yong Jae Lee. 2024. Vgbench: Evaluating large language models on vector graphics understanding and generation.arXiv preprint arXiv:2407.10972. SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation Conference’17, July 2...

    2024 arXiv

  57. [67]

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al . 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479(2025)

    2025 arXiv

  58. [69]

    Make a brief description of the style transfer that how it modifies the image 1 to image 2

  59. [70]

    Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general

  60. [71]

    You should refer to the score rubric

    After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric

  61. [72]

    Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)

    The output format should look as follows: "Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)"

  62. [73]

    Which of the following basic shapes is present in the image?

    Please do not generate any other opening, closing, and explanations. The instruction to evaluate: transfer the provided Image 1 to {style} Response to evaluate: the given Image 2 Score Rubrics: {rubric} Feedback: Prompt for Style Transfer You are a professional SVG designer wi...

    2017

  63. [74]

    Perceptual Question: Focus on the visual features of the icon such as shapes, number of elements, or spatial arrangement

  64. [75]

    Focus on what the icon represents or where it might typically appear

    Semantic Question: Explore the meaning, function, or use- case of the icon. Focus on what the icon represents or where it might typically appear. Format requirements: - Question: [question text] Options: A) [option A]; B) [option B]; C) [option C]; D) [option D] Answer: [corre...

  65. [76]

    2.Output the answer in the format ’Answer: X’ in the last line, where X is one of A, B, C, or D

    Answer the given multiple choice question below according to the svg code. 2.Output the answer in the format ’Answer: X’ in the last line, where X is one of A, B, C, or D. SVG Code: {svg_image} Question: {question} Options: {options_str} Important Notes: - You should answer ex...

    2017

  66. [77]

    Remove metadata and editor information - Clear metadata, comments, and unnecessary attributes generated by design software - Remove hidden elements and empty tags

  67. [78]

    Path optimization - Simplify path data, reduce control points - Lower decimal precision (1-2 places is usually sufficient) - Merge similar paths

  68. [79]

    Attribute and style processing - Remove redundant or default attribute values - Merge duplicate styles - Optimize color representation (e.g., #000 instead of #000000)

  69. [80]

    Structure optimization - Remove unnecessary grouping and nesting - Optimize IDs and class names - Ensure viewBox is set correctly

  70. [81]

    Compression and fine-tuning - Remove unnecessary whitespace and units - Use short commands instead of long format commands After optimization, please strictly return the complete optimized code in the following format:Answer: {SVG code},Provide the complete optimized code only...

  71. [2018]

    InProceedings of the IEEE conference on computer vision and pattern recognition

    The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition. 586–595

  72. [2023]

    Diffsketcher: Text guided vector sketch synthesis through latent diffusion models.Advances in Neural Information Processing Systems36 (2023), 15869– 15889

    2023

  73. [2024]

    Can Large Language Models Understand Symbolic Graphics Programs? arXiv preprint arXiv:2408.08313(2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.