Pith. sign in

REVIEW 3 major objections 4 minor 22 references

SVGEval argues that current multimodal models judge SVG renderings well on semantics and aesthetics but poorly on spatial and structural quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 17:24 UTC pith:X4QSCQI2

load-bearing objection Useful four-aspect SVG quality benchmark with a broadly supported spatial/structural gap, but the scorer's headline numbers are unverifiable without a stated train/eval split. the 3 major comments →

arxiv 2608.01977 v1 pith:X4QSCQI2 submitted 2026-08-03 cs.CV

SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation

Yiming Wang , Ye Chen , Hanqi Chen , Bingbing Ni This is my paper
classification cs.CV
keywords SVG evaluationperceptual qualitymultimodal large modelsvisual groundingbenchmarkspatial layoutgeometric structureexplainable scoring
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

SVGEval is a benchmark and scorer for judging the perceptual quality of text-to-SVG generation. Its central claim is that evaluation should be grounded in the rendered image, not just in SVG code, and that quality splits into four separable aspects: semantic agreement with the prompt, aesthetic finish, spatial layout, and geometric structure. Using this benchmark across representative multimodal models, the paper documents a consistent weakness: models do reasonably well on semantic and aesthetic judgments but are markedly worse on spatial-layout and geometric-structure judgments. The paper then trains an explainable scorer that takes prompt, rendered image, and SVG code, outputs per-aspect 1–5 scores with textual rationales, and shows that visual grounding plus rationale supervision is what drives most of the improvement. If the claim holds, SVG evaluation and model training would need to shift from raster-image metrics and code-only checks to vision-and-code joint assessment.

Core claim

The paper's discovery is that current multimodal evaluation of SVG is systematically lopsided: models can identify whether an image matches a prompt and whether it looks polished, but they consistently misjudge spatial layout and geometric integrity, which are the dimensions that most often determine whether a vector graphic is actually usable. The proposed solution is the SVGEval rubric and dataset, with gold labels defined over the rendered outcome, and an explainable scorer trained with explicit visual renderings, SVG code, and reasoning supervision that produces evidence-grounded rationales. The scorer's MAE of 0.5772 on 1–5 scoring, with adjacent accuracy of 71.95%, beats all general-pu

What carries the argument

The central object is the SVGEval benchmark itself: a two-part, difficulty-progressive test built from 1,000 real prompts and roughly 10,000 candidate SVGs (real and model-generated), each instance pairing a text prompt, a rasterized rendering, and SVG source code. Part I asks yes/no perception questions; Part II asks for 1–5 scores on four dimensions—Aesthetic, Semantic, Spatial, Structural—with gold labels determined by multi-round human annotation and expert adjudication over the rendered outcome. The mechanism that carries the argument is the dimensional decomposition plus the hierarchical attribution rule: penalize explicit geometry and layout defects first under Spatial and Structural,

Load-bearing premise

The load-bearing premise is that the benchmark's gold labels, defined over the rendered outcome under a fixed rasterization pipeline and approved by human annotators, are a reliable ground truth; if rasterization artifacts or label subjectivity shift those scores, the measured model gap and the scorer's training signal inherit the error.

What would settle it

Re-render a random subset of SVGEval instances with a different SVG rasterizer and anti-aliasing settings, then have fresh annotators score them without seeing the original labels; if model rankings and the Spatial/Structural gap flip, the fixed-rendering premise fails. Alternatively, retrain the scorer without visual renderings but with substantially more code-only data; if its MAE approaches that of the vision-grounded scorer, the claim that visual grounding is crucial would be weakened.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the benchmark findings are right, raster-image metrics and CLIP-style similarity scores will keep overrating text-to-SVG outputs because they miss the precise spatial and structural defects that humans punish.
  • Evaluator models should be judged and trained with both rendered images and SVG code: code-only evaluation loses calibration, while image-only evaluation loses structural cues such as path closure and layer ordering.
  • Reasoning supervision is not an add-on: score-only training produces a black-box regressor with worse calibration and failures that are hard to audit.
  • High accuracy on binary diagnosis does not transfer to calibrated ordinal scoring; detecting discrete defects is different from rating quality on a 1–5 scale.
  • The four-aspect decomposition (Aesthetic, Semantic, Spatial, Structural) can serve as a diagnostic lens for attributing generation failures to specific perceptual causes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural but untested extension is to use the scorer's textual rationales as a reward signal or preference label for steering SVG generation models, since the rationales name concrete defects that could be optimized against.
  • Because the gold labels are defined over rendered outputs under a fixed rasterization pipeline, re-rendering the same SVGs with a different rasterizer or anti-aliasing settings could shift scores; a stress test of that sort would reveal how much of the reported model gap is rendering-dependent.
  • The four-aspect rubric likely transfers to other structured visual formats such as diagrams, icons, and UI layouts, where layout and topology matter more than texture; adapting the benchmark to those domains is a direct test of the framework's generality.
  • One testable prediction implied by the design: models trained to maximize CLIP or pixel similarity will show small gains on SVGEval unless explicitly supervised on spatial and structural dimensions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces SVGEval, a vision-grounded benchmark for evaluating text-to-SVG generation quality. The benchmark defines four rubrics—Aesthetic, Semantic, Spatial, Structural—and two tasks: a Part I binary yes/no perception task (500 instances) and a Part II 1–5 Likert multi-aspect scoring task (500 instances). Models are evaluated with prompt, rendered image, and SVG code, while gold labels are defined over the rendered outcome. The authors evaluate a range of MLLMs, reporting that models tend to be stronger on Semantic and Aesthetic judgments and weaker on Spatial and Structural judgments. They then train SVGEval-Scorer, an explainable quality scorer based on Qwen3-VL, using reasoning distillation from a teacher model conditioned on gold scores. The scorer is reported to achieve MAE 0.5772 and Adj. Acc. 71.95%, with ablations showing that removing reasoning supervision, SVG code input, or the rendered image degrades performance. The paper concludes that vision-grounded, rationale-supervised evaluation is important for SVG quality assessment.

Significance. If the scorer results are valid, the paper makes a useful contribution: it provides a human-annotated, vision-grounded benchmark for SVG quality that goes beyond raster-image metrics, and an explainable scorer with structured rationales. The benchmark observation—that current MLLMs struggle more on Spatial and Structural judgments than on Semantic and Aesthetic judgments—is potentially valuable for guiding future SVG-generation research. The human gold labels are external to the models, so the benchmark observation is not circular. The scorer design is reasonable: using both rendered image and SVG code is a sensible way to exploit complementary cues, and the ablations target the right components. However, the lack of a stated train/evaluation split for the scorer is a serious gap that currently undermines the headline numerical claims. The paper also does not report inter-annotator agreement or confidence intervals, and the 'consistent' cross-dimensional pattern has notable exceptions. With a clarified split and more careful reporting of per-model variability, the contributions would be solid.

major comments (3)
  1. [§4.2, §5, Tables 2–3] The training/evaluation setup for SVGEval-Scorer is underspecified. Section 4.1 states that teacher rationales are generated using the prompt, rendered image, SVG code, and the gold score, and Section 4.2 states that supervised fine-tuning is performed on the synthesized dataset. Section 5 reports the scorer's MAE 0.5772 and Adj. Acc. 71.95%, but no train/eval split is described. If the synthesized triples are derived from the same 500 Part II instances whose gold scores are used for evaluation, then the reported results are train-on-test and are invalid. This affects not only Table 2 but also the per-dimension results and all three ablations in Table 3. The authors must state exactly how many instances were used for training, how the train/eval split was created, and ensure the evaluation instances are disjoint from the distillation set. Without this, the headline scorer improvement can
  2. [§5.2, Table 3] The claim that models 'exhibit substantially larger errors on Spatial and Structural than on Aesthetic and Semantic' is not as consistent as stated. In Table 3, Aesthetic MAE is the worst dimension for Doubao-Seed (1.5000), Grok-4.1-Fast (1.7419), and GLM-4.5V (1.6967). Also, Gemini-3 Pro has Structural MAE 0.6111, which is lower than its Semantic MAE 0.6160. The overall trend may hold in aggregate, but the 'consistent gap' language overstates the evidence. A per-model breakdown or a paired statistical test across models would better support the claim; alternatively, the conclusion should be softened to reflect the exceptions.
  3. [§3.2, §5] No inter-annotator agreement, confidence intervals, or variance estimates are reported for the Part II human gold scores. Since the Likert-scale ratings are explicitly acknowledged as subjective (Section 7), MAE differences of 0.05–0.1 between models—and the small per-dimension differences that motivate the 'consistent gap'—may be within label noise. Reporting Cohen's kappa or Krippendorff's alpha, or at least score distributions, would strengthen the reliability of the benchmark and the conclusions drawn from it.
minor comments (4)
  1. [§1 and throughout] The paper uses 'T able' and 'Fig. 3a' formatting inconsistently (e.g., 'T able 1' in the text and 'T able 3' in captions). Please ensure all table/figure references are correctly formatted.
  2. [§3.2] The real-to-generated sampling ratio of 1:4 and the balanced positive/negative ratio for Part I are introduced as design choices, but no sensitivity analysis is provided. A brief justification or ablation would help the reader understand how much these choices affect benchmark difficulty.
  3. [§5.3] The authors note that removing SVG code input 'only mildly affects overall performance' but causes larger drops on Spatial and Structural. This is interesting, but the magnitudes are not statistically compared. Reporting confidence intervals or error bars for the ablations would make the differences more convincing.
  4. [General] No data or model weights are released. Given that the scorer's evaluation depends on the exact train/eval split, releasing at least the training/evaluation indices would improve reproducibility. The paper should also clarify whether the benchmark annotations will be made publicly available.

Circularity Check

2 steps flagged

Scorer's headline result is potentially train-on-test because no held-out split is reported; rationales are score-conditioned post hoc justifications.

specific steps
  1. fitted input called prediction [Sec. 4.1–4.2 (training data synthesis and SFT); Sec. 5.1/Table 2 (evaluation)]
    "Given the prompt, rendered image, SVG code, a target dimension, and its gold score, a teacher model ... is instructed to act as an analyst: 'The score on dimension d is s (assigned by human experts). Identify concrete visual/code evidence that supports this score according to the rubric.' ... We perform supervised fine-tuning on the synthesized dataset."

    The scorer is SFT on image–rationale–score triples whose score labels are human gold scores; the only gold-scored set described is the 500-instance Part II benchmark (Sec. 3.2). Section 5 reports the scorer's MAE/Adj. Acc. on that same Part II task without stating any train/eval split. Thus the headline MAE 0.5772 and the ablations can be a memory of the training labels rather than an independent prediction; the 'improvement' over MLLMs is forced by construction unless a disjoint split is demonstrated.

  2. self definitional [Sec. 4.1, 'Score-Conditioned Rationalization'; Sec. 4 Goal]
    "Given the prompt, rendered image, SVG code, a target dimension, and its gold score, a teacher model ... is instructed to act as an analyst: 'The score on dimension d is s (assigned by human experts). Identify concrete visual/code evidence that supports this score according to the rubric.'"

    The rationale is generated to justify a predetermined gold score, so the 'evidence' is selected after the fact to match the label. The paper then presents these rationales as 'evidence-grounded' and attributes the scorer's gains to 'reasoning supervision' (Sec. 5.3, 'w/o reasoning' ablation). The explanation is thus defined in terms of the score it is supposed to ground; it cannot independently demonstrate that the score follows from visual/code evidence.

full rationale

The human-labeled benchmark observations (Part I accuracy and general-MLLM Part II MAE) are externally grounded and not circular: gold scores come from multi-round human annotation, and the MLLMs are evaluated on fixed instances they have not been trained on. The circularity is localized to the SVGEval-Scorer contribution. Section 4.1 constructs training triples from the same gold scores used to define Part II, and Section 4.2 fine-tunes on them. Section 5 reports the scorer's Part II MAE/Adj. Acc. with no train/eval split and describes no separate gold-scored training set, so the reported 0.5772 MAE is, under the stated protocol, a fit to the evaluation labels rather than an independent prediction; the same applies to the three ablations. Additionally, the rationale supervision is score-conditioned: the teacher is told the gold score and asked to produce supporting evidence, so the resulting rationales are post hoc and cannot independently ground the scores, yet the paper presents them as evidence-grounded and credits them for the performance gain. The benchmark's model-gap finding (Spatial/Structural harder) is independent and would survive a corrected scorer evaluation, which is why the score is 6 rather than higher.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The paper has no formal derivation; its claims rest on the validity of the human-annotation protocol, the four-aspect rubric, the rasterization pipeline, and the teacher-rationale procedure. Design parameters include the label balance and the real-to-generated sampling ratio.

free parameters (2)
  • Part II real-to-generated sampling ratio = 1:4
    Hand-chosen in Sec. 3.2 to make the score distribution relatively balanced across bins; not derived from any theory.
  • Part I label balance = approximately 1:1 yes/no
    Curated in Sec. 3.2 to avoid yes/no response bias; a deliberate design choice rather than a measured quantity.
axioms (5)
  • domain assumption Gold labels are defined over a fixed rasterized rendering of the SVG and are treated as the ground truth for quality.
    Sec. 3.1 states the rasterized rendering is the primary evaluation target; Sec. 7 admits subtle rasterization artifacts can influence judgments, so any systematic divergence from human perception propagates through all results.
  • domain assumption The four dimensions (Aesthetic, Semantic, Spatial, Structural), with the hierarchical attribution rule, form a sufficient and separable decomposition of SVG quality.
    Sec. 3.1 defines the rubric and admits dimensions overlap; no validation is provided that four dimensions capture all relevant quality.
  • domain assumption Multi-round human annotation with expert adjudication produces reliable gold scores despite known subjectivity.
    Sec. 3.2 and Sec. 7 describe the process but report no inter-annotator agreement or label-noise statistics.
  • domain assumption Multimodal models can validly judge SVG quality when given prompt, rendering, and code together.
    Sec. 3.1 fixes the evaluation setting; the benchmark's validity depends on this being a meaningful perception task for MLLMs.
  • ad hoc to paper Score-conditioned rationales from a teacher model are a valid training signal for explainable scoring.
    Sec. 4.1 instructs Qwen3-VL-235B-A22B to rationalize a given gold score; this is a post hoc rationalization procedure that may not reflect independent expert reasoning.

pith-pipeline@v1.3.0-daily-deepseek · 10437 in / 14381 out tokens · 149330 ms · 2026-08-04T17:24:23.202600+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation." pith.science (2026). https://pith.science/paper/X4QSCQI2

@misc{pith2026260801977,
  author       = {Pith},
  title        = {Pith review of: SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X4QSCQI2}},
  note         = {Machine review of arXiv:2608.01977}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal large models are increasingly used to generate scalable vector graphics (SVG), but reliable evaluation remains underexplored. Existing protocols are often code-centric or borrow raster-image metrics after rendering SVGs, which fail to reflect human perception and overlook SVG-specific qualities such as geometry and spatial composition. We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment. SVGEval explicitly incorporates visual renderings to evaluate whether models can judge the rendered outcome rather than only inspect SVG code, and provides high-quality annotations obtained via multi-round human labeling with expert refinement. Systematic evaluations across representative multimodal models reveal a clear gap: models perform relatively well on semantic alignment and aesthetics, yet struggle on geometry- and layout-related judgments. Building on SVGEval, we train an explainable SVG quality scorer that outputs multi-aspect scores with textual rationales. Ablations show that explicit visual grounding and reasoning supervision are crucial, especially for spatial and geometric assessment. SVGEval offers a reliable testbed and practical scorer for evaluating and improving SVG generation in the era of multimodal models.

Figures

Figures reproduced from arXiv: 2608.01977 by Bingbing Ni, Hanqi Chen, Ye Chen, Yiming Wang.

Figure 1
Figure 1. Figure 1: Overview of SVGEval. We define a four-aspect rubric and build a two-part benchmark (binary diagnosis and 1–5 scoring) with visual grounding. SVGEval further enables training an explainable scorer that predicts aspect-wise scores with evidence￾grounded rationales from the prompt, rendered image, and SVG code. 3 SVGEval Benchmark 3.1 Evaluation Principle Evaluation Target. Although SVG is represented as a pr… view at source ↗
Figure 2
Figure 2. Figure 2: Data coverage of SVGEval. Left: topic-aware prompt sampling improves category coverage. Right: the candidate pool provides a continuous quality gradient for benchmarking and scorer training. failure patterns (especially spatial and structural defects), forming a continu￾ous quality gradient from severely broken to near-correct; (2) real SVGs act as high-quality anchors, providing a practical upper bound th… view at source ↗
Figure 3
Figure 3. Figure 3: Benchmark results on SVGEval. Models are strong on semantic/aesthetic judgments but weaker on SVG-specific spatial/structural evaluation; SVGEval-Scorer closes the gap, especially on spatial and structural aspects. 4.2 Model Architecture and Training Recipe Base Model and Inputs. According to the results of our benchmark in Sec. 5.2, we adopt a strong open-source multimodal model as the backbone (Qwen3-VL … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 14 linked inside Pith

  1. [1]

    In: Proceedings of the 33rd ACM International Conference on Multimedia

    Chen, H., Zhao, Z., Chen, Y., Liang, Z., Ni, B.: Svgthinker: Instruction-aligned and reasoning-driven text-to-svg generation. In: Proceedings of the 33rd ACM International Conference on Multimedia. p. 11004–11012. MM ’25, ACM (Oct 2025).https://doi.org/10.1145/3746027.3755392,http://dx.doi.org/10. 1145/3746027.3755392

    arXiv 2025

  2. [2]

    Chen, S., Dong, X., Xu, H., Wu, X., Tang, F., Zhang, H., Yan, Y., Wu, L., Zhang, W.,Hou,G.,Shen,Y.,Lu,W.,Zhuang,Y.:Svgenius:Benchmarkingllmsinsvgun- derstanding, editing and generation (2025),https://arxiv.org/abs/2506.03139

    Pith/arXiv arXiv 2025

  3. [3]

    Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X., Wu, Y., Ji, R., Shan, C., He, R.: Mme: A comprehensive evaluation benchmark for multimodal large language models (2025),https://arxiv.org/ abs/2306.13394

    Pith/arXiv arXiv 2025

  4. [4]

    Hessel, J., Holtzman, A., Forbes, M., Bras, R.L., Choi, Y.: Clipscore: A reference- free evaluation metric for image captioning (2022),https://arxiv.org/abs/2104. 08718

    2022

  5. [5]

    Heusel,M.,Ramsauer,H.,Unterthiner,T.,Nessler,B.,Hochreiter,S.:Ganstrained by a two time-scale update rule converge to a local nash equilibrium (2018),https: //arxiv.org/abs/1706.08500

    Pith/arXiv arXiv 2018

  6. [6]

    Kim, S., Shin, J., Cho, Y., Jang, J., Longpre, S., Lee, H., Yun, S., Shin, S., Kim, S., Thorne, J., Seo, M.: Prometheus: Inducing fine-grained evaluation capability in language models (2024),https://arxiv.org/abs/2310.08491

    Pith/arXiv arXiv 2024

  7. [7]

    Lin, K.Q., Zheng, Y., Ran, H., Zhu, D., Mao, D., Li, L., Torr, P., Wang, A.J.: Vcode: a multimodal coding benchmark with svg as symbolic visual representation (2025),https://arxiv.org/abs/2511.02778

    arXiv 2025

  8. [8]

    Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., Zhu, C.: G-eval: Nlg evaluation using gpt-4 with better human alignment (2023),https://arxiv.org/abs/2303.16634

    Pith/arXiv arXiv 2023

  9. [9]

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision (2021),https://arxiv.org/abs/ 2103.00020

    Pith/arXiv arXiv 2021

  10. [10]

    Rodriguez, J.A., Puri, A., Agarwal, S., Laradji, I.H., Rodriguez, P., Rajeswar, S., Vazquez, D., Pal, C., Pedersoli, M.: Starvector: Generating scalable vector graphics code from images and text (2025),https://arxiv.org/abs/2312.11556

    Pith/arXiv arXiv 2025

  11. [11]

    Rodriguez, J.A., Zhang, H., Puri, A., Shariff, A., lin, M., Xie, X., Zhang, T., Pra- manik, R., Rajeswar, S., Taslakian, P., Gella, S., Vazquez, D., Pal, C., Pedersoli, M.: Vectorgym: A multi-task benchmark for SVG code generation and manipula- tion (2026),https://openreview.net/forum?id=DBFbNT65xO

    2026

  12. [12]

    SVG Repo: Svg repo: Free svg icons and vectors.https://www.svgrepo.com (2026), accessed: 2026-02-28

    2026

  13. [13]

    org/TR/SVG2/(2018),https://www.w3.org/TR/SVG2/, w3C Candidate Recom- mendation, accessed 2026-03-01

    W3C SVG Working Group: Scalable vector graphics (svg) 2.https://www.w3. org/TR/SVG2/(2018),https://www.w3.org/TR/SVG2/, w3C Candidate Recom- mendation, accessed 2026-03-01

    2018

  14. [14]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wang, Z., Zhou, S., He, S., Huang, H., Yang, L., Zhang, Z., Cheng, X., Ji, S., Jin, T., Zhao, H., Zhao, Z.: Spatialclip: Learning 3d-aware image representations from spatially discriminative language. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 29656–29666 (June 2025)

    2025

  15. [15]

    Wang et al

    Wu, H., Zhang, Z., Zhang, E., Chen, C., Liao, L., Wang, A., Li, C., Sun, W., Yan, Q., Zhai, G., Lin, W.: Q-bench: A benchmark for general-purpose foundation models on low-level vision (2024),https://arxiv.org/abs/2309.14181 16 Y. Wang et al

    Pith/arXiv arXiv 2024

  16. [16]

    Xing, X., Hu, J., Liang, G., Zhang, J., Xu, D., Yu, Q.: Empowering llms to un- derstand and generate complex vector graphics (2025),https://arxiv.org/abs/ 2412.11102

    Pith/arXiv arXiv 2025

  17. [17]

    Yang, Y., Cheng, W., Chen, S., Zeng, X., Yin, F., Zhang, J., Wang, L., Yu, G., Ma, X., Jiang, Y.G.: Omnisvg: A unified scalable vector graphics generation model (2025),https://arxiv.org/abs/2504.06263

    arXiv 2025

  18. [18]

    Yuksekgonul, M., Bianchi, F., Kalluri, P., Jurafsky, D., Zou, J.: When and why vision-language models behave like bags-of-words, and what to do about it? (2023), https://arxiv.org/abs/2210.01936

    Pith/arXiv arXiv 2023

  19. [19]

    Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric (2018),https://arxiv.org/ abs/1801.03924

    Pith/arXiv arXiv 2018

  20. [20]

    Zhang, Z., Wu, H., Zhang, E., Zhai, G., Lin, W.: Q-bench+: A benchmark for multi-modal foundation models on low-level vision from single images to pairs (2024),https://arxiv.org/abs/2402.07116

    Pith/arXiv arXiv 2024

  21. [21]

    Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I.: Judging llm-as-a-judge with mt-bench and chatbot arena (2023),https://arxiv.org/abs/2306.05685

    Pith/arXiv arXiv 2023

  22. [22]

    Zou, B., Cai, M., Zhang, J., Lee, Y.J.: Vgbench: Evaluating large language models on vector graphics understanding and generation (2024),https://arxiv.org/abs/ 2407.10972

    Pith/arXiv arXiv 2024