Pith. sign in

REVIEW 4 major objections 5 minor 48 references

MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MT-Web2Code, the first multi-turn benchmark for iterative web UI repair, finds that 13 frontier coding agents can generate whole pages but fail to reconstruct one region without disturbing the rest, and that their errors grow over successiv

desk verdict A genuinely novel multi-turn UI-editing benchmark with a solid deterministic micro track, but the headline macro claims rest on an unvalidated VLM judge. read the letter →

arxiv 2608.03474 v1 pith:HC3QEBQA submitted 2026-08-04 cs.CV

Qiming Li , Shujie Hu , Haohan Liu , Xiaocheng Feng , Songxiang Liu , Guanglu Wan This is my paper
classification cs.CV
keywords multi-turnbenchmarkwebUIcodingregionalreconstructionlocalizedmodificationreverse-corruptiontrajectoriesVLM-as-judgepixelalignmentevaluationerrorpropagation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MT-Web2Code is a benchmark for a task that existing benchmarks skip: editing an existing webpage over multiple turns, either reconstructing a missing semantic region from a visual reference or fixing several localized defects while leaving everything else untouched. The paper argues that this matches real frontend work better than single-turn full-page generation, and that current vision-language coding agents are not reliable at it. To get deterministic repair trajectories without per-turn human labeling, the paper builds a Reverse-Corruption Trajectory Engine: golden pages are broken in a fixed sequence, and the reverse order becomes the repair task, so every turn has a unique correct answer. Evaluation is dual-axis, separating target-region fidelity from preservation of unaffected content. Across 102 pages in 16 domains, 13 agents show a consistent pattern: weak regional reconstruction, imprecise localized edits, and error snowballing over turns.

What carries the argument

The load-bearing mechanism is the Reverse-Corruption Trajectory Engine. It starts from a golden page and applies a deterministic sequence of corruptions—structural deletion for macro tasks, bundled stylistic/structural operator edits for micro tasks—so that reversing the corruption order produces a repair trajectory in which every turn has a unique ground-truth target. Two supporting components make this work: Element Fingerprints (hashes of tag, text, and attributes) that track nodes across deletions and layout reflows, and anchor-based grounding that marks the insertion bounding box for a deleted region using its nearest surviving siblings. The evaluation side is carried by the dual-axis p

What would settle it

Have human raters score a sample of macro-level reconstructions on the same five dimensions and compare their ordering of agents with Kimi-K2.6's ordering; if humans disagree with the judge, or the judge's scores do not rank order agents the way humans do, the benchmark's conclusion that agents struggle to reconstruct regions is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that multi-turn regional reconstruction and localized modification form a distinct capability, one that current benchmarks do not measure and current coding agents do not master. MT-Web2Code operationalizes this with two tracks: Macro-Level Regional Reconstruction deletes a whole semantic region (for example a nav or section) and asks the agent to recreate it inside a red bounding box from a cropped golden reference, while Micro-Level Localized Modification injects three fine-grained defects per turn (layout, elements, text, color, spacing) and asks the agent to fix them with no reference crop. Each task runs for five turns, with the agent's own output feeding th

Load-bearing premise

The headline conclusions rest on a single AI model (Kimi-K2.6) giving trustworthy visual-quality scores, and the paper validates that judge only against other AI judges, not against human raters.

Editorial extensions

If this is right

  • Single-turn UI generation scores do not transfer to multi-turn editing: the best macro-level reconstructor is not the best micro-level editor, and several strong models collapse on chained tasks.
  • Evaluation of UI coding agents should report in-box fidelity and out-of-box preservation separately, because aggregate page similarity hides localized failure.
  • Error snowballing is a measurable, separate failure mode: models perform comparably when each turn starts from a golden state but degrade when they must build on their own previous output.
  • The deterministic metrics could provide dense per-turn reward signals for training iterative UI coding agents, removing the need for human preference annotation.
  • Judge capability is separate from coding capability: a strong coder can be a lenient, unreliable judge, so benchmark designers must validate judges rather than assume model quality transfers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reverse-corruption recipe generalizes, the same cheap trajectory construction could be applied to other structured-output domains—mobile app screens, dashboards, emails—to generate multi-turn repair benchmarks without human annotation.
  • The finding that adding captions to the visual reference hurts some models suggests a testable hypothesis: textual priors compete with visual grounding, and models trained to balance the two might benefit where current ones degrade.
  • The macro-level results depend on one judge model; a human-rated validation study could either strengthen the benchmark or require recalibrating its absolute scores, though the qualitative error-snowballing pattern is less likely to flip.
  • Because the micro-level metric is deterministic and pixel-grounded, it could be plugged directly into a reinforcement-learning training loop as a reward; if it works, the boundary between benchmark and training signal disappears.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces MT-Web2Code, a benchmark for multi-turn web UI repair consisting of 102 tasks across 16 vertical domains. Tasks come in two granularities: Macro-Level Regional Reconstruction (53 tasks, 5 turns each), where an agent must regenerate a deleted semantic region from a visual reference, and Micro-Level Localized Modification (49 tasks, 5 turns each), where an agent must repair several localized structural/stylistic defects. Trajectories are generated by a Reverse-Corruption Trajectory Engine that injects deterministic defects into golden pages and defines the repair target uniquely at each turn. Evaluation is dual-axis: the macro track uses a 5-dimensional VLM rubric plus an overall rating averaged into an in-box score, while the micro track uses deterministic SSIM-based pixel alignment inside and outside a repair footprint; both combine as Score = 0.8 × S_inbox + 0.2 × S_outbox (Eq. 12). Experiments on 13 frontier coding agents report that models struggle with regional reconstruction, preserve out-of-box content well in micro tasks but vary in in-box repair accuracy, and suffer from error snowballing across turns.

Significance. If the evaluation is trustworthy, the benchmark fills a real gap: existing web UI benchmarks are largely single-turn full-page generation, while iterative regional reconstruction and localized modification are closer to practical frontend workflows. The reverse-corruption construction is a sensible way to obtain deterministic repair trajectories without turn-level human annotation, and the dual-axis scoring (in-box fidelity vs. out-of-box preservation) directly targets the phenomenon of collateral damage during editing. The micro-level metrics are deterministic and recomputable from the stated formulas, which is a strength. The headline conclusion that current agents 'struggle' with regional reconstruction, however, rests on a VLM judge whose scores are not anchored to human ratings, so the quantitative macro-level findings are not yet established. The benchmark and protocol are promising and potentially useful for future training-signal research, but the paper needs additional validation before the macro-level claims can be accepted.

major comments (4)
  1. [Section 5.2, Table 5; Eq. (12)] The macro-level conclusion that agents 'struggle to faithfully reconstruct target regions' depends entirely on the VLM-judged rubric score (Eq. 8) weighted at 0.8 in the final score (Eq. 12). Table 5 validates Kimi-K2.6 only against two other VLM judges (Gemini-3.5-Flash, Gemini-3.1-Pro-Preview) on K2.6 reconstructions. There is no human-rater calibration, no inter-annotator agreement, and no evidence for the assertion in Section 5.2 that Kimi-K2.6 scores 'better align with human perception.' Without a human ground truth, the absolute scores (e.g., 65.5 vs. 63.7) and the 'struggle' headline are unanchored. Please add a human study on a subset (e.g., scoring 30-50 reconstructions by multiple raters, reporting judge-human correlation/calibration and IAA) or otherwise externally ground the rubric.
  2. [Section 4.1; Section 3.5] There is a circularity risk: Kimi-K2.6 is used as the corrupt agent, as the difficulty filter for micro tasks, as the VLM judge for macro scores, and is itself one of the 13 evaluated agents. Table 5 only cross-validates judges against other VLM judges, all of which may share rendering-based biases; it does not rule out a family-specific or corruption-specific bias in the macro scores. Concretely, please (a) evaluate the judge's scores against human labels on outputs from all 13 agents, and/or (b) re-score macro outputs with an independent judge not included in the evaluated set and report ranking stability and rank correlations. This is needed to support the claim that the macro ranking reflects coding ability rather than judge bias.
  3. [Eqs. (4), (7), (9); Section 3.5] Several load-bearing thresholds and protocol details are unspecified: tau_w, tau_min^h, tau_max^h in Eq. (4), tau_vis in Eq. (7), tau in Eq. (9), and the 0.8/0.2 weighting in Eq. (12). The rubric's five-band anchors and equal dimension weights are also not operationally defined. Without these values, the benchmark cannot be reproduced and the sensitivity of results to cutoff choices cannot be assessed. Please report all exact values, provide the full judge prompt and cropping/masking procedure, and ideally include a sensitivity analysis (e.g., varying tau_vis or the score weight) to show that the main conclusions are robust.
  4. [Table 3; Figure 4; Section 4.2] All results are stated as averages over three independent runs, but no error bars, variances, or significance tests are reported. Several model differences are small (e.g., Macro Kimi-K2.6 63.7 vs. Claude-4.7-Opus 63.0; Micro Doubao-Seed-2.0-Pro 83.5 vs. Qwen3.5-Plus 83.3), and the text claims relative 'outperform' behavior. Reporting only point averages makes these comparisons unverifiable. Please provide per-run standard deviations or confidence intervals, and use a paired test where rankings are claimed. Similarly, Figure 4 shows per-turn error snowballing without error bars; the gap over turns could be within noise.
minor comments (5)
  1. [General] The paper states evaluation code and data 'will soon be released' but provides no repository URL or release plan. For a benchmark paper, a URL or supplementary archive is expected at submission time, especially since the macro-judge protocol depends on prompt details that cannot be fully conveyed in the text.
  2. [Section 3.4, Eq. (6)] The notation in Eq. (6) is unclear: 'dim(o_i)' is used before the defect operator dimensions are formally defined, and the partial order symbol '≺' is not explicitly defined for DOM nodes. Please clarify.
  3. [Table 3] The star notation on 'Kimi-K2.6*' in the Micro-Level block is unexplained. It likely denotes that Kimi-K2.6 served as judge/curator, but this must be stated in the caption or text.
  4. [Table 4] The Diff. rows are described as 'performance change (%)' but the numbers are percentage points (e.g., +6.8). Please correct the wording.
  5. [Section 3.2] The Element Fingerprint ID in Eq. (3) is defined as a hash of tag/text/attr; collisions are possible in principle. While not likely to affect the experiments, a note on collision handling would make the deterministic tracking claim precise.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark pipeline is self-contained, and the unvalidated VLM judge is a validity concern, not a by-construction reduction.

full rationale

I walked the claimed derivation chain: golden pages are corrupted by the Reverse-Corruption Trajectory Engine, agents reverse the trajectory, and scores come from either a deterministic pixel-grounded metric (micro) or a VLM rubric (macro). No equation defines a result in terms of itself. The macro rubric dimensions are explicitly 'these dimensions along which Stage-2 defects are organized,' but that is task-aligned rubric design, not a circular reduction: the golden page is an independent reference, and the judge's output is a posterior assessment, not an input. Kimi-K2.6 is used as corrupt agent, judge, and one of the 13 evaluated agents; this is a possible conflict of interest and a reproducibility weakness, but it is not circular because no fitted parameter or self-derived ground truth enters the scoring. The micro score is computed by SSIM against golden renders and requires no model judgment. Section 5.2 claims Kimi-K2.6 'better align[s] with human perception' without any human-rater calibration (Table 5 only compares VLM judges against each other). That is an unsupported load-bearing assertion, which I flag as a correctness risk, not as circularity: human-alignment is not an input to the scores, and the claim could be false without making any equation self-referential. No load-bearing self-citations appear; references to Kimi K2.5, GLM, and other models are external technical reports. The central claims that agents struggle at reconstruction and localization are empirical outcomes of the benchmark, not equivalences to its construction choices. Therefore the paper does not exhibit circularity by the standards of Eq. X = Eq. Y or fitted-parameter-renamed-as-prediction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claims rest mostly on design choices such as thresholds and weights, and on assumptions about rendering determinism, SSIM validity, VLM-judge reliability, and fingerprint uniqueness. The heaviest unverified burden is the VLM judge used for macro scores.

free parameters (5)
  • Macro region thresholds tau_w, tau_min^h, tau_max^h = not stated
    Eq (4) hand-set bounds that make deleted regions visually substantial yet localized; no sensitivity analysis is provided.
  • Micro visibility threshold tau_vis = not stated
    Eq (7) resamples defects below a minimum pixel delta; the cutoff determines which defects appear and is not reported.
  • Repair footprint threshold tau = not stated
    Eq (9) defines the pixel-difference mask Omega between golden and broken pages; scores change with this threshold.
  • Score weighting 0.8 / 0.2 = 0.8 and 0.2
    Eq (12) emphasizes in-box fidelity over out-of-box preservation; no ablation or justification is given for these weights.
  • Rubric scale levels and equal dimension weights = 1.0, 0.75, 0.5, 0.25, 0
    Section 3.5: the five-band perceptual scale and equal weighting of the five dimensions are chosen for the macro judge, with no validation that they are equally important or independent.
assumptions (5)
  • domain assumption The rendering function V(.) is deterministic under a fixed coordinate system.
    Invoked in Section 3.1; needed for pixel alignment and stable bounding boxes. Real browsers vary with font availability, viewport, and subpixel rendering.
  • domain assumption SSIM and pixel-difference metrics are valid measures of UI repair quality.
    Section 3.5 uses these for micro scoring without perceptual validation for web layouts.
  • domain assumption VLM rubric judgments reflect human perception of reconstruction quality.
    Section 3.5 and 5.2: macro scores rely on Kimi-K2.6, and cross-validation only compares VLM judges, not human raters.
  • domain assumption Element Fingerprint hash(tag, text, attr) uniquely identifies elements across structural edits.
    Section 3.2: if hashes collide or text/attr changes, anchors and bounding boxes become wrong; no collision analysis is reported.
  • domain assumption The corruption and reversal process produces realistic repair trajectories.
    Section 3.1: forward corruption may not match the distribution of real frontend defects, so multi-turn results may not generalize.
invented entities (1)
  • Element Fingerprint ID
    purpose: Position-agnostic token stamped into each DOM node before corruption and removed before evaluation, used to track elements across deletions and reflows.
    Defined in Eq (3); uniqueness and collision resistance are assumed, not externally validated, and no collision analysis is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification." pith.science (2026). https://pith.science/paper/HC3QEBQA

@misc{pith2026260803474,
  author       = {Pith},
  title        = {Pith review of: MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HC3QEBQA}},
  note         = {Machine review of arXiv:2608.03474}
}
read the original abstract

Recent advances in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing benchmarks predominantly focus on single-turn full-page generation from scratch, overlooking the iterative workflow of real-world frontend engineering, where developers repeatedly reconstruct missing regions and modify localized elements within existing codebases. To bridge this gap, we introduce MT-Web2Code, the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification, which contains 102 tasks spanning 16 vertical domains. To construct deterministic repair trajectories without costly turn-level human annotation, we develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages. We further propose a dual-axis evaluation protocol that measures target-region fidelity and the preservation of unaffected content, where regional reconstruction is assessed by a 5-dimensional VLM-based rubric and localized modification by deterministic pixel-grounded alignment. Experiments on 13 frontier coding agents reveal that current agents struggle to faithfully reconstruct target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns. Beyond benchmarking, our deterministic evaluation metrics provide fine-grained feedback signals that may facilitate future research on training iterative UI coding agents. Our evaluation code and data will soon be released.

Figures

Figures reproduced from arXiv: 2608.03474 by the authors.

Figure 1
Figure 1. Comparison of task formulations. Unlike prior [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The 16 vertical domains of MT-Web2Code, which are sampled to a balanced count. where It contains the turn-specific instruction and resource assets. Each repair step conditions on the agent’s own output from the preceding turn, thereby capturing error propagation across interactions. Since every repair turn reverses a deter￾ministic corruption set Ct, its ground-truth target Ht−1 is uniquely defined, bypassing the ev… view at source ↗
Figure 3
Figure 3. Overview of MT-Web2Code. Starting from a golden page, the Reverse-Corruption Trajectory Engine generates two complementary evaluation tracks: (I) Macro-Level Regional Reconstruction, which evaluates regional visual grounding and structural reasoning capability using a rubric-based protocol; and (II) Micro-Level Localized Modification, which evaluates fine-grained visual grounding and visual-code alignment using a pi… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Per-turn performance comparison of Kimi-K2.6 be [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 19 canonical work pages

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    doi:10.1016/s0020-7373(84)80003-6 1984

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

Show all 48 references
  1. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  2. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  3. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  4. [12]

    2026 , eprint=

    Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification , author=. 2026 , eprint=

    2026

  5. [13]

    D esign2 C ode: Benchmarking Multimodal Code Generation for Automated Front-End Engineering

    Si, Chenglei and Zhang, Yanzhe and Li, Ryan and Yang, Zhengyuan and Liu, Ruibo and Yang, Diyi. D esign2 C ode: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associat...

    2025 doi

  6. [17]

    Proceedings of the ACM on Web Conference 2025 , pages=

    Webcode2m: A real-world dataset for code generation from webpage designs , author=. Proceedings of the ACM on Web Conference 2025 , pages=

    2025

  7. [22]

    Advances in Neural Information Processing Systems , volume=

    Webgen-bench: Evaluating llms on generating interactive and functional websites from scratch , author=. Advances in Neural Information Processing Systems , volume=

  8. [23]

    2025 , eprint=

    A Survey on Code Generation with LLM-based Agents , author=. 2025 , eprint=

    2025

  9. [24]

    2026 , howpublished =

    2026

  10. [25]

    2025 , howpublished =

    2025

  11. [26]

    2025 , eprint =

    Bai, Shuai and Cai, Yuxuan and Chen, Ruizhe and Chen, Keqin and Chen, Xionghui and Cheng, Zesen and Deng, Lianghao and Ding, Wei and Gao, Chang and Ge, Chunjiang and others , journal =. 2025 , eprint =

    2025

  12. [27]

    2026 , month = feb, howpublished =

    2026

  13. [28]

    A Survey on

    Gu, Jiawei and Jiang, Xuhui and Shi, Zhichao and Tan, Hexiang and Zhai, Xuehao and Xu, Chengjin and Li, Wei and Shen, Yinghan and Ma, Shengjie and Liu, Honghao and others , journal =. A Survey on. 2026 , doi =

    2026

  14. [31]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  15. [32]

    2025 , eprint=

    WebRenderBench: Enhancing Web Interface Generation through Layout-Style Consistency and Reinforcement Learning , author=. 2025 , eprint=

    2025

  16. [33]

    Anthropic . 2025. Claude 4.5 : Large Language Model Family. https://www.anthropic.com/claude. Accessed: 2026-01-27

    2025

  17. [34]

    Anthropic . 2026. Claude Code : AI -Powered Coding Assistant. https://claude.com/product/claude-code. Accessed: 2026-01-27

    2026

  18. [35]

    Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng, L.; Ding, W.; Gao, C.; Ge, C.; et al. 2025. Qwen3-VL Technical Report. arXiv preprint arXiv:2511.21631

    2025 arXiv

  19. [36]

    ByteDance Seed Team . 2026. Seed2.0 Model Card : Towards Intelligence Frontier for Real-World Complexity. Model card. Accessed: 2026-05-14

    2026

  20. [37]

    Dai, Y.; Lai, Y.; Huang, M.; Guo, H.; Li, D.; Peng, H.; Li, H.; Zhao, Y.; Lyu, H.; Ge, Z.; Zhang, X.; and Jiang, D. 2026. WebVR : Benchmarking Multimodal LLM s for WebPage Recreation from Videos via Human-Aligned Visual Rubrics. arXiv:2603.13391

    2026

  21. [38]

    Dong, Y.; Jiang, X.; Qian, J.; Wang, T.; Zhang, K.; Jin, Z.; and Li, G. 2025. A Survey on Code Generation with LLM-based Agents. arXiv:2508.00083

    2025 arXiv

  22. [39]

    Ge, T.; Liu, Y.; Ye, J.; Li, T.; and Wang, C. 2025. Advancing vision-language models in front-end development via data synthesis. arXiv preprint arXiv:2503.01619

    2025 arXiv

  23. [40]

    GLM-V Team ; Hong, W.; Yu, W.; Gu, X.; Wang, G.; Gan, G.; Tang, H.; Cheng, J.; Qi, J.; Ji, J.; Pan, L.; et al. 2025. GLM-4.1V-Thinking and GLM-4.5V : Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning. arXiv preprint arXiv:2507.01006

    2025 arXiv

  24. [41]

    Google . 2026 a . Gemini 3 Developer Guide. https://ai.google.dev/gemini-api/docs/gemini-3. Accessed: 2026-01-27

    2026

  25. [42]

    Google . 2026 b . Gemini 3.1 Pro Preview . https://ai.google.dev/gemini-api/docs/models/gemini-3.1-pro-preview. Accessed: 2026-05-14

    2026

  26. [43]

    Gu, J.; Jiang, X.; Shi, Z.; Tan, H.; Zhai, X.; Xu, C.; Li, W.; Shen, Y.; Ma, S.; Liu, H.; et al. 2026. A Survey on LLM -as-a-Judge. The Innovation, 7(6): 101253

    2026

  27. [44]

    Gui, Y.; Li, Z.; Wan, Y.; Shi, Y.; Zhang, H.; Chen, B.; Su, Y.; Chen, D.; Wu, S.; Zhou, X.; et al. 2025. Webcode2m: A real-world dataset for code generation from webpage designs. In Proceedings of the ACM on Web Conference 2025, 1834--1845

    2025

  28. [45]

    He, Z.; Hong, W.; Yang, Z.; Pan, Z.; Liu, M.; Gu, X.; and Tang, J. 2026. Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification. arXiv:2603.26648

    2026 arXiv

  29. [46]

    Hong, W.; Gu, X.; Pan, Z.; Yang, Z.; Wang, Y.; Wang, Y.; Yue, Y.; Wang, Y.; Wang, Y.; Wang, Y.; et al. 2026. Glm-5v-turbo: Toward a native foundation model for multimodal agents. arXiv preprint arXiv:2604.26752

    2026 arXiv

  30. [48]

    Lauren c on, H.; Tronchon, L.; and Sanh, V. 2024. Unlocking the conversion of web screenshots into html code with the websight dataset. arXiv preprint arXiv:2403.09029

    2024 arXiv

  31. [49]

    Lu, Z.; Yang, Y.; Ren, H.; Hou, H.; Xiao, H.; Wang, K.; Shi, W.; Zhou, A.; Zhan, M.; and Li, H. 2026. Webgen-bench: Evaluating llms on generating interactive and functional websites from scratch. Advances in Neural Information Processing Systems, 38

    2026

  32. [50]

    MiniMax . 2025. VIBE : Visual and Interactive Benchmark for Execution. https://huggingface.co/datasets/MiniMaxAI/VIBE. Hugging Face dataset; accessed 2026-05-14

    2025

  33. [51]

    OpenAI . 2026. GPT-5.4 Model. https://developers.openai.com/api/docs/models/gpt-5.4. Accessed: 2026-05-14

    2026

  34. [52]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR

    2021

  35. [53]

    Si, C.; Zhang, Y.; Li, R.; Yang, Z.; Liu, R.; and Yang, D. 2025. D esign2 C ode: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds., Proceedings of the 2025 Conference of the Nations of the Americas Chap...

    2025

  36. [54]

    Team, K.; Bai, T.; Bai, Y.; Bao, Y.; Cai, S.; Cao, Y.; Charles, Y.; Che, H.; Chen, C.; Chen, G.; et al. 2026. Kimi K2. 5: Visual Agentic Intelligence. arXiv preprint arXiv:2602.02276

    2026 arXiv

  37. [55]

    E.; Zhang, A

    Yang, J.; Jimenez, C. E.; Zhang, A. L.; Lieret, K.; Yang, J.; Wu, X.; Press, O.; Muennighoff, N.; Synnaeve, G.; Narasimhan, K. R.; et al. 2024. Swe-bench multimodal: Do ai systems generalize to visual software domains? arXiv preprint arXiv:2410.03859

    2024 arXiv

  38. [56]

    Q.; Wang, Y.; Jiang, Z.; Deng, M.; Wang, J.; Tao, T.; Li, J.; et al

    Yun, S.; Lin, H.; Thushara, R.; Bhat, M. Q.; Wang, Y.; Jiang, Z.; Deng, M.; Wang, J.; Tao, T.; Li, J.; et al. 2024. Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs. arXiv preprint arXiv:2406.20098

    2024 arXiv

  39. [57]

    Z.ai . 2025. GLM-4.6V : Open-Source Multimodal Models with Native Multimodal Function Calling. https://z.ai/blog/glm-4.6v. Accessed: 2026-05-14

    2025

  40. [58]

    Zhang, L.; Liu, A.; Liu, Y.; and Zhou, X. 2026. DiffSpot: Can VLMs Spot Fine-Grained Visual Differences in Web Interfaces? arXiv preprint arXiv:2605.29615

    2026 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.