Pith. sign in

REVIEW 5 major objections 5 minor 60 references

LaTCoder: Converting Webpage Design to Code with Layout-as-Thought

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that splitting a webpage design into blocks, coding each block with chain-of-thought prompting, and stitching the blocks together by position preserves the original layout far better than generating the whole page at once.

desk verdict Worth a serious referee: solid block-wise design-to-code pipeline with consistent gains, but the unvalidated division step and verifier metric overlap cap how much I trust the headline numbers. read the letter →

arxiv 2508.03560 v1 pith:HAJ7ZPUX submitted 2025-08-05 cs.SE

Yi Gui , Zhen Li , Zhongyi Zhang , Guohao Wang , Tianpeng Lv , Gaoyang Jiang , Yi Liu , Dongping Chen
show 5 more authors
Yao Wan Hongyu Zhang Wenbin Jiang Xuanhua Shi Hai Jin
This is my paper
classification cs.SE
keywords design-to-codelayoutpreservationmultimodallargelanguagemodelsblock-wisecodegenerationchain-of-thoughtpromptingHTML/CSSUIautomationCC-HARDbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LaTCoder is a method for turning a webpage screenshot into HTML/CSS code that preserves the page's layout far more reliably than generating the whole page in one pass. The paper's claim is that layout preservation is improved by treating the design as a sequence of thoughts: divide the design into rectangular blocks, ask a multimodal large language model to write each block's code with a chain-of-thought prompt, and then assemble the block codes using either absolute positioning or an MLLM-based merge. On the harder CC-HARD benchmark the method raises TreeBLEU by 66.67% and cuts MAE by 38.53% for DeepSeek-VL2, with similarly large gains for Gemini and GPT-4o, and human annotators prefer LaTCoder over each baseline in at least 60% of pairwise comparisons. The authors introduce CC-HARD, a web-crawl-sourced benchmark with deeper DOM trees and more tags, because existing benchmarks were too easy to separate methods. If the claim holds, block-wise generation with position anchoring is a reliable way to make design-to-code conversion usable for real-world, layout-complex webpages.

What carries the argument

The central mechanism is Layout-as-Thought (LaT), a decompose-generate-assemble pipeline. The divider scans the design image for horizontal and vertical solid-colored lines, uses OCR to keep text regions intact, and merges blocks below a minimum area, producing a list of bounding boxes that anchor every block to its place in the page. The generator feeds each cropped block to the multimodal model with a chain-of-thought prompt that asks the model to analyze the section, write HTML code, compare text and color against the screenshot, and then integrate a final version. The assembler then either wraps each block's code in a parent div positioned by its bounding box, or asks the model to merge all block codes guided by the bounding boxes; a reference-free verifier defined as 0.5 times (1 minus MAE over 255) plus 0.5 times CLIP similarity selects the better of the two outputs. Every block shares one fixed HTML template with utility CSS classes so that the pieces stay consistent when reassembled.

What would settle it

Run LaTCoder on a test set of designs with overlapping cards, diagonal decorations, or gradient or patterned backgrounds, and check whether the TreeBLEU and MAE gains over direct prompting vanish or invert when the divider's line detection fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the main failure of multimodal models in design-to-code is not recognizing elements but keeping their positions, and that failure can be sidestepped rather than trained away. The authors report that on CC-HARD, direct prompting with DeepSeek-VL2 scores 0.09 TreeBLEU and 66.91 MAE, while LaTCoder scores 0.15 and 41.13; with GPT-4o, TreeBLEU rises from 0.09 to 0.16 and MAE falls from 66.18 to 36.80, a 60% structural improvement and a 43.23% pixel-error reduction. The same decompose-generate-assemble pattern also lifts Gemini's TreeBLEU by 30% and cuts its MAE by 43.03% on CC-HARD. In human pairwise evaluation on CC-HARD with GPT-4o, LaTCoder is preferred in 61.7% of cases over direct prompting, 68.8% over self-revision, 67.2% over text-augmented prompting, and 79.7% over DCGen, so the advantage is visible to people, not only to automatic metrics.

Load-bearing premise

The load-bearing premise is that a webpage design can be split into meaningful rectangular blocks by scanning for solid-colored horizontal and vertical lines, and if a design has nested grids, overlapping content, or decorations that this line detector cannot capture, the block boxes inherit those errors and the layout-preservation gains collapse.

Editorial extensions

If this is right

  • On CC-HARD, LaTCoder with GPT-4o raises TreeBLEU from 0.09 to 0.16 and lowers MAE from 66.18 to 36.80, a 60% structural gain and a 43.23% pixel-error reduction.
  • The pipeline works across open-source and commercial backbones, and the gains are largest for smaller models: with DeepSeek-VL2-tiny, TreeBLEU rises 175% and visual score rises 179.17% relative to direct prompting.
  • The two assembly strategies are complementary in measured behavior: absolute positioning gives lower MAE by strictly preserving block positions, MLLM-based assembly keeps smoother transitions between blocks, and the dynamic verifier combines the two to improve the final output.
  • The chain-of-thought block-generation prompt is a real contributor: removing it drops TreeBLEU from 0.16 to 0.13 and visual score from 0.80 to 0.71 on CC-HARD with GPT-4o.
  • Because every method degrades on CC-HARD relative to Design2Code-HARD, layout complexity rather than text length is a primary difficulty for current design-to-code models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The specific divider could be replaced by any region proposer; a semantic layout parser or learned detector would likely extend the same decompose-and-anchor idea to designs with nested grids, overlapping cards, and non-rectangular decorations, where solid-line scanning fails.
  • The block-wise pipeline multiplies the number of model calls per page, but those calls are independent and can be parallelized, so latency could be lowered by batching blocks or by routing easy blocks to a smaller model and content-dense blocks to a stronger one.
  • The verify score's equal weighting of MAE and CLIP is a design choice; a verifier that also penalizes missing blocks or rewards DOM-tree similarity could change which assembly strategy wins the dynamic selection.
  • The new benchmark's construction by human sampling from a web-crawl corpus suggests a scalable recipe for harder layout-oriented benchmarks, though the paired-code creation step remains the bottleneck in scaling them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes LaTCoder, a decompose-generate-assemble pipeline for converting webpage design screenshots into HTML/CSS. It first divides the design into image blocks using a heuristic that detects horizontal/vertical solid-colored lines (Algorithm 1) with OCR-guided text protection, then prompts MLLMs to generate code for each block via a CoT-style prompt, and finally assembles the block code using either absolute-positioning (APS) or MLLM-based (MS) assembly, followed by a dynamic selection step that scores candidates with a reference-free verifier combining MAE and CLIP similarity. The method is evaluated on Design2Code-HARD and a newly curated CC-HARD benchmark with DeepSeek-VL2, Gemini, and GPT-4o as backbones. The paper reports consistent improvements in TreeBLEU, CLIP, Visual Score, and MAE over strong baselines, as well as human preference for LaTCoder in over 60% of pairwise comparisons.

Significance. If the reported gains hold, LaTCoder offers a simple, general divide-and-conquer strategy for design-to-code generation that alleviates the long-context and layout-capture weaknesses of current MLLMs, with benefits visible even for small open-source backbones. The paper has several concrete strengths: it fixes random seeds and temperature for reproducibility, releases code and data links, includes ablations (assembly strategies, CoT prompt, model scale, merge threshold), and performs a human preference study with majority voting. However, the central mechanism—layout-aware division—is never directly validated, the dynamic verifier selects on two of the four headline metrics, the CoT ablation is confounded, and no significance tests accompany the claimed improvements. These issues are addressable but currently limit confidence in the stated effect sizes and in the attribution of the gains to 'layout-as-thought' rather than to shorter per-block code generation.

major comments (5)
  1. [Section 3.1 / Algorithm 1 / Section 5.2] The layout-division step is the load-bearing component of the 'layout-as-thought' claim, yet it is never quantitatively validated. The algorithm relies on several ad-hoc thresholds (τ=50px, grid interval=5px, ignored edge points=10, recursion depth=3, θ=300×300px), and the paper reports no BBox statistics, no comparison against ground-truth DOM element boxes, and no ablation with an alternative partition (e.g., DOM-driven, random, or monolithic). Table 1 acknowledges 13.75% non-standard cases in Design2Code-HARD, and CC-HARD is claimed to be more layout-complex, yet the paper provides no evidence that the detected BBoxes correspond to meaningful layout regions in these cases. The parameter study in Table 6 varies only θ and thus does not test the detection thresholds or the division itself. I recommend adding a validation study of the division step, including agreement metrics with DOM boxes and at least one alternative partitioning ablation, to substantiate that the observed gains come from layout-aware blocks rather than merely from generating shorter code snippets per block.
  2. [Section 3.3 / Eq. (1) / Table 3] The dynamic selection verifier uses Verify Score = 0.5 × (1 − MAE/255) + 0.5 × CLIP, and the final 'LaTCoder' row in Table 3 is selected between the APS and MS candidates using exactly this score. Consequently, the reported improvements in MAE and CLIP for LaTCoder over its own variants are partly by construction, because the verifier optimizes those same metrics. This circularity inflates the headline MAE/CLIP numbers and makes the 'dynamic selection improves results' statement in Section 5.2 difficult to interpret. The verifier should be validated against human judgments, and the paper should either report metrics for each candidate separately without selection, or use a selection metric that is not part of the evaluation suite. The TreeBLEU and Visual Score improvements are less affected by this issue, but the claim of an 'optimal output' currently rests on a self-fulfilling criterion.
  3. [Section 5.2 / Table 4] The ablation for the CoT-based prompt is confounded. The 'simplified prompt' omits not only the step-by-step reasoning but also all task-specific instructions—the Tailwind template, the no-centering rule, content consistency requirements, and the library specifications. Thus the performance drop from 0.16 to 0.13 TreeBLEU in Table 4 cannot be attributed to the CoT mechanism rather than to the loss of detailed prompt engineering. To isolate the effect of CoT, the ablation should keep all requirements and instructions fixed while only removing or adding the step-by-step analysis and comparison stages, or alternatively add CoT steps to the simplified prompt.
  4. [Section 5.1 / Table 3] The abstract and Section 5.1 repeatedly use the word 'significantly' (e.g., 'TreeBLEU scores increased by 66.67%' and 'LaTCoder significantly boosts MLLMs' performance'), but no statistical significance tests are reported. The standard deviations in Table 3 are large; for example, GPT-4o on Design2Code-HARD improves TreeBLEU from 0.16±0.11 to 0.20±0.11 with n=80, which may not be statistically significant. I recommend adding paired tests (e.g., bootstrap or permutation over the 80/128 samples) for all metric comparisons in Table 3, and tempering the 'significant' language in the abstract and contributions if the tests do not support it.
  5. [Section 4.1 / CC-HARD] The new CC-HARD benchmark is described only as: 'we instruct two experts to manually obtain more challenging samples from the Common Crawl dataset and generate paired data.' This is insufficient for a benchmark introduced with the paper. There is no documented selection protocol (what made a sample 'challenging'?), no inter-annotator agreement measure, no description of how the paired HTML was obtained or validated, and no specification of how many candidates were screened. Since CC-HARD is central to the paper's claims, the curation process should be described in detail, and ideally the raw selection criteria and expert instructions should be released alongside the dataset.
minor comments (5)
  1. [Section 4.1] The phrase 'one of our text benchmarks' should read 'one of our test benchmarks.'
  2. [Figure 2] The example BBox entry for block 2 shows both 'left': 0 and 'right': 0, which appears to be a typo; the right coordinate should presumably be a positive pixel value.
  3. [Figure 5] The percentages for the Direct baseline row sum to 99.9% (27.3 + 10.9 + 61.7), which is likely a rounding error; please adjust the displayed values.
  4. [Section 5.3] The human evaluation section does not report the number of samples evaluated, the demographic or expertise of the six annotators, or inter-annotator agreement (e.g., Fleiss' kappa). These details are important for assessing the 'over 60%' preference claim.
  5. [Contributions / Section 1] The code link points to a directory within an existing repository (github.com/CGCL-codes/naturalcc/tree/main/examples/latcoder). A dedicated repository or a tagged release would improve reproducibility and long-term accessibility.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild partial circularity: the dynamic verifier selects outputs using a score that is exactly a combination of the headline MAE and CLIP metrics, so the final LaTCoder row partly optimizes those same metrics; the central claim retains independent support from TreeBLEU, Visual Score, and human evaluation.

  1. fitted input called prediction [Section 3.3, 'Dynamic Strategy Selection' (Verify Score formula) and Table 3 note (3); final LaTCoder rows in Table 3.]
    "We design a verifier to evaluate the similarity between the generated webpage and the webpage design. ... Verify Score = 1/2 × (1 − MAE/255) + 1/2 × CLIP. We empirically set equal coefficients (0.5) for the two components in this formula. ... The three variants of LaTCoder correspond to different assembly strategies: using MLLMs, absolute positioning, and getting the best of the first two strategies with a dynamic verifier."

    The final LaTCoder row is produced by choosing, per sample, whichever of the two assembly outputs (APS or MS) has the higher Verify Score. Since Verify Score is defined as 0.5*(1 - MAE/255) + 0.5*CLIP, the selected output's MAE and CLIP values are partly minimized/maximized by the selection rule itself. Reporting those same two metrics as headline improvements (e.g., the GPT-4o MAE reductions in Table 3) is therefore partially a constructed optimum rather than an independent prediction for those metrics. This is a partial circularity only: TreeBLEU, Visual Score, and the human preference study are not part of the verifier score, and the largest DeepSeek-VL2 gains come from the APS variant without dynamic selection, so the central claim is not fully forced by this step.

full rationale

I walked the derivation chain from division, through block-wise synthesis, to assembly and dynamic selection. The only concrete reduction I can exhibit is the verifier's use of MAE and CLIP as the selection objective while the same two metrics appear as headline outcomes in Table 3. Because the selection rule is explicitly 'getting the best of the first two strategies with a dynamic verifier' and the verify score is exactly 0.5*(1 - MAE/255) + 0.5*CLIP, the final row's MAE and CLIP gains are partially by construction. This is real but mild: the APS variant already produces most of the gain, the deep-seek headline numbers do not use the verifier, and TreeBLEU, Visual Score, and human preference are outside the selection score and improve independently. I also considered TreeBLEU's provenance from the authors' prior WebCode2M paper; that is self-citation, but the metric is a publicly defined DOM-tree similarity measure and does not encode LaTCoder's outputs, so it is not load-bearing circularity. The layout-divider's unvalidated heuristic thresholds are a substantive robustness concern, but they are an assumption about the input distribution, not a step that reduces the derivation to its own inputs. On balance, the paper is largely self-contained against external baselines and its central decompose-generate-assemble claim has independent content, so the circularity score is 2.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The method rests on a hand-tuned image processing pipeline with seven explicit parameters, several strong domain assumptions about webpage layout structure, and an empirically weighted verifier. No new physical or conceptual entities are introduced. The performance claims therefore depend as much on the division algorithm's assumptions as on the multimodal model's capability.

free parameters (7)
  • minimum dividing distance tau = 50 pixels
    Hand-set threshold that determines which solid-colored rows and columns count as dividing lines; changes block granularity and all downstream metrics. Section 4.4.
  • grid sampling interval = 5 pixels
    Heuristic sampling rate for solid-line detection; affects detection efficiency and accuracy. Section 4.4.
  • ignored edge points = 10 pixels
    Pixels at image borders are ignored when checking solid color; can affect line detection near boundaries. Section 4.4.
  • max recursive depth = 3
    Limits recursive division of sub-images to prevent overly fine blocks. Section 4.4.
  • minimum block area theta = 300x300 pixels
    Blocks below this area are merged into adjacent blocks; the paper's parameter study shows performance varies with theta. Section 4.4 and Table 6.
  • OCR bounding-box merge distance = 20 pixels
    Adjacent text boxes within 20 pixels are merged to preserve text regions; affects line detection around text. Section 4.4.
  • verify score weights = 0.5 for MAE term, 0.5 for CLIP
    Equal weights are described as 'empirically set' for dynamic selection; no ablation or held-out tuning is reported. Section 3.3.
assumptions (6)
  • domain assumption Webpage layouts are largely expressible by solid-color horizontal and vertical dividing lines consistent with the CSS box model.
    Section 3.1 searches only for solid-colored lines. Table 1 suggests non-standard layouts occur in only 4 to 14 percent of three datasets, but they are not absent.
  • domain assumption OCR (EasyOCR) accurately identifies text regions so that dividing lines do not split text.
    Section 3.1 and Section 4.4 rely on OCR text boxes to veto dividing lines; OCR errors would corrupt block boundaries and downstream assembly.
  • domain assumption MLLMs can generate code for a cropped image block with no surrounding context, and can assemble blocks correctly given bounding boxes.
    Core to block-wise synthesis and assembly. Section 5.4 documents failures in practice, including centering errors and Gemini 'laziness' during assembly.
  • domain assumption The verify score, defined as 0.5 times (1 minus MAE over 255) plus 0.5 times CLIP, is a valid reference-free proxy for webpage similarity.
    Section 3.3 defines the score and uses it for dynamic selection; its weights are not validated against human preference.
  • domain assumption CC-HARD, manually sampled by two experts from Common Crawl, is representative of complex real-world layouts.
    Section 4.1; the absence of a documented selection protocol makes representativeness an assumption rather than a verified property.
  • domain assumption TreeBLEU, CLIP, Visual Score, and MAE collectively measure layout preservation in a way aligned with human judgments.
    Section 4.2 selects these metrics and states they align with human preferences, but no correlation analysis is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LaTCoder: Converting Webpage Design to Code with Layout-as-Thought." pith.science (2026). https://pith.science/paper/HAJ7ZPUX

@misc{pith2026250803560,
  author       = {Pith},
  title        = {Pith review of: LaTCoder: Converting Webpage Design to Code with Layout-as-Thought},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HAJ7ZPUX}},
  note         = {Machine review of arXiv:2508.03560}
}
read the original abstract

Converting webpage designs into code (design-to-code) plays a vital role in User Interface (UI) development for front-end developers, bridging the gap between visual design and functional implementation. While recent Multimodal Large Language Models (MLLMs) have shown significant potential in design-to-code tasks, they often fail to accurately preserve the layout during code generation. To this end, we draw inspiration from the Chain-of-Thought (CoT) reasoning in human cognition and propose LaTCoder, a novel approach that enhances layout preservation in webpage design during code generation with Layout-as-Thought (LaT). Specifically, we first introduce a simple yet efficient algorithm to divide the webpage design into image blocks. Next, we prompt MLLMs using a CoTbased approach to generate code for each block. Finally, we apply two assembly strategies-absolute positioning and an MLLM-based method-followed by dynamic selection to determine the optimal output. We evaluate the effectiveness of LaTCoder using multiple backbone MLLMs (i.e., DeepSeek-VL2, Gemini, and GPT-4o) on both a public benchmark and a newly introduced, more challenging benchmark (CC-HARD) that features complex layouts. The experimental results on automatic metrics demonstrate significant improvements. Specifically, TreeBLEU scores increased by 66.67% and MAE decreased by 38% when using DeepSeek-VL2, compared to direct prompting. Moreover, the human preference evaluation results indicate that annotators favor the webpages generated by LaTCoder in over 60% of cases, providing strong evidence of the effectiveness of our method.

Figures

Figures reproduced from arXiv: 2508.03560 by the authors.

Figure 1
Figure 1. A real-world bad case from the famous project, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The workflow of LaTCoder. dataset [1] and generate paired data to curate the dataset, which we refer to as CC-HARD. We evaluate our method and four state￾of-the-art baseline approaches using different backbone MLLMs on a public dataset (i.e., Design2Code-Hard) and our newly in￾troduced CC-HARD. Experimental results show that integrating various MLLMs as backbones into our method significantly boosts performance acro… view at source ↗
Figure 3
Figure 3. A toy example of dividing line detection. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The simplified prompt for generating image block [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Pairwise human preference evaluation of baseline [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Case study of samples generated by LaTCoder and other baseline methods with GPT-4o as the backbone MLLM: LaTCoder significantly outperforms the others, particularly in preserving the layout of the design. 6 Related Work UI Automation. Early researches, constrained by l…
Figure 7
Figure 7. Figure 7: Prompt for block-wise code synthesis [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 42 canonical work pages

  1. [1]

    The Common Crawl dataset

    2024. The Common Crawl dataset. https://data.commoncrawl.org/

    work page 2024

  2. [2]

    The screen-to-shot project on the Github

    2024. The screen-to-shot project on the Github . https://github.com/abi/ screenshot-to-code/

    work page 2024

  3. [3]

    Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrit- twieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P

    Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrit- twieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael ...

    arXiv 2023

  4. [4]

    Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. https://api.semanticscholar.org/CorpusID:268232499

    2024

  5. [5]

    Manmatha

    Srikar Appalaraju, Peng Tang, Qi Dong, Nishant Sankaran, Yichu Zhou, and R. Manmatha. 2023. DocFormerv2: Local Features for Document Understanding. In Proceedings of the AAAI Conference on Artificial Intelligence

    work page 2023

  6. [6]

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of Multimodal Large Language Models: A Survey. ArXiv abs/2404.18930 (2024)

    arXiv 2024

  7. [7]

    Tony Beltramelli. 2018. pix2code: Generating Code from a Graphical User Inter- face Screenshot. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS 2018, Paris, France, June 19-22, 2018 . ACM, 3:1–3:6

    work page 2018

  8. [8]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

    work page 2021

Show all 60 references
  1. [9]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Pi- otr Nyczyk, and Torsten Hoefler. 2024. Graph of Thoughts: Solving Elaborate Problems with Large Language Models. In Proceed...

    2024

  2. [10]

    Zhangqian Bi, Yao Wan, Zheng Wang, Hongyu Zhang, Batu Guan, Fangxin Lu, Zili Zhang, Yulei Sui, Hai Jin, and Xuanhua Shi. 2024. Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback. In Proceedings of the Findings of the Associati...

    2024

  3. [11]

    Shang Chai, Liansheng Zhuang, and Fengying Yan. 2023. LayoutDM: Transformer- based Diffusion Model for Layout Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 18349–18358

    2023

  4. [12]

    Dongping Chen, Ruoxi Chen, Shu Pu, Zhaoyi Liu, Yanru Wu, Caixi Chen, Benlin Liu, Yue Huang, Yao Wan, Pan Zhou, et al. 2024. Interleaved Scene Graph for Interleaved Text-and-Image Generation Assessment. InProceedings of the Twelfth International Conference on Learning Represent...

    2024

  5. [13]

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Pan Zhou, Yao Wan, and Lichao Sun. 2024. MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark. In Proceedings of the International Conference on Machine Learning

    2024

  6. [14]

    Tu Anh Dinh, Carlos Mullov, Leonard Barmann, Zhaolin Li, Danni Liu, Simon Reiß, Jueun Lee, Nathan Lerzer, Fabian Ternava, Jianfeng Gao, Alexander Waibel, Tamim Asfour, Michael Beigl, Rainer Stiefelhagen, Carsten Dachsbacher, Klemens Bohm, and Jan Niehues. 2024. SciEx: Benchmar...

    2024

  7. [15]

    Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. ArXiv abs/2204.05999 (2022)

    2022 arXiv

  8. [16]

    Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Bohua Chen, Yi Su, Dong- ping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, and Xiangliang Zhang

  9. [17]

    Yi Gui, Yao Wan, Zhen Li, Zhongyi Zhang, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, and Xiangliang Zhang. 2025. UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs. In Proceedings of the International World Wid...

    2025

  10. [18]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence.ArXiv abs/2401.14196 (2024)

    2024 arXiv

  11. [19]

    Hongcheng Guo, Wei Zhang, Junhao Chen, Yaonan Gu, Jian Yang, Junjia Du, Binyuan Hui, Tianyu Liu, Jianxin Ma, Chang Zhou, and Zhoujun Li. 2024. IW- Bench: Evaluating Large Multimodal Models for Converting Image-to-Web.ArXiv abs/2409.18980 (2024)

    2024

  12. [20]

    Girshick

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. 2020. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2 (2020), 386–397

    2020

  13. [21]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks ...

    2021

  14. [22]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Proba- bilistic Models. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual

    2020

  15. [23]

    Fan Hu, Yanlin Wang, Lun Du, Xirong Li, Hongyu Zhang, Shi Han, and Dongmei Zhang. 2023. Revisiting Code Search in a Two-Stage Paradigm. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM 2023, Singapore, 27 February 2023 - 3 March ...

    2023

  16. [24]

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking. In Proceed- ings of the 30th ACM International Conference on Multimedia

    2022

  17. [25]

    Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi

  18. [26]

    Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conver- sion of Web Screenshots into HTML Code with the WebSight Dataset. ArXiv abs/2403.09029 (2024)

    2024 arXiv

  19. [27]

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Mar- tin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2Struct: Screenshot Parsing as Pretraining for Visual Lan- guage Understanding. In Proceedings of ...

    2023

  20. [28]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s Verify Step by Step. In Proceedings of the Twelfth International Conference on Learning Representations, ICLR 2024, Vie...

    2024

  21. [29]

    Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang Yang, Jian-Guang Lou, and Dongmei Zhang. 2023. LayoutPrompter: Awaken the Design Ability of Large Language Models. In Advances in Neural Information Processing Systems 36: Annual Confer- ence on Neural Information Processing Systems ...

    2023

  22. [30]

    Haoyu Lu, Wen Liu, Bo Zhang, Bing-Li Wang, Kai Dong, Bo Liu (Benjamin Liu), Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. DeepSeek-VL: Towards Real-World Vision-Language Understanding. ArXiv abs/2403...

    2024 arXiv

  23. [31]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. WizardCoder: Em- powering Code Large Language Models with Evol-Instruct.ArXiv abs/2306.08568 (2023)

    2023 arXiv

  24. [32]

    OpenAI. 2023. GPT-4 Technical Report. ArXiv abs/2303.08774 (2023)

    2023 arXiv

  25. [33]

    Geliang Ouyang, Jingyao Chen, Zhihe Nie, Yi Gui, Yao Wan, Hongyu Zhang, and Dongping Chen. 2025. nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguist...

    2025

  26. [34]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke E

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, J...

    2022 arXiv

  27. [35]

    Shu Pu, Yaochen Wang, Dongping Chen, Yuhang Chen, Guohao Wang, Qi Qin, Zhongyi Zhang, Zhiyuan Zhang, Zetong Zhou, Shuang Gong, et al. 2025. Judge Anything: MLLM as a Judge Across Any Modality. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mini...

    2025

  28. [36]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings...

    2021

  29. [37]

    Alex Robinson. 2019. Sketch2code: Generating a website from a paper mockup. ArXiv abs/1905.13750 (2019). LaTCoder: Converting Webpage Design to Code with Layout-as-Thought KDD ’25, August 3–7, 2025, Toronto, ON, Canada

    2019 arXiv

  30. [38]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, CVPR 2022, New Orleans, LA, USA, June 1...

    2022

  31. [39]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. InProceedings of the Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 -...

    2015

  32. [40]

    Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. 2000. The Earth Mover’s Distance as a Metric for Image Retrieval. International Journal of Computer Vision 40 (2000), 99–121

    2000

  33. [41]

    Evan Shelhamer, Jonathan Long, and Trevor Darrell. 2014. Fully convolutional networks for semantic segmentation. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 3431–3440

    2014

  34. [42]

    Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang

  35. [43]

    Zhihong Sun, Yao Wan, Jia Li, Hongyu Zhang, Zhi Jin, Ge Li, and Chen Lyu

  36. [44]

    Yao Wan, Zhangqian Bi, Yang He, Jianguo Zhang, Hongyu Zhang, Yulei Sui, Guandong Xu, Hai Jin, and Philip Yu. 2024. Deep learning for code intelligence: Survey, benchmark and toolkit. Comput. Surveys 56, 12 (2024), 1–41

    2024

  37. [45]

    Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Pap...

    2025

  38. [46]

    Yao Wan, Jingdong Shu, Yulei Sui, Guandong Xu, Zhou Zhao, Jian Wu, and Philip Yu. 2019. Multi-modal attention network learning for semantic source code retrieval. In Proceedings of 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 13–25

    2019

  39. [47]

    Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R. Lyu. 2024. Automatically Generating UI Code from Screen- shot: A Divide-and-Conquer-Based Approach. ArXiv abs/2406.16386 (2024)

    2024 arXiv

  40. [48]

    Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S Yu. 2018. Improving automatic source code summarization via deep rein- forcement learning. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering . 397–407

    2018

  41. [49]

    Yuxuan Wan, Yi Dong, Jingyu Xiao, Yintong Huo, Wenxuan Wang, and Michael R. Lyu. 2024. MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs. ArXiv abs/2412.15310 (2024)

    2024 arXiv

  42. [50]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neur...

    2022

  43. [51]

    Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zhiyao Xu, and Michael R. Lyu. 2024. Interaction2Code: How Far Are We From Automatic Interactive Webpage Gener- ation? ArXiv abs/2411.03292 (2024)

    2024

  44. [52]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025. Qwen2.5-Omni Technical Report. ArXiv abs/2503.20215 (2025)

    2025 arXiv

  45. [53]

    Wenhua Wang, Yuqun Zhang, Yulei Sui, Yao Wan, Zhou Zhao, Jian Wu, S Yu Philip, and Guandong Xu. 2020. Reinforcement-learning-guided source code sum- marization using hierarchical attention.IEEE Transactions on software Engineering 48, 1 (2020), 102–119

    2020

  46. [54]

    Xing, Xiaodan Liang, and Zhiqiang Shen

    Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, and Zhiqiang Shen. 2024. Web2Code: A Large-scale ...

    2024

  47. [55]

    Ti Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang

  48. [57]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Pr...

    2023

  49. [60]

    en " > 3 < head > 4 < meta charset =

    Bridging Design and Development with Automated Declarative UI Code Generation. ArXiv abs/2409.11667 (2024). KDD ’25, August 3–7, 2025, Toronto, ON, Canada Yi Gui et al. Prompt for Block-wise Code Synthesis You are an expert Tailwind developer. Based on the reference screenshot...

    2024 arXiv

  50. [2023]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    LayoutDM: Discrete Diffusion Model for Controllable Layout Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10167–10176

  51. [2024]

    In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA, USA, October 27 - November 1, 2024

    Sifting through the Chaff: On Utilizing Execution Feedback for Ranking the Generated Code Candidates. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA, USA, October 27 - November 1, 2024 . ACM, 229–241

    2024

  52. [2025]

    In Proceedings of the International World Wide Web Conference, WWW 2025, Sydney, April 28–May 2, 2024

    WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs. In Proceedings of the International World Wide Web Conference, WWW 2025, Sydney, April 28–May 2, 2024

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.