Pith. sign in

REVIEW 3 major objections 6 minor 60 references

SAM2-UNeXT: An Improved High-Resolution Baseline for Adapting Foundation Models to Downstream Segmentation Tasks

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Block-by-block webpage code generation, anchored to the design's detected layout regions, preserves layout far better than whole-page generation, especially for smaller multimodal models.

desk verdict The arXiv record is not what it claims: abstract/title describe SAM2-UNeXT, body is LaTCoder, and the LaTCoder results have internal number inconsistencies that undermine the main empirical claim. read the letter →

arxiv 2508.03566 v1 pith:2D3R6BYP submitted 2025-08-05 cs.CV

Xinyu Xiong , Zihuang Wu , Lei Zhang , Lei Lu , Ming Li , Guanbin Li This is my paper
classification cs.CV
keywords design-to-codelayoutpreservationmultimodallargelanguagemodelschain-of-thoughtblock-wisegenerationlayout-awaredivisionCC-HARDbenchmarkTreeBLEU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dozens of multimodal language models (MLLMs) now promise to turn a webpage screenshot into HTML and CSS, but they routinely scramble the layout because they try to generate the entire page in one pass. LaTCoder instead treats the design as a set of layout 'thoughts': a lightweight algorithm divides the image into rectangular blocks along solid-colored lines (using OCR so lines never cut through text), each block's code is generated separately with a chain-of-thought prompt, and the blocks are reassembled at their original coordinates. The paper reports that, across GPT-4o, Gemini, and DeepSeek-VL2 on two benchmarks, this block-wise pipeline improves DOM-tree structure similarity and pixel error relative to direct prompting and prior divide-and-conquer baselines, with the largest relative gains for the smallest model. It also contributes CC-HARD, a harder benchmark with deeper DOM trees and more complex layouts, and a human preference study that favors LaTCoder in most pairwise comparisons.

What carries the argument

The load-bearing object is the Layout-as-Thought (LaT) pipeline, a block-wise code-generation process in which each image block is treated as an independent 'thought' for a multimodal model to convert into code. The dividing algorithm is the critical mechanism: it scans rows and columns at a 5-pixel grid interval for solid-colored lines at least 50 pixels apart, uses OCR to ensure no dividing line crosses text regions, ignores edge pixels, merges blocks smaller than 300×300 pixels, and recurses to depth 3, producing a set of bounding boxes (BBoxes) that anchor every block. Each cropped block is then fed to an MLLM with a chain-of-thought prompt that instructs it to analyze the section, generate code, and check text, color, and style consistency. Finally, two assembly strategies — absolute positioning (APS) and MLLM-based assembly (MS) — are combined via a reference-free verifier that computes a score as the equal-weight sum of normalized MAE and CLIP similarity, dynamically selecting the better output.

What would settle it

Take a set of webpage designs with gradient backgrounds, overlapping sections, and irregular grids where the dividing algorithm is shown to produce wrong or misaligned blocks; if the full LaTCoder pipeline performs no better than monolithic generation on those cases, the claim that block-wise layout anchoring is what drives the improvement is refuted. A sharper version: replace the algorithm's detected bounding boxes with ground-truth boxes; if results improve substantially, then the divider, not anchored generation, is the true bottleneck.

Watch

Extended reading notes

Core claim

The central claim is that Layout-as-Thought (LaT) — decomposing a webpage design into rectangular image blocks via a lightweight, OCR-aware line-detection algorithm, generating each block's HTML/CSS code independently with a chain-of-thought prompt, and reassembling the blocks using their recorded bounding boxes — alleviates the layout-information loss inherent in monolithic design-to-code generation. LaTCoder reports consistent improvements over direct prompting and prior divide-and-conquer baselines across three backbone MLLMs (GPT-4o, Gemini, DeepSeek-VL2) and two benchmarks: on Design2Code-HARD it raises TreeBLEU from 0.16 to 0.20 for GPT-4o, and on the new CC-HARD benchmark it raises TreeBLEU from 0.09 to 0.16, reducing MAE by roughly 43%. The paper further claims that the relative gains are largest for the weakest model (DeepSeek-VL2), and that human annotators prefer LaTCoder's output in at least 60% of pairwise comparisons against each baseline.

Load-bearing premise

The load-bearing premise is that a webpage design can be split into rectangular blocks separated by solid-colored lines that never cross text; if a design has gradients, overlaps, or irregular elements, the division step can mangle it and the rest of the pipeline inherits the error.

Editorial extensions

If this is right

  • Adopting block-wise, layout-anchored generation should improve layout fidelity and pixel accuracy for any MLLM doing design-to-code, with no fine-tuning required.
  • Smaller open-source MLLMs such as DeepSeek-VL2 see the largest relative gains, which narrows the gap to commercial models on this task.
  • The CC-HARD benchmark provides a more discriminating evaluation of layout and structural complexity than Design2Code-HARD, as shown by larger performance gaps across methods.
  • Absolute-positioning assembly is responsible for the largest MAE reductions, while MLLM-based assembly yields better aesthetic continuity; combining them via the verify score improves overall output.
  • The CoT-based block prompt is a major contributor: replacing it with a simplified prompt drops TreeBLEU from 0.16 to 0.13 and visual score from 0.80 to 0.71 on CC-HARD.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same divide-and-conquer principle could extend to other structured visual-to-code tasks, such as mobile UI screens, dashboards, or document layouts, although the hand-set division thresholds would likely need retuning.
  • The line-based divider is the fragile part of the pipeline; using a learned layout parser or structural knowledge from the source would be a natural next step for irregular, gradient, or overlapping designs.
  • The finding that decomposition gives the largest boost to the weakest model suggests that block-wise generation acts as an inference-time substitute for model capacity, which may generalize to other long-context generation problems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript, despite the metadata header naming a different title, presents LaTCoder, a divide-and-conquer pipeline for converting webpage screenshots into HTML/CSS code with multimodal LLMs. The method first partitions the design into rectangular blocks using a hand-tuned line-detection algorithm, generates code for each block with a Chain-of-Thought style prompt, assembles the block codes through either absolute positioning (APS) or MLLM-based assembly (MS), and finally selects between the two assembled outputs with a verifier that combines MAE and CLIP similarity. The authors evaluate LaTCoder with DeepSeek-VL2, Gemini, and GPT-4o on Design2Code-HARD and on a new CC-HARD benchmark, reporting improvements in TreeBLEU, CLIP, Visual Score, and MAE, plus a pairwise human preference study. The core technical idea is plausible and the new benchmark is a useful contribution, but the headline full-method results are confounded by test-time selection on the evaluation metrics, and several supporting analyses are incomplete.

Significance. If validated, LaTCoder would provide a simple and effective block-wise generation baseline for design-to-code, and CC-HARD would be a useful stress test for layout preservation. The paper also takes reproducibility seriously: it fixes temperature to 0, sets random seeds, and releases code and benchmark links. The main significance is conditional, however, because the full-method rows in Table 3 are produced by a verifier whose score is a convex combination of two of the four reported evaluation metrics; this makes the comparison against single-shot baselines non-equivalent unless the selection effect is quantified and separated. The APS-only results, which are not affected by this confound, still show substantial MAE and TreeBLEU gains, so the underlying approach is worth further scrutiny rather than dismissal.

major comments (3)
  1. [§3.3, Eq. (1), Table 3] The dynamic selection verifier is defined as Verify Score = 0.5*(1-MAE/255) + 0.5*CLIP, and MAE and CLIP are also two of the four headline metrics in Table 3. Therefore the 'LaTCoder' row in Table 3 is not the output of a single generation pipeline but the per-sample best of two candidates according to a weighted combination of the evaluation metrics themselves, plus empirically set weights (0.5/0.5). This guarantees an advantage on those metrics relative to either candidate and conflates the contribution of block-wise generation/assembly with the contribution of test-time selection on the target metric. I note that the full-method MAE being lower than both the APS and MS average MAEs is not by itself arithmetically impossible, since per-sample min-selection can lower the average; however, the paper does not report per-sample selection counts or the metrics on the exact selected outputs, so the reported full-method values cannot be independently verified. The authors should either make APS/MS without selection the primary comparison, or use a verifier based on a metric not included in the evaluation table, and should report the selection statistics.
  2. [§3.1, Table 1] The layout-aware division algorithm assumes rectangular blocks separated by solid-colored horizontal/vertical lines that do not cross text regions. Table 1 explicitly reports that 13.75% of Design2Code-HARD and 4.33% of Design2Code contain irregular layouts, overlapping layouts, or gradient backgrounds. The paper does not state how many samples fail the division step or what fallback is used, even though every downstream step inherits the block decomposition. Because the aggregate scores may be dominated by the 86.25% of 'regular' cases, the authors should provide a per-case failure analysis of the divider and report performance separately on the non-standard subset, or describe a fallback for samples that do not satisfy the rectangular-line assumption.
  3. [Table 3, §5.3, Figure 5] Several of the claimed improvements are small relative to the reported standard deviations. For example, with GPT-4o on Design2Code-HARD the CLIP score changes from 0.84±0.08 for direct prompting to 0.87±0.07 for LaTCoder, and on CC-HARD the CLIP gain is 0.79±0.10 to 0.81±0.09. No paired significance tests are reported for any automatic metric, so statements such as 'significantly boosts' in Section 5.1 are not statistically supported. The human evaluation in Section 5.3 and Figure 5 covers only GPT-4o on CC-HARD and omits the total number of paired comparisons, the handling of ties, and inter-annotator agreement. Please add paired tests for the automatic metrics and report the full protocol and statistics for the human study.
minor comments (6)
  1. [Metadata] The submission metadata identifies the paper as arXiv:2508.03566 (cs.CV), 'SAM2-UNeXT', but the body is an ACM KDD '25 paper titled 'LaTCoder' with arXiv:2508.03560 (cs.SE). The editor should verify that the correct manuscript and identifier were submitted.
  2. [§3.1, Algorithm 1] The text says the line-detection algorithm is applied recursively, but Algorithm 1 as written has no recursion or termination condition for recursion. Please clarify the recursive call and the stopping criterion.
  3. [§3.3, Eq. (1)] The verify-score formula is not numbered and the symbols MAE and CLIP are not formally defined at that point. Consider numbering the equation and noting that MAE is normalized by 255 and CLIP is cosine similarity in [0,1].
  4. [Table 3] The table footnote says 'best performance among three variances'; 'variances' should be 'variants', and the text should be explicit that for DeepSeek-VL2 only the APS variant is available.
  5. [§5.3] The abstract and Section 5.3 state that annotators prefer LaTCoder in 'over 60% of cases', but the figure does not report the denominator or the number of samples. Please include the exact counts and the majority-voting details.
  6. [§5.2] The claim that MLLM-based assembly 'often produces better overall aesthetics and smoother transitions' is a subjective assertion without quantitative or visual evidence beyond one case study. Please provide support or temper the claim.

Circularity Check

1 steps flagged · score 5.0 of 10

Partial circularity: Section 3.3's dynamic verifier selects outputs using the same MAE and CLIP metrics that Section 4.2 reports as evaluation metrics, so the full 'LaTCoder' row's gains on those metrics are partly forced by the selector; the block-wise APS comparison nevertheless provides independent support.

  1. self definitional [Section 3.3 'Dynamic Strategy Selection'; Section 4.2 'Evaluation Metrics']
    "We combine MAE and CLIP [36] similarity into a composite metric for the verifier. The metric, called the verify score, is defined as follows: Verify Score = 1/2×(1− MAE/255) + 1/2× CLIP. We empirically set equal coefficients (0.5) for the two components in this formula. ... CLIP [36]. This metric evaluates content similarity by computing the CLIP cosine similarity between the screenshot of the generated page and the original design. ... MAE measures the average absolute pixel color value difference between images."

    The dynamic selector is defined as choosing the output that maximizes Verify Score. Both terms in Verify Score are computed between the generated screenshot and the original design, which is precisely how the evaluation metrics MAE and CLIP in Table 3 and Section 4.2 are computed. Therefore, when the paper reports the 'LaTCoder' row (the output of this selector) and credits it with better CLIP/MAE than 'LaTCoder (APS)' or 'LaTCoder (MS)', that margin is in part guaranteed by construction: the selector directly optimizes the same quantities used to score the table. Section 5.2's statement that 'filtering the results through a verifier further improves the generated outcomes' is thus, for MAE and CLIP, a restatement of the selection rule rather than an independent empirical finding.

full rationale

The paper's central engineering contribution—dividing a design into blocks, generating each block with CoT prompts, and assembling with absolute positioning—is evaluated independently in the 'LaTCoder (APS)' rows, which beat direct prompting on MAE (e.g., 36.21 vs 61.62 for GPT-4o on Design2Code-HARD) without any selection step. This is genuine, non-circular evidence. No load-bearing self-citation or imported uniqueness theorem is present; citations such as WebCode2M are for the TreeBLEU metric and dataset context, not for the main claim. The only circularity I find is the dynamic verifier: it selects the final output by maximizing a weighted sum of MAE and CLIP, and the paper then reports MAE and CLIP as headline evaluation metrics. The resulting improvement of the full 'LaTCoder' row over its two assembly candidates is in part a direct consequence of optimizing the same numeric functions used for evaluation, not of an independent layout-reasoning gain. Additionally, Table 3's footnote computes the displayed improvement as 'LaTCoder’s best performance among three variances', which adds a best-of-three selection to the numbers quoted in Section 5.1; this is a reporting choice that stacks further selection on the evaluation metrics. Because the APS-vs-baseline comparison remains independent, I assign a moderate score rather than a high one.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central method depends on hand-set thresholds in the dividing algorithm and verifier, and on domain assumptions about webpage structure and MLLM reliability. No new entities are introduced.

free parameters (7)
  • Grid sampling interval = 5 pixels
    Spacing of sampled points for solid-line detection in Section 4.4.
  • Minimum dividing distance tau = 50 pixels
    Minimum distance between adjacent dividing lines in Algorithm 1 and Section 4.4.
  • Ignored edge points = 10
    Number of border pixels ignored when checking if a line is solid-colored, Section 4.4.
  • Maximum recursion depth = 3
    Depth limit for recursive division, Section 4.4.
  • Minimum block area theta = 300x300
    Threshold for merging small blocks, tuned in Table 6.
  • OCR merge distance = 20 pixels
    Distance within which text bounding boxes are merged, Section 4.4.
  • Verifier weights = 0.5 and 0.5
    Equal weights for MAE and CLIP in the verify score, empirically set in Section 3.3.
assumptions (4)
  • domain assumption Webpage layouts follow the CSS box model and can be partitioned into rectangular blocks separated by solid-colored lines.
    Invoked in Section 3.1, where dividing line detection scans for solid-colored rows/columns; Table 1 shows some samples violate this.
  • domain assumption MLLMs given a small image block and a chain-of-thought prompt will generate reasonably accurate HTML/CSS for that block.
    The whole block-wise synthesis in Section 3.2 relies on this; the paper's own error analysis (Section 5.4) shows MLLMs sometimes misplace content.
  • domain assumption The composite verifier score (MAE + CLIP) correlates with human judgment of webpage similarity.
    Used in Section 3.3 for dynamic selection; the paper found MLLM-as-a-judge unreliable and fell back to this pixel-level metric.
  • domain assumption EasyOCR provides correct text region bounding boxes for avoiding splits.
    Used in the dividing algorithm optimization in Section 4.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAM2-UNeXT: An Improved High-Resolution Baseline for Adapting Foundation Models to Downstream Segmentation Tasks." pith.science (2026). https://pith.science/paper/2D3R6BYP

@misc{pith2026250803566,
  author       = {Pith},
  title        = {Pith review of: SAM2-UNeXT: An Improved High-Resolution Baseline for Adapting Foundation Models to Downstream Segmentation Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2D3R6BYP}},
  note         = {Machine review of arXiv:2508.03566}
}
read the original abstract

Recent studies have highlighted the potential of adapting the Segment Anything Model (SAM) for various downstream tasks. However, constructing a more powerful and generalizable encoder to further enhance performance remains an open challenge. In this work, we propose SAM2-UNeXT, an advanced framework that builds upon the core principles of SAM2-UNet while extending the representational capacity of SAM2 through the integration of an auxiliary DINOv2 encoder. By incorporating a dual-resolution strategy and a dense glue layer, our approach enables more accurate segmentation with a simple architecture, relaxing the need for complex decoder designs. Extensive experiments conducted on four benchmarks, including dichotomous image segmentation, camouflaged object detection, marine animal segmentation, and remote sensing saliency detection, demonstrate the superior performance of our proposed method. The code is available at https://github.com/WZH0120/SAM2-UNeXT.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 14 linked inside Pith

  1. [1]

    The Common Crawl dataset

    2024. The Common Crawl dataset. https://data.commoncrawl.org/

    2024

  2. [2]

    The screen-to-shot project on the Github

    2024. The screen-to-shot project on the Github . https://github.com/abi/ screenshot-to-code/

    2024

  3. [3]

    Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrit- twieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P

    Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrit- twieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael ...

    arXiv 2023

  4. [4]

    Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. https://api.semanticscholar.org/CorpusID:268232499

    2024

  5. [5]

    Manmatha

    Srikar Appalaraju, Peng Tang, Qi Dong, Nishant Sankaran, Yichu Zhou, and R. Manmatha. 2023. DocFormerv2: Local Features for Document Understanding. In Proceedings of the AAAI Conference on Artificial Intelligence

    2023

  6. [6]

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of Multimodal Large Language Models: A Survey. ArXiv abs/2404.18930 (2024)

    arXiv 2024

  7. [7]

    Tony Beltramelli. 2018. pix2code: Generating Code from a Graphical User Inter- face Screenshot. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS 2018, Paris, France, June 19-22, 2018 . ACM, 3:1–3:6

    2018

  8. [8]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

    2021

Show all 60 references
  1. [9]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Pi- otr Nyczyk, and Torsten Hoefler. 2024. Graph of Thoughts: Solving Elaborate Problems with Large Language Models. In Proceed...

    2024

  2. [10]

    Zhangqian Bi, Yao Wan, Zheng Wang, Hongyu Zhang, Batu Guan, Fangxin Lu, Zili Zhang, Yulei Sui, Hai Jin, and Xuanhua Shi. 2024. Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback. In Proceedings of the Findings of the Associati...

    2024

  3. [11]

    Shang Chai, Liansheng Zhuang, and Fengying Yan. 2023. LayoutDM: Transformer- based Diffusion Model for Layout Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 18349–18358

    2023

  4. [12]

    Dongping Chen, Ruoxi Chen, Shu Pu, Zhaoyi Liu, Yanru Wu, Caixi Chen, Benlin Liu, Yue Huang, Yao Wan, Pan Zhou, et al. 2024. Interleaved Scene Graph for Interleaved Text-and-Image Generation Assessment. InProceedings of the Twelfth International Conference on Learning Represent...

    2024

  5. [13]

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Pan Zhou, Yao Wan, and Lichao Sun. 2024. MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark. In Proceedings of the International Conference on Machine Learning

    2024

  6. [14]

    Tu Anh Dinh, Carlos Mullov, Leonard Barmann, Zhaolin Li, Danni Liu, Simon Reiß, Jueun Lee, Nathan Lerzer, Fabian Ternava, Jianfeng Gao, Alexander Waibel, Tamim Asfour, Michael Beigl, Rainer Stiefelhagen, Carsten Dachsbacher, Klemens Bohm, and Jan Niehues. 2024. SciEx: Benchmar...

    2024

  7. [15]

    Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. ArXiv abs/2204.05999 (2022)

    2022 arXiv

  8. [16]

    Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Bohua Chen, Yi Su, Dong- ping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, and Xiangliang Zhang

  9. [17]

    Yi Gui, Yao Wan, Zhen Li, Zhongyi Zhang, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, and Xiangliang Zhang. 2025. UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs. In Proceedings of the International World Wid...

    2025

  10. [18]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence.ArXiv abs/2401.14196 (2024)

    2024 arXiv

  11. [19]

    Hongcheng Guo, Wei Zhang, Junhao Chen, Yaonan Gu, Jian Yang, Junjia Du, Binyuan Hui, Tianyu Liu, Jianxin Ma, Chang Zhou, and Zhoujun Li. 2024. IW- Bench: Evaluating Large Multimodal Models for Converting Image-to-Web.ArXiv abs/2409.18980 (2024)

    2024

  12. [20]

    Girshick

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. 2020. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2 (2020), 386–397

    2020

  13. [21]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks ...

    2021

  14. [22]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Proba- bilistic Models. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual

    2020

  15. [23]

    Fan Hu, Yanlin Wang, Lun Du, Xirong Li, Hongyu Zhang, Shi Han, and Dongmei Zhang. 2023. Revisiting Code Search in a Two-Stage Paradigm. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM 2023, Singapore, 27 February 2023 - 3 March ...

    2023

  16. [24]

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking. In Proceed- ings of the 30th ACM International Conference on Multimedia

    2022

  17. [25]

    Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi

  18. [26]

    Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conver- sion of Web Screenshots into HTML Code with the WebSight Dataset. ArXiv abs/2403.09029 (2024)

    2024 arXiv

  19. [27]

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Mar- tin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2Struct: Screenshot Parsing as Pretraining for Visual Lan- guage Understanding. In Proceedings of ...

    2023

  20. [28]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s Verify Step by Step. In Proceedings of the Twelfth International Conference on Learning Representations, ICLR 2024, Vie...

    2024

  21. [29]

    Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang Yang, Jian-Guang Lou, and Dongmei Zhang. 2023. LayoutPrompter: Awaken the Design Ability of Large Language Models. In Advances in Neural Information Processing Systems 36: Annual Confer- ence on Neural Information Processing Systems ...

    2023

  22. [30]

    Haoyu Lu, Wen Liu, Bo Zhang, Bing-Li Wang, Kai Dong, Bo Liu (Benjamin Liu), Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. DeepSeek-VL: Towards Real-World Vision-Language Understanding. ArXiv abs/2403...

    2024 arXiv

  23. [31]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. WizardCoder: Em- powering Code Large Language Models with Evol-Instruct.ArXiv abs/2306.08568 (2023)

    2023 arXiv

  24. [32]

    OpenAI. 2023. GPT-4 Technical Report. ArXiv abs/2303.08774 (2023)

    2023 arXiv

  25. [33]

    Geliang Ouyang, Jingyao Chen, Zhihe Nie, Yi Gui, Yao Wan, Hongyu Zhang, and Dongping Chen. 2025. nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguist...

    2025

  26. [34]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke E

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, J...

    2022 arXiv

  27. [35]

    Shu Pu, Yaochen Wang, Dongping Chen, Yuhang Chen, Guohao Wang, Qi Qin, Zhongyi Zhang, Zhiyuan Zhang, Zetong Zhou, Shuang Gong, et al. 2025. Judge Anything: MLLM as a Judge Across Any Modality. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mini...

    2025

  28. [36]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings...

    2021

  29. [37]

    Alex Robinson. 2019. Sketch2code: Generating a website from a paper mockup. ArXiv abs/1905.13750 (2019). LaTCoder: Converting Webpage Design to Code with Layout-as-Thought KDD ’25, August 3–7, 2025, Toronto, ON, Canada

    2019 arXiv

  30. [38]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, CVPR 2022, New Orleans, LA, USA, June 1...

    2022

  31. [39]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. InProceedings of the Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 -...

    2015

  32. [40]

    Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. 2000. The Earth Mover’s Distance as a Metric for Image Retrieval. International Journal of Computer Vision 40 (2000), 99–121

    2000

  33. [41]

    Evan Shelhamer, Jonathan Long, and Trevor Darrell. 2014. Fully convolutional networks for semantic segmentation. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 3431–3440

    2014

  34. [42]

    Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang

  35. [43]

    Zhihong Sun, Yao Wan, Jia Li, Hongyu Zhang, Zhi Jin, Ge Li, and Chen Lyu

  36. [44]

    Yao Wan, Zhangqian Bi, Yang He, Jianguo Zhang, Hongyu Zhang, Yulei Sui, Guandong Xu, Hai Jin, and Philip Yu. 2024. Deep learning for code intelligence: Survey, benchmark and toolkit. Comput. Surveys 56, 12 (2024), 1–41

    2024

  37. [45]

    Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Pap...

    2025

  38. [46]

    Yao Wan, Jingdong Shu, Yulei Sui, Guandong Xu, Zhou Zhao, Jian Wu, and Philip Yu. 2019. Multi-modal attention network learning for semantic source code retrieval. In Proceedings of 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 13–25

    2019

  39. [47]

    Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R. Lyu. 2024. Automatically Generating UI Code from Screen- shot: A Divide-and-Conquer-Based Approach. ArXiv abs/2406.16386 (2024)

    2024 arXiv

  40. [48]

    Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S Yu. 2018. Improving automatic source code summarization via deep rein- forcement learning. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering . 397–407

    2018

  41. [49]

    Yuxuan Wan, Yi Dong, Jingyu Xiao, Yintong Huo, Wenxuan Wang, and Michael R. Lyu. 2024. MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs. ArXiv abs/2412.15310 (2024)

    2024 arXiv

  42. [50]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neur...

    2022

  43. [51]

    Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zhiyao Xu, and Michael R. Lyu. 2024. Interaction2Code: How Far Are We From Automatic Interactive Webpage Gener- ation? ArXiv abs/2411.03292 (2024)

    2024

  44. [52]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025. Qwen2.5-Omni Technical Report. ArXiv abs/2503.20215 (2025)

    2025 arXiv

  45. [53]

    Wenhua Wang, Yuqun Zhang, Yulei Sui, Yao Wan, Zhou Zhao, Jian Wu, S Yu Philip, and Guandong Xu. 2020. Reinforcement-learning-guided source code sum- marization using hierarchical attention.IEEE Transactions on software Engineering 48, 1 (2020), 102–119

    2020

  46. [54]

    Xing, Xiaodan Liang, and Zhiqiang Shen

    Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, and Zhiqiang Shen. 2024. Web2Code: A Large-scale ...

    2024

  47. [55]

    Ti Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang

  48. [57]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Pr...

    2023

  49. [60]

    en " > 3 < head > 4 < meta charset =

    Bridging Design and Development with Automated Declarative UI Code Generation. ArXiv abs/2409.11667 (2024). KDD ’25, August 3–7, 2025, Toronto, ON, Canada Yi Gui et al. Prompt for Block-wise Code Synthesis You are an expert Tailwind developer. Based on the reference screenshot...

    2024 arXiv

  50. [2023]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    LayoutDM: Discrete Diffusion Model for Controllable Layout Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10167–10176

  51. [2024]

    In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA, USA, October 27 - November 1, 2024

    Sifting through the Chaff: On Utilizing Execution Feedback for Ranking the Generated Code Candidates. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA, USA, October 27 - November 1, 2024 . ACM, 229–241

    2024

  52. [2025]

    In Proceedings of the International World Wide Web Conference, WWW 2025, Sydney, April 28–May 2, 2024

    WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs. In Proceedings of the International World Wide Web Conference, WWW 2025, Sydney, April 28–May 2, 2024

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.