Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DesignCoder claims that making an MLLM first reconstruct the nested component hierarchy of a design, then generate code per subtree, and finally repair the rendered result by visual comparison, yields React Native code that is both more…

desk verdict DesignCoder is a sensible three-stage pipeline for UI code generation, but its headline numbers rest on shaky evaluation choices that need fixing before I'd trust the effect sizes. read the letter →

arxiv 2506.13663 v1 pith:J3FZ4WTO submitted 2025-06-16 cs.SE

Yunnong Chen , Shixian Ding , YingYing Zhang , Wenkai Chen , Jinzhou Du , Lingyun Sun , Liuqing Chen This is my paper · ORCID
classification cs.SE
keywords UIcodegenerationmultimodallargelanguagemodelshierarchicalunderstandingchain-of-thoughtself-correctionReactNativedesignmockupvisualfidelity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DesignCoder claims that generating UI code from design mockups fails mainly because multimodal language models cannot grasp the nested hierarchy of a screen, and that this can be fixed by a three-stage pipeline. The pipeline first builds a UI Grouping Chain—a chain-of-thought that divides the screen into regions, names the elements semantically, and groups them into a component tree. It then generates React Native code per subtree using design metadata, and finally renders the page, compares it visually against the original mockup, and repairs the code component by component. The paper reports that on 300 real mockups this pipeline beats specialized and general-purpose baselines on both visual similarity and code-structure similarity, and that senior developers rate the output higher on availability, modification time, readability, and maintainability.

What carries the argument

UI Grouping Chain: a multimodal chain-of-thought that turns a flat list of design layers into a hierarchical component tree through three steps—UI division, semantic extraction, and component grouping—each followed by a post-processing correction. The tree is then treated as the skeleton for divide-and-conquer code generation and for the self-correction loop, where a UI automation tool captures the rendered page, visual comparison prompts produce targeted repair suggestions, and each component's code is repaired independently before merging.

What would settle it

Recompute TreeBLEU, Container Match, and Tree Edit Distance using reference trees written independently by developers (not derived from CodeFun) and check whether DesignCoder still beats Prototype2Code on the same 300 mockups; a drop to parity would indicate that the reported structural gains are artifacts of the reference construction.

Watch

Extended reading notes

Core claim

The central claim is that explicit hierarchical reasoning about UI structure, before any code is written, is the key to both visual fidelity and structural quality in MLLM-based code generation. DesignCoder constructs a component tree via a multimodal chain-of-thought where the model first segments the flattened layer list into semantic regions, annotates each element's meaning, and then groups elements into components; this tree, rather than the raw screenshot, drives code generation. The paper supports the claim with a 300-mockup benchmark showing the full pipeline improves over the best baseline on MSE by 37.63%, CLIP by 9.52%, SSIM by 12.82%, and on TreeBLEU by 30.19%, Container Match by 29.31%, and Tree Edit Distance by 24.67%. A five-developer user study adds that the generated code is rated closer to production-ready than the output of the strongest MLLM-based baseline.

Load-bearing premise

The load-bearing premise is that the reference component trees used to score code structure—produced by the CodeFun platform and manually corrected by two authors—are a neutral and valid standard, rather than a standard that shares grouping biases with DesignCoder.

Editorial extensions

If this is right

  • If correct, MLLM-based UI code generation can reach production-like fidelity on design mockups without manual layout coding.
  • Component-tree-driven generation yields reusable, responsive components, reducing the time developers need to modify generated code.
  • The self-correction loop generalizes beyond code: any renderable artifact could be improved by comparing its rendered output against the original design.
  • Semantic, MLLM-driven grouping could replace rule-based approaches for novel UI patterns that do not fit predefined rules.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editor's inference: The same hierarchy-first, render-then-repair recipe could transfer to web (HTML/CSS) and desktop UI generation, since the defects it targets—omission, distortion, and misalignment—are not specific to React Native.
  • Editor's inference: The gain from the self-correction stage may scale with the visual grounding ability of the underlying MLLM, so the reported ablation numbers could shift meaningfully if the foundation model changes.
  • Editor's inference: The structural ground truth is derived from the CodeFun platform, so if the authors' corrections inherited CodeFun's grouping biases, the structural-similarity advantages over rule-based baselines could be partly an artifact of the reference construction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DesignCoder, a three-stage pipeline for generating React Native code from UI mockups: a UI Grouping Chain that divides the UI, extracts element semantics, and builds a component tree; a hierarchical divide-and-conquer code and style generation step that consumes design metadata; and an iterative visual self-correction step that compares rendered screenshots against the original mockup. The method is evaluated on 300 mockups (250 from the Figma community and 50 from an unnamed company) against Prototype2Code, DeclarUI, GPT-4o, Claude-3.5, LLaVA, and CodeFun, using visual metrics (MSE, CLIP, SSIM) and structural metrics (TreeBLEU, Container Match, Tree Edit Distance). The paper reports large improvements over the best baseline, e.g., a 37.6% reduction in MSE and a 30.2% increase in TreeBLEU on the Figma dataset, and a user study with five professional developers. The central claim is that DesignCoder produces both more visually faithful and more structurally maintainable UI code than existing methods.

Significance. If the reported results hold, DesignCoder would be a practically valuable contribution to industrial UI code generation, combining hierarchy reasoning, metadata use, and automated visual repair. The paper has clear strengths: a new 300-mockup dataset from two sources, a systematic ablation isolating the grouping chain and the refinement module, a user study with experienced developers, and a level of implementation detail that would support reproduction. However, the headline numbers currently rest on two unresolved evaluation-validity issues: the treatment of non-compiling baseline outputs is under-specified and could inflate all reported improvements, and the structural ground truth is derived from CodeFun, which is itself a reported baseline. The main comparison tables also lack variance information and significance tests despite the use of the word 'significantly.' These issues are fixable but require additional experiments and reporting; they are not mere presentation concerns.

major comments (4)
  1. [Section 3.2.3, Tables 1 and 2] The statement 'For code that failed to compile, we assigned a value of zero to all its evaluation metrics' is ambiguous and potentially consequential. For MSE, zero is the best possible score, so assigning zero to non-compiling code would actually reward failure; if instead a worst-case value was intended, that is not what the text says. More importantly, the paper never reports the per-method compile success rate. Because DesignCoder's self-correction loop (Section 2.3) is explicitly designed to fix compile errors, its compile rate is likely near 100%, while generic baselines such as GPT-4o or LLaVA may fail to compile at non-negligible rates. Assigning worst-case scores to all non-compiling baseline outputs would inflate the reported relative improvements in both visual and structural metrics. Please report compile rates for every method, clarify exactly how non-compiling outputs were scored for each metric, and recompute the headline comparisons on the subset of outputs that compile (or use an explicit, transparently specified penalty with sensitivity analysis).
  2. [Section 3.2.2, Tables 1 and 2] The structural ground truth is constructed by running CodeFun on the design files and then having two authors manually correct the resulting component trees, while CodeFun is simultaneously listed as a baseline in Tables 1 and 2. Since DesignCoder also consumes design metadata and generates code from a component tree, the structural metrics (TreeBLEU, Container Match, Tree Edit Distance) may be biased in DesignCoder's favor if the reference trees inherit CodeFun's grouping conventions. The manual correction step is not documented in enough detail to rule this out: there is no inter-annotator agreement, no description of how many corrections were made, and no independent validation of the reference trees. Please (i) describe the correction protocol and report inter-annotator agreement, (ii) quantify how much the corrected trees differ from CodeFun's raw output, and (iii) report structural metrics against an independently constructed reference, for example trees written by front-end developers not involved in the method, or use multiple references.
  3. [Section 3.3, Tables 1 and 2] No variance, confidence interval, or significance test is reported for any metric in Tables 1 and 2, yet the abstract and Section 3.3 use the term 'significantly.' With 300 mockups (and 50 in the company dataset), it is feasible to compute per-sample standard errors and paired tests, e.g., paired Wilcoxon or bootstrap confidence intervals for the difference between DesignCoder and each baseline. Without such statistics, the reader cannot determine whether the reported advantages are stable or driven by a small number of outlier cases. This is particularly important for the structural metrics, where the arbitrary penalties for trees with no matching nodes (Section 3.2.2) can interact with per-case variability.
  4. [Section 3.5.1, Table 4] The user study involves five participants and 15 prototypes, and the reported Mann-Whitney U p-values appear to treat individual ratings as independent observations. With only five raters and no random-effects model or per-participant aggregation, these p-values almost certainly overstate the significance of the differences. Please report the unit of analysis, per-participant averages, and an appropriate paired test (e.g., Wilcoxon signed-rank on aggregated per-participant scores). Also state whether the participants were blinded to which method produced each code sample, since the evaluation involves subjective ratings of readability and maintainability.
minor comments (4)
  1. [Section 3.2.2, Eq. (8)] TreeBLEU is described as the proportion of height-1 subtrees in the given tree that match the reference tree, but the equation only has the matched count in the numerator and the reference subtree count in the denominator. Please clarify whether this is precision, recall, or a combined score, and describe how the mobile DOM tree is extracted from React Native code for this comparison.
  2. [Section 2.3 and Figure 2] The number of self-correction iterations is denoted 'N' in Figure 2 but its value is never reported. Please state the value used in the experiments and include a sensitivity analysis over N, since the cost of the method scales with this number.
  3. [Throughout] There are minor typographical and formatting issues: 'LLaV A-v1.5-7B' in Section 3.2.1 has an unintended space, 'Tree Edit Distace' appears in the Table 3 title, and the reference to CodeFun is inconsistently capitalized as 'Codefun' and 'CodeFun.' The ACM reference header also uses a 2018 placeholder that should be updated.
  4. [Section 4] The threats-to-validity section acknowledges that 'complete visual similarity does not necessarily indicate high-quality generated code' and that 'manually annotated code structures may not fully reflect best practices.' These caveats should be operationalized by making the corrected reference trees and the evaluation scripts publicly available so readers can inspect the ground truth and reproduce the structural metrics.

Circularity Check

1 steps flagged · score 4.0 of 10

Structural ground truth is derived from CodeFun, which is itself a compared baseline, making the structural-similarity claims partly self-referential; the generation pipeline itself is not circular.

  1. self definitional [Section 3.2.2 (Evaluation Metrics), Eqs. (8)-(9), and Tables 1-2]
    "To obtain the ground truth of UI code, we introduce Codefun [1], an enterprise-level code generation platform, which converts well-structured design files into front-end code. Subsequently, two authors manually inspected the generated component tree, focusing on hierarchy and component tags."

    The structural ground truth (the reference tree t_r in Eq. (8), and the reference for Tree Edit Distance and Container Match) is produced by CodeFun, and CodeFun is itself a scored baseline in Table 2. Structural similarity therefore measures how closely each method agrees with one baseline's component-tree and tag conventions, rather than with an independent gold standard. The claimed TreeBLEU/Container Match/Tree Edit Distance improvements (30.19%, 29.31%, 29.47%) are thus partly 'agreement with CodeFun' scores. The two authors' manual correction mitigates but does not remove the coupling: the raw CodeFun output is still compared, and the correction is performed by the same evaluating authors.

full rationale

The core method is not circular: DesignCoder takes mockups and metadata, builds component trees through the UI Grouping Chain, generates code by divide-and-conquer, and self-corrects by rendering comparison. No parameter is fitted and then renamed as a prediction, and no equation defines the output in terms of the input. Self-citations are present but not load-bearing: [8], [10], [11], and [30] are background, procedure, or baseline references, and no uniqueness theorem is imported from the authors' prior work. The one significant circularity is in the structural evaluation: the reference trees come from CodeFun, a platform that is simultaneously a baseline, so the structural metrics are partly a measure of agreement with that baseline rather than an independent standard. This affects the TreeBLEU/TED/CM claims but does not undermine the visual metrics or the user study. Separately, the stated rule that non-compiling code receives zero on all metrics is a serious validity confound for the reported improvements (and for MSE, where lower is better, zero would be a perfect score, making the penalty ambiguous), but that is an evaluation-fairness issue rather than a circular derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on domain assumptions about metadata reliability, ground-truth construction, rendering fidelity, and MLLM capability rather than on fitted parameters or new physical entities. The hand-chosen division range and unspecified self-correction iteration count are heuristics; the structural ground truth derived from CodeFun is the most consequential evaluation choice.

free parameters (3)
  • division count range (3 to 10)
    Algorithm 1 requires a segmentation into 3 to 10 regions, otherwise it rolls back and re-segments. This threshold is chosen by hand and is not justified by data.
  • self-correction iterations N
    Figure 2 shows the refinement loop repeated 'x N' times, but the paper never specifies N or a stopping criterion, and the evaluation does not control for the number of correction rounds.
  • foundation model choice (GPT-4o) = GPT-4o
    Every stage of the pipeline uses GPT-4o; all reported gains are coupled to this specific model and prompt set, and no alternative model variation is tested.
assumptions (4)
  • domain assumption Design metadata extracted by Figma CLI (id, type, bbox, style) is accurate and complete for all mockups.
    Section 3.1 assumes the metadata reflects the real UI structure; in practice mockup layers often contain nested or semantically empty containers.
  • domain assumption The CodeFun output corrected by two authors is a valid ground-truth code structure.
    Section 3.2.2 relies on this corrected tree as the reference for TreeBLEU, Tree Edit Distance, and Container Match, without external validation of the corrected structure.
  • domain assumption Rendered screenshots from Appium on an Android emulator are a faithful proxy for the design mockup in visual comparison.
    Section 2.3 compares original and rendered component images to drive the self-correction step; font rendering, screen density, and emulator artifacts can distort this comparison.
  • domain assumption MLLMs (specifically GPT-4o) can reliably perform semantic extraction, grouping, code generation, and repair in this prompting structure.
    The whole method depends on one closed model at every stage; no analysis of failure modes or model variability is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models." pith.science (2026). https://pith.science/paper/J3FZ4WTO

@misc{pith2026250613663,
  author       = {Pith},
  title        = {Pith review of: DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J3FZ4WTO}},
  note         = {Machine review of arXiv:2506.13663}
}
read the original abstract

Multimodal large language models (MLLMs) have streamlined front-end interface development by automating code generation. However, these models also introduce challenges in ensuring code quality. Existing approaches struggle to maintain both visual consistency and functional completeness in the generated components. Moreover, they lack mechanisms to assess the fidelity and correctness of the rendered pages. To address these issues, we propose DesignCoder, a novel hierarchical-aware and self-correcting automated code generation framework. Specifically, we introduce UI Grouping Chains, which enhance MLLMs' capability to understand and predict complex nested UI hierarchies. Subsequently, DesignCoder employs a hierarchical divide-and-conquer approach to generate front-end code. Finally, we incorporate a self-correction mechanism to improve the model's ability to identify and rectify errors in the generated code. Extensive evaluations on a dataset of UI mockups collected from both open-source communities and industry projects demonstrate that DesignCoder outperforms state-of-the-art baselines in React Native, a widely adopted UI framework. Our method achieves a 37.63%, 9.52%, 12.82% performance increase in visual similarity metrics (MSE, CLIP, SSIM) and significantly improves code structure similarity in terms of TreeBLEU, Container Match, and Tree Edit Distance by 30.19%, 29.31%, 24.67%. Furthermore, we conducted a user study with professional developers to assess the quality and practicality of the generated code. Results indicate that DesignCoder aligns with industry best practices, demonstrating high usability, readability, and maintainability. Our approach provides an efficient and practical solution for agile front-end development, enabling development teams to focus more on core functionality and product innovation.

Figures

Figures reproduced from arXiv: 2506.13663 by the authors.

Figure 1
Figure 1. Motivating examples are provided. Red boxes 1, 2, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overview of DesignCoder. Task Description Task Description Example output (Division) "division_name" "element_list" : , [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The prompt used to construct UI grouping chain. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Examples of the rendering results from both the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Examples illustrating the effectiveness of DesignCoder in rendering and code structure, compared with two baseline [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Examples illustrating DesignCoder’s responsiveness across different screen sizes, compared with two baseline methods. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

    cs.SE 2026-07 accept novelty 7.0 of 10

    UI2App introduces a benchmark showing that vision-language models can reconstruct web page visuals but largely fail to infer the underlying interaction logic from screenshots alone.

  2. WebDesignIter: Co-Evolving Design Knowledge for Repository-Level Front-End Code Generation

    cs.SE 2026-07 conditional novelty 6.0 of 10

    A co-evolving design-and-code knowledge graph improves multi-iteration front-end repository generation by ~9.5 Pass@2 points and cuts input tokens 25–30× versus agent baselines.

  3. SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI

    cs.HC 2025-09 conditional novelty 6.0 of 10

    Structured, parameterized UI specifications extracted from screenshots improve LLM-based generation fidelity and iterative control compared with text prompting.

Reference graph

Works this paper leans on

36 extracted references · 22 canonical work pages · cited by 3 Pith papers

  1. [1]

    Codefun website

    2024. Codefun website. https://code.fun/. [Accessed on Dec. 27, 2024]

    work page 2024

  2. [2]

    Codia Design to Code

    2024. Codia Design to Code. https://codia.ai/design-to-code/. [Accessed on Dec. 27, 2024]

    work page 2024

  3. [3]

    Figma Software

    2024. Figma Software. https://www.figma.com/. [Accessed on Dec. 27, 2024]

    work page 2024

  4. [4]

    Imgcook website

    2024. Imgcook website. https://www.imgcook.com/. [Accessed on Dec. 27, 2024]

    work page 2024

  5. [5]

    Sketch Software

    2024. Sketch Software. https://www.sketch.com/. [Accessed on Dec. 27, 2024]

    work page 2024

  6. [6]

    Tony Beltramelli. 2018. pix2code: Generating code from a graphical user inter- face screenshot. InProceedings of the ACM SIGCHI symposium on engineering interactive computing systems. 1–6

    2018

  7. [7]

    Chunyang Chen, Ting Su, Guozhu Meng, Zhenchang Xing, and Yang Liu. 2018. From ui design image to gui skeleton: a neural machine translator to bootstrap mobile gui implementation. InProceedings of the 40th International Conference on Software Engineering. 665–676

    work page 2018

  8. [8]

    Liuqing Chen, Yunnong Chen, Shuhong Xiao, Yaxuan Song, Lingyun Sun, Yankun Zhen, Tingting Zhou, and Yanfang Chang. 2024. EGFE: End-to-end Grouping of Fragmented Elements in UI Designs with Multimodal Learning. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–12

    work page 2024

Show all 36 references
  1. [9]

    Wen-Yin Chen, Pavol Podstreleny, Wen-Huang Cheng, Yung-Yao Chen, and Kai- Lung Hua. 2022. Code generation from a graphical user interface via attention- based encoder–decoder model.Multimedia Systems28, 1 (2022), 121–130

    2022

  2. [10]

    Yunnong Chen, Shuhong Xiao, Jiazhi Li, Tingting Zhou, Yanfang Chang, Yankun Zhen, Lingyun Sun, and Liuqing Chen. 2024. Fragmented Layer Grouping in GUI Designs Through Graph Learning Based on Multimodal Information.arXiv preprint arXiv:2412.05555(2024)

    2024 arXiv

  3. [11]

    Yunnong Chen, Yankun Zhen, Chuning Shi, Jiazhi Li, Liuqing Chen, Zejian Li, Lingyun Sun, Tingting Zhou, and Yanfang Chang. 2023. UI layers merger: merging UI layers via visual learning and boundary prior.Frontiers of Information Technology & Electronic Engineering24, 3 (2023), 373–387

    2023

  4. [12]

    Sidong Feng, Mingyue Yuan, Jieshan Chen, Zhenchang Xing, and Chunyang Chen. 2023. Designing with Language: Wireframing UI Design Intent with Generative Large Language Models.arXiv preprint arXiv:2312.07755(2023)

    2023 arXiv

  5. [13]

    Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Bohua Chen, Dong- ping Chen, Siyuan Wu, Xing Zhou, et al . 2025. WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs. InTHE WEB CONFERENCE 2025

    2025

  6. [14]

    Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Shaoling Dong, Xing Zhou, and Wenbin Jiang. 2024. Vision2ui: A real-world dataset with layout for code generation from ui designs.arXiv preprint arXiv:2404.06369(2024)

    2024 arXiv

  7. [15]

    Yi Gui, Yao Wan, Zhen Li, Zhongyi Zhang, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, et al. 2025. UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs. InTHE WEB CONFERENCE 2025

    2025

  8. [16]

    Xue Jiang, Yihong Dong, Yongding Tao, Huanyu Liu, Zhi Jin, Wenpin Jiao, and Ge Li. 2024. ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generation.arXiv preprint arXiv:2411.07112 (2024). Accepted by ICSE 2025

    2024 arXiv

  9. [17]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al

  10. [18]

    Zhu Lin, Feng Yingchaojie, Zhu Hang, Wang Sijia, Zhu Minfeng, Yu Chenhao, Zhang Yuhui, Xu Daxing, Zhao Deming, Feng Yujun, et al. 2024. Figma2Code: Automatic code generation method for Figma design drafts.Journal of Computer- Aided Design & Computer Graphics(2024)

    2024

  11. [19]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning.Advances in neural information processing systems36 (2023), 34892–34916

    2023

  12. [20]

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. 2024. Grounding dino: Marry- ing dino with grounded pre-training for open-set object detection. InEuropean Conference on Computer Vision. Springer, 38–55

    2024

  13. [21]

    Meta. 2024. React Native. https://reactnative.dev/

    2024

  14. [22]

    Soumik Mohian and Christoph Csallner. 2020. Doodle2App: Native app code by freehand UI sketching. InProceedings of the IEEE/ACM 7th International Conference on Mobile Software Engineering and Systems. 81–84

    2020

  15. [23]

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster R-CNN: To- wards real-time object detection with region proposal networks.IEEE transactions on pattern analysis and machine intelligence39, 6 (2016), 1137–1149

    2016

  16. [24]

    Alex Robinson. 2019. Sketch2code: Generating a website from a paper mockup. arXiv preprint arXiv:1905.13750(2019)

    2019 arXiv

  17. [25]

    Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2code: How far are we from automating front-end engineering?arXiv e-prints(2024), arXiv–2403

    2024

  18. [26]

    Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R Lyu. 2024. Automatically generating UI code from screenshot: A divide-and-conquer-based approach.arXiv preprint arXiv:2406.16386(2024)

    2024 arXiv

  19. [27]

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing13, 4 (2004), 600–612

    2004

  20. [28]

    Jason Wu. 2024. UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Lon...

    2024

  21. [29]

    Jason Wu, Xiaoyi Zhang, Jeff Nichols, and Jeffrey P Bigham. 2021. Screen parsing: Towards reverse engineering of ui models from screenshots. InThe 34th Annual ACM Symposium on User Interface Software and Technology. 470–483

    2021

  22. [30]

    Shuhong Xiao, Yunnong Chen, Jiazhi Li, Liuqing Chen, Lingyun Sun, and Tingting Zhou. 2024. Prototype2code: End-to-end front-end code generation from ui design prototypes. InInternational Design Engineering Technical Conferences and Computers and Information in Engineering Conf...

    2024

  23. [31]

    Shuhong Xiao, Yunnong Chen, Yaxuan Song, Liuqing Chen, Lingyun Sun, Yankun Zhen, Yanfang Chang, and Tingting Zhou. 2024. UI semantic component group detection: Grouping UI elements with similar semantics in mobile graphical user interface.Displays83 (2024), 102679

    2024

  24. [32]

    Mulong Xie, Zhenchang Xing, Sidong Feng, Xiwei Xu, Liming Zhu, and Chunyang Chen. 2022. Psychologically-inspired, unsupervised inference of perceptual groups of GUI widgets from GUI images. InProceedings of the 30th ACM joint European software engineering conference and sympos...

    2022

  25. [33]

    Yong Xu, Lili Bo, Xiaobing Sun, Bin Li, Jing Jiang, and Wei Zhou. 2021. im- age2emmet: Automatic code generation from web user interface image.Journal of Software: Evolution and Process33, 8 (2021), e2369

    2021

  26. [34]

    Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang

  27. [2023]

    InProceedings of the IEEE/CVF international conference on computer vision

    Segment anything. InProceedings of the IEEE/CVF international conference on computer vision. 4015–4026

  28. [2024]

    Accepted by FSE 2025

    Bridging design and development with automated declarative ui code generation.arXiv preprint arXiv:2409.11667(2024). Accepted by FSE 2025

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.