Pith. sign in

REVIEW 5 major objections 6 minor 46 references

LEGOUI: Designing with UI-DSL Bricks to Balance Transparency and Controllability

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LegoUI claims that exposing a UI generator's intermediate design decisions as editable, provenance-tagged specification items gives beginner users more transparency, control, and intent alignment than one-shot prompt-to-UI tools.

desk verdict Useful staged-generation pattern and a credible user study, but the 95% extraction accuracy is inflated by circular ground truth and a lenient metric; deserves serious review. read the letter →

arxiv 2608.04293 v1 pith:4CJAI3CB submitted 2026-08-04 cs.HC

Yinsi Zhou , Mingyue Yuan , Hongyue Xu , Jieshan Chen , Dong Wen , Shidong Pan , Xiwei Xu , Wenjie Zhang
show 3 more authors
Aaron Quigley Zhenchang Xing Gelareh Mohammadi
This is my paper · ORCID
classification cs.HC
keywords UIgenerationUI-DSLtransparencycontrollabilityprovenancestagedhuman-AIcollaborationiterativedesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most prompt-to-UI tools produce a finished interface in a single pass, leaving the model's intermediate design decisions invisible; when the result misses the intent, users can only rewrite the prompt and regenerate. LegoUI claims a better structure: generation proceeds through staged reasoning steps, and every decision — components, layout regions, interaction rules, relations, and style — is recorded as an editable item in a UI-specific language, tagged with its provenance (from the prompt, inferred by the model, or added by the user). Users can inspect, accept, reject, or add items before any interface is rendered, and later stages generate from the revised specification, so intent accumulates across iterations rather than being reinterpreted each time. If established, this matters because beginner designers could catch and steer misinterpretations early instead of debugging through prompts, at the price of extra review work. The paper supports this with over 95% extraction accuracy on 40 real-world prompts and a 15-participant study in which LegoUI was rated above the one-shot baselines on transparency, controllability, and intent alignment.

What carries the argument

The load-bearing mechanism is the UI-DSL, a domain-specific language whose items are atomic design commitments organized into seven specifications: intent, global, component pool, layout, interaction, relation, and style. Each item is a structured tuple — for example, a component $C=(N_c,\tau_c,A_c,P_c)$ with identifier, type, attribute assertions, and provenance; an interaction rule $E_c[\Phi]\Rightarrow A\langle P_i\rangle$; a relation edge $c_1\to c_2:k(a)\langle P_r\rangle$; a style override $t\mapsto\{h_1,\ldots,h_k\}\langle P_s\rangle$ — and every item carries provenance entries recording whether it came from the user's prompt, from model inference, or from a user edit. The mechanism's work is to make the design state persistent and actionable: accepting an item preserves it for later stages, rejecting it removes it from the assumptions later inference reads, and adding an item injects an explicit user requirement into the same reasoning context. Because generation always proceeds from the current specification state, the same machinery supports exploring alternative versions without discarding accumulated intent, and each rendered interface can be traced back to the commitments that produced it.

What would settle it

Build an independent ground truth for the same 40 prompts: annotators who have never seen LegoUI's outputs or category definitions extract explicit requirements against an ontology fixed in advance, and recompute accuracy with strict matches only, omissions inside the denominator, and partial matches treated as errors; if accuracy drops well below 95 percent, the extraction-fidelity foundation of the transparency claim is not established. For the user-study claim, a second check would run the identical staged items with provenance stripped and with accept/reject/add disabled, and test whether reported transparency and intent alignment fall back to one-shot levels.

Watch

Extended reading notes

Core claim

The paper's discovery claim is that prompt-to-interface generation can be treated as an explicit, evolving reasoning process rather than an end-to-end transformation, and that doing so improves how beginner users understand and steer generation. LegoUI externalizes each intermediate design commitment as a UI-DSL item — an atomic, provenance-tagged statement in one of seven specifications (intent, global, component pool, layout, interaction, relation, style) — and presents these items as negotiable commitments before they are rendered into an interface. Users accept, reject, or add items, and every subsequent stage consumes the resulting specification state, so rejected inferences drop out of later generation, added requirements become explicit assumptions, and previously accepted decisions persist across regenerations. The paper reports that this shifts design work from post-hoc correction of a finished artifact to earlier intervention in the assumptions behind it, and it grounds the claim in a technical evaluation (explicit requirements extracted at over 95% accuracy with zero redundancy) and a within-subject study of 15 participants who rated LegoUI higher than Claude, Bolt, Vercel v0, and Lovable on transparency, controllability, usability, and intent alignment. The effect is uneven: component and layout decisions were easy for participants to judge, while interaction and relation decisions, whose consequences spread across components and pages, were harder to evaluate and added inspection cost.

Load-bearing premise

The reported 95% extraction accuracy rests on a reference standard produced by annotators who used LegoUI's own requirement-analysis tooling to build the PromptChecklists, with partial matches counted as correct and omissions excluded from the accuracy denominator, so if the system's DSL categories shaped what the annotators counted as explicit requirements, part of the accuracy measures self-consistency rather than faithful extraction.

Editorial extensions

If this is right

  • Users can inspect how the system interpreted their prompt before that interpretation hardens into a rendered interface, and can reject unwanted inferences before they propagate into later stages.
  • Regenerated versions build on the accumulated specification state — accepted, rejected, and user-added items — instead of reinterpreting the original prompt from scratch, so stable commitments persist across iterations.
  • Control becomes stage-appropriate: component and layout decisions invite confident user judgment, while interaction and relation decisions need extra visual or behavioral context to be judged well.
  • The workflow cost shifts earlier: staged inspection adds review work, while one-shot tools keep the advantage of speed and first-pass polish; the two paradigms fit different phases, exploration versus refinement.
  • Future systems can preserve the staged structure while adding selective, consequence-oriented transparency, linking each decision to the UI regions and behaviors it affects rather than showing every item with equal weight.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The inspect-accept-reject-add loop is a general template beyond UI: any generative tool where users must judge model-made commitments (document drafting, slide decks, data-analysis reports) faces the same black-box tension, and a provenance-tagged intermediate specification could let users separate their own input from model inference there as well.
  • A control condition the paper does not run — presenting the same staged items without provenance labels, or without accept/reject/add controls — would isolate whether the transparency benefit comes from the UI-DSL mechanism itself or from simply slowing generation into reviewable stages.
  • The paper's finding that interaction and relation stages are the hardest to judge suggests a concrete design target: linking each item to the UI regions and behaviors it affects should cut inspection cost where it is highest; measuring whether that trade-off pays off is a direct follow-up study.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents LegoUI, a staged generative UI design framework that externalizes intermediate design decisions into a UI-specific DSL with provenance, allowing users to inspect, accept, reject, or add items across generation stages. It reports a technical evaluation of the requirement-analysis stage on 40 prompts, claiming over 95% accuracy, near-complete coverage, and zero redundancy, and a within-subject user study with 15 participants comparing LegoUI to one-shot tools on perceived transparency, controllability, intent alignment, and workload. The central claim is that staged, inspectable generation improves transparency and controllability compared to one-shot GenUI systems.

Significance. If the central claims hold, the paper makes a useful contribution to GenUI research: it offers a concrete mechanism for exposing intermediate reasoning in UI generation, with a persistent, provenance-aware DSL that supports iteration and cross-stage reuse. The system design is clearly described, the DSL grammar in Appendix D is a genuine design artifact, and the user study combines Likert questionnaires, preference rankings, and semi-structured interviews, giving qualitative insight into how participants used staged control. The paper also discusses trade-offs honestly, including the interaction cost of inspectability in Section 9.5. However, the quantitative foundation for the transparency claim is not yet independent: the technical evaluation's reference standard is partly generated by the system under test, the metrics are lenient, and the abstract overstates the section-level results. The user study findings are suggestive, but the statistical reporting needs strengthening before the significance claims can be fully assessed.

major comments (5)
  1. [Section 7.1.2] The reference PromptChecklist is constructed by annotators who used the system to perform the requirement-analysis step and organized it according to the system's top-level DSL structure. Because the same UI-DSL categories define both the extraction target and the annotation scheme, the reported agreement partly reflects self-consistency rather than independent extraction fidelity. I recommend constructing the ground truth from the raw standardized prompts without exposing annotators to system output, and reporting inter-annotator agreement against that independent standard.
  2. [Section 7.1.3] The Accuracy metric is (MT+PM)/(MT+PM+ML+EX), so Omitted items are excluded from the denominator and Partial Matches count as correct. As a result, the reported 95% accuracy can coexist with many omissions, and exact-match performance is not separately reported. Please report precision and recall with omissions included in the appropriate denominator, and give exact-match rates separately from partial-match rates.
  3. [Abstract and Section 7.2] The claim of over 95% accuracy is not supported by Table 4, where globalSpec has 92.77% accuracy and styleSpec has 89.29%. The abstract and summary should be revised to present section-level results and should not state an aggregate figure that exceeds the observed section-level values.
  4. [Section 7.1.1 and Tables 3-4] layoutSpec (N=23) and relationSpec (N=14) rest on very small samples, and relation items were injected by adding four GPT-4o-generated prompts after the 36 collected prompts contained no relationSpec. The perfect scores for these sections should be de-emphasized, and the dataset should be described as an artificially balanced corpus rather than 40 real-world design prompts.
  5. [Sections 8.2-8.4] Tasks 2 and 3 compare systems within subjects without explicit order counterbalancing, and multiple pairwise t-tests are reported without correction for multiple comparisons. With N=15, this creates nontrivial Type I error risk; please report confidence intervals, correct for multiple comparisons or justify the uncorrected approach, and state how system order was handled in the within-subject design.
minor comments (6)
  1. [Section 7.1.1] The phrase after verifying that the 36 ground-truth prompts contained no relationSpec is unclear; please clarify that relationSpec was absent and explain the implications of adding relation-heavy prompts for the naturalness of the corpus.
  2. [Table 6] Several cells contain concatenated values, such as 4.474.07 and 4.534.40; these formatting errors should be fixed.
  3. [Appendix D] Appendix D refers to Section 5.4 for provenance handling, but the relevant subsection is Section 5.3; the cross-reference should be corrected.
  4. [Data Availability] The technical evaluation claims would be easier to verify if the coding protocol, annotation instructions, and de-identified extracted checklists were released publicly rather than available only upon request.
  5. [Section 7.1.1] The prompt-standardization criteria and the number and content of the four supplementary prompts are design choices that are not varied or justified; a brief discussion of their influence on the results would improve the evaluation.
  6. [Section 8.4.1] The statement that Claude Chat was reliably above the baseline one-shot tools uses reliable for differences that may not all be statistically significant; please align this wording with the reported significance tests.

Circularity Check

2 steps flagged · score 5.0 of 10

Technical evaluation's >95% requirement accuracy is partly self-consistency: GT is scaffolded by LegoUI's own DSL and requirement-analysis stage, and the accuracy metric omits omissions; the user-study comparison remains independent.

  1. self definitional [Section 7.1.2 (Ground Truth Construction)]
    "For each standardized prompt, evaluators used our system to perform the requirement analysis step, which generates a PromptChecklist organized according to the system’s top-level DSL structure: intentSpec, globalSpec, componentPool, layoutSpec, interactionSpec, relationSpec, styleSpec ... At this stage, we extract only explicitly stated information, including elements shown directly or mentioned in the prompt text."

    The paper's stated evaluation goal is to determine 'whether the system can faithfully capture key requirements'. The reference PromptChecklist used to measure capture is itself produced through the same requirement-analysis stage being tested, organized into the system's own top-level DSL sections. Comparing LegoUI's output against a reference that was generated through LegoUI's requirement-analysis step measures internal consistency rather than faithful extraction from arbitrary user prompts. Because the DSL categories also define what annotators enumerate as explicit requirements, the ground truth is scaffolded by the system's output structure; the claimed high accuracy is therefore partly a check of the system against a reference shaped by the system.

  2. self definitional [Section 7.1.3 (Scoring Procedure)]
    "Accuracy = (MT + PM) / (MT + PM + ML + EX); Coverage = (MT + PM) / (MT + PM + OM); Redundancy Rate = EX / (MT + PM + ML + EX). Partial Matches (PM) are included in accuracy since they capture the correct semantic unit even if attributes are missing."

    The metric computation makes 'accuracy' insensitive to missing requirements: omitted GT items (OM) are excluded from the accuracy denominator, while Partial Matches (PM) are counted as correct. As defined, a system that fails to extract an explicit requirement, or drops attributes and details, does not lose accuracy. Thus the abstract's 'captured explicit requirements with over 95% accuracy' is guaranteed by the formula to remain high even when extraction is incomplete; Coverage is the only omission-sensitive metric and it is computed against the system-scaffolded GT. The high accuracy number is therefore partly an artifact of the scoring definition rather than an independent measure of faithful capture.

full rationale

Score 5: the circularity is real but confined to the technical-evaluation pillar. The user study (Section 8) is independent: outputs were anonymized and randomized, ratings and interviews came from 15 participants, and the comparison to Bolt, Claude, Lovable, and Vercel v0 does not rely on the DSL categories or on the system's own ground truth. No load-bearing self-citation or imported uniqueness theorem appears; the design goals cite the external GenUI Study [10], and the references to the authors' own PrototypeFlow [42] are contextual rather than premises. The central user-experience finding therefore has independent content. However, the quantitative support for the requirement-analysis mechanism is substantially self-referential: the GT is built under the system's own DSL/requirement-analysis stage and the accuracy metric omits omissions, so the >95% accuracy claim reduces partly to self-consistency. This warrants a mid-range score rather than 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on a self-authored DSL, a lenient evaluation metric, and a small self-reported user study. No parameters were fitted to data in the numerical sense, but the evaluation design contains several hand-chosen thresholds and category definitions that materially affect the reported numbers.

free parameters (3)
  • Prompt standardization criteria = 150-200 words, 6-10 components, 2-4 interactions, 1-3 style constraints
    These bounds were chosen by the authors to make prompts comparable; they shape the difficulty of the extraction task and are not derived from any external standard.
  • Number and content of 4 supplementary prompts = 4 GPT-4o-generated prompts added after discovering the 36 prompts contained no relationSpec
    Added to balance the corpus; directly determines the relationSpec results (N=14 items), including the perfect scores reported.
  • UI-DSL category set (7 dimensions) = intent, global, component, layout, interaction, relation, style
    The specification's seven dimensions are authored for this paper and define what counts as a correctly extracted requirement; changing this set changes accuracy.
assumptions (4)
  • domain assumption LLMs with zero-shot prompting and fragmentary grammar examples can reliably decompose UI prompts into DSL items.
    The technical evaluation tests this assumption rather than justifying it; the entire 95% accuracy claim depends on it.
  • domain assumption Ground-truth checklists produced by annotators who used the LegoUI system are unbiased references for explicit requirements.
    Section 7.1.2: evaluators used the system to generate PromptChecklists then annotated; this risks aligning the reference with the system's output categories.
  • domain assumption Self-reported Likert ratings of transparency, controllability and alignment measure the constructs the paper claims.
    Section 8 uses questionnaires and interviews; no behavioral metrics for actual control or design quality are reported.
  • domain assumption Results from 15 mostly beginner participants generalize to the intended beginner-user population.
    Section 8.1: small, self-selected sample, no power analysis, task order fixed across participants.
invented entities (1)
  • UI-DSL (IntentSpec, GlobalSpec, ComponentPool, LayoutSpec, InteractionSpec, RelationSpec, StyleSpec)
    purpose: Externalizes intermediate design decisions as editable, provenance-carrying items for user inspection and intervention.
    The DSL is defined in Appendix D and evaluated only within this paper; no external test shows whether this particular representation is the right one.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LEGOUI: Designing with UI-DSL Bricks to Balance Transparency and Controllability." pith.science (2026). https://pith.science/paper/4CJAI3CB

@misc{pith2026260804293,
  author       = {Pith},
  title        = {Pith review of: LEGOUI: Designing with UI-DSL Bricks to Balance Transparency and Controllability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CJAI3CB}},
  note         = {Machine review of arXiv:2608.04293}
}
read the original abstract

Generative user interface design tools enable rapid prototyping but often operate as black boxes with limited transparency and controllability. When outputs diverge from the designer's intent, users are left tweaking prompts via trial-and-error with little insight into the model's reasoning. We present LegoUI, a staged generative framework that structures the interface design process into sequential, interpretable steps along key design dimensions, capturing each step's result in a UI domain-specific language (UI-DSL) enriched with provenance. This approach exposes the model's intermediate reasoning and enables user intervention and iterative refinement. In a technical evaluation on 40 real-world design prompts, LegoUI's requirement analysis stage captured explicit requirements with over 95% accuracy, near-complete coverage, and zero redundancy. In user studies, participants using LegoUI reported significantly greater transparency, controllability, and alignment with their intent compared to existing one-shot generative UI tools.

Figures

Figures reproduced from arXiv: 2608.04293 by the authors.

Figure 1
Figure 1. The LegoUI system for staged UI generation. LegoUI starts from a user prompt A and incrementally constructs a UI specification through successive reasoning stages. Inferred design decisions are written into a persistent UI-DSL and surfaced as editable tasks B , organized by design dimensions C . Users intervene on these decisions through explicit actions D , which update the active specification state that guides su… view at source ↗
Figure 2
Figure 2. Overview of our pipeline for UI generation, illustrated through a factory-style production line metaphor. From a user input, the system performs [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Average ratings of anonymized UIs across five evaluation dimensions (Accessibility, Intent Alignment, Trustworthiness, Usability, and Visual [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparative results of UI evaluations across five dimensions. (a) shows the overall average score matrix by dimension, where each cell reports the [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: User preference results across prompts. (a–c) show best-choice and second-choice votes for Dashboard, Productivity, and Portfolio prompts respectively. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Mean scores with standard deviations for all questionnaire items, grouped by dimension (Fairness & Process Transparency, Usability & Cognitive [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Mean ratings (1–5 scale) of post-generation editing experience across five UX dimensions: control, explanation and transparency, intent alignment, [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: P4’s staged workflow across requirement analysis, layout, interaction, relation, and style stages. The figure shows both AI inferences (green) and user [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: P4’s UI generation and subsequent refinement. After the first UI draft was generated, the participant added new component and relation specifications [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: Distribution of participant responses to the 5-point Likert scale questionnaire evaluating the interaction task in LegoUI. [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Full distribution of Likert responses for post-generation editing questionnaire items across all five dimensions and three systems. [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 32 canonical work pages

  1. [1]

    Anthropic. 2024. Artifacts are now generally available. Today, we’re making Artifacts available for all Claude.ai users across our Free, Pro, and Team plans. And now, you can create and view Artifacts on our iOS and Android apps. (Aug. 2024). https://www.anthropic.com/news/artifacts

    work page 2024

  2. [2]

    Jazbo Beason, Ruijia Cheng, Eldon Schoop, and Jeffrey Nichols. 2025. Athena: intermediate representations for iterative scaffolded app generation with an llm.arXiv preprint arXiv:2508.20263

    arXiv 2025

  3. [3]

    Tony Beltramelli. 2018. Pix2code: generating code from a graphical user interface screenshot. InProceedings of the ACM SIGCHI symposium on engineering interactive computing systems, 1–6

    work page 2018

  4. [4]

    Carlos Bernal-Cárdenas, Kevin Moran, Michele Tufano, Zichang Liu, Linyong Nan, Zhehan Shi, and Denys Poshyvanyk. 2019. Guigle: a gui search engine for android apps. In2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 71–74. doi:10.1109/ICSE-Companion.2019.00041

    arXiv 2019

  5. [5]

    Bolt.new. 2025. Bolt.new: ai-powered builder for websites, web apps, and mobile apps. An AI-powered tool that transforms natural language prompts into websites, web apps, or mobile apps in minutes. (2025). https://bolt.new/

    work page 2025

  6. [6]

    Sara Bunian, Kai Li, Chaima Jemmali, Casper Harteveld, Yun Fu, and Magy Seif Seif El-Nasr. 2021. Vins: visual search for mobile user interface design. In (CHI ’21) Article

    work page 2021

  7. [7]

    Yining Cao, Peiling Jiang, and Haijun Xia. 2025. Generative and malleable user interfaces with generative and evolving task-driven data model. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 1–20

    work page 2025

  8. [8]

    Chunyang Chen, Sidong Feng, Zhenchang Xing, Linda Liu, Shengdong Zhao, and Jinshui Wang. 2019. Gallery d.c.: design search and knowledge discovery through auto-created gui component gallery.Proc. ACM Hum.-Comput. Interact., 3, CSCW, Article 180, (Nov. 2019), 22 pages. doi:10.1145/3359282

    work page doi:10.1145/3359282 2019

Show all 46 references
  1. [9]

    Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xin Xia, Liming Zhu, John Grundy, and Jinshui Wang. 2020. Wireframe-based ui design search through image autoencoder. ACM Trans. Softw. Eng. Methodol., 29, 3, Article 19, (June 2020), 31 pages. doi:10.1145/3391613. LEGOUI: Designing...

    2020 doi

  2. [10]

    Xiang’Anthony Chen, Tiffany Knearem, and Yang Li. 2025. The genui study: exploring the design of generative ui tools to support ux practitioners and beyond. InProceedings of the 2025 ACM Designing Interactive Systems Conference, 1179–1196

    2025

  3. [11]

    Yunnong Chen, Chengwei Shi, and Liuqing Chen. 2025. Specifyui: supporting iterative ui design intent expression through structured specifications and generative ai.arXiv preprint arXiv:2509.07334

    2025 arXiv

  4. [12]

    Chin-Yi Cheng, Forrest Huang, Gang Li, and Yang Li. 2023. Play: parametrically conditioned layout generation using latent diffusion.arXiv preprint arXiv:2301.11529

    2023 arXiv

  5. [13]

    Hai Dang, Lukas Mecke, Florian Lehmann, Sven Goller, and Daniel Buschek. 2022. How to prompt? opportunities and challenges of zero-and few-shot learning for human-ai interaction in creative applications of generative models.arXiv preprint arXiv:2209.01390

    2022 arXiv

  6. [14]

    Peitong Duan, Jeremy Warner, Yang Li, and Bjoern Hartmann. 2024. Generating automatic feedback on ui mockups with large language models. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems, 1–20

    2024

  7. [15]

    Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models.Transactions of the Association for Computational Linguistics, 9, 1012–1031

    2021

  8. [16]

    Yi Gui et al. 2025. Uicopilot: automating ui synthesis via hierarchical code generation from webpage designs. InProceedings of the ACM on Web Conference 2025, 1846–1855

    2025

  9. [17]

    Canny, and Jeffrey Nichols

    Forrest Huang, John F. Canny, and Jeffrey Nichols. 2019. Swire: sketch-based user interface retrieval. In (CHI ’19). Association for Computing Machinery, Glasgow, Scotland Uk, 1–10.isbn: 9781450359702. doi:10.1145/3290605.3300334

    2019

  10. [18]

    Tae Soo Kim, DaEun Choi, Yoonseo Choi, and Juho Kim. 2022. Stylette: styling the web with natural language. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–17

    2022

  11. [19]

    Kristian Kolthoff, Felix Kretzer, Christian Bartelt, Alexander Maedche, and Simone Paolo Ponzetto. 2025. Guide: llm-driven gui generation decomposition for automated prototyping. In2025 IEEE/ACM 47th International Conference on Software Engineering: Companion Proceedings (ICSE...

    2025 doi

  12. [20]

    Kristian Kolthoff, Felix Kretzer, Lennart Fiebig, Christian Bartelt, Alexander Maedche, and Simone Paolo Ponzetto. 2024. Zero-shot prompting approaches for llm-based graphical user interface generation.arXiv preprint arXiv:2412.11328

    2024 arXiv

  13. [21]

    Yoonjoo Lee, Kihoon Son, Tae Soo Kim, Jisu Kim, John Joon Young Chung, Eytan Adar, and Juho Kim. 2024. One vs. many: comprehending accurate information from multiple erroneous and inconsistent ai generations. InProceedings of the 2024 ACM Conference on Fairness, Accountability...

    2024

  14. [22]

    Alan Leung, Ruijia Cheng, Jason Wu, Jeffrey Nichols, and Titus Barik. 2025. Squire: interactive ui authoring via slot query intermediate representations. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, 1–17

    2025

  15. [23]

    Jie Li, Hancheng Cao, Laura Lin, Youyang Hou, Ruihao Zhu, and Abdallah El Ali. 2024. User experience design professionals’ perceptions of generative artificial intelligence. InProceedings of the 2024 CHI conference on human factors in computing systems, 1–18

    2024

  16. [24]

    Toby Jia-Jun Li, Amos Azaria, and Brad A Myers. 2017. Sugilite: creating multimodal smartphone automation by demonstration. InProceedings of the 2017 CHI conference on human factors in computing systems, 6038–6049

    2017

  17. [25]

    Toby Jia-Jun Li, Jingya Chen, Haijun Xia, Tom M Mitchell, and Brad A Myers. 2020. Multi-modal repairs of conversational breakdowns in task-oriented dialogs. InProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, 1094–1107

    2020

  18. [26]

    Toby Jia-Jun Li, Lindsay Popowski, Tom Mitchell, and Brad A Myers. 2021. Screen2vec: semantic embedding of gui screens and gui components. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–15

    2021

  19. [27]

    Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, and Zhiwei Guan. 2020. Widget captioning: generating natural language description for mobile user interface elements. arXiv preprint arXiv:2010.04295

    2020 arXiv

  20. [28]

    Lovable. 2025. Lovable: ai-powered platform for building full-stack websites via natural language. An AI-powered platform that enables users to create full-stack websites through natural language prompts. (2025). https://lovable.dev/

    2025

  21. [29]

    Yuwen Lu, Alan Leung, Amanda Swearngin, Jeffrey Nichols, and Titus Barik. 2025. Misty: ui prototyping through interactive conceptual blending. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 1–17

    2025

  22. [30]

    Yuwen Lu, Chengzhi Zhang, Iris Zhang, and Toby Jia-Jun Li. 2022. Bridging the gap between ux practitioners’ work practices and ai-enabled design support tools. InCHI Conference on Human Factors in Computing Systems Extended Abstracts, 1–7

    2022

  23. [31]

    Arpit Narechania, Arjun Srinivasan, and John Stasko. 2021. Nl4dv: a toolkit for generating analytic specifications for data visualization from natural language queries.IEEE Transactions on Visualization and Computer Graphics, 27, 2, 369–379. doi:10.1109/TVCG.2020.3030378

    2021

  24. [32]

    Savvas Petridis, Michael Terry, and Carrie Jun Cai. 2023. Promptinfuser: bringing user interface mock-ups to life with large language models. InExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, 1–6

    2023

  25. [33]

    Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, and Jeffrey Heer. 2017. Vega-lite: a grammar of interactive graphics.IEEE Transactions on Visualization and Computer Graphics, 23, 1, 341–350. doi:10.1109/TVCG.2016.2599030

    2017

  26. [34]

    Siegfried Schreiber. 1994. Specification and generation of user interfaces with the boss-system. InInternational Conference on Human-Computer Interaction. Springer, 107–120

    1994

  27. [35]

    Atefeh Shokrizadeh, Boniface Bahati Tadjuidje, Shivam Kumar, Sohan Kamble, and Jinghui Cheng. 2025. Dancing with chains: ideating under constraints with uidec in ui/ux design. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 1–23

    2025

  28. [36]

    Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2code: benchmarking multimodal code generation for automated front-end engineering.arXiv preprint arXiv:2403.03163

    2024 arXiv

  29. [37]

    Priyan Vaithilingam, Elena L Glassman, Jeevana Priya Inala, and Chenglong Wang. 2024. Dynavis: dynamically synthesized ui widgets for visualization editing. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems, 1–17

    2024

  30. [38]

    Priyan Vaithilingam and Philip J Guo. 2019. Bespoke: interactively synthesizing custom guis from command-line applications by demonstration. InProceedings of the 32nd annual ACM symposium on user interface software and technology, 563–576

    2019

  31. [39]

    Vercel / v0. 2025. V0.app: collaborative ai assistant to design, iterate, and scale full-stack applications. A collaborative AI assistant enabling prompt-driven design, iteration, and scaling of full-stack web applications. (2025). https://v0.app/

    2025

  32. [40]

    Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R Lyu. 2024. Automatically generating ui code from screenshot: a divide-and-conquer-based approach.arXiv preprint arXiv:2406.16386

    2024 arXiv

  33. [41]

    Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey P Bigham, and Jeffrey Nichols. 2024. Uicoder: finetuning large language models to generate user interface code through automated feedback.arXiv preprint arXiv:2406.07739

    2024 arXiv

  34. [42]

    Mingyue Yuan, Jieshan Chen, Yongquan Hu, Sidong Feng, Mulong Xie, Gelareh Mohammadi, Zhenchang Xing, and Aaron Quigley. 2024. Towards human-ai synergy in ui design: enhancing multi-agent based ui generation with intent clarification and alignment.arXiv preprint arXiv:2412.20071

    2024

  35. [43]

    Sukmin Yun et al. 2024. Web2code: a large-scale webpage-to-code dataset and evaluation framework for multimodal llms.Advances in neural information processing systems, 37, 112134–112157

    2024

  36. [44]

    J Diego Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts. InProceedings of the 2023 CHI conference on human factors in computing systems, 1–21

    2023

  37. [45]

    distinct article pages with header and article content

    Tianming Zhao, Chunyang Chen, Yuanning Liu, and Xiaodong Zhu. 2021. Guigan: learning to generate gui designs using generative adversarial networks. In2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 748–760. 22 Zhou et al. A Illustrative Exampl...

    2021

  38. [423]

    doi:10.1145/3411764.3445762

    Association for Computing Machinery, Yokohama, Japan, 14 pages.isbn: 9781450380966. doi:10.1145/3411764.3445762

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.