REVIEW 9 cited by
Bridging Design and Development with Automated Declarative UI Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Declarative UI frameworks have gained widespread adoption in mobile app development, offering benefits such as improved code readability and easier maintenance. Despite these advantages, the process of translating UI designs into functional code remains challenging and time-consuming. Recent advancements in multimodal large language models (MLLMs) have shown promise in directly generating mobile app code from user interface (UI) designs. However, the direct application of MLLMs to this task is limited by challenges in accurately recognizing UI components and comprehensively capturing interaction logic. To address these challenges, we propose DeclarUI, an automated approach that synergizes computer vision (CV), MLLMs, and iterative compiler-driven optimization to generate and refine declarative UI code from designs. DeclarUI enhances visual fidelity, functional completeness, and code quality through precise component segmentation, Page Transition Graphs (PTGs) for modeling complex inter-page relationships, and iterative optimization. In our evaluation, DeclarUI outperforms baselines on React Native, a widely adopted declarative UI framework, achieving a 96.8% PTG coverage rate and a 98% compilation success rate. Notably, DeclarUI demonstrates significant improvements over state-of-the-art MLLMs, with a 123% increase in PTG coverage rate, up to 55% enhancement in visual similarity scores, and a 29% boost in compilation success rate. We further demonstrate DeclarUI's generalizability through successful applications to Flutter and ArkUI frameworks.
Forward citations
Cited by 9 Pith papers
-
Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation
On a new 1,440-screenshot fill-in-the-blank benchmark, five frontier multimodal models default to repeated UI patterns instead of visual deviations, with mean bias rates from 69.8% on card widths to 80.2% on font sizes.
-
DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models
A hierarchy-aware, self-correcting LLM pipeline for generating React Native UI code improves visual fidelity and structural similarity over baselines on 300 mockups.
-
MLLM-Based UI2Code Automation Guided by UI Layout Information
LayoutCoder improves screenshot-to-code generation on real websites by parsing the layout into a tree, generating code per region, and fusing it deterministically, beating the best baseline by 10.14 BLEU and 3.95 CLIP...
-
P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark
P2P is a multi-agent framework that automatically generates HTML-rendered academic posters from papers, backed by a 30k instruction dataset and a 121-pair evaluation benchmark.
-
MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs
A resource-list representation and a 500-site benchmark let multimodal LLMs generate web code with real links, images, and routes, lifting resource matching from ~0% to 66-80%.
-
SAM2-UNeXT: An Improved High-Resolution Baseline for Adapting Foundation Models to Downstream Segmentation Tasks
The full text describes LaTCoder, a block-wise chain-of-thought method for webpage design-to-code, reporting improved layout preservation on two benchmarks, while the title and abstract are for a different paper.
-
LaTCoder: Converting Webpage Design to Code with Layout-as-Thought
LaTCoder cuts a webpage screenshot into blocks, generates code for each block with chain-of-thought prompting, and assembles the blocks, improving layout fidelity over direct whole-page generation.
-
Multimodal graph representation learning for website generation based on visual sketch
Adding a graph built from OCR text boxes and segmented visual blocks to a Flamingo-style vision-language model improves HTML generation on the WebSight benchmark, but gains do not generalize to the Design2Code benchmark.
-
Synergizing LLMs and Knowledge Graphs: A Novel Approach to Software Repository-Related Question Answering
An LLM plus a repository knowledge graph answers software repository questions with 84% accuracy when few-shot chain-of-thought prompting is added, outperforming an intent-based bot and web-search GPT-4o.
Discussion (0). Continue with ORCID to comment.