{"id":"a38aea19-4f88-4d4b-afe2-73882ac98cc5","arxiv_id":"2506.13663","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A hierarchy-aware, self-correcting LLM pipeline for generating React Native UI code improves visual fidelity and structural similarity over baselines on 300 mockups.","lead":"DesignCoder turns UI design mockups into React Native code by first using a multimodal language model to group interface elements into a hierarchy, then generating code for each group and finally repairing the rendered result through visual comparison. A generalist might read it because automating front-end development remains a high-value, unsolved industrial task, and this pipeline is a concrete attempt with evaluation on real mockups.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation may inflate DesignCoder's gains by penalizing non-compiling baseline code without reporting compile rates; headline improvements could reflect compile-repair rather than fidelity.","rationale":"DesignCoder is a clearly described pipeline, and the ablation study and user study provide supporting evidence. However, the evaluation protocol has a load-bearing gap: the treatment of non-compiling code is not transparent. The paper explicitly assigns a zero/worst-case score to failed compilations (Section 3.2.3) but does not report how often each method fails to compile. Since DesignCoder's self-correction loop is designed to fix compile errors (Section 2.3), the comparison may be uneven; baselines without such a repair step are penalized on every failed case. Recomputing metrics only on mutually successful cases would reveal whether the headline improvements are due to genuine fidelity or to penalization. This is a specific, testable threat to the central claim, and it is independent of the reader's CodeFun ground-truth concern, though both point to evaluation validity. The reader's conditional verdict remains apt; no verdict change is needed, but the acceptance conditions should include release of per-case compile outcomes and recomputation on the compiling subset.","tokens_in":14768,"tokens_out":6947,"duration_ms":72979,"concrete_test":"Recompute Tables 1 and 2 on the subset of the 300 mockups for which all methods produce compiling code; also report compile success rates per method and the exact score assigned to non-compiling code for each metric. If DesignCoder's improvements (especially MSE and TreeBLEU) shrink by more than, say, half on this subset, the headline result is an artifact of compile-failure penalization rather than fidelity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2.3 states: 'For code that failed to compile, we assigned a value of zero to all its evaluation metrics.' The paper never reports the per-method compile success rate. Because DesignCoder's self-correction step (Section 2.3, Figure 2 'Rendering Error Message / Bug Fix') is explicitly designed to repair compile errors, its compile success rate should be near 100%. If any baseline (e.g., GPT-4o, Claude-3.5, Prototype2Code) has a non-negligible fraction of non-compiling outputs, those cases receive worst-case scores and drag down the baseline averages, inflating the reported relative improvements (37.63% MSE, 9.52% CLIP, 12.82% SSIM; 30.19% TreeBLEU, etc.). The ambiguity of 'zero' for MSE (where lower is better) compounds the problem: a zero MSE would be a perfect score, not a penalty, so the exact handling is unclear. Without per-case compile outcomes and metrics computed on the subset of mutually compiling outputs, the central claim that DesignCoder outperforms baselines in visual and structural fidelity is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DesignCoder, a three-stage pipeline for generating React Native code from UI mockups: a UI Grouping Chain that divides the UI, extracts element semantics, and builds a component tree; a hierarchical divide-and-conquer code and style generation step that consumes design metadata; and an iterative visual self-correction step that compares rendered screenshots against the original mockup. The method is evaluated on 300 mockups (250 from the Figma community and 50 from an unnamed company) against Prototype2Code, DeclarUI, GPT-4o, Claude-3.5, LLaVA, and CodeFun, using visual metrics (MSE, CLIP, SSIM) and structural metrics (TreeBLEU, Container Match, Tree Edit Distance). The paper reports large improvements over the best baseline, e.g., a 37.6% reduction in MSE and a 30.2% increase in TreeBLEU on the Figma dataset, and a user study with five professional developers. The central claim is that DesignCoder produces both more visually faithful and more structurally maintainable UI code than existing methods.","tokens_in":15014,"tokens_out":4772,"duration_ms":53254,"significance":"If the reported results hold, DesignCoder would be a practically valuable contribution to industrial UI code generation, combining hierarchy reasoning, metadata use, and automated visual repair. The paper has clear strengths: a new 300-mockup dataset from two sources, a systematic ablation isolating the grouping chain and the refinement module, a user study with experienced developers, and a level of implementation detail that would support reproduction. However, the headline numbers currently rest on two unresolved evaluation-validity issues: the treatment of non-compiling baseline outputs is under-specified and could inflate all reported improvements, and the structural ground truth is derived from CodeFun, which is itself a reported baseline. The main comparison tables also lack variance information and significance tests despite the use of the word 'significantly.' These issues are fixable but require additional experiments and reporting; they are not mere presentation concerns.","major_comments":[{"comment":"The statement 'For code that failed to compile, we assigned a value of zero to all its evaluation metrics' is ambiguous and potentially consequential. For MSE, zero is the best possible score, so assigning zero to non-compiling code would actually reward failure; if instead a worst-case value was intended, that is not what the text says. More importantly, the paper never reports the per-method compile success rate. Because DesignCoder's self-correction loop (Section 2.3) is explicitly designed to fix compile errors, its compile rate is likely near 100%, while generic baselines such as GPT-4o or LLaVA may fail to compile at non-negligible rates. Assigning worst-case scores to all non-compiling baseline outputs would inflate the reported relative improvements in both visual and structural metrics. Please report compile rates for every method, clarify exactly how non-compiling outputs were scored for each metric, and recompute the headline comparisons on the subset of outputs that compile (or use an explicit, transparently specified penalty with sensitivity analysis).","section":"Section 3.2.3, Tables 1 and 2"},{"comment":"The structural ground truth is constructed by running CodeFun on the design files and then having two authors manually correct the resulting component trees, while CodeFun is simultaneously listed as a baseline in Tables 1 and 2. Since DesignCoder also consumes design metadata and generates code from a component tree, the structural metrics (TreeBLEU, Container Match, Tree Edit Distance) may be biased in DesignCoder's favor if the reference trees inherit CodeFun's grouping conventions. The manual correction step is not documented in enough detail to rule this out: there is no inter-annotator agreement, no description of how many corrections were made, and no independent validation of the reference trees. Please (i) describe the correction protocol and report inter-annotator agreement, (ii) quantify how much the corrected trees differ from CodeFun's raw output, and (iii) report structural metrics against an independently constructed reference, for example trees written by front-end developers not involved in the method, or use multiple references.","section":"Section 3.2.2, Tables 1 and 2"},{"comment":"No variance, confidence interval, or significance test is reported for any metric in Tables 1 and 2, yet the abstract and Section 3.3 use the term 'significantly.' With 300 mockups (and 50 in the company dataset), it is feasible to compute per-sample standard errors and paired tests, e.g., paired Wilcoxon or bootstrap confidence intervals for the difference between DesignCoder and each baseline. Without such statistics, the reader cannot determine whether the reported advantages are stable or driven by a small number of outlier cases. This is particularly important for the structural metrics, where the arbitrary penalties for trees with no matching nodes (Section 3.2.2) can interact with per-case variability.","section":"Section 3.3, Tables 1 and 2"},{"comment":"The user study involves five participants and 15 prototypes, and the reported Mann-Whitney U p-values appear to treat individual ratings as independent observations. With only five raters and no random-effects model or per-participant aggregation, these p-values almost certainly overstate the significance of the differences. Please report the unit of analysis, per-participant averages, and an appropriate paired test (e.g., Wilcoxon signed-rank on aggregated per-participant scores). Also state whether the participants were blinded to which method produced each code sample, since the evaluation involves subjective ratings of readability and maintainability.","section":"Section 3.5.1, Table 4"}],"minor_comments":[{"comment":"TreeBLEU is described as the proportion of height-1 subtrees in the given tree that match the reference tree, but the equation only has the matched count in the numerator and the reference subtree count in the denominator. Please clarify whether this is precision, recall, or a combined score, and describe how the mobile DOM tree is extracted from React Native code for this comparison.","section":"Section 3.2.2, Eq. (8)"},{"comment":"The number of self-correction iterations is denoted 'N' in Figure 2 but its value is never reported. Please state the value used in the experiments and include a sensitivity analysis over N, since the cost of the method scales with this number.","section":"Section 2.3 and Figure 2"},{"comment":"There are minor typographical and formatting issues: 'LLaV A-v1.5-7B' in Section 3.2.1 has an unintended space, 'Tree Edit Distace' appears in the Table 3 title, and the reference to CodeFun is inconsistently capitalized as 'Codefun' and 'CodeFun.' The ACM reference header also uses a 2018 placeholder that should be updated.","section":"Throughout"},{"comment":"The threats-to-validity section acknowledges that 'complete visual similarity does not necessarily indicate high-quality generated code' and that 'manually annotated code structures may not fully reflect best practices.' These caveats should be operationalized by making the corrected reference trees and the evaluation scripts publicly available so readers can inspect the ground truth and reproduce the structural metrics.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a software engineering venue and the core idea is promising, but the two load-bearing evaluation issues—the under-specified handling of non-compiling baseline outputs and the use of CodeFun-derived reference trees with CodeFun also as a baseline—must be resolved before the quantitative claims can be trusted. The lack of variance reporting in the main tables compounds this. I would encourage the editor to request artifact release (dataset, corrected reference trees, and evaluation code) as part of the revision, since the dataset is described as newly collected and is central to the paper's contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the short version: DesignCoder has a real new idea—the UI Grouping Chain, which splits UI division, semantic extraction, and grouping into MLLM prompts with post-processing—and the paper shows a coherent engineering story. But the evaluation as written does not establish the headline \"significantly improves\" claims. Two problems stand out.\n\nFirst, Section 3.2.3 says failed-to-compile outputs get zero on all metrics. That's undefined for MSE (zero is a perfect score), and the paper never reports compile rates per method. Since DesignCoder's self-correction loop is designed to fix compile errors, baselines without that loop could be carrying a non-compiling penalty that inflates the reported 37% MSE and structural gains. The stress-test note got this right. This is fixable—report per-case compile outcomes and recompute on mutually compiling subsets—but without it the central comparison is ambiguous.\n\nSecond, the structural ground truth comes from CodeFun, which is also a baseline. The authors manually correct the CodeFun trees, but any grouping bias inherited from CodeFun would tilt TreeBLEU, Container Match, and TED in DesignCoder's favor if DesignCoder's grouping follows the same logic. The paper acknowledges manual correction but doesn't measure inter-annotator agreement or independence from CodeFun. That's a real circularity worry, not a fatal one.\n\nThe paper does some things well: the dataset is 300 mockups from two sources, the user study has 5 senior developers with p-values (though small n), and the ablation roughly supports the contribution of each stage. The divide-and-conquer generation and vision-based self-correction are not new by themselves, but the grouping chain plus pipeline is a legitimate increment.\n\nThe tables report no variance or confidence intervals for the main metrics, so \"significantly\" is doing unsupported work in the abstract. No code or data is released, so the numbers can't be reproduced.\n\nWho's this for? People building UI code generation systems will get useful ideas from the prompt design and the correction loop. A serious referee can give them a path to a solid paper if the evaluation is tightened.\n\nMy recommendation: send it to peer review—it deserves referee time—but the decision should be conditional on major revision, not acceptance as-is.\n\nBest.","headline":"DesignCoder is a sensible three-stage pipeline for UI code generation, but its headline numbers rest on shaky evaluation choices that need fixing before I'd trust the effect sizes.","tokens_in":15519,"tokens_out":1845,"would_cite":false,"duration_ms":18576,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DesignCoder claims that making an MLLM first reconstruct the nested component hierarchy of a design, then generate code per subtree, and finally repair the rendered result by visual comparison, yields React Native code that is both more…","keywords":["UI code generation","multimodal large language models","hierarchical UI understanding","chain-of-thought","self-correction","React Native","design mockup","visual fidelity"],"falsifier":"Recompute TreeBLEU, Container Match, and Tree Edit Distance using reference trees written independently by developers (not derived from CodeFun) and check whether DesignCoder still beats Prototype2Code on the same 300 mockups; a drop to parity would indicate that the reported structural gains are artifacts of the reference construction.","tokens_in":14603,"feed_emoji":"🧩","tokens_out":4253,"duration_ms":37489,"temperature":0.7,"pith_summary":"DesignCoder claims that generating UI code from design mockups fails mainly because multimodal language models cannot grasp the nested hierarchy of a screen, and that this can be fixed by a three-stage pipeline. The pipeline first builds a UI Grouping Chain—a chain-of-thought that divides the screen into regions, names the elements semantically, and groups them into a component tree. It then generates React Native code per subtree using design metadata, and finally renders the page, compares it visually against the original mockup, and repairs the code component by component. The paper reports that on 300 real mockups this pipeline beats specialized and general-purpose baselines on both visual similarity and code-structure similarity, and that senior developers rate the output higher on availability, modification time, readability, and maintainability.","feed_headline":"Hierarchy-aware pipeline lifts mockup-to-code fidelity by 37%","feed_subtitle":"DesignCoder's grouping-chain plus self-correction beats specialized and general MLLM baselines in React Native.","key_machinery":"UI Grouping Chain: a multimodal chain-of-thought that turns a flat list of design layers into a hierarchical component tree through three steps—UI division, semantic extraction, and component grouping—each followed by a post-processing correction. The tree is then treated as the skeleton for divide-and-conquer code generation and for the self-correction loop, where a UI automation tool captures the rendered page, visual comparison prompts produce targeted repair suggestions, and each component's code is repaired independently before merging.","core_discovery":"The central claim is that explicit hierarchical reasoning about UI structure, before any code is written, is the key to both visual fidelity and structural quality in MLLM-based code generation. DesignCoder constructs a component tree via a multimodal chain-of-thought where the model first segments the flattened layer list into semantic regions, annotates each element's meaning, and then groups elements into components; this tree, rather than the raw screenshot, drives code generation. The paper supports the claim with a 300-mockup benchmark showing the full pipeline improves over the best baseline on MSE by 37.63%, CLIP by 9.52%, SSIM by 12.82%, and on TreeBLEU by 30.19%, Container Match by 29.31%, and Tree Edit Distance by 24.67%. A five-developer user study adds that the generated code is rated closer to production-ready than the output of the strongest MLLM-based baseline.","pith_inferences":["Editor's inference: The same hierarchy-first, render-then-repair recipe could transfer to web (HTML/CSS) and desktop UI generation, since the defects it targets—omission, distortion, and misalignment—are not specific to React Native.","Editor's inference: The gain from the self-correction stage may scale with the visual grounding ability of the underlying MLLM, so the reported ablation numbers could shift meaningfully if the foundation model changes.","Editor's inference: The structural ground truth is derived from the CodeFun platform, so if the authors' corrections inherited CodeFun's grouping biases, the structural-similarity advantages over rule-based baselines could be partly an artifact of the reference construction."],"forward_implications":["If correct, MLLM-based UI code generation can reach production-like fidelity on design mockups without manual layout coding.","Component-tree-driven generation yields reusable, responsive components, reducing the time developers need to modify generated code.","The self-correction loop generalizes beyond code: any renderable artifact could be improved by comparing its rendered output against the original design.","Semantic, MLLM-driven grouping could replace rule-based approaches for novel UI patterns that do not fit predefined rules."],"supporting_citations":[{"why":"Prototype2Code is the state-of-the-art mockup-to-code baseline that DesignCoder must beat on visual and structural metrics.","marker":"[30]"},{"why":"DeclarUI is the MLLM-based mobile UI generation baseline using the same foundation model, providing the comparison for the user study.","marker":"[34]"},{"why":"CodeFun is the enterprise platform used both as a baseline and as the starting point for constructing the ground-truth component trees.","marker":"[1]"},{"why":"The divide-and-conquer approach for UI code generation from screenshots motivates the region-wise decomposition used in DesignCoder.","marker":"[26]"},{"why":"Design2Code supplies the CLIP-based visual similarity metric and prompting best practices for MLLM image-to-code generation.","marker":"[25]"},{"why":"The screen parsing work supplies the Container Match metric for evaluating element grouping alignment.","marker":"[29]"},{"why":"WebCode2M supplies the TreeBLEU metric for measuring code structure similarity between generated and reference trees.","marker":"[13]"}],"fun_headline_variants":["Hierarchy-aware codegen lifts UI fidelity 37%, structure 30%","DesignCoder: self-correcting UI code from hierarchy reasoning","UI mockup to code: hierarchy-first beats general MLLMs","37% fidelity gain via UI hierarchy and self-correction","DesignCoder: hierarchy-aware and self-correcting UI codegen"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reference component trees used to score code structure—produced by the CodeFun platform and manually corrected by two authors—are a neutral and valid standard, rather than a standard that shares grouping biases with DesignCoder.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchy-aware codegen lifts UI fidelity 37%, structure 30%","DesignCoder: self-correcting UI code from hierarchy reasoning","UI mockup to code: hierarchy-first beats general MLLMs","37% fidelity gain via UI hierarchy and self-correction","DesignCoder: hierarchy-aware and self-correcting UI codegen"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000302,"raw_usage":{"total_tokens":1793,"prompt_tokens":1055,"completion_tokens":738,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":671,"completion_tokens_details":{"reasoning_tokens":646}},"tokens_in":671,"tokens_out":738,"duration_ms":7064,"temperature":1.0,"reasoning_tokens":646,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:27:47.156264+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute TreeBLEU, Container Match, and Tree Edit Distance using reference trees written independently by developers (not derived from CodeFun) and check whether DesignCoder still beats Prototype2Code on the same 300 mockups; a drop to parity would indicate that the reported structural gains are artifacts of the reference construction.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prototype2Code is the state-of-the-art mockup-to-code baseline that DesignCoder must beat on visual and structural metrics."},{"cited_title":"Codefun website","cited_arxiv_id":null,"evidence_quote":"CodeFun is the enterprise platform used both as a baseline and as the starting point for constructing the ground-truth component trees."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Design2Code supplies the CLIP-based visual similarity metric and prompting best practices for MLLM image-to-code generation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The screen parsing work supplies the Container Match metric for evaluating element grouping alignment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"WebCode2M supplies the TreeBLEU metric for measuring code structure similarity between generated and reference trees."}],"review_version":1}