Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A benchmark of 4,075 blinded expert pairwise votes ranks AI text-to-app tools by visual design quality and puts Orchids first.

desk verdict A genuinely useful first benchmark for AI text-to-app visual quality, but the Figma Make prompt mismatch and an overstated 'clear separation' claim keep the leaderboard from being the final word. read the letter →

arxiv 2508.20410 v3 pith:W5KX3T2P submitted 2025-08-28 cs.CL

Sam Jung , Agustin Garcinuno , Spencer Mateega This is my paper
classification cs.CL
keywords AItext-to-apptoolsbenchmarkvisualdesignqualityexpertpairwisecomparisonTrueSkillrankingUI/UXevaluationleaderboard
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the visual quality of AI text-to-app tools has been left unverified by marketing claims and feature lists, and that it can be measured with a blinded, expert-judged benchmark. UI-Bench standardizes 30 realistic client prompts, generates sites from 10 tools, and collects 4,075 pairwise decisions from 194 professional designers. The central result is a reproducible leaderboard: Orchids ranks first, Figma Make second, and Lovable third. If correct, the benchmark gives practitioners and developers a common yardstick for design quality rather than anecdotes, and a target to improve against.

What carries the argument

The load-bearing mechanism is the blinded pairwise comparison protocol combined with TrueSkill. Each expert sees two full-page outputs side by side with randomized placement and hidden tool identities, and must answer a single forced-choice client-delivery question. TrueSkill, a Bayesian rating system, updates per-prompt skill estimates for the two tools on every binary vote, starting each at mu=25 and sigma=8.33; a tool's overall score is the mean of its 30 prompt-level mu values. The client-delivery framing converts raw pairwise preference into a professional design-quality signal, and the standardized prompt set makes the comparison reproducible across tools and over time.

What would settle it

Re-run the evaluation protocol with Figma Make receiving the full 150+ word prompts instead of the condensed 500-character versions. If its TrueSkill mean falls below Lovable's or loses its clear second-place separation, the published leaderboard order is an artifact of the prompt mismatch. More cheaply, release the per-prompt scores and check whether Figma Make's advantage appears even on prompts where the condensation changed the brief.

Watch

Extended reading notes

Core claim

The paper claims that UI-Bench is the first large-scale benchmark to evaluate the visual design quality of AI text-to-app tools using blinded expert pairwise comparison. Rather than relying on automated aesthetic proxies or open crowd votes, it asks professional designers, after viewing full pages, which of two sites they would deliver to a client. Over 30 prompts in five categories, scoring with a TrueSkill model yields calibrated confidence intervals and separates leading tools from a cluster of mid-tier and low-tier systems. On this metric Orchids leads with mean rating 30.12 (95% CI [26.61, 33.55]), ahead of Figma Make and Lovable, while the lower-ranked tools overlap and fall below 24.

Load-bearing premise

The rankings assume every tool was tested under identical conditions, but Figma Make received shortened prompts because of its 500-character input limit, so its second-place rating may partly reflect the prompt change rather than the tool's design ability.

Editorial extensions

If this is right

  • Tool developers get a concrete, reproducible optimization target: winning blinded client-delivery judgments on the 30 standardized prompts instead of chasing feature checklists.
  • Buyers can compare tools on measured output quality and re-check rankings as tools update, because the prompt set is fixed and public.
  • The overlapping credible intervals among mid-tier tools mean the benchmark currently separates only the top cluster; more votes are needed to resolve ranks 4 to 8.
  • Per-prompt TrueSkill ratings allow category-level diagnostics, so a team can identify which prompt types a tool handles well and which drag it down.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because expert taste correlates only moderately with lay-user appeal (a point the paper itself raises), the leaderboard should be read as a measure of professional polish, not of consumer preference or conversion potential.
  • A re-run of the same protocol with a lay-designer panel would likely reorder the mid-tier tools, since novices weight different visual cues than professional designers.
  • The released pairwise judgments could be used to train a text-to-app-specific automatic aesthetic scorer, giving the field a cheaper proxy calibrated to expert preference instead of generic image metrics.
  • The absence of per-category score breakouts leaves open the possibility that Orchids' lead is concentrated in a few prompt categories; publishing category-level TrueSkill means would clarify whether the ranking reflects a broad capability or a niche strength.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. UI-Bench is a benchmark for evaluating the visual design quality of AI text-to-app tools. Thirty synthetic client briefs across five categories are given to ten tools; 4,075 blinded pairwise expert comparisons on the client-delivery question are aggregated with a per-prompt TrueSkill model, and tools are ranked by the mean rating across prompts. The paper reports Orchids first (µ=30.12), Figma Make second (µ=27.46), and releases prompts, an evaluation framework, and a public leaderboard. The stated contribution is a reproducible standard for advancing AI-driven web design.

Significance. The contribution is potentially valuable: the paper addresses a genuine gap, since existing web benchmarks focus on code fidelity/functionality rather than holistic design quality, and the use of blinded expert pairwise comparison with a Bayesian rating model is a sensible methodology. The release of prompts and an open-source framework is a concrete asset, and the authors are transparent about the Figma Make input-limit deviation. However, the headline leaderboard is currently weakened by an uncontrolled input variable for the #2-ranked system, and some qualitative claims about separation go beyond what the reported CIs support. With a matched-input sensitivity analysis and more careful uncertainty reporting, UI-Bench could become a useful standard.

major comments (3)
  1. [§7.1, Table 1] The Figma Make prompt mismatch is a confound for the #2 rank. The protocol (§4.1) standardizes prompts so that outcomes reflect tool capability; Figma Make instead received 500-character ChatGPT summaries while all other tools received the full 150+ word briefs. The paper acknowledges this and releases the summaries, which is commendable, but the leaderboard treats Figma Make as directly comparable. A matched-input sensitivity check is needed: for example, generate with one or two other tools on both the full and the summarized inputs, or report a re-ranked leaderboard excluding Figma Make. Without this, the second-place position is the result of an uncontrolled variable and does not support a fair comparison.
  2. [§5.1, Table 1] The claim of 'clear separation between the leading systems and the trailing cluster' is stronger than the reported 95% CIs support. Orchids, Figma Make, and Lovable have intervals [26.61,33.55], [24.11,30.81], and [23.77,30.51], respectively, which overlap substantially; the mid-tier CIs also overlap with the top. The text itself notes mid-tier clustering, but the concluding 'clear separation' is not justified. Please report pairwise posterior probabilities of superiority (or rank probabilities) rather than relying on interval inspection, and qualify the ranking claims accordingly.
  3. [§3.2/§5.1] The statistical aggregation is underspecified. The global score is the mean of 30 per-prompt TrueSkill µ values, but the reported σ and 95% CI are not derived explicitly. If σ is the standard deviation of the per-prompt means, it is not a valid posterior credible interval for the global skill because it ignores per-prompt posterior variance and correlations. Clarify the model and computation; if a joint model is used, state it. The CIs are a central element of the leaderboard, so this needs to be precise.
minor comments (6)
  1. [§1.1] Typo: 'Rapid Emergence of of AI App and Website Tools' should read 'of AI App and Website Tools'.
  2. [§7.1] Typo: 'sumarized' should be 'summarized'.
  3. [§6.1.2] The sentence 'We make no claims on AI failure modes related to losses clustering around: typography, spacing...' is confusing; it appears to enumerate observed failure modes while disclaiming claims. Please rephrase to state what is claimed and what is deferred.
  4. [§5.1] The phrase 'n = 8,150 unique tool comparisons' is misleading: 4,075 pairwise matches imply 8,150 tool appearances, not unique comparisons. Rephrase.
  5. [Abstract] Missing space: 'nopublicbenchmark' should be 'no public benchmark'.
  6. [Appendix A / §4.1] The authoring prompt specifies 'Scenario (120–180 words)' whereas §4.1 requires 150–180 words. Align the two specifications.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the leaderboard is an empirical ranking from blinded expert votes; the Figma Make prompt mismatch is a disclosed validity limitation, not a circular derivation.

full rationale

UI-Bench's central claim is that it provides a reproducible benchmark and that its TrueSkill-derived leaderboard ranks AI text-to-app tools by expert pairwise preference. The ranking is produced directly from 4,075 blinded pairwise judgments, with ratings computed by a standard TrueSkill model using fixed priors (µ=25.0, σ=8.33). There is no step where a parameter is fitted to a subset of the data and then reported as a prediction of a closely related quantity; the reported scores are posterior means of the collected votes, not forecasts derived from fitted inputs. The only self-citation (FinanceQA, Mateega et al. 2025) appears in a list of static LLM benchmarks in related work and plays no role in the methodology or the leaderboard, so it is not load-bearing. The paper's acknowledged limitation that Figma Make received condensed 500-character prompts while other tools received full 150+ word prompts is a legitimate concern about the fairness or construct validity of that tool's ranking, but it is not circularity: the ranking is still computed from the votes that were actually collected, and the differing input is an experimental-design confound, not a case of the output being defined in terms of the input or a fitted parameter being relabeled as a prediction. No equation in the paper reduces to its own inputs, and no uniqueness claim is imported from the authors' prior work to force a conclusion. The benchmark is self-contained empirical evaluation, so the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The benchmark's ranking depends on the validity of expert preference as a proxy for design quality, the representativeness of the prompts, and the comparability of conditions across tools. The Figma Make prompt mismatch is a known violation of equivalence that affects the #2 ranking.

free parameters (2)
  • TrueSkill initial mu and sigma = mu=25.0, sigma=8.33
    Standard TrueSkill priors, chosen by hand rather than fitted; they set the rating scale but likely do not change the ordinal ranking.
  • Adaptive phase top-5 sample size = 5
    The adaptive prompt-sampling phase draws from a top-5 by a weighted score; this is a protocol choice, not fitted to data.
assumptions (4)
  • domain assumption Expert pairwise preference on the client-delivery question is a valid measure of design quality.
    Used to justify the primary endpoint; grounded in consensual assessment and comparative judgment literature, but not independently validated for this specific task.
  • standard math TrueSkill aggregation across pairwise matches yields a meaningful per-tool skill score.
    The model assumes a generative process for match outcomes; the paper averages per-prompt ratings, which is an additional modeling choice.
  • domain assumption The 30 synthetic prompts are representative of real text-to-app client briefs.
    Prompts were generated by GPT-5 with light editorial review; representativeness is not empirically demonstrated.
  • domain assumption All tools received equivalent evaluation conditions.
    Violated for Figma Make due to shortened prompts; acknowledged in Limitations. This is the weakest assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools." pith.science (2026). https://pith.science/paper/W5KX3T2P

@misc{pith2026250820410,
  author       = {Pith},
  title        = {Pith review of: UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W5KX3T2P}},
  note         = {Machine review of arXiv:2508.20410}
}
read the original abstract

AI text-to-app tools promise high quality applications and websites in minutes, yet no public benchmark rigorously verifies those claims. We introduce UI-Bench, the first large-scale benchmark that evaluates visual excellence across competing AI text-to-app tools through expert pairwise comparison. Spanning 10 tools, 30 prompts, 300 generated sites, and 4,000+ expert judgments, UI-Bench ranks systems with a TrueSkill-derived model that yields calibrated confidence intervals. UI-Bench establishes a reproducible standard for advancing AI-driven web design. We release (i) the complete prompt set, (ii) an open-source evaluation framework, and (iii) a public leaderboard. The generated sites rated by participants will be released soon. View the UI-Bench leaderboard at https://uibench.ai/leaderboard.

Figures

Figures reproduced from arXiv: 2508.20410 by the authors.

Figure 1
Figure 1. Example comparison interface used in UI-Bench. Raters view two webpages in a blinded layout with randomized placement and a full-view requirement. 4.4.2 Comparison Presentation Each comparison is presented in a controlled, blinded UI. As shown in [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    Cookie-Bench is a reference-free 1,000-query web development benchmark paired with Cookie-Frame, a metacognition-inspired three-stage framework (static perception, agent interaction, dynamic scoring) that aligns with ...

Reference graph

Works this paper leans on

46 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Teresa M. Amabile. Social psychology of creativity: A consensual assessment technique. Journal of Personality and Social Psychology, 43 0 (5): 0 997--1013, 1982. doi:10.1037/0022-3514.43.5.997

    doi:10.1037/0022-3514.43.5.997 1982

  2. [2]

    Teresa M. Amabile. Creativity in Context: Update to the Social Psychology of Creativity. Westview Press, Boulder, CO, 1996

    work page 1996

  3. [3]

    Bradley and Milton E

    Ralph A. Bradley and Milton E. Terry. Rank analysis of incomplete block designs. Biometrika, 39 0 (3-4): 0 324--345, 1952. doi:10.1093/biomet/39.3-4.324

    doi:10.1093/biomet/39.3-4.324 1952

  4. [4]

    Evaluating large language models trained on code, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code, 2021. URL https://arxiv.org/abs/2107.03374

    arXiv 2021

  5. [5]

    Chiang, L

    W.-L. Chiang, L. Zheng, Y. Sheng, A. N. Angelopoulos, T. Li, D. Li, H. Zhang, B. Zhu, M. I. Jordan, J. E. Gonzalez, and I. Stoica. Chatbot arena: An open platform for evaluating LLMs by human preference, 2024. URL https://arxiv.org/abs/2403.04132

    arXiv 2024

  6. [6]

    Christiano, Jan Leike, Tom B

    Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems (NeurIPS), pages 4299--4307, 2017

    work page 2017

  7. [7]

    Training verifiers to solve math word problems, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168

    arXiv 2021

  8. [8]

    Webthetics: Quantifying webpage aesthetics with deep learning

    Qi Dou, Xiansong Zheng, Tao Sun, and Pheng-Ann Heng. Webthetics: Quantifying webpage aesthetics with deep learning. International Journal of Human-Computer Studies, 124: 0 56--66, 2019. doi:10.1016/j.ijhcs.2018.11.006

    work page doi:10.1016/j.ijhcs.2018.11.006 2019

Show all 46 references
  1. [9]

    Hashimoto

    Yann Dubois, Bal \'a zs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. Length-controlled AlpacaEval : A simple way to debias automatic evaluators, 2024. URL https://arxiv.org/abs/2404.04475

    2024 arXiv

  2. [10]

    How content volume on landing pages influences consumer behavior: Empirical evidence

    Nim Dvir and Ruti Gafni. How content volume on landing pages influences consumer behavior: Empirical evidence. In Proceedings of InSITE 2018, pages 35--53, 2018. doi:10.28945/4016

    2018 doi

  3. [11]

    B. J. Fogg, Jonathan Marshall, Othman Laraki, Alex Osipovich, Chris Varma, Nicholas Fang, Jyoti Paul, Amy Rangnekar, John Shon, Peter Swani, and Marissa Treinen. What makes web sites credible? a report on a large quantitative study. In Proceedings of the SIGCHI Conference on H...

    2001

  4. [12]

    WebCode2M : A real-world dataset for code generation from webpage designs

    Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Bohua Chen, Dongping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, and Xiangliang Zhang. WebCode2M : A real-world dataset for code generation from webpage designs. In Proceedings of The Web Conference (WWW), 2025

    2025

  5. [13]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), 2021. URL https://arxiv.org/abs/2009.03300

    2021 arXiv

  6. [14]

    TrueSkill : A Bayesian skill rating system

    Ralf Herbrich, Tom Minka, and Thore Graepel. TrueSkill : A Bayesian skill rating system. In Advances in Neural Information Processing Systems (NIPS), volume 19, pages 569--576, 2006. URL https://papers.nips.cc/paper/3079-trueskilltm-a-bayesian-skill-rating-system.pdf

    2006

  7. [15]

    CLIPScore : A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore : A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7514--7528, 2021. doi:10.1865...

    2021 doi

  8. [16]

    GANs trained by a two time-scale update rule converge to a local Nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), 2017. URL https://arxiv.org/abs/1706.08500

    2017 arXiv

  9. [17]

    Jankowski, J

    J. Jankowski, J. Hamari, and J. W a tr \'o bski. A gradual approach for maximising user conversion without compromising experience with high visual intensity website elements. Internet Research, 29 0 (1): 0 194--217, 2019. doi:10.1108/INTR-09-2016-0271

    2019 doi

  10. [18]

    Jayasumana, X

    S. Jayasumana, X. Liu, X. E. Wang, L. Ma, S. Savarese, and L. El Ghaoui. Rethinking FID : Towards a better evaluation metric for image generation, 2024

    2024

  11. [19]

    Kirstain, A

    Y. Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In Advances in Neural Information Processing Systems (NeurIPS), 2023. URL https://papers.neurips.cc/paper_files/paper/2023/file/7...

    2023

  12. [20]

    Assessing dimensions of perceived visual aesthetics of web sites

    Talia Lavie and Noam Tractinsky. Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human-Computer Studies, 60 0 (3): 0 269--298, 2004. doi:10.1016/j.ijhcs.2003.09.002

    2004 doi

  13. [21]

    T. Lee, M. Yasunaga, C. Meng, Y. Mai, J. S. Park, A. Gupta, Y. Zhang, D. Narayanan, H. B. Teufel, M. Bellagente, M. Kang, T. Park, J. Leskovec, J.-Y. Zhu, F.-F. Li, J. Wu, S. Ermon, and P. Liang. Holistic evaluation of text-to-image models. In Advances in Neural Information Pr...

    2023 arXiv

  14. [22]

    Lu, Zhe Lin, Hailin Jin, Jianchao Yang, and James Z

    Xiaoran L. Lu, Zhe Lin, Hailin Jin, Jianchao Yang, and James Z. Wang. RAPID : Rating pictorial aesthetics using deep learning. In Proceedings of ACM Multimedia, pages 457--466, 2014. doi:10.1145/2647868.2654927

    2014

  15. [23]

    WebGen-Bench : Evaluating LLMs on generating interactive and functional websites from scratch, 2025

    Zimu Lu, Yunqiao Yang, Houxing Ren, Han Xiao, Ke Wang, Weikang Shi, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. WebGen-Bench : Evaluating LLMs on generating interactive and functional websites from scratch, 2025

    2025

  16. [24]

    FinanceQA : A benchmark for evaluating financial analysis capabilities of large language models, 2025

    Spencer Mateega, Carlos Georgescu, and Danny Tang. FinanceQA : A benchmark for evaluating financial analysis capabilities of large language models, 2025

    2025

  17. [25]

    Moshagen and M

    M. Moshagen and M. T. Thielsch. Facets of visual aesthetics. International Journal of Human-Computer Studies, 68 0 (10): 0 689--709, 2010. doi:10.1016/j.ijhcs.2010.05.006

    2010 doi

  18. [26]

    Lev Muchnik, Sinan Aral, and Sean J. Taylor. Social influence bias: A randomized experiment. Science, 341 0 (6146): 0 647--651, 2013. doi:10.1126/science.1240466

    2013 doi

  19. [27]

    D. C. L. Ngo, L. S. Teo, and J. G. Byrne. Modelling interface aesthetics. Information Sciences, 152: 0 25--46, 2003. doi:10.1016/S0020-0255(02)00404-8

    2003 doi

  20. [28]

    Color compatibility from large datasets

    Patrick O 'Donovan, Aseem Agarwala, and Aaron Hertzmann. Color compatibility from large datasets. ACM Transactions on Graphics, 30 0 (4): 0 63:1--63:12, 2011. doi:10.1145/2010324.1964958

    2011

  21. [29]

    Comparative judgement for assessment

    Alastair Pollitt. Comparative judgement for assessment. International Journal of Technology and Design Education, 22 0 (2): 0 157--170, 2012. doi:10.1007/s10798-011-9189-x

    2012 doi

  22. [30]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021. URL https://arxiv.org/abs/2103.00020

    2021 arXiv

  23. [31]

    Krzysztof Reinecke, Tom Yeh, Luke Miratrix, Rahmat Mardiko, Yao Zhao, Jenny Liu, and Krzysztof Z. Gajos. Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on ...

  24. [32]

    Robins and J

    D. Robins and J. Holmes. Aesthetics and credibility in web site design. Information Processing & Management, 44 0 (1): 0 386--399, 2008. doi:10.1016/j.ipm.2007.02.003

    2008 doi

  25. [33]

    Improved techniques for training GANs

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs . In Advances in Neural Information Processing Systems (NIPS), 2016. URL https://proceedings.neurips.cc/paper/2016/file/8a3363abe792db2d8761d6403605ae...

    2016

  26. [34]

    Design2code: Benchmarking multimodal code generation for automated front-end engineering

    Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2code: Benchmarking multimodal code generation for automated front-end engineering. In Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Ling...

    2025

  27. [35]

    Beyond the imitation game: Quantifying and extrapolating the capabilities of language models ( BIG -bench)

    Aarohi Srivastava et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models ( BIG -bench). Transactions on Machine Learning Research, 2023. URL https://arxiv.org/abs/2206.04615

    2023 arXiv

  28. [36]

    L. L. Thurstone. A law of comparative judgment. Psychological Review, 34 0 (4): 0 273--286, 1927. doi:10.1037/h0070288

    1927 doi

  29. [37]

    C. G. von Wangenheim, J. V. A. Porto, J. C. R. Hauck, and A. F. Borgatto. Do we agree on user interface aesthetics of Android apps?, 2018

    2018

  30. [38]

    Whitehouse and A

    C. Whitehouse and A. Pollitt. Using adaptive comparative judgement to obtain a highly reliable rank order in summative assessment. Technical report, AQA Centre for Education Research and Policy, Manchester, UK, 2012. URL https://filestore.aqa.org.uk/content/research/CERP_RP_CW...

    2012

  31. [39]

    The best AI website builders in 2025

    Owain Williams. The best AI website builders in 2025. https://www.techradar.com/pro/best-ai-website-builder, 2025. Accessed 2025-08-10

    2025

  32. [40]

    Jingyu Xiao, Ming Wang, Man Ho Lam, Yuxuan Wan, Junliang Liu, Yintong Huo, and Michael R. Lyu. Designbench: A comprehensive benchmark for MLLM -based front-end code generation, 2025

    2025

  33. [41]

    J. Xu, Z. Shi, Y. Wang, Y. Zhang, D. Lin, D. Zhang, and W. Zhu. Imagereward: Learning and evaluating human preferences for text-to-image generation. In Advances in Neural Information Processing Systems (NeurIPS), 2023. URL https://papers.nips.cc/paper_files/paper/2023/file/336...

    2023

  34. [42]

    Xing, Xiaodan Liang, and Zhiqiang Shen

    Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, and Zhiqiang Shen. Web2Code : A large-scale webpa...

    2024

  35. [43]

    HellaSwag : Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4791--4800, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag : Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4791--4800, 2019. doi:10.18653/v1/P19-1472. UR...

    2019 doi

  36. [44]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM -as-a-judge with MT -bench and chatbot arena, 2023. URL https://arxiv.org/abs/2306.05685

    2023 arXiv

  37. [45]

    AGIEval : A human-centric benchmark for evaluating foundation models, 2023

    Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. AGIEval : A human-centric benchmark for evaluating foundation models, 2023. URL https://arxiv.org/abs/2304.06364

    2023 arXiv

  38. [46]

    Frontendbench: A benchmark for evaluating LLMs on front-end development via automatic evaluation, 2025

    Hongda Zhu, Yiwen Zhang, Bing Zhao, Jingzhe Ding, Siyao Liu, Tong Liu, Dandan Wang, Yanan Liu, and Zhaojian Li. Frontendbench: A benchmark for evaluating LLMs on front-end development via automatic evaluation, 2025

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.