Pith. sign in

REVIEW 5 major objections 4 minor 42 references

Five evidence-grounded personas, allowed to deliberate and socially weighted, track human UI-quality ratings far better than a single LLM judge (r = 0.716 to 0.922).

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:19 UTC pith:3LOP77PZ

load-bearing objection A real method contribution for GenUI evaluation — persona panel with evidence grounding beats a single judge by a wide margin — but the validation evidence is thinner than the abstract implies, with an uncharacterized human target and no uncertainty quantification. the 5 major comments →

arxiv 2607.28439 v2 pith:3LOP77PZ submitted 2026-07-30 cs.CL

Beyond a Single Judge: The Evidence-Grounded, Social-Weighted Persona Panel for Generative UI Evaluation

Zheng Wu , Yibo Luo , Pu Zhang , Cheng Yang , Zhuosheng Zhang This is my paper
classification cs.CL
keywords generative UI evaluationLLM-as-a-judgepersona panelevidence groundingbounded confidence opinion dynamicsDelphi aggregationhuman judgment fidelityUI quality dimensions
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Generative UI lets LLMs render interfaces from text, but judging quality is stuck between costly human raters and a single LLM judge that reflects one implicit viewpoint. This paper proposes ESPP, a three-stage judge: a panel of five psychologically diverse personas, each grounded in its own question-answer evidence, rates a screenshot independently; the panel then exchanges opinions under a trait-derived, semantically gated bounded-confidence rule; and a Delphi-inspired weighting combines revised ratings. On a 7,000-screenshot benchmark with human ground truth, ESPP raises Pearson correlation with human mean scores from 0.716 (single pass) to 0.922, while a 5-prompt ensemble reaches only 0.789, recovering about a third of the gap. The authors argue that most of the gain comes from genuine persona and evidence grounding, not from averaging multiple LLM calls, and that retaining per-persona ratings exposes dimension-localized disagreements between user subgroups that a single score erases.

Core claim

The central claim is that a socially-weighted panel of evidence-grounded personas is a substantially more faithful automatic proxy for mean human judgment of generative-UI screenshots than a single LLM judge, without needing human raters at evaluation time. Concretely, Pearson correlation with per-screenshot human means rises from 0.716 (naive single-pass) to 0.922 (full ESPP); the 0.789 of the prompt-ensemble control isolates genuine persona/evidence grounding as the dominant source. The ablation shows removing the discussion/aggregation stages costs little, while removing P-Q-A grounding accounts for most of the remaining gap to baseline.

What carries the argument

The load-bearing object is the persona panel itself: five LLM-backed synthetic raters, each parameterized by a Big-Five trait vector, cognitive style, demographics, and a bank of persona-specific question-answer pairs (P-Q-A) that behaviorally instantiate those traits. Three mechanisms carry the argument: (1) evidence grounding, which conditions each rating on the persona's own P-Q-A evidence so a rating is anchored rather than a free rationalization; (2) semantic bounded-confidence opinion dynamics, an adaptation of Hegselmann-Krause in which a persona revises toward peers only if numerically close and if the peer's argument references the persona's own topical hooks, with receptivity a det

Load-bearing premise

The load-bearing premise is that the constructed 1,000-persona pool is a valid behavioral sample of real users on the five rating dimensions; the paper compares the panel's aggregate only to the mean of five human raters per screenshot and never validates persona-level ratings against individual human raters, so a systematic persona bias that happens to correlate with the human mean could produce the headline r=0.922.

What would settle it

Compare each persona's individual ratings to those of human raters matched on the same Big-Five, cognitive-style, and demographic profile: if persona ratings do not reproduce the between-rater distribution (not just the pooled mean) of real users, the claim that the panel tracks human judgment collapses. Simpler: re-run the benchmark with a different judge model or with the human ground truth expanded to more than five raters; if r drops markedly, the result is tied to the specific judge or the small rater target.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Automatic evaluation of generative UI can approach the fidelity of human mean ratings without employing human raters at run time; the paper's numbers put ESPP at r = 0.922 vs. 0.716 for a single judge.
  • Simply averaging several LLM judge passes is not sufficient; the prompt-ensemble control recovers only about one-third of the gap, pointing to viewpoint diversity plus grounding as the active ingredient.
  • Per-persona ratings preserve information a single aggregate judge destroys: user subgroups agree on overall model rankings but diverge sharply on specific dimensions, especially Control.
  • The panel's psychological mechanisms are behaviorally operative, not decorative: trait-derived receptivity predicts revision magnitude, and discussion converges partially rather than collapsing to consensus.
  • The resulting leaderboard suggests Transparency is a systematic, industry-wide weakness (lowest dimension for all 14 models) and Understanding is near-ceiling for all.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the claimed fidelity holds up beyond this benchmark, the same three-stage structure—grounded personas, gated discussion, soft weighting—could transfer to other subjective evaluation tasks (e.g., writing quality, product reviews, content moderation) where population heterogeneity matters.
  • A direct validation the paper leaves implicit: comparing each persona's ratings against actual human raters matched on the same traits, rather than only against the pooled 5-rater mean, would test whether the persona pool is behaviorally representative or merely correlating with the mean.
  • The r=0.922 could in principle be inflated by the narrow 3.45–3.73 leaderboard range and the 5-rater target; a testable extension is to measure calibration on screenshots with high rater disagreement, where the panel's behavior is most consequential.
  • The finding that low-tech-literacy and high-agreeableness personas are more susceptible to cosmetic dark-pattern manipulations suggests a tunable robustness knob: panels could be skewed toward more critical personas for adversarial settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes ESPP, a three-stage method for evaluating generative UI screenshots: a diversity-constrained panel of evidence-grounded personas independently rates each screenshot (Stage 1), revises ratings through semantic bounded-confidence opinion dynamics with trait-derived receptivity (Stage 2), and aggregates via Delphi-inspired social weighting (Stage 3). The method is validated on UIPersonaBench, a new benchmark of 500 instructions rendered by 14 models (7,000 screenshots), with human ground truth from 5 raters per screenshot. The headline result is an increase in Pearson correlation with the human mean from 0.716 (naive single-pass judge) to 0.922 (full ESPP), with a 5-pass prompt-ensemble control at 0.789 recovering only about a third of the gap. The paper also reports cumulative ablations, checks that trait-derived receptivity predicts rating revision, shows partial rather than full consensus in discussion, claims dimension-localized disagreement between user subgroups, and tests robustness to cosmetic dark-pattern overlays.

Significance. If the result holds, the contribution is substantial: a practical automatic proxy for mean human ratings of GenUI quality, with a large improvement over a single-pass judge and a well-chosen prompt-ensemble control. The paper's strengths are its relatively large benchmark, the explicit control isolating persona/evidence grounding from simple averaging, the cumulative ablation, the behavioral validation of the receptivity mechanism, and the public release of code. The main concerns are external: the human ground truth is only a 5-rater mean with no inter-rater reliability or rater-level validation, the 7,000 observations are clustered within 500 instructions with no adjustment, no held-out split or sensitivity analysis is reported for the many design choices, and the headline correlations lack confidence intervals and significance tests. These gaps are load-bearing for the central claim that ESPP is a substantially more faithful proxy for human judgment.

major comments (5)
  1. [§Experimental Setup, Human Ground Truth; §Main Results, Table 1] The entire external validation rests on a Pearson correlation with the mean of only 5 human raters per screenshot. No inter-rater reliability (ICC/alpha), rater-level correlations, or leave-one-rater-out analyses are reported. If these raters are heterogeneous or idiosyncratic, the target itself is noisy, and a method tuned to it may not generalize to another rater sample. Please report inter-rater agreement, per-rater agreement with the methods, and robustness of the headline r to subsampling the raters.
  2. [§Experimental Setup, Metrics; §Main Results, Table 1] The 7,000 screenshots are nested in 500 instructions, and the same 5-persona panel is reused for all 14 renderings of an instruction. Pearson r is computed over all 7,000 observations as if independent. A substantial part of the correlation could be between-instruction agreement about instruction difficulty rather than within-instruction discrimination among models. Please report cluster-robust confidence intervals or within-instruction correlations, and show that the 0.922 vs 0.789 contrast survives when computed on within-instruction deviations.
  3. [Stage 0; Further Analysis, Table 4 and Figure 5] The persona pool and its subgroups are constructed by the authors from synthetic trait labels; the subgroup differences shown in Table 4 and Figure 5 are therefore internal properties of the construction procedure, not evidence about real user populations. The claim that a single judge erases 'real' disagreement requires validation of the persona subgroups against actual rater demographics or individual human ratings. Without this, the subgroup analysis is descriptive of the method rather than evidence about human users.
  4. [Stage 0; §Experimental Setup] All design choices — panel size N=5, bounded-confidence threshold ε, receptivity constants ρ_min/ρ_max, weighting λ_e=λ_r=0.5, diversity-constrained sampling, and the P-Q-A bank construction — appear to be fixed on UIPersonaBench. No held-out instructions, held-out models, or repeated benchmark variants are reported, so the risk of overfitting the design to this specific benchmark is not addressed. The prompt-ensemble control rules out mere averaging, but not selection of hyperparameters and prompt structure on the same data. Please provide a development/test split or sensitivity analysis over these choices.
  5. [§Further Analysis (Figure 3, Table 4); §Main Results, Table 1] The text states that the receptivity-revision association is 'statistically robust' and that ESPP improves over controls on every metric, but no confidence intervals, significance tests, or effect-size uncertainties are provided. For the 14-model rank-level comparisons in Table 4, the effective sample size is 14, so differences such as ρ(Control)=0.70 vs 0.80 are unlikely to be significant. Please provide bootstrap or permutation intervals for the headline r differences and for the subgroup rank correlations.
minor comments (4)
  1. [Throughout] The abstract and body contain several typos and spacing errors, e.g. 'generateremains' in the abstract and 'Journal of Artifical Societies' in the references. Please proofread.
  2. [§Main Results, Table 2; Further Analysis, Table 4] The authors caution not to over-read the narrow 3.45–3.73 overall score range, yet Table 4 uses rank correlations over these 14 models as evidence about subgroup agreement. Please clarify the interpretational status of these 14-model rankings.
  3. [Stage 0] The construction of the 1,000-persona pool is described at a high level. Please provide details of the P-Q-A elicitation, the stratified sampling over (d,b), and the exact diversity constraint used to select each panel.
  4. [Further Analysis: Adversarial Robustness] The 'small probe set' is not described. Please state the number of screenshots, the number of re-runs per manipulation, and the sample sizes behind Figure 6b.

Circularity Check

0 steps flagged

No significant circularity: the headline fidelity result is anchored to external human labels; only a minor non-load-bearing self-citation is present.

full rationale

The paper's central claim—that the ESPP panel tracks human judgment at r=0.922 versus r=0.716 for a naive single-pass judge—is not circular. The criterion is an external human mean over 7,000 screenshots, and the reported improvement is measured against that external anchor rather than constructed from the method's own definitions. The prompt-ensemble control (r=0.789) provides a non-circular baseline for separating genuine persona/evidence grounding from mere averaging. The design constants (N=5, λe=λr=0.5, the receptivity formula) are declared and hand-set, not fitted to the human labels, so no fitted parameter is being renamed as a prediction. The further analyses of receptivity, partial consensus, and subgroup disagreement are internal mechanism checks or simulation results; they may overstate interpretive reach, but they are not reductions of the central derivation to its inputs. The lack of person-level validation, inter-rater reliability, and held-out instructions are external-validity and benchmark-construction limitations, not circularity. The only overlapping-author citation (Zheng et al. 2023, which includes Wu, Z.) supports background claims about LLM-as-a-judge and is not load-bearing. Overall, the derivation chain is self-contained against an external benchmark, so the circularity score is low.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central claim rests on a chain of assumptions: (1) the 5-rater human mean is a valid criterion; (2) the persona pool/P-Q-A grounding is a valid behavioral proxy for real users; (3) LLM personas follow their instructions in the intended trait-dependent way; (4) the hand-set constants (Eq. 1, Eq. 2, ε, N, diversity constraints) are reasonable. None of these is derived from independent data; the paper provides internal behavior checks for one of them. This does not by itself invalidate the method, but it means the reported r=0.922 is conditional on several unverified modeling choices.

free parameters (4)
  • Receptivity constants ρ_min, ρ_max (Eq. 1) = not stated
    Hand-set endpoints of the trait-derived receptivity function; injected as a qualitative behavioral instruction, not used as a numeric weight, so impact is indirect, but no sensitivity analysis is reported.
  • Bounded-confidence threshold ε (Stage 2 semantic gate) = not stated
    The numeric gate |s_i − s_j| < ε controls which peer comments are licensed to move a persona; the actual value is never given, and no sensitivity analysis is reported.
  • Social-weighting hyperparameters λ_e = λ_r = 0.5 (Eq. 2) = 0.5
    Chosen 'to keep the reweighting mild by construction'; no ablation or sensitivity check on these values.
  • Panel size N=5 and diversity-constrained sampling (Stage 0) = N=5, 1000-persona pool
    Panel composition rule (one close pair, one distant pair) is a hand-set design choice; the paper does not test how N or the diversity constraint affects fidelity.
axioms (6)
  • domain assumption Five raters per screenshot, averaged, constitute a valid human ground-truth score for a 1-5 UI quality rating.
    §Experimental Setup, Human Ground Truth: each screenshot gets 5 human ratings; the mean is treated as the criterion for all fidelity metrics. No inter-rater reliability (ICC, Krippendorff's α, or human-human correlation) is reported, so the noise in the target is unknown.
  • domain assumption The 1,000-persona pool, generated from Big-Five labels, cognitive-style vectors, demographics, and P-Q-A banks, is a valid behavioral sample of the real user population for these UI dimensions.
    Stage 0 constructs personas 'so that (b,c,d,e) are behaviorally instantiated'; the paper never compares individual persona ratings to individual human raters or to known population distributions—only the aggregate panel score to the 5-rater mean.
  • domain assumption An LLM prompted with a persona profile and P-Q-A evidence behaves according to the intended psychological traits.
    The receptivity monotonicity check (§Does Trait-Derived Receptivity Drive Opinion Change?) shows the LLM follows the instruction in the expected direction, but this is a self-referential compliance check, not independent evidence that the simulated traits match real cognitive processes.
  • ad hoc to paper The trait-derived receptivity function ρ(p_i) = ρ_min + (a_i+n_i)/4 (ρ_max − ρ_min) (Eq. 1) adequately represents persuadability as a function of agreeableness and neuroticism.
    The functional form is a linear combination with hand-set endpoints, motivated by FFM correlates of persuadability (Costa and McCrae 2014) but not derived from any dataset; the cited FFM literature does not specify this equation.
  • ad hoc to paper The semantic gate (on-topic argument referencing the persona's own hook set H_i) captures the assimilation-contrast asymmetry of social judgment theory.
    Sherif and Hovland 1961 is cited as motivation, but the paper provides no direct validation that the LLM-implemented semantic gate behaves like the theory (e.g., no test of whether off-topic near arguments actually fail to move personas in a controlled setting).
  • ad hoc to paper A single-round, soft reweighting (Eq. 2) with λ_e=λ_r=0.5 captures the 'spirit' of the Delphi method sufficiently for the claimed social-weighting benefit.
    Dalkey and Helmer 1963 describe an iterative expert-elicitation process; the paper drops iteration and expert selection and replaces them with a closed-form heuristic. This is a loose adaptation and the fidelity contribution of the weighting step is small in the ablation (0.031 of the r-gap).

pith-pipeline@v1.3.0-alltime-deepseek · 11686 in / 12670 out tokens · 127968 ms · 2026-08-03T01:19:18.509795+00:00 · methodology

0 comments
read the original abstract

Generative UI (GenUI) lets large language models synthesize a complete, renderable interface directly from a natural-language instruction, but evaluating the quality of what they generate remains an open problem. Human evaluation is costly and rater-variant, while LLM-as-a-judge is scalable but reflects only a single implicit viewpoint, unable to capture how different populations of real users actually perceive the same interface. We propose the Evidence-Grounded, Social-Weighted Persona Panel (ESPP), a three-stage GenUI evaluation method in which a panel of psychologically diverse, evidence-grounded personas independently rates a screenshot, exchanges opinions under a trait-derived, semantically-gated bounded-confidence mechanism, and is aggregated via Delphi-inspired social weighting into a single judgment. ESPP tracks human judgment substantially more closely than a naive single-pass judge, raising Pearson $r$ from $0.716$ to $0.922$, and a prompt-ensemble control recovers only about a third of this gap, isolating genuine persona and evidence grounding as the dominant source of improvement. Beyond this fidelity gain, retaining each panelist's individual rating further reveals that user subgroups agree on overall model rankings yet diverge sharply on specific rating dimensions, a structural disagreement a single homogeneous judge would systematically erase. The codes are available at https://github.com/Wuzheng02/ESPP.

Figures

Figures reproduced from arXiv: 2607.28439 by Cheng Yang, Pu Zhang, Yibo Luo, Zheng Wu, Zhuosheng Zhang.

Figure 1
Figure 1. Figure 1: Three paradigms for evaluating Generative UI. Hu [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the Evidence-Grounded, Social-Weighted Persona Panel. A diversity-constrained panel [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Stage-2 rating revision as a function of a persona’s [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Post-discussion vs. pre-discussion panel disagree [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-model Stage-1 score gap between subgroups, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 11 linked inside Pith

  1. [1]

    2025 , eprint =

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities , author =. 2025 , eprint =

    2025

  2. [2]

    2025 , eprint =

    Kimi K2: Open Agentic Intelligence , author =. 2025 , eprint =

    2025

  3. [3]

    2026 , eprint =

    GLM-5: from Vibe Coding to Agentic Engineering , author =. 2026 , eprint =

    2026

  4. [4]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2606.19348 , year=

    Deepseek-v4: Towards highly efficient million-token context intelligence , author=. arXiv preprint arXiv:2606.19348 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2602.15763 , year=

    Glm-5: from vibe coding to agentic engineering , author=. arXiv preprint arXiv:2602.15763 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2507.20534 , year=

    Kimi k2: Open agentic intelligence , author=. arXiv preprint arXiv:2507.20534 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2507.06261 , year=

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

    Pith/arXiv arXiv

  9. [9]

    2026 , month = apr, howpublished =

    Introducing. 2026 , month = apr, howpublished =

    2026

  10. [10]

    2026 , month = feb, howpublished =

    Introducing. 2026 , month = feb, howpublished =

    2026

  11. [11]

    2026 , month = mar, howpublished =

    Introducing. 2026 , month = mar, howpublished =

    2026

  12. [12]

    2025 , month = dec, howpublished =

    Introducing. 2025 , month = dec, howpublished =

    2025

  13. [13]

    2026 , howpublished =

    2026

  14. [14]

    arXiv preprint arXiv:2605.26494 , year=

    The minimax-m2 series: Mini activations unleashing max real-world intelligence , author=. arXiv preprint arXiv:2605.26494 , year=

    Pith/arXiv arXiv

  15. [15]

    Management science , volume=

    An experimental application of the Delphi method to the use of experts , author=. Management science , volume=. 1963 , publisher=

    1963

  16. [16]

    Personality assessment , pages=

    The NEO Inventories1 , author=. Personality assessment , pages=. 2014 , publisher=

    2014

  17. [17]

    Journal of Artifical Societies and Social Simulation (JASSS) vol , volume=

    OPINION DYNAMICS AND BOUNDED CONFIDENCE MODELS, ANALYSIS, AND SIMULATION , author=. Journal of Artifical Societies and Social Simulation (JASSS) vol , volume=

  18. [18]

    , author=

    Social judgment: Assimilation and contrast effects in communication and attitude change. , author=. 1961 , publisher=

    1961

  19. [19]

    Findings of the Association for Computational Linguistics: ACL 2026 , pages=

    Generative interfaces for language models , author=. Findings of the Association for Computational Linguistics: ACL 2026 , pages=

    2026

  20. [20]

    arXiv preprint arXiv:2604.09577 , year=

    Generative UI: LLMs are effective UI generators , author=. arXiv preprint arXiv:2604.09577 , year=

    Pith/arXiv arXiv

  21. [21]

    Transactions on Computational and Scientific Methods , volume=

    Generative UI Design with Diffusion Models: Exploring Automated Interface Creation and Human-Computer Interaction , author=. Transactions on Computational and Scientific Methods , volume=

  22. [22]

    UI2Code\^

    Yang, Zhen and Hong, Wenyi and Xu, Mingde and Fan, Xinyue and Wang, Weihan and Cheng, Jiele and Gu, Xiaotao and Tang, Jie , journal=. UI2Code\^

  23. [23]

    Advances in Neural Information Processing Systems , volume=

    Layoutgpt: Compositional visual planning and generation with large language models , author=. Advances in Neural Information Processing Systems , volume=

  24. [24]

    Design2code: Benchmarking multimodal code generation for automated front-end engineering , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  25. [25]

    Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology , pages=

    UIClip: a data-driven model for assessing user interface design , author=. Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology , pages=

  26. [26]

    International Conference on Learning Representations , volume=

    Prometheus: Inducing fine-grained evaluation capability in language models , author=. International Conference on Learning Representations , volume=

  27. [27]

    arXiv preprint arXiv:2407.00215 , year=

    Llm critics help catch llm bugs , author=. arXiv preprint arXiv:2407.00215 , year=

    Pith/arXiv arXiv

  28. [28]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Large language models are not fair evaluators , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  29. [29]

    arXiv preprint arXiv:2310.10076 , year=

    Verbosity bias in preference labeling by large language models , author=. arXiv preprint arXiv:2310.10076 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2404.18796 , year=

    Replacing judges with juries: Evaluating llm generations with a panel of diverse models , author=. arXiv preprint arXiv:2404.18796 , year=

    Pith/arXiv arXiv

  31. [31]

    Advances in neural information processing systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume=

  32. [32]

    Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

    Generative agents: Interactive simulacra of human behavior , author=. Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

  33. [33]

    Political Analysis , volume=

    Out of one, many: Using language models to simulate human samples , author=. Political Analysis , volume=. 2023 , publisher=

    2023

  34. [34]

    Proceedings of the 41st International Conference on Machine Learning , pages=

    Improving factuality and reasoning in language models through multiagent debate , author=. Proceedings of the 41st International Conference on Machine Learning , pages=

  35. [35]

    Proceedings of the 35th annual ACM symposium on user interface software and technology , pages=

    Social simulacra: Creating populated prototypes for social computing systems , author=. Proceedings of the 35th annual ACM symposium on user interface software and technology , pages=

  36. [36]

    arXiv preprint arXiv:2210.03629 , year=

    React: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    Pith/arXiv arXiv

  37. [37]

    End user development , pages=

    End-user development: An emerging paradigm , author=. End user development , pages=. 2006 , publisher=

    2006

  38. [38]

    All that’s ‘human’is not gold: Evaluating human evaluation of generated text , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages=

  39. [39]

    arXiv preprint arXiv:2307.00184 , year=

    Personality traits in large language models , author=. arXiv preprint arXiv:2307.00184 , year=

    Pith/arXiv arXiv

  40. [40]

    Proceedings of the National Academy of Sciences , volume=

    Can generative AI improve social science? , author=. Proceedings of the National Academy of Sciences , volume=. 2024 , publisher=

    2024

  41. [41]

    Journal of political philosophy , year=

    The law of group polarization , author=. Journal of political philosophy , year=

  42. [42]

    Advanced personality , pages=

    Cognitive-experiential self-theory , author=. Advanced personality , pages=. 1998 , publisher=

    1998