REVIEW 3 major objections 3 minor 3 references
The Role of Urban Designers in the Era of AIGC: An Experimental Study Based on Public Participation
T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read When designers structure prompts and masks, AI-generated urban garden designs score higher and align better with user intent.
desk verdict A real experiment on prompt structure in AI-generated urban design, but the headline claim about 'designer involvement' is not identified by the analysis. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism carrying the argument is the experimental contrast among three designer-involvement levels, operationalized by two controls: prompt format (freeform vs structured prompt modules prepared by designers) and mask format (freeform vs designer-predefined modification areas), applied within a fine-tuned Stable Diffusion pipeline with soft inpainting and two domain knowledge graphs encoded as tags. These controls convert 'designer involvement' into measurable inputs, and the 1-10 aesthetic and alignment ratings from 160 participants convert output quality into comparison data. The structured-prompt control is the load-bearing component: at moderate involvement it produced significantly higher scores than predefined masks, indicating that how design intent is phrased matters more than where edits are allowed.
What would settle it
Run the same three-scene pocket-garden task with a fourth arm in which non-designers write their own structured prompt modules and define their own masks, or use structured prompts generated by a generic prompt-optimizer instead of by designers; if that arm scores as high as the intensive-guidance group, the paper's claim that designers specifically improve output collapses.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that the quality and goal-alignment gap in AIGC-generated urban designs is driven mainly by upstream designer guidance. In the experiment, low-freedom conditions (designer-defined masks plus structured prompt modules) produced the best mean aesthetic and alignment scores; high-freedom conditions (freeform masks and freeform prompts) produced the lowest and most scattered scores. At moderate involvement, structured prompts with freeform masks outperformed freeform prompts with structured masks, and this difference was statistically significant. Post-hoc polishing under freeform prompts raised scores but not significantly, which the paper reads as evidence that early structuring beats later refinement. The paper concludes that designers' value in the AIGC era lies in translating public intentions into structured prompts and bounded modification areas, with professional background playing no significant interaction with guidance effects.
Load-bearing premise
The argument assumes that designer involvement is fully captured by structured prompt modules and predefined mask areas, so that freeform prompts with freeform masks count as 'no designer involvement'—if the quality difference actually comes from prompt format alone, the designer-necessity claim weakens.
Editorial extensions
If this is right
- If designer involvement lifts quality through structured prompts and predefined masks, participatory AIGC workflows should place designers at the input stage, before generation, rather than only in post-processing.
- The finding that non-professionals scored higher under low-freedom conditions implies that deep designer involvement can make public participation more effective, not less.
- Structured prompts being more effective than structured masks at moderate involvement points to prompt-design guidance as the highest-yield intervention for AI-assisted urban design.
- If post-hoc refinement alone does not significantly rescue freeform outputs, then investment in upstream guidance may be more cost-effective than iterative polishing.
Reading between the lines
- Editorial inference: the manipulation equates designer involvement with prompt and mask structure, so the causal claim about designers is only as strong as the assumption that freeform input is a fair proxy for 'no designer'; a direct arm where designers personally supervise generation would test whether the human, rather than the format, carries the effect.
- Editorial inference: the quality gap between structured and freeform prompts may shrink as text-to-image models become better at instruction following, which would push designers' protected role toward tasks the models still miss, such as spatial strategy and cultural context.
- Editorial inference: the same factorial design could be applied to other generative outputs (3D massing, plan layouts, section drawings) and to other participant populations to test whether structured prompts generalize or are specific to pocket-garden aesthetics.
- Editorial inference: because the authors did not measure whether participants' design intent itself changed under structured prompts, a remaining question is whether structured guidance improves alignment by better capturing intent or by steering participants toward designer-approved intentions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports an experimental study of how varying levels of designer involvement affect the aesthetic quality and goal alignment of AIGC-generated urban pocket-garden images. Using a fine-tuned Stable Diffusion model with soft inpainting and a public participation WebUI, the authors recruited 160 participants and collected 1–10 aesthetic and alignment ratings under conditions that differ in prompt structure (freeform vs. structured) and mask predefinition (freeform vs. structured). The stated headline finding is that designer involvement significantly improves the quality and alignment of AIGC-generated images, with the main value of designers lying in providing structured prompts and predefined modification areas. The paper also reports secondary analyses comparing participants with and without design backgrounds, and discusses implications for the future role of urban designers in AI-assisted participatory planning.
Significance. If the central claim were adequately supported, the study would be a useful empirical contribution to human–AI collaboration in urban design and public participation, with practical implications for how AIGC tools should be integrated into participatory workflows. The study's strengths include a relatively large participant sample (160), a priori power analysis, a realistic experimental setup with multiple urban scenarios, and a concrete system (fine-tuned diffusion model plus WebUI) that others could adapt. However, the current statistical and design issues mean the headline causal claim is not established; the paper's contribution at this stage is more of a demonstrated proof-of-concept for a participatory AIGC pipeline than a rigorous test of the role of designers.
major comments (3)
- [Results and discussion (Impact of AIGC on the Role of Designers); Table 1] The abstract and the 'Results and discussion' section state that designer involvement 'significantly enhances the quality and alignment of AIGC-generated images,' but the reported inferential statistics do not support the word 'significantly.' The main effect of guidance degree on aesthetic scores has p = .064 and on alignment scores p = .075 (Table 1), and the Tukey HSD comparison between the low- and high-freedom groups yields p = .055 and .069. All of these exceed the conventional .05 threshold, and the manuscript itself uses 'marginally significant' in the Results section. The conclusions should be reworded to reflect marginal or suggestive evidence, or the authors must provide additional analyses that meet the stated significance criterion.
- [Research Design (Levels of Designer Involvement); Result Analysis with different degrees of freedom] The design is a 2x2 factorial (PromptType × MaskType), but the analysis collapses the four cells into an ordinal 'degree of freedom' factor. This aggregation conflates the two manipulations and prevents identification of which component (prompt structure or mask predefinition) drives the observed quality differences. The only significant contrast reported, 'structured prompt & freeform mask' versus 'freeform prompt & structured mask' (p = .013 and .026), differs in both factors simultaneously, so it cannot be attributed to either factor alone. A two-way ANOVA with PromptType and MaskType as factors, including their interaction, should be reported to separate the effects. Until this is done, the claim that 'structured prompts' (rather than mask predefinition, or some interaction) are the effective ingredient is not supported.
- [Research Design (Levels of Designer Involvement); Hypothesis 2] Hypothesis 2 frames 'designer involvement' as the causal variable, but the manipulation operationalizes designer involvement only through pre-authored prompt modules and pre-defined mask areas. There is no experimental condition in which a designer directly supervises, refines, or iterates on the generation process. The data are therefore equally consistent with an interpretation in which prompt format or mask predefinition alone (independent of any human designer) improve outputs. The causal attribution to 'designers' is underidentified. The authors should either add an experimental arm with direct designer interaction, or reframe the claim as an effect of structured input scaffolding rather than of designer involvement per se.
minor comments (3)
- [Result Analysis with different degrees of freedom] The text says 'the significance level of difference in mean aesthetic scores between various degrees of freedom was 0.065' while Table 1 reports .064 for aesthetic scores; please ensure consistency between the text and table values.
- [Result Analysis with different degrees of freedom] The heading 'Result Analysis with different degrees of freedom' appears twice in the paper, and the phrase 'Comparison of guiding methods in moderate freedom' is followed by a figure labeled 'Bloxplot' (a typo for 'Boxplot'). Please correct the repeated heading and the figure label.
- [Difference analysis before and after polishing of the same guiding method] The term 'polishing' is used interchangeably with 'refinement' in different sections; the 'Extended Analysis' refers to 'refinement' while the results section uses 'polishing.' Please unify the terminology to avoid confusion.
Circularity Check
No circularity found: the study's empirical comparison is not definitionally self-referential, and no fitted parameters, self-citations, or uniqueness claims carry the conclusion.
full rationale
The paper makes no mathematical derivation and fits no parameters; the central claim is a causal interpretation of an experiment in which 'designer involvement' is operationalized as structured prompts and predefined masks, while the outcomes are participant-rated aesthetic and alignment scores. Although this operationalization raises construct-validity questions (designer involvement is conflated with prompt structure, and the low-vs-high contrast is only marginally significant), no load-bearing step reduces to its own input: the independent variable is not defined in terms of the outcome, no result is imported from the authors' prior work, and no uniqueness theorem or ansatz is smuggled in by citation. The comparison between structured-prompt/freeform-mask and freeform-prompt/structured-mask cells is an empirical contrast, not a tautology. Any concern that 'designer involvement' may capture prompt format rather than human expertise belongs to internal validity, not circularity, and therefore does not raise the circularity score.
Assumptions & free parameters
assumptions (4)
- domain assumption Participant ratings on 1 to 10 scales are valid measures of aesthetic quality and alignment with design intent.
- domain assumption The three selected pocket garden scenes are representative of urban roadside green spaces.
- domain assumption Structured prompts and predefined masks cleanly operationalize designer involvement, while freeform inputs operationalize absence of designer involvement.
- domain assumption Stable Diffusion with the authors' fine-tuning and knowledge-graph tags can produce outputs that meet basic urban planning standards.
Cite this review
Pith. "Pith review of The Role of Urban Designers in the Era of AIGC: An Experimental Study Based on Public Participation." pith.science (2026). https://pith.science/paper/LXRAJPEN
@misc{pith2026241117194,
author = {Pith},
title = {Pith review of: The Role of Urban Designers in the Era of AIGC: An Experimental Study Based on Public Participation},
year = {2026},
howpublished = {\url{https://pith.science/paper/LXRAJPEN}},
note = {Machine review of arXiv:2411.17194}
}
read the original abstract
This study explores the application of Artificial Intelligence Generated Content (AIGC) technology in urban planning and design, with a particular focus on its impact on placemaking and public participation. By utilizing natural language pro-cessing and image generation models such as Stable Diffusion, AIGC enables efficient transformation from textual descriptions to visual representations, advancing the visualization of urban spatial experiences. The research examines the evolving role of designers in participatory planning processes, specifically how AIGC facilitates their transition from traditional creators to collaborators and facilitators, and the implications of this shift on the effectiveness of public engagement. Through experimental evaluation, the study assesses the de-sign quality of urban pocket gardens generated under varying levels of designer involvement, analyzing the influence of de-signers on the aesthetic quality and contextual relevance of AIGC outputs. The findings reveal that designers significantly improve the quality of AIGC-generated designs by providing guidance and structural frameworks, highlighting the substantial potential of human-AI collaboration in urban design. This research offers valuable insights into future collaborative approaches between planners and AIGC technologies, aiming to integrate technological advancements with professional practice to foster sustainable urban development.
Reference graph
Works this paper leans on
-
[2021]
Proceedings of the 2021 CHI Conference on Human Factors in Compu- ting Systems, 1–10
Understanding conversational and expressive style in a multimodal embodied conversational agent. Proceedings of the 2021 CHI Conference on Human Factors in Compu- ting Systems, 1–10. Borji, A. 2022. Generated faces in the wild: Quantitative comparison of stable diffusion, midjourney and dall- e.arXiv, 2210.00586. Dolhopolov, S.; Honcharenko, T.; Sachenko,...
arXiv 2021
-
[2022]
Placemaking AI: Participatory Urban Design with Generative Adversarial Networks. Proceedings of the 27th Conference on Computer Aided Architectural Design Re- search in Asia (CAADRIA), 2, 485–494. Lane, M. B. 2005. Public participation in planning: an intel- lectual history. Australian Geographer, 36(3), 283–299. Najafi, M.; and Shariff, M. K. B. M. 2011....
work page 2022
-
[2024]
Proceedings of the 5th In- ternational Workshop IT Project Management (ITPM 2024)
Enhancing Urban Planning with LoRa and GANs: A Project Management Perspective. Proceedings of the 5th In- ternational Workshop IT Project Management (ITPM 2024). Forester, J. 1982. Planning in the Face of Power. Journal of the American Planning Association, 48(1), 67–80. Healey, P. 1992. Planning through debate: The communica- tive turn in planning theory...
work page 2024
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.