{"id":"3abe3abc-e867-4daf-95b0-e5fc53f33478","arxiv_id":"2508.12333","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Sketchar, a ChatGPT and DALL-E based prototyping tool, helped game designers without artistic backgrounds generate reference images and character documents that they rated as more supportive of creativity than sketching alone.","lead":"This paper introduces Sketchar, a tool that lets game designers turn text descriptions into AI-generated character profiles and reference images. A study of designers found that those without art training found the tool more expressive than sketching on their own, though real illustrator collaboration was only simulated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The collaboration benefit is untested: the only quantitative support is a CSI Collaboration score gathered while participants imagined an illustrator, and the paper's own §5.5 concedes the perception may be speculative.","rationale":"The reader's weakest assumption—that the collaboration claim rests on imagined collaboration—is also the most load-bearing concern I find. The paper is unusually transparent about this: §5.5 explicitly states that single individuals used Sketchar, that collaboration had to be imagined, and that the CSI Collaboration subscale may not apply to asynchronous designer–illustrator handoff. That transparency is why this is a conditional concern rather than a fatal flaw. However, the abstract frames the contribution as enhancing communication with illustrators, and the significant Collaboration dimension is the headline quantitative result; if the imagined-handoff perception does not survive a real handoff, the central claim narrows to 'GenAI image generation supports non-artist designers' expression,' which the qualitative findings (KF2, KF3) and the non-artist CSI difference do support. The expert evaluation is post hoc and non-independent, so it does not fill the collaboration gap. A dyadic follow-up with blinded alignment ratings would settle whether the Collaboration CSI score reflects actual communication value or a demand-characteristic response to being told to imagine an illustrator. Because the limitation is acknowledged and the fix is feasible, I keep the reader's CONDITIONAL verdict unchanged.","tokens_in":23419,"tokens_out":4976,"duration_ms":56608,"concrete_test":"Run a dyadic study where designers prepare a handoff artifact for a character: (A) text-only proposal versus (B) Sketchar-generated profile plus reference images. Illustrators, blind to condition, produce a character sketch from the artifact. Independent raters score alignment between the designer's pre-registered intent and the illustrator's output, and the study counts clarification iterations and perceived communicative efficiency. A significant reduction in iterations or higher alignment for Sketchar would support the bridge claim; no difference would indicate that the Collaboration CSI result was an artifact of imagined handoff rather than real communication value.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline contribution is that Sketchar's generated reference images act as a communication bridge between game designers and illustrators, and the quantitative anchor for this is the CSI Collaboration dimension, which 'scored higher than any other dimension' (§4.4). But that anchor was measured under an imaginary handoff. §5.5 states: 'in our testing phase, single individuals worked with Sketchar, meaning that the collaboration had to be imagined to be taking place,' and 'the CSI Collaboration subscale is generally used to evaluate synchronous collaboration with another person... so the perception that the tool supports collaboration may be speculative.' The expert evaluation (§4.5) also does not test a workflow: five illustrators rated whether generated images would be useful references in their workflow, with no designer–illustrator iteration, feedback, or negotiation—the very processes the tool is claimed to facilitate. Thus the load-bearing condition for the central claim—that Sketchar actually improves cross-role communication, not just perceived expressiveness—is untested in the current data. The non-artist expressiveness result (p = .011) is better supported, but it does not establish the communication-bridge claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Sketchar, a web-based generative-AI tool that combines ChatGPT for structuring character profiles and DALL-E for generating reference images, aimed at helping game designers who lack illustration skills communicate character concepts to illustrators. The authors report formative interviews with 10 game design professionals, a qualitative study with 13 game designers, a quantitative study with 17 participants comparing Sketchar against a baseline using the Creativity Support Index (CSI), and an expert evaluation with 5 illustrators who rated generated images as reference material. The main reported findings are that the CSI score for Sketchar was significantly higher than for the baseline (p = .028), that the Collaboration dimension scored highest, that designers without artistic backgrounds rated Sketchar higher than those with artistic backgrounds (p = .011), and that expert illustrators deemed an average of 6.84 out of 10 generated images per prompt usable as references. The paper claims that Sketchar's generated reference images foster refinement of design details and can be incorporated into real-world workflows.","tokens_in":23621,"tokens_out":3197,"duration_ms":36442,"significance":"If the claims are supported, the paper addresses a real and under-served problem: facilitating communication in game character design across the designer-illustrator divide. The formative interview grounding, the mixed-method design, the randomized task order in the quantitative study, and the candid acknowledgement of limitations are strengths. The qualitative findings (KF1-KF3) provide plausible, concrete evidence that non-artist designers feel better able to express and refine character ideas with generated references. The non-artist expressiveness result is a useful contribution. However, the headline communication-bridge claim rests on a CSI Collaboration subscale administered under an imagined asynchronous handoff and on an expert evaluation that did not involve a designer-illustrator workflow; as written, those data support perceived usefulness of generated images, not demonstrated improvement in cross-role collaboration.","major_comments":[{"comment":"The central collaboration claim is not supported by the reported data. The quantitative anchor for the communication-bridge contribution is the CSI Collaboration dimension, which 'scored higher than any other dimension' (§4.4), but the collaboration subscale is designed for synchronous collaboration with another person, and the authors told participants to imagine handing images to an illustrator. The paper's own §5.5 states that 'the collaboration had to be imagined to be taking place' and that 'the perception that the tool supports collaboration may be speculative.' Because no designer-illustrator pair worked together, the significant overall CSI difference and the high Collaboration subscale do not establish that Sketchar actually improves cross-role communication; they establish that participants perceived it might.","section":"§4.4 and §5.5"},{"comment":"The expert evaluation does not test a real-world workflow. Five illustrators rated pre-selected generated images and found an average of 6.84 of 10 images per set suitable as references, but they did not receive the designers' design documents, were not given feedback or revision requests, and did not interact with the designers. This procedure can show that experienced illustrators see some utility in the images, but it cannot support the abstract's stronger claim that the images 'can be incorporated into real-world workflows.' A workflow claim requires at least one complete designer-to-illustrator handoff, iteration, or negotiation.","section":"§4.5"},{"comment":"The baseline condition is described inconsistently, and this weakens the causal interpretation of the p = .028 CSI result. The abstract and introduction describe the control as a 'sketching-only' baseline, but the methods section says participants were asked to 'submit a character design proposal to the artist based on their experience and abilities' (§4.1.2 and §4.2.2), and §5.5 refers to the control as 'participants' original workflows.' If the baseline was not constrained to sketching and was not matched in structure, the significant CSI difference may reflect any structured digital workflow support rather than the specific GenAI-mediated features of Sketchar. The paper should clarify exactly what the baseline condition was and temper causal phrasing accordingly.","section":"§4.2.2 and Abstract"},{"comment":"Factor-level claims about the Collaboration dimension are asserted without reported statistical detail. The text says a Tukey test showed Collaboration scores higher than other dimensions but then states 'We did not statistically test each factor score. Instead, we compare them directly.' No test statistics, adjusted p-values, or effect sizes are given for the individual CSI dimensions. Given that the Collaboration subscale is the load-bearing evidence for the paper's main claim, the authors should either report the supporting statistics or explicitly frame the dimension comparisons as descriptive.","section":"§4.4"}],"minor_comments":[{"comment":"The final paragraph of §4.4 uses 'Sketcher' instead of 'Sketchar' in the sentence about the exploration dimension; please correct the spelling for consistency.","section":"§4.4"},{"comment":"The caption says the displayed results involve 'six experts (P1, P2, P3, P9, P10, and P11),' but P1-P13 are identifiers from the qualitative user study, not the expert evaluators (E1-E5). Clarify whether these are user-study participants or illustrators.","section":"Figure 8 caption"},{"comment":"The experience categories in Table 2 are ambiguous: 'No more than 3 years' and 'No less than 1 year' overlap and do not partition the sample clearly. Use mutually exclusive ranges such as 'less than 1 year,' '1-3 years,' and 'more than 3 years.'","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is suitable for the venue and addresses a timely topic, but the main communication-bridge claim currently outruns the evidence. The authors could either add a genuine designer-illustrator collaboration study or substantially reframe the abstract and discussion to present the collaboration finding as a perceived-benefit hypothesis raised by the data. I would also encourage them to tighten the baseline description so that the quantitative comparison is transparent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things about this paper. The solid result is the non-artist expressiveness finding: game designers without artistic training gave Sketchar significantly higher CSI scores than the sketching-only baseline, and the qualitative interviews back that up. That is a genuine, useful contribution. The headline collaboration claim — that generated reference images bridge designer–illustrator communication — is not actually tested. The CSI Collaboration score came from participants imagining handing images to an illustrator, and the paper's own §5.5 says the perception 'may be speculative.' Read the abstract accordingly.\n\nWhat's new: Sketchar wires ChatGPT and DALL-E into a hierarchical prompt pipeline, with editable profiles, simulated character chat, and family trees. The formative interviews with ten professionals give a credible basis for the design goals. The studies are reasonable for an initial systems paper: 13 qualitative participants, 17 quantitative, plus five expert illustrators who judged about 68% of generated images as usable references. The stats are modest but fine for this kind of work.\n\nSoft spots, in order of severity. First, the communication-bridge claim carries the paper's significance but is unsupported by the current data; a real designer–illustrator iteration study is needed. Second, the baseline is sketching-only, not raw GenAI, so you can't tell whether Sketchar's added structure beats just prompting DALL-E. The authors acknowledge both of these in §5.5, which is credit to them. Third, no code or data is released, and the statistical reporting is thin — no effect sizes, factor-level comparisons are visual only. Sample is small and all-Chinese, which they also note. None of these are fatal, but they cap how strongly the conclusions can be stated.\n\nWho this is for: anyone working on GenAI co-creativity or creativity support tools. It's a legitimate CHI PLAY paper that would benefit from a revision that either tests real collaboration or reframes the contribution around non-artist expressiveness. I'd send it to peer review.","headline":"Sketchar's real finding is that non-artist designers feel more expressive with it; the collaboration benefit is asserted, not shown.","tokens_in":24134,"tokens_out":2832,"would_cite":true,"duration_ms":29704,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A generative-AI character prototyping tool gives non-artist game designers a concrete visual brief to hand to illustrators, and a study finds it beats sketching alone on creativity support.","keywords":["game character design","generative AI","creativity support index","human-AI collaboration","text-to-image generation","designer-illustrator communication","rapid prototyping","large language models"],"falsifier":"Run the same character-design task with pairs of real designers and illustrators, giving half of the pairs a sketching-only brief and half a Sketchar brief, and count revision rounds and requirement mismatches until the illustrator's final asset is approved; if Sketchar does not reduce the number of iterations or mismatches, the claimed communication-bridge benefit is not supported.","tokens_in":23243,"feed_emoji":"🎮","tokens_out":6916,"duration_ms":68268,"temperature":0.7,"pith_summary":"This paper claims that a generative-AI tool can close a specific communication gap in game development: the gap between designers, who think in narrative and mechanics, and illustrators, who work in visual form. The tool, Sketchar, turns a designer's natural-language description into a structured character profile and a set of reference images, giving the designer something visual to hand to an illustrator even when the designer cannot draw. In a mixed-method study, designers rated the Sketchar workflow significantly higher than a sketching-only baseline on the Creativity Support Index, especially on collaboration, and designers without artistic backgrounds rated it higher than artists did. The paper concludes that such reference images refine design details and can enter real-world design workflows.","feed_headline":"AI prototype tool beats sketching for non-artist character designers","feed_subtitle":"CSI scores beat the sketching baseline, especially on collaboration; experts found most AI images usable.","key_machinery":"The load-bearing mechanism is a hierarchical content-generation pipeline driven by prompt engineering. In the first layer, a large language model takes the designer's raw input (name, role, background story, game type) and summarizes it into integrated information; in the second layer it distills keywords; in the third layer those keywords, plus a chosen render style and role details, are sent to a text-to-image model that produces five reference images. A human-in-the-loop step lets the designer edit the profile or regenerate, and the result is packaged as a shareable character \"ID card,\" with optional simulated conversations and a character family tree to support exploration. The identity doing the work is this translation from narrative intent to visual prototype through an intermediate keyword representation, which is also what makes the artifact legible to an illustrator.","core_discovery":"The central claim is that a designer who can describe a character in words can, with Sketchar's help, produce both a structured design document and a set of reference images that an illustrator can use as a starting point, and that this capability measurably improves the creative-design experience over sketching alone. The evidence is a two-part study: with 13 designers in qualitative sessions and 17 in a within-subjects quantitative comparison, Sketchar's Creativity Support Index total significantly exceeded the baseline (p = .028), with the Collaboration dimension the highest factor; non-artists scored Sketchar significantly higher than artists did (p = .011), especially on Expressiveness and Results Worth Effort. Five professional illustrators judged, on average, 6.84 of 10 images in each generated set as relevant reference material for real game character development. The paper frames this as a new application of generative AI: not replacing the illustrator, but giving the non-artist designer a concrete visual object to think with and hand off.","pith_inferences":["Beyond the paper, the same two-stage \"words to keywords to image\" pipeline could be tested in other interdisciplinary handoffs, such as UX designer to developer or writer to animator, where one party lacks the other's medium.","Because the artist/non-artist gap was large, a testable prediction is that Sketchar's adoption and measured benefit will scale with the skill distance between the designer and the illustrator, not with overall team size.","The paper leaves to future work a direct comparison with a raw LLM-plus-image-generator workflow; the natural experiment is to see whether the structured profile and ID-card packaging add value over the same models used directly.","If real designer-illustrator pairs are used, the Collaboration subscale should be validated against an objective outcome such as number of revision rounds or illustrator comprehension score, since the Creativity Support Index collaboration scale normally assumes synchronous collaboration."],"forward_implications":["A designer who cannot draw can nevertheless hand an illustrator a concrete visual brief, changing the handoff from words-and-hope to image-plus-document.","Because the Collaboration dimension drove the Creativity Support Index gain, designers perceive the tool as supporting the designer-illustrator relationship even when no illustrator is physically present.","With expert illustrators accepting most generated images as references, the images can plausibly enter professional pipelines as early-stage prototypes rather than as final art.","The structured profile generated alongside the image gives the design team a standardized character document, addressing the complaint that design briefs vary in format and clarity.","The tool lends itself to batching near-identical NPC variants, an efficiency the paper identifies as a natural next use."],"supporting_citations":[{"why":"Defines the designer-illustrator collaboration and the miscommunication problem that motivates the tool.","marker":"[19]"},{"why":"Supplies the Creativity Support Index survey used to compare Sketchar against the sketching baseline.","marker":"[13]"},{"why":"Describes the We-toon writer-artist communication system, the closest comparison for Sketchar's mediation claim and the model for its future collaborative study.","marker":"[35]"},{"why":"Documents professional artists' criticisms of large-scale text-to-image models, explaining the lower ratings from artistically trained participants.","marker":"[36]"},{"why":"CharacterMeet, the closest existing LLM-plus-visual character construction tool, used to position Sketchar's profile-based iteration.","marker":"[59]"},{"why":"CharacterChat, a chatbot-based character creation system without visual generation, used as a contrast for Sketchar's combined text-and-image output.","marker":"[62]"},{"why":"Compares face-generation quality across image models, supporting the choice of the text-to-image model and the acknowledged image-detail limitation.","marker":"[7]"},{"why":"Provides the thematic-analysis procedure used to code the formative interviews and user-study transcripts.","marker":"[10]"}],"fun_headline_variants":["GenAI tool turns non-artists into character design collaborators","Sketchar: AI assist for character design boosts non-artist collaboration","Non-artist designers create stronger characters with AI sketching aid","AI prototype tool empowers designers without art skills to communicate visually","Character design AI tool helps illustrators and designers speak the same language"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The key claim rests on participants' imagined collaboration with an illustrator; designers were told to imagine the handoff rather than work with an actual illustrator, and if that imagined handoff does not resemble real designer-illustrator iteration, the communication-bridge benefit would not be established.","fun_headline_variants_meta":{"raw":{"variants":["GenAI tool turns non-artists into character design collaborators","Sketchar: AI assist for character design boosts non-artist collaboration","Non-artist designers create stronger characters with AI sketching aid","AI prototype tool empowers designers without art skills to communicate visually","Character design AI tool helps illustrators and designers speak the same language"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000247,"raw_usage":{"total_tokens":1527,"prompt_tokens":916,"completion_tokens":611,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":525}},"tokens_in":532,"tokens_out":611,"duration_ms":5814,"temperature":1.0,"reasoning_tokens":525,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:22:33.472391+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same character-design task with pairs of real designers and illustrators, giving half of the pairs a sketching-only brief and half a Sketchar brief, and count revision rounds and requirement mismatches until the illustrator's final asset is approved; if Sketchar does not reduce the number of iterations or mismatches, the claimed communication-bridge benefit is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the designer-illustrator collaboration and the miscommunication problem that motivates the tool."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes the We-toon writer-artist communication system, the closest comparison for Sketchar's mediation claim and the model for its future collaborative study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents professional artists' criticisms of large-scale text-to-image models, explaining the lower ratings from artistically trained participants."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CharacterMeet, the closest existing LLM-plus-visual character construction tool, used to position Sketchar's profile-based iteration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CharacterChat, a chatbot-based character creation system without visual generation, used as a contrast for Sketchar's combined text-and-image output."}],"review_version":2}