{"id":"3a4f7375-6567-4d58-abd0-e4570273c257","arxiv_id":"2502.07556","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SketchFlex combines sketch-aware prompt recommendation with decompose-and-recompose shape refinement to help novices generate multi-object images from rough region sketches.","lead":"SketchFlex is an interactive tool that turns rough region sketches and simple text into detailed, spatially arranged images by auto-completing prompts and refining object shapes. A 12-person study reports better alignment and coherence than text-only or region-only baselines, though the abstract's cognitive-load claim is not directly measured.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation confounds prompt recommendation with manual single-object adjustment; cognitive-load claim lacks any direct measure.","rationale":"The reader's weakest assumption concerns GPT-4o spatial reasoning and dataset priors; that is a real robustness risk, but the more load-bearing issue is that the study design cannot attribute the measured gains to the proposed mechanism at all. Even a perfect prompt recommender would not be supported by the reported comparison, because the R2I baseline omits the single-object decomposition and adjustment features that directly manipulate the spatial layout and therefore the IoU metric. The paper's own qualitative evidence shows these manual controls affect the output (Sec 5.4.2, Figure 11) and are the features users most credit (Sec 5.3.2). Given the abstract's strong causal language about reducing cognitive load and matching intentions, the absence of a direct cognitive-load measure and the lack of any component-level quantitative comparison mean the evidence is consistent with a much weaker claim: a system with extra manual spatial controls outperforms a baseline without them. I recommend keeping the CONDITIONAL verdict but adding specific conditions: isolate the prompt recommendation, and measure cognitive load directly. My concern agrees partially with the reader: both point at the semantic-space mechanism, but the reader focuses on LLM unreliability while I focus on the evaluation's inability to isolate that mechanism.","tokens_in":26355,"tokens_out":5737,"duration_ms":50768,"concrete_test":"Run a component ablation user study (N >= 12, counterbalanced) with four conditions: (a) R2I baseline, (b) R2I + prompt recommendation only, (c) R2I + single-object adjustment only, and (d) full SketchFlex, keeping the same generation backbone and seeds across conditions. Measure Task 1/Task 2 IoU and a validated cognitive-load scale such as NASA-TLX. If (b) does not significantly beat (a), the semantic-space mechanism is unvalidated; if (c) is close to (d), manual adjustment, not prompt recommendation, drives the benefit. Recompute Sec 5.3 subjective ratings under this decomposition to see which feature users credit for intention alignment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central technical contribution is the sketch-aware prompt recommendation built on the semantic space (Sec 4.1), but the user study (Sec 5) varies at least three independent factors simultaneously between SketchFlex and the R2I baseline: automatic prompt recommendation, single-object decomposition/refinement, and single-object adjustment (scale/position). The close-ended task scores IoU against a reference image, and the single-object adjustment stage lets users directly move and resize masks to match that reference, which can improve IoU without any semantic prompt improvement. Sec 5.4.2's Figure 11 shows that single-object adjustment alone changes the composition, and Sec 5.3.2 quotes users attributing intention alignment to precise shape fixing and spatial adjustments rather than prompt content. The ablations in Figures 5 and 11 are qualitative only, with no metrics or significance tests. Consequently, the component the paper names as its key novelty is never isolated in a quantitative condition, so the reported SketchFlex advantage may be due to the extra manual spatial controls and shape anchoring rather than to the semantic-space prompt recommendation. In addition, the abstract claims significantly reducing cognitive load, yet no validated cognitive-load instrument (e.g., NASA-TLX) appears in Sec 5; the support is informal participant comments about time savings. The headline claim is therefore underdetermined.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SketchFlex is an interactive system for rough-sketch region-based text-to-image generation. It combines sketch-aware prompt recommendation, in which GPT-4o completes a semantic space populated with attributes and relationships from Visual Genome and VAW, with a decompose-and-recompose sketch refinement pipeline that generates per-object shape candidates, lets users adjust their masks, and anchors the final generation through Canny/ControlNet. The paper reports a 12-participant user study comparing SketchFlex with a text-to-image baseline and a region-to-image baseline on closed-ended tasks (IoU and subjective ratings) and an open-ended creative task. The authors claim that SketchFlex produces more cohesive images, better matches user intentions, and significantly reduces cognitive load relative to the region-based baseline.","tokens_in":26638,"tokens_out":3121,"duration_ms":30712,"significance":"If the reported results hold, SketchFlex provides a practical and novel interaction pathway for novice users to control object placement, shape, and inter-object relationships in diffusion-based image generation. The release of code and data is a concrete strength, and the semantic-space formulation is a reasonable way to connect rough sketches to coherent prompts. The core ideas are plausible and the qualitative examples are compelling. The significance is currently conditional, however, because the evaluation does not isolate the contribution of the sketch-aware prompt recommendation, the harder-task IoU advantage is not statistically significant, and the abstract's cognitive-load claim is not supported by any direct workload measurement.","major_comments":[{"comment":"The R2I baseline is not an independent system: it is the same SketchFlex application with the advanced features disabled. The close-ended IoU task additionally rewards the manual single-object adjustment stage, because participants can move and resize masks to match the reference image, and this can improve IoU even if the prompt recommendation contributes nothing. The paper's named central contribution, sketch-aware prompt recommendation, is never evaluated in isolation: the ablations in Figs. 5 and 11 are qualitative only, with no metrics or significance tests. Please add a condition that isolates prompt recommendation from the manual shape-adjustment controls, or report quantitative ablations on the same backbone and tasks.","section":"§5.1.3, §5.2, §5.4.2"},{"comment":"On the harder, more representative Task 2, SketchFlex's IoU (0.456) is not significantly better than the R2I baseline (0.396, p = 0.19). The overall headline advantage therefore rests mainly on Task 1, where the fixed two-object setup is easier and where manual mask adjustment can trivially improve IoU. Report effect sizes and confidence intervals for each pairwise comparison, and discuss the statistical power of the n = 12 design when making broad claims about superiority.","section":"§5.2, Table 3"},{"comment":"The abstract's claim that SketchFlex \"significantly reduc[es] cognitive load\" is not supported by the reported measurements. Section 5 contains no NASA-TLX or other validated workload instrument; the evidence consists of feature-specific Likert items and interview comments about time savings, which conflate perceived helpfulness with cognitive workload. Either add a direct workload measure or revise the abstract and conclusions to remove the cognitive-load claim.","section":"Abstract, §5.4.1"}],"minor_comments":[{"comment":"There is a typo in \"reseach\" in the opening sentence of Section 2.3.","section":"§2.3"},{"comment":"The participant experience description is internally inconsistent: it says 2 participants had more than one year of experience and 1 participant had over a year of experience. Please clarify the intended categories.","section":"§5.1.1"},{"comment":"Section 5.1.4 describes Task 2 as involving four main objects, while Section 5.2 says Task 2 features three smaller objects. Please reconcile the object count.","section":"§5.1.4, §5.2"},{"comment":"The feature-influence examples in Figure 11 are informative, but the figure would benefit from explicit labels indicating which row corresponds to which task and which changes were made between columns; this would make the qualitative ablation easier to interpret.","section":"§5.4.2, Figure 11"},{"comment":"The acknowledged failure cases (three or more similar objects, thin strokes, inclusion relationships) directly bound the central claim about multi-object coherence; please state in the Discussion how these limitations affect the scope of the claimed advantage.","section":"§6.6"}],"recommendation":"major_revision","confidential_remarks":"The paper is squarely within CHI's scope and the system is likely to interest the community. The main issue is evaluation rigor: the fixed tool order, the self-referential R2I baseline, the unisolated prompt-recommendation component, the non-significant Task 2 IoU result, and the unsupported cognitive-load claim together mean the abstract overstates what is demonstrated. I believe these can be fixed within the manuscript's scope by adding an ablation condition, reporting effect sizes, and toning down the cognitive-load language."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper builds a coherent system: sketch-aware prompt recommendation grounded in Visual Genome/VAW, decompose-and-recompose shape refinement, and an interface that lets novices adjust individual object masks. That integration is genuinely new, and the formative study plus the qualitative examples give a decent picture of how the tool works in practice. The user study shows large IoU gains over T2I and R2I baselines, and the subjective ratings are consistently in SketchFlex's favor. For a CHI-style systems paper, this is a solid core.\n\nThe soft spots are real but not fatal. The stress-test note is on target: the R2I baseline is the same app with SketchFlex's features disabled, so the comparison varies prompt recommendation, single-object decomposition, and single-object adjustment all at once. The single-object adjustment stage lets users move and resize masks to match the reference image, which can raise IoU on its own, independent of any semantic prompt improvement. The paper never isolates its headline component, the semantic-space prompt recommendation, in a quantitative condition. The ablations in Figures 5 and 11 are qualitative only. So the reported SketchFlex advantage is underdetermined.\n\nThe cognitive-load claim in the abstract is also overreach. No validated instrument like NASA-TLX appears in Section 5; the support is informal participant comments about time savings and facility. That is a weaker basis than the abstract implies. And in Task 2, the IoU gain over R2I is not significant (p=0.19), a fact the paper reports honestly but does not emphasize enough in its conclusions.\n\nThe paper itself acknowledges several of these issues in Section 6.6 — learning effects, thin-stroke failures, missing objects with similar semantics. That honesty is a point in its favor; the work is not trying to hide its limits.\n\nWho is this for? Researchers working on human-AI co-creation tools for image generation, especially novice-friendly spatial control. They will find the system design and the qualitative feedback useful, and the semantic-space construction via crowd-sourced datasets is worth borrowing. The evaluation lessons also matter: a comparison that varies multiple features at once cannot support a claim about any single feature.\n\nMy recommendation: send it to peer review, but require the authors to either add a condition that isolates prompt recommendation from the manual adjustment controls, or substantially soften the claims. The cognitive-load sentence should be revised to say the system was perceived as less demanding, without a direct measurement. With those changes, the paper is a reasonable contribution. As it stands, the central designed component is not quantitatively validated.","headline":"A plausible systems contribution with a real integration story, but the evaluation confounds its own components and the cognitive-load claim needs a direct measure.","tokens_in":27147,"tokens_out":1287,"would_cite":false,"duration_ms":14550,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SketchFlex claims rough region sketches plus auto-refined prompts and shape anchors produce more cohesive, intention-aligned images than text-to-image or region-to-image baselines.","keywords":["text-to-image generation","region-based sketch control","spatial-semantic coherence","prompt recommendation","diffusion models","interactive image generation","decompose-and-recompose","human-AI co-creative tools"],"falsifier":"A controlled ablation that gives the multimodal language model only the rough sketch and the semantic-space template, without the retrieved attribute and relationship examples, would settle whether the crowd-sourced data carries the coherence gain; the paper's own Figure 5 ablation compares full SketchFlex against no-prompt-recommendation and no-sketch-refinement conditions, but not retrieval alone. A second check would count missing-object and wrong-relationship errors on a fixed sketch set where thin strokes and inclusion relationships are deliberately included, testing whether the decompose-and-recompose shape anchoring can handle those cases even with perfect prompts.","tokens_in":26167,"feed_emoji":"🎨","tokens_out":5078,"duration_ms":43708,"temperature":0.7,"pith_summary":"The paper introduces SketchFlex, an interactive system that lets novice users specify an image by drawing rough colored region sketches and typing a few words, then automatically fills in detailed, spatially coherent prompts and refines the rough shapes into precise anchors. The central claim is that this two-step intervention, semantic prompt completion plus shape refinement, makes generated images more cohesive and better matched to user intent than either pure text-to-image generation or standard region-based generation, while lowering the mental effort of prompt crafting. If true, it gives non-experts a practical way to control object placement, shape, and relationships in diffusion-model image generation without learning depth maps, canny references, or prompt-engineering skills. The paper reports a 12-participant study in which SketchFlex achieved higher IoU alignment (0.535 overall, versus 0.314 for text-to-image and 0.418 for region-to-image) and significantly higher subjective ratings on image quality, cohesiveness, and intention alignment.","feed_headline":"Rough region sketches beat text-only prompts at image layout control","feed_subtitle":"Auto-filled prompts and shape anchors lift object alignment IoU to 0.535, with higher user-rated cohesion.","key_machinery":"The load-bearing mechanism is semantic-space-guided prompt completion coupled with decompose-and-recompose shape anchoring. The semantic space is a structured vocabulary covering type, attribute, state, direction, relationship, lighting, and style, used to organize per-region prompts; a multimodal large language model fills it from the sketch and initial prompt, while retrieval from crowd-sourced image-annotation datasets supplies realistic attribute and relationship references. On the shape side, the rough sketch is decomposed into individual object regions, candidate refined shapes are generated for each object, filtered by IoU and CLIP score, and presented to the user for selection and spatial adjustment. The selected masks are recomposed into a joint mask, and the canny edges of that anchor are fed into a ControlNet-conditioned diffusion model, with a cross-attention update strengthening relationship prompts inside joint masks and suppressing them outside. This decompose-and-recompose loop is what converts rough scribbles into precise anchors while keeping iterative refinement local to individual objects.","core_discovery":"SketchFlex's core claim is that the two bottlenecks for novice text-to-image users, writing prompts that specify relationships between multiple objects and preparing precise spatial conditions, can be removed by a pipeline that first uses a multimodal large language model, guided by a curated semantic space and retrieved crowd-sourced object attributes and relationships, to turn a rough color-coded sketch and minimal prompt into complete region prompts. The pipeline then refines each foreground object through a decompose-and-recompose strategy: generate candidate single-object images, filter them by IoU and CLIP score, let the user select and adjust masks, and recompose the selected shapes into a joint mask whose canny edges condition the diffusion model. The paper argues that the resulting images are more cohesive and more aligned with user intentions than end-to-end text-to-image and region-based baselines, with the user study reporting IoU of 0.535 versus 0.314 and 0.418 respectively, and significantly higher subjective ratings on all three outcome dimensions.","pith_inferences":["The pipeline's ceiling likely tracks the spatial reasoning ability of the underlying multimodal language model, so swapping in open-weight models would test whether the semantic-space scaffolding carries the benefit or whether the LLM's sketch reading is essential.","The decompose-and-recompose shape-anchoring step could be reused as a general front-end for other spatial conditioning backbones, not only canny-based ControlNet.","The failure cases named in Section 6.6, thin strokes, inclusion relationships, and three or more similar objects, imply the system's benefit is strongest for clear, well-separated objects; extending to overlapping or contained objects likely needs layer or depth control rather than more prompt tuning.","A larger user study with fixed layout constraints rather than open reference images could separate tool capability from user skill, since the reported variability in Task 2 suggests subjective alignment choices partly drive the IoU differences."],"forward_implications":["Novice users can produce images with specific object arrangement and interaction without learning depth maps, canny references, or segmentation controls.","Iterative editing becomes local: changing one object's shape or position need not regenerate the whole composition.","Prompt recommendation reduces the cognitive load of per-region prompt writing, since users can rely on auto-filled attributes, states, and relationships.","The same backbone model can serve text-to-image, region-to-image, and SketchFlex conditions, isolating the effect of the interaction design from model quality.","The performance gap narrows on complex multi-object scenes, with Task 2 IoU of 0.456 versus 0.396 and a non-significant p-value of 0.19, indicating limits with three or more small objects."],"supporting_citations":[{"why":"Visual Genome supplies the crowd-sourced object attributes and relationships used as reference priors in prompt recommendation.","marker":"[35]"},{"why":"VAW provides visual attribute annotations that enrich the semantic space for the multimodal language model.","marker":"[50]"},{"why":"Dense Diffusion is the state-of-the-art region-based generation method used as the R2I baseline in the user study.","marker":"[30]"},{"why":"MultiDiffusion is the region-based approach whose failure cases, such as missing objects and unnatural relationships, motivate SketchFlex's decompose-and-recompose design.","marker":"[10]"},{"why":"Segment Anything is used to extract refined object masks during the recompose stage and to segment evaluation masks for IoU measurement.","marker":"[31]"},{"why":"Fast SAM accelerates single-object decomposition and mask extraction during the refinement process.","marker":"[70]"},{"why":"Canny edge detection produces the shape anchors that condition the diffusion model through ControlNet.","marker":"[14]"},{"why":"The ColorfulXL-Lighting model is the shared backbone across all comparison conditions, ensuring consistent image quality and style.","marker":"[4]"}],"fun_headline_variants":["SketchFlex: Rough sketches beat text prompts for multi-object layout control","Auto-filled prompts from rough sketches boost image cohesion","Rough sketch to canny anchors: IoU up to 0.535 in T2I","SketchFlex: Less prompt effort, better object alignment in images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline depends on the multimodal language model correctly reading a color-coded rough sketch, and on the retrieved real-world attributes and relationships being relevant priors, so if the model misreads the sketch or the dataset examples are off, the recommended prompts can insert wrong objects or unnatural relationships instead of fixing them.","fun_headline_variants_meta":{"raw":{"variants":["SketchFlex: Rough sketches beat text prompts for multi-object layout control","Auto-filled prompts from rough sketches boost image cohesion","Rough sketch to canny anchors: IoU up to 0.535 in T2I","SketchFlex: Less prompt effort, better object alignment in images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000375,"raw_usage":{"total_tokens":1982,"prompt_tokens":910,"completion_tokens":1072,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":993}},"tokens_in":526,"tokens_out":1072,"duration_ms":7532,"temperature":1.0,"reasoning_tokens":993,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:19:57.007155+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled ablation that gives the multimodal language model only the rough sketch and the semantic-space template, without the retrieved attribute and relationship examples, would settle whether the crowd-sourced data carries the coherence gain; the paper's own Figure 5 ablation compares full SketchFlex against no-prompt-recommendation and no-sketch-refinement conditions, but not retrieval alone. A second check would count missing-object and wrong-relationship errors on a fixed sketch set where thin strokes and inclusion relationships are deliberately included, testing whether the decompose-and-recompose shape anchoring can handle those cases even with perfect prompts.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"VAW provides visual attribute annotations that enrich the semantic space for the multimodal language model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dense Diffusion is the state-of-the-art region-based generation method used as the R2I baseline in the user study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MultiDiffusion is the region-based approach whose failure cases, such as missing objects and unnatural relationships, motivate SketchFlex's decompose-and-recompose design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Fast SAM accelerates single-object decomposition and mask extraction during the refinement process."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Canny edge detection produces the shape anchors that condition the diffusion model through ControlNet."},{"cited_title":"colorful-realistic-xl-v1-sdxl","cited_arxiv_id":null,"evidence_quote":"The ColorfulXL-Lighting model is the shared backbone across all comparison conditions, ensuring consistent image quality and style."}],"review_version":1}