{"id":"879eac32-5025-4eaa-92a7-76310c4f6fa2","arxiv_id":"2502.08642","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A conditional diffusion model generates image-conditioned vector sketches in under a second by denoising stroke coordinates, trained on a synthetic dataset produced by a new SDS-based optimizer with depth ControlNet.","lead":"SwiftSketch is a diffusion model that turns an input image into a vector sketch in under a second, by denoising stroke control points instead of pixels. It is trained on a synthetic dataset of 35,000 image-sketch pairs produced by a new slow-but-high-quality optimizer called ControlSketch, and it could enable interactive sketching tools and fast sketch-data generation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quality claim rests on a self-referential evaluation: SwiftSketch is never in a user study, ControlSketch is treated as ground truth, and CLIP is both conditioner and metric; external CLIP Top-1 drops to 0.56, so 'high-quality, generalizing' is not established.","rationale":"I agree with the reader's conditional verdict, but I locate the load-bearing weakness slightly differently. The reader emphasizes that the synthetic training set may inherit ControlSketch's biases or artifacts. That is a real concern, but the more immediate and testable problem is that the paper never directly demonstrates that SwiftSketch's outputs are high quality to humans. The quantitative evaluation uses ControlSketch as the ground truth, uses CLIP both as the conditioning backbone and the main recognition metric, and omits SwiftSketch from the user study. These choices make the quality claim partly self-referential. The external generalization gap (CLIP Top-1 0.56 on SketchyCOCO, versus 0.95 on seen categories) is consistent with the worry that SwiftSketch has learned a narrow synthetic distribution rather than a general image-to-vector-sketch mapping. None of this is internally inconsistent, and the speed claim is well supported, so the original CONDITIONAL verdict remains appropriate. The proposed user study would settle whether the quality claim survives direct human evaluation; if it does not, the verdict should move toward REJECT or at least require the authors to supply the missing evidence before acceptance.","tokens_in":18220,"tokens_out":4246,"duration_ms":41223,"concrete_test":"Run a 2AFC perceptual study with at least 40 participants comparing SwiftSketch, ControlSketch, and CLIPasso on the same 24 categories used in the existing user study (Sec. E) plus 8 SketchyCOCO categories. Ask both 'which better depicts the input image' and 'which is higher quality,' and report preference fractions with confidence intervals. If SwiftSketch is not preferred over CLIPasso or is not within a small margin of ControlSketch on these questions, the central quality claim fails. As a secondary check, have artists rate a random subset of ControlSketch sketches against a professional sketch corpus such as OpenSketch; if ControlSketch is not rated as professional, the dataset premise underlying SwiftSketch is invalid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two parts: speed and quality. Speed is credible: 50 diffusion steps plus one refinement pass, about 0.5 s (Sec. 4.4, Table 1). Quality is the weak link. SwiftSketch is trained to mimic ControlSketch on a 35k synthetic dataset produced by the authors' own SDS-with-depth-ControlNet pipeline (Sec. 4.2). ControlSketch is then called 'the ground truth in our case' (Sec. 5.1), and it is the only optimization method evaluated against humans; the user study in Sec. E compares ControlSketch with CLIPasso only, not SwiftSketch. The Table 1 metrics are also partially circular: the conditioning encoder is a frozen CLIP ResNet (Sec. 4.3) and the main recognition metric is a CLIP zero-shot classifier. This makes it possible that SwiftSketch's 'high quality' is actually 'high recognizability on synthetic SDXL images under a CLIP-based metric.' The external-data numbers support this worry: CLIP Top-1 falls from 0.95 seen / 0.70 unseen to 0.56 on SketchyCOCO, while ControlSketch itself falls only to 0.91. So on real images, both teacher and student fail to maintain the claimed quality; Sec. 7 concedes unrecognizable failure cases and over-simplification. Unless the evaluation is validated by direct human assessment of SwiftSketch outputs and by an independent check of ControlSketch's quality, the headline claim that SwiftSketch 'approaches' optimization-based quality in general is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SwiftSketch, a diffusion model that generates vector sketches by denoising stroke control points sampled from a Gaussian, conditioned on a frozen CLIP image embedding through a transformer decoder. To obtain training data, the authors introduce ControlSketch, an SDS-based optimization method augmented with a depth-aware ControlNet, and use it to synthesize over 35,000 image-sketch pairs across 100 classes. SwiftSketch is trained on 15 classes with a fixed stroke count of 32, produces sketches in about 0.5 seconds, and is compared against CLIPasso, Photo-Sketching, Chan et al., InstantStyle, and ControlSketch using CLIP recognition accuracy, MS-SSIM, and DreamSim. The paper claims high-quality, generalizable vector sketch generation in under a second.","tokens_in":18515,"tokens_out":4365,"duration_ms":42447,"significance":"The speed claim is credible and practically valuable: replacing a 5-10 minute optimization with a 0.5 second feed-forward pass, while retaining vector output, would make real-time interactive sketching feasible. The synthetic dataset pipeline is also a useful contribution, and the paper states that data and code will be released. However, the quality and generalization claims are not yet established to the standard required by the central contribution. The evaluation is largely self-referential, the perceptual study does not include SwiftSketch, and the external SketchyCOCO results show a substantial drop in recognition accuracy. These issues are fixable with additional experiments, but they currently limit the strength of the paper's main claim.","major_comments":[{"comment":"The quality evaluation is self-referential. SwiftSketch is trained exclusively on ControlSketch-generated pairs (Sec. 4.2), and ControlSketch is described as 'the ground truth in our case' in Sec. 5.1. On the synthetic test sets, the main reference is therefore the output distribution SwiftSketch was trained to imitate. This establishes fidelity to the teacher but not independent sketch quality. The external SketchyCOCO numbers in Table 1 expose the fragility of the claim: SwiftSketch CLIP Top-1 drops from 0.95 on seen categories and 0.70 on unseen categories to 0.56 on SketchyCOCO, whereas ControlSketch drops only to 0.91. In addition, the recognition metric uses CLIP, which is also the conditioning encoder for SwiftSketch (Sec. 4.3), creating a common-mode bias. The paper should report a direct human evaluation of SwiftSketch outputs and validate ControlSketch quality independently, for example against human sketches or with a non-CLIP recognizer, before claiming that SwiftSketch approaches optimization-based quality in general.","section":"Section 5.1, Table 1"},{"comment":"The only perceptual study compares ControlSketch with CLIPasso; SwiftSketch is not included. Since the central claim is that SwiftSketch produces high-quality sketches in real time, this study does not support that claim. A 2AFC or rating study that includes SwiftSketch, ideally on both synthetic and external real images, is needed to establish perceptual quality of the proposed method itself.","section":"Supplementary Sec. E"},{"comment":"The claim that SwiftSketch 'generalizes across diverse concepts' is only partially supported. The model was trained on 15 classes, and Table 1 shows CLIP Top-1 of 0.70 on unseen categories of the same synthetic distribution and 0.56 on external SketchyCOCO images. Section 7 and Figure 10 acknowledge failure cases that are noisy or unrecognizable. The generalization claim should be scoped to the regime in which it is actually demonstrated, or the paper should provide additional evidence that the drop does not undermine the stated practical-usefulness claim.","section":"Abstract, Sec. 4.4, Sec. 5, Table 1"},{"comment":"The quality premise of the dataset is load-bearing but not rigorously validated. The supplementary states that 'all the sketches in our data were manually verified', but no protocol is given: no number of annotators, no criteria, no inter-annotator agreement, and no comparison with professional human sketch distributions. Because SwiftSketch inherits its entire output distribution from ControlSketch, the representativeness of ControlSketch as 'professional-quality' must be justified more concretely, for example by a user study comparing ControlSketch outputs with human sketches or by a quantitative style-distribution analysis.","section":"Sec. 4.2, Supplementary Sec. A"}],"minor_comments":[{"comment":"The title contains an erroneous space in 'V ector', and typos such as 'supplamentary' (Sec. 5.1), 'differnet' (Sec. 3), and 'sapmels' (Fig. 22 caption) should be corrected.","section":"Title and throughout"},{"comment":"The caption says the stroke count is shown on top, but the visible labels '12s', '17s' read as times; please relabel the sequence to avoid confusion.","section":"Figure 8"},{"comment":"The inference timing of approximately 0.5 seconds is reported without specifying the GPU and measurement protocol; please state the hardware and the number of runs averaged.","section":"Sec. 4.4"},{"comment":"The sentence stating that Chan et al. and InstantStyle 'achieve the highest scores across most metrics' should be reconciled with Table 1, where no single method is best in every column; the claim is approximately true but should be phrased precisely.","section":"Sec. 5.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The speed contribution is solid, and the synthetic dataset pipeline is a reasonable engineering contribution. The main gap is that the quality evaluation is not yet independent enough: SwiftSketch is absent from the user study, ControlSketch is treated as ground truth, and external recognition drops substantially. These issues are addressable with additional experiments, so I do not recommend rejection. I would also encourage the editor to ask the authors to clarify the release status of the dataset and code, since the supplementary promises public release."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SwiftSketch is a real speed win and a genuinely new formulation—diffusion over stroke control points, trained on synthetic image-sketch pairs. The quality argument, though, is only half-supported. The paper's own numbers show the student model generalizes poorly to unseen categories and external data, and the evaluation leans on a self-referential loop.\n\nWhat's new: first image-conditioned diffusion model for vector sketch generation that operates directly on stroke control points, with a transformer decoder. The synthetic dataset pipeline (ControlSketch with depth ControlNet) is a useful contribution in itself; the 35k dataset and sorting heuristic enable training. The speed claim is solid: 0.5s versus 5-10 minutes for optimization. The ablation is informative—L1 loss is load-bearing, refinement helps unseen categories, and the sorting scheme is not critical for quality but supports abstraction.\n\nSoft spots, in order of importance. First, the quality claim rests on a benchmark where ControlSketch, the teacher that generated the training data, is called 'the ground truth in our case' (Sec 5.1). SwiftSketch is never in the user study; the 2AFC study compares ControlSketch to CLIPasso only. So 'approaches optimization-based quality' is not directly human-validated. Second, the CLIP-as-conditioner-and-metric overlap makes the recognition numbers partly circular; a sketch that looks right to CLIP is more likely to be recognized by CLIP. Third, external generalization is weak: CLIP Top-1 drops to 0.56 on SketchyCOCO (ControlSketch gets 0.91). The authors acknowledge this in Sec 7 and attribute it to limited training categories; that is plausible, but it undercuts the 'generalizes across diverse concepts' claim. Fourth, no code, data, or checkpoints are released, despite the conclusion promising the dataset pipeline will be made public; that makes reproduction impossible and the six-day training cost a barrier.\n\nThese are addressable rather than fatal. The core speed contribution is real and the architecture is sensible. The paper would benefit from a human eval of SwiftSketch itself, an independent (non-CLIP) fidelity metric, and release of at least the dataset and checkpoints.\n\nWho it's for: people working on sketch generation and fast vector graphics; also a good case study for evaluation circularity in generative-model papers. It deserves a serious referee—the method is new and the speed claim matters—but the referee should push on the evaluation.","headline":"Real speed win and a novel formulation, but the quality claim is only half-supported by a self-referential evaluation.","tokens_in":19131,"tokens_out":3068,"would_cite":true,"duration_ms":25632,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SwiftSketch turns an input image into a vector sketch in under a second by denoising stroke control points with a transformer-decoder diffusion model, cutting generation time from roughly 10 minutes to about 0.5 seconds while approaching…","keywords":["vector sketch generation","diffusion model","image-to-sketch","stroke control points","score distillation sampling","ControlNet","synthetic dataset","transformer decoder"],"falsifier":"Run SwiftSketch on a few hundred photographed objects whose classes are absent from the 100 synthetic categories and report CLIP Top-1 recognition and human recognizability judgments: if accuracy sits near the 0.56 external-data value rather than the 0.95 seen-category value, the claim that the model generalizes across diverse concepts is settled in the negative, and the paper's own carrot and Eiffel Tower failure cases are the first data points of that test.","tokens_in":1747,"feed_emoji":"✏️","tokens_out":1714,"duration_ms":140720,"temperature":0.7,"pith_summary":"The paper tries to establish that image-conditioned vector sketching can be made interactive. Instead of spending minutes optimizing stroke positions against a frozen vision-language model, a diffusion model can learn the output distribution of that optimization and reproduce it in about half a second. The key step is to diffuse directly in stroke-coordinate space: each sketch is a set of cubic Bezier control points that are noised and denoised like pixels, with a transformer decoder that respects the discrete, sequential nature of strokes and the global dependencies between them. Because no large professional-quality paired sketch dataset exists, the paper builds one, synthesizing over 35,000 image-sketch pairs across 100 classes using ControlSketch, an SDS-based optimizer augmented with depth-aware ControlNet spatial control, and trains SwiftSketch on that data. If correct, this replaces a costly per-image optimization with a learned feed-forward model, and the dataset-construction pipeline becomes a reusable asset for other vector-generation tasks.","feed_headline":"Sketch generation drops from 10 minutes to half a second","feed_subtitle":"SwiftSketch denoises stroke control points straight from a Gaussian to draw any input object as a vector sketch.","key_machinery":"The load-bearing object is the diffusion process over stroke-coordinate space: a sketch is a tensor $S_0 \\in \\mathbb{R}^{2\\times 4 \\times n}$ of Bezier control points, noised by a cosine-derived noise scheduler and denoised by a transformer decoder with eight self- and cross-attention layers. The input image enters through a frozen CLIP ResNet embedding extracted from the fourth layer, refined by a lightweight CNN, and injected via cross-attention alongside the timestep. Two other named mechanisms carry the method: ControlSketch, an SDS-based optimizer that enhances the score-distillation loss with a depth-aware ControlNet so strokes follow the object's geometry, and the stroke-sorting heuristic that orders strokes by contour intersection count and attention score, enabling variable levels of abstraction.","core_discovery":"The authors' framing is that professional-looking vector sketches can be distilled from a slow optimization process into a fast generative model. SwiftSketch trains a diffusion model to map Gaussian noise in the space of stroke control points, where each stroke is a cubic Bezier curve and each sketch consists of 32 ordered strokes, to the sketch distribution, conditioned on a CLIP image embedding injected through cross-attention; a learned refinement network applies a final cleaning pass. Training uses dual objectives, an L1 loss on control points plus an LPIPS loss on the rasterized sketch, and a learned stroke ordering that foregrounds contour and salient strokes, which supports progressive levels of abstraction. The authors report that SwiftSketch approaches ControlSketch's quality on training categories, with CLIP Top-1 recognition of 0.95 for seen classes versus 0.97 for ControlSketch, while cutting per-sketch cost from about 10 minutes to about 0.5 seconds, at the cost of degraded recognition on unseen categories (0.70) and on external SketchyCOCO data (0.56).","pith_inferences":["The paper's real contribution may be the distillation loop itself: the slow optimization teacher and fast diffusion student form a bootstrapping scheme that could be iterated by re-feeding student outputs into the teacher's data pool without human labor.","The generalization gap, CLIP Top-1 of 0.70 on unseen synthetic categories versus 0.56 on real SketchyCOCO images, suggests the student has partly memorized the synthetic teacher's style; fine-tuning on a small set of real human sketches would test whether external recognition and style diversity recover.","Because the model is trained only on masked single-object images at a fixed 32 strokes, extending training to variable stroke counts and scene-level inputs is a natural test of whether the denoising-in-coordinate-space formulation scales to other vector-generation tasks.","Since the depth ControlNet is the main carrier of geometric fidelity in ControlSketch, improving the teacher, for example by swapping in different ControlNet condition types, should directly improve the student's output quality."],"forward_implications":["Vector sketching becomes a real-time operation, so interactive tools can re-sketch on every edit without a minutes-long optimization wait.","Because the output is vector SVG with ordered strokes, users can control abstraction simply by truncating the stroke sequence, with early strokes already conveying the concept.","The synthetic-pair pipeline is a general recipe: any SDS-based sketch optimizer could be swapped into ControlSketch's place to produce training data for a fast student model.","The 35,000-pair ControlSketch dataset, covering 100 classes, is released and built by a fully automated pipeline, addressing the scarcity of professional-quality paired sketch data.","The student's fidelity on seen categories tracks the teacher's, so the quality ceiling of the whole system is set by ControlSketch's optimization, not by the diffusion model."],"supporting_citations":[{"why":"CLIPasso is the optimization baseline both ControlSketch and SwiftSketch are compared against, and its attention-based stroke initialization is the scheme ControlSketch improves.","marker":"[47]"},{"why":"Defines the SDS loss that ControlSketch builds on to extract feedback from a pretrained text-to-image diffusion model.","marker":"[34]"},{"why":"The depth-aware ControlNet that gives ControlSketch its spatial control and carries the geometric fidelity of the generated sketches.","marker":"[58]"},{"why":"Provides the CLIP image encoder used for SwiftSketch's conditioning and the zero-shot classifier used in all quantitative evaluations.","marker":"[36]"},{"why":"SDXL generates the input images of the synthetic dataset used to train SwiftSketch.","marker":"[33]"},{"why":"The differentiable rasterizer that bridges vector strokes and pixel space in both ControlSketch's optimization and SwiftSketch's losses.","marker":"[26]"},{"why":"SketchyCOCO is the external real-image dataset used to assess generalization of the trained model.","marker":"[14]"},{"why":"LPIPS serves both as a training objective term for SwiftSketch and as a fidelity metric in the quantitative evaluation.","marker":"[59]"}],"fun_headline_variants":["SwiftSketch turns 10-minute sketch optimization into half a second","Fast vector sketches: diffusion model cuts time from 10 min to 0.5 s","Diffusion model draws vector sketches in 0.5s instead of 10min","From 10 minutes to 0.5 seconds: SwiftSketch generation"],"cache_read_input_tokens":21120,"weakest_assumption_plain":"The load-bearing premise is that ControlSketch's synthetic outputs, SDXL-generated images sketched by a depth-conditioned SDS optimizer and then manually filtered, form a distribution that genuinely represents professional-quality sketching; if that distribution is stylistically narrow or hides artifacts, SwiftSketch inherits those flaws, and the paper's own external-evaluation drop, CLIP Top-1 of 0.56 on SketchyCOCO, already indicates the learned style does not fully transfer to real photographs.","fun_headline_variants_meta":{"raw":{"variants":["SwiftSketch turns 10-minute sketch optimization into half a second","Fast vector sketches: diffusion model cuts time from 10 min to 0.5 s","Diffusion model draws vector sketches in 0.5s instead of 10min","From 10 minutes to 0.5 seconds: SwiftSketch generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2598,"prompt_tokens":963,"completion_tokens":1635,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":1549}},"tokens_in":579,"tokens_out":1635,"duration_ms":10326,"temperature":1.0,"reasoning_tokens":1549,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T23:57:52.968261+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SwiftSketch on a few hundred photographed objects whose classes are absent from the 100 synthetic categories and report CLIP Top-1 recognition and human recognizability judgments: if accuracy sits near the 0.56 external-data value rather than the 0.95 seen-category value, the claim that the model generalizes across diverse concepts is settled in the negative, and the paper's own carrot and Eiffel Tower failure cases are the first data points of that test.","supporting_citations":[{"cited_title":"Bo, Ro- man Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir","cited_arxiv_id":null,"evidence_quote":"CLIPasso is the optimization baseline both ControlSketch and SwiftSketch are compared against, and its attention-based stroke initialization is the scheme ControlSketch improves."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"The depth-aware ControlNet that gives ControlSketch its spatial control and carries the geometric fidelity of the generated sketches."},{"cited_title":"Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023","cited_arxiv_id":null,"evidence_quote":"SDXL generates the input images of the synthetic dataset used to train SwiftSketch."},{"cited_title":"Differentiable vector graphics rasterization for editing and learning","cited_arxiv_id":null,"evidence_quote":"The differentiable rasterizer that bridges vector strokes and pixel space in both ControlSketch's optimization and SwiftSketch's losses."},{"cited_title":"Sketchycoco: Image generation from freehand scene sketches","cited_arxiv_id":null,"evidence_quote":"SketchyCOCO is the external real-image dataset used to assess generalization of the trained model."}],"review_version":1}