Pith. sign in

REVIEW 3 major objections 5 minor 21 references

Draw This First

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Drawing order can be encoded as a color field and read back as text-ordered strokes.

desk verdict A genuinely new order-as-color system with careful derendering evaluation, but the headline claim overreaches: text-to-sketch order is never quantitatively tested. read the letter →

arxiv 2608.12064 v1 pith:THFBDEGI submitted 2026-08-12 cs.CV cs.LG

Dazhi Zhong , Rowan Bradbury , Grant Davis This is my paper · ORCID
classification cs.CVcs.LG
keywords sketchgenerationstrokeorderorder-as-colorcodecimagederenderingtext-conditioneddrawinglatentflowmatchingdiffusiontransformervectorization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the order in which strokes are drawn can be treated as a 2D image field instead of a sequence: it encodes global drawing progress into hue and within-stroke progress into brightness, generates or derenders that encoded image with a pretrained latent flow-matching transformer, and reads the field back out with a retrained decoder to assemble polylines sorted by draw order. The system thereby gives a text-conditioned channel for stroke order: captions state which regions or parts come first, and changing only the caption changes the order while leaving the geometry fixed. The clearest evidence is that reversing a caption flips Kendall tau from 0.373 to -0.267 with unchanged mask IoU, and that on an external part-annotated benchmark the model reaches 0.778 stroke-order Kendall when the stated part order is given. In the paper's own tests, order control degrades with instruction granularity: region-level instructions are followed reliably, part-level adherence is 0.46, and stroke order inside a named unit is near zero. The contribution is a representation-level inversion: order is no longer a generation axis but a property of a generated image that can be read back.

What carries the argument

The load-bearing object is the order-as-color codec of Eq. (1): each inked pixel's hue is $H = a\cdot 342/360$, where $a\in[0,1]$ is the accumulated arc length from the first stroke to that pixel divided by total arc length; saturation is 1 on ink and 0 on background; and value is $V = 1-u/2$, where $u$ is the arc fraction within the current stroke. This single 8-bit RGB image lets a pretrained latent flow-matching transformer generate or derender the order field inside its native image latent space. A retrained decoder with a pyramid CNN head emits the predicted global arc field, a foreground mask, and eight-dimensional stroke-instance embeddings; HDBSCAN clusters the embeddings, a nearest-neighbour walk recovers polylines, and the global arc field sorts clusters into the final stroke order. The codec is what converts a sequential ordering problem into a spatial image-generation problem, so text can steer order through the diffusion model and the decoder can read it back.

What would settle it

Take a set of drawings whose true stroke-level order is known and issue text instructions that name individual strokes or atomic units in a specific order. If the resulting Kendall tau stays near zero, while region-level instructions reach 0.8, the claim of following text instructions for drawing order is true only at region granularity. A cleaner test: generate the same sketch twice under 'draw top to bottom' and 'draw bottom to top', then check whether every matching stroke's relative order flips; any stroke whose order does not invert marks the control boundary.

Watch

Extended reading notes

Core claim

The central discovery is that vector stroke order can be carried lossily through an image-generative model. The paper shows that if every inked pixel is colored by a global arc fraction and a within-stroke arc fraction, then a frozen image encoder, a diffusion transformer, and a finetuned decoder can recover enough of that field to reconstruct ordered vectors with Kendall tau 0.91–0.94 at the decoder ceiling and 0.449 with recorded-order captions end to end. Order is an independent controllable dimension: geometry metrics stay flat while order swings from positive to negative when the text instruction is reversed, and the model continues to generate recognizable sketches because it retains the base model's world knowledge. The authors state the proposal as: instead of drawing strokes in order, predict a 2D field that defines the order, then vectorize and sort by it.

Load-bearing premise

The assumption that carries the whole system is that an 8-bit color map of draw order can pass through the lossy image encoder and diffusion model with enough of its fine-grained hue and brightness structure intact for the decoder to sort strokes correctly; if the image prior smooths away that structure, the order signal is gone.

Editorial extensions

If this is right

  • Because text, not geometry, carries the order, the same drawing can be replayed in multiple orders from one model by changing only the caption.
  • Order control is bounded by the granularity of the training permutation: coarse units are reliable, part-level adherence is about 0.46, and within-unit stroke order is essentially uninstructable.
  • The decoder ceiling (Kendall tau 0.91–0.94 on four datasets) shows that the vectorization path is not the bottleneck; losses in end-to-end order come from generation given the caption.
  • The model can derender an input image into replayable ordered vectors while retaining the base model's open-vocabulary text-to-sketch ability, as measured by CLIP recognition of generated QuickDraw categories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension beyond the paper: because the order field is a continuous scalar independent of stroke geometry, the same codec could convert offline handwriting or assembly video into ordered trajectories, provided a dataset with recorded ground-truth order exists.
  • The caption-reversal result suggests a testable design principle: when a language instruction conflicts with an image condition, the diffusion model treats the text as the stronger order prior; measuring this conflict systematically could guide conditioning for other generative tasks.
  • A direct follow-up experiment would train the same architecture with permutations at the stroke or point level rather than the region level; if the codec survives, the paper's own limitation list predicts within-unit tau should rise from near zero.
  • If per-point order labels were available, the global arc field could be replaced by any learned monotone embedding, turning the method into a general order-as-image representation for sequential data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes a sketch-order representation and pipeline. It encodes stroke order as an HSV 'codec image' (global arc in hue, within-stroke progress in value), trains a LoRA-adapted latent flow-matching transformer to generate this intermediate from text or image, and finetunes the VAE decoder with new heads that emit a global arc field, foreground mask, and per-stroke instance embeddings. A non-learned vectorizer (HDBSCAN plus nearest-neighbor tracing and RDP simplification) turns these fields into ordered polylines. The model is trained on 47,318 commissioned drawings with programmatic captions that describe a permuted region-level order. The main quantitative results are: (i) a decoder ceiling of Kendall tau around 0.91-0.94 on multi-stroke sets; (ii) caption interventions in derendering that move or reverse stroke order while geometry stays fixed; (iii) region-level instruction adherence tau=0.838, falling to 0.461/0.444 at part/pass level; (iv) an external ControlSketch-Part test with stated-order tau=0.778; and (v) QuickDraw text-to-sketch CLIP recognition of 0.70-0.75 Top-1 without any order evaluation. The paper claims that the system 'can predict an ordered vector sketch from a text description or derender an image into ordered vectors; for either, it follows text instructions specifying the order of drawing.'

Significance. If the order channel works as claimed, this is a novel and useful capability: it makes stroke order a free-form, language-conditioned output variable rather than an implicit generation axis, and it reuses a strong pretrained image prior. The evaluation is more careful than typical for this area: the authors separate decoder ceiling from end-to-end loss, include caption-deletion, recorded-order, and reversed-order controls with geometry held fixed, report a condition funnel, and validate on an externally annotated dataset whose part annotations come from a different pipeline. These controls give me moderate confidence that language genuinely influences coarse stroke order in the derendering setting. The significance is reduced, however, by two gaps: the text-to-sketch branch of the central claim has no order evaluation, and the demonstrated order control is coarse (named units only), with the paper itself stating that 'below named units there is no control at all' (Section 6). The core idea and evaluation design are solid enough to warrant revision rather than rejection.

major comments (3)
  1. [Section 4.3, Table 2] The abstract's claim that 'for either, it follows text instructions specifying the order of drawing' is not supported for the text-to-sketch branch, because every quantitative order test in the paper—Table 1, Table 3, and the ControlSketch-Part evaluation—conditions on an input sketch, while Section 4.3 evaluates text-only generation only with CLIP recognition over 50 QuickDraw categories and never issues or measures an order instruction. Please add a text-to-sketch order experiment (for example, generating with explicit order captions and scoring part-order adherence against an annotation or human judgment), or narrow the claim to image-conditioned derendering.
  2. [Abstract, Section 5, Table 3] The phrase 'follows text instructions specifying the order of drawing' overstates the measured precision. Region-level instruction adherence is high (tau=0.838), but part-level adherence drops to 0.461, pass-level to 0.444, and within-unit residual tau is near zero (0.177/0.134 on holdout; 0.058/0.068 on ControlSketch-Part). Section 6 itself concedes that 'below named units there is no control at all.' The claim should be qualified as following coarse named-unit order, not arbitrary or stroke-level order.
  3. [Section 4, Tables 1, 3, 4] All headline order numbers are n=100 point estimates with no confidence intervals and no description of how the 100 drawings per dataset are selected (random, first N, or stratified), and the Wilcoxon tests in Appendix B are reported only for one comparison. Because the paper's central quantitative claims—especially the part-level and ControlSketch-Part numbers—are used to calibrate how much instruction following the system achieves, the authors should report bootstrap confidence intervals or per-sample distributions and state the selection procedure.
minor comments (5)
  1. [Table 4] The holdout path-count ratio of 12.53 conflicts with Section 6's statement that recovered paths run 1.7-2.5 times the true stroke count; please reconcile or correct the table entry.
  2. [Section 3.3] The sentence 'The pixel branch is architecturally unchanged, while training jointly with the new head' is ambiguous about whether the reconstruction branch is also finetuned; clarify which weights are updated.
  3. [Section 3.4 and Algorithm 1] HDBSCAN clustering over the eight-dimensional embedding channels is a key step for segmentation, but the appendix gives no hyperparameter values or sensitivity analysis for HDBSCAN or the capping radius in Algorithm 1; please add these to the appendix.
  4. [Section 4] The primary order metric is called Kendall tau, but the paper does not state whether this is tau-a or tau-b; ties can arise from the vectorizer's duplicated arc values, and the choice affects comparability across tables.
  5. [Section 5] The 'pass' level of instruction is referenced in Table 3 but not defined until Section 6; define it earlier or add a pointer.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the order targets come from recorded artist stroke data and are evaluated against independent held-out and external benchmarks.

full rationale

The paper's derivation chain is self-contained rather than circular. The order target is defined externally by the recorded Apple Pencil input stream (Section 3.1: 'Each drawing is stored as the raw Apple Pencil input stream plus a simplified per-point form'), and the order-as-color codec in Eq. (1) merely encodes that recorded order into HSV. The VAE decoder is trained with a supervised loss against ground-truth arc field, mask, and segmentation (Eq. (2)), and the DiT is trained with a standard flow-matching objective on the latent of the codec image; no objective uses the model's own output as its own label. The order metrics (Kendall tau against recorded stroke order after Hungarian geometric matching) compare predictions to held-out artist recordings, including Creative Birds, Creative Creatures, FS-COCO, and QuickDraw, none of which enter DiT training (Section 4). The reversed-caption controls (Table 1) and the external ControlSketch-Part test with stated and reversed part order (Section 5) provide independent, out-of-distribution evidence that the caption is the operative order channel. The only self-citation, bbml [Bradbury and Zhong, 2025], is an implementation framework (Section 3.4) and is not load-bearing for any scientific claim. The paper's own Limitations qualify the granularity of order control (within-unit tau near zero; a trailing constraint lost entirely), and the abstract's text-to-sketch order claim is under-supported because Section 4.3 reports only CLIP recognition without an order metric; however, these are empirical evaluation gaps, not reductions of the result to its inputs. No equation or fitted parameter is equivalent by construction to the quantity it purports to predict.

Assumptions & free parameters 8 free parameters · 4 assumptions · 2 invented entities

The central claim rests on one private dataset, one pretrained backbone, and a synthetic caption distribution, none of which are independently verified outside the paper. The loss weights and clustering thresholds are hand-set, and the order-as-color codec is validated only internally. This is a typical systems-paper ledger: no invented physical entities, but several unverified modeling choices.

free parameters (8)
  • arc loss weight w_arc = 8
    Set by hand in Eq. (2) to balance arc-field reconstruction against pixel and perceptual terms; the central order readout depends on this scale.
  • pixel and LPIPS loss weights w1, wlpips = 1 each
    Set by hand in Eq. (2); not fitted to any external benchmark.
  • stroke-pixel weight factor = 50x
    Hand-chosen emphasis on ink pixels and their radius-1 surround in Appendix A.
  • push-pull margin 2dd = 3.0
    Discriminative embedding loss margin in Appendix A; affects whether HDBSCAN can separate stroke instances.
  • push-pull radius dv = 0.5
    Embedding contraction radius in Appendix A.
  • RDP simplification epsilon = 0.5 px
    Hand-set tolerance in Section 3.4; changes stroke shape and count.
  • HDBSCAN clustering parameters = not specified
    Needed for stroke segmentation from embeddings in Section 3.4; the parameters are not reported, which also hurts reproducibility.
  • hue scale factor = 342/360
    Hand-chosen mapping in Eq. (1); it reserves the top of the hue range for background and sets the resolution of the order channel.
assumptions (4)
  • domain assumption The commissioned Apple Pencil recordings, in simplified polyline form, are a reliable ground truth for stroke order and geometry.
    Section 3.1 treats the recorded stroke sequence as the target for captions and for Kendall tau; if the recordings contain artifacts or nonstandard order, all order metrics inherit that noise.
  • domain assumption The Qwen-Image VAE and DiT latent space can carry the order-as-color codec faithfully enough for order recovery.
    Section 3.2 and Appendix E test this for the frozen VAE and show degradation; the method assumes the diffusion model can also generate these codes at inference.
  • ad hoc to paper Programmatic captions built from bounding-box region and subject names are sufficient for the model to learn a generalizable language-to-order mapping.
    The order-controllability claim rests on this synthetic caption distribution; there is no evidence from natural human order descriptions.
  • domain assumption The pretrained backbone retains its text-to-sketch world knowledge after LoRA fine-tuning.
    Section 4.3's CLIP recognition supports this only for QuickDraw categories; the assumption is broader than the evidence.
invented entities (2)
  • order-as-color HSV codec image
    purpose: Rasterizes a sketch so that hue carries global progress and value carries within-stroke progress, making order a pixel property that an image model can generate.
    Introduced in Section 3.2 and validated only internally through decoder-ceiling and frozen-VAE ablations; no external benchmark verifies this representation.
  • order-native decoder with pyramid head and instance-embedding channels
    purpose: Reads the latent code and emits global arc, foreground mask, and stroke-instance embeddings so that vectorization can sort strokes by predicted order.
    Reported in Section 3.3 and Appendix A; internal metrics show good field recovery but there is no independent dataset or external check of the embedding head.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Draw This First." pith.science (2026). https://pith.science/paper/THFBDEGI

@misc{pith2026260812064,
  author       = {Pith},
  title        = {Pith review of: Draw This First},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/THFBDEGI}},
  note         = {Machine review of arXiv:2608.12064}
}
read the original abstract

We invert the typical formulation of sketch generation: instead of drawing strokes in order, we predict a 2D field that defines the order in which strokes are drawn. We use a pretrained latent flow-matching transformer to supply the image prior to predict an intermediate representation, while training the VAE's decoder to predict the order field, stroke mask, and stroke segmentation. We vectorize the predicted segmentation into polylines and sort them by the field, producing an ordered vector sketch. Our model can predict an ordered vector sketch from a text description or derender an image into ordered vectors; for either, it follows text instructions specifying the order of drawing.

Figures

Figures reproduced from arXiv: 2608.12064 by the authors.

Figure 1
Figure 1. Decoder architecture and objective. The source strokes derive four targets: ink raster, stroke [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) Order permutation with local order preservation from Section 3.1. (b) The DiT trains [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Draw order under text intervention: four held-out sketches, each derendered under a hand [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Hierarchical annotation on one sketch. (a) Solid boxes: regions; dashed: subjects. (b) [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: The encoding, on one held-out drawing (15 strokes). (a) Source ink. (b) [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Representative 50k outputs, deterministic median selections per domain. (a) Birds and [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: One deterministically-sampled drawing from each of the 50 commissioned artists. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 7 canonical work pages

  1. [6]

    Teaching an Agent to Sketch One Part at a Time

    Xiaodan Du, Ruize Xu, David Yunis, Yael Vinker, and Greg Shakhnarovich. Teaching an agent to sketch one part at a time.arXiv preprint arXiv:2603.19500,

    work page Pith review arXiv

  2. [9]

    Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg

    arXiv:2211.11319. Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. The quick, draw! dataset,

    arXiv

  3. [10]

    InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write

    arXiv:2402.05804. Elmokhtar Mohamed Moussa, Thibault Lelore, and Harold Mouchère. SET, SORT! a novel sub-stroke level transformers for offline handwriting to online conversion. InInternational Conference on Document Analysis and Recognition (ICDAR), pages 81–97,

    work page Pith review arXiv

  4. [11]

    VideoSketcher: Sequential Sketch Generation Using Video Model Priors

    hal-04182547. Hui Ren, Yuval Alaluf, Omer Bar-Tal, Alexander Schwing, Antonio Torralba, and Yael Vinker. VideoSketcher: Sequential sketch generation using video model priors.arXiv preprint arXiv:2602.15819,

    work page Pith review arXiv

  5. [12]

    Zecheng Tang et al

    arXiv:2312.11556. Zecheng Tang et al. Strokenuwa: Tokenizing strokes for vector graphic synthesis. InInternational Conference on Machine Learning (ICML),

    arXiv

  6. [13]

    Yael Vinker et al

    arXiv:2401.17093. Yael Vinker et al. Clipasso: Semantically-aware object sketching.ACM Transactions on Graphics (SIGGRAPH), 41(4),

    arXiv

  7. [14]

    Jiawei Wang et al

    arXiv:2411.17673. Jiawei Wang et al. Vq-sgen: A vector quantized stroke representation for creative sketch generation. In International Conference on Computer Vision (ICCV),

    arXiv

  8. [15]

    VQ-SGen: A Vector Quantized Stroke Representation for Creative Sketch Generation

    arXiv:2411.16446. Qiang Wang et al. Sketchknitter: Vectorized sketch generation with diffusion models. InInternational Conference on Learning Representations (ICLR),

    work page Pith review arXiv

Show all 21 references
  1. [16]

    Qwen-image technical report.arXiv preprint arXiv:2508.02324,

    Chenfei Wu et al. Qwen-image technical report.arXiv preprint arXiv:2508.02324,

    arXiv

  2. [17]

    7 Ximing Xing, Haitao Zhou, Chuang Wang, Jing Zhang, Dong Xu, and Qian Yu

    arXiv:2304.14400. 7 Ximing Xing, Haitao Zhou, Chuang Wang, Jing Zhang, Dong Xu, and Qian Yu. Svgdreamer: Text guided svg generation with diffusion model. InConference on Computer Vision and Pattern Recognition (CVPR),

    arXiv

  3. [18]

    Ximing Xing et al

    arXiv:2312.16476. Ximing Xing et al. Empowering llms to understand and generate complex vector graphics. InConference on Computer Vision and Pattern Recognition (CVPR),

    arXiv

  4. [19]

    Yiying Yang et al

    arXiv:2412.11102. Yiying Yang et al. Omnisvg: A unified scalable vector graphics generation model. InAdvances in Neural Information Processing Systems (NeurIPS),

    arXiv

  5. [20]

    Jin Zhou, Yi Zhou, Hongliang Yang, Pengfei Xu, and Hui Huang

    arXiv:2504.06263. Jin Zhou, Yi Zhou, Hongliang Yang, Pengfei Xu, and Hui Huang. Strokefusion: Vector sketch generation via joint stroke-udf encoding and latent sequence diffusion. InAAAI Conference on Artificial Intelligence,

  6. [2018]

    Edward J

    arXiv:1704.03477. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. InInternational Conference on Learning Representations (ICLR),

    arXiv

  7. [2020]

    Ellie Arar, Yarden Frenkel, Daniel Cohen-Or, Ariel Shamir, and Yael Vinker

    arXiv:2006.09930. Ellie Arar, Yarden Frenkel, Daniel Cohen-Or, Ariel Shamir, and Yael Vinker. Swiftsketch: A diffusion model for image-to-vector sketch generation. InSIGGRAPH,

    2006 arXiv

  8. [2021]

    David Ha and Douglas Eck

    arXiv:2011.10039; DoodlerGAN. David Ha and Douglas Eck. A neural representation of sketch drawings. InInternational Conference on Learning Representations (ICLR),

    2011 arXiv

  9. [2022]

    Moises Diaz, Gioele Crispo, Antonio Parziale, Angelo Marcelli, and Miguel A. Ferrer. Writing order recovery in complex and long static handwriting.arXiv preprint arXiv:2406.03194,

    arXiv

  10. [2023]

    Ayan Das et al

    arXiv:2304.03785. Ayan Das et al. Sketchode: Learning neural sketch representation in continuous time. InInternational Conference on Learning Representations (ICLR),

    arXiv

  11. [2024]

    Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, Jorma Laaksonen, and Michael Felsberg

    arXiv:2403.09344. Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, Jorma Laaksonen, and Michael Felsberg. Doodleformer: Creative sketch drawing with transformers.arXiv preprint arXiv:2112.03258,

    arXiv

  12. [2025]

    Hmrishav Bandyopadhyay, Ankan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Tao Xiang, and Yi-Zhe Song

    arXiv:2502.08642. Hmrishav Bandyopadhyay, Ankan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Tao Xiang, and Yi-Zhe Song. Sketchinr: A first look into sketches as implicit neural representations. InConference on Computer Vision and Pattern Recognition (CVPR),

    arXiv

  13. [2026]

    arXiv:2503.23752. A Decoder objective and training details Writingz=E(x),x ′ =D p(z), and(a′,m′,e′) =D ord(z), L=w 1 wfg⊙(x′−x) 1| {z } foreground-weighted pixelL 1 +w lpips LPIPS(x′,x) +w arc h (a′−a)⊙m 1 + 0.25 BCEw(m′,m) + 0.1L pp(e′,s) i , (2) wheres is the ground-truth st...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.