Pith. sign in

REVIEW 3 major objections 25 references

CAD sequences can be rebuilt step by step from three-view images by treating modeling as offline reinforcement learning with live visual feedback.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 21:32 UTC pith:OP7NLWGJ

load-bearing objection Clean offline-RL framing of CAD sequence recovery with stepwise orthographic + sketch feedback; gains look real, but the train-time IoU vs. inference pHash hinge is under-checked. the 3 major comments →

arxiv 2607.04119 v1 pith:OP7NLWGJ submitted 2026-07-05 cs.CV cs.AI

SOV-CAD: Stepwise Orthographic Views Guided CAD Modeling Sequence Reconstruction

Zhaopeng Feng , Chen Zhi , Xuhong Zhang , Zhengwen Feng , Xinkui Zhao This is my paper
classification cs.CV cs.AI
keywords CAD reconstructionmodeling sequenceDecision Transformeroffline reinforcement learningorthographic viewsstepwise visual supervisionsketch canvasgeometric alignment reward
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most CAD reconstruction systems try to emit an entire modeling program in one shot, which ignores how human designers actually work: they look at the growing model, adjust the active sketch, and choose the next command based on what they see. This paper claims that the same loop can be automated by feeding a Decision Transformer the target's orthographic three-views, the current model's three-views, and the sketch canvas at every step, together with a geometric-alignment reward. The resulting policy, SOV-CAD, is trained offline on pre-rendered trajectories and at inference updates its own visual state after each predicted command. On standard CAD sequence benchmarks the method recovers geometry more accurately than prior holistic or autoregressive baselines while using less training data. A sympathetic reader cares because editable, intent-preserving CAD programs are far more useful than static meshes, and a stepwise visual policy is a concrete route toward that goal.

Core claim

Formulating CAD sequence reconstruction as offline reinforcement learning with a Decision Transformer that conditions on stepwise orthographic three-views of the partial model, the active sketch, and IoU-based return-to-go tokens produces more accurate and data-efficient modeling sequences than holistic generation methods.

What carries the argument

Stepwise multimodal state tokens: at each step the target three-views (prefix), the current model's three-views plus sketch canvas (state), and a return-to-go derived from IoU improvement are concatenated and fed to a Decision Transformer that predicts the next CAD command and parameters.

Load-bearing premise

Pre-computed offline trajectories with exact IoU rewards, plus a perceptual-hash surrogate for return-to-go at inference, are assumed to be a faithful enough stand-in for the live visual feedback humans use when modeling.

What would settle it

Train and evaluate the identical Decision Transformer on the same trajectories but replace the stepwise visual state and reward with only the target three-views; if median Chamfer Distance and IoU become statistically indistinguishable from the full SOV-CAD model, the claim that stepwise visual feedback is the decisive ingredient fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. SOV-CAD formulates CAD modeling-sequence reconstruction from orthographic views as an offline RL problem solved by a Decision Transformer. At each step the policy conditions on prefix tokens from the target three-views, state tokens from the current solid’s three-views plus the active sketch canvas, and a return-to-go token derived from remaining IoU (training) or a pHash surrogate (inference, Eq. 5). Actions are discretized CAD commands and parameters. Experiments on CADParser and a 50 % DeepCAD subset report lower median Chamfer Distance, higher IoU and lower invalid rates than adapted DeepCAD, SkexGen, HNC-CAD and the original CADParser numbers; ablations isolate stepwise views, sketches and the reward signal.

Significance. If the results hold under stricter controls, the paper supplies a practical, data-efficient alternative to holistic sequence generators by injecting intermediate visual feedback into a Decision Transformer. The public code release and the clear ablation of stepwise views / sketches / reward are concrete strengths that make the contribution usable and falsifiable. The work is of interest to the CAD-reconstruction and offline-RL communities, even though the train–test reward mismatch remains an open methodological point.

major comments (3)
  1. §III.D Eq. (5) and Tables I–II: the central claim that stepwise visual feedback drives the gains rests on the untested premise that pHash return-to-go is a faithful surrogate for the IoU-based return-to-go used in training. The paper never reports the correlation (or rank correlation of candidate actions) between pHash and true IoU on intermediate solids. Without that measurement the near-identical “w/ geometric” vs “w/o geometric” numbers cannot be confidently attributed to the claimed mechanism rather than to other architectural differences.
  2. §IV.A–B and Table II: all DeepCAD baselines were adapted by the authors (latent-space mappers or codebook selectors) while SOV-CAD* was pretrained on a CADParser+DeepCAD mix and fine-tuned on only half the DeepCAD data. The comparison is therefore not controlled for training regime or data volume; a same-data, same-protocol re-run (or at least an ablation that trains SOV-CAD from scratch on the identical DeepCAD split) is needed before the SOTA claim can be accepted.
  3. §IV.A and Table I: CADParser numbers are taken from the original paper because its code is unavailable. Without a re-implementation under the same rendering, quantization and evaluation pipeline, the numerical superiority claimed in Table I remains only partially verifiable.

Circularity Check

0 steps flagged

No circularity: empirical offline-RL pipeline with external geometric rewards and independent benchmarks; nothing reduces by construction to its own inputs.

full rationale

SOV-CAD is a purely empirical systems paper. It formulates CAD sequence reconstruction as an offline MDP whose states are rendered orthographic views plus sketch canvases, whose actions are discretized CAD commands, and whose rewards are defined externally as stepwise IoU deltas (or a pHash surrogate at inference). The Decision Transformer is trained by ordinary teacher-forced cross-entropy on pre-computed trajectories; evaluation uses standard geometric metrics (CD, IoU, IR) against held-out models and against independently re-implemented baselines. No equation equates a claimed prediction to a fitted constant, no uniqueness theorem is imported from the authors’ prior work, and the few self-citations are ordinary references to public datasets or architectures. The train/inference reward mismatch (IoU vs. pHash) is an untested modeling assumption, not a circular reduction. Consequently the derivation chain is self-contained against external benchmarks and exhibits zero circularity of the kinds enumerated.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 2 invented entities

The central performance claim rests on a small set of modeling choices (command discretization, IoU reward, three-view + sketch state, offline DT) and standard deep-learning assumptions; no new physical entities are postulated. Free parameters are ordinary hyper-parameters and quantization choices; axioms are domain conventions of CAD kernels and offline RL; the invented entities are the specific multimodal state and reward design.

free parameters (6)
  • Transformer embedding dimension dm = 768
    Set to 768; controls capacity of all tokens and is not derived from data or theory.
  • number of Transformer layers / heads = 8 / 12
    Chosen as 8 layers, 12 heads; standard architectural free choice.
  • learning rate and schedule = 1e-4
    1e-4 with cosine decay and 1000-step warm-up; selected by authors.
  • parameter quantization levels = 256
    Continuous CAD parameters discretized into 256 bins following DeepCAD; arbitrary granularity choice.
  • maximum sequence length Nc = 64
    Hard limit of 64 commands; truncates longer expert trajectories.
  • image resolution and ViT patch size = 384 / 32
    384×384 inputs, frozen ViT-Base/32; preprocessing free parameters.
axioms (4)
  • domain assumption CAD construction is a Markov Decision Process whose state can be adequately summarized by three orthographic views plus a 2-D sketch canvas.
    Stated in §III.A; underpins the entire state representation.
  • domain assumption Offline trajectories generated from ground-truth command sequences with exact IoU rewards are sufficient to train a policy that generalizes to new targets without online CAD interaction.
    Core premise of the offline-RL formulation (§III.A).
  • ad hoc to paper Perceptual-hash similarity of orthographic views is a monotonic proxy for volumetric IoU and can therefore replace the true return-to-go at inference (Eq. 5).
    Introduced only for inference; never validated against true IoU on held-out geometry.
  • standard math Standard Transformer sequence modeling and frozen ViT image embeddings are adequate feature extractors for this task.
    Implicit throughout §III.B; relies on prior literature (Decision Transformer, ViT).
invented entities (2)
  • stepwise multimodal state (three orthographic views of partial solid + active sketch canvas) no independent evidence
    purpose: Supplies the continuous visual feedback that lets the Decision Transformer condition each action on intermediate geometry.
    Defined in §III.B Eq. (2); no independent existence outside this architecture.
  • IoU-delta reward and remaining-IoU return-to-go for CAD sequences no independent evidence
    purpose: Provides the scalar progress signal that turns sequence modeling into offline RL.
    Defined in §III.A; the particular delta-IoU formulation is paper-specific.

pith-pipeline@v1.1.0-grok45 · 14514 in / 3237 out tokens · 37341 ms · 2026-07-11T21:32:52.769442+00:00 · methodology

0 comments
read the original abstract

Reconstructing Computer-Aided Design (CAD) modeling sequences from images is crucial for preserving design intent and supporting parametric editing. However, existing methods typically generate full CAD sequences holistically, overlooking the iterative, feedback-driven nature of human design workflows. We address this limitation by introducing the rich stepwise visual supervision: at each modeling step, the system observes the target's orthographic projections, the projections of the incrementally constructed model, and the active sketch, enabling informed action selection. To effectively leverage this on-the-fly feedback, we propose SOV-CAD, a framework that formulates CAD reconstruction as a sequential decision-making task and employs offline reinforcement learning with a Decision Transformer architecture. This design incorporates continuous visual feedback guided by geometric alignment rewards, resulting in a more accurate and human-like modeling process. Extensive experiments show that SOV-CAD surpasses state-of-the-art methods in CAD sequence reconstruction while exhibiting strong data efficiency. Code of SOV-CAD is available at: https://github.com/LukePhong/SOV-CAD

Figures

Figures reproduced from arXiv: 2607.04119 by Chen Zhi, Xinkui Zhao, Xuhong Zhang, Zhaopeng Feng, Zhengwen Feng.

Figure 1
Figure 1. Figure 1: An illustration of the step-by-step CAD model construction process, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An overview of our model architecture. (a) The target model’s three orthographic views are encoded into prefix tokens. (b) At each step [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Results of reconstructing CAD modeling sequences using orthogonal three-view images on the DeepCAD test set, comparing ours with other methods. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 2 linked inside Pith

  1. [1]

    Fusion 360 gallery: a dataset and environment for programmatic cad construction from human design sequences,

    Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, and Wojciech Matusik, “Fusion 360 gallery: a dataset and environment for programmatic cad construction from human design sequences,”ACM Trans. Graph., vol. 40, no. 4, July 2021

    2021

  2. [2]

    Img2cad: Reverse engineering 3d cad models from images through vlm-assisted conditional factorization,

    Yang You, Mikaela Angelina Uy, Jiaqi Han, Rahul Thomas, Haotong Zhang, Yi Du, Hansheng Chen, Francis Engelmann, Suya You, and Leonidas Guibas, “Img2cad: Reverse engineering 3d cad models from images through vlm-assisted conditional factorization,” 2025

    2025

  3. [3]

    Wonder3d: Single image to 3d using cross-domain diffusion,

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al., “Wonder3d: Single image to 3d using cross-domain diffusion,”arXiv preprint arXiv:2310.15008, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Hola: B-rep generation using a holistic latent representation,

    Yilin Liu, Duoteng Xu, Xingyao Yu, Xiang Xu, Daniel Cohen-Or, Hao Zhang, and Hui Huang, “Hola: B-rep generation using a holistic latent representation,”ACM TOG (SIGGRAPH), pp. 116:1–116:25, 2025

    2025

  5. [5]

    Mamba- cad: State space model for 3d computer-aided design generative mod- eling,

    Xueyang Li, Yunzhong Lou, Yu Song, and Xiangdong Zhou, “Mamba- cad: State space model for 3d computer-aided design generative mod- eling,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 5, pp. 5013–5021, Apr. 2025

    2025

  6. [6]

    Cadcrafter: Generating computer- aided design models from unconstrained images,

    Cheng Chen, Jiacheng Wei, Tianrun Chen, Chi Zhang, Xiaofeng Yang, Shangzhan Zhang, Bingchen Yang, Chuan-Sheng Foo, Guosheng Lin, Qixing Huang, and Fayao Liu, “Cadcrafter: Generating computer- aided design models from unconstrained images,” inProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), June 2025, pp. 11073–11082

    2025

  7. [7]

    Text-to-cad generation through infusing visual feedback in large language models,

    Ruiyu Wang, Yu Yuan, Shizhao Sun, and Jiang Bian, “Text-to-cad generation through infusing visual feedback in large language models,” inInternational Conference on Machine Learning, 2025

    2025

  8. [8]

    Generating cad code with vision-language models for 3d designs,

    Kamel Alrashedy, Pradyumna Tambwekar, Zulfiqar Zaidi, Megan Lang- wasser, Wei Xu, and Matthew Gombolay, “Generating cad code with vision-language models for 3d designs,” 2025

    2025

  9. [9]

    Seek-cad: A self-refined generative modeling for 3d parametric cad using local inference via deepseek,

    Xueyang Li, Jiahao Li, Yu Song, Yunzhong Lou, and Xiangdong Zhou, “Seek-cad: A self-refined generative modeling for 3d parametric cad using local inference via deepseek,” 2026

    2026

  10. [10]

    Decision transformer: reinforcement learning via sequence modeling,

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch, “Decision transformer: reinforcement learning via sequence modeling,” inProceedings of the 35th International Conference on Neural Infor- mation Processing Systems. 2021, NIPS ’21, Curran Associates Inc

    2021

  11. [11]

    Deepcad: A deep generative network for computer-aided design models,

    Rundi Wu, Chang Xiao, and Changxi Zheng, “Deepcad: A deep generative network for computer-aided design models,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2021, pp. 6772–6782

    2021

  12. [12]

    SkexGen: Autore- gressive generation of CAD construction sequences with disentangled codebooks,

    Xiang Xu, Karl D.D. Willis, Joseph G. Lambourne, Chin-Yi Cheng, Pradeep Kumar Jayaraman, and Yasutaka Furukawa, “SkexGen: Autore- gressive generation of CAD construction sequences with disentangled codebooks,” inInternational Conference on Machine Learning, 2022

    2022

  13. [13]

    Cadparser: A learning approach of sequence modeling for b-rep cad,

    Shengdi Zhou, Tianyi Tang, and Bin Zhou, “Cadparser: A learning approach of sequence modeling for b-rep cad,” inProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, Edith Elkind, Ed. 8 2023, pp. 1804–1812, International Joint Conferences on Artificial Intelligence Organization, Main Track

    2023

  14. [14]

    Hierarchical neural coding for control- lable cad model generation,

    Xiang Xu, Pradeep Kumar Jayaraman, Joseph G. Lambourne, Karl D.D. Willis, and Yasutaka Furukawa, “Hierarchical neural coding for control- lable cad model generation,” inProceedings of the 40th International Conference on Machine Learning. 2023, ICML’23, JMLR.org

    2023

  15. [15]

    From 2d cad drawings to 3d parametric models: a vision-language approach,

    Xilin Wang, Jia Zheng, Yuanchao Hu, Hao Zhu, Qian Yu, and Zi- han Zhou, “From 2d cad drawings to 3d parametric models: a vision-language approach,” inProceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances ...

    2025

  16. [16]

    Flexcad: Unified and versatile controllable cad generation with fine-tuned large language models,

    Zhanwei Zhang, Shizhao Sun, Wenxiao Wang, Deng Cai, and Jiang Bian, “Flexcad: Unified and versatile controllable cad generation with fine-tuned large language models,” inICLR, 2025

    2025

  17. [17]

    Plankassembly: Robust 3d reconstruction from three orthographic views with learnt shape programs,

    Wentao Hu, Jia Zheng, Zixin Zhang, Xiaojun Yuan, Jian Yin, and Zihan Zhou, “Plankassembly: Robust 3d reconstruction from three orthographic views with learnt shape programs,” inICCV, 2023

    2023

  18. [18]

    Caddreamer: Cad object generation from single-view images,

    Yuan Li, Cheng Lin, Yuan Liu, Xiaoxiao Long, Chenxu Zhang, Ningna Wang, Xin Li, Wenping Wang, and Xiaohu Guo, “Caddreamer: Cad object generation from single-view images,” inProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), June 2025, pp. 21448–21457

    2025

  19. [19]

    Cad-assistant: Tool-augmented vllms as generic cad task solvers,

    Dimitrios Mallis, Ahmet Serdar Karadeniz, Sebastian Cavada, Danila Rukhovich, Niki Maria Foteinopoulou, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada, “Cad-assistant: Tool-augmented vllms as generic cad task solvers,”International Conference on Computer Vision (ICCV), 2025

    2025

  20. [20]

    Cad-mllm: Unifying multimodality-conditioned cad generation with mllm,

    Jingwei Xu, Chenyu Wang, Zibo Zhao, Wen Liu, Yi Ma, and Shenghua Gao, “Cad-mllm: Unifying multimodality-conditioned cad generation with mllm,”arXiv preprint arXiv:2411.04954, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Cad-gpt: Synthesising cad construction sequence with spatial reasoning-enhanced multimodal llms,

    Siyu Wang, Cailian Chen, Xinyi Le, Qimin Xu, Lei Xu, Yanzhou Zhang, and Jie Yang, “Cad-gpt: Synthesising cad construction sequence with spatial reasoning-enhanced multimodal llms,”Proceedings of the AAAI Conference on Artificial Intelligence, p. 7880–7888, Apr. 2025

    2025

  22. [22]

    Reinforcement learning-based parametric cad models reconstruction from 2d orthographic drawings,

    Chao Zhang, Arnaud Polette, Romain Pinqui ´e, Mirai Iida, Henri De Charnace, and Jean-Philippe Pernot, “Reinforcement learning-based parametric cad models reconstruction from 2d orthographic drawings,” Computer-Aided Design, vol. 188, pp. 103925, 2025

    2025

  23. [23]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021

    2021

  24. [24]

    Implementation and benchmarking of perceptual image hash functions,

    Christoph Zauner, “Implementation and benchmarking of perceptual image hash functions,” 2010

    2010

  25. [25]

    Open cascade technology,

    Open CASCADE Technology, “Open cascade technology,” https://dev.opencascade.org/, 2025, Accessed: 2025-06-15

    2025