{"id":"e1b5fe5f-508a-4315-97be-800e49d8e77b","arxiv_id":"2412.14042","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CAD-Recode is the first LLM-based CAD reverse engineering method that converts point clouds into executable CadQuery Python code, and it reports state-of-the-art Chamfer distance and IoU on DeepCAD, Fusion360, and CC3D.","lead":"CAD-Recode turns a 3D point cloud of an object into Python code that, when executed, rebuilds the object as an editable CAD model. It pairs a small language model with a lightweight point-cloud projector and trains on a million procedurally generated CAD examples, reporting large gains over prior methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation protocol mismatch: borrowed baseline numbers and test-time reranking by Chamfer distance may inflate CAD-Recode's reported margin, though the no-sampling ablation partially mitigates.","rationale":"The paper makes a credible empirical case: an LLM decoder with a point-cloud projector can generate executable CadQuery code, and the procedural dataset provides a large training signal. The internal ablations (Table 3) are the strongest evidence: training on 160k procedural samples beats training on DeepCAD with the same amount of data, and scaling to 1M helps further. The test-time reranking is shown to mostly reduce invalidity, not to inflate CD/IoU, and even without it CAD-Recode beats CAD-SIGNet by a large margin. Thus the central architecture claim is not seriously threatened by the reranking concern. The load-bearing weakness is the external evaluation: baseline numbers are harvested from different papers with potentially different protocols, and no code or dataset is released to allow independent reproduction. This makes the precise margin of improvement (ten-fold CD, +10-15 IoU) uncertain. A conditional acceptance is appropriate until the authors release code/data or a unified benchmark re-run is performed. We therefore agree with the reader's identification of the evaluation protocol as the weakest assumption, while noting that the paper's own ablations partially mitigate it.","tokens_in":26008,"tokens_out":15764,"duration_ms":139660,"concrete_test":"Re-run at least CAD-SIGNet and CAD-Diffuser under CAD-Recode's exact evaluation protocol: same point-cloud sampling densities, 8192-point Chamfer distance, and the same ten-candidate reranking that selects the code with minimal CD to the input. Recompute Table 1 and compare. If the mean-CD ratio between the best baseline and CAD-Recode drops from the reported ~10x to below ~5x, the headline margin is substantially a protocol artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of 'significantly outperforming existing methods' rests on Table 1, where most baseline numbers are borrowed from prior papers (Section 5.1) and CAD-Recode is evaluated with a test-time reranking that generates ten codes from different input samplings and selects the one with minimum Chamfer distance to the input cloud (Section 4.3). Because the selection criterion is the evaluation metric itself, the reported mean CD is a minimum over ten random draws, not the model's expected single-sample performance. The borrowed baselines were not re-run under this exact protocol (same point-cloud sampling, same 8192-point CD, same reranking), so the comparison may not be apples-to-apples. However, Table 3 shows that CAD-Recode without reranking still outperforms CAD-SIGNet (0.75 vs 6.81 mean CD on DeepCAD), so the qualitative conclusion is robust to this protocol difference. The concern is therefore about the magnitude of the claimed improvement and the fairness of comparisons to other baselines, not about whether the method works at all.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CAD-Recode, a method that maps point clouds to executable Python/CadQuery code representing sketch-extrude CAD sequences. The architecture combines a lightweight point-cloud projector (furthest point sampling, Fourier positional encoding, and a linear projection) with a fine-tuned Qwen2-1.5B LLM decoder. The authors introduce a procedurally generated 1M-sample training dataset and evaluate on DeepCAD, Fusion360, and the real-world CC3D dataset, reporting large improvements in Chamfer distance, IoU, and invalidity ratio over prior reverse-engineering methods. They also demonstrate that the generated code can be interpreted by an off-the-shelf LLM for CAD question answering and interactive editing.","tokens_in":26210,"tokens_out":7867,"duration_ms":68570,"significance":"If confirmed, the paper's contributions are significant: a CAD-code representation that is directly executable and LLM-readable, a demonstration that procedural training data can outperform a hand-collected dataset of the same size, and an order-of-magnitude improvement on point-cloud CAD reverse engineering. The no-sampling ablation in Table 3 is a valuable internal control and supports the qualitative conclusion that the method works even without Chamfer-based reranking. The paper also provides detailed dataset-generation algorithms, validation steps (syntax checks, BRepCheck, duplicate detection), and a transparent single-stage training setup, which are strengths. However, the reported magnitude of the improvement is currently hard to verify because the main comparisons mix borrowed baseline numbers with a test-time reranking protocol that optimizes the evaluation metric, and because all results are single-run point estimates.","major_comments":[{"comment":"The test-time procedure of Section 4.3 generates ten code candidates from different point-cloud samplings and selects the one with minimum Chamfer distance to the input cloud. Since this selection criterion is the same metric on which the method is then evaluated, the reported CD values are minima over ten draws, rather than single-sample expectations. Most baseline numbers in Table 1 are borrowed from earlier papers ([34], [21], [12]) and were not produced under this protocol. The no-sampling rows in Table 3 show that the qualitative conclusion is robust for CAD-SIGNet, but the headline 'ten-fold mean CD improvement' and the IoU margins in Table 1 may still be materially inflated by the protocol asymmetry. Please re-run all baselines under a common evaluation harness with identical point-cloud sampling, identical 8192-point CD computation, and identical ten-candidate Chamfer selection, or state the headline comparisons in the single-sample setting and report both.","section":"Sections 4.3 and 5.1, Table 1"},{"comment":"The paper does not specify whether CD and IoU are averaged over all ten generated candidates, over the single selected candidate, or only over executable/valid candidates. This matters because invalid predictions exist (Figure 14) and because Table 3 shows test-time sampling changes the invalidity ratio from 4.9% to 0.4% for the 1M model. If the reported CD/IoU are computed only after discarding invalid codes (or only on the selected candidate), the comparison with baselines that report over all predictions is not apples-to-apples. Please state the exact aggregation rule and, ideally, report both per-candidate and selected-candidate numbers.","section":"Section 5.1, metric definitions"},{"comment":"All quantitative results are single-run point estimates with no error bars, confidence intervals, or multiple seeds. The paper repeatedly claims that CAD-Recode 'significantly outperforms' prior methods and that the procedural dataset is beneficial; these claims are supported only by point estimates. Please add multiple-seed runs (or equivalent variance information) for the main comparisons and the key ablations, and perform a significance test or state the number of seeds used.","section":"Section 5.1, Tables 1-4 and Table 6"}],"minor_comments":[{"comment":"The reference to 'Table 6 of the main paper' should be Table 5, which contains the CAD-QA results.","section":"Appendix F.2"},{"comment":"Generate2DSketch() returns the boundary components of a single generated sketch, yet the loop 'for sketch in sketches' implies multiple sketches; please clarify the pseudocode to make the generation procedure unambiguous.","section":"Appendix B, Algorithm 2"},{"comment":"Table 4 reports median CD but the caption only says 'CD'; the appendix Table 6 clarifies this, but the main table should state 'median CD' explicitly.","section":"Section 5.1, Table 4"},{"comment":"The sentence that CAD-Recode achieves 'comparable performance to the state-of-the-art on the command type accuracy' understates the actual numbers, since CAD-Diffuser attains 88.5% versus 83.9% for CAD-Recode; please rephrase to be precise about the comparison.","section":"Appendix D, Table 7 discussion"},{"comment":"The caption contains a typo: 'CADSIGNet' should be 'CAD-SIGNet'.","section":"Figure 17 caption"},{"comment":"The paper states that the 1M dataset will be made publicly accessible but does not provide a repository link or a release timeline; adding this information would improve reproducibility.","section":"Abstract and Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of a computer vision journal and the core idea is promising. My recommendation is driven by the need to make the evaluation protocol and statistical claims reproducible; I do not see a fundamental correctness error. The authors' reliance on their own prior work as baselines is understandable, but the borrowed-number comparisons should be replaced or supplemented with a common evaluation harness before the reported margins are taken at face value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version. The core contribution is credible and the empirical support is stronger than the evaluation caveats might suggest. CAD-Recode translates point clouds into CadQuery Python code that, when executed, reconstructs the CAD model. That representation is a genuinely useful idea: it is directly executable, editable, and, as they show, interpretable by off-the-shelf LLMs. The architecture is simple—a point cloud projector (FPS, Fourier encoding, linear layer) feeding a fine-tuned Qwen2-1.5B. The 1M procedural dataset is a real contribution, even if not yet public.\n\nWhere the paper earns its keep: the ablations are honest. Training on the procedural dataset beats training on DeepCAD, scaling to 1M helps, and they report the no-test-time-sampling numbers. In Table 3, the no-sampling row still shows about a 9x improvement in mean CD over CAD-SIGNet on DeepCAD (0.75 vs 6.81). So the qualitative conclusion that the method is better holds up without the reranking. The appendix is transparent about failure modes: invalid predictions from quantization of tiny features, or from ground-truth operations (revolution, B-spline) outside the training distribution. They even note the command/parameter accuracy metrics are not sequence-aware because multiple sequences can describe the same model. That is honest.\n\nThe soft spots are exactly where the stress-test points: evaluation protocol mismatch. Baseline numbers are borrowed from other papers and not re-run under the same point-cloud sampling or the same best-of-10 reranking. The test-time reranking selects the candidate with minimum Chamfer distance to the input cloud—the same metric being reported—so the mean CD is a minimum over ten draws, not an expectation. That can inflate the margin, though not the direction. Add single-run results with no error bars and no code/data release yet. These don't sink the paper, but they cap how much you should trust exact numbers.\n\nWho this is for: anyone working in CAD reverse engineering, LLM-based 3D understanding, or program synthesis for geometry. It deserves a serious referee. I'd send it out, and in review ask for released code and dataset, re-run or clearly stated baseline protocols, and multi-seed results. Recommendation: engage with it. The representation idea alone is worth having in the literature.","headline":"Solid new idea, honest ablations, but borrowed baselines and metric-aware reranking inflate the margin; still deserves a serious referee.","tokens_in":26758,"tokens_out":4059,"would_cite":true,"duration_ms":34488,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CAD-Recode claims that CAD reverse engineering can be recast as a code-generation problem: a point cloud is translated into executable Python code (using the CadQuery library) that reconstructs the CAD model.","keywords":["CAD reverse engineering","point clouds","large language models","CadQuery","Python code generation","procedural dataset","sketch-extrude sequences","parametric CAD"],"falsifier":"Run a head-to-head comparison in which CAD-SIGNet (and other baselines) are given the same ten-candidate, Chamfer-reranked inference and identical input sampling, and see whether the DeepCAD/Fusion360/CC3D margins persist. Also probe the generalization boundary: feed CAD-Recode point clouds of models containing revolutions, fillets, or B-splines — features absent from its procedural training set — and measure how often the predicted code fails to reconstruct the shape.","tokens_in":25796,"feed_emoji":"⚙️","tokens_out":6357,"duration_ms":49831,"temperature":0.7,"pith_summary":"CAD-Recode claims that CAD reverse engineering can be recast as a code-generation problem: a point cloud is translated into executable Python code (using the CadQuery library) that reconstructs the CAD model. The paper argues this works because pre-trained large language models already know Python syntax, so a relatively small LLM (Qwen2-1.5B) can be fine-tuned with a lightweight point-cloud projector to produce valid CAD programs. To overcome the scarcity of human CAD sequences, the authors procedurally generate one million sketch-extrude models as CadQuery code. On the DeepCAD, Fusion360, and real-world CC3D datasets, the method reports substantially lower Chamfer distance and higher IoU than prior sketch-extrude reverse-engineering methods, and the generated code is directly interpretable by off-the-shelf LLMs for question answering and editing. If these results hold, they imply that LLMs plus a large procedural training set can make CAD reverse engineering practical, producing models that are both accurate and editable.","feed_headline":"LLM turns point clouds into editable CAD code","feed_subtitle":"Fine-tuned on one million procedurally generated models, the method beats earlier CAD reverse engineering baselines.","key_machinery":"The mechanism is a two-part architecture: (1) a point-cloud projector that downsamples the input to 256 points, applies Fourier positional encoding to the coordinates, and linearly projects them into a token sequence; and (2) a fine-tuned Qwen2-1.5B LLM that autoregressively generates CadQuery Python code from those tokens. The third load-bearing element is the procedural dataset generator, which builds 1M valid sketch-extrude sequences via randomized heuristics (sketch primitives, boolean operations, planes, extrusions, then union), verifies each with syntactic and geometric checks, and quantizes parameters to a fixed grid. Together these let the model sidestep custom CAD-language design and instead reuse the LLM's pre-existing code knowledge.","core_discovery":"The central claim is that representing CAD sketch-extrude sequences as CadQuery Python code lets a pre-trained LLM serve as the decoder for point-cloud-to-CAD reconstruction, provided it is trained on a sufficiently large and controllable procedural dataset. The paper reports that CAD-Recode, trained on 1M generated models, achieves a mean Chamfer distance of 0.30 on the DeepCAD test set and 0.35 on Fusion360, versus 3.43 and 7.37 for the previous best method CAD-SIGNet, with IoU rising from 77.6% to 92.0% and from 65.6% to 87.8% respectively; on the scanned CC3D set, median CD drops from 2.90 to 0.31. The authors also show the predicted code can be fed to a general-purpose LLM to answer CAD-specific geometry questions (76.5% accuracy on SGP-Bench, versus 63.2% with CAD-SIGNet's output) and to generate interactive editing sliders. The underlying assertion is that the code representation absorbs most of the 'design intent' burden, so the network only needs to learn a geometry-to-token mapping rather than a new CAD-specific syntax.","pith_inferences":["If the LLM's Python prior is the main source of the gains, then scaling to a larger LLM or a code-specialised model should improve geometry fidelity further; this is a direct testable extension of the paper's stated scaling plans.","The procedural dataset distribution is fully controllable, so it could be re-weighted to target specific failure modes (thin features, non-extrusion operations) and thereby reduce the invalid-prediction cases the authors document.","The Chamfer-reranking step effectively turns the generative model into a search over a latent code space; a learned verifier could do the same job at a fraction of the compute, and might also detect invalid code before execution.","The approach's reliance on CadQuery ties it to that library's kernel; porting the representation to another programmatic CAD API would be a quick way to test how much of the benefit comes from the code format itself versus the LLM's prior."],"forward_implications":["CAD reverse engineering becomes a language-generation task, so future gains in LLM code ability should transfer directly to reconstruction quality.","The procedural dataset generator can be extended to more operations (revolution, fillet, patterns), potentially scaling to more complex real-world models.","The code output enables a new workflow: a designer scans an object, gets an editable CadQuery script, and modifies parameters through sliders or natural language.","Because the output is executable code, reconstruction validity can be checked by simply running it, which provides a clean training signal for geometric correctness.","The ten-candidate reranking suggests a general verification-based inference strategy that other CAD-sequence models could adopt."],"supporting_citations":[{"why":"Supplies the DeepCAD dataset and sketch-extrude representation that the method converts to CadQuery code and uses as a training and evaluation baseline.","marker":"[48]"},{"why":"Provides the previous state-of-the-art method CAD-SIGNet and the test-time sampling strategy that CAD-Recode adapts for inference.","marker":"[21]"},{"why":"Defines the Qwen2-1.5B pre-trained LLM that serves as the fine-tuned decoder backbone.","marker":"[55]"},{"why":"Defines the CadQuery Python library that supplies the executable code representation central to the method.","marker":"[10]"},{"why":"Is the source of the baseline numbers and the IoU metric used in the main comparison table.","marker":"[34]"},{"why":"Provides the Fusion360 gallery dataset used for evaluation.","marker":"[47]"},{"why":"Provides the real-world CC3D scan dataset used to test robustness under noise and missing parts.","marker":"[35]"},{"why":"Supplies the SGP-Bench CAD question-answering benchmark used to measure the interpretability of the generated code.","marker":"[38]"}],"fun_headline_variants":["LLM writes CAD code from point clouds, beats prior methods","Point clouds become editable CAD via LLM-generated Python code","CAD-Recode: LLM turns point clouds into Python CAD models","From point cloud to CAD: LLM generates the code"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains rely on the evaluation protocol being fair: CAD-Recode uses test-time reranking over ten generated code candidates chosen by Chamfer distance to the input, while the comparison numbers for other methods are taken from earlier papers that may have used different point-cloud sampling or no equivalent reranking.","fun_headline_variants_meta":{"raw":{"variants":["LLM writes CAD code from point clouds, beats prior methods","Point clouds become editable CAD via LLM-generated Python code","CAD-Recode: LLM turns point clouds into Python CAD models","From point cloud to CAD: LLM generates the code"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000465,"raw_usage":{"total_tokens":2352,"prompt_tokens":1004,"completion_tokens":1348,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":1277}},"tokens_in":620,"tokens_out":1348,"duration_ms":10181,"temperature":1.0,"reasoning_tokens":1277,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:31:51.236297+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a head-to-head comparison in which CAD-SIGNet (and other baselines) are given the same ten-candidate, Chamfer-reranked inference and identical input sampling, and see whether the DeepCAD/Fusion360/CC3D margins persist. Also probe the generalization boundary: feed CAD-Recode point clouds of models containing revolutions, fillets, or B-splines — features absent from its procedural training set — and measure how often the predicted code fails to reconstruct the shape.","supporting_citations":[{"cited_title":"Deepcad: A deep generative network for computer-aided design models","cited_arxiv_id":null,"evidence_quote":"Supplies the DeepCAD dataset and sketch-extrude representation that the method converts to CadQuery code and uses as a training and evaluation baseline."},{"cited_title":"Cad-signet: Cad language inference from point clouds using layer-wise sketch instance guided attention","cited_arxiv_id":null,"evidence_quote":"Provides the previous state-of-the-art method CAD-SIGNet and the test-time sampling strategy that CAD-Recode adapts for inference."},{"cited_title":"Qwen2 technical report, 2024","cited_arxiv_id":null,"evidence_quote":"Defines the Qwen2-1.5B pre-trained LLM that serves as the fine-tuned decoder backbone."},{"cited_title":"Cadquery: A python paramet- ric cad scripting framework","cited_arxiv_id":null,"evidence_quote":"Defines the CadQuery Python library that supplies the executable code representation central to the method."},{"cited_title":"Draw step by step: Reconstructing cad construction sequences from point clouds via multimodal diffusion","cited_arxiv_id":null,"evidence_quote":"Is the source of the baseline numbers and the IoU metric used in the main comparison table."},{"cited_title":"Fusion 360 gallery: A dataset and environ- ment for programmatic cad construction from human design sequences","cited_arxiv_id":null,"evidence_quote":"Provides the Fusion360 gallery dataset used for evaluation."},{"cited_title":"Sharp challenge 2023: Solving cad history and parameters recovery from point clouds and 3d scans","cited_arxiv_id":null,"evidence_quote":"Provides the real-world CC3D scan dataset used to test robustness under noise and missing parts."}],"review_version":1}