{"id":"c96da8a0-ea9f-421d-88a8-4dcb257d29ff","arxiv_id":"2412.11892","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CAD2Program fine-tunes a vision-language model to read rasterized 2D CAD drawings and write text programs that reconstruct 3D parametric cabinet models, matching vector-based baselines on standard benchmarks.","lead":"This paper presents CAD2Program, a system that turns ordinary 2D CAD drawings into 3D parametric cabinet models by feeding the drawing as a picture to a vision-language model and reading out a text program. It shows that raster images can carry enough information for this task and that using natural-language-style output handles many more part types than earlier fixed-slot methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The raster-input premise is only validated on simplified three-view plank drawings (Sec. 4.2); no vector baseline is run on the cabinet dataset, so the central claim that pixel input loses no task-relevant information in the intended regime is unsupported.","rationale":"The reader identified the same load-bearing premise: rasterization must not lose task-relevant information, and the PlankAssembly(ViT) experiment on simple three-view drawings does not prove this for arbitrary layouts or mixed annotation layers. The reader's CONDITIONAL verdict is appropriate because the paper is otherwise well-constructed: the annotation ablation, output-representation comparison, and proxy-language ablation are sensible, and the reported gains are large enough to be credible if the artifacts and baseline are provided. My stress-test pass does not find an internal contradiction or a fatal flaw; rather, it sharpens the condition. The decisive missing experiment is an end-to-end vector baseline on the cabinet dataset, which would directly test whether the raster-only input is truly competitive in the regime the paper claims to address. Until that baseline exists, the central claim rests on an extrapolation from a simplified setting. I therefore keep the verdict CONDITIONAL: the paper should be accepted if the authors supply the vector baseline and, ideally, the dataset/code for independent reproduction. No ad hominem or theatrical language is needed; the issue is purely the scope of the empirical support.","tokens_in":14803,"tokens_out":7909,"duration_ms":81856,"concrete_test":"","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that treating the 2D CAD drawing as a raster image and encoding it with a ViT achieves competitive performance against vector-graphics-input methods while imposing fewer restrictions. The only direct support for this is the PlankAssembly(ViT) comparison in Sec. 4.2 (Figure 6), which uses TinyViT on a fixed 512x512 canvas with exactly three aligned, annotation-free views. That setting does not exercise the properties the paper cites as the motivation for the cabinet domain: varying numbers of views, unaligned views, mixed geometry/annotation layers, 373 primitive IDs, model-specific parameters, or the need to read dimension text. The CAD2Program results on the cabinet dataset (Tables 1-4, Fig. 7) have no vector-input baseline at all, so the paper never measures whether the rasterized representation preserves all information required for these harder inputs. A vector encoder could exploit exact layer metadata and coordinate precision; the ViT must re-estimate them from pixels. The annotation-layer ablation (Table 1) shows the model can use dimension text, but not that it recovers everything a vector encoder sees. This is a load-bearing gap: if a vector baseline on the cabinet dataset outperforms CAD2Program, the central claim fails; if it does not, the claim is plausible but currently unverified. The weakness is external underdetermination, not internal inconsistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"CAD2Program fine-tunes Mini-InternVL-1.5-2B to reconstruct 3D parametric cabinet models from 2D CAD drawings. The input is the raw rasterized drawing, encoded by a ViT, and the output is an autoregressively generated Python-style text program listing primitive model IDs, common box parameters, and model-specific parameters. The paper introduces a new dataset of 368K cabinet models with 373 primitives and 702 model-specific parameters, and reports experiments on the raster-input assumption (via a ViT variant of PlankAssembly), the usefulness of annotation layers, output representation (command template vs. text), proxy language, and base VLM. The central claims are that a general ViT can match vector-graphics encoders and that a general-purpose text representation handles a large set of primitives more flexibly than fixed-slot command templates.","tokens_in":14983,"tokens_out":5335,"duration_ms":47997,"significance":"The paper's result, if fully established, would be practically significant: it removes the need for vector-format parsing, geometry-layer separation, and exactly three aligned views, and scales reconstruction to hundreds of primitive types. The component-level comparisons are well designed in several respects: the PlankAssembly (ViT) experiment reuses the same code, data, and decoder as the vector baseline; the annotation-layer ablation is performed on the new dataset with matched architecture; and the output-representation comparison holds the architecture fixed while changing only the sequence format. The release of a large cabinet dataset and the use of publicly available base models are additional strengths. The main gap is that the raster-input premise is only tested on the simple PlankAssembly setting, not on the cabinet setting that motivates the paper.","major_comments":[{"comment":"The only direct evidence for the claim that a raster image encoder is as effective as a vector-graphics encoder comes from PlankAssembly (ViT) on exactly three aligned, annotation-free views on a fixed 512x512 canvas. The cabinet dataset used in Tables 1-4 and Figure 7 is precisely the regime in which the paper claims fewer restrictions (varying view counts, unaligned views, mixed geometry/annotation layers, 373 primitives, reading of dimension text), yet no vector-input baseline is run there. If a vector encoder with layer metadata and coordinate precision outperforms CAD2Program on this dataset, the central claim fails; if it does not, the claim is currently unverified. I request either a vector-based baseline on the cabinet dataset or an oracle-style experiment that feeds exact vector entities and layer metadata into the same decoder, so that the information loss from rasterization is measured in the intended setting.","section":"§4.2, Figure 6"},{"comment":"All quantitative results are reported for a single training run, without standard deviations, confidence intervals, or significance tests. Several comparative statements rest on small margins (e.g., Table 2 retrieval 93.84 vs. 93.36; Table 4 F1 84.90 vs. 82.76, retrieval 93.98 vs. 93.80), and the claim of 'comparable performance' cannot be distinguished from training noise at these margins. Please report at least three seeds and the corresponding variance for the main comparisons, or a paired test on the test set.","section":"Tables 1-4"}],"minor_comments":[{"comment":"The text 'Figure 9 illustrates the two layers' should refer to Figure 1, which is the figure that actually shows the geometry and annotation layers; Figure 9 is an appendix figure.","section":"Section 1"},{"comment":"The parameter 'uCove=18' in Listing 1 is inconsistent with 'uCover' used in Listing 2 and the example in Figure 10; this is likely a typo.","section":"Listing 1"},{"comment":"The claim that treating values as text avoids quantization error should state the numerical precision of the serialized values in the dataset (e.g., one decimal place), since the text tokenizer itself imposes representational limits; the claim is only true relative to the chosen binning of the command-template baseline.","section":"Section 3.1"},{"comment":"The traditional-approach curve is taken directly from prior work, but the paper should specify whether the same noise-level definitions and plotting conventions were used for PlankAssembly (ViT) so that the comparison is fair.","section":"§4.2, Figure 6"},{"comment":"The interpretation of Table 2 should state explicitly that the command-template comparison covers only the common-parameter subset, since the command-template variant does not predict model-specific parameters; the current presentation implies text output is as effective across the full task.","section":"§4.3, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The main risk is not internal inconsistency but missing evidence for the central claim. If the authors can add a vector-input baseline on the cabinet dataset, the paper would be considerably stronger. The reliance on PlankAssembly as both baseline and metric is transparent but should be noted; a reviewer from a different subcommunity might ask for an independent implementation or at least a careful statement of what is reused."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-constructed empirical paper, and the text-based output is the strongest part. The idea of feeding the whole drawing as a raster image to a fine-tuned VLM and predicting a Python-like shape program is new relative to PlankAssembly, and the experiments are carefully matched. What the paper does well: the annotation-layer ablation (Table 1) is a clean demonstration that the model exploits dimension text and functional symbols, with a large jump in accuracy. The output-representation comparison (Table 2) is also fair—same architecture, text vs. fixed-slot command—and it shows text is at least as good while being far more flexible. The PlankAssembly (ViT) comparison in Sec. 4.2 is a good idea, but it is the only place where raster and vector inputs are directly compared, and it uses simple three-view, annotation-free drawings on a fixed canvas. The cabinet dataset, where the claims about varying views and mixed layers actually matter, has no vector baseline. So the central premise—that pixel input loses no task-relevant information in the intended regime—remains unverified. That is the main weak spot. There are also no error bars, single runs throughout, and no code or dataset release, which makes the headline numbers hard to check. The self-citation of PlankAssembly is transparent and the metric choice follows that work, so I don't see that as a serious problem. The limitations paragraph is honest about the cabinet-only scope. Overall, the paper is credible and the questions it asks are relevant, but the load-bearing comparison is missing. I would want a vector-input baseline on the cabinet data, or at least released artifacts for independent reproduction, before treating the raster advantage as established. A serious referee should engage with it; my recommendation is conditional acceptance with those requests.","headline":"Well-ablated VLM approach to 2D-CAD-to-3D-parametric reconstruction with a genuinely flexible text output, but the raster-vs-vector claim is only validated on simple planks, not the intended cabinet domain.","tokens_in":15613,"tokens_out":2140,"would_cite":true,"duration_ms":19549,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A vision-language model rebuilds 3D cabinets from 2D drawings","keywords":["CAD reconstruction","vision-language model","parametric modeling","raster image input","text shape program","orthographic drawings","annotation layers","autoregressive generation"],"falsifier":"Render a held-out set of cabinets in view layouts and annotation styles never seen in training—e.g., unaligned views, an unusual number of views, or dimension text in a nonstandard font—and compare CAD2Program against a vector-based baseline on those drawings. If the raster-input model's F1 falls far below the vector model's, the claim that pixel input is competitive while imposing fewer restrictions is refuted.","tokens_in":14506,"feed_emoji":"🛠️","tokens_out":13245,"duration_ms":107161,"temperature":0.7,"pith_summary":"CAD2Program claims that 3D parametric models can be reconstructed from 2D CAD drawings by treating the drawing as a raster image and letting a fine-tuned vision-language model write a text program describing the 3D model. The authors show that this pixel-based input matches the accuracy of prior vector-graphics methods on clean three-view drawings, while dropping their requirements: drawings need not be cleaned into a separate geometry layer, and the annotation layer—dimension text and functional symbols—becomes a source of information rather than noise. The output is a Python-style shape program, which the paper argues is more flexible than fixed-size command templates because it naturally handles 373 primitive types and 702 model-specific parameters. On a new dataset of 368K cabinet drawings, the reconstructed 3D boxes reach about 83 percent F1 and model-specific parameters are estimated correctly about 97 percent of the time, with the annotation layer boosting F1 by roughly 20 points. If the claim holds, CAD-to-3D reconstruction becomes less a problem of designing special data representations and more a problem of collecting data and fine-tuning an existing model.","feed_headline":"A vision-language model rebuilds 3D cabinets from 2D drawings","feed_subtitle":"Raster-in, text-out reconstruction hits about 83 percent F1, using annotations as signal, not noise.","key_machinery":"The central mechanism is a fine-tuned vision-language model with a ViT image encoder, an MLP projector, and a small autoregressive language model, trained to map a rasterized 2D CAD drawing directly to a text shape program. Each primitive instance in that program is written as a `Bbox(...)` line holding the common parameters (position, size, rotation) followed by a model call with its model-specific parameters written as `key=value` pairs, so no fixed-size command template is needed. To help the model identify which primitive a drawing region refers to, each model ID is represented by a special token built from the primitive's name and a rendered snapshot image, encoded with a contrastive vision-language model. The whole pipeline is fine-tuned end-to-end on a dataset of 368K cabinet drawings, with the language model generating the program tokens auto-regressively.","core_discovery":"The core claim is that a single vision-language model can solve the task end-to-end by treating the input as an image and the output as text. The input side passes the full drawing—both the geometry lines and the annotation symbols—through a ViT image encoder, and the output side auto-regressively generates a text shape program in which each primitive appears as a bounding-box line plus a model call with named parameters, e.g., `model_2 = <model_115813862>(N=1, NKA=928, DBXX=1, BT=18)`. This text representation removes the fixed-size command template used in prior sequence modeling, eliminates parameter quantization by writing values as text, and allows new primitive types to be added without restructuring the output. The paper demonstrates that the vector-specific PlankAssembly system performs comparably when its encoder is swapped for a generic ViT on the same three-view drawings, and that the full CAD2Program trained on cabinet drawings works with a different proxy language and a different vision-language backbone. The result is a method that uses annotations as input rather than stripping them, and that scales to hundreds of primitives and hundreds of model-specific parameters.","pith_inferences":["The same raster-in, text-out recipe could plausibly extend to other engineering documents whose meaning lives in mixed layers—P&ID diagrams, electrical schematics, or dimensioned mechanical drawings—though the paper only demonstrates cabinets.","The special token for model IDs uses a rendered snapshot with default parameter values, so primitives whose appearance changes sharply with their parameters could be hard to retrieve; a parameter-aware embedding would be a natural extension.","Since the output is executable text, the same fine-tuned weights could be prompted to edit a model program or answer questions about a drawing, effectively turning the model into a CAD assistant—a use the authors flag as future work.","The paper's 'fewer restrictions' claim is demonstrated inside the dataset's distribution of layouts and annotations; a stress test with deliberately arbitrary view arrangements and annotation styles would assess how far the generalization extends."],"forward_implications":["Reconstruction no longer requires a clean geometry layer: drawings can be fed in as-is, and the annotation layer measurably improves accuracy instead of being treated as noise.","The text-shaped output removes the fixed-slot constraint, allowing one model to work with 373 primitive types and 702 model-specific parameters without lengthening or restructuring the output sequence.","Common parameters are emitted as text rather than discretized tokens, so the quantization error of fixed-bin encoders disappears.","Because the output is free-form text, the same approach extends to new product categories and new primitive libraries by collecting a dataset, and the proxy language can be swapped without retraining the core method (Python and YAML give similar results).","A general-purpose vision encoder is as effective as a vector-specific encoder for three-view orthographic drawings, as shown when the PlankAssembly encoder is replaced by a ViT."],"supporting_citations":[{"why":"Provides the PlankAssembly baseline, the three-view vector-input method and dataset used to compare raster versus vector encoders, and the F1 evaluation protocol.","marker":"Hu et al. 2023"},{"why":"Supplies the open-source vision-language foundation model (Mini-InternVL) that CAD2Program fine-tunes with a ViT encoder and LLM decoder.","marker":"Chen et al. 2024a,b"},{"why":"Establishes the vision transformer (ViT) architecture used as the image encoder that reads the rasterized drawing.","marker":"Dosovitskiy et al. 2020"},{"why":"Represents the domain-specific sequence modeling approach with a fixed command template, the output-side contrast for the paper's text shape program.","marker":"Wu, Xiao, and Zheng 2021"},{"why":"Provides the contrastive image-text embeddings used to build the special token for each primitive model ID from a rendered snapshot and name.","marker":"Yang et al. 2023"}],"fun_headline_variants":["Vision-language model reads drawings, writes 3D programs","CAD2Program: From raster drawing to text-based 3D model","Treat CAD as image, output as text: new reconstruction","Annotations as input: a flexible VLM for 3D CAD","No vector parsing needed: VLM reconstructs from 2D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the premise that a CAD drawing can be read correctly from its pixels alone, so the vision encoder must recover geometry, dimensions, and annotation symbols without any vector entities or layer metadata.","fun_headline_variants_meta":{"raw":{"variants":["Vision-language model reads drawings, writes 3D programs","CAD2Program: From raster drawing to text-based 3D model","Treat CAD as image, output as text: new reconstruction","Annotations as input: a flexible VLM for 3D CAD","No vector parsing needed: VLM reconstructs from 2D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000355,"raw_usage":{"total_tokens":1946,"prompt_tokens":980,"completion_tokens":966,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":878}},"tokens_in":596,"tokens_out":966,"duration_ms":9277,"temperature":1.0,"reasoning_tokens":878,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:28:48.416186+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a held-out set of cabinets in view layouts and annotation styles never seen in training—e.g., unaligned views, an unusual number of views, or dimension text in a nonstandard font—and compare CAD2Program against a vector-based baseline on those drawings. If the raster-input model's F1 falls far below the vector model's, the claim that pixel input is competitive while imposing fewer restrictions is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the vision transformer (ViT) architecture used as the image encoder that reads the rasterized drawing."}],"review_version":1}