Pith. sign in

REVIEW 5 major objections 6 minor 13 references

A single 27B multimodal model trained on six industrial CAD corpora becomes a shared substrate that beats both its base and a frontier closed model on CAD vision, CadQuery, COM APIs, and assemblies—without forgetting general skills.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-31 19:24 UTC pith:MTIRAEIJ

load-bearing objection Solid applied SFT paper: real CadQuery/COM jumps on a scarce industrial stack, with the usual sibling-eval and small-n caveats—not a methods breakthrough, but worth engaging if you build CAD agents. the 5 major comments →

arxiv 2607.28050 v1 pith:MTIRAEIJ submitted 2026-07-30 cs.AI

IndustryForge-27B: A Domain-Enhanced Multimodal Foundation Model for Industrial CAD

classification cs.AI
keywords industrial CADmultimodal foundation modelmulti-task SFTCadQueryCOM APIassembly generationCAD visual QAindustrial agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Industrial CAD automation needs models that can read engineering drawings and 3D screenshots, emit correct parametric scripts and Windows COM API code, and handle both single parts and assemblies. General multimodal models fail on those axes, and single-task fine-tunes are too narrow for the mixed calls real agents make. This paper builds IndustryForge-27B by multi-task supervised fine-tuning of Qwen3.5-VL-27B on about 52k curated multimodal samples spanning CAD visual QA, CadQuery parts and assemblies, and COM code for Inventor and SolidWorks. On four CAD-domain benchmarks the model gains roughly 34 percentage points over its base and outperforms GPT-5.4 on every task, including the first usable non-zero assembly pass rate among the compared models, while slightly improving average scores on eleven general benchmarks. The authors present the result not as an end-to-end CAD system but as a common foundation so downstream agents need not re-solve basic CAD reading and coding from scratch.

Core claim

Multi-task supervised fine-tuning of a 27B vision-language model on six integrated industrial-CAD sub-corpora (~52k samples) produces a single foundation that substantially raises CAD-domain performance—mean +33.65 pp over the base, winning all four domain benchmarks against GPT-5.4—while retaining and slightly improving general capability (+1.56 pp mean on eleven public benchmarks, no catastrophic forgetting), thereby serving as a usable common substrate for full-stack industrial CAD agents.

What carries the argument

Unified multi-task SFT on six industrial-CAD sub-corpora (CAD-VQA, text2cadquery, text2cadquery-assembly, com_2d, com_3d, com_assembly), mixed flat and trained with LoRA, DeepSpeed ZeRO-3, sequence parallel, and padding-free packing, with sandbox execution, Chamfer-Distance filtering, and UUID/AST decontamination as quality gates.

Load-bearing premise

The four CAD benchmarks—especially the small COM set and the still-low assembly pass rate—plus decontamination against related pipelines are assumed independent and representative enough that the large gains prove a generally superior industrial CAD foundation rather than in-distribution wins on author-adjacent tests.

What would settle it

Re-evaluate IndustryForge-27B and the same baselines on a held-out industrial CAD suite built from unrelated plant drawings, COM workflows, and multi-part assemblies never seen in the six sub-corpora or sibling agent pipelines; if the mean domain lead over the base and over GPT-5.4 collapses or assembly pass rate stays near zero under that protocol, the substrate claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Downstream CAD agents can start from one shared checkpoint for drawing understanding, CadQuery, COM operation, and assembly instead of each re-solving basic CAD literacy.
  • COM-as-code operation of SolidWorks/Inventor becomes a first-class foundation skill rather than a rare, near-zero capability of general models.
  • Assembly-level generation moves from near-total failure (0–4%) to a non-zero usable baseline (~15%) that closed-loop agents can iterate on.
  • CAD-oriented multi-task SFT can spill over positively into geometric visual reasoning and structured figure-text tasks on general benchmarks without wiping out broad knowledge.
  • A planned feedback loop can recycle high-quality agent traces back into the next substrate round, linking single-shot SFT to closed-loop self-improvement.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the substrate pattern holds, other heavy Windows professional apps with scarce public COM/API data (beyond CAD) may benefit from the same curated sandbox-executed multi-task recipe.
  • The still-dominant single-part mix and ~15% assembly pass rate imply that scale and verification of assembly data—not just more SFT steps—will be the next bottleneck for production-ready multi-part systems.
  • Positive transfer on math_vision and chart-like tasks suggests industrial drawing SFT is a practical way to strengthen multi-view spatial reasoning in general multimodal models.
  • Open release of the substrate could shrink duplicated CAD-foundation work across teams and make fair head-to-head agent comparisons easier.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents IndustryForge-27B, a 27B multimodal model obtained by multi-task supervised fine-tuning of Qwen3.5-VL-27B on six industrial-CAD sub-corpora (~52k samples) covering CAD visual QA, CadQuery part/assembly code, and SolidWorks/Inventor COM APIs. It reports large gains on four author-curated CAD-domain benchmarks (mean +33.65 pp over the base; wins 4/4 vs GPT-5.4), with CadQuery rising 7.82%→77.88% and COM 28.33%→73.33%, while eleven general benchmarks show a small mean gain (+1.56 pp) and no catastrophic forgetting. The stated contribution is not an end-to-end CAD system but a shared foundation-model substrate for downstream agents (IterCAD, AssemCAD, ComAct, SimLoop).

Significance. If the domain gains transfer beyond the authors’ sibling evaluation stacks, the work would be a useful systems contribution: a single open multimodal checkpoint that jointly covers drawing understanding, parametric CadQuery, Windows COM operation, and a non-zero assembly pass rate—capabilities that general VLMs and single-task CAD fine-tunes currently split. Strengths include sandbox execution plus Chamfer-Distance filtering of training code, UUID/AST/Chamfer decontamination against listed benches, a clear pass@CD≤1e-3 metric, and explicit measurement of general-capability retention. The absolute assembly result (15.38%) and the COM lift where public data are scarcest are practically interesting if they hold under independent evaluation.

major comments (5)
  1. [§3.2.6, §4.1–4.2] §3.2.2–3.2.4 and §4.1–4.2: Three of the four domain benches (Zero-To-CAD / text2cadquery, com_QA_mini, CadQuery Assembly) are co-sourced with the same IterCAD / ComAct / AssemCAD pipelines that supply SFT data. UUID and AST-normalised / Chamfer decontamination (§3.2.6) block exact and trivial-variant leakage but do not address shared generators, COM helper idioms, geometry families, or template structure. The headline “+33.65 pp / wins 4/4 vs GPT-5.4” and the “common substrate” claim are load-bearing on transfer; the manuscript needs either an external or held-out plant-style split, or a quantitative analysis showing that pass@CD and COM artefact-match are not inflated by in-family homology.
  2. [§4.1, Figure 4] §4.1 and Figure 4: COM CAD is evaluated on com_QA_mini_0311 with only 60 samples. A jump 28.33%→73.33% (+45 pp) and +43.33 pp over GPT-5.4 is central to the differentiation narrative, but with n=60 binomial uncertainty is large and a few template families could dominate. Report confidence intervals, item-level breakdown by API family (2D/3D/assembly), and ideally a larger or independently sourced COM set before treating this as a settled closed-source gap.
  3. [§4.2, Table 1, Limitations] §4.2 and Limitations: Assembly pass@CD≤1e-3 reaches 15.38% from 0/3.85%, which is directionally important but still low for the claim of a “first usable non-zero” industrial foundation and for supporting upper-layer assembly agents. The training mix is only ~1k CadQuery-assembly + 5k COM-assembly against ~42k single-part samples (Table 1, Figure 3). Either strengthen the assembly evaluation (more tasks, partial-credit / constraint metrics, failure taxonomy) or temper the substrate/assembly claims to match the absolute level and data imbalance.
  4. [§1.2, §3.3.3, §4] §3.3 and §4: The paper argues multi-task SFT is necessary because single-task fine-tuning is “too narrow” for agent call diversity (§1.2), yet there is no ablation of joint training vs. single-corpus specialists (or vs. mixture-of-adapters) on the four domain benches and on cross-task interference. Without that, the unified-recipe claim and the flat sampling choice over a highly unbalanced mix (com_2d 20k vs assembly 1k) remain untested relative to the obvious alternative the introduction criticises.
  5. [§3.2.1, Table 1, References] §3.2.1 and Table 1: CAD-VQA is listed as “Independent [TODO]” with construction details and source still unfinished; References [6–9] are also TODO placeholders. CAD-VQA barely moves (84.10→88.23) while carrying one of the four domain axes and the “drawing understanding” skill in Figure 1. Incomplete data provenance for a reported benchmark and missing citations for the sibling systems the substrate is defined against are not presentation nits—they block reproducibility of a core claimed skill.
minor comments (6)
  1. [Title page, §4.1] Date line and arXiv stamp say July 31, 2026 / 30 Jul 2026 while citing “gpt-5.4” as the contemporary closed-source comparator; clarify model identity and evaluation date so readers can interpret the comparison.
  2. [§3.3] §3.3.3 states a “flat proportion” mix but does not give epochs, effective tokens per subset, LoRA rank/targets, learning rate, or IMAGE_MAX_TOKEN_NUM. These are needed to reproduce the v4.1 recipe on 8×A100.
  3. [§4.1, Table 2] Table 2: gpt-5.4 column is populated on the general suite in the table but the text says gpt-5.4 was not evaluated on the general suite (§4.1). Resolve the contradiction.
  4. [Figure 1, §1.3] Figure 1 and §1.3 count “four shared skills” but radiate five capability tags (COM split by application). Align the taxonomy in text and figure.
  5. [§4.4] Case studies (Figures 5–9) are described in captions but the qualitative evidence would be stronger with full prompts, full code, and CD values in an appendix or supplement.
  6. [Throughout] Minor prose issues: “Qwen3.5-27B” vs “Qwen3.5-VL-27B” naming inconsistency; “outperformsthe” spacing; author line “Y urui Dong”.

Circularity Check

0 steps flagged

Empirical multi-task SFT paper: measured gains are not forced by definition; only mild ecosystem self-citation for data provenance, not a circular derivation.

full rationale

IndustryForge-27B’s central claims are empirical: multi-task SFT of Qwen3.5-VL-27B on ~52k curated CAD/COM samples, then head-to-head measurement on four domain benches and eleven public general benches versus the untouched base and gpt-5.4. There is no algebraic or definitional chain in which a fitted quantity is renamed a prediction, no uniqueness theorem imported from the authors that forces the architecture or scores, and no ansatz smuggled in as a theorem. Training data is co-sourced with IterCAD / AssemCAD / ComAct and those works are cited as sibling arXiv preprints (several still TODO), which creates ecosystem dependence and a legitimate contamination/generalization concern—but that is eval-homology risk, not circularity of derivation. The paper states UUID-, AST-, and Chamfer-level decontamination against the reported benches and reports absolute metrics (e.g. assembly 15.38%, COM n=60) rather than deriving them from the training mix. General-suite retention (+1.56 pp) is likewise measured on external public benchmarks. Under the circularity criteria, this is a normal non-finding with at most minor non-load-bearing self-citation for data lineage.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 2 invented entities

Load-bearing content is empirical ML practice plus domain engineering assumptions, not a formal derivation. The claim rests on: (1) standard SFT transfer assumptions; (2) trust that sandbox execution + Chamfer thresholds define “correct” CAD; (3) that author-curated corpora and small proprietary-API evals represent industrial CAD; (4) engineering choices (flat task mix, LoRA, frozen visual tower as mentioned in spillover discussion) treated as sufficient to avoid forgetting. No new physical entities; free parameters are training/data-threshold choices rather than fitted physical constants.

free parameters (4)
  • Chamfer Distance pass threshold (10^-3) = ≤ 1e-3
    Defines success on CadQuery and assembly metrics; choice directly sets reported pass rates and training QC filters.
  • Flat multi-task sampling ratios over six unbalanced corpora = uniform/flat mix (unstated exact probabilities)
    No learned or tuned task weights despite 1k–20k size skew; mixture choice can dominate which skills move.
  • LoRA / sequence_parallel_size / IMAGE_MAX_TOKEN_NUM and related SFT hyperparameters = sequence_parallel_size=8; other values not fully reported
    Capacity and long-context behavior depend on these knobs; paper states SP=8 and stack names but not full fitted schedule.
  • COM and assembly corpus acceptance filters (sandbox success + structural checks) = execution-success and structural/geometry gates (thresholds not fully numeric)
    Which of the raw ComAct/AssemCAD traces enter the ~52k set is a discrete selection policy that shapes the fitted model.
axioms (6)
  • domain assumption Supervised fine-tuning on explicit multimodal QA/code pairs is sufficient to install industrial CAD skills without continued pretraining of the base distribution.
    Stated in §3.3.2 as the reason for pure SFT from v1 onward.
  • domain assumption Sandbox execution plus Chamfer/structural match is an adequate proxy for manufacturable, assemblable engineering correctness.
    Used as both training QC and primary domain metrics (§3.2, §4.1); ignores tolerance stacks, materials, and many shop constraints.
  • domain assumption UUID-level and AST-normalized dedup (plus Chamfer NN on COM artefacts) eliminates meaningful train/test leakage relative to sibling benchmarks.
    §3.2.6; central to interpreting generalization vs in-family memorization.
  • domain assumption Windows COM automation of SolidWorks/Inventor/AutoCAD is the right native action interface for industrial software agents (COM-as-Action).
    Inherited from ComAct framing in §2.3 and §3.2.4; scopes the API skill the substrate claims.
  • standard math Standard transformer multimodal SFT optimization (next-token on packed instruction data) with LoRA and ZeRO-3 converges to the reported skills without hidden curriculum tricks.
    Background ML practice assumed throughout §3.3; no new learning theorem.
  • ad hoc to paper GPT-5.4 is a fair strong closed-source comparator for public CAD code/vision tasks at time of writing.
    Used as the headline closed baseline in §4; model identity, decoding, tools, and CAD tool access are not specified in detail.
invented entities (2)
  • IndustryForge-27B (unified industrial-CAD multimodal substrate) no independent evidence
    purpose: Single 27B model pre-installing CAD vision, CadQuery, COM, and assembly skills as a shared base for multiple agent projects.
    The paper’s primary artifact; defined by the multi-task SFT corpus and recipe rather than a new physical or mathematical object.
  • CAD-VQA-4k corpus no independent evidence
    purpose: Inject CAD-drawing reading priors via course-style VQA over multi-view engineering graphics.
    Independently constructed per authors, but pipeline/source still TODO; existence is internal to this project until released.

pith-pipeline@v1.2.0-daily-grok45 · 15687 in / 4638 out tokens · 103331 ms · 2026-07-31T19:24:08.641295+00:00 · methodology

0 comments
read the original abstract

Automating industrial CAD design and manufacturing places distinctive demands on multimodal foundation models: the model must see engineering drawings and 3D geometry screenshots, write correct parametric-modelling scripts and Windows COM API code, and cover the full range from single parts to assemblies. General-purpose multimodal models fall short on these tasks, while single-task fine-tuning is too narrow to support the diverse calls that upper-layer agents issue. We build IndustryForge-27B on top of Qwen3.5-VL-27B by curating and integrating six industrial-CAD sub-corpora totalling $\sim$52k multimodal samples---CAD Visual QA (CAD-VQA), parametric CAD code (text2cadquery), assembly-level CAD code (text2cadquery-assembly), and three COM sub-corpora for Inventor / SolidWorks (com_2d / com_3d / com_assembly)---and training with a unified multi-task SFT recipe. Across four CAD-domain benchmarks IndustryForge-27B lifts the base model by $+33.65$~pp on average and outperforms the strong closed-source model GPT-5.4 on all four; across eleven general-capability benchmarks it retains, and slightly improves upon, the base model ($+1.56$~pp mean, no catastrophic forgetting). IndustryForge-27B will serve as the common substrate for downstream industrial-agent projects, providing a unified starting point for a full-stack industrial agent that spans from CAD design to industrial-software operation, from parts to assemblies, and from single-shot generation to closed-loop self-improvement.

Figures

Figures reproduced from arXiv: 2607.28050 by Botian Shi, Hairong Zhang, Hongbin Zhou, Jiaxin Ai, Licheng Wen, Nianchen Deng, Pinlong Cai, Shu Zou, Siqi Li, Tao Hu, Xinyu Cai, Xuemeng Yang, Yurui Dong.

Figure 1
Figure 1. Figure 1: IndustryForge-27B at a glance. Left — Capability Radiation. Five industrial-CAD skills (Drawing Understanding, CadQuery Modeling, SolidWorks COM, Inventor COM, Assembly) radiate from the IndustryForge￾27B badge and share a common “full-stack CAD loop” dashed orbit that visualises the end-to-end design → modeling → assembly → software-operation pipeline the substrate supports. Right — Six-axis radar. Three … view at source ↗
Figure 2
Figure 2. Figure 2: Framework overview. A vertical three-tier stack. Bottom: six domain sub-corpora (CAD-VQA 4k, text2cadquery 17k, text2cadquery-assembly 1k, com_2d 20k, com_3d 5k, com_assembly 5k), colour-coded by their upstream project (IterCAD / AssemCAD / ComAct / independent). Middle: the IndustryForge-27B foundation model, itself built on Qwen3.5-VL-27B and trained on ∼52k samples. Top: four downstream agents that will… view at source ↗
Figure 3
Figure 3. Figure 3: Data distribution. Outer ring: six sub-corpora sliced by sample count; consecutive slices belong to the same inner-ring family, giving visual alignment between the two levels. Inner ring: four task families (CAD-VQA, CadQuery Parts, COM Parts, Assembly). Two takeaways: (1) the COM family (com_2d + com_3d) is the largest single-part contributor by sample count; (2) the Assembly family is small in count (6 k… view at source ↗
Figure 4
Figure 4. Figure 4: Domain benchmark results. Grouped bars per benchmark: Qwen3.5-27B (grey), gpt-5.4 (orange), IndustryForge-27B (blue-purple). Value labels above each bar. Mean = 63.71%, +33.65 pp over the base model, and IndustryForge-27B wins on 4/4 benchmarks against gpt-5.4. Comparison models. Qwen3.5-27B (the untouched base model, serving as the “no-change baseline”); gpt-5.4 (a strong closed-source frontier model repr… view at source ↗
Figure 5
Figure 5. Figure 5: Case 1 — CAD VQA (part identification / dimension reading). Prose-style QA layout: an input image and question on the left; three stacked answer rows on the right showing each model’s chain-of-thought excerpt and final answer with PASS / FAIL badges. IndustryForge-27B correctly reads the multi-view geometry and matches the target answer; the base model and gpt-5.4 mis-count features. all?” once at the foun… view at source ↗
Figure 6
Figure 6. Figure 6: Case 2 — Text-to-CadQuery (single part). Left card: task description, ground-truth STL preview, and CD ≤ 10−3 target. Right: three stacked rows with per-model code excerpts (the discriminating lines highlighted) and the corresponding STL renders. IndustryForge-27B produces a CadQuery script that passes at 10−3 ; the base model errors out on the sketch API and gpt-5.4 mis-dimensions the target feature [PIT… view at source ↗
Figure 7
Figure 7. Figure 7: Case 3 — COM (SolidWorks). A single-cylinder task executed through the SolidWorks COM API. Each row shows the model’s key COM calls (sketch → extrude → move body) and the resulting STL. IndustryForge-27B uses the correct helper structure and passes; gpt-5.4 mis-selects the extrude direction; the base model fails at COM plane construction. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Case 4 — CadQuery Assembly (multi-part with cq_warehouse). A foot-mounted vertical bearing housing with clamp cap and shaft. The discriminating detail is a single line of coordinate reasoning that centres the shaft on the bearing axis rather than on the global origin or on the bearing’s local upward extrude. Only IndustryForge-27B’s placement (offset = bearing_center_z − shaft_length/2) passes at 10−3 [P… view at source ↗
Figure 9
Figure 9. Figure 9: Case 5 — Cross-benefit on math_vision L3. A painted-solid surface-area puzzle. The base model runs into a “rathole” (23 rounds of “Wait, let me re-examine. . . ” over 40k characters, the correct value 11.5 appearing at 92% of the response but never committed to \boxed{}); gpt-5.4 under-counts contact interfaces and lands on 13.5; IndustryForge-27B enumerates all six contact groups, computes hidden = 26, ex… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 2 linked inside Pith

  1. [1]

    CadQuery: a python parametric CAD scripting framework based on OpenCascade, 2024

    CadQuery Team. CadQuery: a python parametric CAD scripting framework based on OpenCascade, 2024. https://github.com/CadQuery/cadquery

  2. [2]

    OpenSCAD: The programmers solid 3d cad modeller, 2024.https://openscad.org

    OpenSCAD Team. OpenSCAD: The programmers solid 3d cad modeller, 2024.https://openscad.org

  3. [3]

    DeepCAD: A deep generative network for computer-aided design models.ICCV, 2021

    Rundi Wu, Chang Xiao, and Changxi Zheng. DeepCAD: A deep generative network for computer-aided design models.ICCV, 2021

  4. [4]

    Text2CAD: Generating sequential CAD designs from beginner-to-expert level text prompts.NeurIPS, 2024

    Mohammad Sadil Khan and Others. Text2CAD: Generating sequential CAD designs from beginner-to-expert level text prompts.NeurIPS, 2024

  5. [5]

    Xiang Xu, Karl D. D. Peng, Karl D. D. Willis, and Others. SkexGen: Autoregressive generation of CAD construction sequences with disentangled codebooks.ICML, 2022

  6. [6]

    ComAct: COM-as-Action for industrial CAD software agents.arXiv preprint, 2026

    ComAct Team. ComAct: COM-as-Action for industrial CAD software agents.arXiv preprint, 2026. [TODO: replace with final citation]

  7. [7]

    IterCAD: Closed-loop CAD code generation with compiler and visual feedback.arXiv preprint, 2026

    IterCAD Team. IterCAD: Closed-loop CAD code generation with compiler and visual feedback.arXiv preprint, 2026. [TODO: replace with final citation]

  8. [8]

    AssemCAD: Production-ready assembly generation via what-how-verify decomposition

    AssemCAD Team. AssemCAD: Production-ready assembly generation via what-how-verify decomposition. arXiv preprint, 2026. [TODO: replace with final citation]

  9. [9]

    CAD-LLM: Leveraging large language models for parametric CAD generation.arXiv preprint,

    Anonymous. CAD-LLM: Leveraging large language models for parametric CAD generation.arXiv preprint,

  10. [10]

    Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

  11. [11]

    Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and Others. Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025

  12. [12]

    InternVL: Scaling up vision foundation models and aligning for generic visual-linguistic tasks.CVPR, 2024

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and Others. InternVL: Scaling up vision foundation models and aligning for generic visual-linguistic tasks.CVPR, 2024. 14

  13. [2024]

    [TODO: replace with final citation]