REVIEW 3 major objections 5 minor 1 cited by
BrepLLM enables large language models to parse and reason over raw boundary representation (B-rep) CAD data, and reports state-of-the-art captioning and classification results with a compact 2.9B-parameter model.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 15:33 UTC pith:ZTKCHXX4
load-bearing objection New modality, sound architecture, but the SOTA numbers rest on a 200-sample split with no error bars and selection on the test set. the 3 major comments →
BrepLLM: Enabling Large Language Models to Understand Boundary Representations
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that a boundary representation graph—faces as nodes with sampled UV geometry, edges as adjacency with length-adaptive point features—can be tokenized and fed into an LLM to produce fluent, dimensionally accurate CAD captions and category labels. The authors show that after contrastive pretraining of a hierarchical face/edge/topology encoder against CLIP text embeddings, and after a three-stage fine-tuning that first bridges geometry to the 2D vision-language space, then aligns through LoRA, and finally adds a residual mixture-of-query-experts, the resulting model outperforms point-cloud-based LLMs on both tasks. The result is attributed to the rich, exact geometric a
What carries the argument
The load-bearing mechanism is the hierarchical BrepEncoder: an adaptive UV sampling strategy discretizes each face into 10-dimensional attribute tensors (coordinates, normals, curvature, visibility, face type, area) and each edge into 8-dimensional tensors, building a face-edge adjacency graph. A PointTransformerV3 extracts per-face features, an NNConv edge-conditioned layer propagates boundary information, and EGAT-Conv layers aggregate global topology; concatenating these yields a 128-dimensional node token per face plus a global graph token. The global token is aligned to frozen CLIP text embeddings via InfoNCE, and the node tokens are progressively aligned to a Q-Former and Phi-2 LLM thr
Load-bearing premise
The whole evaluation rests on the assumption that 269,444 auto-generated question-answer pairs are faithful enough to human knowledge of CAD, and that a held-out set of 200 models is large enough to resolve the few-point gaps the paper reports.
What would settle it
Re-run the reported evaluation on a fresh random 200-model split (or a larger held-out set) and measure the distribution of the score gaps; if the gap to the best point-cloud baseline is consistently below one point or changes sign across splits, the claimed advantage of native B-rep reasoning does not hold.
If this is right
- If correct, CAD question-answering and part captioning no longer require rendering point clouds or predicting command scripts; the raw B-rep can be the direct input modality.
- A compact 2.9B model matching or beating 7B-13B point-cloud models suggests that representation fidelity matters more than scale for geometry-to-language tasks, potentially lowering hardware requirements.
- The three-stage curriculum (CLIP alignment, vision-bridge MLP, then residual query experts) gives a reusable recipe for plugging other structured geometry modalities into frozen LLMs.
- Brep2Text, with 269,444 QA pairs, becomes a benchmark for future B-rep language understanding and for comparing native-geometry approaches to point-cloud proxies.
Where Pith is reading between the lines
- The same adaptive-UV graph encoding could be applied to other exact geometry formats, such as NURBS surfaces or trimmed patches, to let LLMs reason about free-form geometry rather than only CAD primitives.
- If the B-rep signal is as rich as claimed, a testable extension is zero-shot or few-shot transfer to downstream tasks like constraint satisfaction, assemblability, or manufacturability analysis, where point-cloud representations are known to be lossy.
- Because the dataset is synthesized by an LLM without reported human validation, a likely failure mode is that the model learns to imitate the synthetic captioning style rather than true geometric reasoning; this could be tested by probing with novel geometry attributes not present in the training distribution.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes BrepLLM, a framework that feeds raw boundary-representation (B-rep) CAD models directly into an LLM, bypassing command-sequence or point-cloud intermediaries. It introduces adaptive UV sampling to turn B-rep faces and edges into a graph, a hierarchical BrepEncoder that produces face, edge, and topology features, and a contrastive pre-training step that aligns a global Brep token with a frozen CLIP text encoder. The LLM (Phi-2) is then fine-tuned in three stages through an MLP projector, a Q-Former with LoRA, and a Mixture-of-Query Experts module. The paper also introduces Brep2Text, a dataset of 269,444 automatically generated B-rep–language QA pairs, and reports state-of-the-art (SOTA) results on captioning and generative classification tasks against point-cloud baselines on a 200-model test split.
Significance. If the SOTA claim holds, this is a significant contribution: it demonstrates that raw structured B-rep geometry alone can support LLM captioning and classification, and the proposed hierarchical encoder and dataset could become useful resources for CAD-focused multimodal research. The paper includes several strengths: a clear and plausible architecture, reproduction of point-cloud baselines on the same split, a large new dataset, and both automatic and human evaluation. However, the empirical evidence for the central claim is currently not strong enough. The headline margins over the best baseline are small (1.33–2.60 points) and are reported as point estimates on an n=200 split with no confidence intervals, significance tests, or repeated-seed variance. Moreover, the ablations that select the final configuration are computed on the same test split, so the final numbers are selected rather than independently validated. These issues are fixable, but they currently undermine the SOTA claim.
major comments (3)
- [§5.1, Tables 1–2] The headline SOTA margins are within sampling noise. With n=200, the classification accuracy difference 57.05% vs 54.90% has an approximate standard error of 5.0 percentage points, giving z≈0.4. No confidence intervals, p-values, or repeated-seed results are reported for any metric. The captioning scores in Table 1 also lack any variance measure. Since the reported deltas are the entire basis for the SOTA claim, the paper should report bootstrap CIs, significance tests, or results across multiple test splits/seeds.
- [§5.5, Tables 4–7] The ablations are conducted on the same 200-model test split used for the main results, and they are used to select the final configuration (stage composition, MQE placement, adaptive UV sampling). For example, Table 4 shows a final gain of +0.64 for adaptive sampling, Table 5 shows +2.42 for the hierarchical encoder, and Table 6 shows 57.05% for the full pipeline. Because the final configuration was chosen after inspecting these same test-set numbers, the reported SOTA is a selected maximum over several configurations and may overstate true performance. A separate validation split or a nested/selection-aware evaluation protocol is needed before the headline claim is credible.
- [§4 and §5.2] The training QA pairs in Brep2Text are generated by Qwen-Max (§4), and Qwen-Max is also used as the captioning judge (§5.2). Although the reference answers derive from human-written Text2CAD captions, using the same model family for training-data generation and evaluation risks rewarding stylistic mimicry of Qwen-Max-generated text rather than genuine geometric understanding. No human validation rate of the automatically generated questions/labels is reported. Please provide a validation sample with human agreement, use an additional independent LLM judge or human evaluation with inter-annotator statistics, and report the correlation between the Qwen-Max score and human precision.
minor comments (5)
- [Abstract and §3.3] The abstract and introduction describe a two-stage pipeline, while Section 3.3 and the conclusion describe a three-stage fine-tuning strategy. Please harmonize this terminology.
- [Table 3 caption] Typo: 'sxample' should be 'sample'. Please proofread the qualitative-results section.
- [Eq. (2)] Notation is inconsistent: MC is used in the text, while ℓC appears without a clear definition of C. Please define all symbols consistently.
- [References] Reference [1] has a malformed author list, and reference [23] is missing venue information. Please correct these entries.
- [Appendix] The text repeatedly refers to 'See Appendix' for training details, but no appendix is present in the submitted version. Please include it or clarify availability.
Circularity Check
No significant circularity: the reported SOTA numbers are external benchmark comparisons, not definitional reductions.
full rationale
BrepLLM's central claims are empirical comparisons against external point-cloud 3D-LLMs on a held-out 200-model split, with the model trained via standard CLIP contrastive loss (Eq. 7) and autoregressive language-modeling objectives. No equation-level identity connects a fitted parameter to a reported metric, and there is no self-citation chain or imported uniqueness theorem; components such as CLIP, PointTransformerV3, BLIP-2 Q-Former, and Phi-2 are external. The one self-referential element is that Qwen-Max generated the Brep2Text QA pairs and also served as a captioning judge; this is an evaluation-bias concern (possible style mimicry) rather than a circular derivation, and the captioning claim is also supported by frozen external metrics (Sentence-BERT, SimCSE) and human evaluation, while classification is judged against Text2CAD-derived ground-truth categories. Statistical concerns about the 200-model test split and ablations performed on the same split affect confidence in the SOTA delta but are correctness/risk issues, not circularity.
Axiom & Free-Parameter Ledger
free parameters (6)
- Contrastive temperature τ (InfoNCE scale) =
learned, value not reported
- UV sampling counts N_face, M_edge =
range [16,32] (Eqs. 1–2)
- Q-Former query count Q =
32
- Node token length cap T_max =
128 (truncate/pad)
- MQE expert count k and Top-G routing =
not specified (appendix)
- LoRA rank/alpha (Q-Former, LLM) =
not specified (appendix)
axioms (6)
- domain assumption UV-sampled face-adjacency graph faithfully represents B-rep geometry and topology for language grounding
- domain assumption Qwen-Max-generated Brep2Text QA pairs are valid ground truth for B-rep–language semantics
- domain assumption Frozen CLIP (ViT-L/14) text space is a valid semantic anchor for CAD-related language
- domain assumption The 200-model held-out split is representative and sufficient for 1–3-point metric deltas
- domain assumption Off-the-shelf components (PointTransformerV3, NNConv, EGATConv, Phi-2, Q-Former, 2D-VLM priors) behave as described in their source papers
- standard math Symmetric InfoNCE/CLIP contrastive loss yields a meaningful shared embedding under the dual-tower setup
invented entities (2)
-
Mixture-of-Query Experts (MQE) with residual query sets E_1..E_k
no independent evidence
-
Brep2Text dataset (269,444 B-rep–text QA pairs)
no independent evidence
read the original abstract
Current token-sequence-based Large Language Models (LLMs) struggle to directly process 3D Boundary Representation (B-rep) models that contain complex geometric and topological information. To this end, we propose BrepLLM, the first multimodal framework that enables LLMs to directly parse and reason over raw B-rep data. BrepLLM adopts a two-stage training pipeline: cross-modal alignment pre-training and two-stage LLM fine-tuning. In the first stage, we design an adaptive UV sampling strategy to convert B-reps into graph representations that integrate geometric and topological information. Subsequently, we construct a hierarchical BrepEncoder to extract features from geometric elements (faces and edges) and topology, generating a global token and a sequence of node tokens. Then, via contrastive learning, we conduct an initial alignment between this global token and the text embeddings of a frozen CLIP text encoder (ViT-L/14). In the second stage, we integrate the pre-trained BrepEncoder into the LLM and employ a two-stage progressive strategy to align the sequence of node tokens: (1) training an MLP-based semantic mapping network that utilizes the prior knowledge of a 2D-VLM to align the B-rep representation to the 2D visual semantic space; (2) utilizing LoRA for parameter-efficient fine-tuning of the Q-Former and the LLM backbone network to achieve the final 3D-language generation capability. Furthermore, we construct the Brep2Text dataset, which contains 269,444 B-rep and text question-answer pairs. Experiments demonstrate that BrepLLM achieves SOTA performance on 3D object classification and captioning tasks. The project page is available at https://user-deng.github.io/BrepLLM/.
Figures
Forward citations
Cited by 1 Pith paper
-
BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning
BrepCoder trains one multimodal LLM to read B-rep CAD geometry and output CAD code, handling reverse engineering, completion, error correction, and CAD-QA.
Reference graph
Works this paper leans on
-
[1]
Phi-2: The surprising power of small language models, 2023
Marah Abdin, Aneja Jyoti, Bubeck Sebastien, C ´esar Teodoro Mendes Caio, Weizhu Chen, Allie Del Giorno, Ro- nen Eldan, Sivakanth Gopi, Suriya Gunasekar, Mojan Java- heripi, Piero Kauffmann, Tat Lee Yin, Yuanzhi Li, Anh Nguyen, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Michael Santacroce, Harkirat Singh Behl, Adam Tau- mann Kalai, Xin Wang,...
2023
-
[2]
Akshay Badagabettu, Sai Sravan Yarlagadda, and Amir Barati Farimani. Query2cad: Generating cad models using natural language queries.arXiv preprint arXiv:2406.00144, 2024. 2
Pith/arXiv arXiv 2024
-
[3]
Text2shape: Generating shapes from natural language by learning joint embeddings
Kevin Chen, Christopher B Choy, Manolis Savva, An- gel X Chang, Thomas Funkhouser, and Silvio Savarese. Text2shape: Generating shapes from natural language by learning joint embeddings. InAsian conference on computer vision, pages 100–116. Springer, 2018. 3
2018
-
[4]
An investigation on utilizing large language model for in- dustrial computer-aided design automation.Procedia CIRP, 128:221–226, 2024
Haoxuan Deng, Samir Khan, and John Ahmet Erkoyuncu. An investigation on utilizing large language model for in- dustrial computer-aided design automation.Procedia CIRP, 128:221–226, 2024. 2, 3
2024
-
[5]
Benjamin T Jones, Felix H ¨ahnlein, Zihan Zhang, Maaz Ah- mad, Vladimir Kim, and Adriana Schulz. A solver-aided hi- erarchical language for llm-driven cad design.arXiv preprint arXiv:2502.09819, 2025. 3
Pith/arXiv arXiv 2025
-
[6]
Text2cad: Generating sequential cad designs from beginner- to-expert level text prompts.Advances in Neural Information Processing Systems, 37:7552–7579, 2024
Mohammad Sadil Khan, Sankalp Sinha, Talha Uddin, Di- dier Stricker, Sk Aziz Ali, and Muhammad Zeshan Afzal. Text2cad: Generating sequential cad designs from beginner- to-expert level text prompts.Advances in Neural Information Processing Systems, 37:7552–7579, 2024. 2
2024
-
[7]
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models.arXiv preprint arXiv:2407.07895, 2024. 3
Pith/arXiv arXiv 2024
-
[8]
Llm4cad: Multi-modal large language models for 3d computer-aided design generation
Xingang Li, Yuewan Sun, and Zhenghui Sha. Llm4cad: Multi-modal large language models for 3d computer-aided design generation. InInternational Design Engineering Technical Conferences and Computers and Information in Engineering Conference, page V006T06A015. American Society of Mechanical Engineers, 2024. 2
2024
-
[9]
Cad- assistant: Tool-augmented vllms as generic cad task solvers
Dimitrios Mallis, Ahmet Serdar Karadeniz, Sebastian Cavada, Danila Rukhovich, Niki Foteinopoulou, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. Cad- assistant: Tool-augmented vllms as generic cad task solvers. arXiv preprint arXiv:2412.13810, 2024. 2
Pith/arXiv arXiv 2024
-
[10]
Shapellm: Universal 3d object understanding for embodied interaction,
Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction,
-
[11]
Learning transferable visual models from natural language supervi- sion
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 2, 5
2021
-
[12]
Clip-forge: Towards zero-shot text-to-shape genera- tion
Aditya Sanghi, Hang Chu, Joseph G Lambourne, Ye Wang, Chin-Yi Cheng, Marco Fumero, and Kamal Rahimi Malek- shan. Clip-forge: Towards zero-shot text-to-shape genera- tion. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 18603–18613,
-
[13]
Meshclip: Efficient cross-modal infor- mation processing for 3d mesh data in zero/few-shot learn- ing.Information Processing & Management, 60(6):103497,
Yupeng Song, Naifu Liang, Qing Guo, Jicheng Dai, Junwei Bai, and Fazhi He. Meshclip: Efficient cross-modal infor- mation processing for 3d mesh data in zero/few-shot learn- ing.Information Processing & Management, 60(6):103497,
-
[14]
Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors, 2024
Yuan Tang, Xu Han, Xianzhi Li, Qiao Yu, Yixue Hao, Long Hu, and Min Chen. Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors, 2024. 6
2024
-
[15]
Cad-gpt: Synthesising cad construction sequence with spatial reasoning-enhanced mul- timodal llms
Siyu Wang, Cailian Chen, Xinyi Le, Qimin Xu, Lei Xu, Yanzhou Zhang, and Jie Yang. Cad-gpt: Synthesising cad construction sequence with spatial reasoning-enhanced mul- timodal llms. InProceedings of the AAAI Conference on Artificial Intelligence, pages 7880–7888, 2025. 2
2025
-
[16]
Cad-llm: Large language model for cad generation
Sifan Wu, Amir Khasahmadi, Mor Katz, Pradeep Kumar Ja- yaraman, Yewen Pu, Karl Willis, and Bang Liu. Cad-llm: Large language model for cad generation. InProceedings of the neural information processing systems conference. neurIPS, 2023. 2
2023
-
[17]
Cad- vlm: Bridging language and vision in the generation of para- metric cad sketches
Sifan Wu, Amir Hosein Khasahmadi, Mor Katz, Pradeep Ku- mar Jayaraman, Yewen Pu, Karl Willis, and Bang Liu. Cad- vlm: Bridging language and vision in the generation of para- metric cad sketches. InEuropean Conference on Computer Vision, pages 368–384. Springer, 2024. 2
2024
-
[18]
Point transformer v3: Simpler, faster, stronger
Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Heng- shuang Zhao. Point transformer v3: Simpler, faster, stronger. 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 4840–4851, 2023. 4
2024
-
[19]
Jingwei Xu, Zibo Zhao, Chenyu Wang, Wen Liu, Yi Ma, and Shenghua Gao. Cad-mllm: Unifying multimodality- conditioned cad generation with mllm.arXiv preprint arXiv:2411.04954, 2024. 2
Pith/arXiv arXiv 2024
-
[20]
Pointllm: Empowering large language models to understand point clouds, 2024
Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiang- miao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds, 2024. 6
2024
-
[21]
Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding
Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1179–1189, 2023. 3
2023
-
[22]
Cadtalk: An algorithm and benchmark for semantic commenting of cad programs
Haocheng Yuan, Jing Xu, Hao Pan, Adrien Bousseau, Niloy J Mitra, and Changjian Li. Cadtalk: An algorithm and benchmark for semantic commenting of cad programs. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3753–3762, 2024. 2
2024
-
[23]
Cad-editor: Text-based cad editing through adapting large language mod- els with synthetic data
Yu Yuan, Shizhao Sun, Qi Liu, and Jiang Bian. Cad-editor: Text-based cad editing through adapting large language mod- els with synthetic data. 2
-
[24]
Pointclip: Point cloud understanding by clip
Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xu- peng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8552–8562, 2022. 3
2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.