Pith.

Artificial Intelligence · arXiv:2610.12452 v1 · submitted 8 Oct 2026 · 24 references

BrickBench: Evaluating Agentic Brick Design

Peter Kulits, Yiqing Xu, R. Kenny Jones, Cordelia Schmid, Jiajun Wu

The paper says

“We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built.”

more of the abstract

To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our benchmark and environment at http://www.brickben.ch

from the abstract

Pith says

Machine review · deepseek-v4-flash · 9 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A genuinely useful agentic-design benchmark with a properly validated design-quality metric; trust the design findings, read "physically built" as the rigid-body proxy it is. This line is wrong

2 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 24 references say.

The paper says, at Sec. 3.2; Abstract; Fig. 1

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The paper is transparent that the validity bar is a proxy: Sec. 3.2 states that stud holding forces and axle play are not modeled and that an assembly that balances as a rigid body passes even when a built one might sag or fall apart, and Sec. 6 calls the bar necessary but not sufficient ([b82], [b267]–[b271]).

more of objection 1

What remains unresolved is a distinct, unmeasured failure mode. Sec. 3.1 defines hinge connectors that allow rotation and flips, ball connectors with free rotation, and axle connectors that allow sliding; Sec. 3.2 then simulates each connected component as a single rigid body, freezing every one of those degrees of freedom. An assembly with an articulated subassembly (a crane arm on a hinge, a wheel on an axle, a ball-jointed tail) passes Valid whenever its locked pose is statically balanced, regardless of whether the physical build would fold, rotate, or slide under gravity, and the paper reports no physical build, joint-release simulation, or connection-strength check of any generated assembly. Because the BrickAgent validator is exactly the function the agents receive as feedback and optimize against, the bias direction is known (Valid overstates buildability) while its magnitude is never measured. This gap underlies the categorical statements in the Abstract (can also be physically built) and in Fig. 1 (Each assembly above can be built). I recommend one of two fixes: (a) a bounded empirical check, such as physically building a sample of valid assemblies per setting or re-running the stability test with joints released, with the pass rate reported; or (b) recalibrating the Abstract, Fig. 1, and Sec. 5.1 so the claims describe the Sec. 3.2 rigid-body check as a necessary proxy condition rather than physical buildability.

The paper says, at Sec. 4.4; Fig. A3

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The judge model behind the pairwise ELO metric is never identified. The VQA paragraph names Gemma 4 31B, but the ELO paragraph refers only to 'the judge,' and the prompts reproduced in Fig. A3 are anonymous. Design ELO is one of the paper's three headline metrics, and its use depends on the human-alignment study of Sec. 5.3 and on the ratings assigned to Opus 5.5 and GPT-6.1 Sol that were produced after that study.

more of objection 2

Without the judge's identity, version, and decoding settings, the metric cannot be reproduced, and the Fig. 5b validation cannot be checked for whether the validated judge is the same model that scored the two later agents. A sentence naming the model and inference settings for both the VQA and ELO calls is required.

7 minor objections

The paper says, at Table 1; Sec. 5.1

Pith says: minor objection 1 of 7

Valid and VQA are reported as point estimates without uncertainty. With 100 prompts per setting, a Valid value of 1.00 versus 0.95 is within sampling noise (a binomial 95% interval for 95/100 is roughly [0.89, 0.98]), so statements such as 'Astra leads on every metric' should either carry intervals or be tempered.

The paper says, at Sec. 3.2

Pith says: minor objection 2 of 7

Several validator hyperparameters are not specified numerically: the collision 'overlap by more than a tolerance' value, the 'high coefficient of friction,' and the simulation duration are all left open. The 3 LDU movement threshold is given, but the others should be stated in the text or appendix, or pinned to specific values in the released code, for exact reproducibility of the gate metric.

show 5 more

The paper says, at Fig. 5b caption

Pith says: minor objection 3 of 7

The caption says 'we align the two by a constant factor' when rescaling Design ELO to the human ratings, but the factor is not reported; state its value so the rescaling is transparent.

The paper says, at Sec. 5.4

Pith says: minor objection 4 of 7

The human-design study relies on only five raters, and no inter-rater agreement is reported. Since the 323/360 result carries the 'gap to humans' conclusion, per-rater rates or an agreement statistic should be added.

The paper says, at Sec. 5.1

Pith says: minor objection 5 of 7

The sentence 'Of the 137 invalid assemblies across the reference set' has an ambiguous denominator: nine anchored agents times 300 prompts is 2700 assemblies, while the full agent set is larger. Specify which set the 137 counts come from.

The paper says, at Fig. 1

Pith says: minor objection 6 of 7

The caption lists five agents, but the rendered panels are not individually labeled in the figure as printed; add labels so the assemblies can be attributed to agents.

The paper says, at References

Pith says: minor objection 7 of 7

The LEGO-Maker reference contains a stray space ('ACM T ransactions on Graphics'), and the inline citation form 'Waßmann & Weicker (2012)' differs in style from the reference list; harmonize both.

What would settle it

Re-run the Sec. 3.2 validity pipeline on the reference-set assemblies (all 2,700, or a stratified subset) with each connected component simulated as an articulated multi-body system: free the hinge/ball/axle DOFs defined in Sec. 3.1, add per-connection force/torque limits for stud connections (e.g., Waßmann & Weicker 2012 or BrickFEM, both cited in Sec. 6), and replace the ad-hoc "high" friction with realistic plastic-on-plastic parameters. Compare the recomputed Valid fraction, under the same ≤3 LDU criterion, with Table 1. If the passing fraction drops more than a few points (Astra 1.00 to <0.95), the abstract's "can also be physically built" and the near-perfect Valid scores overstate buildability; if it holds, the proxy is adequate. Physically assembling a small Alt-Build sample (set 10698's parts exist in production) can calibrate the simulation.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 9 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 3 min 10 s · 19,912 tokens read, 15,427 written
  3. Editor wrote the verdict above 30 s · 20,802 tokens read, 4,534 written
  4. Referee major revision: 2 major objections, 7 minor 1 min 45 s · 20,231 tokens read, 15,498 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

You should know two things about BrickBench. First, it is the most usable agentic brick-design benchmark I have seen so far: it gives coding agents a grounded, tool-rich environment and then separates semantic alignment from design quality, and it validates the design metric properly — Design ELO agrees with human raters on 34 of 36 agent pairings (Kendall tau 0.89), and raters pick the human build in 323 of 360 comparisons. Second, the word "physically built" is one notch stronger than what the validator actually checks, and the gap is acknowledged but never measured.

What's new is the benchmark itself — 300 prompts across three settings with different part constraints — plus the BrickAgent environment with named-part validity feedback, and the finding that with tools, leading agents produce valid assemblies for nearly every prompt and satisfy about 95% of semantic questions, while validity collapses without the environment (Astra drops from 1.00 to 0.40, Luna to under 0.01). The design-quality bottleneck is the most interesting result, and it does not depend on the validity proxy, since ELO and VQA are scored on all assemblies regardless of validity.

Soft spots, in order of size. The validity test simulates each connected component as a single rigid body in PyBullet, freezing hinge, ball, and axle joints and ignoring stud holding force. Any articulated subassembly passes if its locked pose balances. The authors state this plainly in Sections 3.2 and 6 — "necessary but not sufficient" — but they never estimate the false-positive rate, and the abstract's "can also be physically built" is stronger than the evidence. Since agents optimize against exactly this validator, the unmeasured bias in Valid deserves an abstract-level caveat. Minor items: Valid and VQA are point estimates without error bars; the design judge is a VLM but is never named (Gemma 4 is named only for VQA); and the human-vs-agent comparison used five raters and part-count-matched rather than prompt-matched human assemblies — the authors are upfront about the latter. None of these break the central claims.

This is for people building agents for structured 3D design and for benchmark authors who care about validating a subjective quality metric. It deserves a serious referee; the revision I would want is to temper or quantify the physical-buildability claim and name the judge.

The strongest objection

The load-bearing premise is that the Sec. 3.2 Valid test operationalizes "physically built." The authors disclose the core gap: connections are not tested for strength, so an assembly that balances as a rigid body passes even when a physical one might sag or come apart ([b82], [b267]–[b271]), and the reader's verdict already weighs that. What remains unresolved is a more concrete mechanism: Sec. 3.1 defines stud, hinge, ball, axle, and fixed connectors with real degrees of freedom (rotation, flip, slide), yet Sec. 3.2 simulates each connected component as a single rigid body, freezing every joint. Any assembly with an articulated sub-assembly — a hinge-held crane arm, a ball-jointed tail, an axle-mounted wheel — passes Valid whenever its locked pose is statically balanced, regardless of whether the physical build would fold, rotate, or slide under gravity. Because the BrickAgent validator is exactly the function the agents see as feedback and optimize against, the unmodeled dimensions are systematically exploited on the valid side: the direction of bias is known (Valid overstates buildability) and its magnitude is never measured. No physical build, joint-release simulation, or connection-strength check of any generated assembly is reported. If the false-positive rate is substantial, then the abstract's "can also be physically built," the introduction's "valid assemblies for nearly every prompt," and the Valid≈1.00 entries of Table 1 all overstate the central claim that agents reliably produce buildable LEGO sets. The paper's careful wording — "verifiable physical requirements" — keeps it internally consistent, but the gap between proxy and construct is entirely unquantified.

Three testable claims structure the paper: (1) agents with BrickAgent produce assemblies that pass its validity checks, near-perfectly for the top agents; (2) the best satisfies ~95% of the DSG-decomposed VQA questions; (3) design quality separates agents and stays below human standards. Claim (3) is the best supported: Design ELO's judge agrees with 4,080 human pairwise judgments on 34/36 agent pairs (Kendall τ=0.89), and the 323/360 human-discrimination result, though subject-mismatched and acknowledged as such ([b248], [b250]), points the same way. Claim (2) is plausible as a ranking; the absolute 95% is uncalibrated, and the anonymous ELO judge (Sec. 4.4) hurts reproduction without breaking the argument. Claim (1) carries the load because it underlies "can also be physically built." The authors are explicit that the bar is "necessary but not sufficient" ([b271]), which the reader's verdict already weighs. My stress-test sharpens the mechanism: Sec. 3.2's single-rigid-body treatment freezes the connector DOFs that Sec. 3.1 defines, so jointed assemblies are scored as locked sculptures, and since the BrickAgent validator is the agents' optimization target, the proxy gap is exploited, not randomly measured. The false-positive rate is never quantified; no physical build or structural re-simulation is reported. This is a correctness-risk concern about magnitude, not an internal inconsistency: the paper's own findings are scoped to "verifiable physical requirements," and the design-metric validation is real credit. The proposed re-simulation test settles the magnitude, so the reader's CONDITIONAL verdict stands and needs no change; I agree with the reader's choice of weakest assumption.

The referee's report

If the results hold, BrickBench is a solid contribution to agentic benchmark design. The strongest elements are concrete and externally grounded: Design ELO is validated against 4,080 human pairwise judgments with agreement on 34/36 agent pairs; the human-versus-agent study supplies a falsifiable measure of the design gap; the environment ablation cleanly shows that grounded feedback, not raw model capability, drives buildability; and the VQA question graph makes semantic failures attributable. The release of the benchmark and environment materially strengthens the contribution, as does the detailed per-setting reporting. The principal risk is the physical-validity proxy: because the benchmark's stated purpose is to evaluate designs that 'can also be physically built,' the Sec. 3.2 rigid-body check is load-bearing for the validity claims, and its unmeasured false-positive rate would propagate into the headline numbers. This risk does not touch the design-quality or human-gap findings, which rest on the human studies.

Pith's reading of the paper (a machine's summary, not the authors')

BrickBench tries to establish where coding agents stand on a harder version of text-to-LEGO design: assemblies of up to 4,000 parts that must be buildable, not just renderable. The paper claims that, inside its BrickAgent environment, a programmatic workspace with part search, connector-based placement, rendering, and an interpretable validator, leading agents deliver valid assemblies for nearly every prompt, with the best satisfying about 95% of decomposed semantic questions. The same agents without the environment collapse: validity drops from 100% to 40% for the strongest agent and below 1% for a weaker one. The sharpest separation, however, comes from design quality, measured by pairwise judgments, where human raters picked the human-made assembly in 323 of 360 comparisons. The paper's conclusion is that agents have largely learned to build what works, but not yet what makes a great design.

What carries the argument. Three mechanisms carry the argument. First, the BrickNet connector system: every part in the 14,441-type LDraw library carries typed connectors (stud, hinge, axle, ball, or fixed), each with a mating subtype, a polarity, and a frame, so two parts connect when compatible connectors align, which turns an assembly into a graph of connected components. Second, the validity validator: an assembly must respect its setting's part limits, have no colliding geometry, and remain at rest under gravity when each connected component is simulated as a rigid body in PyBullet, with the paper's own caveat that connection strength and axle play are not modeled. Third, the evaluation stack: DSG-style question graphs decompose each prompt into atomic yes/no questions answered by a vision-language model over renders, and a Bradley-Terry model turns pairwise judgments into Align ELO and Design ELO ratings anchored to a fixed reference set.

Core claim. The paper claims four results. First, physical validity and prompt alignment are near-solved for frontier coding agents when they work inside the provided environment: five agents deliver a valid assembly for every prompt, no reference agent falls below 0.80 on validity, and the best satisfies 0.954 of the VQA questions, the fraction of atomic yes/no questions answered correctly from eight renders of the assembly. Second, design quality, measured by Design ELO from pairwise judgments, separates agents far more sharply, spanning 545 points between the weakest and strongest, a spread the paper reads as the judge preferring the stronger build in about 96% of pairs, and the metric agrees with human raters on 34 of 36 agent pairs (Kendall $\tau = 0.89$). Third, the environment is the load-bearing factor for buildability: without BrickAgent's tools, the strongest agent's valid share falls from 1.00 to 0.40 and a weaker agent's below 0.01, while alignment and design scores barely move. Fourth, a gap to human design remains: human raters familiar with LEGO identified the human-made assembly in 323 of 360 comparisons, and no agent was taken for a human designer more than one time in five. The paper's headline conclusion is that agents are learning to build what works but not yet what makes a great design.

The premise it rests on. The load-bearing premise is that the validity tests, collision-free geometry and remaining at rest under a rigid-body simulation that ignores stud-holding force and axle play, adequately capture what it means for an assembly to be physically built; the paper itself says in Section 6 that this bar is necessary but not sufficient, and if real builds fail through weak connections rather than tipping, every Valid score overstates buildability.

If the paper is right (Pith's inferences, not the authors')

Reliable construction in this domain is now a tooling result: coding agents with grounded build-and-validate feedback produce buildable assemblies, so other physical-design domains such as CAD or robotic assembly can adopt the same verifier-loop recipe.

Because validity and alignment saturate near the top of the field, the next measurable progress in generative 3D design will show up in design quality rather than in pass/fail checks.

The human-gap result gives a concrete target: until a rater can no longer tell agent builds from human ones at above-chance rates, agents have not matched human standards of part usage, proportion, and surface detail.

Because Design ELO agrees with human rankings on 34 of 36 agent pairs, pairwise vision-language judging is a workable automated proxy for design quality, enabling large-scale evaluation without hiring expert raters.

The environment ablation implies that buildability constraints cost agents some expressivity: one weaker agent's alignment and design scores rose when the constraints were removed, so validity checks do not come free.

If the validity bar were upgraded to model stud-holding force and axle play, the paper's own stated gap, Valid scores would likely fall, and the 'can be physically built' claim is best tested by physically building a sampled set of the agent assemblies with real bricks.

The same protocol transfers: a question-graph alignment metric plus a preference-judge design metric could score any discrete, constraint-heavy construction task, from furniture assembly to circuit layout, with the validity test swapped for that domain's physics.

The human baseline is deliberately part-count-matched rather than prompt-matched; commissioning human builds of the same 300 prompts would likely narrow the reported human-agent gap and give a fairer ceiling for the benchmark.

Design ELO's agreement with human raters suggests a direct training signal: agents could be fine-tuned or prompted against the validated design judge as a reward, which might close the design gap faster than scaling base models.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our benchmark and environment at http://www.brickben.ch

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-09

The authors' main claim, as the classifier picked it out 1.00

“We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.80 · human participants 0.11 · controlled comparison 0.31 · randomized assignment 0.02 · states quantitative result 0.07 · compares with prior results 0.63 · mentions code or data availability 0.96

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. contribution 1.00 0.92 0.03 asserted 0.98
Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. method 0.65 0.15 0.03 asserted 0.97
To do so, it must select parts from a discrete library and reason jointly about local and global constraints. method 0.91 0.13 0.02 asserted 0.99
We score validity, alignment, and design across three settings that vary in scale and part availability. method 0.91 0.95 0.04 asserted 0.98
We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. contribution 0.79 0.93 0.03 asserted 1.00
We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. result 1.00 0.96 0.90 asserted 1.00
We release our benchmark and environment at http://www.brickben.ch contribution 0.99 0.85 0.03 asserted 0.98

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 0.90

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-09: 1 request, 5,385 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. We evaluate coding agents on their ability to design LEGO assemblies from a text
Figure 2 of the paper
Figure 2. BrickNet validation. We evaluate GPT-5.6 Luna on the BrickNet text-to-assembly validation set (Kulits & Schmid, 2026). While it was not specialized for this dataset, we find it exceeds task-specific baselines including BrickGPT (Pun et al., 2025) across metrics and nears the alignment of the ground-truth objects. (a) Sample inputs and model outputs. (b) Caption align￾ment following BrickNet’s evaluation protocol. PE (Bolya et al., 2025) and SigLIP 2 (Tschannen et al., 2025) score the similarity between the embeddings of the renders and of the caption, and VQAScore (Lin et al., 2024)…
Figure 3 of the paper
Figure 3. BrickBench summary. We report summary metrics averaged across the three evalua￾tion settings. See Tab. 1 for disaggregated results. The ELO-plot lines represent 95% intervals. Luna runs at max and the others at xhigh. Each agent is given a budget of 300 turns per prompt. The baselines evaluated are BrickNet-14B (Kulits & Schmid, 2026) and BrickGPT (Pun et al., 2025), both restricted to the Model setting which is the only one applicable. Nine of the agents form the reference set of the ELO scale (Sec. 4.4). The baselines and ablated runs of Sec. 5.2 are rated against them and do not…
Figure 4 of the paper
Figure 4. BrickBench samples. We show one assembly from each of the eleven agents for three prompts in each setting. We observe the agents vary in their interpretation of the prompts. One assembly is missing for Muse because it did not deliver one within the turn budget. 9
Figure 5 of the paper
Figure 5. ELO. (a) We plot ELO against mean assembly cost. (b) To validate our model-based Design ELO computation, we perform a perceptual study where participants are tasked with judging design quality between a pair of assemblies for a given prompt. The prompt isn’t given to the judge, and they make their decision based only on the images. We find that the rankings agree on 34/36 agent pairs (Kendall 𝜏 = 0.89), suggesting the metric is an effective proxy for human judgment. The study covers the nine reference agents; Opus 5.5 and GPT-6.1 Sol were evaluated after it. BrickNet-14B is…
Figure 6 of the paper
Figure 6. Human-design evaluation. We perform a perceptual study comparing agent designs with human designs. For each assembly in the Model and Set settings, we select a random assembly from the BrickNet dataset with a comparable number of parts and ask a human rater to identify the one made by a person. We find that raters are able to distinguish between the agent-designed and human-designed assemblies (a); the study covers the nine reference agents and the two baselines. While agent-designed assemblies often satisfy the prompt criteria, there is a clear gap in fine-grained design to those…

Figures are the authors', reproduced from the paper.

References (24)
  1. [1] Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Doll \'a r, and Christoph Feichtenhofer. Perception encod... on Pith
  2. [2] Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I . the method of paired comparisons. Biometrika, 39 0 (3/4): 0 324--345, 1952 on Pith
  3. [3] Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. In International Conference on Learning Repr... on Pith
  4. [4] Hyunsoo Chung, Jungtaek Kim, Boris Knyazev, Jinhwi Lee, Graham W. Taylor, Jaesik Park, and Minsu Cho. Brick-by-Brick : Combinatorial construction with deep reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2021 on Pith
  5. [5] Erwin Coumans and Yunfei Bai. PyBullet , a Python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016--2021 on Pith
  6. [6] Jiahao Ge, Mingjun Zhou, Hanyou Zheng, Hao Xu, and Chi-Wing Fu. LEGO -maker: Autoregressive image-conditioned LEGO model creation. ACM Transactions on Graphics (TOG), 44 0 (6), 2025 on Pith
  7. [7] Gemma Team . Gemma 4 technical report. arXiv preprint arXiv:2607.02770, 2026 on PithPith's reviewarXiv
  8. [8] Yunqi Gu, Ian Huang, Jihyeon Je, Guandao Yang, and Leonidas Guibas. BlenderGym : Benchmarking foundational model systems for graphics editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025 on Pith
  9. [9] Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A. Ross, Cordelia Schmid, and Alireza Fathi. SceneCraft : An LLM agent for synthesizing 3D scenes as Blender code. In International Conference on Machine Learning (ICML), 2024 on Pith
  10. [10] Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE -bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations (ICLR), 2024 on Pith
  11. [11] Peter Kulits and Cordelia Schmid. BrickNet : Graph-backed generative brick assembly. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 39252--39261, June 2026 on Pith
  12. [12] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision (ECCV), 2024 on Pith
  13. [13] Liang Ma, Jiajun Wen, Min Lin, Rongtao Xu, Xiwen Liang, Bingqian Lin, Jun Ma, Yongxin Wang, Ziming Wei, Haokun Lin, Mingfei Han, Meng Cao, Bokui Chen, Ivan Laptev, and Xiaodan Liang. PhyBlock : A progressive benchmark for physical understanding and planning via 3D block assemb... on Pith
  14. [14] Dimitrios Mallis, Ahmet Serdar Karadeniz, Sebastian Cavada, Danila Rukhovich, Niki Foteinopoulou, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. CAD -assistant: Tool-augmented VLLMs as generic CAD task solvers. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025 on Pith
  15. [15] Martin Pletz and Matthias Drvoderic. BrickFEM : An automated finite element model for static and dynamic simulations of simple Lego sets. engrXiv preprint, 2023. doi:10.31224/2898 on Pithdoi:10.31224/2898
  16. [16] Ava Pun, Kangle Deng, Ruixuan Liu, Deva Ramanan, Changliu Liu, and Jun-Yan Zhu. Generating physically stable and buildable brick structures from text. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 14798--14809, October 2025 on Pith
  17. [17] Kexian Tang, Junyao Gao, Yanhong Zeng, Haodong Duan, Yanan Sun, Zhening Xing, Wenran Liu, Kai Chen, and Kaifeng Lyu. LEGO -puzzles: How good are MLLMs at multi-step spatial reasoning? In European Conference on Computer Vision (ECCV), 2026 on Pith
  18. [18] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier H \'e naff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. SigLIP 2: Multilingual vision-language... on PithPith's reviewarXiv
  19. [19] Aaron Walsman, Muru Zhang, Klemen Kotar, Karthik Desingh, Ali Farhadi, and Dieter Fox. Break and make: Interactive structural understanding using LEGO bricks. In European Conference on Computer Vision (ECCV), 2022 on Pith
  20. [20] Martin Wa mann and Karsten Weicker. Maximum flow networks for stability analysis of LEGO structures. In European Symposium on Algorithms (ESA), 2012 on Pith
show 4 more
  1. [21] Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gordon Wetzstein. GPT-4V(ision) is a human-aligned evaluator for text-to- 3D generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024 on Pith
  2. [22] Tian Xia, Tianrun Gao, Wenhao Deng, Long Wei, Xiaowei Qian, Chenglei Yu, and Tailin Wu. BuildArena : A physics-aligned interactive benchmark of LLMs for engineering construction. In International Conference on Machine Learning (ICML), 2026 on Pith
  3. [23] Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Jing Hua Toh, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld : Benchmarking multimodal agents for o... on Pith
  4. [24] Haozhe Zhang, Kaichen Liu, Miaomiao Chen, Lei Li, Shaojie Yang, Cheng Peng, and Hanjie Chen. BenchCAD : A comprehensive, industry-standard benchmark for programmatic CAD . arXiv preprint arXiv:2605.10865, 2026 on PithPith's reviewarXiv
Cited by 3 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment