Choose What to Manipulate: Revealing Data Scaling Laws in Bounding-Box Guided Policies for Semantic Manipulation

Diyun Xiang; Jinming Ma; Junbo Tan; Mingliang Zhou; Shoujie Li; Xueqian Wang; Yanzhao Yu; Yihao Wu

Choose What to Manipulate: Revealing Data Scaling Laws in Bounding-Box Guided Policies for Semantic Manipulation

T0 review · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Semantic manipulation policies generalize better when trained on more bounding-box-annotated object classes, and this improvement follows a power law—so an object-diversity-first data strategy is the most efficient path to robust robot skil

desk verdict Strong system and data pipeline, but the headline scaling law conflates object diversity with total data; the practical recipe may still hold, but the evidence needs a constant-data control. read the letter →

arxiv 2602.11885 v2 pith:5SQGFKYC submitted 2026-02-12 cs.RO

Yihao Wu , Jinming Ma , Junbo Tan , Yanzhao Yu , Shoujie Li , Mingliang Zhou , Diyun Xiang , Xueqian Wang This is my paper · ORCID

classification cs.RO

keywords semanticmanipulationdiffusionpolicybounding-boxguidancedatascalinglawsimitationlearningobjectdiversityrobotgeneralizationcollectionstrategy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that generalization in semantic manipulation—measured as a normalized success score on unseen objects in unseen environments—scales approximately as a power law with the number of object classes labeled by bounding boxes. The authors build a handheld device (Label-UMI) with an automated pipeline to produce such labels efficiently, and a decoupled policy (BBox-DP) that detects the target and conditions a diffusion policy on a bounding-box-augmented image. On four real-world tasks with 6,400 demonstrations, they find saturating returns and distill a data strategy: around 16 object classes with roughly 50 demonstrations per class yields about 85% success. This is presented as a practical scaling law for semantic manipulation.

What carries the argument

The core mechanism is a semantic-motion decoupled framework. An object detection module (YOLO) converts a semantic instruction (text or voice) into a visual bounding box overlaid on the raw image. The diffusion policy then learns to follow this bounding box as a 'visual instruction,' decoupling semantic understanding from motion generation. This design transfers generalization burden to the detection module and allows the policy to focus on following the box. The scaling-law study systematically varies the number of annotated object classes while measuring generalization, fitted as a power law on the log-transformed optimality gap.

What would settle it

Train policies with a fixed total number of demonstrations while varying only the number of bounding-box object classes (e.g., 800 demos across 1, 2, 4, 8, and 16 classes). If generalization no longer improves with class count, the power law is attributable to data volume, not object diversity.

Watch

Extended reading notes

Core claim

The central claim is that generalization in semantic manipulation improves as a power law of the number of bounding-box-annotated object classes used in training. The paper demonstrates this by varying the number of object classes (1, 2, 4, 8, 16) and measuring policy performance on unseen objects and environments. The fits show a consistent power-law relationship between the optimality gap (1 minus normalized score) and the number of object classes, with correlation coefficients reported in Figure 8. From this, the authors derive an object-diversity-first data collection strategy: prioritize collecting around 16 object classes with about 50 demonstrations per class, which reaches approximat

Load-bearing premise

The scaling-law experiment increases the number of object classes while also increasing the total number of demonstrations, so the observed power law may be driven by data volume rather than object diversity per se.

Editorial extensions

If this is right

Dataset designers should prioritize object diversity over sheer demonstration count when building semantic manipulation datasets; roughly 16 object classes with 50 demonstrations each is a practical target.
The power-law relationship makes performance improvement predictable, enabling budget-driven data collection: diminishing returns from adding object classes can be anticipated.
Bounding-box visual instructions outperform text conditioning and point/keypoint guidance in cluttered scenes, suggesting that explicit spatial grounding is a key ingredient for generalization.
The Label-UMI handheld device with automated annotation reduces labeling cost, making large-scale semantic manipulation studies feasible in the wild.
The achieved ~85% success across four real-world tasks provides a concrete entry point for deploying semantic manipulation policies in cluttered environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The fitted power law may partly reflect total data volume, because in the scaling experiment the number of demonstrations grows with the number of object classes; a constant-total-data comparison would isolate object diversity as the causal factor.
Because the framework decouples detection from policy, improving the detection module (e.g., using zero-shot detectors) could further lift generalization without retraining the policy—a testable extension.
The paper's observation that generalization correlates with shape diversity of the object set hints at a second axis of the scaling law; quantifying shape diversity alongside class count could refine data collection guidance.
The power-law trend may extend to other visual grounding formats (e.g., keypoints, masks) or other policy classes, but the paper only establishes it for bounding boxes with diffusion policies.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Let me give you the short version: the headline result—a power-law scaling law in the number of bounding-box object classes—is not actually supported by the data as presented, because the analysis conflates object diversity with total demonstration count. The system work is real and the experiments are extensive, but the central quantitative claim needs a control before it can be taken as a scaling law.

The Label-UMI device and the automated annotation pipeline look like genuinely useful contributions for collecting semantically labeled demonstrations. The BBox-DP architecture is sensible: decoupling detection from the policy and using bounding boxes as visual instructions is clean, and the results against text-conditioned and keypoint baselines are strong and statistically tested (Welch's t-tests, all significant). The four-task real-robot study with 6,400 demos is more thorough than most.

The problem is in Section V-C. They fit the power law using only the n=0 condition, where number of objects N' and demonstrations per object K=100 are both varied. That means total demonstrations D = N'*K, so N' and D are perfectly collinear. A least-squares fit of log(1-S) vs log N' is algebraically the same as vs log D up to a shift. So the fitted exponent tells you nothing about whether it's diversity of objects or just sheer data volume driving improvement. The paper has the right data in Fig. 7 to break this collinearity—cells like (N'=2,K=50) against (N'=1,K=100)—but no such constant-total-data comparison is reported. Also, the fit has five points and no error bars, despite variance across the five repetitions. That's load-bearing because the 'object-diversity-first' strategy rests entirely on the causal reading. The validation in Section V-D is also somewhat circular: they apply the strategy derived from the same law and don't compare against an alternative (e.g., more demos per object at fixed total data).

That said, the practical recipe—around 16 object classes and 50 demos per class—might still be sensible, and the cross-task validation in V-E is a useful sanity check. But as it stands, the scaling law is an overstatement. The paper would be much stronger with a constant-total-data comparison and confidence intervals on the exponent, ideally with code/data released.

For a reader working on data collection for manipulation policies, this paper is worth a look for the system and the empirical comparison. The scaling-law claim should not be cited until the confound is addressed. I'd send it to peer review because the system is substantial and the question is important, but I'd insist on the control. Reading group? Maybe—it's a good case study in how easy it is to over-interpret a collinear fit.

Circularity Check

1 steps flagged · score 5.0 of 10

The claimed power-law scaling with number of bounding-box objects is not identifiable: in the n=0 condition used for the fit, total demonstrations are perfectly collinear with object count, so the fit is algebraically equivalent to a total-data scaling law.

renaming known result [Section V-C (Fig. 8), with Section V-A (data construction) and Section V-D (strategy)]
"To study how the number of bounding boxes objects affects policy generalization... we define a triple (m,n)_j. Here, m=0,1,2,3,4 represents the number of object with bounding boxes 2^m sampled from the pool of 16, n=0, -1, -2, -3, -4, -5 denotes the fraction of demonstrations per object as 2^n... Using data from the experiments with 100% demonstration fraction, we fit a linear model to the log-transformed values... the policy's generalization to new objects, new environments, or both scales approximately as a power law with the number of bounding-box objects."

At n=0 the per-object demonstration fraction is 100%, and Section V-A states 100 valid demonstrations per object. Hence total demonstrations D = 100 * 2^m for the five plotted points (m=0..4), so log D = log(2^m) + log(100). A least-squares fit of log(1-S) versus log(2^m) is therefore identical, up to an additive intercept, to a fit of log(1-S) versus log D. The fitted exponent cannot distinguish 'number of bounding-box object classes' from 'total demonstration count.' Presenting this as a scaling law specifically in the number of bounding-box objects, and deriving the 'object-diversity-first' strategy from it, renames the known total-data scaling law rather than isolating diversity as the causal driver.

full rationale

The hardware (Label-UMI), the annotation pipeline, and the BBox-DP policy are independent engineering contributions and are validated against external baselines (Octo, OpenVLA, OpenVLA-OFT, Text-DP, Keypoint-DP) with real-world experiments. The central scaling-law claim, however, is built on a five-point fit using only the n=0 (100% demonstrations-per-object) condition, where the number of bounding-box objects and total demonstration count are perfectly collinear by construction: D = 100 * N'. Consequently, the log-log regression against N' is algebraically indistinguishable from a regression against total data volume. The paper does not report any constant-total-data comparison (e.g., N'=1,K=100 vs N'=2,K=50 vs N'=4,K=25), which would be needed to separate object diversity from data volume. The 'object-diversity-first' collection strategy inherits this confound, although its cross-task validation on two additional tasks is a genuine external test rather than purely retrospective. No self-citation loops, definitional circularity, or ansatz-smuggling through citations were found. The score reflects one load-bearing construction confound in the central scaling-law result, not a wholesale collapse of the paper's contributions.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claims rest on the automated annotation pipeline, the diffusion policy architecture, and the scaling-law fit. The main hidden load-bearing choices are the power-law functional form, detector capability on test objects, and the under-specified score metric; none of these are independently verified with released artifacts.

free parameters (4)

η, λ stage/time weights in S = not reported (η+λ=1)
Eq. (2) defines the normalized score with η+λ=1, but the weights are never given; all success/score numbers depend on these unspecified weights.
t_min, t_max time thresholds = not reported
Eq. (2) uses a clip on completion time between t_min and t_max; thresholds are unstated, affecting every normalized score.
power-law exponent α and prefactor β = not given per task; only correlation r shown in Fig. 8
Fitted to five aggregate points at the 100% demonstration fraction; no out-of-sample validation and no confidence intervals.
denoising steps K and policy/YOLO training hyperparameters = not reported
Needed to reproduce BBox-DP and the YOLO detector; absent from the manuscript.

assumptions (5)

domain assumption The optimality gap 1−S scales as a power law Y=βX^α in N'; no alternative functional form is considered.
Section V-C fits a line in log-log coordinates and interprets r as evidence, assuming the power-law form rather than testing it against logarithmic or saturating-exponential curves.
domain assumption A YOLO detector trained on pipeline annotations supplies correct bounding boxes for all evaluated objects, including 'unseen' test objects.
The paper never reports detector accuracy or whether test objects were held out from detector training; if boxes are wrong or test objects are detector-seen, the generalization numbers are affected.
domain assumption ORB-SLAM3-based trajectories from Label-UMI are accurate enough to train a diffusion policy.
Adopted from UMI [21]; no trajectory accuracy validation is provided in this paper.
domain assumption SAM2 with laser-point prompts produces accurate segmentation masks and bounding boxes.
The automated annotation pipeline relies on this for all labels; no annotation-accuracy numbers are reported.
domain assumption The stage-score aggregation S in Eq. (2) is a meaningful proxy for task success.
Weights η,λ are unknown and stage scoring is human-assigned, so comparability across tasks is unverified.

invented entities (1)

Label-UMI handheld annotation device
purpose: Collect demonstration videos with laser-point cues for automated bounding-box labeling
Described with a component list, but no open schematics, independent replication, or third-party validation is provided in the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Choose What to Manipulate: Revealing Data Scaling Laws in Bounding-Box Guided Policies for Semantic Manipulation." pith.science (2026). https://pith.science/paper/5SQGFKYC

@misc{pith2026260211885,
  author       = {Pith},
  title        = {Pith review of: Choose What to Manipulate: Revealing Data Scaling Laws in Bounding-Box Guided Policies for Semantic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5SQGFKYC}},
  note         = {Machine review of arXiv:2602.11885}
}

read the original abstract

Diffusion-based policies generalize poorly in semantic manipulation, a key obstacle to real-world deployment, because text-only instructions cannot reliably steer the policy toward the target object in cluttered, dynamic scenes. We instead use bounding-box instructions to specify the target directly, and study how performance scales with data. To this end, we build Label-UMI, a handheld segmentation device with an automated annotation pipeline for efficiently collecting semantically labeled demonstrations, and propose a semantic-motion-decoupled framework that couples object detection with a bounding-box-guided diffusion policy; a first-frame anchoring mechanism keeps execution robust to missed detections and noisy boxes. We find that generalization follows a bounded, saturating data-scaling law with diminishing returns, validated on four real-world tasks with 6,400 demonstrations, and distill an object-diversity-first collection strategy reaching 85\% success in cluttered scenes. All data and code will be released.

Figures

Figures reproduced from arXiv: 2602.11885 by the authors.

**Figure 1.** Overview of framework. (a) The Label-UMI device collects multimodal demonstration data including IMU, visual observations, and voice commands. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** (a) Structure of Label-UMI. Composition of the Label-UMI Device [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Overview of Data Acquisiton pipeline. (a) YOLO model acquisition process. We randomly sample a subset from the total dataset. The first video [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Overview of the BBox-DP. (a) Semantic detection part. The raw image and the image overlaid with bounding boxes (generated by YOLO) are [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Real-robot experiments on four semantic manipulation tasks: (a) [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 8.** Figure 8: Power-law relationship. is performed (j = 1). Policies are trained for each valid (m, n)j combination with more than 50 total demonstrations, resulting in 76 trained policies. Each policy is evaluated on 16 unseen objects in an unseen environment, with 5 trials per obj…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 16 linked inside Pith

[1]

Scaling laws for neural language models,

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020

arXiv 2001
[2]

Gpt-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat,et al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

arXiv 2023
[3]

Minigpt-4: En- hancing vision-language understanding with advanced large language models,

D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: En- hancing vision-language understanding with advanced large language models,”arXiv preprint arXiv:2304.10592, 2023

arXiv 2023
[5]

Diff-control: A stateful diffusion-based policy for imitation learning,

X. Liu, Y . Zhou, F. Weigend, S. Sonawani, S. Ikemoto, and H. B. Amor, “Diff-control: A stateful diffusion-based policy for imitation learning,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2024, pp. 7453–7460

2024
[6]

Octo: An open-source generalist robot policy,

O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu,et al., “Octo: An open-source generalist robot policy,”arXiv preprint arXiv:2405.12213, 2024

arXiv 2024
[7]

Rdt-1b: a diffusion foundation model for bimanual manipulation,

S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “Rdt-1b: a diffusion foundation model for bimanual manipulation,”arXiv preprint arXiv:2410.07864, 2024

arXiv 2024
[8]

Visual instruction tuning,

H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34 892–34 916, 2023

2023
[9]

Openvla: An open-source vision-language-action model,

M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong,et al., “Openvla: An open-source vision-language-action model,” inConference on Robot Learning. PMLR, 2025, pp. 2679–2713

2025

Show all 36 references

[10]

Fine-tuning vision-language- action models: Optimizing speed and success,

M. J. Kim, C. Finn, and P. Liang, “Fine-tuning vision-language- action models: Optimizing speed and success,”arXiv preprint arXiv:2502.19645, 2025

2025 arXiv
[11]

Chatvla: Unified multimodal understanding and robot control with vision-language-action model,

Z. Zhou, Y . Zhu, M. Zhu, J. Wen, N. Liu, Z. Xu, W. Meng, R. Cheng, Y . Peng, C. Shen,et al., “Chatvla: Unified multimodal understanding and robot control with vision-language-action model,”arXiv preprint arXiv:2502.14420, 2025

2025 arXiv
[12]

Rt-trajectory: Robotic task generalization via hindsight trajectory sketches,

J. Gu, S. Kirmani, P. Wohlhart, Y . Lu, M. G. Arenas, K. Rao, W. Yu, C. Fu, K. Gopalakrishnan, Z. Xu,et al., “Rt-trajectory: Robotic task generalization via hindsight trajectory sketches,”International Conference on Learning Representations 2024, 2023

2024
[13]

Hamster: Hierarchical action models for open-world robot manipulation,

Y . Li, Y . Deng, J. Zhang, J. Jang, M. Memmel, R. Yu, C. R. Garrett, F. Ramos, D. Fox, A. Li,et al., “Hamster: Hierarchical action models for open-world robot manipulation,”International Conference on Learning Representations 2025, 2025

2025
[14]

Kite: Keypoint- conditioned policies for semantic manipulation,

P. Sundaresan, S. Belkhale, D. Sadigh, and J. Bohg, “Kite: Keypoint- conditioned policies for semantic manipulation,”The Conference on Robot Learning 2023, 2023

2023
[15]

Vip: Vision instructed pre-training for robotic manipulation,

Z. Li, L. Ren, J. Yang, Y . Zhao, X. Wu, Z. Xu, X. Bai, and H. Zhao, “Vip: Vision instructed pre-training for robotic manipulation,”arXiv preprint arXiv:2410.07169, 2024

2024 arXiv
[16]

Yolov8: A novel object detection algorithm with enhanced performance and robustness,

R. Varghese and M. Sambath, “Yolov8: A novel object detection algorithm with enhanced performance and robustness,” in2024 Inter- national conference on advances in data engineering and intelligent computing systems (ADICS). IEEE, 2024, pp. 1–6

2024
[17]

Roboturk: A crowdsourcing platform for robotic skill learning through imitation,

A. Mandlekar, Y . Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay,et al., “Roboturk: A crowdsourcing platform for robotic skill learning through imitation,” inConference on Robot Learning. PMLR, 2018, pp. 879–893

2018
[18]

Open-television: Teleoperation with immersive active visual feedback,

X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,”arXiv preprint arXiv:2407.01512, 2024

2024 arXiv
[19]

Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,

T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,” in2018 IEEE international conference on robotics and automation. Ieee, 2018, pp. 5628–5635

2018
[20]

Learning fine-grained bimanual manipulation with low-cost hardware,

T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023

2023 arXiv
[21]

Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,

C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song, “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,”arXiv preprint arXiv:2402.10329, 2024

2024 arXiv
[22]

Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,

H. Ha, Y . Gao, Z. Fu, J. Tan, and S. Song, “Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,”arXiv preprint arXiv:2407.10353, 2024

2024 arXiv
[23]

Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,”The International Journal of Robotics Research, p. 02783649241273668, 2023

2023
[24]

Zero-shot robotic manipulation with pretrained image- editing diffusion models,

K. Black, M. Nakamoto, P. Atreya, H. Walke, C. Finn, A. Kumar, and S. Levine, “Zero-shot robotic manipulation with pretrained image- editing diffusion models,”arXiv preprint arXiv:2310.10639, 2023

2023 arXiv
[25]

Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review,

G. Du, K. Wang, S. Lian, and K. Zhao, “Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review,”Artificial Intelligence Review, vol. 54, no. 3, pp. 1677–1734, 2021

2021
[26]

An intelligent emulsion explosive grasping and filling system based on yolo-simam-grcnn,

J. Yi, P. Liu, J. Gao, R. Yuan, and J. Wu, “An intelligent emulsion explosive grasping and filling system based on yolo-simam-grcnn,” Scientific Reports, vol. 14, no. 1, p. 28425, 2024

2024
[27]

Earl: Eye-on-hand reinforcement learner for dynamic grasping with active pose estimation,

B. Huang, J. Yu, and S. Jain, “Earl: Eye-on-hand reinforcement learner for dynamic grasping with active pose estimation,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2023, pp. 2963–2970

2023
[28]

Manibox: Enhancing spatial grasping generalization via scalable simulation data generation,

H. Tan, X. Xu, C. Ying, X. Mao, S. Liu, X. Zhang, H. Su, and J. Zhu, “Manibox: Enhancing spatial grasping generalization via scalable simulation data generation,”arXiv preprint arXiv:2411.01850, 2024

2024
[29]

Flow as the cross-domain manipulation interface,

M. Xu, Z. Xu, Y . Xu, C. Chi, G. Wetzstein, M. Veloso, and S. Song, “Flow as the cross-domain manipulation interface,”arXiv preprint arXiv:2407.15208, 2024

2024 arXiv
[30]

Robotic control via embodied chain-of-thought reasoning,

M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine, “Robotic control via embodied chain-of-thought reasoning,”The Con- ference on Robot Learning 2024, 2024

2024
[31]

Energy-based models are zero-shot planners for compositional scene rearrangement,

N. Gkanatsios, A. Jain, Z. Xian, Y . Zhang, C. Atkeson, and K. Fragki- adaki, “Energy-based models are zero-shot planners for compositional scene rearrangement,”Robotics Science and Systems, 2023

2023
[32]

A single multi-task deep neural network with post-processing for object detection with reasoning and robotic grasp detection,

D. Park, Y . Seo, D. Shin, J. Choi, and S. Y . Chun, “A single multi-task deep neural network with post-processing for object detection with reasoning and robotic grasp detection,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 7300–7306

2020
[33]

A multi- task convolutional neural network for autonomous robotic grasping in object stacking scenes,

H. Zhang, X. Lan, L. Wan, C. Yang, and N. Zheng, “A multi- task convolutional neural network for autonomous robotic grasping in object stacking scenes,” in2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2019, pp. 6435–6442

2019
[34]

Denoising diffusion implicit models,

J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,”arXiv preprint arXiv:2010.02502, 2020

2010 arXiv
[35]

Data scaling laws in imitation learning for robotic manipulation,

F. Lin, Y . Hu, P. Sheng, C. Wen, J. You, and Y . Gao, “Data scaling laws in imitation learning for robotic manipulation,”arXiv preprint arXiv:2410.18647, 2024

2024 arXiv
[36]

Learning transferable visual models from natural language supervision,

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PmLR, 2021, pp. 8748–8763

2021
[37]

Open-world object manipulation using pre-trained vision-language models,

A. Stone, T. Xiao, Y . Lu, K. Gopalakrishnan, K.-H. Lee, Q. Vuong, P. Wohlhart, S. Kirmani, B. Zitkovich, F. Xia,et al., “Open-world object manipulation using pre-trained vision-language models,”The Conference on Robot Learning 2023, 2023

2023

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

Scaling laws for neural language models,

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020

arXiv 2001

[2] [2]

Gpt-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat,et al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

arXiv 2023

[3] [3]

Minigpt-4: En- hancing vision-language understanding with advanced large language models,

D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: En- hancing vision-language understanding with advanced large language models,”arXiv preprint arXiv:2304.10592, 2023

arXiv 2023

[4] [5]

Diff-control: A stateful diffusion-based policy for imitation learning,

X. Liu, Y . Zhou, F. Weigend, S. Sonawani, S. Ikemoto, and H. B. Amor, “Diff-control: A stateful diffusion-based policy for imitation learning,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2024, pp. 7453–7460

2024

[5] [6]

Octo: An open-source generalist robot policy,

O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu,et al., “Octo: An open-source generalist robot policy,”arXiv preprint arXiv:2405.12213, 2024

arXiv 2024

[6] [7]

Rdt-1b: a diffusion foundation model for bimanual manipulation,

S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “Rdt-1b: a diffusion foundation model for bimanual manipulation,”arXiv preprint arXiv:2410.07864, 2024

arXiv 2024

[7] [8]

Visual instruction tuning,

H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34 892–34 916, 2023

2023

[8] [9]

Openvla: An open-source vision-language-action model,

M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong,et al., “Openvla: An open-source vision-language-action model,” inConference on Robot Learning. PMLR, 2025, pp. 2679–2713

2025