Pith. sign in

REVIEW 3 major objections 5 minor 49 references

Pretrained robot policies share a bias hierarchy: color dominates, verbs and size get ignored, and targeted data collection fixes it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:59 UTC pith:YDJKTJXQ

load-bearing objection Useful diagnostic and a practical data-collection payoff, but the factor hierarchy's fine details depend on an under-validated VLM judge. the 3 major comments →

arxiv 2607.21582 v1 pith:YDJKTJXQ submitted 2026-07-23 cs.RO cs.CV

Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation

Yu Qi , Zhang Ye , Xinyi Xu , Yuxuan Lu , Amitoj Sandhu , Boce Hu , Haojie Huang , Jonathan Tremblay
show 1 more author
Lawson L.S. Wong
This is my paper
classification cs.RO cs.CV
keywords instruction factor biascompositional generalizationrobotic manipulationfactor dominance ratefactor dominance hierarchydata collection strategyvision-language-action modelsshortcut learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper pinpoints why robot policies fail compositional generalization: they over-rely on salient instruction factors like color while under-grounding verbs and size. The authors introduce Factor Dominance Rate (FDR) for pairwise factor bias and Factor Dominance Hierarchy (FDH) for a global ranking, then show the hierarchy is consistent across six foundation policies. They turn the diagnosis into an actionable data collection strategy that reallocates a fixed demonstration budget toward under-grounded factors. In simulation and on a real robot, this strategy outperforms uniform baselines and reaches higher success with half the demonstrations. If confirmed, this gives robot learning a measurable diagnostic for shortcut behavior and a simple allocation rule for data budgets.

Core claim

The central claim is that fine-tuned robot policies suffer from instruction factor bias—a systematic tendency to latch onto dominant semantic factors instead of grounding language—and that this bias can be quantified and exploited for better data collection. Across six policies, the pairwise FDR scores aggregate into a consistent FDH: color is the most dominant factor, followed by object, spatial, verb, and size. The authors further claim that a bias-aware strategy, which concentrates demonstrations on the factors ranked lowest by FDH, improves compositional generalization beyond random or L-shaped coverage, achieving comparable or better success with half the demonstrations on a real robot.

What carries the argument

The explanatory devices are two metrics built from factor-correlation experiments: Factor Dominance Rate (FDR), the normalized difference between rollout counts where the policy overfits to one factor versus another, and Factor Dominance Hierarchy (FDH), a global ranking obtained by Copeland scoring over pairwise FDR wins with a tie threshold. The actionable mechanism is a data collection strategy (V) that fixes under-grounded factors and sweeps the other factors, adding diagonal and subdiagonal coverage, thereby concentrating the budget on the weakest factors.

Load-bearing premise

The entire diagnosis rests on a single vision-language model's labels for which factor drove each rollout; if that judge's biases vary across models, the claimed hierarchy and the choice of under-grounded factors would shift.

What would settle it

Re-run the factor-correlation experiments on the same six policies with a second judge model or with per-model human annotation; if the pairwise FDR signs or the FDH ordering change materially, the cross-architecture consistency is an artifact of the judge rather than a property of the policies.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the hierarchy color≥object≥spatial≥verb≥size holds across architectures, then demonstration budgets for tabletop manipulation can be allocated by rank without re-estimating bias per model.
  • A bias-aware collection strategy can match or beat exhaustive coverage under small budgets, suggesting that dense coverage of a structured subset beats thin coverage of the full instruction space.
  • The framework turns failure attribution from binary success rates into a per-factor diagnosis, enabling targeted data repair rather than blind scaling.
  • The consistent under-grounding of verbs and size points to a training-data gap that may be addressed by deliberate augmentation of verb-object and size-related demonstrations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same diagnostic could be applied beyond tabletop manipulation—to navigation instructions or human-robot collaboration—to see whether color-like perceptual salience universally outranks action semantics.
  • Editorial inference: the proposed strategy is one instance of a broader coverage principle; a principled active-learning loop could dynamically re-estimate FDR during data collection and outperform the fixed static hierarchy.
  • Editorial inference: the size factor's resistance to grounding suggests it may require a different representational change (e.g., explicit metric reasoning) rather than mere data allocation.
  • Editorial inference: the claim that bias-aware allocation is more sample-efficient could be stress-tested in even lower budget regimes (e.g., 25 or 50 demonstrations) where density effects should be amplified.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a diagnostic framework for compositional generalization in language-conditioned robotic manipulation. It decomposes instructions into five factors (color, verb, object, size, spatial attribute), introduces Factor Dominance Rate (FDR) for pairwise bias and Factor Dominance Hierarchy (FDH) for global ranking, and evaluates six foundation policies. The authors report a consistent hierarchy color ≥ object ≥ spatial ≥ verb ≥ size, and use it to design a bias-aware data collection strategy (V) that reallocates a fixed demonstration budget toward under-grounded factors. They validate V against L and Random baselines in ManiSkill simulations and on a UR5 real-robot setup with GELLO demonstrations, reporting gains in sample efficiency and compositional generalization.

Significance. If the diagnostic is valid, it addresses a real gap: most prior evaluations report aggregate success rates rather than identifying which instruction factors a policy ignores. The multi-backbone comparison and the translation of a measured bias into a concrete data-collection rule are valuable, and the real-robot experiments add credibility. The paper also avoids circularity: the metrics come from held-out conflict experiments and V is tested against baselines under matched budgets. However, the central hierarchy is built on VLM labels validated on a single model and on FDR values without uncertainty intervals, so the strength of the claimed findings currently exceeds the evidence.

major comments (3)
  1. [Sec. 4.1/B.2; Eq. (1); Table 5] The entire FDR/FDH diagnostic and the subsequent choice of under-grounded factors for the V strategy (Sec. 5) depend on labels produced by Gemini-2.5-Flash, yet the human-agreement check is conducted on only 80 rollouts of a single backbone (π0.5, 88.7% overall; Table 5). No per-model or per-factor-pair agreement is reported. If the judge's labeling errors are systematically correlated with visual salience (e.g., ambiguous motions being read as 'followed color' because color is the most discriminable attribute), the top rank of color in Table 2 and the ordering used to allocate data in Sec. 5 could be partly a measurement artifact. Please report per-model and per-factor-pair agreement, or otherwise show that the FDR values are robust to plausible judge error rates across all six backbones.
  2. [Table 1, Eq. (3), and B.3 (Table 7)] No uncertainty is reported for any FDR entry in Table 1. Several values lie near the τ=5% Copeland tie threshold used in Eq. (3) — e.g., π0.5 verb-color −4.0, object-spatial −3.8, and π0 spatial-size −13.0 — so sampling variability over the 400 rollouts can flip pairwise wins and change the FDH. Moreover, the threshold-sensitivity analysis in Table 7 shows that at τ=10% the hierarchy changes materially: π0.5 becomes color>{object≈verb}>{spatial≈size} and GR00T becomes {color≈object≈spatial}>verb>size. This contradicts the claim in B.3 that color≥object≥spatial≥verb≥size 'remains consistent across different thresholds.' Please provide bootstrap CIs or raw counts and discuss threshold sensitivity explicitly.
  3. [Sec. 6.2, Table 3] The real-robot comparison is based on 48 rollouts per condition with no confidence intervals or significance tests. Differences such as Bun(n=100) V=54.2% vs Random=37.5% correspond to 8 rollouts and are not shown to be statistically reliable. Since the abstract's practical claim ('stronger performance with half the demonstrations') rests on these results, the authors should report binomial CIs or an exact test, or temper the claim.
minor comments (5)
  1. [Sec. 4.1] 'Finaly we calculte' should be 'Finally we calculate'; similar typos occur throughout the manuscript and should be corrected.
  2. [Sec. 5 and Fig. 5] 'We hypothesis' should be 'We hypothesize'. The Figure 5 caption 'Methods' is uninformative; consider 'Sampling strategies across a two-factor grid'.
  3. [B.2, Table 5] 'for each models require 4000 videos' is grammatically unclear. Also, 'held-out sample of rollouts of pi0.5' should specify that it is a single-model sample and not a held-out evaluation set across models.
  4. [C.5] The Cup task description repeats the Pizza sentence ('put different sauce onto the Pizza'), apparently a copy-paste error. Please replace with the correct Cup description.
  5. [B.3] 'Copleland scores' is a typo. Additionally, Eq. (1) and Eq. (2) duplicate the FDR definition and could be consolidated to avoid confusion.

Circularity Check

0 steps flagged

No circular derivation: FDR/FDH are held-out measurements and the V strategy is an independently tested intervention; only a minor non-load-bearing self-citation appears.

full rationale

The paper's derivation chain is not circular. FDR is defined directly from counts of judge-labeled rollouts in held-out factor-conflict experiments (Eq. 1), and FDH is an aggregation of those measured pairwise scores via Copeland ranking (Eq. 3). No fitted parameter is renamed as a prediction, and no equation maps FDH into the reported success rates; the V strategy is an intervention whose gains are evaluated against matched-budget baselines and can fail, as acknowledged in the Verb-Size setting. The only self-citation (Ref. [31], RoboLab, which includes author Tremblay) appears in a related-work list of simulation benchmarks and is not load-bearing. The main limitation is the VLM judge's validation on only 80 rollouts of a single model, but that is an external-validity/correctness concern, not circularity: the judge is an external annotator, not an output of the paper's equations. The central claims are therefore self-contained and empirically testable against baselines.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central contribution is a measurement framework (FDR/FDH) plus an allocation rule. The framework rests on hand-set thresholds (τ) and hand-selected carrier values; the rule rests on the transferability of the measured ordering. No new physical entities are postulated.

free parameters (3)
  • Copeland tie threshold τ = 5%
    Section 4.2 defines a pairwise win as FDR > τ, with τ = 5%, hand-chosen. The authors report sensitivity at τ = 10% (Appendix B.3), and some orderings change (e.g., π0.5 at τ = 10% collapses several distinctions).
  • Carrier factor values in FDR experiments = e.g., carrier verb ∈ {Grasp, Lift}, carrier object ∈ {cube, sphere}, carrier color ∈ {yellow, red}
    Appendix B fixes 'carrier' factors to 2 values per pairwise experiment. These choices are arbitrary and can change the measured FDR magnitudes, e.g., verbs grasp/lift are perceptually simpler than push/rotate.
  • Base configuration f* in V and L strategies = not specified explicitly (per task base values)
    Algorithms 1 and 2 start from a base factor configuration f*; the specific f* determines which column/diagonal is collected. The paper does not state how f* is chosen, which affects V's coverage.
axioms (4)
  • domain assumption The five instruction factors (verb, color, object, size, spatial) are independent and reusable semantic components; every Cartesian product is a valid instruction.
    Section 3 defines the instruction template and factor vocabulary; the entire benchmark relies on this decomposability.
  • domain assumption Factor dominance measured in diagonal-correlation conflict experiments (with one factor deliberately paired with another during training) is a valid proxy for how much a policy 'grounds' each factor.
    Section 4.1's FDR construction: a policy is judged to overfit to the factor it acts on in an OOD/impossible scene. This assumes behavior under conflict reveals the factor used in normal instruction-following.
  • domain assumption Gemini-2.5-Flash can accurately classify agent-view videos into 'success', 'overfit to f1', 'overfit to f2'.
    All FDR numbers come from this VLM judge (Section B.2); human agreement is measured on a single 80-rollout subset of π0.5 at 88.7% overall.
  • domain assumption The FDH ordering measured on the six models transfers to other models and real-robot tasks.
    Section 6 uses the global ordering (color≥object≥spatial≥verb≥size) to select under-grounded factors without re-estimating bias per setting; the real-robot evaluation uses only GR00T-N1.7.

pith-pipeline@v1.3.0-alltime-deepseek · 16353 in / 14139 out tokens · 123703 ms · 2026-08-01T06:59:57.894273+00:00 · methodology

0 comments
read the original abstract

Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferring to salient cues rather than grounding language. We introduce a diagnostic framework that localizes this failure to individual \textit{instruction factors}, \textit{e.g.,} reusable semantic components such as color, verb, object, size, and spatial attribute. Our framework formalizes instruction factor bias, the tendency of fine-tuned policies to over-rely on dominant factors as shortcuts, and quantifies it through two metrics: Factor Dominance Rate (FDR), capturing pairwise bias between factors, and Factor Dominance Hierarchy (FDH), aggregating these into a global ranking. Evaluation on six foundation policies reveals broadly consistent ordering, \textit{i.e.}, color $\geq$ object $\geq$ spatial $\geq$ verb $\geq$ size, with color dominant, and verb and size most under-grounded. We further show the diagnosis is actionable: a bias-aware data collection strategy that reallocates a fixed budget toward under-grounded factors outperforms baselines in simulation and on a real robot using half the demonstrations, thereby enabling more sample-efficient and generalizable policy learning.

Figures

Figures reproduced from arXiv: 2607.21582 by Amitoj Sandhu, Boce Hu, Haojie Huang, Jonathan Tremblay, Lawson L.S. Wong, Xinyi Xu, Yu Qi, Yuxuan Lu, Zhang Ye.

Figure 1
Figure 1. Figure 1: (a) We decompose language instructions into factors. (b) We identify the problem of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Simulation setup in Maniskill. Data Collection Strategy in Robotic Manipulation. Collecting robot demonstrations at scale remains a major bottleneck in imitation learn￾ing [32, 33]. A large body of work reduces this cost by improving data acquisition pipelines, including scalable teleoperation, motion cap￾ture, and sim-and-real co-training, active learning [34, 35, 36, 37, 38]. Complementary to these colle… view at source ↗
Figure 3
Figure 3. Figure 3: Factor correlation experiment setups. (a) The (verb, object) setting. During inference, only two objects are in the scene with the same color: the target object(“car”) and the verb-paired object during training(“sphere”). We compute FDR by checking whether it overfits to the training￾paired verb or to the training-paired object. (b) The (color, object) setting follows similar training and inference setup; … view at source ↗
Figure 4
Figure 4. Figure 4: Simulation Experiment Results. Each panel plots the success rate under factor change f or demonstration budget n. In plot 11, 12, f = 50, in all other plots, n = 200. Each checkpoint is evaluated under three random seeds, each seed with 200 rollouts. Dots are per-seed runs, lines represent their mean values. More results and details are provided in the supplementary materials. added per under-grounded fact… view at source ↗
Figure 5
Figure 5. Figure 5: Methods. To test whether the consistency revealed in Finding 3 translates into an effective data collection strategy, we conducted experiments across mul￾tiple models in both simulation in Sec. 6.1 and real-robot experiments in Sec. 6.2. Additionally, following the exact setup as Gao et al. [40]. We keep the sum of total factor change f and demonstration budget n the same across methods for fair comparison… view at source ↗
Figure 7
Figure 7. Figure 7: Real-robot Experiment. We conduct experiments on three manipulation tasks: Bun, Pizza and Cup. Each task contains 16 instruction variants with randomized spatial configurations. object location randomization are defined in Sec. 3. Experiment parameters and results are reported in [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 6
Figure 6. Figure 6: UR5 Setup. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 9
Figure 9. Figure 9: Scaling behav￾iors on Bun. by +16.7 points on average. The advantage holds at both demonstration budgets: on Bun, V at n = 100 (54.2%) already surpasses both L (37.5%) and Random (47.9%) at n = 200, showing that bias-aware allocation is substantially more sample-efficient: it attains higher success with half the demonstrations. As the budget grows from n = 100 to n = 200, all methods improve while V retain… view at source ↗
Figure 8
Figure 8. Figure 8: Bun results visualization. Partial coverage may yield better performance than full coverage under a small demo budget. In [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Additional Results. Bun. In this task, the robot is asked to pick up a bun with specific color and place it into a specific location of the steamer. Random distractors are placed near steam￾ers and objects. The position of distractors are random. Instruction example: “put (green, white, yellow, purple) bun into the (first, second, third, fourth) steamer basket on the left. ” Pizza. In this setting, robot … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 24 linked inside Pith

  1. [1]

    Geirhos, J.-H

    R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wich- mann. Shortcut learning in deep neural networks.Nature Machine Intelligence, 2(11):665–673, 2020

    2020

  2. [2]

    Lake and M

    B. Lake and M. Baroni. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. InInternational conference on machine learning, pages 2873–2882. PMLR, 2018

    2018

  3. [3]

    Agrawal, D

    A. Agrawal, D. Batra, D. Parikh, and A. Kembhavi. Don’t just assume; look and answer: Overcoming priors for visual question answering. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4971–4980, 2018

    2018

  4. [4]

    Y . Fang, Y . Feng, D. Jing, J. Liu, Y . Yang, Z. Wei, D. Szafir, and M. Ding. When vision overrides language: Evaluating and mitigating counterfactual failures in vlas.arXiv preprint arXiv:2602.17659, 2026

    Pith/arXiv arXiv 2026

  5. [5]

    Y . Xing, X. Luo, J. Xie, L. Gao, H. Shen, and J. Song. Shortcut learning in generalist robot policies: The role of dataset diversity and fragmentation.arXiv preprint arXiv:2508.06426, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    F. Lin, Y . Hu, P. Sheng, C. Wen, J. You, and Y . Gao. Data scaling laws in imitation learning for robotic manipulation. InInternational Conference on Learning Representations, volume 2025, pages 54877–54910, 2025

    2025

  7. [7]

    S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning.Advances in Neural Information Processing Systems, 36:44776–44791, 2023

    2023

  9. [9]

    S. Choi, Y . Lee, Y . Park, C. D. Kim, R. Krishna, D. Fox, and Y . Yu. vla-eval: A unified evaluation harness for vision-language-action models.arXiv preprint arXiv:2603.13966, 2026

    Pith/arXiv arXiv 2026

  10. [10]

    E. Liu, S. Liang, L. Lu, X. Zeng, X. Cao, A. Liu, and S. Pang. Roboview-bias: Benchmarking visual bias in embodied agents for robotic manipulation.arXiv preprint arXiv:2509.22356, 2025

    arXiv 2025

  11. [11]

    Hupkes, V

    D. Hupkes, V . Dankers, M. Mul, and E. Bruni. Compositionality decomposed: How do neural networks generalise?Journal of Artificial Intelligence Research, 67:757–795, 2020

    2020

  12. [12]

    Keysers, N

    D. Keysers, N. Sch ¨arli, N. Scales, H. Buisman, D. Furrer, S. Kashubin, N. Momchev, D. Sinopalnikov, L. Stafiniak, T. Tihon, et al. Measuring compositional generalization: A comprehensive method on realistic data.arXiv preprint arXiv:1912.09713, 2019

    Pith/arXiv arXiv 1912

  13. [13]

    B. Lin, D. Bouneffouf, and I. Rish. A survey on compositional generalization in applications. arXiv preprint arXiv:2302.01067, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    C. Li, C. Jing, Z. Li, M. Zhai, Y . Wu, and Y . Jia. In-context compositional generalization for large vision-language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17954–17966, 2024

    2024

  15. [15]

    Devin, A

    C. Devin, A. Gupta, T. Darrell, P. Abbeel, and S. Levine. Learning modular neural network policies for multi-task and multi-robot transfer. In2017 IEEE international conference on robotics and automation (ICRA), pages 2169–2176. IEEE, 2017. 17

    2017

  16. [16]

    D. Xu, S. Nair, Y . Zhu, J. Gao, A. Garg, L. Fei-Fei, and S. Savarese. Neural task programming: Learning to generalize across hierarchical tasks. In2018 IEEE international conference on robotics and automation (ICRA), pages 3795–3802. IEEE, 2018

    2018

  17. [17]

    Y .-L. Kuo, B. Katz, and A. Barbu. Deep compositional robotic planners that follow natural lan- guage commands. In2020 IEEE international conference on robotics and automation (ICRA), pages 4906–4912. IEEE, 2020

    2020

  18. [18]

    Qiu and H

    W. Qiu and H. Zhu. Programmatic reinforcement learning without oracles. InInternational Conference on Learning Representations, 2022

    2022

  19. [19]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  20. [20]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  21. [21]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. backslashpi0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. pi0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    2024

  25. [25]

    Cadene, C

    R. Cadene, C. Dancette, M. Cord, D. Parikh, et al. Rubi: Reducing unimodal biases for visual question answering.Advances in neural information processing systems, 32, 2019

    2019

  26. [26]

    M. Shi, L. Chen, J. Chen, Y . Lu, C. Liu, G. Ren, P. Luo, D. Huang, M. Yao, and H. Li. Is diversity all you need for scalable robotic manipulation?IEEE Transactions on Robotics, 2026

    2026

  27. [27]

    S. Wu, X. Luo, J. Zhang, J. Xie, J. Song, H. T. Shen, and L. Gao. Policy contrastive decoding for robotic foundation models.arXiv preprint arXiv:2505.13255, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    T. Chen, Z. Chen, B. Chen, Z. Cai, Y . Liu, Z. Li, Q. Liang, X. Lin, Y . Ge, Z. Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Nasiriany, A

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots.arXiv preprint arXiv:2406.02523, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manipulation tasks.IEEE Robotics and Automation Letters, 7(3):7327–7334, 2022. 18

    2022

  31. [31]

    X. Yang, R. Dagli, A. Zook, H. Hadfield, A. Goyal, S. Birchfield, F. Ramos, and J. Tremblay. Robolab: A high-fidelity simulation benchmark for analysis of task generalist policies.arXiv preprint arXiv:2604.09860, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Mandlekar, Y

    A. Mandlekar, Y . Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, et al. Roboturk: A crowdsourcing platform for robotic skill learning through imita- tion. InConference on Robot Learning, pages 879–893. PMLR, 2018

    2018

  33. [33]

    Kelly, C

    M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer. Hg-dagger: Interactive imitation learning with human experts. In2019 International Conference on Robotics and Automation (ICRA), pages 8077–8083. IEEE, 2019

    2019

  34. [34]

    C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation.arXiv preprint arXiv:2403.07788, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Lepert, J

    M. Lepert, J. Fang, and J. Bohg. Phantom: Training robots without robots using only human videos.arXiv preprint arXiv:2503.00779, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Maddukuri, Z

    A. Maddukuri, Z. Jiang, L. Y . Chen, S. Nasiriany, Y . Xie, Y . Fang, W. Huang, Z. Wang, Z. Xu, N. Chernyadev, et al. Sim-and-real co-training: A simple recipe for vision-based robotic ma- nipulation.arXiv preprint arXiv:2503.24361, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Saxena, M

    V . Saxena, M. Bronars, N. R. Arachchige, K. Wang, W. C. Shin, S. Nasiriany, A. Mandlekar, and D. Xu. What matters in learning from large-scale datasets for robot manipulation. Inter- national Conference on Learning Representations 2025, 2025

    2025

  38. [38]

    Salganicoff, L

    M. Salganicoff, L. H. Ungar, and R. Bajcsy. Active learning for vision-based robot grasping. Machine Learning, 23(2):251–278, 1996

    1996

  39. [39]

    Belkhale, Y

    S. Belkhale, Y . Cui, and D. Sadigh. Data quality in imitation learning.Advances in neural information processing systems, 36:80375–80395, 2023

    2023

  40. [40]

    J. Gao, A. Xie, T. Xiao, C. Finn, and D. Sadigh. Efficient data collection for robotic manipu- lation via compositional generalization.arXiv preprint arXiv:2403.05110, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    T. Mu, Z. Ling, F. Xiang, D. Yang, X. Li, S. Tao, Z. Huang, Z. Jia, and H. Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations.arXiv preprint arXiv:2107.14483, 2021

    Pith/arXiv arXiv 2021

  42. [42]

    Comanici, E

    G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced rea- soning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Bjorck, F

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    Zheng, J

    J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y . Feng, Y . Zheng, J. Zou, Y . Chen, J. Zeng, et al. X- vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274, 2025

    Pith/arXiv arXiv 2025

  46. [46]

    Y . Liao, P. Zhou, S. Huang, D. Yang, S. Chen, Y . Jiang, Y . Hu, J. Cai, S. Liu, J. Luo, et al. Genie envisioner: A unified world foundation platform for robotic manipulation.arXiv preprint arXiv:2508.05635, 2025. 19

    Pith/arXiv arXiv 2025

  47. [47]

    Hamming distance.https://en.wikipedia.org/wiki/Hamming_distance,

    wikipedia. Hamming distance.https://en.wikipedia.org/wiki/Hamming_distance,

  48. [48]

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12156–12163. IEEE, 2024. 20

    2024

  49. [2025]

    Accessed: 2026-05-25

    2026