RoboBenchMart: Benchmarking Robots in Retail Environment

Alexander Krapukhin; Andrei Spiridonov; Andrey Kuznetsov; Denis Shepelev; Gregorii Bukhtuev; Konstantin Soshin; Vlad Shakhuro

REVIEW 3 major objections 5 minor 1 cited by

A new benchmark shows generalist robot models still fail at grocery-store tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 22:25 UTC pith:KWMLC6TO

load-bearing objection Useful open retail benchmark, but the zero-composite-success negative claim needs a learnability control before it supports the 'not truly general' conclusion. the 3 major comments →

arxiv 2511.10276 v2 pith:KWMLC6TO submitted 2025-11-13 cs.RO cs.AI

RoboBenchMart: Benchmarking Robots in Retail Environment

Konstantin Soshin , Alexander Krapukhin , Andrei Spiridonov , Gregorii Bukhtuev , Andrey Kuznetsov , Vlad Shakhuro , Denis Shepelev This is my paper

classification cs.RO cs.AI

keywords robot manipulation benchmarkretail dark storevision-language-action modelsgeneralizationprocedural scene generationtrajectory generationmobile manipulationlong-horizon tasks

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces RoboBenchMart, an open simulation suite for retail 'dark store' environments, and uses it to probe whether today's general-purpose robot policies transfer from tabletop and household settings to cluttered grocery warehouses. The authors generate thousands of demonstrations with motion planning and reinforcement learning, fine-tune three state-of-the-art vision-language-action models, and measure success on atomic and multi-step retail tasks. Their central finding is negative: even after fine-tuning, the models rarely complete unseen-scene or unseen-item tasks, and all three score zero on composite multi-step tasks. The paper concludes that current generalist models are not truly general across domains and argues that retail-specific pretraining may be required. If correct, the result bounds what can be expected from generalist policies without domain-targeted data, and provides a reusable testbed for measuring progress.

Core claim

RoboBenchMart is the first open benchmark dedicated to retail manipulation, built around a procedural store generator and an automatic trajectory sampler for a wheeled mobile manipulator with a seven-degree-of-freedom arm. Five atomic tasks—picking to a basket, picking from the floor, transferring between boards, and opening/closing a fridge door—are combined into two composite tasks: collecting three specified items, and opening a fridge, picking an item, then closing it. After fine-tuning Octo, π0, and π0.5 on roughly 2,500–3,000 generated demonstrations, the authors report that success rates deteriorate from in-domain settings to unseen store layouts to unseen scenes with unseen items; on

What carries the argument

The suite's load-bearing component is the Store Trajectory Sampler, which converts procedural store layouts into demonstrations: arm motion is planned by screw motion and RRT-based search with collision checks, base motion by task-specific heuristics, and a separate PPO controller supplements plan failures; together they produce roughly 2,976 trajectories. The evaluation protocol then fine-tunes each generalist model only on atomic tasks and tests composite tasks by an oracle that decomposes them into atomic subtasks executed sequentially. That oracle-decomposed evaluation is the mechanism that isolates each model's per-skill competence from its ability to chain skills, and it is the step th

Load-bearing premise

The paper's conclusion that generalist models struggle in retail rests on the fine-tuning setup being a fair stand-in for real domain adaptation—only 248 demonstrations per task-object triplet, training on atomic tasks only, composite evaluation via oracle decomposition—and the paper itself notes a correction to its reported total trajectory count from 2,480 to 2,976.

What would settle it

Train a current generalist vision-language-action model on an order of magnitude more retail demonstrations, or on end-to-end composite demonstrations, and re-run the same tasks. If the model then completes Pick 3 Items and Pick from Fridge at non-negligible rates, the paper's claim that generalist models are insufficient for retail would fail; conversely, if performance stays near zero despite the larger data, the claim is strengthened.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If the benchmark's negative results hold, current generalist vision-language-action models cannot be deployed for dark-store restocking or order picking without substantial domain-specific fine-tuning or pretraining.
Composite-task performance of zero across all three models implies that even near-competent atomic skills, such as 85% success at closing a fridge door, do not compose into multi-step retail workflows when trained only on atomic subtasks.
The sharp drop between in-domain, unseen-scene, and unseen-item conditions localizes the generalization gap: layout and texture changes hurt, and novel object-task combinations are the hardest failure point.
The open release of store layouts, trajectory-generation code, assets, and fine-tuned baselines gives the community a common protocol for measuring whether any future model improves on these numbers.
Scenarios the authors excluded in evaluation—unseen shelf arrangements and completely unseen items—are currently beyond reach of tested policies, so the benchmark defines a staged difficulty ladder for future work.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the negative result may partly reflect the fine-tuning budget, since 248 demonstrations per task-item-fixture triplet is small relative to retail layout diversity; scaling demonstrations by an order of magnitude and re-measuring is a direct testable extension.
Editorial inference: because composite tasks are evaluated via oracle decomposition into atomic subtasks, the zero composite scores do not distinguish 'cannot execute a skill sequence' from 'the atomic policy fails early in the chain'; an end-to-end evaluation on composite demonstrations would separate those possibilities.
Editorial inference: a concrete falsifier would be a model trained on full composite demonstrations rather than atomic-only ones; if it then succeeds on the two composite tasks at high rates, the paper's claim about long-horizon insufficiency would need to be narrowed to a claim about atomic-only fine-tuning.
Editorial inference: since the benchmark uses simplified shelves with large gaps and rigid-body packages only, real-world retail clutter may be even harder than the simulation; the results are best read as a lower bound on the retail gap rather than a measurement of deployable failure rates.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful open retail benchmark, but the zero-composite-success negative claim needs a learnability control before it supports the 'not truly general' conclusion.

read the letter

The paper gives the community a genuine new artifact: a dark-store retail benchmark with procedural store layouts, 370 released 3D grocery assets, a trajectory-generation pipeline, and evaluations of Octo, pi0, and pi0.5 after fine-tuning. That is more than FetchBot or IPA-3D1K offer, and the code and data are public. Credit is due for the mesh-simplification work and for publishing the corrected trajectory count in Appendix A (2,976 not 2,480); that kind of transparency helps.

The soft spot is the one the stress-test note flags: there is no learnability baseline. The central claim is that current generalist VLAs 'are not yet truly general across domains,' supported by zero composite success and near-zero unseen-item performance. But with only 248 demos per (task, item, fixture), 2-3 training objects per task, and a trajectory sampler that itself succeeds only 60% of the time, the experiment cannot distinguish 'generalists fail at retail generalization' from 'fine-tuning with this little data and these tasks fails for any policy.' The paper does report 'Train Seeds' numbers in Appendix C, which show even in-distribution success is sometimes low (Octo pick-to-basket 30%, pi0.5 open-fridge 4% on one seed), but those are still the generalists; there is no scratch-trained baseline or scripted policy to show the tasks are learnable at all with this data budget. That gap is load-bearing for the negative generalization claim.

Also worth noting: composite tasks are evaluated via oracle decomposition, so zero composite success should be read as 'chained atomic failures' rather than a separate long-horizon deficit. The single embodiment and simplified packing reduce realism but are stated limitations, not hidden ones. The 60% sampler rate without error bars is a minor data-quality concern.

The benchmark itself is worth adopting regardless of the negative result. A reader working on retail robotics, benchmark design, or VLA generalization will get value from the assets and evaluation harness. The empirical claim needs a control experiment before it becomes a strong negative result. I would send it to peer review, but I would push the authors to add a learnability baseline and soften the abstract accordingly.

Referee Report

3 major / 5 minor

Summary. The paper introduces RoboBenchMart, a simulated benchmark for mobile manipulation in retail dark-store environments. It contributes a procedural store plan generator, a trajectory sampling pipeline combining motion planning and reinforcement learning, a dataset of 370 grocery assets, and a benchmark protocol in which Octo, π0, and π0.5 are fine-tuned on generated atomic-task demonstrations and evaluated on atomic and composite retail tasks under In-Domain, Unseen Scenes, and Unseen Scenes & Items scenarios. The headline empirical finding is that, even after fine-tuning on 248 demonstrations per (task, item, fixture) triplet with 2–3 training objects per task, all three generalist VLA models achieve low atomic success rates and zero composite-task success, leading the authors to conclude that current generalist models are not yet truly general across domains.

Significance. If the empirical finding is robust, RoboBenchMart fills a real gap: it is, to my knowledge, the first open benchmark dedicated to retail mobile manipulation, and the released infrastructure — procedural layout generation, trajectory generation, 3D assets, and fine-tuned baseline checkpoints — is a useful community resource. The negative result across three diverse generalist models would be a valuable calibration point for the field. However, the central claim is currently under-supported. The fine-tuning setup lacks a learnability control: there is no demonstration that a task-specific policy can learn these tasks from the same 248-demonstration budget, so the low success rates may reflect data scarcity or task difficulty rather than a lack of cross-domain generalization in generalist VLAs. The composite-task evaluation via an oracle decomposition also conflates long-horizon capability with atomic-task reliability. These issues are load-bearing because the paper's main conclusion is an empirical negative finding. With the addition of appropriate controls and corrected reporting, the benchmark could be a solid contribution.

major comments (3)

[§5.3, Table 2, Appendix A.3] The central empirical claim — that generalist VLA models 'struggle to complete typical retail tasks' — is not supported without a learnability baseline. In Table 2, even in the In-Domain scenario, Octo's Pick-to-basket success is 17% and Pick-from-floor is 2%; π0.5 reaches only 44% on Pick-from-floor and 50% on Open-fridge. The paper provides no control showing that a task-specific policy trained from scratch on the same 248 trajectories per (task, item, fixture) can achieve high success on the training distribution. Without such a baseline, the low numbers are equally consistent with (i) generalists lacking cross-domain generalization, (ii) the fine-tuning protocol giving insufficient signal, and (iii) the tasks being too difficult for the data budget — especially since the trajectory sampler itself succeeds only 60% of the time (§4.1, §4.2). I recommend adding a scratch-trained behavio
[§5.3, §A.2, Table 2] The composite-task evaluation is confounded by the oracle decomposition described in §A.2: a composite task is scored as successful only if every atomic subtask succeeds in sequence. Given the atomic success rates in Table 2 (often below 50%), the product of probabilities makes zero composite success heavily driven by atomic failures. The paper's conclusion that models show 'inadequate support for long-horizon, compositional execution' (§5.4) is therefore not supportable from the reported composite numbers alone. Please report per-stage success rates, and consider an evaluation variant that resets or re-positions the robot between subtasks to isolate long-horizon execution from atomic reliability.
[Appendix A.3] The authors themselves state that the main-text trajectory count is wrong: §5.3 reports 'totaling 2,480 demonstrations,' while Appendix A.3 reports the correct total as 2,976 and says 'We will update the paper to reflect this upon acceptance.' This is a data-consistency error in the central experimental description and must be corrected in the present version, not deferred. The discrepancy also affects any downstream statements about data budget and training cost.

minor comments (5)

[§5.1] The exclusion of scenarios 4 (Unseen Shelf Arrangement) and 6 (Completely Unseen Items) is justified by saying 'current policies consistently fail even under the simpler conditions considered above.' This is a reasonable practical choice, but it is framed using the negative result that the benchmark is meant to establish. Please rephrase to avoid a circular justification, e.g., by reporting that these scenarios were piloted and the models achieved no non-trivial success.
[Table 2] The 'n/a' entries for Open-fridge and Close-fridge under Unseen Scenes & Items are unexplained. Clarify whether these tasks were not evaluated in that scenario and why.
[§5.2 and Table 3] Naming is inconsistent: the paper uses 'Open fridge' and 'Close fridge' as atomic tasks, but Appendix A.1 refers to 'Open showcase' and 'Close showcase,' and the composite task 'Pick from fridge' is decomposed into 'Open fridge,' 'Pick to basket,' 'Close fridge' while §5.2 lists it as 'open the fridge, pick an item, and close the fridge.' Please harmonize terminology.
[§3.3 and Appendix E] Section 3.3 says assets are 'licensed for unrestricted research use,' while Appendix E states all assets are CC BY-NC 4.0. Non-commercial is a restriction; please align the wording with the actual license.
[Appendix A.3] The paper should report confidence intervals or standard errors for the success rates in Table 2, given that only 50 trials per (task, item, fixture) were used. This would help readers judge whether differences like π0's 29% vs 22% on Pick-from-floor are meaningful.

Circularity Check

0 steps flagged

No significant circularity; the paper's central claim is an empirical benchmark evaluation, not a construction.

full rationale

We walked the derivation chain: procedural store layout generation, trajectory sampling via motion planning/RL, fine-tuning of Octo/pi0/pi0.5 on the generated trajectories, and evaluation on held-out scenarios. No step defines a derived quantity in terms of the paper's conclusion. Success rates in Table 2 are measurements, not fitted parameters. Composite tasks are evaluated via oracle decomposition into atomic subtasks (Appendix A.2), but the zero composite success is not enforced by that protocol; it is a measured outcome that could in principle be nonzero if atomic policies succeeded reliably in sequence. The 248/2,976 trajectory budget and the 60% sampler success rate are data-generation design choices, not parameters fitted to the tested models. The paper contains no load-bearing self-citation: references to Octo, pi0, ManiSkill3, mplib, scene synthesizer, and procedural street modeling are external and do not smuggle in the retail-generalization conclusion. Appendix A's correction of the trajectory count (2,480 vs 2,976) is transparency, not circularity. The strongest validity criticism, noted by the reader and skeptic context, is the absence of an in-distribution learnability control (e.g., a task-specific policy trained on the same trajectories); this is a real external-validity threat because poor results could reflect data scarcity or task difficulty rather than lack of cross-domain generalization. However, that is an experimental-design gap, not a reduction of the conclusion to its own inputs, and it does not satisfy the requirement to exhibit a specific equation or fitted parameter that makes the result true by construction. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 3 invented entities

The benchmark is infrastructure plus measurement. It does not introduce a new scientific law or physical model; instead it postulates a set of design choices (trajectory budget, reward functions, anchor poses, simulator/embodiment choice, oracle decomposition) that jointly determine the negative result. Those design choices are the paper's effective free parameters, and they are described but not quantitatively justified.

free parameters (3)

Trajectory budget of 248 demos per (task, item, fixture) = 248 per triplet
The dataset size per task-object pair is a hand-chosen budget (§5.3, A.3) that strongly influences whether fine-tuned models can succeed; it is not derived from any analysis of required data volume. The negative conclusion depends on this choice.
Task-specific reward functions for RL trajectory generation = not quantified
RL rewards 'typically include terms for proximity to the target product or pose, successful object placement, and avoidance of collisions' (§4.2). These hand-crafted rewards affect which trajectories are generated and therefore what the models are trained on, but they are not specified in detail.
Anchor poses for motion planning = not quantified
The sequence of initial/intermediate/final anchor poses is 'heuristically defined' per task (§4.1). The 60% trajectory-generation success rate is averaged over these choices, which are not fully specified.

axioms (5)

domain assumption Knowledge of successful trajectory generation in the simulator implies usable imitation-learning demonstrations.
The trajectory sampler's 60% success rate is treated as sufficient to fine-tune VLAs; no analysis of action distribution quality or suboptimality is provided (§4.1-4.2, §5.3).
domain assumption The ManiSkill3 simulator with Fetch robot is a valid proxy for real dark-store retail work.
The paper explicitly notes it excludes suction grippers, deformable items, human workers, and uses simplified shelf packing (§5.4, §6). These are acknowledged scope limits, but the negative conclusion is drawn only within that proxy.
domain assumption Oracle decomposition into atomic subtasks is a valid way to evaluate composite tasks.
Composite tasks are evaluated by 'an oracle that decomposes each composite task into a sequence of atomic subtasks, executed sequentially upon the successful completion of the preceding subtask' (§A.2). This assumption shapes the result that all composite success rates are zero.
standard math Standard math for tensor fields, motion planning, and PPO/IL fine-tuning.
Standard algorithms (RRT-Connect, screw motion, PPO, fine-tuning of Octo/π0/π0.5) are invoked as background without proof.
domain assumption SketchFab assets licensed for unrestricted research use are sufficient for a retail benchmark.
Product assets are manually scaled to catalog dimensions and simplified; no validation that they cover realistic item diversity and graspability, which the paper itself notes is limited by Fetch's parallel jaw and required gaps (§3.3, §6).

invented entities (3)

Procedural Store Plan Generator no independent evidence
purpose: Generates varied dark-store floor layouts from a tensor field.
This is a new software artifact from this paper; evidence for its realism is visual (Figure 2) plus the design rationale that it mirrors procedural street modeling.
Store Trajectories Sampler no independent evidence
purpose: Auto-generates demonstrations for retail tasks using motion planning and RL.
A new data-generation pipeline; the paper reports only an average 60% success rate with no per-task breakdown or validation that trajectories are diverse/graspable.
370-item 3D grocery asset collection with 21 categories independent evidence
purpose: Provides visual and geometric object variety for the benchmark.
Assets are released on Hugging Face, so the data itself is an artifact a third party can inspect; however, no external measurement confirms the assets match real retail item statistics.

pith-pipeline@v1.3.0-alltime-deepseek · 15490 in / 9791 out tokens · 69455 ms · 2026-08-03T22:25:37.543034+00:00 · methodology

0 comments

read the original abstract

Most existing robotic manipulation benchmarks focus on tabletop or household scenarios. While these setups have driven impressive progress, it remains unclear whether generalist VLAs that excel there can truly generalize to domains with different geometry, semantics, and workflows. We introduce RoboBenchMart, an open-source simulated benchmark targeting retail dark-store environments, where a mobile manipulator must perform complex manipulation tasks with diverse grocery items. This setting presents significant challenges, including dense object clutter and varied spatial configurations, with items positioned at different heights, depths, and in close proximity. By targeting on the retail domain, our benchmark addresses a setting with strong potential for near-term automation impact. Using generated trajectories, we model a standard, realistic fine-tuning setup for current generalist VLAs and evaluate several state-of-the-art models. We find that they still struggle even on common retail tasks, indicating that these models are not yet truly general across domains. To support further research, we release the RoboBenchMart suite, which includes a procedural store layout generator, a trajectory generation pipeline, evaluation tools, and fine-tuned baseline models.

Figures

Figures reproduced from arXiv: 2511.10276 by Alexander Krapukhin, Andrei Spiridonov, Andrey Kuznetsov, Denis Shepelev, Gregorii Bukhtuev, Konstantin Soshin, Vlad Shakhuro.

**Figure 2.** Figure 2: Examples of generated store with fixtures arranged by our pipeline. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: Example of product arrangement and shelf depletion over time produced by our simulator. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Examples of collected product assets. (3) Tensor Field Aggregation: The final tensor field over the store layout is computed as a weighted sum of the basis tensors: T(p) = P j e −d ||p−pj || Tj (p), where the weights decay exponentially with the distance from p to the basis point pj , and the parameter d > 0 controls the rate of decay. In the final shelving unit arrangement stage, shelves are placed accor… view at source ↗

**Figure 5.** Figure 5: Examples of ceiling, wall, and floor textures used in our store generation pipeline, illustrating just a subset of possible [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗

**Figure 6.** Figure 6: Example of different geometry approximations for assets (original [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗

**Figure 8.** Figure 8: Examples of heuristically generated anchor poses used in our motion planner. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png] view at source ↗

**Figure 9.** Figure 9: Examples of input images used during training and [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗

**Figure 10.** Figure 10: Simulation time for scenes with varying numbers [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models
cs.LG 2026-06 unverdicted novelty 7.0

Act2Answer protocol reveals VLA models retain simple concepts but show larger gaps on complex semantics than source VLMs, with VQA co-training linked to better retention and knowledge signals peaking in middle layers.

Reference graph

Works this paper leans on

5 extracted references · 5 linked inside Pith · cited by 1 Pith paper

[4]

Murray, R

CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks.IEEE Robotics and Automation Letters (RA-L), 7(3): 7327–7334. Murray, R. M.; Li, Z.; and Sastry, S. S. 2017.A mathemati- cal introduction to robotic manipulation. CRC press. Nasiriany, S.; Maddukuri, A.; Zhang, L.; Parikh, A.; Lo, A.; Joshi, A.; Mandleka...

Pith/arXiv arXiv 2017
[2008]

Graph., 27(3): 1–10

Interactive procedural street modeling.ACM Trans. Graph., 27(3): 1–10. Clark, J. H. 1976. Hierarchical geometric models for visible surface algorithms.Communications of the ACM, 19(10): 547–554. Collaboration, O. X.-E.; O’Neill, A.; Rehman, A.; Gupta, A.; Maddukuri, A.; Gupta, A.; Padalkar, A.; Lee, A.; Poo- ley, A.; Gupta, A.; Mandlekar, A.; Jain, A.; Tu...

Pith/arXiv arXiv 1976
[2020]

Karaman, S.; and Frazzoli, E

RLBench: The Robot Learning Benchmark & Learn- ing Environment.IEEE Robotics and Automation Letters. Karaman, S.; and Frazzoli, E. 2011. Sampling-based algo- rithms for optimal motion planning.The International Jour- nal of Robotics Research, 30(7): 846–894. Khazatsky, A.; Pertsch, K.; Nair, S.; Balakrishna, A.; Dasari, S.; Karamcheti, S.; Nasiriany, S.; ...

Pith/arXiv arXiv 2011
[2022]

In6th Annual Conference on Robot Learning

BEHA VIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation. In6th Annual Conference on Robot Learning. Lindermayr, J.; Odabasi, C.; Jordan, F.; Graf, F.; Knak, L.; Kraus, W.; Bormann, R.; and Huber, M. F. 2023. IPA-3D1K: a large retail 3D model dataset for robot picking. In2023 IEEE/RSJ International Conference on In...

Pith/arXiv arXiv 2023
[2024]

opening and closing show- case

VLABench: A Large-Scale Benchmark for Language- Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks. arXiv:2412.18194. Zucker, M.; Ratliff, N.; Dragan, A. D.; Pivtoraiko, M.; Klin- gensmith, M.; Dellin, C. M.; Bagnell, J. A.; and Srinivasa, S. S. 2013. CHOMP: Covariant Hamiltonian optimization for motion planning.The International Journal ...

Pith/arXiv arXiv 2013

[1] [4]

Murray, R

CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks.IEEE Robotics and Automation Letters (RA-L), 7(3): 7327–7334. Murray, R. M.; Li, Z.; and Sastry, S. S. 2017.A mathemati- cal introduction to robotic manipulation. CRC press. Nasiriany, S.; Maddukuri, A.; Zhang, L.; Parikh, A.; Lo, A.; Joshi, A.; Mandleka...

Pith/arXiv arXiv 2017

[2] [2008]

Graph., 27(3): 1–10

Interactive procedural street modeling.ACM Trans. Graph., 27(3): 1–10. Clark, J. H. 1976. Hierarchical geometric models for visible surface algorithms.Communications of the ACM, 19(10): 547–554. Collaboration, O. X.-E.; O’Neill, A.; Rehman, A.; Gupta, A.; Maddukuri, A.; Gupta, A.; Padalkar, A.; Lee, A.; Poo- ley, A.; Gupta, A.; Mandlekar, A.; Jain, A.; Tu...

Pith/arXiv arXiv 1976

[3] [2020]

Karaman, S.; and Frazzoli, E

RLBench: The Robot Learning Benchmark & Learn- ing Environment.IEEE Robotics and Automation Letters. Karaman, S.; and Frazzoli, E. 2011. Sampling-based algo- rithms for optimal motion planning.The International Jour- nal of Robotics Research, 30(7): 846–894. Khazatsky, A.; Pertsch, K.; Nair, S.; Balakrishna, A.; Dasari, S.; Karamcheti, S.; Nasiriany, S.; ...

Pith/arXiv arXiv 2011

[4] [2022]

In6th Annual Conference on Robot Learning

BEHA VIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation. In6th Annual Conference on Robot Learning. Lindermayr, J.; Odabasi, C.; Jordan, F.; Graf, F.; Knak, L.; Kraus, W.; Bormann, R.; and Huber, M. F. 2023. IPA-3D1K: a large retail 3D model dataset for robot picking. In2023 IEEE/RSJ International Conference on In...

Pith/arXiv arXiv 2023

[5] [2024]

opening and closing show- case

VLABench: A Large-Scale Benchmark for Language- Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks. arXiv:2412.18194. Zucker, M.; Ratliff, N.; Dragan, A. D.; Pivtoraiko, M.; Klin- gensmith, M.; Dellin, C. M.; Bagnell, J. A.; and Srinivasa, S. S. 2013. CHOMP: Covariant Hamiltonian optimization for motion planning.The International Journal ...

Pith/arXiv arXiv 2013