Labimus is the first benchmark for humanoid dexterous manipulation in organic chemistry laboratories, exposing a gap between task completion and required experimental precision.
Autobio: A simulation and benchmark for robotic automation in digital biology laboratory
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
abstract
Vision-language-action (VLA) models have shown promise as generalist robotic policies by jointly leveraging visual, linguistic, and proprioceptive modalities to generate action trajectories. While recent benchmarks have advanced VLA research in domestic tasks, professional science-oriented domains remain underexplored. We introduce AutoBio, a simulation framework and benchmark designed to evaluate robotic automation in biology laboratory environments--an application domain that combines structured protocols with demanding precision and multimodal interaction. AutoBio extends existing simulation capabilities through a pipeline for digitizing real-world laboratory instruments, specialized physics plugins for mechanisms ubiquitous in laboratory workflows, and a rendering stack that support dynamic instrument interfaces and transparent materials through physically based rendering. Our benchmark comprises biologically grounded tasks spanning three difficulty levels, enabling standardized evaluation of language-guided robotic manipulation in experimental protocols. We provide infrastructure for demonstration generation and seamless integration with VLA models. Baseline evaluations with two SOTA VLA models reveal significant gaps in precision manipulation, visual reasoning, and instruction following in scientific workflows. By releasing AutoBio, we aim to catalyze research on generalist robotic systems for complex, high-precision, and multimodal professional environments. The simulator and benchmark are publicly available to facilitate reproducible research.
citation-role summary
background 3
citation-polarity summary
roles
background 3polarities
background 3representative citing papers
SpatialWorld is a new multi-simulator benchmark showing top multimodal agents achieve under 18% success on interactive spatial tasks requiring active exploration and long-horizon planning.
BiCoord is a new benchmark for long-horizon tightly coordinated bimanual manipulation that includes quantitative metrics and shows existing policies like DP, RDT, Pi0 and OpenVLA-OFT struggle on such tasks.
ST-BiBench reveals a coordination paradox in which MLLMs show strong high-level strategic reasoning yet fail at fine-grained 16-dimensional bimanual action synthesis and multi-stream fusion.
A modular benchmark of 100 dexterous manipulation tasks across 3 arms and 6 hands with 3,180 demonstrations reveals that current policies (Diffusion Policy, DP3, OpenVLA, π0.5) achieve only 34% mean success, exposing unsolved challenges in contact-rich and precise manipulation.
A wet-lab simulation benchmark and success-checked augmentation loop for turning 30 demonstrations per task into policy training data, reported to lift SmolVLA from 44% to 75% simulated success.
RoboTwin 2.0 automates diverse synthetic data creation for dual-arm robots via MLLMs and five-axis domain randomization, leading to 228-367% gains in manipulation success.
A simulation-trained model predicts voxel occupancy from single RGB views for transparent object grasping and transfers to real robotic setups without fine-tuning.
citing papers explorer
-
Labimus: A Simulation and Benchmark for Humanoid Dexterous Manipulation in Chemical Laboratory
Labimus is the first benchmark for humanoid dexterous manipulation in organic chemistry laboratories, exposing a gap between task completion and required experimental precision.
-
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
SpatialWorld is a new multi-simulator benchmark showing top multimodal agents achieve under 18% success on interactive spatial tasks requiring active exploration and long-horizon planning.
-
BiCoord: A Bimanual Manipulation Benchmark towards Long-Horizon Spatial-Temporal Coordination
BiCoord is a new benchmark for long-horizon tightly coordinated bimanual manipulation that includes quantitative metrics and shows existing policies like DP, RDT, Pi0 and OpenVLA-OFT struggle on such tasks.
-
ST-BiBench: Benchmarking Multi-Stream Multimodal Coordination in Bimanual Embodied Tasks for MLLMs
ST-BiBench reveals a coordination paradox in which MLLMs show strong high-level strategic reasoning yet fail at fine-grained 16-dimensional bimanual action synthesis and multi-stream fusion.
-
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
A modular benchmark of 100 dexterous manipulation tasks across 3 arms and 6 hands with 3,180 demonstrations reveals that current policies (Diffusion Policy, DP3, OpenVLA, π0.5) achieve only 34% mean success, exposing unsolved challenges in contact-rich and precise manipulation.
-
Pipette: An Embodied Simulation Platform, Benchmark, and Data-Efficient Augmentation Framework for Wet-Lab Robotics
A wet-lab simulation benchmark and success-checked augmentation loop for turning 30 demonstrations per task into policy training data, reported to lift SmolVLA from 44% to 75% simulated success.
-
RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
RoboTwin 2.0 automates diverse synthetic data creation for dual-arm robots via MLLMs and five-axis domain randomization, leading to 228-367% gains in manipulation success.
-
Trans2Occ: Voxel Occupancy Estimation and Grasp for Transparent Objects from Simulation to Reality
A simulation-trained model predicts voxel occupancy from single RGB views for transparent object grasping and transfers to real robotic setups without fine-tuning.