Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Vision in Action: Learning Active Perception from Human Demonstrations

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A policy trained on a single active head camera, with gaze behavior learned from VR demonstrations, outperforms fixed chest and wrist camera setups by 45% on occluded bimanual tasks.

desk verdict Good systems paper, but the headline 45% number rests on a confounded baseline comparison. read the letter →

arxiv 2506.15666 v1 pith:CFTSVW66 submitted 2025-06-18 cs.RO

Haoyu Xiong , Xiaomeng Xu , Jimmy Wu , Yifan Hou , Jeannette Bohg , Shuran Song This is my paper
classification cs.RO
keywords ActivePerceptionBimanualManipulationImitationLearningTeleoperationSystemsVRVisuomotorpolicyRobotneckPointcloudrendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a robot can learn useful active perception strategies—searching, tracking, focusing—directly from human demonstrations, rather than from hand-coded heuristics. The authors build Vision in Action (ViA), a bimanual manipulation system whose "head" is an off-the-shelf 6-DoF arm carrying a camera, and collect demonstrations through a VR teleoperation interface that shares the robot's observations with the operator. Across three multi-stage tasks with heavy visual occlusion, a policy trained on the single head-camera view succeeds 45% more often than one trained on fixed chest and wrist cameras, and adding wrist views actually hurts. The claim is that giving the robot an actively controlled viewpoint and letting the policy learn the operator's gaze behavior closes the observation mismatch between human and robot.

What carries the argument

The load-bearing mechanism is the shared-observation VR teleoperation interface with an intermediate 3D point-cloud scene representation. It renders stereo views instantly from the operator's latest head pose in the world frame while asynchronously updating the robot head pose from aggregated movements, reducing motion-to-photon latency below 7 ms and letting the robot head track the human's search behavior. This recorded head pose is then part of the action space the diffusion policy must predict, so the policy learns to move the camera to find, track, and focus on task-relevant regions.

What would settle it

Re-run the three tasks with the bag task using more than two unseen test objects and additional distractors inside the bag; if the 45% success advantage over the fixed chest-and-wrist setup shrinks dramatically on those new configurations, the active-perception edge is specific to the tested objects rather than a general learning effect. A second check: train ViA on demonstrations collected with a direct stereo-RGB streaming interface instead of the point-cloud rendering; if success does not drop, the intermediate representation is not the decisive design element.

Watch

Extended reading notes

Core claim

The central claim is that active perception for manipulation can be acquired by imitation rather than engineered: a visuomotor policy conditioned on a single active head camera, trained with behavior cloning on human VR demonstrations, outperforms standard camera configurations on occluded bimanual tasks. The paper reports a 45% average improvement in task success over a fixed chest-plus-wrist setup, and a specific result that augmenting the active head view with wrist views reduces success by 18.33% on average, supporting the authors' position that the moving viewpoint itself—not additional static coverage—is what carries the task-relevant information.

Load-bearing premise

The whole pipeline rests on the assumption that head movements demonstrated inside the lower-fidelity point-cloud VR scene are a faithful and sufficient training signal for real-world active perception, so that the policy's learned gaze transfers to the physical robot's noisier observations.

Editorial extensions

If this is right

  • A policy that controls its own viewpoint can succeed at tasks where static or arm-mounted cameras are occluded, making active perception practical for real-world bimanual manipulation.
  • Adding more camera views is not automatically beneficial in low-data imitation learning; task-relevant views from a controlled head camera can dominate redundant wrist views.
  • Learned gaze behavior transfers through behavior cloning to a diffusion policy without task-specific search heuristics or reinforcement learning.
  • The asynchronous point-cloud teleoperation interface reduces VR motion sickness by decoupling rendering latency from robot actuation latency, which may make VR data collection more comfortable and scalable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension beyond the three reported tasks is to test whether the active-head advantage persists in mobile manipulation or in tasks with little occlusion, where a fixed camera might already provide sufficient information.
  • Because the interface removes the observation mismatch between human and robot, the recorded head poses could be reused as supervisory labels for explicit attention or next-best-view models without retraining the whole policy.
  • Replacing the point-cloud scene with denser dynamic fusion (for example, 4D Gaussian splatting) could raise rendering fidelity; if the low-latency rendering property is preserved, it might improve both operator comfort and final task performance.
  • The policy currently lacks memory of previously searched viewpoints, which the paper itself identifies as a limitation; adding such memory is a concrete next step for search-heavy tasks like the Lime and Pot task.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents Vision in Action (ViA), a system for learning active perception from human demonstrations for bimanual manipulation. The hardware uses a 6-DoF robot arm as a movable neck with an RGB-D camera, and a VR teleoperation interface that renders a point-cloud scene in real time based on the operator's head pose while asynchronously updating the robot's camera. The policy is a Diffusion Policy with a DINOv2 visual encoder that takes the active head camera image and proprioceptive state and outputs neck and arm actions. The system is evaluated on three multi-stage occluded manipulation tasks (bag retrieval, cup arrangement, lime-and-pot alignment), comparing camera setups, visual representations, and the teleoperation interface. The paper reports that the active head camera improves average task performance by 45% over a fixed chest-plus-wrist camera setup and that adding wrist views decreases performance by 18.33%, with a user study showing reduced motion sickness.

Significance. The contribution is potentially significant. The idea of using a standard robot arm as a 6-DoF neck and the asynchronous point-cloud VR interface to capture human gaze strategies is elegant and addresses a real bottleneck in imitation learning. The three tasks are well chosen to expose occlusion and active search. If the quantitative claims hold, the system would be a useful step toward learning active perception from demonstration. Strengths include the concrete hardware/interface design, the internal consistency of training all baselines on the same demonstrations, and the inclusion of a user study. The main weaknesses are the potential confound in the camera-setup comparison and the absence of statistical validation.

major comments (3)
  1. [Section 4.1] The comparison between ViA and the Chest & Wrist Cameras baseline is confounded by the demonstration collection protocol. All demonstrations were collected while the human operator viewed only the active head camera feed and used that view to guide arm and neck actions (Section 3.2). The Chest & Wrist policy is therefore trained on observations that were not available to the demonstrator when choosing actions; the arm-action labels encode information visible only in the head view. This likely underestimates what a fixed-camera policy could achieve if demonstrations were collected under that fixed-camera condition, and it inflates the reported 45% advantage. The authors should either collect separate demonstrations for each camera condition or provide evidence (e.g., teleoperation success under chest/wrist views) that the fixed-camera information is sufficient for the demonstrated actions.
  2. [Section 4.1, Figures 5-6] No error bars, confidence intervals, or significance tests are reported. With 10-20 rollouts per condition (Section 4), the differences between conditions (e.g., 80% vs 65% in the cup task) are within plausible sampling noise. All quantitative claims (45% improvement, 18.33% decrease) should be accompanied by standard errors, bootstrap confidence intervals, or a statement of per-seed variance, and ideally multiple training seeds.
  3. [Sections 3.2 and 6] The human demonstrator observes a rendered point-cloud view, whereas the policy is trained on the raw RGB image from the active head camera (Section 3.3). The paper calls this a shared observation space, but the point-cloud rendering may contain depth/geometric cues not present in RGB and is acknowledged to be lower fidelity than RGB streaming. This observation mismatch could limit the transfer of the learned active perception strategy to the physical robot and should be addressed explicitly, for example by ablating whether the policy is trained on the rendered views or by analyzing how much of the human's head movements are recoverable from the RGB stream alone.
minor comments (5)
  1. [Section 3.1] The iPhone 15 Pro is cited as [6], which is "Robot Utility Models"; this appears to be a citation error and should be replaced with a hardware/manual reference or removed.
  2. [Figures 5 and 6] The bar labels are dense and hard to read; the figures would benefit from clearer annotations and a description of how the final "average task performance" numbers are computed from the stage-wise success rates.
  3. [Section 4] The paper reports 150, 125, and 260 demonstrations but does not state the number of training seeds or the variance across seeds; please add this information to the supplement.
  4. [Figure 1 caption] The repeated "Target Visible / Target Not Visible" text in the caption appears garbled and should be cleaned up.
  5. [Section 3.2] The paper would benefit from a brief discussion of why the point-cloud rendering refresh rate (10 Hz) and control frequency (<10 Hz) are sufficient for the three tasks, since this may not hold for faster manipulation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: ViA's 45% active-head advantage is an empirical, held-out comparison; the shared-demonstration chest/wrist baseline is a validity confound, not a circular reduction.

full rationale

The paper's load-bearing claim is an empirical evaluation on held-out test configurations: unseen bag objects and fresh cup/lime configurations, with success measured by physical task completion. No parameter is fitted to the reported success rates, and the 45% improvement is not computed from the head-camera input by any equation or definition. The chest/wrist baseline is trained on the same recorded demonstrations, which is standard for an observation-modality ablation, though it does not isolate the data-collection viewpoint; this is a legitimate external-validity concern about whether the margin would persist under fixed-camera demonstrations, but it is not circularity under the provided taxonomy. The Diffusion Policy backbone is a same-group citation but is a widely reproduced external method, not an unverified self-supporting uniqueness argument. No ansatz is smuggled in via citation, no known result is renamed, and no author-imported uniqueness theorem is invoked. The limitation about point-cloud fidelity is an honest acknowledgment of transfer risk, not a circular step. Overall, the derivation chain is self-contained as an empirical system evaluation, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper is empirical; it introduces no fitted physical constants. The central comparison is measured, not derived. The load-bearing premises are domain assumptions about the teleoperation interface, the visual representation, and task representativeness.

assumptions (3)
  • domain assumption Human head movements captured through the VR point-cloud interface encode task-relevant active perception strategies that can be learned by a diffusion policy.
    The method trains the policy on these demonstrations; its success depends on this transfer. Evaluated only on three tabletop tasks (Section 3.2, Section 4).
  • domain assumption The point-cloud rendered view, despite acknowledged lower fidelity, preserves enough task-relevant information for the operator to perform fine manipulation.
    Limitations concede lower fidelity; if the operator cannot see fine details, demonstrations may miss critical cues for tasks like precise alignment.
  • domain assumption DINOv2 pretrained visual features are a sufficient semantic representation for active search and manipulation in these tasks.
    The policy relies on a frozen DINOv2 encoding; the paper does not ablate it beyond comparing to ResNet and DP3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vision in Action: Learning Active Perception from Human Demonstrations." pith.science (2026). https://pith.science/paper/CFTSVW66

@misc{pith2026250615666,
  author       = {Pith},
  title        = {Pith review of: Vision in Action: Learning Active Perception from Human Demonstrations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CFTSVW66}},
  note         = {Machine review of arXiv:2506.15666}
}
read the original abstract

We present Vision in Action (ViA), an active perception system for bimanual robot manipulation. ViA learns task-relevant active perceptual strategies (e.g., searching, tracking, and focusing) directly from human demonstrations. On the hardware side, ViA employs a simple yet effective 6-DoF robotic neck to enable flexible, human-like head movements. To capture human active perception strategies, we design a VR-based teleoperation interface that creates a shared observation space between the robot and the human operator. To mitigate VR motion sickness caused by latency in the robot's physical movements, the interface uses an intermediate 3D scene representation, enabling real-time view rendering on the operator side while asynchronously updating the scene with the robot's latest observations. Together, these design elements enable the learning of robust visuomotor policies for three complex, multi-stage bimanual manipulation tasks involving visual occlusions, significantly outperforming baseline systems.

Figures

Figures reproduced from arXiv: 2506.15666 by the authors.

Figure 1
Figure 1. Vision in Action (ViA) uses an active head camera to search for the target object (yellow banana) inside the bag. The wrist cameras are ineffective in this visually occluded scenario, as they are constrained by the arm motions. Perception is inherently active [1]. Consider the task of retrieving a banana from a bag ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. VR Teleoperation Comparison. [Left] Traditional RGB streaming suffers from motion-to-photon latency due to both RGB data transmission latency and robot control latency, often leading to VR motion sickness. [Right] Our system mitigates this by: (a, e) streaming a 3D point cloud in the world frame from RGB￾D data, (b, c) performing real-time view rendering based on the user’s latest head pose, and (d) asynchronously u… view at source ↗
Figure 3
Figure 3. Task Definitions. We introduce three multi-stage tasks that highlight the critical role of active perception in everyday scenarios. [Left] Third-person view with red arrows indicating head movements and blue arrows indicating arm movements. [Middle] Active head camera views across task stages (upper row), and third-person view of robot actions (lower row). [Right] Test scenarios, including training and testing objec… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Policy Learning Camera Setup Comparison. [ViA] uses a single active head camera that dynami￾cally adjusts its viewpoint to capture task-relevant visual information (e.g., finding a cup hidden inside a shelf). In contrast, [Wrist & Chest cameras] policy often fails due …
Figure 5
Figure 5. Figure 5: Policy Learning Camera Setup Comparison Results. We report stage-wise success rates across the three tasks to demonstrate the effectiveness of our active head camera [ViA] compared to two baseline configurations: [Active Head & Wrist Cameras] and [Chest & Wrist Cameras…
Figure 6
Figure 6. Figure 6: Policy Learning Visual Representation Comparison Results. We report stage-wise success rates across the three tasks to demonstrate the effectiveness of our method [ViA], in comparison to two baseline approaches: [ResNet-DP] and [DP3]. • [ResNet-DP]: A baseline using a …
Figure 7
Figure 7. Figure 7: Teleoperation Interface Comparison. We evaluate our teleoperation interface design based on three metrics: reported levels of motion sickness, aver￾age duration to complete each demonstration, and over￾all user preference. In this experiment, we evaluate our VR tele￾op…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World

    cs.RO 2026-04 unverdicted novelty 6.5 of 10

    EgoVerse releases 1,362 hours of standardized egocentric human data across 1,965 tasks and shows via multi-lab experiments that robot policy performance scales with human data volume when the data aligns with robot ob...

  2. B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Predicting continuous B-spline action curves instead of discrete chunks lets visuomotor policies finish real and simulated manipulation tasks substantially faster without large success-rate loss.

  3. Act, Sense, Act: Learning Active Perception from Large-Scale Egocentric Human Data

    cs.RO 2026-02 conditional novelty 6.0 of 10

    CoMe-VLA combines cognitive subtask labels and dual-track memory with human egocentric pretraining, reaching 83% mean success on five active-perception manipulation tasks.

Reference graph

Works this paper leans on

68 extracted references · 44 canonical work pages · cited by 3 Pith papers

  1. [1]

    R. Bajcsy. Active perception.Proceedings of the IEEE, 76(8):966–1005, 1988. doi:10.1109/ 5.5968

    work page 1988

  2. [2]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    arXiv 2023

  3. [3]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. InConference on Robot Learning (CoRL), 2024

    2024

  4. [4]

    Kareer, D

    S. Kareer, D. Patel, R. Punamiya, P. Mathur, S. Cheng, C. Wang, J. Hoffman, and D. Xu. Egomimic: Scaling imitation learning via egocentric video, 2024. URLhttps://arxiv. org/abs/2410.24221

    arXiv 2024

  5. [5]

    C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation.arXiv preprint arXiv:2403.07788, 2024

    arXiv 2024

  6. [6]

    Etukuru, N

    H. Etukuru, N. Naka, Z. Hu, S. Lee, J. Mehu, A. Edsinger, C. Paxton, S. Chintala, L. Pinto, and N. M. M. Shafiullah. Robot utility models: General policies for zero-shot deployment in new environments.arXiv preprint arXiv:2409.05865, 2024

    arXiv 2024

  7. [7]

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  8. [8]

    F. Lin, Y . Hu, P. Sheng, C. Wen, J. You, and Y . Gao. Data scaling laws in imitation learning for robotic manipulation.arXiv preprint arXiv:2410.18647, 2024

    arXiv 2024

Show all 68 references
  1. [9]

    K. Shaw, Y . Li, J. Yang, M. K. Srirama, R. Liu, H. Xiong, R. Mendonca, and D. Pathak. Bimanual dexterity for complex tasks. In8th Annual Conference on Robot Learning, 2024

    2024

  2. [10]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. InProceedings of Robotics: Science and Systems (RSS), 2023

    2023

  3. [11]

    Shigemi.ASIMO and Humanoid Robot Research at Honda, pages 55–90

    S. Shigemi.ASIMO and Humanoid Robot Research at Honda, pages 55–90. Springer Nether- lands, Dordrecht, 2019. ISBN 978-94-007-6046-2. doi:10.1007/978-94-007-6046-2 9. URL https://doi.org/10.1007/978-94-007-6046-2_9

    2019 doi

  4. [12]

    Kaneko, K

    K. Kaneko, K. Harada, F. Kanehiro, G. Miyamori, and K. Akachi. Humanoid robot hrp-3. In2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2471– 2478, 2008. doi:10.1109/IROS.2008.4650604

    2008

  5. [13]

    Elmogy, C

    M. Elmogy, C. Habel, and J. Zhang. Online motion planning for hoap-2 humanoid robot navigation. In2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3531–3536, 2009. doi:10.1109/IROS.2009.5354572

    2009

  6. [14]

    Cheng, J

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang. Open-television: Teleoperation with immer- sive active visual feedback.arXiv preprint arXiv:2407.01512, 2024

    2024 arXiv

  7. [15]

    B. Sen, M. Wang, N. Thakur, A. Agarwal, and P. Agrawal. Learning to look around: Enhancing teleoperation and learning with a human-like actuated neck, 2024. URLhttps://arxiv. org/abs/2411.00704

    2024 arXiv

  8. [16]

    Zheng, Y

    W. Zheng, Y . Sun, H. Wu, H. Sun, and D. Zhang. The interaction of top–down and bottom–up attention in visual working memory.Scientific Reports, 14(1):17397, 2024

    2024

  9. [17]

    Itti and C

    L. Itti and C. Koch. Computational modelling of visual attention.Nature Reviews Neuro- science, 2(3):194–203, Mar 2001. 10

    2001

  10. [18]

    L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis.IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11):1254–1259, Nov 1998

    1998

  11. [19]

    J. K. Tsotsos, S. M. Culhane, W. Y . K. Wai, Y . Lai, N. Davis, and F. Nuflo. Modeling visual attention via selective tuning.Artificial Intelligence, 78(1–2):507 – 545, 1995. ISSN 0004-

    1995

  12. [20]

    Rasolzadeh, M

    B. Rasolzadeh, M. Bj ¨orkman, K. Huebner, and D. Kragic. An active vision system for detect- ing, fixating and manipulating objects in the real world.The International Journal of Robotics Research, 29(2-3):133–154, 2010. doi:10.1177/0278364909346069

    2010 doi

  13. [21]

    Grotz, T

    M. Grotz, T. Habra, R. Ronsse, and T. Asfour. Autonomous view selection and gaze stabiliza- tion for humanoid robots. In2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1427–1434, 2017. doi:10.1109/IROS.2017.8205944

    2017

  14. [22]

    J. Bohg, K. Welke, B. Le ´on, M. Do, D. Song, W. Wohlkinger, M. Madry, A. Ald ´oma, M. Przybylski, T. Asfour, H. Mart ´ı, D. Kragic, A. Morales, and M. Vincze. Task- based grasp adaptation on a humanoid robot.IFAC Proceedings Volumes, 45(22): 779–786, 2012. ISSN 1474-6670. doi...

    2012 doi

  15. [23]

    R. Pito. A solution to the next best view problem for automated surface acquisition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10):1016–1030, 1999. doi: 10.1109/34.799908

    1999 doi

  16. [24]

    Bajcsy, Y

    R. Bajcsy, Y . Aloimonos, and J. K. Tsotsos. Revisiting active perception.CoRR, abs/1603.02729, 2016

    2016 arXiv

  17. [25]

    D. H. Ballard. Animate vision.Artificial intelligence, 48(1):57–86, 1991

    1991

  18. [26]

    Aloimonos, I

    J. Aloimonos, I. Weiss, and A. Bandyopadhyay. Active vision.International Journal of Com- puter Vision, 1:333 – 356, 1988

    1988

  19. [27]

    H. Shi, W. Wang, S. Song, and C. K. Liu. Toddlerbot: Open-source ml-compatible humanoid platform for loco-manipulation, 2025. URLhttps://arxiv.org/abs/2502.00893

    2025

  20. [28]

    Righetti, M

    L. Righetti, M. Kalakrishnan, P. Pastor, J. Binney, J. Kelly, R. V oorhies, G. Sukhatme, and S. Schaal. An autonomous manipulation system based on force control and optimization. Autonomous Robots, 36:11–30, 01 2014. doi:10.1007/s10514-013-9365-9

    2014 doi

  21. [29]

    Asfour, K

    T. Asfour, K. Welke, P. Azad, A. Ude, and R. Dillmann. The karlsruhe humanoid head. In Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots, pages 447– 453, 2008. doi:10.1109/ICHR.2008.4755993

    2008

  22. [30]

    Pahlavan and J.-O

    K. Pahlavan and J.-O. Eklundh. A head-eye system—analysis and design.CVGIP: Image Un- derstanding, 56(1):41–56, 1992. ISSN 1049-9660. doi:https://doi.org/10.1016/1049-9660(92) 90084-G. URLhttps://www.sciencedirect.com/science/article/pii/ 104996609290084G. Purposive, Qualitati...

    1992 doi

  23. [31]

    Kappler, F

    D. Kappler, F. Meier, J. Issac, J. Mainprice, C. G. Cifuentes, M. W¨uthrich, V . Berenz, S. Schaal, N. Ratliff, and J. Bohg. Real-time perception meets reactive motion generation.IEEE Robotics and Automation Letters, 3(3):1864–1871, 2018. doi:10.1109/LRA.2018.2795645

    2018

  24. [32]

    Olson and R

    T. Olson and R. Potter. Real time vergence control. InProceedings CVPR ’89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 404–409, 1989. doi: 10.1109/CVPR.1989.37878. 11

    1989

  25. [33]

    J. L. Crowley, P. Bobet, and M. Mesrabi. Gaze control for a binocular camera head. In G. Sandini, editor,Computer Vision — ECCV’92, pages 588–596, Berlin, Heidelberg, 1992. Springer Berlin Heidelberg. ISBN 978-3-540-47069-4

    1992

  26. [34]

    Lenz and S

    C. Lenz and S. Behnke. Bimanual telemanipulation with force and haptic feedback through an anthropomorphic avatar system.Robotics and Autonomous Systems, 161:104338, 2023. ISSN 0921-8890. doi:https://doi.org/10.1016/j.robot.2022.104338. URLhttps://www. sciencedirect.com/scienc...

    2023

  27. [35]

    J. Lv, Y . Feng, C. Zhang, S. Zhao, L. Shao, and C. Lu. SAM-RL: Sensing-Aware Model- Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering. InProceedings of Robotics: Science and Systems, Daegu, Republic of Korea, July 2023. doi: 10.15607/RSS.2...

    2023 doi

  28. [36]

    R. Zeng, Y . Wen, W. Zhao, and Y .-J. Liu. View planning in robot active vision: A survey of systems, algorithms, and applications.Computational Visual Media, 6(3):225–245, Sept

  29. [37]

    Krainin, B

    M. Krainin, B. Curless, and D. Fox. Autonomous generation of complete 3d object models us- ing next best view manipulation planning. In2011 IEEE International Conference on Robotics and Automation, pages 5031–5037, 2011. doi:10.1109/ICRA.2011.5980429

    2011

  30. [38]

    S. Dass, J. Hu, B. Abbatematteo, P. Stone, and R. Mart ´ın-Mart´ın. Learning to look: Seeking information for decision making via policy factorization.arXiv preprint arXiv:2410.18964, 2024

    2024 arXiv

  31. [39]

    Cheng, A

    R. Cheng, A. Agarwal, and K. Fragkiadaki. Reinforcement learning of active vision for manip- ulating objects under occlusions, 2019. URLhttps://arxiv.org/abs/1811.08067

    2019 arXiv

  32. [40]

    Shang and M

    J. Shang and M. S. Ryoo. Active vision reinforcement learning under limited visual observ- ability, 2023. URLhttps://arxiv.org/abs/2306.00975

    2023 arXiv

  33. [41]

    Jayaraman and K

    D. Jayaraman and K. Grauman. Learning to look around: Intelligently exploring unseen envi- ronments for unknown tasks, 2017. URLhttps://arxiv.org/abs/1709.00507

    2017 arXiv

  34. [42]

    Uppal, A

    S. Uppal, A. Agarwal, H. Xiong, K. Shaw, and D. Pathak. Spin: Simultaneous perception interaction and navigation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18133–18142, June 2024

    2024

  35. [43]

    J. Wu, W. Chong, R. Holmberg, A. Prasad, Y . Gao, O. Khatib, S. Song, S. Rusinkiewicz, and J. Bohg. Tidybot++: An open-source holonomic mobile manipulator for robot learning. In Conference on Robot Learning, 2024

    2024

  36. [44]

    Mandlekar, D

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart´ın-Mart´ın. What matters in learning from offline human demonstrations for robot manipulation.arXiv preprint arXiv:2108.03298, 2021

    2021 arXiv

  37. [45]

    Jiang, R

    Y . Jiang, R. Zhang, J. Wong, C. Wang, Y . Ze, H. Yin, C. Gokmen, S. Song, J. Wu, and L. Fei- Fei. Behavior robot suite: Streamlining real-world whole-body manipulation for everyday household activities, 2025. URLhttps://arxiv.org/abs/2503.05652

    2025 arXiv

  38. [46]

    X. Xu, D. Bauer, and S. Song. Robopanoptes: The all-seeing robot with whole-body dexterity. arXiv preprint arXiv:2501.05420, 2025

    2025

  39. [47]

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12156–12163. IEEE, 2024. 12

    2024

  40. [48]

    Y . Liu, X. Xu, W. Chen, H. Yuan, H. Wang, J. Xu, R. Chen, and L. Yi. Enhancing generalizable 6d pose tracking of an in-hand object with tactile sensing.IEEE Robotics and Automation Letters, 9(2):1106–1113, 2023

    2023

  41. [49]

    A. Iyer, Z. Peng, Y . Dai, I. Guzey, S. Haldar, S. Chintala, and L. Pinto. Open teach: A versatile teleoperation system for robotic manipulation.arXiv preprint arXiv:2403.07870, 2024

    2024 arXiv

  42. [50]

    M. Seo, S. Han, K. Sim, S. H. Bang, C. Gonzalez, L. Sentis, and Y . Zhu. Deep imitation learn- ing for humanoid loco-manipulation through human teleoperation. InIEEE-RAS International Conference on Humanoid Robots (Humanoids), 2023

    2023

  43. [51]

    Y . Liu, S. Mu, X. Chao, Z. Li, Y . Mu, T. Chen, S. Li, C. Lyu, X.-p. Zhang, and W. Ding. Avr: Active vision-driven robotic precision manipulation with viewpoint and focal length optimiza- tion.arXiv preprint arXiv:2503.01439, 2025

    2025

  44. [52]

    Chuang, A

    I. Chuang, A. Lee, D. Gao, M.-M. Naddaf-Sh, and I. Soltani. Active vision might be all you need: Exploring active vision in bimanual robotic manipulation, 2025. URLhttps: //arxiv.org/abs/2409.17435

    2025 arXiv

  45. [53]

    Zhang, Z

    T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep im- itation learning for complex manipulation tasks from virtual reality teleoperation. In2018 IEEE International Conference on Robotics and Automation (ICRA), pages 5628–5635, 2018. doi:10.1109/I...

    2018

  46. [54]

    Y . Ze, Z. Chen, W. Wang, T. Chen, X. He, Y . Yuan, X. B. Peng, and J. Wu. Generalizable humanoid manipulation with 3d diffusion policies, 2025. URLhttps://arxiv.org/ abs/2410.10803

    2025 arXiv

  47. [55]

    U. A. Chattha, U. I. Janjua, F. Anwar, T. M. Madni, M. F. Cheema, and S. I. Janjua. Motion sickness in virtual reality: An empirical evaluation.IEEE Access, 8:130486–130499, 2020. doi:10.1109/ACCESS.2020.3007076

    2020

  48. [56]

    J. Zhao, R. S. Allison, M. Vinnikov, and S. Jennings. Estimating the motion-to-photon latency in head mounted displays. In2017 IEEE Virtual Reality (VR), pages 313–314, 2017. doi: 10.1109/VR.2017.7892302

    2017

  49. [57]

    Wilder-Smith, V

    M. Wilder-Smith, V . Patil, and M. Hutter. Radiance fields for robotic teleoperation.arXiv, 2024

    2024

  50. [58]

    H. Ha, Y . Gao, Z. Fu, J. Tan, and S. Song. UMI on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers. InProceedings of the 2024 Conference on Robot Learning, 2024

    2024

  51. [59]

    M. Yang, J. Zhang, and L. Yu. Perceptual tolerance to motion-to-photon latency with head movement in virtual reality. In2019 Picture Coding Symposium (PCS), pages 1–5, 2019. doi:10.1109/PCS48520.2019.8954518

    2019

  52. [60]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  53. [61]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. InPro- ceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  54. [62]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierar- chical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 13

    2009

  55. [63]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3d diffusion policy.arXiv e-prints, pages arXiv–2403, 2024

    2024

  56. [64]

    G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang. 4d gaussian splatting for real-time dynamic scene rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20310–20320, June 2024

    2024

  57. [65]

    X. Xu, H. Ha, and S. Song. Dynamics-guided diffusion model for robot manipulator design. arXiv preprint arXiv:2402.15038, 2024

    2024 arXiv

  58. [66]

    Schneider, D

    R. Schneider, D. Honerkamp, T. Welschehold, and A. Valada. Task-driven co-design of mobile manipulators.arXiv preprint arXiv:2412.16635, 2024. 14

    2024 arXiv

  59. [2020]

    doi:10.1007/s41095-020-0179-3

    ISSN 2096-0662. doi:10.1007/s41095-020-0179-3. URLhttps://doi.org/10. 1007/s41095-020-0179-3

    doi

  60. [3702]

    Special V olume on Computer Vision

    doi:http://dx.doi.org/10.1016/0004-3702(95)00025-9. Special V olume on Computer Vision

    doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.