Pith. sign in

REVIEW 4 major objections 6 minor 71 references

UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

T0 review · 4 major / 6 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single multimodal language model can choose views, ground affordances, and reason about robot base pose for zero-shot last-mile navigation.

desk verdict Clean three-stage MLLM last-mile recipe with a real but modest OVMM gain; useful systems work, not a statistical slam dunk. read the letter →

arxiv 2607.06537 v2 pith:N4VP26JF submitted 2026-07-07 cs.RO

Zhuofan Zhang , Tianxu Wang , Guoxi Zhang , Yixiong Lin , Xilin Wang , Hongming Xu , Qing Li , Song-Chun Zhu
show 1 more author
Lifeng Fan
This is my paper · ORCID
classification cs.RO
keywords mobilemanipulationlast-milenavigationzero-shotopen-vocabularymultimodallargelanguagemodelaffordancegroundingbase-posereasoningOVMM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Mobile robots often stop near a target after object navigation but still cannot manipulate it because their base is poorly placed for reach, clearance, or task geometry. This paper argues that last-mile navigation—the final base adjustment that makes manipulation feasible—can be solved without task-specific training or pose labels by decomposing the problem into three stages that a shared multimodal large language model handles: picking a usable camera view from recent memory, grounding a task-conditioned affordance point, and predicting a geometry-aware base pose. On the open-vocabulary mobile-manipulation benchmark the method raises overall success over the prior zero-shot leader and remains competitive even with a small robotics-tuned model; real-robot trials on a quadruped with an arm confirm the pipeline can run outside simulation. A sympathetic reader cares because the same off-the-shelf model family can now close the navigation-to-manipulation gap for open-vocabulary instructions that include fine spatial relations such as “in front of the monitor.”

What carries the argument

UniLM-Nav’s three-stage shared-MLLM pipeline: view selection from a short-term RGB-D memory, task-conditioned 2D affordance grounding lifted to robot-centric 3D, and geometry-aware base-pose reasoning that predicts (x, y) while orienting the robot toward the affordance.

What would settle it

On the same OVMM validation protocol, replace the full three-stage UniLM-Nav pipeline with a strong object-navigation endpoint plus simple turn-to-face or floor-point heuristics and check whether overall success still exceeds the prior zero-shot leader MoTo by a clear margin; if the gap disappears, the claimed benefit of the unified MLLM decomposition is falsified.

Watch

Extended reading notes

Core claim

UniLM-Nav shows that last-mile navigation for open-vocabulary mobile manipulation can be performed zero-shot by a single multimodal large language model that first selects a reference view from short-term memory, then grounds a task-relevant affordance and lifts it to 3D, and finally reasons over that geometry plus robot configuration to output a manipulation-ready base pose, outperforming prior zero-shot and several trained baselines on OVMM.

Load-bearing premise

The pipeline assumes object navigation already leaves the robot roughly one to two meters from the target with that target visible in a handful of recent camera frames; if the target never appears in that memory, last-mile navigation has no usable input.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. UniLM-Nav proposes a zero-shot, open-vocabulary last-mile navigation framework that bridges object-goal navigation and manipulation-ready base positioning. The method decomposes the problem into three MLLM-driven stages—view selection from a short-term RGB-D memory (K=5), task-conditioned 2D affordance grounding with depth lifting to a robot-centric 3D point, and geometry-aware base-pose reasoning that predicts (x,y) while computing heading geometrically toward the affordance. On the HomeRobot OVMM validation set, UniLM-Nav with Gemini-3-Flash-Preview reports 23.77% Overall SR, exceeding MoTo (20.64%) by 3.13 points and training-based MoManipVLA (15.80%); a 4B embodied backend (RoboBrain-2.5-4B) remains competitive at 19.19%. Component ablations on a 20% stratified subset, multi-backend comparisons, failure taxonomy, and four real-world tasks (10 trials each) on a Unitree B2+Z1 platform are provided to support the design.

Significance. Last-mile navigation is a genuine bottleneck in modular open-vocabulary mobile manipulation: proximity after object nav often fails to yield a reachable, collision-free, task-aligned base pose. A training-free pipeline that uses a shared MLLM for view selection, fine-grained spatial affordance (beyond object-level cues), and explicit geometry-conditioned base reasoning is a useful systems contribution. Strengths include (i) a clean three-stage formulation with released-style prompts in the appendix, (ii) multi-backend analysis showing that embodied fine-tuning (RoboBrain-2.5-4B) can rival much larger general models, (iii) qualitative error breakdown of view/affordance/base failures, and (iv) real-robot deployment with spatial-relation tasks. If the SOTA margin and component attributions hold under stronger experimental controls, the work would be a solid reference for MLLM-based navigation–manipulation handoff.

major comments (4)
  1. Table 1 reports the headline 23.77% Overall SR (Gemini) vs MoTo 20.64% as a single point estimate with no multi-seed variance, confidence intervals, or sensitivity to sampling temperature/prompt wording. Given that OVMM episodes are stochastic (navigation, perception, contact), a 3.13-point absolute gain cannot be assessed for robustness from one run. Please report at least repeated evaluation variance or bootstrap-style uncertainty on Overall SR for UniLM-Nav and the primary baseline under matched seeds/policies.
  2. §5.2 and Fig. 3 / Tables 2,4–6: all component ablations (w/o last-mile, w/o view selection, w/o base-pose reasoning, merging stages, heading strategy, thinking models) and the detailed MLLM backend comparison are run only on a scene-stratified 20% subset, where Gemini already reaches 25.42% Overall SR—higher than the full-set 23.77%. Component importance and backend rankings may therefore be inflated or reordered by subset selection. Either re-run the key ablations (at least w/o view selection and w/o geometry-aware base-pose reasoning) on the full validation set, or show that the 20% subset is distributionally matched and that full-set deltas are consistent.
  3. §5.1 Setup and Table 1: navigation and grasping use default HomeRobot policies, while placement uses a simple MLLM-predicted arm-extension/lift policy integrated with base-pose reasoning. MoTo and other baselines may differ in placement execution. The Overall SR gain therefore conflates last-mile base positioning with a custom placement controller and a stronger proprietary MLLM. Please isolate the contribution of the predicted base pose alone (e.g., fixed placement policy across methods, or an ablation that keeps UniLM-Nav’s placement but replaces base pose with MoTo/HomeRobot heuristics) so the 3.13-point claim can be attributed to the proposed decomposition.
  4. §3–4 and Limitations: the pipeline assumes object navigation leaves the target visible in the last K=5 frames within ~1–2 m. Error analysis (Appendix B.4) attributes 61% of failures to pre–last-mile navigation, so headline OVMM gains partly reflect interaction with an upstream policy that often fails before UniLM-Nav runs. The paper should quantify conditional success given successful FindObj/FindRec near-target states (last-mile-only metrics), and more clearly bound the claim to the near-target regime rather than end-to-end OVMM superiority without that conditioning.
minor comments (6)
  1. Abstract/§5.1: “Gemini-3-Flash-Previw” / “Gemini3-Flash-Preview” spelling is inconsistent; standardize the model name throughout.
  2. Fig. 3(a) uses hard-to-read Unicode-style labels in the manuscript text dump; ensure the published figure has clear axis labels and method names.
  3. Eq. (3) predicts (x,y) but the surrounding text and prompts also discuss theta and arm parameters; clarify in the main text which quantities are MLLM outputs vs. geometric post-processing (heading is geometric; arm reach/lift are MLLM-predicted for placement).
  4. Table 1 “Average SR” definition (average of partial rates and Overall SR) is unconventional; state explicitly whether this is an official OVMM metric or an author-defined aggregate.
  5. Related work could more sharply contrast UniLM-Nav with Lin et al. (AAAI 2026) affordance-guided base placement and MoMa-Kitchen, beyond a one-line mention.
  6. Real-world §5.4: report failure modes for the two harder spatial-relation tasks (4/10) in the main text, not only success counts, to match the sim error analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: UniLM-Nav is an empirical modular pipeline whose SOTA claim is an external OVMM success-rate comparison, not a quantity defined from its own fitted inputs.

full rationale

The paper presents a zero-shot engineering pipeline (view selection from short-term memory Mt, 2D affordance grounding, depth lift to pa, then MLLM base-pose prediction with geometrically computed heading) and evaluates it against the external HomeRobot OVMM benchmark and real-robot trials. Table 1 reports Overall SR numbers (23.77% with Gemini-3-Flash-Preview vs MoTo 20.64%) that are measured outcomes of running the pipeline with fixed HomeRobot navigation/grasp policies; they are not algebraic rearrangements of parameters fitted to the same metric. Ablations (Fig. 3, Tables 2/4/5/6) remove modules or swap MLLM backends and re-measure success rates; none redefine a fitted coefficient as a 'prediction.' Self-citations (e.g., MoTo [18], HomeRobot [23], RoboBrain [24]) appear only as related-work baselines or optional backends, not as uniqueness theorems that force the method. The near-target K=5 assumption is an explicit scope limitation, not a circular definition of the reported SR. Therefore the derivation chain contains no self-definitional, fitted-as-prediction, or load-bearing self-citation circularity; score 0 is appropriate.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central empirical claim rests on standard robotics assumptions (RGB-D, odometry, obstacle map, pre-existing object-nav policy), a fixed short-term memory size, robot kinematic parameters supplied to the MLLM, and the off-the-shelf reasoning ability of current MLLMs. No new physical constants or particles are postulated; free parameters are engineering choices (K, arm-reach heuristics) rather than data-fitted scientific constants.

free parameters (3)
  • short-term memory buffer size K = 5
    Default K=5 recent frames used for view selection; chosen by authors, not derived.
  • optimal manipulation distance heuristic = 70–80% of max arm reach
    Prompt instructs base placement so object lies within 70–80% of max arm reach; hand-chosen rule.
  • arm-lift offset formula = +0.2 m
    arm_lift = target height + object_height/2 + 0.2 m gripper offset; fixed additive constant.
assumptions (4)
  • domain assumption Object-goal navigation terminates with the target visible inside a 1–2 m near-target radius and recent observations.
    Stated in Problem Statement (Section 3) and Limitations; required for the memory buffer to contain usable views.
  • domain assumption An off-the-shelf MLLM can select views, ground 2D affordances, and reason about metric base poses when given explicit 3D coordinates and robot configuration.
    Core methodological premise of Sections 4.1–4.3; performance varies strongly with backend choice (Table 2).
  • domain assumption Default HomeRobot navigation and grasping policies plus a simple MLLM-predicted arm extension/lift suffice for the remaining stages.
    Experimental Setup (Section 5.1); isolates last-mile contribution but couples reported success to those policies.
  • domain assumption Depth maps and camera extrinsics are accurate enough to lift a 2D pixel to a usable robot-centric 3D affordance.
    Geometry-Aware Base-Pose Reasoning (Section 4); real-world section notes residual odometry/depth/calibration error requiring a second grounding step.
invented entities (1)
  • UniLM-Nav three-stage pipeline independent evidence
    purpose: Decompose last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning under a shared MLLM.
    The named framework is the paper’s central construct; it is an engineering composition, not a new physical entity, and is evaluated empirically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation." pith.science (2026). https://pith.science/paper/N4VP26JF

@misc{pith2026260706537,
  author       = {Pith},
  title        = {Pith review of: UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N4VP26JF}},
  note         = {Machine review of arXiv:2607.06537}
}
read the original abstract

Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

Figures

Figures reproduced from arXiv: 2607.06537 by the authors.

Figure 1
Figure 1. The robot is tasked with placing the bottle on the table in front of the monitor. Object-goal [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. UniLM-Nav decomposes last-mile navigation into view selection, affordance ground [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Framework ablation of UniLM-Nav. (a) Success rates of UniLM-Nav and its variants on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Left: Manual error breakdown on the 20% OVMM subset used in our ablation studies. Right: Representative last-mile navigation failures involving view selection, affordance grounding, and base-pose reasoning. The green annotations indicate preferred views, affordance reg…
Figure 5
Figure 5. Figure 5: View Selection Prompt for Pick and Place [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Affordance Grounding Prompt for Pick and Place [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Base-pose Reasoning Prompt for Pick and Place [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Qualitative example of view selection from the short-term memory [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Qualitative success case on the OVMM benchmark. After object navigation ends, UniLM-Nav predicts the affordance point in the first-person view, shown as the red point, and infers the robot’s base position conditioned on the grounded affordance, task instruction, and ro…
Figure 11
Figure 11. Figure 11: Real-world experiment: Pick up the cup from the table. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Real-world experiment: Place the cake on the plate. [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Real-world experiment: Place the bottle on the table, in front of the monitor. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Real-world experiment: Imagine facing the monitor, place the cake at the bottom-left [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 8 linked inside Pith

  1. [1]

    R. Yang, Y . Kim, R. Hendrix, A. Kembhavi, X. Wang, and K. Ehsani. Harmonic mobile manipulation. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3658–3665. IEEE, 2024

    2024

  2. [2]

    Z. Wu, Y . Zhou, X. Xu, Z. Wang, and H. Yan. Momanipvla: Transferring vision-language- action models for general mobile manipulation. InThe IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1714–1723, 2025

    2025

  3. [3]

    A. Xiao, N. Janaka, T. Hu, A. Gupta, K. Li, C. Yu, and D. Hsu. Robi butler: Multimodal remote interaction with a household robot assistant.arXiv preprint arXiv:2409.20548, 2024

    arXiv 2024

  4. [4]

    Abbatematteo, E

    B. Abbatematteo, E. Rosen, S. Thompson, T. Akbulut, S. Rammohan, and G. Konidaris. Com- posable interaction primitives: A structured policy class for efficiently learning sustained- contact manipulation skills. In2024 IEEE international conference on robotics and automation (ICRA), pages 7522–7529. IEEE, 2024

    2024

  5. [5]

    Y . Peng, Z. Wang, Y . Zhang, S. Zhang, N. Cai, F. Wu, and M. Chen. Revolutionizing battery disassembly: The design and implementation of a battery disassembly autonomous mobile manipulator robot (beam-1). In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6367–6374. IEEE, 2024

    2024

  6. [6]

    ˇStibinger, G

    P. ˇStibinger, G. Broughton, F. Majer, Z. Rozsyp´alek, A. Wang, K. Jindal, A. Zhou, D. Thakur, G. Loianno, T. Krajn´ık, et al. Mobile manipulator for autonomous localization, grasping and precise placement of construction material in a semi-structured environment.IEEE Robotics and Automation Letters (RA-L), 6(2):2595–2602, 2021

    2021

  7. [7]

    Chang, T

    M. Chang, T. Gervet, M. Khanna, S. Yenamandra, D. Shah, S. Y . Min, K. Shah, C. Paxton, S. Gupta, D. Batra, et al. Goat: Go to any thing. InRobotics: Science and Systems (RSS),

  8. [8]

    doi:10.15607/RSS.2024.XX.073

    doi:10.15607/rss.2024.xx.073 2024

Show all 71 references
  1. [9]

    Zhang, X

    P. Zhang, X. Gao, Y . Wu, K. Liu, D. Wang, Z. Wang, B. Zhao, Y . Ding, and X. Li. Moma- kitchen: A 100k+ benchmark for affordance-grounded last-mile navigation in mobile manipu- lation. InInternational Conference on Computer Vision (ICCV), pages 6315–6326, 2025

    2025

  2. [10]

    R.-Z. Qiu, Y . Song, X. Peng, S. A. Suryadevara, G. Yang, M. Liu, M. Ji, C. Jia, R. Yang, X. Zou, et al. Wildlma: Long horizon loco-manipulation in the wild. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 10011–10019. IEEE, 2025

    2025

  3. [11]

    Huang, D

    X. Huang, D. Batra, A. Rai, and A. Szot. Skill transformer: A monolithic policy for mobile manipulation. InInternational Conference on Computer Vision (ICCV), pages 10852–10862, 2023

    2023

  4. [12]

    Yokoyama, A

    N. Yokoyama, A. Clegg, J. Truong, E. Undersander, T.-Y . Yang, S. Arnaud, S. Ha, D. Batra, and A. Rai. Asc: Adaptive skill coordination for robotic mobile manipulation.IEEE Robotics and Automation Letters (RA-L), 9(1):779–786, 2023

    2023

  5. [13]

    J. Yang, I. Huang, B. Vu, M. Bajracharya, R. Antonova, and J. Bohg. Mobi-π: Mobilizing your robot learning policy. InConference on Robot Learning (CoRL), volume 305 ofProceedings of Machine Learning Research, pages 3516–3536. PMLR, 2025. 9

    2025

  6. [14]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InInternational Conference on Computer Vision (ICCV), pages 4015–4026, 2023

    2023

  7. [15]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. DINOv2: Learning robust visual features without supervi- sion.Transactions on Machine Learning Research, 2024

    2024

  8. [16]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational Conference on Machine Learning (ICML), volume 139 ofProceedings of Mach...

    2021

  9. [17]

    Hurst, A

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  10. [18]

    Openai o3 and o4-mini system card.https://openai.com/index/ o3-o4-mini-system-card/, 2025

    OpenAI. Openai o3 and o4-mini system card.https://openai.com/index/ o3-o4-mini-system-card/, 2025

    2025

  11. [19]

    Z. Wu, A. Ma, X. Xu, H. Yin, Y . Liang, Z. Wang, J. Lu, and H. Yan. Moto: A zero-shot plug-in interaction-aware navigation for general mobile manipulation. InConference on Robot Learning (CoRL), volume 305 ofProceedings of Machine Learning Research, pages 2933–

  12. [20]

    Lin, J.-F

    T.-J. Lin, J.-F. Yeh, H.-T. Su, C.-Y . Lin, Y .-T. Chen, and W. H. Hsu. Affordance-guided coarse- to-fine exploration for base placement in open-vocabulary mobile manipulation. InAAAI Con- ference on Artificial Intelligence (AAAI), pages 18443–18451, 2026

    2026

  13. [21]

    Z. Cai, Y . Wang, Q. Sun, R. Wang, C. Gu, W. Yin, Z. Lin, Z. Yang, C. Wei, O. Qian, et al. Holis- tic evaluation of multimodal llms on spatial intelligence.arXiv preprint arXiv:2508.13142, 2025

    2025

  14. [22]

    S. Bai, Y . Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    2025 arXiv

  15. [23]

    Gemini 3 flash: frontier intelligence built for speed.https://blog.google/ products-and-platforms/products/gemini/gemini-3-flash/, 2025

    Google. Gemini 3 flash: frontier intelligence built for speed.https://blog.google/ products-and-platforms/products/gemini/gemini-3-flash/, 2025

    2025

  16. [24]

    Yenamandra, A

    S. Yenamandra, A. Ramachandran, K. Yadav, A. S. Wang, M. Khanna, T. Gervet, T.-Y . Yang, V . Jain, A. Clegg, J. M. Turner, et al. Homerobot: Open-vocabulary mobile manipulation. InConference on Robot Learning (CoRL), volume 229 ofProceedings of Machine Learning Research, pages...

    1975

  17. [25]

    H. Tan, E. Zhou, Z. Li, Y . Xu, Y . Ji, X. Chen, C. Chi, P. Wang, H. Jia, Y . Ao, et al. Robobrain 2.5: Depth in sight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    2026

  18. [26]

    Introducing gpt-5.4.https://openai.com/index/introducing-gpt-5-4/, 2026

    OpenAI. Introducing gpt-5.4.https://openai.com/index/introducing-gpt-5-4/, 2026

    2026

  19. [27]

    Unitree b2.https://unitree.com/b2, 2026

    Unitree Robotics. Unitree b2.https://unitree.com/b2, 2026. Product page, accessed 2026-06-02

    2026

  20. [28]

    Unitree z1.https://unitree.com/z1, 2026

    Unitree Robotics. Unitree z1.https://unitree.com/z1, 2026. Product page, accessed 2026-06-02

    2026

  21. [29]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), 2023. doi:10.15607/RSS.2023.XIX.025. 10

    2023 doi

  22. [30]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile ALOHA: Learning bimanual mobile manipulation using low-cost whole-body teleoperation. InConference on Robot Learning (CoRL), volume 270 ofProceedings of Machine Learning Research, pages 4066–4083. PMLR, 2025

    2025

  23. [31]

    S. Yan, Z. Zhang, M. Han, Z. Wang, Q. Xie, Z. Li, Z. Li, H. Liu, X. Wang, and S.-C. Zhu. M2 diffuser: Diffusion-based trajectory optimization for mobile manipulation in 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–17, 2025. doi: 10.1109/TP...

    2025 doi

  24. [32]

    P. Liu, Y . Orru, J. Vakil, C. Paxton, N. M. M. Shafiullah, and L. Pinto. Demonstrating OK- Robot: What really matters in integrating open-knowledge models for robotics. InRobotics: Science and Systems (RSS), 2024. doi:10.15607/RSS.2024.XX.091

    2024 doi

  25. [33]

    Rosen, S

    E. Rosen, S. James, S. Orozco, V . Gupta, M. Merlin, S. Tellex, and G. Konidaris. Synthesizing navigation abstractions for planning with portable manipulation skills. InConference on Robot Learning (CoRL), volume 229 ofProceedings of Machine Learning Research, pages 2278–

  26. [34]

    F. Wang, S. Lyu, P. Zhou, A. Duan, G. Guo, and D. Navarro-Alarcon. Instruction-augmented long-horizon planning: Embedding grounding mechanisms in embodied mobile manipulation. InAAAI Conference on Artificial Intelligence (AAAI), pages 14690–14698, 2025

    2025

  27. [35]

    Quartey, E

    B. Quartey, E. Rosen, S. Tellex, and G. Konidaris. Verifiably following complex robot in- structions with foundation models. InInternational Conference on Robotics and Automation (ICRA), pages 1–8. IEEE, 2025

    2025

  28. [36]

    T.-H. Lee, F. Mahmudova, and K. Desingh. Learning category-level last-meter navigation from rgb demonstrations of a single-instance.arXiv preprint arXiv:2512.11173, 2025

    2025 arXiv

  29. [37]

    L. Qin, M. Wang, P. Li, W. Zhou, and H. Li. Active perception meets rule-guided rl: A two-phase approach for precise object navigation in complex environments. InInternational Conference on Computer Vision (ICCV), pages 7603–7612, 2025

    2025

  30. [38]

    K. Chai, H. Lee, and J. J. Lim. N2m: Bridging navigation and manipulation by learning pose preference from rollout.arXiv preprint arXiv:2509.18671, 2025

    2025 arXiv

  31. [39]

    Nasiriany, F

    S. Nasiriany, F. Xia, W. Yu, T. Xiao, J. Liang, I. Dasgupta, A. Xie, D. Driess, A. Wahid, Z. Xu, et al. Pivot: Iterative visual prompting elicits actionable knowledge for vlms. InInternational Conference on Machine Learning (ICML), volume 235 ofProceedings of Machine Learning ...

    2024

  32. [40]

    K. Fang, F. Liu, P. Abbeel, and S. Levine. MOKA: Open-world robotic manipulation through mark-based visual prompting. InRobotics: Science and Systems (RSS), 2024. doi:10.15607/ RSS.2024.XX.062

    2024

  33. [41]

    Y . Tang, S. Zhang, X. Hao, P. Wang, J. Wu, Z. Wang, and S. Zhang. Affordgrasp: In-context affordance reasoning for open-vocabulary task-oriented grasping in clutter. InInternational Conference on Intelligent Robots and Systems (IROS), pages 9433–9439. IEEE, 2025

    2025

  34. [42]

    J. Yang, S. Yang, A. W. Gupta, R. Han, L. Fei-Fei, and S. Xie. Thinking in space: How multi- modal large language models see, remember, and recall spaces. InThe IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10632–10643, 2025

    2025

  35. [43]

    B. Chen, Z. Xu, S. Kirmani, B. Ichter, D. Sadigh, L. Guibas, and F. Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. InThe IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14455–14465, 2024

    2024

  36. [44]

    M. Jia, Z. Qi, S. Zhang, W. Zhang, X. Yu, J. He, H. Wang, and L. Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. InInternational Con- ference on Learning Representations (ICLR), 2026. 11

    2026

  37. [45]

    Zhang, Z

    P. Zhang, Z. Huang, Y . Wang, J. Zhang, L. Xue, Z. Wang, Q. Wang, K. Chandrasegaran, R. Zhang, Y . Choi, et al. Theory of space: Can foundation models construct spatial beliefs through active exploration? InInternational Conference on Learning Representations (ICLR), 2026

    2026

  38. [46]

    D. Wu, F. Liu, Y .-H. Hung, and Y . Duan. Spatial-mllm: Boosting mllm capabilities in visual- based spatial intelligence.Advances in Neural Information Processing Systems (NeurIPS), 38: 13569–13597, 2025

    2025

  39. [47]

    R. Yang, H. Chen, J. Zhang, M. Zhao, C. Qian, K. Wang, Q. Wang, T. V . Koripella, M. Mova- hedi, M. Li, et al. Embodiedbench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. InInternational Conference on Machine Learning (ICML), ...

    2025

  40. [48]

    E. Zhou, J. An, C. Chi, Y . Han, S. Rong, C. Zhang, P. Wang, Z. Wang, T. Huang, L. Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. Advances in Neural Information Processing Systems (NeurIPS), 38:28404–28481, 2025

    2025

  41. [49]

    Melnik, M

    A. Melnik, M. B ¨uttner, L. Harz, L. Brown, G. C. Nandi, A. PS, G. K. Yadav, R. Kala, and R. Haschke. Uniteam: Open vocabulary mobile manipulation challenge.arXiv preprint arXiv:2312.08611, 2023

    2023 arXiv

  42. [50]

    Introducing gpt-4.1 in the api.https://openai.com/index/gpt-4-1/, 2025

    OpenAI. Introducing gpt-4.1 in the api.https://openai.com/index/gpt-4-1/, 2025

    2025

  43. [51]

    Introducing gpt-5.2.https://openai.com/index/introducing-gpt-5-2/, 2025

    OpenAI. Introducing gpt-5.2.https://openai.com/index/introducing-gpt-5-2/, 2025

    2025

  44. [52]

    Qwen3.5: Towards native multimodal agents.https://qwen.ai/blog?id=qwen3

    Qwen. Qwen3.5: Towards native multimodal agents.https://qwen.ai/blog?id=qwen3. 5, 2025

    2025

  45. [53]

    Qwen3.6-27b: Flagship-level coding in a 27b dense model.https://qwen.ai/blog? id=qwen3.6-27b, 2025

    Qwen. Qwen3.6-27b: Flagship-level coding in a 27b dense model.https://qwen.ai/blog? id=qwen3.6-27b, 2025

    2025

  46. [54]

    W. Wang, Z. Gao, L. Gu, H. Pu, L. Cui, X. Wei, Z. Liu, L. Jing, S. Ye, J. Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025

    2025 arXiv

  47. [55]

    B. R. Team, M. Cao, H. Tan, Y . Ji, X. Chen, M. Lin, Z. Li, Z. Cao, P. Wang, E. Zhou, et al. Robobrain 2.0 technical report.arXiv preprint arXiv:2507.02029, 2025

    2025 arXiv

  48. [56]

    Gemini 335: Stereo vision camera for robotics.https://orbbec.com/products/ stereo-vision-camera/gemini-335/, 2026

    Orbbec. Gemini 335: Stereo vision camera for robotics.https://orbbec.com/products/ stereo-vision-camera/gemini-335/, 2026. Product page, accessed 2026-06-02

    2026

  49. [57]

    T. Shan, B. Englot, D. Meyers, W. Wang, C. Ratti, and D. Rus. Lio-sam: Tightly-coupled lidar inertial odometry via smoothing and mapping. InInternational Conference on Intelligent Robots and Systems (IROS), pages 5135–5142. IEEE, 2020

    2020

  50. [58]

    Macenski, F

    S. Macenski, F. Martin, R. White, and J. Gin´es Clavero. The marathon 2: A navigation system. InInternational Conference on Intelligent Robots and Systems (IROS), 2020. 12 The Appendix is organized into the following sections:Method Details(Section A),More Exper- iments & Resu...

    2020

  51. [59]

    Visibility: The{object name}is clearly visible and identifiable (not blurry, not heavily occluded, and enough of the object is visible to pick a precise grasp/target point)

  52. [60]

    Avoid images where obstacles/clutter block the approach path to the{object name}or where the{object name}appears unlikely to be reachable

    Reachability/Navigability: The robot can realistically move close enough to the{object name}to pick it up. Avoid images where obstacles/clutter block the approach path to the{object name}or where the{object name}appears unlikely to be reachable. Return a JSON object with key ‘...

  53. [61]

    Visibility: The surface of the{place recep name}is clearly visible (not blurry, not heavily occluded, enough surface area visible to select a stable point for placing the object)

  54. [62]

    Avoid images where obstacles/clutter block access to the front/edge of the{place recep name}or where approaching the {place recep name}would likely be impossible

    Reachability/Navigability: The robot can realistically move close enough to the{place recep name}to perform placement. Avoid images where obstacles/clutter block access to the front/edge of the{place recep name}or where approaching the {place recep name}would likely be impossi...

  55. [63]

    Visible-on-object point: The pixel must lie ON the{object name}(not background or other items), and should be on a clearly visible, unoccluded part of the object (avoid blurry/ambiguous regions and heavy occlusions)

  56. [64]

    Prefer points on the object that are not blocked by obstacles between the robot and other object

    Reachable for pickup: Choose a point such that the robot can realistically approach near and pick up the object. Prefer points on the object that are not blocked by obstacles between the robot and other object. Avoid cases where the object (or the selected point) is behind lar...

  57. [65]

    The point should NOT overlap/collide with any other unrelated objects

    Safe & stable surface: The point must lie on a flat, supported, and stable region of the{place recep name}. The point should NOT overlap/collide with any other unrelated objects

  58. [66]

    Do NOT select points on or near the {place recep name}’s edge

    Avoid the edge (anti-drop): Prefer an interior placement region rather than the boundary. Do NOT select points on or near the {place recep name}’s edge

  59. [67]

    Reachable placement (base + arm constraints): The point should be reachable for the robot to approach and place the object. Prefer locations with clear free space around the receptacle and no obvious obstacles blocking the robot’s approach and placing path (e.g., clutter, furn...

  60. [68]

    Pick up the cup from the table. (Fig. 11)

  61. [69]

    Place the cake on the plate. (Fig. 12)

  62. [70]

    Place the bottle on the table, in front of the monitor. (Fig. 13)

  63. [71]

    in front of the monitor

    Imagine facing the monitor, place the cake at the bottom-left corner of the table. (Fig. 14) The first two tasks mainly evaluate whether UniLM-Nav can effectively bridge object navigation and manipulation in real scenes, where the robot needs to approach the task-relevant obje...

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.