SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation

Guang Li; Junjie An; Shuyang Yu; Xiao Liu; Xiaoqing Guan; Yiqun Zhou; Yi Wu; You Wang; Yuechen Wu

REVIEW 2 major objections 4 minor 37 references

SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation

T0 review · 2 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A navigation system that feeds continuous 3D scene embeddings directly into a vision–language model’s hidden space outperforms text-based scene descriptions, and a controlled ablation shows the text interface is a measurable bottleneck.

desk verdict Useful paper with a genuinely new result, but the central representation-gap claim is undercut by a parameter-count confound in the key ablation. read the letter →

arxiv 2607.14586 v1 pith:VEFHIN23 submitted 2026-07-16 cs.RO

Yi Wu , Junjie An , Xiao Liu , Yiqun Zhou , Yuechen Wu , Xiaoqing Guan , Shuyang Yu , You Wang

show 1 more author

Guang Li

This is my paper · ORCID

classification cs.RO

keywords embodiednavigationvision-languagemodelssofttokeninjection3Dscenerepresentationobject-goalfrontier-basedexplorationzero-shottransfergap

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the way 3D scene information is transmitted to a vision–language model (VLM) is a real performance bottleneck in embodied navigation. It claims that serializing the scene as text—coordinates, object lists, or rich text—loses geometric and semantic structure that matters for finding goals. To test this, it introduces SoftNav, which injects one continuous embedding per detected object or frontier directly into the VLM’s hidden space as soft tokens, using a small MLP projector. With the 3D encoder and VLM frozen and only about 1,200 training samples, SoftNav beats all prior methods on the HM3D-OVON benchmark and transfers zero-shot to two other benchmarks and a real robot.

What carries the argument

The key machinery is soft-token injection through a lightweight MLP projector. PQ3D, a frozen 3D scene encoder, produces a 768-dimensional query embedding for each detected object and frontier, fused from voxel geometry, multi-view appearance, and text. A two-layer MLP with GELU and LayerNorm maps each embedding to the VLM’s 2048-dimensional hidden space, where it replaces special placeholder tokens in the input sequence. These soft tokens carry continuous geometric and semantic content that text serialization discards. The VLM (Qwen2.5-VL-3B) attends jointly over visual memory tokens, soft tokens, and a textual navigation prompt, and outputs a coordinate that is snapped to the nearest front

What would settle it

Run the same controlled ablation (same encoder, same VLM, same training set) on a benchmark where the goal object is always correctly detected and segmented, but vary only the interface between text coordinates and soft tokens; if a text format—even an oracle format with perfect coordinates and object names—matches or beats soft-token SR and SPL, the claimed representation gap would be disproven. Equivalently, if replacing PQ3D with a different 3D encoder eliminates the soft-token advantage, the gap would be encoder-specific, not interface-general.

Watch

Extended reading notes

Core claim

The central discovery is a representation gap: when the same 3D encoder and the same VLM are held constant, replacing text-coordinate prompts with direct soft-token injection improves success rate by +7.5 percentage points and path efficiency by +9.2 percentage points on the val_seen split (p=0.028 and p<0.0001). Adding LoRA adaptation to the text baseline does not close the gap, and enriching the text with distance, direction, and confidence descriptors does not either. This indicates that the loss happens in the text serialization itself, not in model capacity. SoftNav, which projects PQ3D’s 768-dimensional object and frontier embeddings into the VLM’s 2048-dimensional hidden space via a t

Load-bearing premise

The entire advantage of SoftNav rests on the assumption that PQ3D’s incremental instance segmentation and frontier detection produce a complete and accurate set of entity embeddings for the goal object and relevant frontiers; if the 3D encoder misses or mis-segments the target, neither soft tokens nor text can compensate, as the paper itself shows in its shared-failure example.

Editorial extensions

If this is right

If the representation gap is real, then any VLM-based navigation system that relies on text-serialized 3D information is leaving measurable success and efficiency on the table; switching to soft-token injection could raise SR and SPL without changing the perception or reasoning models.
The result implies that a lightweight, trainable projector plus LoRA—about 17 million parameters and roughly 1,200 samples—is enough to align a specialized 3D encoder with a frozen VLM, making the approach practical for low-data regimes.
Zero-shot transfer across three benchmarks and a real robot suggests that continuous 3D entity embeddings encode task-general navigation semantics, so a single policy can handle open-vocabulary, multi-modal, and scene-graph-grounded goals without retraining.
The failure case where both soft tokens and text fail because the 3D encoder cannot detect the occluded goal shows that perception quality remains a hard ceiling; the interface fix only helps when the encoder actually sees the target.
The token-selection ablation (selecting fewer than all 128 tokens never helps) suggests that, at this training scale, every detected object and frontier carries useful signal, so aggressive filtering can be counterproductive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The representation-gap argument likely generalizes beyond navigation: any task that pipes structured 3D (or other continuous) information into a language model through text—scene graphs, point clouds, map layouts—may benefit from direct embedding injection, and the paper’s controlled-ablation methodology is a template for testing that.
If the bottleneck is truly the text interface, then richer text formats are unlikely to ever fully close the gap; the paper’s Rich-Text variant showing no consistent improvement supports this, suggesting that future work should focus on learned continuous interfaces rather than better serialization.
A testable extension would be to swap PQ3D for a different 3D encoder (e.g., one producing fixed-dimensional embeddings) and check whether the soft-token advantage persists; the paper leaves this as future work, and it would confirm that the finding is about the interface, not PQ3D specifically.
The modest SPL drop on unseen splits hints that spatial cue exploitation depends on LoRA adaptation quality; scaling the training set beyond 1,187 samples might close that gap, which could be probed by doubling or tripling the distilled data and remeasuring val_unseen SPL.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth reading. It does something concrete that I haven't seen before: it holds the 3D encoder and VLM fixed, varies only whether the 3D information is transmitted as text coordinates or as injected soft tokens, and measures the gap. That is a clean way to isolate the interface, and the result is a large, statistically significant improvement in SPL on val_seen. The per-episode analysis and bootstrap CIs are careful. The low-data recipe (~1,200 samples, ~17M trainable params) and zero-shot transfer to other benchmarks and a real robot are impressive, even if the zero-shot label is qualified by benchmark-specific PQ3D weights. I also credit the authors for being honest about the shared failure mode when the 3D encoder misses the target: Figure 5c shows that neither transfer modality can compensate, which is a useful boundary on the claim.

The main soft spot is exactly what the stress-test note says. Table V compares Text-Hint + LoRA (only LoRA, ~13M params) to SoftNav (MLP projector + LoRA, ~17M params). That is not a capacity-matched comparison. The paper claims "the majority of the improvement stems from preserving continuous representations rather than from additional capacity," but there is no ablation that holds capacity fixed. The +5.9 SR / +7.0 SPL gain over the Text-Hint + LoRA baseline could be partly or wholly due to the extra 4M parameters. This does not destroy the paper's value, but it does mean the central causal claim "rather than model capacity alone" is not supported as written. The authors need to run a matched baseline — for example, a text baseline with an equally sized trainable projector or MLP, or a soft-token variant without LoRA — before they can make that claim.

Other issues are minor but worth noting: significance is reported cleanly only for val_seen; the real-world evaluation is small and human-judged; the concurrent Dynam3D is described but not quantitatively compared; and no code or data are released. The Rich-Text ablation is a good attempt to test information content versus modality, but it is also underpowered and inconsistent across splits.

Overall: the paper deserves a serious referee. It identifies a plausible and previously unquantified bottleneck, and the SoftNav results are strong enough to be interesting even if the representation-gap explanation needs qualification. I would send it to peer review but ask for a capacity-matched ablation and clearer reporting of statistical significance across splits.

For a reading group, I'd bring it in as an example of a well-structured ablation that just misses the last control. I'd cite it if I were working on VLM-based navigation, mainly for the soft-token injection recipe and the per-episode analysis.

Referee Report

2 major / 4 minor

Summary. SoftNav proposes to replace text-serialized 3D scene information with direct injection of entity-level continuous embeddings into a frozen VLM. Using PQ3D's object/frontier query embeddings, a lightweight MLP projector maps 768-d features into the VLM's hidden space as soft tokens, while LoRA adapters and the projector are trained on ~1,187 samples distilled from MTU3D on HM3D-OVON. The method reports 74.2/68.3/66.7% SR on OVON val splits and zero-shot transfer to GOAT-Bench and SG3D, plus a small real-world deployment. The central claim is that a 'representation gap' in text-based 3D-to-VLM interfaces is the primary bottleneck, supported by the controlled ablation in Table V.

Significance. If the central claim holds, the paper makes a meaningful contribution: it quantifies, for the first time in this setting, the information loss from text serialization of 3D scene representations and demonstrates that a lightweight projector can bridge it. The controlled ablation holding the 3D encoder and VLM fixed, the shared teacher labels, the paired bootstrap inference, and the per-episode analysis are notable methodological strengths. The reported zero-shot generalization with only ~17M trainable parameters is also impressive. However, the causal interpretation — that the gap is due to the interface rather than to model capacity — rests on a comparison that is not parameter-matched, which weakens the main claim as currently supported.

major comments (2)

[§IV-C, Table V] The central claim that the improvement stems from 'preserving continuous representations rather than from additional capacity' is not supported because the soft-token condition is not parameter-matched with the text baseline. Soft tokens add the ~4M MLP projector plus ~13M LoRA, while Text-Hint + LoRA has only ~13M. The +7.5 pp SR / +9.2 pp SPL gain on val_seen could in part be attributable to the extra 4M parameters. A capacity-matched text baseline — e.g., an MLP of similar size applied to text-coordinate embeddings, or a soft-token variant without LoRA / with a matched total parameter budget — is needed to separate interface from capacity. Without it, the conclusion 'rather than model capacity alone' (Section V) is overstated.
[§IV-C, Table V and Table VII] Statistics are reported only for val_seen, yet Table V's caption says 'across all OVON splits' and the abstract/conclusion claim a general representation gap. On val_unseen the SPL delta is −2.6 pp and the SR gain (+3.4 pp) is far from the val_seen gains; on val_seen_synonyms the SR gain is +1.6 pp. Paired bootstrap CIs and p-values should be reported for every split, or the conclusion should be qualified to the split(s) where the effect is actually established. As written, the claim of a modality-driven gap on unseen layouts is not supported by the evidence.

minor comments (4)

[§IV-A.4 / §IV-B.2] The term 'zero-shot' is used for policy transfer to GOAT-Bench and SG3D, but PQ3D uses benchmark-specific stage-2 weights in those evaluations. The policy is zero-shot, but the full perception stack is not; please clarify this distinction in the abstract and main text.
[§IV-C.1, Table V] The Rich-Text baseline is not given LoRA training; it is compared as an untuned prompt against tuned SoftNav. This is acceptable for probing prompt content, but the sentence 'enriching the text channel cannot narrow the gap' should be phrased as 'in our untrained text baselines'.
[Throughout] Typos and spacing: '~1,200samples' should be '~1,200 samples'; 'V oroNav' should be 'VoroNav'; 'LLaV A' should be 'LLaVA'. Please also ensure all table captions state the exact statistical tests used.
[§IV-C.2, Table VI] The token selection ablation is only on val_seen. Since the full model is also best on the other splits in Table V, it would be useful to confirm the same ordering on val_unseen, where the SPL trade-off is most pronounced.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the soft-token vs text comparison is empirical and not forced by construction.

full rationale

The central derivation chain is: PQ3D produces 768-d query embeddings; an MLP projector maps them to 2048-d VLM hidden space; the VLM predicts a frontier coordinate; the coordinate is snapped to the nearest frontier. None of these steps defines the target result in terms of the input or fits a parameter to the outcome being predicted. The representation-gap claim rests on Table V, which varies only the information pathway while holding the 3D encoder, VLM, and training data constant. The soft-token advantage (+7.5 pp SR, +9.2 pp SPL on val_seen) is not guaranteed by construction: Text-Hint could have matched or beaten soft tokens, and the paper reports Rich-Text sometimes nominally outperforming SoftNav on val_unseen. The training labels from MTU3D successful episodes are a teacher-student distillation source, not a circular derivation: the student is then evaluated against the teacher and other methods on held-out episodes and zero-shot benchmarks, so the comparison is not equivalent to the training signal. There is no self-citation chain invoked to forbid alternatives; PQ3D and MTU3D are used as external components, and the paper explicitly notes validating with alternative encoders as future work. The main weakness is that Table V is not parameter-matched: SoftNav adds a ~4M-parameter MLP projector on top of LoRA, so the claim that the improvement is 'rather than from additional capacity' is not fully controlled. That is a correctness/confound concern, not circularity, because the observed gain is still an empirical outcome rather than a quantity forced by the fitting procedure.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The scoring and conclusion do not rest on classical fitted constants, but several hyperparameters (r=2, K=16, projector width 1408) are hand-set. The central causal claim relies on the presupposition that PQ3D embeddings are sufficient scene representations and that the tested text serializations fairly represent the text interface. No new physical entities are introduced; soft tokens are a technical mechanism, not an invented entity.

free parameters (4)

Distance-ratio guardrail r = 2
Set empirically in Eq. (4) to prevent selecting excessively distant frontiers; no sensitivity analysis reported.
Visual memory length K = 16
Rolling buffer length 'set empirically' in Sec III-A; no ablation is provided.
Projector intermediate dimension = 1408
Chosen as the midpoint of linspace(768,2048,3), a 'simple heuristic' in Sec III-C2; no search is reported.
LoRA rank and alpha = r=8, alpha=32
Standard LoRA configuration used without a sweep (Sec IV-A3).

assumptions (4)

domain assumption PQ3D's 768-dimensional query embeddings encode geometric, semantic, and spatial-relational information sufficient for frontier selection.
The entire method feeds these embeddings as soft tokens; if they lack navigation-relevant information, SoftNav cannot recover it (Sec III-B).
domain assumption The text serialization formats tested (coordinates, rich text) are representative of the text interface.
The representation-gap conclusion generalizes from these specific formats; no exhaustive search over serializations is performed (Sec IV-C).
domain assumption MTU3D successful-episode frontier selections provide unbiased, sufficient supervision.
Training labels are the teacher's choices; systematic teacher errors would be inherited by SoftNav (Sec III-E2).
domain assumption Qwen2.5-VL-3B hidden space can accept projected 2048-d soft tokens without catastrophic interference.
No analysis of VLM internal dynamics is given; empirical results support it, but the mechanism is not proven (Sec III-C).

how reviews work

0 comments

Cite this review

Pith. "Pith review of SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation." pith.science (2026). https://pith.science/paper/VEFHIN23

@misc{pith2026260714586,
  author       = {Pith},
  title        = {Pith review of: SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VEFHIN23}},
  note         = {Machine review of arXiv:2607.14586}
}

read the original abstract

In goal-directed embodied navigation, where an agent must locate a specified target in an unseen environment, 3D scene understanding and navigation reasoning must work in concert. Current approaches transmit 3D scene information to vision-language models (VLMs) through text, suggesting a representation gap in our tested configurations; a controlled ablation confirms that direct embedding-level transfer significantly outperforms the evaluated text serialization formats. We introduce SoftNav, which injects entity-level 3D continuous representations -- one token per detected object or frontier -- into a VLM's hidden space as soft tokens through a lightweight projector. With the 3D encoder and VLM frozen, only ~1,200 samples and ~17M trainable parameters are needed. On HM3D-OVON, SoftNav achieves 74.2%/68.3%/66.7% SR across three splits, surpassing all prior methods in both SR and SPL; the same navigation policy transfers zero-shot to GOAT-Bench (67.2% SR), SG3D (47.2% s-SR), and real-world robot deployment without retraining or architectural modification. Injecting 3D scene tokens directly into VLMs bridges the representation gap, enabling transferable navigation with minimal training.

Figures

Figures reproduced from arXiv: 2607.14586 by the authors.

**Figure 2.** Overview of SoftNav. PQ3D extracts query embeddings from streaming RGB-D observations; the MLP projector [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Per-episode SPL on OVON. Red: SoftNav; blue: [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: UMAP visualization of PQ3D query embeddings (50 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Representative trajectories on OVON val_unseen. (a) Both succeed; SoftNav is more efficient. (b) Only SoftNav succeeds. (c) Both fail—the support beam is heavily occluded, limiting 3D encoder cues [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Real-world deployment on a Unitree Go2. (a) Corridor (8/10): multi-exposure overlay and point cloud map. (b) Hall [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 3 linked inside Pith

[1]

HM3D- OVON: A dataset and benchmark for open-vocabulary object goal navigation,

N. Yokoyama, R. Ramrakhya, A. Das, D. Batra, and S. Ha, “HM3D- OVON: A dataset and benchmark for open-vocabulary object goal navigation,” inProc. IROS, 2024

2024
[2]

GOAT-Bench: A benchmark for multi-modal lifelong navigation,

M. Khanna, R. Ramrakhya, G. Chhablani, et al., “GOAT-Bench: A benchmark for multi-modal lifelong navigation,” inProc. CVPR, 2024

2024
[3]

Task-oriented sequential grounding and navigation in 3D scenes,

Z. Zhang, Z. Zhu, J. Li, et al., “Task-oriented sequential grounding and navigation in 3D scenes,”arXiv:2408.04034, 2024

arXiv 2024
[4]

Object goal navigation using goal-oriented semantic exploration,

D. S. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,” inProc. NeurIPS, 2020

2020
[5]

VLFM: Vision-language frontier maps for zero- shot semantic navigation,

N. Yokoyama et al., “VLFM: Vision-language frontier maps for zero- shot semantic navigation,” inProc. ICRA, 2024

2024
[6]

Move to understand a 3D scene: Bridging visual ground- ing and exploration for efficient and versatile embodied navigation,

Z. Zhu et al., “Move to understand a 3D scene: Bridging visual ground- ing and exploration for efficient and versatile embodied navigation,” inProc. ICCV, 2025

2025
[7]

Unifying 3D vision-language understanding via prompt- able queries,

Z. Zhu et al., “Unifying 3D vision-language understanding via prompt- able queries,” inProc. ECCV, 2024

2024
[8]

NavGPT: Explicit reasoning in vision- and-language navigation with large language models,

G. Zhou, Y . Hong, and Q. Wu, “NavGPT: Explicit reasoning in vision- and-language navigation with large language models,” inProc. AAAI, 2024

2024

Show all 37 references

[9]

NaVid: Video-based VLM plans the next step for vision-and-language navigation,

J. Zhang et al., “NaVid: Video-based VLM plans the next step for vision-and-language navigation,” inProc. RSS, 2024

2024
[10]

L3MVN: Leveraging large language models for visual target navigation,

B. Yu, H. Kasaei, and M. Cao, “L3MVN: Leveraging large language models for visual target navigation,” inProc. IROS, 2023

2023
[11]

SayNav: Grounding large language models for dynamic planning to navigation in new environments,

A. Rajvanshi et al., “SayNav: Grounding large language models for dynamic planning to navigation in new environments,” inProc. ICAPS, 2024

2024
[12]

SG-Nav: Online 3D scene graph prompting for LLM-based zero-shot object navigation,

H. Yin, X. Xu, Z. Wu, J. Zhou, and J. Lu, “SG-Nav: Online 3D scene graph prompting for LLM-based zero-shot object navigation,” inProc. NeurIPS, 2024

2024
[13]

A frontier-based approach for autonomous exploration,

B. Yamauchi, “A frontier-based approach for autonomous exploration,” inProc. CIRA, 1997, pp. 146–151

1997
[14]

PONI: Potential functions for ObjectGoal navigation with interaction-free learning,

S. K. Ramakrishnan, D. S. Chaplot, Z. Al-Halah, J. Malik, and K. Grauman, “PONI: Potential functions for ObjectGoal navigation with interaction-free learning,” inProc. CVPR, 2022

2022
[15]

PIRLNav: Pre- training with imitation and RL finetuning for ObjectNav,

R. Ramrakhya, D. Batra, E. Wijmans, and A. Das, “PIRLNav: Pre- training with imitation and RL finetuning for ObjectNav,” inProc. CVPR, 2023

2023
[16]

Uni-NaVid: A video-based vision-language-action model for unifying embodied navigation tasks,

J. Zhang et al., “Uni-NaVid: A video-based vision-language-action model for unifying embodied navigation tasks,” inProc. RSS, 2025

2025
[17]

V oroNav: V oronoi-based zero-shot object navigation with large language model,

P. Wu et al., “V oroNav: V oronoi-based zero-shot object navigation with large language model,” inProc. ICML, 2024

2024
[18]

MapGPT: Map-guided prompting with adaptive path planning for vision-and-language navigation,

J. Chen, B. Lin, R. Xu, Z. Chai, X. Liang, and K.-Y . K. Wong, “MapGPT: Map-guided prompting with adaptive path planning for vision-and-language navigation,” inProc. ACL, 2024

2024
[19]

ESC: Exploration with soft commonsense constraints for zero-shot object navigation,

K. Zhou et al., “ESC: Exploration with soft commonsense constraints for zero-shot object navigation,” inProc. ICML, 2023

2023
[20]

Flamingo: A visual language model for few-shot learning,

J.-B. Alayrac et al., “Flamingo: A visual language model for few-shot learning,” inProc. NeurIPS, 2022

2022
[21]

Visual instruction tuning,

H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” in Proc. NeurIPS, 2023

2023
[22]

BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

J. Li et al., “BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” inProc. ICML, 2023

2023
[23]

3D-LLM: Injecting the 3D world into large language models,

Y . Hong et al., “3D-LLM: Injecting the 3D world into large language models,” inProc. NeurIPS, 2023

2023
[24]

An embodied generalist agent in 3D world,

J. Huang et al., “An embodied generalist agent in 3D world,” inProc. ICML, 2024

2024
[25]

LL3DA: Visual interactive instruction tuning for omni- 3D understanding,

S. Chen et al., “LL3DA: Visual interactive instruction tuning for omni- 3D understanding,” inProc. CVPR, 2024

2024
[26]

EmbodiedGPT: Vision-language pre-training via em- bodied chain of thought,

Y . Mu et al., “EmbodiedGPT: Vision-language pre-training via em- bodied chain of thought,” inProc. NeurIPS, 2023

2023
[27]

Dynam3D: Dynamic layered 3D tokens empower VLM for vision-and-language navigation,

Z. Wang, S. Lee, and G. H. Lee, “Dynam3D: Dynamic layered 3D tokens empower VLM for vision-and-language navigation,” inProc. NeurIPS, 2025

2025
[28]

AstraNav-Memory: Contexts compression for long memory,

B. Ren et al., “AstraNav-Memory: Contexts compression for long memory,”arXiv:2512.21627, 2025

2025
[29]

Siméoni et al., “DINOv3,”arXiv:2508.10104, 2025

O. Siméoni et al., “DINOv3,”arXiv:2508.10104, 2025

2025 arXiv
[30]

Qwen2.5-VL technical report,

S. Bai et al., “Qwen2.5-VL technical report,”arXiv:2502.13923, 2025

2025 arXiv
[31]

LoRA: Low-rank adaptation of large language models,

E. J. Hu et al., “LoRA: Low-rank adaptation of large language models,” inProc. ICLR, 2022

2022
[32]

Habitat: A platform for embodied AI research,

M. Savva et al., “Habitat: A platform for embodied AI research,” in Proc. ICCV, 2019

2019
[33]

The design of stretch: A compact, lightweight mobile manipulator for indoor human environments,

C. C. Kemp, A. Edsinger, H. M. Clever, and B. Matulevich, “The design of stretch: A compact, lightweight mobile manipulator for indoor human environments,” inProc. ICRA, 2022, pp. 3150–3157

2022
[34]

TANGO: Training- free embodied AI agents for open-world tasks,

F. Ziliotto, T. Campari, L. Serafini, and L. Ballan, “TANGO: Training- free embodied AI agents for open-world tasks,” inProc. CVPR, 2025

2025
[35]

MSGNav: Unleashing the power of multi-modal 3D scene graph for zero-shot embodied navigation,

X. Huang et al., “MSGNav: Unleashing the power of multi-modal 3D scene graph for zero-shot embodied navigation,” inProc. CVPR, 2026

2026
[36]

Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation,

S. Y . Gadre, M. Wortsman, G. Ilharco, L. Schmidt, and S. Song, “Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation,” inProc. CVPR, 2023

2023
[37]

Embodied VideoAgent: Persistent memory from ego- centric videos and embodied sensors enables dynamic scene under- standing,

Y . Fan et al., “Embodied VideoAgent: Persistent memory from ego- centric videos and embodied sensors enables dynamic scene under- standing,” inProc. ICCV, 2025

2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

HM3D- OVON: A dataset and benchmark for open-vocabulary object goal navigation,

N. Yokoyama, R. Ramrakhya, A. Das, D. Batra, and S. Ha, “HM3D- OVON: A dataset and benchmark for open-vocabulary object goal navigation,” inProc. IROS, 2024

2024

[2] [2]

GOAT-Bench: A benchmark for multi-modal lifelong navigation,

M. Khanna, R. Ramrakhya, G. Chhablani, et al., “GOAT-Bench: A benchmark for multi-modal lifelong navigation,” inProc. CVPR, 2024

2024

[3] [3]

Task-oriented sequential grounding and navigation in 3D scenes,

Z. Zhang, Z. Zhu, J. Li, et al., “Task-oriented sequential grounding and navigation in 3D scenes,”arXiv:2408.04034, 2024

arXiv 2024

[4] [4]

Object goal navigation using goal-oriented semantic exploration,

D. S. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,” inProc. NeurIPS, 2020

2020

[5] [5]

VLFM: Vision-language frontier maps for zero- shot semantic navigation,

N. Yokoyama et al., “VLFM: Vision-language frontier maps for zero- shot semantic navigation,” inProc. ICRA, 2024

2024

[6] [6]

Move to understand a 3D scene: Bridging visual ground- ing and exploration for efficient and versatile embodied navigation,

Z. Zhu et al., “Move to understand a 3D scene: Bridging visual ground- ing and exploration for efficient and versatile embodied navigation,” inProc. ICCV, 2025

2025

[7] [7]

Unifying 3D vision-language understanding via prompt- able queries,

Z. Zhu et al., “Unifying 3D vision-language understanding via prompt- able queries,” inProc. ECCV, 2024

2024

[8] [8]

NavGPT: Explicit reasoning in vision- and-language navigation with large language models,

G. Zhou, Y . Hong, and Q. Wu, “NavGPT: Explicit reasoning in vision- and-language navigation with large language models,” inProc. AAAI, 2024

2024