Pith. sign in

REVIEW 3 major objections 5 minor 99 references

Multimodal Spatial Language Maps for Robot Navigation and Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fusing audio, visual, and language features into a single 3D map lets robots navigate to goals described by words, images, or sounds, and disambiguate ambiguous goals by up to 50%.

desk verdict A useful, candid journal extension of the authors' own VLMaps/AVLMaps; the audio-disambiguation claims rest on synthetic audio and an acknowledged pose-as-source approximation. read the letter →

arxiv 2506.06862 v1 pith:WXH75JIN submitted 2025-06-07 cs.RO cs.AIcs.CVcs.LGcs.SDeess.AS

Chenguang Huang , Oier Mees , Andy Zeng , Wolfram Burgard This is my paper
classification cs.ROcs.AIcs.CVcs.LGcs.SDeess.AS
keywords robotnavigationmultimodalspatiallanguagemapsaudio-visual-languagezero-shotgoalopen-vocabularymappingcross-modalreasoning3Dvoxellargemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a robot can build a single 3D map of a space that stores not only geometry but also the meanings of what it saw and heard, so that later it can find places described in plain language, by an image, or by a sound. The proposed visual-language maps and audio-visual-language maps are built from an ordinary RGB-D exploration video with an audio track, using pretrained models to embed pixels and sound segments, and can be queried in an open vocabulary. The authors argue this enables zero-shot spatial goal navigation, supports sharing one map across robots of different shapes by generating custom obstacle maps, and improves disambiguation of ambiguous goals by up to 50% in top-1 recall. Experiments in simulation and on real mobile and tabletop robots support these claims.

What carries the argument

The load-bearing object is a spatially indexed feature map: a top-down grid or 3D voxel grid in which each cell stores averaged dense embeddings from a vision-language model, while audio segments are embedded by an audio-language model and attached to the pose where they were heard, and image-level features are attached to poses for area queries. Each localization module converts its match scores into a heatmap $H \in [0,1]^{\bar H \times \bar W \times \bar Z}$ whose values decay linearly with distance from the matched location, with a high decay rate for the primary target and low rates for auxiliary constraints. Cross-modal reasoning is the element-wise product $H_{\text{target}} = H_1 \odot \cdots \odot H_L$, and a large language model turns natural-language commands into executable code that calls these heatmap primitives and does simple spatial arithmetic. This machinery is what lets a query like "the sound of a crying baby near the sofa" become a single fused goal location.

What would settle it

In a scene with two identical objects of the same category and one distinctive sound played near only one of them, run the cross-modal query "object near the sound" many times; if the robot selects the wrong object at rates no better than chance, the paper's claim that audio disambiguates visual goals is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that audio, visual, and language cues can be fused into a unified 3D voxel map, rather than kept in separate representations, by attaching embedding vectors from pretrained multimodal models to the geometry reconstructed during exploration. At query time, each modality proposes a heatmap of likely locations, and the maps are combined by element-wise multiplication, which lets one modality (such as the sound of glass breaking) narrow down the candidates of another (such as the correct backpack). The paper shows this works zero-shot, without any task-specific fine-tuning, and that the resulting disambiguation improves top-1 recall by up to 50% in ambiguous scenarios compared with unimodal baselines. The same map also supports open-vocabulary spatial goals and embodiment-specific obstacle maps.

Load-bearing premise

The load-bearing assumption is that sounds heard during map building can be separated from silence and noise, embedded by a pretrained audio-language model, matched to the query text, and treated as fixed landmarks at the robot's pose at the moment they were heard.

Editorial extensions

If this is right

  • A robot that built one map of a home can later be told "go in between the sofa and the TV" and navigate there directly, without retraining.
  • The same map can be reused by a ground robot and a drone: each defines its own obstacle categories and gets a tailored path-planning grid from the same representation.
  • Multimodal goals such as "the table near the sound of glass breaking" become addressable, and ambiguous duplicates are disambiguated using the extra modality.
  • Swapping in a stronger audio-language model improves downstream navigation, so the map representation improves as the underlying pretrained models improve.
  • Tabletop manipulators can use the same map to approach visual, audio, and object goals within tight spatial tolerances.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the map's audio entries record where the robot heard a sound, not where the sound originated, so pairing this representation with live binaural sound-source localization would let a robot both recall past sounds and chase transient ones.
  • The heatmap multiplication assumes the modalities are conditionally independent; when a sound and an object are physically correlated, the fused peak may shift, suggesting a learned or probabilistic fusion rule as a testable alternative.
  • The paper's modular recipe (embed, voxelize, localize, fuse) is a general template, so modalities such as temperature or odor could be added by supplying a pretrained encoder and a silence-like segmentation scheme.
  • Given the paper's finding that newer vision-language models did not clearly help while newer audio-language models did, a plausible bottleneck is the quality of dense pixel-level visual-language features, which could be probed directly by measuring map localization accuracy against per-pixel feature fidelity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes multimodal spatial language maps, a 3D voxel representation that fuses visual-language features (LSeg/CLIP), CLIP-based area features, audio-language features (AudioCLIP/wav2clip/CLAP), and visual localization features (NetVLAD/SuperPoint/SuperGlue) with a geometric reconstruction. Two instances are presented: VLMaps, which supports open-vocabulary spatial goal navigation from natural language and embodiment-specific obstacle-map generation, and AVLMaps, which extends the representation to audio and image queries and fuses modality heatmaps through element-wise multiplication for cross-modal disambiguation. The experiments cover zero-shot spatial goal navigation in Habitat/Matterport3D, cross-embodiment navigation in AI2THOR, simulated multimodal goal navigation, real-world mobile navigation and tabletop manipulation, and a scaling study over newer audio and vision-language foundation models. The central claims are zero-shot multimodal goal navigation and a top-1 recall improvement of up to 50% in ambiguous goal scenarios.

Significance. If the claims are substantiated, the work is a useful systems contribution: it shows how several pretrained multimodal foundation models can be composed into a persistent 3D map that supports language, image, and audio queries, and it demonstrates positive scaling when AudioCLIP is replaced by CLAP. The cross-embodiment obstacle-map idea is practical, and the comparisons against LM-Nav, CoW, ConceptFusion, and unimodal baselines are appropriate where they are provided. The paper also contains explicit discussion of limitations and releases code and videos, which aids reproducibility. However, the audio component of the central claim is not yet validated under real acoustic conditions: every audio experiment injects or mixes sound into recordings rather than using physically emitted sound, and the map stores the robot pose at the time a sound was heard rather than the sound-source location. This gap, together with missing baselines and variance information in several core experiments, makes the headline claims stronger than the evidence currently supports.

major comments (3)
  1. [Sec. 3.5, Eqs. (9)-(10); Sec. 4.3; Sec. 4.8; Sec. 4.9; Fig. 17; Sec. 5] The audio localization module associates each silence-segmented audio segment's embedding with the robot odometry at the segment's timestamp, not with a sound-source position. The paper itself states in Sec. 5 that 'a sound can be heard in all locations inside a room, but we only associate its features with the robot's current location,' and Fig. 17's caption says that the visualized sound locations 'are not sound-source locations but the places where the sounds were heard.' No experiment uses physically emitted sound: simulation adds random audio tracks post hoc (Sec. 4.3), the real mobile experiment mixes semantic soundtracks with recorded environmental noise because speaker playback was masked by robot vibration (Sec. 4.8), and the tabletop experiment inserts audio into recorded observation data (Sec. 4.9). Consequently, the recall gains in Tab. 8 and the ambiguous sound-goal navigation successes in Tab. 10 measure retrieval of injection positions in recordings, not localization of sound sources in the physical environment. This directly weakens the abstract's claim that AVLMaps ground audio cues to spatial locations for navigation. The authors should either add experiments with a speaker emitting sounds from known locations during mapping and report distance-to-source error, or explicitly reframe the contribution as navigation to locations where sounds were previously heard by the robot.
  2. [Sec. 4.3-4.6, Tables 5-10] The core multimodal goal navigation results are presented without adequate baselines or statistical support. Sound-goal navigation (Tab. 5) and combined multimodal goal navigation (Tab. 6) have no comparison method, so the reader cannot assess whether the audio module's contribution goes beyond a retrieval rule that already knows which audio segment corresponds to the query. The disambiguation experiments in Tab. 10 use a single multimodal baseline, and no error bars, confidence intervals, or significance tests are reported for any success or recall rate. Given the small task counts (21 trajectories in Sec. 4.1, 119 indexing tasks in Sec. 4.5, 20 navigation tasks in Sec. 4.8) and the manual construction of task sets, the conclusion in Sec. 6 that the method 'significantly outperforms baselines' is not supported by the reported statistics. The paper should add unimodal ablations and chance-level performance, and report per-scene variance or confidence intervals.
  3. [Sec. 3.6, Eq. (11); Sec. 4.8] The cross-modal disambiguation mechanism is an element-wise product of heatmaps under an explicit conditional-independence assumption, and its behavior is controlled by hand-tuned decay rates (0.1 for the primary heatmap and 0.01 for the auxiliary heatmap) plus the noise-gate and silence-segmentation parameters described in Sec. 4.8. The paper neither tests the independence assumption nor reports sensitivity to these parameters, even though the headline disambiguation improvements are produced by this fusion rule. A sensitivity analysis over decay-rate ratios and noise-gate thresholds, and ideally a comparison to alternative fusion operators (e.g., max fusion or learned weights), would be needed to establish that the reported gains are robust rather than an artifact of the chosen configuration.
minor comments (5)
  1. [Sec. 3.5, Eq. (10)] Eq. (10) appears to contain a typo: the subscript 'i' in 'p_si' is not bound outside the maximum, and subtracting an additional distance term from a quantity that already contains the maximum over the same positions is not the natural analogue of Eq. (8). Please revise to H_s(p) = max(bar_H_s(p), 0) or clarify the intended formula.
  2. [Sec. 4.3] The subsection title starts with 'Exerimental setup'; please correct the spelling.
  3. [Table 2 and Fig. 12] The API name is given as 'get_max_pose_3d' in Table 2 but appears as 'get_max_pos_3d' in the code examples in Fig. 12 and Fig. 13; please make the names consistent. Also, the code example in Fig. 12 contains a line with '/* robot.get_map(sound="dog")' inside Python code, which is not valid Python syntax.
  4. [Abstract and Sec. 4.5] The abstract's statement 'improve recall by 50%' is ambiguous: it should specify whether the gain is absolute percentage points or relative. For example, Tab. 9 shows a gain of about 54.7 percentage points at the 0.5 m tolerance, while Tab. 8 shows a gain of about 27.7 percentage points at <0.5 m.
  5. [References] The reference list contains duplicate entries for Chen et al. 2023a and 2023b (the 'Open-vocabulary queryable scene representations for real world planning' paper appears twice with different keys); please merge them.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the audio-spatial limitation is explicit, self-citations are non-load-bearing, and the central evaluations are against external baselines.

full rationale

No circular reasoning is exhibited in the paper's derivation chain. VLMaps and AVLMaps are empirical mapping-and-retrieval systems whose components are external pretrained models (LSeg, CLIP, AudioCLIP, CLAP, NetVLAD, SuperPoint, SuperGlue) and whose evaluations are run against external baselines such as LM-Nav, CoW, AudioCLIP, wav2clip, and ConceptFusion. The self-citations to the authors' prior VLMaps work (Huang et al. 2023b) and to HOV-SG (Werby et al. 2024) are used as components and baselines, not as an unverified uniqueness theorem or as a substitute for independent validation; VLMaps was independently peer-reviewed and is code-reproducible. The heatmap fusion in Sec. 3.6 (Eq. 11) and the LLM code generation in Sec. 3.7 are stated mechanisms with hand-set decay rates, not parameters fitted to the reported recall or success numbers. The one conceptually delicate point is the audio localization module (Sec. 3.5): it associates each silence-segmented audio clip with the odometry at its timestamp (Eqs. 9-10), and the simulation ground truth in Sec. 4.3 is also the video frame position where the audio was inserted, so the sound-goal numbers measure text-to-audio retrieval among injected segments rather than physical sound-source localization. However, this is explicitly acknowledged in Sec. 5 ('a sound can be heard in all locations inside a room, but we only associate its features with the robot's current location') and in the Fig. 17 caption ('The locations of sounds are not sound-source locations but the places where the sounds were heard'), so it is a stated and visible limitation rather than a concealed circular derivation. The central cross-modal disambiguation claims also include visual-object and area-object tasks whose ground truth is not defined by the method's stored association, so the core contribution retains independent empirical content.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central results depend on several hand-set hyperparameters (heatmap decay rates, noise gate settings) and on strong assumptions about foundation model alignment, conditional independence of heatmaps, and the ability to associate sounds with robot poses. These are the main things a reader must accept that the paper does not derive or independently validate.

free parameters (5)
  • primary heatmap decay rate = 0.1 per 0.05 m voxel
    Controls how quickly the main target heatmap decays with distance; set by hand in Sec 3.6 and directly affects fusion outcomes.
  • auxiliary heatmap decay rate = 0.01 per 0.05 m voxel
    Sets the influence radius of disambiguating modality heatmaps; chosen by hand in Sec 3.6.
  • noise gate threshold = -10 dB
    Used in real-world audio preprocessing to suppress background noise; selected based on empirical experience (Sec 4.8).
  • noise gate attack/hold/release times = 250 ms / 1000 ms / 170 ms
    Shaping parameters for the noise gate; chosen empirically to avoid fragmenting target audio (Sec 4.8).
  • silence segmentation threshold = 0.1
    Used to segment audio into clips; set empirically in Sec 4.8.
assumptions (5)
  • standard math Back-projection of depth pixels to 3D points uses a standard pinhole camera model and Euclidean geometry.
    Used throughout Sec 3.1 and 3.5 for fusing image features into the map; accepted geometric basis.
  • domain assumption Pretrained multimodal encoders (LSeg, CLIP, AudioCLIP, CLAP) produce embeddings that are semantically aligned across text, image, and audio modalities.
    The entire query-matching pipeline relies on this alignment; failures directly degrade localization performance (Sec 3.1, 3.5).
  • domain assumption Camera odometry and RGB-D registration are accurate enough for back-projecting features to the right 3D locations.
    VLMaps and AVLMaps project pixel features into a map using estimated poses; pose drift would corrupt the map (Sec 3.1).
  • ad hoc to paper Heatmaps from different modalities are conditionally independent, so element-wise multiplication is a valid fusion strategy.
    Stated as an underlying assumption in Sec 3.6; it can be violated when modalities are correlated, such as an object and a sound that always co-occur.
  • ad hoc to paper Sounds heard during exploration can be attributed to the robot's current pose and remain stationary until inference.
    Audio localization associates each sound segment's embedding with the timestamped robot pose, not with the true sound source location; Sec 3.5 and limitations in Sec 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Spatial Language Maps for Robot Navigation and Manipulation." pith.science (2026). https://pith.science/paper/WXH75JIN

@misc{pith2026250606862,
  author       = {Pith},
  title        = {Pith review of: Multimodal Spatial Language Maps for Robot Navigation and Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WXH75JIN}},
  note         = {Machine review of arXiv:2506.06862}
}
read the original abstract

Grounding language to a navigating agent's observations can leverage pretrained multimodal foundation models to match perceptions to object or event descriptions. However, previous approaches remain disconnected from environment mapping, lack the spatial precision of geometric maps, or neglect additional modality information beyond vision. To address this, we propose multimodal spatial language maps as a spatial map representation that fuses pretrained multimodal features with a 3D reconstruction of the environment. We build these maps autonomously using standard exploration. We present two instances of our maps, which are visual-language maps (VLMaps) and their extension to audio-visual-language maps (AVLMaps) obtained by adding audio information. When combined with large language models (LLMs), VLMaps can (i) translate natural language commands into open-vocabulary spatial goals (e.g., "in between the sofa and TV") directly localized in the map, and (ii) be shared across different robot embodiments to generate tailored obstacle maps on demand. Building upon the capabilities above, AVLMaps extend VLMaps by introducing a unified 3D spatial representation integrating audio, visual, and language cues through the fusion of features from pretrained multimodal foundation models. This enables robots to ground multimodal goal queries (e.g., text, images, or audio snippets) to spatial locations for navigation. Additionally, the incorporation of diverse sensory inputs significantly enhances goal disambiguation in ambiguous environments. Experiments in simulation and real-world settings demonstrate that our multimodal spatial language maps enable zero-shot spatial and multimodal goal navigation and improve recall by 50% in ambiguous scenarios. These capabilities extend to mobile robots and tabletop manipulators, supporting navigation and interaction guided by visual, audio, and spatial cues.

Figures

Figures reproduced from arXiv: 2506.06862 by the authors.

Figure 1
Figure 1. AVLMaps provide an open-vocabulary 3D map representation for storing cross-modal information from audio, visual, and language cues. When combined with large language models, AVLMaps consumes multimodal prompts from audio, vision, and language to solve zero-shot spatial goal navigation by effectively leveraging complementary information sources to disambiguate goals. use vision language models (VLMs) as critics to ma… view at source ↗
Figure 2
Figure 2. The creation and language-conditioned indexing of a VLMap. A VLMap is created by fusing pretrained visual-language features into the reconstruction of the environment to enable visual-spatial-language-based reasoning. By providing a list of open-vocabulary labels, we retrieve segmentation masks for semantic classes required by downstream applications. observation data into the representation of the scene as we do in… view at source ↗
Figure 3
Figure 3. The overview of building customized obstacle maps for different robot embodiments. By specifying different obstacle categories in natural language for different embodiments, different obstacle maps can be built to ensure the most efficient path planning for different embodiments. and CLIP). Specifically, given a list of obstacle categories described with natural language, we can localize those obstacles at runtime t… view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: The full context prompt (prompt in gray) VLMap used for achieving spatial goal navigation tasks in the experiments. Prepared using sagej.cls [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: VLMaps enable a robot to perform complex zero-shot spatial goal navigation tasks given natural language commands, without additional data collection or model finetuning. APIs Functions move_to(pos) move to a position on the map. move_to_left(object_name) move to the le…
Figure 6
Figure 6. Figure 6: The query and the generated results from the LLM for spatial goal navigation tasks. During the query, the context prompt in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: System overview. AVLMaps are constructed from RGB-D, audio, and odometry inputs, converting raw data into visual localization features, visual-language features, and audio-language features. During inference time, each module’s output is unified with cross-modal reason…
Figure 8
Figure 8. Figure 8: The overview of the visual localization module. We follow the hierarchical localization scheme by using NetVLAD and SuperGLUE to localize the query image’s location before generating a heatmap as the indexing result. The object localization results are a list of points…
Figure 9
Figure 9. Figure 9: The overview of the object localization module. Similar to Sec. 3.1, during mapping, the RGB images in the exploration video are input to a vision-language model, LSeg (Li et al. 2021), to generate pixel-level features. Corresponding pixels are back-projected with dept…
Figure 11
Figure 11. Figure 11: , in the query “the sound of baby crying near the sofa”, the heatmap for “the sound of baby crying” is the primary heatmap, while the heatmap for “the sofa” is the auxiliary. We set the decay rate for the primary heatmap higher (e.g., 0.1 in this work for voxel map wi…
Figure 13
Figure 13. Figure 13: The query and the generated results from the LLM. During the query, the context prompt in [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 12
Figure 12. Figure 12: The full context prompt (prompt in gray) AVLMaps used for achieving all navigation tasks in the experiments. # move in between the image ./006899.png and the backpack near the sound of glass breaking img = robot.load_image("./006899.png") img_map = robot.get_major_map…
Figure 14
Figure 14. Figure 14: VLMaps enable different embodiments to define their own obstacle maps for navigation. The left image shows the top-down view of an environment. The middle columns show the observations of agents during navigation. The images on the right demonstrate the obstacles maps…
Figure 15
Figure 15. Figure 15: Real-world navigation experiments are conducted in a room with multiple ambiguous goals such as tables, chairs, backpacks, and paper boxes (left). The robot setup is also shown in the left image. We leverage dense SLAM techniques to build a 3D reconstruction of the sc…
Figure 16
Figure 16. Figure 16: Audio denoising for real-world experiment. We pre-process the audio by applying a noise gate. Simply applying a threshold for filtering out low-level noise might lead to frequent fluctuation of the audio level, leading to fragmentation of the target audio (see the lef…
Figure 17
Figure 17. Figure 17: Visualization of example heatmaps in AVLMaps for multimodal goal reasoning for ambiguous object goals. The first column shows the positions of ambiguous objects (green bounding boxes) and the location of a sound (the icon of a speaker) or an image (the icon of a camer…
Figure 18
Figure 18. Figure 18: Visualization of tabletop goal-reaching experiments. We set up two tabletop scenes and inserted sounds to different locations in the observation data (left). We show the indexing heatmap results of sound goals, visual goals, object goals, and ambiguous goal reaching r…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

99 extracted references · 75 canonical work pages

  1. [1]

    In: Proc

    Ahn M, Brohan A, Brown N, Chebotar Y, Cortes O, David B, Finn C, Gopalakrishnan K, Hausman K, Herzog A, Ho D, Hsu J, Ibarz J, Ichter B, Irpan A, Jang E, Ruano RMJ, Jeffrey K, Jesmonth S, Joshi NJ, Julian RC, Kalashnikov D, Kuang Y, Lee KH, Levine S, Lu Y, Luu L, Parada C, Pastor P, Quiambao J, Rao K, Rettinghouse J, Reyes DM, Sermanet P, Sievers N, Tan C,...

    2022

  2. [2]

    https://arxiv.org/abs/1807.06757

    Anderson P, Chang A, Chaplot DS, Dosovitskiy A, Gupta S, Koltun V, Kosecka J, Malik J, Mottaghi R, Savva M and Zamir AR (2018 a ) On evaluation of embodied navigation agents. https://arxiv.org/abs/1807.06757

    arXiv 2018

  3. [3]

    In: Proc

    Anderson P, Shrivastava A, Truong J, Majumdar A, Parikh D, Batra D and Lee S (2021) Sim-to-real transfer for vision-and-language navigation. In: Proc. of the Conference on Robot Learning (CoRL)

    2021

  4. [4]

    In: Proc

    Anderson P, Wu Q, Teney D, Bruce J, Johnson M, S \"u nderhauf N, Reid I, Gould S and Van Den Hengel A (2018 b ) Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2018

  5. [5]

    In: Proc

    Arandjelovic R, Gronat P, Torii A, Pajdla T and Sivic J (2016) Netvlad: Cnn architecture for weakly supervised place recognition. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2016

  6. [6]

    In: Proc

    Brohan A, Brown N, Carbajal J, Chebotar Y, Dabis J, Finn C, Gopalakrishnan K, Hausman K, Herzog A, Hsu J, Ibarz J, Ichter B, Irpan A, Jackson T, Jesmonth S, Joshi N, Julian R, Kalashnikov D, Kuang Y, Leal I, Lee KH, Levine S, Lu Y, Malla U, Manjunath D, Mordatch I, Nachum O, Parada C, Peralta J, Perez E, Pertsch K, Quiambao J, Rao K, Ryoo MS, Salazar G, S...

    2023

  7. [7]

    In: Proc

    Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I and Amodei D (2020) Language models are few-sho...

    2020

  8. [8]

    In: International Conference on 3D Vision (3DV)

    Chang A, Dai A, Funkhouser T, Halber M, Niessner M, Savva M, Song S, Zeng A and Zhang Y (2017) Matterport3D : Learning from RGB-D data in indoor environments. In: International Conference on 3D Vision (3DV)

    2017

Show all 99 references
  1. [9]

    In: Proc

    Chaplot DS, Gandhi DP, Gupta A and Salakhutdinov RR (2020) Object goal navigation using goal-oriented semantic exploration. In: Proc. of the Advances in Neural Information Processing Systems (NeurIPS)

    2020

  2. [10]

    In: Proc

    Chen B, Xia F, Ichter B, Rao K, Gopalakrishnan K, Ryoo MS, Stone A and Kappler D (2023 a ) Open-vocabulary queryable scene representations for real world planning. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2023

  3. [11]

    In: Proc

    Chen B, Xia F, Ichter B, Rao K, Gopalakrishnan K, Ryoo MS, Stone A and Kappler D (2023 b ) Open-vocabulary queryable scene representations for real world planning. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2023

  4. [12]

    In: Proc

    Chen C, Jain U, Schissler C, Gari SVA, Al-Halah Z, Ithapu VK, Robinson P and Grauman K (2020) Soundspaces: Audio-visual navigation in 3d environments. In: Proc. of European Conference on Computer Vision (ECCV)

    2020

  5. [13]

    In: Proc

    Chen C, Majumder S, Al-Halah Z, Gao R, Ramakrishnan SK and Grauman K (2021 a ) Learning to set waypoints for audio-visual navigation. In: Proc. of International Conference on Learning Representations (ICLR)

    2021

  6. [14]

    Chen C, Schissler C, Garg S, Kobernik P, Clegg A, Calamia P, Batra D, Robinson P and Grauman K (2022) Soundspaces 2.0: A simulation platform for visual-acoustic learning. Proc. of the Advances in Neural Information Processing Systems (NeurIPS)

    2022

  7. [15]

    https://arxiv.org/abs/2107.03374

    Chen M, Tworek J, Jun H, Yuan Q, de Oliveira Pinto HP, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G, Ray A, Puri R, Krueger G, Petrov M, Khlaaf H, Sastry G, Mishkin P, Chan B, Gray S, Ryder N, Pavlov M, Power A, Kaiser L, Bavarian M, Winter C, Tillet P, Such FP, Cummings...

    2021 arXiv

  8. [16]

    https://arxiv.org/abs/2402.02651

    Chen W, Mees O, Kumar A and Levine S (2024 a ) Vision-language models provide promptable representations for reinforcement learning. https://arxiv.org/abs/2402.02651

    2024 arXiv

  9. [17]

    In: Proc

    Chen Z, Gebru ID, Richardt C, Kumar A, Laney W, Owens A and Richard A (2024 b ) Real acoustic fields: An audio-visual room acoustics dataset and benchmark. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  10. [18]

    Cognitive psychology

    Chun MM and Jiang Y (1998) Contextual cueing: Implicit learning and memory of visual context guides spatial attention. Cognitive psychology

    1998

  11. [19]

    In: Proc

    DeTone D, Malisiewicz T and Rabinovich A (2018) Superpoint: Self-supervised interest point detection and description. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2018

  12. [20]

    In: Proc

    Doshi R, Walke H, Mees O, Dasari S and Levine S (2024) Scaling cross-embodied learning: One policy for manipulation, navigation, locomotion and aviation. In: Proc. of the Conference on Robot Learning (CoRL)

    2024

  13. [21]

    In: Proc

    Elizalde B, Deshmukh S, Al Ismail M and Wang H (2023) Clap learning audio concepts from natural language supervision. In: Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    2023

  14. [22]

    In: Proc

    Endres F, Hess J, Engelhard N, Sturm J, Cremers D and Burgard W (2012) An evaluation of the rgb-d slam system. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2012

  15. [23]

    In: Proc

    Engelmann F, Manhardt F, Niemeyer M, Tateno K, Pollefeys M and Tombari F (2024) OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views . In: Proc. of International Conference on Learning Representations (ICLR)

    2024

  16. [24]

    Communications of the ACM

    Fischler MA and Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM

    1981

  17. [25]

    Fried D, Hu R, Cirik V, Rohrbach A, Andreas J, Morency LP, Berg-Kirkpatrick T, Saenko K, Klein D and Darrell T (2018) Speaker-follower models for vision-and-language navigation

    2018

  18. [26]

    In: Proc

    Gadre SY, Wortsman M, Ilharco G, Schmidt L and Song S (2023) Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  19. [27]

    In: Proc

    Gan C, Gu Y, Zhou S, Schwartz J, Alter S, Traer J, Gutfreund D, Tenenbaum JB, McDermott JH and Torralba A (2022) Finding fallen objects via asynchronous audio-visual integration. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2022

  20. [28]

    https://arxiv.org/abs/2007.04954

    Gan C, Schwartz J, Alter S, Schrimpf M, Traer J, Freitas JD, Kubilius J, Bhandwaldar A, Haber N, Sano M, Kim K, Wang E, Mrowca D, Lingelbach M, Curtis A, Feigelis KT, Bear D, Gutfreund D, Cox D, DiCarlo JJ, McDermott JH, Tenenbaum JB and Yamins DLK (2020 a ) Threedworld: A pla...

    2020 arXiv

  21. [29]

    In: Proc

    Gan C, Zhang Y, Wu J, Gong B and Tenenbaum JB (2020 b ) Look, listen, and act: Towards audio-visual embodied navigation. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2020

  22. [30]

    In: Proc

    Ganapathi A, Florence P, Varley J, Burns K, Goldberg K and Zeng A (2022) Implicit kinematic policies: Unifying joint and cartesian action spaces in end-to-end robot learning. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2022

  23. [31]

    In: Proc

    Ghiasi G, Gu X, Cui Y and Lin TY (2022) Scaling open-vocabulary image segmentation with image-level labels. In: Proc. of European Conference on Computer Vision (ECCV)

    2022

  24. [32]

    In: Proc

    Gu Q, Kuwajerwala A, Morin S, Jatavallabhula KM, Sen B, Agarwal A, Rivera C, Paul W, Ellis K, Chellappa R, Gan C, de Melo CM, Tenenbaum JB, Torralba A, Shkurti F and Paull L (2024) Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. In: Proc. of the IEE...

    2024

  25. [33]

    In: Proc

    Gu X, Lin TY, Kuo W and Cui Y (2021) Open-vocabulary object detection via vision and language knowledge distillation. In: Proc. of International Conference on Learning Representations (ICLR)

    2021

  26. [34]

    In: Proc

    Guhur PL, Tapaswi M, Chen S, Laptev I and Schmid C (2021) Airbert: In-domain pretraining for vision-and-language navigation. In: Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2021

  27. [35]

    In: Proc

    Guzhov A, Raue F, Hees J and Dengel A (2022) Audioclip: Extending clip to image, text and audio. In: Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    2022

  28. [36]

    In: Proc

    Hirose N, Glossop C, Sridhar A, Shah D, Mees O and Levine S (2024) Lelan: Learning a language-conditioned navigation policy from in-the-wild video. In: Proc. of the Conference on Robot Learning (CoRL)

    2024

  29. [37]

    In: Proc

    Hong Y, Wang Z, Wu Q and Gould S (2022) Bridging the gap between learning in discrete and continuous environments for vision-and-language navigation. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2022

  30. [38]

    In: Proc

    Huang C, Mees O, Zeng A and Burgard W (2023 a ) Audio visual language maps for robot navigation. In: Proc. of the International Symposium of Experimental Robotics (ISER)

    2023

  31. [39]

    In: Proc

    Huang C, Mees O, Zeng A and Burgard W (2023 b ) Visual language maps for robot navigation. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2023

  32. [40]

    IEEE Robotics and Automation Letters

    Huang C, Yan S and Burgard W (2025) Bye: Build your encoder with one sequence of exploration data for long-term dynamic scene understanding. IEEE Robotics and Automation Letters

    2025

  33. [41]

    In: Proc

    Huang W, Abbeel P, Pathak D and Mordatch I (2022 a ) Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In: Proc. of the International Conference on Machine Learning (ICML)

    2022

  34. [42]

    In: Proc

    Huang W, Xia F, Xiao T, Chan H, Liang J, Florence P, Zeng A, Tompson J, Mordatch I, Chebotar Y, Sermanet P, Jackson T, Brown N, Luu L, Levine S, Hausman K and brian ichter (2022 b ) Inner monologue: Embodied reasoning through planning with language models. In: Proc. of the Con...

    2022

  35. [43]

    In: Proc

    Jatavallabhula KM, Kuwajerwala A, Gu Q, Omama M, Iyer G, Saryazdi S, Chen T, Maalouf A, Li S, Keetha NV, Tewari A, Tenenbaum J, de Melo C, Krishna M, Paull L, Shkurti F and Torralba A (2023) ConceptFusion: Open-set multimodal 3D mapping . In: Proc. of Robotics: Science and Sys...

    2023

  36. [44]

    In: Proc

    Kamath A, Singh M, LeCun Y, Synnaeve G, Misra I and Carion N (2021) Mdetr-modulated detection for end-to-end multi-modal understanding. In: Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2021

  37. [45]

    In: Proc

    Kerr J, Kim CM, Goldberg K, Kanazawa A and Tancik M (2023) Lerf: Language embedded radiance fields. In: Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2023

  38. [46]

    In: Proc

    Kim CM, Wu M, Kerr J, Goldberg K, Tancik M and Kanazawa A (2024) Garfield: Group anything with radiance fields. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  39. [47]

    In: Proc

    Kim MJ, Pertsch K, Karamcheti S, Xiao T, Balakrishna A, Nair S, Rafailov R, Foster EP, Sanketi PR, Vuong Q, Kollar T, Burchfiel B, Tedrake R, Sadigh D, Levine S, Liang P and Finn C (2025) Openvla: An open-source vision-language-action model. In: Proc. of the Conference on Robo...

    2025

  40. [48]

    In: Proc

    Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo WY, Dollar P and Girshick R (2023) Segment anything. In: Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2023

  41. [49]

    https://arxiv.org/abs/1712.05474

    Kolve E, Mottaghi R, Han W, VanderBilt E, Weihs L, Herrasti A, Deitke M, Ehsani K, Gordon D, Zhu Y, Kembhavi A, Gupta A and Farhadi A (2017) Ai2-thor: An interactive 3d environment for visual ai. https://arxiv.org/abs/1712.05474

    2017 arXiv

  42. [50]

    PLoS one

    K \"o rding KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB and Shams L (2007) Causal inference in multisensory perception. PLoS one

    2007

  43. [51]

    In: Proc

    Krantz J, Gokaslan A, Batra D, Lee S and Maksymets O (2021) Waypoint models for instruction-guided navigation in continuous environments. In: Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2021

  44. [52]

    In: Proc

    Krantz J, Wijmans E, Majumdar A, Batra D and Lee S (2020) Beyond the nav-graph: Vision-and-language navigation in continuous environments. In: Proc. of European Conference on Computer Vision (ECCV)

    2020

  45. [53]

    In: Proc

    Li B, Weinberger KQ, Belongie S, Koltun V and Ranftl R (2021) Language-driven semantic segmentation. In: Proc. of International Conference on Learning Representations (ICLR)

    2021

  46. [54]

    In: Proc

    Li B, Weinberger KQ, Belongie S, Koltun V and Ranftl R (2022) Language-driven semantic segmentation. In: Proc. of International Conference on Learning Representations (ICLR)

    2022

  47. [55]

    In: Proc

    Liang F, Wu B, Dai X, Li K, Zhao Y, Zhang H, Zhang P, Vajda P and Marculescu D (2023 a ) Open-vocabulary semantic segmentation with mask-adapted clip. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  48. [56]

    In: Proc

    Liang J, Huang W, Xia F, Xu P, Hausman K, Ichter B, Florence P and Zeng A (2023 b ) Code as policies: Language model programs for embodied control. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2023

  49. [57]

    MacMahon M, Stankiewicz B and Kuipers B (2006) Walk the talk: Connecting language, knowledge, and action in route instructions. Def

    2006

  50. [58]

    In: International Conference on 3D Vision (3DV)

    McCormac J, Clark R, Bloesch M, Davison A and Leutenegger S (2018) Fusion++: Volumetric object-level slam. In: International Conference on 3D Vision (3DV)

    2018

  51. [59]

    In: Proc

    McCormac J, Handa A, Davison A and Leutenegger S (2017) Semanticfusion: Dense 3d semantic mapping with convolutional neural networks. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2017

  52. [60]

    Journal of Experimental Psychology: Learning, Memory, and Cognition

    McNamara TP, Hardy JK and Hirtle SC (1989) Subjective hierarchies in spatial memory. Journal of Experimental Psychology: Learning, Memory, and Cognition

    1989

  53. [61]

    In: Proc

    Mees O, Borja-Diaz J and Burgard W (2023) Grounding language with visual affordances over unstructured data. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2023

  54. [62]

    IEEE Robotics and Automation Letters

    Mees O, Hermann L and Burgard W (2022 a ) What matters in language conditioned robotic imitation learning over unstructured data. IEEE Robotics and Automation Letters

    2022

  55. [63]

    IEEE Robotics and Automation Letters

    Mees O, Hermann L, Rosete-Beas E and Burgard W (2022 b ) Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters

    2022

  56. [64]

    Cognition

    Newman EL, Caplan JB, Kirschen MP, Korolev IO, Sekuler R and Kahana MJ (2007) Learning your way around town: How virtual taxicab drivers learn to use both layout and landmark information. Cognition

    2007

  57. [65]

    In: Proc

    Octo Model Team , Ghosh D, Walke H, Pertsch K, Black K, Mees O, Dasari S, Hejna J, Xu C, Luo J, Kreiman T, Tan Y, Chen LY, Sanketi P, Vuong Q, Xiao T, Sadigh D, Finn C and Levine S (2024) Octo: An open-source generalist robot policy. In: Proc. of Robotics: Science and Systems (RSS)

    2024

  58. [66]

    In: Proc

    O’Neill A, Rehman A, Maddukuri A, Gupta A, Padalkar A, Lee A, Pooley A, Gupta A, Mandlekar A, Jain A, Tung A, Bewley A, Herzog A, Irpan A, Khazatsky A, Rai A, Gupta A, Wang A, Singh A, Garg A, Kembhavi A, Xie A, Brohan A, Raffin A, Sharma A, Yavary A, Jain A, Balakrishna A, Wa...

    2024

  59. [67]

    In: Proc

    Paul S, Roy-Chowdhury A and Cherian A (2022) Avlen: Audio-visual-language embodied navigation in 3d environments. In: Proc. of the Advances in Neural Information Processing Systems (NeurIPS)

    2022

  60. [68]

    In: Proc

    Peng S, Genova K, Jiang CM, Tagliasacchi A, Pollefeys M and Funkhouser T (2023) Openscene: 3d scene understanding with open vocabularies. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  61. [69]

    In: Proceedings of the 23rd ACM international conference on Multimedia

    Piczak KJ (2015) Esc: Dataset for environmental sound classification. In: Proceedings of the 23rd ACM international conference on Multimedia

    2015

  62. [70]

    In: Proc

    Qin M, Li W, Zhou J, Wang H and Pfister H (2024) Langsplat: 3d language gaussian splatting. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  63. [71]

    In: Proc

    Quigley M (2009) Ros: an open-source robot operating system. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2009

  64. [72]

    In: Proc

    Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J, Krueger G and Sutskever I (2021) Learning transferable visual models from natural language supervision. In: Proc. of the International Conference on Machine Learning (ICML)

    2021

  65. [73]

    In: Proc

    Rosete-Beas E, Mees O, Kalweit G, Boedecker J and Burgard W (2022) Latent plans for task agnostic offline reinforcement learning. In: Proc. of the Conference on Robot Learning (CoRL)

    2022

  66. [74]

    In: Proc

    Runz M, Buffier M and Agapito L (2018) Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects. In: Proc. of IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

    2018

  67. [75]

    In: Proc

    Salas-Moreno RF, Newcombe RA, Strasdat H, Kelly PH and Davison AJ (2013) Slam++: Simultaneous localisation and mapping at the level of objects. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2013

  68. [76]

    In: Proc

    Sarlin PE, Cadena C, Siegwart R and Dymczyk M (2019) From coarse to fine: Robust hierarchical localization at large scale. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  69. [77]

    In: Proc

    Sarlin PE, DeTone D, Malisiewicz T and Rabinovich A (2020) Superglue: Learning feature matching with graph neural networks. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  70. [78]

    In: Proc

    Savva M, Kadian A, Maksymets O, Zhao Y, Wijmans E, Jain B, Straub J, Liu J, Koltun V, Malik J, Parikh D and Batra D (2019) Habitat: A P latform for E mbodied AI R esearch. In: Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2019

  71. [79]

    International Journal of Computer Vision

    Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D and Batra D (2020) Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision

    2020

  72. [80]

    In: Proc

    Shafiullah NMM, Paxton C, Pinto L, Chintala S and Szlam A (2023) CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory . In: Proc. of Robotics: Science and Systems (RSS)

    2023

  73. [81]

    In: Proc

    Shah D, Osi\'nski B, ichter b and Levine S (2023) Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In: Proc. of the Conference on Robot Learning (CoRL)

    2023

  74. [82]

    In: Proc

    Shridhar M, Manuelli L and Fox D (2022) Cliport: What and where pathways for robotic manipulation. In: Proc. of the Conference on Robot Learning (CoRL)

    2022

  75. [83]

    In: Proc

    Szot A, Clegg A, Undersander E, Wijmans E, Zhao Y, Turner J, Maestre N, Mukadam M, Chaplot D, Maksymets O, Gokaslan A, Vondrus V, Dharur S, Meier F, Galuba W, Chang A, Kira Z, Koltun V, Malik J, Savva M and Batra D (2021) Habitat 2.0: Training home assistants to rearrange thei...

    2021

  76. [84]

    In: Proc

    Tellex S, Kollar T, Dickerson S, Walter M, Banerjee A, Teller S and Roy N (2011) Understanding natural language commands for robotic navigation and mobile manipulation. In: Proc. of the National Conference on Artificial Intelligence (AAAI)

    2011

  77. [85]

    Autonomous Robots

    Thrun S, Burgard W and Fox D (1998) A probabilistic approach to concurrent mapping and localization for mobile robots. Autonomous Robots

    1998

  78. [86]

    In: Proc

    Wang ML, Sawata R, Clarke S, Gao R, Wu S and Wu J (2024) Hearing anything anywhere. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  79. [87]

    In: Proc

    Werby A, Huang C, Büchner M, Valada A and Burgard W (2024) Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation . In: Proc. of Robotics: Science and Systems (RSS)

    2024

  80. [88]

    In: Proc

    Wu HH, Seetharaman P, Kumar K and Bello JP (2022) Wav2clip: Learning robust audio representations from clip. In: Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    2022

  81. [89]

    In: Proc

    Wu J, Sun X, Zeng A, Song S, Rusinkiewicz S and Funkhouser T (2021) Spatial intention maps for multi-agent mobile manipulation. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2021

  82. [90]

    In: Proc

    Xu B, Li W, Tzoumanikas D, Bloesch M, Davison A and Leutenegger S (2019) Mid-fusion: Octree-based object-level multi-instance dynamic slam. In: Proc. of the IEEE International Conference on Robotics & Automation (ICRA)

    2019

  83. [91]

    IEEE Robotics and Automation Letters

    Yan Z, Li S, Wang Z, Wu L, Wang H, Zhu J, Chen L and Liu J (2025) Dynamic open-vocabulary 3d scene graphs for long-term language-guided mobile manipulation. IEEE Robotics and Automation Letters

    2025

  84. [92]

    IEEE Robotics and Automation Letters

    Younes A, Honerkamp D, Welschehold T and Valada A (2023) Catch me if you hear me: Audio-visual navigation in complex unmapped environments with moving sounds. IEEE Robotics and Automation Letters

    2023

  85. [93]

    In: Proc

    Zakka K, Zeng A, Florence P, Tompson J, Bohg J and Dwibedi D (2022) Xirl: Cross-embodiment inverse reinforcement learning. In: Proc. of the Conference on Robot Learning (CoRL)

    2022

  86. [94]

    In: Proc

    Zawalski M, Chen W, Pertsch K, Mees O, Finn C and Levine S (2024) Robotic control via embodied chain-of-thought reasoning. In: Proc. of the Conference on Robot Learning (CoRL)

    2024

  87. [95]

    PhD Thesis, Princeton University

    Zeng A (2019) Learning visual affordances for robotic manipulation. PhD Thesis, Princeton University

    2019

  88. [96]

    In: Proc

    Zeng A, Attarian M, Ichter B, Choromanski KM, Wong A, Welker S, Tombari F, Purohit A, Ryoo MS, Sindhwani V, Lee J, Vanhoucke V and Florence P (2023) Socratic models: Composing zero-shot multimodal reasoning with language. In: Proc. of International Conference on Learning Repre...

    2023

  89. [97]

    In: Proc

    Zhou Z, Atreya P, Lee A, Walke H, Mees O and Levine S (2024) Autonomous improvement of instruction following skills via foundation models. In: Proc. of the Conference on Robot Learning (CoRL)

    2024

  90. [98]

    In: Proc

    Zitkovich B, Yu T, Xu S, Xu P, Xiao T, Xia F, Wu J, Wohlhart P, Welker S, Wahid A, Vuong Q, Vanhoucke V, Tran H, Soricut R, Singh A, Singh J, Sermanet P, Sanketi PR, Salazar G, Ryoo MS, Reymann K, Rao K, Pertsch K, Mordatch I, Michalewski H, Lu Y, Levine S, Lee L, Lee TWE, Lea...

    2023

  91. [99]

    International Journal of Computer Vision

    Zuo X, Samangouei P, Zhou Y, Di Y and Li M (2024) Fmgs: Foundation model embedded 3d gaussian splatting for holistic 3d scene understanding. International Journal of Computer Vision

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.