Pith. sign in

REVIEW 4 major objections 5 minor 46 references

LiLMaps: Learnable Implicit Language Maps

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LiLMaps claims that adapting a small language decoder to each scene and smoothing noisy vision-language measurements in a sparse octree yields large-scale incremental 3D language maps that beat pretrained methods such as OpenScene and…

desk verdict Plausible incremental method for online 3D language mapping, but the reported gains rest on a per-scene fitted comparison and a self-referential measurement update that needs ablating. read the letter →

arxiv 2501.03304 v2 pith:LY5KV54F submitted 2025-01-06 cs.RO cs.LG

Evgenii Kruzhkov , Sven Behnke This is my paper · ORCID
classification cs.ROcs.LG
keywords implicitlanguagemapsincrementalmappingvision-languagefeaturesadaptivedecoderoptimizationsparseoctreemeasurementupdateopen-vocabularysceneunderstandingSLAM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a robot can build a compact, queryable 3D language map incrementally, as it moves, by storing vision-language features in a sparse octree and training a small per-scene decoder rather than relying on a fixed pretrained one. To handle the two problems that arise online, the authors add an adaptive decoder optimization that retrains only when genuinely new language features appear, using a short replay list of known features to avoid forgetting, and a measurement update rule that blends each new observation with the map's own previous prediction to dampen inconsistent labels across views. If correct, the result is an implicit language map that supports natural-language queries and object detection in environments never seen in batch, with memory cost well below the raw image feature set. The experiments report better mapping quality than OpenScene and VLMaps on Matterport3D scenes, and better class-level F1 than fixed pretrained decoder heads.

What carries the argument

The load-bearing mechanism is a sparse octree language field with two coupled learned parts: interpolated corner features and a per-voxel vector $F$, decoded by a three-layer MLP. The paper's two named techniques are adaptive language decoder optimization and the measurement update strategy. The update equation $\varphi^*_n = \alpha \bar{\varphi}_{n-1} + (1-\alpha) \varphi_n$, with $\alpha = \mathrm{CosineSimilarity}(\varphi_n, \bar{\varphi}_n)/(0.5 + \mathrm{CosineSimilarity}(\varphi_n, \bar{\varphi}_n))$, defines a self-referential training target: the map learns to move toward a blend of the newest observation and its own earlier reconstruction. The decoder optimization uses a small replay set of known features, so the decoder can be trained on new features without forgetting old ones.

What would settle it

On a sequence whose first frames mislabel a painting as a wall and whose later frames label it correctly, compare the final map built with Eq. (4)–(5) against one built with the exact running average of Eq. (3) or with a fixed $\alpha$. If the adaptive rule keeps the wrong label or scores lower, the claimed superiority of the measurement update over simple averaging fails; reversing the frame order and observing different final maps would also show order-dependent behavior.

Watch

Extended reading notes

Core claim

LiLMaps represents the environment as a sparse three-level octree: low-dimensional learnable features are stored at voxel corners, one high-dimensional feature vector $F$ is stored per coarse-level voxel, and a three-layer MLP decoder predicts a CLIP-scale language feature from the interpolated and summed corner encoding modulated by $F$. Training uses a cosine-similarity loss between predicted and observed features. The adaptive decoder optimization maintains a set of known language features; when a new frame contains features dissimilar to anything seen before, it initializes encodings and an $F$ vector, optimizes the decoder plus those new parameters while replaying stored features to prevent catastrophic forgetting, and updates the known-feature list. The measurement update strategy replaces the raw observation in the loss with a weighted blend of the new observation and the map's previously predicted feature, with blend weight $\alpha = \mathrm{CosineSimilarity}(\varphi_n, \bar{\varphi}_n)/(0.5 + \mathrm{CosineSimilarity}(\varphi_n, \bar{\varphi}_n))$ increasing when new and stored features disagree. On Matterport3D sequences, LiLMaps reports higher accuracy, recall, precision, and mean IoU than OpenScene and VLMaps, and the adaptive decoder reaches more classes above 90% F1 than fixed pretrained OpenScene heads.

Load-bearing premise

The load-bearing premise is that the map's own previous reconstruction is good enough to serve as a blending partner for new observations; if early language labels are wrong, the adaptive alpha can keep or amplify the stored error instead of correcting it.

Editorial extensions

If this is right

  • LiLMaps can be grafted onto existing implicit SLAM systems with minimal changes, giving them a queryable language layer with small computational overhead.
  • A robot can answer object queries in a large indoor scene directly from the reconstructed implicit map, without an offline batch reconstruction.
  • The adaptive decoder can be pretrained on all labels of a scene and then updated online, or trained entirely online, and it preserves objects mapped early in the sequence when new objects arrive.
  • The number of stored language features stays tiny relative to the input: fewer than 0.5% of pixels in a single 640x480 image across hundreds of frames, keeping the representation memory-efficient.
  • Adaptive decoder optimization runs at about 4 frames per second and can be executed in parallel with mapping, so real-time incremental language mapping is within reach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not in the paper: the self-referential blend makes the final map sensitive to frame order, so the reported gains on noisy semantic data should be checked by reversing sequences or by replacing Eq. (4)–(5) with exact averaging and rerunning Table 2.
  • Not in the paper: the known-feature replay list is essentially a continual-learning memory, so the same adaptive decoder recipe could apply to other incremental perception tasks where object classes appear online.
  • Not in the paper: because the visual language encoder is external, the gap between ground-truth and semantic-input results in Table 2 suggests encoder quality, not map storage, is the main bottleneck; a stronger per-pixel encoder should improve LiLMaps without any architectural change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LiLMaps, an implicit 3D language mapping method that stores learnable features in a sparse octree and reconstructs CLIP-style language features with a small MLP decoder. The two main contributions are an adaptive decoder optimization procedure (Algorithm 1) that updates the decoder when new language features appear, and a measurement update strategy (Sec. 3.3) that blends incoming features with the map's own previous reconstruction before training the map. Experiments on five Matterport3D scenes compare the method with OpenScene and VLMaps in terms of accuracy, recall, precision, and mIoU, and compare the adaptive decoder with fixed pretrained decoders via per-class F1 scores.

Significance. If the claims hold, LiLMaps would be a useful step toward compact, incremental, queryable 3D language maps for robot navigation and LLM-based interaction, since it does not require retraining the whole scene when new object categories appear. The paper gives concrete pseudocode for the adaptive optimization and evaluates on multiple scenes against established baselines. However, the current evidence is weakened by the self-referential update target, the absence of statistical controls, and evaluations that mostly measure per-scene reconstruction rather than generalization. The central ideas are plausible, but the experimental support needs to be strengthened before the claimed improvements can be accepted.

major comments (4)
  1. [3.3] Equation (5) does not implement the behavior described in the text. The text says that α is set to higher values when new measurements φ_n are more different from the previously stored features \bar φ_{n−1}; the formula α = CosineSimilarity(φ_i, \bar φ_i)/(0.5 + CosineSimilarity(φ_i, \bar φ_i)) is monotonically increasing in cosine similarity, so α is largest when φ_i and \bar φ_i are most similar. The authors should correct either the formula or the prose, and state explicitly which behavior was used in the reported experiments.
  2. [3.3] The update in Eq. (4) makes the map's own previous reconstruction \bar φ_{n−1} part of the training target for Eq. (1). This is a self-referential loop: an error stored in the octree or decoder at step n−1 is re-entered into the loss at step n, weighted by α. The paper itself lists relevant failure modes in Sec. 1 (paintings recognized as walls, beds misclassified as sofas). With a single fixed input order per scene in Table 2, the observed improvement of LiLMapsSEM over LiLMaps*SEM could be an artifact of the order in which biased observations arrive rather than of noise reduction. I request an ablation against exact averaging in Eq. (3), a fixed α, and reversed or permuted observation orders, with the resulting range of metrics reported.
  3. [4.1] Table 2 reports single point estimates per scene with no error bars, confidence intervals, or significance tests. Since the update rule in Eq. (4) is history-dependent, the five fixed trajectories are not sufficient to establish that the measurement update reliably improves mapping quality. Please provide per-sequence variances, multiple runs, or at least a paired comparison across scenes; the same concern applies to the F1 class counts in Table 3.
  4. [4] The quantitative comparisons are all performed on the same scenes on which the map and decoder are optimized, so beating fixed pretrained models such as OpenScene and the OpenScene heads is expected by construction. This does not by itself validate the representation or the update rule. To support the abstract's claim of 'solid improvements,' the paper needs a matched comparison in which every method receives the same per-scene adaptation, or an evaluation on held-out scenes and labels to measure generalization.
minor comments (5)
  1. [3.3] Equation (5) uses subscript i while Eq. (4) uses subscript n; please unify the notation and define α per point or per frame explicitly.
  2. [3.2] In Algorithm 1, the line 'fvectors1 ← SHUFFLE(allFVectors)' and the regularization loss in Line 29 are not explained. It is unclear why shuffling is needed and how the regularization enforces similarity to existing F vectors.
  3. [4.2] There are typos and formatting inconsistencies, e.g., 'LiLMapssimple pretraiend' should be 'LiLMaps_simple pretrained', and the subscript/superscript formatting for LiLMaps_simple variants is inconsistent throughout the section.
  4. [4.2] The arrow notation in Table 3 (e.g., '↑100% – 90%') is not defined in the caption and is hard to parse; please add a clear explanation of the column headers.
  5. [4.2] In the discussion of Fig. 6, the specific threshold values on the x-axis are not stated in the text, and the claim that stored features are 'less than 0.5% of the number of pixels in a single image' would be easier to verify with exact storage counts.

Circularity Check

1 steps flagged · score 4.0 of 10

Measurement-update target is self-referential (Eqs. 4-5): the map's own reconstruction is part of its supervision, a partial feedback circularity; no self-citation chain and GT-based comparisons retain independent content.

  1. self definitional [Sec. 3.3, Eqs. (4)-(5) and use in Eq. (1)]
    "we propose to use as target for training in Eq. (1) a weighted average φ∗ n between observations φn and the features ¯φn−1 already stored in the map: φ∗ n = α ¯φn−1 + (1−α)φn with ¯φ0 = 0, where α is set dynamically to higher values if new measurements φn are more different from the previously optimized map features ¯φn−1 and lower otherwise: α = CosineSimilarity(φi, ¯φi)/(0.5 + CosineSimilarity(φi, ¯φi))."

    Eq. (1) optimizes the map to match the target φ∗. By Eq. (4), that target is constructed from ̅φ_{n−1}, the map's own previously reconstructed features, so the supervision signal for the next mapping step contains the current map state. Any error already stored in the octree/decoder — e.g., the painting-as-wall and bed-as-sofa inconsistencies the paper itself names in Sec. 1 — is re-entered into the loss as part of the desired output. At the optimization fixed point, the map regresses onto its own recurrent state (̅φ = α̅φ + (1−α)φ), rather than onto an independent target.

full rationale

No self-citation chain is load-bearing here: the paper does not invoke the authors' own prior theorems to force its choices, and all benchmark comparisons are evaluated on Matterport3D. The main Table 2 comparison is a per-scene reconstruction benchmark in which the map is allowed to be fit to the observed scene; this is the intended incremental-mapping setting rather than a hidden claim of generalization to held-out scenes. Likewise, Table 3 shows that a decoder optimized on the scene's labels beats fixed pretrained heads, which is an expected consequence of online adaptation rather than a concealed re-derivation. The one genuine circular element is the Measurement Update Strategy: because the target in Eq. (1) is defined using the map's own previous output (Eq. 4), the training signal is self-referential and stored errors can be replayed. This is a partial feedback circularity that warrants a score of 4, while the rest of the system still has independent content and external benchmarks.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical entity; it relies on standard octree feature grids and an MLP decoder. The main ledger burden is hyperparameters chosen without ablations, plus the self-referential smoothing target in Eq. (4) and the assumption that CLIP cosine similarity is a reliable semantic distance.

free parameters (5)
  • Cosine similarity threshold tau = 0.02
    Used to identify unique and unknown language features in Algorithm 1; no sensitivity analysis on mapping quality is provided, only on the final stored feature count.
  • Denominator constant in adaptive alpha = 0.5
    Appears in Eq. (5) and sets the smoothing rate; the paper does not ablate this value or justify it beyond empirical preference.
  • Octree levels and fine resolution = levels 8, 9, 10; 0.05 m
    Positional encoding hyperparameters chosen in Table 1; no study of their effect on language map quality.
  • Learnable feature dimensions = m = 16, L = 512
    Dimensions of corner features and per-voxel F vectors; chosen values are reported without ablation.
  • Optimization iterations = Nopt = 100; mapping loss iterations = 100
    Iteration counts in Table 1; no sensitivity analysis is reported.
assumptions (5)
  • domain assumption CLIP feature space supports cosine similarity as a reliable semantic distance for clustering and reconstructing language features.
    Cosine similarity is used in Eq. (1), in the unique/unknown feature detection of Algorithm 1, and implicitly in the accuracy metric of Sec. 4.1.
  • ad hoc to paper The map's own previous reconstruction is a reliable target for blending with new observations.
    Eq. (4) defines the training target as a weighted sum of the new observation and the map's current reconstruction; this self-referential choice can propagate early errors.
  • domain assumption External camera poses are accurate and provided by an external SLAM method.
    The input pipeline projects depth and language features using camera poses; no sensitivity to pose noise is tested.
  • domain assumption Language features of the same object are static and consistent across frames except for noise.
    The measurement update strategy in Sec. 3.3 assumes frame-to-frame variation is noise rather than a change in the environment or viewpoint semantics.
  • domain assumption The selected octree levels 8 to 10 with 0.05 m resolution are sufficient for the scenes considered.
    This scaling choice in Table 1 constrains the achievable detail and scale of the implicit map; no resolution study is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LiLMaps: Learnable Implicit Language Maps." pith.science (2026). https://pith.science/paper/LY5KV54F

@misc{pith2026250103304,
  author       = {Pith},
  title        = {Pith review of: LiLMaps: Learnable Implicit Language Maps},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LY5KV54F}},
  note         = {Machine review of arXiv:2501.03304}
}
read the original abstract

One of the current trends in robotics is to employ large language models (LLMs) to provide non-predefined command execution and natural human-robot interaction. It is useful to have an environment map together with its language representation, which can be further utilized by LLMs. Such a comprehensive scene representation enables numerous ways of interaction with the map for autonomously operating robots. In this work, we present an approach that enhances incremental implicit mapping through the integration of vision-language features. Specifically, we (i) propose a decoder optimization technique for implicit language maps which can be used when new objects appear on the scene, and (ii) address the problem of inconsistent vision-language predictions between different viewing positions. Our experiments demonstrate the effectiveness of LiLMaps and solid improvements in performance.

Figures

Figures reproduced from arXiv: 2501.03304 by the authors.

Figure 1
Figure 1. Reconstructed implicit language map built with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Implicit language mapping. Vision-language features [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Left: Environments reconstructed without measure￾ment update; Middle: Ground Truth; Right: Environments recon￾structed with measurement update. In this work, we propose to use as target for training in Eq. (1) a weighted average φ ∗ n between observations φn and the features φ¯n−1 already stored in the map: φ ∗ n = n − 1 n φ¯n−1 + φn n , with φ¯0 = 0. (3) This averaging is especially useful for noisy measurements su… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Left: Language map produced by OpenScene 3D [31]; Middle: Ground Truth; Right: Language map created by LiLMaps. is defined as the number of points with correctly recon￾structed language features divided by the total number of points. The values are compared with the Op…
Figure 5
Figure 5. Figure 5: Language map incrementally created with our adaptive [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Number of language features stored for the adaptive [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: 3D language-based object detection performed on our [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 39 canonical work pages

  1. [1]

    Do as I can, not as I say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 2

    arXiv 2022

  2. [2]

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko S ¨underhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3674–3683, 2018. 2

    work page 2018

  3. [3]

    CodeSLAM – learn- ing a compact, optimisable representation for dense visual SLAM

    Michael Bloesch, Jan Czarnowski, Ronald Clark, Stefan Leutenegger, and Andrew J Davison. CodeSLAM – learn- ing a compact, optimisable representation for dense visual SLAM. In IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 2560–2568, 2018. 2

    work page 2018

  4. [4]

    nuScenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuScenes: A multi- modal dataset for autonomous driving. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 11621–11631, 2020. 7, 8

    work page 2020

  5. [5]

    Matterport3D: Learning from rgb- d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Hal- ber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from rgb- d data in indoor environments. International Conference on 3D Vision (3DV), pages 667–676, 2017. 1, 5, 6, 7, 8

    work page 2017

  6. [6]

    Open-vocabulary queryable scene representations for real world planning

    Boyuan Chen, Fei Xia, Brian Ichter, Kanishka Rao, Keerthana Gopalakrishnan, Michael S Ryoo, Austin Stone, and Daniel Kappler. Open-vocabulary queryable scene representations for real world planning. In IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 11509–11522, 2023. 2

    work page 2023

  7. [7]

    Leveraging large language models for robot 3D scene un- derstanding

    William Chen, Siyi Hu, Rajat Talak, and Luca Carlone. Leveraging large language models for robot 3D scene un- derstanding. arXiv preprint arXiv:2209.05629, 2022. 2

    arXiv 2022

  8. [8]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas A

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas A. Funkhouser, and Matthias Nießner. Scan- Net: Richly-annotated 3d reconstructions of indoor scenes. In IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 2432–2443, 2017. 7, 8

    work page 2017

Show all 46 references
  1. [9]

    Plan-Seq-Learn: Language model guided RL for solving long horizon robotics tasks

    Murtaza Dalal, Tarun Chiruvolu, Devendra Chaplot, and Ruslan Salakhutdinov. Plan-Seq-Learn: Language model guided RL for solving long horizon robotics tasks. In 12th International Conference on Learning Representations (ICLR), 2024. 2

    2024

  2. [10]

    Scaling open-vocabulary image segmentation with image- level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scaling open-vocabulary image segmentation with image- level labels. In European Conference on Computer Vision (ECCV), pages 540–557. Springer, 2022. 2

    2022

  3. [11]

    Semantic Abstraction: Open- world 3D scene understanding from 2D vision-language models

    Huy Ha and Shuran Song. Semantic Abstraction: Open- world 3D scene understanding from 2D vision-language models. In Conference on Robot Learning (CoRL) , volume 205 of Proceedings of Machine Learning Research , pages 643–653. PMLR, 2022. 2

    2022

  4. [12]

    Iain Haughton, Edgar Sucar, Andr ´e Mouton, Edward Johns, and Andrew J. Davison. Real-time mapping of physical scene properties with an autonomous robot experimenter. In Conference on Robot Learning (CoRL) , volume 205 of Proceedings of Machine Learning Research, pages 118–127...

    2022

  5. [13]

    Bridg- ing the gap between learning in discrete and continuous envi- ronments for vision-and-language navigation

    Yicong Hong, Zun Wang, Qi Wu, and Stephen Gould. Bridg- ing the gap between learning in discrete and continuous envi- ronments for vision-and-language navigation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15439–15449, 2022. 2

    2022

  6. [14]

    Toward general-purpose robots via foundation models: A survey and meta-analysis

    Yafei Hu, Quanting Xie, Vidhi Jain, Jonathan Francis, Jay Patrikar, Nikhil Keetha, Seungchan Kim, Yaqi Xie, Tianyi Zhang, Zhibo Zhao, et al. Toward general-purpose robots via foundation models: A survey and meta-analysis. arXiv preprint arXiv:2312.08782, 2023. 2

    2023 arXiv

  7. [15]

    Visual language maps for robot navigation

    Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In IEEE International Conference on Robotics and Automation (ICRA), 2023. 2, 3, 5, 7

    2023

  8. [16]

    Omama, Ganesh Iyer, Soroush Saryazdi, Tao Chen, Alaa Maalouf, Shuang Li, Nikhil Varma Keetha, Ayush Tewari, Joshua B

    Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd. Omama, Ganesh Iyer, Soroush Saryazdi, Tao Chen, Alaa Maalouf, Shuang Li, Nikhil Varma Keetha, Ayush Tewari, Joshua B. Tenenbaum, Celso Miguel de Melo, K. Madhava Krishna, Liam Paull, Florian Shkurti, and Anto...

    2023

  9. [17]

    3D Gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3D Gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics (TOG), 42(4):139–1, 2023. 2, 3

    2023

  10. [18]

    LERF: Language embed- ded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. LERF: Language embed- ded radiance fields. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 19729–19739, 2023. 2

    2023

  11. [19]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 4015–4026, 2023. 2

    2023

  12. [20]

    Weinberger, Serge J

    Boyi Li, Kilian Q. Weinberger, Serge J. Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. In 10th International Conference on Learning Representations (ICLR), 2022. 2

    2022

  13. [21]

    GS-Octree: Octree-based 3D Gaussian splatting for robust object-level 3D reconstruction under strong lighting

    Jiaze Li, Zhengyu Wen, Luo Zhang, Jiangbei Hu, Fei Hou, Zhebin Zhang, and Ying He. GS-Octree: Octree-based 3D Gaussian splatting for robust object-level 3D reconstruction under strong lighting. Computer Graphics Forum (CGF) , 43(7):i–xxii, 2024. 3

    2024

  14. [22]

    Per-pixel features: Mating segment-anything with CLIP, 2023

    Ming-Feng Li. Per-pixel features: Mating segment-anything with CLIP, 2023. 2, 3

    2023

  15. [23]

    Interactive language: Talking to robots in real time

    Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters (RA-L) , 2023. 2

    2023

  16. [24]

    Gaussian splatting SLAM

    Hidenobu Matsuki, Riku Murai, Paul HJ Kelly, and Andrew J Davison. Gaussian splatting SLAM. In IEEE/CVF Confer- 9 ence on Computer Vision and Pattern Recognition (CVPR) , pages 18039–18048, 2024. 2

    2024

  17. [25]

    Feature- realistic neural fusion for real-time, open set scene under- standing

    Kirill Mazur, Edgar Sucar, and Andrew J Davison. Feature- realistic neural fusion for real-time, open set scene under- standing. In IEEE International Conference on Robotics and Automation (ICRA), pages 8201–8207, 2023. 2, 3

    2023

  18. [26]

    NeRF: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 2, 3

    2021

  19. [27]

    Instant neural graphics primitives with a mul- tiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a mul- tiresolution hash encoding. ACM Transactions on Graphics (TOG), 41(4):1–15, 2022. 3

    2022

  20. [28]

    NeRF-VO: Real-time sparse visual odometry with neural radiance fields

    Jens Naumann, Binbin Xu, Stefan Leutenegger, and Xingx- ing Zuo. NeRF-VO: Real-time sparse visual odometry with neural radiance fields. IEEE Robotics and Automation Let- ters (RA-L), 9(8):7278–7285, 2024. 2

    2024

  21. [29]

    Foundation model based open vocabulary task planning and executive system for general purpose service robots

    Yoshiki Obinata, Naoaki Kanazawa, Kento Kawaharazuka, Iori Yanokura, Soonhyo Kim, Kei Okada, and Masayuki In- aba. Foundation model based open vocabulary task planning and executive system for general purpose service robots. arXiv preprint arXiv:2308.03357, 2023. 2

    2023 arXiv

  22. [30]

    iSDF: Real-time neural signed distance fields for robot per- ception

    Joseph Ortiz, Alexander Clegg, Jing Dong, Edgar Sucar, David Novotn´y, Michael Zollh ¨ofer, and Mustafa Mukadam. iSDF: Real-time neural signed distance fields for robot per- ception. In Robotics: Science and Systems XVIII (RSS) ,

  23. [31]

    OpenScene: 3D scene understanding with open vocabular- ies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. OpenScene: 3D scene understanding with open vocabular- ies. In IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 815–824, 2023. 2, 6, 7

    2023

  24. [32]

    LangSplat: 3D language Gaussian splat- ting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. LangSplat: 3D language Gaussian splat- ting. In IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 20051–20060, 2024. 2

    2024

  25. [33]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  26. [34]

    Planning with large language models via corrective re-prompting

    Shreyas Sundara Raman, Vanya Cohen, Eric Rosen, Ifrah Idrees, David Paulius, and Stefanie Tellex. Planning with large language models via corrective re-prompting. In NeurIPS Foundation Models for Decision Making Workshop (FMDM), 2022. 2

    2022

  27. [35]

    Per- ceptual grouping in contrastive vision-language models

    Kanchana Ranasinghe, Brandon McKinzie, Sachin Ravi, Yinfei Yang, Alexander Toshev, and Jonathon Shlens. Per- ceptual grouping in contrastive vision-language models. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 5548–5561, 2023. 2

    2023

  28. [36]

    KiloNeRF: Speeding up neural radiance fields with thousands of tiny MLPs

    Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. KiloNeRF: Speeding up neural radiance fields with thousands of tiny MLPs. InIEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 14335–14345, 2021. 2, 3

    2021

  29. [37]

    LLM-Planner: Few-shot grounded planning for embodied agents with large language models

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. LLM-Planner: Few-shot grounded planning for embodied agents with large language models. In IEEE/CVF International Conference on Com- puter Vision (ICCV), pages 2998–3009, 2023. 2

    2023

  30. [38]

    iMAP: Implicit mapping and positioning in real-time

    Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew J Davi- son. iMAP: Implicit mapping and positioning in real-time. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 6229–6238, 2021. 2

    2021

  31. [39]

    Neural geometric level of detail: Real-time rendering with implicit 3D shapes

    Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacob- son, Morgan McGuire, and Sanja Fidler. Neural geometric level of detail: Real-time rendering with implicit 3D shapes. In IEEE/CVF Conference on Computer Vision and Patte...

    2021

  32. [40]

    GO- Surf: Neural feature grid optimization for fast, high-fidelity RGB-D surface reconstruction

    Jingwen Wang, Tymoteusz Bleja, and Lourdes Agapito. GO- Surf: Neural feature grid optimization for fast, high-fidelity RGB-D surface reconstruction. In International Conference on 3D Vision (3DV), pages 433–442. IEEE, 2022. 3

    2022

  33. [41]

    SAM3D: Segment anything in 3D scenes

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. SAM3D: Segment anything in 3D scenes. arXiv preprint arXiv:2306.03908, 2023. 2

    2023 arXiv

  34. [42]

    iLabel: Interactive neural scene labelling

    Shuaifeng Zhi, Edgar Sucar, Andre Mouton, Iain Haughton, Tristan Laidlow, and Andrew J Davison. iLabel: Interactive neural scene labelling. arXiv preprint arXiv:2111.14637 ,

    arXiv

  35. [43]

    SHINE-Mapping: Large-scale 3D mapping us- ing sparse hierarchical implicit neural representations

    Xingguang Zhong, Yue Pan, Jens Behley, and Cyrill Stach- niss. SHINE-Mapping: Large-scale 3D mapping us- ing sparse hierarchical implicit neural representations. In IEEE International Conference on Robotics and Automation (ICRA), pages 8371–8377, 2023. 2

    2023

  36. [44]

    LoopSplat: Loop closure by registering 3D Gaussian splats

    Liyuan Zhu, Yue Li, Erik Sandstr ¨om, Konrad Schindler, and Iro Armeni. LoopSplat: Loop closure by registering 3D Gaussian splats. arXiv preprint arXiv:2408.10154, 2024. 2

    2024 arXiv

  37. [45]

    SemGauss-SLAM: Dense semantic Gaus- sian splatting slam

    Siting Zhu, Renjie Qin, Guangming Wang, Jiuming Liu, and Hesheng Wang. SemGauss-SLAM: Dense semantic Gaus- sian splatting slam. arXiv preprint arXiv:2403.07494, 2024. 2

    2024 arXiv

  38. [46]

    NICE-SLAM: Neural implicit scalable encoding for SLAM

    Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hu- jun Bao, Zhaopeng Cui, Martin R Oswald, and Marc Polle- feys. NICE-SLAM: Neural implicit scalable encoding for SLAM. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12786–12796, 2022. 2 10

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.