Pith. sign in

REVIEW 5 major objections 5 minor 50 references

BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read BYE shows that a robot can learn to re-identify moved objects from a single exploration pass, reaching 95.6% association accuracy in simulation and 100% in real-world tests.

desk verdict A solid per-scene object-association method whose claims are narrower than the abstract suggests; the reported numbers rely on oracle masks/odometry, and the paper would benefit from variance and sensitivity reporting. read the letter →

arxiv 2412.02449 v1 pith:E7T3RVXJ submitted 2024-12-03 cs.RO cs.AIcs.CVcs.LG

Chenguang Huang , Shengchao Yan , Wolfram Burgard This is my paper · ORCID
classification cs.ROcs.AIcs.CVcs.LG
keywords objectassociationlong-termdynamicscenesper-scenepointcloudencodercontrastivelearninginstance-levelmappingmemorybankvision-languagemodelensemblingrobotexploration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a robot can build a usable instance-level object memory of a scene from one exploratory pass, then use it to re-identify objects after they have been moved, without predefined object categories, shape priors, or large association datasets. The method, called BYE, trains a small point-cloud encoder on partial views of each object seen during that single pass, using contrastive learning to pull views of the same object together and push different objects apart. At query time, embeddings of new partial views are matched by nearest neighbours in a memory bank of reference embeddings. The authors report 95.6% association success in the AI2THOR simulator and 100% in their real-world tabletop and furniture layouts, outperforming pure vision-language-model baselines by about 7 percentage points.

What carries the argument

The load-bearing mechanism is a per-scene point cloud encoder trained with a contrastive loss. From one reference exploration, each instance mask is back-projected through depth and odometry into a partial point cloud (3D coordinates plus colour), centred at its mean; these partial views, labelled by instance ID, are the training set. Following the contrastive scheme of [43], the encoder is trained with the NT-Xent loss [45] to pull views of the same object together and push views of different objects apart in a 128-dimensional embedding space, and the projection head used during training is discarded at inference. All reference embeddings are stored with their instance IDs as an object memory bank. At query time, a new partial observation is embedded, its 10 nearest neighbours in the bank are found, and the reference ID with the most votes is the association; the ensemble variant sums this score matrix with a vision-language-model cosine-similarity matrix and applies a standard one-to-one assignment step.

What would settle it

Run BYE on a reference trial whose per-object outlines are deliberately corrupted, for example one object split into two outlines or two objects merged into one, and measure association accuracy; a collapse to baseline levels would show the one-sequence claim depends on near-perfect segmentation rather than on the encoder itself.

Watch

Extended reading notes

Core claim

BYE establishes that instance-level object association across scene changes can be learned from one sequence of exploration data alone, without categories, shape priors, or synthetic association datasets. The paper's claim is that partial point cloud observations of objects in a single reference trial are sufficient training signal: contrastive learning makes the encoder produce embeddings that are stable under viewpoint changes but discriminative between instances. The resulting memory bank, combined by score averaging with a vision-language-model semantic similarity matrix and resolved by a one-to-one assignment step, yields high association success in the tested settings: 95.6% in AI2THOR across 252 movable objects and 100% in the real-world scenarios. The authors present this as evidence that a lightweight scene-specific expert model can complement general foundation-model semantics for long-term scene understanding.

Load-bearing premise

The load-bearing assumption is that the reference exploration comes with accurate per-object outlines and camera poses, and that the changed scene contains only moved objects, with nothing added or removed.

Editorial extensions

If this is right

  • If the central claim holds, a robot that has explored a room once can later return and correctly say which objects were moved, without ever being told object categories or shapes.
  • The training pipeline removes the need for large synthetic association datasets, so each new environment can get its own expert encoder from its own exploration data.
  • At roughly 11 ms per sample, the association can run online during a new exploration, so a robot could update its map while moving.
  • The reported gain from ensembling shows that foundation-model semantics and scene-specific instance features carry complementary information, so combining them is more reliable than either in isolation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same contrastive recipe could be reapplied incrementally as a robot revisits a scene, turning one-shot memory into a lifelong object-memory update mechanism.
  • Inference: because the paper's evaluation fixes the object set, a natural extension is to treat low nearest-neighbour affinity in the memory bank as a novelty signal for detecting newly introduced objects.
  • Inference: the per-class results suggest the largest gains over foundation models come on small or long-tailed objects; a targeted stress test with many duplicate objects of the same category would show how far the geometric and colour cues alone can go.
  • Inference: if the method is deployed with real perception, its sensitivity to mask and odometry errors should be measured, since the reference labels come from masks and odometry rather than from the encoder itself.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents BYE, a per-scene point cloud encoder trained on a single exploration sequence for instance-level object association in changing scenes. From RGB-D plus instance masks and odometry, the authors build an instance-level point cloud map, generate partial point cloud observations, train a PointNet/DGCNN encoder with a SimCLR-style contrastive loss, and assemble a memory bank of embeddings for all reference observations. At test time, new partial observations are embedded and matched to the memory bank by k-NN voting, optionally fused with CLIP-based instance features via a score-matrix sum and Hungarian assignment. Experiments in AI2THOR (252 movable objects) report 95.6% association success for BYE (PointNet+CLIP) versus 88.9% for CLIP alone, and real-world tabletop/furniture experiments report 100% success for the ensemble. The authors release code and data and claim a 7% improvement over foundation-model baselines.

Significance. The core idea—training a scene-specific, class-agnostic encoder from one exploration pass—is a useful and low-cost alternative to association methods that require synthetic datasets and category priors. The paper includes a fair set of foundation-model baselines, a real-world deployment with SAM2 and DROID-SLAM, runtime numbers, and a commitment to releasing code and data. The claimed 95.6%/100% accuracy is encouraging but currently qualified by the oracle-input assumption in simulation, the absence of perceptual-noise sensitivity analysis, a possibly biased voting scheme, and missing details in the ensemble and loss specification. If these points are addressed, the method could be a solid practical contribution to instance re-identification for long-term scene understanding, although the restriction to pure relocations (no added/removed objects) should be reflected in the claims.

major comments (5)
  1. [Sec. IV-A and Sec. V-B] The method assumes known instance masks and odometry for the reference trial (Sec. IV-A), and the main simulation results in Table I therefore use oracle perception from AI2THOR. In the real-world experiments (Sec. V-B), masks and poses are produced by SAM2 and DROID-SLAM, but the paper never reports the accuracy of these components and does not study how mask splits, merges, or mislabels, or pose errors, affect the contrastive training labels and the memory-bank votes. Since the training labels and the memory bank are both constructed from these inputs, the headline success rates do not yet establish robust association under imperfect perception. Please add a sensitivity analysis with corrupted masks/poses, or at least report the measured mask and odometry errors in the real-world trials.
  2. [Sec. IV-E] The ensemble matrix A = A^BYE + A^VLM sums A^BYE, whose entries are probabilities in [0,1], with A^VLM, whose entries are raw cosine similarities roughly in [-1,1], without stating any normalization or weighting between the two matrices. Unless the scales are calibrated, one term will dominate the Hungarian assignment, and the claimed 7% improvement from ensembling is not well-defined. Please specify the normalization/weights and, ideally, ablate the combination weight or report the individual matrices.
  3. [Sec. IV-B and Sec. IV-E] The memory bank contains a variable number R_i of partial observations per instance, and the k-NN voting in Sec. IV-E counts the reference labels among the 10 nearest neighbors with equal weight. Because R_i depends on object size, visibility, and trajectory coverage, the frequency estimate P(f(j)=i|z_{t=1:n}) is biased toward instances with more observations in the memory bank. Please normalize the vote counts by R_i (or use per-instance prototypes) and report whether this changes the results in Table I.
  4. [Sec. IV-C, Eq. (1)] Equation (1) is not internally consistent: 'exp(gi, g+)/τ' is not a defined operation, and the temperature τ is never specified anywhere in the training details. Since the NT-Xent loss is the central training objective, the formula should be corrected (e.g., exp(sim(gi,g+)/τ)) and the value of τ should be reported. Without this information the training pipeline is not fully reproducible.
  5. [Sec. V, Tables I-II] All association success rates are single runs without repeated training seeds, confidence intervals, or per-scene breakdowns, and the real-world evaluation has only 62 association tasks. Given the small margins in Table I (for example, BYE (PointNet) at 85.7% versus CLIP at 88.9% before ensembling), the statistical significance of the reported improvements is unclear. Please provide repeated-seed statistics or confidence intervals, and report the per-scene results that are currently averaged.
minor comments (5)
  1. [Sec. IV-E] The probability formula P(f(j)=i|z_{t=1:n}) treats all observations of a new object equally, but no justification is given for why a simple frequency count is appropriate; a short discussion of the i.i.d. assumption would help.
  2. [Sec. V-B] The real-world setup is described only briefly; please state how SAM2 masks were linked into consistent instance IDs across frames, how the ground-truth correspondences for the 62 association tasks were obtained, and whether the one training trial per setting was captured under the same conditions as the test trials.
  3. [Sec. V-C, Table III] The runtime table reports that BYE (PointNet) is slower than BYE (DGCNN) at batch size 1 (19.7 ms vs 16.7 ms per sample); this is unexpected and deserves a one-sentence explanation or a correctness check.
  4. [Fig. 2 and Fig. 3] The text in Figures 2 and 3 is very small and hard to read; please enlarge the fonts and increase the figure resolution.
  5. [Sec. VI] The conclusion is honest about the main limitations, but the abstract and title use 'long-term dynamic scene understanding' while the evaluated scenario only covers relocations of a fixed object set; please align the claims with the actual scope.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BYE's per-scene training and KNN retrieval are evaluated on new trials after object relocation, with independent foundation-model baselines.

full rationale

The paper's derivation chain is self-contained rather than circular. Given a reference exploration trial with RGB, depth, instance masks, and odometry, the method constructs partial point cloud observations (Sec. IV-B), trains a PointNet/DGCNN encoder from scratch with SimCLR-style contrastive learning using instance IDs as labels (Sec. IV-C), builds an object memory bank from the same reference observations (Sec. IV-D), and then retrieves associations for new-trial observations by KNN voting plus an optional CLIP-based score matrix (Sec. IV-E). The test data are not the same observations used for training: they come from a separate trial after objects have been relocated, so the encoder must generalize to new partial views and layouts. The reported success rates are therefore not forced by construction; the comparison against zero-shot foundation models (CLIP, LSeg, OVSeg, DINOv2) provides independent grounding. The paper honestly states its limitations in Sec. VI, including dependence on instance masks and inability to handle added or removed objects; these are robustness and scope concerns, not evidence of circularity. Self-citations, notably to HOV-SG for open-vocabulary feature construction, are descriptive components rather than load-bearing uniqueness claims or imported constraints. No equation or fitted parameter is renamed as a prediction, and no central result reduces to its own inputs.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim relies on several assumptions: clean per-frame instance masks (which provide the labels), static objects during a trial, and an evaluation restricted to relocations. The contrastive training itself is the main 'free' component; hyperparameters such as voxel size, point count, rotation augmentation, KNN k, and the unreported temperature tau are hand-chosen and not derived. No new physical entities are introduced.

free parameters (5)
  • voxel downsampling resolution = 0.01 m
    Chosen by hand in Sec IV-B; affects point density in training and memory bank construction.
  • farthest point sampling target = 1024 points
    Chosen in Sec IV-C to balance workload across object sizes.
  • rotation augmentation range = 0 to 30 degrees per axis
    Chosen in Sec IV-C; bounds the encoder's rotation invariance, a key property for matching after relocation.
  • memory bank KNN k = 10
    Chosen in Sec IV-E; controls vote aggregation for association.
  • contrastive temperature tau = not reported
    Appears in the NT-Xent loss (Eq (1)) but is never assigned a value, leaving an unstated free hyperparameter that re-implementers must choose.
assumptions (5)
  • domain assumption Instance masks for the reference trial are correct and instance-consistent across frames.
    Sec IV-A: 'we assume known instance masks and odometry as input'; these masks define the contrastive labels in Sec IV-B and Sec IV-C.
  • domain assumption Objects remain static during each exploration trial.
    Sec III: 'each object remains static during a single trial'; otherwise fused instance maps would be corrupted.
  • domain assumption Only object locations change between reference and new trials; no objects are added or removed.
    Sec IV-E restricts to relocation 'without adding new objects or removing old ones'; Sec VI admits the method does not handle additions or removals.
  • domain assumption Object identity is determined by local geometry and color in a translation-invariant way.
    Sec IV-B centers point clouds at their means; the encoder has no absolute position input, so the claim depends on position-independent object appearance.
  • ad hoc to paper Partial observations from the reference trial are sufficient to learn an embedding that generalizes to new trials.
    The core hypothesis of Sec IV-C; the encoder is trained only on reference-trial data and must transfer to new viewpoints and poses.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding." pith.science (2026). https://pith.science/paper/E7T3RVXJ

@misc{pith2026241202449,
  author       = {Pith},
  title        = {Pith review of: BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E7T3RVXJ}},
  note         = {Machine review of arXiv:2412.02449}
}
read the original abstract

Dynamic scene understanding remains a persistent challenge in robotic applications. Early dynamic mapping methods focused on mitigating the negative influence of short-term dynamic objects on camera motion estimation by masking or tracking specific categories, which often fall short in adapting to long-term scene changes. Recent efforts address object association in long-term dynamic environments using neural networks trained on synthetic datasets, but they still rely on predefined object shapes and categories. Other methods incorporate visual, geometric, or semantic heuristics for the association but often lack robustness. In this work, we introduce BYE, a class-agnostic, per-scene point cloud encoder that removes the need for predefined categories, shape priors, or extensive association datasets. Trained on only a single sequence of exploration data, BYE can efficiently perform object association in dynamically changing scenes. We further propose an ensembling scheme combining the semantic strengths of Vision Language Models (VLMs) with the scene-specific expertise of BYE, achieving a 7% improvement and a 95% success rate in object association tasks. Code and dataset are available at https://byencoder.github.io.

Figures

Figures reproduced from arXiv: 2412.02449 by the authors.

Figure 1
Figure 1. BYE enables reliable and robust object association in long-term dy [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the pipeline of BYE for long-term dynamic environment understanding. With the reference trial of exploration data, we first build an [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The architecture of the point cloud encoder. We first follow the architecture of DGCNN [ [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The process of querying the object memory bank with new exploration [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The tabletop and furniture setups in the real world. We take one [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The qualitative object association results in furniture setup in the real [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 25 canonical work pages

  1. [1]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0,

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain et al., “Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 6892–6903

    work page 2024

  2. [2]

    Co-fusion: Real-time segmentation, tracking and fusion of multiple objects,

    M. R ¨unz and L. Agapito, “Co-fusion: Real-time segmentation, tracking and fusion of multiple objects,” in 2017 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2017, pp. 4471–4478

    work page 2017

  3. [3]

    Dynaslam: Tracking, mapping, and inpainting in dynamic scenes,

    B. Bescos, J. M. F ´acil, J. Civera, and J. Neira, “Dynaslam: Tracking, mapping, and inpainting in dynamic scenes,” IEEE Robotics and Au- tomation Letters , vol. 3, no. 4, pp. 4076–4083, 2018

    work page 2018

  4. [4]

    Ds- slam: A semantic visual slam towards dynamic environments,

    C. Yu, Z. Liu, X.-J. Liu, F. Xie, Y . Yang, Q. Wei, and Q. Fei, “Ds- slam: A semantic visual slam towards dynamic environments,” in 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2018, pp. 1168–1174

    work page 2018

  5. [5]

    Mid-fusion: Octree-based object-level multi-instance dynamic slam,

    B. Xu, W. Li, D. Tzoumanikas, M. Bloesch, A. Davison, and S. Leutenegger, “Mid-fusion: Octree-based object-level multi-instance dynamic slam,” in 2019 International Conference on Robotics and Automation (ICRA) . IEEE, 2019, pp. 5231–5237

    work page 2019

  6. [6]

    Objects can move: 3d change detection by geometric transformation consistency,

    A. Adam, T. Sattler, K. Karantzalos, and T. Pajdla, “Objects can move: 3d change detection by geometric transformation consistency,” in European Conference on Computer Vision . Springer, 2022, pp. 108– 124

    work page 2022

  7. [7]

    Has anything changed? 3d change detection by 2d segmentation masks,

    A. Adam, K. Karantzalos, L. Grammatikopoulos, and T. Sattler, “Has anything changed? 3d change detection by 2d segmentation masks,” arXiv preprint arXiv:2312.01148 , 2023. 8

    arXiv 2023

  8. [8]

    Living scenes: Multi- object relocalization and reconstruction in changing 3d environments,

    L. Zhu, S. Huang, K. Schindler, and I. Armeni, “Living scenes: Multi- object relocalization and reconstruction in changing 3d environments,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 28 014–28 024

    work page 2024

Show all 50 references
  1. [9]

    Shapenet: An information- rich 3d model repository,

    A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su et al., “Shapenet: An information- rich 3d model repository,” arXiv preprint arXiv:1512.03012 , 2015

    2015 arXiv

  2. [10]

    Langsplat: 3d language gaussian splatting,

    M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister, “Langsplat: 3d language gaussian splatting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 20 051–20 060

    2024

  3. [11]

    3d gaussian splatting for real-time radiance field rendering

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering.” ACM Trans. Graph. , vol. 42, no. 4, pp. 139–1, 2023

    2023

  4. [12]

    Ai2-thor: An interactive 3d environment for visual ai,

    E. Kolve, R. Mottaghi, W. Han, E. VanderBilt, L. Weihs, A. Herrasti, M. Deitke, K. Ehsani, D. Gordon, Y . Zhuet al., “Ai2-thor: An interactive 3d environment for visual ai,” arXiv preprint arXiv:1712.05474 , 2017

    2017 arXiv

  5. [13]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  6. [14]

    Open-vocabulary semantic segmentation with mask-adapted clip,

    F. Liang, B. Wu, X. Dai, K. Li, Y . Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu, “Open-vocabulary semantic segmentation with mask-adapted clip,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 7061–7070

    2023

  7. [15]

    Scaling open-vocabulary image segmentation with image-level labels,

    G. Ghiasi, X. Gu, Y . Cui, and T.-Y . Lin, “Scaling open-vocabulary image segmentation with image-level labels,” in European Conference on Computer Vision . Springer, 2022, pp. 540–557

    2022

  8. [16]

    Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action,

    D. Shah, B. Osi ´nski, S. Levine et al., “Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action,” in Conference on Robot Learning . PMLR, 2023, pp. 492–504

    2023

  9. [17]

    Visual language maps for robot navigation,

    C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual language maps for robot navigation,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) , London, UK, 2023

    2023

  10. [18]

    Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning,

    Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa et al., “Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning,” in 2024 IEEE International Conference on Robotics and Automation (ICRA)...

    2024

  11. [19]

    Audio visual language maps for robot navigation,

    C. Huang, O. Mees, A. Zeng, and W. Burgard, “Audio visual language maps for robot navigation,” in Proceedings of the International Sympo- sium on Experimental Robotics (ISER) , Chiang Mai, Thailand, 2023

    2023

  12. [20]

    Hi- erarchical Open-V ocabulary 3D Scene Graphs for Language-Grounded Robot Navigation,

    A. Werby, C. Huang, M. B ¨uchner, A. Valada, and W. Burgard, “Hi- erarchical Open-V ocabulary 3D Scene Graphs for Language-Grounded Robot Navigation,” in Proceedings of Robotics: Science and Systems , Delft, Netherlands, July 2024

    2024

  13. [21]

    V oxposer: Composable 3d value maps for robotic manipulation with language models,

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei, “V oxposer: Composable 3d value maps for robotic manipulation with language models,” arXiv preprint arXiv:2307.05973 , 2023

    2023 arXiv

  14. [22]

    Lerf: Language embedded radiance fields,

    J. Kerr, C. M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik, “Lerf: Language embedded radiance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 19 729–19 739

    2023

  15. [23]

    Openscene: 3d scene understanding with open vocabularies,

    S. Peng, K. Genova, C. M. Jiang, A. Tagliasacchi, M. Pollefeys, and T. Funkhouser, “Openscene: 3d scene understanding with open vocabularies,” 2023

    2023

  16. [24]

    Conceptfusion: Open-set multimodal 3d mapping,

    K. M. Jatavallabhula, A. Kuwajerwala, Q. Gu, M. Omama, T. Chen, S. Li, G. Iyer, S. Saryazdi, N. Keetha, A. Tewari et al., “Conceptfusion: Open-set multimodal 3d mapping,” 2023

    2023

  17. [25]

    OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views,

    F. Engelmann, F. Manhardt, M. Niemeyer, K. Tateno, M. Pollefeys, and F. Tombari, “OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views,” in International Conference on Learning Representations , 2024

    2024

  18. [26]

    Garfield: Group anything with radiance fields,

    C. M. Kim, M. Wu, J. Kerr, K. Goldberg, M. Tancik, and A. Kanazawa, “Garfield: Group anything with radiance fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 21 530–21 539

    2024

  19. [27]

    Fusion++: V olumetric object-level slam,

    J. McCormac, R. Clark, M. Bloesch, A. Davison, and S. Leutenegger, “Fusion++: V olumetric object-level slam,” in2018 international confer- ence on 3D vision (3DV) . IEEE, 2018, pp. 32–41

    2018

  20. [28]

    Flowfusion: Dynamic dense rgb-d slam based on optical flow,

    T. Zhang, H. Zhang, Y . Li, Y . Nakamura, and L. Zhang, “Flowfusion: Dynamic dense rgb-d slam based on optical flow,” in 2020 IEEE international conference on robotics and automation (ICRA) . IEEE, 2020, pp. 7322–7328

    2020

  21. [29]

    Dynamicfusion: Recon- struction and tracking of non-rigid scenes in real-time,

    R. A. Newcombe, D. Fox, and S. M. Seitz, “Dynamicfusion: Recon- struction and tracking of non-rigid scenes in real-time,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 343–352

    2015

  22. [30]

    Dynamic 3d gaus- sians: Tracking by persistent dynamic view synthesis,

    J. Luiten, G. Kopanas, B. Leibe, and D. Ramanan, “Dynamic 3d gaus- sians: Tracking by persistent dynamic view synthesis,” arXiv preprint arXiv:2308.09713, 2023

    2023 arXiv

  23. [31]

    Phys- gaussian: Physics-integrated 3d gaussians for generative dynamics,

    T. Xie, Z. Zong, Y . Qiu, X. Li, Y . Feng, Y . Yang, and C. Jiang, “Phys- gaussian: Physics-integrated 3d gaussians for generative dynamics,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4389–4398

    2024

  24. [32]

    Robust change detection based on neural descriptor fields,

    J. Fu, Y . Du, K. Singh, J. B. Tenenbaum, and J. J. Leonard, “Robust change detection based on neural descriptor fields,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2022, pp. 2817–2824

    2022

  25. [33]

    Indoor scene change captioning based on multimodality data,

    Y . Qiu, Y . Satoh, R. Suzuki, K. Iwata, and H. Kataoka, “Indoor scene change captioning based on multimodality data,”Sensors, vol. 20, no. 17, p. 4761, 2020

    2020

  26. [34]

    3d vsg: Long-term semantic scene change prediction through 3d variable scene graphs,

    S. Looper, J. Rodriguez-Puigvert, R. Siegwart, C. Cadena, and L. Schmid, “3d vsg: Long-term semantic scene change prediction through 3d variable scene graphs,” in 2023 IEEE International Confer- ence on Robotics and Automation (ICRA) . IEEE, 2023, pp. 8179–8186

    2023

  27. [35]

    Khronos: A unified approach for spatio-temporal metric-semantic slam in dynamic envi- ronments,

    L. Schmid, M. Abate, Y . Chang, and L. Carlone, “Khronos: A unified approach for spatio-temporal metric-semantic slam in dynamic envi- ronments,” in Proc. of Robotics: Science and Systems (RSS) , Delft, Netherlands, July 2024

    2024

  28. [36]

    Pointnet: Deep learning on point sets for 3d classification and segmentation,

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 652–660

    2017

  29. [37]

    Dynamic graph cnn for learning on point clouds,

    Y . Wang, Y . Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Transactions on Graphics (tog) , vol. 38, no. 5, pp. 1–12, 2019

    2019

  30. [38]

    Deepsdf: Learning continuous signed distance functions for shape rep- resentation,

    J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape rep- resentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 165–174

    2019

  31. [39]

    Learning implicit fields for generative shape modeling,

    Z. Chen and H. Zhang, “Learning implicit fields for generative shape modeling,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 5939–5948

    2019

  32. [40]

    Coarse- to-fine point cloud registration with se (3)-equivariant representations,

    C.-W. Lin, T.-I. Chen, H.-Y . Lee, W.-C. Chen, and W. H. Hsu, “Coarse- to-fine point cloud registration with se (3)-equivariant representations,” in 2023 IEEE international conference on robotics and automation (ICRA). IEEE, 2023, pp. 2833–2840

    2023

  33. [41]

    Neural descriptor fields: Se (3)-equivariant object representations for manipulation,

    A. Simeonov, Y . Du, A. Tagliasacchi, J. B. Tenenbaum, A. Ro- driguez, P. Agrawal, and V . Sitzmann, “Neural descriptor fields: Se (3)-equivariant object representations for manipulation,” in 2022 Inter- national Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 6394–6400

    2022

  34. [42]

    Neuse: Neural se (3)-equivariant embedding for consistent spatial understanding with objects,

    J. Fu, Y . Du, K. Singh, J. B. Tenenbaum, and J. J. Leonard, “Neuse: Neural se (3)-equivariant embedding for consistent spatial understanding with objects,” in Proceedings of Robotics: Science and Systems (RSS) , 2023

    2023

  35. [43]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning . PMLR, 2020, pp. 1597–1607

    2020

  36. [44]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems , vol. 30, 2017

    2017

  37. [45]

    Improved deep metric learning with multi-class n-pair loss objective,

    K. Sohn, “Improved deep metric learning with multi-class n-pair loss objective,” Advances in neural information processing systems , vol. 29, 2016

    2016

  38. [46]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby et al. , “Dinov2: Learning robust visual features without supervision,” arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  39. [47]

    Language-driven semantic segmentation,

    B. Li, K. Q. Weinberger, S. Belongie, V . Koltun, and R. Ranftl, “Language-driven semantic segmentation,” in International Conference on Learning Representations , 2022. [Online]. Available: https: //openreview.net/forum?id=RriDjddCLN

    2022

  40. [48]

    Density-based spatial clustering of applications with noise,

    M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “Density-based spatial clustering of applications with noise,” in Int. Conf. knowledge discovery and data mining , vol. 240, no. 6, 1996

    1996

  41. [49]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R¨adle, C. Rolland, L. Gustafson et al., “Sam 2: Segment anything in images and videos,” arXiv preprint arXiv:2408.00714 , 2024

    2024 arXiv

  42. [50]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,

    Z. Teed and J. Deng, “Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,” Advances in neural information processing systems, vol. 34, pp. 16 558–16 569, 2021

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.