Pith. sign in

REVIEW 4 major objections 4 minor 42 references

Voxel-Aggregated Feature Synthesis: Efficient Dense Mapping for Simulated 3D Reasoning

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VAFS claims that dense 3D semantic mapping in simulation can be made an order of magnitude faster by embedding one synthetic view per object, and that the resulting map answers text queries more accurately than fusion-based baselines.

desk verdict A genuinely faster way to build CLIP maps in simulation, but the accuracy win is mostly inherited from the simulator's ground-truth segmentation rather than earned by the method. read the letter →

arxiv 2411.10616 v2 pith:ULJ7YMRJ submitted 2024-11-15 cs.CV

Owen Burns , Rizwan Qureshi This is my paper · ORCID
classification cs.CV
keywords dense3DmappingsemanticsimulationCLIPembeddingsvoxelaggregationviewsynthesisopen-setembodiedagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dense 3D semantic mapping normally embeds every RGB-D frame and fuses the results, which is redundant because consecutive frames overlap heavily. This paper proposes Voxel-Aggregated Feature Synthesis (VAFS), which instead asks the simulator for a segmented point cloud, renders one synthetic view per object, and embeds those views, reducing the number of embeddings from the number of frames to the number of objects. The paper claims this cuts runtime by an order of magnitude while raising IoU on text-based semantic queries, and it presents a comparison on a benchmark scene where VAFS finished in 189 seconds versus 1536 seconds for ConceptFusion and 5831 seconds for LeRF. A sympathetic reader would care because it makes frequently updated semantic maps practical for simulated embodied agents, which previously avoided dense mapping because of cost.

What carries the argument

The central mechanism is the synthetic per-object view synthesized by Algorithm 1. For each object, the algorithm collects the simulator's points, estimates an average normal, converts that normal to camera elevation and azimuth, and renders the object in isolation; this single image replaces the hundreds of overlapping frames that fusion-based methods would embed. The other load-bearing piece is the voxel aggregation of Equation (6): features of all points inside each voxel are summed and normalized, and the voxel position becomes the point centroid, so the map keeps a uniform density and explicitly represents the borders between objects. Together these pieces move the cost of mapping from vision, segmentation and embedding of every frame, to the simulator's physics engine, which already knows object membership.

What would settle it

Render a simulated object with clearly distinct parts, such as a chair with a fabric backrest and a wooden seat, build the VAFS map, and measure IoU for a "fabric" query restricted to the backrest: if the object-level embedding spreads the query response across the whole object, part-level semantics are lost. A second check is to remove the simulator's object-reference oracle and feed VAFS segments from an off-the-shelf instance segmenter; if accuracy falls to the level of the fusion baselines, the claimed speed-accuracy advantage is attributable to the oracle rather than to view synthesis.

Watch

Extended reading notes

Core claim

VAFS constructs a dense 3D semantic map entirely from simulator ground truth: at each time step it groups the point cloud by object reference, renders a synthetic view of each object from its average surface normal, and embeds the view with a frozen vision-language model. Each object embedding is contextualized by comparing it to an embedding of the whole scene, then assigned to every point belonging to that object. A voxel-aggregation step averages features within each voxel and replaces the points with the voxel centroid, keeping density uniform and making object boundaries explicit. In the reported experiments, this pipeline produced higher IoU than two fusion-based baselines on all six tested text queries, with roughly an 8x speedup over ConceptFusion and a 30x speedup over LeRF on the test scene. The authors describe the result as a "ground truth" semantic map because object identity comes from the simulator rather than from vision.

Load-bearing premise

The load-bearing premise is that every point in a simulator object is semantically identical, so one synthetic view per object captures what the object means; if an object is composed of distinct parts, the single object-level embedding cannot answer part-level semantic queries, and the map's accuracy on those queries would drop.

Editorial extensions

If this is right

  • VAFS makes it feasible to recompute a semantic map whenever the scene changes, since an update costs embeddings for only the objects that moved or appeared.
  • The method transfers the semantic-question-answering load from dense multiview fusion to a single-image embedding, so the ceiling on accuracy is set by the foundation model's ability to recognize an isolated object view.
  • In simulation, agents can receive a true object-level semantic map without training a segmentation module, which removes a major source of error and computation from the perception pipeline.
  • The reported speedups imply that dense 3D mapping no longer needs to be a one-time offline step in simulation; it can be part of the agent's online loop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not develop is part-level rendering: if an object label is replaced by sub-segment labels, the same pipeline would answer queries like "chair arm" or "bottle cap", at the cost of more synthetic views.
  • The paper's efficiency claim depends on the simulator oracle; an editor-level extrapolation is that applying the view-synthesis idea to predicted segments would preserve part of the speedup but inherit segmentation errors, so the gain over fusion methods would shrink.
  • Since runtime scales with object count rather than trajectory length, VAFS should be especially attractive in long-horizon tasks with many objects but limited camera motion, which is exactly the regime where fusion methods waste computation on redundant frames.
  • The weighting of object features against a global view is borrowed from fusion methods; an alternative worth testing would be to condition each object view on the agent's current task, making the map task-relevant rather than generic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes Voxel-Aggregated Feature Synthesis (VAFS), a dense 3D semantic mapping method for simulated environments. Instead of segmenting and embedding every RGB-D frame and fusing the resulting features, VAFS uses the simulator's object references (Eq. 2) to group point-cloud points into objects, renders a single synthetic view of each object from its average normal (Algorithm 1), embeds those views plus a global view with CLIP (Eq. 4), combines object and global features (Eq. 5), and voxel-pools the result to keep point density uniform (Eq. 6). The paper reports runtime and IoU comparisons against ConceptFusion and LeRF on one RoCoBench scene (Tables 1 and 2), claiming an order-of-magnitude speedup and higher IoU on six semantic queries. The intended contribution is to make dense 3D mapping practical for simulation-based embodied-agent research by exploiting ground-truth segmentation available from the simulator.

Significance. The central efficiency insight is sound: if the simulator can supply reliable object segments, then one CLIP embedding per object plus one global embedding can replace per-frame pixel-wise embeddings, reducing the number of expensive model calls from the number of frames to the number of objects in the scene. The paper states its assumptions explicitly (Section 3.2), which is a strength, and the proposed algorithm is simple enough to be reproducible once the missing experimental details are filled in. However, the evaluation as presented is not sufficient to support the accuracy claim: the comparison is confounded by the use of simulator-provided object references, and the runtime table contains an arithmetic inconsistency. The contribution is a promising baseline for simulated agents rather than a demonstrated SOTA method.

major comments (4)
  1. [Table 1] The VAFS row is internally inconsistent: 2D feature computation 175 s plus 3D fusion 145 s does not equal the reported total of 189 s (175+145=320). Because the central speed claim is an 'order of magnitude decrease in runtime,' this arithmetic error must be corrected, and the meaning of '3D fusion' for VAFS must be clarified: VAFS performs voxel aggregation of a single fused object-feature assignment rather than the per-frame feature fusion used by the baselines.
  2. [Section 4, Table 2] The accuracy comparison is uncontrolled. VAFS receives ground-truth object references from the simulator (Eq. 2) and assigns its synthesized feature to every point in each object segment by construction (Eq. 5), whereas ConceptFusion and LeRF must infer object boundaries from RGB-D observations and fuse noisy per-frame features. The high IoU in Table 2 is therefore substantially predetermined by the oracle segmentation. To support the claim that VAFS 'exceeds the accuracy of prior dense 3D mapping techniques,' the authors should run VAFS without simulator-provided object references (e.g., using a 2D segmenter on the rendered views or a point-cloud clustering method), or give the baselines access to equivalent segmentation information, or restrict the claim to the runtime advantage.
  3. [Section 3.2 and Algorithm 1] The assumption that all points in an object share the same semantic meaning, together with the rendering of one synthetic view per object from the average normal, means VAFS cannot represent semantically distinct object parts (e.g., a chair back vs. seat). Since the paper does not test part-level queries, the broad claim of 'better performance than fusion-based approaches' is not established for such cases. The authors should either report part-level query results to characterize the failure mode or explicitly state in the conclusion that VAFS targets object-level semantics only.
  4. [Section 4] The experimental protocol is under-specified. The paper does not state which CLIP encoder is used for fe, the rendering resolution and camera parameters used in Algorithm 1, the prompt templates or normalization used for the queries in Table 2, how IoU is computed (point-level vs. voxel-level and threshold), or how ConceptFusion and LeRF were configured in this simulated environment. Without these details the numbers in Tables 1 and 2 cannot be reproduced or independently verified, and the use of a single scene with six queries is too narrow to support the strength of the stated claims.
minor comments (4)
  1. [Section 5] In Section 5, the method is referred to as 'V ACC' instead of 'V AFS'.
  2. [Eq. (1)] Equation (1) overloads the symbol T: it denotes both the input observation set and the set of time indices; use different symbols for clarity.
  3. [Figure 2] The caption of Figure 2 does not explain the color scale used in the relevancy maps or how the maps are generated, which makes the qualitative claim about reduced blurring difficult to assess.
  4. [Abstract] The abstract's phrase 'ground truth semantic map' is misleading, because the semantic features come from CLIP embeddings rather than from the simulator's ground truth; only the object segmentation is ground truth from the simulator.

Circularity Check

1 steps flagged · score 6.0 of 10

Accuracy comparison reduces to simulator-provided object segmentation by construction; speed claim remains independent.

  1. self definitional [Section 3.2 Eq. (2); Section 3.3 Algorithm 1 and Eq. (5); Tables 1-2 and Conclusion]
    "P = {{¯pk, ¯rk, ok}|¯p, ¯r ∈ R3, k∈ Sp, o∈ So} ... fp : (S, t, k) → {¯pk, ¯rk, ik} (2) ... We then group the points by their object reference, and render a synthetic view vo of the object corresponding to those points ... Mu_t = {{¯p, co_t}|¯p ∈ Pt, o∈ Pt ⇒ co_t ≜ o} (5)"

    Eq. (5) assigns the same feature co_t to every point whose simulator-provided object reference is o. The object reference o in Eq. (2) is a ground-truth segmentation from the simulator, and Algorithm 1 groups points by that reference. Therefore the spatial support of any object-level semantic query is exactly the simulator's input segment (up to CLIP's threshold on the synthetic view). The Table 2 IoU scores for 'banana', 'apple', etc. thus measure whether CLIP recognizes a synthetic view of the oracle segment, not whether VAFS solves dense mapping (segmentation plus embedding).

full rationale

The paper is transparent about using simulator-provided object references, and its speed advantage is a legitimate, independent algorithmic contribution: replacing per-frame embedding with one synthetic view per object and voxel aggregation is a real computational savings. However, the accuracy claim in the abstract and conclusion is not an independent prediction. Equation (5) constructs the concept cloud by assigning one feature per simulator-provided object segment, so the object-level localization used in the IoU evaluation is the input segmentation itself. The only learned or measured component is CLIP's zero-shot similarity between a query text and the synthetic view of the known object segment. The comparison to ConceptFusion and LeRF is therefore asymmetric: those baselines must also solve segmentation and multi-view fusion, while VAFS receives ground-truth object boundaries for free. This makes the central accuracy comparison partially circular, though not in the classic fitted-parameter sense; no parameters are fitted to the IoU data, and CLIP is an external model. Overall, a score of 6 reflects that one of the two central claims (accuracy superiority) reduces by construction to the input segmentation, while the other central claim (runtime) remains independent and self-contained.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the simulator's oracle object segmentation and the uniform-semantics assumption. No new physical entities are introduced. The only hand-chosen hyperparameter is the voxel size/increment.

free parameters (1)
  • voxel_size_and_increment = 0.1 (starting size and increment)
    Chosen per Section 4 as 0.1 for all results; no ablation shows sensitivity of IoU to voxel resolution.
assumptions (4)
  • domain assumption Simulator provides per-point coordinate, color, and object reference (Eq. 2).
    The entire method depends on ground-truth segmentation from the physics engine; without it there is no way to group points into objects.
  • domain assumption All points in a given object have the same semantic meaning relevant to encoding (Sec. 3.2).
    VAFS assigns one feature per object, so it cannot represent parts or composite semantics; this is the main limitation for open-set queries.
  • ad hoc to paper A single synthetic view per object, rendered from the average normal, contains enough information for the embedding (Algorithm 1).
    No analysis of view coverage or occlusion; complex objects may not be fully represented by one average-normal view.
  • domain assumption The pretrained feature extractor fe maps a synthetic view to a feature that supports text-query similarity (Assumption 3 in Sec. 3.2).
    The method inherits the zero-shot capability and biases of CLIP or similar; if the synthetic view is out of distribution, features degrade.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Voxel-Aggregated Feature Synthesis: Efficient Dense Mapping for Simulated 3D Reasoning." pith.science (2026). https://pith.science/paper/ULJ7YMRJ

@misc{pith2026241110616,
  author       = {Pith},
  title        = {Pith review of: Voxel-Aggregated Feature Synthesis: Efficient Dense Mapping for Simulated 3D Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULJ7YMRJ}},
  note         = {Machine review of arXiv:2411.10616}
}
read the original abstract

We address the issue of the exploding computational requirements of recent State-of-the-art (SOTA) open set multimodel 3D mapping (dense 3D mapping) algorithms and present Voxel-Aggregated Feature Synthesis (VAFS), a novel approach to dense 3D mapping in simulation. Dense 3D mapping involves segmenting and embedding sequential RGBD frames which are then fused into 3D. This leads to redundant computation as the differences between frames are small but all are individually segmented and embedded. This makes dense 3D mapping impractical for research involving embodied agents in which the environment, and thus the mapping, must be modified with regularity. VAFS drastically reduces this computation by using the segmented point cloud computed by a simulator's physics engine and synthesizing views of each region. This reduces the number of features to embed from the number of captured RGBD frames to the number of objects in the scene, effectively allowing a "ground truth" semantic map to be computed an order of magnitude faster than traditional methods. We test the resulting representation by assessing the IoU scores of semantic queries for different objects in the simulated scene, and find that VAFS exceeds the accuracy and speed of prior dense 3D mapping techniques.

Figures

Figures reproduced from arXiv: 2411.10616 by the authors.

Figure 1
Figure 1. The high-level workflow of VAFS. At each time step, we associate points [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Relevancy maps for semantic queries. can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 1 [2] Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. Openflamingo: An open-source fram… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 26 canonical work pages

  1. [1]

    Do as i 3 Synthesize Aggregate Synthesize Aggregate t

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i 3 Synthesize Aggregate Synthesize Aggregate t ... t+n P1 P2 P3 C1 C2 C3 P1 P2 P3 C1 C2 C3 Figure 1. The high-level workflow of V AFS. At each time step, we associate points P with segments ...

    arXiv 2022

  2. [2]

    Openflamingo: An open-source frame- work for training large autoregressive vision-language mod- els, 2023

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. Openflamingo: An open-source frame- work for training large autoregressive vision-language mod- els, 2023. 2

    2023

  3. [3]

    Understanding human perception by human-made illusions

    Claus-Christian Carbon. Understanding human perception by human-made illusions. Frontiers in Human Neuroscience, 8, 2014. 1

    work page 2014

  4. [4]

    End-to- end object detection with transformers, 2020

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers, 2020. 2

    2020

  5. [5]

    Xia, Brian Ichter, Kanishka Rao, Keerthana Gopalakrishnan, Michael S

    Boyuan Chen, F. Xia, Brian Ichter, Kanishka Rao, Keerthana Gopalakrishnan, Michael S. Ryoo, Austin Stone, and Daniel Kappler. Open-vocabulary queryable scene representations for real world planning.2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 11509–11522,

    work page 2023

  6. [6]

    Schwing, and Alexander Kir- illov

    Bowen Cheng, Alexander G. Schwing, and Alexander Kir- illov. Per-pixel classification is not all you need for semantic segmentation, 2021. 2

    work page 2021

  7. [7]

    Scal- ing open-vocabulary image segmentation with image-level labels, 2022

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels, 2022. 2

    work page 2022

  8. [8]

    Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull

    Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Kr- ishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, Chuang Gan, Celso Miguel de Melo, Joshua B. Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull. Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning, 2023. 2

    work page 2023

Show all 42 references
  1. [9]

    Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull

    Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Kr- ishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, Chuang Gan, Celso Miguel de Melo, Joshua B. Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull. Conce...

    2023

  2. [10]

    Semantic abstraction: Open- world 3d scene understanding from 2d vision-language mod- els, 2022

    Huy Ha and Shuran Song. Semantic abstraction: Open- world 3d scene understanding from 2d vision-language mod- els, 2022. 2

    2022

  3. [11]

    3d-llm: Inject- ing the 3d world into large language models, 2023

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Inject- ing the 3d world into large language models, 2023. 1, 2

    2023

  4. [12]

    Toward general-purpose robots via foundation mod- els: A survey and meta-analysis, 2023

    Yafei Hu, Quanting Xie, Vidhi Jain, Jonathan Francis, Jay Patrikar, Nikhil Keetha, Seungchan Kim, Yaqi Xie, Tianyi Zhang, Shibo Zhao, Yu Quan Chong, Chen Wang, Katia Sycara, Matthew Johnson-Roberson, Dhruv Batra, Xiaolong Wang, Sebastian Scherer, Zsolt Kira, Fei Xia, and Yonat...

    2023

  5. [13]

    Planning-oriented autonomous driv- ing, 2023

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-oriented autonomous driv- ing, 2023. 1

    2023

  6. [14]

    Visual language maps for robot navigation

    Chen Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. 2023 IEEE In- ternational Conference on Robotics and Automation (ICRA), pages 10608–10615, 2022. 2

    2023

  7. [15]

    Conceptfusion: Open-set multimodal 3d mapping

    Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Alaa Maalouf, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, et al. Conceptfusion: Open-set multimodal 3d mapping. arXiv preprint arXiv:2302.07241, 2023. 1, 2, 3

    2023 arXiv

  8. [16]

    Le, Yunhsuan Sung, Zhen Li, and Tom Duerig

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V . Le, Yunhsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision, 2021. 2

    2021

  9. [17]

    Multi-agent trajectory prediction by combin- ing egocentric and allocentric views

    Xiaosong Jia, Liting Sun, Hang Zhao, Masayoshi Tomizuka, and Wei Zhan. Multi-agent trajectory prediction by combin- ing egocentric and allocentric views. In 5th Annual Confer- ence on Robot Learning, 2021. 2

    2021

  10. [18]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 19672–19682, 2023. 2, 3

    2023

  11. [19]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, and Hanzi Mao et. al. Segment anything, 2023. 2

    2023

  12. [20]

    Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl

    Boyi Li, Kilian Q. Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation, 2022. 2

    2022

  13. [21]

    Ni, and Heung-Yeung Shum

    Feng Li, Hao Zhang, Huaizhe xu, Shilong Liu, Lei Zhang, Lionel M. Ni, and Heung-Yeung Shum. Mask dino: Towards a unified transformer-based framework for object detection and segmentation, 2022. 2

    2022

  14. [22]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023. 2

    2023

  15. [23]

    Alvarez, Ping Luo, and Tong Lu

    Zhiqi Li, Wenhai Wang, Enze Xie, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, Ping Luo, and Tong Lu. Panoptic segformer: Delving deeper into panoptic segmen- tation with transformers, 2022. 2

    2022

  16. [24]

    Ov-nerf: Open-vocabulary neural radiance fields with vision and language foundation models for 3d semantic understanding

    Guibiao Liao, Kaichen Zhou, Zhenyu Bao, Kanglin Liu, and Qing Li. Ov-nerf: Open-vocabulary neural radiance fields with vision and language foundation models for 3d semantic understanding. ArXiv, abs/2402.04648, 2024. 2

    2024 arXiv

  17. [25]

    Xing, and Shijian Lu

    Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El-Saddik, Christian Theobalt, Eric P. Xing, and Shijian Lu. 3d open-vocabulary segmenta- tion with foundation models. ArXiv, abs/2305.14093, 2023. 2

    2023 arXiv

  18. [26]

    Roco: Di- alectic multi-robot collaboration with large language models,

    Zhao Mandi, Shreeya Jain, and Shuran Song. Roco: Di- alectic multi-robot collaboration with large language models,

  19. [27]

    Kirill Mazur, Edgar Sucar, and Andrew J. Davison. Feature- realistic neural fusion for real-time, open set scene under- standing, 2022. 2

    2022

  20. [28]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: 5 Representing scenes as neural radiance fields for view syn- thesis, 2020. 2

    2020

  21. [29]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, and Ilge Akkaya et. al. Gpt-4 technical re- port, 2024. 2

    2024

  22. [30]

    Openscene: 3d scene understanding with open vocabularies,

    Songyou Peng, Kyle Genova, Chiyu ”Max” Jiang, An- drea Tagliasacchi, Marc Pollefeys, and Thomas Funkhouser. Openscene: 3d scene understanding with open vocabularies,

  23. [31]

    Langsplat: 3d language gaussian splatting,

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting,

  24. [32]

    Alec Radford, Jong Wook Kim, and Chris Hallacy et. al. Learning transferable visual models from natural language supervision, 2021. 2

    2021

  25. [33]

    Clip-fields: Weakly supervised semantic fields for robotic memory

    Nur Muhammad (Mahi) Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, and Arthur Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory. ArXiv, abs/2210.05663, 2022. 2

    2022 arXiv

  26. [34]

    Davi- son

    Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew J. Davi- son. imap: Implicit mapping and positioning in real-time,

  27. [35]

    Vi- sual slam algorithms: A survey from 2010 to 2016

    Takafumi Taketomi, Hideaki Uchiyama, and Sei Ikeda. Vi- sual slam algorithms: A survey from 2010 to 2016. IPSJ transactions on computer vision and applications , 9:1–11,

    2010

  28. [36]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 2

    2023 arXiv

  29. [37]

    Neural feature fusion fields: 3d distillation of self- supervised 2d image representations

    Vadim Tschernezki, Iro Laina, Diane Larlus, and Andrea Vedaldi. Neural feature fusion fields: 3d distillation of self- supervised 2d image representations. 2022 International Conference on 3D Vision (3DV), pages 443–453, 2022. 2

    2022

  30. [38]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. 1

    2023

  31. [39]

    Language to rewards for robotic skill synthesis,

    Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, and Fei...

  32. [40]

    Deformable detr: Deformable transformers for end-to-end object detection, 2021

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection, 2021. 2

    2021

  33. [41]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023. 1

    2023

  34. [42]

    Natural language processing in the era of large language models

    Arkaitz Zubiaga. Natural language processing in the era of large language models. Frontiers in Artificial Intelligence, 6,

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.