Pith. sign in

REVIEW 5 major objections 5 minor 55 references

Hi-Dyna Graph: Hierarchical Dynamic Scene Graph for Robotic Autonomy in Human-Centric Environments

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hybrid scene graph that pairs a fixed global layout with live local relations lets a mobile manipulator carry out long-horizon cafeteria tasks—delivery and cleanup—with no task-specific training.

desk verdict A sensible hybrid graph architecture with a real robot demo, but the headline autonomy claim is not supported by the reported evidence and the paper's own appendix undermines it. read the letter →

arxiv 2506.00083 v1 pith:GPOS6NHI submitted 2025-05-30 cs.RO cs.AI

Jiawei Hou , Xiangyang Xue , Taiping Zeng This is my paper
classification cs.ROcs.AI
keywords hierarchicaldynamicscenegraphembodiedrobotautonomyserviceroboticshuman-centricenvironmentsgenerationLLMreasoningtopologicalmappingmobilemanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a service robot can handle long-horizon tasks in a busy human environment—delivering coffee, tidying tables—using a new scene representation called Hi-Dyna Graph, with no task-specific training or reward design. The representation splits the world into two coupled layers: a slowly changing global topological graph of rooms and large furniture, and fast-updating local subgraphs that track people, small objects, and the relations between them. A large language model reads this unified graph as text, notices when something task-relevant happens (coffee appears on the counter, a patron leaves), and emits navigation and pick-place commands. Real-world deployment on a mobile manipulator in a campus cafeteria and laboratory is offered as evidence that the pipeline works end to end.

What carries the argument

The central device is the hierarchical split between a global static scene graph and local dynamic subgraphs, joined by an anchoring step that uses semantic and spatial constraints. The static branch filters out transient objects rather than trusting every detection, keeping only objects larger than a volume threshold or in protected semantic classes; the dynamic branch refreshes its relations on a 10-second sliding window. The load-bearing mechanism inside the dynamic branch is the relation-pair prior, which biases Top-k relation candidates toward humans-as-subjects and large-furniture-as-objects (a 70/30 allocation) on the assumption that those are the pairs that drive activity understanding. The LLM then converts the text-serialized graph into executable skill sequences.

What would settle it

Move a piece of large furniture (say, a booth or counter exceeding the $2\,\text{m}^3$ threshold) to a new location and check whether the static graph updates: the relatively-static filter predicts the graph keeps the old node and the robot navigates to the stale pose. A second test is to create a task whose trigger is a relation with a human as the object (for example, a person receiving a tray from someone), where the 70/30 relation-pair prior should under-sample exactly the triggering pair. The paper's own appendix adds a third test: occlude the prepared coffee on the counter for several seconds and check whether the delivery trigger is permanently lost.

Watch

Extended reading notes

Core claim

Hi-Dyna Graph models a changing indoor scene as a pair of graphs, $G = \{G_s, G_d\}$. $G_s$ is a global static graph built from posed RGB-D images that keeps only regions and objects that are 'relatively static'—those with bounding-box volume above $v_{thr} = 2\,\text{m}^3$ or belonging to designated semantic classes. $G_d$ is a local dynamic graph built from video streams, where instances are segmented, tracked, and paired for relation prediction with a prior that reserves 70% of Top-k pairs for humans-as-subjects or large-furniture-as-objects. The dynamic subgraph is anchored into the global graph either by 3D back-projection with bounding-box overlap above 60% or by semantic matching of static categories, and it is cleared and rebuilt every 10-second sliding window. An LLM reasoner, fed the combined structure as text prompts, infers latent task triggers (coffee ready, table abandoned) and composes them into 'navigate / pick / place' sequences grounded in the robot's skills; the authors report the robot completing coffee delivery and tidy-up tasks autonomously without further training.

Load-bearing premise

The system assumes a world divided into things that barely move and things that move often, and it fixes that division by hand-picked thresholds: objects with volume above $2\,\text{m}^3$, a chosen list of semantic classes, and a 70/30 bias toward certain relation pairs. If large furniture gets rearranged, if a small object is actually a permanent fixture, or if the key interaction has a person as the object, the graph either freezes a stale node or misses the trigger, and the paper reports no sensitivity analysis over these choices.

Editorial extensions

If this is right

  • A robot can keep a usable model of a changing building without rebuilding its map: the dynamic-update graph matches static from-scratch reconstruction in vertex and edge accuracy over 30 minutes.
  • Long-horizon service tasks such as coffee delivery and table tidying can be triggered purely by changes detected in local dynamic subgraphs, with the global graph supplying navigation context.
  • The fixed 70/30 subject-object prior and the static-object filter each improve open-vocabulary relation prediction on the OpenPVSG benchmark over the same pipeline without them.
  • Because the LLM works from the text-formatted graph, new skills or new task phrasings can be added without retraining the perception or relation-prediction modules.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to make the static/dynamic split scene-adaptive instead of hand-chosen: learning $v_{thr}$, the semantic class list, and the 70/30 split from observed motion statistics could let the same architecture transfer to warehouses, hospitals, or homes where 'static' means different things.
  • The paper's own failure analysis points to verification rather than perception as the next bottleneck: adding closed-loop checks (did the grasp succeed, was the trigger missed during occlusion) to the LLM loop would probably matter more than further improving relation accuracy.
  • The global-anchor/local-patch pattern is a general recipe: any robot with a slowly changing map and fast-moving local activity could use the same two-timescale graph idea, not just scene-graph pipelines.
  • A direct test of the prior's blind spot would be a task whose trigger has a human as the object of the relation (for example, a person receiving a tray), where the 70/30 subject-object allocation should under-sample exactly the triggering pair.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Hi-Dyna Graph, a hierarchical dynamic scene graph for service-robot autonomy in human-centric environments. A static global topological graph of regions and large objects is built from posed RGB-D data, while dynamic local subgraphs are generated from environmental or egocentric video streams via open-vocabulary segmentation, tracking, and relation prediction. The two levels are fused by spatial or semantic anchoring and fed to an LLM-based reasoner that generates navigation and pick-place instructions. The authors report relation-prediction results on OpenPVSG, comparisons of graph structure stability against static scene graph baselines, ablations of their static-object filter and pair-prior strategy, and two real-world demonstrations (coffee delivery and tidy-up) on a mobile manipulator.

Significance. If substantiated, the proposed hybrid representation is a useful step toward practical long-horizon service-robot autonomy: it couples a cheap, persistent spatial prior with transient relational semantics, and the use of an LLM as a graph-driven reasoner is timely. The paper has strengths: it evaluates on an external benchmark (OpenPVSG), provides ablations, reports a real-world deployment, and includes a candid limitations appendix. However, the central autonomy claim is currently supported only by qualitative demonstrations, and the quantitative evidence has confounds that prevent attribution of the observed gains to the proposed architecture. The work is therefore of interest to the community, but the evidence as presented is not yet sufficient for the strength of the claims.

major comments (5)
  1. [§5.2 and Appendix A.5] The abstract claims that the robot 'autonomously complete[s] complex tasks with no further training or complex rewarding in a dynamic scene,' but the only supporting evidence is two qualitative demonstrations with no success rate, no repeated trials, and no failure or recovery statistics. Appendix A.5 explicitly concedes that relation-prediction errors can leave an item 'undelivered indefinitely' and that errors 'propagate silently, often resulting in full task abandonment.' This directly contradicts the unconditional autonomy claim. The authors must either provide task-level success/recovery metrics over repeated trials or substantially temper the claim in the abstract and introduction.
  2. [Table 1, §5.3] The OpenPVSG comparison is confounded: PVSG uses a ResNet-50 backbone, while the proposed method uses FC-CLIP with a ConvNeXt-Large backbone pretrained on LAION-2B. The ablation 'w/o CNN-CLIP' still uses a different ViT-based CLIP backbone, so it does not isolate the effect of the proposed relation-prediction architecture or pair-prior. The observed gains in R/mR could largely reflect the stronger segmentation backbone rather than the Hi-Dyna Graph contributions. A controlled comparison with the same backbone and trackers for both the baseline and the proposed method is needed to support the claimed superiority.
  3. [Table 2, §5.3] The graph-structure evaluation shows that Hi-Dyna Graph is competitive, not superior, to static baselines that rebuild from scratch: at several time steps the static baselines achieve higher vertex or edge accuracy. The paper's claim of 'superior scene representation effectiveness' (abstract) is thus not supported by this table. Moreover, the vertex/edge accuracy metrics are not precisely defined in the main text, and no error bars or significance tests are provided despite the checklist claiming their presence.
  4. [§4.1, §4.2, Fig. 5] The method relies on several hand-set parameters: the volume threshold vthr = 2 m^3, the semantic class set Cs, and the 70/30 allocation of relation-pair proposals toward humans-as-subjects and large-furniture-as-objects. The paper does not analyze sensitivity to these parameters, even though the 'relatively static scene assumption' is load-bearing for the whole approach. If large furniture is moved, if persistent small objects are excluded, or if key interactions involve humans as objects, the static graph can become stale or the dynamic graph can miss the triggering relation. A sensitivity analysis or explicit failure characterization is required to establish the robustness claimed in Fig. 5.
  5. [§4.4 and §5.2] The LLM-driven reasoning and execution pipeline is not evaluated against any baseline, such as an LLM planner consuming raw observation text or a static graph alone. The two real-world demonstrations therefore do not show that the hierarchical dynamic graph, rather than the LLM's priors and the hand-crafted prompt, drives the observed behavior. Without an ablation or a task-level comparison, the causal contribution of the proposed representation to the autonomy claim remains unmeasured.
minor comments (5)
  1. [Abstract and Figure 1] There is a typo 'Hi-Dyna Grap's' in the abstract; it should read 'Hi-Dyna Graph's'.
  2. [§4.1] The sentence ending 'with algorithmic details described bellow' contains a typo: 'bellow' should be 'below'.
  3. [§5.4] The word 'volumn' should be 'volume' in the description of the threshold vthr.
  4. [Fig. 5] The figure legend states 'vertices precision,' and the main text says 'vertices accuracy' and 'vertices precision' interchangeably; please use consistent terminology and define the metric precisely.
  5. [Table 2] The formatting of bold and underlined numbers is inconsistent (e.g., the 0-min column has an underlined 0.74 that is not explained in the caption); also, no error bars or number of runs are reported, despite the checklist indicating otherwise.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: central relation-prediction and static-graph results are benchmarked on external data and baselines; self-citations are peripheral and not load-bearing.

full rationale

Hi-Dyna Graph's central quantitative claims do not reduce to its inputs. The local dynamic relation predictor is trained and evaluated on the external OpenPVSG benchmark, with ablations over the segmentation backbone and the relation-pair prior; the static graph is compared against ConceptGraph*, HOV-SG, and Topo-Field on vertex/edge accuracy over time. The relatively static scene assumption (volume threshold vthr, semantic class set Cs, and the 70/30 pair-prior split) is a stated design prior rather than a parameter fitted to the evaluation labels, and Table 1 explicitly ablates the pair prior instead of treating it as a forced result. Self-citations to Topo-Field [3] (used as one topology-construction option and as a baseline) and ELA-ZSON [28] (navigation strategy) are not load-bearing for the paper's central claim: neither is invoked as a uniqueness theorem or as a substitute for the external benchmark evidence, and both support auxiliary components rather than the predicted scene-graph accuracy. Appendix A.5's failure cases and Section 6's limitations weaken the autonomy claim as an empirical matter, but that is a correctness/evidence concern, not circularity. No equation in the paper defines a target quantity in terms of itself or of a fitted input that is then renamed as a prediction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The graph architecture rests on several hand-set thresholds and behavioral priors rather than fitted parameters; the most consequential is the relatively static assumption. No sensitivity analysis is provided. The system also assumes the reliability of off-the-shelf perception, tracking, alignment, and LLM components, which the paper's own failure cases partially contradict.

free parameters (6)
  • Volume threshold v_thr = 2 m^3
    Objects with bounding-box volume below the threshold are excluded from the static graph; set by the authors in Section 5.4 and A.3, with no sensitivity analysis.
  • Static semantic class set C_s = couch, fridge, TV, and similar categories
    Objects in these classes are always kept in the static graph even if small; chosen by hand in Section 4.1.
  • Relation pair prior split = 70% prioritized / 30% others
    70% of Top-k relation pairs are drawn from human-as-subject or large-furniture-as-object pairs, 30% from others, as described in Section 4.2.
  • Bounding-box merge threshold b_thr = 60%
    Overlap threshold for merging dynamic and static vertices in spatial alignment, specified in Section 4.3.
  • Sliding window duration Delta t = 10 seconds
    Length of recent video used for dynamic graph updates, specified in Section 5.2 and A.2.
  • Temporal consolidation gap = 2 seconds
    Consecutive relation segments separated by less than 2 seconds are merged, as specified in Section 4.2.
assumptions (5)
  • domain assumption Relatively static scene assumption: large furniture and designated semantic categories remain stationary, and humans tend to be subjects while large furniture tends to be objects in interactions.
    Invoked in Sections 4.1 and 4.2; if violated, the static graph has stale nodes and the dynamic graph misses important pairs.
  • domain assumption Unified coordinate alignment: posed RGB-D inputs and environmental camera poses from GLOMAP are accurate enough for back-projection and graph anchoring.
    Sections 4.1 and 4.3 rely on back-projecting embeddings and merging vertices by 3D overlap; calibration errors would break fusion.
  • domain assumption Open-vocabulary perception reliability: FC-CLIP segmentation, UniTrack association, and relation prediction are accurate enough to capture task triggers.
    Used in Sections 4.2 and 5.3; appendix A.5 admits relation prediction failures in crowded scenes.
  • domain assumption LLM grounding: the text-formatted graph is sufficient for an LLM to infer task triggers and generate executable skill sequences.
    Equation 3 and Section 4.4; no evaluation of LLM output quality is provided.
  • domain assumption Manipulation reliability: 6-DoF pose estimation from a single RGB-D image is sufficient for grasping in deployed scenarios.
    Section 4.4 and A.2 rely on Polaris; A.5 states grasp failures propagate without recovery.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hi-Dyna Graph: Hierarchical Dynamic Scene Graph for Robotic Autonomy in Human-Centric Environments." pith.science (2026). https://pith.science/paper/GPOS6NHI

@misc{pith2026250600083,
  author       = {Pith},
  title        = {Pith review of: Hi-Dyna Graph: Hierarchical Dynamic Scene Graph for Robotic Autonomy in Human-Centric Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GPOS6NHI}},
  note         = {Machine review of arXiv:2506.00083}
}
read the original abstract

Autonomous operation of service robotics in human-centric scenes remains challenging due to the need for understanding of changing environments and context-aware decision-making. While existing approaches like topological maps offer efficient spatial priors, they fail to model transient object relationships, whereas dense neural representations (e.g., NeRF) incur prohibitive computational costs. Inspired by the hierarchical scene representation and video scene graph generation works, we propose Hi-Dyna Graph, a hierarchical dynamic scene graph architecture that integrates persistent global layouts with localized dynamic semantics for embodied robotic autonomy. Our framework constructs a global topological graph from posed RGB-D inputs, encoding room-scale connectivity and large static objects (e.g., furniture), while environmental and egocentric cameras populate dynamic subgraphs with object position relations and human-object interaction patterns. A hybrid architecture is conducted by anchoring these subgraphs to the global topology using semantic and spatial constraints, enabling seamless updates as the environment evolves. An agent powered by large language models (LLMs) is employed to interpret the unified graph, infer latent task triggers, and generate executable instructions grounded in robotic affordances. We conduct complex experiments to demonstrate Hi-Dyna Grap's superior scene representation effectiveness. Real-world deployments validate the system's practicality with a mobile manipulator: robotics autonomously complete complex tasks with no further training or complex rewarding in a dynamic scene as cafeteria assistant. See https://anonymous.4open.science/r/Hi-Dyna-Graph-B326 for video demonstration and more details.

Figures

Figures reproduced from arXiv: 2506.00083 by the authors.

Figure 1
Figure 1. Hi-Dyna Graph creates a hybrid graph of global static layouts and local dynamic relations. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of our proposed Hi-Dyna Graph. The hierarchical dynamic scene graph construction process consists of two branches, including the global static graph and local dynamic graph generation. The separate parts are combined to form a united scene graph representation. By employing LLM as reasoning approach, the scene graph is fed as prompts, together with other context, to drive the robotic mobile manipulator to m… view at source ↗
Figure 3
Figure 3. Example of the generated hierarchical dynamic scene graphs. We show the detailed evolving process of dynamic subgraphs in the cafeteria and laboratory. On the right, we show the quantity counts of the graph vertices and edges during the process. We further show the mobile manipulation demonstrations, including navigation, pick, and place tasks. Cafeteria Environmental Camera Laboratory Environmental Camera Entire Bu… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Environmental and platform setups. Environmental cameras are installed in the activity￾critical regions. We show the top-down view of regions and a brief platform structure. 5.2 Robotic Deployment Setups. For environmental setup, we install environmental cameras in act…
Figure 5
Figure 5. Figure 5: Comparison of static graph vertices accuracy as time goes by. The left plot illustrates the temporal variations in vertices accuracy across static scene graphs constructed using different methods from multiple video sequences. The right plot demonstrates the time-depen…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 45 canonical work pages

  1. [1]

    Hierarchical open-vocabulary 3d scene graphs for language-grounded robot navigation

    Abdelrhman Werby, Chenguang Huang, Martin Büchner, Abhinav Valada, and Wolfram Bur- gard. Hierarchical open-vocabulary 3d scene graphs for language-grounded robot navigation. Robotics: Science and Systems, 2024

    work page 2024

  2. [2]

    Clio: Real-time task-driven open-set 3d scene graphs.IEEE Robotics and Automation Letters, 9(10):8921–8928, 2024

    Dominic Maggio, Yun Chang, Nathan Hughes, Matthew Trang, Dan Griffith, Carlyn Dougherty, Eric Cristofalo, Lukas Schmid, and Luca Carlone. Clio: Real-time task-driven open-set 3d scene graphs.IEEE Robotics and Automation Letters, 9(10):8921–8928, 2024

    work page 2024

  3. [3]

    Topo-field: Topometric mapping with brain-inspired hierarchical layout-object-position fields

    Jiawei Hou, Wenhao Guan, Longfei Liang, Jianfeng Feng, Xiangyang Xue, and Taiping Zeng. Topo-field: Topometric mapping with brain-inspired hierarchical layout-object-position fields. IEEE Robotics and Automation Letters, 10(6):5385–5392, 2025

    work page 2025

  4. [4]

    Topomap: Topological mapping and navigation based on visual slam maps

    Fabian Blochliger, Marius Fehr, Marcin Dymczyk, Thomas Schneider, and Rol Siegwart. Topomap: Topological mapping and navigation based on visual slam maps. In2018 IEEE International Conference on Robotics and Automation (ICRA), pages 3818–3825. IEEE, 2018

    work page 2018

  5. [5]

    Hybrid topological and 3d dense mapping through autonomous explo- ration for large indoor environments

    Clara Gomez, Marius Fehr, Alex Millane, Alejandra C Hernandez, Juan Nieto, Ramon Barber, and Roland Siegwart. Hybrid topological and 3d dense mapping through autonomous explo- ration for large indoor environments. In2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9673–9679. IEEE, 2020

    work page 2020

  6. [6]

    McGill University (Canada), 2015

    Qiwen Zhang.Autonomous indoor exploration and mapping using hybrid metric/topological maps. McGill University (Canada), 2015

    work page 2015

  7. [7]

    Uncertainty reduction via heuristic search planning on hybrid metric/topological map

    Qiwen Zhang, Ioannis Rekleitis, and Gregory Dudek. Uncertainty reduction via heuristic search planning on hybrid metric/topological map. In2015 12th Conference on Computer and Robot Vision, pages 222–229. IEEE, 2015

    work page 2015

  8. [8]

    Hmaps-hybrid height- voxel maps for environment representation

    Luís Garrote, Cristiano Premebida, David Silva, and Urbano J Nunes. Hmaps-hybrid height- voxel maps for environment representation. In2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1197–1203. IEEE, 2018

    work page 2018

Show all 55 references
  1. [9]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. InThe European Conference on Computer Vision (ECCV), 2020

    2020

  2. [10]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July 2023

    2023

  3. [11]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20310...

    2024

  4. [12]

    HyperReel: High-fidelity 6-DoF video with ray- conditioned sampling

    Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O’Toole, and Changil Kim. HyperReel: High-fidelity 6-DoF video with ray- conditioned sampling. InCVPR, 2023

    2023

  5. [13]

    Hexplane: A fast representation for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 130–141, 2023

    2023

  6. [14]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12479–12488, 2023. 10

    2023

  7. [15]

    Panoptic video scene graph generation

    Jingkang Yang, Wenxuan Peng, Xiangtai Li, Zujin Guo, Liangyu Chen, Bo Li, Zheng Ma, Kaiyang Zhou, Wayne Zhang, Chen Change Loy, and Ziwei Liu. Panoptic video scene graph generation. InCVPR, 2023

    2023

  8. [16]

    4d panoptic scene graph generation

    Jingkang Yang, Jun Cen, Wenxuan Peng, Fangzhou Liu, Shuai amd Hong, Xiangtai Li, Kaiyang Zhou, Qifeng Chen, and Ziwei Liu. 4d panoptic scene graph generation. InNeurIPS, 2023

    2023

  9. [17]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 5692–5701, 2021

    2021

  10. [18]

    Neural scene flow fields for space-time view synthesis of dynamic scenes

    Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6498–6508, 2021

    2021

  11. [19]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. InProceedings of the IEEE/CVF conference on computer vision ...

    2022

  12. [20]

    Neural radiance flow for 4d view synthesis and video processing

    Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B Tenenbaum, and Jiajun Wu. Neural radiance flow for 4d view synthesis and video processing. In2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14304–14314. IEEE Computer Society, 2021

    2021

  13. [21]

    Hypernerf: a higher-dimensional representation for topologically varying neural radiance fields.ACM Transactions on Graphics (TOG), 40(6):1–12, 2021

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M Seitz. Hypernerf: a higher-dimensional representation for topologically varying neural radiance fields.ACM Transactions on Graphics (TOG), 40(6):1...

    2021

  14. [22]

    Nerfies: Deformable neural radiance fields

    Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. InProceedings of the IEEE/CVF international conference on computer vision, pages 5865–5874, 2021

    2021

  15. [23]

    De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction.arXiv preprint arXiv:2309.13101, 2023

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction.arXiv preprint arXiv:2309.13101, 2023

    2023 arXiv

  16. [24]

    Spacetime gaussian feature splatting for real-time dynamic view synthesis

    Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8508–8520, June 2024

    2024

  17. [25]

    Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang-Chieh Chen. Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip. InNeurIPS, 2023

    2023

  18. [26]

    Do different tracking tasks require different appearance models? In2021 Conference on Neural Information Processing Systems (NeurIPS), 2021

    Z Wang, H Zhao, Y Li, S Wang, P Torr, and L Bertinetto. Do different tracking tasks require different appearance models? In2021 Conference on Neural Information Processing Systems (NeurIPS), 2021

    2021

  19. [27]

    Pair then relation: Pair-net for panoptic scene graph generation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Jinghao Wang, Zhengyu Wen, Xiangtai Li, Zujin Guo, Jingkang Yang, and Ziwei Liu. Pair then relation: Pair-net for panoptic scene graph generation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  20. [28]

    Ela-zson: Efficient layout-aware zero-shot object navigation agent with hierarchical planning.arXiv preprint arXiv:2505.06131, 2025

    Jiawei Hou, Yuting Xiao, Xiangyang Xue, and Taiping Zeng. Ela-zson: Efficient layout-aware zero-shot object navigation agent with hierarchical planning.arXiv preprint arXiv:2505.06131, 2025

    2025

  21. [29]

    Polaris: Open-ended interactive robotic manipulation via syn2real visual grounding and large language models

    Tianyu Wang, Haitao Lin, Junqiu Yu, and Yanwei Fu. Polaris: Open-ended interactive robotic manipulation via syn2real visual grounding and large language models. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9676–9683. IEEE, 2024. 11

    2024

  22. [30]

    Global Structure- from-Motion Revisited

    Linfei Pan, Daniel Barath, Marc Pollefeys, and Johannes Lutz Schönberger. Global Structure- from-Motion Revisited. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  23. [31]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF conference on computer vision a...

    2022

  24. [32]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Scaling egocentric vision: The epic-kitchens dataset. InProceedings of the European conference on compute...

    2018

  25. [33]

    Annotating objects and relations in user-generated videos

    Xindi Shang, Donglin Di, Junbin Xiao, Yu Cao, Xun Yang, and Tat-Seng Chua. Annotating objects and relations in user-generated videos. InProceedings of the 2019 on International Conference on Multimedia Retrieval, pages 279–287, 2019

    2019

  26. [34]

    Learning 3d semantic scene graphs from 3d indoor reconstructions

    Johanna Wald, Helisa Dhamo, Nassir Navab, and Federico Tombari. Learning 3d semantic scene graphs from 3d indoor reconstructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3961–3970, 2020

    2020

  27. [35]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning

    Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. In2024 IEEE International Conference...

    2024

  28. [36]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  29. [37]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)...

    2019

  30. [38]

    e n v i r o n m e n t

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting twenty-thousand classes using image-level supervision. InECCV, 2022. 12 A Appendix / supplemental material A.1 Environmental Setups The environment we deploy our robotics and conduct expe...

    2022

  31. [39]

    , " role

    Cafe area 2) C la ss roo m cluster 3) Faculty offices 4) Common spaces " , " role ": " Embodied service robot for campus cafe named ’ CafeBot ’" , " p r i m a r y _ o b j e c t i v e ": " Handle delivery tasks between cafe counter and various d e s t i n a t i o n s while m a ...

  32. [40]

    Query global topology

  33. [41]

    Find C a f e _ S t a t i o n -> E l e v a t o r _ L o b b y -> Classroom_A1 - A6 cluster

  34. [42]

    C al cu lat e shortest path avoiding crowded zones during class breaks

  35. [43]

    "" } , " l o c a l _ r e l a t i o n a l _ g r a p h

    Update path when d et ect in g tem po ra ry o b s t r u c t i o n """ } , " l o c a l _ r e l a t i o n a l _ g r a p h ": { " d y n a m i c _ n o d e s ": { " human ": [" student " , " pr ofe ss or " , " staff " , " visitor "] , " objects ": { " static ": [" table " , " door ...

  36. [44]

    Coffee order ready for delivery

  37. [45]

    Cup needs s t a b i l i z a t i o n during t ra ns po rt

  38. [46]

    "" } } # ===== R ea so ni ng Chain ===== R E A S O N I N G _ M E C H A N I S M = {

    Priority el ev ati on if customer w a i t i n g _ t i m e > 5 min """ } } # ===== R ea so ni ng Chain ===== R E A S O N I N G _ M E C H A N I S M = { " c o r e _ o p e r a t i o n _ c h a i n ": [ " Navigate -> Grasp -> Navigate -> Place " ] , " t a s k _ d e c o m p o s i t i...

  39. [47]

    "" # New minimal example

    Basic Op er at ion R ea son in g Chain : - M an da tor y d e c o m p o s i t i o n st ru ct ur e : { R E A S O N I N G _ M E C H A N I S M [ ’ c o r e _ o p e r a t i o n _ c h a i n ’]} - Standard workflow example : { R E A S O N I N G _ M E C H A N I S M [ ’ t a s k _ d e c ...

  40. [48]

    T o p o l o g i c a l N a v i g a t i o n : - Maintain mental map : { S C E N E _ U N D E R S T A N D I N G [ ’ g l o b a l _ t o p o l o g i c a l _ m a p ’][ ’ structure ’]} - Use regional c o n n e c t i o n s like { S C E N E _ U N D E R S T A N D I N G [ ’ g l o b a l _ t...

  41. [49]

    S i t u a t i o n a l Rea so ni ng : - Track r e l a t i o n s h i p s : { S C E N E _ U N D E R S T A N D I N G [ ’ l o c a l _ r e l a t i o n a l _ g r a p h ’] [ ’ re la ti ons hi p_ ed ge s ’][ ’ spatial ’]} - Make i n f e r e n c e s like { S C E N E _ U N D E R S T A N ...

  42. [50]

    keys () } - Follow workflow : { R O B O T _ C A P A B I L I T I E S [ ’ ta sk _d ec omp os it io n ’][ ’ example_workflow ’]} When r ec eiv in g requests :

    Skill O r c h e s t r a t i o n : - Compose p r i m i t i v e s : { R O B O T _ C A P A B I L I T I E S [ ’ primitive_skills ’]. keys () } - Follow workflow : { R O B O T _ C A P A B I L I T I E S [ ’ ta sk _d ec omp os it io n ’][ ’ example_workflow ’]} When r ec eiv in g requests :

  43. [51]

    Parse request into semantic c o m p o n e n t s

  44. [52]

    Cross - verify with spatial r e l a t i o n s h i p s

  45. [53]

    Generate e x e c u t a b l e skill sequence

  46. [54]

    Limitations

    Monitor e n v i r o n m e n t changes for a d a p t a t i o n """ } 19 NeurIPS Paper Checklist 1.Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: This paper claims tha...

  47. [55]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.