Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Off-the-shelf generative vision models can sample plausible 3D structures of occluded rooms, giving robots probabilistic priors over hidden geometry and target locations for planning without any fine-tuning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pretrained generative models can sample plausible 3D point clouds of hidden room regions from a doorway view, providing object and occupancy priors a motion planner can use without fine-tuning.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Promising zero-shot generative prior pipeline, but planning success is measured on the pipeline's own hallucinated point clouds, so the central environment-sampler claim is not yet supported. the 4 major comments →

arxiv 2510.11014 v2 pith:BP6MGIHC submitted 2025-10-13 cs.RO cs.AIcs.CV

MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models

Subhransu S. Bhattacharjee , Hao Lu , Dylan Campbell , Rahul Shome This is my paper
classification cs.RO cs.AIcs.CV
keywords generative priorszero-shotpartial observabilitymotion planning under uncertaintyspatio-semantic priorsenvironment sampling3D point cloud generationobject search
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a robot seeing only a doorway view of a room can still plan inside that room using priors sampled from pretrained generative vision models. The central claim is that a composite pipeline—language-model-guided image outpainting, monocular depth estimation, semantic segmentation, and 3D back-projection—acts as an 'environment sampler' that produces diverse, semantically labeled point-cloud hypotheses of the unobserved space. From these samples the authors derive spatio-semantic priors: per-configuration probabilities of collision and of reaching a target object, whose sample averages converge to the true probabilities if the sampler is well-behaved. On a new ten-scene benchmark of doorway-occluded indoor rooms, the sampled room-level object statistics align with ground-truth co-occurrence statistics, and a simulated robot solves object-reaching problems using these priors. The stakes: if the central assumption holds, robots can bootstrap reasoning about unseen space from internet-trained generative knowledge rather than hand-built maps.

Core claim

The paper's central claim is that a composite, zero-shot pipeline—vision-language-conditioned image outpainting followed by monocular depth, semantic segmentation, and back-projection—constitutes an environment sampler under a formal definition, and that the point-cloud samples it draws carry enough spatial and semantic structure to support configuration-space planning in occluded rooms. The authors formalize spatio-semantic priors via indicator queries: for any configuration, the sample average of 'in collision' or 'reaches object' over generated scenes estimates the true probability under environment uncertainty. They show that on their benchmark the scene-level object statistics of these

What carries the argument

The environment sampler abstraction. Definition 3 defines an environment sampler as any procedure drawing workspace realizations from the uncertain workspace distribution, and Definition 5 turns such a sampler into a generative spatio-semantic prior: the sample average of indicator functions over N draws, which converges to the true probability by the weak law of large numbers. The paper's pipeline operationalizes this sampler as the composition of a vision-language-model-guided outpainting model (which expands the partial view), a monocular metric depth estimator (which adds geometry), a semantic segmentation model (which labels object categories), and ray-preserving back-projection with al

Load-bearing premise

The central assumption is that a well-trained generative model conditioned on a partial observation acts as a true sampler of the hidden workspace distribution: its point-cloud samples converge asymptotically to the actual distribution of unobserved geometry and object locations, which is required for the sample-average probabilities used in planning to be calibrated.

What would settle it

Compute a ground-truth occupancy map of an occluded room (e.g., from a full scan) and compare it, in the hidden region only, to the occupancy predicted by many samples from the pipeline: if the average predicted occupancy has near-zero correlation with the true occupancy (or is no better than the room-type prior), the environment-sampler claim is falsified. The paper's own evaluations only check room-level object co-occurrence, not per-point geometry.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the central assumption holds, a robot can compute per-configuration collision probabilities in regions it has never observed, enabling uncertainty-aware planning from a single egocentric image.
  • The same samples yield target-object discovery probabilities, so object-search goals can be expressed as reaching configurations with high joint probability of being collision-free and reaching the object.
  • The formulation extends to entire trajectories by multiplying per-configuration probabilities, giving a principled cost for planners optimizing probability of task success.
  • Constraining the language-model prompt to contextually expected but non-visible objects measurably improves recovery of room-level object statistics, indicating that prompt design is a controlling factor.
  • The benchmark and pipeline provide an evaluation protocol for generative environment priors: KL divergence of object-in-room statistics plus simulated planning success.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Since the formalism is agnostic to the source of samples, any generative model with convergence guarantees could replace the outpainting chain; the paper's contribution is the interface between sampling and planning, not the specific models.
  • A necessary next test is spatial calibration: does the predicted probability of occupancy in the hidden region match the true occupancy at a per-voxel level? The paper evaluates room-level co-occurrence only, so the claim that geometry is useful for collision checking rests on the planning experiments, not on direct metric comparison.
  • One testable extension: measure whether planning success on real robots improves as sample count N grows, in the way the weak-law argument suggests; this would separate sampling error from model bias.
  • The prompting mechanism suggests a broader principle: generative priors for robotics can be steered by language models that know commonsense room contents, pointing toward compositional priors where scene grammar replaces hand-coded maps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes a zero-shot pipeline for sampling spatio-semantic priors of unobserved room structure from partial observations. Given a partial egocentric view through a doorway, the pipeline uses a VLM (Gemini) to propose a room type and a list of ten relevant objects, prompts the FLUX-Fill-dev outpainting model to expand the RGB image, estimates monocular metric depth with DepthPro, segments the image with SegFormer, and back-projects the RGB-D result into semantically labeled point clouds. The authors formalize spatio-semantic probability as the probability that a robot configuration satisfies a semantic property (collision or target acquisition) under environment uncertainty, and they introduce an 'environment sampler' assumption to connect generative models with planning. They contribute a 10-scene Matterport3D benchmark of doorway-occluded rooms, evaluate the sampled object statistics against dataset-level HM3D-Semantics object-in-room frequencies via KL divergence, and demonstrate object-reaching planning with PRM* on a simulated Stretch robot using the generated point clouds.

Significance. If the central claim holds, this is a notable step toward using off-the-shelf generative vision models as a source of zero-shot environment priors for planning under partial observability. The formalization of spatio-semantic priors is clean, the benchmark is a useful community resource, and the pipeline is constructed from publicly available models with an explicit intention to release code and data. The paper also clearly identifies its own limitation regarding model biases and runtime costs. However, the current evidence for the central claim is substantially incomplete: the generator is never directly validated against the true geometry and semantics of the occluded region, the KL evaluation uses dataset-level statistics rather than per-scene ground truth, and the planning evaluation is self-referential. These gaps currently prevent the paper from substantiating its strongest claims.

major comments (4)
  1. [Section 3.2, Central Assumption] The load-bearing premise of the paper is that the composite FLUX-outpainting → DepthPro → SegFormer → ICP chain is an environment sampler whose samples converge in probability to the true conditional workspace distribution. The convergence guarantee cited [37] applies to diffusion models solving stochastic inverse problems with a known forward model, not to this composite of a VLM-conditioned outpainter, monocular depth estimator, and semantic segmenter. No experiment measures whether the generated point clouds in the occluded region match the actual Matterport3D geometry. Without such a fidelity check, Definition 5 and the claim that the sample average approximates the true spatio-semantic probability are unsupported. I strongly recommend adding a direct per-scene evaluation: for each of the 10 scenes, compare the generated occluded-region point clouds against the known Matterport3D mes
  2. [Section 6, Eq. (1)] The KL divergence is computed against dataset-level HM3D-Semantics object-in-room frequencies, not against the actual contents of the specific hidden room. The per-scene ground-truth distribution appears to be derived from aggregate statistics over many rooms of the same type (as stated in Section 4), so an atypical scene would be penalized even if the samples exactly reproduce that scene's true contents. Additionally, the D_KL values in Table 2 are reported as single point estimates with no variance across the 100 samples or seeds, making it impossible to assess whether the differences between ablations (e.g., DKL-Det vs DKL-UnconDet) are significant. Please clarify precisely what the ground-truth distribution represents and report confidence intervals or standard deviations over the sample set.
  3. [Section 5.5, Table 3] The planning evaluation is self-referential. The PRM* planner's world model is exactly the point clouds produced by the pipeline, with observed-region ground-truth depth injected. Consequently, p_plan measures the planner's ability to reach a target object that exists in the hallucinated sample, not the probability that the trajectory would succeed in the actual room. A planner can 'succeed' by reaching a target that appears in the generated sample but is not present in the true hidden geometry. No experiment checks whether the planned trajectories are collision-free and goal-reaching when executed against the actual Matterport3D mesh. To break this circularity, the authors should validate the planned trajectories in the true mesh (e.g., report collision fraction and goal success when evaluated in the real geometry), or at minimum compare against a non-generative baseline such as an empt
  4. [Section 6] The paper lacks any non-generative baseline. The ablations in Table 2 compare different prompting strategies within the proposed pipeline, but there is no comparison to simple alternative priors, such as using observed-region object statistics, a fixed room-type prior, or a uniform distribution over object groups. Similarly, the planning results in Table 3 are not compared against a planner that uses only the observed region or a naive prior. Without such a baseline, it is unclear whether the generative pipeline provides any benefit over the room label alone, which undermines the claim that the point cloud samples are 'usable' and 'diverse' priors for planning.
minor comments (4)
  1. [Throughout] The manuscript contains many typographical artifacts, e.g., 'T o' in the abstract and Section 1, and 'W e' throughout. The title on the first page ('Into the Unknown: Towards using Generative Models for Sampling Priors of Environment Uncertainty for Planning in Configuration Spaces') differs from the arXiv title ('MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models'); please align them.
  2. [Section 5.5] The planner description is ambiguous: the text says PRM* is used, then that 'the resulting trajectories are subsequently optimized by formulating the problem as a mixed-integer quadratic program.' Please clarify the relationship between PRM* and the MIQP: does the MIQP refine PRM* solutions, and how are the probability-of-success factors encoded in the optimization objective?
  3. [Section 4] The dataset section states 'we only use the first 20 house scans (alphabetically) of the Matterport3D dataset' but the final dataset contains 10 scene images. Please clarify the selection process and whether these 10 images come from 10 distinct houses.
  4. [Section 5.2] The text says 'We repeat this process for 10 times per seed and prompt and obtain a total of 100 samples.' This is confusing: there is one prompt per scene, so does this mean 10 generations per prompt? Also, the 'global random seed of 1234 incremented by one for each sample' needs clarification—are the 10 generations per prompt given 10 distinct seeds?

Circularity Check

0 steps flagged

No circular derivation chain: the central premise is an explicitly labeled assumption, the semantic KL evaluation is against external HM3D-Semantics statistics, and the planning experiments are framed as in-simulation usability checks rather than fidelity predictions.

full rationale

I examined the derivation chain in Secs. 3-6 for the seven enumerated patterns. Definitions 1-5 are stipulative formalisms: an environment sampler is defined as a sampler of W, and the Central Assumption (Sec. 3.2) explicitly states that g_theta(O) will be assumed to be such a sampler. The convergence of the sample-average prior (Def. 4/5) is a mathematical consequence of that assumption, not an empirical prediction, and the paper labels it an assumption rather than importing a uniqueness theorem. The KL evaluation in Sec. 6 is calibrated against external HM3D-Semantics object-scene statistics, with ablations over constrained/unconstrained/no-object prompting; these are not quantities fitted to the evaluation target. The planning results in Sec. 5.5/Table 3 do use the generated point clouds as their own world model, so p_plan measures planner self-consistency on the samples rather than fidelity to real occluded geometry; however the paper does not present p_plan as a geometric fidelity check, and it explicitly notes in the introduction that it is 'in lieu of being able to compare against the unknown true spatio-semantic uncertainty of the unobserved world' and lists biases and limited evaluation scope in the conclusion. That is a validity/experimental-design caveat, not a reduction of an output to an input by construction. The only self-citations ([7] planner, [11] prompting methodology) are used as engineering tools, not as load-bearing justification for the central claim that generative models can serve as environment samplers. The cited convergence result [37] is for a different model class and is a correctness risk, but it is not a self-citation and does not make the derivation circular. Hence no specific circular step can be quoted.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The central probabilistic interpretation rests on the generative-model convergence assumption and on treating HM3D room-type object statistics as the ground-truth distribution. No new physical entities are introduced; the formal definitions (environment sampler, spatio-semantic prior) are bookkeeping, not postulates. The free parameters are prompt lists, filtering thresholds, hypernym groups, planning radii, and post-processing constants.

free parameters (6)
  • Object prompt list (10 objects per scene) = varied per scene, generated by Gemini
    The constrained/unconstrained prompts control what the generative model is asked to imagine; the detection distribution (and KL score) depends directly on these lists, and they are not derived from ground truth.
  • Detection filtering thresholds = min 1% image area; 20% confidence; Otsu thresholding
    Hand-chosen in Stage 3; changing thresholds changes predicted object distributions and downstream planning statistics.
  • Object group hypernyms = 10 collapsed ADE20K categories
    Semantic classes are collapsed by hand; the KL comparison only covers these 10 groups, which constrains the evaluation.
  • Target acquisition radius = 1 m
    Used in the planning success definition; the reported p_plan values depend directly on this radius.
  • Post-processing geometric parameters = floor trim 20 cm; depth cull 20 m; translation-only ICP
    Hand-set constants for point-cloud cleaning and alignment; different values would change occupancy samples and collision-checking.
  • Number of samples and seed schedule = 100 samples per scene; seed 1234 incremented
    The evaluation uses a fixed seed schedule and no variance across seeds is reported, so the stability of the KL and planning numbers is unknown.
axioms (6)
  • ad hoc to paper Central Assumption 3.2: a well-trained conditional generative model gθ(O) is an environment sampler whose samples converge asymptotically to the true conditional workspace distribution W_O.
    This is the load-bearing premise of the paper; it is assumed, with a citation to [37] for diffusion inverse problems, but not demonstrated for the FLUX-outpainting + depth + segmentation stack.
  • standard math The weak law of large numbers applies to the sampled point clouds as i.i.d. realizations of W.
    Used in Definition 4 to equate sample averages with probabilities; requires LLN conditions not checked for the highly correlated generative samples.
  • domain assumption HM3D-Semantics dataset-level object-in-room statistics are a valid ground truth for per-scene priors.
    Section 4 and Section 6 compute GT probabilities P_S from HM3D labels across scenes, not from the actual hidden contents of the specific test crop.
  • domain assumption Pretrained models (FLUX, DepthPro, SegFormer, Gemini) are 'sufficiently well-trained' for zero-shot transfer.
    The whole pipeline transfers these internet-scale models zero-shot; no fine-tuning or calibration is performed, and the paper explicitly relies on their training quality.
  • domain assumption Monocular depth predictions are metric-scale given FoV metadata, and translation-only ICP alignment is sufficient to register generated clouds with observed GT.
    Stage 4; if depth scale or alignment is wrong, occupancy samples in hidden regions would be geometrically invalid, breaking the planning evaluation.
  • domain assumption The convergence result of [37] transfers from diffusion forward models to flow-matching outpainting.
    The paper repeatedly invokes [37] to justify asymptotic convergence, but FLUX is not a diffusion forward model and the full pipeline includes non-generative stages (depth, segmentation, ICP).

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models." pith.science (2026). https://pith.science/paper/BP6MGIHC

@misc{pith2026251011014,
  author       = {Pith},
  title        = {Pith review of: MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BP6MGIHC}},
  note         = {Machine review of arXiv:2510.11014}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Autonomous robots often view rooms only partially, through a doorway, where the walls and scene structure hide the geometry and task-relevant semantics needed for safe navigation and goal-directed action. We ask whether off-the-shelf pretrained generative vision models can derive this missing structure as zero-shot offline priors for robot reasoning. Such priors should support spatio-semantic queries over unobserved structure, estimating the target object likelihood in hidden regions and the probability that those regions are occupied. Given an egocentric RGB observation and target query, our pipeline uses VLM-guided outpainting, monocular depth estimation, and semantic segmentation to sample semantically labeled 3D point cloud hypotheses of the hidden room. We introduce MatterDoor, a Matterport3D-derived benchmark of doorway-occluded indoor scenes, and evaluate the resulting priors with generative metrics and simulated Stretch robot object-reaching tasks. Our results suggest that useful spatio-semantic priors for planning can be derived without problem-specific fine-tuning.

Figures

Figures reproduced from arXiv: 2510.11014 by Dylan Campbell, Hao Lu, Rahul Shome, Subhransu S. Bhattacharjee.

Figure 1
Figure 1. Figure 1: In the top row, the two images represent partial views for the office (left) and bedroom (right). Shown alongside are two simulated motions from an uncertainty-aware planner using priors generated from our pipeline. The three sections from top to bottom show intermediate outputs from the proposed pipeline are the expanded RGB images, monocular depth for the RGB images, and expanded point cloud samples. Eac… view at source ↗
Figure 2
Figure 2. Figure 2: A generative model pipeline is presented which provides structured priors in 3D to reason and plan beyond the FoV and uncover the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Motivating examples from Matterport where large portions of rooms are occluded or visible only through doorways. Crops are shown in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FlatLands: Generative Floormap Completion From a Single Egocentric View

    cs.CV 2026-03 conditional novelty 7.0

    A new multi-source real indoor benchmark shows conditional generative models outperform deterministic and ensemble baselines at single-view BEV floor completion, with uncertainty concentrated at layout boundaries.

Reference graph

Works this paper leans on

70 extracted references · cited by 1 Pith paper

  1. [1]

    LQG-MP: Op- timized path planning for robots with motion uncertainty and imperfect state information,

    J. van den Berg, P. Abbeel, and K. Goldberg, “LQG-MP: Op- timized path planning for robots with motion uncertainty and imperfect state information,” Int. J. Robot. Res., 2011

    2011

  2. [2]

    Using occupancy grids for mobile robot perception and navigation,

    A. Elfes, “Using occupancy grids for mobile robot perception and navigation,” Computer, 1989

    1989

  3. [3]

    Monte-carlo planning in large POMDPs,

    D. Silver and J. V eness, “Monte-carlo planning in large POMDPs,” in NeurIPS, 2010

    2010

  4. [4]

    DimSam: Diffusion models as samplers for task and motion planning under partial observability,

    X. Fang, C. R. Garrett, C. Eppner, T. Lozano-P´erez, L. P. Kaelbling, and D. Fox, “DimSam: Diffusion models as samplers for task and motion planning under partial observability,” inIROS, 2024. T able 2: Per-ablation evaluation reportingDKL divergence as described in Eq. (1). Best value per scene is in bold. Bedroom1 Bedroom2 Office1 Office2 Kitchen1 Kitch...

    2024

  5. [5]

    Semantic linking maps for active visual object search,

    Z. Zeng, A. R ¨ofer, and O. C. Jenkins, “Semantic linking maps for active visual object search,” in IJCAI, 2021

    2021

  6. [6]

    Uncertainty-aware occupancy map prediction using generative networks for robot navigation,

    K. Katyal, K. Popek, C. Paxton, P. Burlina, and G. D. Hager, “Uncertainty-aware occupancy map prediction using generative networks for robot navigation,” in ICRA, 2019

    2019

  7. [7]

    Sampling-based motion planning for optimal probability of collision under environment uncertainty,

    H. Lu, H. Kurniawati, and R. Shome, “Sampling-based motion planning for optimal probability of collision under environment uncertainty,” in IROS, 2024

    2024

  8. [8]

    Provably safe robot navigation with obstacle uncertainty,

    B. Axelrod, L. P. Kaelbling, and T. Lozano-P´erez, “Provably safe robot navigation with obstacle uncertainty,” Int. J. Robot. Res. , 2018

    2018

  9. [9]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” in NeurIPS W orkshops, 2021

    2021

  10. [10]

    SDXL: Improving latent diffusion models for high-resolution image synthesis,

    D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, and R. Rombach, “SDXL: Improving latent diffusion models for high-resolution image synthesis,” in ICLR, 2024

    2024

  11. [11]

    Believing is Seeing: Unobserved object detection using generative models,

    S. S. Bhattacharjee, D. Campbell, and R. Shome, “Believing is Seeing: Unobserved object detection using generative models,” in CVPR, 2025

    2025

  12. [12]

    Flux.1 Kontext: Flow matching for in-context image generation and editing in latent space,

    Black Forest Labs, S. Batifol, A. Blattmann, F. Boesel, S. Consul, C. Diagne, T. Dockhorn, J. English, Z. English, P. Esser, S. Kulal, K. Lacey, Y. Levi, C. Li, D. Lorenz, J. M¨uller, D. Podell, R. Rom- bach, H. Saini, A. Sauer, and L. Smith, “Flux.1 Kontext: Flow matching for in-context image generation and editing in latent space,” 2025, preprint

    2025

  13. [13]

    Depth pro: Sharp monocular metric depth in less than a second,

    A. Bochkovskii, A. Delaunoy, H. Germain, M. Santos, Y. Zhou, S. R. Richter, and V . Koltun, “Depth pro: Sharp monocular metric depth in less than a second,” in ICLR, 2025

    2025

  14. [14]

    SegFormer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W . W ang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” in NeurIPS, 2021

    2021

  15. [15]

    NeRF: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. T ancik, J. T. Barron, R. Ra- mamoorthi, and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,” in ECCV, 2020

    2020

  16. [16]

    3D gaus- sian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk¨uhler, and G. Drettakis, “3D gaus- sian splatting for real-time radiance field rendering,” ACM TOG, 2023

    2023

  17. [17]

    N-view computational methods,

    R. Hartley and A. Zisserman, “N-view computational methods,” in Multiple View Geometry in Computer Vision . Cambridge University Press, 2004

    2004

  18. [18]

    Semantic scene completion from a single depth image,

    S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in CVPR, 2017

    2017

  19. [19]

    Matterport3D: Learn- ing from RGB-D data in indoor environments,

    A. X. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y. Zhang, “Matterport3D: Learn- ing from RGB-D data in indoor environments,” in 3DV, 2017

    2017

  20. [20]

    Partially observable markov decision processes and robotics,

    H. Kurniawati, “Partially observable markov decision processes and robotics,” Annu. Rev. Control Robot. Auton. Syst., 2022

    2022

  21. [21]

    Seeing is Believing: Belief-space planning with foundation models as uncertainty estimators,

    L. Zhao, W . McClinton, A. Curtis, N. Kumar, T. Silver, L. P. Kael- bling, and L. L. S. W ong, “Seeing is Believing: Belief-space planning with foundation models as uncertainty estimators,” arXiv, 2025

    2025

  22. [22]

    Prior-assisted propagation of spatial information for object search,

    M. Lorbach, S. H¨ofer, and O. Brock, “Prior-assisted propagation of spatial information for object search,” in IROS, 2014

    2014

  23. [23]

    Motion planning diffusion: Learning and planning of robot motions with diffusion models,

    J. Carvalho, A. Le, M. Baierl, D. Koert, and J. Peters, “Motion planning diffusion: Learning and planning of robot motions with diffusion models,” in IROS, 2023

    2023

  24. [24]

    Dream to control: Learning behaviors by latent imagination,

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” in ICLR, 2020

    2020

  25. [25]

    Deep varia- tional reinforcement learning for POMDPs,

    M. Igl, L. Zintgraf, T. Le, F. W ood, and S. Whiteson, “Deep varia- tional reinforcement learning for POMDPs,” in ICLR, 2018

    2018

  26. [26]

    Diffuscene: Denoising diffusion models for generative indoor scene synthesis,

    J. T ang, Y. Nie, L. Markhasin, A. Dai, J. Thies, and M. Niessner, “Diffuscene: Denoising diffusion models for generative indoor scene synthesis,” in CVPR, 2024

    2024

  27. [27]

    Octomap: An efficient probabilistic 3D mapping framework based on octrees,

    A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W . Burgard, “Octomap: An efficient probabilistic 3D mapping framework based on octrees,” Auton. Robots, 2013

    2013

  28. [28]

    Visual semantic navigation using scene priors,

    W . Y ang, X. W ang, A. Farhadi, A. Gupta, and R. Mottaghi, “Visual semantic navigation using scene priors,” in ICLR, 2019

    2019

  29. [29]

    V ariational diffu- sion models,

    D. P. Kingma, T. Salimans, B. Poole, and J. Ho, “V ariational diffu- sion models,” in NeurIPS, 2021

    2021

  30. [30]

    Generative adversar- ial nets,

    I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. W arde- Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversar- ial nets,” in NeurIPS, 2014

    2014

  31. [31]

    V ariational inference with nor- malizing flows,

    D. J. Rezende and S. Mohamed, “V ariational inference with nor- malizing flows,” in ICML, ser. Proc. Mach. Learn. Res., 2015

    2015

  32. [32]

    Density estimation using real NVP,

    L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real NVP,” in ICLR, 2017

    2017

  33. [33]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in NeurIPS, 2020

    2020

  34. [34]

    Score-based generative modeling through stochastic differential equations,

    Y. Song and S. Ermon, “Score-based generative modeling through stochastic differential equations,” in ICLR, 2021

    2021

  35. [35]

    CamCtrl3D: Single-image scene exploration with precise 3D camera control,

    S. Popov, A. Raj, Y. Li, M. Krainin, W . T. Freeman, and M. Rubin- stein, “CamCtrl3D: Single-image scene exploration with precise 3D camera control,” in 3DV, 2025

    2025

  36. [36]

    GEN3C: 3D-informed world-consistent video generation with precise camera control,

    X. Ren, T. Shen, J. Huang, H. Ling, Y. Lu, M. Nimier-David, T. M¨uller, A. Keller, S. Fidler, and J. Gao, “GEN3C: 3D-informed world-consistent video generation with precise camera control,” in CVPR, 2025

    2025

  37. [37]

    Diffusion with forward models: Solving stochastic inverse problems without direct supervision,

    A. T ewari, T. Yin, G. Cazenavette, S. Rezchikov, J. B. T enenbaum, F. Durand, W . T. Freeman, and V . Sitzmann, “Diffusion with forward models: Solving stochastic inverse problems without direct supervision,” in NeurIPS, 2023

    2023

  38. [38]

    Depth Anything V2,

    L. Y ang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao, “Depth Anything V2,” in NeurIPS, 2024

    2024

  39. [39]

    Scaling rectified flow transformers for high-resolution image syn- thesis,

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M¨uller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, D. Podell, T. Dockhorn, Z. English, K. Lacey, A. Goodwin, Y. Marek, and R. Rombach, “Scaling rectified flow transformers for high-resolution image syn- thesis,” in ICML, 2024

    2024

  40. [40]

    Flow matching for generative modeling,

    Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and L. Le, “Flow matching for generative modeling,” in ICLR, 2023

    2023

  41. [41]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W . Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agar- wal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in ICML, 2021

    2021

  42. [42]

    A survey on object instance segmentation,

    R. Sharma, M. Saqib, C.-T. Lin, and M. Blumenstein, “ A survey on object instance segmentation,” SN Comput. Sci., 2022

    2022

  43. [43]

    Amodal instance segmentation,

    K. Li and J. Malik, “ Amodal instance segmentation,” in ECCV, 2016

    2016

  44. [44]

    Segan: Segmenting and generating the invisible,

    K. Ehsani, R. Mottaghi, and A. Farhadi, “Segan: Segmenting and generating the invisible,” in CVPR, 2018

    2018

  45. [45]

    Probabilistic algorithms in robotics,

    S. Thrun, “Probabilistic algorithms in robotics,” AI Mag., 2000

    2000

  46. [46]

    Habitat-Matterport 3D Dataset (HM3D): 1000 large-scale 3D environments for embodied AI,

    S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, J. Turner, A. Clegg, E. Undersander, W . Galuba, A. X. Chang, M. Savva, and D. Ba- tra, “Habitat-Matterport 3D Dataset (HM3D): 1000 large-scale 3D environments for embodied AI,” in NeurIPS Datasets and Benchmarks Track, 2021

    2021

  47. [47]

    Habitat-Matterport 3D Seman- tics Dataset,

    K. Y adav, R. Ramrakhya, S. K. Ramakrishnan, T. Gervet, J. M. Turner, A. Gokaslan, N. Maestre, A. X. Chang, D. Batra, M. Savva, A. W . Clegg, and D. S. Chaplot, “Habitat-Matterport 3D Seman- tics Dataset,” in CVPR, 2023

    2023

  48. [48]

    The Design of Stretch: A compact, lightweight mobile manipulator for indoor human environments,

    C. C. Kemp, A. Edsinger, H. M. Clever, and B. Matulevich, “The Design of Stretch: A compact, lightweight mobile manipulator for indoor human environments,” in ICRA, 2022

    2022

  49. [49]

    Akenine-M¨oller, E

    T. Akenine-M¨oller, E. Haines, N. Hoffman, A. Pesce, M. Iwanicki, and S. Hillaire, Real-Time Rendering, 4th ed. CRC Press, 2018

    2018

  50. [50]

    Open3D: A modern library for 3d data processing,

    Q.-Y. Zhou, J. Park, and V . Koltun, “Open3D: A modern library for 3d data processing,” arXiv, 2018

    2018

  51. [51]

    Gemini: A family of highly capable multi- modal models,

    Gemini T eam Google, “Gemini: A family of highly capable multi- modal models,” arXiv, 2023

    2023

  52. [52]

    Scene parsing through ADE20K dataset,

    B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. T orralba, “Scene parsing through ADE20K dataset,” in CVPR, 2017

    2017

  53. [53]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W . Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., 2020

    2020

  54. [54]

    SVDQuant: Absorbing outliers by low-rank components for 4-bit diffusion models,

    M. Li, Y. Lin, Z. Zhang, T. Cai, X. Li, J. Guo, E. Xie, C. Meng, J.-Y. Zhu, and S. Han, “SVDQuant: Absorbing outliers by low-rank components for 4-bit diffusion models,” in ICLR, 2025

    2025

  55. [55]

    Diffusers: State-of-the-art diffusion models,

    P. von Platen, S. Patil, A. Lozhkov, P. Cuenca, N. Lambert, K. Ra- sul, M. Davaadorj, D. Nair, S. Paul, W . Berman, Y. Xu, S. Liu, and T. W olf, “Diffusers: State-of-the-art diffusion models,” 2022

    2022

  56. [56]

    A threshold selection method from gray-level his- tograms,

    N. Otsu, “ A threshold selection method from gray-level his- tograms,” IEEE Trans. Syst., Man, Cybern., 1979

    1979

  57. [57]

    Open vocabulary scene parsing,

    H. Zhao, X. Puig, B. Zhou, S. Fidler, and A. T orralba, “Open vocabulary scene parsing,” in ICCV, 2017

    2017

  58. [58]

    Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,

    M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,”Commun. ACM, 1981

    1981

  59. [59]

    Generalized-ICP,

    A. V . Segal, D. H¨ahnel, and S. Thrun, “Generalized-ICP,” in RSS, 2009

    2009

  60. [60]

    Sampling-based algorithms for opti- mal motion planning,

    S. Karaman and E. Frazzoli, “Sampling-based algorithms for opti- mal motion planning,” Int. J. Robot. Res., 2011

    2011

  61. [61]

    The open motion plan- ning library,

    I. A. Sucan, M. Moll, and L. E. Kavraki, “The open motion plan- ning library,” IEEE Robot. Autom. Mag., 2012

    2012

  62. [62]

    Robot operating system 2: Design, architecture, and uses in the wild,

    S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W . W oodall, “Robot operating system 2: Design, architecture, and uses in the wild,” Sci. Robot., 2022

    2022

  63. [63]

    Reducing the barrier to entry of complex robotic software: A MoveIt! case study,

    D. Coleman, I. A. S ¸ucan, S. Chitta, and N. Correll, “Reducing the barrier to entry of complex robotic software: A MoveIt! case study,” J. Softw. Eng. Robot., 2014

    2014

  64. [64]

    FCL: A general purpose library for collision and proximity queries,

    J. Pan, S. Chitta, and D. Manocha, “FCL: A general purpose library for collision and proximity queries,” in ICRA, 2012

    2012

  65. [65]

    Gurobi Optimization, LLC, Gurobi Optimizer Reference Manual, 2023

    2023

  66. [66]

    Peyr´e and M

    G. Peyr´e and M. Cuturi, Computational Optimal Transport: With Applications to Data Science. Now Publishers, 2019

    2019

  67. [67]

    Active visual object search in unknown environments using uncertain semantics,

    A. Aydemir, A. Pronobis, M. G¨obelbecker, and P. Jensfelt, “ Active visual object search in unknown environments using uncertain semantics,” IEEE Trans. Robot., 2013

    2013

  68. [68]

    A survey on active simultaneous localization and mapping: State of the art and new frontiers,

    J. A. Placed, J. Strader, H. Carrillo, N. Atanasov, V . Indelman, L. Carlone, and J. A. Castellanos, “ A survey on active simultaneous localization and mapping: State of the art and new frontiers,”IEEE Trans. Robot., 2023

    2023

  69. [69]

    Ob- ject goal navigation using goal-oriented semantic exploration,

    D. S. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov, “Ob- ject goal navigation using goal-oriented semantic exploration,” in NeurIPS, 2020

    2020

  70. [70]

    Semantic robot programming for goal-directed manipulation in cluttered scenes,

    Z. Zeng, Z. Zhou, Y. Sui, and O. C. Jenkins, “Semantic robot programming for goal-directed manipulation in cluttered scenes,” in ICRA, 2018

    2018

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.