A multi-view transformer predicts dense perspective fields that feed a geometric optimizer to estimate camera intrinsics and gravity from arbitrary numbers of real-world views.
hub
Joint 2D-3D-Semantic Data for Indoor Scene Understanding
21 Pith papers cite this work, alongside 686 external citations. Polarity classification is still indexing.
21
Pith papers citing it
686
external citations · Pith
abstract
We present a dataset of large-scale indoor spaces that provides a variety of mutually registered modalities from 2D, 2.5D and 3D domains, with instance-level semantic and geometric annotations. The dataset covers over 6,000m2 and contains over 70,000 RGB images, along with the corresponding depths, surface normals, semantic annotations, global XYZ images (all in forms of both regular and 360{\deg} equirectangular images) as well as camera information. It also includes registered raw and semantically annotated 3D meshes and point clouds. The dataset enables development of joint and cross-modal learning models and potentially unsupervised approaches utilizing the regularities present in large-scale indoor spaces. The dataset is available here: http://3Dsemantics.stanford.edu/
hub tools
citation-role summary
dataset 3
background 1
citation-polarity summary
representative citing papers
HM3D offers 1000 building-scale 3D environments that are larger and higher-fidelity than existing datasets, enabling better-performing embodied AI agents for tasks like PointGoal navigation.
DepthMaster unifies metric monocular depth estimation for perspective and panoramic images by patching panoramas into perspective views, adding a consistency loss and virtual cameras, and training mostly on perspective data to reach SOTA zero-shot results on 13 datasets.
LangTail uses entity-level semantic priors from language models aligned via contrastive learning in a hierarchical clustering setup to resolve long-tail ambiguity, yielding +13.5, +12.9, and +8.9 mIoU gains on ScanNet-v2, S3DIS, and nuScenes.
VGGT-360 delivers geometry-consistent zero-shot panoramic depth by converting panoramas into multi-view 3D reconstructions via VGGT models and three plug-and-play correction modules, then reprojecting the result.
Presents a novel integer linear programming approach for automatic reconstruction of volumetric, parametric, multi-story building models from unstructured point clouds without requiring initial room segmentation.
PointGS achieves semantic-consistent unsupervised 3D point cloud segmentation by using 3D Gaussian Splatting to bridge discrete points and continuous 2D images for distilling SAM semantics.
A hybrid pipeline combines semantic segmentation of point clouds with topology-aware reconstruction to generate BIM models, introduces the vIoU evaluation metric, and releases the DeKH dataset with demonstrated gains over RANSAC baselines.
STRNet improves goal-conditioned visual navigation by replacing simplistic encoders and pooling with a spatio-temporal fusion module that performs spatial graph reasoning and hybrid temporal modeling.
EAGLE-360 introduces a global-to-local exploration framework for 360° visual search, adapting RoPE Rolling, creating a new VQA dataset, and using SFT+GRPO training to claim SOTA performance with 8x accuracy gain.
Argus plus Realsee3D deliver state-of-the-art metric camera pose, depth, and point-cloud reconstruction from unordered indoor panoramas via learned covisibility anchoring and geometric factorization.
FoundObj uses foundation-model priors as RL rewards to discover multi-class 3D objects from point clouds without scene-level labels.
Holo360D provides 109,495 real-world panoramas with continuous trajectories and high-completeness depth maps, plus a benchmark showing fine-tuning on this dataset improves feed-forward 3D reconstruction models.
PanoSAMic modifies SAM with multi-stage feature encoding, spatio-modal fusion, spherical attention, and dual-view fusion to achieve SOTA panoramic semantic segmentation on public RGB and RGB-D datasets.
PointCaM proposes a cut-and-mix mechanism with an Unknown-Point Simulator and Estimator to improve open-set recognition on point clouds by simulating out-of-distribution data and using multi-level features.
A survey diagnosing panoramic scene understanding as a field that converged on compatibility-preserving geometric adaptation rather than sphere-native modeling, while its evaluation protocols systematically fail to measure spherical understanding.
ORION applies ordinal neural collapse to organize visual encoder features along an action-ordinal axis, then integrates the encoder into a diffusion navigation policy, yielding higher success rates than end-to-end and standard neural collapse baselines in simulation and real-world tests.
EvObj learns evolving object-centric representations for unsupervised 3D instance segmentation by dynamically refining object candidates and completing partial geometries to bridge the synthetic-to-real domain gap, outperforming baselines on real and synthetic datasets.
Gaussian and related cropping strategies for point cloud subclouds improve 3D neural network performance over spherical cropping on large outdoor scenes.
INSIGHT transfers 2D semantic understanding from foundation models and traditional CV tools into 3D point clouds and compressed scene graphs for indoor public-safety mapping without target-domain labels.
A survey of deep learning architectures for 3D sensed data classification covering RGB-D, multi-view, volumetric and end-to-end methods along with datasets and future directions.
citing papers explorer
-
CalibAnyView: Beyond Single-View Camera Calibration in the Wild
A multi-view transformer predicts dense perspective fields that feed a geometric optimizer to estimate camera intrinsics and gravity from arbitrary numbers of real-world views.
-
Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
HM3D offers 1000 building-scale 3D environments that are larger and higher-fidelity than existing datasets, enabling better-performing embodied AI agents for tasks like PointGoal navigation.
-
DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images
DepthMaster unifies metric monocular depth estimation for perspective and panoramic images by patching panoramas into perspective views, adding a consistency loss and virtual cameras, and training mostly on perspective data to reach SOTA zero-shot results on 13 datasets.
-
Resolving Long-Tail Ambiguity in Unsupervised 3D Point Cloud Segmentation with Language Priors
LangTail uses entity-level semantic priors from language models aligned via contrastive learning in a hierarchical clustering setup to resolve long-tail ambiguity, yielding +13.5, +12.9, and +8.9 mIoU gains on ScanNet-v2, S3DIS, and nuScenes.
-
VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation
VGGT-360 delivers geometry-consistent zero-shot panoramic depth by converting panoramas into multi-view 3D reconstructions via VGGT models and three plug-and-play correction modules, then reprojecting the result.
-
Automatic reconstruction of fully volumetric 3D building models from point clouds
Presents a novel integer linear programming approach for automatic reconstruction of volumetric, parametric, multi-story building models from unstructured point clouds without requiring initial room segmentation.
-
PointGS: Semantic-Consistent Unsupervised 3D Point Cloud Segmentation with 3D Gaussian Splatting
PointGS achieves semantic-consistent unsupervised 3D point cloud segmentation by using 3D Gaussian Splatting to bridge discrete points and continuous 2D images for distilling SAM semantics.
-
BIMStruct3D: A Fully Automated Hybrid Learning Scan-to-BIM Pipeline with Integrated Topology Refinement
A hybrid pipeline combines semantic segmentation of point clouds with topology-aware reconstruction to generate BIM models, introduces the vIoU evaluation metric, and releases the DeKH dataset with demonstrated gains over RANSAC baselines.
-
STRNet: Visual Navigation with Spatio-Temporal Representation through Dynamic Graph Aggregation
STRNet improves goal-conditioned visual navigation by replacing simplistic encoders and pooling with a spatio-temporal fusion module that performs spatial graph reasoning and hybrid temporal modeling.
-
EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$
EAGLE-360 introduces a global-to-local exploration framework for 360° visual search, adapting RoPE Rolling, creating a new VQA dataset, and using SFT+GRPO training to claim SOTA performance with 8x accuracy gain.
-
Argus: Metric Panoramic 3D Reconstruction for Indoor Scenes
Argus plus Realsee3D deliver state-of-the-art metric camera pose, depth, and point-cloud reconstruction from unordered indoor panoramas via learned covisibility anchoring and geometric factorization.
-
FoundObj: Self-supervised Foundation Models as Rewards for Label-free 3D Object Segmentation
FoundObj uses foundation-model priors as RL rewards to discover multi-class 3D objects from point clouds without scene-level labels.
-
Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyond
Holo360D provides 109,495 real-world panoramas with continuous trajectories and high-completeness depth maps, plus a benchmark showing fine-tuning on this dataset improves feed-forward 3D reconstruction models.
-
PanoSAMic: Panoramic Image Segmentation from SAM Feature Encoding and Dual View Fusion
PanoSAMic modifies SAM with multi-stage feature encoding, spatio-modal fusion, spherical attention, and dual-view fusion to achieve SOTA panoramic semantic segmentation on public RGB and RGB-D datasets.
-
PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning
PointCaM proposes a cut-and-mix mechanism with an Unknown-Point Simulator and Estimator to improve open-set recognition on point clouds by simulating out-of-distribution data and using multi-level features.
-
Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling
A survey diagnosing panoramic scene understanding as a field that converged on compatibility-preserving geometric adaptation rather than sphere-native modeling, while its evaluation protocols systematically fail to measure spherical understanding.
-
Ordinal Neural Collapse as a Representation Prior for Visual Navigation
ORION applies ordinal neural collapse to organize visual encoder features along an action-ordinal axis, then integrates the encoder into a diffusion navigation policy, yielding higher success rates than end-to-end and standard neural collapse baselines in simulation and real-world tests.
-
EvObj: Learning Evolving Object-centric Representations for 3D Instance Segmentation without Scene Supervision
EvObj learns evolving object-centric representations for unsupervised 3D instance segmentation by dynamically refining object candidates and completing partial geometries to bridge the synthetic-to-real domain gap, outperforming baselines on real and synthetic datasets.
-
From Spherical to Gaussian: A Comparative Analysis of Point Cloud Cropping Strategies in Large-Scale 3D Environments
Gaussian and related cropping strategies for point cloud subclouds improve 3D neural network performance over spherical cropping on large outdoor scenes.
-
INSIGHT: Indoor Scene Intelligence from Geometric-Semantic Hierarchy Transfer for Public~Safety
INSIGHT transfers 2D semantic understanding from foundation models and traditional CV tools into 3D point clouds and compressed scene graphs for indoor public-safety mapping without target-domain labels.
-
A review on deep learning techniques for 3D sensed data classification
A survey of deep learning architectures for 3D sensed data classification covering RGB-D, multi-view, volumetric and end-to-end methods along with datasets and future directions.