Pith. sign in

REVIEW 2 major objections 2 minor 65 cited by

ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data

T0 review · 2 major / 2 minor · reviewed 2026-05-15 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ARKitScenes is the largest indoor RGB-D dataset captured with widely available mobile LiDAR sensors and includes laser-scanned depth plus manual 3D bounding box labels.

desk verdict ARKitScenes releases a large mobile RGB-D dataset from real Apple LiDAR hardware plus laser ground truth and 3D boxes, which is the useful part even if the paper stays mostly descriptive. read the letter →

arxiv 2111.08897 v3 pith:PRRIYH5H submitted 2021-11-17 cs.CV cs.AI

Gilad Baruch , Zhuoyuan Chen , Afshin Dehghan , Tal Dimry , Yuri Feigin , Peter Fu , Thomas Gebauer , Brandon Joffe
show 3 more authors
Daniel Kurz Arik Schwartz Elad Shulman
This is my paper · ORCID
classification cs.CVcs.AI
keywords RGB-Ddatasetindoorsceneunderstanding3DobjectdetectiondepthupsamplingmobileLiDARboundingboxesreal-worldARKitScenes
checked against Foundation.DimensionForcingFoundation.HierarchyEmergence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents ARKitScenes, a dataset of RGB-D captures collected from Apple iPads and iPhones that have LiDAR sensors. It augments the raw mobile data with high-resolution depth maps from a stationary laser scanner and manual 3D oriented bounding box labels for a large set of furniture categories. The authors test the data on two tasks, 3D object detection and color-guided depth upsampling, and report that it improves existing methods while exposing challenges closer to everyday conditions. A sympathetic reader would care because the captures come from devices already owned by millions of people, moving 3D scene understanding from controlled lab settings toward practical mobile use.

What carries the argument

The ARKitScenes dataset that pairs mobile RGB-D captures with laser-scanner depth maps and manual 3D bounding box annotations for indoor furniture.

What would settle it

A controlled test in which models trained on ARKitScenes show no improvement over models trained on prior datasets when evaluated on independent mobile RGB-D captures from varied indoor rooms would falsify the usefulness claim.

Watch

Extended reading notes

Core claim

ARKitScenes is the first RGB-D dataset captured with the widely available depth sensor on iPads and iPhones and the largest indoor scene understanding dataset released. It supplies raw and processed mobile device data, high-resolution depth maps from a stationary laser scanner, and manually labeled 3D oriented bounding boxes for furniture. Evaluation on 3D object detection and color-guided depth upsampling shows the dataset pushes state-of-the-art performance and introduces new real-world challenges.

Load-bearing premise

The mobile RGB-D captures, laser-scanned depth maps, and manual 3D bounding box labels are sufficiently accurate and representative of real-world indoor scenes to advance state-of-the-art methods.

Editorial extensions

If this is right

  • 3D object detection models achieve higher accuracy on large furniture taxonomies when trained with the labeled mobile data.
  • Color-guided depth upsampling produces higher-resolution outputs by using the laser scans as precise ground truth.
  • The dataset scale supports training larger machine-learning models for indoor scene understanding.
  • Methods developed on the data must handle noise and viewpoint variation typical of handheld mobile captures.
  • The combination of mobile and laser data creates a bridge between consumer hardware and high-precision references.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • App developers could fine-tune models on this data to add room-layout awareness to consumer AR experiences without extra hardware.
  • The dataset could be used to study how well algorithms generalize from mobile captures to other depth sensors.
  • Future releases might add semantic segmentation labels or dynamic object tracks to extend the current static bounding-box focus.
  • Cross-validation across different device models within the captures could reveal hardware-specific biases in depth sensing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces ARKitScenes as the first RGB-D dataset captured with Apple's widely available LiDAR sensor on mobile iPads/iPhones and, to the authors' knowledge, the largest indoor scene understanding dataset released. It supplies raw and processed mobile RGB-D captures, registered high-resolution depth maps from a stationary laser scanner, and manually annotated 3D oriented bounding boxes over a furniture taxonomy. The authors compare scale and characteristics to prior datasets (ScanNet, Matterport3D) and demonstrate utility on two downstream tasks: 3D object detection and color-guided depth upsampling, claiming the data pushes SOTA boundaries while introducing real-world challenges.

Significance. If the scale, registration quality, and annotation accuracy hold, the release supplies a high-value resource whose mobile capture characteristics better match everyday consumer hardware than prior lab-style datasets. This can accelerate development of robust 3D scene understanding methods for mobile applications, with the laser-scanned depths and 3D boxes providing strong supervision signals for detection and upsampling benchmarks.

major comments (2)
  1. [§4] §4 (Dataset Statistics): the central claim that ARKitScenes is the largest indoor dataset requires an explicit side-by-side table (number of scenes, frames, annotated objects, capture conditions) against ScanNet and Matterport3D; without these numbers the size/diversity assertion is unsupported.
  2. [§6] §6 (Downstream Tasks): the demonstrations for 3D object detection and depth upsampling must report concrete metrics (mAP, RMSE, etc.) and baselines; the abstract states only that the data 'pushes boundaries' without evidence, which is load-bearing for the utility claim.
minor comments (2)
  1. Figure captions should explicitly state what each panel shows (RGB, mobile depth, laser depth, projected boxes) and include scale bars or units.
  2. [§3] The taxonomy of furniture classes and the exact annotation protocol (number of annotators, quality control) should be listed in a dedicated subsection or table.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the positive recommendation of minor revision and the constructive comments. We address each point below.

read point-by-point responses

  1. Referee: [§4] §4 (Dataset Statistics): the central claim that ARKitScenes is the largest indoor dataset requires an explicit side-by-side table (number of scenes, frames, annotated objects, capture conditions) against ScanNet and Matterport3D; without these numbers the size/diversity assertion is unsupported.

    Authors: We agree that an explicit comparison table will strengthen the claim. In the revised manuscript we will insert a side-by-side table in §4 that reports number of scenes, frames, annotated objects, and capture conditions for ARKitScenes, ScanNet, and Matterport3D. revision: yes

  2. Referee: [§6] §6 (Downstream Tasks): the demonstrations for 3D object detection and depth upsampling must report concrete metrics (mAP, RMSE, etc.) and baselines; the abstract states only that the data 'pushes boundaries' without evidence, which is load-bearing for the utility claim.

    Authors: We will revise the abstract to include the key quantitative results (mAP for detection and RMSE for upsampling) and will ensure §6 explicitly lists all metrics together with the baselines used. This will provide the concrete evidence requested. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The paper is a dataset release paper whose central claims concern the scale, sensor type, and annotation quality of ARKitScenes itself. No mathematical derivations, fitted parameters, or predictions appear in the manuscript. Claims of being the first LiDAR-based RGB-D dataset and the largest indoor scene-understanding release are supported by explicit size statistics and direct comparisons to ScanNet, Matterport3D, and similar prior releases, none of which reduce to self-citation chains or self-definitional loops. The two downstream-task demonstrations (3D object detection and depth upsampling) are empirical evaluations on the released data rather than derivations that collapse to their own inputs. The work is therefore self-contained against external benchmarks with no load-bearing circular steps.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central contribution is a new data collection and annotation effort rather than a derivation; the main assumptions concern sensor accuracy and label quality, which are domain-standard for RGB-D datasets.

assumptions (1)
  • domain assumption Mobile RGB-D sensors such as Apple's LiDAR produce depth data of sufficient quality for indoor scene understanding tasks
    Invoked when positioning the dataset as enabling real-world applications

how reviews work

0 comments
Cite this review

Pith. "Pith review of ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data." pith.science (2026). https://pith.science/paper/PRRIYH5H

@misc{pith2026211108897,
  author       = {Pith},
  title        = {Pith review of: ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PRRIYH5H}},
  note         = {Machine review of arXiv:2111.08897}
}
read the original abstract

Scene understanding is an active research area. Commercial depth sensors, such as Kinect, have enabled the release of several RGB-D datasets over the past few years which spawned novel methods in 3D scene understanding. More recently with the launch of the LiDAR sensor in Apple's iPads and iPhones, high quality RGB-D data is accessible to millions of people on a device they commonly use. This opens a whole new era in scene understanding for the Computer Vision community as well as app developers. The fundamental research in scene understanding together with the advances in machine learning can now impact people's everyday experiences. However, transforming these scene understanding methods to real-world experiences requires additional innovation and development. In this paper we introduce ARKitScenes. It is not only the first RGB-D dataset that is captured with a now widely available depth sensor, but to our best knowledge, it also is the largest indoor scene understanding data released. In addition to the raw and processed data from the mobile device, ARKitScenes includes high resolution depth maps captured using a stationary laser scanner, as well as manually labeled 3D oriented bounding boxes for a large taxonomy of furniture. We further analyze the usefulness of the data for two downstream tasks: 3D object detection and color-guided depth upsampling. We demonstrate that our dataset can help push the boundaries of existing state-of-the-art methods and it introduces new challenges that better represent real-world scenarios.

Discussion (0). Continue with ORCID to comment.

Forward citations

Showing 60 of 65 Pith papers that cite this

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. See all 65 Pith citations

  1. WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlife

    cs.CV 2026-06 unverdicted novelty 8.0 of 10

    WildBox provides over 237k 3D wildlife annotations from drone video and benchmarks reveal zero-shot 3D detection at 0 AP but fine-tuned performance of 8.68 AP-BEV and 13.17 AP3D, with depth estimation causing most errors.

  2. SpatialBench: Is Your Spatial Foundation Model an All-Round Player?

    cs.CV 2026-05 unverdicted novelty 8.0 of 10

    SpatialBench evaluates 41 spatial foundation models across 6 paradigms and 5 task suites, finds they are not all-round players, and introduces the DA-Next-5M dataset plus DA-Next baseline model.

  3. FUSER: Feed-Forward MUltiview 3D Registration Transformer and SE(3)$^N$ Diffusion Refinement

    cs.CV 2025-12 unverdicted novelty 8.0 of 10

    FUSER is the first feed-forward multiview 3D registration transformer that jointly processes all scans to predict global poses, followed by SE(3)^N diffusion refinement for higher accuracy.

  4. Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors

    cs.CV 2026-08 conditional novelty 7.0 of 10

    A dual-branch transformer with masked joint attention completes dense metric depth from sparse dToF sensors, trained entirely on synthetic data and generalizing zero-shot to real devices.

  5. Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    A self-evolving training loop that generates its own spatial QA data with executable code and difficulty feedback lifts Qwen3-VL-4B/8B to 62.7/63.3 on VSI-Bench using an order of magnitude less data.

  6. Honey, I Shrunk the Arc de Triomphe!

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    MetricScenes dataset from web photos and stereo imagery, plus a two-stage Poisson depth completion method, allows fine-tuning MoGe-2 to mitigate scale-collapse in metric monocular geometry while preserving benchmark p...

  7. SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    SOCO is a new benchmark for semantic object correspondence that provides taxonomy, annotations, and language labels to evaluate part-level understanding in vision and multimodal foundation models.

  8. Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    VLMs excel at semantic and grouping tasks while VGMs are stronger on dense geometry and camera motion, with naive fusion yielding balanced representations.

  9. VGenST-Bench: A Benchmark for Spatio-Temporal Reasoning via Active Video Synthesis

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    VGenST-Bench is a new video benchmark for MLLM spatio-temporal reasoning built via generative synthesis, a multi-agent pipeline with human oversight, a 3x2x2 taxonomy, and hierarchical tasks separating perception from...

  10. CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Proposes Spatial Narrative Score (SNS) evaluation for VLMs' camera motion understanding and introduces CaMo model achieving consistent performance on SNS and direct QA.

  11. SplatWeaver: Learning to Allocate Gaussian Primitives for Generalizable Novel View Synthesis

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    SplatWeaver uses cardinality Gaussian experts and pixel-level routing to dynamically allocate varying numbers of Gaussian primitives for generalizable novel view synthesis.

  12. Reasoning over Video: Evaluating How MLLMs Extract, Integrate, and Reconstruct Spatiotemporal Evidence

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    VAEX-BENCH shows state-of-the-art MLLMs perform substantially worse on abstractive spatiotemporal reasoning tasks than on matched extractive tasks in video understanding.

  13. ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    ZipMap achieves linear-time bidirectional 3D reconstruction by zipping image collections into a compact stateful representation via test-time training layers.

  14. POMA-3D: The Point Map Way to 3D Scene Understanding

    cs.CV 2025-11 unverdicted novelty 7.0 of 10

    POMA-3D learns self-supervised 3D scene representations from point maps and improves performance on geometric 3D tasks including navigation and scene retrieval.

  15. SpaceVista: All-Scale Visual Spatial Reasoning from mm to km

    cs.CV 2025-10 conditional novelty 7.0 of 10

    SpaceVista contributes a 1M-QA, 38K-video all-scale spatial reasoning dataset spanning mm to km, a manually verified benchmark, and a fine-tuned 7B MLLM with scale experts and progressive reward training.

  16. A Scene is Worth a Thousand Features: Feed-Forward Camera Localization from a Collection of Image Features

    cs.CV 2025-10 unverdicted novelty 7.0 of 10

    FastForward represents scenes as collections of 3D-anchored image features and performs camera pose estimation via feed-forward correspondence prediction, achieving competitive accuracy with minimal mapping time.

  17. Cubify Anything: Scaling Indoor 3D Object Detection

    cs.CV 2024-12 conditional novelty 7.0 of 10

    A new large-scale, class-agnostic 3D box dataset for indoor scenes shows that an image-based transformer detector can beat point-cloud-based detectors at scale.

  18. G$^2$TAM: Geometry Grounded Track Anything Model

    cs.CV 2026-07 accept novelty 6.5 of 10

    Spatially aligned geometric features serve as implicit memory so one model reconstructs scenes and produces promptable, cross-view consistent instance masks from unordered RGB only.

  19. ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes

    cs.CV 2026-08 conditional novelty 6.0 of 10

    ThinkAfford decomposes 3D affordance grounding into high-recall proposal generation and GRPO-trained vision-language selection, reporting state-of-the-art AP25 on SceneFun3D.

  20. Rethinking 3D Segmentation from Individual LiDAR Scans: Incidence-Aware Sampling on the SIP Benchmark

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Incidence-aware manifold sampling improves resolution-averaged semantic segmentation of single-scan construction LiDAR by preserving thin, rarely observed structures.

  21. ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A new benchmark shows the best multimodal LLM reaches 62% versus 91% human accuracy on qualitative spatial-temporal reasoning from videos.

  22. SeeSE3: Emergence of 3D Space in Vision Features

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Self-supervised vision features, especially DINOv2, contain a subspace that a small trained adapter can map to 3D camera motion, enabling pose estimation and latent-space navigation without explicit 3D reconstruction.

  23. Glob3R: Global Structure-from-Motion with 3D Foundation Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A frozen Pi3X backbone plus dense warping tracks and keyframe sliding-window global optimization yields more accurate, scalable SfM than feed-forward or classical baselines alone.

  24. DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    DPPE decouples rotation and translation in camera positional encodings for multi-view transformers to resolve late-stage training stagnation and improve generalization in novel view synthesis.

  25. Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Stream3D-VLM adds autoregressive streaming control, VSFI geometry integration, GAVC compression, and a 1M-pair benchmark to enable real-time 3D VLM performance that beats prior models on 29 online and offline tasks.

  26. PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    PAR3D is a part-aware 3D-MLLM framework with ScenePart dataset, Part-Aware 3D Representation Learning, and Hierarchical Segmentation Query Generation to improve part-level 3D scene understanding.

  27. GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Gravity-aligned RGB reconstruction factors multi-view scenes into rigid object meshes with 6-DoF poses plus a cleaned background for direct physics simulation.

  28. Zero-Shot 3D Question Answering via Hierarchical View-to-Token Transportation

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    KeyVT improves zero-shot 3D question answering by hierarchically selecting semantically and geometrically relevant views and using optimal transport to extract representative tokens from them.

  29. ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    ForeSplat introduces MetaGrad, a meta-gradient training rule that makes feed-forward 3DGS predictions optimization-aware so they converge faster and to higher quality upon refinement.

  30. UniT: Unified Geometry Learning with Group Autoregressive Transformer

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    UniT unifies online and offline 3D geometry perception via a Group Autoregressive Transformer that processes observation groups with anchor-free point map prediction and a scale-adaptive loss.

  31. TInR: Exploring Tool-Internalized Reasoning in Large Language Models

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    TInR-U internalizes tool knowledge into LLMs via bidirectional alignment, supervised fine-tuning, and reinforcement learning, outperforming standard tool-integrated reasoning in both in-domain and out-of-domain evaluations.

  32. SVCBench: A Streaming Video Counting Benchmark for Spatial-Temporal State Maintenance

    cs.CV 2026-03 accept novelty 6.0 of 10

    Streaming multi-point counting on 406 videos with three trajectory metrics reveals large human-model gaps in spatial-temporal state maintenance, worst on periodic events.

  33. Dual Tuning for Reasoning Efficacy-Driven Data Curation in Multimodal LLM Training

    cs.CL 2026-02 unverdicted novelty 6.0 of 10

    Dual Tuning is a data curation method that jointly scores training examples for benefit and for reasoning-gain to choose between reasoning and direct-answer post-training modes for multimodal LLMs.

  34. MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A sparse-prompt pretraining framework on 20M heterogeneous image-depth pairs yields a scaling trend and state-of-the-art metric depth across many downstream tasks.

  35. OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding

    cs.CV 2025-12 conditional novelty 6.0 of 10

    OpenGround grounds open-world 3D targets by planning a task chain and dynamically expanding the object lookup table through online 2D segmentation and 3D lifting, achieving SOTA zero-shot ScanRefer accuracy and 46.2% ...

  36. Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encoding

    cs.CV 2025-12 unverdicted novelty 6.0 of 10

    Chorus pretrains a shared 3D Gaussian scene encoder via multi-teacher distillation to capture holistic features from high-level semantics to fine-grained structure, with strong transfer on segmentation and point-cloud...

  37. VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

    cs.CV 2025-11 conditional novelty 6.0 of 10

    A 1,680-question video benchmark shows leading multimodal models lag humans by ~15 points on visual knowledge, and a See-Think-Answer RL-trained model narrows the gap.

  38. DoReMi: Bridging 3D Domains via Topology-Aware Domain-Representation Mixture of Experts

    cs.CV 2025-11 unverdicted novelty 6.0 of 10

    DoReMi uses self-supervised pre-training on topological and texture variations plus domain-aware experts with spatial-guided routing and entropy-controlled allocation to reach 80.1% mIoU on ScanNet and 77.2% mIoU on S3DIS.

  39. Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots

    cs.CV 2025-09 reject novelty 6.0 of 10

    Social 3D Scene Graphs extend 3D scene graphs with human activities and relations, but the evaluation ground truth is derived from the model's own outputs.

  40. Mem4D: Decoupling Static and Dynamic Memory for Dynamic Scene Reconstruction

    cs.CV 2025-08 conditional novelty 6.0 of 10

    Mem4D decouples static and dynamic memory to improve online monocular 3D reconstruction of dynamic scenes, showing metric-depth gains on Sintel and Bonn but worse static reconstruction than CUT3R.

  41. Music-Aligned Holistic 3D Dance Generation via Hierarchical Motion Modeling

    cs.MM 2025-07 conditional novelty 6.0 of 10

    A new captured music-dance dataset with facial expressions and a hierarchical residual VQ plus masked-transformer model that generates expressive 3D dance from music.

  42. Streaming 4D Visual Geometry Transformer

    cs.CV 2025-07 unverdicted novelty 6.0 of 10

    A causal transformer with key-value caching and distillation from a bidirectional VGGT model enables efficient online 4D geometry reconstruction from videos.

  43. PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection

    cs.CV 2025-07 conditional novelty 6.0 of 10

    PLOT generates 3D bounding-box labels for objects from monocular video alone, using point tracking, estimated camera motion, and temporal fusion of pseudo-LiDAR, without auxiliary sensors or model training.

  44. Puzzles: Unbounded Video-Depth Augmentation for Scalable End-to-End 3D Reconstruction

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Puzzles synthesizes posed video-depth clips from single images and keyframes, letting 3D reconstruction models match full-data accuracy using only 10% of the data.

  45. Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotations

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Video-grounded RL with two-stage 2D grounding and SAM2 lifting enables 3D object localization and QA without dense 3D instance supervision.

  46. BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A dual-stream diffusion model trained with Blender-render conditioning, source masking, and object jittering performs 3D-grounded multi-object editing and compositing better than existing baselines on three video datasets.

  47. Towards In-the-wild 3D Plane Reconstruction from a Single Image

    cs.CV 2025-06 conditional novelty 6.0 of 10

    ZeroPlane trains a Transformer plane reconstructor on 560K images spanning 10 indoor and outdoor datasets and outperforms prior methods in zero-shot evaluations on NYUv2, 7-Scenes, ParallelDomain, and ApolloScape.

  48. MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A new 1,188-question multimodal benchmark covering deductive, inductive, and abductive reasoning shows that leading MLLMs score around 60% and are especially weak at abductive reasoning.

  49. VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction

    cs.CV 2025-05 unverdicted novelty 6.0 of 10

    VLM-3R augments VLMs with implicit 3D tokens from monocular video via geometry encoding and 200K+ 3D reconstructive QA pairs, plus a new 138K-pair temporal benchmark, to support spatial and embodied reasoning.

  50. RadarRGBD A Multi-Sensor Fusion Dataset for Perception with RGB-D and mmWave Radar

    eess.IV 2025-05 conditional novelty 6.0 of 10

    RadarRGBD provides over 2,700 frames of RGB-D, high-resolution mmWave radar point clouds, and raw radar matrices across indoor and outdoor scenes, plus a depth-completion fine-tuning method.

  51. Matrix3D: Large Photogrammetry Model All-in-One

    cs.CV 2025-02 conditional novelty 6.0 of 10

    A single multi-modal diffusion transformer trained with masked learning performs pose estimation, depth prediction, and novel view synthesis in one model, reporting SOTA pose and NVS numbers.

  52. Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A single-pass transformer generalizes DUSt3R's pointmap regression from two views to all-to-all multi-view attention, reconstructing 1000+ images and estimating camera poses in one forward pass.

  53. Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Orient Anything predicts an object's front-facing 3D orientation in a single image using a model trained on 2M rendered views, with zero-shot transfer to real images.

  54. Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Reloc3r trains a symmetric, scale-free relative pose regression transformer on 8M image pairs and uses motion averaging for absolute poses, outperforming prior regression methods on six localization benchmarks.

  55. LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences

    cs.CV 2024-12 conditional novelty 6.0 of 10

    LSceneLLM chooses task-relevant 3D regions via LLM attention, magnifies their details, and improves large-scene 3D question answering, planning, and captioning.

  56. SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A ground-truth-free diffusion distillation method makes metric depth predictions from UniDepth sharper while preserving their scale.

  57. Grounded 3D-Aware Spatial Vision-Language Modeling

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    GR3D is a VLM that combines explicit 2D, implicit 2D, and monocular 3D grounding mechanisms to improve performance on spatial understanding benchmarks.

  58. GEM: Generative Supervision Helps Embodied Intelligence

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    GEM adds generative depth supervision to VLM pre-training and reports improved results on embodied benchmarks plus real-world robot execution.

  59. HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    HorizonStream is a long-horizon Transformer that factorizes geometric evidence influence into channel-wise linear attention for long-range temporal propagation and local spatiotemporal attention for short-range matchi...

  60. IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    IVGT implicitly represents scenes as continuous neural fields from pose-free multi-view images to enable coherent surface extraction, novel view synthesis, and related tasks via transformer-based feature retrieval and...

See all 65 Pith citations

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages · cited by 65 Pith papers (see all)

  1. [1]

    3d-sis: 3d semantic instance segmentation of rgb-d scans

    Ji Hou, Angela Dai, and Matthias Nießner. 3d-sis: 3d semantic instance segmentation of rgb-d scans. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4421–4430, 2019

    work page 2019

  2. [2]

    Gspn: Generative shape proposal network for 3d instance segmentation in point cloud

    Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), pages 3947–3956, 2019

    work page 2019

  3. [3]

    Sgpn: Similarity group proposal network for 3d point cloud instance segmentation

    Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neumann. Sgpn: Similarity group proposal network for 3d point cloud instance segmentation. InProc. Conference on Computer Vision and Pattern Recognition (CVPR), pages 2569–2578, 2018

    work page 2018

  4. [4]

    Deep hough voting for 3d object detection in point clouds

    Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. Deep hough voting for 3d object detection in point clouds. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2019

    work page 2019

  5. [5]

    Qi, Xinlei Chen, and Leonidas J

    Charles R. Qi, Xinlei Chen, and Leonidas J. Guibas Or Litany. Imvotenet: Boosting 3d object detection in point clouds with image votes. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2020

    work page 2020

  6. [6]

    Svga- net: Sparse voxel-graph attention network for 3d object detection from point clouds

    Qingdong He, Zhengning Wang, Hao Zeng, Yi Zeng, Shuaicheng Liu, and Bing Zeng. Svga- net: Sparse voxel-graph attention network for 3d object detection from point clouds. arXiv preprint arXiv:2006.04043, 2020

    work page arXiv 2006

  7. [7]

    Group-free 3d object detection via transformers

    Ze Liu, Zheng Zhang, Yue Cao, Han Hu, and Xin Tong. Group-free 3d object detection via transformers. arXiv preprint arXiv:2104.00678, 2021

    work page arXiv 2021

  8. [8]

    ShapeNet: An Information-Rich 3D Model Repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015

    work page Pith review arXiv 2015

Show all 46 references
  1. [9]

    Sun3d: A database of big spaces reconstructed using sfm and object labels

    Jianxiong Xiao, Andrew Owens, and Antonio Torralba. Sun3d: A database of big spaces reconstructed using sfm and object labels. In Proc. International Conference on Computer Vision (ICCV), pages 1625–1632, 2013

    2013

  2. [10]

    A category-level 3d object dataset: Putting the kinect to work

    Allison Janoch, Sergey Karayev, Yangqing Jia, Jonathan T Barron, Mario Fritz, Kate Saenko, and Trevor Darrell. A category-level 3d object dataset: Putting the kinect to work. InConsumer depth cameras for computer vision, pages 141–165. Springer, 2013

    2013

  3. [11]

    3d semantic parsing of large-scale indoor spaces

    Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), pages 1534–1543, 2016

    2016

  4. [12]

    Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding

    Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) , pages 90...

    2019

  5. [13]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proc. Conference on Computer Vision and Pattern Recog...

    2020

  6. [14]

    Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    2017

  7. [15]

    Sun rgb-d: A rgb-d scene understanding benchmark suite

    S Song, S Lichtenberg, and J Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) , 2015

    2015

  8. [16]

    Indoor scene segmentation using a structured light sensor

    Nathan Silberman and Rob Fergus. Indoor scene segmentation using a structured light sensor. In 2011 IEEE international conference on computer vision workshops (ICCV workshops), pages 601–608. IEEE, 2011

    2011

  9. [17]

    https://www.apple.com/newsroom/2020/03/apple-unveils-new-ipad-pro-with-lidar- scanner-and-trackpad-support-in-ipados/. 10

    2020

  10. [18]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. Ieee, 2009

    2009

  11. [19]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. 2013

    2013

  12. [20]

    Kesten, M

    R. Kesten, M. Usman, J. Houston, T . Pandya, K. Nadhamuni, A. Ferreira, M. Yuan, B. Low, A. Jain, P . Ondruska, S. Omari, S. Shah, A. Kulkarni, A. Kazakova, C. Tao, L. Platinsky, W . Jiang, and V . Shet. Lyft level 5 perception dataset 2020. 2019

    2020

  13. [21]

    Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  14. [22]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017

    2017 arXiv

  15. [23]

    Scenenn: A scene meshes dataset with annotations

    Binh-Son Hua, Quang-Hieu Pham, Duc Thanh Nguyen, Minh-Khoi Tran, Lap-Fai Yu, and Sai- Kit Yeung. Scenenn: A scene meshes dataset with annotations. In 2016 Fourth International Conference on 3D Vision (3DV), pages 92–101. IEEE, 2016

    2016

  16. [24]

    Pigraphs: Learning interaction snapshots from observations

    Manolis Savva, Angel X Chang, Pat Hanrahan, Matthew Fisher, and Matthias Nießner. Pigraphs: Learning interaction snapshots from observations. ACM Transactions on Graphics (TOG), 35(4):1–12, 2016

    2016

  17. [25]

    A naturalistic open source movie for optical flow evaluation

    Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Proc. European Conference on Computer Vision (ECCV) , pages 611–625. Springer, 2012

    2012

  18. [26]

    High-resolution stereo datasets with subpixel-accurate ground truth

    Daniel Scharstein, Heiko Hirschmüller, York Kitajima, Greg Krathwohl, Nera Neši´ c, Xi Wang, and Porter Westling. High-resolution stereo datasets with subpixel-accurate ground truth. In German conference on pattern recognition, pages 31–42. Springer, 2014

    2014

  19. [27]

    Structure aware single-stage 3d object detection from point cloud

    Chenhang He, Hui Zeng, Jianqiang Huang, Xian-Sheng Hua, and Lei Zhang. Structure aware single-stage 3d object detection from point cloud. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  20. [28]

    Hvnet: Hybrid voxel network for lidar based 3d object detection

    Maosheng Ye, Shuangjie Xu, and Tongyi Cao. Hvnet: Hybrid voxel network for lidar based 3d object detection. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  21. [29]

    Point-gnn: Graph neural network for 3d object detection in a point cloud

    Weijing Shi and Raj Rajkumar. Point-gnn: Graph neural network for 3d object detection in a point cloud. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  22. [30]

    Mlcvnet: Multi-level context votenet for 3d object detection

    Qian Xie, Yu-Kun Lai, Jing Wu, Zhoutao Wang, Yiming Zhang, Kai Xu, and Jun Wang. Mlcvnet: Multi-level context votenet for 3d object detection. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  23. [31]

    Chen, and Jian Wu

    Jintai Chen, Biwen Lei, Qingyu Song, Haochao Ying, Danny Z. Chen, and Jian Wu. A hierarchi- cal graph network for 3d object detection on point clouds. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  24. [32]

    Frodo: From detections to 3d objects

    Martin Runz, Kejie Li, Meng Tang, Lingni Ma, Chen Kong, Tanner Schmidt, Ian Reid, Lourdes Agapito, Julian Straub, Steven Lovegrove, and Richard Newcombe. Frodo: From detections to 3d objects. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020

    2020

  25. [33]

    Generative sparse detection networks for 3d single-shot object detection

    JunYoung Gwak, Christopher Choy, and Silvio Savarese. Generative sparse detection networks for 3d single-shot object detection. arXiv preprint arXiv:2006.12356, 2020

    2006

  26. [34]

    Frustum pointnets for 3d object detection from rgb-d data

    Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb-d data. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), pages 918–927, 2018

    2018

  27. [35]

    Pv-rcnn: Point-voxel feature set abstraction for 3d object detection

    Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), pages 10529–10538, 2020. 11

    2020

  28. [36]

    Objectron: A large scale dataset of object-centric videos in the wild with pose annotations

    Adel Ahmadyan, Liangkai Zhang, Jianing Wei, Artsiom Ablavatski, and Matthias Grundmann. Objectron: A large scale dataset of object-centric videos in the wild with pose annotations. arXiv preprint arXiv:2012.09988, 2020

    2012

  29. [37]

    Depth map super-resolution by deep multi-scale guidance

    Tak-Wai Hui, Chen Change Loy, , and Xiaoou Tang. Depth map super-resolution by deep multi-scale guidance. In Proc. European Conference on Computer Vision (ECCV) , pages 353–369, 2016

    2016

  30. [38]

    Cohen, Dani Lischinski, and Matt Uyttendaele

    Johannes Kopf, Michael F . Cohen, Dani Lischinski, and Matt Uyttendaele. Joint bilateral upsampling. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007), 26(3):to appear, 2007

    2007

  31. [39]

    Image guided depth upsampling using anisotropic total generalized variation

    David Ferstl, Christian Reinbacher, Rene Ranftl, Matthias Rüther, and Horst Bischof. Image guided depth upsampling using anisotropic total generalized variation. InProc. International Conference on Computer Vision (ICCV) , pages 993–1000, 2013

    2013

  32. [40]

    A taxonomy and evaluation of dense two-frame stereo correspondence algorithms

    Daniel Scharstein and Richard Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision (IJCV) , 47(1):7–42, 2002

    2002

  33. [41]

    High-accuracy stereo depth maps using structured light

    Daniel Scharstein and Richard Szeliski. High-accuracy stereo depth maps using structured light. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) , volume 1, pages I–I. IEEE, 2003

    2003

  34. [42]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    1981

  35. [43]

    H3dnet: 3d object detection using hybrid geometric primitives

    Zaiwei Zhang, Bo Sun, Haitao Yang, and Qixing Huang. H3dnet: 3d object detection using hybrid geometric primitives. In Proc. European Conference on Computer Vision (ECCV) , 2020

    2020

  36. [44]

    Qi, Li Yi, Hao Su, and Leonidas J

    Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  37. [45]

    Multi-scale progressive fusion learning for depth map super-resolution

    Chuhua Xian, Kun Qian, Zitian Zhang, and Charlie CL Wang. Multi-scale progressive fusion learning for depth map super-resolution. arXiv preprint arXiv:2011.11865, 2020

    2011

  38. [46]

    Megadepth: Learning single-view depth prediction from internet photos

    Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) , 2018. 12

    2018

Pith tools

Reviewed May 15, 2026 · model on record in the stance chip above.