Pith. sign in

REVIEW 3 major objections 5 minor 90 references

RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A vision-language module can refine the depth of any monocular 3D box—from closed-set detectors, open-vocabulary detectors, or auto-labelers—by treating depth error as a visible misalignment of the projected box and correcting it with…

desk verdict A useful empirical paper with a clean new task, but the central 'semantic alignment' mechanism is plausible rather than proven on real detector outputs. read the letter →

arxiv 2608.09147 v1 pith:USKUYFRY submitted 2026-08-10 cs.CV

Zhihao Zhang , Gengwei Zhang , Tianlong Chen , Xiaoming Liu This is my paper · ORCID
classification cs.CV
keywords monocular3Dobjectdetectiondepthrefinementvision-languagemodelsemanticalignmentactiontokensopen-vocabularyauto-labelingboundingbox
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the depth of any monocular 3D bounding box can be corrected after the fact by asking a vision-language model to look at the box projected onto the image and decide, categorically, which way and how far the box must move. The central insight is that depth error leaves a visual signature: a box placed too far projects too small, a box too close projects too large, so depth refinement becomes a semantic alignment judgment rather than a numerical regression. RefineAny3D renders the candidate box as a wireframe, lets a VLM reason explicitly about the misalignment, and emits six discrete action tokens covering direction and magnitude, chaining them iteratively. The paper reports consistent gains across closed-set detectors on KITTI, open-vocabulary detectors on Omni3D, and auto-labeling pipelines, while an oracle study identifies depth as the dominant bottleneck. If correct, this makes 3D box refinement a plug-and-play capability that improves any upstream detector without retraining it.

What carries the argument

The central object is the projected wireframe alignment check paired with a factored action-token vocabulary. RefineAny3D projects the eight corners of the candidate box through the camera intrinsics and renders them as a wireframe on the image, so metric scale and intrinsics are absorbed into a purely 2D question: does the box tightly enclose the object? Six special tokens answer that question categorically—three direction tokens for closer, ok, and farther, and three magnitude tokens for small, medium, and large—with magnitude defined relative to the object's mean linear extent so a step means the same thing across object sizes. The VLM first produces a short reasoning text that identifies the object, recalls a rough size, and points to the visual evidence, then decodes the action; the depth is updated as a signed object-relative step and the loop repeats until ok or a cap of two steps. Training uses about 3 million synthetic depth-perturbation samples built from Omni3D ground-truth boxes, with nine perturbation slots per annotation that balance all token combinations; the two-stage recipe first warms up the six new token embeddings with semantic initialization, then fine-tunes the language tower while freezing the vision encoder to preserve general visual priors.

What would settle it

Evaluate RefineAny3D's first-step direction accuracy on a held-out set of real detector predictions in two conditions: the raw coupled predictions, and the same predictions with dimensions and yaw replaced by ground truth. If direction accuracy in the raw condition is no better than random while the decoupled-condition accuracy is high, the claimed transfer from synthetic depth-only training to realistic detector residuals is not happening.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that object depth in monocular 3D detection is not just an attribute to regress but a residual that can be repaired after the fact from 2D visual evidence. The paper first isolates the bottleneck with an oracle study: replacing a strong detector's predicted depth with ground truth raises AP3D by +29.92 on Omni3D, while substituting a state-of-the-art depth foundation model lowers it by 3.68. RefineAny3D then closes part of that gap by rendering the candidate 3D box as a wireframe overlay and training a VLM to judge alignment and emit direction and magnitude action tokens. The claimed transfer rests on the judgment being local and camera-agnostic: the same module improves MonoCoP on KITTI, raising Easy AP3D from 32.06 to 35.62; improves DetAny3D with oracle 2D boxes on Omni3D, raising AP3D from 34.38 to 38.73; and improves auto-labeled pseudo-labels, yielding a downstream detector with +2.37 Moderate AP3D, all without retraining for any upstream method. On a controlled benchmark with novel categories and rescaled cameras, the model retains most of its direction accuracy, supporting the generalization claim.

Load-bearing premise

The load-bearing premise is that depth error is visually readable from the projected wireframe's misalignment, and that a model trained on synthetic depth-only perturbations of ground-truth boxes will keep reading that signal on real detector outputs whose depth, size, orientation, and position errors are coupled.

Editorial extensions

If this is right

  • A single refinement module can be attached to any monocular 3D pipeline that outputs boxes and camera intrinsics, improving AP3D without retraining the detector or changing its architecture.
  • Depth foundation models should not be treated as drop-in replacements for object-level depth in 3D detection; at tight 3D IoU thresholds their bias propagates into detection error.
  • Auto-labeling pipelines can be upgraded by refining pseudo-labels before training downstream detectors, narrowing the gap to ground-truth-trained accuracy without new annotations.
  • Because the module terminates with an abstain, leaving roughly 38 percent of objects unchanged, it can act as a post-hoc safety net that does not disturb already-aligned predictions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same alignment-token recipe could be extended to refine yaw and dimensions, not just depth; the paper's own oracle study shows those residual errors are small for in-domain detectors, but on open-vocabulary or auto-labeled boxes they may become large enough to matter.
  • The failure of a lightweight classifier trained with identical action targets suggests that the transfer to real detector outputs is carried by the VLM's broad pretrained priors; distilling those priors into a smaller model would be a natural efficiency test.
  • A stronger data recipe that perturbs depth, dimensions, yaw, and projected position jointly, rather than depth alone, could either make the model more robust to coupled residuals or reveal the ceiling of the visual-alignment formulation.
  • The reformulation principle, turning a metric residual into a discrete visually checkable decision, may transfer to other geometric refinement problems such as camera pose or 6-DoF object pose.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RefineAny3D, a vision-language model that refines the object depth of candidate 3D boxes from any upstream monocular detector by emitting discrete action tokens (direction and magnitude) grounded in chain-of-thought reasoning over a projected wireframe overlay. The authors recast depth refinement as semantic alignment in image space, train on a large synthetic dataset constructed by perturbing ground-truth depth along the camera ray while holding all other attributes fixed, and apply the model iteratively at inference. They report consistent AP3D gains on a closed-set detector (MonoCoP on KITTI), an open-vocabulary detector (DetAny3D on Omni3D), and an auto-labeling pipeline (LabelAny3D), alongside ablations, an oracle study, and comparisons with geometric fitting and a DINOv2 action classifier.

Significance. The paper has several genuine strengths: the oracle study (Table 8) cleanly shows depth as the dominant bottleneck; the comparison against geometric projection fitting and a DINOv2 classifier (Table 6) is a thoughtful attempt to rule out simpler alternatives; the iteration-convergence analysis (Appendix F) and the honesty about limitations (Appendix J) are commendable. If the central mechanism were validated, the module would be a practical plug-and-play contribution with value for closed-set and open-vocabulary detection and for auto-labeling. However, the load-bearing assumption of the paper—that a model trained on synthetic depth-only perturbations transfers its alignment judgment to real detector outputs with coupled residuals in depth, dimensions, yaw, and position—is asserted rather than directly measured. The current evidence leaves open the possibility that the observed end-task gains come from category-size priors embedded in the CoT supervision or from mean-depth shifts rather than from the claimed visual alignment mechanism. The significance of the work is therefore conditional on additional evidence that this mechanism actually operates on real detector outputs.

major comments (3)
  1. [Appendix A.2] The central claim of the paper—that depth refinement reduces to a visual alignment judgment that transfers from synthetic depth-only perturbations to real detector outputs—is not directly measured. The Refine3D benchmark (Sec. 5.1 and Appendix E) tests only synthetically depth-perturbed ground-truth boxes using the same perturbation schedule as the training data (Sec. D.3), so it cannot distinguish visual-alignment reasoning from category-size priors. The only evidence for real-output transfer is the DINOv2 classifier's failure (Table 6, Appendix A.2) plus the assertion of 'broad visual priors' (A.3). However, the chain-of-thought supervision includes category-specific size recalls (e.g., 'a car is typically 3.5–4.5 m long', Fig. 3 and Sec. D.4), which provides an alternative pathway for the model to produce correct depth corrections without reading wireframe misalignment. The paper should report per-step direction/magnitude accuracy (or at least first-step depth-error reduction) on real detector predictions matched to ground truth on KITTI and Omni3D, and compare against a variant that receives the category label but not the wireframe overlay. Without these numbers, the +3.56 AP3D gain on KITTI Easy could in principle be produced by a model that regresses toward the category mean depth.
  2. [Appendix A.2] The DINOv2 comparison is not a controlled test of the visual-alignment mechanism. The DINOv2 classifier is a small discriminative network trained on the same action supervision but without chain-of-thought reasoning or category-specific world-knowledge recalls. Its failure under distribution shift therefore does not demonstrate that RefineAny3D's success comes from visual alignment; it is equally consistent with the VLM leveraging the size priors embedded in the CoT targets. To isolate the mechanism, the authors should ablate the 'Recall' step from the CoT (or remove category labels from the input) and evaluate on real detector outputs, ideally also with the wireframe overlay removed from the input. If the model still refines depth accurately without these cues, the claim that it is performing semantic alignment in image space is not supported.
  3. [Tables 2, 4] The paper does not report error bars or statistical significance (the checklist acknowledges this), and several headlined gains are small: +0.67 AP3D on KITTI Hard (Table 2) and +0.72 on Hard in the auto-labeling experiment (Table 4). Given the high variance of AP3D at IoU≥0.7 for small numbers of hard samples, the claim of 'consistent gains' across all difficulty levels is not statistically established. The authors should provide at least bootstrapped confidence intervals over test samples, or multiple training seeds for the downstream detector, for the main experimental tables.
minor comments (5)
  1. [Eq. (5)] The magnitude step sizes (0.20, 0.55, 1.10 times the mean object extent) are selected as midpoints of the training perturbation ranges, but the sensitivity of final AP3D to these coefficients is not analyzed; the token semantics are therefore tied to the training schedule, which should be discussed as a design choice rather than a parameter-free consequence of the formulation.
  2. [Sec. E.4] The Novel Camera split rescales the image and the intrinsics jointly, which leaves the relationship between the projected wireframe and the object unchanged up to image resolution; this is more a resolution-generalization test than a camera-generalization test. Consider varying the focal length or camera height independently to make the split more challenging and meaningful.
  3. [Sec. 3.2] The ⟨depth_ok⟩ outcome has no magnitude token defined in the training schedule, but the inference contract always emits both a direction and a magnitude token; it should be clarified whether the magnitude token is conditionally generated and ignored for ⟨depth_ok⟩, and how this is handled in constrained decoding.
  4. [Sec. D.4] The CoT generator is given the ground-truth direction/magnitude token and composes 'visual evidence' from the label plus a caption, which can produce hallucinated evidence that does not reflect the actual image; the paper should explicitly note this as a data-generation caveat that may affect the extent to which the model learns grounded visual reasoning.
  5. [Sec. 5.2] The claim of being a detector-agnostic refinement module is supported by experiments on a single closed-set detector (MonoCoP) in the main tables; applying RefineAny3D to one or two additional closed-set detectors (e.g., MonoDETR or MonoDGP) on KITTI would strengthen the claim of generality.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RefineAny3D's real-detector AP gains are external, though the synthetic Refine3D benchmark is self-consistent with the training schedule.

full rationale

The paper's derivation chain is self-contained against external benchmarks. The training signal is synthetic: GT depth is perturbed along the camera ray and the target action tokens are read off the perturbation schedule (Sec. D.3, Table 10), while the update rule (Eqs. 4-6) uses the same object-relative magnitudes. This creates an internally consistent task definition, but the central claims are not tested solely on that schedule: RefineAny3D is applied to real detector outputs (MonoCoP on KITTI, Table 2; DetAny3D on Omni3D, Table 3; LabelAny3D pseudo-labels, Table 4) and evaluated with GT-based AP3D, which is an external outcome that cannot be manufactured by the action-token construction. The Refine3D benchmark (Appendix E) uses the same perturbation protocol as training, making it a controlled in-distribution test rather than an independent validation; the real generalization evidence is the detector AP gains under novel categories/cameras. Appendix A.2 discloses that a DINOv2 classifier trained with identical supervision fails on real detector outputs, and the paper attributes the VLM's success to broad pretraining priors without reporting per-step accuracy on real residuals; this is a genuine generalization/correctness risk, not a circular reduction. No load-bearing self-citation or imported uniqueness theorem is used: MonoCoP [75] appears as a baseline, not as justification for the method. The paper therefore contains no step where a prediction reduces by definition to its training input.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on one visual-signature assumption, a Markovian locality assumption for multi-step generalization, and a transfer assumption from synthetic perturbations to real detector residuals. The free parameters are the magnitude step ratios and perturbation ranges that define token semantics and the training distribution; these are hand-specified but described transparently. No new physical entities are introduced; the action tokens are output vocabulary, not independent world entities. The method quantifies its contribution well via oracle and ablation studies, but the transfer assumption is the most fragile unverified premise.

free parameters (4)
  • magnitude step ratios = 0.20 / 0.55 / 1.10 times sobject
    These are chosen as bucket midpoints of the perturbation schedule ranges, not fitted to maximize AP. They appear in equation (5) and the inference contract in Appendix D.3. They are a design choice, but the schedule itself defines the correct action classes, so the step sizes are not free parameters that were tuned against the final evaluation metric.
  • depth perturbation ranges r = [0.10, 0.30), [0.30, 0.80), [0.80, 1.50], symmetric negative ranges, dead zones [0, 0.10)
    The perturbation slot ranges in Table 10 define the action-token labels and the difficulty of the training and Refine3D benchmark. They are chosen by hand, and the resulting distribution of supervision is stratified. The Refine3D test metric DepthErr is normalized by sobject and evaluated on these same ranges, so the benchmark difficulty is determined by these choices.
  • Tmax = 2 = 2
    The step cap is set based on a sweep in Appendix F.1 showing AP plateaus at K=2 on KITTI Moderate. This is a hyperparameter chosen to trade accuracy against inference cost, not a physical constant.
  • filter thresholds = truncation <= 0.05, visibility >= 0.8 (0.6 nuScenes), 2D IoU > 0.85, min 2D footprint 32x32 px
    These curation thresholds in Appendix D.2 determine which 335K annotations survive and therefore shape the training distribution. They are chosen by hand and not swept. They could affect generalization, but no evidence of overfitting to the filter is presented.
assumptions (4)
  • domain assumption A 3D box projected with correct depth tightly encloses the visible object, and depth error produces a consistent visual signature (too small means too far, too large means too close).
    This is the core premise of Section 3.1 and the whole method. It is stated as an observation and used to justify recasting depth refinement as visual alignment. The paper gives an analytical argument in Appendix A.1 showing that tight 2D box fitting is biased, but the visual signature assumption itself is not proven; it is supported by the empirical success of the VLM. Heavy occlusion, truncation, and unusual shape categories weaken the signature, as the paper's own limitation section admits.
  • domain assumption The autoregressive factorization P(S(t)|I(t)) = P(r(t)|I(t)) * P(a(t)|I(t), r(t)) implies that single-step supervision on local actions generalizes to multi-step iterative inference.
    Invoked in Section 3.1 and defended in Section B.2. The argument is that each action depends only on the current visual state, so training on single steps is consistent with multi-step deployment. This is a Markov-style assumption: the model's next action only depends on the rendered image of the current box, not on the history. The paper provides no counterfactual test of this assumption beyond the convergence experiment in Appendix F, which is consistent with it.
  • domain assumption The VLM's visual priors, preserved by freezing the vision encoder, transfer the alignment judgment from synthetic depth-only perturbations to real coupled detector residuals.
    This is the key generalization assumption that Appendix A.2 highlights: a DINOv2 classifier trained on the same data fails under this shift, and the paper attributes the VLM's success to broad visual priors. The assumption is plausible but not independently verified; the paper does not quantify how much of the improvement is due to priors versus the specific CoT supervision or model scale.
  • domain assumption Ground-truth depth is the correct refinement target and refining only depth while keeping other attributes fixed is sufficient to improve downstream AP3D.
    Motivated by the oracle study in Table 8: oracle depth gives +29.92 AP while oracle dimensions and angle give near-zero changes. The method therefore only adjusts depth. This is a strong empirical justification on Omni3D with Cube R-CNN, but it is not checked for every upstream detector; for MonoCoP on KITTI the gains confirm the assumption.
invented entities (2)
  • Six action tokens: depth_closer, depth_farther, depth_ok, step_small, step_medium, step_large
    purpose: Discrete output vocabulary for depth refinement decisions, replacing numerical depth regression.
    These are new tokens introduced in Section 3.2 with learnable embeddings. They are internal design elements, not physical entities, and their behavior is only evaluated through the paper's own benchmarks. There is no independent falsifiable prediction outside the paper beyond the reported accuracy gains.
  • Refine3D benchmark
    purpose: Controlled evaluation benchmark for the standalone refinement capability.
    The benchmark is constructed by the paper using Omni3D; it is an evaluation instrument, not an entity with external evidence. It is documented in Appendix E.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection." pith.science (2026). https://pith.science/paper/USKUYFRY

@misc{pith2026260809147,
  author       = {Pith},
  title        = {Pith review of: RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/USKUYFRY}},
  note         = {Machine review of arXiv:2608.09147}
}
read the original abstract

Monocular 3D object detection spans two regimes: closed-set detectors operating within a fixed category vocabulary, and open-vocabulary detectors that localize arbitrary categories by leveraging depth foundation models for 3D geometry. We find that current depth foundation models, despite their strong zero-shot generalization, lack the object-level precision 3D detection demands: substituting a state-of-the-art depth foundation model for a strong detector's predicted depth degrades accuracy, even falling below the detector's own prediction. Rather than pushing detectors or depth models to be more accurate end-to-end, we treat object-level depth refinement as a stand-alone task and present RefineAny3D, a vision-language model that corrects depth without ever predicting a numerical value. Our key insight is that depth error has a direct visual signature in image space: when projected onto the image, a correctly placed box tightly encloses the object, while a too-far box projects too small and a too-close box projects too large. Depth refinement thus reduces to a visual alignment problem rather than a metric regression problem, which we instantiate by extending the VLM's vocabulary with action tokens that replace numerical depth output with categorical decisions, and by supervising the model on a large-scale chain-of-thought dataset that grounds each decision in explicit visual evidence. Applied as a single post-hoc step, RefineAny3D delivers consistent gains across closed-set detectors, open-vocabulary detectors, and 3D auto-labeling tools, and generalizes to novel categories, scenes, and cameras without retraining.

Figures

Figures reproduced from arXiv: 2608.09147 by the authors.

Figure 1
Figure 1. RefineAny3D: 3D box refinement as semantic alignment in image space. (a) Object depth is the dominant bottleneck of monocular 3D detection: replacing predicted depth with ground truth boosts AP3D by +29.92, while substituting a depth foundation model (DFM) degrades it by −3.68. Relying on DFM alone is insufficient for accurate object-level depth. (b) RefineAny3D reformulates depth refinement as a semantic alignment … view at source ↗
Figure 2
Figure 2. Overview of RefineAny3D. RefineAny3D recasts depth refinement as semantic alignment in image space: when a candidate 3D box is projected onto the image, depth error has a direct visual signature, with a too-far box projecting too small and a too-close box too large. We render the projected box as a wireframe overlay on the image, and a vision-language model performs chain-of-thought reasoning over the visual misalig… view at source ↗
Figure 3
Figure 3. Examples of CoT-based supervision. Each training sample pairs a rendered image of a projected 3D wireframe with a chain-of-thought reasoning target that grounds the corrective action in explicit visual evidence before emitting the action tokens. stabilize them before joint optimization. In Stage 2, we jointly fine-tune the LLM backbone and the action-token embeddings while freezing the vision encoder, preserving the… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Single-step and iterative depth refinement. RefineAny3D refines DetAny3D [72]’s initial prediction (red) toward the refined result (green). Most cases are corrected in a single step; for some cases the model applies multiple action steps until the projected box aligns …
Figure 5
Figure 5. Figure 5: Data construction pipeline for training RefineAny3D. We collect diverse indoor and outdoor scenes (Selection), followed by geometric and VLM-based semantic filtering to remove low-quality or ambiguous samples (Filtering). We then generate training signals by sampling a…
Figure 6
Figure 6. Figure 6: Examples of generated perturbed data. samples share the same crop region but differ only in the rendered wireframe, illustrating how a depth perturbation manifests visually as a wireframe that aligns with the object, sits in front of it, or sits behind it. D.4 Chain-of…
Figure 7
Figure 7. Figure 7: Iterative refinement convergence on KITTI Moderate. AP3D as a function of forced iteration count K. A single refinement step recovers most of the gain (+2.95 AP3D over the baseline); the second step adds another +0.54, after which performance plateaus and slightly drif…
Figure 8
Figure 8. Figure 8: Effect of training data scale. Both DirAcc and FullAcc improve sharply from 250K to 1M and plateau beyond. I More Ablations We sweep the size of the Stage 2 training subsample from 250K to the full ∼3M pool ( [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Additional qualitative examples of single-step and iterative depth refinement on Omni3D. RefineAny3D refines DetAny3D [72]’s initial prediction (red) toward the refined result (green). Most cases are corrected in a single step; harder cases require multiple action step…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

90 extracted references · 59 canonical work pages

  1. [1]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631,

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631,

    arXiv

  2. [2]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923,

    Shuai Bai, Keqin Chen, Xuejing Liu, et al. Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923,

    arXiv

  3. [3]

    Omni3d: A large benchmark and model for 3D object detection in the wild

    Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3d: A large benchmark and model for 3D object detection in the wild. InCVPR, 2023. 1, 3, 6, 7, 8, 17, 19, 22, 25, 26, 29

    2023

  4. [4]

    M3D-RPN: Monocular 3D region proposal network for object detection

    Garrick Brazil and Xiaoming Liu. M3D-RPN: Monocular 3D region proposal network for object detection. InICCV, 2019. 3

    2019

  5. [5]

    Kinematic 3d object detection in monocular video

    Garrick Brazil, Gerard Pons-Moll, Xiaoming Liu, and Bernt Schiele. Kinematic 3d object detection in monocular video. InECCV, 2020. 3

    2020

  6. [6]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. InCVPR, 2020. 3, 29

    2020

  7. [7]

    Vip-llava: Making large multimodal models understand arbitrary visual prompts

    Mu Cai, Haotian Liu, Siva Karthik Mustikovela, Gregory P Meyer, Yuning Chai, Dennis Park, and Yong Jae Lee. Vip-llava: Making large multimodal models understand arbitrary visual prompts. InCVPR, 2024. 3

    2024

  8. [8]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InECCV, 2020. 3

    2020

Show all 90 references
  1. [9]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. InCVPR, 2024. 3

    2024

  2. [10]

    End-to-end autonomous driving: Challenges and frontiers.TPAMI, 2024

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers.TPAMI, 2024. 1

    2024

  3. [11]

    Group detr: Fast detr training with group-wise one-to-many assignment

    Qiang Chen, Xiaokang Chen, Jian Wang, Shan Zhang, Kun Yao, Haocheng Feng, Junyu Han, Errui Ding, Gang Zeng, and Jingdong Wang. Group detr: Fast detr training with group-wise one-to-many assignment. InCVPR, 2023. 3

    2023

  4. [12]

    Monocular 3D object detection for autonomous driving

    Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. Monocular 3D object detection for autonomous driving. InCVPR, 2016. 1

    2016

  5. [13]

    Spatialrgpt: Grounded spatial reasoning in vision-language models

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision-language models. InNeurIPS, 2024. 3

    2024

  6. [14]

    Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction.arXiv preprint arXiv:2505.20279, 2025

    Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Shijie Zhou, Dilin Wang, et al. Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction.arXiv preprint arXiv:2505.20279, 2025. 3

    2025 arXiv

  7. [15]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. InCVPR, 2012. 3, 7, 8, 14, 24, 29

    2012

  8. [16]

    Omni-rgpt: Unifying image and video region-level understanding via token marks

    Miran Heo, Min-Hung Chen, De-An Huang, Sifei Liu, Subhashree Radhakrishnan, Seon Joo Kim, Yu- Chiang Frank Wang, and Ryo Hachiuma. Omni-rgpt: Unifying image and video region-level understanding via token marks. InCVPR, 2025. 3

    2025

  9. [17]

    3d-llm: Injecting the 3d world into large language models.NeurIPS, 2023

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models.NeurIPS, 2023. 3

    2023

  10. [18]

    G 2vlm: Geometry grounded vision language model with unified 3d reconstruction and spatial reasoning.arXiv preprint arXiv:2511.21688, 2025

    Wenbo Hu, Jingli Lin, Yilin Long, Yunlong Ran, Lihan Jiang, Yifan Wang, Chenming Zhu, Runsen Xu, Tai Wang, and Jiangmiao Pang. G 2vlm: Geometry grounded vision language model with unified 3d reconstruction and spatial reasoning.arXiv preprint arXiv:2511.21688, 2025. 3

    2025

  11. [19]

    Monodtr: Monocular 3D object detection with depth-aware transformer

    Kuan-Chih Huang, Tsung-Han Wu, Hung-Ting Su, and Winston H Hsu. Monodtr: Monocular 3D object detection with depth-aware transformer. InCVPR, 2022. 3

    2022

  12. [20]

    3d-r1: Enhancing reasoning in 3d vlms for unified scene understanding.arXiv preprint arXiv:2507.23478, 2025

    Ting Huang, Zeyu Zhang, and Hao Tang. 3d-r1: Enhancing reasoning in 3d vlms for unified scene understanding.arXiv preprint arXiv:2507.23478, 2025. 3 10

    2025 arXiv

  13. [21]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Xu Tang, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025. 3

    2025 arXiv

  14. [22]

    MonoMAE: Enhancing monocular 3D detection through depth-aware masked autoencoders

    Xueying Jiang, Sheng Jin, Xiaoqin Zhang, Ling Shao, and Shijian Lu. MonoMAE: Enhancing monocular 3D detection through depth-aware masked autoencoders. InNeurIPS, 2024. 7

    2024

  15. [23]

    Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024. 3

    2024 arXiv

  16. [24]

    Deviant: Depth equivariant network for monocular 3D object detection

    Abhinav Kumar, Garrick Brazil, Enrique Corona, Armin Parchami, and Xiaoming Liu. Deviant: Depth equivariant network for monocular 3D object detection. InECCV, 2022. 3

    2022

  17. [25]

    GrooMeD-NMS: Grouped mathematically differen- tiable nms for monocular 3D object detection

    Abhinav Kumar, Garrick Brazil, and Xiaoming Liu. GrooMeD-NMS: Grouped mathematically differen- tiable nms for monocular 3D object detection. InCVPR, 2021. 3

    2021

  18. [26]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. InCVPR, 2024. 2

    2024

  19. [27]

    Spatial forcing: Implicit spatial representation alignment for vision-language-action model

    Fuhao Li, Wenxuan Song, Han Zhao, Jingbo Wang, Pengxiang Ding, Donglin Wang, Long Zeng, and Haoang Li. Spatial forcing: Implicit spatial representation alignment for vision-language-action model. arXiv preprint arXiv:2510.12276, 2025. 3

  20. [28]

    Diversity matters: Fully exploiting depth clues for reliable monocular 3D object detection

    Zhuoling Li, Zhan Qu, Yang Zhou, Jianzhuang Liu, Haoqian Wang, and Lihui Jiang. Diversity matters: Fully exploiting depth clues for reliable monocular 3D object detection. InCVPR, 2022. 3

    2022

  21. [29]

    Unimode: Unified monocular 3d object detection

    Zhuoling Li, Xiaogang Xu, SerNam Lim, and Hengshuang Zhao. Unimode: Unified monocular 3d object detection. InCVPR, 2024. 3

    2024

  22. [30]

    Draw-and-understand: Leveraging visual prompts to enable mllms to comprehend what you want.arXiv preprint arXiv:2403.20271, 2024

    Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shang- hang Zhang, and Hongsheng Li. Draw-and-understand: Leveraging visual prompts to enable mllms to comprehend what you want.arXiv preprint arXiv:2403.20271, 2024. 3

    2024 arXiv

  23. [31]

    Monotakd: Teaching assistant knowledge distillation for monocular 3d object detection

    Hou-I Liu, Christine Wu, Jen-Hao Cheng, Wenhao Chai, Shian-Yun Wang, Gaowen Liu, Hugo Latapie, Jhih-Ciang Wu, Jenq-Neng Hwang, Hong-Han Shuai, et al. Monotakd: Teaching assistant knowledge distillation for monocular 3d object detection. InCVPR, 2025. 3, 7

    2025

  24. [32]

    Edge assisted real-time object detection for mobile augmented reality

    Luyang Liu, Hongyu Li, and Marco Gruteser. Edge assisted real-time object detection for mobile augmented reality. InMobiCom, 2019. 1

    2019

  25. [33]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023. 3

    2023 arXiv

  26. [34]

    Monocular 3D object detection with bounding box denoising in 3D by perceiver

    Xianpeng Liu, Ce Zheng, Kelvin B Cheng, Nan Xue, Guo-Jun Qi, and Tianfu Wu. Monocular 3D object detection with bounding box denoising in 3D by perceiver. InICCV, 2023. 3

    2023

  27. [35]

    SMOKE: Single-stage monocular 3D object detection via keypoint estimation

    Zechen Liu, Zizhang Wu, and Roland Tóth. SMOKE: Single-stage monocular 3D object detection via keypoint estimation. InCVPR Workshop, 2020. 3

    2020

  28. [36]

    Geometry uncertainty projection network for monocular 3D object detection

    Yan Lu, Xinzhu Ma, Lei Yang, Tianzhu Zhang, Yating Liu, Qi Chu, Junjie Yan, and Wanli Ouyang. Geometry uncertainty projection network for monocular 3D object detection. InICCV, 2021. 3, 14

    2021

  29. [37]

    Delving into localization errors for monocular 3D object detection

    Xinzhu Ma, Yinmin Zhang, Dan Xu, Dongzhan Zhou, Shuai Yi, Haojie Li, and Wanli Ouyang. Delving into localization errors for monocular 3D object detection. InCVPR, 2021. 3

    2021

  30. [38]

    Spatiallm: Training large language models for structured indoor modeling

    Yongsen Mao, Junhao Zhong, Chuan Fang, Jia Zheng, Rui Tang, Hao Zhu, Ping Tan, and Zihan Zhou. Spatiallm: Training large language models for structured indoor modeling. InNeurIPS, 2025. 3

    2025

  31. [39]

    3D bounding box estimation using deep learning and geometry

    Arsalan Mousavian, Dragomir Anguelov, John Flynn, and Jana Košecká. 3D bounding box estimation using deep learning and geometry. InCVPR, 2017. 14

    2017

  32. [40]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023. 15

    2023 arXiv

  33. [41]

    Learning occupancy for monocular 3D object detection

    Liang Peng, Junkai Xu, Haoran Cheng, Zheng Yang, Xiaopei Wu, Wei Qian, Wenxiao Wang, Boxi Wu, and Deng Cai. Learning occupancy for monocular 3D object detection. InCVPR, 2024. 1 11

    2024

  34. [42]

    UniDepthV2: Universal monocular metric depth estimation made simpler, 2025

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniDepthV2: Universal monocular metric depth estimation made simpler, 2025. 1, 3

    2025

  35. [43]

    Monodgp: Monocular 3D object detection with decoupled-query and geometry-error priors.arXiv preprint arXiv:2410.19590, 2024

    Fanqi Pu, Yifan Wang, Jiru Deng, and Wenming Yang. Monodgp: Monocular 3D object detection with decoupled-query and geometry-error priors.arXiv preprint arXiv:2410.19590, 2024. 3, 7

    2024 arXiv

  36. [44]

    Monoground: Detecting monocular 3D objects from the ground

    Zequn Qin and Xi Li. Monoground: Detecting monocular 3D objects from the ground. InCVPR, 2022. 3

    2022

  37. [45]

    Loc3r-vlm: Language- based localization and 3d reasoning with vision-language models, 2026

    Kevin Qu, Haozhe Qi, Mihai Dusmanu, Mahdi Rad, Rui Wang, and Marc Pollefeys. Loc3r-vlm: Language- based localization and 3d reasoning with vision-language models, 2026. 3

    2026

  38. [46]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025. 3

    2025 arXiv

  39. [47]

    PointRCNN: 3D object proposal generation and detection from point cloud

    Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. PointRCNN: 3D object proposal generation and detection from point cloud. InCVPR, 2019. 1

    2019

  40. [48]

    Geometry- based distance decomposition for monocular 3D object detection

    Xuepeng Shi, Qi Ye, Xiaozhi Chen, Chuangrong Chen, Zhixiang Chen, and Tae-Kyun Kim. Geometry- based distance decomposition for monocular 3D object detection. InICCV, 2021. 1

    2021

  41. [49]

    Geometry- based distance decomposition for monocular 3D object detection

    Xuepeng Shi, Qi Ye, Xiaozhi Chen, Chuangrong Chen, Zhixiang Chen, and Tae-Kyun Kim. Geometry- based distance decomposition for monocular 3D object detection. InICCV, 2021. 14

    2021

  42. [50]

    What does clip know about a red circle? visual prompt engineering for vlms

    Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? visual prompt engineering for vlms. InCVPR, 2023. 3

    2023

  43. [51]

    Eva-clip: Improved training techniques for clip at scale.arXiv preprint arXiv:2303.15389, 2023

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale.arXiv preprint arXiv:2303.15389, 2023. 3

    2023 arXiv

  44. [52]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms, 2024

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms, 2024. 25

    2024

  45. [53]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InCVPR, 2025. 3

    2025

  46. [54]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 2

    2024 arXiv

  47. [55]

    Moge-2: Accurate monocular geometry with metric scale and sharp details

    Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. In NeurIPS, 2025. 1, 3, 18

    2025

  48. [56]

    Embodiedscan: A holistic multi-modal 3d perception suite towards embodied ai

    Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, et al. Embodiedscan: A holistic multi-modal 3d perception suite towards embodied ai. InCVPR, 2024. 1

    2024

  49. [57]

    Probabilistic and geometric depth: Detecting objects in perspective

    Tai Wang, ZHU Xinge, Jiangmiao Pang, and Dahua Lin. Probabilistic and geometric depth: Detecting objects in perspective. InCoRL, 2022. 3

    2022

  50. [58]

    N3d-vlm: Native 3d grounding enables accurate spatial reasoning in vision-language models

    Yuxin Wang, Lei Ke, Boqiang Zhang, Tianyuan Qu, Hanxun Yu, Zhenpeng Huang, Meng Yu, Dan Xu, and Dong Yu. N3d-vlm: Native 3d grounding enables accurate spatial reasoning in vision-language models. arXiv preprint arXiv:2512.16561, 2025. 1, 3

    2025

  51. [59]

    Ov-uni3detr: Towards unified open-vocabulary 3d object detection via cycle-modality propagation

    Zhenyu Wang, Yali Li, Taichi Liu, Hengshuang Zhao, and Shengjin Wang. Ov-uni3detr: Towards unified open-vocabulary 3d object detection via cycle-modality propagation. InECCV, 2024. 3

    2024

  52. [60]

    Monopgc: Monocular 3D object detection with pixel geometry contexts

    Zizhang Wu, Yuanzhu Gan, Lei Wang, Guilian Chen, and Jian Pu. Monopgc: Monocular 3D object detection with pixel geometry contexts. InICRA, 2023. 3

    2023

  53. [61]

    FD3D: Exploiting foreground depth map for feature-supervised monocular 3D object detection

    Zizhang Wu, Yuanzhu Gan, Yunzhe Wu, Ruihao Wang, Xiaoquan Wang, and Jian Pu. FD3D: Exploiting foreground depth map for feature-supervised monocular 3D object detection. InAAAI, 2024. 3, 7

    2024

  54. [62]

    Pointllm: Empower- ing large language models to understand point clouds

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empower- ing large language models to understand point clouds. InECCV, 2024. 3

    2024

  55. [63]

    MonoCD: Monocular 3D object detection with complementary depths

    Longfei Yan, Pei Yan, Shengzhou Xiong, Xuanyu Xiang, and Yihua Tan. MonoCD: Monocular 3D object detection with complementary depths. InCVPR, 2024. 7 12

    2024

  56. [64]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    An Yang, Anfeng Li, Baosong Yang, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    arXiv

  57. [65]

    Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v.arXiv preprint arXiv:2310.11441, 2023

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v.arXiv preprint arXiv:2310.11441, 2023. 3

    2023 arXiv

  58. [66]

    Lisa++: An improved baseline for reasoning segmentation with large language model.arXiv preprint arXiv:2312.17240,

    Senqiao Yang, Tianyuan Qu, Xin Lai, Zhuotao Tian, Bohao Peng, Shu Liu, and Jiaya Jia. Lisa++: An improved baseline for reasoning segmentation with large language model.arXiv preprint arXiv:2312.17240,

    arXiv

  59. [67]

    3d-mood: Lifting 2d to 3d for monocular open-set object detection

    Yung-Hsu Yang, Luigi Piccinelli, Mattia Segu, Siyuan Li, Rui Huang, Yuqian Fu, Marc Pollefeys, Hermann Blum, and Zuria Bauer. 3d-mood: Lifting 2d to 3d for monocular open-set object detection. InICCV,

  60. [68]

    Open vocabulary monocular 3d object detection

    Jin Yao, Hao Gu, Xuweiyi Chen, Jiayun Wang, and Zezhou Cheng. Open vocabulary monocular 3d object detection. In3DV, 2026. 1, 3, 7, 23, 25

    2026

  61. [69]

    Dwyer, and Zezhou Cheng

    Jin Yao, Radowan Mahmud Redoy, Sebastian Elbaum, Matthew B. Dwyer, and Zezhou Cheng. Labelany3d: Label any object 3d in the wild. InNeurIPS, 2025. 1, 3, 8

    2025

  62. [70]

    Center-based 3D object detection and tracking

    Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3D object detection and tracking. In CVPR, 2021. 1

    2021

  63. [71]

    Videorefer suite: Advancing spatial-temporal object understanding with video llm

    Yuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng, Boqiang Zhang, Long Li, Xin Li, Deli Zhao, Wenqiao Zhang, Yueting Zhuang, et al. Videorefer suite: Advancing spatial-temporal object understanding with video llm. InCVPR, 2025. 3

    2025

  64. [72]

    Detect anything 3d in the wild.arXiv preprint arXiv:2504.07958, 2025

    Hanxue Zhang, Haoran Jiang, Qingsong Yao, Yanan Sun, Renrui Zhang, Hao Zhao, Hongyang Li, Hongzi Zhu, and Zetong Yang. Detect anything 3d in the wild.arXiv preprint arXiv:2504.07958, 2025. 1, 3, 7, 8, 25, 26, 27

    2025

  65. [73]

    Monodetr: Depth-guided transformer for monocular 3D object detection

    Renrui Zhang, Han Qiu, Tai Wang, Ziyu Guo, Ziteng Cui, Yu Qiao, Hongsheng Li, and Peng Gao. Monodetr: Depth-guided transformer for monocular 3D object detection. InICCV, 2023. 1, 3, 7

    2023

  66. [74]

    Objects are different: Flexible monocular 3D object detection

    Yunpeng Zhang, Jiwen Lu, and Jie Zhou. Objects are different: Flexible monocular 3D object detection. InCVPR, 2021. 3

    2021

  67. [75]

    Unleashing the power of chain-of-prediction for monocular 3d object detection

    Zhihao Zhang, Abhinav Kumar, Girish Chandar Ganesan, and Xiaoming Liu. Unleashing the power of chain-of-prediction for monocular 3d object detection. InCVPR, 2026. 3, 7, 8, 14, 24

    2026

  68. [76]

    Detrs beat yolos on real-time object detection

    Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, and Jie Chen. Detrs beat yolos on real-time object detection. InCVPR, 2024. 3

    2024

  69. [77]

    3d-vla: 3d vision-language-action generative world model.arXiv preprint arXiv:2403.09631, 2024

    Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: 3d vision-language-action generative world model.arXiv preprint arXiv:2403.09631, 2024. 3

    2024 arXiv

  70. [78]

    Monoatt: Online monocular 3D object detection with adaptive token transformer

    Yunsong Zhou, Hongzi Zhu, Quan Liu, Shan Chang, and Minyi Guo. Monoatt: Online monocular 3D object detection with adaptive token transformer. InCVPR, 2023. 3

    2023

  71. [79]

    Single image 3d object detection and pose estimation for grasping

    Menglong Zhu, Konstantinos G Derpanis, Yinfei Yang, Samarth Brahmbhatt, Mabel Zhang, Cody Phillips, Matthieu Lecce, and Kostas Daniilidis. Single image 3d object detection and pose estimation for grasping. InICRA, 2014. 1

    2014

  72. [80]

    move closer to camera

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InCoRL, 2023. 3 13 Table 6:Comparison with non-VLM refinement mecha...

    2023

  73. [82]

    16 Table 8: Oracle study under the Omni3D evaluation protocol

    Render the current box b(t) as a wireframe on the input image to obtain I(t) = render(I, π(b(t);K)). 16 Table 8: Oracle study under the Omni3D evaluation protocol. Each row replaces one attribute of Cube R-CNN’s predictions with either the ground truth or the output of a depth...

  74. [83]

    Query the VLM with I(t) and a fixed instruction prompt; the model first emits a chain-of- thought reasoning trace, then the action tokens(a (t) d , a(t) m )

  75. [84]

    Compute the depth update ∆d(t) = dir(a(t) d )·mag(a (t) m , sobj) and update the box: d(t+1) = d(t) + ∆d(t), with (ˆx(t+1),ˆy(t+1)) re-derived along the camera ray to keep the box’s image location consistent with the new depth

  76. [85]

    We use Tmax = 2 in all experiments

    Continue if a(t) d ∈ {⟨depth_closer⟩,⟨depth_farther⟩} ; terminate if a(t) d = ⟨depth_ok⟩ or if the maximum number of iterations Tmax is reached. We use Tmax = 2 in all experiments. The chain-of-thought reasoning trace is generated at every step but used only for action-token d...

  77. [86]

    Indoor Scenes

    Selection Outdoor Scenes ... Indoor Scenes ... 3M Samples

  78. [87]

    Filtering Geometric Filter - Object truncation - Object occlusion VLM Semantic Filter - Object truncation - Object occlusion - Object visibility - Image quality 335K Samples

  79. [88]

    Perturbing Action Tokens <depth_ok> <depth_further> <depth_closer> 3M Samples Random Sampler <step_small> <step_medium> <step_large>

  80. [89]

    a black car viewed from behind on a street

    CoT Reasoning Identify Object World Knowledge Recall Find Visual Evidence Make Decision 3M CoTs Figure 5:Data construction pipeline for training RefineAny3D.We collect diverse indoor and outdoor scenes (Selection), followed by geometric and VLM-based semantic filtering to remo...

  81. [90]

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  82. [2025]

    2, 6, 7, 15, 19, 21, 25, 29

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.