Pith. sign in

REVIEW 3 major objections 5 minor 94 references

UrbanCraft: Urban View Extrapolation via Hierarchical Sem-Geometric Priors

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UrbanCraft claims the first framework that synthesizes large and text-ambiguous extrapolated urban views with distribution consistent with the observed scene, guided by hierarchical semantic-geometric priors.

desk verdict A clever, plausible conditioning scheme for urban extrapolated view synthesis, but the SOTA claim rests on a dataset-circular FID metric with a tiny margin, so the evidence supports only a qualitative advance. read the letter →

arxiv 2505.23434 v1 pith:YJVONF7L submitted 2025-05-29 cs.CV

Tianhang Wang , Fan Lu , Sanqing Qu , Guo Yu , Shihang Du , Ya Wu , Yuan Huang , Guang Chen This is my paper
classification cs.CV
keywords extrapolatedviewsynthesisurbanscenereconstruction3DGaussianSplattingdiffusionmodelscoredistillationsamplingoccupancygridsemantic-geometricpriorsControlNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes on Extrapolated View Synthesis (EVS) for urban scenes reconstructed from forward-facing vehicle cameras: synthesizing photorealistic images from viewpoints far outside the training camera trajectory, such as looking left, right, or downward. It claims to be the first effective framework for large and text-ambiguous extrapolated views, where previous text-only diffusion guidance fails because a text prompt cannot specify local layout, occlusion, or orientation. UrbanCraft's answer is to condition a pretrained text-to-image diffusion model on hierarchical semantic-geometric priors, a scene-level occupancy grid rendered at the extrapolated pose plus instance-level 3D bounding-box rotation maps, and to distill the conditioned model into a 3D Gaussian Splatting scene through a new Hierarchical Sem-Geometric-Guided Variational Score Distillation (HSG-VSD) loss. If correct, this makes urban reconstructions usable from viewpoints no training image ever saw, and turns coarse semantic-geometric maps into enough control to keep generated content consistent with the observed scene.

What carries the argument

The load-bearing object is the Hierarchical Sem-Geometric (HSG) representation and its use in score distillation. HSG is a control signal assembled by rendering a scene-level occupancy grid, giving semantic and depth maps, and an instance-level projected rotation map, the flattened 3x3 rotation matrix of each object's 3D bounding box, at the extrapolated camera pose, yielding a 13-channel condition. This signal is fed through ControlNet into a Stable Diffusion v2.1 model finetuned on urban datasets as UrbanCraft2D; a fixed base prompt is used so text does not dominate. HSG-VSD then replaces the text-only score distillation gradient with a gradient comparing the ControlNet-conditioned denoiser against a LoRA-adapted denoiser, both conditioned on the rendered sem-geometric signal, so that each optimization step pulls the rendered image toward the distribution of the observable scene. A supplementary geometry score distillation loss on rendered depth and normals regularizes Gaussian covariance shapes to suppress surface cavities and floating artifacts.

What would settle it

Render an extrapolated view in a scene whose occupancy grid has been deliberately truncated, for example cutting the perception range in half, while keeping the true scene otherwise unchanged; if blurring artifacts appear exactly in the truncated regions and persist after HSG-VSD optimization, then the claim that hierarchical sem-geometric priors enable consistent large extrapolation fails for exactly the regime the paper targets.

Watch

Extended reading notes

Core claim

UrbanCraft claims that the missing ingredient for urban extrapolated view synthesis is not a bigger or better diffusion model but hierarchical semantic-geometric control. The paper shows that scene-level occupancy grids, providing coarse semantic and depth layout, together with instance-level projected rotation maps from 3D bounding boxes, providing object orientation and spatial relationships, form a 13-channel control signal that can be injected into a finetuned Stable Diffusion model via ControlNet. This conditioned model, named UrbanCraft2D, is then distilled into a static-plus-dynamic 3D Gaussian Splatting scene using HSG-VSD, a variational score distillation loss in which ControlNet features condition both the pretrained denoiser and a LoRA-adapted denoiser at each extrapolated camera pose. The paper reports state-of-the-art FID and KID on extrapolated views while keeping on-par quality on conventional test cameras, and presents this as the first framework that repairs large, text-ambiguous extrapolated views without collapsing vehicle geometry or losing color and depth consistency with the observed scene.

Load-bearing premise

The method assumes that accurate occupancy grid maps and 3D bounding-box annotations for the scene are available and can be rendered correctly at the extrapolated camera pose; if those maps are incomplete or wrong, the control signal is wrong and the whole pipeline degrades, which the paper's own limitation section confirms by tying blurring artifacts to incomplete occupancy grids.

Editorial extensions

If this is right

  • Extrapolated views that are far left, far right, or downward can be rendered without the black holes, blurring, and collapsing vehicle geometry that text-only diffusion baselines produce.
  • Text ambiguity stops being the main failure mode because local layout comes from occupancy grids and object poses, so a single prompt can guide many different extrapolated views consistently.
  • With scene-completion algorithms supplying occupancy grids and detectors providing 3D boxes, the framework can be applied to custom driving datasets that lack dense multi-view imagery.
  • Instance-level control is strong enough to support editing, such as changing a car's color while preserving its geometry and position, which the paper demonstrates as an additional capability.
  • The two-stage training recipe, reconstruction first and distillation repair second, plus a LoRA-based style transfer, lets a model pretrained on one urban dataset be adapted to another.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The single biggest lever on final quality may be the completeness of the occupancy grid rather than the diffusion model itself: the paper's own failure analysis ties blurring to incomplete grids, which suggests a testable recipe of corrupting the grid, such as truncating its perception range, and measuring how FID degrades.
  • The HSG recipe is domain-generic: any setting where cheap geometric primitives, such as maps, boxes, or depth, can be rendered from a novel pose could use the same ControlNet-plus-variational-score-distillation conditioning, so the approach should transfer to indoor navigation, aerial or satellite view synthesis, or other structured scenes, not just street driving.
  • Because the paper evaluates distribution consistency with FID and KID relative to training images, those numbers do not directly measure geometric correctness at extrapolated views; a stronger public test would hold out ground-truth images from a parallel vehicle pass and measure pixel-level error there.
  • The reported speed of about two iterations per second, together with the ablation showing the SDS-style variant suffers over-saturation and over-smoothing, suggests the variational branch is the accuracy bottleneck and that a faster SDS-style variant may be the practical path to real-time urban scene repair.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes UrbanCraft, a framework for extrapolated view synthesis (EVS) in urban scenes. It introduces a pretrained 2D diffusion model (UrbanCraft2D) conditioned on hierarchical semantic-geometric priors (occupancy grids, semantic/depth maps, and instance-level rotation maps), and distills this model into a 3D Gaussian representation via a proposed HSG-VSD loss. The authors claim state-of-the-art EVS performance, reporting quantitative results on KITTI-360 and qualitative results on KITTI-360 and NuScenes.

Significance. The work addresses a practically important problem: rendering urban scenes from viewpoints far outside the training camera distribution. The method is well motivated, combining scene-level and instance-level control signals in a single diffusion prior, and the qualitative results show plausible repair of unseen regions, including improved vehicle geometry and color consistency. The paper also provides extensive supplementary material, including failure cases and public-resource acknowledgments. However, the central quantitative claim of state-of-the-art performance is not well supported: the main EVS metrics (FID/KID) are computed against the very training images used to fine-tune UrbanCraft2D (Supp. 6.1), the margins over VEGS are tiny and lack error bars, and on conventional test-view metrics the method is on par with VEGS. The paper's own limitations (Supp. 8.2) admit blurring from incomplete occupancy grids and Gaussian sparks in large unseen views, further tempering the headline claim.

major comments (3)
  1. [Sec. 4.3, Table 1; Supp. 6.1] The central state-of-the-art claim rests on FID/KID computed between EVS renders and training images, but UrbanCraft2D is pretrained on the same KITTI-360 and NuScenes datasets used for evaluation (Supp. 6.1). This makes the EVS metric close to circular: the diffusion prior has seen the test scenes, and the HSG-VSD loss explicitly forces distribution consistency with the observable/training views, so matching FID/KID is essentially optimizing the metric itself. The reported gap over VEGS is tiny (FID 278.75 vs 279.27; KID 0.8906 vs 0.9245) and is reported without error bars or significance testing. On conventional test views with ground truth (Table 1), UrbanCraft is essentially on par with VEGS (PSNR 24.15 vs 24.00; LPIPS 0.219 vs 0.216). The evidence therefore does not substantiate the state-of-the-art claim. Please provide held-out-scene or cross-dataset evaluation, error bars over multiple runs, and/or metrics that do not rely on the training distribution.
  2. [Sec. 3.1; Supp. 8.2] The method assumes accurate occupancy grids and 3D bounding boxes as inputs, and the authors' own failure analysis (Supp. 8.2) states that incomplete occupancy grids cause blurring artifacts and that large unseen views produce Gaussian sparks due to an optimization imbalance. These are not merely edge cases: they directly affect the claimed capability to synthesize large- and text-ambiguous extrapolated camera views. The paper should quantify how often these failures occur (e.g., by difficulty level or scene) and discuss whether the hierarchical priors actually improve robustness to imperfect annotations, or at least temper the claim accordingly.
  3. [Supp. 6.6; Table 1] The paper mentions the existing EVS benchmark [71] only in a supplementary discussion and does not report results on that benchmark or compare with other EVS baselines beyond VEGS and 3DGS variants. Given the paper's title and central claim, a direct comparison on a standardized benchmark with fixed train/test splits is necessary to support the state-of-the-art assertion. Without it, the evidence is limited to a single dataset (KITTI-360) under the authors' own evaluation protocol.
minor comments (5)
  1. [Table 1 caption] The caption says 'HSD-VSD distillation'; this should be 'HSG-VSD distillation'.
  2. [Figure 8 caption] The caption says 'sharp cone phonemes'; this appears to be a typo for 'sharp cone phenomena'.
  3. [Eq. (1)] The expression 'epsilon_p(x_t, t, y, psi(SG(T)) - epsilon_phi(...))' has an ambiguous parenthesis structure; a closing parenthesis is needed before the minus sign for clarity.
  4. [Sec. 4.1] The phrase 'extent it to a more comprehensive setting' appears to be a typo for 'extend it to a more comprehensive setting'.
  5. [Abstract / Sec. 3.1] The term 'sem-geometric' is used in the abstract but not defined until Section 3.1; please define it at first use.

Circularity Check

1 steps flagged · score 6.0 of 10

The EVS evaluation is partly circular: UrbanCraft2D is fine-tuned on the same training images used as the FID/KID reference, and HSG-VSD explicitly optimizes EVS renders to match that distribution.

  1. fitted input called prediction [Sec. 3.1 (Pretraining) + Sec. 3.2 Eq. (1) + Sec. 4.3 / Table 1]
    "Specifically, we finetune Stable Diffusion v2.1 [54] with urban scenes datasets including nuScenes [31] and KITTI-360 [23]. ... We follow VEGS [26] to employ FID and KID scores with respect to the training images to assess the reconstruction quality of EVS renderings."

    UrbanCraft2D is a diffusion model fine-tuned on exactly the KITTI-360/nuScenes training images that later define the FID/KID reference in Table 1. Eq. (1) (HSG-VSD) then optimizes extrapolated renderings by score distillation toward the conditional distribution p0(x0|y, psi(SG(T))) learned from those same training images, while L_reconst directly fits the training cameras. The reported FID/KID are computed 'between EVS and training images', so the metric measures proximity to the very distribution the optimization is forced to match.

full rationale

The quantitative SOTA claim rests on Table 1's FID/KID computed between EVS renderings and training images (Sec. 4.3). UrbanCraft2D is fine-tuned on KITTI-360 and nuScenes (Sec. 3.1), i.e., on the same images that define the metric reference. Eq. (1) (HSG-VSD) then uses this model to pull extrapolated renders toward p0(x0|y, psi(SG(T))), a distribution learned from those training images, while L_reconst fits training cameras directly. Thus the FID/KID gap over VEGS measures how well the method matched its own training distribution rather than an independent extrapolation property. The margin is small (278.75 vs 279.27 FID; 0.8906 vs 0.9245 KID) and reported without error bars or significance tests. On conventional test cameras with ground truth, UrbanCraft is roughly on par with VEGS (PSNR 24.15 vs 24.00; LPIPS 0.219 vs 0.216), so the only claimed advantage is on the circular EVS metric. The paper's own Supp. 8.2 admits Gaussian sparks and blurring artifacts from incomplete occupancy grids, further limiting the headline claim. No load-bearing self-citations were found; citations to the authors' prior work are contextual or used as baselines, not as the justification for the central result.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or metaphysical entities are introduced. The free parameters are loss weights that must be tuned by hand. The axioms are domain assumptions about the availability and correctness of occupancy maps, object detections, and normal priors, all of which the method depends on for its control signals. The invented-entities list is empty because UrbanCraft2D and HSG-VSD are methods/models, not postulated entities.

free parameters (3)
  • lambda1 (reconstruction loss weight) = 1e4
    Chosen to balance the large magnitude of the distillation loss; the paper notes that a unified training failed with this scale, so it is a hand-tuned hyperparameter affecting the result.
  • lambda2 (distillation loss weight) = 1.0
    Set to 1.0 in the loss formulation; interacts with lambda1 to control the optimization balance.
  • lambda_r (SSIM weight) = 0.2
    Standard weight for the D-SSIM term in the reconstruction loss, used in the paper's total loss.
assumptions (4)
  • domain assumption Occupancy grid maps provide an accurate coarse semantic-geometric prior of the scene.
    Invoked in Sec 3.1: the method renders the occupancy grid at extrapolated camera poses to build the HSG control signal. The limitations section admits blurring when the occupancy is incomplete or outside the perceptual range.
  • domain assumption 3D bounding box annotations are available and correct for instance-level orientation priors.
    Used in Sec 3.1 to build the instance-level rotation map R, which is concatenated with semantic and depth controls for UrbanCraft2D. Object detection quality is not evaluated or modeled.
  • domain assumption Surface normal priors estimated from training images can regularize 3D Gaussian covariance orientation.
    Assumed in Sec 3.3 following VEGS [26]: the paper uses a rendered covariance map supervised with estimated surface normals to improve extrapolated geometry.
  • domain assumption UrbanCraft2D, fine-tuned on KITTI-360 and NuScenes, provides a distribution consistent with the observable scenes at extrapolated viewpoints.
    The whole method depends on the pretrained 2D diffusion model capturing scene-specific appearance and layout; this is the foundation of the HSG-VSD distillation loss in Sec 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UrbanCraft: Urban View Extrapolation via Hierarchical Sem-Geometric Priors." pith.science (2026). https://pith.science/paper/YJVONF7L

@misc{pith2026250523434,
  author       = {Pith},
  title        = {Pith review of: UrbanCraft: Urban View Extrapolation via Hierarchical Sem-Geometric Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YJVONF7L}},
  note         = {Machine review of arXiv:2505.23434}
}
read the original abstract

Existing neural rendering-based urban scene reconstruction methods mainly focus on the Interpolated View Synthesis (IVS) setting that synthesizes from views close to training camera trajectory. However, IVS can not guarantee the on-par performance of the novel view outside the training camera distribution (\textit{e.g.}, looking left, right, or downwards), which limits the generalizability of the urban reconstruction application. Previous methods have optimized it via image diffusion, but they fail to handle text-ambiguous or large unseen view angles due to coarse-grained control of text-only diffusion. In this paper, we design UrbanCraft, which surmounts the Extrapolated View Synthesis (EVS) problem using hierarchical sem-geometric representations serving as additional priors. Specifically, we leverage the partially observable scene to reconstruct coarse semantic and geometric primitives, establishing a coarse scene-level prior through an occupancy grid as the base representation. Additionally, we incorporate fine instance-level priors from 3D bounding boxes to enhance object-level details and spatial relationships. Building on this, we propose the \textbf{H}ierarchical \textbf{S}emantic-Geometric-\textbf{G}uided Variational Score Distillation (HSG-VSD), which integrates semantic and geometric constraints from pretrained UrbanCraft2D into the score distillation sampling process, forcing the distribution to be consistent with the observable scene. Qualitative and quantitative comparisons demonstrate the effectiveness of our methods on EVS problem.

Figures

Figures reproduced from arXiv: 2505.23434 by the authors.

Figure 1
Figure 1. UrbanCraft for urban view extrapolation. (a) Illustration of Extrapolated View Synthesis (EVS) problem in urban scenes reconstructed with forward-facing cameras. Unlike traditional test cameras that resemble training camera poses, we access view synthesis using cameras that are remote from the training camera distribution. (b) Qualitative comparisons on EVS to baselines. Abstract Existing neural rendering-based urba… view at source ↗
Figure 2
Figure 2. Overview of UrbanCraft. We introduce UrbanCraft, a method that repairs unseen extrapolated views with hierarchical sem￾geometric priors. Our framework contains three stages: (a) pretrained of a 2D diffusion model, named UrbanCraft2D, including stable diffusion model ϵp and corresponding ControlNet ψ(·) and (b) distillation of the UrbanCraft2D by proposed HSG-VSD to enforce the optimization process to be consistent w… view at source ↗
Figure 4
Figure 4. Illustration of the effectiveness of the proposed Hierar [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (13 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison on KITTI-360 [23] for extrapolated view synthesis under different difficulty levels (Easy, Middle and Hard) across three settings. EVS-D and EVS-LR refer to extrapolated views facing downwards and left/right, respectively, while EVS-LR￾D represen…
Figure 6
Figure 6. Figure 6: Qualitative comparison about extrapolated view synthesis of multi-view distribution consistency of partially visible vehicle. Ours [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Fine-grained comparison with VEGS. Note that ours can [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Qualitative ablation comparisons results on the pro [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison about NuScenes [31] and KITTI￾360 [23] pretrained UrbanCraft2D, respectively. because KITTI-360 provides only forward-facing images, whereas NuScenes offers images from six different view￾ing angles, covering a much broader distribution. Conse￾qu…
Figure 10
Figure 10. Figure 10: Visualization of occupancy grid maps. ban reconstruction applications. 6.3. Hyper-parameters Setting The training process of UrbanCraft consists of two stages: i) Initial Reconstruction: In this stage, we focus on the coarse reconstruction of urban 3D representations …
Figure 11
Figure 11. Figure 11: Qualitative comparison about multi-view distribution consistency for extrapolated view synthesis. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Hyper-parameters setting about the training iterations. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison of RGB images and corresponding depth maps on KITTI-360 [ [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Qualitative comparison about failure cases for extrapolated view synthesis. [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Comparison of different methods on the barely-visible [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Qualitative comparison between UrbanArchi* and pro [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Qualitative comparison on NuScenes [31] for extrapo￾lated view synthesis. 9. UrbanCraft on NuScenes Dataset In the main manuscript, to ensure a fair comparison, we uti￾lize the same KITTI-360 [23] dataset as VEGS [26] for both qualitative and quantitative analysis. To…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

94 extracted references · 54 canonical work pages

  1. [71]

    Extrapolated urban view synthesis benchmark, 2024

    Xiangyu Han, Zhen Jia, Boyi Li, Yan Wang, Boris Ivanovic, Yurong You, Lingjie Liu, Yue Wang, Marco Pavone, Chen Feng, and Yiming Li. Extrapolated urban view synthesis benchmark, 2024. 5, 14

    work page 2024

  2. [1]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 1, 2, 3

    2021

  3. [2]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4578–4587, 2021. 1

    2021

  4. [3]

    Putting nerf on a diet: Semantically consistent few-shot view synthesis

    Ajay Jain, Matthew Tancik, and Pieter Abbeel. Putting nerf on a diet: Semantically consistent few-shot view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5885–5894, 2021. 2

    2021

  5. [4]

    Reg- nerf: Regularizing neural radiance fields for view synthesis from sparse inputs

    Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Reg- nerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5480–5490, 2022. 2

    2022

  6. [5]

    Freenerf: Im- proving few-shot neural rendering with free frequency reg- ularization

    Jiawei Yang, Marco Pavone, and Yue Wang. Freenerf: Im- proving few-shot neural rendering with free frequency reg- ularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8254–8263, 2023

    2023

  7. [6]

    Sparsenerf: Distilling depth ranking for few-shot novel view synthesis

    Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Zi- wei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9065–9076,

  8. [7]

    Block-nerf: Scalable large scene neural view synthesis

    Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Prad- han, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. Block-nerf: Scalable large scene neural view synthesis. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8248–8258, 2022. 1

    2022

Show all 94 references
  1. [8]

    Mega-nerf: Scalable construction of large- scale nerfs for virtual fly-throughs

    Haithem Turki, Deva Ramanan, and Mahadev Satya- narayanan. Mega-nerf: Scalable construction of large- scale nerfs for virtual fly-throughs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12922–12931, 2022

    2022

  2. [9]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2024

  3. [10]

    S-nerf: Neural radiance fields for street views

    Ziyang Xie, Junge Zhang, Wenye Li, Feihu Zhang, and Li Zhang. S-nerf: Neural radiance fields for street views. arXiv preprint arXiv:2303.00749, 2023. 1

    2023 arXiv

  4. [11]

    Emernerf: Emergent spatial-temporal scene decomposition via self-supervision

    Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Se- ung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, et al. Emernerf: Emergent spatial-temporal scene decomposition via self-supervision. arXiv preprint arXiv:2311.02077, 2023

    2023 arXiv

  5. [12]

    Suds: Scalable urban dynamic scenes

    Haithem Turki, Jason Y Zhang, Francesco Ferroni, and Deva Ramanan. Suds: Scalable urban dynamic scenes. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12375–12385, 2023. 1

    2023

  6. [13]

    Point- nerf: Point-based neural radiance fields

    Qiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi, Zhixin Shu, Kalyan Sunkavalli, and Ulrich Neumann. Point- nerf: Point-based neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5438–5448, 2022. 1

    2022

  7. [14]

    Npbg++: Accelerating neural point-based graphics

    Ruslan Rakhimov, Andrei-Timotei Ardelean, Victor Lem- pitsky, and Evgeny Burnaev. Npbg++: Accelerating neural point-based graphics. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 15969–15979, 2022

    2022

  8. [15]

    Neural mesh-based graphics

    Shubhendu Jena, Franck Multon, and Adnane Boukhayma. Neural mesh-based graphics. In European Conference on Computer Vision, pages 739–757. Springer, 2022

    2022

  9. [16]

    Urban radiance field representa- tion with deformable neural mesh primitives

    Fan Lu, Yan Xu, Guang Chen, Hongsheng Li, Kwan-Yee Lin, and Changjun Jiang. Urban radiance field representa- tion with deformable neural mesh primitives. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 465–476, 2023. 1

    2023

  10. [17]

    Neural scene graphs for dynamic scenes

    Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 2856–2865, 2021. 1

    2021

  11. [18]

    Mars: An instance-aware, mod- ular and realistic simulator for autonomous driving

    Zirui Wu, Tianyu Liu, Liyi Luo, Zhide Zhong, Jianteng Chen, Hongmin Xiao, Chao Hou, Haozhe Lou, Yuantao Chen, Runyi Yang, et al. Mars: An instance-aware, mod- ular and realistic simulator for autonomous driving. In CAAI International Conference on Artificial Intelligence, pages 3–

  12. [19]

    Multi-level neural scene graphs for dynamic urban environments

    Tobias Fischer, Lorenzo Porzi, Samuel Rota Bulo, Marc Pollefeys, and Peter Kontschieder. Multi-level neural scene graphs for dynamic urban environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21125–21135, 2024. 1

    2024

  13. [20]

    Graph attention multi-layer perceptron

    Wentao Zhang, Ziqi Yin, Zeang Sheng, Yang Li, Wen Ouyang, Xiaosen Li, Yangyu Tao, Zhi Yang, and Bin Cui. Graph attention multi-layer perceptron. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discov- ery and Data Mining, pages 4560–4570, 2022. 2

    2022

  14. [21]

    Graph attention networks

    Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio’, and Yoshua Bengio. Graph attention networks. ArXiv, abs/1710.10903, 2017. 2

    2017 arXiv

  15. [22]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Computer Vision and Pattern Recog- nition (CVPR), 2012. 2

    2012

  16. [23]

    Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d

    Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2022. 2, 4, 5, 6, 13, 16, 18

    2022

  17. [24]

    Panoptic nerf: 3d-to-2d label transfer for panoptic urban scene segmentation

    Xiao Fu, Shangzhan Zhang, Tianrun Chen, Yichong Lu, Lanyun Zhu, Xiaowei Zhou, Andreas Geiger, and Yiyi Liao. Panoptic nerf: 3d-to-2d label transfer for panoptic urban scene segmentation. In International Conference on 3D Vi- sion (3DV), 2022. 2

    2022

  18. [25]

    Rcdn: Towards robust camera-insensitivity collaborative perception via dynamic feature-based 3d neu- ral modeling

    Tianhang Wang, Fan Lu, Zehan Zheng, Guang Chen, and Changjun Jiang. Rcdn: Towards robust camera-insensitivity collaborative perception via dynamic feature-based 3d neu- ral modeling. Advances in Neural Information Processing Systems, 2024. 2

    2024

  19. [26]

    Vegs: View extrapolation of urban scenes in 3d gaussian splatting using learned priors

    Sungwon Hwang, Min-Jung Kim, Taewoong Kang, Jayeon Kang, and Jaegul Choo. Vegs: View extrapolation of urban scenes in 3d gaussian splatting using learned priors. Euro- pean Conference on Computer Vision (ECCV), 2024. 2, 4, 5, 8, 18

    2024

  20. [27]

    Se- mantickitti: A dataset for semantic scene understanding of lidar sequences

    Jens Behley, Martin Garbade, Andres Milioto, Jan Quen- zel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Se- mantickitti: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 9297–9307,

  21. [28]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Neural Information Processing Systems, 36, 2024. 5, 13

    2024

  22. [29]

    Ss- cbench: A large-scale 3d semantic scene completion bench- mark for autonomous driving

    Yiming Li, Sihang Li, Xinhao Liu, Moonjun Gong, Kenan Li, Nuo Chen, Zijun Wang, Zhiheng Li, Tao Jiang, Fisher Yu, Yue Wang, Hang Zhao, Zhiding Yu, and Chen Feng. Ss- cbench: A large-scale 3d semantic scene completion bench- mark for autonomous driving. In 2024 IEEE/RSJ Interna...

    2024

  23. [30]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2, 4

    2023

  24. [31]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and patte...

    2020

  25. [32]

    Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pa...

  26. [33]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5470–5479, 2022

    2022

  27. [34]

    Zip-nerf: Anti-aliased grid-based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19697–19705, 2023

    2023

  28. [35]

    Tri-miprf: Tri-mip represen- tation for efficient anti-aliasing neural radiance fields

    Wenbo Hu, Yuling Wang, Lin Ma, Bangbang Yang, Lin Gao, Xiao Liu, and Yuewen Ma. Tri-miprf: Tri-mip represen- tation for efficient anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19774–19783, 2023. 2

    2023

  29. [36]

    Tensorf: Tensorial radiance fields

    Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. InEuropean con- ference on computer vision, pages 333–350. Springer, 2022. 2

    2022

  30. [37]

    Plenoxels: Radiance fields without neural networks

    Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5501–5510, 2022

    2022

  31. [38]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  32. [39]

    Instant neural graphics primitives with a mul- tiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a mul- tiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022

    2022

  33. [40]

    Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction

    Cheng Sun, Min Sun, and Hwann-Tzong Chen. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5459– 5469, 2022. 2

    2022

  34. [41]

    Neural scene graphs for dynamic scenes

    Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) , pages 2856–2865, June 2021. 2

    2021

  35. [42]

    Unisim: A neural closed-loop sensor simulator

    Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Mani- vasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Ur- tasun. Unisim: A neural closed-loop sensor simulator. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 1389–1399, 2023

    2023

  36. [43]

    Street gaussians: Modeling dynamic urban scenes with gaussian splatting

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In ECCV, 2024. 2

    2024

  37. [44]

    Diffusionerf: Regularizing neural radiance fields with denoising diffu- sion models

    Jamie Wynn and Daniyar Turmukhambetov. Diffusionerf: Regularizing neural radiance fields with denoising diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4180– 4189, 2023. 2

    2023

  38. [45]

    Ray priors through reprojection: Improving neu- ral radiance fields for novel view extrapolation

    Jian Zhang, Yuanqing Zhang, Huan Fu, Xiaowei Zhou, Bowen Cai, Jinchi Huang, Rongfei Jia, Binqiang Zhao, and Xing Tang. Ray priors through reprojection: Improving neu- ral radiance fields for novel view extrapolation. In Proceed- ings of the IEEE/CVF Conference on Computer Visi...

    2022

  39. [46]

    Ner- fvs: Neural radiance fields for free view synthesis via geom- etry scaffolds

    Chen Yang, Peihao Li, Zanwei Zhou, Shanxin Yuan, Bing- bing Liu, Xiaokang Yang, Weichao Qiu, and Wei Shen. Ner- fvs: Neural radiance fields for free view synthesis via geom- etry scaffolds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2023

  40. [47]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  41. [48]

    Zero-shot text-guided object genera- tion with dream fields

    Ajay Jain, Ben Mildenhall, Jonathan T Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object genera- tion with dream fields. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 867–876, 2022. 2

    2022

  42. [49]

    Avatarclip: Zero-shot text- driven generation and animation of 3d avatars

    Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, and Ziwei Liu. Avatarclip: Zero-shot text- driven generation and animation of 3d avatars. ACM Trans- actions on Graphics (TOG), 41(4):1–19, 2022

    2022

  43. [50]

    Clip-mesh: Generating textured meshes from text using pretrained image-text models

    Nasir Mohammad Khalid, Tianhao Xie, Eugene Belilovsky, and Tiberiu Popa. Clip-mesh: Generating textured meshes from text using pretrained image-text models. InSIGGRAPH Asia 2022 conference papers, pages 1–8, 2022

    2022

  44. [51]

    Clip-forge: Towards zero-shot text-to-shape genera- tion

    Aditya Sanghi, Hang Chu, Joseph G Lambourne, Ye Wang, Chin-Yi Cheng, Marco Fumero, and Kamal Rahimi Malek- shan. Clip-forge: Towards zero-shot text-to-shape genera- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 18603–18613, 2022

    2022

  45. [52]

    Urban architect: Steerable 3d urban scene generation with layout prior

    Fan Lu, Kwan-Yee Lin, Yan Xu, Hongsheng Li, Guang Chen, and Changjun Jiang. Urban architect: Steerable 3d urban scene generation with layout prior. arXiv preprint arXiv:2404.06780, 2024. 2

    2024 arXiv

  46. [53]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 2

    2022 arXiv

  47. [54]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 4, 5

    2022

  48. [55]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  49. [56]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22246–22256, 2023. 2

    2023

  50. [57]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2023

  51. [58]

    Latent-nerf for shape-guided generation of 3d shapes and textures

    Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12663–12673, 2023

    2023

  52. [59]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. Interna- tional Conference on Learning Representations, 2022. 2, 15

    2022

  53. [60]

    Dreamgaussian: Generative gaussian splatting for efficient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. International Conference on Learning Representations, 2023

    2023

  54. [61]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36, 2024. 3, 15

    2024

  55. [62]

    MagicDrive: Street view generation with diverse 3d geometry control

    Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhen- guo Li, Dit-Yan Yeung, and Qiang Xu. MagicDrive: Street view generation with diverse 3d geometry control. In Inter- national Conference on Learning Representations, 2024. 2

    2024

  56. [63]

    Set-the-scene: Global-local training for generating controllable nerf scenes

    Dana Cohen-Bar, Elad Richardson, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. Set-the-scene: Global-local training for generating controllable nerf scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2920–2929, 2023. 3

    2023

  57. [64]

    Componerf: Text-guided multi-object compositional nerf with editable 3d scene lay- out

    Haotian Bai, Yuanhuiyi Lyu, Lutao Jiang, Sijia Li, Haonan Lu, Xiaodong Lin, and Lin Wang. Componerf: Text-guided multi-object compositional nerf with editable 3d scene lay- out. arXiv preprint arXiv:2303.13843, 2023. 3

    2023 arXiv

  58. [65]

    Text2room: Extracting textured 3d meshes from 2d text-to-image models

    Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7909–7920, 2023. 3

    2023

  59. [66]

    Neural sparse voxel fields

    Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. Advances in Neural Information Processing Systems, 2020. 4

    2020

  60. [67]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 4

    2022

  61. [68]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 4, 13

    2021 arXiv

  62. [69]

    Geonet: Geometric neural network for joint depth and surface normal estimation

    Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, and Jiaya Jia. Geonet: Geometric neural network for joint depth and surface normal estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 283–291, 2018. 5

    2018

  63. [70]

    Nerf-ds: Neural ra- diance fields for dynamic specular objects

    Zhiwen Yan, Chen Li, and Gim Hee Lee. Nerf-ds: Neural ra- diance fields for dynamic specular objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8285–8295, 2023

    2023

  64. [72]

    Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans

    Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10786–10796, 2021. 5

    2021

  65. [73]

    3d common corruptions and data augmentation

    O ˘guzhan Fatih Kar, Teresa Yeo, Andrei Atanov, and Amir Zamir. 3d common corruptions and data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 18963–18974, 2022. 5

    2022

  66. [74]

    Pytorch: An im- perative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An im- perative style, high-performance deep learning library. Ad- vances in neural information processing systems ...

    2019

  67. [75]

    Diffusers: State-of-the-art diffu- sion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffu- sion models. https://github.com/huggingface/ diffusers...

    2022

  68. [76]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2017. 5

    2017

  69. [77]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 5, 7

    2017

  70. [78]

    Demystifying mmd gans

    Mikołaj Bi ´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. arXiv preprint arXiv:1801.01401, 2018. 5, 7

    2018 arXiv

  71. [79]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 8

    2018

  72. [80]

    Monoscene: Monoc- ular 3d semantic scene completion

    Anh-Quan Cao and Raoul de Charette. Monoscene: Monoc- ular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 13

    2022

  73. [81]

    V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion

    Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anand- kumar. V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2023

  74. [82]

    Sym- phonize 3d semantic scene completion with contextual in- stance queries

    Haoyi Jiang, Tianheng Cheng, Naiyu Gao, Haoyang Zhang, Tianwei Lin, Wenyu Liu, and Xinggang Wang. Sym- phonize 3d semantic scene completion with contextual in- stance queries. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 13 Ur...

    2024

  75. [83]

    UrbanCraft2D

    UrbanCraft: Methodology 3 3.1. UrbanCraft2D . . . . . . . . . . . . . . . . 3 3.2. Distillation-Guided Urban Scene Optimization 4 3.3. Urban 3D Representation Initialization . . . 5 3.4. Optimization . . . . . . . . . . . . . . . . . 5

  76. [84]

    Experimental Setup

    Experiments 5 4.1. Experimental Setup . . . . . . . . . . . . . . 5 4.2. Qualitative Evaluation . . . . . . . . . . . . 5 4.3. Quantitative evaluation . . . . . . . . . . . . 8 4.4. Ablation Study . . . . . . . . . . . . . . . . 8

  77. [85]

    Implementation Details. 13 6.1. UrbanCraft2D Pretraining Process . . . . . . 13 6.2. Occupancy Grid Maps . . . . . . . . . . . . 13 6.3. Hyper-parameters Setting . . . . . . . . . . 14 6.4. Training Time . . . . . . . . . . . . . . . . 14 6.5. Loss Explanation . . . . . . . . ...

  78. [86]

    Geometry Consistency

    Ablation Study 14 7.1. Geometry Consistency . . . . . . . . . . . . 14 7.2. HSG-SDS . . . . . . . . . . . . . . . . . . . 15

  79. [87]

    Challenge Task: Barely-visible Vehicle

    Discussion 15 8.1. Challenge Task: Barely-visible Vehicle . . . 15 8.2. Failure Cases Analysis and Limitation . . . . 15 8.3. Compare with 3D Generation Baselines . . . 17

  80. [88]

    Public Resources Used 19 10.1

    UrbanCraft on NuScenes Dataset 18 10 . Public Resources Used 19 10.1 . Public Datasets Used . . . . . . . . . . . . . 19 10.2 . Public Implementations Used . . . . . . . . 19

  81. [89]

    Implementation Details. 6.1. UrbanCraft2D Pretraining Process We pretrained the proposed UrbanCraft2D using both the NuScenes [31] and KITTI-360 [23] datasets. Experimental results demonstrate that UrbanCraft2D trained on NuScenes performs better than when trained on KITTI-360...

  82. [90]

    Visualization of occupancy grid maps

    [4] [5] Figure 10. Visualization of occupancy grid maps. ban reconstruction applications. 6.3. Hyper-parameters Setting The training process of UrbanCraft consists of two stages: i) Initial Reconstruction : In this stage, we focus on the coarse reconstruction of urban 3D repre...

  83. [91]

    Geometry Consistency In Figure 11, we illustrate the visualization results of depth maps and RGB images

    Ablation Study 7.1. Geometry Consistency In Figure 11, we illustrate the visualization results of depth maps and RGB images. It is evident that our method signif- icantly outperforms existing approaches in reconstructing geometric structures and preserving texture details. In ...

  84. [92]

    Challenge Task: Barely-visible V ehicle Figure 15 showcases our method’s effectiveness in handling barely visible objects, particularly for vehicle restoration

    Discussion 8.1. Challenge Task: Barely-visible V ehicle Figure 15 showcases our method’s effectiveness in handling barely visible objects, particularly for vehicle restoration. Unlike VEGS, which fails to reconstruct the car structure, our approach accurately preserves the veh...

  85. [93]

    To further validate the effectiveness of the proposed UrbanCraft, we developed a dedicated DataLoader for the nuScenes [31] dataset and ap- plied it to all baseline methods

    UrbanCraft on NuScenes Dataset In the main manuscript, to ensure a fair comparison, we uti- lize the same KITTI-360 [23] dataset as VEGS [26] for both qualitative and quantitative analysis. To further validate the effectiveness of the proposed UrbanCraft, we developed a dedica...

  86. [94]

    Public Resources Used In this section, we acknowledge the public resources used, during the course of this work. 10.1. Public Datasets Used • KITTI-360 1 . . . . . . . . . . . . . . . . . . . . . . CC BY-NC-SA 3.0 • NuScenes 2 . . . . . . . . . . . . . . . . . . . . . . . Apac...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.