Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FreeSim synthesizes high-quality camera images from viewpoints more than three meters off the recorded driving trajectory by combining generative enhancement with progressive reconstruction.

desk verdict A clever generation-reconstruction pipeline for off-trajectory driving views, but the headline FID numbers are confounded because the generative post-enhancer is applied only to FreeSim. read the letter →

arxiv 2412.03566 v1 pith:74NL3NTY submitted 2024-12-04 cs.CV

Lue Fan , Hao Zhang , Qitai Wang , Hongsheng Li , Zhaoxiang Zhang This is my paper · ORCID
classification cs.CV
keywords free-viewpointsynthesiscamerasimulationautonomousdriving3DGaussianSplattingdiffusionenhancementprogressivereconstructionoff-trajectoryviewsWaymoOpenDataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FreeSim is a camera-simulation method for autonomous driving that targets viewpoints off the recorded ego trajectory, where existing reconstruction-based methods degrade because no training images exist. It reframes the problem: instead of generating an image directly from a new camera pose, a diffusion-based enhancement model restores a degraded rendering of that viewpoint. Because real off-trajectory images do not exist, the model is trained on synthetically degraded renderings made by extrapolating from piece-wise Gaussian reconstructions and by perturbing Gaussian primitives. A progressive reconstruction loop then repeatedly adds generated off-trajectory images to the training set, moving outward in small lateral steps, which lets the simulation reach deviations above three meters while keeping FID scores far below baselines.

What carries the argument

The central machinery is the pair construction that turns pose-conditioned novel-view generation into image enhancement: degraded renderings are produced by extrapolated rendering of piece-wise Gaussian fields (holding out the last frames of 20-frame sub-segments) plus Gaussian primitive translation and rotation perturbation, paired with recorded ground-truth images. A Stable Diffusion 1.5 U-Net with two ControlNets, one for the degraded image and one for the sparse LiDAR projection, is trained to restore these degraded renderings. A progressive trajectory-shifting schedule then feeds generated images back into PVG reconstruction, with a final post-enhancement pass to mitigate rolling-shutter distortion and generative randomness.

What would settle it

Record a scene with two laterally separated passes, reconstruct with FreeSim using only one pass, then render the second pass as if it were off-trajectory; if the enhancement model performs markedly worse on these true off-trajectory renderings than on its synthetic training degradations, the data-construction mimicry fails and the progressive loop is being guided by the wrong pseudo-ground-truth.

Watch

Extended reading notes

Core claim

The paper claims that high-quality off-trajectory view synthesis in driving scenes is achievable without any ground-truth off-trajectory images, by combining a generative enhancement model with a progressive reconstruction schedule. The enhancement model learns to map a slightly degraded rendering of an unrecorded viewpoint to a clean image, and the progressive schedule ensures the renderings it is asked to enhance are never catastrophically degraded: generated images from nearby off-trajectory views are folded back into the Gaussian reconstruction, extending the reachable trajectory step by step. On Waymo scenes this yields FID values of 14.6, 17.0, and 18.6 for lateral shifts of 1, 2, and 3 meters, compared with 22.9, 34.3, and 47.5 for the PVG reconstruction baseline.

Load-bearing premise

The method assumes that the degradation patterns created by extrapolated rendering and by moving Gaussian primitives look like the degradation that a truly off-trajectory viewpoint would produce, and the paper offers no quantitative comparison between these synthetic degradations and real off-trajectory renderings.

Editorial extensions

If this is right

  • Simulators can generate photorealistic images from lane-change-like trajectories up to at least three meters from the recorded path, closing a capability gap in closed-loop testing.
  • The trained enhancement model generalizes to other Gaussian-based reconstruction methods besides the PVG renderings it was trained on, such as StreetGS.
  • Small lateral steps are important: a single 3-meter step raises FID at 3 meters from 18.6 to 29.7, while step sizes around 0.5 to 1.0 meters keep quality high.
  • The sparse LiDAR projection condition matters most at large deviations, with FID at 3 meters rising from 18.6 to 21.3 when it is removed.
  • Ground-truth off-trajectory images are not needed for training the enhancement model; only recorded trajectory data and synthetic degradation are required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If synthetic degradation is a faithful proxy, the same enhancement-plus-progressive-reconstruction recipe should extend to other extrapolation axes such as vertical shifts or rotations, though the paper demonstrates lateral shifts only.
  • A direct test of the mimicry assumption would be to collect a small set of true off-trajectory images from a second pass or chase vehicle and compare the enhancement model's output on true versus synthetic degradation; the paper does not report such a comparison.
  • FID improvement alone does not guarantee multi-view consistency across newly generated viewpoints; a stress test would render a traversal of adjacent off-trajectory views and check for flicker or geometric drift, which matters for closed-loop simulation.
  • The approach is likely portable to other single-trajectory capture domains, such as aerial or warehouse robotics, where off-trajectory ground truth is equally unavailable.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FreeSim, a hybrid generation-reconstruction method for camera simulation in driving scenes at viewpoints beyond the recorded ego trajectory. The method first trains a generative enhancement model (Stable Diffusion with two ControlNets) on a large synthetic dataset of degraded-rendering/ground-truth pairs created by piecewise Gaussian reconstruction, extrapolated rendering, and Gaussian primitive perturbation. It then alternately generates images at progressively shifted lateral viewpoints and adds them to the reconstruction training set, followed by a post-enhancement pass. The main claims are that FreeSim achieves high-quality off-trajectory synthesis at lateral deviations up to 3 meters, with quantitative results reported as FID scores (14.6/17.0/18.6 at 1m/2m/3m vs. PVG's 22.9/34.3/47.5), and that the progressive strategy is necessary for large deviations.

Significance. If the central claim were established, FreeSim would be a meaningful advance for closed-loop sensor simulation in autonomous driving, where off-trajectory rendering is important and prior reconstruction-based methods degrade quickly. The paper contributes a large-scale data-construction pipeline for training an image-enhancement diffusion model on driving scenes, and a progressive viewpoint-expansion strategy that is intuitively well-motivated. The authors also provide extensive ablations of their design choices and acknowledge several limitations. However, the current quantitative evidence is confounded by the asymmetric application of the generative post-enhancer, and the central assumption that synthetically degraded renderings mimic true off-trajectory degradation is not verified; these issues must be addressed before the reported FID improvements can be attributed to the proposed method.

major comments (3)
  1. [Sec. 3.3, Table 1] The reported FID comparison does not isolate the proposed progressive generation-reconstruction pipeline from the generative post-enhancer. FreeSim's results in Table 1 are produced after applying the enhancement model as post-processing, as described in Sec. 3.3 ("Post-enhancement to mitigate rolling shutter distortion and generative randomness"), whereas the baseline methods are evaluated directly on their Gaussian-field renderings. Since the enhancement model maps degraded renderings to the distribution of recorded ground-truth images, it can artificially lower FID even when the underlying reconstructed geometry is poor. The paper does not report (a) FID of FreeSim without post-enhancement, nor (b) FID of a baseline with the same post-enhancement applied. This control is essential to support the claim that the progressive strategy, rather than the off-the-shelf diffusion post-processor, is responsible for the improvement.
  2. [Sec. 3.1.1] The data-construction strategy rests on the assumption that extrapolated rendering on held-out frames and Gaussian primitive perturbation produce degradation patterns that are representative of true off-trajectory renderings ("The rendering of such extrapolated views can simulate the degraded rendering patterns similar to off-trajectory renderings"). No quantitative evidence is provided that the simulated degradation distribution matches real off-trajectory degradation; the paper only shows qualitative ghosting examples. If this mimicry fails, the enhancement model will not transfer to actual large deviations, and the progressive reconstruction loop would be guided by incorrect pseudo-ground-truth. The authors should validate this assumption by comparing degradation statistics (e.g., per-pixel error, edge/boundary distortion, frequency spectra) between simulated degraded renderings and renderings from genuinely off-trajectory viewpoints, or by evaluating the final method on scenes with multi-pass or adjacent-lane data if such data can be obtained.
  3. [Sec. 4.2] The evaluation of off-trajectory quality relies solely on FID computed between rendered images and recorded-view ground-truth images. FID is a distributional metric and can be low even if individual synthesized views are geometrically incorrect, as long as the set matches the texture statistics of the training distribution. The claim of "high-quality" off-trajectory synthesis under 3-meter deviations is therefore not directly backed by a metric that measures geometric or viewpoint fidelity. The paper does not report per-scene FID variance, nor any geometry-aware or pixel-aligned measure (e.g., depth-map consistency, warped-view PSNR, or LPIPS against a pseudo-GT). Given the absence of off-trajectory ground truth, the authors should either report additional consistency metrics that probe geometric correctness, or temper the claim to be explicitly about distribution-level realism.
minor comments (5)
  1. [Sec. 3.1.1] The term "Piece-wise Gaussian Reconstruction" should be "piecewise Gaussian reconstruction" for consistency with standard terminology; the hyphenated form appears throughout.
  2. [Table 2] The row "Blending p = 0.5" is ambiguous because the probability of enabling image blending is already p=0.1 in the default setting, but the table does not state whether p=0.5 is the probability or the fixed blending weight alpha. Clarify the notation.
  3. [Sec. 4.1] The text says "we adopt a 1k-iteration schedule" for piecewise reconstruction and later "we further conduct the optimization for another 30k steps" for the progressive reconstruction; the relationship between the 1k-iteration schedule and the 30k-step optimization should be clarified.
  4. [Fig. 5] The caption lists "S3GaussianStreetGS" without a space or slash; this appears to be a formatting error rather than a method name.
  5. [Eq. (1) and Sec. 3.2] The notation I_d, I_l, I* is introduced but the subscript "l" for LiDAR is not defined before first use; state explicitly that I_l is the LiDAR projection condition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the FID-based evaluation and the post-enhancement confound are validity limitations, not by-construction reductions.

full rationale

FreeSim's method chain is not circular in the sense defined here. The enhancement model is trained on degraded renderings synthesized from piece-wise Gaussian fields (extrapolated held-out frames plus Gaussian primitive perturbation), paired with recorded ground-truth images, and is then applied to renderings of laterally shifted viewpoints. This is a stated modeling assumption (Sec. 3.1.1: extrapolated renderings 'can simulate the degraded rendering patterns similar to off-trajectory renderings'), not a derivation that takes its conclusion as an input. The progressive reconstruction loop adds generated images as pseudo-ground-truth to refit the Gaussian field; the paper provides an internal control showing non-progressive reconstruction is worse (FID 20.1/26.3/29.7 vs 14.6/17.0/18.6 in Table 2), which gives independent content to the progressive-strategy claim. The self-citations are not load-bearing: FreeVS [27] is cited for an optional LiDAR condition that is ablated, and the FID protocol is standard in prior off-trajectory evaluation. The main concerns raised by the skeptic are evaluation-validity issues rather than circular reductions: the reported FID compares outputs to the recorded-image distribution that the enhancement model was trained to match, and post-enhancement is applied to FreeSim but not to the baselines in Table 1. These are real threats to the interpretability of the headline numbers, and the paper acknowledges the absence of off-trajectory ground truth, but no specific equation or fitted parameter is shown to make the claimed result equivalent to its own inputs by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method introduces no new theoretical quantities or entities. The central claim depends on several untested modeling assumptions, notably the fidelity of synthetic degradations and the convergence of the pseudo-label loop, plus a collection of hand-chosen hyperparameters that control the pipeline.

free parameters (6)
  • Viewpoint expansion step size = 0.5 m
    Chosen by hand; ablation shows 0.5m gives best FID trade-off (Sec. 4.3, Tab. 2).
  • Expansion interval = 5k iterations
    Set to 5k; not ablated.
  • Total optimization steps for reconstruction = 30k
    Default from PVG with extensions; not ablated.
  • Image blending parameters = alpha=0.5, p=0.1
    p=0.1 chosen via ablation; alpha fixed.
  • Gaussian perturbation magnitude = up to 50% primitives, 0.2m translation, 15 deg rotation
    Chosen to create diverse degradations; no sensitivity analysis.
  • Piece-wise sub-segment length = 20 frames, hold out last 4
    Chosen for efficiency; no sensitivity analysis.
assumptions (4)
  • domain assumption Extrapolated rendering and Gaussian perturbation produce degradation patterns statistically similar to true off-trajectory renderings.
    Sec. 3.1.1 motivates this but provides no quantitative gap analysis; the enhancement model's transfer depends on it.
  • domain assumption Progressive addition of generated pseudo-ground-truth views converges to a scene representation that does not drift over iterations.
    Sec. 3.3 relies on this; no convergence guarantee or error analysis is given.
  • domain assumption FID measured against recorded views is an adequate proxy for off-trajectory rendering quality.
    Sec. 4.2 adopts this from prior work; it is a distributional metric and not a per-view correctness check.
  • domain assumption PVG is a sufficiently accurate reconstruction backbone for creating training data that generalizes across methods.
    Sec. 3.1.1 uses PVG for all 1.5M pairs; generalizability to other methods is only qualitatively shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes." pith.science (2026). https://pith.science/paper/74NL3NTY

@misc{pith2026241203566,
  author       = {Pith},
  title        = {Pith review of: FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/74NL3NTY}},
  note         = {Machine review of arXiv:2412.03566}
}
read the original abstract

We propose FreeSim, a camera simulation method for autonomous driving. FreeSim emphasizes high-quality rendering from viewpoints beyond the recorded ego trajectories. In such viewpoints, previous methods have unacceptable degradation because the training data of these viewpoints is unavailable. To address such data scarcity, we first propose a generative enhancement model with a matched data construction strategy. The resulting model can generate high-quality images in a viewpoint slightly deviated from the recorded trajectories, conditioned on the degraded rendering of this viewpoint. We then propose a progressive reconstruction strategy, which progressively adds generated images of unrecorded views into the reconstruction process, starting from slightly off-trajectory viewpoints and moving progressively farther away. With this progressive generation-reconstruction pipeline, FreeSim supports high-quality off-trajectory view synthesis under large deviations of more than 3 meters.

Figures

Figures reproduced from arXiv: 2412.03566 by the authors.

Figure 1
Figure 1. The proposed FreeSim can obtain high-quality camera simulation results in viewpoints largely deviated from the recorded [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall framework of FreeSim. The black line indicates the recorded trajectory. Starting from it, the training viewpoints are progressively expanded to far away unrecorded trajectories (blue and red lines). The generative model produces high-quality images for the new viewpoints after each expansion. The progressive expansion can be conducted repeatedly, and we only illustrate two stages for simplicity. We use r… view at source ↗
Figure 4
Figure 4. This pattern is caused by those Gaussian primitives with inaccurate depth, which is rasterized into wrong im￾age positions after the viewpoint deviations. To simulate the ghosting, we randomly sample a small portion of Gaus￾sian primitives in a scene and move them the same random distance along the horizontal direction (i.e., width direction in image space). Thus, the perturbed Gaussian primitives produce ghosting a… view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison. The two scenes are captured by the front-left camera and the front camera, respectively. To showcase our performance, we adopt a quite large viewpoint change. However, those views with orange bounding boxes get completely ruined with such a larg…
Figure 6
Figure 6. Figure 6: Demonstration of different step sizes in progressive reconstruction. All images are enhanced by the proposed post￾enhancement. Empirically, step sizes less than 1 meter usually produce good results. Method Recorded views Unrecorded views (FID ↓) PSNR ↑ SSIM ↑ @1m @2m @…
Figure 7
Figure 7. Figure 7: LiDAR condition improves robustness in extreme viewpoints. Here the deviated viewpoint is about to collide with the stairs. In such an extreme view, LiDAR input corrects the color of lines, eliminates the artifact, and makes the stairs more clear. Ways to make degraded…
Figure 10
Figure 10. Figure 10: A typical failure case. During the progressive recon￾struction, the window frame converges to a different shape, caused by a severe “ghosting” issue ( [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 8
Figure 8. Figure 8: Effectiveness of post enhancement. Post enhancement enhances the detailed texture such as tree and grass and objects near the camera such as the car. Generalization of generative model. Although our train￾ing data is solely created by PVG [4], we find it can be general…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ExtraGS: Geometric-Aware Trajectory Extrapolation with Uncertainty-Guided Generative Priors

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    ExtraGS combines Gaussian-SDF road surfaces, far-field Gaussians, and spherical-harmonics uncertainty gating to generate geometrically consistent extrapolated driving views.

  2. Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model

    cs.RO 2024-12 conditional novelty 5.0 of 10

    A reactive closed-loop driving simulator that uses a diffusion renderer with retrieval from real recordings, plus a nuPlan behavioral controller, to generate sensor images in response to an end-to-end driving model's actions.

  3. Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happen

    cs.LG 2025-05 conditional novelty 4.0 of 10

    Crash prediction should learn from near-miss events and synthetic counterfactual scenarios, not just recorded crashes.

Reference graph

Works this paper leans on

44 extracted references · 10 canonical work pages · cited by 3 Pith papers

  1. [1]

    Imagen 3

    Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brich- tova, Andrew Bunner, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuqing Du, Zach Eaton-Rosen, et al. Imagen 3. arXiv preprint arXiv:2408.07009, 2024. 2

    arXiv 2024

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  3. [3]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  4. [4]

    Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering

    Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv:2311.18561,

    arXiv

  5. [5]

    Omnire: Omni ur- ban scene reconstruction

    Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Go- jcic, Sanja Fidler, Marco Pavone, et al. Omnire: Omni ur- ban scene reconstruction. arXiv preprint arXiv:2408.16760,

    arXiv

  6. [6]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13142–13153, 2023. 3

    2023

  7. [7]

    Cat3d: Create anything in 3d with multi-view diffusion models

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405.10314, 2024. 2, 3

    arXiv 2024

  8. [8]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 3

    2020

Show all 44 references
  1. [9]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. 2022. 2

    2022

  2. [10]

    S3gaussian: Self-supervised street gaussians for autonomous driving

    Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. S3gaussian: Self-supervised street gaussians for autonomous driving. arXiv preprint arXiv:2405.20323, 2024. 1, 2, 7

    2024 arXiv

  3. [11]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 1, 2

    2023

  4. [12]

    Autosplat: Constrained gaussian splatting for autonomous driving scene reconstruction

    Mustafa Khan, Hamidreza Fazlali, Dhruv Sharma, Tongtong Cao, Dongfeng Bai, Yuan Ren, and Bingbing Liu. Autosplat: Constrained gaussian splatting for autonomous driving scene reconstruction. arXiv preprint arXiv:2407.02598, 2024. 3

    2024 arXiv

  5. [13]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 5

    2013 arXiv

  6. [14]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vi- sion

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Anirud- dha Mukherjee, Hao Kang, Xiangrui Kong, Gang Hua, Tianyi Zhang, Bedrich Benes, and Aniket Bera. Dl3dv-10k: A large-scale s...

    2024

  7. [15]

    Re- conx: Reconstruct any scene from sparse views with video diffusion model

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Re- conx: Reconstruct any scene from sparse views with video diffusion model. arXiv preprint arXiv:2408.16767, 2024. 2

    2024 arXiv

  8. [16]

    Real-time neural rasterization for large scenes

    Jeffrey Yunfan Liu, Yun Chen, Ze Yang, Jingkang Wang, Sivabalan Manivasagam, and Raquel Urtasun. Real-time neural rasterization for large scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8416–8427, 2023. 2

    2023

  9. [17]

    3dgs- enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors

    Xi Liu, Chaoyi Zhou, and Siyu Huang. 3dgs- enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors. arXiv preprint arXiv:2410.16266, 2024. 2, 3

    2024 arXiv

  10. [18]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023. 1

    2023 arXiv

  11. [19]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 2

    2021

  12. [20]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  13. [21]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  14. [22]

    Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vi...

    2021

  15. [23]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 5

    2022

  16. [24]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Et- tinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhan...

    2020

  17. [25]

    Neurad: Neural rendering for autonomous driving

    Adam Tonderski, Carl Lindstr ¨om, Georg Hess, William Ljungbergh, Lennart Svensson, and Christoffer Petersson. Neurad: Neural rendering for autonomous driving. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14895–14904, 2024. 2

    2024

  18. [26]

    Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer...

    2025

  19. [27]

    Freevs: Generative view synthesis on free driv- ing trajectory

    Qitai Wang, Lue Fan, Yuqi Wang, Yuntao Chen, and Zhaox- iang Zhang. Freevs: Generative view synthesis on free driv- ing trajectory. arXiv preprint arXiv:2410.18079, 2024. 2, 4, 6, 7

    2024 arXiv

  20. [28]

    Drivedreamer: Towards real-world- driven world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jia- gang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world- driven world models for autonomous driving. arXiv preprint arXiv:2309.09777, 2023. 7

    2023 arXiv

  21. [29]

    Driving into the future: Multiview visual forecasting and planning with world model for au- tonomous driving

    Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for au- tonomous driving. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pa...

    2024

  22. [30]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320,...

    2024

  23. [31]

    Mars: An instance-aware, mod- ular and realistic simulator for autonomous driving

    Zirui Wu, Tianyu Liu, Liyi Luo, Zhide Zhong, Jianteng Chen, Hongmin Xiao, Chao Hou, Haozhe Lou, Yuantao Chen, Runyi Yang, et al. Mars: An instance-aware, mod- ular and realistic simulator for autonomous driving. In CAAI International Conference on Artificial Intelligence, pages 3–

  24. [32]

    Street gaussians for modeling dynamic ur- ban scenes

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic ur- ban scenes. arXiv preprint arXiv:2401.01339, 2024. 1, 2, 7, 8

    2024 arXiv

  25. [33]

    Emernerf: Emergent spatial- temporal scene decomposition via self-supervision

    Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Se- ung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. Emernerf: Emergent spatial- temporal scene decomposition via self-supervision. arXiv preprint arXiv:2311.02077, 2023. 2, 5, 6, 7

    2023 arXiv

  26. [34]

    Unisim: A neural closed-loop sensor simulator

    Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Mani- vasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Ur- tasun. Unisim: A neural closed-loop sensor simulator. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 1389–1399, 2023. 2, 3, 6

    2023

  27. [35]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024. 1, 2

    2024

  28. [36]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024. 3

    2024 arXiv

  29. [37]

    Mvimgnet: A large-scale dataset of multi-view images

    Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognit...

    2023

  30. [38]

    Sgd: Street view synthesis with gaussian splatting and diffusion prior

    Zhongrui Yu, Haoran Wang, Jinze Yang, Hanzhang Wang, Zeke Xie, Yunfeng Cai, Jiale Cao, Zhong Ji, and Mingming Sun. Sgd: Street view synthesis with gaussian splatting and diffusion prior. arXiv preprint arXiv:2403.20079, 2024. 2, 6

    2024 arXiv

  31. [39]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 5

    2023

  32. [40]

    Drivedreamer4d: World models are effective data machines for 4d driving scene rep- resentation

    Guosheng Zhao, Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Boyuan Wang, Youyi Zhang, Wenjun Mei, and Xingang Wang. Drivedreamer4d: World models are effective data machines for 4d driving scene rep- resentation. arXiv preprint arXiv:2410.13571, 2024. 2, 7

    arXiv

  33. [41]

    Hugs: Holistic urban 3d scene understanding via gaus- sian splatting

    Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaus- sian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages ...

    2024

  34. [42]

    Stereo magnification: Learning view synthesis using multiplane images

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817, 2018. 3

    2018 arXiv

  35. [43]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2024

  36. [44]

    Sparsefusion: Dis- tilling view-conditioned diffusion for 3d reconstruction

    Zhizhuo Zhou and Shubham Tulsiani. Sparsefusion: Dis- tilling view-conditioned diffusion for 3d reconstruction. In CVPR, 2023. 2

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.