{"id":"6bd59e1c-d989-4324-bb82-2fba776e35ce","arxiv_id":"2412.00730","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"SEED4D contributes a CARLA-based data generator and two large synthetic ego-exo driving datasets for 3D and 4D reconstruction benchmarks.","lead":"SEED4D is an open-source generator and dataset that creates synthetic driving scenes with both vehicle-mounted (ego) and outside (exo) camera views, including moving traffic. It provides 212,000 static images and 16.8 million dynamic images to train and test 3D and 4D reconstruction models for autonomous driving.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 4's few-image benchmark is not a valid comparison: per-scene optimizers are tested on a 5-scene subset, feed-forward methods on Town02, and depth baselines are ranked on unmasked metrics despite masked variants; the benchmark cannot support its claimed rankings.","rationale":"The reader's weakest assumption was sim-to-real transfer, which the authors themselves flag. I find that concern real but not the most load-bearing: a synthetic dataset can be useful even before sim-to-real transfer is demonstrated, and the paper's limitations section already concedes photorealism and dynamics limits. The more decisive problem is internal to the benchmark contribution. Table 4 is presented as evidence that SEED4D supports few-image-to-3D reconstruction, but the comparison mixes two incompatible evaluation regimes (per-scene optimization on five scenes vs. cross-scene generalization on a 100-scene split) and then further mixes masked and unmasked metrics only for depth baselines. This is not a matter of outside consensus; it is an inconsistency within the paper's own stated protocol. The masked depth baseline numbers show the effect is large (PSNR roughly triples), so the ranking is not robust. A matched re-evaluation with error bars would settle it. If the re-evaluation preserves the ranking, the benchmark can stand; if not, the paper should be conditional on reworking Table 4. Because the reader already issued CONDITIONAL and called for tightened benchmarking, my read does not change the verdict, so I mark UNCHANGED while making the specific test explicit.","tokens_in":23284,"tokens_out":10055,"duration_ms":95661,"concrete_test":"Reproduce Table 4 under a matched protocol: (1) evaluate K-Planes, NeRFacto, and SplatFacto on all 100 Town02 test scenes, or on a random 20-scene sample with per-scene mean plus/minus standard deviation, rather than an unspecified five scenes; (2) evaluate PixelNeRF, SplatterImage, and 6Img-to-3D both on the full Town02 split and on the exact five scenes used for the per-scene methods; (3) compute masked and unmasked PSNR, SSIM, LPIPS, and DRMSE for every method using one common evaluation mask, and report rankings under both conventions. If the ordering of 6Img-to-3D relative to K-Planes changes, or if per-scene variance exceeds the reported inter-method gaps, the benchmark ranking is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central resource claim is credible, but the benchmark half of the central claim is undercut by internal protocol flaws in Table 4. First, K-Planes, NeRFacto, and SplatFacto are per-scene optimizations run on an unspecified five-scene subset, while PixelNeRF, SplatterImage, and 6Img-to-3D are trained on the training towns and evaluated on the Town02 test split. The two groups are evaluated on different scenes under different data regimes, and no per-scene error bars or standard deviations are given, so the ranking in Table 4 cannot be attributed to method quality. Second, ZoeDepth and Metric3D are rendered by projecting depth into colored point clouds and rasterizing exo views, which leaves holes; Table 4 reports masked variants (ZoeDepth‡, Metric3D‡) but the caption states that only unmasked values are used for ranking. That is internally inconsistent: depth baselines are penalized for missing pixels while no mask is applied to the other methods, and the masked versions of the depth baselines improve PSNR from 5.466 to 14.202 and from 6.314 to 13.699, respectively. Third, the DRMSE column mixes depth-estimation error for depth baselines with reconstruction-derived depth error for 6Img-to-3D, so cross-method DRMSE comparisons are not meaningful. These issues do not invalidate the dataset or generator, but they do invalidate the benchmark as a source of evidence for the paper's stated few-image-to-3D contribution until the protocol is corrected.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SEED4D, a CARLA-based data generator for synthetic ego-exo dynamic 4D driving data, together with two released datasets: a static 3D dataset of 2,002 scenes with about 212k images, and a dynamic 4D dataset of roughly 10.5k trajectories with about 16.8M images. Each image is accompanied by camera poses, depth, semantic and instance segmentation, optical flow, and LiDAR, and the data generator supports camera setups mimicking NuScenes, KITTI360, and Waymo. The authors also benchmark existing methods for multi-view novel view synthesis, monocular metric depth estimation, and single-shot few-image-to-3D reconstruction on the static dataset. The central claim is that SEED4D fills a gap: no existing autonomous-driving dataset provides large-scale multi-view ego-exo data suitable for supervising few-image-to-3D and 4D prediction models.","tokens_in":23616,"tokens_out":9644,"duration_ms":80693,"significance":"If the generator and datasets are released as described, they constitute a useful community resource: they supply the missing non-ego supervision views for driving scenes at a scale unmatched by existing ego-exo autonomous-driving data, with dense ground-truth annotations and NeRFStudio-compatible poses. Concrete strengths include the open-source generator, the reproducible generation pipeline, the explicit train/test town split, and an honest Limitations section acknowledging that CARLA output is not photorealistic and that vehicle dynamics are limited. The benchmark half of the paper, however, is not yet a reliable source of evidence: as detailed below, Table 4 compares methods under incompatible protocols, the masking treatment of depth baselines is internally inconsistent, and no variance information is reported anywhere. Because benchmarks are a stated contribution, the manuscript needs revision before the few-image-to-3D benchmark claim is supported.","major_comments":[{"comment":"The few-image-to-3D benchmark compares methods under incompatible protocols. The per-scene optimization methods (K-Planes, NeRFacto, SplatFacto) are evaluated on an unspecified subset of five scenes, while the feed-forward methods (PixelNeRF, SplatterImage, 6Img-to-3D) are trained on the training towns and evaluated on the Town02 test split. The resulting ranking cannot be attributed to method quality because the two groups are tested on different scenes under different data regimes. Please specify the evaluation scenes (which towns, how selected, which input views per scene) and either run all methods on an identical test set with identical inputs or restructure the table so that the two evaluations are clearly separated.","section":"§4, Table 4"},{"comment":"The depth baseline masking is internally inconsistent. The masked variants ZoeDepth‡ and Metric3D‡ improve PSNR from 5.466 to 14.202 and from 6.314 to 13.699, respectively, yet the caption states that only the unmasked values are used for ranking. This places the depth baselines at a systematic disadvantage because the unmasked values penalize holes from point-cloud rasterization, while no masking protocol is applied to the other methods. Either apply the same masking protocol to all methods and rank on masked values, or remove the ‡ rows from the main comparison table and discuss them separately.","section":"§4, Table 4 caption"},{"comment":"The DRMSE column conflates two different quantities: for ZoeDepth and Metric3D it is monocular depth estimation error computed against ground-truth depth maps, while for 6Img-to-3D (and potentially pixelNeRF and SplatterImage) it is depth rendered from the reconstructed geometry. These measure different things, so the cross-method DRMSE comparison is not meaningful. The column should be recomputed under a unified protocol that defines the same ground-truth depth target and the same way of extracting depth from each method's output, or it should be removed.","section":"§4, Table 4, DRMSE column"},{"comment":"No variance information is reported for any benchmark result, and the number of evaluation scenes is undocumented for Table 2 while the five-scene subset used for K-Planes, NeRFacto, and SplatFacto in Table 4 is not described (which towns, how selected, whether the same scenes are used as in Table 2). For a benchmark paper, per-scene results or at least standard deviations and an explicit scene list are necessary for reproducibility and for judging whether the reported differences (e.g., K-Planes at 25.744 PSNR versus SplatFacto at 24.458 in Table 2) are significant.","section":"§4, Tables 2–4"}],"minor_comments":[{"comment":"The counting convention for the ego cameras should be stated explicitly: the dataset totals (212k static images, 16.8M dynamic images) correspond to 6 ego views plus the exo views per scene or timestep, i.e., excluding the additional 110-degree rear camera that is described as part of the 'six plus one' setup.","section":"§3.2, §7.1, Abstract"},{"comment":"The text lists 'SplatFacto-big' among the methods evaluated on five scenes, but no such row appears in Table 4; in addition, the SplatFacto row in Table 4 cites reference [112] while Table 2 cites [47] for the same method. Please align the text, the table, and the citations.","section":"§4, Table 4"},{"comment":"The yaw formula in Algorithm 2 (line 7) is written as yaw = sign(x) * arccos(y / (x^2 + y^2)^0.5), which is not the standard arccos(x / sqrt(x^2 + y^2)); please rewrite the formula unambiguously and check that the implementation matches the intended spherical Fibonacci orientation.","section":"§11.2, Algorithm 2"},{"comment":"There is a duplicated phrase: 'due to the CARLA Simulator [25], our data generator and the data generator provide reliable ground truth annotations'; the second occurrence of 'the data generator' should be removed.","section":"§1"},{"comment":"The sentence 'results are shown Table in 3' should read 'results are shown in Table 3'.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The benchmark problems identified in Section 4 are concentrated and fixable within the manuscript's scope; the dataset and generator contributions are the core value of the paper and appear credible from the text, though I have not verified the actual releases or hosting. I would hold the paper to a corrected benchmark protocol: a common evaluation set for all few-image-to-3D methods, explicit per-scene results or error bars, a unified masking treatment, and a consistent definition of DRMSE. One point to monitor: the authors' own method 6Img-to-3D tops Table 4; this is legitimate, but the paper should confirm that it received no advantage in input views or train/test composition relative to PixelNeRF and SplatterImage. The novelty claim relative to NeRDS360 is addressed in Section 2 and is acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the dataset and generator are the real contribution; the few-image benchmark in Table 4 has enough internal inconsistency that its rankings should not be cited as evidence. The resource itself looks solid and fills a clear gap.\n\nWhat is new: a CARLA-based generator that outputs NeRFStudio-format ego and exo views for driving scenes, plus two released datasets with LiDAR, depth, flow, and segmentation. The static set (2k scenes, 212k images) and dynamic set (10k trajectories, 16.8M images) are large for synthetic data, and the ego-exo combination is absent from prior AD datasets. The spherical Fibonacci placement is straightforward, and the camera setups matching NuScenes/KITTI360/Waymo are a practical touch. Credit where due: the authors are explicit about limitations—CARLA is not photorealistic and the vehicle dynamics are limited—and they correctly note that no suitable 4D prediction method exists to benchmark yet. The inclusion of their own 6Img-to-3D among baselines is disclosed and it is an evaluation, not a fitted result; that is not a circularity problem.\n\nSoft spots: the benchmark section is the weak part. Table 4 compares per-scene optimizers (K-Planes, NeRFacto, SplatFacto) evaluated on an unspecified five-scene subset with feed-forward methods (PixelNeRF, SplatterImage, 6Img-to-3D) trained on other towns and evaluated on Town02. Different scenes, different data regimes, no error bars. The caption states that only unmasked values are used for ranking, while ZoeDepth and Metric3D are penalized for rasterization holes; the masked variants improve PSNR from roughly 5.5 to 14.2 and 6.3 to 13.7, so the decision to rank on unmasked values is not a minor detail. Also, the DRMSE column mixes depth-estimation error with reconstruction-derived depth error, so cross-method DRMSE comparisons are not meaningful. These are fixable: pick a common test split, report per-scene numbers with variance, apply the same masking rule to all methods or none, and separate the depth-only baselines from the reconstruction methods. The multi-view NVS benchmark (Table 2) is cleaner, and the depth-only comparison (Table 3) is fine as a zero-shot sanity check.\n\nBottom line: the resource is worth having and the generator is likely to be used by others. The paper deserves a serious referee, but the benchmark needs revision before the few-image rankings can be taken at face value. I would cite the dataset; I would not cite Table 4 as it stands.","headline":"A genuinely useful synthetic ego-exo driving dataset and generator, saddled with a benchmark section whose rankings should not be trusted until the protocol is fixed.","tokens_in":24168,"tokens_out":2098,"would_cite":true,"duration_ms":18134,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SEED4D is a large-scale synthetic ego-exo dynamic 4D dataset and generator for autonomous driving, built so models can learn 3D and 4D scene reconstruction from both driver and outside cameras.","keywords":["synthetic data","ego-exo views","4D reconstruction","autonomous driving","novel view synthesis","few-image-to-3D","LiDAR","benchmark"],"falsifier":"Train a few-image-to-3D model on SEED4D and evaluate it zero-shot on real street images with known geometry; if its reconstruction and depth error is no better than a model trained on unrelated real images, the claim that this synthetic ego-exo data closes a real supervision gap is falsified.","tokens_in":23081,"feed_emoji":"🚗","tokens_out":9591,"duration_ms":80309,"temperature":0.7,"pith_summary":"The paper tries to establish that the missing ingredient for joint 3D and 4D perception in driving—a large collection of dynamic urban scenes captured at the same time from egocentric (in-vehicle) and exocentric (outside-vehicle) cameras—can be produced synthetically at scale. It presents an open, customizable generator built on an urban driving simulator, plus two datasets generated with it: 2,002 static scenes with 212k images and 10,458 dynamic trajectories with 16.8M images, every frame carrying ground-truth depth, flow, segmentation, and LiDAR. The paper then defines benchmarks for few-image-to-3D reconstruction (building a 3D scene from a handful of images), novel view synthesis, and monocular depth estimation, and leaves 4D prediction as an open challenge that the dynamic dataset is sized to support.","feed_headline":"SEED4D adds 16.8M ego-exo images for 4D driving research","feed_subtitle":"An open generator pairs driver and outside cameras with ground truth to train 3D and 4D scene models.","key_machinery":"The load-bearing object is the data generator: a customizable pipeline built on an open urban driving simulator that can place any number of cameras on any vehicle or anywhere in the scene and record them over time. Exocentric cameras are arranged on a half-sphere around each vehicle using a spherical Fibonacci lattice, a point distribution scheme that spaces the viewpoints evenly; every exo camera keeps a fixed relative pose to its vehicle. Camera poses are exported in a radiance-field-friendly format with intrinsics, extrinsics, and distortion, so the generated scenes drop directly into existing radiance-field tooling. Two datasets produced by this generator carry the argument: a static set built for few-image-to-3D and a dynamic set of 10-second trajectories built for temporal reconstruction and prediction.","core_discovery":"The paper's central claim is that no existing autonomous-driving dataset offers the combination of complex, dynamic, multi-view ego-exo data, and that SEED4D is the first large-scale answer. The paper demonstrates the claim by defining benchmark protocols that only such data make possible: outward-facing ego images serve as input, while 100 inward-facing exo images supervise novel view synthesis on the static scenes. Scores are reported for per-scene radiance-field methods, monocular metric depth estimators, and few-image-to-3D models; the benchmarks are demonstrations of the new evaluation regime rather than assertions about which method is best. The dynamic dataset extends the same idea through time, providing 10-second multi-view trajectories intended to push reconstruction methods toward actual 4D prediction.","pith_inferences":["The generator's camera-placement machinery is not tied to road scenes, so the same tool could produce ego-exo data for infrastructure cameras, pedestrian views, or off-road environments, an extension the paper leaves to future work.","A direct test of the resource's value would be pre-training a few-image-to-3D model on SEED4D and measuring its zero-shot transfer to real road images, with and without style transfer.","If 4D prediction matures, the dynamic dataset's fixed 10-second trajectories and dense ground truth make it a natural controlled setting for comparing appearance-based video predictors with explicit 3D reconstruction methods."],"forward_implications":["Few-image-to-3D models can be trained with ego views as input and exo views as supervision, a protocol no current real-world driving dataset supports.","The dynamic set provides a common 10-second multi-view testbed for 4D reconstruction and forecasting, a task the paper notes has no vision-based method ready to run on it yet.","Because the generator can reproduce the camera geometry of several established driving sensor suites, experiments on SEED4D can be configured to match familiar real-world hardware.","Every image comes with pixel-aligned ground truth, so depth, flow, and segmentation models can be evaluated on exactly the same scenes and viewpoints as reconstruction models."],"supporting_citations":[{"why":"Supplies the open urban driving simulator that generates all scenes, vehicles, sensors, and ground-truth annotations.","marker":"[25]"},{"why":"Provides the radiance-field pose format and the method implementations used in the novel-view-synthesis benchmarks.","marker":"[96]"},{"why":"Defines the egocentric camera configuration that the static and dynamic datasets adopt for the ego views.","marker":"[13]"},{"why":"One of the established driving sensor setups the generator can reproduce, used to show compatibility with real-world suites.","marker":"[59]"},{"why":"Another established driving sensor setup the generator reproduces, expanding the range of supported ego-camera layouts.","marker":"[91]"},{"why":"Provides the few-image-to-3D method specialized for ego-exo driving reconstruction that anchors the single-shot benchmark.","marker":"[34]"},{"why":"Supplies the Gaussian-splatting scene representation used as a baseline in both multi-view and few-image benchmarks.","marker":"[47]"},{"why":"The closest prior dataset of surround exo vehicle views without ego views or temporal data, and the source of the sparse-view baseline implementation.","marker":"[44]"},{"why":"Provides the sparse-view radiance-field baseline that conditions on projected image features from few input images.","marker":"[114]"},{"why":"Supplies the image-to-Gaussian-splatting single-view reconstruction baseline evaluated in the benchmark.","marker":"[94]"}],"fun_headline_variants":["SEED4D: first large-scale ego-exo 4D driving dataset","SEED4D: 16.8M synthetic images push 3D driving into 4D","SEED4D: ego-exo dynamic benchmark for self-driving 4D models","SEED4D: open source ego-exo camera data for 4D driving"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The value of the whole resource rests on the simulator's synthetic scenes being a workable stand-in for real urban driving—both in how they look and in how vehicles move—so that what models learn on SEED4D transfers to real roads.","fun_headline_variants_meta":{"raw":{"variants":["SEED4D: first large-scale ego-exo 4D driving dataset","SEED4D: 16.8M synthetic images push 3D driving into 4D","SEED4D: ego-exo dynamic benchmark for self-driving 4D models","SEED4D: open source ego-exo camera data for 4D driving"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001373,"raw_usage":{"total_tokens":5560,"prompt_tokens":934,"completion_tokens":4626,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":4532}},"tokens_in":550,"tokens_out":4626,"duration_ms":28101,"temperature":1.0,"reasoning_tokens":4532,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:04:06.280889+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a few-image-to-3D model on SEED4D and evaluate it zero-shot on real street images with known geometry; if its reconstruction and depth error is no better than a model trained on unrelated real images, the claim that this synthetic ego-exo data closes a real supervision gap is falsified.","supporting_citations":[{"cited_title":"Nerfstudio: A mod- ular framework for neural radiance field development","cited_arxiv_id":null,"evidence_quote":"Provides the radiance-field pose format and the method implementations used in the novel-view-synthesis benchmarks."},{"cited_title":"pixelNeRF: Neural radiance fields from one or few images","cited_arxiv_id":null,"evidence_quote":"Provides the sparse-view radiance-field baseline that conditions on projected image features from few input images."},{"cited_title":"Splatter image: Ultra-fast single-view 3d recon- struction","cited_arxiv_id":null,"evidence_quote":"Supplies the image-to-Gaussian-splatting single-view reconstruction baseline evaluated in the benchmark."}],"review_version":1}