{"id":"841c9e7f-ddd9-4dde-88f4-9b6357d1c982","arxiv_id":"2411.13631","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Regularizing sparse-input radiance fields with visibility priors, simpler-solution depth supervision, and sparse flow priors improves novel view synthesis and depth estimation on multiple benchmarks.","lead":"This thesis proposes several new ways to train 3D scene models from very few camera views: a visibility prior computed with plane sweep volumes, depth supervision from deliberately simplified 'augmented' models, and sparse SIFT flow priors for moving scenes. It reports state-of-the-art results on multiple view-synthesis benchmarks and introduces a new dynamic-scene dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Simple-RF's depth-reliability mask (Eq. 4.8) is validated only indirectly; the thesis skips a direct ground-truth check on NeRF-Synthetic by falsely claiming GT depth is unavailable, leaving the central 'reliable priors' claim under-supported.","rationale":"The reader identified prior reliability as the weakest assumption, highlighting both ViP-NeRF's photometric consistency and Simple-RF's reprojection-MSE ranking. My concern is more specific and, I think, more load-bearing: the reprojection-MSE proxy in Eq. 4.8 is not merely a prior that can be wrong in rare cases; it is the decision rule that controls all cross-model depth supervision in Simple-RF, which is the thesis's main contribution family. If this proxy is unreliable, the method's performance could be driven by the fixed capacity reductions rather than by the claimed 'reliability' mechanism, weakening the thesis's central premise. The thesis provides only an indirect ablation (w/o reliable depth) and qualitative mask visualizations, not a direct precision/recall analysis against any depth reference. Strikingly, the thesis states that NeRF-Synthetic has no ground-truth depth, which is factually incorrect: the original Blender-based NeRF-Synthetic dataset includes depth maps for all views. This is a concrete, checkable error that also explains why the authors never validated their depth supervision on a dataset with real geometry. My proposed test directly measures the mask's precision and the depth improvement on GT depth, which would settle whether the 'reliable priors' claim holds. I do not think this concern changes the verdict from CONDITIONAL: the empirical results still show improvement, and the test could turn out in the thesis's favor. But it does sharpen the condition on which acceptance rests.","tokens_in":53499,"tokens_out":7745,"duration_ms":54815,"concrete_test":"Run Simple-ZipNeRF on NeRF-Synthetic (Blender scenes) and evaluate the reliability mask decisions from Eq. 4.8 against the ground-truth depth maps that ship with the dataset. On training views, compute the fraction of pixels where the mask selects the model with the lower actual depth error (precision), stratified by region type (textured, textureless, specular, occlusion boundary). Also compute the depth MAE of Simple-ZipNeRF against GT depth and compare it with ZipNeRF. If the mask precision is near chance in textureless or specular regions, or if the depth MAE improvement over ZipNeRF is negligible, the reliability premise underlying Simple-RF is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Simple-RF framework's core mechanism is the depth reliability mask of Eq. 4.8: it assumes that lower reprojection MSE to the nearest training view implies higher depth accuracy, and uses this to decide which of the main or augmented models' depths supervise the other. This proxy is plausible but never directly validated. The thesis only provides ablations showing that removing the mask hurts performance, not evidence that the mask selects the more accurate depth. The proxy can fail systematically: in textureless regions, reprojection MSE is low for many wrong depths; with large baselines (2-4 views), the reprojection may not discriminate; and at occlusion boundaries, neither depth reprojects well. The thesis also avoids the one dataset with ground-truth depth: it states that NeRF-Synthetic provides no GT depth, but the Blender-based NeRF-Synthetic dataset does provide depth maps for every view. This omission means the claimed 'superior geometry estimation' and 'reliable depth supervision' were never tested against real geometry. If the mask frequently selects the augmented model's depth where the main model is actually more accurate (or vice versa), wrong gradients enter the radiance field, and the gains in Tables 4.2-4.8 could stem from other components (e.g., capacity reduction, coarse-fine consistency) rather than the asserted reliability mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The thesis addresses sparse-input novel view synthesis for static and dynamic scenes. It proposes four families of methods: ViP-NeRF, which regularizes a NeRF with a dense visibility prior computed from plane sweep volumes; Simple-RF, which trains reduced-capacity 'augmented' radiance fields (NeRF, TensoRF, ZipNeRF) in tandem with the main model and supervises via a depth-reliability mask; SF-DeRF, a fast dynamic radiance field with an explicit factorized motion field regularized by sparse SIFT-based flow priors; and DeCOMPnet, an MPI-based temporal view synthesis method that decomposes camera and object motion for frame-rate upsampling. The thesis claims state-of-the-art performance on RealEstate-10K, NeRF-LLFF, MipNeRF360, NeRF-Synthetic, N3DV, InterDigital, and MPI Sintel, and introduces the IISc VEED-Dynamic dataset.","tokens_in":53846,"tokens_out":4888,"duration_ms":49511,"significance":"If the empirical claims hold, the thesis makes a useful contribution by showing that reliable geometric priors can be computed in a scene-specific, pre-training-free manner: the visibility prior and the simplicity-based depth priors avoid the generalization problems of learned dense depth priors, and the sparse flow priors avoid the unreliability of dense optical flow in multi-camera dynamic scenes. The breadth of the work is a strength: it spans static and dynamic scenes, implicit and explicit radiance fields, and an application-oriented temporal view synthesis setting. The thesis is also transparent about test-set choices and metric changes, and the priors are computed independently of the supervised targets, so the central claims are not circular. The main weaknesses are that the depth-reliability mask of Simple-RF is never validated directly against ground-truth depth, all comparisons are single-run without error bars, and the dynamic-view-synthesis state-of-the-art claim rests on a narrow set of baselines.","major_comments":[{"comment":"The depth-reliability mask is the load-bearing component of the Simple-RF framework, but the manuscript never validates the assumption that lower reprojection MSE implies more accurate depth. The ablations in Table 4.5 show that removing the mask degrades performance, but they do not establish that the mask selects the more accurate depth; other components (capacity reduction, coarse-fine consistency) may be responsible for the gains. Critically, Sec. 4.4.2 states that NeRF-Synthetic 'ground truth depth is not provided in the dataset either,' which is incorrect: the Blender-based NeRF-Synthetic dataset includes depth maps for every view, and Simple-ZipNeRF is evaluated on this dataset in Table 4.8. The one dataset that could directly test the reliability assumption is therefore available but avoided. I request a direct evaluation of the mask against ground-truth depth on NeRF-Synthetic, or a corrected and substantiated justification of why this is infeasible.","section":"§4.3.1.4, Eq. (4.8); §4.4.2"},{"comment":"All empirical comparisons are single-run and depth quality is measured against pseudo ground truth from dense-view NeRF/ZipNeRF models rather than true depth. Given that some headline improvements are small (e.g., Table 4.3, 4-view row: Simple-NeRF vs ViP-NeRF, LPIPS 0.0847 vs 0.0892), the lack of repeated runs, confidence intervals, or significance tests makes it hard to assess whether the state-of-the-art claims are robust. I ask for at least a few seeds for the main comparisons, or an explicit discussion of training variance.","section":"§4.4.2, Tables 4.2–4.4 and 3.1–3.4"},{"comment":"The claim that SF-DeRF 'outperforms the state-of-the-art dynamic view synthesis models with fewer input viewpoints' is supported only against K-Planes and HexPlane. Other dynamic radiance field methods discussed in Sec. 5.1 (e.g., D-NeRF, TiNeuVox) are not included in the quantitative comparisons. If the published version contains additional baselines, they should be reproduced or clearly cited in the thesis so that the state-of-the-art claim is commensurate with the evidence.","section":"§5.3.3, Table 5.2"}],"minor_comments":[{"comment":"The Simple-ZipNeRF row appears to have concatenated values ('21.030.239'), likely a formatting or rendering error; please fix the table so all entries are cleanly separated.","section":"Table 4.7"},{"comment":"Several mathematical expressions contain glyph artifacts, such as '/x31' in place of an indicator function and '∇' used as a stop-gradient placeholder. These should be typeset properly for a camera-ready version.","section":"Throughout (e.g., Eq. (3.10), Eq. (4.8))"},{"comment":"The word 'meta-verse' should be 'metaverse'; also, the thesis says in Sec. 1.1.2 that source code will be released, but Sec. 1.2 only gives a publications page URL. Please include a direct pointer to the code repositories if they are available.","section":"Abstract and Sec. 1.1"}],"recommendation":"major_revision","confidential_remarks":"The thesis is a compilation of four already-published papers, and the core ideas have passed peer review. The main concern for this version is the validation gap around the Simple-RF depth-reliability mask: the incorrect statement about NeRF-Synthetic ground-truth depth is easily fixable, but the omitted direct validation of Eq. (4.8) is load-bearing for the 'reliable priors' narrative. The single-run evaluation is a secondary but real weakness. I do not see circularity in the priors; the reader's concern about depth pseudo-ground-truth is reasonable and is part of the requested revision. Overall, the manuscript is technically solid and broad, and it should be publishable after the requested additional experiments and corrections."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thesis is a well-organized compilation of four papers you've probably already seen (SIGGRAPH 2023, SIGGRAPH Asia 2023, SIGGRAPH 2024, ISMAR 2022). If you're expecting a new result, there isn't one. The chapters are the published works with added narrative glue. What the thesis does well is frame them under one coherent philosophy: use priors that can be computed without pre-training on large datasets, and validate them across implicit and explicit representations. That framing has value. The visibility prior for NeRF and the SIFT sparse flow prior for dynamic scenes are genuinely useful ideas. Simple-RF's augmentation trick—training a lower-capacity model in tandem and using its depth to supervise the main model—is more general than the original paper alone suggested, and the extension to TensoRF and ZipNeRF is non-trivial.\n\nThe experiments are extensive, code links are there, and the thesis is transparent about the test-set change in Chapter 4 and about using pseudo ground truth for depth on most datasets. That transparency is worth crediting.\n\nThe soft spots are real but not fatal. First, all comparisons are single-run with no error bars, so the SOTA claims rest on average differences that could shift with seeds. Second, depth evaluation leans on pseudo ground truth from dense-view NeRFs; that's a reasonable proxy, but not true ground truth. The stress-test note is right that the thesis makes a false statement in Sec. 4.4.2: NeRF-Synthetic does provide ground truth depth maps. The stated reason for skipping depth evaluation on that dataset is wrong, and that's a missed opportunity to directly validate the depth reliability mask (Eq. 4.8). The mask is only validated indirectly through ablations showing that removing it hurts performance. That's evidence, but it doesn't show the mask selects the more accurate depth—and in textureless or heavily occluded regions, the reprojection-MSE heuristic can fail. The claim of \"reliable depth supervision\" would be stronger with a direct correctness check. Third, there are many hand-tuned thresholds (γ, eτ, λ's, augmentation capacities). You can't tune all of them per scene and still call the prior fully automatic.\n\nThe thesis is likely the author's PhD dissertation, so the intended reader is someone who wants the full arc of the research program, not a reader looking for a single new submission. If a journal version is in the works, it deserves a serious referee—the individual papers were peer-reviewed, and a consolidated article with the depth-validation gap fixed could be useful. I'd engage with it, but I'd ask the authors to close the GT-depth oversight before signing off.","headline":"A solid, clearly written thesis that compiles four published methods; the novel ideas are mostly in the reliability of pre-training-free priors, but the evaluation has weak spots around depth validation.","tokens_in":54364,"tokens_out":2964,"would_cite":true,"duration_ms":34256,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reliable priors make sparse-input view synthesis work","keywords":["novel view synthesis","sparse input views","neural radiance fields","visibility prior","plane sweep volumes","radiance field regularization","dynamic view synthesis","temporal view synthesis"],"falsifier":"A specular or textureless scene captured from only two viewpoints provides a direct test: if the visibility prior mislabels visibly matched pixels as occluded because their colors change with view, ViP-NeRF's render quality should fall below a depth-smoothness baseline on the same inputs; similarly, a dynamic multi-view clip with large inter-camera occlusion should show sparse-flow matching dragging keypoints to wrong canonical points, measurable as a rise in depth error against a dense-view reference.","tokens_in":53297,"feed_emoji":"🎥","tokens_out":14239,"duration_ms":141278,"temperature":0.7,"pith_summary":"The thesis targets novel view synthesis from very few input images, where radiance fields overfit and render blurry, ghosted, or float-filled frames. Its central claim is that failures come less from the 3D representation than from unreliable priors: learned dense depth and dense optical flow generalize poorly, while priors computed from the input geometry are dense, reliable, and enough to regularize the model. Across static scenes it contributes a plane-sweep visibility prior on relative depth, and a family of 'simpler solution' augmentations that supervise depth from reduced-capacity models on the same scene. For dynamic scenes it contributes a factorized motion field trained with sparse keypoint flows instead of dense optical flow. The reported results across forward-facing, 360-degree, and multi-view dynamic datasets put the proposed methods ahead of prior sparse-input baselines, and the final chapter extends the same warp-and-infill machinery to frame-rate upsampling of rendered videos.","feed_headline":"Two input views suffice when priors are reliable","feed_subtitle":"Geometric visibility and simpler-solution depth supervision beat learned dense priors for sparse-input radiance fields.","key_machinery":"The load-bearing mechanism is reliability-weighted supervision: every prior is applied only where it can be checked. For static scenes, the visibility prior is computed by warping one input view into another at several depth planes (a plane sweep volume) and thresholding the minimum per-pixel matching error, so the loss acts only on pixels judged visible in the second view. For Simple-RF, depth from reduced-capability augmented models is gated by a reprojection mask that compares each model's depth patch against the nearest training view, and the augmented models are built by lowering the positional-encoding degree, the hash-table size, or the tensor rank. For dynamic scenes, the motion field is a four-dimensional deformation stored as six factorized planes that maps every time to a canonical volume, and it is supervised by sparse keypoint correspondences that pull matched pixels to the same canonical 3D point. For temporal view synthesis, multi-plane images (a stack of images at discrete depth planes) together with masked correlation estimate object motion in 3D after nullifying camera motion.","core_discovery":"The paper's central discovery is that visibility—whether a pixel's surface appears in a second view—is a dense and reliable prior for sparse-input neural radiance fields, and that relative depth cues of this kind beat absolute-depth priors produced by pre-trained networks. It shows further that depth supervision need not come from external models: reduced-capability 'augmented' radiance fields trained alongside the main model provide better depth in smooth or Lambertian regions, and a reprojection-error check decides when each model's depth is trustworthy. For dynamic scenes, the thesis finds that dense optical flow across cameras is unreliable as a motion prior, whereas sparse keypoint matches, used to pull corresponding points to the same canonical 3D location, stabilize a factorized deformation field with only three input views. In temporal view synthesis, it shows that decoupling camera and object motion and estimating object motion in the 3D multi-plane image space improves future-frame prediction and disocclusion infilling.","pith_inferences":["The reliability-gating principle likely transfers beyond view synthesis: any under-constrained inverse problem with a cheap geometric check (reprojection, consistency, loop closure) could audit a learned prior the same way.","Because the visibility prior constrains relative depth ordering rather than absolute scale, combining it with a monocular absolute-depth estimate could give both robustness and metric scale without dense learned priors.","The scene-specific augmentation recipe could extend to 3D Gaussian splatting once its sparse-input initialization is solved, potentially closing a gap this thesis leaves open for that representation.","For dynamic scenes, the sparse keypoint ceiling could be raised by densifying correspondences with a network trained on the same scene, turning the reliable-sparse idea into a self-supervised densification loop."],"forward_implications":["With two to four input views on forward-facing scenes, visibility regularization outperforms learned dense depth priors on both rendering quality and depth accuracy.","The same 'simpler solutions' supervision recipe improves three different radiance fields—NeRF, TensoRF, and ZipNeRF—and removes floaters and duplication artifacts characteristic of sparse-input training.","For dynamic multi-view scenes with three cameras, a factorized deformation field regularized by sparse keypoint flows surpasses a model trained with dense optical flow priors, which actively hurts performance.","Frame-rate upsampling of rendered video improves when object motion is estimated in 3D multi-plane image space after nullifying camera motion, rather than predicted as 2D video motion.","Reliability gating is a necessary ingredient: ablations that disable the visibility prior, the reliability masks, or the coarse-fine consistency loss all degrade performance."],"supporting_citations":[{"why":"Defines the neural radiance field and volume rendering that the static-scene chapters regularize.","marker":"[123]"},{"why":"Supplies the sparse depth prior that ViP-NeRF and Simple-RF build on, and the DS-NeRF baseline for comparisons.","marker":"[46]"},{"why":"Provides the learned dense depth prior baseline that the thesis shows to be less reliable than its visibility prior.","marker":"[142]"},{"why":"Introduces plane sweep volumes and multi-plane images used for the visibility prior and for temporal view synthesis, and contributes the RealEstate-10K dataset.","marker":"[216]"},{"why":"The factorized 4D radiance field baseline that SF-DeRF extends with an explicit motion model and sparse-flow regularization.","marker":"[53]"},{"why":"N3DV, one of the two multi-view dynamic datasets used to evaluate SF-DeRF against prior dynamic view synthesis methods.","marker":"[98]"},{"why":"InterDigital, the second multi-view dynamic dataset used for the SF-DeRF evaluation.","marker":"[143]"},{"why":"TensoRF, the explicit radiance field that Simple-RF regularizes through a reduced-capacity augmentation.","marker":"[33]"},{"why":"ZipNeRF, the hash-grid radiance field that Simple-ZipNeRF regularizes for sparse 360-degree inputs.","marker":"[18]"},{"why":"The keypoint matching method used to build the reliable sparse flow priors that regularize SF-DeRF's motion field.","marker":"[111]"}],"fun_headline_variants":["Visibility priors beat depth nets for sparse NeRF","Few views, reliable prior: visibility is key","Sparse views rely on visibility, not dense depth","Two views suffice if you trust visibility","Relative visibility outdoes absolute depth priors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every prior assumes photometric consistency between views: if pixel intensities change across views—specular surfaces, lighting shifts, or heavy occlusion—the matching that underpins the prior can label visible pixels as occluded or pick the wrong model's depth, pushing the radiance field in the wrong direction.","fun_headline_variants_meta":{"raw":{"variants":["Visibility priors beat depth nets for sparse NeRF","Few views, reliable prior: visibility is key","Sparse views rely on visibility, not dense depth","Two views suffice if you trust visibility","Relative visibility outdoes absolute depth priors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000691,"raw_usage":{"total_tokens":3324,"prompt_tokens":1334,"completion_tokens":1990,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":950,"completion_tokens_details":{"reasoning_tokens":1920}},"tokens_in":950,"tokens_out":1990,"duration_ms":16877,"temperature":1.0,"reasoning_tokens":1920,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:18:18.592070+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A specular or textureless scene captured from only two viewpoints provides a direct test: if the visibility prior mislabels visibly matched pixels as occluded because their colors change with view, ViP-NeRF's render quality should fall below a depth-smoothness baseline on the same inputs; similarly, a dynamic multi-view clip with large inter-camera occlusion should show sparse-flow matching dragging keypoints to wrong canonical points, measurable as a rise in depth error against a dense-view reference.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces plane sweep volumes and multi-plane images used for the visibility prior and for temporal view synthesis, and contributes the RealEstate-10K dataset."}],"review_version":1}