{"id":"6c70e613-6e57-401f-b9ab-687071b046c2","arxiv_id":"2506.23153","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dynamic NeRF method that handles small camera motion by regularizing rendering weight distributions with Gumbel-Softmax sampling and by jointly optimizing camera parameters.","lead":"This paper presents a method for generating new camera views of moving scenes from videos where the camera barely moves, such as phone clips. It adds a new depth-based training rule that helps the model learn correct scene shape, and it also learns camera poses during training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The depth priors in Eqs. 16-17 treat MiDaS output as absolute metric depth, but MiDaS predicts affine-invariant disparity; with no scale/shift alignment described, the 'true distribution' target is not the true depth, so the central geometry claim lacks support.","rationale":"The reader's weakest assumption correctly points at the dependence on MiDaS. The most specific load-bearing version of this concern is not only that MiDaS can fail semantically (non-opaque, thin geometry) but that even a successful relative-depth prediction is not an absolute metric depth. The equations in Sec. IV-B use D(r) as if it were the true distance along the ray; the density loss literally takes points with t < D(r) as free space. MiDaS is an affine-invariant disparity estimator, so D(r) is likely a scalar multiple plus offset of the true depth (or inverse depth). Without per-image alignment, the target distribution is displaced. This is testable by substituting ground-truth depth. Since the paper provides no code and no ablation against ground-truth depth, the conditional verdict is appropriate; we do not move to reject because the authors may have implicitly normalized depth in an unstated way, and the method may still work well in practice. Our recommendation is unchanged: conditional acceptance with the specific condition that the depth alignment be clarified and tested.","tokens_in":19239,"tokens_out":8265,"duration_ms":90362,"concrete_test":"Re-run the method on the Nvidia Dynamic Scenes benchmark with ground-truth metric depth (available from the multi-view setup) substituted for the MiDaS depth in L_weight and L_density, keeping everything else fixed. If the full-model PSNR/SSIM or the rendered depth maps change substantially (e.g., >0.5 dB or visibly different geometry), this confirms that the method is sensitive to the absolute depth scale and that MiDaS's affine ambiguity is a live issue. Additionally, inspect the code (when released) for any per-image scale/shift normalization of the MiDaS output; if none exists, the loss is ill-posed with respect to the claimed target.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that DDR and the density loss make the rendering-weight distribution align with the true distribution. Both losses use the MiDaS output D(r) as ground-truth absolute depth: L_weight in Eq. 16 compares rendered sample depth \\hat{T} to D(r), and L_density in Eq. 17 carves all volume density for t < D(r). MiDaS (Ranftl et al. 2022) outputs inverse depth (disparity) that is only defined up to an unknown per-image scale and shift; it is not metric depth. The paper never describes a scale/shift alignment or a conversion to metric depth in Sec. IV-F or the implementation details, even though the NDC coordinate system gives absolute distances. Consequently, even a perfect MiDaS prediction would supply a transformed target: the density-loss boundary would be at the wrong absolute location, and the L1 depth loss would pull geometry toward a depth that differs by a per-image affine map. The method's reported success on Nvidia Dynamic Scenes may still hold because the appearance loss can overfit, but the claimed 'accurate scene geometry' and 'alignment with true distribution' are not established. The self-reported limitations (non-opaque objects, thin geometry, motion blur) are precisely regimes where MiDaS relative depth is unreliable, which compounds rather than resolves the concern. This is the most load-bearing weakness because the entire contribution is a depth-based shape regularizer; if the supervision signal is not absolute depth, the mechanism does not do what is claimed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses novel view synthesis for dynamic scenes from monocular videos with small camera motion. It proposes Distribution-based Depth Regularization (DDR), which uses Gumbel-Softmax to convert the discrete rendering-weight distribution into a continuous mixture and penalizes sampled depth error against MiDaS depth, plus a density carving loss that zeroes density before the object boundary. Camera parameters are initialized by RCVD and refined jointly. The method is evaluated on the Nvidia Dynamic Scenes dataset under a three-camera small-motion protocol and on self-captured iPhone videos, with ablations for each component.","tokens_in":19642,"tokens_out":5005,"duration_ms":52674,"significance":"If the claims held, the method would be a practically useful contribution because small camera motion is common in casual capture. The paper includes a thorough comparison against seven baselines, ablations of each loss term, additional large-motion experiments, a novel weight-visualization tool, and self-captured real videos. The gains on the small-motion protocol are substantial (full-image PSNR 21.47 vs. 20.33 for DVS*). However, the main regularizer's theoretical justification and its supervision signal both need additional support before these results can be interpreted as evidence for accurate scene geometry.","major_comments":[{"comment":"MiDaS outputs affine-invariant disparity rather than metric depth, and the paper does not describe any scale/shift alignment. The method treats D(r) as absolute depth in the L1 rendering-weight loss and as a hard boundary in the density loss. Since the NDC coordinates used for rendering are metric, a per-image affine transform between MiDaS disparity and true depth means that the target distribution is not the true distribution. The central claim that DDR 'ensures the rendering weight distribution to align with the true distribution' is therefore not established, and the density loss could carve away valid geometry or retain floaters if the boundary is misplaced. The authors should either align MiDaS output to metric depth (e.g., by fitting scale/shift on the training rays or dataset-wide) or re-cast the method as a relative-depth regularizer, and they should provide quantitative geometry evaluation against independent ground truth.","section":"Sec. IV-B, Eqs. (16)-(17)"},{"comment":"The described procedure is not a sample from the mixture distribution. Equation (15) computes a weighted sum of one sample from each triangular sub-distribution using Gumbel-Softmax weights, which is a soft-selection average rather than a categorical draw followed by a continuous sample from the chosen sub-distribution. With the reported temperature epsilon = 2, the Gumbel-Softmax weights are far from one-hot, so \\hat{T} is not a draw from P(t). The 'expectation of the error' formulation in Eq. (16) is therefore not an unbiased estimator of E_{t~P(t)}[d(D,t)] as written; it is a weighted average over all sub-distributions. The authors should either implement true Gumbel-Softmax sampling (one category sample plus a continuous sample from that component) or provide a corrected derivation of what the objective actually estimates.","section":"Sec. IV-B, Eq. (15)"},{"comment":"The total objective is never specified. The paper defines Lrgb, Lweight, Ldensity, and Lgrad but never writes the combined loss L = Lrgb + lambda_w * Lweight + lambda_d * Ldensity + lambda_g * Lgrad, and no weighting coefficients are reported. This is a key reproducibility gap: the contribution of each regularizer cannot be assessed without these weights. The paper should report the full loss and all hyperparameters, including the parameterization of the camera residual variables Delta P and Delta f used in Eqs. (18)-(19).","section":"Sec. IV-F and Sec. IV-B"},{"comment":"The evidence for the geometry claim is largely self-referential. The depth maps shown in Figs. 7, 9, and 11 are produced by a model supervised with MiDaS depth, so they are not independent evidence of 'accurate scene geometry.' No quantitative depth error against ground truth is provided, even though the Nvidia Dynamic Scenes dataset supports evaluating depth accuracy. Furthermore, the limitations admitted in Sec. VI-A (non-opaque objects, densely thin geometry, motion blur) are exactly the regimes where MiDaS relative depth is unreliable, so the failure modes are consistent with a supervision-signal problem. A quantitative depth metric is needed to support the paper's central geometry claim.","section":"Sec. V and Sec. VI-A"}],"minor_comments":[{"comment":"The claim of being the 'first dynamic NeRF method that effectively handles monocular videos of dynamic scenes with small camera motion' should be qualified because RoDynRF also performs camera parameter joint learning and is evaluated under small motion; the paper's novelty is the geometry regularization, not the handling of camera motion per se.","section":"Sec. I and Table I"},{"comment":"Equation (12) is typographically ambiguous: the division by epsilon after both the numerator and denominator should be parenthesized so that the Gumbel-Softmax formula is unambiguous.","section":"Eq. (12)"},{"comment":"The protocol for simulating small camera motion is unclear: the text says three adjacent cameras are selected, while the figure suggests different views are used for the first, middle, and final thirds of the video. Please clarify whether the three views are fixed across time or change over time.","section":"Fig. 6 and Sec. V-A"},{"comment":"The sentence in Sec. V-D that 'weight loss has a slightly lower impact compared to gradient loss' is confusing because the full-image SSIM is lower for w/o Lgrad (0.582) than for w/o Lweight (0.633), while dynamic-region PSNR is lower for w/o Lweight. The comparison should be stated per metric.","section":"Table III and Sec. V-D"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The genuinely new piece is the DDR loss: instead of matching the expected depth, they turn the discrete rendering weights into a continuous mixture of triangular sub-distributions and use Gumbel-Softmax to sample, then minimize the expected error. That is a real departure from DS-NeRF and DDNeRF, and the paper earns points for positioning it that way. The ablations show each component pulls its weight, and the numbers on the Nvidia small-motion protocol are consistently better than the adapted baselines. The camera joint learning with RCVD as initialization is a sensible answer to COLMAP failing on tiny baselines, and the rendering-weight visualization is a small but usable diagnostic.\n\nThe soft spots are real and one is load-bearing. MiDaS predicts affine-invariant disparity, not metric depth. The paper never describes a scale/shift alignment between MiDaS output and the NDC coordinates it uses for sampling. So in Eq. 16 the L1 target is not true depth, and in Eq. 17 the density carve boundary sits at the wrong absolute location unless an alignment is happening off-screen. The paper also gives no quantitative depth evaluation, so 'accurate scene geometry' is asserted rather than shown. Relative depth priors can still help a NeRF train, and the appearance loss may absorb a lot, but the central claim that the weight distribution aligns with the true distribution lacks support until this is fixed. I'd call this a major-revision issue, not a desk-reject.\n\nThere are secondary issues. Eq. 15 actually averages samples from every sub-distribution weighted by Gumbel-Softmax weights, which is not the same as drawing one sample from the selected component; it's a reasonable relaxation but the text should say so. The total loss and all weighting coefficients are never given, and there are no error bars, so the numbers are hard to check. The 'first' claim should be softened, and the quantitative protocol uses three cameras from a 12-camera rig rather than a true continuous small-motion capture, though the qualitative iPhone captures help.\n\nAll that said, this is a serious paper with a clear contribution and an honest limitations section. It deserves a careful referee. I'd send it to review with a request for major revision: specify the alignment, release code or full implementation details, report depth error against ground truth, and reconcile Eq. 15 with the sampling story.","headline":"A novel and plausible depth-regularization idea, but the MiDaS-to-NDC alignment gap makes the central geometry claim unproven; worth refereeing as a major revision.","tokens_in":20127,"tokens_out":4880,"would_cite":true,"duration_ms":55004,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T45","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that constraining the shape of the rendering weight distribution, not just its mean, lets a dynamic NeRF reconstruct correct scene geometry from monocular videos with small camera motion, and that joint camera-parameter…","keywords":["dynamic neural radiance fields","novel view synthesis","small camera motion","depth regularization","Gumbel-Softmax","rendering weight distribution","camera parameter joint learning","monocular video"],"falsifier":"Run the method on the Nvidia Dynamic Scenes small-motion protocol and compare the rendered depth against the dataset's ground-truth depth maps. If the per-ray depth error is no better than a model trained with the ordinary mean-depth loss, while the PSNR improvement remains, then DDR's claimed geometry alignment is not what carries the gain.","tokens_in":19066,"feed_emoji":"🎥","tokens_out":8916,"duration_ms":83564,"temperature":0.7,"pith_summary":"This paper takes on a regime where most dynamic NeRF methods collapse: monocular videos in which the camera barely moves. Its central claim is that standard depth regularization fails because it matches the mean of the rendering weight distribution along a ray, leaving the distribution's shape free to drift into wrong geometry. The proposed Distribution-based Depth Regularization (DDR) instead samples points from the rendering weight distribution with Gumbel-Softmax, converts the discrete weights into a continuous mixture of triangular distributions, and minimizes the expected depth error against monocular depth maps from MiDaS. A companion density loss forces the volume density of points before the object boundary to zero, and camera parameters are initialized with RCVD and refined jointly during training. If correct, the method yields accurate geometry and high-quality novel views under small camera motion, and the paper reports it as the first dynamic NeRF to handle such inputs effectively.","feed_headline":"Depth-distribution loss fixes dynamic NeRFs on tiny camera motion","feed_subtitle":"Matching depth means fails; sampling from the weight distribution keeps geometry accurate.","key_machinery":"The central mechanism is Distribution-based Depth Regularization (DDR), built on the per-ray rendering weights $w_i = T_i(1-\\exp(-\\sigma_i\\delta_i))$. DDR defines the target as a continuous mixture distribution $P(t)=\\sum_i w_i p_i(t)$ over triangular bases $p_i(t)$, samples from it with Gumbel-Softmax—a differentiable approximation of categorical sampling—and minimizes the expected absolute depth error $\\mathbb{E}_{t\\sim P(t)}[|D(r)-t|]$ against the MiDaS depth $D(r)$. The density loss $\\sum_{\\tau}\\sum_{x\\in X_f(\\tau)}\\|\\sigma_\\tau(x)\\|_1$ zeros out the volume density of points before the object boundary, and camera parameters are refined by residual updates initialized from RCVD. A rendering-weight visualization tool displays the scene geometry as a weight map and is used to argue that the learned distribution is closer to the ideal single-peak shape.","core_discovery":"The paper establishes that the object a depth regularizer should match is not the expected depth but the distribution of rendering weights along the ray. Its DDR loss makes that distribution continuous with triangular sub-distributions centered at each sample, draws points from it via Gumbel-Softmax, and penalizes the L1 distance from each drawn depth to the MiDaS depth; the paper argues this 'expectation of the error' shapes the weights into the desired single-peaked form, whereas the usual 'error of the expectation' leaves the shape unconstrained. DDR is paired with a volume-density loss that sets $\\sigma \\approx 0$ before the object boundary, and with camera extrinsics and focal length initialized by RCVD and refined jointly with the MLPs. On the Nvidia Dynamic Scenes small-motion protocol the method reports the best full-image PSNR 21.47 and SSIM 0.718, and the paper claims this makes it the first dynamic NeRF to effectively handle small-motion monocular videos.","pith_inferences":["Editorial extension: the same loss should transfer to few-shot static NeRF, where sparse SfM point clouds are the bottleneck; a direct comparison against DS-NeRF's KL-divergence loss on static sparse-view benchmarks would test this.","Editorial extension: the paper evaluates geometry only through renderings and MiDaS targets, so a direct depth-error comparison against ground-truth depth maps would isolate whether DDR genuinely improves geometry or only appearance.","Editorial extension: the failure cases the paper lists—non-opaque objects, thin structures, and motion blur—coincide with unreliable MiDaS depth, suggesting a confidence-weighted version of the DDR loss as a testable remedy.","Editorial extension: the disocclusion failures with slowly moving foregrounds imply that a learned or generative prior for never-observed background is the next necessary component for near-static cameras."],"forward_implications":["Dynamic scene reconstruction no longer needs wide-baseline parallax or SfM point clouds; monocular depth plus joint pose refinement is enough to stabilize geometry under small camera motion.","Any depth-supervised NeRF variant that currently matches mean depth can be re-expressed in this distribution-matching form, making DDR a drop-in replacement for its depth term.","Applications that consume NeRF geometry, such as editing, relighting, or extracting depth from dynamic scenes, gain reliability precisely in the low-motion regime where such geometry was previously unreliable.","The method remains competitive under large camera motion, so the small-motion fix does not sacrifice the conventional setting."],"supporting_citations":[{"why":"Supplies the monocular depth maps that define the DDR target in Eq. 16 and the object boundary in Eq. 17.","marker":"[12]"},{"why":"Gumbel-Softmax provides the differentiable categorical sampling that turns discrete rendering weights into trainable samples.","marker":"[64]"},{"why":"RCVD predicts the initial camera extrinsics and focal length used when COLMAP fails.","marker":"[13]"},{"why":"NSFF is the scene-flow baseline whose monocular training protocol and depth-constrained pipeline frame the comparison.","marker":"[5]"},{"why":"DVS supplies a scene-flow baseline and the large-motion evaluation protocol in Table IV.","marker":"[4]"},{"why":"RoDynRF is the only baseline that also learns camera parameters, isolating the contribution of DDR.","marker":"[54]"},{"why":"DS-NeRF provides the alternative depth-distribution regularization that DDR is compared against in Table V.","marker":"[59]"},{"why":"The 12-camera dynamic scenes dataset supplies the small- and large-motion evaluation protocols.","marker":"[67]"},{"why":"NeRF-- contributes the residual camera-parameter optimization strategy used for joint learning.","marker":"[65]"}],"fun_headline_variants":["Match depth distribution, not mean, for dynamic NeRFs on tiny motion","Gumbel-Softmax depth distribution keeps dynamic NeRFs stable on small motion","Dynamic NeRF learns true geometry from tiny camera motion via distribution depth","Distribution-based depth loss: the fix for dynamic NeRF with still cameras","Small motion? Distribution depth regularizer keeps dynamic NeRFs accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the MiDaS monocular depth maps are reliable enough to serve both as the target depth and as the definition of the object boundary; if MiDaS is wrong, the density loss carves away real geometry or leaves floaters, and the paper's own failure cases are exactly the cases where MiDaS is unreliable.","fun_headline_variants_meta":{"raw":{"variants":["Match depth distribution, not mean, for dynamic NeRFs on tiny motion","Gumbel-Softmax depth distribution keeps dynamic NeRFs stable on small motion","Dynamic NeRF learns true geometry from tiny camera motion via distribution depth","Distribution-based depth loss: the fix for dynamic NeRF with still cameras","Small motion? Distribution depth regularizer keeps dynamic NeRFs accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001529,"raw_usage":{"total_tokens":6154,"prompt_tokens":1007,"completion_tokens":5147,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":5049}},"tokens_in":623,"tokens_out":5147,"duration_ms":32665,"temperature":1.0,"reasoning_tokens":5049,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:48:25.809080+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the method on the Nvidia Dynamic Scenes small-motion protocol and compare the rendered depth against the dataset's ground-truth depth maps. If the per-ray depth error is no better than a model trained with the ordinary mean-depth loss, while the PSNR improvement remains, then DDR's claimed geometry alignment is not what carries the gain.","supporting_citations":[{"cited_title":"Towards robust monocular depth estimation: Mixing datasets for zero-shot cross- dataset transfer,","cited_arxiv_id":null,"evidence_quote":"Supplies the monocular depth maps that define the DDR target in Eq. 16 and the object boundary in Eq. 17."},{"cited_title":"Robust consistent video depth estimation,","cited_arxiv_id":null,"evidence_quote":"RCVD predicts the initial camera extrinsics and focal length used when COLMAP fails."},{"cited_title":"Neural scene flow fields for space-time view synthesis of dynamic scenes,","cited_arxiv_id":null,"evidence_quote":"NSFF is the scene-flow baseline whose monocular training protocol and depth-constrained pipeline frame the comparison."},{"cited_title":"Dynamic view synthesis from dynamic monocular video,","cited_arxiv_id":null,"evidence_quote":"DVS supplies a scene-flow baseline and the large-motion evaluation protocol in Table IV."},{"cited_title":"Robust dynamic radiance fields,","cited_arxiv_id":null,"evidence_quote":"RoDynRF is the only baseline that also learns camera parameters, isolating the contribution of DDR."},{"cited_title":"Depth-supervised nerf: Fewer views and faster training for free,","cited_arxiv_id":null,"evidence_quote":"DS-NeRF provides the alternative depth-distribution regularization that DDR is compared against in Table V."},{"cited_title":"Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera,","cited_arxiv_id":null,"evidence_quote":"The 12-camera dynamic scenes dataset supplies the small- and large-motion evaluation protocols."}],"review_version":1}