{"id":"ac17b143-e070-4161-9703-bb020db9fdde","arxiv_id":"2507.13344","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A sliding iterative denoising scheme that alternates spatial and temporal passes, combined with skeleton conditioning, lets a diffusion model create spatio-temporally consistent multi-view human videos from sparse input videos.","lead":"Diffuman4D turns a few camera recordings of a moving person into a full set of consistent views from many angles, by repeatedly refining each view with information from its neighbors in space and time. The method could make free-viewpoint replays practical in sports and film with only a handful of cameras instead of dozens.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sliding iterative denoising (Sec. 3.2) is not a well-defined diffusion sampling procedure: overlapping windows force latents with different denoising histories — hence different noise levels — into the same network call, and no per-sample timestep bookkeeping is specified.","rationale":"The reader identified the same load-bearing weakness I would: the sliding iterative denoising schedule rests on an unstated and non-obvious assumption about how timesteps or noise levels are assigned to latents with unequal denoising histories inside a single network call. This is not a cosmetic omission; it determines whether the method is a valid diffusion sampling process of the kind the paper claims. I reproduced the argument with a small W=3, S=1 example: overlapping windows immediately create mixed noise levels in one batch, so either the model uses a timestep that is wrong for some latents, or the architecture and training must support per-latent timesteps, neither of which is stated. This concern is directly load-bearing for the central claim that the sliding mechanism provides a large receptive field and thereby improves 4D consistency. Because the reader's verdict was already CONDITIONAL with this concern, my independent pass does not move the verdict; it reinforces it. I also note the paper's own supplementary Sec. C states that the directly generated videos 'still inevitably exhibit spatio-temporal inconsistencies' and that subsequent 4DGS reconstruction is used to alleviate them, which further cautions against interpreting the headline consistency claim too strongly, but the formal timestep gap is the more fundamental issue. No code or data are released, and there are no error bars on the stochastic pipeline, so the exact schedule cannot currently be resolved from the manuscript. The paper has genuine strengths: two dataset evaluations, ablations of the denoising strategy and conditioning scheme, and a plausible design. The correct path is to require the authors to specify the per-sample timestep handling and validate it, not to reject the empirical contribution out of hand.","tokens_in":16324,"tokens_out":10115,"duration_ms":126609,"concrete_test":"Run the Sec. 3.2 schedule literally on one N=12 sequence with W=3, S=1, P=2 (so D=12). Before every network call, log the number of denoising updates already applied to each latent in the window. Because windows overlap, this log will contain a window with unequal counts (e.g., after the first two windows, counts {1,1,0}); that window is exactly where no single timestep t can describe the batch. If instead the implementation uses per-latent timesteps, verify that the training objective also uses per-latent timesteps. Then reproduce Table 2's sliding-iterative row under the corrected, well-defined schedule; if PSNR drops by more than about 0.5 dB relative to the as-written version, the reported gain depends on the undefined mixed-noise process rather than on the claimed large-receptive-field consistency mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim attributes 4D consistency to the sliding iterative denoising process in Sec. 3.2: a length-W window slides with stride S over the latent grid, denoising P steps per window, and each sample is claimed to receive D = 2*P*W/S denoising steps in total. The paper never specifies what diffusion timestep or noise level is used at each network call. With any overlap W > S, a new window contains latents that were already partially denoised in earlier windows together with latents that were not. For example, with W=3 and S=1, after a window processes samples {1,2,3} for P steps, the next window contains {2,3,4}, where samples 2 and 3 have one more denoising update than sample 4. Feeding these latents to the network with a single shared timestep is wrong for at least one noise level; feeding per-latent timesteps is an architectural and training change that the paper does not describe, and standard training corrupts all samples in a batch with the same timestep. Thus D = 2*P*W/S cannot be mapped to any standard DDPM/DDIM schedule, and the alternating spatial/temporal halves in the same section aggravate the problem by mixing histories across two different sequence contexts. The supplementary mention of DPM-Solver++ with 24 sampling steps does not resolve this: W, S, and P are not given, and no per-sample timestep update rule appears anywhere. The mechanism may work as a heuristic, and the ablations and two-dataset evaluation are encouraging, but as written the central sampling process is formally underspecified and cannot be reproduced or validated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Diffuman4D proposes a two-stage system for free-viewpoint video synthesis of humans from sparse-view video. In the first stage, a latent diffusion model, initialized from Stable Diffusion 2.1 and fine-tuned on a processed DNA-Rendering subset, generates dense multi-view videos. Input views, skeleton renderings, Plücker coordinates, and a conditional mask are concatenated in latent space. The central novelty is a sliding iterative denoising schedule (Sec. 3.2) in which a window of length W slides with stride S over a latent grid, applying P denoising steps per window, first along the spatial dimension and then along the temporal dimension; the paper claims each latent receives D = 2PW/S denoising steps, yielding a large receptive field at bounded GPU cost. In the second stage, the synthesized videos are used to reconstruct a 4D Gaussian Splatting representation with LongVolcap. Experiments on DNA-Rendering and ActorsHQ compare against LongVolcap, GauHuman, GPS-Gaussian, and a reproduced CAT4D, with ablations over the denoising strategy and conditioning scheme.","tokens_in":16712,"tokens_out":8083,"duration_ms":88731,"significance":"The contribution is potentially significant: if the sliding schedule is well-defined, it offers a practical way to extend multi-view video diffusion to long sequences while keeping memory affordable, and the paper's integration with a 4DGS reconstruction backend is sensible. The paper has clear strengths: evaluation on two public datasets, ablations for both main components, a reproduced CAT4D baseline trained under the same settings, and a plan to release a processed DNA-Rendering dataset. The reported numbers, if reproducible, are substantially better than the baselines. However, the central sampling schedule is under-specified in a way that affects the validity of the method as a diffusion process, and the empirical support would be stronger with error bars and a direct consistency metric.","major_comments":[{"comment":"The sliding iterative denoising schedule is not a well-defined diffusion sampling procedure as written. With overlapping windows (W > S), a new window contains latents that have undergone different numbers of prior denoising updates (e.g., with W=3, S=1, samples 2 and 3 have one more update than sample 4 when the window moves), yet the paper never specifies the diffusion timestep or noise level assigned to each latent in each network call. The formula D = 2PW/S counts network applications, but it cannot be mapped to a standard DDIM/DDPM or DPM-Solver++ schedule unless per-sample timestep bookkeeping is provided; the supplementary's statement that DPM-Solver++ with 24 steps is used does not resolve this because W, S, and P are not reported. Please provide exact pseudocode for the schedule, including per-window/per-sample timestep assignment, boundary handling at sequence ends, and the integer constraints on D/2 and W/S.","section":"Sec. 3.2"},{"comment":"The quantitative evidence for the central claim is incomplete: the denoising-strategy ablation reports only PSNR, SSIM, and LPIPS, which are per-frame image-similarity metrics and do not directly measure spatio-temporal consistency. Since the paper's core claim is that sliding iterative denoising enhances 4D consistency, the evaluation should include a direct consistency metric, such as temporal flicker, cross-view re-projection error, or a warping-based metric, rather than relying on per-frame averages and qualitative figures.","section":"Sec. 4.4 / Table 2"},{"comment":"No error bars or significance tests are reported for any of the quantitative comparisons, although the method is stochastic (diffusion sampling with classifier-free guidance). The claim that Diffuman4D 'significantly outperforms' the baselines would be more defensible with means and variances over multiple independent sampling runs and a statement of the number of trials; this applies especially to the small-scale ablations in Tables 2 and 3.","section":"Tables 1–3"}],"minor_comments":[{"comment":"The table title contains a typo: 'DNA-Rednering' should be 'DNA-Rendering'.","section":"Table 1"},{"comment":"The caption says that 'GPS-Gaussian uses 8 input views while all other methods use 4 input views,' which is inconsistent with Table 1 where GPS-Gaussian is evaluated in both 4-view and 8-view settings; please clarify the input-view counts for each method in each panel.","section":"Fig. 6 caption"},{"comment":"There are formatting errors such as 'with24 sampling steps' (missing space) and '4D reconstrcution' (misspelled); similar spacing/encoding issues affect 'V AE' in Sec. 3.1 and 'Pl ¨ucker' throughout.","section":"Supplementary Sec. A"},{"comment":"The notation is inconsistent: the text uses N for target views and T for frames, while Fig. 3b says M=2, N=5 for an 'M-view, N-frame video'; please unify the notation for the number of views and frames.","section":"Sec. 3.2 and Fig. 3b"},{"comment":"The formula for total denoising steps is typeset differently as 'D = 2×P ×W/S' in the text and 'D = 2P W/S' in Fig. 3; please use one consistent form and state the required divisibility conditions (e.g., W/S and D/2 integer).","section":"Sec. 3.2"},{"comment":"Please clarify whether the reproduced CAT4D† uses its original sliding-window strategy or an adapted version of the proposed alternating spatial/temporal procedure, since the current wording says it uses 'the same sampling sequences and conditional-view selection strategy described in Sec. 3.2' but not the same denoising schedule.","section":"Sec. 4.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope and the empirical results are encouraging, but the central sampling schedule is under-specified in a way that currently prevents the method from being reproduced or formally verified as a diffusion process. I recommend asking the authors for pseudocode and exact hyperparameters (W, S, P), as well as error bars and a direct consistency metric, before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the sliding iterative denoising idea is real: alternating spatial and temporal passes with a sliding window, and doing a few denoising steps per window, is a practical way to stretch receptive field without blowing up memory. The ablations in Table 2 and the large gains over their own CAT4D reproduction support that. Second, the paper as written does not define the sampling algorithm well enough to reproduce. The stress-test note is on target.\n\nThe good parts: this is the first human-specific multi-view video diffusion that combines skeleton-Plücker mixed conditioning with sliding-window denoising. The numbers are strong: 25.39 PSNR vs 21.45 for CAT4D on 4-view DNA-Rendering, and the gap holds on ActorsHQ zero-shot. Ablations of the denoising strategy and conditioning both show consistent improvement. Recalibrating DNA-Rendering and releasing the processed data is a service to the community.\n\nThe soft spots are real but fixable. The core sampling schedule in Sec. 3.2 is underspecified: with W greater than S, overlapping windows put latents with different denoising histories into the same network call, and the paper never says what timestep each call uses. The D=2*P*W/S claim does not map to any standard DDIM or DPM-Solver schedule unless you add bookkeeping that is not described. The supplementary gives 24 sampling steps but no W, S, P values. This is the load-bearing mechanism, so it has to be pinned down before anyone can validate or build on it. Also, there are no error bars over seeds for a stochastic pipeline, and no direct metric for 4D consistency, just PSNR/SSIM/LPIPS on the rendered 4DGS. These are addressable. Minor by comparison: the method is evaluated through LongVolcap for all methods, which is fair, and I do not see circularity in the evaluation.\n\nBottom line: the paper deserves a serious referee. The novelty and the scale of the empirical improvement justify referee time. But I would send it back with a request to specify the timestep handling, report hyperparameters for W/S/P, release code, and add variance estimates. If the sampling schedule turns out to be a heuristic that works despite mismatched noise levels, that is fine; just say so and characterize it. As written, it is a conditional accept at best.","headline":"Sliding iterative denoising is a genuinely useful idea, but the missing timestep schedule for overlapping windows makes the core algorithm unreproducible as written.","tokens_in":17259,"tokens_out":2830,"would_cite":true,"duration_ms":32550,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sliding iterative denoising, alternating across viewpoints and time, produces 4D-consistent novel-view human videos from sparse-view inputs.","keywords":["4D view synthesis","sparse-view video","diffusion model","novel view synthesis","sliding window denoising","human performance capture","Gaussian splatting","spatio-temporal consistency"],"falsifier":"Regenerate a test sequence with per-sample noise levels tracked exactly (each latent denoised according to its true step count) and compare the output consistency against the paper's shared-timestep sliding schedule; if the sliding variant does not produce fewer inter-window inconsistencies than independent-window denoising with the same total compute, the claimed large-receptive-field benefit is not supported.","tokens_in":16130,"feed_emoji":"🎥","tokens_out":4559,"duration_ms":47345,"temperature":0.7,"pith_summary":"The paper claims that a sliding iterative denoising process, which alternately denoises a 4D latent grid along spatial and temporal dimensions, lets a spatio-temporal diffusion model obtain a large receptive field and thereby generate 4D-consistent novel-view human videos from sparse-view inputs. This matters because prior multi-view video diffusion models generate views in independent groups and suffer from inconsistencies across groups, which degrades downstream reconstruction. The paper also adds a skeleton and camera conditioning scheme to stabilize human pose and improve appearance quality. On the DNA-Rendering and ActorsHQ datasets, the method reports higher PSNR and SSIM and lower LPIPS than the compared optimization-based, SMPL-based, feed-forward, and generation baselines. If correct, this gives a practical recipe for generating long, consistent multi-view human video from just a few cameras.","feed_headline":"A sliding-window denoise keeps sparse-view human videos 4D-consistent","feed_subtitle":"Alternating denoising across views and time yields sharper novel-view videos from four cameras.","key_machinery":"The sliding iterative denoising process is the central mechanism: a context window of length $W$ slides over the $(N+M) \\times T$ latent grid with stride $S$, alternately along the spatial (multi-view) and temporal (video) dimensions, performing a few denoising steps per window position and reversing direction to enable bidirectional context aggregation. This is what extends the diffusion model's receptive field without exceeding GPU memory. The second mechanism is skeleton-Plücker mixed conditioning, where 3D skeletons projected into each view are encoded into the latent space and concatenated with Plücker coordinates, giving pixel-aligned human pose and camera control that resolve front-back ambiguity and improve pose accuracy.","core_discovery":"The central claim is that consistency in long-sequence multi-view video generation can be achieved by treating all target views and frames as a single 4D latent grid and denoising it with a sliding window that iterates over the spatial and temporal dimensions. Each sample in the grid is a latent encoding the image, camera pose, and human pose for a given viewpoint and timestamp. The window slides counter-clockwise then clockwise in the spatial dimension and similarly in the temporal dimension, and each sample receives a total of $D = 2 \\times P \\times W / S$ denoising steps, equal to the diffusion inference steps. The authors argue this gives the model a large receptive field, lets nearby samples share more joint denoising steps (matching the correlation structure of 4D data), and keeps GPU memory bounded by the window size. They further condition generation on projected 3D skeletons plus Plücker coordinates to constrain pose, and feed the resulting dense-view videos into a 4D Gaussian Splatting reconstruction pipeline. Experiments show consistent improvements over the compared baselines at both 4-view and 8-view settings.","pith_inferences":["The sliding schedule implicitly assumes that all latents in a window can be denoised with a single shared timestep even though samples have undergone different numbers of denoising passes across sliding iterations; a reader could test whether this assumption holds and whether it affects generation quality.","The sliding iterative denoising mechanism is not human-specific and could transfer to other multi-view video diffusion tasks, such as general dynamic scenes, where long-sequence consistency is also a bottleneck.","The paper sets window length and stride to fixed values ($W=3$, $S=1$) without ablating them; the trade-off between consistency and inference time across these hyperparameters is a testable extension.","Releasing the processed DNA-Rendering dataset could standardize sparse-view human evaluation, since the reproduced baseline CAT4D was retrained on the same processed data."],"forward_implications":["The method enables 1024p, 4D-consistent novel-view human video synthesis from as few as 4 input views, with visual quality reported to be comparable to dense 48-view reconstruction.","Alternating spatial and temporal denoising with sliding windows introduces a smoothness inductive bias that removes the group-boundary jumps seen in multi-group and median-filtering denoising strategies.","Skeleton conditioning combined with Plücker coordinates reduces camera misalignment and front-back ambiguity, improving the accuracy of generated human poses.","The generated dense-view videos can be fed into an off-the-shelf 4D Gaussian Splatting pipeline, yielding a real-time renderable free-viewpoint video representation.","The processed DNA-Rendering dataset with recalibrated cameras, color correction, foreground masks, and estimated skeletons is planned for release, which could support future sparse-view human reconstruction research."],"supporting_citations":[{"why":"The prior multi-view video diffusion model whose sliding-window and median-filtering strategy is the direct baseline and whose alternating denoising scheme the paper extends.","marker":"[68]"},{"why":"The 4D Gaussian Splatting reconstruction method (LongVolcap) used to turn generated videos into a 4D representation and also one of the optimization-based baselines.","marker":"[73]"},{"why":"The DNA-Rendering dataset used for training and quantitative evaluation, which the paper reprocesses for camera calibration, color correction, masks, and skeletons.","marker":"[10]"},{"why":"The ActorsHQ dataset used for zero-shot generalization evaluation of the trained model on unseen actors.","marker":"[23]"},{"why":"The pretrained latent diffusion backbone (Stable Diffusion 2.1) from which the model is initialized and whose sampling procedure is followed.","marker":"[52]"},{"why":"The human skeleton estimation model (Sapiens) used to predict 2D skeletons that are triangulated into the 3D skeleton conditioning signal.","marker":"[27]"},{"why":"The Plücker coordinate embedding used as camera pose conditioning, contributing to the skeleton-Plücker mixed conditioning scheme.","marker":"[82]"},{"why":"The multi-view latent diffusion architecture (CAT3D) whose 3D self-attention design the proposed model follows.","marker":"[18]"}],"fun_headline_variants":["Sliding denoise on a latent grid keeps human 4D video consistent","Sparse views to consistent 4D video via sliding-window denoise","Iterative sliding denoise for consistent human view synthesis","4D human video from sparse views with sliding latent denoising"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that during sliding, latents that have undergone different numbers of denoising passes can be fed together with a single shared timestep, and the resulting schedule still behaves like a well-defined diffusion sampling process.","fun_headline_variants_meta":{"raw":{"variants":["Sliding denoise on a latent grid keeps human 4D video consistent","Sparse views to consistent 4D video via sliding-window denoise","Iterative sliding denoise for consistent human view synthesis","4D human video from sparse views with sliding latent denoising"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000335,"raw_usage":{"total_tokens":1887,"prompt_tokens":1006,"completion_tokens":881,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":805}},"tokens_in":622,"tokens_out":881,"duration_ms":9236,"temperature":1.0,"reasoning_tokens":805,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:25:00.622239+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Regenerate a test sequence with per-sample noise levels tracked exactly (each latent denoised according to its true step count) and compare the output consistency against the paper's shared-timestep sliding schedule; if the sliding variant does not produce fewer inter-window inconsistencies than independent-window denoising with the same total compute, the claimed large-receptive-field benefit is not supported.","supporting_citations":[{"cited_title":"Representing long volumet- ric video with temporal gaussian hierarchy","cited_arxiv_id":null,"evidence_quote":"The 4D Gaussian Splatting reconstruction method (LongVolcap) used to turn generated videos into a 4D representation and also one of the optimization-based baselines."},{"cited_title":"Humanrf: High-fidelity neural radiance fields for humans in motion","cited_arxiv_id":null,"evidence_quote":"The ActorsHQ dataset used for zero-shot generalization evaluation of the trained model on unseen actors."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"The pretrained latent diffusion backbone (Stable Diffusion 2.1) from which the model is initialized and whose sampling procedure is followed."},{"cited_title":"Srinivasan, Jonathan T","cited_arxiv_id":null,"evidence_quote":"The multi-view latent diffusion architecture (CAT3D) whose 3D self-attention design the proposed model follows."}],"review_version":1}