{"id":"3bfac3fc-87a0-43e5-85e7-95294f02919a","arxiv_id":"2509.06579","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An autoregressive multi-view diffusion model that generates novel views sequentially with flexible input-output configurations, using causal masking, relative pose encoding, and KV caching.","lead":"CausNVS is a diffusion model that generates new camera views of a scene one at a time rather than all at once, and accepts any number of input images. It makes novel view synthesis flexible and streaming-friendly, which matters for AR/VR, robotics, and world models that need to imagine new views on the fly.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Long-rollout stability claim rests on an unablated noise-conditioning augmentation, and the DL3DV long-sequence evaluation shows a ~4 dB PSNR drop, so drift mitigation is not established.","rationale":"The reader identified the noise-conditioning augmentation as the weakest assumption, and I agree that this is the most load-bearing concern. The central claim explicitly promises stability over rollouts many times longer than the training horizon; the paper's only dedicated evidence is the DL3DV short-vs-long split, which shows a 3-4 dB degradation. The paper attributes stability to the noise-conditioning design, but presents no ablation isolating that component. This is not a manufactured concern: the paper states in Sec. 3 that the augmentation 'helps the model reweight context and correct flawed predictions,' yet no experiment verifies this. The causal-vs-non-causal ablation in Table 2 supports the overall autoregressive formulation and is a genuine strength, and the CaPE-based KV-caching argument is technically coherent. However, the absence of code or weights means the reader cannot independently confirm the reported long-rollout numbers, making the unablated mechanism the key risk. The proposed concrete test directly addresses this by removing the augmentation and measuring the effect on the long-sequence split; if the degradation is negligible, the paper's explanation is wrong but the empirical claim may still hold, whereas if the degradation is large, the mechanism is confirmed as important. Either way, the current manuscript does not provide enough evidence to distinguish these cases, so the CONDITIONAL verdict is appropriate and no change is needed.","tokens_in":13542,"tokens_out":13217,"duration_ms":116927,"concrete_test":"Run inference on the DL3DV long-sequence split twice: once with noise conditioning augmentation enabled (assign small noise levels to previously generated views, as described in Sec. 3) and once with augmentation disabled (set the conditioning noise level to 0 for all previously generated frames). Keep all other settings fixed and report PSNR and LPIPS both as a mean over the split and per target-frame index. If the no-augmentation run shows a mean PSNR drop larger than ~1 dB or a noticeably steeper per-frame decline, the augmentation is load-bearing for the stability claim. If the two runs are statistically indistinguishable, the stated drift-mitigation mechanism is unsupported and stability must be attributed to other components such as causal masking or the sliding-window attention.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim includes stable autoregressive rollouts up to 10x the training length. The mechanism asserted for this stability is per-frame noise conditioning during training plus noise conditioning augmentation at inference (Sec. 3, 'Noise-Level Conditioning'). This mechanism is never isolated: no ablation removes or varies the augmentation, and no experiment quantifies its contribution. The only long-rollout evidence is the DL3DV short-vs-long comparison in Table 1, where PSNR drops from 18.32 to 14.42 (N=3) and 18.75 to 15.22 (N=6), a 3.5-4 dB degradation. This is a substantial decline that is difficult to reconcile with the word 'stable.' Moreover, the paper does not report per-frame PSNR as a function of rollout position, nor does it compare against a non-causal baseline on the same long-sequence protocol. Without such comparisons, the degradation could be due to systematic drift, to the specific trajectories in the long split, or to the sliding-window forgetting earlier views. The training-time noise conditioning is a plausible way to make the model robust to imperfect context, but it is not proven to be the load-bearing component; the stability could instead come from the causal masking itself or from the pose-aware windowing. If the augmentation is ineffective, the paper's explanation of drift mitigation is wrong and the long-rollout claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces CausNVS, a multi-view latent diffusion model for novel view synthesis in which target views are generated autoregressively rather than jointly. The model applies causal masking across frame tokens in newly added frame-wise attention layers, samples per-frame noise levels during training, uses pairwise-relative camera pose encodings (CaPE) for coordinate-invariant conditioning, and employs a pose-aware sliding window with KV caching at inference. The central claims are that a single model supports arbitrary numbers of input and output views without retraining, that it can generate views sequentially for streaming settings, and that it remains stable for rollouts up to 10x the training length. The paper evaluates on RealEstate10K, LLFF, and DL3DV, comparing with fixed-view baselines, and includes an ablation contrasting causal and non-causal training.","tokens_in":13800,"tokens_out":7607,"duration_ms":59817,"significance":"If the central claims are supported, the paper makes a notable contribution: it identifies and addresses a genuine limitation of current multi-view diffusion models (fixed input/output view counts and forced joint denoising) and provides a causal formulation suited to streaming or interactive 3D applications. The combination of per-frame noise conditioning, CaPE, and KV caching is a clean design. The strongest evidence is the causal-vs-non-causal ablation (Table 2), which shows that the non-causal variant fails at short sequences and in autoregressive mode, demonstrating a concrete benefit of the causal formulation. The paper is honest about its training scale and does not overclaim SOTA on all benchmarks; the main uncertainty surrounds the long-rollout stability claim, which currently rests on mixed quantitative evidence and an unablated mechanism.","major_comments":[{"comment":"The long-rollout stability claim is not quantitatively supported. The DL3DV short-vs-long comparison, which is the only evaluation explicitly targeting autoregressive drift, shows a drop of about 3.5-4 dB in PSNR at N=3 (14.42 vs 18.32) and N=6 (15.22 vs 18.75), and the long-sequence results fall below SEVA by 1.5-2.6 dB. The paper reports no per-frame PSNR as a function of rollout position and no non-causal baseline on the same long-sequence protocol, so it is not possible to tell whether the degradation reflects systematic drift, harder target poses in the long split, or the sliding-window forgetting earlier views. Without these controls, the statement in the Table 1 caption that the model maintains 'stable quality over long generation rollouts' is stronger than the evidence.","section":"§4.1, Table 1"},{"comment":"The paper's stated mechanism for mitigating autoregressive drift—per-frame noise conditioning during training plus noise conditioning augmentation at inference—is never isolated. No ablation removes or varies the augmentation, and there is no experiment quantifying its effect on long rollouts. Since the paper explicitly attributes drift mitigation to this component, writing that it 'stabilizes subsequent predictions and improves robustness to accumulated errors,' the absence of a targeted ablation leaves the explanation for the long-rollout behavior unsupported. The stability could equally be due to the causal masking or to the pose-aware sliding window, which are also introduced in this work.","section":"§3, Noise-Level Conditioning"},{"comment":"The interpretation of the causal vs non-causal ablation is contradicted at one evaluation point. The text states that the non-causal model 'performs well only under conditions close to its training setup and degrades significantly when evaluated with different sequence lengths,' but at F=32, N=1 the non-causal parallel model achieves 19.27 dB, which is higher than the causal AR model's 17.44 dB and also higher than the non-causal model's own F=8 result (17.52 dB). The non-causal model clearly fails at F=2 and F=4 and in autoregressive mode, so the central point about flexibility stands; however, the unqualified claim about degradation is not consistent with the full table. Please qualify the statement or explain why the F=32 comparison should not be taken at face value.","section":"§4.2, Table 2"}],"minor_comments":[{"comment":"There is a typo in the sentence describing the training objective: 'the model ˆεθ is trained o predict' should read 'trained to predict.'","section":"§3, Training Objective"},{"comment":"The sentence 'These layers are zero-initialized (weights to identity, biases to zero)' is ambiguous, because zero-initialized weights are normally the opposite of identity-initialized weights; please clarify the intended initialization scheme.","section":"Appendix A"},{"comment":"Reference [19] is a Medium blog post on KV caching; the transformer literature contains more authoritative descriptions of key-value caching, and citing a primary source would be more appropriate.","section":"References"},{"comment":"The paper uses both 'SEVA' and 'SEV A' with inconsistent spacing; please unify the spelling.","section":"Throughout"},{"comment":"The analysis of CaPE attention scores is described only in the caption; please explain in the main text how the scores are computed and why the cyclic/linear variation with rotation/translation constitutes an SE(3)-aware inductive bias.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within the scope of the journal and the core idea is interesting. The main risk is that the long-rollout claim may not survive a thorough re-evaluation; the missing noise-conditioning ablation and the per-frame degradation analysis should be requested before acceptance. The Table 2 F=32 result should also be addressed in the revision, as it currently weakens the narrative about degradation in the non-causal model."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper is a genuine step forward, not a repackaging. The combination of causal masking, per-frame noise conditioning, and relative pose encoding (CaPE) into a single multi-view diffusion model is new, and it delivers the headline capability—one model that generates any N-to-M view configuration autoregressively, without retraining. The KV-caching-with-relative-pose story is also coherent; using CaPE to keep cached attention valid across a shifting window is a clean solution to a real problem.\n\nThe strongest evidence is Table 2. The causal model keeps working at F=2 and F=64, and improves with more inputs. The non-causal baseline collapses at F=2 and drifts badly in autoregressive mode. That ablation directly supports the paper's central flexibility claim, and I found it convincing.\n\nWhere the paper is soft:\n\n1. The long-rollout stability claim is over-sold. The stress-test note is right: the noise-conditioning augmentation is asserted as the drift-mitigation mechanism but never isolated. No ablation removes or varies it, no per-frame PSNR over rollout position is reported, and no non-causal baseline is run on the same long-sequence protocol. Meanwhile, the DL3DV long-vs-short numbers drop about 4 dB (18.32 to 14.42 for N=3). That is a real degradation. Calling it \"stable\" is too generous. It may be that the augmentation does help, but the paper does not show it. The long-sequence evidence is also on one dataset with one protocol.\n\n2. No error bars or significance tests anywhere. For a paper whose conclusions depend on small differences (e.g., N=4 on Re10K), this matters.\n\n3. No code or weights. For a method with several interacting components, reproducibility is not just a courtesy.\n\n4. The framing is more confident than the numbers. \"Consistently strong visual quality\" is hard to square with being below SEVA on LLFF at N=3 (16.85 vs 19.48). The comparison is not entirely unfair to them—SEVA trains at F=21 and uses scale sweeping—but the abstract should reflect the mixed results.\n\nMinor point: the CaPE attention-score analysis in Fig. 3 is a nice touch, and citing EscherNet for CaPE is appropriate, not a problem.\n\nOverall, the core idea is likely correct and useful. The paper deserves a serious referee, not a desk reject. I would send it to review with a strong request for the drift ablation, per-frame degradation curves, and error bars. If the authors can show the noise-conditioning augmentation actually does something, this becomes a solid contribution.","headline":"Solid and genuinely novel autoregressive multi-view diffusion formulation, with the core flexibility claim backed by a clean causal-vs-noncausal ablation; but the long-rollout stability claim rests on an unablated drift-mitigation mechanism and the benchmark numbers are more mixed than the framing suggests.","tokens_in":14371,"tokens_out":1341,"would_cite":true,"duration_ms":14503,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A causal multi-view diffusion model, CausNVS, generates novel views sequentially from arbitrary input and output counts, using per-frame noise and relative camera pose encodings for stable long rollouts.","keywords":["autoregressive novel view synthesis","multi-view diffusion","causal masking","per-frame noise conditioning","relative camera pose encoding","KV caching","3D consistency","world models"],"falsifier":"Use the same training data and architecture but replace per-frame independent noise levels with a single shared noise level, then evaluate rollouts of length 32 and 64; if image quality and 3D consistency degrade no faster than in the full model, the claimed drift-mitigation mechanism is not doing the work.","tokens_in":13276,"feed_emoji":"🎥","tokens_out":7525,"duration_ms":66961,"temperature":0.7,"pith_summary":"Novel view synthesis is usually done by jointly denoising a fixed set of target views, which forces the model to know all future camera poses ahead of time and limits it to one input-output configuration. CausNVS turns this into an autoregressive problem: a single model trained on eight-frame sequences learns to emit views one at a time, conditioned only on the input views and whatever it has already generated. The paper claims this works for arbitrary numbers of input views and output views across diverse camera trajectories, and that rollouts remain visually coherent for sequence lengths several times the training horizon. If correct, the same checkpoint becomes a streaming-capable 3D prior that needs no retraining, no padding, and no recomputation of past attention when the camera moves.","feed_headline":"One model now synthesizes any number of 3D views, one by one","feed_subtitle":"Trained on 8-frame clips, it rolls out stable 3D-consistent views for trajectories up to 10 times that length.","key_machinery":"The load-bearing mechanism is the combination of frame-wise attention with causal masking inside a pretrained latent-diffusion backbone, per-frame independent noise levels during training so conditioning on partially noisy or previously generated views is in-distribution at inference, and CaPE, a parameter-free relative camera pose encoding that rotates query and key features from pairwise pose matrices. CaPE makes attention scores invariant to global coordinate shifts, which is what allows key-value caching and pose-aware sliding windows to remain valid as the reference frame changes. The causal mask also makes every prefix length a valid training configuration, so one model covers many different input-to-output settings.","core_discovery":"The central discovery is that multi-view diffusion can be made causal without giving up quality or flexibility. Training with causal masking across frame-wise attention tokens, independently sampled noise levels per frame, and pairwise-relative camera pose encodings yields a model that generates target views autoregressively, with each new view attending only to earlier frames. At inference, previously generated views are re-fed with small assigned noise levels, and attention is restricted to a sliding window of the nearest views in pose space while cached key-value states carry longer-range context. The authors report that this combination keeps generation stable for rollouts well beyond the training length, improves as more input views are supplied, and supports customized trajectories including revisiting viewpoints.","pith_inferences":["Because the causal formulation makes every prefix length a training configuration, applying the same recipe to video or action-conditioned world models, with pose plus control tokens in the frame-wise attention, would be a direct test of whether the benefits extend beyond camera-only synthesis.","The paper reports single-view numbers without the ground-truth scale sweeping used by some baselines; a matched evaluation would show whether the single-view gap is a protocol artifact or a genuine method limitation.","A direct ablation that varies the noise assigned to already-generated views while keeping the sliding window fixed would separate how much of the long-rollout stability comes from noise conditioning augmentation rather than from the windowing and caching mechanism."],"forward_implications":["A single CausNVS checkpoint covers a wide range of input and output view counts without retraining or padding, because causal teacher forcing exposes the model to every shorter prefix during training.","Autoregressive rollout stays stable for sequences roughly ten times the training horizon, with only mild degradation compared with shorter-sequence generation.","Adding more input views consistently improves image quality and 3D consistency, resolving single-view scale ambiguity from multi-view cues rather than from ground-truth scale sweeping.","Key-value caching in the frame-wise attention layers means inference cost grows with the window and the number of generated frames, not with the full history, making streaming generation feasible.","Non-causal models trained identically perform well only near their training sequence length and drift quickly when evaluated autoregressively, while the causal variant keeps consistent performance across configurations."],"supporting_citations":[{"why":"Supplies the per-frame noise-conditioning scheme and the causal scheduling idea the method adapts to views.","marker":"[5]"},{"why":"Introduces the relative camera pose encoding, CaPE, that keeps attention valid across coordinate shifts.","marker":"[17]"},{"why":"Provides the pretrained latent diffusion backbone and VAE that CausNVS finetunes for multi-view generation.","marker":"[27]"},{"why":"Represents the fixed-view multi-view diffusion line of work the method contrasts with and builds on.","marker":"[9]"},{"why":"Defines the flexible-NVS benchmark and fixed-size baseline against which CausNVS is evaluated.","marker":"[57]"},{"why":"Motivates assigning small noise to already-generated views as noise conditioning augmentation at inference.","marker":"[12]"},{"why":"Supplies the key-value caching mechanism used to reuse attention computations across generated frames.","marker":"[19]"},{"why":"Supplies one of the two scene-level datasets used for training and evaluation.","marker":"[58]"},{"why":"Supplies the other scene-level training dataset and the source of the long-rollout evaluation splits.","marker":"[20]"}],"fun_headline_variants":["Autoregressive diffusion rolls out 3D views one by one","Causal masking enables flexible 3D view sequences","Generate any 3D views in order with CausNVS","Sequential 3D views from a causal diffusion model","Beyond fixed views: autoregressive 3D synthesis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Long-rollout stability rests on the assumption that per-frame noise conditioning makes the model robust enough to condition on its own imperfect outputs; the paper uses this design but never isolates its contribution.","fun_headline_variants_meta":{"raw":{"variants":["Autoregressive diffusion rolls out 3D views one by one","Causal masking enables flexible 3D view sequences","Generate any 3D views in order with CausNVS","Sequential 3D views from a causal diffusion model","Beyond fixed views: autoregressive 3D synthesis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1337,"prompt_tokens":863,"completion_tokens":474,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":392}},"tokens_in":479,"tokens_out":474,"duration_ms":4631,"temperature":1.0,"reasoning_tokens":392,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:15:38.929649+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use the same training data and architecture but replace per-frame independent noise levels with a single shared noise level, then evaluate rollouts of length 32 and 64; if image quality and 3D consistency degrade no faster than in the full model, the claimed drift-mitigation mechanism is not doing the work.","supporting_citations":[{"cited_title":"Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems (NeurIPS), 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the per-frame noise-conditioning scheme and the causal scheduling idea the method adapts to views."},{"cited_title":"Eschernet: A generative model for scalable view synthesis","cited_arxiv_id":null,"evidence_quote":"Introduces the relative camera pose encoding, CaPE, that keeps attention valid across coordinate shifts."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained latent diffusion backbone and VAE that CausNVS finetunes for multi-view generation."},{"cited_title":"Srinivasan, Jonathan T","cited_arxiv_id":null,"evidence_quote":"Represents the fixed-view multi-view diffusion line of work the method contrasts with and builds on."},{"cited_title":"Cascaded diffusion models for high fidelity image generation.Journal of Machine Learning Research (JMLR), 2022","cited_arxiv_id":null,"evidence_quote":"Motivates assigning small noise to already-generated views as noise conditioning augmentation at inference."},{"cited_title":"Kv caching explained","cited_arxiv_id":null,"evidence_quote":"Supplies the key-value caching mechanism used to reuse attention computations across generated frames."},{"cited_title":"Stereo magnification: Learning view synthesis using multiplane images.Proceedings of SIGGRAPH, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies one of the two scene-level datasets used for training and evaluation."},{"cited_title":"Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision","cited_arxiv_id":null,"evidence_quote":"Supplies the other scene-level training dataset and the source of the long-rollout evaluation splits."}],"review_version":2}