Pith. sign in

REVIEW 2 major objections 5 minor 67 references

Online Neural Space Time Memory for Dynamic Novel View Synthesis

T0 review · 2 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's core claim is that online novel view synthesis can keep minute-long memory of occluded regions and run in real time by updating memory only periodically—once per second—while applying it on every frame.

desk verdict Real architectural step forward for online dynamic NVS, but the headline "minute-scale memory at real time" is quantified only in a 1:2 update regime, not the claimed 1:29 deployment regime. read the letter →

arxiv 2607.15271 v1 pith:BINEDFD6 submitted 2026-07-16 cs.CV cs.GRcs.LG

Baback Elmieh , Lynn Tsai , Zeman Li , Srinivas Kaza , Tiancheng Sun , Gabor Csapo , Ali Behrouz , Yuan Deng
show 3 more authors
Stephen Lombardi Steven M. Seitz Xuan Luo
This is my paper
classification cs.CVcs.GRcs.LG
keywords onlinenovelviewsynthesisdynamicscenestest-timetrainingneuralspace-timememorycachingcross-viewattentionocclusionrecallreal-timerendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Novel view synthesis from live multi-view video faces a trade-off: to render a correct image when part of the scene is occluded, the system must remember what it saw earlier, but gradient-based test-time training on every frame is too slow for real time. This paper argues that the two operations do not need to happen at the same rate. Its method, Neural Space-Time Memory (NSTM), updates an internal memory once per second while synthesizing new views at 30 frames per second, using cross-view attention to reconcile the older memory with the current input. On a forced-occlusion test over 60 timesteps (roughly a minute of subsampled video), NSTM stays above 20 dB masked PSNR while a per-frame test-time-training baseline drops about 5.5 dB. If the result transfers to true 30 FPS streams, persistent long-horizon memory becomes a practical ingredient for live telepresence and free-viewpoint broadcast.

What carries the argument

The load-bearing mechanism is the decoupling of memorization frequency (set to 1 FPS) from synthesis frequency (30 FPS). Memorization is a fast-weight test-time-training update—one gradient step of an L2 inner loss over the incoming frames—while synthesis is a lightweight query against the frozen memory, made pose-aware by cross-view attention between current inputs and target rays. Two stabilizers complete the design: an auxiliary memory loss that forces the memory, queried in isolation from current inputs, to reconstruct the scene from history alone, and memory caching, which recursively averages historical weight checkpoints so active weights do not drift out of distribution. The L2 inner

What would settle it

Run the same forced-occlusion Memory Stress Test on raw 30 FPS multi-view video (with memorization every 30 frames) and measure masked PSNR at 60 seconds; if it drops below ~20 dB or the gap over per-frame TTT baselines narrows, the claim of minute-scale real-time memory does not transfer to natural frame rates. Presenting a distinctive pattern only on a non-memorization frame and asking for recall 60 steps later would test whether the periodic schedule can capture events that occur between updates.

Watch

Extended reading notes

Core claim

The central discovery is that frequency-decoupled test-time training—heavy memory updates on a slow periodic schedule, lightweight queries on every frame—can give an online dynamic novel view synthesis system both real-time throughput and minute-long recall of occluded content. NSTM accomplishes this with three coupled mechanisms: cross-view attention that fuses current motion with the stored memory state; an auxiliary memory loss with isolated readout tokens that prevents the model from bypassing memory during training; and recursive memory caching over historical weight checkpoints, paired with an L2 inner objective, to keep fast weights stable over long rollouts. The result is constant pe

Load-bearing premise

The paper's central claim assumes that results on 1.2 FPS subsampled sequences, where each timestep spans ~0.83 s and updates occur every two frames, carry over to true 30 FPS streams with a 29:1 synthesis-to-memorization ratio and hence 25× larger inter-update motion.

Editorial extensions

If this is right

  • Online view synthesis no longer requires per-frame gradient updates; a 1:29 memorization-to-synthesis schedule keeps per-frame cost at the level of memory application (about 27 ms at 256×256) independent of stream length.
  • Because per-frame cost is O(1) in history length, sustained live streaming does not accumulate latency the way full-history attention does, removing a key scalability barrier for long free-viewpoint sessions.
  • The L2 inner loss and memory caching together prevent the drift that plagues per-frame TTT baselines over long horizons, preserving the identity of occluded regions (e.g., back-of-subject patterns) beyond 50 seconds.
  • Training-time 1:1 alternating supervision generalizes to inference-time 29:1 schedules without additional training, because each synthesis step between updates reads the same frozen memory independently.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same frequency-decoupling principle could apply to other online reconstruction tasks with asymmetric update/apply costs and high temporal redundancy, such as streaming depth estimation or robot perception, where throttling expensive updates while querying at sensor rate might yield similar gains.
  • A direct testable extension is to run the forced-occlusion protocol on raw 30 FPS video rather than the 1.2 FPS subsampled sequences; if masked PSNR at minute scale degrades when inter-update motion is roughly 25× larger, the claim would hold only for low-frame-rate or slow-motion streams.
  • The acknowledged 'missed events' failure suggests a natural benchmark: insert a brief, distinctive cue only between two memorization steps and measure whether the network recovers it later; this would quantify how much scene history the periodic schedule actually captures, rather than assuming all history is internalized.
  • Future work could learn the memorization schedule itself, triggering expensive updates adaptively when inter-frame change exceeds a threshold, rather than fixing a constant 1 FPS, potentially improving both recall and efficiency.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents NSTM, an online dynamic novel-view-synthesis method that decouples the frequency of test-time-training memory updates from per-frame synthesis. Memory is updated periodically (target 1 FPS) while synthesis is applied per frame (target 30 FPS) using cross-view attention to fuse current inputs with the stored fast-weight memory. An auxiliary memory loss supervises isolated memory readouts, memory caching aggregates historical fast-weight states, and an L2 inner loss replaces the dot-product inner loss to prevent magnitude drift. Experiments on MVHumanNet++ (filtered to 390 60-frame sequences subsampled at 1.2 FPS) show that NSTM maintains mPSNR above 20 dB over 60 timesteps in a forced-occlusion memory stress test, outperforming LVSM, LaCT-NVS, and Token-Mem; ablations support the role of each component. The headline contribution is amortized real-time operation at 28.1 ms/frame from a 1 FPS memorization / 30 FPS synthesis schedule.

Significance. If the claims hold, the paper makes a useful contribution: the decoupled update/apply schedule is a simple but plausible idea, and the ablations provide credible causal evidence for the L2 inner loss, memory caching, and the auxiliary memory loss. The forced-occlusion stress test is a well-designed protocol for isolating memory persistence. The main weakness is that the quantitative evidence for persistent memory is obtained at a 1:2 memorization/synthesis ratio on 1.2 FPS subsampled sequences, while the real-time claim is made for a 1:29 ratio at 30 FPS; the paper's independence argument is plausible but not quantitatively validated. The work is of interest to the streaming-reconstruction and dynamic-NVS communities, but the headline claims need stronger direct support.

major comments (2)
  1. [Sec 4.1, C.5, Tab. 1 vs Tab. 5] The central quantitative claim of amortized real-time with minute-scale memory is not evaluated in the claimed regime. All memory benchmarks use MVHumanNet++ sequences subsampled to 1.2 FPS with a 1:2 memorization/synthesis ratio, giving a maximum memory age of ~0.83 s and only one independent synthesis step per memory state. The deployment claim is 1 FPS memorization / 30 FPS synthesis (1:29 ratio), with a maximum memory age of ~1.0 s and 29 synthesis steps per memory state. The paper asserts in Sec 3.2 that the synthesis steps are 'completely independent of one another,' but this is only demonstrated in supplemental videos, not quantitatively. The statement in Sec 4.1 that the subsampled tests 'accurately reflect true performance' on high-FPS videos is exactly the assumption that needs testing. Please provide either (a) quantitative memory metrics on raw 30 FPS video under the actual 1
  2. [Sec 4.4, Tab. 5] The real-time claim is expressed as an amortized average latency of 28.1 ms/frame, computed as (58.14 + 29×27.01)/30. The memorization step alone takes 58.14 ms, which exceeds the 33 ms frame interval at 30 FPS. The paper does not describe how this step is scheduled without frame drops or unacceptable latency — e.g., via an asynchronous pipeline, multiple GPUs, or output buffering — nor does it report the end-to-end latency of frames around a memorization step. If the memorization step is synchronous, the system cannot maintain a 30 FPS frame cadence. Please state the operational definition of 'real-time' and provide a latency distribution or a detailed scheduling diagram demonstrating that the 58 ms update does not block the frame pipeline.
minor comments (5)
  1. [Sec 3.3] The recursion for memory caching states '∀n > 1'; it should be '∀n ≥ 1' for the base case to make sense.
  2. [Table 2 caption] The citation for MVHumanNet++ is inconsistent: the main text and Sec 4.1 cite [28], but Table 2's caption and Sec C.4 cite [55]. Please unify.
  3. [Table 3 caption] There is a duplicated phrase: 'outperforming Token-Mem and LaCT-NVS by and LaCT-NVS by 1.74 dB...' should be 'by 1.74 dB and 2.95 dB, respectively'.
  4. [Fig. 4] The legend in Fig. 4 says 'Token-Attn' while the text and other figures use 'Token-Mem'. Please make the naming consistent.
  5. [Sec 4.1] The filtering to 390 test scenes with at least 60 frames is mentioned only in the supplementary material (Sec C.4). Since this changes the evaluation set relative to the standard split, it should be stated in the main paper's protocol description.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed components are independently ablated and evaluated on held-out data; remaining concerns are external-validity gaps, not definitional reductions.

full rationale

The derivation chain is not circular. The claimed components—periodic memorization, per-frame synthesis with cross-view attention, the auxiliary memory loss, memory caching, and the L2 inner loss—are each defined independently, and their causal contributions are directly tested by ablations in Table 2 (removing L_mem lowers PSNR from 30.09 to 28.23; removing caching lowers it to 29.24; applying caching to the coupled LaCT baseline fails with 20.25 PSNR). Baselines are external (LVSM, LaCT-NVS, CUT3R-derived Token-Mem), and evaluations use a held-out 390-scene subset of MVHumanNet++ with a standard train/test split. The auxiliary memory loss is a training signal, not a test-set fit, and the paper does not rename a fitted parameter as a prediction. Self-citations for memory caching and L2 objectives (e.g., [3,6,7]) are not load-bearing because the present paper supplies its own ablation evidence for these choices. The acknowledged gaps—1.2 FPS subsampling in Sec. 4.1 and C.5, the untested 29:1 inference schedule, and the missed-events limitation in Sec. 5 / Fig. 8—are external-validity or generalization concerns about whether minute-scale performance transfers to true 30 FPS streams, not cases where a reported result reduces to an input by construction.

Assumptions & free parameters 6 free parameters · 6 assumptions · 3 invented entities

The paper adds a handful of hand-scheduled hyperparameters (update period, loss weights, K unspecified) rather than a fitted physical parameter. The TTT and cross-attention machinery are architectural assumptions from the prior literature, and the memory token plus the averaged fast-weight state are new modeling entities whose support is entirely internal to the paper's ablations.

free parameters (6)
  • lambda_lpips = 0.5
    Perceptual-loss weight in both memory and synthesis losses (Eq. 5, Sec. C.2). Chosen by hand, standard practice, but it affects the trained behavior and is not swept.
  • memory supervision weight lambda_mem = appears as 1.0 in the alternating schedule (Eq. 7)
    The alternating 1:1 training ratio effectively sets the relative strength of Lmem and Lsynth to 1, rather than learning or tuning it (Sec. 3.2).
  • memorization period during training = every 2 timesteps
    The frequency ratio between memory updates and synthesis is a key architectural hyperparameter, set to 1:2 during training and then extrapolated to 1:29 at inference (Sec. 3.1, C.5). Its value determines the system's compute and motion-misalignment handling and is not swept.
  • memory caching stride K = not stated
    The caching formula Sec. 3.3 depends on K, the spacing of snapshots aggregated. The value used in experiments is not reported, so part of the empirical recipe is underspecified.
  • number of transformer blocks L = 24
    Model capacity hyperparameter chosen for the architecture (Sec. C.6); standard scale selection, reported but not justified.
  • learned memory-attention balance alpha = learned per layer (value not reported)
    A learned scale balancing cross-view attention and TTT memory contributions (Sec. C.6, Fig. 7). Learned from data, and part of the model's capacity rather than a free parameter fit to the evaluation metric.
assumptions (6)
  • domain assumption TTT fast-weight updates define a valid compressive memory of the scene
    The whole method relies on the TTT premise (Sec. 3, Eq. 2-4) that gradient updates of a fast-weight MLP on inner-loss binding keys to values internalize scene context; no convergence or capacity bound is given.
  • domain assumption Cross-view attention over input tokens and target rays can resolve deformation between memory state and current pose
    The synthesis step (Sec. 3.1) assumes transformer attention implicitly handles the motion misalignment between the periodic memory and the current frame; the ablation (Tab. 2, caching-fails-on-coupled) supports this indirectly but it is not proven.
  • ad hoc to paper The alternating 1:1 memory/synthesis training supervision transfers to the 1:29 inference ratio
    Sec. 3.2 asserts the synthesis steps between memorization steps are independent and read the same frozen memory, so the ratio transfer is valid. This is plausible but an assumption about generalization across the train/inference ratio gap.
  • domain assumption MVHumanNet++ foregrounds, backgrounds, and multi-view calibration are accurate enough that alpha-compositing and masked metrics measure the subject's identity
    All memory-retention claims use either masked metrics or RGB-alpha compositing against random backgrounds (Sec. C.7); any matte or calibration error leaks into the reported numbers.
  • domain assumption Natural disocclusion statistics of the 390-scene subset represent the target application
    Only 17 of 390 scenes contain full rotations, so the paper constructs a forced-occlusion stress test (Sec. 4.1) and evaluates the natural-rotation protocol on only 17 scenes; whether this generalizes to live telepresence scenarios is assumed.
  • domain assumption L2 inner loss keeps fast-weight magnitudes bounded over long horizons
    The paper claims L2 avoids magnitude inflation of the dot-product inner loss (Sec. 1, Tab. 2). Empirically supported by the ablation against LaCT-NVS, and supplemented by Newton-Schulz orthonormalization and L2 weight normalization, but no theoretical guarantee is given for the 60+ step regime.
invented entities (3)
  • Memory token
    purpose: Isolated target-ray tokens that self-attend only among themselves and query the memory, forcing pure memory readout for the auxiliary memory loss (Sec. 3.2, Fig. 2).
    Introduced purely as an architectural training device; its only measurable effect is the ablation gain of 1.01 PSNR when Lmem is removed.
  • Aggregated memory state W_hat
    purpose: Averaged checkpoint of fast-weight states used to stabilize long-horizon updates (Sec. 3.3).
    Adapted from the memory-caching literature, applied here to 2D multi-view video; its observable effect is the +0.85 PSNR ablation gain.
  • Learned layer scale alpha
    purpose: Balances the cross-view attention branch and the TTT memory branch in each layer (Fig. 7).
    A standard learned architectural scalar; no external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Online Neural Space Time Memory for Dynamic Novel View Synthesis." pith.science (2026). https://pith.science/paper/BINEDFD6

@misc{pith2026260715271,
  author       = {Pith},
  title        = {Pith review of: Online Neural Space Time Memory for Dynamic Novel View Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BINEDFD6}},
  note         = {Machine review of arXiv:2607.15271}
}
read the original abstract

Online novel view synthesis from multi-view streaming videos faces a fundamental trade-off: maintaining a persistent, long-horizon memory to reconstruct temporarily occluded regions while operating under strict real-time constraints. While Test-Time Training (TTT) offers a powerful memory mechanism, standard models mandate gradient-based memory updates at every frame to adapt to the changing motion in dynamic scenes. The computational cost of heavy memory updates precludes real-time application and can lead to instability over long contexts. Given that memory updates are more demanding than memory application and video content is largely redundant, we propose to decouple the frequencies of these two processes. Our approach performs periodic memory updates while applying the memory on a per-frame basis, using cross-view attention to manage deformations between the prior memory state and the current frame. To lock in the historical context, we introduce two critical mechanisms: an auxiliary Memory Loss that forces persistent internalization of the scene, and a Memory Caching strategy that regularizes active weights against catastrophic drift. Our method demonstrates real-time, state-of-the-art performance on scenes with dynamic human motion as well as minute-scale online memorization.

Figures

Figures reproduced from arXiv: 2607.15271 by the authors.

Figure 1
Figure 1. Neural Space-Time Memory (NSTM) synthesizes novel views in amortized real-time from 30 FPS multi-view videos, while updating its memory at 1 FPS. Middle: NSTM captures the back logo as the subject rotates (see memory readout), faithfully recalling it later from memory given only frontal views. Right: While full self-attention in typical feed-forward models (e.g., LVSM [22]) scales quadratically with history sequence… view at source ↗
Figure 2
Figure 2. NSTM Training Scheme. Left: Our alternating training scheme explicitly enforces memorization. In the Memory Supervision step, isolated memory tokens (target camera rays) perform strict self-attention —- sharing weights with the cross-view attention layer — without attending to input views. Once the memory is updated by the inputs, these isolated tokens query the memory to reconstruct the target view. This forces a p… view at source ↗
Figure 3
Figure 3. Memory Stress Test at T = 60. Despite only observing the back at T = 0, NSTM accurately recalls the hoodie patterns and hairstyles while fusing with current motion. LVSM makes its best stateless guess in unseen regions, LaCT-NVS suffers from out-of￾distribution memory drift, and Token-Mem fails to recall accurate back views. See our supplementary website for the video results. is an RGB image and Pt,i ∈ SE(3) is the… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Memory Stress Test Over Time. We show the back view at T=0. For subsequent timestamps, we show only frontal views and task the model with synthesizing the occluded back. NSTM maintains high-fidelity recall over time while baseline models suffer from drift or collapse t…
Figure 5
Figure 5. Figure 5: Qualitative ablation study. At T = 60, each model synthesizes the back view that has been occluded after T = 0. LaCT-NVS demonstrates the instability of the inner dot-product loss. In contrast, LaCT-NVS w/ L2 greatly improves stability through our L2 inner loss choice,…
Figure 6
Figure 6. Figure 6: Memory from Natural Rotations Qualitative Results. We evaluate our method against the baselines on sequences feature natural rotations that reveal the back of the subject, which the models must synthesize. Left column shows at which timestep the back is revealed. NSTM …
Figure 7
Figure 7. Figure 7: Online Inference Pipeline and NSTM Layer Design. Left: During inference, we run the memorization step periodically while performing the synthesis step per-frame, which enables us to achieve amortized real-time speed. Right: We introduce the NSTM Layer consisting of cro…
Figure 8
Figure 8. Figure 8: Limitations. Left: Update Frequency and Missed Events. Our periodic updates may miss events between memorization steps. The pattern shown at t = 1 was presented only at this non-update timestep and is therefore not recoverable at t = 60. Right: Memory Primacy Bias Our …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 11 linked inside Pith

  1. [1]

    Neural point-based graph- ics

    Kara-Ali Aliev, Artem Sevastopolsky, Maria Kolos, Dmitry Ulyanov, and Victor Lempitsky. Neural point-based graph- ics. InEuropean conference on computer vision, pages 696–

  2. [2]

    Simple linear attention language models bal- ance the recall-throughput tradeoff

    Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, James Zou, Atri Rudra, and Christopher Re. Simple linear attention language models bal- ance the recall-throughput tradeoff. InInternational Confer- ence on Machine Learning, pages 1763–1840. PMLR, 2024. 3

    2024

  3. [3]

    Atlas: Learning to optimally memorize the con- text at test time, 2025

    Ali Behrouz, Zeman Li, Praneeth Kacham, Majid Daliri, Yuan Deng, Peilin Zhong, Meisam Razaviyayn, and Vahab Mirrokni. Atlas: Learning to optimally memorize the con- text at test time, 2025. 2, 3, 4

    2025

  4. [4]

    Nested learning: The illusion of deep learning ar- chitectures

    Ali Behrouz, Meisam Razaviyayn, Peilin Zhong, and Vahab Mirrokni. Nested learning: The illusion of deep learning ar- chitectures. InThe Thirty-ninth Annual Conference on Neu- ral Information Processing Systems, 2025. 3

    2025

  5. [5]

    Titans: Learning to memorize at test time

    Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time. InThe Thirty-ninth An- nual Conference on Neural Information Processing Systems,

  6. [6]

    Memory caching: Rnns with growing memory, 2026

    Ali Behrouz, Zeman Li, Yuan Deng, Peilin Zhong, Meisam Razaviyayn, and Vahab Mirrokni. Memory caching: Rnns with growing memory, 2026. 2, 3, 6

    2026

  7. [7]

    It’s all connected: A journey through test-time memorization, attentional bias, retention, and online opti- mization

    Ali Behrouz, Meisam Razaviyayn, Peilin Zhong, and Vahab Mirrokni. It’s all connected: A journey through test-time memorization, attentional bias, retention, and online opti- mization. InThe Fourteenth International Conference on Learning Representations, 2026. 2, 3, 4, 6

    2026

  8. [8]

    Immersive light field video with a layered mesh representation.ACM Trans- actions on Graphics (TOG), 39(4):86–1, 2020

    Michael Broxton, John Flynn, Ryan Overbeck, Daniel Erick- son, Peter Hedman, Matthew Duvall, Jason Dourgarian, Jay Busch, Matt Whalen, and Paul Debevec. Immersive light field video with a layered mesh representation.ACM Trans- actions on Graphics (TOG), 39(4):86–1, 2020. 2

    2020

Show all 67 references
  1. [9]

    Hexplane: A fast representa- tion for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representa- tion for dynamic scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2023. 3

    2023

  2. [10]

    Simoncelli

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unifying structure and texture similarity.CoRR, abs/2004.07728, 2020. 12

    2004 arXiv

  3. [11]

    Hymba: A hybrid-head architecture for small language mod- els

    Xin Dong, Yonggan Fu, Shizhe Diao, Wonmin Byeon, ZIJIA CHEN, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Min-Hung Chen, Yoshi Suhara, Yingyan Celine Lin, et al. Hymba: A hybrid-head architecture for small language mod- els. InThe Thirteenth International Conference on Learnin...

    2025

  4. [12]

    Deepstereo: Learning to predict new views from the world’s imagery

    John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. Deepstereo: Learning to predict new views from the world’s imagery. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5515–5524,

  5. [13]

    Deepview: View synthesis with learned gra- dient descent

    John Flynn, Michael Broxton, Paul Debevec, Matthew Du- Vall, Graham Fyffe, Ryan Overbeck, Noah Snavely, and Richard Tucker. Deepview: View synthesis with learned gra- dient descent. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2367...

    2019

  6. [14]

    Quark: Real-time, high-resolution, and general neural view synthesis.ACM Transactions on Graphics (TOG), 43(6):1–20, 2024

    John Flynn, Michael Broxton, Lukas Murmann, Lucy Chai, Matthew DuVall, Cl ´ement Godard, Kathryn Heal, Srinivas Kaza, Stephen Lombardi, Xuan Luo, et al. Quark: Real-time, high-resolution, and general neural view synthesis.ACM Transactions on Graphics (TOG), 43(6):1–20, 2024. 2

    2024

  7. [15]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 12479–12488, 2023. 3

    2023

  8. [16]

    Log-linear attention.arXiv preprint arXiv:2506.04761, 2025

    Han Guo, Songlin Yang, Tarushii Goel, Eric P Xing, Tri Dao, and Yoon Kim. Log-linear attention.arXiv preprint arXiv:2506.04761, 2025. 3

    2025

  9. [17]

    Liq- uid structural state-space models

    Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang, Makram Chahine, Alexander Amini, and Daniela Rus. Liq- uid structural state-space models. InThe Eleventh Interna- tional Conference on Learning Representations, 2023. 3

    2023

  10. [18]

    Deep blending for free-viewpoint image-based rendering.ACM Transactions on Graphics (ToG), 37(6):1–15, 2018

    Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. Deep blending for free-viewpoint image-based rendering.ACM Transactions on Graphics (ToG), 37(6):1–15, 2018. 2

    2018

  11. [19]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  12. [20]

    2d gaussian splatting for geometrically ac- curate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically ac- curate radiance fields. InACM SIGGRAPH 2024 conference papers, pages 1–11, 2024. 2

    2024

  13. [21]

    Neural differential equations for learning to program neural nets through continuous learning rules.Advances in Neural Information Processing Systems, 35:38614–38628, 2022

    Kazuki Irie, Francesco Faccio, and J ¨urgen Schmidhuber. Neural differential equations for learning to program neural nets through continuous learning rules.Advances in Neural Information Processing Systems, 35:38614–38628, 2022. 3

    2022

  14. [22]

    Lvsm: A large view synthesis model with minimal 3d inductive bias

    Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. InThe Thirteenth International Conference on Learning Representations, 2025. 1, 3, 4, 7, 8, 12, 14, 15

    2025

  15. [23]

    Barron, Noah Snavely, and Aleksander Holyn- ski

    Haian Jin, Rundi Wu, Tianyuan Zhang, Ruiqi Gao, Jonathan T. Barron, Noah Snavely, and Aleksander Holyn- ski. ZipMap: Linear-time stateful 3d reconstruction via test- 9 time training. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 2, 5

    2026

  16. [24]

    Muon: An optimizer for hidden layers in neural networks, 2024

    Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. 4

    2024

  17. [25]

    Polysketchformer: Fast transformers via sketching poly- nomial kernels

    Praneeth Kacham, Vahab Mirrokni, and Peilin Zhong. Polysketchformer: Fast transformers via sketching poly- nomial kernels. InInternational Conference on Machine Learning, pages 22748–22770. PMLR, 2024. 3

    2024

  18. [26]

    Transformers are rnns: Fast autoregressive transformers with linear attention

    Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Franc ¸ois Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. InInternational confer- ence on machine learning, pages 5156–5165. PMLR, 2020. 3

    2020

  19. [27]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, George Drettakis, et al. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1,

  20. [28]

    Mvhumannet++: A large-scale dataset of multi-view daily dressing human captures with richer annotations for 3d human digitization.arXiv preprint arXiv:2505.01838, 2025

    Chenghong Li, Hongjie Liao, Yihao Zhi, Xihe Yang, Zheng- wentai Sun, Jiahao Chang, Shuguang Cui, and Xiaoguang Han. Mvhumannet++: A large-scale dataset of multi-view daily dressing human captures with richer annotations for 3d human digitization.arXiv preprint arXiv:2505.01838...

    2025 arXiv

  21. [29]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF conference on computer vi- si...

    2022

  22. [30]

    Spacetime gaus- sian feature splatting for real-time dynamic view synthesis

    Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaus- sian feature splatting for real-time dynamic view synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8508–8520, 2024. 3

    2024

  23. [31]

    Efficient neural radiance fields for interactive free-viewpoint video

    Haotong Lin, Sida Peng, Zhen Xu, Yunzhi Yan, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Efficient neural radiance fields for interactive free-viewpoint video. InSIGGRAPH Asia 2022 conference papers, pages 1–9, 2022. 2

    2022

  24. [32]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. InProceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 3

    2023

  25. [33]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age.arXiv preprint arXiv:2309.03453, 2023

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age.arXiv preprint arXiv:2309.03453, 2023. 3

    2023 arXiv

  26. [34]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 2

    2021

  27. [35]

    Wind, Tianyi Wu, Daniel Wut- tke, and Christian Zhou-Zheng

    Bo Peng, Ruichong Zhang, Daniel Goldstein, Eric Alcaide, Xingjian Du, Haowen Hou, Jiaju Lin, Jiaxing Liu, Janna Lu, William Merrill, Guangyu Song, Kaifeng Tan, Saiteja Ut- pala, Nathan Wilce, Johan S. Wind, Tianyi Wu, Daniel Wut- tke, and Christian Zhou-Zheng. RWKV-7 ”goose” w...

    2025

  28. [36]

    Free view synthesis

    Gernot Riegler and Vladlen Koltun. Free view synthesis. InEuropean conference on computer vision, pages 623–640. Springer, 2020. 2

    2020

  29. [37]

    Scene representation transformer: Geometry-free novel view syn- thesis through set-latent scene representations

    Mehdi SM Sajjadi, Henning Meyer, Etienne Pot, Urs Bergmann, Klaus Greff, Noha Radwan, Suhani V ora, Mario Luˇci´c, Daniel Duckworth, Alexey Dosovitskiy, et al. Scene representation transformer: Geometry-free novel view syn- thesis through set-latent scene representations. InPr...

    2022

  30. [38]

    Linear transformers are secretly fast weight programmers

    Imanol Schlag, Kazuki Irie, and J ¨urgen Schmidhuber. Linear transformers are secretly fast weight programmers. InInter- national conference on machine learning, pages 9355–9366. PMLR, 2021. 3

    2021

  31. [39]

    Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023. 3

    2023 arXiv

  32. [40]

    Scene representation networks: Continuous 3d- structure-aware neural scene representations.Advances in neural information processing systems, 32, 2019

    Vincent Sitzmann, Michael Zollh ¨ofer, and Gordon Wet- zstein. Scene representation networks: Continuous 3d- structure-aware neural scene representations.Advances in neural information processing systems, 32, 2019. 2

    2019

  33. [41]

    3dgstream: On-the-fly training of 3d gaussians for efficient streaming of photo-realistic free- viewpoint videos

    Jiakai Sun, Han Jiao, Guangyuan Li, Zhanjie Zhang, Lei Zhao, and Wei Xing. 3dgstream: On-the-fly training of 3d gaussians for efficient streaming of photo-realistic free- viewpoint videos. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, p...

    2024

  34. [42]

    Retentive network: A successor to transformer for large language mod- els.arXiv preprint arXiv:2307.08621, 2023

    Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language mod- els.arXiv preprint arXiv:2307.08621, 2023. 3

    2023 arXiv

  35. [43]

    Learning to (learn at test time): Rnns with expressive hidden states, 2025

    Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. Learning to (learn at test time): Rnns with expressive hidden states, 2025. 3, 4

    2025

  36. [44]

    De- ferred neural rendering: Image synthesis using neural tex- tures.Acm Transactions on Graphics (TOG), 38(4):1–12,

    Justus Thies, Michael Zollh ¨ofer, and Matthias Nießner. De- ferred neural rendering: Image synthesis using neural tex- tures.Acm Transactions on Graphics (TOG), 38(4):1–12,

  37. [45]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 3

    2017

  38. [46]

    Vggt: Vi- sual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Vi- sual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025. 2

    2025

  39. [47]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021. 2

    2021 arXiv

  40. [48]

    Ibr- net: Learning multi-view image-based rendering

    Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul P Srinivasan, Howard Zhou, Jonathan T Barron, Ricardo 10 Martin-Brualla, Noah Snavely, and Thomas Funkhouser. Ibr- net: Learning multi-view image-based rendering. InPro- ceedings of the IEEE/CVF conference on computer vision an...

    2021

  41. [49]

    Efros, and Angjoo Kanazawa

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3D Perception Model with Persistent State . In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10510–10522, Los Alamitos, CA, USA,

  42. [50]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20697– 20709, 2024. 2

    2024

  43. [51]

    Neus2: Fast learning of neural implicit surfaces for multi-view recon- struction

    Yiming Wang, Qin Han, Marc Habermann, Kostas Dani- ilidis, Christian Theobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view recon- struction. InProceedings of the IEEE/CVF international conference on computer vision, pages 3295–3306, 2023. 2

    2023

  44. [52]

    Learning efficient fuse-and-refine for feed-forward 3d gaussian splatting.arXiv preprint arXiv:2503.14698, 2025

    Yiming Wang, Lucy Chai, Xuan Luo, Michael Niemeyer, Manuel Lagunas, Stephen Lombardi, Siyu Tang, and Tiancheng Sun. Learning efficient fuse-and-refine for feed-forward 3d gaussian splatting.arXiv preprint arXiv:2503.14698, 2025. 3

    2025

  45. [53]

    Novel view synthesis with diffusion models.arXiv preprint arXiv:2210.04628, 2022

    Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, and Mohammad Norouzi. Novel view synthesis with diffusion models.arXiv preprint arXiv:2210.04628, 2022. 3

    2022 arXiv

  46. [54]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 20310–20320...

    2024

  47. [55]

    Mvhumannet: A large-scale dataset of multi-view daily dressing human captures

    Zhangyang Xiong, Chenghong Li, Kenkun Liu, Hongjie Liao, Jianqiao Hu, Junyi Zhu, Shuliang Ning, Lingteng Qiu, Chongjie Wang, Shijie Wang, Shuguang Cui, and Xiaoguang Han. Mvhumannet: A large-scale dataset of multi-view daily dressing human captures. InProceedings of the IEEE/C...

    2024

  48. [56]

    Point- nerf: Point-based neural radiance fields

    Qiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi, Zhixin Shu, Kalyan Sunkavalli, and Ulrich Neumann. Point- nerf: Point-based neural radiance fields. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5438–5448, 2022. 2

    2022

  49. [57]

    4k4d: Real-time 4d view synthesis at 4k resolution

    Zhen Xu, Sida Peng, Haotong Lin, Guangzhao He, Jiaming Sun, Yujun Shen, Hujun Bao, and Xiaowei Zhou. 4k4d: Real-time 4d view synthesis at 4k resolution. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20029–20040, 2024. 3

    2024

  50. [58]

    4dgt: Learning a 4d gaus- sian transformer using real-world monocular videos.arXiv preprint arXiv:2506.08015, 2025

    Zhen Xu, Zhengqin Li, Zhao Dong, Xiaowei Zhou, Richard Newcombe, and Zhaoyang Lv. 4dgt: Learning a 4d gaus- sian transformer using real-world monocular videos.arXiv preprint arXiv:2506.08015, 2025. 2

    2025

  51. [59]

    Instant gaussian stream: Fast and generalizable streaming of dy- namic scene reconstruction via gaussian splatting

    Jinbo Yan, Rui Peng, Zhiyan Wang, Luyang Tang, Jiayu Yang, Jie Liang, Jiahao Wu, and Ronggang Wang. Instant gaussian stream: Fast and generalizable streaming of dy- namic scene reconstruction via gaussian splatting. InPro- ceedings of the Computer Vision and Pattern Recognitio...

    2025

  52. [60]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20331–20341, 2024. 3

    2024

  53. [61]

    Multiview neu- ral surface reconstruction by disentangling geometry and ap- pearance.Advances in Neural Information Processing Sys- tems, 33:2492–2502, 2020

    Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lipman. Multiview neu- ral surface reconstruction by disentangling geometry and ap- pearance.Advances in Neural Information Processing Sys- tems, 33:2492–2502, 2020. 2

    2020

  54. [62]

    V ol- ume rendering of neural implicit surfaces.Advances in neu- ral information processing systems, 34:4805–4815, 2021

    Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. V ol- ume rendering of neural implicit surfaces.Advances in neu- ral information processing systems, 34:4805–4815, 2021. 2

    2021

  55. [63]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024. 3

    2024 arXiv

  56. [64]

    Loger: Long-context geometric reconstruction with hybrid memory.arXiv preprint arXiv:2603.03269,

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Chen Sun, Ming-Hsuan Yang, Forrester Cole, Trevor Darrell, and De- qing Sun. Loger: Long-context geometric reconstruction with hybrid memory.arXiv preprint arXiv:2603.03269,

    arXiv

  57. [65]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric.CoRR, abs/1801.03924,

    arXiv

  58. [66]

    Memory Stress Test

    Tianyuan Zhang, Sai Bi, Yicong Hong, Kai Zhang, Fujun Luan, Songlin Yang, Kalyan Sunkavalli, William T Freeman, and Hao Tan. Test-time training done right.arXiv preprint arXiv:2505.23884, 2025. 2, 3, 4, 5, 7, 12, 14 A. Supplementary Material Our Supplementary Material presents...

    2025 arXiv

  59. [2025]

    7, 12, 14

    IEEE Computer Society. 7, 12, 14

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.