Pith. sign in

REVIEW 2 major objections 6 minor 50 references

Efficient Neural Video Representation with Temporally Coherent Modulation

T0 review · 2 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read NVTM represents a video as flow-aligned 2D grids, letting temporally matching pixels share one modulation code for faster, more parameter-efficient encoding.

desk verdict Real contribution on dynamic video INR; speed claim needs a caveat about RAFT precomputation and Eq. (3) has a minor formal gap. read the letter →

arxiv 2505.00335 v1 pith:JRADNJOW submitted 2025-05-01 cs.CV cs.AI

Seungjun Shin , Suji Kim , Dokwan Oh This is my paper · ORCID
classification cs.CVcs.AI
keywords implicitneuralrepresentationvideocompressionparametricencodingtemporallycoherentmodulationopticalflowalignmentreconstructioninpaintingframeinterpolation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes NVTM, an implicit neural video representation built from a small stack of 2D latent grids plus a learned alignment flow. The flow maps pixels that correspond to the same scene content across frames onto the same grid location, so each group of temporally matching pixels receives one shared modulation code instead of redundant per-frame parameters. This targets two weaknesses of earlier video INRs at once: grid-type methods train fast but waste parameters because they ignore motion, while frame-wise NeRV-style methods are parameter-efficient but slow to fit. On dynamic HD sequences, the paper reports that NVTM reaches the same reconstruction quality about three times faster than NeRV-style methods, and that it beats the grid-based NVP by 1.54 dB PSNR on UVG (Dynamic) with 10% fewer parameters, and by 1.84 dB on MCL-JCV (Dynamic). The consequence is that neural video representation does not have to choose between fast encoding and parameter efficiency.

What carries the argument

The central object is the temporally coherent modulation latent: a single vector $z_{xyt}$ that modulates the base network and is shared by pixels the alignment flow judges to be the same scene content. The mechanism that creates it is the alignment flow network $F_H(t)$, a SIREN whose weights are produced by a hypernetwork over time; its output is scaled by $\log(t-t_k)$ to compensate for the temporal distance to the keyframe of a short group of frames (GOP). Equation (4), $(x_k,y_k) = (x,y) + \log(t-t_k)F_H(t)(x,y)$, warps each 3D coordinate to the keyframe's 2D coordinate, and Eq. (5) adaptively normalizes those coordinates before they look up the per-GOP 2D latent grid $G_k$. The grid lookup is what lets one latent serve many pixels; the adaptive normalization is what keeps grid cells from being wasted on sparse outlier regions.

What would settle it

Train NVTM on a synthetic dynamic video with known ground-truth optical flow, first with no occlusions and then with large occluded bands that force unrelated pixels into the same aligned grid cell. If the PSNR margin over NVP stays roughly 1.5 dB in both cases, the shared-latent mechanism is robust to broken correspondence; if the margin collapses when occlusions are added, the method's gain depends on flow accuracy exactly where the paper's weakest assumption says it does.

Watch

Extended reading notes

Core claim

On its own terms, NVTM establishes that the temporal redundancy in video can be removed from a parametric encoding by conditioning the coordinates, not by adding parameters. For each group of frames, an alignment flow network $F_H(t)$ predicts a displacement, and every 3D coordinate $(x,y,t)$ is mapped to a keyframe-time coordinate $(x_k,y_k) = (x,y) + \log(t-t_k) F_H(t)(x,y)$. The normalized coordinate indexes a 2D latent grid $G_k$, producing the modulation latent $z_{xyt} = G_k(x'_k,y'_k)$ that conditions the base network; because corresponding pixels land on the same grid cell, they share the latent. The reported consequences are a training-speed advantage of over 3 times versus NeRV-style methods and a parameter-efficiency advantage of 1.54 dB and 0.019 PSNR/LPIPS over NVP on UVG (Dynamic) with 10% fewer parameters, plus 1.84 dB and 0.013 on MCL-JCV (Dynamic). The same shared-latent mechanism is then shown to serve super-resolution, frame interpolation, and inpainting, and the GOP grid stack can be compressed with HEVC to reach bitrates competitive with standard codecs.

Load-bearing premise

The load-bearing premise is that the alignment flow correctly pairs the same scene point across frames; if it maps unrelated pixels to the same grid cell, one shared code encodes mismatched content, and the speed and parameter gains disappear.

Editorial extensions

If this is right

  • At the 5-minute mark on UVG (Dynamic), NVTM already exceeds 29 dB PSNR while every NeRV-style baseline is still below 25 dB at 0.1 bpp, showing that fast encoding does not require giving up quality.
  • Grid-type video INRs can be made parameter-efficient without losing their training speed: NVTM uses 10% fewer parameters than NVP and still gains 1.54 dB on UVG (Dynamic) and 1.84 dB on MCL-JCV (Dynamic) in PSNR.
  • Because decoding is coordinate-based, one trained model supports spatial and temporal resampling, which the paper demonstrates as 2x super-resolution and 2x frame interpolation with PSNR and LPIPS gains over NVP and 3D ModSIREN.
  • Video inpainting follows from the same shared-latent mechanism: masked areas are decoded from temporally corresponding visible pixels, as demonstrated on DAVIS2017 with random box masks.
  • The per-GOP latent grids form a temporally coherent sequence, so compressing them with HEVC yields bitrate-distortion points comparable to H.264/HEVC and prior INR codecs while keeping the fast-encoding advantage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • In principle, the same warped-coordinate shared-code mechanism should transfer to other parametric encodings such as hash grids or triplane features, where temporal redundancy is currently absorbed by adding more grid levels; this is an extension, not something the paper tests.
  • The paper's own GOP-size ablation suggests a motion-energy-based GOP scheduler could widen the gains on mixed-motion footage; the authors note the possibility but do not implement it.
  • On low-motion or occlusion-heavy footage, a hybrid that keeps a static grid for unreliable-correspondence regions could preserve the advantage; the paper's Beauty result marks the boundary where this would matter.
  • If the HEVC-on-grid-stack compression result generalizes, the latent grids are temporally coherent in a codec-friendly sense, which could inform future INR codec designs beyond this architecture.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes NVTM, an implicit neural representation for video that groups temporally corresponding pixels via a learned alignment flow guided by RAFT optical flow, so that a shared modulation latent from per-GOP 2D grids can modulate a base SIREN network. The method is evaluated on dynamic subsets of UVG and MCL-JCV, where it reports faster encoding times and higher reconstruction quality than NVP and NeRV-style baselines, plus results on video compression, super resolution, frame interpolation, and inpainting. The central claims are empirical: a speed advantage of over 3x versus NeRV-style methods and parameter-efficiency gains of about 1.5-1.8 dB over NVP on dynamic sequences.

Significance. If the empirical claims hold, NVTM is a meaningful advance in video INR: it is the first grid-type parametric encoding that explicitly removes temporal redundancy by coordinate alignment, achieving both faster training and better parameter efficiency on dynamic HD content than prior grid-based methods. The paper is unusually thorough: it includes ablations over GOP size, neighbor set, adaptive normalization, and static features; reports random-seed stability; provides sequence-level tables; and extends to multiple downstream tasks. There is no equation-level circularity, since the alignment network is trained against an external pretrained RAFT system and reconstruction is measured on held-out coordinates. The main reservations are that the headline speed comparison omits the cost of RAFT flow precomputation, and that the formal definition of the flow scaling in Eq. (3) is incomplete.

major comments (2)
  1. [Section 4.2 and Table 2] The speed comparison is defined as training time only (Table 2 caption: 'i.e., the training time'), but NVTM requires RAFT optical flow as auxiliary supervision for the alignment flow network (Section 3.2, Section 4.1). The paper never reports the wall-clock cost of computing RAFT flows on 600-frame HD UVG sequences or 100-frame MCL-JCV sequences, nor does it state whether such cost is included in the 'encoding time' values. If, as is typical, flow estimation takes a non-negligible fraction of the 1-minute or 5-minute training intervals, the abstract's 'fastest encoding speed' and 'speed increase of over 3 times' would not describe the end-to-end encoding pipeline. The paper should either include the RAFT precomputation time in the speed comparison and re-evaluate the 3x claim, or explicitly qualify all speed statements as training-time-only and justify that the excluded preprocessing is acceptable in the intended deployment scenario.
  2. [Section 3.2, Eq. (3)] Eq. (3), Flow_{t->tk}(x,y) = log(t - tk) F_H(t)(x,y), is undefined at the keyframe t = tk because log(0) is undefined. Since the keyframe is defined as the first frame of each GOP and every pixel in that frame has t = tk, the paper must specify how the alignment flow and the aligned coordinate (x_k, y_k) are computed for the keyframe (e.g., by setting flow to zero). Additionally, the domain of t is not stated: if t is a normalized time in [0,1], log(t-tk) is negative for frames immediately after the keyframe, reversing the sign of the learned flow; if t is an integer frame index, log(t-tk)=0 at t=tk+1, which would force zero flow for the adjacent frame and contradict the intended temporal alignment. This is a load-bearing formal gap in the method definition and needs clarification.
minor comments (6)
  1. [Section 5 and Section E.3] The conclusion's statement that 'NVTM outperforms existing methods of implicit neural video representation on various video related tasks' is too broad: Table 16 shows NVP outperforms NVTM on the Beauty sequence (36.24 vs 35.63 PSNR). The claim should be scoped to dynamic video sequences, in line with the datasets actually used.
  2. [Table 17] In the 'Output type' column, 'Pxiel' should be 'Pixel'.
  3. [Section 4.6] The sentence 'we analysis on zxyz corresponding to pixels' should be 'we analyze the modulation latent z_xyz corresponding to pixels.' Also, the notation z_xyz appears in the analysis but z_xyt is used in the methodology; this inconsistency should be fixed.
  4. [Figure 4(b)] The caption says 'random-valued flow in a notated scale of source video resolution'; it is unclear what 'notated scale' means and how the random flow was generated. Please clarify.
  5. [Eq. (5)] The Clip operator is not defined; it should state whether clipping is to the closed interval [0,1] and whether the value is clamped after normalization.
  6. [Various] There are several typos and grammatical issues, e.g., 'demostrate' in Section 2.2, 'a 10-' and 'a diminishing duration of the entire iterations' in Section B.3, and Figure 1's '1 minutes' should be '1 minute'.

Circularity Check

1 steps flagged · score 1.0 of 10

The central speed and quality claims are externally benchmarked; the only minor circularity is a self-consistency analysis that restates the construction.

  1. self definitional [This occurs in Section 4.6, 'Temporal Consistency Modulation' analysis, relying on Eq. (6) and the Figure 2 caption.]
    "We can observe that the latent values derived from similar pixel areas across different frames are represented as similar embeddings. These findings validate our intention that our network produces identical modulation latents from similar pixels in consecutive frames."

    By Eq. (6), the latent is zxyt = concat{G_{k+p}(x'_{k+p}, y'_{k+p}) | p in P}, where (x'_k, y'_k) are the aligned 2D coordinates. The Figure 2 caption states that temporally correlated coordinates are mapped to the same 2D coordinate, 'thereby ensuring they share the same modulation latent representation.' Thus the observation that corresponding pixels yield similar or identical latents is entailed by the definition of the latent, not an independent empirical validation. This is a minor self-consistency check and is not the source of the paper's speed or reconstruction claims, which are measured on held-out reconstruction and benchmarked against external baselines.

full rationale

No equation-level circularity is present in the main derivation chain. The alignment flow is trained with auxiliary supervision from RAFT, an external pretrained system, and the resulting model is evaluated on reconstruction, compression, super-resolution, interpolation, and inpainting tasks against external baselines; none of these predictions is defined in terms of the quantity it claims to predict. The only circularity found is the Section 4.6 t-SNE 'validation,' which restates the construction that aligned coordinates share a grid latent. The paper also discloses its limitation on low-motion sequences (Appendix E.3), which is consistent with a genuine empirical claim rather than a tautology. The omission of RAFT precomputation time from the Table 2 encoding-speed comparison is an evaluation-scope caveat, not a circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim rests on two categories of assumptions: (1) standard INR machinery (modulated-SIREN, multi-resolution hash grids, hyper-SIREN) whose behavior is inherited from cited prior work, and (2) the domain assumption that RAFT-style optical flow accurately identifies corresponding pixels, which is load-bearing and content-dependent. The only hand-tuned quantities are the GOP size, adaptive-normalization threshold, auxiliary loss weight, and neighbor set; a hidden log-scale choice in Eq. (3) further weakens reproducibility.

free parameters (5)
  • GOP size n = 10
    Default setting (Section 4.1); Table 6 shows n=5 often better for high-motion sequences, indicating the choice is content-dependent.
  • Adaptive normalization density threshold r_th = 0.5
    Hand-set threshold used to find the high-density region for normalizing aligned coordinates (Eq. 5); no sensitivity study is provided.
  • Auxiliary loss weight w_aux = 0.5
    Hand-set trade-off between reconstruction error and RAFT optical-flow alignment (Section 4.1); no ablation on this weight.
  • Neighbor index set P = {0,1}
    Concatenates latents from the current and next GOP; Table 5c shows a 1.09dB gain over P={0}, so the final value is tuned for performance.
  • Log scale factor in Eq. (3) = not specified
    The formula Flow = log(t-tk) * FH(t)(x,y) is undefined at t=tk and ambiguous in sign; the exact implementation is not given, making the effective scaling a hidden choice.
assumptions (4)
  • domain assumption RAFT optical flow is a sufficiently accurate proxy for pixel correspondences across video frames.
    Used as auxiliary supervision (Section 4.1) to train the alignment flow network; errors in RAFT directly corrupt the aligned coordinates.
  • domain assumption Corresponding pixels across frames are similar enough in appearance that sharing one modulation latent improves reconstruction.
    Core premise of temporally coherent modulation; the method loses to NVP on low-motion Beauty (Table 16), showing the assumption is content-dependent.
  • domain assumption A lightweight hyper-SIREN can represent the time-dependent alignment flow needed for each GOP.
    The flow network is a single-layer hyper-network and a 5-layer 8-neuron SIREN (Section B.1); no evidence that this class captures general optical flow.
  • standard math Multi-resolution 2D grids (DenseGrid in tiny-cuda-nn) are valid parametric encoders for the aligned latent field.
    Relies on the established tiny-cuda-nn grid implementation as background technique.
invented entities (1)
  • Temporally coherent modulation latent z_xyt
    purpose: A per-pixel modulation code shared across temporally corresponding pixels, used to condition the base network via modulated-SIREN.
    Internal design construct. The t-SNE analysis in Figure 4 shows clustering, but there is no externally falsifiable prediction attached to this entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Neural Video Representation with Temporally Coherent Modulation." pith.science (2026). https://pith.science/paper/JRADNJOW

@misc{pith2026250500335,
  author       = {Pith},
  title        = {Pith review of: Efficient Neural Video Representation with Temporally Coherent Modulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JRADNJOW}},
  note         = {Machine review of arXiv:2505.00335}
}
read the original abstract

Implicit neural representations (INR) has found successful applications across diverse domains. To employ INR in real-life, it is important to speed up training. In the field of INR for video applications, the state-of-the-art approach employs grid-type parametric encoding and successfully achieves a faster encoding speed in comparison to its predecessors. However, the grid usage, which does not consider the video's dynamic nature, leads to redundant use of trainable parameters. As a result, it has significantly lower parameter efficiency and higher bitrate compared to NeRV-style methods that do not use a parametric encoding. To address the problem, we propose Neural Video representation with Temporally coherent Modulation (NVTM), a novel framework that can capture dynamic characteristics of video. By decomposing the spatio-temporal 3D video data into a set of 2D grids with flow information, NVTM enables learning video representation rapidly and uses parameter efficiently. Our framework enables to process temporally corresponding pixels at once, resulting in the fastest encoding speed for a reasonable video quality, especially when compared to the NeRV-style method, with a speed increase of over 3 times. Also, it remarks an average of 1.54dB/0.019 improvements in PSNR/LPIPS on UVG (Dynamic) (even with 10% fewer parameters) and an average of 1.84dB/0.013 improvements in PSNR/LPIPS on MCL-JCV (Dynamic), compared to previous grid-type works. By expanding this to compression tasks, we demonstrate comparable performance to video compression standards (H.264, HEVC) and recent INR approaches for video compression. Additionally, we perform extensive experiments demonstrating the superior performance of our algorithm across diverse tasks, encompassing super resolution, frame interpolation and video inpainting. Project page is https://sujiikim.github.io/NVTM/.

Figures

Figures reproduced from arXiv: 2505.00335 by the authors.

Figure 1
Figure 1. Fast encoding speed with high image quality. (Left) The encoding speed in UVG, where all models are configured at 0.1bpp and evaluated on the same resource conditions. NVTM learns quickly and achieves 30dB 3×faster than the NeRV￾series. (Right) Video reconstruction results on ReadySetGo sequence after train￾ing for 1 minutes. While E-NeRV and HNeRV exhibit blurry outputs, NVP and NVTM, based on parametric encoding, … view at source ↗
Figure 2
Figure 2. Overview of NVTM. NVTM generates the same modulation latent for tem￾porally correlated pixels between consecutive frames, and the latent is used to modulate the base network. To obtain this latent, 1) input video is split into GOP units, 2) net￾work F generates an alignment flow to transform 3D coordinate (x, y, t) to specific time tk in k-th GOP unit, 3) 2D aligned coordinated (xk, yk) is obtained by adding (x, y) … view at source ↗
Figure 3
Figure 3. Video inpainting and compression performance. (a) Visualization of video in￾painting on Blackswan and Camel sequences in DAVIS2017. Although the masked regions are excluded during encoding, the NVTM successfully decodes them by uti￾lizing temporally coherent modulation latent from adjacent frames. (b) BPP-PSNR plot of video compression on UVG (Dynamic). We encode all models with each video sequence and evaluate as f… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: (a) t-SNE visualization of modulation latent zxyz from our alignment module on corresponding pixels (1st and 5th frame). We select areas with similar pixel in￾formation, i.e. RGB values, and for ease of verification, these are denoted as {Horse, Grass, Sign}. The laten…
Figure 5
Figure 5. Figure 5: Visualization results of video inpainting. Each are the first frame of Camel, Blackswan, Cows, Drift-Chicane, Soapbox and Tennis sequence on DAVIS2017 [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: Sequence-wise video compression results on UVG (Dynamic) [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]
Figure 7
Figure 7. Figure 7: Trade-off on performance and training speed. A to F represents different types of parameter distribution, and B(*) denotes the type we experimented. On upper figure, red line represents PSNR (left y-axis), and the blue line represents encoding speed (right y-axis). The…
Figure 8
Figure 8. Figure 8: Comparison on optical flow and our alignment flow. Our network generates the alignment flow across different times within the GOP (Eq. 3), then maps the 2D aligned coordinates (Eq. 4). We compare the optical flow, which we used for guidance, with our flow output on Bos…
Figure 9
Figure 9. Figure 9: PSNR variation across random seeds on video reconstruction [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]
Figure 10
Figure 10. Figure 10: Video reconstruction result of first frame on UVG sequences (Bosphorus, Jockey, ReadySetGo and YachtRide from top to bottom). For each sequence, the first row indicates the decoded results, and the second row stands out the FLIP results, while darker colors indicating…
Figure 11
Figure 11. Figure 11: Video reconstruction result of first frame on MCL-JCV sequences (04, 05, 11, 20, 21 from top to bottom). For each sequence, the first row indicates the decoded results, and the second row stands out the FLIP results, while darker colors indicating better performance …
Figure 12
Figure 12. Figure 12: Visualization of video super resolution results for the first frame on the UVG sequences (Bosphorus, Jockey, ReadySetGo and YachtRide from top to bottom). For each sequence, the first row indicates the doubled spatial decoded results and we zoomed them for better clar…
Figure 13
Figure 13. Figure 13: Visualization of Video frame interpolation results for the second frame (i.e. first interpolated frame) on the UVG sequences (Bosphorus, Jockey, ReadySetGo and YachtRide from top to bottom). For each sequence, the first row indicates the doubled temporal decoded resul…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 41 canonical work pages

  1. [1]

    Andersson, P., Nilsson, J., Akenine-M¨ oller, T., Oskarsson, M.,˚Astr¨ om, K., Fairchild, M.D.: Flip: A difference evaluator for alternating images. Proc. ACM Comput. Graph. Interact. Tech. 3(2), 15–1 (2020)

    work page 2020

  2. [2]

    In: 2023 IEEE International Conference on Image Processing (ICIP)

    Bai, Y., Dong, C., Wang, C., Yuan, C.: Ps-nerv: Patch-wise stylized neural representations for videos. In: 2023 IEEE International Conference on Image Processing (ICIP). pp. 41–45. IEEE (2023)

    work page 2023

  3. [3]

    arXiv preprint arXiv: 2302.03130 (2023)

    Bauer, M., Dupont, E., Brock, A., Rosenbaum, D., Schwarz, J.R., Kim, H.: Spatial functa: Scaling functa to imagenet classification and generation. arXiv preprint arXiv: 2302.03130 (2023)

    arXiv 2023

  4. [4]

    In: Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16

    Chabra, R., Lenssen, J.E., Ilg, E., Schmidt, T., Straub, J., Lovegrove, S., Newcombe, R.: Deep local shapes: Learning local sdf priors for detailed 3d reconstruction. In: Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. pp. 608–625. Springer (2020)

    work page 2020

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    Chen, H., Gwilliam, M., Lim, S.N., Shrivastava, A.: Hnerv: A hybrid neural representation for videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    work page 2023

  6. [6]

    Advances in Neural Information Processing Systems 34, 21557–21568 (2021)

    Chen, H., He, B., Wang, H., Ren, Y., Lim, S.N., Shrivastava, A.: Nerv: Neu- ral representations for videos. Advances in Neural Information Processing Systems 34, 21557–21568 (2021)

    work page 2021

  7. [7]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Chen, Y., Liu, S., Wang, X.: Learning continuous image representation with local implicit image function. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8628–8638 (2021)

    work page 2021

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Chen, Z., Chen, Y., Liu, J., Xu, X., Goel, V., Wang, Z., Shi, H., Wang, X.: Videoinr: Learning video implicit neural representation for continuous space-time super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2047–2057 (2022)

    work page 2022

Show all 50 references
  1. [9]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Chibane, J., Alldieck, T., Pons-Moll, G.: Implicit functions in feature space for 3d shape reconstruction and completion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 6970–6981 (2020)

    2020

  2. [10]

    ACM Transactions on Graphics (TOG) 39(4), 75–1 (2020)

    Chu, M., Xie, Y., Mayer, J., Leal-Taix´ e, L., Thuerey, N.: Learning tem- poral coherence via self-supervision for gan-based video generation. ACM Transactions on Graphics (TOG) 39(4), 75–1 (2020)

    2020

  3. [11]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Deng, C.L., Tartaglione, E.: Compressing explicit voxel grid representations: fast nerfs become also small. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 1236–1245 (2023)

    2023

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition

    Dong, Z., Guo, C., Song, J., Chen, X., Geiger, A., Hilliges, O.: Pina: Learn- ing a personalized implicit neural avatar from a single rgb-d video sequence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition. pp. 20470–20480 (2022) 16 S. Shi...

    2022

  5. [13]

    Transactions on Machine Learning Research 2022(11) (2022)

    Dupont, E., Loya, H., Alizadeh, M., Golinski, A., Teh, Y., Doucet, A.: Coin++: neural compression across modalities. Transactions on Machine Learning Research 2022(11) (2022)

    2022

  6. [14]

    In: Neural Compression: From Information Theory to Applications–Workshop@ ICLR 2021 (2021)

    Dupont, E., Golinski, A., Alizadeh, M., Teh, Y.W., Doucet, A.: Coin: Com- pression with implicit neural representations. In: Neural Compression: From Information Theory to Applications–Workshop@ ICLR 2021 (2021)

    2021

  7. [15]

    In: International Conference on Machine Learning

    Dupont, E., Kim, H., Eslami, S.A., Rezende, D.J., Rosenbaum, D.: From data to functa: Your data point is a function and you can treat it like one. In: International Conference on Machine Learning. pp. 5694–5725. PMLR (2022)

    2022

  8. [16]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Fang, S., Xu, W., Wang, H., Yang, Y., Wang, Y., Zhou, S.: One is all: Bridg- ing the gap between neural radiance fields architectures with progressive volume distillation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 37, pp. 597–605 (2023)

    2023

  9. [17]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Figueirˆ edo, P., Paliwal, A., Kalantari, N.K.: Frame interpolation for dy- namic scenes with implicit flow encoding. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 218–228 (2023)

    2023

  10. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., Kanazawa, A.: Plenoxels: Radiance fields without neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5501–5510 (2022)

    2022

  11. [19]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Genova, K., Cole, F., Sud, A., Sarna, A., Funkhouser, T.: Local deep implicit functions for 3d shape. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4857–4866 (2020)

    2020

  12. [20]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Girish, S., Shrivastava, A., Gupta, K.: Shacira: Scalable hash-grid compres- sion for implicit neural representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17513–17524 (2023)

    2023

  13. [21]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition

    Gomes, C., Azevedo, R., Schroers, C.: Video compression with entropy- constrained neural representations. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition. pp. 18497–18506 (2023)

    2023

  14. [22]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    He, B., Yang, X., Wang, H., Wu, Z., Chen, H., Huang, S., Ren, Y., Lim, S.N., Shrivastava, A.: Towards scalable neural representation for diverse videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6132–6142 (2023)

    2023

  15. [23]

    Networks 910(37), 5 (1999)

    Installations, T., Line, L.: Subjective video quality assessment methods for multimedia applications. Networks 910(37), 5 (1999)

    1999

  16. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Jiang, C., Sud, A., Makadia, A., Huang, J., Nießner, M., Funkhouser, T., et al.: Local implicit grid representations for 3d scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6001–6010 (2020)

    2020

  17. [25]

    arXiv preprint arXiv:2210.06823 (2022)

    Kim, S., Yu, S., Lee, J., Shin, J.: Scalable neural video representations with learnable positional features. arXiv preprint arXiv:2210.06823 (2022)

    2022 arXiv

  18. [26]

    In: Proceedings of the 31st ACM International Conference on Multimedia

    Lee, J.C., Rho, D., Ko, J.H., Park, E.: Ffnerv: Flow-guided frame-wise neu- ral representations for videos. In: Proceedings of the 31st ACM International Conference on Multimedia. pp. 7859–7870 (2023) Efficient Neural Video Representation with Temporally Coherent Modulation 17

    2023

  19. [27]

    In: Com- puter Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, Oc- tober 23–27, 2022, Proceedings, Part XXXV

    Li, Z., Wang, M., Pi, H., Xu, K., Mei, J., Liu, Y.: E-nerv: Expedite neural video representation with disentangled spatial-temporal context. In: Com- puter Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, Oc- tober 23–27, 2022, Proceedings, Part XXXV. pp. 267–284....

    2022

  20. [28]

    Advances in Neural Information Processing Systems33, 15651–15663 (2020)

    Liu, L., Gu, J., Zaw Lin, K., Chua, T.S., Theobalt, C.: Neural sparse voxel fields. Advances in Neural Information Processing Systems33, 15651–15663 (2020)

    2020

  21. [29]

    arXiv preprint arXiv:1711.05101 (2017)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  22. [30]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    Maiya, S.R., Girish, S., Ehrlich, M., Wang, H., Lee, K.S., Poirson, P., Wu, P., Wang, C., Shrivastava, A.: Nirvana: Neural implicit representations of videos with adaptive networks and autoregressive patch-wise modeling. In: Proceedings of the IEEE/CVF Conference on Computer V...

    2023

  23. [31]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Mehta, I., Gharbi, M., Barnes, C., Shechtman, E., Ramamoorthi, R., Chan- draker, M.: Modulated periodic activations for generalizable local functional representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 14214–14223 (2021)

    2021

  24. [32]

    In: Proceedings of the 11th ACM Multimedia Systems Conference

    Mercat, A., Viitanen, M., Vanne, J.: Uvg dataset: 50/120fps 4k sequences for video codec analysis and development. In: Proceedings of the 11th ACM Multimedia Systems Conference. pp. 297–302 (2020)

    2020

  25. [33]

    In: European conference on computer vision (2020)

    Mildenhall, B., Srinivasan, P., Tancik, M., Barron, J., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: European conference on computer vision (2020)

    2020

  26. [34]

    M¨ uller, T.: tiny-cuda-nn (4 2021), https://github.com/NVlabs/ tiny-cuda-nn

    2021

  27. [35]

    ACM Transactions on Graphics (ToG) 41(4), 1–15 (2022)

    M¨ uller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primi- tives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG) 41(4), 1–15 (2022)

    2022

  28. [36]

    arXiv:1704.00675 (2017)

    Pont-Tuset, J., Perazzi, F., Caelles, S., Arbel´ aez, P., Sorkine-Hornung, A., Van Gool, L.: The 2017 davis challenge on video object segmentation. arXiv:1704.00675 (2017)

    2017 arXiv

  29. [37]

    In: Proceedings of the Asian Conference on Computer Vision

    Rho, D., Cho, J., Ko, J.H., Park, E.: Neural residual flow fields for efficient video representations. In: Proceedings of the Asian Conference on Computer Vision. pp. 3447–3463 (2022)

    2022

  30. [38]

    Advances in Neu- ral Information Processing Systems 33, 7462–7473 (2020)

    Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. Advances in Neu- ral Information Processing Systems 33, 7462–7473 (2020)

    2020

  31. [39]

    Advances in Neural Information Processing Systems35, 8144– 8158 (2022)

    Su, K., Chen, M., Shlizerman, E.: Inras: Implicit neural representation for audio scenes. Advances in Neural Information Processing Systems35, 8144– 8158 (2022)

    2022

  32. [40]

    IEEE Transactions on circuits and systems for video technology 22(12), 1649–1668 (2012)

    Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on circuits and systems for video technology 22(12), 1649–1668 (2012)

    2012

  33. [41]

    arXiv preprint arXiv:2211.01839 (2022) 18 S

    Szatkowski, F., Piczak, K.J., Spurek, P., Tabor, J., Trzci´ nski, T.: Hyper- sound: Generating implicit neural representations of audio signals with hy- pernetworks. arXiv preprint arXiv:2211.01839 (2022) 18 S. Shin, S. Kim, D. Oh

    2022 arXiv

  34. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Takikawa, T., Litalien, J., Yin, K., Kreis, K., Loop, C., Nowrouzezahrai, D., Jacobson, A., McGuire, M., Fidler, S.: Neural geometric level of detail: Real- time rendering with implicit 3d shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2021

  35. [43]

    In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16

    Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. pp. 402–419. Springer (2020)

    2020

  36. [44]

    Linux journal 2006(146), 10 (2006)

    Tomar, S.: Converting video formats with ffmpeg. Linux journal 2006(146), 10 (2006)

    2006

  37. [45]

    264/avc video quality assessment dataset

    Wang, H., Gan, W., Hu, S., Lin, J.Y., Jin, L., Song, L., Wang, P., Kat- savounidis, I., Aaron, A., Kuo, C.C.J.: Mcl-jcv: a jnd-based h. 264/avc video quality assessment dataset. In: 2016 IEEE international conference on image processing (ICIP). pp. 1509–1513. IEEE (2016)

    2016

  38. [46]

    264/avc video coding standard

    Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A.: Overview of the h. 264/avc video coding standard. IEEE Transactions on circuits and systems for video technology 13(7), 560–576 (2003)

    2003

  39. [47]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Yang, J., Zheng, W.S., Yang, Q., Chen, Y.C., Tian, Q.: Spatial-temporal graph convolutional network for video-based person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3289–3299 (2020)

    2020

  40. [48]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreason- able effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 586–595 (2018)

    2018

  41. [49]

    bframes=0

    Zhao, Q., Asif, M.S., Ma, Z.: Dnerv: Modeling inherent dynamics via dif- ference neural representation for videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2031–2040 (2023) Efficient Neural Video Representation with Temporally ...

    2023

  42. [50]

    bframes=0

    reported with a different resolution and its code is unavailable, we only com- pared with HNeRV [5]. NVTM has large margins compared to HNeRV, except Jockey sequence. Table 14: Comparison of PSNR on frame interpolation result (×2) in UVG. Method Bosphorus Jockey ReadySetGo Yac...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.