Pith. sign in

REVIEW 4 major objections 6 minor 65 references

How to Design and Train Your Implicit Neural Representation for Video Compression

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read When video INR codecs are compared under equal training time instead of equal epochs, the ranking reverses, and the proposed RNeRV configuration leads the NeRV family by 1.27% PSNR on UVG.

desk verdict A useful benchmarking library and a sensible reframing around training time, but the headline PSNR gains rest on baseline tuning the authors admit is imperfect and the abstract numbers don't match the tables. read the letter →

arxiv 2506.24127 v1 pith:7EJPMUSD submitted 2025-06-30 cs.CV

Matthew Gwilliam , Roy Zhang , Namitha Padmanabhan , Hongyang Du , Abhinav Shrivastava This is my paper
classification cs.CV
keywords implicitneuralrepresentationvideocompressionNeRVhyper-networkweighttokenmaskingencodingtimerate-distortiontraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Implicit neural representation (INR) video codecs store a video as a small network's weights, decoding fast but encoding slowly because each video must be trained individually. This paper argues that comparing these codecs by size and quality alone is misleading, since methods that win at equal epochs may lose badly once everyone gets the same wall-clock training budget. To support that case, the authors build a library that isolates the components of NeRV-style methods, distill design principles from it, and assemble a configuration called Rabbit NeRV (RNeRV) that achieves +1.27% PSNR on average over the best alternative on seven UVG 1080p videos when all methods are limited to 300 NeRV epochs of training time. For the encoding-speed problem itself, they show a hyper-network that predicts INR weights from video clips can encode in one forward pass, and they introduce weight token masking so the stored weights can be halved at inference with a small quality cost.

What carries the argument

The load-bearing machinery is a modular library that decomposes every NeRV-style codec into paired positional-encoding and stem options (sinusoidal, grid-based, content-adaptive, or none), upsample block designs (basic conv–PixelShuffle, group-wise conv, ConvNext), skip-connection styles (E-NeRV's learned t-skips with layer norm versus HiNeRV's grid skips), and a parameter-distribution schedule governed by expansion, reduction, and kernel-size terms; ablating one component at a time under fixed wall-clock budgets is what generates RNeRV. The second mechanism is the hyper-network with weight token masking: a transformer reads video clip tokens plus learnable weight tokens and predicts per-layer modulations of a shared base hypo-network, and masking half the predicted tokens during training makes the stored bitstream size selectable at encoding time.

What would settle it

Retrain a competitor such as HiNeRV with its own hyperparameter search at the same parameter budgets and the same 30-minute per-video wall-clock budget on an RTX A5000, then average PSNR over the seven UVG videos; if any alternative meets or beats RNeRV's average, the equal-training-time claim collapses. A more direct check is an independent re-implementation of RNeRV and the library baselines from the paper's code: if the +1.27% average margin does not reproduce under the specified training schedule, the claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that encoding time—the wall-clock cost of training a network per video—is a first-class axis of evaluation for INR video codecs, and that the NeRV family's components can be disentangled and recombined predictably once that axis is measured. To support this, the authors re-implement ONeRV, E-NeRV, NeRV, FFNeRV, HiNeRV, HNeRV, and DiffNeRV in one library, ablate positional encoding and stem choices, block designs, skip connections, and parameter distributions, and assemble the best combination into Rabbit NeRV (RNeRV). At a 30-minute training budget on an RTX A5000, RNeRV reports +1.27% PSNR and +0.72% MS-SSIM on average over the best alternative for each of seven UVG 1080p videos. For the slow-encoding objection, the paper shows that a hyper-network can predict the weights of a small 'hypo-network' from 8-frame video clips in one forward pass, and introduces weight token masking: randomly dropping half of each layer's predicted weight tokens during training, which lets the encoder choose between full and half-size storage at inference, with reported 1.7% PSNR and MS-SSIM gains at 0.037 bpp on UCF-101.

Load-bearing premise

The load-bearing premise is that the library's re-implemented baselines, tuned to the same 1.5M and 3M parameter budgets, are fair representatives of each method's best result under a 30-minute training budget; the paper itself notes in Appendix 7.1 that some configurations are likely suboptimal, so a better-tuned competitor could change the +1.27% PSNR headline.

Editorial extensions

If this is right

  • At equal epochs, HiNeRV dominates the library, but at equal 30-minute budgets FFNeRV and RNeRV move ahead, so published rankings that ignore encoding time overstate the value of expensive components.
  • Component choices transfer across methods: the FFNeRV-style grid stem with no stem MLP, E-NeRV-style local skips with layer norms, and parameter redistribution toward later layers combine into RNeRV, which performs best at short and medium training budgets.
  • Hyper-networks can compress entire videos, not just sampled frames, and encode with a single forward pass, which removes per-sample training from the encoding path at the cost of lower reconstruction quality.
  • Weight token masking gives a 2x storage knob at inference: storing all predicted tokens gives higher quality, while dropping the masked half roughly halves the bitstream with a small quality penalty, and the masked model beats the unmasked baseline at the smaller size.
  • Increasing only the shared hypo-network parameters by 0.4% improves PSNR by 2.5% and MS-SSIM by 2.7% at equal bpp, indicating that decoder-side shared parameters can buy quality without enlarging the transmitted bitstream.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the equal-training-time protocol could be adopted as a standard benchmark for INR video codecs, and if it is, architecture search over stems, blocks, and parameter distributions will likely favor cheap, fast upsampling over quality-optimal but slow components.
  • Editorial inference: weight token masking is a representation-level dropout that could generalize beyond video to any INR emitting structured weight blocks, such as image or 3D scene codecs, enabling progressive or scalable bitstreams.
  • Editorial inference: the paper's parameter-distribution findings imply an optimal per-budget distribution curve; a follow-up study could map the Pareto frontier of expansion, reduction, and kernel-size settings for each wall-clock budget.
  • Editorial inference: since RNeRV is a configuration of existing components rather than a fundamentally new architecture, an independent re-implementation with the authors' stated hyperparameters is the most direct test of whether the +1.27% margin is a property of the components or of the tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies implicit neural representation (INR) methods for video compression, focusing on the NeRV family. The authors build a unified library of existing NeRV variants, decompose their components (positional encoding, stem, blocks, skip connections, parameter distribution), and propose a new configuration called Rabbit NeRV (RNeRV). The central claim is that under equal training time (300 NeRV epochs, about 30 minutes on an RTX A5000), RNeRV achieves +1.27% PSNR on average over the best-performing alternative per video on 7 UVG 1080p videos. The paper also proposes HyperNeRV, a hyper-network that predicts INR weights for video clips, and a Weight Token Masking training strategy that allows variable bitrate at inference; a 1.7% improvement in PSNR and MS-SSIM at 0.037 bpp on UCF-101 is claimed. Additional contributions include a qualitative analysis of learned representations by extending the XINC framework to multiple NeRV variants.

Significance. If the central empirical claims hold, the paper makes a useful contribution by reframing video INR evaluation around training time rather than only size and quality, and by providing an open library that could facilitate fairer comparisons in this area. The Weight Token Masking idea is a simple but potentially useful mechanism for flexible bitrate in hyper-network-based compression. The release of code and a project website is a concrete reproducibility asset. However, the significance is heavily dependent on whether the reimplemented baselines are genuinely representative and whether the equal-time comparison is robust to hyperparameter choices; the manuscript currently provides only limited evidence on these points.

major comments (4)
  1. [Section 4.3 / Appendix 6] The headline claim of +1.27% PSNR under equal training time is not supported by per-video numerical results. Figures 8-10 show only HoneyBee and Jockey, while the averaged curves in Figures 13-14 are presented without per-video breakdowns, and the averaging rule (averaging bpp and quality across videos separately, then pairing them) is described only in Appendix 6. Because the advertised number is the average over the best-performing alternative per video, a table listing each video, the best baseline, and RNeRV's PSNR/MS-SSIM/bpp is needed to audit the claim. Without it, the reader cannot verify the headline or assess whether a single under-tuned baseline inflates the margin.
  2. [Section 4.1 / Table 1] The claim that the library 'reproduces' existing methods is not backed by a comparison with the numbers reported by the original papers. Table 1 reports only results from the authors' own reimplementations, with settings that are 'faithful to the original papers when possible' but with at least one known deviation (FFNeRV expansion changed from 8 to 4). A table comparing the library implementations against published PSNR/MS-SSIM results at comparable settings would establish that the baselines are representative. As written, the reader cannot tell whether the reimplementations are strong enough to make the equal-time comparison meaningful.
  3. [Appendix 7.1] The manuscript itself concedes in Appendix 7.1 that some tested configurations are 'likely suboptimal' and that finding a true optimum over f_c_dim, r, exp, and component choices is 'quite difficult.' This concession applies to the very baselines used in the equal-time benchmark, not only to exploratory ablations. Since a single under-tuned baseline on one video can change the reported '+1.27%' margin, the paper should provide sensitivity evidence, such as multiple reasonable hyperparameter settings per baseline under the same time budget, or a tuning curve showing that the reported result is not an artifact of baseline configuration.
  4. [Abstract / Table 2] The abstract states that Weight Token Masking yields '1.7% improvements to both PSNR and MS-SSIM' at 0.037 bpp, but the numbers in Table 2 for the 2.45k-parameter row give 22.81/22.55 = 1.53% for PSNR and 0.6130/0.6011 = 1.98% for MS-SSIM. The claim should be stated accurately, for example as approximately 1.5-2.0% depending on the metric, or the underlying aggregation should be explicitly reported.
minor comments (6)
  1. [Figure 1 caption] The caption contains a typo: 'peak singal-noise ratio' should be 'peak signal-to-noise ratio.'
  2. [Introduction, Section 1] The sentence beginning 'However, with equal training time, such by allowing all methods the amount of time...' is grammatically incomplete and should be rewritten.
  3. [Section 2] There is a double comma in 'In this work,, we focus on video compression.'
  4. [Table 2] The column headers 'Train Val Min Max' are ambiguous. Clarify that 'Min' and 'Max' refer to parameter counts under masking, and define the masking flags in the table caption or in the text.
  5. [Section 4.1 and Table 1] The notation 'NeRV' for the updated NeRV from the HNeRV paper and 'ONeRV' for the original NeRV is potentially confusing; the definitions should be repeated near their first use in the main text.
  6. [Appendix 6] The word 'performancne' in the first sentence of Appendix 6 is a typo for 'performance.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's headline results are empirical benchmark measurements on external UVG and UCF-101 data, not derivations from fitted parameters or from a self-citation chain.

full rationale

The central claims are (i) RNeRV, an architecture assembled from components of prior NeRV-family methods, achieves +1.27% PSNR over the best per-video alternative under a fixed 30-minute training budget on 7 UVG 1080p videos, and (ii) weight token masking gives +1.7% PSNR/MS-SSIM at 0.037 bpp on UCF-101. Both are measured outcomes, reported as tables and curves, against external datasets and external methods (E-NeRV, FFNeRV, HiNeRV, DiffNeRV) plus the authors' own NeRV/HNeRV. No step defines an input in terms of the target output, no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported to forbid alternatives. RNeRV is a combination of previously published components, and the paper's ablations (Figures 5-7) are tests, not circular justifications. The self-citations (NeRV, HNeRV, NeRV-Enc, XINC) provide building blocks and baselines, but the load-bearing comparison is an external benchmark, so those citations do not create circularity. The main weakness is baseline tuning: Appendix 7.1 concedes that some tested configurations are 'likely suboptimal' and that finding a true global optimum is 'quite difficult,' so the +1.27% figure could change with better-tuned competitors. That is a correctness and reproducibility risk, not a circularity of derivation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on hand-chosen architecture hyperparameters and on the assumption that reimplemented baselines are tuned fairly. No new physical or theoretical entities are introduced; weight tokens and shared parameters are design components, not independent hypothetical entities.

free parameters (6)
  • RNeRV first-layer expansion (exp) = 4
    Hand-set to keep total decoder size at 1.5M/3M params; not swept for RNeRV itself.
  • RNeRV channel reduction (r) = 1.2 (1.5M), 1.4 (3M)
    Chosen to redistribute parameters toward later layers as suggested by ablations (Fig. 6).
  • Weight token masking ratio = 0.5
    Half the clips during training have the latter half of tokens masked; this ratio is fixed, not optimized.
  • FFNeRV expansion adjustment in library = 4 (down from 8)
    Adjusted by the authors to fit the 1.5M/3M parameter budgets; may disadvantage the FFNeRV baseline.
  • HyperNeRV transformer size = 47.9M parameters, 6 blocks, 720 token dim
    A large backbone trained on 10,000 Kinetics-400 videos; no ablation of backbone size is reported.
  • HypoNeRV fc_dim/token dimensions = fc_dim 16 or 20; token dims 256/144/72 etc.
    Manually chosen architecture details of the hypo-network; Table 3 varies only total hypo size.
assumptions (5)
  • domain assumption The UVG 7-video set and the UCF-101 one-video-per-class subset are representative of general video compression performance.
    All headline INR comparisons use these two small test sets with no statistical significance testing; Section 4.
  • domain assumption Wall-clock training time on a single NVIDIA RTX A5000 is a fair proxy for encoding speed of INR codecs.
    The equal-training-time comparisons in Figures 8-10 and the headline 1.27% gain assume this proxy; Section 4.
  • domain assumption Reimplemented baselines in the library faithfully represent each method's performance under the fixed parameter budget.
    The authors tuned baselines themselves and concede in Appendix 7.1 that some configurations are likely suboptimal.
  • domain assumption PSNR and MS-SSIM adequately capture the perceived quality differences between codecs for the purposes of these claims.
    No other quality metrics or human studies are used; standard in the field but still an assumption.
  • domain assumption The hyper-network trained on Kinetics-400 clips generalizes to UCF-101 videos without per-video fine-tuning.
    HyperNeRV results in Tables 2-4 rely on zero-shot transfer; no analysis of distribution shift is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How to Design and Train Your Implicit Neural Representation for Video Compression." pith.science (2026). https://pith.science/paper/7EJPMUSD

@misc{pith2026250624127,
  author       = {Pith},
  title        = {Pith review of: How to Design and Train Your Implicit Neural Representation for Video Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7EJPMUSD}},
  note         = {Machine review of arXiv:2506.24127}
}
read the original abstract

Implicit neural representation (INR) methods for video compression have recently achieved visual quality and compression ratios that are competitive with traditional pipelines. However, due to the need for per-sample network training, the encoding speeds of these methods are too slow for practical adoption. We develop a library to allow us to disentangle and review the components of methods from the NeRV family, reframing their performance in terms of not only size-quality trade-offs, but also impacts on training time. We uncover principles for effective video INR design and propose a state-of-the-art configuration of these components, Rabbit NeRV (RNeRV). When all methods are given equal training time (equivalent to 300 NeRV epochs) for 7 different UVG videos at 1080p, RNeRV achieves +1.27% PSNR on average compared to the best-performing alternative for each video in our NeRV library. We then tackle the encoding speed issue head-on by investigating the viability of hyper-networks, which predict INR weights from video inputs, to disentangle training from encoding to allow for real-time encoding. We propose masking the weights of the predicted INR during training to allow for variable, higher quality compression, resulting in 1.7% improvements to both PSNR and MS-SSIM at 0.037 bpp on the UCF-101 dataset, and we increase hyper-network parameters by 0.4% for 2.5%/2.7% improvements to PSNR/MS-SSIM with equal bpp and similar speeds. Our project website is available at https://mgwillia.github.io/vinrb/ and our code is available at https://github.com/mgwillia/vinrb.

Figures

Figures reproduced from arXiv: 2506.24127 by the authors.

Figure 1
Figure 1. Compression performance of INR-based video codecs from the NeRV family. We examine not only size (bits per pixel) and quality (peak singal-noise ratio), but also encoding speed. Since INRs must be trained for each sample, the encod￾ing speed is dominated by the training time. (top) PSNR/bpp with time as dot size. (bottom) PSNR/bpp for equal time (30 minutes on RTXA5000), averaged over 7 UVG videos at 1080p. at low s… view at source ↗
Figure 2
Figure 2. Disentangling components within the NeRV family. We isolate the various components of NeRV-like architectures and categorize critical parts of the design space – positional encoding, stem, and upsample blocks. There are other components shown in the figure (skip connections) as well as many not shown, including critical choices like parameter distribution. In this work we analyze the impact of these various componen… view at source ↗
Figure 3
Figure 3. Weight Token Masking. We make hyper-networks have flexible encoding size at inference time. If we mask some of the predicted weight tokens for some portion of the samples (in our case, 50%), we can also choose to mask the same set of tokens at inference for any given input clip (group of frames), with a small decrease in reconstruction quality. That is, we can flexibly reduce the bitrate by 2x post-training, during … view at source ↗
Figures from the paper (21 more)
Figure 4
Figure 4. Figure 4: Quality vs. wall time, for UVG HoneyBee (top) and Jockey (bottom) at 1080p (Section 4.1). papers. We train all INR methods with the Adam opti￾mizer [20], with a cosine annealing learning rate and single epoch warmup. We run on single GPUs, and wherever we benchmark wal…
Figure 6
Figure 6. Figure 6: Quality vs. size, for Honey￾Bee (top) and Jockey (bottom) at 1080p for NeRV with different parameter distribu￾tions (Section 4.2) [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 10
Figure 10. Figure 10: Quality vs. size, for Honey￾Bee (top) and Jockey (bottom) at 1080p with “long” training time (Section 4.3). This motivates two directions of investigation – (1) which method is best with equal time, and (2) can we mix-and￾match method components for good quality with …
Figure 11
Figure 11. Figure 11: XINC contribution maps on 1080p Jockey. XINC dissects an INR to understand what parts of the visual signal are represented by each neuron (2D convolution kernel). We show contribution maps for the last (head) layer, sorted by magnitude for ease of comparison. Darker r…
Figure 12
Figure 12. Figure 12: XINC motion analysis for the last (head) layer, for 1080p Jockey. We show fluctuation in total kernel contributions in response to motion between adjacent frames at various points in the video. While HiNeRV’s changes are driven by fine differences between frames, FFNe…
Figure 13
Figure 13. Figure 13: Quality vs. size, averaged across the 7 UVG videos we use in this paper, for “short” training time. 0.003 0.005 0.007 0.010 0.013 0.015 0.018 bpp 24 26 28 30 PSNR Train ~2 minutes 0.003 0.005 0.007 0.010 0.013 0.015 0.018 bpp 0.75 0.80 0.85 MS-SSIM Train ~2 minutes ON…
Figure 16
Figure 16. Figure 16: Quality vs. size, for Honey￾Bee (top) and Jockey (bottom) at 1080p for NeRV with different parameter distribu￾tions [PITH_FULL_IMAGE:figures/full_fig_p013_16.png]
Figure 18
Figure 18. Figure 18: Quality vs. size, for Honey￾Bee (top) and Jockey (bottom) at 1080p for FFNeRV for various position-stem combi￾nations [PITH_FULL_IMAGE:figures/full_fig_p014_18.png]
Figure 23
Figure 23. Figure 23: Quality vs. size, for HoneyBee (top) and Jockey (bottom) at 1080p for E￾NeRV with different skip connections and without flow warping. sion for transformer encoder layers: 720 and 2880; 12 heads, 6 blocks, for HyperNeRV of size 47.9M. • Optimizer: Adam • Learning rate…
Figure 26
Figure 26. Figure 26: Quality vs. size, for Honey￾Bee (top) and Jockey (bottom) at 1080p for NeRV with different skip connections and without flow warping [PITH_FULL_IMAGE:figures/full_fig_p015_26.png]
Figure 27
Figure 27. Figure 27: Quality vs. size, for Honey￾Bee (top) and Jockey (bottom) at 1080p with “short” training time [PITH_FULL_IMAGE:figures/full_fig_p015_27.png]
Figure 30
Figure 30. Figure 30: Quality vs. size, for HoneyBee (top) and Jockey (bottom) at 1080p [PITH_FULL_IMAGE:figures/full_fig_p016_30.png]
Figure 32
Figure 32. Figure 32: Reconstruction quality over training epochs, for UVG [PITH_FULL_IMAGE:figures/full_fig_p017_32.png]
Figure 34
Figure 34. Figure 34: We show a detailed walkthrough of the NeRV stem, in [PITH_FULL_IMAGE:figures/full_fig_p018_34.png]
Figure 35
Figure 35. Figure 35: We show an example NeRV block with upsample stride [PITH_FULL_IMAGE:figures/full_fig_p019_35.png]
Figure 36
Figure 36. Figure 36: We show example token predictions, FC processing, repeat/reshape, and modulation with the “shared” parameters for a single [PITH_FULL_IMAGE:figures/full_fig_p019_36.png]
Figure 37
Figure 37. Figure 37: XINC contribution maps on the last (head) layer of HypoNeRV, for Jockey at 256×256 resolution. We sort kernels by total contribution magnitude and select a subset of uniformly sampled kernels in the head layer. For the sake of interpretability, we remove locations at …
Figure 39
Figure 39. Figure 39: XINC motion analysis for fluctuation in total contribu [PITH_FULL_IMAGE:figures/full_fig_p020_39.png]
Figure 38
Figure 38. Figure 38: Visualization of PixelShuffle’s effect on kernel contri￾bution patterns in the last (head) layer of HypoNeRV, for Jockey at 256x256. Left: Contribution map for a single kernel, showing the sparse activation pattern created by PixelShuffle’s channel-to￾space rearrangem…
Figure 40
Figure 40. Figure 40: XINC contribution maps from the last (head) layer on Jockey at 1080p. We supplement [PITH_FULL_IMAGE:figures/full_fig_p021_40.png]
Figure 41
Figure 41. Figure 41: XINC motion analysis for the last (head) layer of all dif [PITH_FULL_IMAGE:figures/full_fig_p021_41.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 61 canonical work pages

  1. [1]

    Scale-space flow for end-to-end optimized video com- pression

    Eirikur Agustsson, David Minnen, Nick Johnston, Jo- hannes Balle, Sung Jin Hwang, and George Toderici. Scale-space flow for end-to-end optimized video com- pression. InCVPR, 2020. 1, 2

    work page 2020

  2. [2]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016. 7

    work page 2016

  3. [3]

    Nerv: Neural representations for videos.Advances in Neural Infor- mation Processing Systems, 34:21557–21568, 2021

    Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser Nam Lim, and Abhinav Shrivastava. Nerv: Neural representations for videos.Advances in Neural Infor- mation Processing Systems, 34:21557–21568, 2021. 2, 3, 5

    work page 2021

  4. [4]

    Cnerv: Content-adaptive neural representation for visual data, 2022

    Hao Chen, Matt Gwilliam, Bo He, Ser-Nam Lim, and Abhinav Shrivastava. Cnerv: Content-adaptive neural representation for visual data, 2022. 2

    work page 2022

  5. [5]

    Hnerv: A hybrid neural representa- tion for videos

    Hao Chen, Matthew Gwilliam, Ser-Nam Lim, and Ab- hinav Shrivastava. Hnerv: A hybrid neural representa- tion for videos. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 10270–10279, 2023. 2, 3, 5

    work page 2023

  6. [6]

    Fast encoding and decoding for implicit video representation, 2024

    Hao Chen, Saining Xie, Ser-Nam Lim, and Abhinav Shrivastava. Fast encoding and decoding for implicit video representation, 2024. 2, 3, 5, 1

    work page 2024

  7. [7]

    Transformers as meta-learners for implicit neural representations,

    Yinbo Chen and Xiaolong Wang. Transformers as meta-learners for implicit neural representations,

    work page

  8. [8]

    Coin: Com- pression with implicit neural representations.arXiv preprint arXiv:2103.03123, 2021

    Emilien Dupont, Adam Goli ´nski, Milad Alizadeh, Yee Whye Teh, and Arnaud Doucet. Coin: Com- pression with implicit neural representations.arXiv preprint arXiv:2103.03123, 2021. 2

    arXiv 2021

Show all 65 references
  1. [9]

    Coin++: Data agnostic neural compression.arXiv preprint arXiv:2201.12904, 1(2):4, 2022

    Emilien Dupont, Hrushikesh Loya, Milad Alizadeh, Adam Golinski, Yee Whye Teh, and Arnaud Doucet. Coin++: Data agnostic neural compression.arXiv preprint arXiv:2201.12904, 1(2):4, 2022. 2

    2022 arXiv

  2. [10]

    Shacira: Scalable hash-grid compression for implicit neural representations, 2023

    Sharath Girish, Abhinav Shrivastava, and Kamal Gupta. Shacira: Scalable hash-grid compression for implicit neural representations, 2023. 2

    2023

  3. [11]

    Adversarial text to continuous image gener- ation

    Kilichbek Haydarov, Aashiq Muhamed, Xiaoqian Shen, Jovana Lazarevic, Ivan Skorokhodov, Chamu- ditha Jayanga Galappaththige, and Mohamed Elho- seiny. Adversarial text to continuous image gener- ation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  4. [12]

    Towards scalable neural repre- sentation for diverse videos

    Bo He, Xitong Yang, Hanyu Wang, Zuxuan Wu, Hao Chen, Shuaiyi Huang, Yixuan Ren, Ser-Nam Lim, and Abhinav Shrivastava. Towards scalable neural repre- sentation for diverse videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), page...

    2023

  5. [13]

    Gaussian error lin- ear units (gelus), 2023

    Dan Hendrycks and Kevin Gimpel. Gaussian error lin- ear units (gelus), 2023. 1

    2023

  6. [14]

    The ki- netics human action video dataset, 2017

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, and Andrew Zisserman. The ki- netics human action video dataset, 2017. 5

    2017

  7. [15]

    Efficient video compression via content-adaptive super-resolution.ICCV, 2021

    Mehrdad Khani, Vibhaalakshmi Sivaraman, and Mo- hammad Alizadeh. Efficient video compression via content-adaptive super-resolution.ICCV, 2021. 2

    2021

  8. [16]

    Generalizable implicit neural representations via instance pattern composers.arXiv preprint arXiv:2211.13223, 2022

    Chiheon Kim, Doyup Lee, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Generalizable implicit neural representations via instance pattern composers.arXiv preprint arXiv:2211.13223, 2022. 3

    2022 arXiv

  9. [17]

    C3: High-performance and low-complexity neural compression from a single image or video, 2023

    Hyunjik Kim, Matthias Bauer, Lucas Theis, Jonathan Richard Schwarz, and Emilien Dupont. C3: High-performance and low-complexity neural compression from a single image or video, 2023. 1, 2

    2023

  10. [18]

    Springer Nature Switzerland, 2024

    Jina Kim, Jihoo Lee, and Je-Won Kang.SNeRV: Spectra-Preserving Neural Representation for Video, page 332–348. Springer Nature Switzerland, 2024. 2

    2024

  11. [19]

    Scalable neural video representations with learnable positional features

    Subin Kim, Sihyun Yu, Jaeho Lee, and Jinwoo Shin. Scalable neural video representations with learnable positional features. InAdvances in Neural Information Processing Systems, 2022. 2

    2022

  12. [20]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. 5

    2017

  13. [21]

    Hinerv: Video compression with hi- erarchical encoding-based neural representation

    Ho Man Kwan, Ge Gao, Fan Zhang, Andrew Gower, and David Bull. Hinerv: Video compression with hi- erarchical encoding-based neural representation. In Advances in Neural Information Processing Systems, pages 72692–72704. Curran Associates, Inc., 2023. 1, 2, 3, 5

    2023

  14. [22]

    Nvrc: Neural video representation compression, 2024

    Ho Man Kwan, Ge Gao, Fan Zhang, Andrew Gower, and David Bull. Nvrc: Neural video representation compression, 2024. 2

    2024

  15. [23]

    Cool-chic: Coordinate- based low complexity hierarchical image codec, 2023

    Th ´eo Ladune, Pierrick Philippe, F ´elix Henry, Gordon Clare, and Thomas Leguay. Cool-chic: Coordinate- based low complexity hierarchical image codec, 2023. 2

    2023

  16. [24]

    Mpeg: A video compression standard for multimedia applications.Commun

    Didier Le Gall. Mpeg: A video compression standard for multimedia applications.Commun. ACM, 1991. 2 9

    1991

  17. [25]

    Ffnerv: Flow-guided frame-wise neural representations for videos

    Joo Chan Lee, Daniel Rho, Jong Hwan Ko, and Eun- byung Park. Ffnerv: Flow-guided frame-wise neural representations for videos. InProceedings of the 31st ACM International Conference on Multimedia, page 7859–7870. ACM, 2023. 2, 3, 5

    2023

  18. [26]

    Deep contextual video compression, 2021

    Jiahao Li, Bin Li, and Yan Lu. Deep contextual video compression, 2021. 1, 2

    2021

  19. [27]

    Hybrid spatial- temporal entropy modelling for neural video compres- sion

    Jiahao Li, Bin Li, and Yan Lu. Hybrid spatial- temporal entropy modelling for neural video compres- sion. InProceedings of the 30th ACM International Conference on Multimedia. ACM, 2022. 2

    2022

  20. [28]

    Neural video com- pression with diverse contexts

    Jiahao Li, Bin Li, and Yan Lu. Neural video com- pression with diverse contexts. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 22616–22626, 2023

    2023

  21. [29]

    Neural video compres- sion with feature modulation

    Jiahao Li, Bin Li, and Yan Lu. Neural video compres- sion with feature modulation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 26099–26108, 2024. 2

    2024

  22. [30]

    E-nerv: Expedite neural video representation with disentangled spatial- temporal context, 2022

    Zizhang Li, Mengmeng Wang, Huaijin Pi, Kechun Xu, Jianbiao Mei, and Yong Liu. E-nerv: Expedite neural video representation with disentangled spatial- temporal context, 2022. 2, 3, 4, 5

    2022

  23. [31]

    Neural video compression using spatio-temporal priors.arXiv preprint arXiv:1902.07383, 2019

    Haojie Liu, Tong Chen, Ming Lu, Qiu Shen, and Zhan Ma. Neural video compression using spatio-temporal priors.arXiv preprint arXiv:1902.07383, 2019. 2

    1902 arXiv

  24. [32]

    Nirvana: Neural implicit representations of videos with adaptive networks and autoregressive patch-wise modeling

    Shishira R Maiya, Sharath Girish, Max Ehrlich, Hanyu Wang, Kwot Sin Lee, Patrick Poirson, Pengx- iang Wu, Chen Wang, and Abhinav Shrivastava. Nirvana: Neural implicit representations of videos with adaptive networks and autoregressive patch-wise modeling. InProceedings of the ...

    2023

  25. [33]

    Latent-inr: A flexible framework for implicit repre- sentations of videos with discriminative semantics

    Shishira R Maiya, Anubhav Gupta, Matthew Gwilliam, Max Ehrlich, and Abhinav Shrivastava. Latent-inr: A flexible framework for implicit repre- sentations of videos with discriminative semantics. InEuropean Conference on Computer Vision, pages 285–302. Springer, 2024. 3

    2024

  26. [34]

    Practical full resolution learned lossless image compression

    Fabian Mentzer, Eirikur Agustsson, Michael Tschan- nen, Radu Timofte, and Luc Van Gool. Practical full resolution learned lossless image compression. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2019. 6, 1

    2019

  27. [35]

    Uvg dataset: 50/120fps 4k sequences for video codec analysis and development

    Alexandre Mercat, Marko Viitanen, and Jarno Vanne. Uvg dataset: 50/120fps 4k sequences for video codec analysis and development. InProceedings of the 11th ACM multimedia systems conference, pages 297–302,

  28. [36]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis, 2020. 2

    2020

  29. [37]

    Instant neural graphics primitives with a multiresolution hash encoding.ACM Transac- tions on Graphics, 41(4):1–15, 2022

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding.ACM Transac- tions on Graphics, 41(4):1–15, 2022. 2

    2022

  30. [38]

    Explaining the implicit neural canvas: Connecting pixels to neurons by tracing their contri- butions

    Namitha Padmanabhan, Matthew Gwilliam, Pulkit Kumar, Shishira R Maiya, Max Ehrlich, and Abhinav Shrivastava. Explaining the implicit neural canvas: Connecting pixels to neurons by tracing their contri- butions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  31. [39]

    Anderson, and Lubomir Bourdev

    Oren Rippel, Sanjay Nair, Carissa Lew, Steve Bran- son, Alexander G. Anderson, and Lubomir Bourdev. Learned video compression. InICCV, 2019. 1, 2

    2019

  32. [40]

    Anderson, Kedar Tat- wawadi, Sanjay Nair, Craig Lytle, and Lubomir Bour- dev

    Oren Rippel, Alexander G. Anderson, Kedar Tat- wawadi, Sanjay Nair, Craig Lytle, and Lubomir Bour- dev. Elf-vc: Efficient learned flexible-rate video cod- ing. InICCV, 2021. 2

    2021

  33. [41]

    Combining frame and gop embeddings for neural video representation

    Jens Eirik Saethre, Roberto Azevedo, and Christopher Schroers. Combining frame and gop embeddings for neural video representation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 9253–9263, 2024. 2

    2024

  34. [42]

    Baraniuk

    Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, and Richard G. Baraniuk. Wire: Wavelet implicit neural representations, 2023. 2

    2023

  35. [43]

    Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang

    Wenzhe Shi, Jose Caballero, Ferenc Husz ´ar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolu- tional neural network, 2016. 2

    2016

  36. [44]

    Implicit neural representations with periodic activation functions.Ad- vances in neural information processing systems, 33: 7462–7473, 2020

    Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions.Ad- vances in neural information processing systems, 33: 7462–7473, 2020. 2

    2020

  37. [45]

    Adversarial generation of continuous images,

    Ivan Skorokhodov, Savva Ignatyev, and Mohamed El- hoseiny. Adversarial generation of continuous images,

  38. [46]

    Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012. 5

    2012

  39. [47]

    Implicit neural represen- tations for image compression, 2022

    Yannick Str ¨umpler, Janis Postels, Ren Yang, Luc van Gool, and Federico Tombari. Implicit neural represen- tations for image compression, 2022. 2

    2022

  40. [48]

    Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand

    Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (hevc) standard.IEEE Transactions on Circuits and Systems for Video Technology, 2012. 2 10

    2012

  41. [49]

    Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Sing- hal, Ravi Ramamoorthi, Jonathan T

    Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Sing- hal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains.NeurIPS,

  42. [50]

    Multiscale structural similarity for image quality as- sessment

    Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality as- sessment. InThe Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, pages 1398–

    2003

  43. [51]

    Wiegand, G.J

    T. Wiegand, G.J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the h.264/avc video coding stan- dard.IEEE Transactions on Circuits and Systems for Video Technology, 2003. 2

    2003

  44. [52]

    Qs-nerv: Real-time quality-scalable de- coding with neural representation for videos

    Chang Wu, Guancheng Quan, Gang He, Xin-Quan Lai, Yunsong Li, Wenxin Yu, Xianmeng Lin, and Cheng Yang. Qs-nerv: Real-time quality-scalable de- coding with neural representation for videos. InPro- ceedings of the 32nd ACM International Conference on Multimedia, pages 2584–2592, 2024. 2

    2024

  45. [53]

    Signal processing for implicit neu- ral representations, 2022

    Dejia Xu, Peihao Wang, Yifan Jiang, Zhiwen Fan, and Zhangyang Wang. Signal processing for implicit neu- ral representations, 2022. 2

    2022

  46. [54]

    Vq-nerv: A vector quantized neural representation for videos, 2024

    Yunjie Xu, Xiang Feng, Feiwei Qin, Ruiquan Ge, Yong Peng, and Changmiao Wang. Vq-nerv: A vector quantized neural representation for videos, 2024. 2

    2024

  47. [55]

    Ds-nerv: Implicit neural video representation with decomposed static and dynamic codes

    Hao Yan, Zhihui Ke, Xiaobo Zhou, Tie Qiu, Xidong Shi, and Dadong Jiang. Ds-nerv: Implicit neural video representation with decomposed static and dynamic codes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23019–23029, 2024. 2

    2024

  48. [56]

    Gener- ating videos with dynamics-aware implicit generative adversarial networks, 2022

    Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung-Woo Ha, and Jinwoo Shin. Gener- ating videos with dynamics-aware implicit generative adversarial networks, 2022. 3

    2022

  49. [57]

    Boosting neural representations for videos with a con- ditional decoder

    Xinjie Zhang, Ren Yang, Dailan He, Xingtong Ge, Tongda Xu, Yan Wang, Hongwei Qin, and Jun Zhang. Boosting neural representations for videos with a con- ditional decoder. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 2556–25...

    2024

  50. [58]

    Implicit neural video compression, 2021

    Yunfan Zhang, Ties van Rozendaal, Johann Brehmer, Markus Nagel, and Taco Cohen. Implicit neural video compression, 2021. 2

    2021

  51. [59]

    Salman Asif, and Zhan Ma

    Qi Zhao, M. Salman Asif, and Zhan Ma. Dnerv: Mod- eling inherent dynamics via difference neural repre- sentation for videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 2031–2040, 2023. 2, 5

    2023

  52. [60]

    Salman Asif, and Zhan Ma

    Qi Zhao, M. Salman Asif, and Zhan Ma. Pnerv: Enhancing spatial consistency via pyramidal neural representation for videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 19103–19112, 2024. 2 11 How to Design and Train Your I...

    2024

  53. [61]

    short” train- ing time in Figure 13, and for “medium

    Additional Results We report PSNR and MS-SSIM vs. bpp for an average across the 7 videos we use in this paper, for “short” train- ing time in Figure 13, and for “medium” training time in Figure 14. When we compute an average, we average in both size and quality simultaneously....

  54. [62]

    short” training time. Figure 28.Quality vs. size, for Honey- Bee (top) and Jockey (bottom) at 1080p with “medium

    Implementation Details 7.1. NeRV Compression. We follow the quantization procedure as explained in HiN- eRV [21]. Any time we report bpp, we measure the ac- tual space requirement after compressing with torchac [34]. We then measure the quality after loading the compressed mod...

  55. [63]

    See a detailed diagram of the first NeRV block corresponding to that stem in Figure 35

    NeRV Walkthrough See a detailed diagram of the basic NeRV stem from HN- eRV [5] in Figure 34. See a detailed diagram of the first NeRV block corresponding to that stem in Figure 35. From there, the features are upsampled by subsequent NeRV blocks, with upsampling dictated by t...

  56. [64]

    unique parameters

    INR Hyper-Network Walkthrough We provide a brief walkthrough of the hyper-network setup, although it is explained in prior work as well [6]. To under- stand the INR hyper-network setup, it is easiest to start with the prediction of the hyper-network: the hypo-network. For our ...

  57. [65]

    Adapting XINC XINC, introduced in [38], is a framework designed to inves- tigate how neurons in an image or video INR encode signals they are trained to represent

    XINC Analysis Walkthrough 10.1. Adapting XINC XINC, introduced in [38], is a framework designed to inves- tigate how neurons in an image or video INR encode signals they are trained to represent. By dissecting the contributions of each neuron in every layer of the network to e...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.