Pith. sign in

REVIEW 4 major objections 6 minor 46 references

URVC claims a single real-time neural video codec can simultaneously adapt temporal prediction to bitrate and motion, let users steer spatial bit allocation at test time with no retraining or transmitted mask, and switch between signal-fide

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:21 UTC pith:ACRXQACC

load-bearing objection A solid real-time neural codec paper with well-ablated components, but the zero-shot ROI claim is under-tested and the perceptual-switching theory is thinner than the empirics. the 4 major comments →

arxiv 2607.15033 v1 pith:ACRXQACC submitted 2026-07-16 cs.MM

URVC: A Unified Real-Time Neural Video Coding Model with Temporal, Spatial, and Perceptual Adaptivity

Xihua Sheng , Chang Wen Chen This is my paper
classification cs.MM
keywords neural video codingreal-time codectemporal predictionspatial rate controlregion-of-interest codingperceptual qualitymodule bankrate-distortion optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that real-time neural video codecs do not have to be rigid. It proposes URVC, a codec that restores three kinds of adaptivity that traditional codecs have but learned real-time codecs lost: temporal prediction that picks among diverse prediction candidates according to rate-distortion cost; spatial rate control that splits features into structural and detail components so a user can steer bits to chosen regions at test time; and a perceptual-switching mechanism that lets one backbone serve both fidelity and perceptual-quality modes by swapping module banks. If right, a single real-time model replaces several separately trained codecs. Experiments claim an average -22.4% BD-rate over VTM-17.0, real-time 0.027s encode/0.028s decode for 1080p, perceptual-mode savings of -57.3% DISTS / -48.7% LPIPS vs DCVC-FM, and zero-shot ROI coding with 20-30% bitrate savings.

Core claim

The central discovery is that the fixed, once-trained strategy of real-time neural codecs can be opened up by introducing learnable per-QP quantization vectors as a module bank indexed by QP, and using them in three places: as fusion weights over multi-branch temporal prediction candidates, as separate structural/detail modulators for spatial bit allocation, and as switchable fidelity/perceptual parameter sets on a shared frozen backbone. The paper argues that the perception-distortion tradeoff constrains only the conditional distribution a codec realizes, not its architecture, so two modes can share a backbone and differ only in quantization vectors, the factorized entropy model, and the fr

What carries the argument

The load-bearing mechanism is the module bank: a set of channel-wise quantization vectors and a factorized entropy model indexed by QP, which modulate features at every stage. Temporal adaptivity comes from three structurally diverse prediction branches whose outputs are weighted by QP-indexed quantization vectors, making candidate selection a differentiable proxy for rate-distortion mode decision. Spatial adaptivity comes from decomposing features into a smooth structural part and a detail residual, each quantized by its own vector; at test time the encoder's decomposition can be replaced by a binary ROI mask, assigning the detail vector to foreground and the structural vector to background

Load-bearing premise

The perceptual mode assumes that a backbone trained for signal fidelity can express the perception-optimal conditional distribution by only rescaling channel amplitudes and swapping the frame generator; if the frozen backbone's features cannot represent what perceptual quality requires, the switching gain collapses.

What would settle it

Train the same architecture twice — once with the proposed two-stage perceptual switching, and once with the entire network trained from scratch for perceptual quality at matched parameter count. If the switched perceptual mode fails to match the from-scratch perceptual BD-rate in DISTS/LPIPS, then the shared-backbone sufficiency assumption is refuted. A simpler check: verify that the reported -22.4% average BD-rate is reproduced when the official DCVC-RT training protocol (64-frame cascaded training at 512x512) is used, since the paper's own reproduction DCVC-RT* is weaker than the released m

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A user can control where bits go in a compressed video at deployment time, without retraining, without ROI supervision, and with zero bitrate overhead for the region map.
  • One deployed model covers both fidelity-oriented and perception-oriented applications; switching modes costs negligible extra storage and no extra compute.
  • The per-QP module bank makes variable-rate coding and hierarchical quality structures natural within a single model.
  • The temporal multi-candidate mechanism gives real-time codecs a way to adapt to motion complexity that previously required explicit motion estimation and compression.
  • If the gains hold, real-time neural codecs become viable for interactive and user-driven streaming scenarios rather than fixed one-size-fits-all compression.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same mask-substitution trick could carry other test-time hints, such as saliency maps or text prompts, because the decoder never needs the mask and the encoder only swaps which pre-learned quantization vectors apply where.
  • Editorial inference: interpolating between the fidelity and perceptual module banks might yield a continuous perception-distortion knob, letting a single model trace the whole tradeoff curve rather than only two endpoints.
  • Editorial inference: the zero-shot ROI evidence covers two motion-control videos and one downstream generative model; whether 20-30% savings persist across diverse content, mask granularities, and downstream tasks is a natural testable extension.
  • Editorial inference: the perceptual-switching claim's upper bound is a from-scratch perceptual codec; comparing against that would clarify how much of the perception-optimal distribution the shared backbone can express.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes URVC, a real-time neural video codec that augments a DCVC-RT-style implicit coding architecture with three forms of adaptivity: (i) rate-aware temporal prediction via a multi-branch context predictor whose branch contributions are controlled by QP-indexed quantization vectors; (ii) decomposition-based spatial rate control that splits encoder features into smooth/detail components, quantized with separate QP-indexed vectors, and that can be replaced at test time by a user-defined ROI mask; and (iii) perceptual switching, in which a shared backbone is combined with two QP-indexed module banks and two frame generators to yield fidelity and perceptual modes. Experiments report -22.4% average BD-rate vs VTM-17.0, surpassing the official DCVC-RT; real-time 1080p coding (27/28 ms per frame); large DISTS/LPIPS gains for the perceptual mode; and 20-30% bitrate savings in a two-video motion-control task. Ablations support each proposed component. The central claim is that URVC simultaneously achieves temporal, spatial, and perceptual adaptivity within a single real-time model.

Significance. The paper addresses a genuine gap: real-time neural codecs have generally been rigid, and the proposed framework makes strides toward user-controllable bit allocation and mode switching without retraining. The architectural contributions are clearly described, the experiments span three standard datasets plus complexity and ablation studies, and the timing measurements show the system remains in the real-time regime. The zero-shot ROI coding idea—no mask transmission, no retraining—is practically attractive and, if validated, would be a meaningful advance. However, the perceptual evaluation is partially circular and the ROI validation is too thin to fully support the central claims; these issues require experimental reinforcement before the paper can be recommended for acceptance.

major comments (4)
  1. [Section IV-C and Table II] LPIPS appears both in the perceptual training loss (Sec. IV-C) and as an evaluation metric (Table II), so the reported -48.7% LPIPS BD-rate gain is substantially a self-measurement. Even though DISTS is not part of the loss, the authors should also report an independent perceptual metric (e.g., FID, NIQE, or a no-reference metric) and the PSNR difference between fidelity and perceptual modes at matched bitrates, so readers can assess the perception-distortion tradeoff. Without this, the headline 'substantially better perceptual quality' is not safely established.
  2. [Section III-C, Eqs. (14)-(15)] The test-time substitution of a user mask for the learned down/up structural-detail decomposition is an out-of-distribution operation. The vectors q_s/q_d and the entropy models are optimized for a frequency-like decomposition; applying q_s to a textured background and q_d to a foreground region changes encoder features in ways not seen in training, while the decoder still applies the original down/up decomposition. The paper does not demonstrate that actual bit allocation follows the mask; Section IV-E reports only downstream CLIP/motion similarity on two sequences and total bitrate savings, with no per-region quality or bitrate and no non-ROI matched-rate baseline. Please add per-region PSNR/bitrate measurements, more sequences, and a comparison with a simple global-QP adaptation to substantiate the zero-shot ROI claim.
  3. [Section III-D and Table V] The theoretical justification for perceptual switching is insufficient. The perception-distortion tradeoff constrains conditional distributions, not architectures; it does not imply that diagonal rescaling of intermediate features (Eq. 16) plus a switched frame generator can realize the perception-optimal conditional distribution. The ablation M5->M6 shows these modules help, but it does not compare against a fully retrained perceptual model with the same backbone and data. Such a comparison (or an upper-bound study) is needed to support the claim that 'only' a secondary module bank and frame generator are sufficient. At minimum, soften the theoretical claim and discuss the capacity limitation.
  4. [Table I and Section IV-B] The headline comparison against the official DCVC-RT relies on the released model's numbers, while the authors' own reproduction (DCVC-RT*) is 7.3 percentage points worse (-11.5% vs -18.8%). To make the 'surpasses' claim trustworthy, the authors should state exactly how the official numbers were produced (same test script, same VTM version and configuration, same YUV420 conversion) and provide bootstrap confidence intervals. Without code/weights, the reader cannot independently check the fairness of this comparison.
minor comments (6)
  1. [Fig. 1] The figure is very small and labels for module banks and D&Q blocks are hard to read. Please enlarge and consider splitting into separate subfigures.
  2. [Section III-B] The Ambiguity Decomposition (Eq. 4) is for linearly weighted estimators, whereas the proposed fusion is nonlinear (concatenation + convolution). Please clarify that this is an analogy rather than a rigorous decomposition of the actual objective.
  3. [Table III] Specify how encode/decode times were measured: number of frames averaged, whether arithmetic coding is included, GPU warm-up, and the bitrate/QP range. Without these details, the real-time claim is hard to reproduce.
  4. [Section IV-E, Eq. (18)] The motion similarity metric may be dominated by large background motion. Please report foreground-only or masked metrics, or justify why the global optical-flow cosine is appropriate.
  5. [Reproducibility] Consider releasing code and trained weights, at least for the main instantiation, to allow independent verification of the BD-rate and timing results.
  6. [Section IV-B] Define "intra period -1" for readers unfamiliar with the convention (all frames except the first are inter-coded). This is standard in the field but the abbreviation is unexplained.

Circularity Check

1 steps flagged

Minor self-measurement via LPIPS in both training loss and evaluation; main RD and DISTS claims remain externally benchmarked and independent.

specific steps
  1. other [Section IV-C (Training Details) and Section IV-F / Table II (perceptual BD-rate evaluation)]
    "For perceptual quality training, the distortion term is a weighted combination of YUV MSE, LPIPS, and a GAN loss [27]. ... our codec with the perceptual mode achieves an average BD-rate of −57.3% in DISTS and −48.7% in LPIPS, substantially outperforming both DCVC-FM and the released DCVC-RT"

    LPIPS is included in the perceptual mode's training loss, and the same metric is then used as a headline evaluation metric in Table II. The −48.7% LPIPS BD-rate therefore partly measures the model's training objective rather than an independent property: the optimizer directly minimizes LPIPS, so improvement on this metric is partly by construction. DISTS is not in the training loss and provides independent support, so this is a partial self-measurement rather than a collapse of the perceptual claim.

full rationale

The central temporal, spatial, and fidelity claims are not derived from fitted parameters. Table I is an external comparison against VTM-17.0 and the released DCVC-RT; Eq. 4–8 are standard ensemble/gradient identities used to motivate an architecture that is trained with ordinary rate–distortion losses. The spatial rate-control method (Eq. 11–15) is a design with a test-time substitution; the zero-shot ROI result is an empirical claim validated on two videos with downstream metrics, so its weakness is validation scope, not circularity. The perceptual-switching section invokes Blau–Michaeli and Eq. 16, but Eq. 16 is only a covariance-rescaling identity and does not itself establish that quantization-vector switching is distributionally sufficient; that is an unsupported architectural assumption, not a definitional reduction. The one genuine circularity-like element is the use of LPIPS both as a perceptual training loss (Section IV-C) and as a headline evaluation metric (Section IV-F / Table II), which makes the −48.7% LPIPS gain partially self-measured. DISTS, which is not part of the training loss, gives independent support, so the overall circularity is minor and the paper's primary contributions retain external content.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The paper's central claims rest on several trained/fitted parameter sets and three unproven transfer assumptions: branch fusion as RD mode selection, shared-backbone channel rescaling as sufficient for perceptual switching, and mask substitution as equivalent to learned structural/detail quantization. These are plausible engineering bets, but none is externally verified.

free parameters (4)
  • QP-indexed quantization vectors (q_s^{E/D}, q_d^{E/D}, q_p, q_c^{0..2}, q_r) = per-QP vectors, not individually reported
    Channel-wise multipliers learned end-to-end and indexed by QP; they carry the rate-distortion adaptivity claims in Eqs. 7, 13, and 15.
  • Factorized entropy models per mode/QP (φ_z) = per-QP/per-mode distributions
    Compress side information z_t; selected by QP and mode; distribution parameters are fitted during training.
  • Mode-specific frame generators θ_r^{Fid}, θ_r^{Per} = trained weights, not released
    Perceptual mode requires a separate decoder head; the reported 24.89M total parameters include these two instances.
  • λ(QP) interpolation and hierarchical QP offsets/distortion weights = λ ∈ [1,768]; offsets [0,8,0,4,0,4,0,4]; weights [1.2,0.5,0.9,0.5]
    Hand-set training schedule and distortion weighting; affects rate-quality balance but is not derived from first principles.
axioms (4)
  • standard math Ambiguity Decomposition theorem (Eq. 4) justifies multi-branch diversity for temporal prediction.
    The identity is valid, but its application assumes the three branches act as well-calibrated base estimators whose fusion weights sum appropriately.
  • ad hoc to paper Fusing three prediction candidates via learnable quantization vectors is equivalent to rate-distortion optimized mode selection.
    No discrete selection occurs; the paper claims the weighted fusion emerges from RD training, but Eq. 8 does not prove the weights select modes by motion complexity or bitrate.
  • ad hoc to paper A fidelity-trained shared backbone, with only quantization vectors, entropy models, and frame generators switched, can realize both fidelity-optimal and perception-optimal conditional distributions.
    Eq. 16 shows channel covariance rescaling, not that the perceptual conditional distribution is reachable within the shared backbone's expressivity.
  • ad hoc to paper A spatial foreground/background mask at test time inherits the properties of the structural/detail down-up decomposition used in training.
    The training decomposition separates low/high spatial frequency; the test decomposition separates content by location. The paper assumes q_s and q_d transfer without retraining, but no analysis establishes equivalence.
invented entities (2)
  • Module banks B^{Fid} and B^{Per} no independent evidence
    purpose: Enable fidelity/perceptual mode switching by swapping QP-indexed quantization vectors and factorized entropy models.
    Central architectural invention; no external evidence beyond the paper's own experiments, and no code or weights are released.
  • Dual frame generators θ_r^{Fid} and θ_r^{Per} no independent evidence
    purpose: Separate decoder heads for signal fidelity and perceptual quality modes.
    The necessity of a separate perceptual frame generator is asserted and ablated, but not derived from theory or verified externally.

pith-pipeline@v1.3.0-alltime-deepseek · 17716 in / 11141 out tokens · 126768 ms · 2026-08-02T00:21:11.951581+00:00 · methodology

0 comments
read the original abstract

Neural video coding has advanced rapidly, achieving competitive compression performance while also enabling real-time coding speed. Yet, existing codecs exhibit severe rigidity when deployed in dynamic environments, failing to adapt to different video content, user requirements, and quality preferences. First, to meet the real-time constraint, they discard explicit motion estimation and motion compression, thereby losing the ability to adapt temporal prediction to motion complexity and bitrate constraints. Second, their spatial bit allocation strategy is coarse and, once trained, is fixed. It cannot adapt to dynamic user requirements at test time, preventing users from freely controlling the spatial distribution of bits. Third, they cannot adapt their quality preference to varying application requirements without deploying separate models. We address all three limitations within a single real-time neural video codec--URVC, transforming a rigid system into a unified framework with temporal, spatial, and perceptual adaptivity. First, we propose a rate-aware adaptive temporal prediction method that generates diverse prediction candidates through a multi-candidate architecture and couples candidate selection directly to rate-distortion optimization. Second, we propose a decomposition-based spatial rate control method that achieves finer-grained spatial bit allocation through feature decomposition and separate quantization, and allows users to perform direct spatial rate control at test time without retraining. Third, we propose a perceptual switching method that only requires learning a secondary module bank alongside a frame generator, enabling a codec to switch between signal fidelity and perceptual quality modes.

Figures

Figures reproduced from arXiv: 2607.15033 by Chang Wen Chen, Xihua Sheng.

Figure 1
Figure 1. Figure 1: Architecture of our proposed real-time neural video codec with rate-aware adaptive temporal prediction method, decomposition-based spatial rate [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the rate-aware adaptive temporal prediction method. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the decomposition-based spatial rate control method. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of the perceptual switching method. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Rate-distortion curves on HEVC, UVG, and MCL-JCV datasets. All frames are coded with intra period –1 in YUV420 color space. The quality metric [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Illustration of video motion control as a typical ROI task. The [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of reference video compression and generated animation with/without our spatial rate control on the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 6
Figure 6. Figure 6: Consequently, the foreground (the person) requires high [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Rate-perception curves on HEVC, UVG, and MCL-JCV datasets. All frames are coded with intra period –1 in YUV420 color space. The reconstructed [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visual comparison between DCVC-FM [22], DCVC-RT [13], our codec in signal fidelity mode, and our codec in perceptual mode. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization of the predicted temporal contexts and prediction resid [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Illustration of the down-up sampling-based decomposition and user [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 2 linked inside Pith

  1. [1]

    An end- to-end learning framework for video compression,

    G. Lu, X. Zhang, W. Ouyang, L. Chen, Z. Gao, and D. Xu, “An end- to-end learning framework for video compression,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

    2020

  2. [2]

    Prediction and reference quality adaptation for learned video compression,

    X. Sheng, L. Li, D. Liu, and H. Li, “Prediction and reference quality adaptation for learned video compression,”IEEE Transactions on Image Processing, 2025. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    2025

  3. [3]

    Fvc: An end-to- end framework towards deep video compression in feature space,

    Z. Hu, D. Xu, G. Lu, W. Jiang, W. Wang, and S. Liu, “Fvc: An end-to- end framework towards deep video compression in feature space,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4569–4585, 2022

    2022

  4. [4]

    Scale-space flow for end-to-end optimized video compres- sion,

    E. Agustsson, D. Minnen, N. Johnston, J. Balle, S. J. Hwang, and G. Toderici, “Scale-space flow for end-to-end optimized video compres- sion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8503–8512, 2020

    2020

  5. [5]

    Learning image and video compression through spatial-temporal energy compaction,

    Z. Cheng, H. Sun, M. Takeuchi, and J. Katto, “Learning image and video compression through spatial-temporal energy compaction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10071–10080, 2019

    2019

  6. [6]

    Learned video compression,

    O. Rippel, S. Nair, C. Lew, S. Branson, A. G. Anderson, and L. Bour- dev, “Learned video compression,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3454–3463, 2019

    2019

  7. [7]

    End-to-end neural video coding using a compound spatiotemporal representation,

    H. Liu, M. Lu, Z. Chen, X. Cao, Z. Ma, and Y . Wang, “End-to-end neural video coding using a compound spatiotemporal representation,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 8, pp. 5650–5662, 2022

    2022

  8. [8]

    End-to-end rate-distortion optimized learned hierarchical bi-directional video compression,

    M. A. Yılmaz and A. M. Tekalp, “End-to-end rate-distortion optimized learned hierarchical bi-directional video compression,”IEEE Transac- tions on Image Processing, vol. 31, pp. 974–983, 2021

    2021

  9. [9]

    DMVC: Decomposed motion modeling for learned video compression,

    K. Lin, C. Jia, X. Zhang, S. Wang, S. Ma, and W. Gao, “DMVC: Decomposed motion modeling for learned video compression,”IEEE Transactions on Circuits and Systems for Video Technology, 2022

    2022

  10. [10]

    Learning cross- scale weighted prediction for efficient neural video compression,

    Z. Guo, R. Feng, Z. Zhang, X. Jin, and Z. Chen, “Learning cross- scale weighted prediction for efficient neural video compression,”IEEE Transactions on Image Processing, 2023

    2023

  11. [11]

    Conditional entropy coding for efficient video compression,

    J. Liu, S. Wang, W.-C. Ma, M. Shah, R. Hu, P. Dhawan, and R. Urtasun, “Conditional entropy coding for efficient video compression,” inEuro- pean Conference on Computer Vision (ECCV), pp. 453–468, Springer, 2020

    2020

  12. [12]

    Rdvc: Efficient deep video compression with regulable rate and complexity optimization,

    X. Wei, J. Lin, J. Xu, W. Gao, and T. Zhao, “Rdvc: Efficient deep video compression with regulable rate and complexity optimization,”IEEE Transactions on Multimedia, 2025

    2025

  13. [13]

    Towards practical real-time neural video compression,

    Z. Jia, B. Li, J. Li, W. Xie, L. Qi, H. Li, and Y . Lu, “Towards practical real-time neural video compression,” inProceedings of the Computer Vision and Pattern Recognition Conference, pp. 12543–12552, 2025

    2025

  14. [14]

    Mining temporal redundancy using long short-term motion aggregation and global-local decorrelation for learned video compression,

    F. Yuan, Z. Pan, J. Lei, B. Peng, H. Xie, F. L. Wang, and S. Kwong, “Mining temporal redundancy using long short-term motion aggregation and global-local decorrelation for learned video compression,”IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  15. [15]

    Canf- vc: Conditional augmented normalizing flows for video compression,

    Y .-H. Ho, C.-P. Chang, P.-Y . Chen, A. Gnutti, and W.-H. Peng, “Canf- vc: Conditional augmented normalizing flows for video compression,” inComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVI, pp. 207–223, Springer, 2022

    2022

  16. [16]

    Maskcrt: Masked conditional residual transformer for learned video compression,

    Y .-H. Chen, H.-S. Xie, C.-W. Chen, Z.-L. Gao, M. Benjak, W.-H. Peng, and J. Ostermann, “Maskcrt: Masked conditional residual transformer for learned video compression,”IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  17. [17]

    Temporal context min- ing for learned video compression,

    X. Sheng, J. Li, B. Li, L. Li, D. Liu, and Y . Lu, “Temporal context min- ing for learned video compression,”IEEE Transactions on Multimedia, vol. 25, pp. 7311–7322, 2022

    2022

  18. [18]

    Vnvc: A versatile neural video cod- ing framework for efficient human-machine vision,

    X. Sheng, L. Li, D. Liu, and H. Li, “Vnvc: A versatile neural video cod- ing framework for efficient human-machine vision,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  19. [19]

    Bi-directional deep contextual video compression,

    X. Sheng, L. Li, D. Liu, and S. Wang, “Bi-directional deep contextual video compression,”IEEE Transactions on Multimedia, 2025

    2025

  20. [20]

    Neural video compression with diverse contexts,

    J. Li, B. Li, and Y . Lu, “Neural video compression with diverse contexts,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22616–22626, 2023

    2023

  21. [21]

    Spatial decomposition and temporal fusion based inter prediction for learned video compression,

    X. Sheng, L. Li, D. Liu, and H. Li, “Spatial decomposition and temporal fusion based inter prediction for learned video compression,”IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  22. [22]

    Neural video compression with feature modulation,

    J. Li, B. Li, and Y . Lu, “Neural video compression with feature modulation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26099–26108, 2024

    2024

  23. [23]

    Roi video coding based on h. 263+ with robust skin-color detection technique,

    M.-J. Chen, M.-C. Chi, C.-T. Hsu, and J.-W. Chen, “Roi video coding based on h. 263+ with robust skin-color detection technique,”IEEE Transactions on Consumer Electronics, vol. 49, no. 3, pp. 724–730, 2003

    2003

  24. [24]

    A neural video codec with spatial rate-distortion control,

    N. Fathima, J. Petersen, G. Sauti `ere, A. Wiggers, and R. Pourreza, “A neural video codec with spatial rate-distortion control,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5365–5374, 2023

    2023

  25. [25]

    Roi-dvc: A region-of-interest based deep video coding framework,

    X. Wu, P. Wang, and X. Wang, “Roi-dvc: A region-of-interest based deep video coding framework,” in2024 IEEE International Conference on Image Processing (ICIP), pp. 1967–1972, IEEE, 2024

    1967

  26. [26]

    Roi-aware dynamic network quantization for neural video compression,

    J. Liu, B. Zhang, and X. Cao, “Roi-aware dynamic network quantization for neural video compression,” inInternational Conference on Pattern Recognition, pp. 333–349, Springer, 2024

    2024

  27. [27]

    Perceptual learned video compression with recurrent conditional gan.,

    R. Yang, R. Timofte, and L. Van Gool, “Perceptual learned video compression with recurrent conditional gan.,” inIJCAI, pp. 1537–1544, 2022

    2022

  28. [28]

    High visual-fidelity learned video compression,

    M. Li, Y . Shi, J. Wang, and Y . Huang, “High visual-fidelity learned video compression,” inProceedings of the 31st ACM International Conference on Multimedia, pp. 8057–8066, 2023

    2023

  29. [29]

    Generative latent coding for ultra-low bitrate image and video compression,

    L. Qi, Z. Jia, J. Li, B. Li, H. Li, and Y . Lu, “Generative latent coding for ultra-low bitrate image and video compression,”IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  30. [30]

    Diffusion-based perceptual neural video com- pression with temporal diffusion information reuse,

    W. Ma and Z. Chen, “Diffusion-based perceptual neural video com- pression with temporal diffusion information reuse,”ACM Transactions on Multimedia Computing, Communications and Applications, vol. 21, no. 12, pp. 1–22, 2025

    2025

  31. [31]

    Diffvc-rt: Towards practical real-time diffusion-based perceptual neural video compression,

    W. Ma and Z. Chen, “Diffvc-rt: Towards practical real-time diffusion-based perceptual neural video compression,”arXiv preprint arXiv:2601.20564, 2026

    arXiv 2026

  32. [32]

    Generative neural video compression via video diffusion prior,

    Q. Mao, H. Cheng, T. Yang, L. Jin, and S. Ma, “Generative neural video compression via video diffusion prior,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 43239– 43248, 2026

    2026

  33. [33]

    Progvc: Progressive-based generative video compression via auto-regressive con- text modeling,

    D. Li, R. Dong, Y . Chen, K. Li, D. Ding, and L. Li, “Progvc: Progressive-based generative video compression via auto-regressive con- text modeling,”arXiv preprint arXiv:2603.17546, 2026

    arXiv 2026

  34. [34]

    Neural network ensembles, cross validation, and active learning,

    A. Krogh and J. Vedelsby, “Neural network ensembles, cross validation, and active learning,”Advances in neural information processing systems, vol. 7, 1994

    1994

  35. [35]

    The perception-distortion tradeoff,

    Y . Blau and T. Michaeli, “The perception-distortion tradeoff,” inPro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 6228–6237, 2018

    2018

  36. [36]

    Video enhance- ment with task-oriented flow,

    T. Xue, B. Chen, J. Wu, D. Wei, and W. T. Freeman, “Video enhance- ment with task-oriented flow,”International Journal of Computer Vision, vol. 127, no. 8, pp. 1106–1125, 2019

    2019

  37. [37]

    Overview of the high efficiency video coding (HEVC) standard,

    G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649– 1668, 2012

    2012

  38. [38]

    UVG dataset: 50/120fps 4k sequences for video codec analysis and development,

    A. Mercat, M. Viitanen, and J. Vanne, “UVG dataset: 50/120fps 4k sequences for video codec analysis and development,” inProceedings of the 11th ACM Multimedia Systems Conference, pp. 297–302, 2020

    2020

  39. [39]

    MCL-JCV: a JND- based H.264/A VC video quality assessment dataset,

    H. Wang, W. Gan, S. Hu, J. Y . Lin, L. Jin, L. Song, P. Wang, I. Katsavounidis, A. Aaron, and C.-C. J. Kuo, “MCL-JCV: a JND- based H.264/A VC video quality assessment dataset,” in2016 IEEE International Conference on Image Processing (ICIP), pp. 1509–1513, IEEE, 2016

    2016

  40. [40]

    Overview of the versatile video coding (VVC) standard and its applications,

    B. Bross, Y .-K. Wang, Y . Ye, S. Liu, J. Chen, G. J. Sullivan, and J.- R. Ohm, “Overview of the versatile video coding (VVC) standard and its applications,”IEEE Transactions on Circuits and Systems for Video Technology, 2021

    2021

  41. [41]

    Image quality assess- ment: Unifying structure and texture similarity,

    K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Image quality assess- ment: Unifying structure and texture similarity,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2567– 2581, 2020

    2020

  42. [42]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586–595, 2018

    2018

  43. [43]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning, pp. 8748–8763, PMLR, 2021

    2021

  44. [44]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  45. [45]

    Ecvc: Exploiting non-local correlations in multiple frames for contextual video compression,

    W. Jiang, J. Li, K. Zhang, and L. Zhang, “Ecvc: Exploiting non-local correlations in multiple frames for contextual video compression,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, pp. 7331–7341, 2025

    2025

  46. [46]

    Seedance 2.0: Advancing video generation for world complexity,

    T. Seedance, D. Chen, L. Chen, X. Chen, Y . Chen, Z. Chen, Z. Chen, F. Cheng, T. Cheng, Y . Cheng,et al., “Seedance 2.0: Advancing video generation for world complexity,”arXiv preprint arXiv:2604.14148, 2026

    Pith/arXiv arXiv 2026