Pith. sign in

REVIEW 5 major objections 4 minor 18 references

Highly Efficient Non-Separable Transforms for Next Generation Video Coding

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Signal-adaptive video transforms can be built from cheap hypercube rotations that match KLT coding gains while using about 6.8 times less memory.

desk verdict HyGT is a genuinely useful parametric transform family that appears to match KLT coding gain with 5–7x less memory, but the paper's refusal to disclose the optimization procedure makes the central claim unreproducible. read the letter →

arxiv 2505.21728 v1 pith:P3NUTPD4 submitted 2025-05-27 eess.IV

Amir Said , Xin Zhao , Marta Karczewicz , Hilmi E. Egilmez , Vadim Seregin , Jianle Chen This is my paper
classification eess.IV
keywords signal-adaptivetransformcodingHypercube-GivensKarhunen-LoèvevideocompressionHEVCsecondarylow-complexityorthogonaltransformsgainGivensrotations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video coding has long faced a trade-off: the Karhunen-Loève transform (KLT) gives excellent signal-adaptive compression, but its dense matrices demand too much memory and computation for practical codecs. This paper proposes a parametric family called Hypercube-Givens Transforms (HyGTs), in which every transform is a product of parallel Givens rotations arranged along hypercube dimensions, so only rotation angles need to be stored. The authors optimize those angles against a variance-based coding-gain model and test the transforms as secondary transforms in HEVC under All-Intra conditions. They report that HyGTs essentially match the KLT's average 6% bit-rate reduction over the HEVC anchor while using 6.8 times less memory for the stored transform set. If this holds, the main practical obstacle to signal-adaptive transform coding—not performance but implementation cost—is removed.

What carries the argument

The central object is the Hypercube-Givens Transform (HyGT), a special case of Parametric Multi-pass Transform (PMPT). A HyGT round is a sequence of $\log_2 N$ passes, each pass applying $N/2$ Givens rotations whose index pairs are the edges of a $\log_2 N$-dimensional hypercube in one direction. Because the $N/2$ rotations in a pass act on disjoint pairs, they commute and can be computed in parallel; the full transform is a product of these sparse orthogonal matrices over $R$ rounds and is therefore orthogonal. The parameter vector is the set of all rotation angles, and the pass structure is fixed, so storing a transform costs $R N \log_2 N / 2$ angle values instead of $N^2$ matrix entries. The optimization machinery is a non-linear search over this angle vector, maximizing a variance-based transform coding gain derived from the high-rate Laplacian model, with multiple initializations to cope with non-convexity.

What would settle it

Re-run the HEVC secondary-transform experiments but choose transforms by directly measuring actual rate-distortion cost on the training data instead of using the variance-based Laplacian proxy. If HyGTs selected by the proxy lose their reported parity with KLTs, or if the proxy ranking disagrees substantially with the true rate-distortion ranking, then the load-bearing approximation is the point of failure.

Watch

Extended reading notes

Core claim

The paper's central claim is that searching for optimal transform parameters, rather than optimal full matrices, yields practical signal-adaptive transforms that are nearly as good as KLTs. For each residual class, a HyGT consists of R rounds of log2(N) parallel Givens passes; each pass decomposes into N/2 independent rotations on disjoint index pairs defined by a hypercube edge pattern, and the free parameters are just the RN log2(N)/2 rotation angles. The optimization maximizes a high-rate Laplacian transform coding gain computed from the variances v(h)=diag(T(h)ΦT(h)^T), repeated from several starting points because the problem is non-convex. In HEVC All-Intra tests, the best HyGT combinations achieve average BD-rate reductions of 6.1–6.2% versus 6.0% for KLT combinations, with a memory usage ratio of about 6.8 for the average of 4x4 and 8x8 transform sets. The paper also shows that one byte per angle suffices, with sine and cosine values produced on the fly from a shared table.

Load-bearing premise

The argument stands or falls on whether a simplified statistical score based only on transformed variances correctly predicts which transform will compress best in the actual video codec, since the paper optimizes HyGT angles with that proxy rather than with full encoder simulations.

Editorial extensions

If this is right

  • HyGT-based secondary transforms can replace KLT-based ones in an HEVC-style codec with essentially the same average bit-rate savings (about 6% in All-Intra) at roughly 5–7 times lower transform memory.
  • The one-byte-per-angle representation means transform parameters are small enough to embed in a decoder's tables, with sine and cosine values generated on the fly from a shared lookup table.
  • Because each pass consists of $N/2$ independent rotations, HyGT computation parallelizes evenly across processors, making non-separable adaptive transforms practical for block sizes and transform counts where matrix KLTs are not.
  • The orthogonal, reversibly-passed structure fits existing secondary-transform coding designs without changing rate-distortion optimization or inverse-transform semantics beyond replacing the matrix product.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step is to optimize HyGT angles with a differentiable rate-distortion loss inside a learned codec; the fixed hypercube topology gives a small parameter space that might train faster and generalize better than dense learned transform matrices.
  • The memory advantage grows with the number of transform classes, since HyGT stores only angles per class while KLT stores $N^2$ entries per class; future standards using more mode-dependent classes would likely see relative savings beyond the reported 6.8x.
  • The same fixed-topology parametric idea could transfer to adaptive filtering or graph-based operators where dense kernel storage is the bottleneck; that is a testable extension, not a claim made in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes Hypercube-Givens Transforms (HyGT), a parametric class of orthogonal transforms built from Givens rotations arranged in hypercube passes, as low-complexity non-separable secondary transforms for video coding. The authors optimize HyGT parameters offline on residuals generated with the JEM-2.0 software, implement the transforms as an extension of HEVC, and evaluate them under All-Intra common test conditions. Table II reports BD-rate results of about -6% on average relative to an HEVC anchor, with a memory-usage ratio of up to 6.8x in favor of HyGT compared to KLT-based secondary transforms. The central claim is that HyGTs achieve KLT-level coding gain while requiring significantly less memory and offering parallelizable implementations.

Significance. If the results are reproducible, this work is significant: it offers a practical path to signal-adaptive non-separable transforms in video coding, where matrix-based KLTs have historically been too complex in memory and operations. The strengths are a clear transform construction with an explicit complexity analysis, a fair comparison (KLT and HyGT trained on the same JEM-2.0 data and evaluated on the same test set), and a genuinely lower parameter count for HyGT. However, the main technical claim rests on an optimization whose details are not provided, and the experimental scope is narrow (only All-Intra, no inter configurations, no error bars). These issues currently limit the paper's reproducibility and the strength of the conclusions.

major comments (5)
  1. [Section V, Eqs. (10)-(11)] The optimization objective and algorithm are not specified. The text says 'use a variation of (11) based on Laplacian distribution' and then 'Further details of the optimization are out of the scope of this paper.' Since the HyGT parameters are entirely the output of this non-convex optimization, the BD-rate results in Table II cannot be independently reproduced, and the claimed parity with KLT cannot be distinguished from an artifact of particular (unreported) optimization choices. Please provide the exact objective function, the training data preparation (including classification and residual statistics), the optimization algorithm, and the hyperparameters, or a complete reference to a public description.
  2. [Section VI, first paragraph and Table II] The experimental anchor is ambiguous. The text states the BD-rate is measured 'benchmarking against the reference software (HM 16.6)' while also saying the transforms are trained and tested in JEM-2.0. If the reported BD-rate is JEM-2.0+transform versus HM 16.6, then the ~6% gain includes all JEM-2.0 coding tools and cannot be attributed to HyGT or KLT. If the intended anchor is JEM-2.0 without the secondary transform, that should be stated explicitly, and the corresponding BD-rate for each scheme should be reported against that anchor. The abstract's claim of a '6% coding gain' should be reworded to reflect the exact comparison.
  3. [Table II, memory usage ratio] The memory usage ratio is computed as the ratio of stored parameter counts (N^2 for KLT versus R N log2(N)/2 for HyGT). This omits the shared sine/cosine table shown in Fig. 6, the different bit depths of stored values (HyGT angles are 8-bit; KLT coefficients are typically higher precision), and any implementation overhead such as addressing logic. The abstract's claim of '6.8 times less memory' is therefore not a direct measure of implementation memory. Please present the memory comparison in terms of actual byte footprint under stated assumptions, or clearly label the ratio as a parameter-count ratio.
  4. [Section VI, experimental scope] Only the All-Intra configuration is tested, and no statistical variability is reported. The BD-rate differences between schemes in Table II are often only 0.1-0.2 percentage points (e.g., -6.0% vs -6.2% on average), which may be within run-to-run noise. Please report results for at least one inter configuration (Random Access or Low-Delay) and provide confidence intervals or multiple-run variability, or temper the generalization that HyGT 'closely approaches' KLT performance.
  5. [Section V, Eqs. (10)-(11)] The optimization maximizes a simplified variance-based coding gain under a Laplacian high-rate model rather than the actual encoder rate-distortion performance. The paper acknowledges that true RD optimization requires 'extremely long sets of simulations' and that the simplified model is an approximation, but it provides no evidence that this proxy reliably ranks transforms in the real codec. Without such evidence, or a description of any final fine-tuning using the actual encoder, the claim that the optimized HyGTs match KLT performance is not fully established. Please provide a validation of the proxy (e.g., comparing proxy-optimal and RD-optimal parameters for a subset of classes) or describe a fine-tuning step.
minor comments (4)
  1. [Abstract and Introduction] There are typographical errors in the provided text, such as 'sty mied' (should be 'stymied') and 'Lo` eve' in the Karhunen-Loève transform name; please proofread carefully.
  2. [Section II, Table I] The complexity expressions in Table I are typeset inconsistently (e.g., missing superscripts in some cells). Please ensure the table is rendered legibly with proper mathematical notation.
  3. [Section VII, Conclusions] The word 'achived' should be 'achieved'.
  4. [Section VI, footnote 2] The URL 'https://hevc.hhi.fraunhofer.de/svn/svn HEVCSoftware/' contains a space; it should be a single hyphenated or underscore-connected path.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the coding-gain claim is an external empirical result, not a restatement of the objective used to train HyGT parameters.

full rationale

The paper's central claim is that HyGTs match KLT-level BD-rate gains (about 6% vs. HEVC) with 5-7x lower memory. That claim is established by full codec simulations in Section VI under All-Intra CTC on standard test sequences, with both HyGTs and KLTs trained on a separate JEM-2.0 residual dataset: 'both HyGTs and KLTs are designed by training on a dataset of transform coefficients generated using the JEM-2.0 video coding software... on a different video dataset given in the common test conditions'. The BD-rate numbers are therefore not the variance-based objective in Section V renamed as a prediction; they are measured encoder outputs on held-out sequences. The KLT comparison is controlled because both transform families are optimized on the same training data and evaluated under identical conditions. Self-citations [7], [17], and [18] are used for textbook rate-distortion formulas and for NSST implementation details; neither is load-bearing for the HyGT efficiency claim. Section V's disclosure that exact optimization details are 'out of the scope' is a reproducibility and missing-support limitation, not circularity: nothing in the paper defines the reported BD-rate to equal the optimized proxy by construction. No step in the derivation chain reduces Eqs. (10)-(11) to the Table II results; the evaluation pipeline is external and falsifiable.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The paper's contribution rests on the validity of a variance-based high-rate proxy for RD performance, the generalization of JEM-trained parameters to HEVC test sequences, and the existence of many near-optimal transforms. The optimization procedure itself is under-specified; the rotation angles are the key fitted parameters.

free parameters (4)
  • HyGT rotation angles = not listed (R*N*log2(N)/2 angles per transform)
    Optimized to maximize variance-based transform coding gain on JEM-2.0 training residuals; the central adaptive component.
  • Number of rounds R = 2, 3, 4, 5 in different schemes
    Chosen by the authors; more rounds increase complexity and slightly improve coding gain (see Table II).
  • Non-convex optimizer initializations = not specified
    The objective is non-convex; the best local solution depends on the initialization set, which is not disclosed.
  • Laplacian model and high-rate approximation constants = implicit in Eq. 11
    Used as the optimization objective; the model choice affects the resulting transforms and reported gains.
assumptions (5)
  • domain assumption Transform coding gain can be approximated by variances of transformed residuals under a high-rate Laplacian model (Eqs. 10-11).
    Invoked in Section V as the optimization objective; if this proxy is poor, optimized HyGTs may not achieve real encoder gains.
  • domain assumption Training on JEM-2.0 residual statistics generalizes to HEVC CTC test sequences.
    The paper trains on JEM residuals and tests on different sequences in CTC; cross-software and cross-dataset transfer is assumed.
  • domain assumption A small set of nearly optimal transforms exists around the KLT (Fig. 1 premise).
    Foundational premise for the parametric search; if false, low-complexity transforms cannot match KLT.
  • standard math Orthogonal matrices can be decomposed into sparse parallel-Givens passes with hypercube indexing.
    Uses standard linear algebra and FFT-style indexing; mathematically valid.
  • domain assumption The non-convex optimization with multiple initializations finds a sufficiently good local optimum.
    Not proven; accepted practice in signal processing, but no guarantee of global optimality.
invented entities (1)
  • PMPT/HyGT transform class independent evidence
    purpose: Parameterized low-complexity non-separable transforms for video coding
    The transform class is newly introduced; its performance is empirically verified by BD-rate and memory measurements in Table II.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Highly Efficient Non-Separable Transforms for Next Generation Video Coding." pith.science (2026). https://pith.science/paper/P3NUTPD4

@misc{pith2026250521728,
  author       = {Pith},
  title        = {Pith review of: Highly Efficient Non-Separable Transforms for Next Generation Video Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P3NUTPD4}},
  note         = {Machine review of arXiv:2505.21728}
}
read the original abstract

For the last few decades, the application of signal-adaptive transform coding to video compression has been stymied by the large computational complexity of matrix-based solutions. In this paper, we propose a novel parametric approach to greatly reduce the complexity without degrading the compression performance. In our approach, instead of following the conventional technique of identifying full transform matrices that yield best compression efficiency, we look for the best transform parameters defining a new class of transforms, called HyGTs, which have low complexity implementations that are easy to parallelize. The proposed HyGTs are implemented as an extension of High Efficiency Video Coding (HEVC), and our comprehensive experimental results demonstrate that proposed HyGTs improve average coding gain by 6% bit rate reduction, while using 6.8 times less memory than KLT matrices.

Figures

Figures reproduced from arXiv: 2505.21728 by the authors.

Figure 1
Figure 1. The new design method exploits the fact that for media [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. “Butterfly” structure for computing one Givens rotat [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. The complete HyGT is composed of one or more rounds, ea [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (1 more)
Figure 6
Figure 6. Figure 6: System for “on-the-fly” conversion of transform para [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    I. E. Richardson, The H.264 Advanced Video Compression Standard , 2nd ed. West Sussex, England: John Wiley and Sons Ltd., 2010

    work page 2010

  2. [2]

    Wien, High Efficiency Video Coding: Coding Tools and Specification

    M. Wien, High Efficiency Video Coding: Coding Tools and Specification. Berlin: Springer-V erlag, 2015

    work page 2015

  3. [3]

    The latest open-source video codec VP 9 – an overview and preliminary results,

    D. Mukherjee, J. Bankoski, R. S. Bultje, A. Grange, J. Han , J. Koleszar, P . Wilkins, and Y . Xu, “The latest open-source video codec VP 9 – an overview and preliminary results,” in Proc. 30th Picture Coding Symp. , San Jose, CA, Dec. 2013

    work page 2013

  4. [4]

    Sayood, Introduction to Data Compression , 2nd ed

    K. Sayood, Introduction to Data Compression , 2nd ed. San Francisco, CA: Morgan Kaufmann Publishers, 2000

    work page 2000

  5. [5]

    JPEG2000: standard fo r interac- tive imaging,

    D. S. Taubman and M. W. Marcellin, “JPEG2000: standard fo r interac- tive imaging,” Proc. IEEE , vol. 90, no. 8, pp. 1336–1357, Aug. 2002

    work page 2002

  6. [6]

    J. W. Woods, Multidimensional Signal, Image, and Video Processing and Coding , 2nd ed. Waltham, MA: Academic Press, 2011

    work page 2011

  7. [7]

    W. A. Pearlman and A. Said, Digital Signal Compression: Principles and Practice . Cambridge, UK: Cambridge University Press, 2011

    work page 2011

  8. [8]

    P . P . V aidyanathan, Multirate Systems and Filter Banks . New Jersey: Prentice Hall, Sep. 1993

    work page 1993

Show all 18 references
  1. [9]

    Strang, Introduction to Linear Algebra , 4th ed

    G. Strang, Introduction to Linear Algebra , 4th ed. Wellesley, MA: Wellesley-Cambridge Press, 2009

    2009

  2. [10]

    Britanak, P

    V . Britanak, P . C. Yip, and K. R. Rao, Discrete Cosine and Sine Trans- forms: General Properties, Fast Algorithms and Integer App roximations. San Diego, CA: Academic Press, 2007

    2007

  3. [11]

    Gilmore, Lie Groups, Lie Algebras, and Some of Their Applications

    R. Gilmore, Lie Groups, Lie Algebras, and Some of Their Applications . New Y ork, NY: John Wiley and Sons, 1974

    1974

  4. [12]

    Design of low-complexity, non-sep arable 2- D transforms based on butterfly structures,

    H. Chen and B. Zeng, “Design of low-complexity, non-sep arable 2- D transforms based on butterfly structures,” in Proc. IEEE Int. Symp. Circuits Syst. , Seoul, South Korea, May 2012, pp. 2921–2924

    2012

  5. [13]

    The solution of singular-valu e and symmetric eigenvalue problems on multiprocessor arrays,

    R. P . Brent and F. T. Luk, “The solution of singular-valu e and symmetric eigenvalue problems on multiprocessor arrays,” SIAM J. Sci. and Stat. Comput., vol. 6, no. 1, pp. 69–84, Jan. 1985

    1985

  6. [14]

    A. V . Oppenheim and R. W. Schafer, Discrete-Time Signal Processing , 3rd ed. Upper Saddle River, NJ: Prentice-Hall, Inc., Aug. 20 09

  7. [15]

    Calculation of average PSNR differen ces between RD- curves,

    G. Bjøntegaard, “Calculation of average PSNR differen ces between RD- curves,” ITU-T SG16 Q.6, Austin, TX, Tech. Rep. VCEG-M33, Ap r. 2001

    2001

  8. [16]

    Common test conditions and software refere nce configura- tions,

    F. Bossen, “Common test conditions and software refere nce configura- tions,” Joint Collaborative Team on Video Coding (JCT-VC) o f ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, San Jose, CA, Tech. Rep . JCTVC-H1100, Feb. 2011

    2011

  9. [17]

    Mode-dependent no n-separable secondary transform,

    X. Zhao, J. Chen, and M. Karczewicz, “Mode-dependent no n-separable secondary transform,” ITU-T SG16/Q6, Tech. Rep. Doc. COM16 - C1044, Oct. 2015

    2015

  10. [18]

    Improved non-separable secondary transform,

    X. Zhao, A. Said, V . Seregin, M. Karczewicz, and J. Chen, “Improved non-separable secondary transform,” Joint Video Explorat ion Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, Geneva , CH, Input Document JVET-C0063, May 2016

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.