Pith. sign in

REVIEW 3 major objections 5 minor 25 references

Low-Complexity Transform Adjustments For Video Coding

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Sparse orthogonal adjustment stages can replace VVC's DST-7 and DCT-8 with near-zero coding loss and a big complexity cut.

desk verdict Solid engineering write-up of JVET transform-adjustment designs, with a clean optimization framework and real throughput gains; the coding-efficiency evidence is thinner than the conclusion. read the letter →

arxiv 2505.23618 v1 pith:TJKJZZQ7 submitted 2025-05-29 eess.IV

classification eess.IV
keywords transformcodingvideocompressionDST-7DCT-8low-complexityapproximationsparseorthogonalmatricesweightedleastsquaresasymmetrictransforms
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to establish that the asymmetric transforms used in Versatile Video Coding, DST-7 and DCT-8, can be replaced by cheap approximations built from a DCT-2-family transform plus sparse orthogonal correction stages without losing coding efficiency. The core argument is that only the low-frequency basis vectors of these transforms matter for compression, so an optimization with exponentially larger weights on low frequencies can fit those rows accurately and leave high-frequency rows approximate. In tests on the VVC reference software, the approximations hold BD-rate changes to within a few hundredths of a percent on average, while worst-case multiplications per coefficient fall from 64 to 36 and 64-point asymmetric transforms become affordable. If the claim holds, the practical cost of multiple-transform coding drops substantially, making the compression gains of DST-7/DCT-8 viable in hardware where full matrix transforms are too expensive.

What carries the argument

The load-bearing object is the factored approximation $H = C B A \approx D$, with $B$ chosen from the DCT-2 family of transforms that share the DCT-2's efficient computational structure, and $A$ or $C$ restricted to sparse orthogonal matrices: a $K$-tap band matrix in the pre-adjustment variant, or a top-left $8\times 8$ sub-block in the post-adjustment variant. The optimization enforces orthogonality through products of Givens rotations and minimizes the weighted least-squares objective $\sum_{i=0}^{N-1}\sum_{j=0}^{N-1} e^{-\alpha i}(H_{i,j}-D_{i,j})^2$ with $\alpha = \ln(10^2)/N$. Two algebraic identities carry extra weight: $T(C_8)=S T(S_7) R$ links DCT-8 to DST-7, and in the sub-block case $C(C_8)=S C(S_7) S$, so the encoder can compute both adjustments in one pass and leave most coefficients unchanged.

What would settle it

Encode a set of high-texture or noisy 4K sequences with the proposed 8x8 sub-block adjustment under VVC all-intra common test conditions and compare BD-rate against the full DST-7/DCT-8; if the loss exceeds the paper's reported noise margin by a clear amount, such as more than 0.5%, the low-frequency-weighting premise would be refuted.

Watch

Extended reading notes

Core claim

The central discovery is that a desired orthogonal transform D can be factored approximately as H = C B A, where B is one of the efficiently computable DCT-2-type transforms (DCT-2, DCT-3, DST-2, or DST-3) and A and C are sparse orthogonal adjustment matrices, either a K-tap band matrix before B or a small top-left sub-block matrix after B. The adjustment parameters come from minimizing a weighted least-squares error that places exponentially larger weight on the low-frequency basis-vector rows, reflecting their dominant role in coding gain. Applied to DST-7, with DCT-8 obtained through the identity $T(C_8)=S T(S_7) R$, the method yields approximations whose low-frequency basis functions match the target almost exactly while high-frequency functions diverge in ways that do not hurt compression. The VVC reference software tests show essentially no coding loss, while the 8x8 sub-block adjustment lowers worst-case complexity to 36 multiplications per coefficient and enables 64-point asymmetric transforms without increasing that worst case.

Load-bearing premise

The approach relies on the assumption that reproducing the low-frequency basis vectors of DST-7 and DCT-8 with much higher fidelity than the high-frequency ones is enough to preserve the coding gain, so approximation errors in the high-frequency rows can be neglected.

Editorial extensions

If this is right

  • With the 8x8 sub-block adjustment, worst-case complexity in the VVC reference software drops from 64 to 36 multiplications per coefficient, and memory decreases by about 1 kilobyte.
  • 64-point DST-7 and DCT-8 can be enabled with no increase in worst-case complexity, adding up to 0.45% BD-rate gains on UHD content.
  • Average all-intra BD-rate differences versus the reference remain within roughly a tenth of a percent for typical classes, and the only class exceeding that range is A1, where the adjustment adds up to 0.45% gain on UHD content.
  • The DST-7/DCT-8 sign and flip relations let an encoder evaluate all horizontal/vertical transform combinations at the cost of one adjustment, with throughput more than five times that of full matrix multiplication in the 8x8 case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same weighted least-squares adjustment recipe could be exported to other compression transforms, such as DST-1 variants or non-separable secondary transforms, wherever a fast base transform exists and only low-frequency fidelity is rate-distortion-relevant.
  • Because the objective downweights high-frequency basis vectors exponentially, the approximation may lose more coding gain on content with unusually strong high-frequency residual energy; a targeted test on textured, noisy, or screen-content sequences would reveal whether the empirical safety margin covers that regime.
  • The reported complexity is in multiplications per coefficient and software throughput ratios; actual silicon area and energy savings depend on how the sparse adjustment stages are mapped to hardware, which the paper does not model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes a method for approximating the DST-7 and DCT-8 transforms used in VVC by composing an efficient DCT-2-family transform with sparse orthogonal "adjustment" matrices. The adjustment matrices are optimized with a weighted least-squares criterion (Eq. 2) that emphasizes low-frequency basis vectors. Two designs are presented: a band-matrix pre-adjustment before DST-3, and an 8x8 sub-block post-adjustment after DCT-2. Experiments in VTM-3.0 under all-intra CTC report BD-rate differences within roughly +/-0.1% of the reference, while Table 1 reports throughput improvements up to about 5.7x, and the conclusion states that enabling 64-point DST-7/DCT-8 does not increase worst-case complexity.

Significance. If the claims are correct, this is a practical and potentially influential complexity-reduction technique for multiple transform selection in video coding. The optimization framework is general and could be reused for other transform pairs, and the 8x8 sub-block design is a clever way to share computation between DST-7 and DCT-8. The reported throughput gains are substantial. However, the manuscript's evidence is narrower than its conclusions: only all-intra results are shown, no statistical characterization is provided, and one table appears internally inconsistent with the exact-transform baseline. The central modeling premise -- that low-frequency fidelity alone preserves coding gain -- is plausible but is not rigorously established in the text. The paper is clearly written and the mathematical framework is coherent, but the experimental support needs strengthening before the claims can be fully accepted.

major comments (3)
  1. [Section 2, Eq. (2) and Fig. 2] The weighting alpha = ln(100)/N in Eq. (2) downweights the highest-frequency basis vectors by a factor of about 100 relative to the lowest-frequency ones, so the objective places essentially no constraint on high-frequency rows. For the 8x8 post-adjustment, where C is the identity except for the top-left 8x8 block, rows 8..N-1 of H are exactly rows of DCT-2, not DST-7 or DCT-8. The paper does not bound this high-frequency approximation error or analyze its effect on quantization distortion. Since the "practically identical coding efficiency" claim is the main result, the authors should either provide an analytical bound or present experiments (e.g., RA, LD, very-high-bitrate) that directly validate the low-frequency-only assumption. This is a load-bearing point, not a presentation issue.
  2. [Table 3 vs. Table 2] There is an apparent inconsistency in the reported gains. Table 3, row "64" of the 8x8 sub-matrix adjustment, reports a CTC average of +0.08% over VTM-3.0. Table 2 shows that exact 64-point DST-7 and DCT-8 provide 2.78% versus 2.73%, i.e., +0.05%. Thus the approximate adjustment appears to outperform the exact transform by 0.03% on the same metric. Under either common BD-rate sign convention, an approximation should not beat the transform it approximates unless the difference is within measurement noise. Please reconcile this discrepancy and provide confidence intervals or repeated-anchor statistics to show the reported differences are meaningful.
  3. [Section 3, Table 3 and footnote 3] The main-text experimental evidence is exclusively all-intra under CTC; random-access, low-delay, and very-high-bitrate results are only cited to JVET documents [7, 8]. The conclusions in Section 4 state that the proposed methods provide "negligible coding performance differences on VTM-3.0" without restricting this statement to all-intra. Since complexity and coding-efficiency trade-offs can differ across configurations, the paper should include these additional results or explicitly narrow the scope of the conclusions. Furthermore, no confidence intervals are reported for any BD-rate numbers; with differences on the order of 0.01-0.08%, this level of statistical detail is necessary to support the claim of identical coding efficiency.
minor comments (5)
  1. [Section 4] The conclusion contains a duplicated word: "new method method" should read "new method."
  2. [Figure 1 caption] The caption sentence "A long vertical direction are transforms used for computation (B), and along horizontal are the desired transforms (D)" appears garbled; it should be rewritten as a complete, grammatical sentence, e.g., "Along the vertical direction are the transforms used for computation (B), and along the horizontal direction are the desired transforms (D)."
  3. [Section 2.3] The phrase "same number of arithmetic operations as needed for one adjustment" should be "the same number of arithmetic operations as are needed for one adjustment" for grammatical correctness.
  4. [Reproducibility] The optimized adjustment matrices are not included in the manuscript; the text refers to JVET documents [7, 8] for parameters. Because these matrices are the central artifact of the paper, please provide them in an appendix or as supplementary material so that the results can be reproduced without consulting external standards documents.
  5. [Table 1] The throughput ratios are reported relative to full matrix multiplication, but no absolute measured throughput or details of the benchmarking platform are given in the paper. A brief statement of the methodology, even if the full tool is described in [18], would help readers assess the practical significance.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: adjustment matrices are fitted to target transform matrices, and coding efficiency is measured on an external VTM-3.0 benchmark.

full rationale

The paper's derivation chain is: choose sparse orthogonal adjustment matrices A and C, minimize a weighted least-squares error to the target transform D (Eq. 2), and then measure the resulting coding efficiency on the VTM-3.0 reference software under common test conditions (Table 3). The optimization is fitted to the mathematical transform matrices DST-7/DCT-8, not to BD-rate or any coding-efficiency outcome. Eq. (2) uses fixed weights alpha = ln(100)/N, and no coding data appear in the objective. The claim of 'practically identical coding efficiency' is an empirical result measured on the external VTM-3.0 benchmark, not a consequence forced by the optimization by construction. The self-citations [7,8,18] provide parameter tables, additional test configurations (random-access, low-delay, very high bitrates), and a throughput measurement tool; these are prior-work references rather than a self-referential derivation, and the paper's central all-intra result is reported directly in Table 3. The apparent inconsistency where the approximate 64-pt adjustment appears to outperform the exact transform (Tables 2 and 3) is a measurement-noise or statistical-characterization concern, not evidence of circularity. No equation or claim reduces a 'prediction' to its own input, so no circular step can be exhibited.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The method depends on standard trigonometric identities linking DCT-2, DCT-3, DST-2, and DST-3; the domain assumption that low-frequency approximation preserves coding gain; and several hand-chosen design choices (weighting exponent, sparsity patterns, regularization constant, tap counts, sub-block size). No new physical entities are introduced. The optimized matrices themselves are the output of the method and are stored in referenced JVET documents rather than in this paper.

free parameters (4)
  • Low-frequency weighting exponent α = ln(100)/N
    Hand-picked row weighting in Eq. (2); shapes the fit but is not derived from coding data.
  • Sparse adjustment matrices A and C = Listed in JVET documents [7,8]; not printed in this paper
    Optimized via weighted least squares to approximate DST-7/DCT-8; the values are the method's deliverable.
  • Sparsity-pattern parameters = 4- or 6-tap band; 8x8 sub-block
    Chosen empirically to balance approximation quality and complexity.
  • Sparsity regularization constant 0.05 = 0.05
    Used in the sparsity-promoting objective; chosen by hand with no sensitivity analysis.
assumptions (6)
  • standard math DCT-3, DST-2, and DST-3 share DCT-2's computational efficiency via reversal and sign-change operations.
    Standard trigonometric transform identities from [5], invoked in Section 2.1 to justify using these as the efficient building blocks B.
  • domain assumption Weighted low-frequency approximation preserves coding gain.
    Central design premise in Eq. (2); empirically checked in Table 3 but not proven.
  • ad hoc to paper Chosen sparsity patterns (band matrix, 8x8 sub-block) suffice for the complexity/quality trade-off.
    Selected by inspection of Fig. 1 and human expertise in Section 2.2; no optimality guarantee.
  • domain assumption VTM-3.0 and CTC measurements represent practical codec behavior.
    Used for all coding results; the software and test conditions are standard but date from 2018.
  • domain assumption Throughput methodology [18] is a valid proxy for implementation complexity.
    Table 1 uses this methodology; it is not reproduced in the paper.
  • ad hoc to paper Sequential parallel Givens rotations find an adequate local optimum of the non-convex objective.
    The optimization problem is non-convex and the paper states no convergence guarantee (Section 2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Low-Complexity Transform Adjustments For Video Coding." pith.science (2026). https://pith.science/paper/TJKJZZQ7

@misc{pith2026250523618,
  author       = {Pith},
  title        = {Pith review of: Low-Complexity Transform Adjustments For Video Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TJKJZZQ7}},
  note         = {Machine review of arXiv:2505.23618}
}
read the original abstract

Recent video codecs with multiple separable transforms can achieve significant coding gains using asymmetric trigonometric transforms (DCTs and DSTs), because they can exploit diverse statistics of residual block signals. However, they add excessive computational and memory complexity on large transforms (32-point and larger), since their practical software and hardware implementations are not as efficient as of the DCT-2. This article introduces a novel technique to design low-complexity approximations of trigonometric transforms. The proposed method uses DCT-2 computations, and applies orthogonal adjustments to approximate the most important basis vectors of the desired transform. Experimental results on the Versatile Video Coding (VVC) reference software show that the proposed approach significantly reduces the computational complexity, while providing practically identical coding efficiency.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    INTRODUCTION In hybrid video codecs, intra and inter predicted residuals may have diverse statistical properties, which are not full y exploited by symmetric transforms such as the traditional d is- crete cosine transform (i.e., DCT-2), yet some irregular fe a- tures can be captured with fixed separable asymmetric trans- forms including discrete sine trans...

  2. [2]

    LOW-COMPLEXITY TRANSFORM ADJUSTMENTS Representing the group of orthogonal N × N matrices [9] as O(N, R), let D be the desired transform and B be the trans- form allowing efficient computations. The objective is to fin d orthogonal transform adjustment matrices A and C such that H = CBA cg ≈ D, A ∈ A , C ∈ C , (1) where cg ≈ denotes a form of approximation m...

  3. [3]

    05 + |Ai,j | + e− αi (Hi,j − Di,j )2    . Fig. 1 show examples of the results of such optimizations when the DCT-2, DCT-3, DST-2, and DST-3 are used as ma- trix B, and the remaining DSTs as matrix D, and when either A = I or C = I (results for other DCTs are not shown be- cause they are identical, but permuted versions). The resulting sparsity indicate...

  4. [4]

    a band matrix A before the DST-3

  5. [5]

    Since the first approach has computations similar to bi- orthogonal filters, band matrices with at most K non-zero row elements are referred as K-tap adjustments

    a top-left sub-block matrix C after the DCT-2. Since the first approach has computations similar to bi- orthogonal filters, band matrices with at most K non-zero row elements are referred as K-tap adjustments. The second approach is similar to secondary transforms proposed by other authors [11, 12, 13], but it should be noted that, while in a W × H block a ...

  6. [6]

    CTC + 64

    EXPERIMENTAL RESUL TS The parameters for band matrix adjustments used for the ex- perimental results shown in this section are listed in [7], w hile the parameters for sub-block matrices are in [8]. 2 In the latest VVC reference software, VTM-3.0 [15], the maximum DST-7 and DCT-8 size is restricted to 32, and it is 64 for DCT-2. As shown in Table 2, the t...

  7. [7]

    The proposed methods are used to reduce the complexity of large DST-7 and DCT-8 and tested on a real codec (VTM-3.0)

    CONCLUSIONS This paper proposes a new method method for low-complexity approximations of desired transforms and introduces two variants of adjustment based on band and sub-block trans- form matrices. The proposed methods are used to reduce the complexity of large DST-7 and DCT-8 and tested on a real codec (VTM-3.0). Inspection of the experimental results ...

  8. [8]

    Improved H.264 intra coding bas ed on bi-directional intra prediction, directional transfor m, and adaptive coefficient scanning,

    Y . Ye and M. Karczewicz, “Improved H.264 intra coding bas ed on bi-directional intra prediction, directional transfor m, and adaptive coefficient scanning,” in Proc. IEEE Int. Conf. Im- age Process., San Diego, CA, Oct. 2008, pp. 2116–2119

Show all 25 references
  1. [9]

    Jointly opti- mized spatial prediction and block transform for video and i m- age coding,

    J. Han, A. Saxena, V . Melkote, and K. Rose, “Jointly opti- mized spatial prediction and block transform for video and i m- age coding,” IEEE Trans. Image Process. , vol. 21, no. 4, pp. 1874–1884, Apr. 2012

  2. [10]

    GBST: Separable transforms based on line graphs for predic - tive video coding,

    H. E. Egilmez, Y .-H. Chao, A. Ortega, B. Lee, and S. Yea, “GBST: Separable transforms based on line graphs for predic - tive video coding,” in Proc. IEEE Int. Conf. Image Process. , Phoenix, AZ, USA, Sept. 2016, pp. 2375–2379

  3. [11]

    Enhanced multiple transform for video coding,

    X. Zhao, J. Chen, M. Karczewicz, L. Zhang, X. Li, and W.- J. Chien, “Enhanced multiple transform for video coding,” in IEEE Data Compression Conf. , Snowbird, UT, USA, Mar. 2016, pp. 73–82

  4. [12]

    Britanak, P

    V . Britanak, P . C. Yip, and K. R. Rao,Discrete Cosine and Sine Transforms: General Properties, Fast Algorithms and Integ er Approximations, Academic Press, San Diego, CA, 2007

  5. [13]

    Low- complexity adaptive multiple transforms for post-HEVC vid eo coding,

    T. Biatek, V . Lorcy, P . Castel, and P . Philippe, “Low- complexity adaptive multiple transforms for post-HEVC vid eo coding,” in Proc. 32nd Picture Coding Symp. , Nuremberg, Germany, Dec. 2016

  6. [14]

    Co m- plexity reduction for adaptive multiple transforms (AMT) u s- ing adjustment stages,

    A. Said, H. E. Egilmez, V . Seregin, and M. Karczewicz, “Co m- plexity reduction for adaptive multiple transforms (AMT) u s- ing adjustment stages,” Input document JVET-J0066, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, San Diego, CA, USA...

  7. [15]

    CE6: efficient implementations of MTS with transform adjustments,

    A. Said, H. E. Egilmez, Y .-H. Chao, V . Seregin, and M. Kar- czewicz, “CE6: efficient implementations of MTS with transform adjustments,” Input document JVET-M0538, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, Marrakech, MA, Jan. 2019

  8. [16]

    B. C. Hall, Lie Groups, Lie Algebras, and Representations: An Elementary Introduction, Springer-V erlag, New Y ork, NY, 2004

  9. [17]

    Highly efficient non-separable transforms for next generation video coding,

    A. Said, X. Zhao, M. Karczewicz, H. E. Egilmez, V . Seregi n, and J. Chen, “Highly efficient non-separable transforms for next generation video coding,” in Proc. 32nd Picture Coding Symp., Nuremberg, Germany, Dec. 2016

  10. [18]

    Rotational transform for image and video compression,

    E. Alshina, A. Alshin, and F. C. Fernandes, “Rotational transform for image and video compression,” in Proc. IEEE Int. Conf. Image Process. , Brussels, Belgium, Sept. 2011, pp. 3689–3692

  11. [19]

    On secondary transforms for prediction residual,

    A. Saxena and F. C. Fernandes, “On secondary transforms for prediction residual,” in Proc. IEEE Int. Conf. Acoustics Speech Signal Process. , Orlando, FL, USA, Sept. 2012, pp. 2489–2492

  12. [20]

    Low latency secondary tr ans- forms for intra/inter prediction residual,

    A. Saxena and F. C. Fernandes, “Low latency secondary tr ans- forms for intra/inter prediction residual,” IEEE Trans. Image Process., vol. 22, no. 10, pp. 4061–4071, Oct. 2013

  13. [21]

    CE6-related: efficient computation of MTS trans- form combinations,

    A. Said, H. E. Egilmez, Y .-H. Chao, V . Seregin, and M. Kar - czewicz, “CE6-related: efficient computation of MTS trans- form combinations,” Input document JVET-M0539, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, Marrakech, MA, Jan. 2019

  14. [22]

    Algorithm description fo r V er- satile Video Coding and Test Model 3 (VTM 3),

    J. Chen, Y . Ye, and S.-H. Kim, “Algorithm description fo r V er- satile Video Coding and Test Model 3 (VTM 3),” Output doc- ument JVET-L1002, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, Macao, CN, Oct. 2018

  15. [23]

    JVET common test conditions and software reference con- figurations for SDR video,

    F. Bossen, J. Boyce, K. S¨ uhring, X. Li, and V . Seregin, “JVET common test conditions and software reference con- figurations for SDR video,” Output document JVET-L1010, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, Macao, CN, Oct. 2018

  16. [24]

    Calculation of average PSNR differen ces be- tween RD-curves,

    G. Bjøntegaard, “Calculation of average PSNR differen ces be- tween RD-curves,” Tech. Rep. VCEG-M33, ITU-T SG16 Q.6, Austin, TX, Apr. 2001

  17. [25]

    CE6-related: software tool for computing trans- form throughput,

    A. Said, H. E. Egilmez, Y .-H. Chao, V . Seregin, and M. Kar - czewicz, “CE6-related: software tool for computing trans- form throughput,” Input document JVET-M0540, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC29/WG11, Marrakech, MA, Jan. 2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.