{"id":"732adb9d-af17-4c19-af41-555963665f28","arxiv_id":"2411.18494","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A linear transform trained with end-to-end rate-distortion optimization beats the DCT and prior data-driven transforms on residual block compression.","lead":"A team trained a new mathematical transform for image and video compression that directly targets the rate-distortion trade-off on prediction residuals. It reports about 12 percent bit-rate savings over the widely used DCT, with code released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No orthonormality constraint is imposed on M; using M^T as the inverse in Eq. (3) assumes M M^T = I, and no orthogonality measure is reported. The reported BD-rate gains may partly come from a non-inverse decoder, so a check of the released matrices is needed.","rationale":"The reader's weakest assumption correctly identifies the orthonormality issue, and I agree it is the most load-bearing concern. The paper's central claim is that a simple learned linear transform, RDLT, significantly outperforms DCT and other data-driven transforms while remaining integrable into existing standards. For that claim to hold, the reconstruction step in Eq. (3) must be a true inverse of the forward transform in Eq. (1). This requires M M^T = I. The paper provides no mechanism to enforce this: the transposed matrix is used as the inverse precisely to 'enforce orthonormality', but the training updates are unconstrained gradient steps on M. A tied-weight linear autoencoder can produce good reconstruction even when the weight matrix is not orthogonal, so the reported RD performance may reflect decoder flexibility rather than transform optimality. This is especially important because the baselines DCT, KLT, and SOT are orthonormal by construction, making the comparison potentially unfair. Other concerns, such as evaluation on a held-out split of the same training distribution and the absence of error bars, are real but less fundamental: they affect generality, not the validity of the reported pipeline. The paper does have supporting evidence: the code is publicly released, the rate at test time is computed with actual arithmetic coding rather than the training rate model, and the comparisons include both KLT and SOT. These are positive signs, but none of them addresses the orthogonality question. The proposed concrete test is decisive: if the released matrices are near-orthonormal, the concern evaporates and the conditional acceptance stands; if they are not, the authors would need to re-run the evaluation with a true inverse transform and show that the BD-rate gains survive. Since the reader already issued a CONDITIONAL verdict and this concern is precisely the condition to verify, no verdict change is needed.","tokens_in":6840,"tokens_out":5825,"duration_ms":57237,"concrete_test":"Download the public repository and extract the trained matrices M for n = 8, 16, 32. Compute the orthogonality error e = ||M M^T - I||_F / n and the minimum singular value of M. If e is not near 0 (say > 1e-3) or the minimum singular value is far from 1, rerun the evaluation pipeline with Eq. (3) replaced by the true inverse xhat = Q * yhat * M^{-1} and recompute BD-rate vs DCT. If the BD-rate savings drop by more than a few percentage points or reconstruction PSNR collapses, the central RD claim depends on using a non-inverse decoder; if savings persist, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the learned matrix M is orthonormal, so that using M^T in the reconstruction Eq. (3) is equivalent to using the true inverse M^{-1}. The paper states that the choice of M^T 'enforces orthonormality on the matrix M', but no orthonormality constraint, penalty, projection, or post-training metric is reported; the training procedure in the 'Learning procedure' section is ordinary SGD on an unconstrained M. Unconstrained training of a tied-weight linear map y = xM, xhat = Q*round(y/Q)*M^T does not keep M orthogonal. If M drifts from orthonormal, M^T is not the inverse transform: a standards-compliant decoder would need M^{-1}. The reported PSNR and BD-rate then measure a tied-weight linear autoencoder, which can use M^T as a learned decoder that compensates for quantization in ways unavailable to a true transform codec. This directly affects the central claim that RDLT is a linear block transform suitable for integration into existing standards, and it makes the comparison with DCT, KLT, and SOT (all using true orthonormal inverses) potentially unfair. Because no orthogonality measure is reported, the 12% and 5% BD-rate savings could in part be an artifact of this decoder-side flexibility rather than a better transform.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes RDLT, a linear block transform for residual blocks, trained end-to-end by minimizing a rate-distortion loss with a learned Gaussian entropy model and variable-rate training. The transform matrix M is applied as y = xM, quantization is modeled as uniform noise during training, and reconstruction uses M^T in Eq. (3). Experiments compare RDLT with DCT, KLT, and SOT on residual blocks of sizes 8, 16, and 32, using arithmetic coding on a held-out split; the authors report about 12% BD-rate saving over DCT and 5% over SOT, and also simulate integration of RDLT into a VVC-like MTS framework. The central claim is that a simple learned linear transform can substantially improve standard-compatible block transform coding.","tokens_in":7049,"tokens_out":6122,"duration_ms":59635,"significance":"If the reported gains hold under a standards-compliant inverse transform, this is a practically interesting result: it would show that a rudimentary linear transform trained with an RD objective and a modern entropy model can beat both the DCT and prior data-driven transforms such as KLT and SOT. The paper has genuine strengths: the rate term is evaluated with actual arithmetic coding rather than only the learned model, the code is promised to be public, and the comparison includes several relevant baselines. However, the central claim depends on the learned M being orthonormal, since Eq. (3) uses M^T as the inverse, and the currently reported experiments do not establish that property. The lack of image-disjoint evaluation and of any uncertainty quantification further weakens the strength of the reported performance margins.","major_comments":[{"comment":"","section":"Proposed method / Learning procedure, Eqs. (1)-(3)"},{"comment":"","section":"Experiments / Experimental settings"},{"comment":"","section":"Experiments / Rate-distortion comparison, Table 1"}],"minor_comments":[{"comment":"","section":"Learning procedure"},{"comment":"","section":"References"},{"comment":"","section":"Title/Abstract"},{"comment":"","section":"Additional investigation / MTS experiment"},{"comment":"","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The orthonormality issue is the central technical concern and should be resolved before publication. If the released code shows that the learned M is numerically close to orthonormal, the paper may become acceptable after adding the orthogonality metric and the evaluation refinements; otherwise, the comparisons must be rerun with an explicit orthonormality constraint. The topic is within the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here is the short version. RDLT is worth knowing about: it is the first linear block transform trained end-to-end by minimizing a rate-distortion loss with a parametric Gaussian entropy model, and the authors report about 12% BD-rate saving over the DCT and 5% over SOT on residual blocks. Prior data-driven transforms (KLT) are optimized for linear approximation, and SOT uses an l0 rate proxy, so the RD-based training objective is genuinely new. The evaluation is also more concrete than many papers in this area: rates come from actual arithmetic coding on held-out data, and the code is public. That is real evidence and should be credited. \n\nThe load-bearing soft spot is orthonormality. Equation (3) uses M^T as the inverse, justified only if M is orthonormal. The authors say that choosing M^T 'enforces orthonormality,' but there is no orthonormality constraint, penalty, projection, or post-training measure of M M^T reported. Ordinary SGD on an unconstrained M will not keep M orthogonal. If M drifts, the decoder is a tied linear autoencoder, not an inverse transform, and the comparison against DCT/KLT/SOT—all with true orthonormal inverses—may be unfair. The reported PSNR and BD-rate could partly reflect decoder-side flexibility rather than a better transform. This is fixable: the authors should report the orthogonality error of the learned matrices and either enforce orthonormality explicitly (e.g., via a Stiefel projection or a penalty) or use M^{-1} in reconstruction for a fair comparison. If the released matrices turn out to be near-orthogonal, the central claim likely survives. As written, it is a significant gap, not a fatal one. \n\nMinor issues: no error bars or multi-seed results; evaluation is on a held-out split of the same distribution used for training; the VVC-like integration is described briefly; and the code link lacks a commit hash. The citation pattern is normal, and the paper is honestly framed. \n\nWho this is for: researchers in image/video compression, especially those working on transform coding or learned codecs. It is a useful new baseline. I would send it to peer review rather than desk-reject it: the idea is sound, the experiments are mostly well done, and the orthonormality concern is a specific, addressable technical point that a good referee can force the authors to clarify.","headline":"A genuinely new learned linear transform for residual coding that reports real BD-rate gains, but the missing orthonormality check makes the headline numbers hard to fully trust.","tokens_in":7651,"tokens_out":1996,"would_cite":true,"duration_ms":20418,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a linear block transform trained end-to-end to minimize a rate-distortion loss beats the DCT by about 12% in BD-rate and the SOT by about 5% on residual blocks.","keywords":["learned linear transform","rate-distortion optimization","residual coding","DCT replacement","Gaussian entropy model","block transform coding","image compression","video compression"],"falsifier":"Retrain RDLT under the paper's setup and compute the orthogonality error ||M M^T - I|| plus the PSNR gap between reconstructing with M^T and with the numerically inverted M; a non-negligible gap would mean part of the reported RD gain is an artifact of using a non-inverse at the decoder.","tokens_in":6553,"feed_emoji":"🗜️","tokens_out":7213,"duration_ms":58813,"temperature":0.7,"pith_summary":"The paper tries to establish that a single linear block transform learned by gradient descent can beat the DCT and prior data-driven transforms at coding residual blocks, without abandoning the block-transform architecture used in video and image standards. It introduces RDLT, a transform matrix updated end-to-end to minimize a rate-distortion loss, with coefficient rate estimated by a parametric Gaussian entropy model. On a mixed dataset of 8x8, 16x16, and 32x32 residual blocks, RDLT reports about 12% BD-rate saving over the DCT and about 5% over SOT. If true, the result matters because it offers a standards-compatible path to coding gains that does not require the nonlinear, decoder-heavy machinery of end-to-end learned codecs.","feed_headline":"Learned linear transform beats DCT by 12% in residual coding","feed_subtitle":"A rate-distortion-trained linear transform cuts BD-rate by about 12% over DCT and 5% over SOT in tests.","key_machinery":"The load-bearing object is the $n^{2}$ by $n^{2}$ transform matrix M itself, treated as a one-layer linear network whose basis functions are applied as y = xM. Training uses additive uniform noise in place of hard rounding, a Gaussian entropy model to estimate coefficient rate, and variable-rate training where a small two-layer network predicts the quantization step-size Q from $\\lambda$. Reconstruction uses the transpose M^T as the inverse, a choice the paper says enforces orthonormality. The mechanism that carries the argument is the Gaussian entropy model: replacing crude proxies like the l0 norm with a differentiable estimate of true coefficient entropy is what lets gradient descent find transforms that outperform KLT and SOT.","core_discovery":"The central claim is that the rate-distortion-optimal linear transform is learnable and measurably better than the usual choices. The paper trains the transform matrix M by back-propagating through a differentiable proxy of quantization and minimizing L = D + lambda R, where the rate term is the negative log-likelihood of the quantized coefficients under a Gaussian entropy model with learnable per-coefficient means and variances. It reports about 12% BD-rate saving over the DCT and about 5% over SOT on residual blocks of sizes 8, 16, and 32, with gains appearing at each block size. It further reports that swapping RDLT for DCT inside the multi-transform selection loop of a VVC-like codec improves rate-distortion performance. The paper's intended significance is that accurate rate estimation, not just variance-based linear optimality, is what makes a linear transform effective in practical compression.","pith_inferences":["Inference: the reported gains are measured on HEVC intra residuals; extending RDLT to inter residuals or other prediction modes may require retraining, and the gain size is an open question.","Inference: because orthonormality is only encouraged by using M^T, an explicit orthogonality constraint or post-training orthogonalization could make the gains reproducible in a real codec where the inverse must be exact.","Inference: the method could be extended to a family of learned transforms, one per intra-prediction mode, along the lines of mode-dependent transforms, potentially compounding the RD gain.","Inference: since the entropy model is trained jointly, one could also tune the quantizer step-size schedule with the transform rather than treating Q as an input, but the paper does not test this."],"forward_implications":["If RDLT's reported gains hold, replacing DCT with the learned matrix inside existing block codecs could yield around 12% BD-rate savings without changing the decoder's block-transform structure.","The per-block-size results suggest the gains are not an artifact of one block size; separate 8x8, 16x16, and 32x32 transforms each beat DCT.","The VVC-like MTS simulation implies RDLT can be added as one transform among several and chosen by rate-distortion optimization, rather than requiring a fixed single transform.","The comparison with SOT indicates that a differentiable Gaussian rate model outperforms an l0-based rate proxy for learning linear transforms.","Because the resulting transform is linear and intended to be invertible through orthonormality, integration into standards could be decoder-compatible."],"supporting_citations":[{"why":"Supplies the end-to-end training recipe: additive uniform noise for differentiable quantization, rate-distortion loss, and back-propagation.","marker":"[9]"},{"why":"Supplies the Gaussian entropy model used to estimate the rate of quantized coefficients during training.","marker":"[22]"},{"why":"Provides the sparse orthonormal transform baseline that RDLT is compared against and outperforms by about 5% BD-rate.","marker":"[21]"},{"why":"Establishes the classic DCT-to-KLT optimality relationship that motivates the DCT baseline and its sub-optimality for residuals.","marker":"[7]"},{"why":"Defines the rate-distortion optimized transform scheme and offline KLT approach that the VVC-like MTS integration builds on.","marker":"[10]"},{"why":"Supplies the HEVC intra prediction used to generate the residual blocks in the training and evaluation dataset.","marker":"[3]"},{"why":"Contributes part of the source image dataset used to build residual blocks for training and evaluation.","marker":"[23]"},{"why":"Contributes another part of the source image dataset used to build residual blocks for training and evaluation.","marker":"[24]"}],"fun_headline_variants":["Rate-distortion trained linear transform beats DCT by 12%","Learned linear transform cuts coding cost 12% over DCT","Optimal linear transform learned from rate-distortion beats DCT","DCT outperformed by 12% with rate-distortion-optimized linear transform","Backprop optimizes linear transform, DCT falls behind by 12%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the learned matrix stays close enough to orthonormal that using its transpose as the inverse reconstruction is valid, because the training loss only nudges M in that direction and the paper reports no orthogonality measure.","fun_headline_variants_meta":{"raw":{"variants":["Rate-distortion trained linear transform beats DCT by 12%","Learned linear transform cuts coding cost 12% over DCT","Optimal linear transform learned from rate-distortion beats DCT","DCT outperformed by 12% with rate-distortion-optimized linear transform","Backprop optimizes linear transform, DCT falls behind by 12%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001077,"raw_usage":{"total_tokens":4527,"prompt_tokens":986,"completion_tokens":3541,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":3445}},"tokens_in":602,"tokens_out":3541,"duration_ms":22051,"temperature":1.0,"reasoning_tokens":3445,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:07:58.251272+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain RDLT under the paper's setup and compute the orthogonality error ||M M^T - I|| plus the PSNR gap between reconstructing with M^T and with the numerically inverted M; a non-negligible gap would mean part of the reported RD gain is an artifact of using a non-inverse at the decoder.","supporting_citations":[{"cited_title":"End-to-end optimized image compression,","cited_arxiv_id":null,"evidence_quote":"Supplies the end-to-end training recipe: additive uniform noise for differentiable quantization, rate-distortion loss, and back-propagation."},{"cited_title":"Variational image compression with a scale hyperprior,","cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian entropy model used to estimate the rate of quantized coefficients during training."},{"cited_title":"Approximation and compression with sparse orthonormal transforms,","cited_arxiv_id":null,"evidence_quote":"Provides the sparse orthonormal transform baseline that RDLT is compared against and outperforms by about 5% BD-rate."},{"cited_title":"Ahmed and K","cited_arxiv_id":null,"evidence_quote":"Establishes the classic DCT-to-KLT optimality relationship that motivates the DCT baseline and its sub-optimality for residuals."},{"cited_title":"Video coding with rate-distortion op- timized transform,","cited_arxiv_id":null,"evidence_quote":"Defines the rate-distortion optimized transform scheme and offline KLT approach that the VVC-like MTS integration builds on."},{"cited_title":"Intra coding of the HEVC standard,","cited_arxiv_id":null,"evidence_quote":"Supplies the HEVC intra prediction used to generate the residual blocks in the training and evaluation dataset."},{"cited_title":"The USC-SIPI image database,","cited_arxiv_id":null,"evidence_quote":"Contributes another part of the source image dataset used to build residual blocks for training and evaluation."}],"review_version":1}