{"id":"7f253882-d0fb-435a-ae44-ee3cb3a56797","arxiv_id":"2505.18107","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A sensitivity-aware mode-decomposition training method plus a moving-average smoother accelerates learned image compression training to about 62% of standard SGD time with comparable or better R-D performance.","lead":"This paper proposes a training method for learned image compression that cuts training time by roughly 40% while matching or slightly improving rate-distortion performance. It does so by modeling parameter trajectories as a few correlated modes, freezing many weights once their dynamics stabilize, and smoothing training with a moving-average scheme.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing control: SMA-only at 70 epochs is never run, so the claimed acceleration from dimension reduction is not established; it may be due to weight averaging.","rationale":"The paper contains extensive experiments across multiple codecs and domains, and the loss curves in Fig. 2 show the full method converges faster than SGD. I credit the empirical result that the combined method works. However, the paper's causal story--that modeling neural training dynamics and embedding parameters accelerates convergence--requires showing that the dimension reduction, not the weight averaging, is responsible. The existing evidence is incomplete: (1) A.3 shows SMA alone matches the final loss of the full method, but only at epoch 120; (2) the w/o SMA ablation shows STDET alone is poor, so SMA is necessary, but this does not show STDET is sufficient or even helpful for speed; (3) the wall-clock reduction is entirely from fewer epochs since per-epoch time is not reduced. Thus the single most load-bearing concern is the missing SMA-only-at-70-epochs control. The reader's weakest assumption (affine coefficient stability) is related but secondary; the paper's good results across many models suggest that assumption holds reasonably in practice, whereas the SMA-only control directly tests whether the central mechanism causes the reported speedup. A one-line experiment would settle it. I therefore keep the CONDITIONAL verdict (no change), adding this as an explicit condition, alongside code release and hyperparameter selection clarification.","tokens_in":35397,"tokens_out":10955,"duration_ms":94036,"concrete_test":"Train ELIC, TCM-S, and FLIC with SMA only (no STDET, no embedding) for exactly 70 epochs for λ=0.0018 (and 50-epoch fine-tuning for other λ values), using the same α=0.8, l=5, LR schedule, and initialization as the paper's Proposed runs. Compare the testing R-D loss at epoch 70 (and final BD-Rate) against the Reported Proposed results. If SMA-only matches Proposed within run-to-run variance (e.g., R-D loss within 0.0005 or BD-Rate within 0.2%) at the same epoch, the claimed acceleration is attributable to SMA, and the STDET dimension-reduction mechanism is not the cause; then the central claim should be weakened accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim attributes faster convergence to reducing the training-space dimension via parameter embedding (STDET). Appendix A.3 reports that SMA alone ('ELIC + SMA') reaches the same final R-D loss (0.3442) as the full method, but only at the SGD budget of 120 epochs; the paper never trains SMA-only for the proposed method's 70-epoch budget. Since Fig. 15 shows the full method reaches 0.3442 by epoch 70, and since A.5 admits embedded parameters do not reduce per-epoch gradient time on current frameworks, the entire wall-clock saving is fewer epochs. If SMA-only also reaches ~0.3442 at epoch 70, then the acceleration is attributable to the moving-average mechanism (a known Lookahead-style technique), not to the parameter/dimension reduction that the title and abstract emphasize. The ablation 'w/o SMA' shows STDET alone fails, so the contribution of the embedding mechanism to convergence speed is unproven. Hyperparameters F=20, P=1%, L=1, α=0.8, l=5 were also selected using final BD-Rate on the evaluation sets (Secs. 4.4.1-4.4.3), but the SMA-only control is the decisive missing experiment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two mechanisms to accelerate the training of learned image compression (LIC) models: STDET, which clusters parameter trajectories into modes, expresses non-reference parameters as affine functions of reference parameters, and progressively freezes ('embeds') parameters with stable affine coefficients; and SMA, which periodically averages sampled SGD iterates and synchronizes them back into training. The authors report that their method reaches the same or slightly better rate-distortion performance as standard SGD in roughly 62% of the wall-clock training time across ELIC, TCM-S, and FLIC, with additional experiments on stereo, remote sensing, screen content, and raw-image codecs. They also provide a noisy-quadratic-model analysis intended to show that the method reduces steady-state training variance relative to SGD.","tokens_in":35759,"tokens_out":3635,"duration_ms":32068,"significance":"Speeding up LIC training is a practically important goal, and the paper demonstrates a large reduction in wall-clock time on several modern codecs with essentially unchanged rate-distortion performance. The experimental scope is broad, the implementations of the compared codecs are publicly available, and the paper reports detailed ablations of the main hyperparameters. The variance analysis is derived from the algorithm's update rules rather than fitted to the measurements, and the paper includes a useful comparison with low-dimensional and sparsity-based training methods. The main weakness is that the reported wall-clock acceleration is not cleanly attributed to the proposed parameter-embedding mechanism, because the SMA-only control at the reduced epoch budget is missing and the paper admits that embedding does not reduce per-epoch gradient time in current frameworks.","major_comments":[{"comment":"The decisive control experiment is missing. Appendix A.3 reports that 'ELIC + SMA' reaches the same final R-D loss (0.3442) as the full method, but the text states that this SMA-only run uses the same 120-epoch budget as standard SGD. Since Fig. 15 shows the full proposed method reaching the same loss by epoch 70, and since Appendix A.5 states that embedded parameters do not reduce per-epoch gradient time on current frameworks, the entire reported wall-clock saving comes from training for fewer epochs. The paper must report an SMA-only run at the proposed method's reduced budget (70 epochs for lambda=0.0018, 50 epochs for fine-tuning). If that run also reaches approximately 0.3442, the acceleration is attributable to the moving-average mechanism rather than to the dimension/parameter reduction that the title and abstract emphasize; if it does not, the result would support the claimed role of STDET.","section":"Appendix A.3 / Fig. 15"},{"comment":"The complexity analysis does not establish that reducing the number of trainable parameters reduces training time. Table 1 reports 'Total trainable params' and 'Final trainable params', but Section A.5 explicitly acknowledges that setting requires_grad=False does not prevent the embedded parameters from being computed in the chain rule, so per-epoch wall-clock time is essentially unchanged. Under this admission, the total-parameter reduction is an accounting metric, not a computational saving. The paper should report per-epoch wall-clock time as a function of the fraction of embedded parameters, or implement sparse backward computation that actually skips embedded parameters, before claiming that dimension reduction accelerates convergence.","section":"Section 4.3 / Appendix A.5"},{"comment":"The hyperparameters F, M, S, P, L, alpha, and l appear to be selected using final BD-Rate on the same evaluation benchmarks that are later used to report the headline results. The ablation tables in Section 4.4 report BD-Rate without specifying a separate validation set, and the chosen configuration in each column ('0%' anchor) is the one used in Table 1. To avoid selection-on-test circularity, the hyperparameters should be chosen on a held-out validation split or on a subset of the training data, and the test-set results in Table 1 should be reported only for the final selected configuration.","section":"Section 4.4, Tables 3-5"},{"comment":"The theoretical variance comparison rests on assumptions that are not verified in the paper. The inequality V*_Proposed <= V*_SGD requires both alpha in (0,1) and E[k_i^2] <= 1, and the proof also uses E[d_i] -> 0. The paper states only that E[d_i] -> 0 'empirically' and provides no evidence for E[k_i^2] <= 1. The authors should report the empirical distribution of k_i (and d_i) from the models in Section 4; if E[k_i^2] exceeds 1 for some modes, the stated inequality no longer follows. In addition, the variance of embedded parameters is written as E[k_i^2] V*_SMA, which omits a possible contribution from Var(d_i) unless d_i is treated as a deterministic offset.","section":"Appendix A.6, Eq. (14)"}],"minor_comments":[{"comment":"The sentence 'following standard practie' contains a typo: 'practie' should be 'practice'.","section":"Appendix A.2"},{"comment":"Two subfigures are both labeled '(c) Proposed SMA', one for l=1 and one for l=2; the labels should be (c) and (d).","section":"Appendix A.4 / Fig. 16"},{"comment":"The heading 'Total trainable params' is ambiguous because the reported values are parameter-epoch products (e.g., 35.42M x 520 epochs = 18,418M), not the number of parameters at any single time. The caption or column header should state this explicitly.","section":"Table 1"},{"comment":"The connection of SMA to the Lookahead optimizer is stated only briefly; a short formal comparison or a citation to the Lookahead update when l>1 would help readers understand the exact relationship.","section":"Section 3.3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is already published in TMLR (04/2025), so the review should focus on whether the arXiv version makes additional claims or whether the concerns below are already addressed in the published version. The central missing experiment, SMA-only at the reduced epoch budget, is inexpensive to run and directly determines whether the paper's stated mechanism is the cause of the observed acceleration. I would not recommend rejection because the empirical acceleration is visible and the method may still be useful even if the attribution changes, but the current text overstates the role of dimension reduction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline result here is real: ELIC, TCM-S, and FLIC trained with the proposed STDET+SMA recipe match the final rate-distortion of standard SGD in about 62% of the wall-clock time (FLIC goes from 520 to 320 hours on one 4090). That is a practically meaningful saving for a field where training runs take weeks, and the paper extends the result to stereo, remote sensing, screen content, and raw-image codecs. The \"CMD instant performance\" heuristic for choosing the number of modes and sampled trajectories without full training is clever and likely useful beyond this paper.\n\nThe problem is that the central mechanism isn't established. The authors attribute the speedup to dimension reduction through parameter embedding. But the appendix concedes that embedded parameters do not reduce per-epoch gradient time in current frameworks, so the entire wall-clock saving comes from needing fewer epochs. The decisive control is missing: we are never shown SMA-only trained for 70 epochs. The paper does run SMA alone for 120 epochs, and it matches the full method; it runs STDET alone for 70 epochs, and it fails. That combination leaves open the possibility that SMA alone would also hit the target in 70 epochs, in which case the novel dimension-reduction component adds nothing to the speedup. This is not a fatal flaw in the empirical result, but it does mean the paper's advertised contribution—reducing training space dimensions to accelerate convergence—is unsupported by the data as presented. The fix is a single experiment: train ELIC+SMA for 70 epochs. If it is worse, the claim holds; if not, the paper needs reframing.\n\nOther soft spots are minor in comparison: most hyperparameters were selected on the evaluation datasets, and no code is provided. The theoretical analysis in the appendix is a standard variance calculation for the moving average and does not address epoch reduction from embedding.\n\nWho this is for: researchers in learned image compression who care about training cost, and anyone who wants a clean case study in how a missing control can undercut a plausible mechanism. It deserves a serious referee, but the referee should require the SMA-only 70-epoch experiment and code before the mechanism claims can be taken seriously.\n\nRecommendation: engage with the paper, but treat the mechanism as unproven until that control is run. The empirical speedup stands on its own, even if the explanation needs to change.","headline":"Real training-time savings, but the paper's own mechanism is undercut by a missing SMA-only control at the same epoch budget.","tokens_in":36252,"tokens_out":5220,"would_cite":false,"duration_ms":34420,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","94A08"],"pacs":[],"model":"deepseek-v4-flash","headline":"Training a learned image codec can run in about 62% of the usual wall-clock time, with the same final rate–distortion quality, when most weights are locked as affine copies of a few reference weights.","keywords":["learned image compression","training acceleration","correlation mode decomposition","dynamic mode decomposition","low-dimensional subspace training","parameter embedding","sampling-then-moving-average","noisy quadratic model"],"falsifier":"Pick a new LIC architecture not reported in the paper, run the method with F=20, and record the maximum relative drift of the fixed coefficients (k_i, d_i) between epoch 20 and the final epoch alongside the final BD-Rate against an SGD-trained anchor. If a large fraction of coefficients drift by more than a few percent after embedding, or if freezing them at epoch 20 costs more than roughly a 1% BD-Rate penalty, the central assumption would be contradicted.","tokens_in":35218,"feed_emoji":"⚡","tokens_out":8412,"duration_ms":63829,"temperature":0.7,"pith_summary":"This paper argues that the expensive training of learned image compression (LIC) models can be substantially accelerated without sacrificing final rate–distortion performance. The key claim is that during most of training, parameter trajectories are so strongly correlated within a few 'modes' that every non-reference weight can be expressed as an affine function of one reference weight per mode, with coefficients that stabilize after roughly the first 20 epochs. Once the coefficients stabilize, the paper embeds the non-reference parameters — they stop being trained and simply follow the reference parameters. Combined with a moving-average step that smooths SGD noise, this reduces the training space dimension and trainable-parameter count by about half, and the authors report the same or slightly better BD-Rate in roughly 62% of the wall-clock time on three modern codecs (ELIC, TCM-S, FLIC) and on four domain-specific codecs. A theoretical analysis on a noisy quadratic model provides one supporting mechanism: the method's steady-state training variance is provably lower than standard SGD's.","feed_headline":"Cut learned codec training time by 38 percent","feed_subtitle":"Fixed affine ties to a few reference weights halve trainable parameters and match rate–distortion.","key_machinery":"The load-bearing mechanism is Correlation Mode Decomposition (CMD), which expresses each non-reference weight as an affine function of a mode reference weight, $w_i(t)\\approx k_i w_m(t)+d_i$, after clustering parameter trajectories into a few correlated modes. The method first performs CMD after $F=20$ epochs of normal training, then progressively 'true-embeds' the least-sensitive 1% of parameters per epoch once their affine coefficients have stabilized, while a small dummy-embedding step reinitializes a few extra parameters to their embedded values. A companion Sampling-then-Moving-Average (SMA) step interpolates periodically sampled SGD states and copies the moving average back into the optimizer, keeping temporal behavior smooth enough for the affine model to remain valid. Together these mechanisms shrink the training-space dimension toward the number of modes, which is as few as 50–120 in practice.","core_discovery":"The central discovery is that LIC training dynamics are highly compressible: after a short warm-up, the trajectory of each weight $w_i(t)$ is well approximated by $w_i(t) \\approx k_i w_m(t) + d_i$, where $w_m(t)$ is a reference trajectory from the same correlation mode. This linear representation, obtained by Correlation Mode Decomposition, lets the authors freeze the affine coefficients $(k_i, d_i)$ and 'embed' the corresponding weights so they are no longer optimized directly; they evolve through the reference parameter instead. The paper further shows that only about 25% of parameters are sensitive to being frozen this way, and that a Sampling-then-Moving-Average (SMA) schedule is needed to keep trajectories smooth enough for the approximation to hold. In experiments, the method reaches final R-D performance comparable to SGD in roughly 62% of the training time, and on a noisy quadratic model it provably attains lower steady-state variance than SGD.","pith_inferences":["If the affine-coefficient stability measured at epoch 20 fails on architectures with very different layer dynamics (for instance, long-range transformer blocks), periodic re-estimation of the coefficients, or a slower embedding schedule, would likely be needed to preserve the time savings outside the paper's model set.","Combining the frozen parameters with sparse backpropagation that skips gradient computation for embedded weights could cut per-epoch cost as well as epoch count; the paper itself flags this as a non-trivial open implementation step.","The same trajectory-compression idea could be applied to the intra-frame coder of learned video compression, since that coder is itself an LIC; the motion-estimation modules would require separate treatment.","The strong correlation between 'CMD instant performance' and final BD-Rate suggests a cheap single-forward-pass diagnostic for how many effective dimensions any codec architecture needs, which may generalize beyond compression."],"forward_implications":["Across the three benchmark codecs ELIC, TCM-S, and FLIC, the method reaches the final R-D point in roughly 62% of the SGD wall-clock time, with slightly negative BD-Rate on Kodak (about -0.7%).","The number of trainable parameters at the final epoch is roughly halved (e.g., FLIC drops from 70.96M to 41.39M), and the total parameter-epochs across all rate points fall to about 51% of the SGD total.","The same affine-embedding recipe transfers to stereo, remote-sensing, screen-content, and raw-image codecs, each trained in about 60–64% of the corresponding SGD time with comparable or better BD-Rate.","On a noisy quadratic model the proposed update rule provably has lower steady-state training variance than standard SGD at the same learning rate, which the paper identifies as the stabilizing mechanism for the embedding.","The method beats the low-dimensional baselines P-SGD, P-BFGS, and TWA (which diverge on LIC) and the sparsity baselines RigL and SRigL (which do not reduce training time), so the combination of mode embedding with moving-average smoothing is claimed to be the key."],"supporting_citations":[{"why":"Supplies ELIC, the base codec on which the coefficient-stability evidence (Fig. 1) and a full BD-Rate comparison are run.","marker":"He et al., 2022"},{"why":"Provides TCM-S, the second modern LIC used to demonstrate the ~62% training-time ratio.","marker":"Liu et al., 2023"},{"why":"Provides FLIC, the largest benchmark model where SGD training drops from 520h to 320h.","marker":"Li et al., 2024a"},{"why":"Supplies the Correlation Mode Decomposition that the affine embedding is built on.","marker":"Brokman et al., 2024"},{"why":"Gives the low intrinsic dimension hypothesis that justifies training LICs in a reduced parameter space.","marker":"Li et al., 2018"},{"why":"Provides the P-SGD/P-BFGS subspace baselines against which the method is compared.","marker":"Li et al., 2022b"},{"why":"Underpins the first-order parameter-sensitivity estimate used to decide embeddability.","marker":"Molchanov et al., 2019"},{"why":"Supplies the noisy quadratic model analysis that the paper extends to show lower steady-state variance.","marker":"Wu et al., 2018"}],"fun_headline_variants":["Modeling training dynamics cuts learned codec training time","Training dynamics modeling accelerates learned image compression","Affine weight ties accelerate learned codec training without quality loss","Freeze most weights, keep quality: faster learned codecs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"After roughly the first 20 epochs, each non-reference weight can be represented by a fixed affine function of its mode's reference weight whose coefficients do not drift meaningfully for the rest of training.","fun_headline_variants_meta":{"raw":{"variants":["Modeling training dynamics cuts learned codec training time","Training dynamics modeling accelerates learned image compression","Affine weight ties accelerate learned codec training without quality loss","Freeze most weights, keep quality: faster learned codecs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000544,"raw_usage":{"total_tokens":2601,"prompt_tokens":938,"completion_tokens":1663,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":1599}},"tokens_in":554,"tokens_out":1663,"duration_ms":15393,"temperature":1.0,"reasoning_tokens":1599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:35:02.342687+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick a new LIC architecture not reported in the paper, run the method with F=20, and record the maximum relative drift of the fixed coefficients (k_i, d_i) between epoch 20 and the final epoch alongside the final BD-Rate against an SGD-trained anchor. If a large fraction of coefficients drift by more than a few percent after embedding, or if freezing them at epoch 20 costs more than roughly a 1% BD-Rate penalty, the central assumption would be contradicted.","supporting_citations":[{"cited_title":"Importance estimation for neural network pruning","cited_arxiv_id":null,"evidence_quote":"Underpins the first-order parameter-sensitivity estimate used to decide embeddability."}],"review_version":1}