Pith. sign in

REVIEW 4 major objections 6 minor 19 references

Enhancing Generalization via Sharpness-Aware Trajectory Matching for Dataset Condensation

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that sharpness-aware trajectory matching—adding a flat-minimum objective to the outer loop—makes synthetic condensed datasets generalize better, and reports consistent accuracy gains on CIFAR, Tiny ImageNet, and…

desk verdict A plausible drop-in sharpness module for trajectory-matching condensation with consistent but modest gains; the efficiency theory is unsound and needs fixing, but the empirical core deserves a serious referee. read the letter →

arxiv 2502.01865 v1 pith:HV6KBDKK submitted 2025-02-03 cs.LG

Boyan Gao , Bo Zhao , Shreyank N Gowda , Xingrun Xing , Yibo Yang , Timothy Hospedales , David A. Clifton This is my paper · ORCID
classification cs.LG
keywords datasetcondensationdistillationtrajectorymatchingsharpness-awareminimizationbileveloptimizationgeneralizationhypergradientapproximationImageNet-1K
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that dataset condensation—learning a tiny synthetic training set that stands in for a large one—should not only match training trajectories but also land the synthetic dataset in a flat region of the bilevel loss landscape. It proposes Sharpness-Aware Trajectory Matching (SATM), which adds a sharpness penalty to the outer-loop objective and uses two cheap gradient approximations so the added cost stays manageable. If the claim is right, condensation methods get a simple, drop-in sharpness module that yields better accuracy on standard benchmarks, better transfer to unseen architectures and continual learning, and the first trajectory-matching gains on ImageNet-1K without extra memory.

What carries the argument

The central machinery is the sharpness-aware outer-loop objective $\min_\varphi \max_{\|\epsilon\|_2\leq\rho} F(\varphi+\epsilon)$ with $F(\varphi)=\|\theta^S_{t+N}(\varphi)-\theta^E_{t+M}\|_2^2/\delta$, where $\epsilon$ is chosen in closed form as $\rho\nabla F(\varphi)/\|\nabla F(\varphi)\|_2$. Two approximation devices carry the efficiency claim: Truncated Unrolling Hypergradient, which differentiates only through the last $N-\iota$ inner-loop steps and is supported by an exponential error bound, and Trajectory Reusing, which initializes the second inner-loop trajectory at the $\tau$-th step of the first, justified by Theorem 4.2's bound $\|\Delta\theta_\tau\|\leq \alpha\tau(2\sigma+\beta\rho)$. Gaussian fluctuation of the synthetic images smooths the outer-loop landscape so that the one-step perturbation approximates the true sharpness direction.

What would settle it

Measure the actual distance $\|\theta_\tau(\varphi)-\theta_\tau(\varphi+\epsilon)\|$ during condensation and compare it with the bound $\alpha\tau(2\sigma+\beta\rho)$ from Theorem 4.2; if the trajectories diverge before step $\tau$, or if running the full second unrolling ($\tau=0$) changes final accuracy by more than run-to-run noise, the reuse assumption—and the sharpness-aware interpretation of the update—fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the generalization bottleneck in trajectory-matching dataset condensation is the sharpness of the outer-loop loss, and that minimizing sharpness together with the trajectory distance improves the resulting synthetic data. In the paper's own terms, replacing the outer-loop objective with $\min_\varphi\max_{\|\epsilon\|_2\leq\rho}F(\varphi+\epsilon)$ and updating along $\nabla F(\varphi+\epsilon)$ steers the synthetic dataset into a flat region of the landscape. The empirical payoff is consistent: SATM outperforms MTT, FTD, and TESLA on CIFAR-10, CIFAR-100, and Tiny ImageNet across image-per-class settings, improves accuracy on ImageNet-1K, and generalizes better to unseen architectures and continual-learning tasks, all with memory and runtime comparable to or better than TESLA.

Load-bearing premise

The load-bearing premise is that, midway through the training run on the synthetic images, the model trained on the original images and the model trained on the slightly perturbed images are still close enough that the second run can be started from the first run's checkpoint; if that fails, the update is no longer genuinely sharpness-reducing.

Editorial extensions

If this is right

  • On CIFAR-10, CIFAR-100, and Tiny ImageNet, SATM beats MTT, FTD, and TESLA at 1, 3, 10, and 50 images per class; for example, CIFAR-10 IPC-1 accuracy rises from 46.2±0.8 (MTT) to 49.0±0.3.
  • On ImageNet-1K, SATM reaches 29.2±1.1 accuracy at 50 IPC versus TESLA's 27.9±1.2, with slightly lower runtime and constant memory usage.
  • Synthetic datasets learned by SATM transfer better to unseen architectures such as AlexNet, VGG11, and ResNet18, and maintain better accuracy in class-incremental continual-learning benchmarks.
  • The efficiency devices (truncated unrolling and trajectory reusing) keep memory cost constant and reduce runtime below TESLA's, making sharpness-aware condensation feasible at ImageNet scale.
  • SATM composes with other sharpness-aware minimizers and with curriculum-style trajectory matching, as shown by the SATM-DA results on CIFAR-100 and Tiny ImageNet.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the trajectory-reuse bound holds broadly, the same two-device approximation could make SAM-style updates practical in other bilevel settings with long unrolls, such as hyperparameter learning and optimizer learning.
  • The Gaussian fluctuation strength $\gamma$ is a free smoothing knob; replacing it with other noise families or adversarial perturbations could yield testable variants with different bias-variance trade-offs on the same benchmarks.
  • The sharpness of a condensed dataset, rather than its in-domain accuracy alone, may be the better predictor of cross-architecture transfer; a direct test would correlate measured sharpness of competing condensed sets with their accuracy on unseen architectures.
  • The paper sets $\tau=\iota$ for convenience, but Theorem 4.2 suggests the reuse horizon should be tuned jointly with the learning rate; separating them could improve performance further than the reported settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Sharpness-Aware Trajectory Matching (SATM), which applies the SAM sharpness penalty to the outer-loop objective of trajectory-matching dataset condensation. To make the two inner-loop unrollings affordable, the authors introduce two approximations: truncated unrolling hypergradient (TUH), which backpropagates only through the last N-iota inner-loop steps, and trajectory reusing (TR), which initializes the second inner-loop trajectory at a middle checkpoint of the first trajectory instead of rerunning the full inner loop from scratch. A first-order closed-form update for the inner-loop learning rate is also proposed. Experiments on CIFAR-10/100, Tiny ImageNet, ImageNet subsets, and ImageNet-1K show consistent, though modest, accuracy gains over MTT, FTD, and TESLA, together with lower or comparable runtime and memory, plus gains in cross-architecture and continual-learning settings.

Significance. If the central claim holds, SATM is a practical, plug-in sharpness module for trajectory-matching dataset condensation that improves generalization without a large efficiency penalty. The empirical evaluation is broad and includes ImageNet-1K, where few trajectory-matching methods scale, and the reported gains are consistent across datasets, IPCs, and out-of-domain tasks. However, the paper's theoretical support for the key efficiency mechanism (trajectory reusing) is not valid as stated, and the pseudocode is not fully consistent with the mathematical formulation, so the claim that SATM minimizes the sharpened objective in Eq. (5) is not established. The theoretical analysis must be repaired or substantially weakened before the paper is publishable.

major comments (4)
  1. [Sec. 4.2 / Theorem 4.2 / Appx. A.1] The proof of Theorem 4.2 is not derived from its assumptions. The step bounding ||∇L(ϕ+ϵ, θ̂_i) − ∇L(ϕ, θ_i)|| by 2σ+βρ uses a first-order Taylor expansion in ϕ and then asserts ||∇ϕ∇θL(ϕ, θ̂_i)|| ≤ β from β-continuity; β-continuity alone does not imply that norm bound without an anchor condition such as ∇ϕ∇θL(ϕ, 0)=0 and a bound on θ̂_i. More importantly, the correct local recursion for the weight gap is ||Δθ_{i+1}|| ≤ (1+ασ)||Δθ_i|| + αβρ, which yields a bound growing like (1+ασ)^τ, not the stated ατ(2σ+βρ). The stated bound is constant in the gap and grows linearly in τ, so it does not actually demonstrate that θ_τ(ϕ) and θ̂_τ(ϕ+ϵ) remain close. Since Theorem 4.2 is the sole support for the trajectory-reusing assumption underlying Eq. (9), the claim that Eq. (9) computes the hypergradient of the perturbed objective is not supported.
  2. [Sec. 4.1 / Algorithm 1] Algorithm 1 is not consistent with the equations defining the sharpness update. Lines 7–8 perturb ϕ by ϕ_Δ before the first inner loop, and line 12 then computes ∇F(ϕ) at this perturbed value; only afterward (line 15) is the perturbation reverted. Consequently, the perturbation direction ϵ in line 13 is computed from a gradient of the noisy dataset, not from ∇F(ϕ) at the unperturbed ϕ as in Eq. (7), and the second inner loop in lines 16–18 runs on ϕ+ϵ without the noise term. The text in Sec. 4.1 describes the fluctuation as a landscape-smoothing mechanism, but the algorithm and the objective actually being minimized (Eq. 5 versus a noise-smoothed variant) need to be reconciled, otherwise the empirical gains cannot be cleanly attributed to the sharpness penalty in Eq. (5).
  3. [Sec. 4.2 / Proposition 4.1 / Appx. A.2] Proposition 4.1 assumes L_CE is locally J-strongly convex in θ, but the proof then switches to the case 'when LCE is K-smooth but nonconvex' and uses ||A_i|| = 1 + αK ≤ 2, which is outside the proposition's assumptions. Additionally, the expression for the truncated hypergradient in Eq. (8) has index/product inconsistencies: the product ∏_{i'=i}^N ∂θ_{i'}/∂θ_{i'-1} combined with ∂θ_i/∂ϕ does not generally give the correct chain-rule term for ∂F(ϕ)/∂ϕ. The proposition's statement and proof need to be corrected to provide the claimed exponential error bound.
  4. [Sec. 4.2 / Eq. (10) / Appx. A.4] The derivation of the closed-form learning-rate hypergradient is not correct as written. The step 'we treat ∂L_CE(θ_{N−1}, ϕ)/∂θ_{N−1} as a constant w.r.t. α' is invalid because θ_{N−1} depends on α through the previous updates, so the resulting sum is not the derivative ∂L(θ_N(ϕ))/∂α. The claim that this first-order expression has 'limited differences' from the second-order derivative is supported only by the empirical plot in Fig. 3; the paper should either provide a corrected derivation or explicitly state that Eq. (10) is a heuristic approximation rather than a closed-form hypergradient.
minor comments (6)
  1. [General] There are numerous typos, e.g., 'Assmue' in Proposition 4.1, 'Shaprness' in Sec. 4.1, 'Trucated' in Appx. A.3, and 'syntactic' in several places; a careful proofread is needed.
  2. [Table 6] The ImageNet-1K memory column reports the same value (26.6 GB) for MTT, TESLA, and SATM, which seems inconsistent with the text's claim that TESLA and SATM achieve constant memory; the table should clarify what exactly is being measured and why the values are identical.
  3. [Appx. A.5 / Table 2] Appendix A.5 states that mini-batch outer-loop updates are used for CIFAR-100 10/50 IPC and Tiny ImageNet 1 IPC, but Table 2 does not indicate which settings use this protocol; this should be disclosed in the main results so that comparisons are made on an equal footing.
  4. [Figure 2] The caption says the top row is 'Cifar100 3IPC' while the plot text says 'Cifar100 10 IPC'; also the bottom 'Dynamic of Sharpness' panel shows only one curve, so it should state clearly that it is the SATM trajectory and whether no comparison is needed.
  5. [Sec. 4.2 / Table 8] The text says τ=ι is used in experiments, but Table 8 varies the retained inner-loop fraction with k = 2..6; please clarify how τ and ι are set for the main results and how they relate to the parameters in Table 11.
  6. [References / Appx. A.7] The reference to Li and Giannakis is inconsistently named 'Vasson' in the main text and 'Vasso' in Appendix A.7; please use the official method name consistently.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the SATM update is a standard SAM-style outer-loop construction; the only weak point is a proof gap, not a circular reduction.

full rationale

The central derivation is self-contained: SATM applies sharpness-aware minimization to the outer-loop objective of trajectory matching by defining F(phi)=L(theta_S(phi)), computing epsilon via a first-order Taylor approximation, and then taking the hypergradient at phi+epsilon (Eqs. 5-9). This is the standard SAM construction transplanted into a bilevel setting, and no accuracy result is used to define the objective. The trajectory-reusing approximation is supported by Theorem 4.2, which is an attempted proof from smoothness assumptions, not an assumption of the conclusion; even though the appendix proof appears dimensionally inconsistent (the sigma-smooth term is bounded as a constant rather than as a recursive function of Delta_theta), that is a correctness or rigor concern, not circularity. Self-citations to the authors' earlier works (Gao et al. 2021, 2022; Zhao et al. 2021) appear in related-work context and are not load-bearing for the proposed method. The sharpness dynamics reported in Fig. 2 measure the quantity being optimized, so they are confirmatory rather than independent evidence, but they are not presented as a prediction derived from a fitted input. Therefore no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The method depends on standard smoothness assumptions for its error bounds, on the empirically motivated flatness-generalization link, and on two ad hoc components (random image noise and trajectory reusing). None of these are machine-checked, and the proofs of Theorem 4.2 and Proposition 4.1 have gaps. No new physical or conceptual entities are introduced.

free parameters (3)
  • rho (sharpness radius) = not reported
    Controls the magnitude of the dataset perturbation for sharpness approximation; tuned per dataset in experiments but not listed in Table 11.
  • gamma (fluctuation noise strength) = not reported
    Scales the Gaussian noise added to each synthetic image (Algorithm 1 line 7); no default or search range given.
  • tau = iota (truncation and trajectory-reuse point) = set equal; see Section 5.2
    Number of inner-loop steps retained for hypergradient computation and where the second trajectory is initialized; studied as a fraction of N.
assumptions (6)
  • standard math L_CE is K-smooth, twice differentiable, and locally J-strongly convex in theta around {theta_{iota+1},...,theta_N}; inner-loop learning rate alpha <= 1/K.
    Assumed by Proposition 4.1 to bound the truncated hypergradient error; not guaranteed for neural nets but standard in optimization analysis.
  • standard math The matching loss L(phi, theta) is sigma-smooth and continuous in both arguments, and the mixed derivative grad_phi grad_theta L is beta-continuous.
    Assumed by Theorem 4.2; the proof uses a first-order Taylor expansion of grad L in phi.
  • domain assumption Flat minima of the outer-loop loss landscape imply condensed datasets that generalize to unseen tasks and architectures.
    Core motivation (Section 4.1); supported empirically in uni-level optimization but not proven for the bilevel trajectory matching case.
  • ad hoc to paper Random Gaussian noise on each synthetic image, phi_delta ~ N(0, gamma ||phi_j||^2 I), smooths the landscape without degrading dataset content.
    Introduced in Algorithm 1 lines 7-8 and Section 4.1; no ablation isolates its effect from the sharpness perturbation.
  • ad hoc to paper For tau steps, the trajectory trained on phi stays close to the trajectory trained on phi + epsilon, so reusing the first trajectory's midpoint is valid.
    The premise behind Trajectory Reusing (Section 4.2); the provided proof (Theorem 4.2) has a gap.
  • standard math The first-order Taylor approximation epsilon = rho grad F(phi) / ||grad F(phi)|| accurately approximates the sharpness maximizer.
    Standard SAM approximation (Eq. 7); valid only for small rho.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Generalization via Sharpness-Aware Trajectory Matching for Dataset Condensation." pith.science (2026). https://pith.science/paper/HV6KBDKK

@misc{pith2026250201865,
  author       = {Pith},
  title        = {Pith review of: Enhancing Generalization via Sharpness-Aware Trajectory Matching for Dataset Condensation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HV6KBDKK}},
  note         = {Machine review of arXiv:2502.01865}
}
read the original abstract

Dataset condensation aims to synthesize datasets with a few representative samples that can effectively represent the original datasets. This enables efficient training and produces models with performance close to those trained on the original sets. Most existing dataset condensation methods conduct dataset learning under the bilevel (inner- and outer-loop) based optimization. However, the preceding methods perform with limited dataset generalization due to the notoriously complicated loss landscape and expensive time-space complexity of the inner-loop unrolling of bilevel optimization. These issues deteriorate when the datasets are learned via matching the trajectories of networks trained on the real and synthetic datasets with a long horizon inner-loop. To address these issues, we introduce Sharpness-Aware Trajectory Matching (SATM), which enhances the generalization capability of learned synthetic datasets by optimising the sharpness of the loss landscape and objective simultaneously. Moreover, our approach is coupled with an efficient hypergradient approximation that is mathematically well-supported and straightforward to implement along with controllable computational overhead. Empirical evaluations of SATM demonstrate its effectiveness across various applications, including in-domain benchmarks and out-of-domain settings. Moreover, its easy-to-implement properties afford flexibility, allowing it to integrate with other advanced sharpness-aware minimizers. Our code will be released.

Figures

Figures reproduced from arXiv: 2502.01865 by the authors.

Figure 1
Figure 1. Test accuracy (%) comparison on continual learning. Left: 5-step class-incremental learning on Cifar10 50IPC, Middle: 10-step class-incremental learning on Tiny ImageNet 3IPC, Right: 20-step class-incremental learning on Tiny ImageNet 3IPC [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Sharpness analysis by visualisation. Hypergradient Norm comparison between MTT and SATM. Top: the hypergradient norm on Cifar100 with 10 IPC; Middle: the hypergradient norm on Tiny ImageNet with 3 IPC. Bottom: Sharpness dynamic on Tiny ImageNet with 3 IPC. once the differentiation steps reach a certain threshold. 6. Conclusions, Limitations and Future Works In this work, we explore the generalization ability of con￾… view at source ↗
Figure 3
Figure 3. The comparison of the learning dynamic of learning rate learning with first and second order differentiation when condensing on the Cifar100-10IPC setting. A.5. Computational Resource We conduct all our experiments on two TESLA V100-32GB GPUs with Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz and one A100-40GB GPU with Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz which are on different servers. Thus, we cannot run the full batc… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Cifar10 with 1IPC 16 [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Cifar10 with 3IPC [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Cifar10 with 10IPC 17 [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Cifar100 with 1IPC 18 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 10 canonical work pages

  1. [3]

    The comparison of the learning dynamic of learning rate learning with first and second order differentiation when condensing on the Cifar100-10IPC setting. A.5. Computational Resource We conduct all our experiments on two TESLA V100-32GB GPUs with Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz and one A100-40GB GPU with Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz whi...

    work page 2023

  2. [9]

    Dataset pruning: Reducing training data by examining generalization influence

    Yang, S., Xie, Z., Peng, H., Xu, M., Sun, M., and Li, P. Dataset pruning: Reducing training data by examining generalization influence. arXiv preprint arXiv:2205.09329,

    arXiv

  3. [10]

    0 2000 4000 6000 Training Iteration 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 Learning Rate Dynamic of Learning Rate Learning First Order Second Order Figure

    We chose the first in our experiments due to the implementation simplicity without causing any significant performance differences. 0 2000 4000 6000 Training Iteration 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 Learning Rate Dynamic of Learning Rate Learning First Order Second Order Figure

    work page 2000

  4. [12]

    Accuracy (%) Comparison of MTT, FTD, SATM, and SATM-FI across different datasets and configurations. A.7. Compatibility with Advanced Sharpness-Aware optimizers. We study the compatibility of the proposed hypergradient approximation method on other sharpness minimization-based methods including EMA, SAM (Foret et al., 2020), GSAM (Zhuang et al., 2021), AS...

    work page 2020

  5. [13]

    For a fair comparison, the hyperparameters of each method are properly tuned for the adaption to all the tasks including Cifar100 with 1 IPC and Tiny ImageNet with 3 IPC

    with our loss landscape smoothing mechanism removed. For a fair comparison, the hyperparameters of each method are properly tuned for the adaption to all the tasks including Cifar100 with 1 IPC and Tiny ImageNet with 3 IPC. We repeat each method 5 times and report the mean and variance in Table A.7. The results imply that all the sharpness methods consist...

    work page 2022

  6. [14]

    Test Accuracy (%) Comparison with the advanced sharpness aware minimization methods including EMA, SAM, GSAM, ASAM and Vasso with the same expert trajectories as MTT. A.8. Experiment Setting Details We conduct experiments on four main image datasets, Cifar10 (Krizhevsky et al., 2009), Cifar100 (Krizhevsky et al., 2009), TinyImageNet (Le & Yang,

    work page 2009

  7. [15]

    Cifar10 categorises 50,000 images with the size 32 × 32 into 10 classes while Cifar100 further categorises each of those 10 classes into 10 fine-grained subcategories

    and ImageNet (Russakovsky et al., 2015). Cifar10 categorises 50,000 images with the size 32 × 32 into 10 classes while Cifar100 further categorises each of those 10 classes into 10 fine-grained subcategories. TinyImageNet comprises 100,000 images distributed across 200 categories, each category consisting of 500 images resized to dimensions of 64 ×

    work page 2015

  8. [17]

    and ImageNet-1K (Russakovsky et al., 2015). The expert trajectories for Cifar10 and 15 Cifar100 are trained with 3-layer ConvNet and collected after each epoch with the initialisation, and those for TinyImageNet and ImageNet are trained with 4-layer and 5-layer ConvNet (Gidaris & Komodakis,

    work page 2015

Show all 19 references
  1. [18]

    respectively. In the in-domain setting, the synthetic datasets are learned and evaluated on the same architectures while in the out-of-domain settings, the learned synthetic datasets are deployed to train different architectures, such as AlexNet (Krizhevsky et al., 2012), VGG1...

    2012

  2. [64]

    We evaluate our methods on four main image datasets, Cifar10 (Krizhevsky et al., 2009), Cifar100 (Krizhevsky et al., 2009), TinyImageNet (Le & Yang,

    We further evaluate SATM on the subset of ImageNet, namely ImageNette, Image Woof, ImageFruit and ImageMeow with each set containing 10 different categories of 128 × 128 images and the whole ImageNet following the protocol from TESLA (Cui et al., 2023). We evaluate our methods...

    2023

  3. [2014]

    The trained neural networks are evaluated on the real test sets for generalization ability comparison of the synthetic datasets

    and ResNet18 (He et al., 2016), which is novel to the synthetic datasets. The trained neural networks are evaluated on the real test sets for generalization ability comparison of the synthetic datasets. A.9. Hyperparameters and Experiment Details The hyperparameters used for c...

    2016

  4. [2015]

    and Zisserman, A

    Simonyan, K. and Zisserman, A. Very deep convolu- tional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    arXiv

  5. [2016]

    Gradient noise convolution (gnc): Smoothing loss function for distributed large-batch sgd

    Haruki, K., Suzuki, T., Hamakawa, Y ., Toda, T., Sakai, R., Ozawa, M., and Kimura, M. Gradient noise convolution (gnc): Smoothing loss function for distributed large-batch sgd. arXiv preprint arXiv:1906.10822,

    1906 arXiv

  6. [2017]

    Sharpness-aware minimization for efficiently improving generalization

    Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412,

    2010 arXiv

  7. [2018]

    Improving task-free continual learning by distributionally robust memory evolution

    Wang, Z., Shen, L., Fang, L., Suo, Q., Duan, T., and Gao, M. Improving task-free continual learning by distributionally robust memory evolution. In ICML, 2022b. Wen, W., Wang, Y ., Yan, F., Xu, C., Wu, C., Chen, Y ., and Li, H. Smoothout: Smoothing out sharp minima to improve ...

    arXiv

  8. [2020]

    Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions

    MacKay, M., Vicol, P., Lorraine, J., Duvenaud, D., and Grosse, R. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. arXiv preprint arXiv:1903.03088,

    1903 arXiv

  9. [2021]

    N., Hao, X., Li, G., Sevilla-Lara, L., and Gowda, S

    9 Gowda, S. N., Hao, X., Li, G., Sevilla-Lara, L., and Gowda, S. N. Watt for what: Rethinking deep learn- ing’s energy-performance relationship. arXiv preprint arXiv:2310.06522,

    arXiv

  10. [2022]

    S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P

    Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836,

    arXiv

  11. [2024]

    Cafe: Learning to condense dataset by aligning features

    Wang, K., Zhao, B., Peng, X., Zhu, Z., Yang, S., Wang, S., Huang, G., Bilen, H., Wang, X., and You, Y . Cafe: Learning to condense dataset by aligning features. In CVPR, 2022a. Wang, T., Zhu, J.-Y ., Torralba, A., and Efros, A. A. Dataset distillation. arXiv preprint arXiv:1811.10959,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.