Pith. sign in

REVIEW 4 major objections 4 minor 69 references

Gradient-matched MAML converges at O(1/T) rate, paper claims

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

DGS-MAML layers gradient matching onto SharpMAML and claims O(1/T) convergence and tighter PAC-Bayes bounds, but the displayed theorems give O(1/sqrt T) under the paper's own parameter choices.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A straightforward SharpMAML+SAGM combination whose headline convergence and PAC-Bayes claims do not survive contact with its own equations; the experiments are too thin to make up for it. the 4 major comments →

arxiv 2508.09418 v1 pith:PCOLNBBF submitted 2025-08-13 cs.LG cs.AI

Domain-Generalization to Improve Learning in Meta-Learning Algorithms

Usman Anjum , Chris Stockman , Cat Luong , Justin Zhan This is my paper
classification cs.LG cs.AI
keywords meta-learningfew-shot learningdomain generalizationsharpness-aware minimizationgradient matchingbi-level optimizationPAC-Bayes generalizationconvergence analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DGS-MAML, a meta-learning algorithm that adds gradient matching to the sharpness-aware MAML (SharpMAML) framework. The authors aim to establish that aligning the gradients of the standard loss and the perturbed loss inside the bi-level optimization drives the meta-model to flatter minima, improving both convergence speed and generalization. The paper claims an $O(1/T)$ convergence rate for the averaged squared gradient norm with $\alpha$, $\delta$, $\gamma$ of order $1/\sqrt{T}$, improving on the claimed $O(1/\sqrt{T})$ rate for SharpMAML, and a tighter PAC-Bayes bound that includes an extra $\delta$ term. If these results hold, few-shot learners could adapt more accurately from very few examples without additional compute, which matters for applications where labeled data is scarce.

Core claim

The paper introduces DGS-MAML, a bi-level meta-learning algorithm that combines MAML's shared-initialization training with gradient matching between the empirical loss and its sharpness-aware perturbed counterpart. The central claim is that this coupling, via the surrogate gap $h(\theta)=L_p(\theta;D)-L(\theta;D)$, drives training to flatter minima and preserves the single-level convergence rate: with $\alpha, \delta, \gamma$ of order $1/\sqrt{T}$, the averaged squared gradient norm decreases as $O(1/T)$, versus the claimed $O(1/\sqrt{T})$ for SharpMAML. It also claims a tighter PAC-Bayes bound from an added $\delta^2$ term in the KL divergence. Experiments on Mini-Imagenet, Omniglot, Double

What carries the argument

DGS-MAML (Domain Generalization Sharpness-Aware Minimization Model-Agnostic Meta-Learning): a bi-level algorithm that updates meta-parameters by matching the gradient of the empirical loss with the gradient of a sharpness-aware perturbed loss. The mechanism rests on the surrogate gap $h(\theta)=L_p(\theta;D)-L(\theta;D)$, where $L_p$ is the loss at a perturbed parameter. Minimizing this gap converts sharpness avoidance into an inner-product maximization: when $\nabla L(\theta)$ and $\nabla L_p(\theta)$ align, the bound improves. The extra hyperparameter $\delta$ controls the balance between perturbation radius and gradient step, and it is the term that appears in the PAC-Bayes KL bound.

Load-bearing premise

The PAC-Bayes guarantee relies on Eq. (32), the assumption that the true loss at the returned parameter is no larger than the expected true loss after a Gaussian perturbation; this flat-minimum dominance is exactly the property that gradient matching is meant to create, so the proof places a key part of its conclusion into its hypotheses.

What would settle it

Train DGS-MAML on Mini-Imagenet 5-way 5-shot with $\alpha=0.05$ and $\delta=0.01$, record the final $\hat{\theta}$, and estimate both sides of Eq. (32) by averaging validation losses over sampled perturbations $\epsilon \sim \mathcal{N}(0,(\alpha^2+\delta^2)I)$. If the left-hand side exceeds the right-hand side by more than a standard error, the paper's key PAC-Bayes assumption does not hold for the parameters it actually returns. A second check: track the averaged squared gradient norm across outer iterations and fit a power law; a slope near $-1/2$ would contradict the claimed $O(1/T)$ rate.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, meta-learners can reach a given gradient-norm target in fewer outer-loop iterations than SharpMAML when $\alpha$, $\delta$, $\gamma$ are tuned to $O(1/\sqrt{T})$, cutting few-shot adaptation cost.
  • The tighter PAC-Bayes bound implies that the $\delta$-regularized surrogate gap improves worst-case generalization guarantees, not just average accuracy.
  • DGS-MAML's runtime stays in the same range as SharpMAML (60–70 seconds per epoch), so the claimed gains do not require extra compute.
  • On Mini-Imagenet, accuracy improves from 0.4509 to 0.4665 (5-way 1-shot) and from 0.5759 to 0.6360 (5-way 5-shot), showing the benefit is largest when baseline accuracy is low.
  • The $\delta$-ablation tables show a single best $\delta$ per setting; future use would benefit from automatic $\delta$ selection.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The $O(1/T)$ rate hinges on the surrogate-gap identity $\nabla h = \nabla L_p - \nabla L$; if the gradient-matching term weakens on non-smooth losses, the rate may degrade toward the SharpMAML baseline in practice.
  • The same gradient-matching mechanism could be ported to other bi-level or federated meta-learning setups where flat minima and quick adaptation are both desired, not only to MAML-style algorithms.
  • Because the PAC-Bayes bound's KL term is monotonically decreasing in $\delta$ for fixed $\alpha$, the theory predicts that a larger $\delta$ tightens the bound, but the experiments show accuracy peaks at a specific $\delta$; reconciling this tension is a testable hypothesis.
  • The convergence proof for SAGM is claimed to be the first; it might be reused to analyze other SAM variants with gradient-matching regularizers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DGS-MAML, a meta-learning algorithm that combines sharpness-aware minimization (SAM) with gradient matching in a bi-level framework. The claimed contributions are (i) a convergence analysis showing that DGS-MAML achieves an O(1/T) rate with respect to the number of iterations T, improving on the claimed O(1/√T) rate of SharpMAML, and (ii) a PAC-Bayes generalization bound that is tighter than SharpMAML's. The method is evaluated on Mini-Imagenet, Omniglot, DoubleMNIST, and TripleMNIST, with reported accuracy improvements over MAML, SharpMAML, and other baselines.

Significance. If the theoretical claims were sound, the work would provide a useful extension of SAGM to meta-learning, with the non-obvious benefit of faster convergence and better generalization bounds at no additional computational cost. The empirical results, while positive, are presented without error bars and with only modest gains; the principal value of the paper is therefore intended to be theoretical. The paper also makes the effort of providing a convergence analysis for single-level SAGM, which was previously missing, and releases source code. However, the two theoretical pillars—the convergence rate and the PAC-Bayes improvement—rest on invalid derivations and unproven assumptions, as detailed below. Because these are load-bearing, the claims as stated are not established.

major comments (4)
  1. [§5.1, Eq. (20)–(22)] The proof of Theorem 1 mishandles the inequality direction when applying Lemma 4. In Eq. (20) the RHS contains the term −(γ²L/2)‖∇h‖². Lemma 4 provides an upper bound on ‖∇h‖². Substituting an upper bound into a negative term yields a lower bound on the RHS, not an upper bound; the valid conclusion from Lemma 4 is only that −(γ²L/2)‖∇h‖² ≤ 0. Therefore Eq. (21) and the explicit bound in Eq. (22) do not follow. The stated O(1/T) corollary may still be derivable by other arguments, but the theorem as proven is unsupported.
  2. [§5.2, Theorem 2 vs. Algorithm 1] The convergence analysis for DGS-MAML does not model the algorithm described in Algorithm 1. Theorem 2 analyzes a stochastic single-level SAGM update of the form θ_{t+1} = θ_t − γ(∇L + ∇L_p). Algorithm 1, however, has an inner loop over tasks with training loss, an outer loop with validation loss, task-specific perturbations ε_m and a global perturbation ε, and two different radii α_l and α_u. None of these elements appear in the assumptions or the proof. Thus the claimed convergence guarantee for the actual DGS-MAML procedure is not established.
  3. [§6, Eq. (32)] The PAC-Bayes proof assumes E_D L(θhat;D) ≤ E_ε E_D L(θhat+ε;D) with ε ∼ N(0,(α²+δ²)I). This is asserted as an assumption that 'will follow' from local minimality, but local minimality of the empirical loss does not imply this inequality for the true distribution, and the Gaussian perturbation has unbounded support. The inequality is essentially the flat-minimum dominance property that DGS-MAML is designed to induce, so the advertised generalization improvement is loaded into the assumptions rather than derived from the algorithm's updates. If Eq. (32) fails for the parameters actually found, the PAC-Bayes guarantee disappears.
  4. [§6, Theorem 3 and Eq. (31)] The total PAC-Bayes bound in Eq. (31) contains a uniform-stability constant U that is never bounded or computed for DGS-MAML. The comparison with SharpMAML (Eqs. 33–36) only analyzes the KL term. A smaller KL does not imply a smaller total PAC-Bayes bound without control of U, and the condition σ_p² ≥ (α²+δ²)e does not provide such control. Therefore the conclusion in §9 that DGS-MAML 'offers a better PAC-Bayes generalization bound' does not follow from the presented analysis.
minor comments (4)
  1. [General] There are numerous typographical errors: 'F ormulation' in the Introduction, 'suppply' in §2.1, 'gradent matching' in §4.1, and 'Kullback-Liebler' in §6 should be 'Kullback-Leibler'.
  2. [Tables 4 and 5] The captions of Tables 4 and 5 are inconsistent with the column headers. Table 4 is captioned '10-way 1-shot and 10-way 5-shot TripleMNIST' but shows '20-way' columns; Table 5 has the reverse inconsistency.
  3. [Algorithm 1] The pseudocode updates θ_t inside the inner loop over tasks and again in the outer loop, but the indexing is ambiguous: both use 'θ_t = θ_{t−1} − γ∇L_GM'. It is not clear whether the inner update is intended to be a temporary task-specific parameter or the meta-parameter, and this ambiguity matters for the convergence interpretation.
  4. [Experimental results] The experimental tables report point estimates without error bars or statistical significance tests. Given the small differences between DGS-MAML and SharpMAML on some datasets, confidence intervals are needed to support the empirical claim.

Circularity Check

1 steps flagged

PAC-Bayes improvement is not derived: Eq. 32 assumes the flat-minimum property DGS-MAML is designed to deliver; the stability constant U is never bounded.

specific steps
  1. self definitional [Section 6, Equation (32)]
    "In our case, we have that some parameter ˆθ∈R d is the learned hypothesis and is a local minimizer. The following assumption needed for our proof will follow: E D∼S L(ˆθ;D)≤E ϵ∼N(0,(α 2+δ2)I)E D∼S L(ˆθ+ϵ;D). (32)"

    Eq. (32) asserts that the unperturbed expected loss at the learned parameter is no larger than the expectation of the Gaussian-perturbed loss. That is exactly the flat-minimum dominance the DGS-MAML objective (surrogate-gap/gradient matching, Eqs. 7–11) is designed to enforce—the paper states the goal is to 'find a flat region in the loss landscape.' The paper provides no derivation of Eq. (32) from the DGS-MAML updates; it simply declares it an 'assumption needed for our proof.' The advertised better PAC-Bayes bound then follows only by assuming the very property the algorithm is claimed to deliver. A local minimizer of the empirical task loss need not satisfy this inequality for the true data distribution, and the Gaussian perturbation has unbounded support.

full rationale

The convergence-rate analysis (Theorems 1–2, Corollaries 1–2) is self-contained: it derives O(1/T) rates for SAGM and DGS-MAML from Lipschitz, bounded-gradient, and bounded-variance assumptions, and does not depend on the paper's own conclusions. The PAC-Bayes analysis, however, does not establish the advertised improvement over SharpMAML. The key step is Eq. (32), which assumes the learned parameter's expected loss is no larger than the expectation of its Gaussian-perturbed loss. This is precisely the flat-minimum dominance that gradient matching is introduced to force, and no proof is given that the DGS-MAML updates produce a parameter satisfying it. The subsequent KL computation and the assertion that adding δ 'further reduces the generalization error' rest on this assumption of the conclusion. In addition, Theorem 3's total PAC-Bayes bound includes a uniform-stability constant U (Eq. 31) that is never bounded for DGS-MAML; comparing only the KL terms (Eqs. 33–36) does not imply a smaller total bound. These are load-bearing gaps in the paper's claimed PAC-Bayes advantage. The experimental results are empirical and cannot supply the missing theoretical derivation. Overall, one central claim partially reduces to its own assumption, so the score is 7 rather than 8; the independent convergence work provides some non-circular content.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claims rest on standard smoothness and boundedness assumptions plus two hand-loaded items: the delta hyperparameter tuned separately for every experiment, and Eq. 32, which assumes the flatness behavior the method claims to deliver. No new architectural or physical entity is introduced.

free parameters (3)
  • delta (gradient-matching weight) = 0.005 to 2.0 per dataset and shot setting
    Tables 7 and 8 show the best delta selected after observing test accuracy; Section 7.3 admits that fine-tuning delta is crucial.
  • alpha (SAM perturbation radius) = 0.05 or 0.005
    Two values are tried and the better one is reported; alpha defines the perturbation radii in both levels.
  • sigma_p^2 (PAC-Bayes prior variance) = not specified; only constrained to sigma_p^2 >= e(alpha^2 + delta^2)
    The KL bound in Eq. 35 depends on a concrete prior variance, but no value is provided.
axioms (5)
  • standard math Loss gradient is L-Lipschitz continuous (Assumption 1)
    Used in the descent inequality, Eq. 13.
  • domain assumption Stochastic gradient is uniformly bounded, ||nabla L(theta;D)||_inf <= C (Assumption 2)
    Used to bound ||nabla L|| and ||nabla L_p|| in Lemmas 3 through 5; C is stated to be in N, though a real bound would be more natural.
  • domain assumption Gradient estimators are unbiased with bounded variances sigma_1^2 and sigma_2^2 (Assumption 3)
    Needed for the bi-level stochastic convergence proof, Eqs. 23 to 28.
  • ad hoc to paper Local-minimizer dominance: E_D L(theta_hat;D) <= E_epsilon E_D L(theta_hat + epsilon;D) (Eq. 32)
    Assumed to make the PAC-Bayes bound apply; this is the flatness property the algorithm is supposed to produce.
  • standard math PAC-Bayes uniform stability theorem of Abbas et al. (Theorem 3)
    Quoted as the template for the generalization bound; the stability constant U is never bounded.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Domain-Generalization to Improve Learning in Meta-Learning Algorithms." pith.science (2026). https://pith.science/paper/PCOLNBBF

@misc{pith2026250809418,
  author       = {Pith},
  title        = {Pith review of: Domain-Generalization to Improve Learning in Meta-Learning Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PCOLNBBF}},
  note         = {Machine review of arXiv:2508.09418}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper introduces Domain Generalization Sharpness-Aware Minimization Model-Agnostic Meta-Learning (DGS-MAML), a novel meta-learning algorithm designed to generalize across tasks with limited training data. DGS-MAML combines gradient matching with sharpness-aware minimization in a bi-level optimization framework to enhance model adaptability and robustness. We support our method with theoretical analysis using PAC-Bayes and convergence guarantees. Experimental results on benchmark datasets show that DGS-MAML outperforms existing approaches in terms of accuracy and generalization. The proposed method is particularly useful for scenarios requiring few-shot learning and quick adaptation, and the source code is publicly available at GitHub.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 56 canonical work pages · 6 internal anchors

  1. [1]

    In: International Conference on Machine Learning, pp

    Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126– 1135 (2017). PMLR

    work page 2017

  2. [2]

    SIAM review60(2), 223–311 (2018)

    Bottou, L., Curtis, F.E., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM review60(2), 223–311 (2018)

    work page 2018

  3. [3]

    In: International Conference on Machine Learning, pp

    Abbas, M., Xiao, Q., Chen, L., Chen, P.-Y., Chen, T.: Sharp-maml: Sharpness- aware model-agnostic meta learning. In: International Conference on Machine Learning, pp. 10–32 (2022). PMLR

    work page 2022

  4. [4]

    arXiv preprint arXiv:2010.01412 (2020)

    Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412 (2020)

    Pith/arXiv arXiv 2010

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Wang, P., Zhang, Z., Lei, Z., Zhang, L.: Sharpness-aware gradient matching for domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3769–3778 (2023) 20

    work page 2023

  6. [6]

    arXiv preprint arXiv:2203.08065 (2022)

    Zhuang, J., Gong, B., Yuan, L., Cui, Y., Adam, H., Dvornek, N., Tatikonda, S., Duncan, J., Liu, T.: Surrogate gap minimization improves sharpness-aware training. arXiv preprint arXiv:2203.08065 (2022)

    Pith/arXiv arXiv 2022

  7. [7]

    In: International Conference on Machine Learning, pp

    Zintgraf, L., Shiarli, K., Kurin, V., Hofmann, K., Whiteson, S.: Fast context adaptation via meta-learning. In: International Conference on Machine Learning, pp. 7693–7702 (2019). PMLR

    work page 2019

  8. [8]

    Advances in neural information processing systems32(2019)

    Rajeswaran, A., Finn, C., Kakade, S.M., Levine, S.: Meta-learning with implicit gradients. Advances in neural information processing systems32(2019)

    work page 2019

  9. [9]

    arXiv preprint arXiv:1909.09157 (2019)

    Raghu, A., Raghu, M., Bengio, S., Vinyals, O.: Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv:1909.09157 (2019)

    Pith/arXiv arXiv 1909

  10. [10]

    Advances in neural information processing systems29(2016)

    Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. Advances in neural information processing systems29(2016)

    work page 2016

  11. [11]

    arXiv preprint arXiv:1803.02999 (2018)

    Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 (2018)

    Pith/arXiv arXiv 2018

  12. [12]

    Advances in neural information processing systems31(2018)

    Yoon, J., Kim, T., Dia, O., Kim, S., Bengio, Y., Ahn, S.: Bayesian model-agnostic meta-learning. Advances in neural information processing systems31(2018)

    work page 2018

  13. [13]

    Advances in neural information processing systems31(2018)

    Finn, C., Xu, K., Levine, S.: Probabilistic model-agnostic meta-learning. Advances in neural information processing systems31(2018)

    work page 2018

  14. [14]

    arXiv preprint arXiv:1801.08930 (2018)

    Grant, E., Finn, C., Levine, S., Darrell, T., Griffiths, T.: Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930 (2018)

    Pith/arXiv arXiv 2018

  15. [15]

    1733–1774 (2022)

    Chen, L., Chen, T.: Is bayesian model-agnostic meta learning better than model- agnostic meta learning, provably? In: International Conference on Artificial Intelligence and Statistics, pp. 1733–1774 (2022). PMLR

    work page 2022

  16. [16]

    Advances in Neural Information Processing Systems31(2018)

    Denevi, G., Ciliberto, C., Stamos, D., Pontil, M.: Learning to learn around a common mean. Advances in Neural Information Processing Systems31(2018)

    work page 2018

  17. [17]

    Advances in Neural Information Processing Systems34, 2173–2186 (2021)

    Farid, A., Majumdar, A.: Generalization bounds for meta-learning via pac-bayes and uniform stability. Advances in Neural Information Processing Systems34, 2173–2186 (2021)

    work page 2021

  18. [18]

    In: Uncertainty in Artificial Intel- ligence, pp

    Wang, Z., Grigsby, J., Sekhon, A., Qi, Y.: St-maml: A stochastic-task based method for task-heterogeneous meta-learning. In: Uncertainty in Artificial Intel- ligence, pp. 2066–2074 (2022). PMLR

    work page 2066

  19. [19]

    Neural Computing and Applications, 1–28 (2025) 21

    Anjum, U., Stockman, C., Luong, C., Zhan, F.: Using adaptive learning and momentum to improve generalization. Neural Computing and Applications, 1–28 (2025) 21

    work page 2025

  20. [20]

    On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

    Wang, R., Xu, K., Liu, S., Chen, P.-Y., Weng, T.-W., Gan, C., Wang, M.: On fast adversarial robustness adaptation in model-agnostic meta-learning. arXiv preprint arXiv:2102.10454 (2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  21. [21]

    Advances in Neural Information Processing Systems33, 17886–17895 (2020)

    Goldblum, M., Fowl, L., Goldstein, T.: Adversarially robust few-shot learning: A meta-learning approach. Advances in Neural Information Processing Systems33, 17886–17895 (2020)

    work page 2020

  22. [22]

    In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp

    Xu, H., Li, Y., Liu, X., Liu, H., Tang, J.: Yet meta learning can adapt fast, it can also break easily. In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 540–548 (2021). SIAM

    work page 2021

  23. [23]

    Advances in Neural Information Processing Systems34, 3096–3107 (2021)

    Fallah, A., Georgiev, K., Mokhtari, A., Ozdaglar, A.: On the convergence the- ory of debiased model-agnostic meta-reinforcement learning. Advances in Neural Information Processing Systems34, 3096–3107 (2021)

    work page 2021

  24. [24]

    Advances in Neural Information Processing Systems33, 18860–18871 (2020)

    Collins, L., Mokhtari, A., Shakkottai, S.: Task-robust model-agnostic meta- learning. Advances in Neural Information Processing Systems33, 18860–18871 (2020)

    work page 2020

  25. [25]

    Advances in Neural Information Processing Systems33, 3557–3568 (2020)

    Fallah, A., Mokhtari, A., Ozdaglar, A.: Personalized federated learning with theo- retical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems33, 3557–3568 (2020)

    work page 2020

  26. [26]

    In: International Conference on Artificial Intelligence and Statistics, pp

    Fallah, A., Mokhtari, A., Ozdaglar, A.: On the convergence theory of gradient- based model-agnostic meta-learning algorithms. In: International Conference on Artificial Intelligence and Statistics, pp. 1082–1092 (2020). PMLR

    work page 2020

  27. [27]

    Sustainability 16(15), 6705 (2024)

    Zarreh, M., Khandan, M., Goli, A., Aazami, A., Kummer, S.: Integrating per- ishables into closed-loop supply chains: a comprehensive review. Sustainability 16(15), 6705 (2024)

    work page 2024

  28. [28]

    Computers & industrial engineering100, 34–51 (2016)

    Makui, A., Heydari, M., Aazami, A., Dehghani, E.: Accelerating benders decom- position approach for robust aggregate production planning of products with a very limited expiration date. Computers & industrial engineering100, 34–51 (2016)

    work page 2016

  29. [29]

    Scientia Iranica26(5), 3007–3031 (2019)

    Aazami, A., Saidi-Mehrabad, M.: Benders decomposition algorithm for robust aggregate production planning considering pricing decisions in competitive envi- ronment: A case study. Scientia Iranica26(5), 3007–3031 (2019)

    work page 2019

  30. [30]

    Interna- tional journal of artificial intelligence16(1), 88–112 (2018)

    Goli, A., Aazami, A., Jabbarzadeh, A.: Accelerated cuckoo optimization algo- rithm for capacitated vehicle routing problem in competitive conditions. Interna- tional journal of artificial intelligence16(1), 88–112 (2018)

    work page 2018

  31. [31]

    Journal of 22 industrial and systems engineering10(3), 140–160 (2017)

    Saeedi Mehrabad, M., Aazami, A., Goli, A.: A location-allocation model in the multi-level supply chain with multi-objective evolutionary approach. Journal of 22 industrial and systems engineering10(3), 140–160 (2017)

    work page 2017

  32. [32]

    In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp

    Azad, N., Aazami, A., Papi, A., Jabbarzadeh, A.: A two-phase genetic algorithm for incorporating environmental considerations with production, inventory and routing decisions in supply chain networks. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 41–42 (2019)

    work page 2019

  33. [33]

    Journal of industrial and systems engineering11(2), 134–150 (2018)

    Heydari, M., Aazami, A.: Minimizing the maximum tardiness and makespan cri- teria in a job shop scheduling problem with sequence dependent setup times. Journal of industrial and systems engineering11(2), 134–150 (2018)

    work page 2018

  34. [34]

    Journal of machine learning research12(7) (2011)

    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research12(7) (2011)

    work page 2011

  35. [35]

    arXiv preprint arXiv:1412.6980 (2014)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    Pith/arXiv arXiv 2014

  36. [36]

    arXiv preprint arXiv:1904.09237 (2019)

    Reddi, S.J., Kale, S., Kumar, S.: On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237 (2019)

    Pith/arXiv arXiv 1904

  37. [37]

    arXiv preprint arXiv:2302.09693 (2023)

    Behdin, K., Song, Q., Gupta, A., Acharya, A., Durfee, D., Ocejo, B., Keerthi, S., Mazumder, R.: msam: Micro-batch-averaged sharpness-aware minimization. arXiv preprint arXiv:2302.09693 (2023)

    Pith/arXiv arXiv 2023

  38. [38]

    K-SAM: Sharpness-Aware Minimization at the Speed of SGD

    Ni, R., Chiang, P.-y., Geiping, J., Goldblum, M., Wilson, A.G., Goldstein, T.: K-sam: Sharpness-aware minimization at the speed of sgd. arXiv preprint arXiv:2210.12864 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  39. [39]

    arXiv preprint arXiv:2110.03141 (2021)

    Du, J., Yan, H., Feng, J., Zhou, J.T., Zhen, L., Goh, R.S.M., Tan, V.Y.: Efficient sharpness-aware minimization for improved training of neural networks. arXiv preprint arXiv:2110.03141 (2021)

    Pith/arXiv arXiv 2021

  40. [40]

    Kaddour, J., Liu, L., Silva, R., Kusner, M.J.: When do flat minima optimiz- ers work? Advances in Neural Information Processing Systems35, 16577–16595 (2022)

    work page 2022

  41. [41]

    In: International Conference on Machine Learning, pp

    Kwon, J., Kim, J., Park, H., Choi, I.K.: Asam: Adaptive sharpness-aware min- imization for scale-invariant learning of deep neural networks. In: International Conference on Machine Learning, pp. 5905–5914 (2021). PMLR

    work page 2021

  42. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Shu, Y., Cao, Z., Wang, C., Wang, J., Long, M.: Open domain generalization with domain-augmented meta-learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9624–9633 (2021)

    work page 2021

  43. [43]

    arXiv preprint arXiv:2006.05929 (2020)

    Zhao, B., Mopuri, K.R., Bilen, H.: Dataset condensation with gradient matching. arXiv preprint arXiv:2006.05929 (2020)

    Pith/arXiv arXiv 2006

  44. [44]

    In: International Conference on Machine Learning, pp

    Killamsetty, K., Durga, S., Ramakrishnan, G., De, A., Iyer, R.: Grad-match: 23 Gradient matching based data subset selection for efficient deep model training. In: International Conference on Machine Learning, pp. 5464–5474 (2021). PMLR

    work page 2021

  45. [45]

    arXiv preprint arXiv:2104.09937 (2021)

    Shi, Y., Seely, J., Torr, P.H., Siddharth, N., Hannun, A., Usunier, N., Synnaeve, G.: Gradient matching for domain generalization. arXiv preprint arXiv:2104.09937 (2021)

    Pith/arXiv arXiv 2021

  46. [46]

    In: 2023 IEEE International Conference on Omni-layer Intelligent Systems (COINS), pp

    Jiang, Z., Gu, J., Liu, M., Pan, D.Z.: Delving into effective gradient matching for dataset condensation. In: 2023 IEEE International Conference on Omni-layer Intelligent Systems (COINS), pp. 1–6 (2023). IEEE

    work page 2023

  47. [47]

    Gradient-Matching Coresets for Rehearsal-Based Continual Learning

    Balles, L., Zappella, G., Archambeau, C.: Gradient-matching coresets for rehearsal-based continual learning. arXiv preprint arXiv:2203.14544 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  48. [48]

    IEEE Transactions on Neural Networks and Learning Systems (2022)

    Zeng, L.-L., Fan, Z., Su, J., Gan, M., Peng, L., Shen, H., Hu, D.: Gradient match- ing federated domain adaptation for brain image classification. IEEE Transactions on Neural Networks and Learning Systems (2022)

    work page 2022

  49. [49]

    In: International Conference on Learning Representations (2016)

    Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: International Conference on Learning Representations (2016)

    work page 2016

  50. [50]

    https://github

    Sun, S.-H.: Multi-digit MNIST for Few-shot Learning (2019). https://github. com/shaohua0116/MultiDigitMNIST

    work page 2019

  51. [51]

    In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol

    Lake, B., Salakhutdinov, R., Gross, J., Tenenbaum, J.: One shot learning of simple visual concepts. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 33 (2011)

    work page 2011

  52. [52]

    Torchmeta: A Meta-Learning library for PyTorch

    Deleu, T., W¨ urfl, T., Samiei, M., Cohen, J.P., Bengio, Y.: Torchmeta: A Meta-Learning library for PyTorch. Available at: https://github.com/tristandeleu/pytorch-meta (2019). https://arxiv.org/abs/ 1909.06576

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  53. [53]

    Advances in neural information processing systems30(2017)

    Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. Advances in neural information processing systems30(2017)

    2017

  54. [54]

    In: Hughes, A.L., McNeill, F., Zobel, C.W

    Lazreg, M.B., Anjum, U., Zadorozhny, V., Goodwin, M.: Semantic decay fil- ter for event detection. In: Hughes, A.L., McNeill, F., Zobel, C.W. (eds.) 17th International Conference on Information Systems for Crisis Response and Man- agement, ISCRAM 2020, May 2020, pp. 14–26. ISCRAM Digital Library, ??? (2020).https://idl.iscram.org/show.php?record=2203

    work page 2020

  55. [55]

    In: Adrot, A., Grace, R., Moore, K.A., Zobel, C.W

    Anjum, U., Zadorozhny, V., Krishnamurthy, P.: TBAM: towards an agent- based model to enrich twitter data. In: Adrot, A., Grace, R., Moore, K.A., Zobel, C.W. (eds.) 18th International Conference on Information Sys- tems for Crisis Response and Management, ISCRAM 2021, Blacksburg, V A, USA, May 2021, pp. 146–158. ISCRAM Digital Library, ??? (2021). 24 https...

    work page 2021

  56. [56]

    Online Soc

    Anjum, U., Zadorozhny, V., Krishnamurthy, P.: Localization of unidentified events with raw microblogging data. Online Soc. Networks Media29, 100209 (2022) https://doi.org/10.1016/J.OSNEM.2022.100209

    work page arXiv 2022

  57. [57]

    In: The Fifteenth International Conference on Information, Process, and Knowledge Management, eKNOW23 (2023)

    Nkhata, G., Anjum, U., Zhan, J.: Sentiment analysis of movie reviews using bert. In: The Fifteenth International Conference on Information, Process, and Knowledge Management, eKNOW23 (2023)

    work page 2023

  58. [58]

    In: Proceedings of the International ISCRAM Conference (2025)

    Anjum, U., Zhan, J.: Weighted trilateration using domain knowledge for event localization in twitter data. In: Proceedings of the International ISCRAM Conference (2025)

    work page 2025

  59. [59]

    In: Khazanchi, J.R.I.D.N.L.D

    Anjum, U., Zadorozhny, V., Krishnamurthy, P.: Localization of events using neural networks in twitter data. In: Khazanchi, J.R.I.D.N.L.D. (ed.) 20th Interna- tional Conference on Information Systems for Crisis Response and Management, ISCRAM 2023, Omaha, Nebraska, USA, May 2023, pp. 909–919. ISCRAM Digital Library, ??? (2023).http://dx.doi.org/10.59297/UVZV1884

    work page doi:10.59297/uvzv1884 2023

  60. [60]

    In: Tsumoto, S., Ohsawa, Y., Chen, L., Poel, D.V., Hu, X., Motomura, Y., Takagi, T., Wu, L., Xie, Y., Abe, A., Raghavan, V

    Guo, X., Anjum, U., Zhan, J.: Cyberbully detection using BERT with augmented texts. In: Tsumoto, S., Ohsawa, Y., Chen, L., Poel, D.V., Hu, X., Motomura, Y., Takagi, T., Wu, L., Xie, Y., Abe, A., Raghavan, V. (eds.) IEEE International Conference on Big Data, Big Data 2022, Osaka, Japan, December 17-20, 2022, pp. 1246–1253. IEEE, ??? (2022). https://doi.org...

    arXiv 2022

  61. [61]

    Briefings in Bioinformatics25(Supplement 1), 232 (2024)

    Woessner, A.E., Anjum, U., Salman, H., Lear, J., Turner, J.T., Campbell, R., Beaudry, L., Zhan, J., Cornett, L.E., Gauch, S.,et al.: Identifying and train- ing deep learning neural networks on biomedical-related datasets. Briefings in Bioinformatics25(Supplement 1), 232 (2024)

    work page 2024

  62. [62]

    Electronics13(19), 3825 (2024)

    Anjum, U., Zhan, J.: A novel tsetlin machine with enhanced generalization. Electronics13(19), 3825 (2024)

    work page 2024

  63. [63]

    A Novel Approach To Implementing Knowledge Distillation In Tsetlin Machines

    Kinateder, C.: A novel approach to implementing knowledge distillation in tsetlin machines. arXiv preprint arXiv:2504.01798 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  64. [64]

    Adaptive Temperature Based on Logits Correlation in Knowledge Distillation

    Matsuyama, K., Anjum, U., Matsuyama, S., Shoda, T., Zhan, J.: Adaptive tem- perature based on logits correlation in knowledge distillation. arXiv preprint arXiv:2503.09030 (2025) 25 10 Appendix 10.1 Lemmas In this section, we show the Lemmas necessary for the formulation of our proofs. Letk=C √ d(1 + α C √ d −δ) in the following. Lemma 1: BoundingE[⟨e∇L(θ...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  65. [65]

    Lemma 2: BoundingLγ2 2 E[|| e∇LSAGM ||2 | Ft]

    (40) This completes the proof. Lemma 2: BoundingLγ2 2 E[|| e∇LSAGM ||2 | Ft]. Lγ2 2 E[|| e∇LSAGM ||2 | Ft] = Lγ2 2 E[|| e∇L(θt) + e∇Lp(θt)||2 | Ft] (41) =Lγ 2E[2|| e∇L(θt)||2 + 2||e∇Lp||2 − ||e∇Lp − e∇L(θt)||2 | Ft] (42) =Lγ 2(2E||∇L||2 + 2E||e∇Lp||2 −E[|| e∇Lp − e∇L(θt)||2 | Ft]) (43) ≤Lγ 2(2(C 2d+σ 2

    work page

  66. [66]

    + 2(k2 +σ 2 2)−E[|| e∇h||2 | Ft] (44) By Lemma 5, we now have: ≤Lγ 2(2(C 2d+σ 2

    work page

  67. [67]

    Lemma 3: Bounding||∇Lp||

    + 2(k2 +σ 2 2) −(C 2d+k 2 +σ 2 1 +σ 2 2 −2 q (C 2d+σ 2 1)(k2 +σ 2 2)) (45) =Lγ 2(C 2d+σ 2 1 +σ 2 2 +k 2 + 2 q (C 2d+σ 2 1)(k2 +σ 2 2)) (46) This completes the proof. Lemma 3: Bounding||∇Lp||. Since||∇L|| ≤C √ dby assumption 2, then: ||∇Lp(θt)||=||∇L(θ t + ˆε−δ∇L(θt))||(47) ≤ s (C+ αC C √ d −δC) 2 +...(48) 26 ≤ r (C+ α√ d −δC) 2d(49) ≤C √ d(1 + α C √ d −δ)...

    work page

  68. [68]

    + 2(k2 +σ 2 2)− ||e∇Lp + e∇L||2 (59) ≤2(C 2d+k 2) + 2(σ2 1 +σ 2 2) −(|| e∇Lp||2 + 2||e∇Lp|| · ||e∇L||+|| e∇L||2) (60) ≤C 2d+k 2 +σ 2 1 +σ 2 2 −2 q (C 2d+σ 2 1)(k2 +σ 2

    work page

  69. [69]

    (61) This completes the proof. 27

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.