Pith. sign in

REVIEW 4 major objections 5 minor 59 references

SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SHIELD is the first certifiably robust continual learning method: a hypernetwork generates each task's classifier weights while interval-bound training gives formal guarantees, reaching 85.64% AutoAttack average accuracy on Rotated MNIST.

desk verdict Real empirical gains, but the 'certified' continual learning claim is not backed by the training procedure—the robustness-preservation condition is only checked post-hoc. read the letter →

arxiv 2506.08255 v4 pith:QIDPBJIY submitted 2025-06-09 cs.LG cs.AIcs.CR

Patryk Krukowski , {L}ukasz Gorczyca , Piotr Helm , Kamil Książek , Przemys{l}aw Spurek This is my paper
classification cs.LGcs.AIcs.CR
keywords certifiedrobustnesscontinuallearninghypernetworksintervalboundpropagationmixupadversarialtask-incrementalcatastrophicforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that continual learning and certified adversarial robustness can be combined in one scalable framework, without replay buffers, gradient logs, or full model copies. The proposed method, SHIELD, uses a shared hypernetwork — a small network that generates the weights of another network — to produce task-specific classifiers from compact trainable embeddings, and trains those classifiers with Interval Bound Propagation so that every prediction inside an $\ell_\infty$-bounded box around each input is formally guaranteed. A new training strategy called Interval MixUp interpolates these perturbation boxes with a radius that shrinks as the virtual sample moves away from real data, which tightens the propagated bounds and pushes decision boundaries away from the data. The promise is that adversarial robustness survives sequential learning instead of being traded away, and the reported results back this up: SHIELD reaches 85.64% AutoAttack average accuracy on Rotated MNIST, 60.91% on Split CIFAR-100, and 56.22% on Split miniImageNet, ahead of every baseline in the comparison.

What carries the argument

The load-bearing machinery is the coupling of a weight-generating hypernetwork with Interval Bound Propagation (IBP), which propagates an input hypercube $[x-\varepsilon, x+\varepsilon]$ through the network in midpoint-radius form to obtain certified logit bounds. The hypernetwork $H(e_t; \Phi)$ maps task embeddings to classifier weights, so task knowledge lives in a few trainable vectors; the regularization $L_{\mathrm{out}}$ from Eq. (10) limits drift of previously generated weights. Theorem 3.1 turns that drift limit into a certificate-preservation argument: robustness survives when the logit drift is at most half the certified margin. Interval MixUp completes the mechanism by interpolating interval boxes between pairs of samples with radius $\varepsilon' = |2\lambda-1|\cdot\varepsilon$, producing virtual certified regions that are smaller far from the data and thereby reducing the wrapping effect and loosening of bounds.

What would settle it

Run AutoAttack only on the few percent of earlier-task test samples for which the supplementary material's condition $M > 2\Delta_{\max}$ fails, and check whether any of them become misclassified under the allowed $\varepsilon$-perturbation; a single misclassified such sample would show that the 'certifiably robust continual learning' claim does not hold for the full test set. A complementary check is to retrain with a smaller regularization coefficient $\beta$ and watch the fraction of samples satisfying Theorem 3.1's condition fall while certified accuracy on task 1 collapses.

Watch

Extended reading notes

Core claim

The central discovery is that the robustness guarantee and the continual-learning mechanics can be made to reinforce each other. SHIELD's hypernetwork produces a fresh set of target-network weights $\theta_t = H(e_t; \Phi)$ for each task from a learned embedding. Because IBP training certifies each task model at the moment it is learned, the only remaining question is whether later hypernetwork updates destroy earlier certificates; the paper answers this with a sufficient condition (Theorem 3.1): if the maximum logit change induced by a hypernetwork update is at most half the certified margin of a previously robust sample, that sample remains certified. The $L_{\mathrm{out}}$ regularization term keeps hypernetwork outputs stable, and the supplementary material verifies post-hoc that the condition holds for 95–99% of previously certified samples across benchmarks. Interval MixUp supplies the accuracy half of the trade-off: interpolating $\ell_\infty$ boxes with the scaled radius $\varepsilon' = |2\lambda-1|\cdot\varepsilon$ yields tighter IBP bounds and smoother decision surfaces, so verified accuracy tracks clean accuracy closely.

Load-bearing premise

The load-bearing premise is that the hypernetwork regularization keeps generated weights stable enough that the Theorem 3.1 inequality — logit drift no larger than half the certified margin — holds for essentially every sample certified on earlier tasks; the paper checks this after training rather than enforcing it during training, and 1–5% of samples fail the check, so modestly larger weight drift would silently void earlier certificates.

Editorial extensions

If this is right

  • Continual learning can be made certifiably robust without replay buffers or stored gradient information: the hypernetwork alone carries prior-task knowledge, so memory cost stays bounded as tasks accumulate.
  • A certificate earned on an early task is not automatically voided by later learning; any sample whose certified margin exceeds the hypernetwork-induced logit drift keeps its formal guarantee through the whole task sequence.
  • Interval MixUp narrows the gap between verified and classical accuracy, meaning certified training no longer forces a large clean-accuracy penalty on hard benchmarks like Split CIFAR-100 and Split miniImageNet.
  • The same machinery extends to class-incremental learning, where task identity is inferred by predictive entropy, giving the first reported certified-robustness results in that setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 1–5% of samples that fail Theorem 3.1's condition are a natural attack surface: an adaptive adversary that identifies such a sample on an early task could find a perturbation that violates the certificate, so the practical guarantee is slightly weaker than the paper's headline wording suggests.
  • The post-hoc verification in the supplementary material could be replaced by a training-time penalty that enforces the margin condition sample-by-sample; this would convert certification preservation from a verified property into an enforced one, at some computational cost.
  • Nothing in the mechanism is specific to IBP: any certified-training scheme with worst-case logit bounds, such as tighter linear-relaxation bounds, could sit inside the same hypernetwork wrapper, since only the weight-drift bound depends on the hypernetwork.
  • The guarantee depends on the task embedding being correct; in the class-incremental deployment, an attacker who can fool the entropy-based task selector could redirect inputs to the wrong task model, a plausible attack vector not examined in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SHIELD, a hypernetwork-based continual learning method integrated with Interval Bound Propagation (IBP) to provide certified adversarial robustness across a sequence of tasks. The hypernetwork generates task-specific target weights from trainable embeddings, and an Interval MixUp loss interpolates ell-infinity balls to smooth decision boundaries. Experiments on Permuted/Rotated MNIST, Split CIFAR-100, and Split miniImageNet report state-of-the-art robustness under AutoAttack/PGD/FGSM relative to DGP and other baselines, plus a theoretical preservation condition (Theorem 3.1) and a post-hoc empirical check in the supplement.

Significance. If the certification claims were fully supported, the paper would be a useful contribution: it combines IBP with a hypernetwork CL architecture, shows strong empirical robustness improvements, and includes a public code release and extensive ablations. The Interval MixUp idea is simple and appears to improve verified accuracy in Figure 3. However, the paper's central 'certifiably robust' claim is weakened by the fact that its main table reports attack accuracy rather than verified accuracy, and by the gap between the theoretical sufficient condition and the training objective.

major comments (4)
  1. [Theorem 3.1 / SM O.1] The proof of Theorem 3.1 in SM O.1 does not establish the claimed guarantee. Equation (33) asserts that f_lx(x;theta+h) lies within [z_lx(x;theta)-Delta_max, z_lx(x;theta)+Delta_max], but z_lx(x;theta) is the IBP lower bound over the entire epsilon-ball around x, not the logit value at the center x. The term Delta_max(x;h), defined in Eq. (14) as ||f(x;theta+h)-f(x;theta)||_infinity, bounds only the change at the single point x; it does not control the change in the interval bounds over the perturbation set. Consequently, the derivation that the margin shrinks by at most 2 Delta_max is not justified. To make the theorem valid, Delta_max would need to be defined as a uniform bound over the epsilon-ball, e.g., sup_{x' in B_eps(x)} ||f(x';theta+h)-f(x';theta)||_infinity, and the proof would need to show that IBP bounds for theta+h are contained in the original bounds expanded by this quantity. As written, the theoretical guarantee is unsupported.
  2. [Table 1 / Section 5] The central claim that SHIELD provides 'certified' robustness is not directly evidenced by the main experimental table. Table 1 reports AutoAttack, PGD, FGSM, and original-sample accuracy, none of which is a certified metric. Verified accuracy, which is the quantity implied by IBP certification, appears only in Figure 3 for SHIELD and SHIELDIM, without comparison to baselines. To substantiate the 'first certifiably robust continual learning method' claim, the paper should report IBP-verified accuracy in Table 1 for SHIELD (and ideally for the baselines, using the same certification procedure), and should not rely on empirical attack accuracy as a proxy for certification.
  3. [Eq. (10) / SM O.2] The sufficient condition (Eq. (14)) is not enforced by the training objective. The regularization L_out in Eq. (10) is an average, over previous task embeddings, of squared hypernetwork output differences; it neither bounds the per-sample logit change Delta_max(x;h) nor controls updates induced by future tasks. The paper's own SM O.2 acknowledges that the condition is only checked post-hoc, with Table 20 showing that 95.63%-99.00% of samples satisfy it. This means 1%-5% of previously certified samples are not guaranteed to remain certifiably robust after subsequent tasks, so the abstract's 'ensuring robustness across tasks' and the conclusion's 'formal guarantees' overstate what is established. The method should be described as an empirically robust continual learning heuristic with a post-hoc margin-preservation check, unless the condition is actively enforced or a probabilistic guarantee is derived.
  4. [Conclusion / Table 17] The conclusion states that SHIELD is 'the first approach to demonstrate substantial certified robustness in the CIL setting,' but the CIL results in Table 17 (and the accompanying text in SM J) report only AutoAttack, PGD, FGSM, and original-sample accuracy. No verified/certified accuracy is reported for the CIL setting, so the claim of demonstrated certified robustness is unsupported. Either report verified accuracy for the CIL experiments or remove the word 'certified' from this claim.
minor comments (5)
  1. [Eq. (15) vs Algorithm 2] Equation (15) defines epsilon' as the MixUp-scaled radius |2lambda-1|*epsilon, but in Algorithm 2 (lines 15, 27) epsilon' is the scheduled base epsilon and epsilon_IM is the scaled radius; please rename one of the two to avoid ambiguity.
  2. [Section 4 / Figure 3] The sentence 'All reported results are based on the best-performing models selected using a single random seed' appears in the paragraph discussing Figure 3, but the supplementary tables report mean +/- std over multiple seeds; clarify which results are single-seed and which are averaged.
  3. [SM Table 15] The note in SM Table 15 states that the authors swapped the FGSM and original-sample columns of the [39] baselines because the original numbers seemed inconsistent. This alteration of baseline results should be stated in the main text or at least in the caption of Table 1, so readers can evaluate the comparison.
  4. [Section 2] In Section 2, the sentence 'Crucially, none of these methods explore hypernetworks' is immediately followed by discussion of hypernetwork-based CL methods [45,17,25,46]; clarify that the statement refers to the robustness-focused continual learning methods (AIR, DGP) and not to continual learning methods in general.
  5. [Abstract] The abstract contains 'enabling efficient over time' which appears to be missing words; it should likely read 'enabling efficiency over time.'

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: Theorem 3.1 is a conditional sufficient condition; the paper's 'ensuring robustness across tasks' overreach is a post-hoc verification gap, not a fitted-input circularity.

full rationale

SHIELD's derivation chain is not circular. The central robustness argument is Theorem 3.1, which states a conditional sufficient condition: if the maximum logit change Delta_max induced by a hypernetwork update is at most half the certified margin, then the margin remains positive. This is a definitional implication of Delta_max and the margin, but it is a valid mathematical statement rather than a fitted parameter or a prediction forced by construction; the proof in SM O.1 simply applies the definition of Delta_max. The regularization term L_out in Eq. (10) is not defined in terms of the certified margin, and no result in the paper is obtained by fitting a parameter and then re-reporting it as a prediction. The paper's own SM O.2 admits that Eq. (14) is not enforced during training and is only verified post-hoc, with Table 20 showing 95.63-99.00% of samples satisfying the condition; this is a support/overclaim gap for the headline 'ensuring robustness across tasks', but it is a correctness concern, not circularity. IBP is imported from external work [13, 35], MixUp from [51], and the baselines from [39]; the self-citations ([24], [25], [35]) appear in related-work or implementation background and are not load-bearing. No equation reduces to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical or mathematical entities are postulated. The method relies on standard IBP and hypernetwork machinery, plus several per-dataset hyperparameters tuned by grid search. The main unproven premise is the smoothness of the hypernetwork output, which the paper checks post-hoc rather than guarantees.

free parameters (5)
  • training perturbation epsilon = 0.01 (SM F) or 25/255 (SM E) for Permuted and Rotated MNIST; 0.005 or 2/255 for CIFAR-100; 2/255 for miniImageNet
    Grid-searched per dataset (SM G); inconsistent between SM E and SM F, affecting certified radius claims.
  • hypernetwork regularization weight beta = 0.001 (Permuted/Rotated), 0.01 (CIFAR-100, miniImageNet), 0.05 (TinyImageNet)
    Selected by validation; controls trade-off between plasticity and stability of generated weights.
  • IBP weight kappa schedule = annealed from 1 to 0.5 (Eq. 30), kappa=0.5 in main experiments
    Standard IBP annealing; ablation in SM H.7 shows kappa=0.5 best for CIFAR-100.
  • MixUp Beta parameter alpha = 0.1 (Permuted/Rotated), 0.3 (CIFAR-100), 0.2 (miniImageNet)
    Grid-searched; controls how often virtual samples are near endpoints.
  • hypernetwork architecture and embedding size = MLP hidden [100,50] or [200,50]; embedding sizes 24 to 512
    Adjusted per dataset, sometimes to match baseline parameter counts (SM E).
assumptions (4)
  • standard math IBP bounds computed with midpoint-radius arithmetic are sound for networks with monotone non-decreasing activations.
    Invoked in Section 3.1 Eqs. (6)-(7); standard result from Gowal et al. [13].
  • domain assumption The hypernetwork output is a smooth function of Phi, so a bound on ||Delta Phi|| via L_out controls the classifier logit change Delta_max.
    Assumed in SM O.2 to justify the training proxy for Theorem 3.1; not proven, only verified empirically on the trained model.
  • domain assumption Task embeddings e_t are sufficient to generate distinct, well-performing task models without replay.
    Core hypernetwork CL premise inherited from von Oswald et al. [45].
  • standard math The worst-case logit selection in Eq. (8) gives a valid certified loss for the whole input box.
    Standard IBP certified training loss, per [13].

how reviews work

0 comments
Cite this review

Pith. "Pith review of SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense." pith.science (2026). https://pith.science/paper/QIDPBJIY

@misc{pith2026250608255,
  author       = {Pith},
  title        = {Pith review of: SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QIDPBJIY}},
  note         = {Machine review of arXiv:2506.08255}
}
abstract

Continual learning under adversarial conditions remains an open problem, as existing methods often compromise either robustness, scalability, or both. We propose a novel framework that integrates Interval Bound Propagation (IBP) with a hypernetwork-based architecture to enable certifiably robust continual learning across sequential tasks. Our method, SHIELD, generates task-specific model parameters via a shared hypernetwork conditioned solely on compact task embeddings, eliminating the need for replay buffers or full model copies and enabling efficient over time. To further enhance robustness, we introduce Interval MixUp, a novel training strategy that blends virtual examples represented as $\ell_{\infty}$ balls centered around MixUp points. Leveraging interval arithmetic, this technique guarantees certified robustness while mitigating the wrapping effect, resulting in smoother decision boundaries. We evaluate SHIELD under strong white-box adversarial attacks, including PGD and AutoAttack, across multiple benchmarks. It consistently outperforms existing robust continual learning methods, achieving state-of-the-art average accuracy while maintaining both scalability and certification. These results represent a significant step toward practical and theoretically grounded continual learning in adversarial settings.

Figures

Figures reproduced from arXiv: 2506.08255 by the authors.

Figure 1
Figure 1. SHIELD uses a hypernetwork to map task-specific embeddings [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (left side) Without Interval MixUp: Training leads to [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison of verified accuracy and classical accuracy [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of different epsilon decay rates used in In [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Comparison of results for different epsilon decay rates across (a) Permuted MNIST, (b) Split CIFAR-100, and (c) Split miniIm [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: AA of SHIELD and HNET under FGSM attacks with increasing perturbation sizes, evaluated on models trained on Permuted [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: AA of SHIELD and HNET under PGD attacks with increasing number of gradient steps, evaluated on models trained on Permuted [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Comparison of AA curves for varying β values, which control hypernetwork regularization strength in SHIELD specifically on TinyImageNet. For FGSM, we set the at￾tack strength to \varepsilon _{\text {attack}} = \frac {4}{255} . For PGD and AutoAttack, we use \varepsilon…
Figure 9
Figure 9. Figure 9: IBP-verified AA as a function of the perturbation radius [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Each row corresponds to a single Interval MixUp-generated sample (hypercube center), and each column shows perturbations [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Empirical verification of Eq. (14) from Theorem 3.1. The dashed red line marks the threshold 1. Left: Permuted MNIST. Right: Split CIFAR-100. In conclusion, our method maintains high certified ro￾bustness across tasks. The empirical analysis shows that knowledge of ce…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 39 canonical work pages

  1. [1]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. InProceedings of the European conference on computer vision (ECCV), pages 139–154, 2018. 1

    work page 2018

  2. [2]

    Obfus- cated gradients give a false sense of security: Circumventing defenses to adversarial examples

    Anish Athalye, Nicholas Carlini, and David Wagner. Obfus- cated gradients give a false sense of security: Circumventing defenses to adversarial examples. InInternational confer- ence on machine learning, pages 274–283. PMLR, 2018. 1

    work page 2018

  3. [3]

    Training Ensembles to Detect Adversarial Examples

    Alexander Bagnall, Razvan Bunescu, and Gordon Stewart. Training ensembles to detect adversarial examples.arXiv preprint arXiv:1712.04006, 2017. 1

    work page Pith review arXiv 2017

  4. [4]

    Decision-based adversarial attacks: Reliable attacks against black-box machine learning models.arXiv preprint arXiv:1712.04248, 2017

    Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models.arXiv preprint arXiv:1712.04248, 2017. 1

    arXiv 2017

  5. [5]

    Dokania, and Philip H

    Arslan Chaudhry, Naeemullah Khan, Puneet K. Dokania, and Philip H. S. Torr. Continual learning in low-rank or- thogonal subspaces, 2020. 2

    work page 2020

  6. [6]

    Zoo: Zeroth order optimization based black- box attacks to deep neural networks without training substi- tute models

    Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black- box attacks to deep neural networks without training substi- tute models. InProceedings of the 10th ACM workshop on artificial intelligence and security, pages 15–26, 2017. 1

    work page 2017

  7. [7]

    Reliable evalua- tion of adversarial robustness with an ensemble of diverse parameter-free attacks, 2020

    Francesco Croce and Matthias Hein. Reliable evalua- tion of adversarial robustness with an ensemble of diverse parameter-free attacks, 2020. 5

    work page 2020

  8. [8]

    A continual learning survey: Defying for- getting in classification tasks.IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying for- getting in classification tasks.IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021. 1

    work page 2021

Show all 59 references
  1. [9]

    Boosting adversarial at- tacks with momentum

    Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial at- tacks with momentum. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 9185–9193, 2018. 1

    2018

  2. [10]

    Evading defenses to transferable adversarial examples by translation-invariant attacks

    Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4312–4321, 2019. 1

    2019

  3. [11]

    Adversarially robust distillation

    Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Gold- stein. Adversarially robust distillation. InProceedings of the AAAI conference on artificial intelligence, pages 3996–4003,

  4. [12]

    Explaining and harnessing adversarial examples.arXiv preprint arXiv:1412.6572, 2014

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples.arXiv preprint arXiv:1412.6572, 2014. 1, 2

    2014 arXiv

  5. [13]

    On the effec- tiveness of interval bound propagation for training verifiably robust models.arXiv preprint arXiv:1810.12715, 2018

    Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arand- jelovic, Timothy Mann, and Pushmeet Kohli. On the effec- tiveness of interval bound propagation for training verifiably robust models.arXiv preprint arXiv:1810.12715, ...

    arXiv

  6. [14]

    Hypernetworks

    David Ha, Andrew Dai, and Quoc V Le. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016. 3, 1

    2016 arXiv

  7. [15]

    Memory efficient experience replay for streaming learning

    Tyler L Hayes, Nathan D Cahill, and Christopher Kanan. Memory efficient experience replay for streaming learning. In2019 International Conference on Robotics and Automa- tion (ICRA), pages 9769–9776. IEEE, 2019. 1

    2019

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1

    2016

  9. [17]

    Poste- rior meta-replay for continual learning.Advances in Neural Information Processing Systems, 34:14135–14149, 2021

    Christian Henning, Maria Cervera, Francesco D’Angelo, Jo- hannes V on Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F Grewe, and João Sacramento. Poste- rior meta-replay for continual learning.Advances in Neural Information Processing Systems, 34:14135–1414...

    2021

  10. [18]

    Re-evaluating continual learning scenarios: A categorization and case for strong baselines.arXiv preprint arXiv:1810.12488, 2018

    Yen-Chang Hsu, Yen-Cheng Liu, Anita Ramasamy, and Zsolt Kira. Re-evaluating continual learning scenarios: A categorization and case for strong baselines.arXiv preprint arXiv:1810.12488, 2018. 1

    2018 arXiv

  11. [19]

    Adversarial machine learning

    Ling Huang, Anthony D Joseph, Blaine Nelson, Benjamin IP Rubinstein, and J Doug Tygar. Adversarial machine learning. InProceedings of the 4th ACM workshop on Security and artificial intelligence, pages 43–58, 2011. 1

    2011

  12. [20]

    Las-at: adversarial training with learn- able attack strategy

    Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: adversarial training with learn- able attack strategy. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 13398–13408, 2022. 1

    2022

  13. [21]

    Ape-gan: Adversarial perturbation elimination with gan

    Guoqing Jin, Shiwei Shen, Dongming Zhang, Feng Dai, and Yongdong Zhang. Ape-gan: Adversarial perturbation elimination with gan. InICASSP 2019-2019 IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 3842–3846. IEEE, 2019. 1

    2019

  14. [22]

    Susceptibility of continual learning against adversarial attacks.arXiv preprint arXiv:2207.05225, 2022

    Hikmat Khan, Pir Masoom Shah, Syed Farhan Alam Zaidi, Qasim Zia, et al. Susceptibility of continual learning against adversarial attacks.arXiv preprint arXiv:2207.05225, 2022. 2, 1

    2022 arXiv

  15. [23]

    Overcoming catastrophic forgetting in neu- ral networks.Proceedings of the national academy of sci- ences, 114(13):3521–3526, 2017

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks.Proceedings of the national academy of sci...

    2017

  16. [24]

    Hy- perinterval: Hypernetwork approach to training weight interval regions in continual learning.arXiv preprint arXiv:2405.15444, 2024

    Patryk Krukowski, Anna Bielawska, Kamil Ksia ˙zek, Paweł Wawrzy´nski, Paweł Batorski, and Przemysław Spurek. Hy- perinterval: Hypernetwork approach to training weight interval regions in continual learning.arXiv preprint arXiv:2405.15444, 2024. 1

    2024 arXiv

  17. [25]

    Hypermask: Adap- tive hypernetwork-based masks for continual learning.arXiv preprint arXiv:2310.00113, 2023

    Kamil Ksia ˙zek and Przemysław Spurek. Hypermask: Adap- tive hypernetwork-based masks for continual learning.arXiv preprint arXiv:2310.00113, 2023. 3, 1

    2023 arXiv

  18. [26]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelli- gence, 40(12):2935–2947, 2017. 1

    2017

  19. [27]

    Continual learning with recursive gradient optimization, 2022

    Hao Liu and Huaping Liu. Continual learning with recursive gradient optimization, 2022. 2

    2022

  20. [28]

    Core50: a new dataset and benchmark for continuous object recognition

    Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. In Conference on robot learning, pages 17–26. PMLR, 2017. 1

    2017

  21. [29]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. InProceedings of the 31st International Conference on Neural Information Processing Systems, page 6470–6479, Red Hook, NY , USA,

  22. [30]

    Towards deep learn- ing models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learn- ing models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017. 1

    2017 arXiv

  23. [31]

    Packnet: Adding mul- tiple tasks to a single network by iterative pruning

    Arun Mallya and Svetlana Lazebnik. Packnet: Adding mul- tiple tasks to a single network by iterative pruning. InPro- ceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765–7773, 2018. 1

    2018

  24. [32]

    Target layer regularization for continual learning using cramer-wold distance.Information Sciences, 609:1369–1380, 2022

    Marcin Mazur, Łukasz Pustelnik, Szymon Knop, Patryk Pa- gacz, and Przemysław Spurek. Target layer regularization for continual learning using cramer-wold distance.Information Sciences, 609:1369–1380, 2022. 1

    2022

  25. [33]

    Catastrophic inter- ference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. InPsychology of learning and motivation, pages 109–165. Elsevier, 1989. 1

    1989

  26. [34]

    Dif- ferentiable abstract interpretation for provably robust neural networks

    Matthew Mirman, Timon Gehr, and Martin Vechev. Dif- ferentiable abstract interpretation for provably robust neural networks. InInternational Conference on Machine Learn- ing, pages 3578–3586. PMLR, 2018. 1

    2018

  27. [35]

    Fast and stable interval bounds propaga- tion for training verifiably robust models.arXiv preprint arXiv:1906.00628, 2019

    Paweł Morawiecki, Przemysław Spurek, Marek ´Smieja, and Jacek Tabor. Fast and stable interval bounds propaga- tion for training verifiably robust models.arXiv preprint arXiv:1906.00628, 2019. 3

    1906 arXiv

  28. [36]

    Deeply supervised discriminative learning for adversarial defense.IEEE trans- actions on pattern analysis and machine intelligence, 43(9): 3154–3166, 2020

    Aamir Mustafa, Salman H Khan, Munawar Hayat, Roland Goecke, Jianbing Shen, and Ling Shao. Deeply supervised discriminative learning for adversarial defense.IEEE trans- actions on pattern analysis and machine intelligence, 43(9): 3154–3166, 2020. 1

    2020

  29. [37]

    Springer, 1993

    Arnold Neumaier.The wrapping effect, ellipsoid arithmetic, stability and confidence regions. Springer, 1993. 6

    1993

  30. [38]

    Bag of tricks for adversarial training.arXiv preprint arXiv:2010.00467, 2020

    Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training.arXiv preprint arXiv:2010.00467, 2020. 1

    2010 arXiv

  31. [39]

    Main- taining adversarial robustness in continuous learning.arXiv preprint arXiv:2402.11196, 2024

    Xiaolei Ru, Xiaowei Cao, Zijia Liu, Jack Murdoch Moore, Xin-Ya Zhang, Xia Zhu, Wenjia Wei, and Gang Yan. Main- taining adversarial robustness in continuous learning.arXiv preprint arXiv:2402.11196, 2024. 1, 2, 7, 8, 3, 5, 6, 11, 12

    2024 arXiv

  32. [40]

    Progressive neural networks

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Raz- van Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. 1

    2016 arXiv

  33. [41]

    Gradient projection memory for continual learning.arXiv preprint arXiv:2103.09762, 2021

    Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning.arXiv preprint arXiv:2103.09762, 2021. 2, 1

    2021 arXiv

  34. [42]

    Online adversarial purification based on self-supervision.arXiv preprint arXiv:2101.09387, 2021

    Changhao Shi, Chester Holtz, and Gal Mishne. Online adversarial purification based on self-supervision.arXiv preprint arXiv:2101.09387, 2021. 1

    2021 arXiv

  35. [43]

    One pixel attack for fooling deep neural networks.IEEE Transactions on Evolutionary Computation, 23(5):828–841,

    Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks.IEEE Transactions on Evolutionary Computation, 23(5):828–841,

  36. [44]

    Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013. 1

    2013 arXiv

  37. [45]

    Continual learning with hy- pernetworks

    Johannes von Oswald, Christian Henning, Benjamin F Grewe, and João Sacramento. Continual learning with hy- pernetworks. InInternational Conference on Learning Rep- resentations, 2019. 4, 1, 8

    2019

  38. [46]

    A comprehensive survey of continual learning: Theory, method and application.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 3, 1

    2024

  39. [47]

    Supermasks in superposition.Advances in Neural Information Processing Systems, 33:15173–15184,

    Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosin- ski, and Ali Farhadi. Supermasks in superposition.Advances in Neural Information Processing Systems, 33:15173–15184,

  40. [48]

    Mitigating adversarial effects through random- ization.arXiv preprint arXiv:1711.01991, 2017

    Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through random- ization.arXiv preprint arXiv:1711.01991, 2017. 1

    2017 arXiv

  41. [49]

    Adaptive test-time defense with the manifold hypothesis.arXiv preprint arXiv:2210.14404, 3, 2022

    Zhaoyuan Yang, Zhiwei Xu, Jing Zhang, Richard Hartley, and Peter Tu. Adaptive test-time defense with the manifold hypothesis.arXiv preprint arXiv:2210.14404, 3, 2022. 1

    2022 arXiv

  42. [50]

    Contin- ual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Contin- ual learning through synaptic intelligence. InInternational conference on machine learning, pages 3987–3995. PMLR,

  43. [51]

    Dauphin, and David Lopez-Paz

    Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion, 2018. 5

    2018

  44. [52]

    Efficient neural network robustness certifi- cation with general activation functions, 2018

    Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certifi- cation with general activation functions, 2018. 6

    2018

  45. [53]

    Defense without for- getting: Continual adversarial defense with anisotropic & isotropic pseudo replay

    Yuhang Zhou and Zhongyun Hua. Defense without for- getting: Continual adversarial defense with anisotropic & isotropic pseudo replay. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24263–24272, 2024. 1, 2, 7, 12, 13

    2024

  46. [54]

    Reliable adversarial distillation with unreli- able teachers.arXiv preprint arXiv:2106.04928, 2021

    Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreli- able teachers.arXiv preprint arXiv:2106.04928, 2021. 1 SHIELD: Secure Hypernetworks for Incremental Expansi...

    2021 arXiv

  47. [56]

    For both Split CIFAR-100 and Split mini- ImageNet, each class appears in only one group, ensuring Table 2

    and dividing it into 20 distinct groups, each containing 5 unique classes. For both Split CIFAR-100 and Split mini- ImageNet, each class appears in only one group, ensuring Table 2. Comparison of learnable parameters (in millions) be- tween our method and the baseline from [39...

  48. [58]

    The AutoAttack configuration is the same as used in the main experiments of the paper

    For PGD and AutoAttack, we use εattack = 2 255, with a PGD step size of δ= 4 255 and 100 iterations. The AutoAttack configuration is the same as used in the main experiments of the paper. We use ResNet-18 as the target network. The hypernet- work is implemented as a multilayer...

  49. [100]

    Importantly, SHIELD is attack-agnostic - it does not rely on generating adversarial examples during training, in contrast to AIR

    SHIELD achieves the highest accuracy on Task 2 and Task 3, outperforming all baselines under both FGSM and PGD attacks in the later tasks. Importantly, SHIELD is attack-agnostic - it does not rely on generating adversarial examples during training, in contrast to AIR. Instead,...

  50. [255]

    The learning rate scheduler matched that of the Split CIFAR-100 setup

    No data augmentation was used. The learning rate scheduler matched that of the Split CIFAR-100 setup. When Interval MixUp was used, mix- Table 5. Comparison of AA after completing all tasks on the Split CIFAR-100 dataset. AA results for SHIELD are averaged over 2 seeds for Aut...

  51. [2017]

    Curran Associates Inc. 2, 1

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.