Pith. sign in

REVIEW 4 major objections 5 minor 69 references

RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Frozen pretrained weights adapt to new tasks with fewer than 50 tunable scalars per task, by generating each layer's weights from shared templates; Neural Mimicry converts any pretrained model into this form in minutes.

desk verdict Novel reconstruction pipeline, but the headline 'fewer than 50 task parameters' omits the classifier head and is unsupported as a total budget. read the letter →

arxiv 2411.16870 v2 pith:HSUOTX25 submitted 2024-11-25 cs.CV cs.LG

Nazia Tasnim , Bryan A. Plummer This is my paper · ORCID
classification cs.CVcs.LG
keywords incrementallearningtask-incrementalparameter-efficientfine-tuningweightdecompositionsharedtemplatebankcoefficienttuningNeuralMimicrycatastrophicforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that task-incremental learning can be made much more parameter-efficient by generating each module's weights on the fly from a small bank of frozen shared templates and per-module scalar coefficients. It reports that fewer than 50 trainable parameters per task are enough to adapt a pretrained backbone, several orders of magnitude less than LoRA-based adapters, and that this reparameterized network matches or exceeds the frozen baseline, lifting ViT-Small by about 3.5% with only 0.0002% task parameters. The key enabling step is Neural Mimicry, a reconstruction pipeline that converts an existing pretrained model into the template-coefficient form in minutes, so the method works with any architecture without training from scratch. If the claim holds, continual learning on edge devices becomes practical because per-task storage and communication costs shrink to a few scalar coefficients.

What carries the argument

RECAST replaces each module weight with a weighted sum over a shared bank of templates: $W_{l,m} = \frac{1}{K}\sum_{k=1}^{K} \sum_{i=1}^{n} T_{g,i} \odot C^{k}_{l,m,i}$, where the templates $T_{g,i}$ are shared within a group of layers and the coefficients $C^{k}_{l,m,i}$ are scalars. Because only these scalars are trained per task, the whole network can be reparameterized with fewer than 50 parameters. The second load-bearing mechanism is Neural Mimicry, which converts an existing pretrained network into this form by gradient descent on the discrepancy between original and reconstructed weights, using Smooth L1 loss with additive coefficient noise; this is what lets RECAST start from any pretrained backbone instead of training from scratch.

What would settle it

Train two sequential tasks whose optimal weight changes are deliberately orthogonal to the span of the shared templates, for example tasks that require swapping or zeroing different input channels, and measure whether coefficient-only tuning still improves accuracy; if the second task cannot be fit above the frozen-backbone level while a full fine-tune improves, the claim that template spans are sufficient is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a frozen pretrained network can be reparameterized as a soft parameter-sharing system in which every layer weight is a linear combination of shared templates scaled by module-specific coefficients, and that task-incremental learning can then be done by tuning only those coefficients. On ViT-Small this yields 85.0% average top-1 accuracy against a frozen-baseline score of 82.4% with 0.0002% task parameters, and combining RECAST with AdaptFormer reaches 90.1%, the best result in the comparison. On ResNet-34 the same recipe improves the baseline by about 2% with 0.0003% task parameters and, when layered under PiggyBack and CLR, adds roughly 1.5% to each of those methods. Neural Mimicry is what makes this practical: it reconstructs existing pretrained weights into templates and coefficients in under five minutes while preserving near-identical ImageNet accuracy, so the framework generalizes across CNN and Transformer backbones, model scales, and pretraining schemes.

Load-bearing premise

The approach assumes that the small set of shared template weights, kept frozen across all tasks, is rich enough that rescaling its few components can produce whatever new weights each task needs; the paper offers no theoretical bound on when that span is sufficient, only experiments on six image datasets.

Editorial extensions

If this is right

  • Per-task storage overhead becomes negligible: only the single frozen backbone and a handful of scalar coefficients must be kept, so communication and memory costs on edge devices drop by orders of magnitude.
  • In the ultra-low parameter regime of 24 to 96 parameters per task, where LoRA-style methods stop improving, coefficient tuning still gives consistent accuracy gains, making the method usable under extreme resource constraints.
  • RECAST can be layered under existing adapter and reparameterization methods such as PiggyBack, CLR, AdaptFormer, MeLo, RoSA, and DoRA, and each combination improves the standalone method by about one to 1.5 percentage points.
  • Neural Mimicry preserves pretrained accuracy, reaching 98 to 100% weight and feature similarity, and completes in minutes, so existing backbones can be converted without retraining from scratch.
  • The method transfers across architectures and scales, including ViT-Small, ViT-Base, ViT-Large, ResNet-34, and a DINO-pretrained ViT, indicating that the template-coefficient decomposition itself, not a particular architecture, drives the gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit is that the real capacity knob is the template space, not the coefficient count: with the reported setting of two templates per bank, each module weight lives in a two-dimensional subspace, so scaling the number of templates per group should be the first lever to try when tasks become more diverse.
  • Because RECAST produces the actual layer weights rather than an additive adapter, the same decomposition could support weight-level compression or quantization of the frozen backbone, though the paper does not test those combinations.
  • The task-incremental evaluation assumes the task identity is known at inference time; extending to class-incremental or task-free settings would require a routing mechanism on top of the coefficients, which is not part of the paper's claims.
  • The paper's own analysis suggests that template diversity and coefficient versatility drive adaptability; an explicit diversity-promoting regularizer in Neural Mimicry, such as penalizing template similarity, is a direct testable extension beyond the Smooth L1 choice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RECAST, a task-incremental learning method that reparameterizes a pretrained network by expressing each target layer weight as a linear combination of shared template matrices and per-module scalar coefficients. A reconstruction procedure called Neural Mimicry fits the templates and coefficients to an existing pretrained model, so the framework can be initialized without training from scratch. At task-incremental training time, the templates are frozen and only the coefficients (optionally together with existing adapters such as LoRA, AdaptFormer, MeLo, DoRA, RoSA, Piggyback, or CLR) are tuned. The paper reports accuracy on six datasets with ResNet-34 and ViT-Small backbones, claiming fewer than 50 task-specific trainable parameters and showing that RECAST combined with adapters improves over the adapters alone. The paper also includes ablations on template sharing, coefficient sets, and reconstruction objectives.

Significance. If the parameter-count issue is resolved, the paper makes a useful empirical contribution. Neural Mimicry is a fast, architecture-agnostic way to reparameterize pretrained weights, and the reported reconstruction fidelity is strong: Table 6 shows 98-100% reconstruction similarity and ImageNet accuracies within 0.3 points of the original models. The combination of RECAST with existing adapters yields the best ViT-Small result (90.1% in Table 2), and the low coefficient count is a genuine practical advantage for communication-constrained settings. The comparison against 16 baselines is broad, and the appendix provides per-task results, dataset details, and reconstruction analyses that support reproducibility. However, the headline efficiency claim is currently overstated because the per-task classification heads are not included in the parameter budget, and the performance claims in the abstract are not consistent with the tables. The central contribution is sound in principle, but the reported metrics need correction before the paper can be accepted.

major comments (4)
  1. [Section 3.3 and Tables 1-2] The reported 'Task Params' count only the coefficient scalars sum_l M_l * n * K, and the paper never specifies how the final class prediction is produced in the TIL protocol. Because the six datasets have disjoint label spaces, a per-task output head is required. For ViT-Small (d=384, 534 total classes), even a linear head adds 205,590 trainable weights and biases across the six tasks (77,000 for the Birds task alone); for ResNet-34 (d=512) the corresponding total is 273,942. If these heads are trained, the claims 'fewer than 50 task-specific trainable parameters' (Abstract) and '0.0002% task parameters' (Section 4.1) undercount the deployed system by roughly four orders of magnitude. If they are not trained, the classification mechanism is unspecified. Please either include the per-task heads in the parameter budget and comparisons, or explicitly state and justify an alternative classification mechanism.
  2. [Abstract and Tables 1-2] The claim that RECAST 'outperforms the state-of-the-art by up to 3%' is not supported by the reported results. In Table 2, standalone RECAST obtains 85.0, while AdaptFormer, MeLo, DoRA, InfLoRA, and RoSA obtain 89.0, 88.7, 89.3, 88.5, and 88.6, respectively. The only configuration that is best overall is RECAST+AdaptFormer at 90.1, which is 0.8 points above the best standalone adapter. The header abstract and the full-text abstract also disagree with each other ('up to 3%' vs 'up to ~1.5%'). Please revise the abstract and introduction so the stated gains match the tables and identify the exact comparison (standalone vs combined vs frozen-backbone baseline) that supports each claim.
  3. [Section 4.1] The sentence 'RECAST improves the baseline ViT-Small by ~3.5%, adding only 0.0002% task parameters' is arithmetically inconsistent with Table 2, which reports 85.0 vs 82.4 (+2.6 points). Similarly, Section 4.1 says RECAST improves ResNet-34 by ~2%, while Table 1 shows 66.3 vs 65.2 (+1.1 points). Please correct the text or the tables and ensure all percentage claims are computed consistently.
  4. [Section 4.1 and Figure 4] The configuration used for the main ViT-Small results ('group size of 6, 2 templates per bank, and 2 coefficient sets') does not state which modules are reparameterized. Figures 7 and 8 distinguish 'RECAST on MLP' from 'RECAST on Attention', and Table 6 reports separate reconstructions for MLP, Attention, and both. The coefficient count depends on this choice: with 12 blocks, it is 48 for attention-only (one QKV module per block), 96 for MLP-only (two FC modules per block), and 144 for both. Indeed, Figure 4 places RECAST in the 24-96 parameter range, which already conflicts with the abstract's 'fewer than 50'. Please specify the module set used for each main-table result and report the resulting coefficient count so the headline parameter claim is verifiable.
minor comments (5)
  1. [Abstract and Section 1] There are several typos: 'REACAST' in Section 1, 'This is can be problematic' in the abstract, and 'perfermos' in the appendix. Please proofread.
  2. [Eq. (3) and Algorithm 2] The objective in Eq. (3) includes a Gaussian noise perturbation on coefficients, but Algorithm 2's update does not show how noise is sampled or added. Please clarify whether the noise is used during reconstruction and, if so, how it enters the gradient step.
  3. [Section 3.3] The phrase 'a combinatorially large number of weight matrices (nK) per module' should presumably read 'n^K'; please correct the notation.
  4. [Table 1] The 'ResNet-34' row is labeled with 0.0 M task parameters, yet Section 4.1 describes finetuned classifier layers as a baseline. If this row is that baseline, the classifier head parameters should be counted; otherwise the row label should be changed to avoid confusion.
  5. [Reproducibility] The footnote 'Code: Repository' is a placeholder without a URL. The reproducibility section states that code is in the supplementary material, but a working link or repository identifier would make the implementation easier to verify.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: RECAST's TIL accuracy gains come from held-out task evaluation; Neural Mimicry is a weight-fitting tool, not a source of predictions; the Plummer et al. self-citation is contextual, not load-bearing. The silent treatment of per-task classification heads is an accounting/correctness concern, not a circular step.

full rationale

The paper's central claim is that tuning only per-module coefficients (Eqs. 1-2) with frozen shared templates yields strong task-incremental accuracy at very low parameter counts. That claim is tested empirically on held-out task test sets reported in Tables 1, 2, and 4, after per-task training on task data. The reported accuracies are therefore not determined by the reconstruction objective. Neural Mimicry (Eq. 3) is explicitly a fitting procedure: it minimizes the discrepancy between the original pretrained weights W_l,m and the RECAST-generated weights W*_l,m. The subsequent '98-100% reconstruction similarity' and matching ImageNet accuracy (Table 6) thus confirm that the optimization converged; they are sanity checks, not independent predictions, and the paper does not use them as evidence for the TIL improvements. The only same-author citation (Plummer et al., 2022) appears in related work as prior template-mixing context and is not used to justify RECAST's design or to exclude alternatives, so it is not load-bearing. One non-circular concern remains: the paper never states how the final per-task classification head is produced or counted, so the 'fewer than 50 task-specific trainable parameters' and '0.0002% task parameters' figures may omit the head, which would be a large per-task cost. That is a correctness/accounting issue, not an instance of circular reasoning, because the efficiency number is a reported metric rather than a derivation from the accuracy results.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities or forces. The templates are a reparameterization of existing weights, not a new postulated object. The main free parameters are the architectural choices G, n, and K, which directly control the number of task-specific coefficients; these are chosen by hand and calibrated to the baseline parameter count, not fitted to downstream performance. The reconstruction loss and noise scale are additional choices that affect the quality of the decomposition.

free parameters (5)
  • Number of template groups G = 6 (ViT-Small), 12 (ViT-Large)
    Chosen by hand; controls parameter sharing and reconstruction capacity. See Section 3.3 and Figure 7.
  • Number of templates per bank n = 2
    Chosen by hand; directly determines the coefficient count per module and is central to the claim of under 50 trainable parameters. See Section 3.3.
  • Number of coefficient sets K = 2
    Chosen by hand; multiplies the number of coefficients per module. See Section 3.3 and Figure 8.
  • Reconstruction noise scale sigma = not specified
    Noise added to coefficients during Neural Mimicry (Eq. 3). The value is not reported, making exact replication difficult.
  • Weight discrepancy loss = Smooth L1
    Selected via ablation (Figure 5). Affects reconstruction fidelity, template diversity, and downstream accuracy.
assumptions (4)
  • domain assumption Pretrained weights are accessible and of sufficient quality to serve as templates.
    Neural Mimicry requires a pretrained model M (Section 3.2); all experiments use ImageNet-pretrained backbones from timm/PyTorch.
  • domain assumption Task identity is known at test time, and tasks arrive sequentially with disjoint output spaces.
    This is the task-incremental learning setting defined in Section 3; the method selects per-task coefficients using the task ID.
  • domain assumption The linear combination of templates and coefficients can represent useful task-specific weight matrices.
    The entire method relies on Eqs. 1-2; no capacity bound is given, and this is only validated empirically on six datasets.
  • standard math Gradient-based optimization finds acceptable templates and coefficients during reconstruction.
    Algorithm 2 uses gradient descent; no convergence guarantee is provided, and the loss landscape is non-convex.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks." pith.science (2026). https://pith.science/paper/HSUOTX25

@misc{pith2026241116870,
  author       = {Pith},
  title        = {Pith review of: RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSUOTX25}},
  note         = {Machine review of arXiv:2411.16870}
}
read the original abstract

Incremental learning aims to adapt to new sets of categories over time with minimal computational overhead. Prior work often addresses this task by training efficient task-specific adaptors that modify frozen layer weights or features to capture relevant information without affecting predictions on previously learned categories. While these adaptors are generally more efficient than finetuning the entire network, they still require tens to hundreds of thousands of task-specific trainable parameters even for relatively small networks, making it challenging to operate on resource-constrained environments with high communication costs like edge devices or mobile phones. Thus, we propose Reparameterized, Compact weight Adaptation for Sequential Tasks (RECAST), a novel method that dramatically reduces task-specific trainable parameters to fewer than 50 - several orders of magnitude less than competing methods like LoRA. RECAST accomplishes this efficiency by learning to decompose layer weights into a soft parameter-sharing framework consisting of shared weight templates and very few module-specific scaling factors or coefficients. This soft parameter-sharing framework allows for effective task-wise reparameterization by tuning only these coefficients while keeping templates frozen.A key innovation of RECAST is the novel weight reconstruction pipeline called Neural Mimicry, which eliminates the need for pretraining from scratch. This allows for high-fidelity emulation of existing pretrained weights within our framework and provides quick adaptability to any model scale and architecture. Extensive experiments across six datasets demonstrate RECAST outperforms the state-of-the-art by up to 3% across various scales, architectures, and parameter spaces Moreover, we show that RECAST's architecture-agnostic nature allows for seamless integration with existing methods, further boosting performance.

Figures

Figures reproduced from arXiv: 2411.16870 by the authors.

Figure 1
Figure 1. (a) Existing IL methods, i.e. Rehearsal, Regularization, Reconfiguration - exhibit various limitations in terms of model complexity, memory requirements, and training overheads. In com￾parison our proposed method, (b) RECAST can be uses as a frozen backbone, allowing efficient reparameterization of any target module with a negligible number of parameter updates (order of 10−6 ) and can accommodate any number of disj… view at source ↗
Figure 2
Figure 2. RECAST decomposes module weights into templates and coefficients, which are learned [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Comparing averaged best Top-1 accuracy across [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Plot showing average classification accuracy of models reconstructed in different ways. VAE-reconstructed models performed slightly worse than the rest, with Smooth L1 loss providing the best performance SmoothL1 MSE SmoothL1 + Noise VAE + MSE VAE + SmoothL1 0 20 40 60…
Figure 8
Figure 8. Figure 8: Both experiments use RECAST on MLP and Attention layers of a small ViT model, with [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 7
Figure 7. Figure 7: Increasing template numbers (lay￾ers in a group) enhances coefficient count and template sharing across the architecture. Low sharing uses G=6, Balanced G=3, and Extreme G=1 (one template bank shared by all network layers). The last setting delivers significant perform…
Figure 9
Figure 9. Figure 9: Analyzing the strategies of changing reconstruction schemes by quantifying their respec [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Intra-group Coefficient Similarity across the groups of similar configuration models, that [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 33 canonical work pages

  1. [1]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part III, pp.\ 144–161, Berlin, Heidelberg, 2018. Springer-Verlag. ISBN 978-3-030-01218-2. doi:10.10...

    doi:10.1007/978-3-030-01219-9_9 2018

  2. [2]

    Gradient based sample selection for online continual learning

    Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_fi...

    work page 2019

  3. [3]

    Nern - learning neural representations for neural networks

    Maor Ashkenazi, Zohar Rimon, Ron Vainshtein, Shir Levi, Elad Richardson, Pinchas Mintz, and Eran Treister. Nern - learning neural representations for neural networks. In -, January 2023. Publisher Copyright: 2023 11th International Conference on Learning Representations, ICLR 2023. All rights reserved.; 11th International Conference on Learning Representa...

    work page 2023

  4. [4]

    Dark experience for general continual learning: a strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546

    work page 2020

  5. [5]

    Online learned continual compression with adaptive quantization modules

    Lucas Caccia, Eugene Belilovsky, Massimo Caccia, and Joelle Pineau. Online learned continual compression with adaptive quantization modules. Proceedings of the 37th International Conference on Machine Learning, 2020

    work page 2020

  6. [6]

    New insights on reducing abrupt representation change in online continual learning

    Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In ICLR, 2022

    work page 2022

  7. [7]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv\'e J\'egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021

    2021

  8. [8]

    Avalanche: A pytorch library for deep continual learning

    Antonio Carta, Lorenzo Pellegrini, Andrea Cossu, Hamed Hemati, and Vincenzo Lomonaco. Avalanche: A pytorch library for deep continual learning. Journal of Machine Learning Research, 24 0 (363): 0 1--6, 2023. URL http://jmlr.org/papers/v24/23-0130.html

    work page 2023

Show all 69 references
  1. [9]

    Efficient lifelong learning with a-gem

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. In ICLR, 2019

    2019

  2. [10]

    Using hindsight to anchor past knowledge in continual learning

    Arslan Chaudhry, Albert Gordo, Puneet K Dokania, Philip HS Torr, and David Lopez-Paz. Using hindsight to anchor past knowledge in continual learning. In AAAI, 2021

    2021

  3. [11]

    Adaptformer: adapting vision transformers for scalable visual recognition

    Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: adapting vision transformers for scalable visual recognition. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook...

    2022

  4. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 248--255, 2009. doi:10.1109/CVPR.2009.5206848

    2009

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2010 arXiv

  6. [14]

    Dytox: Transformers for continual learning with dynamic token expansion

    Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 9275--9285, 2022. doi:10.1109/CVPR52688.2022.00907

    2022

  7. [15]

    Hyperdiffusion: Generating implicit neural fields with weight-space diffusion

    Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nie ner, and Angela Dai. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 14254--14264, 2023

    2023

  8. [16]

    R. M. French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3: 0 128--135, 1999. doi:10.1016/s1364-6613(99)01294-2

    1999 doi

  9. [17]

    Clr: Channel-wise lightweight reprogramming for continual learning

    Yunhao Ge, Yuecheng Li, Shuo Ni, Jiaping Zhao, Ming-Hsuan Yang, and Laurent Itti. Clr: Channel-wise lightweight reprogramming for continual learning. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 18752--18762, 2023. doi:10.1109/ICCV51070.2023.01723

    2023

  10. [18]

    Grossberg

    S. Grossberg. Studies of mind and brain. Boston Studies in the Philosophy of Science, 1982. doi:10.1007/978-94-009-7758-7

    1982 doi

  11. [19]

    Dai, and Quoc V

    David Ha, Andrew M. Dai, and Quoc V. Le. Hypernetworks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings , 2017

    2017

  12. [20]

    Ghostnet: More features from cheap operations

    Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 1577--1586, 2020. doi:10.1109/CVPR42600.2020.00165

    2020

  13. [21]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 770--778, 2016. doi:10.1109/CVPR.2016.90

    2016 doi

  14. [22]

    Lo RA : Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  15. [23]

    Steven C. Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning, pp.\ 11. Curran Associates Inc., Red Hook, NY, USA, 2019

    2019

  16. [24]

    Helpful or harmful: Inter-task association in continual learning

    Hyundong Jin and Eunwoo Kim. Helpful or harmful: Inter-task association in continual learning. In European Conference on Computer Vision, pp.\ 519--535. Springer, 2022

    2022

  17. [25]

    Continual learning on noisy data streams via self-purified replay

    Chris Dongjoo Kim, Jinseo Jeong, Sangwoo Moon, and Gunhee Kim. Continual learning on noisy data streams via self-purified replay. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 517--527, 2021. doi:10.1109/ICCV48922.2021.00058

    2021

  18. [26]

    Kirkpatrick, R

    J. Kirkpatrick, R. Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwińska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy...

    2017 doi

  19. [27]

    Krizhevsky and G

    A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009

    2009

  20. [28]

    K. Lee, K. Lee, J. Shin, and H. Lee. Overcoming catastrophic forgetting with unlabeled data in the wild. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019. doi:10.1109/iccv.2019.00040

    2019

  21. [29]

    Li and D

    Z. Li and D. Hoiem. Learning without forgetting. Lecture Notes in Computer Science, pp.\ 614--629, 2016. doi:10.1007/978-3-319-46493-0_37

    2016 doi

  22. [30]

    Inflora: Interference-free low-rank adaptation for continual learning

    Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-free low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 23638--23647, 2024

    2024

  23. [31]

    Dora: weight-decomposed low-rank adaptation

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: weight-decomposed low-rank adaptation. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024 a

    2024

  24. [32]

    D o RA : Weight-decomposed low-rank adaptation

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. D o RA : Weight-decomposed low-rank adaptation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Fel...

    2024

  25. [33]

    Fixing weight decay regularization in adam

    Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. CoRR, abs/1711.05101, 2017

    2017 arXiv

  26. [34]

    S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. -, 2013

    2013

  27. [35]

    Piggyback: Adapting a single network to multiple tasks by learning to mask weights

    Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting a single network to multiple tasks by learning to mask weights. In Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IV, pp.\ 72–88, Berlin, Heid...

    2018 doi

  28. [36]

    Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Gordon H. Bower (ed.), -, volume 24 of Psychology of Learning and Motivation, pp.\ 109--165. Academic Press, 1989. doi:https://doi.org/10.1016/S0079-74...

    1989 doi

  29. [37]

    Lifelong learning of compositional structures

    Jorge A Mendez and Eric Eaton. Lifelong learning of compositional structures. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=ADWd4TJO13G

    2021

  30. [38]

    Nguyen, Yingzhen Li, Thang D

    Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner. Variational continual learning. In International Conference on Learning Representations, 2018

    2018

  31. [39]

    R o SA : Accurate parameter-efficient fine-tuning via robust adaptation

    Mahdi Nikdan, Soroush Tabesh, Elvir Crn c evi\' c , and Dan Alistarh. R o SA : Accurate parameter-efficient fine-tuning via robust adaptation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), P...

    2024

  32. [40]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, Dec 2008

    2008

  33. [41]

    Continual learning via local module composition

    Oleksiy Ostapenko, Pau Rodr\' guez, Massimo Caccia, and Laurent Charlin. Continual learning via local module composition. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISB...

    2021

  34. [42]

    Neural sculpting: uncovering hierarchically modular task structure in neural networks through pruning and network analysis

    Shreyas Malakarjun Patil, Loizos Michael, and Constantine Dovrolis. Neural sculpting: uncovering hierarchically modular task structure in neural networks through pruning and network analysis. In Proceedings of the 37th International Conference on Neural Information Processing ...

    2023

  35. [43]

    Plummer, Nikoli Dryden, Julius Frost, Torsten Hoefler, and Kate Saenko

    Bryan A. Plummer, Nikoli Dryden, Julius Frost, Torsten Hoefler, and Kate Saenko. Neural parameter allocation search. In International Conference on Learning Representations, 2022

    2022

  36. [44]

    Prabhu, P

    A. Prabhu, P. H. S. Torr, and P. K. Dokania. Gdumb: a simple approach that questions our progress in continual learning. Computer Vision – ECCV 2020, pp.\ 524--540, 2020. doi:10.1007/978-3-030-58536-5_31

    2020 doi

  37. [45]

    Recognizing indoor scenes

    Ariadna Quattoni and Antonio Torralba. Recognizing indoor scenes. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 413--420, 2009. doi:10.1109/CVPR.2009.5206537

    2009

  38. [46]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 5533--5542, 2017. doi:10.1109/CVPR.2017.587

    2017 doi

  39. [47]

    E. Rusu, O. Makarynskyy, A. Gavrilov, and D. Makarynska. Filling gaps in wave records with artificial neural networks. Maritime Transportation and Exploitation of Ocean and Coastal Resources, pp.\ 1085--1091, 2006. doi:10.1201/9781439833728.ch131

    2006 doi

  40. [48]

    Tim Salimans and Diederik P. Kingma. Weight normalization: a simple reparameterization to accelerate training of deep neural networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, pp.\ 901–909, Red Hook, NY, USA, 2016. ...

    2016

  41. [49]

    Learning implicitly recurrent CNN s through parameter sharing

    Pedro Savarese and Michael Maire. Learning implicitly recurrent CNN s through parameter sharing. In International Conference on Learning Representations, 2019

    2019

  42. [50]

    Overcoming catastrophic forgetting with hard attention to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Ma...

    2018

  43. [51]

    Continual learning with deep generative replay

    Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 2994–3003, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 97...

    2017

  44. [52]

    See further for parameter efficient fine-tuning by standing on the shoulders of decomposition

    Chongjie Si, Xiaokang Yang, and Wei Shen. See further for parameter efficient fine-tuning by standing on the shoulders of decomposition. arXiv preprint arXiv:2407.05417, 2024

    2024 arXiv

  45. [53]

    Calibrating cnns for lifelong learning

    Pravendra Singh, Vinay Kumar Verma, Pratik Mazumder, Lawrence Carin, and Piyush Rai. Calibrating cnns for lifelong learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp.\ 15579--1559...

    2020

  46. [54]

    Efficient continual learning with modular networks and task-driven priors

    Tom Veniat, Ludovic Denoyer, and Marc'Aurelio Ranzato. Efficient continual learning with modular networks and task-driven priors. CoRR, abs/2012.12631, 2020

    2012 arXiv

  47. [55]

    V. K. Verma, K. J. Liang, N. Mehta, P. Rai, and L. Carin. Efficient feature transformations for discriminative and generative continual learning. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. doi:10.1109/cvpr46437.2021.01365

    2021

  48. [56]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011

    2011

  49. [57]

    Neural network parameter diffusion

    Kaili Wang, Zhaopan Xu, Yukun Zhou, Zelin Zang, Trevor Darrell, Zhuang Liu, and Yang You. Neural network parameter diffusion. In -, 2024

    2024

  50. [58]

    S-prompts learning with pre-trained transformers: An occam's razor for domain incremental learning

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam's razor for domain incremental learning. In Conference on Neural Information Processing Systems (NeurIPS), 2022 a

    2022

  51. [59]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 139--149, 2022 b

    2022

  52. [60]

    Pytorch image models

    Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019

    2019

  53. [61]

    Lifelong learning with dynamically expandable networks

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In -. ICLR, 2018

    2018

  54. [62]

    Metadiff: Meta-learning with conditional diffusion for few-shot learning

    Baoquan Zhang, Chuyao Luo, Demin Yu, Xutao Li, Huiwei Lin, Yunming Ye, and Bowen Zhang. Metadiff: Meta-learning with conditional diffusion for few-shot learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (15): 0 16687--16695, Mar. 2024. doi:10.1609/aa...

    2024 doi

  55. [63]

    Zhang, J

    J. Zhang, J. Zhang, S. Ghosh, D. Li, S. Tasci, L. Heck, H. Zhang, and C. J. Kuo. Class-incremental learning via deep model consolidation. 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. doi:10.1109/wacv45572.2020.9093365

    2020

  56. [64]

    Interpretable basis decomposition for visual explanation

    Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018

    2018

  57. [65]

    Melo: Low-rank adaptation is better than fine-tuning for medical image diagnosis

    Yitao Zhu, Zhenrong Shen, Zihao Zhao, Sheng Wang, Xin Wang, Xiangyu Zhao, Dinggang Shen, and Qian Wang. Melo: Low-rank adaptation is better than fine-tuning for medical image diagnosis. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pp.\ 1--5, 2024. doi:10....

    2024

  58. [66]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  59. [67]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  60. [68]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  61. [69]

    | m n pzq + fd endstream endobj 18 0 obj << /Filter /FlateDecode /Length 79 >> stream xMͻ О)< >Q

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.