Pith. sign in

REVIEW 5 major objections 5 minor 42 references

Reward Incremental Learning in Text-to-Image Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reward Incremental Distillation lets text-to-image models learn new reward objectives in sequence without forgetting how to draw well.

desk verdict RIL is a real, useful problem definition, but the paper's evidence for RID is confounded by mismatched batch size and learning rate between method and baseline. read the letter →

arxiv 2411.17310 v1 pith:HT65WGEL submitted 2024-11-26 cs.CV cs.LG

Maorong Wang , Jiafeng Mao , Xueting Wang , Toshihiko Yamasaki This is my paper · ORCID
classification cs.CVcs.LG
keywords rewardincrementallearningcatastrophicforgettingtext-to-imagegenerationdiffusionmodelslow-rankadaptationknowledgedistillationEMAteachercontinualfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The authors introduce Reward Incremental Learning (RIL), a setting in which a text-to-image diffusion model is fine-tuned on a sequence of reward objectives (aesthetic quality, human preference, compressibility) rather than a single one. They show that directly adapting existing reward-gradient fine-tuning to this sequential setting causes catastrophic forgetting that appears both in standard metrics (CLIP score, zero-shot FID) and in visible image structure. To counter it, they propose Reward Incremental Distillation (RID), combining a LoRA adapter group, so each task gets its own parameters, with an exponential-moving-average teacher and a one-step distillation loss that anchors the model's final output to the teacher's. Their experiments across six orderings of the three reward tasks report that RID keeps general metrics close to the original Stable Diffusion 1.5 (average CLIP 24.42 vs. 19.76 for the baseline, average zero-shot FID 85.95 vs. 130.92) while staying competitive on target rewards, at under one percent extra training time.

What carries the argument

The central mechanism is Reward Incremental Distillation (RID), built from a LoRA adapter group and an exponential-moving-average (EMA) teacher. When a new reward task arrives, a fresh pair of LoRA matrices is added and all previous adapters and the pretrained weights are frozen, so each task's parameters do not interfere. During fine-tuning, the student model's final denoising output is aligned with the EMA teacher's final output from the same latent $z_1$ via an MSE loss (Eq. 8), with backpropagation truncated to that last step; the full training objective combines the reward gradient with this distillation loss (Eq. 9).

What would settle it

Compare RID against a variant where distillation is applied to an intermediate denoising step (say $z_{25}$ of 50) instead of only $z_1$; if the intermediate version shows substantially less forgetting, then last-step anchoring alone is not the operative mechanism.

Watch

Extended reading notes

Core claim

The paper claims that the forgetting observed in incremental reward fine-tuning of diffusion models can be substantially mitigated by a combination of two ingredients: parameter isolation per task via a group of LoRA adapters, and momentum distillation in which the fine-tuning model is pulled, on its final denoising step, toward the output of an exponential-moving-average copy of itself. The proposed objective is $R_t(f_{W,A,B}(z_1|c)) - \lambda\|f_{W,A,B}(z_1|c) - f^T_{W,A,B}(z_1|c)\|_2$, where the reward term adapts to the new objective and the distillation term preserves general generation quality. The authors argue that aligning only the last step, rather than the full denoising trajectory, keeps computation low and avoids error accumulation, and they report that across all six task orderings the general metrics stay close to the pretrained model while target rewards are preserved or improved.

Load-bearing premise

The method rests on the assumption that forcing the fine-tuned model's final denoising output to match the EMA teacher's final output, from the same latent $z_1$, is enough to preserve both metric-level and visual-structure-level knowledge from earlier tasks.

Editorial extensions

If this is right

  • A single text-to-image model can be adapted to several reward objectives over time without collapsing general image quality, so practitioners do not need to retrain from scratch or maintain multiple models.
  • The forgetting metric defined in Eq. 3 provides a quantitative way to compare continual fine-tuning methods for diffusion models beyond classification accuracy.
  • Because distillation is applied only to the last denoising step, the approach adds little compute (about 2% extra steps, under 1% training time), making it practical for large models.
  • RID outperforms both an adapted reward-gradient baseline and weight-averaging (model soup) in keeping general metrics stable across task orderings.
  • Target rewards are maintained across sequences; for example, after all three tasks, RID retains aesthetic and human-preference improvements over the original model while the baseline falls below it (Table 2).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The last-step distillation assumption could be tested by ablating distillation at different denoising steps; if intermediate-step drift is the main forgetting source, RID would underperform on longer trajectories.
  • Since RID keeps an EMA teacher of the full student, it could be combined with other continual-learning techniques such as replay of previous prompts or parameter regularization, potentially improving retention further.
  • The method's reliance on a differentiable reward and a frozen EMA copy suggests it could extend to non-differentiable rewards by using the teacher's output as a pseudo-target and the reward signal from a learned reward model, though that is beyond the paper.
  • A natural stress test is to push the number of tasks beyond three; the LoRA adapter group grows linearly with tasks, so at some point memory and interference between many adapters may need a pruning or merging step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper defines Reward Incremental Learning (RIL), a continual-learning setting in which a text-to-image diffusion model is sequentially fine-tuned on differentiable reward objectives (aesthetic quality, human preference, compressibility). The authors observe that a baseline constructed by adapting reward-backpropagation methods [5,22] with a per-task LoRA adapter group suffers from both metric-wise and visual-structure-wise catastrophic forgetting. They propose Reward Incremental Distillation (RID), which combines this adapter group with an EMA teacher whose final denoising-step output is matched to the student's final output through an MSE penalty (Eq. 9). Across six task orderings of three reward tasks (Table 3), RID reports average CLIP score 24.42 vs 19.76 and zero-shot MS-COCO FID 85.95 vs 130.92 for the baseline, with original Stable Diffusion V1.5 at 25.77 and 71.54, and lower target-metric forgetting, at under 1% extra training time (Table 4).

Significance. If the central comparison were properly controlled, the paper would make a useful contribution: RIL is a well-motivated problem formulation, the forgetting phenomenon is documented with general metrics (CLIP score, zero-shot FID) that are not part of the training reward, which avoids circularity, and the proposed mechanism is computationally cheap (Table 4) and described with runnable pseudo-code (Alg. 1). The six task orderings in Table 3 give the evaluation reasonable breadth within its chosen scope. The main caveat is that the evidence is currently confounded with the optimization schedule (batch size 128 / lr 1e-3 for the baseline vs 8 / 5e-5 for RID, per Sec. 10), so the magnitude of the benefit attributable to the method itself is not yet established. With matched-hyperparameter ablations and at least one additional continual-learning control, the empirical claim would be substantially stronger.

major comments (5)
  1. [Sec. 10; Tables 1-3] The central comparison is confounded by the optimization schedule. Sec. 10 reports that the adapted baseline is trained with batch size 128 and learning rate 1e-3, while RID uses batch size 8 and learning rate 5e-5, and states that 'Under this configuration, the adapted baseline performs worse.' With 750 training prompts, the baseline performs about 6 optimizer updates per epoch versus about 94 for RID, a roughly 15-fold difference in update counts. Because Tables 1-3 vary the method and the schedule simultaneously, the large gaps in CLIP score (24.42 vs 19.76) and zero-shot FID (85.95 vs 130.92) cannot be attributed to the LoRA adapter group and the EMA distillation; a smaller batch size and a smaller learning rate can by themselves reduce forgetting or trade off task adaptation. A matched comparison (e.g., the baseline at batch 8 / lr 5e-5, and RID without the distillation term under that same schedule), or a matched total-update comparison, is needed to support the abstract's claim that RID's components mitigate forgetting.
  2. [Sec. 4.2, Eq. (8)] The sufficiency of the last-step distillation is not empirically established. The entire forgetting-mitigation mechanism of RID is the single MSE alignment between the student's and the EMA teacher's final denoising-step outputs given the same latent z1 (Eq. 8), yet no ablation is reported that varies the distillation design: no distillation, full-step distillation as in Eq. (7), last-step distillation as in Eq. (8), or alignment at intermediate steps. This matters because the paper motivates the forgetting phenomenon as affecting both metrics and visual structure across the generation trajectory (Sec. 1, Fig. 1); if the drift originates in earlier denoising steps, a one-step output anchor may be too weak, and the claim in Fig. 4 that full-step distillation suffers from accumulated error is plausible but untested. The balancing coefficient lambda in Eq. (9) is also fixed at 0.1 with no sensitivity analysis. Ablations varying the distillation depth and lambda under matched hyperparameters are required to show that the proposed component, rather than the training schedule, produces the reported retention.
  3. [Eq. (6), Alg. 1] A core hyperparameter of the proposed method is missing. The EMA teacher in Eq. (6) depends on the momentum coefficient alpha, and Alg. 1 calls ema_model.update(model), but the value of alpha is never reported in the main text or in Sec. 10, which gives epochs, learning rates, batch sizes, and lambda but not alpha. The optimizer, its hyperparameters, and any learning-rate schedule are also unspecified. Because the EMA teacher is the mechanism claimed to retain previous knowledge, RID cannot be reproduced or ablated without these values; please add a complete implementation-details table.
  4. [Sec. 5.2, Table 3] The reported results are single runs with no variance or significance information, and the direction of the differences is not uniform across target metrics. In Table 3, RID's average aesthetic score (5.55) is below the baseline's (5.62) across the same six orderings, and in two orderings (Compress-HPS-Aesthetic and HPS-Compress-Aesthetic) the baseline's final-task aesthetic score exceeds RID's by a wide margin (7.23 vs 5.77 and 6.18 vs 5.75). The general-metric differences are large and likely robust, but the claim in Sec. 5.2 that RID 'consistently outperforms the adapted baseline' would be substantiated more convincingly with multiple seeds and error bars, especially where target-metric differences are small (e.g., average HPS 0.277 vs 0.258).
  5. [Sec. 4.1, Sec. 6] The comparison set contains no external continual-learning control. The only comparators are the authors' own adapted baseline (reward backpropagation plus the proposed LoRA adapter group, Sec. 4.1) and a model-soup combination with arbitrarily fixed mixing weights alpha = beta = 1/3 (Sec. 6). Since the forgetting phenomenon is presented as a general issue of sequential reward fine-tuning in diffusion models, at least one control from the existing continual-learning literature (e.g., InfLoRA alone, a regularization-based adapter method, or sequential fine-tuning with a single shared LoRA) should be evaluated to show that the combined design of the LoRA adapter group plus EMA distillation, rather than any LoRA-based approach with careful hyperparameters, is what yields the observed retention.
minor comments (5)
  1. [Sec. 5.2, Table 3] The sentence in Sec. 5.2 that RID 'achieves notable improvements across all target metrics' is ambiguous given that Table 3's average aesthetic score is lower for RID (5.55) than for the baseline (5.62); please specify whether the comparison is against Stable Diffusion V1.5 or against the baseline.
  2. [Eq. (3)] The notation in Eq. (3) is confusing: the superscript t denotes the task index while the subscript i denotes the training stage, and T is used both as the total number of tasks and as the index of the current stage, although Table 2 reports forgetting at intermediate stages; please re-notate and define all indices.
  3. [Tables 2-3] In Tables 2 and 3, zero forgetting is sometimes written as '0.00' and sometimes as '-' (e.g., the Compress-HPS-Aesthetic row of Table 3), and the metric of the most recently trained task always shows '-' even though it is the reference value in Eq. (3); the convention should be stated explicitly.
  4. [Fig. 5] The qualitative comparisons would be more informative with larger crops and zoom-ins; at thumbnail size, the claimed visual-structure-wise forgetting in Fig. 5 is difficult for the reader to verify.
  5. [Sec. 2] Sec. 2 does not discuss recent continual-learning methods designed for diffusion models (outside the classification-focused literature it cites); engaging with them would sharpen the novelty claim of the RIL formulation and the positioning of RID.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RID's forgetting claim is grounded in external general metrics; the unequal-hyperparameter confound is a validity issue, not a circular derivation.

full rationale

The claimed derivation chain is: RID optimizes Eq. (9), which is the differentiable reward objective of Eq. (5) plus an MSE last-step distillation to the EMA teacher, Eq. (8). The evaluation then measures retention on target reward metrics and on two general metrics, CLIP score and zero-shot MS-COCO FID (Table 3). Neither general metric enters the training objective, so the observed retention of CLIP/FID is an external, non-circular check of forgetting. Target-metric retention is partly a re-measurement of the training reward, but that is standard for reward fine-tuning and is not a prediction claimed from first principles. The only self-citation that could be questioned is [19] (momentum distillation in online continual learning, sharing two authors); it is used only as motivating background for distillation and is not the mathematical justification, which is given by Eq. (8) and verified on independent metrics, so it is not load-bearing. The paper itself flags a genuine limitation in Supplementary §10: the adapted baseline uses batch size 128 and learning rate 1e-3 while RID uses batch size 8 and 5e-5, and states "Under this configuration, the adapted baseline performs worse." This is a confounding factor for the empirical comparison, but it does not make any reported result equal to its input by construction; there is no fitted parameter renamed as a prediction and no uniqueness claim borrowed from self-citation. Hence the circularity score is 0.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The method is empirical and its central claim depends on the chosen hyperparameters and on transferring established components (InfLoRA and EMA distillation) to diffusion models. The free parameters are mostly training hyperparameters; the key issue is the different batch size and learning rate between baseline and RID, which is not isolated. The axioms include the validity of the differentiable rewards, the assumption that per-task LoRAs prevent interference, the standard truncated backprop approximation, and the paper-specific assumption that last-step EMA distillation is a sufficient knowledge anchor.

free parameters (8)
  • distillation weight lambda = 0.1
    Balancing coefficient between reward and EMA distillation loss in Eq. 9, set by hyperparameter search (Sec. 4.2).
  • EMA momentum alpha = not reported
    Controls the EMA teacher update in Eq. 6; the paper does not state the value used.
  • LoRA rank = 4
    Rank of the per-task LoRA matrices, chosen without reported search (Sec. 5.1).
  • baseline batch size and learning rate = 128 and 1e-3
    Hyperparameters used only for the baseline, differing by 16x and 20x from RID; this asymmetry affects the fairness of the comparison (Sec. 10).
  • RID batch size and learning rate = 8 and 5e-5
    Hyperparameters used only for RID; the paper states the baseline performs worse under this configuration, so the two methods are not compared under matched settings.
  • epochs per reward task = aesthetic 10, HPS 120, compress 15
    Training durations per task, chosen by search and different across rewards (Sec. 10).
  • model soup mixing weights = alpha=beta=0.333
    One fixed choice for averaging three independently fine-tuned models (Sec. 6); may not be optimal.
  • joint tuning weights = 0.01, 2, 1
    Weights in Eq. 10 for the joint tuning comparison, chosen by a small search (Appendix Sec. 9).
assumptions (5)
  • domain assumption Differentiable reward functions (aesthetic predictor, HPSv2, differentiable JPEG) provide meaningful gradients for fine-tuning
    The entire method optimizes through these reward models; if their gradients are uninformative, the method fails. Invoked in Sec. 3.1.
  • domain assumption Per-task LoRA adapters prevent interference, as claimed by InfLoRA
    The LoRA adapter group design relies on the transferability of InfLoRA's findings to diffusion models (Sec. 4.1).
  • domain assumption Truncated backpropagation through the final DDIM step approximates the full reward gradient well enough for learning
    Both baseline and RID use this approximation following [5,22]; it is a known approximation used in prior work.
  • ad hoc to paper EMA teacher tracks the student closely enough that the last-step MSE loss anchors previous knowledge
    The core of RID's forgetting mitigation; the paper provides no ablation showing this anchoring is effective at different distillation depths.
  • standard math Zero-shot MS-COCO FID and CLIP score capture general generation quality for this fine-tuning regime
    Used as general metrics without validation in this setting; they are standard evaluation metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reward Incremental Learning in Text-to-Image Generation." pith.science (2026). https://pith.science/paper/HT65WGEL

@misc{pith2026241117310,
  author       = {Pith},
  title        = {Pith review of: Reward Incremental Learning in Text-to-Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HT65WGEL}},
  note         = {Machine review of arXiv:2411.17310}
}
read the original abstract

The recent success of denoising diffusion models has significantly advanced text-to-image generation. While these large-scale pretrained models show excellent performance in general image synthesis, downstream objectives often require fine-tuning to meet specific criteria such as aesthetics or human preference. Reward gradient-based strategies are promising in this context, yet existing methods are limited to single-reward tasks, restricting their applicability in real-world scenarios that demand adapting to multiple objectives introduced incrementally over time. In this paper, we first define this more realistic and unexplored problem, termed Reward Incremental Learning (RIL), where models are desired to adapt to multiple downstream objectives incrementally. Additionally, while the models adapt to the ever-emerging new objectives, we observe a unique form of catastrophic forgetting in diffusion model fine-tuning, affecting both metric-wise and visual structure-wise image quality. To address this catastrophic forgetting challenge, we propose Reward Incremental Distillation (RID), a method that mitigates forgetting with minimal computational overhead, enabling stable performance across sequential reward tasks. The experimental results demonstrate the efficacy of RID in achieving consistent, high-quality generation in RIL scenarios. The source code of our work will be publicly available upon acceptance.

Figures

Figures reproduced from arXiv: 2411.17310 by the authors.

Figure 1
Figure 1. We define Reward Incremental Learning (RIL), a novel task that aims to fine-tune a diffusion model with a sequence of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of our proposed RID. RID has two main components: LoRA adapter group and momentum distillation. Through [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Comparison between the naive full-step distillation strat [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of generation results using a fixed task sequence in the RIL setting. In generation, instead of reusing the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of generation results at the end of tuning on different task sequences, along with the results of joint tuning [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 30 canonical work pages

  1. [1]

    Training diffusion models with reinforce- ment learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforce- ment learning. arXiv preprint arXiv:2305.13301, 2023. 2, 3

    arXiv 2023

  2. [2]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In ICCV, pages 9650–9660, 2021. 5

    2021

  3. [3]

    Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence

    Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajan- than, and Philip HS Torr. Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence. In ECCV, 2018. 4

    work page 2018

  4. [4]

    Lifelong machine learning

    Zhiyuan Chen and Bing Liu. Lifelong machine learning . Springer Nature, 2017. 3

    work page 2017

  5. [5]

    Directly fine-tuning diffusion models on differentiable re- wards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable re- wards. In ICLR, 2024. 1, 2, 3, 5

    work page 2024

  6. [6]

    A continual learning survey: Defying forgetting in classification tasks

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale ˇs Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. TPAMI, 44(7):3366–3385,

    work page

  7. [7]

    Cogview: Mastering text-to-image gen- eration via transformers

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image gen- eration via transformers. In NeurIPS, pages 19822–19835,

    work page

  8. [8]

    An empirical investigation of catas- trophic forgetting in gradient-based neural networks

    Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catas- trophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013. 2, 3

    arXiv 2013

Show all 42 references
  1. [9]

    Delving deep into rectifiers: Surpassing human-level perfor- mance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level perfor- mance on imagenet classification. In ICCV, pages 1026– 1034, 2015. 5

    2015

  2. [10]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In CVPR, pages 9729–9738, 2020. 5

    2020

  3. [11]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. In EMNLP, 2021. 2, 7

    2021

  4. [12]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In NeurIPS, 2017. 2, 7

    2017

  5. [13]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. In NeurIPS, pages 6840–6851,

  6. [14]

    Re-evaluating continual learning scenarios: A cate- gorization and case for strong baselines

    Y Hsu. Re-evaluating continual learning scenarios: A cate- gorization and case for strong baselines. InContinual Learn- ing Workshop@NIPS, 2018. 4

    2018

  7. [15]

    Aligning text- to-image models using human feedback

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023. 2, 3

    2023 arXiv

  8. [16]

    Inflora: Interference-free low-rank adaptation for continual learning

    Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-free low-rank adaptation for continual learning. In CVPR, pages 23638–23647, 2024. 4, 5

    2024

  9. [17]

    Online continual learning in image classification: An empirical survey

    Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyun- woo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28–51, 2022. 3

    2022

  10. [18]

    Catastrophic inter- ference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation , pages 109–165. 1989. 2, 3

    1989

  11. [19]

    Rethinking momentum knowledge distillation in online continual learning

    Nicolas Michel, Maorong Wang, Ling Xiao, and Toshihiko Yamasaki. Rethinking momentum knowledge distillation in online continual learning. In ICML, pages 35607–35622,

  12. [20]

    Exponential moving average of weights in deep learn- ing: Dynamics and benefits

    Daniel Morales-Brotons, Thijs V ogels, and Hadrien Hen- drikx. Exponential moving average of weights in deep learn- ing: Dynamics and benefits. TMLR, 2024. 5

    2024

  13. [21]

    Ava: A large-scale database for aesthetic visual analysis

    Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: A large-scale database for aesthetic visual analysis. In CVPR, pages 2408–2415, 2012. 3

    2012

  14. [22]

    Aligning text-to-image diffusion models with reward backpropagation

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation. arXiv preprint arXiv:2310.03739, 2023. 1, 2, 3, 5, 7

    2023 arXiv

  15. [23]

    Simulacra aesthetic captions

    John David Pressman, Katherine Crowson, and Simu- lacra Captions Contributors. Simulacra aesthetic captions. Technical report, Stability AI, 2022. https://github. com/JD-P/simulacra-aesthetic-captions . 3

    2022

  16. [24]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, pages 8748–8763, 2021. 3

    2021

  17. [25]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, pages 8821– 8831, 2021. 2

    2021

  18. [26]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2 9

    2022 arXiv

  19. [27]

    icarl: Incremental classi- fier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classi- fier and representation learning. InCVPR, pages 2001–2010,

    2001

  20. [28]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 7

    2022

  21. [29]

    LAION- aesthetics

    Christoph Schuhmann and Romain Beaumont. LAION- aesthetics. laion.ai, 2022. 3, 7

    2022

  22. [30]

    Jpeg-resistant adversarial im- ages

    Richard Shin and Dawn Song. Jpeg-resistant adversarial im- ages. In NeurIPS workshop on machine learning and com- puter security, page 8, 2017. 4

    2017

  23. [31]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 5

    2010 arXiv

  24. [32]

    Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

    Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, 2017. 5

    2017

  25. [33]

    Three scenarios for continual learning

    Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. In NeurIPS workshop on Continual Learning, 2019. 4

    2019

  26. [34]

    A comprehensive survey of continual learning: theory, method and application

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: theory, method and application. TPAMI, 2024. 3

    2024

  27. [35]

    Investigating prompt engineering in diffusion models

    Sam Witteveen and Martin Andrews. Investigating prompt engineering in diffusion models. arXiv preprint arXiv:2211.15462, 2022. 2, 3

    2022 arXiv

  28. [36]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing in- ference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Re- becca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Ko- rnblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing ...

    2022

  29. [37]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  30. [38]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. In NeurIPS, 2024. 2, 3

    2024

  31. [39]

    Towards language-free training for text-to-image generation

    Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. Towards language-free training for text-to-image generation. In CVPR, pages 17907–17917, 2022. 2 10 Reward Incremental Learning in Text-to-Image Generation Supp...

    2022

  32. [40]

    Pseudo Code for EMA Distillation For better understanding, we present the pseudo-code for the EMA distillation as in Alg. 1. For simplicity, we ignored the autoencoder decoding step at the end of the diffusion

  33. [41]

    Apart from the RIL set- ting, we also compare results with a jointly trained approach that optimizes the weighted sum of three reward objectives using the adapted baseline

    Further Analysis and Qualitative Samples Comparison with joint tuning. Apart from the RIL set- ting, we also compare results with a jointly trained approach that optimizes the weighted sum of three reward objectives using the adapted baseline. In this setup, the joint reward R...

  34. [42]

    animation

    Experiment Details Hyperparameter details. We use different hyperparame- ters, specifically epochs and learning rates, for fine-tuning each reward task due to variations in the gradients gener- ated by the reward functions. Specifically, for all experi- ments, we train for 120...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.