R^2MoE: Redundancy-Removal Mixture of Experts for Lifelong Concept Learning

Hongliang Li; Lili Pan; Xiaohan Guo; Yusong Cai; Zejia Liu; Zhengning Wang

REVIEW 4 major objections 5 minor 1 cited by

R^2MoE: Redundancy-Removal Mixture of Experts for Lifelong Concept Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims R2MoE lets a frozen text-to-image model keep learning 101 visual concepts with 0.19% forgetting and 15.2M added parameters.

desk verdict Solid MoE-based continual learning paper with strong numbers, but Eq. 4's routing distillation is undefined as written and needs a fix before the results can be trusted. read the letter →

arxiv 2507.13107 v1 pith:3PV3IEKN submitted 2025-07-17 cs.CV

Xiaohan Guo , Yusong Cai , Zejia Liu , Zhengning Wang , Lili Pan , Hongliang Li This is my paper

classification cs.CV

keywords lifelongconceptlearningtext-to-imagediffusionmixtureofexpertscatastrophicforgettingparameter-efficientfine-tuningroutingdistillationexpertpruningmulti-conceptgeneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

R2MoE claims that a text-to-image model can keep adding new visual concepts throughout its lifetime by giving each concept a low-rank expert, freezing that expert after training, and distilling the old routing behavior into a new gating network when the next concept arrives. On the 101-concept CustomConcept101 benchmark, the paper reports an image-alignment score of 75.3%, which beats the two strongest prior lifelong methods by 10.1 and 2.8 percentage points, while forgetting only 0.19% of old concepts compared with 8.56% and 1.56% for those methods. The same design keeps added parameters to 15.2M for 101 concepts, about 63% fewer than the best prior method. If these numbers hold, a single generative model could be personalized incrementally for many users or objects without retraining old concepts and without a linearly growing parameter budget.

What carries the argument

The load-bearing object is a per-layer mixture of LoRA experts with a text-embedding gating network: $W_n = W_0 + \sum_{i \in I_n} \alpha_{n,i} E_i$, where $W_0$ is the frozen pretrained weight and $\alpha_{n,i}$ are softmax-normalized gating coefficients from Eq. (6). Three mechanisms carry the argument: routing distillation, which trains the new gating network to match the old one on stored concept embeddings via $\mathcal{L}_g(\Theta_n) = \sum_{\tau=1}^{n-1} \| g_n(C_\tau) - g_{n-1}(C_\tau) \|_F^2$; selective expert activation, which forces inclusion of the frozen auxiliary expert and the new expert while choosing $K-1$ old experts by top-$K$ routing; and layer-wise expert pruning, which deletes a newly added expert whenever its own gating coefficient falls below threshold $p$. At inference, hierarchical local attention uses LLM-generated bounding boxes early in denoising and segmentation-model masks late in denoising to combine region-specific expert outputs.

What would settle it

Run R2MoE on CustomConcept101 with the routing-distillation term removed while keeping selective activation, pruning, and hierarchical attention; if forgetting remains near 0.19%, routing distillation is not the cause. Alternatively, inspect the stored gating outputs at the task boundary: if $g_n(C_\tau)$ and $g_{n-1}(C_\tau)$ have different lengths and no alignment rule exists, then Eq. (4) as written cannot be evaluated and the forgetting numbers must depend on an unspecified implementation choice.

Watch

Extended reading notes

Core claim

The paper's central claim is that catastrophic forgetting can be controlled at the router rather than by constraining expert weights. R2MoE grows a sparse mixture of LoRA experts inside the cross-attention Key and Value projections, and after each task it freezes the newly trained expert. When the next concept arrives, a routing-distillation loss forces the new gating network to reproduce the previous gating network's outputs on stored text embeddings of old concepts, so old prompts still route to their old experts. A threshold on gating coefficients then prunes newly added experts that contribute little, and hierarchical local attention separates concepts during generation using LLM-generated layout boxes and a segmentation model. The reported result is near-zero forgetting (0.05% on DreamBooth, 0.19% on CustomConcept101), better image alignment than the compared lifelong methods, and added parameters that grow sublinearly with concept count.

Load-bearing premise

The load-bearing premise is that the routing-distillation loss in Eq. (4) is computable as written: the new gating network outputs $n+1$ coefficients while the old one outputs $n$, and no alignment, padding, truncation, or masking is specified, so if that mismatch is not resolved in a way that preserves the old routing behavior, the reported near-zero forgetting may not be caused by the mechanism the paper credits.

Editorial extensions

If this is right

If the reported numbers hold, continual personalization of a text-to-image model no longer requires retraining old concepts; each new concept is added as one frozen expert plus a router update.
Forgetting would drop from several percent to near zero on standard lifelong benchmarks, changing the practical trade-off in favor of much longer concept sequences.
Parameter growth becomes sublinear: on 101 concepts the method uses 15.2M added parameters, and expert pruning suggests that many future concepts can reuse existing expert capacity.
Multi-concept generation would gain a cheap inference-time control: layout masks from an LLM and a segmenter can be applied to any MoE-based customization model without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: scale the same router-distillation recipe past 101 concepts to see whether gating becomes the binding constraint; near-zero forgetting may persist only while concepts remain separable in text-embedding space.
The paper's Eq. (4) compares gating outputs of different widths ($n+1$ versus $n$ coefficients) without specifying alignment; until that is resolved, the reported 0.19% forgetting should be read as conditional on an unspecified implementation detail.
Because hierarchical local attention delegates layout to an external LLM and segmenter, an ablation that replaces those modules with fixed boxes would isolate how much of the multi-concept gain comes from the MoE itself.
The importance metric for pruning is the gating coefficient, which is learned while the new expert is still being trained; a reader could test whether pruning decisions are stable across random seeds and early stopping points.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Full disclosure: I haven't run the code, but the paper is coherent and the experimental effort is real. The core idea is to use per-concept LoRA experts in cross-attention with a gating network distilled from the previous one, prune redundant experts, and use LLM/SAM-guided attention at inference. The headline results on CustomConcept101 (0.19% forgetting, 15.2M added parameters) are impressive if true.

What's genuinely new is the combination. Individual pieces exist — C-LoRA, CIDM, MoE adapters — but the integrated framework with routing distillation and gating-based pruning is not in prior work. The ablations are informative: removing routing distillation drops DINO-IA from 62.0 to 55.5, so the mechanism matters. The pruning idea is simple and plausible. Releasing code is a plus.

Now the soft spots, in decreasing severity. First, Eq. 4 is not a well-defined loss. At task n, g_n outputs n+1 coefficients, while g_{n-1} outputs n coefficients. The Frobenius norm of their difference is undefined without specifying padding, truncation, or masking. This isn't stylistic — the paper attributes the near-zero forgetting to this distillation, and the ablation says it's important. The authors need to state how they align the vectors. Second, hyperparameters beta, p, K, gamma, and r are not in the main text; they're deferred to the supplementary, but for a main-claim result they should be in the paper or the code should be sufficient to find them. Third, there are small numerical inconsistencies: Fig 1 claims a 4.3% improvement over C-LoRA, while Table I gives 2.8 percentage points; the abstract's 87.8% forgetting reduction matches Table I, but Fig 1's right label says 81.2% and 63.3% parameter reductions without specifying baselines. These are easy to fix. Finally, no error bars; with such small forgetting numbers, a couple of runs would help.

The central argument holds up: MoE with isolated experts plus routing distillation is a reasonable way to reduce forgetting. The evaluation is extensive enough to warrant a serious referee. I'd recommend accepting the paper for peer review, with a request to fix Eq. 4, report hyperparameters, and reconcile the numbers. It's a good contribution to the continual-learning-for-diffusion subfield, but not ready as-is.

Referee Report

4 major / 5 minor

Summary. The paper proposes R2MoE, a mixture-of-experts framework for lifelong visual concept learning in text-to-image diffusion models. The method augments cross-attention K/V projections with LoRA-style experts selected by a per-task gating network, adds a routing distillation loss intended to preserve old routing behavior, prunes experts whose gating coefficient after training falls below a threshold, and uses an LLM/SAM-based hierarchical attention mask at inference. Experiments on DreamBooth and CustomConcept101 report state-of-the-art image alignment, forgetting rates as low as 0.05-0.19%, and a 63.3% parameter reduction relative to CIDM.

Significance. If the central mechanism is well-defined and reproducible, the paper makes a useful contribution: it addresses both catastrophic forgetting and parameter growth in continual personalization of diffusion models, and its near-zero forgetting numbers are striking. The paper ships no code in the submission and omits several key implementation details from the main text, so the claimed gains cannot currently be verified. The routing distillation idea is novel in this setting and could be a valuable building block once the loss is specified unambiguously.

major comments (4)

[Section IV-A, Eq. (4)] The routing distillation loss is not computable as written. In Eq. (3), the new gating network for task n outputs a vector in R^{N+1} (or, under the natural reading of the continual setup, in R^{n+1}), while g_{n-1} outputs a vector of dimension n (or fewer if experts have been pruned). Eq. (4) subtracts these vectors directly without specifying zero-padding, truncation, masking, or restriction to shared coordinates. This is load-bearing: the paper attributes the near-zero forgetting (0.19% on CustomConcept101 and 0.05% on DreamBooth) to routing distillation, and the Table V ablation that reports large drops when RDM is removed presupposes a concrete, correct implementation. Please state exactly how the output dimensions are aligned and how pruned experts are handled in the distillation target.
[Section IV-A to IV-D and Section V-A] Several hyperparameters that the main claims depend on are not reported in the main text or in an accessible supplement: the routing-distillation weight beta in Eq. (5), the pruning threshold p in Eq. (7), the number of selected experts K in Eq. (6), the attention-mixing weight gamma in Eq. (9), and the schedule ratio r for the timestep split in Section IV-D are all left unspecified. The text says "detailed in the supplementary material," but the arXiv submission does not include that supplement. Without these values, the experiments cannot be reproduced and the sensitivity of the 0.19% forgetting number to each choice cannot be assessed.
[Section V-D, Table V and surrounding text] The ablation numbers in the text do not match the table. The text states that removing RDM reduces CLIP-IA and DINO-IA by 15.3% and 17.8%, respectively, but Table V gives R2MoE 78.8/62.0 and w/o RDM 75.8/55.5, which are 3.8% and 10.5% relative reductions. The qualitative conclusion (RDM matters) survives, but the reported percentages should be corrected, and the table row labeled "R2MoE" should be reconciled with the identical row "Ours w/o HLAG" in Table I.
[Section IV-A, Eq. (4) and Section III-B] The routing distillation loss compares gating outputs on stored concept text embeddings C_tau, but at inference the gating network receives text embeddings for arbitrary user prompts. The paper does not justify why preserving the gating output on the finite set of learned identifiers transfers to unseen prompts with the same or similar semantics. Providing at least an empirical check (e.g., the distribution of gating coefficients on held-out prompts before and after task n) would strengthen the claim that routing capability is preserved rather than merely memorized.

minor comments (5)

[Section IV-A, Eq. (4)] There is a typo in the displayed equation: the norm expression has an extra opening parenthesis and is missing the corresponding closing parenthesis: it should be \|g_n(C_τ) - g_{n-1}(C_τ)\|_F^2.
[Abstract and Section I] The abstract and introduction refer to "CILP-IA" in one place; this should be "CLIP-IA".
[Section IV-A] The phrase "we only fintune the parameters" should read "finetune".
[Section V-B] The sentence "with 0.2%, 1.5% improvements in image and text alignment respectively over CIDM" is ambiguous: the numbers appear to be absolute percentage-point changes (85.7 vs 84.1 and 75.4 vs 74.3), not relative improvements. Please clarify the convention consistently across the paper.
[Table I and Table V] The row labeled "Ours w/o HLAG" is reported as 78.8/62.0 in Table I but the same configuration is called "R2MoE" in Table V, while the full method "Ours" in Table I has 78.7/61.9. The relationship between these rows should be stated explicitly; as written it appears that removing HLAG slightly improves CLIP-IA and DINO-IA on DreamBooth, which is in tension with the claim that HLAG helps.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: R2MoE's routing distillation, expert pruning, and attention-guided inference are internal design choices; benchmark gains are externally evaluated and not forced by construction.

full rationale

The paper contains no load-bearing circular step. The routing distillation loss (Eq. 4) is an internal regularizer that trains the current gating network to reproduce the previous gating network's outputs on stored concept embeddings; the reported forgetting rates are then measured on held-out generation quality, so the low-forgetting claim is not equivalent to the loss by construction. The expert-pruning criterion in Sec. IV-C defines expert importance as the gate coefficient and retains experts above threshold p; this is an explicit design choice rather than a fitted parameter renamed as a prediction, and the parameter-count reduction directly follows from the pruning rule rather than being presented as an independent empirical discovery. The hierarchical attention-guided inference (Sec. IV-D) is a test-time control procedure using external GPT-4/SAM tools, and its effect is ablated empirically. Baselines such as CIDM, C-LoRA, and L2DM are external comparisons, and no central premise is justified solely by a self-citation. The only serious flaw is a correctness/implementation concern: Eq. 4 subtracts gating vectors of different lengths (the new gating network outputs N+1 coefficients while the previous one outputs fewer), with no alignment specified; this makes the main anti-forgetting objective under-specified, but this is an incompleteness, not a circularity. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central method depends on trained components and a small set of hyperparameters, of which beta, p, K, gamma, and r are not specified in the submitted manuscript. The key structural assumptions are that cross-attention K/V projections are the right locus for concept knowledge, that mean-pooled text embeddings suffice for routing, and that gating coefficients indicate expert importance. The auxiliary frozen expert E0 is an introduced architectural entity with no external falsifiable prediction.

free parameters (5)

beta (routing distillation weight) = not reported in main text
Balances the LDM generation loss and the routing distillation loss in Eq. 5; the value affects how strongly old routing behavior is preserved.
pruning threshold p = not reported in main text
Determines whether a newly added expert is retained or deleted in Eq. 7; directly controls the reported parameter savings and likely affects forgetting.
number of selected experts K = not reported in main text
Controls the TopK selection in Eq. 6 and therefore how many old experts participate in each task.
attention mixing weight gamma = not reported in main text
Controls the contribution of expert-based region attention in Eq. 9 during inference.
attention schedule ratio r = not reported in main text
Determines the denoising timestep at which bounding-box masks switch to SAM-refined masks in Section IV-D.

assumptions (5)

domain assumption Cross-attention Key and Value projections in the Stable Diffusion U-Net are the right location for concept-specific knowledge.
Section IV-A builds all experts in WK and WV, following Custom Diffusion and C-LoRA. If concept knowledge also resides elsewhere, the expert modules cannot fully capture it.
domain assumption Mean-pooling text embeddings over the token dimension gives the gating network sufficient signal to route concepts.
Section IV-A defines the gating function as averaging over L tokens before an MLP. This discards per-token structure that may matter for multi-concept prompts.
ad hoc to paper The routing distillation loss in Eq. 4 is well-defined across gating networks of different output widths.
The paper compares gn(C_tau) with gn-1(C_tau) without specifying alignment, padding, truncation, or masking, despite the vectors having different dimensions.
ad hoc to paper Gating coefficient magnitude indicates expert importance for pruning.
Section IV-C and Eq. 7 retain a new expert only if its own gating coefficient exceeds threshold p. No causal or validation-based importance measure is provided.
domain assumption Early and late diffusion layers encode low-level attributes while intermediate layers encode high-level semantics, justifying removal of low-importance experts.
Section IV-C cites reference [40] for this layer-role claim and uses it to argue that newly added experts in certain layers may be redundant.

invented entities (2)

Auxiliary frozen expert E0
purpose: Provides an extra routing target during the first task so the gating network receives gradient signal before real experts are learned.
Introduced in Section IV-A as an architectural device. It has no falsifiable prediction outside the method and is kept frozen throughout training.
Per-task gating network Theta_n
purpose: Computes expert coefficients for the n-th task and is distilled to preserve old routing behavior.
Introduced in Section IV-A as an internal component. It is trained jointly with the new expert and has no external testable handle.

how reviews work

0 comments

Cite this review

Pith. "Pith review of R^2MoE: Redundancy-Removal Mixture of Experts for Lifelong Concept Learning." pith.science (2026). https://pith.science/paper/3PV3IEKN

@misc{pith2026250713107,
  author       = {Pith},
  title        = {Pith review of: R^2MoE: Redundancy-Removal Mixture of Experts for Lifelong Concept Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3PV3IEKN}},
  note         = {Machine review of arXiv:2507.13107}
}

read the original abstract

Enabling large-scale generative models to continuously learn new visual concepts is essential for personalizing pre-trained models to meet individual user preferences. Existing approaches for continual visual concept learning are constrained by two fundamental challenges: catastrophic forgetting and parameter expansion. In this paper, we propose Redundancy-Removal Mixture of Experts (R^2MoE), a parameter-efficient framework for lifelong visual concept learning that effectively learns new concepts while incurring minimal parameter overhead. Our framework includes three key innovative contributions: First, we propose a mixture-of-experts framework with a routing distillation mechanism that enables experts to acquire concept-specific knowledge while preserving the gating network's routing capability, thereby effectively mitigating catastrophic forgetting. Second, we propose a strategy for eliminating redundant layer-wise experts that reduces the number of expert parameters by fully utilizing previously learned experts. Third, we employ a hierarchical local attention-guided inference approach to mitigate interference between generated visual concepts. Extensive experiments have demonstrated that our method generates images with superior conceptual fidelity compared to the state-of-the-art (SOTA) method, achieving an impressive 87.8\% reduction in forgetting rates and 63.3\% fewer parameters on the CustomConcept 101 dataset. Our code is available at {https://github.com/learninginvision/R2MoE}

Figures

Figures reproduced from arXiv: 2507.13107 by the authors.

**Figure 1.** R2MoE vs. state-of-the-Art methods on CustomConcept101 Dataset. Left: R 2MoE achieves superior concept retention with minimal degradation in image alignment, significantly outperforming C-LoRA and CIDM by 4.3% and 10.4%, respectively. Right: our approach achieves the lowest parameter overhead, reducing parameter count by 81.2% and 63.3% compared to C-LoRA and CIDM, respectively. these models to create customized con… view at source ↗

**Figure 2.** Train paradigm of the proposed R2MoE framework. The fig illustrates a continual learning process across multiple tasks (1 through n), where each task introduces a distinct visual concept (e.g., ”dog”, ”toy”, ”cat”, ”bear”). When learning the n-th task, the unique identifiers and category information of previously learned concepts, along with the current task’s prompt, are fed into the text encoder to obtain text emb… view at source ↗

**Figure 3.** Expert Coefficient Magnitudes Across Tasks and Network Layers. As task number increases, experts of middle layers receive higher coefficients (highlighted within the red box), while the remaining experts get smaller coefficients. This trend indicates the presence of redundant experts in the model. of trained experts while enforcing deterministic inclusion of the 0-th expert and the newly added expert. The detailed e… view at source ↗

Figures from the paper (4 more)

**Figure 5.** Figure 5: Inference pipeline for our proposed R2MoE framework. First, the initial text prompt is entered into GPT-4 to generate the corresponding region prompt and the bounding box. For t > rT, the model is guided by bounding boxes to control the spatial layout of generated imag…

**Figure 6.** Figure 6: Qualitative comparison with existing state-of-the-art customization methods. R 2MoE produces higher quality results with better subject similarity and text controllability compared to existing methods. For instance, our approach successfully generates images for the fi…

**Figure 7.** Figure 7: Qualitative Comparison for Multi-object Generation. We demonstrate that R2MoE generates multi-subject customized results with superior text controllability and subject fidelity compared to existing lifelong learning methods. For instance, in the first row, R2MoE succes…

**Figure 8.** Figure 8: Qualitative ablation study. The first row presents the ablation results for single-concept generation, while the second row shows the results for multi-concept generation. Fig.8 illustrates the impact of ablating each key component on generation results. The top row di…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Contextualized Multimodal Lifelong Person Re-Identification in Hybrid Clothing States
cs.CV 2025-09 reject novelty 3.0 of 10

CMLReID uses dynamic text prompts and dual-path prototypes to improve lifelong person re-identification in hybrid clothing states, reporting gains of about 5 mAP over four baselines.

Reference graph

Works this paper leans on

43 extracted references · 20 canonical work pages · cited by 1 Pith paper

[1]

High- resolution image synthesis with latent diffusion models,

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

2022
[2]

Sdxl: Improving latent diffusion models for high-resolution image synthesis,

D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, and R. Rombach, “Sdxl: Improving latent diffusion models for high-resolution image synthesis,” arXiv preprint arXiv:2307.01952 , 2023

arXiv 2023
[3]

Scaling rectified flow transformers for high-resolution image synthesis,

P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boesel et al. , “Scaling rectified flow transformers for high-resolution image synthesis,” in Forty-first international conference on machine learning , 2024

work page 2024
[4]

Photorealistic text-to-image diffusion models with deep language understanding,

C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al., “Photorealistic text-to-image diffusion models with deep language understanding,” Advances in neural information processing systems , vol. 35, pp. 36 479–36 494, 2022

2022
[5]

Denoising diffusion probabilistic models,

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

2020
[6]

An image is worth one word: Person- alizing text-to-image generation using textual inversion,

R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Person- alizing text-to-image generation using textual inversion,” arXiv preprint arXiv:2208.01618, 2022

arXiv 2022
[7]

Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,

N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 22 500–22 510

2023
[8]

Controlling text-to-image diffusion by orthogo- nal finetuning,

Z. Qiu, W. Liu, H. Feng, Y . Xue, Y . Feng, Z. Liu, D. Zhang, A. Weller, and B. Sch ¨olkopf, “Controlling text-to-image diffusion by orthogo- nal finetuning,” Advances in Neural Information Processing Systems , vol. 36, pp. 79 320–79 362, 2023

work page 2023

Show all 43 references

[9]

Personalized residuals for concept-driven text-to-image generation,

C. Ham, M. Fisher, J. Hays, N. Kolkin, Y . Liu, R. Zhang, and T. Hinz, “Personalized residuals for concept-driven text-to-image generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 8186–8195

2024
[10]

Sgdm: an adaptive style- guided diffusion model for personalized text to image generation,

Y . Xu, X. Xu, H. Gao, and F. Xiao, “Sgdm: an adaptive style- guided diffusion model for personalized text to image generation,” IEEE Transactions on Multimedia , vol. 26, pp. 9804–9813, 2024

2024
[11]

A two-stage personalized virtual try-on framework with shape control and texture guidance,

S. Zhang, M. Ni, S. Chen, L. Wang, W. Ding, and Y . Liu, “A two-stage personalized virtual try-on framework with shape control and texture guidance,” IEEE Transactions on Multimedia , vol. 26, pp. 10 225– 10 236, 2024

2024
[12]

Videodreamer: Customized multi-subject text-to-video generation with disen-mix finetuning on language-video foundation models,

H. Chen, X. Wang, G. Zeng, Y . Zhang, Y . Zhou, F. Han, Y . Wu, and W. Zhu, “Videodreamer: Customized multi-subject text-to-video generation with disen-mix finetuning on language-video foundation models,” IEEE Transactions on Multimedia , 2025

2025
[13]

Multi- concept customization of text-to-image diffusion,

N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J.-Y . Zhu, “Multi- concept customization of text-to-image diffusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 1931–1941

2023
[14]

Svdiff: Compact parameter space for diffusion fine-tuning,

L. Han, Y . Li, H. Zhang, P. Milanfar, D. Metaxas, and F. Yang, “Svdiff: Compact parameter space for diffusion fine-tuning,” arXiv preprint arXiv:2303.11305, 2023

2023 arXiv
[15]

Animediff: Customized image generation of anime characters using diffusion model,

Y . Jiang, Q. Liu, D. Chen, L. Yuan, and Y . Fu, “Animediff: Customized image generation of anime characters using diffusion model,” IEEE Transactions on Multimedia , 2024

2024
[16]

Mix-of-show: Decentralized low- rank adaptation for multi-concept customization of diffusion models,

Y . Gu, X. Wang, J. Z. Wu, Y . Shi, Y . Chen, Z. Fan, W. Xiao, R. Zhao, S. Chang, W. Wu et al. , “Mix-of-show: Decentralized low- rank adaptation for multi-concept customization of diffusion models,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024
[17]

Orthogonal adaptation for modular customization of diffusion models,

R. Po, G. Yang, K. Aberman, and G. Wetzstein, “Orthogonal adaptation for modular customization of diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 7964–7973

2024
[18]

Multi-lora composition for image generation,

M. Zhong, Y . Shen, S. Wang, Y . Lu, Y . Jiao, S. Ouyang, D. Yu, J. Han, and W. Chen, “Multi-lora composition for image generation,” CoRR, 2024

2024
[19]

Multi-view user preference modeling for personalized text-to-image generation,

H. Zhang, T. Wu, and Y . Wei, “Multi-view user preference modeling for personalized text-to-image generation,” IEEE Transactions on Mul- timedia, 2025

2025
[20]

Continual diffusion: Continual customization of text-to-image diffusion with c-lora,

J. S. Smith, Y .-C. Hsu, L. Zhang, T. Hua, Z. Kira, Y . Shen, and H. Jin, “Continual diffusion: Continual customization of text-to-image diffusion with c-lora,” Transactions on Machine Learning Research , 2024

2024
[21]

Create your world: Lifelong text-to-image diffusion,

G. Sun, W. Liang, J. Dong, J. Li, Z. Ding, and Y . Cong, “Create your world: Lifelong text-to-image diffusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

2024
[22]

How to continually adapt text-to-image diffusion models for flexible customization?

J. Dong, W. Liang, H. Li, D. Zhang, M. Cao, H. Ding, S. H. Khan, and F. Shahbaz Khan, “How to continually adapt text-to-image diffusion models for flexible customization?” Advances in Neural Information Processing Systems, vol. 37, pp. 130 057–130 083, 2024

2024
[23]

Conceptguard: Continual personalized text-to-image generation with forgetting and confusion mitigation,

Z. Guo and T. Jin, “Conceptguard: Continual personalized text-to-image generation with forgetting and confusion mitigation,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 2945–2954

2025
[24]

Catastrophic interference in connec- tionist networks: The sequential learning problem,

M. McCloskey and N. J. Cohen, “Catastrophic interference in connec- tionist networks: The sequential learning problem,” in Psychology of learning and motivation . Elsevier, 1989, vol. 24, pp. 109–165

1989
[25]

Gpt-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

2023 arXiv
[26]

Segment anything,

A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll´ar, and R. Girshick, “Segment anything,” arXiv:2304.02643, 2023

2023 arXiv
[27]

Instantbooth: Personalized text-to-image generation without test-time finetuning,

J. Shi, W. Xiong, Z. Lin, and H. J. Jung, “Instantbooth: Personalized text-to-image generation without test-time finetuning,” arXiv preprint arXiv:2304.03411, 2023

2023 arXiv
[28]

Hierarchical text-conditional image generation with clip latents,

A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022

2022 arXiv
[29]

Lora: Low-rank adaptation of large language models,

E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

2021 arXiv
[30]

Adaptive mixtures of local experts,

R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural computation, vol. 3, no. 1, pp. 79–87, 1991

1991
[31]

Outrageously large neural networks: The sparsely- gated mixture-of-experts layer,

N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely- gated mixture-of-experts layer,” 2017. [Online]. Available: https: //arxiv.org/abs/1701.06538

2017 arXiv
[32]

Mixture-of-loras: An efficient multitask tuning for large language models,

W. Feng, C. Hao, Y . Zhang, Y . Han, and H. Wang, “Mixture-of-loras: An efficient multitask tuning for large language models,” arXiv preprint arXiv:2403.03432, 2024

2024 arXiv
[33]

Mixture of lora experts,

X. Wu, S. Huang, and F. Wei, “Mixture of lora experts,” arXiv preprint arXiv:2404.13628, 2024

2024 arXiv
[34]

Expert gate: Lifelong learning with a network of experts,

R. Aljundi, P. Chakravarty, and T. Tuytelaars, “Expert gate: Lifelong learning with a network of experts,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3366– 3375

2017
[35]

Lifelong language pretraining with distribution-specialized experts,

W. Chen, Y . Zhou, N. Du, Y . Huang, J. Laudon, Z. Chen, and C. Cui, “Lifelong language pretraining with distribution-specialized experts,” in International Conference on Machine Learning . PMLR, 2023, pp. 5383–5395

2023
[36]

Boosting continual learning of vision-language models via mixture-of-experts adapters,

J. Yu, Y . Zhuge, L. Zhang, P. Hu, D. Wang, H. Lu, and Y . He, “Boosting continual learning of vision-language models via mixture-of-experts adapters,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 23 219–23 230

2024
[37]

Lifelong knowledge editing for vision language models with low-rank mixture- of-experts,

Q. Chen, C. Wang, D. Wang, T. Zhang, W. Li, and X. He, “Lifelong knowledge editing for vision language models with low-rank mixture- of-experts,” arXiv preprint arXiv:2411.15432 , 2024

2024 arXiv
[38]

Coin: A benchmark of continual instruction tuning for multimodel large language models,

C. Chen, J. Zhu, X. Luo, H. Shen, J. Song, and L. Gao, “Coin: A benchmark of continual instruction tuning for multimodel large language models,” Advances in Neural Information Processing Systems , 2024

2024
[39]

Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models,

D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu et al., “Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models,” arXiv preprint arXiv:2401.06066, 2024

2024 arXiv
[40]

An image is worth multiple words: Multi-attribute inversion for constrained text-to- image synthesis,

A. Agarwal, S. Karanam, T. Shukla, and B. V . Srinivasan, “An image is worth multiple words: Multi-attribute inversion for constrained text-to- image synthesis,” in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . IEEE, 2025, pp. 6053–6062

2025
[41]

Overcoming catastrophic forgetting in neural networks,

J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Pro- ceedings of the national academy of sciences , vol. 114, no. 13, pp. 3521...

2017
[42]

Learning without forgetting,

Z. Li and D. Hoiem, “Learning without forgetting,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 12, pp. 2935– 2947, 2017

2017
[43]

Classifier-free diffusion guidance,

J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022

2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

High- resolution image synthesis with latent diffusion models,

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

2022

[2] [2]

Sdxl: Improving latent diffusion models for high-resolution image synthesis,

D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, and R. Rombach, “Sdxl: Improving latent diffusion models for high-resolution image synthesis,” arXiv preprint arXiv:2307.01952 , 2023

arXiv 2023

[3] [3]

Scaling rectified flow transformers for high-resolution image synthesis,

P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boesel et al. , “Scaling rectified flow transformers for high-resolution image synthesis,” in Forty-first international conference on machine learning , 2024

work page 2024

[4] [4]

Photorealistic text-to-image diffusion models with deep language understanding,

C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al., “Photorealistic text-to-image diffusion models with deep language understanding,” Advances in neural information processing systems , vol. 35, pp. 36 479–36 494, 2022

2022

[5] [5]

Denoising diffusion probabilistic models,

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

2020

[6] [6]

An image is worth one word: Person- alizing text-to-image generation using textual inversion,

R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Person- alizing text-to-image generation using textual inversion,” arXiv preprint arXiv:2208.01618, 2022

arXiv 2022

[7] [7]

Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,

N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 22 500–22 510

2023

[8] [8]

Controlling text-to-image diffusion by orthogo- nal finetuning,

Z. Qiu, W. Liu, H. Feng, Y . Xue, Y . Feng, Z. Liu, D. Zhang, A. Weller, and B. Sch ¨olkopf, “Controlling text-to-image diffusion by orthogo- nal finetuning,” Advances in Neural Information Processing Systems , vol. 36, pp. 79 320–79 362, 2023

work page 2023