Pith. sign in

REVIEW 5 major objections 5 minor 36 references

DivControl: Knowledge Diversion for Controllable Image Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DivControl decomposes a ControlNet into shared and per-condition modules, cutting training cost 36.4x while improving control quality.

desk verdict Plausible modular-ControlNet idea with a real but narrow experimental base; the abstract oversells efficiency and the zero-shot evidence has a table/taxonomy mismatch. read the letter →

arxiv 2507.23620 v1 pith:GASZPICY submitted 2025-07-31 cs.CV cs.LG

Yucheng Xie , Fu Feng , Ruixiao Shi , Jing Wang , Yong Rui , Xin Geng This is my paper
classification cs.CVcs.LG
keywords controllableimagegenerationknowledgediversionControlNetSVDdecompositiondynamicgatezero-shotgeneralizationrepresentationalignmentdiffusiontransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DivControl claims that the weights of a ControlNet-style condition encoder can be split by singular value decomposition into two reusable families: condition-agnostic components (learngenes) shared by all inputs and condition-specific components (tailors) selected per condition. A lightweight dynamic gate reads a text description of the condition and softly activates the relevant tailors, so a condition never seen in training can be generated zero-shot if its instruction embedding resembles trained ones, and can be adapted by retraining only the tailors otherwise. The paper reports that this pretraining scheme beats prior unified controllers on eight base conditions while using 165 GPU hours instead of 6,000, and reaches state-of-the-art fidelity on high-shift unseen conditions with 200 images and about 0.23 GPU hours of finetuning. The reason to care is that it converts per-condition retraining into modular assembly, which changes the economics of adding new control modalities.

What carries the argument

The load-bearing object is the SVD-factorized weight matrix $W_\star^{(l)} = \sum_i u^{(l,i)}_\star \sigma^{(l,i)}_\star {v^{(l,i)}_\star}^\top$, whose rank-1 components $\Theta^{(l,i)}_\star$ are partitioned into learngenes and tailors. The dynamic gate computes $\alpha = \mathrm{softmax}(\mathcal{G}(e_{\text{txt}}))$ from a text-encoded condition instruction and combines tailors as $\widehat{W} = G + \sum_k \alpha \cdot T_k$, which is what lets a novel condition's embedding pick out semantically related tailors. The representation alignment loss $L_{\text{REPA}}$ pulls early ControlNet features toward a frozen vision encoder embedding of the condition image, improving convergence and control fidelity.

What would settle it

Take a held-out condition, randomly permute the words of its condition instruction (or replace it with an unrelated instruction), and measure zero-shot CLIP-I or LPIPS. If output fidelity does not drop, the semantic routing claim is false because the gate is not actually using instruction semantics. Conversely, if severing individual tailors changes only the associated condition's outputs, disentanglement is confirmed; if all conditions degrade equally, the split is not condition-specific.

Watch

Extended reading notes

Core claim

The paper's central claim is that explicit knowledge diversion inside a diffusion condition encoder yields both better controllability and better transferability than monolithic or LoRA-style unified controllers. Each weight matrix is written as a sum of rank-1 SVD components; the components are partitioned into shared learngenes and condition-specific tailors, and training routes condition embeddings through a soft gate to a weighted combination of tailors. The same trained model then handles seen conditions with improved fidelity, unseen low-shift conditions without gradient updates, and unseen high-shift conditions by initializing fresh tailors while freezing everything else. The paper also introduces a representation alignment loss that matches an early ControlNet feature to a frozen vision encoder embedding of the condition image, accelerating convergence and tightening condition fidelity.

Load-bearing premise

The central load-bearing premise is that the SVD pieces of each weight matrix naturally separate into condition-agnostic and condition-specific knowledge, and that a text embedding of a condition instruction can route an unseen condition to the correct tailors; the paper measures routing behavior only indirectly through gate-activation similarities and does not directly verify component disentanglement.

Editorial extensions

If this is right

  • Adding a new condition costs about 0.23 GPU hours on 200 images (tailor-only finetuning) instead of hundreds of GPU hours per condition.
  • A single pretrained model can generate under multiple simultaneous conditions by aggregating their instruction embeddings and activating the corresponding tailors.
  • Zero-shot generation works for conditions semantically close to training conditions, such as LINEART derived from SKETCH, without any gradient update.
  • Because only tailors are condition-specific, the shared learngenes can be reused for future conditions, so the cost of expanding the condition set does not grow with full model size.
  • Joint training with REPA improves convergence, so shorter pretraining reaches or exceeds the fidelity of much longer training runs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The semantic-routing hypothesis is testable beyond this architecture: if instruction embeddings genuinely select tailors, then interpolating between two condition embeddings should produce outputs that mix the two controls, which the paper does not demonstrate.
  • Disentanglement also suggests an editing primitive: suppressing a condition's behavior could be done by zeroing its gate weight or tailors rather than retraining, a safety-relevant operation the paper does not explore.
  • A direct measurement of component interchangeability, such as swapping tailors across conditions and measuring fidelity loss, would quantify how disentangled the SVD split really is; the gate-activation similarity in Figure 6 is only indirect evidence.
  • The 36.4x cost reduction compares against CtrLoRA's reported 6000 GPU hours; a matched-compute comparison would clarify how much of the gain comes from the decomposition itself versus the shorter schedule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. DivControl proposes a decomposable pretraining framework for controllable image generation. It applies SVD to factorize a ControlNet into shared 'learngenes' and condition-specific 'tailors,' routes conditions through a dynamic gate that reads text-embedding condition instructions, and adds a representation alignment loss (REPA) that aligns condition embeddings with early diffusion features. The method is trained on eight base conditions and evaluated on eight novel conditions. The paper claims state-of-the-art controllability with a 36.4x reduction in training cost compared to CtrLoRA, as well as strong zero-shot and few-shot generalization to unseen conditions.

Significance. The idea of modularizing a control network into reusable and condition-specific components is timely and potentially useful for scalable controllable generation. The paper provides a concrete training algorithm, a benchmark covering multiple control conditions, and an ablation of the main components. The dynamic gate conditioned on text embeddings is a clean mechanism for zero-shot routing. However, the current experimental support for the headline claims is incomplete: the cost-reduction claim is based on a selective baseline comparison, the zero-shot evidence omits two of the six stated low-shift conditions and the reported table caption is internally inconsistent, and a key hyperparameter appears in contradictory forms. These issues are fixable, but they currently prevent the paper from fully establishing its central claims.

major comments (5)
  1. [Abstract, Section 5.1, Table 1] The 36.4x training-cost reduction claim is based on comparing DivControl (165 GPU hours) to CtrLoRA-SD (6000 GPU hours), while the same table reports CtrLoRA-PA at the identical 165 GPU hours. Because a same-cost baseline exists and is outperformed only modestly, the abstract and Figure 1(d) should qualify the claim as a comparison against CtrLoRA-SD specifically, and the discussion in Section 5.1 should explicitly address the margin over CtrLoRA-PA at equal cost.
  2. [Table 2, Figure B.1, Section 5.2.1] The zero-shot generalization claim is not quantitatively supported for all low-shift conditions. Figure B.1 defines six low-shift novel conditions (ILLUSION, GRAYSCALE, BLUR, INPAINTING, LINEART, BRUSH), but Table 2 reports numbers only for BLUR, BRUSH, GRAYSCALE, and INPAINTING, omitting ILLUSION and LINEART. Moreover, Table 2's caption states '6 high-shift novel conditions and 2 low-shift novel conditions,' which contradicts both Figure B.1 and the table's own columns (4 low-shift and 4 high-shift). The averaged row therefore mixes conditions of different shift types and cannot by itself substantiate the abstract's 'strong zero-shot' claim. Please add quantitative results for ILLUSION and LINEART, correct the caption, and report low-shift and high-shift averages separately.
  3. [Section 4.2, Table B.1] There is a direct inconsistency in the core hyperparameters: the main text sets the number of learngenes N_G and tailors N_T to 576 with 288 active tailor components, while Table B.1 reports N_G=512 and N_T=512. Since these values determine the SVD rank r=N_G+N_T and the gating budget, this discrepancy must be resolved before the results can be reproduced.
  4. [Section 3.2.2, Figure 6] The central assumption that SVD components separate into condition-agnostic and condition-specific semantics is not directly measured. Figure 6 shows that gate activations cluster for semantically related conditions, which is indirect behavioral evidence; it does not establish that the learned SVD components themselves are disentangled or that text-embedding routing will generalize to arbitrary unseen instructions. I recommend adding a direct analysis of component specialization (e.g., which tailors are active for which conditions, whether each tailor has consistent semantics) and reporting zero-shot results for all six low-shift conditions, including LINEART and ILLUSION.
  5. [Section 5.2.2, Table 2] The averaged row in Table 2 mixes zero-shot low-shift conditions (BLUR, BRUSH, GRAYSCALE, INPAINTING) with few-shot high-shift conditions (JPEG, PALETTE, PIXEL, SHUFFLE). Consequently, the reported '+1.72 CLIP-I improvement over CtrLoRA' cannot be attributed to either zero-shot or few-shot performance. Please report separate averages for the two groups, and clarify which entries in the table are zero-shot versus finetuned.
minor comments (5)
  1. [Eq. (4)] In Eq. (4), the notation mixes α in R^{N_T} with a sum over K active tailors; please clarify the indexing (e.g., α_k) and how the top-K selected by Eq. (A.1) enter the weighted sum.
  2. [Algorithm 1, line 10] Line 10 uses G(∇_U L_batch) to update tailors, but G already denotes the dynamic gate; please rename the gate or clarify that G here is a gradient mask/selection operator.
  3. [Table 4] The metric FDD appears in Table 4 but is not defined in the text; please define it or replace with a standard metric.
  4. [Introduction, Section 1] The phrase 'with average CLIP-I gains of 0.05' appears to be a typo or an underspecified quantity; Table 1 shows gains of about 0.4 percentage points on average CLIP-I, so please correct the number or clarify the units.
  5. [Figure 6] The figure reports 'inter-condition similarity derived from dynamic gate activations' but does not specify the similarity metric or how the 10x10 matrix is normalized; please add a definition in the caption or text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the zero-shot claim is an empirical held-out evaluation, and self-citations to KIND supply a framework rather than a constructed equivalence.

full rationale

DivControl's central pipeline is: factorize ControlNet weights via SVD (a mathematical identity), learn a dynamic gate mapping condition-instruction text embeddings to tailor weights, train with the denoising loss plus REPA, and then apply the trained gate to novel conditions. No equation in the paper defines the claimed prediction (e.g., zero-shot routing for an unseen condition) in terms of the fitted gate output by construction; the novel conditions are genuinely not among the eight base conditions used for training, so evaluating the gate on them is an external generalization test rather than a restatement of the training objective. The SVD decomposition and the learngene/tailor dichotomy are adopted from KIND [30], a same-author citation, but the paper does not rely on KIND as an unverified uniqueness or existence theorem: Table 3 provides an internal ablation showing that the diversion mechanism itself changes performance, and the main results are measured against external baselines (CtrLoRA, UniControl, PixArt) on COCO2017. The REPA loss aligns DINOv2 condition embeddings with early ControlNet features; this is a training regularizer, not a direct optimization of the evaluation metrics (LPIPS/SSIM/CLIP-I) on generated images, so the reported controllability numbers are not fitted inputs renamed as predictions. The most serious concerns in the paper are evidentiary rather than circular: Table 2's caption states '6 high-shift novel conditions and 2 low-shift novel conditions' while the table reports four low-shift and four high-shift columns, and two of the six low-shift conditions named in Figure B.1 (LINEART, ILLUSION) receive only qualitative zero-shot results. These inconsistencies weaken the strength of the zero-shot generalization claim but do not make the derivation equivalent to its inputs by construction. Overall, the paper's claimed contributions are empirically testable and tested against held-out conditions and external baselines, so no significant circularity is present.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central premise is that SVD-based factorization creates reusable and separable modules, which is an empirical assumption rather than a proven property of the decomposition. No new physical or scientific entities are introduced; 'learngenes' and 'tailors' are re-used conceptual constructs from prior work. The free parameters are mostly hand-chosen hyperparameters, and the inconsistency between sections reduces confidence in the reported configurations.

free parameters (6)
  • NG (number of learngenes) = 512 (Appendix B.2), stated as 576 in Section 4.2
    Determines the capacity of shared condition-agnostic knowledge; chosen by hand and inconsistent across sections.
  • NT (number of tailors) = 512 (Appendix B.2), stated as 576 in Section 4.2
    Determines the capacity of condition-specific modules; chosen by hand and inconsistent across sections.
  • active tailor components per condition (K) = 288
    Top-K threshold for the dynamic gate; chosen by hand to balance reuse and specificity.
  • REPA loss weight lambda = 0.05
    Tuned via ablation in Table 4; not predicted from theory.
  • REPA alignment depth (layer) = layer 4
    Chosen after ablation across depths in Table 4.
  • training steps = 450K
    Set to fit the advertised 165 GPU hour budget; no convergence criterion justified.
assumptions (5)
  • domain assumption SVD-style factorization of weight matrices yields components that can be meaningfully assigned to condition-agnostic and condition-specific knowledge
    Assumed throughout Section 3.2.1 and Algorithm 1. If the components do not separate knowledge, the gating mechanism cannot work as intended.
  • domain assumption Pretrained text encoder embeddings of condition instructions capture semantic relations between conditions
    Assumed in Section 3.2.2 and used for zero-shot routing in Eq. (7). The paper's zero-shot claims depend on this without validating the embedding space.
  • domain assumption Training on eight basic conditions on Subject200K is sufficient to learn transferable and generalizable components
    Underlies the unified control and transfer claims; no analysis of dataset sufficiency or condition diversity is provided.
  • domain assumption REPA alignment with DINOv2 features improves condition fidelity
    Borrowed from prior work (REPA); the paper provides an ablation but no mechanistic explanation for why it should improve control.
  • standard math Standard neural network training assumptions (convergence of AdamW, appropriate learning rate, etc.)
    Routine for deep learning experiments; no special justification needed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DivControl: Knowledge Diversion for Controllable Image Generation." pith.science (2026). https://pith.science/paper/GASZPICY

@misc{pith2026250723620,
  author       = {Pith},
  title        = {Pith review of: DivControl: Knowledge Diversion for Controllable Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GASZPICY}},
  note         = {Machine review of arXiv:2507.23620}
}
abstract

Diffusion models have advanced from text-to-image (T2I) to image-to-image (I2I) generation by incorporating structured inputs such as depth maps, enabling fine-grained spatial control. However, existing methods either train separate models for each condition or rely on unified architectures with entangled representations, resulting in poor generalization and high adaptation costs for novel conditions. To this end, we propose DivControl, a decomposable pretraining framework for unified controllable generation and efficient adaptation. DivControl factorizes ControlNet via SVD into basic components-pairs of singular vectors-which are disentangled into condition-agnostic learngenes and condition-specific tailors through knowledge diversion during multi-condition training. Knowledge diversion is implemented via a dynamic gate that performs soft routing over tailors based on the semantics of condition instructions, enabling zero-shot generalization and parameter-efficient adaptation to novel conditions. To further improve condition fidelity and training efficiency, we introduce a representation alignment loss that aligns condition embeddings with early diffusion features. Extensive experiments demonstrate that DivControl achieves state-of-the-art controllability with 36.4$\times$ less training cost, while simultaneously improving average performance on basic conditions. It also delivers strong zero-shot and few-shot performance on unseen conditions, demonstrating superior scalability, modularity, and transferability.

Figures

Figures reproduced from arXiv: 2507.23620 by the authors.

Figure 1
Figure 1. (a) Traditional ControlNet requires training a dedicated [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the DivControl. (a) Each weight matrix in ControlNet is factorized via SVD into condition-agnostic learngenes and condition-specific tailors. A dynamic gate routes each input to relevant tailors while jointly updating shared learngenes, enabling modular and disentangled representation across conditions. (b) Shallow features in ControlNet are aligned with condition semantics extracted by a pre-trained vis… view at source ↗
Figure 3
Figure 3. Generalization to Novel Control Conditions. (a) For low-shift conditions, DivControl leverages instruction embed￾dings to dynamically activate semantically aligned tailors, en￾abling zero-shot generation. (b) For high-shift conditions, it reuses condition-agnostic learngenes while introducing randomly initial￾ized tailors, supporting efficient few-shot adaptation. types ⋆ ∈ S. These coefficients modulate tailor comp… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Zero-shot Generalization on Low-shift Novel Conditions. We visualize qualitative results on unseen conditions that are semantically aligned with training conditions. DivControl achieves effective zero-shot generation by leveraging knowledge diversion and dynamic gating…
Figure 5
Figure 5. Figure 5: Multi-Conditional Controllable Image Generation. DivControl leverages knowledge diversion to encapsulate condition￾specific knowledge into tailors, enabling flexible composition of multiple conditions. This facilitates high-fidelity, semantically aligned generation und…
Figure 6
Figure 6. Figure 6: Inter-condition similarity derived from dynamic gate ac [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 23 canonical work pages

  1. [1]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022. 1, 2

    arXiv 2022

  2. [2]

    Pixart-δ: Fast and controllable image generation with latent consistency mod- els

    Junsong Chen, Simian Luo, and Enze Xie. Pixart-δ: Fast and controllable image generation with latent consistency mod- els. InICML 2024 Workshop on Theoretical Foundations of Foundation Models. 2, 3, 5

    work page 2024

  3. [3]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of International Conference on Machine Learn- ing (ICML’24), pages 1–13, 2024. 1, 3

    work page 2024

  4. [4]

    Genes in Intelligent Agents

    Fu Feng, Jing Wang, Congzhi Zhang, Wenqian Li, Xu Yang, and Xin Geng. Genes in intelligent agents.arXiv preprint arXiv:2306.10225, 2023. 3

    work page Pith review arXiv 2023

  5. [5]

    Transferring core knowledge via learngenes.arXiv preprint arXiv:2401.08139,

    Fu Feng, Jing Wang, and Xin Geng. Transferring core knowledge via learngenes.arXiv preprint arXiv:2401.08139,

    arXiv

  6. [6]

    Wave: Weight template for adaptive initialization of variable-sized models

    Fu Feng, Yucheng Xie, Jing Wang, and Xin Geng. Wave: Weight template for adaptive initialization of variable-sized models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’25), pages 1–10, 2025. 3

    work page 2025

  7. [7]

    Lora: Low- rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low- rank adaptation of large language models. InProceedings of the International Conference on Learning Representations (ICLR’22), pages 1–13, 2022. 2

    work page 2022

  8. [8]

    Cock- tail: Mixing multi-modality control for text-conditional im- age generation

    Minghui Hu, Jianbin Zheng, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, Dacheng Tao, and Tat-Jen Cham. Cock- tail: Mixing multi-modality control for text-conditional im- age generation. InThirty-seventh Conference on Neural In- formation Processing Systems, 2023. 2

    work page 2023

Show all 36 references
  1. [9]

    No other representation component is needed: Diffusion transformers can provide representation guidance by themselves.arXiv preprint arXiv:2505.02831,

    Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, and Jingdong Wang. No other representation component is needed: Diffusion transformers can provide representation guidance by themselves.arXiv preprint arXiv:2505.02831,

  2. [10]

    Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers

    Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504.10483, 2025. 4

    2025

  3. [11]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  4. [12]

    Midjourney.com.https : / / www

    Midjourney. Midjourney.com.https : / / www . midjourney.com, 2022. Accessed: 2024-11-14. 1

    2022

  5. [13]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InProceedings of the AAAI conference on artificial intelligence, pages 4296–4304, 202...

    2024

  6. [14]

    Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models. InInternational Conference on Machine Learning, pa...

    2022

  7. [15]

    Gpt-4: Openai language model.https:// openai.com/research/gpt-4, 2023

    OpenAI. Gpt-4: Openai language model.https:// openai.com/research/gpt-4, 2023. 5

    2023

  8. [16]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research Jou...

    2024

  9. [17]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV’23), pages 4195–4205, 2023. 1, 2, 3

    2023

  10. [18]

    Unicontrol: A unified diffu- sion model for controllable visual generation in the wild

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffu- sion model for controllable visual generation in the wild. In NeurIPS, 2023. 2, 3, 5, 6

    2023

  11. [19]

    Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1

    2022 arXiv

  12. [20]

    Scaling vision with sparse mix- ture of experts.Advances in Neural Information Processing Systems, 34:8583–8595, 2021

    Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, Andr ´e Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mix- ture of experts.Advances in Neural Information Processing Systems, 34:8583–8595, 2021. 2, 3

    2021

  13. [21]

    Flux.1-dev-controlnet-union-pro.https: / / huggingface

    Shakker Labs. Flux.1-dev-controlnet-union-pro.https: / / huggingface . co / Shakker - Labs / FLUX . 1 - dev-ControlNet-Union-Pro, 2024. 5

    2024

  14. [22]

    Fad: Frequency adaptation and diversion for cross- domain few-shot learning.arXiv preprint arXiv:2505.08349,

    Ruixiao Shi, Fu Feng, Yucheng Xie, Jing Wang, and Xin Geng. Fad: Frequency adaptation and diversion for cross- domain few-shot learning.arXiv preprint arXiv:2505.08349,

    arXiv

  15. [23]

    Ominicontrol: Minimal and uni- versal control for diffusion transformer.arXiv preprint arXiv:2411.15098, 2024

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and uni- versal control for diffusion transformer.arXiv preprint arXiv:2411.15098, 2024. 2, 5

    2024 arXiv

  16. [24]

    U-repa: Aligning diffu- sion u-nets to vits.arXiv preprint arXiv:2503.18414, 2025

    Yuchuan Tian, Hanting Chen, Mengyu Zheng, Yuchen Liang, Chao Xu, and Yunhe Wang. U-repa: Aligning diffu- sion u-nets to vits.arXiv preprint arXiv:2503.18414, 2025. 4

    2025

  17. [25]

    Unicombine: Unified multi-conditional combination with diffusion transformer.arXiv preprint arXiv:2503.09277, 2025

    Haoxuan Wang, Jinlong Peng, Qingdong He, Hao Yang, Ying Jin, Jiafu Wu, Xiaobin Hu, Yanjie Pan, Zhenye Gan, Mingmin Chi, et al. Unicombine: Unified multi-conditional combination with diffusion transformer.arXiv preprint arXiv:2503.09277, 2025. 2

    2025 arXiv

  18. [26]

    Auxiliary-loss-free load balancing strategy for mixture-of-experts.arXiv preprint arXiv:2408.15664, 2024

    Lean Wang, Huazuo Gao, Chenggang Zhao, Xu Sun, and Damai Dai. Auxiliary-loss-free load balancing strategy for mixture-of-experts.arXiv preprint arXiv:2408.15664, 2024. 11

    2024 arXiv

  19. [27]

    Learngene: From open-world to your learning task

    QiuFeng Wang, Xin Geng, ShuXia Lin, Shi-Yu Xia, Lei Qi, and Ning Xu. Learngene: From open-world to your learning task. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI’22), pages 8557–8565, 2022. 3

    2022

  20. [28]

    Learngene: Inheriting condensed knowledge from the ancestry model to descendant models.arXiv preprint arXiv:2305.02279, 2023

    Qiufeng Wang, Xu Yang, Shuxia Lin, and Xin Geng. Learngene: Inheriting condensed knowledge from the ancestry model to descendant models.arXiv preprint arXiv:2305.02279, 2023. 3

    2023 arXiv

  21. [29]

    Representation entanglement for genera- tion: Training diffusion transformers is much easier than you think.arXiv preprint arXiv:2507.01467, 2025

    Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, et al. Representation entanglement for genera- tion: Training diffusion transformers is much easier than you think.arXiv preprint arXiv:2507.01467, 2025. 4

    2025

  22. [30]

    Kind: Knowledge integration and diversion for training decomposable models

    Yucheng Xie, Fu Feng, Ruixiao Shi, Jing Wang, Yong Rui, and Xin Geng. Kind: Knowledge integration and diversion for training decomposable models. InForty-second Interna- tional Conference on Machine Learning, 2025. 2, 3

    2025

  23. [31]

    Ctrlora: An extensible and efficient framework for control- lable image generation.arXiv preprint arXiv:2410.09400,

    Yifeng Xu, Zhenliang He, Shiguang Shan, and Xilin Chen. Ctrlora: An extensible and efficient framework for control- lable image generation.arXiv preprint arXiv:2410.09400,

    arXiv

  24. [32]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. InProceedings of the International Conference on Learning Representations (I...

    2024

  25. [33]

    Controlnet-xs: Rethinking the control of text-to- image diffusion models as feedback-control systems

    Denis Zavadski, Johann-Friedrich Feiden, and Carsten Rother. Controlnet-xs: Rethinking the control of text-to- image diffusion models as feedback-control systems. In European Conference on Computer Vision, pages 343–362. Springer, 2024. 1, 2, 3

    2024

  26. [34]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’23), pages 3836–3847, 2023. 1, 2, 3

    2023

  27. [35]

    Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Sys- tems, 36:11127–11150, 2023

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Sys- tems, 36:11127–11150, 2023. 2

    2023

  28. [36]

    Mixture-of-experts with expert choice routing.Ad- vances in Neural Information Processing Systems, 35:7103– 7114, 2022

    Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M Dai, Quoc V Le, James Laudon, et al. Mixture-of-experts with expert choice routing.Ad- vances in Neural Information Processing Systems, 35:7103– 7114, 2022. 2, 3 A. More Details on Methods A.1. Dyn...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.