Cross-attention maps serve as a tractable surrogate for manifold proximity, enabling automatic synthesis of anchors that suppress normal-space drift in diffusion unlearning.
Diffusion models already have a semantic latent space.arXiv preprint arXiv:2210.10960
13 Pith papers cite this work. Polarity classification is still indexing.
13
Pith papers citing it
abstract
Diffusion models achieve outstanding generative performance in various domains. Despite their great success, they lack semantic latent space which is essential for controlling the generative process. To address the problem, we propose asymmetric reverse process (Asyrp) which discovers the semantic latent space in frozen pretrained diffusion models. Our semantic latent space, named h-space, has nice properties for accommodating semantic image manipulation: homogeneity, linearity, robustness, and consistency across timesteps. In addition, we introduce a principled design of the generative process for versatile editing and quality boost ing by quantifiable measures: editing strength of an interval and quality deficiency at a timestep. Our method is applicable to various architectures (DDPM++, iD- DPM, and ADM) and datasets (CelebA-HQ, AFHQ-dog, LSUN-church, LSUN- bedroom, and METFACES). Project page: https://kwonminki.github.io/Asyrp/
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
Jeffrey guidance applies Jeffrey's rule of conditioning to diffusion models to target prescribed marginal distributions while preserving conditional structure, demonstrated via embedding matching and fairness enforcement.
Base-Anchored Filtering suppresses weakly backbone-aligned LoRA spectral channels to cut memorization while preserving or improving generation quality, without data or re-training.
Higher-variance classes are learned first in diffusion models; strong class imbalance reverses the order and imposes distinct delayed learning times on minority classes.
Diffusion models show grokking on modular addition by composing periodic operand representations in simple data regimes or by separating arithmetic computation from visual denoising across timesteps in varied regimes.
Feature self-guidance disperses internal features of flow models during batch generation and applies manifold regularization to increase output diversity while preserving condition alignment.
AnchorSteer couples self-discovered semantic concept vectors with structural anchoring in diffusion models to achieve controllable music editing with preserved structure.
SafeMark integrates a thresholded watermark-decoding loss into diffusion editors to enable text-guided edits that preserve embedded watermarks with high bit accuracy.
ConceptAgent is a black-box multi-agent system that awakens erased concepts in diffusion models by initializing denoising trajectories from surrogate-guided noisy states.
Gaussian probing infers harmful model specialization from parameter perturbations and internal representation responses to Gaussian latent ensembles rather than from generated outputs.
Latent diffusion models exhibit geometric decoupling where curvature in out-of-distribution generation is misallocated to unstable semantic boundaries instead of image details, identifying geometric hotspots as the structural cause of editing instability.
Text Slider uses LoRA adapters on pre-trained text encoders to identify low-rank directions for efficient, plug-and-play continuous concept control in diffusion-based image and video synthesis.
Self-supervised representation conditioning on diffusion models boosts unconditional generation quality and yields a controllable space with smooth, disentangled variations.
citing papers explorer
-
AutoAnchor: Stable Diffusion Unlearning Using Cross-Attention as a Manifold Surrogate
Cross-attention maps serve as a tractable surrogate for manifold proximity, enabling automatic synthesis of anchors that suppress normal-space drift in diffusion unlearning.
-
Towards More General Control of Diffusion Models Using Jeffrey Guidance
Jeffrey guidance applies Jeffrey's rule of conditioning to diffusion models to target prescribed marginal distributions while preserving conditional structure, demonstrated via embedding matching and fairness enforcement.
-
Filtering Memorization from Parameter-Space in Diffusion Models
Base-Anchored Filtering suppresses weakly backbone-aligned LoRA spectral channels to cut memorization while preserving or improving generation quality, without data or re-training.
-
The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models
Higher-variance classes are learned first in diffusion models; strong class imbalance reverses the order and imposes distinct delayed learning times on minority classes.
-
Grokking of Diffusion Models: Case Study on Modular Addition
Diffusion models show grokking on modular addition by composing periodic operand representations in simple data regimes or by separating arithmetic computation from visual denoising across timesteps in varied regimes.
-
Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance
Feature self-guidance disperses internal features of flow models during batch generation and applies manifold regularization to increase output diversity while preserving condition alignment.
-
AnchorSteer: Self-Discovered Concept Injection for Structure-Preserving Music Editing
AnchorSteer couples self-discovered semantic concept vectors with structural anchoring in diffusion models to achieve controllable music editing with preserved structure.
-
Are Watermarked Images Editable? SafeMark for Watermark-Preserving Text-Guided Image Editing
SafeMark integrates a thresholded watermark-decoding loss into diffusion editors to enable text-guided edits that preserve embedded watermarks with high bit accuracy.
-
Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Framework
ConceptAgent is a black-box multi-agent system that awakens erased concepts in diffusion models by initializing denoising trajectories from surrogate-guided noisy states.
-
Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM
Gaussian probing infers harmful model specialization from parameter perturbations and internal representation responses to Gaussian latent ensembles rather than from generated outputs.
-
Geometric Decoupling: Diagnosing the Structural Instability of Latent
Latent diffusion models exhibit geometric decoupling where curvature in out-of-distribution generation is misallocated to unstable semantic boundaries instead of image details, identifying geometric hotspots as the structural cause of editing instability.
-
Text Slider: Efficient and Plug-and-Play Continuous Concept Control for Image/Video Synthesis via LoRA Adapters
Text Slider uses LoRA adapters on pre-trained text encoders to identify low-rank directions for efficient, plug-and-play continuous concept control in diffusion-based image and video synthesis.
-
Towards Controllable Image Generation through Representation-Conditioned Diffusion Models
Self-supervised representation conditioning on diffusion models boosts unconditional generation quality and yields a controllable space with smooth, disentangled variations.