REVIEW 3 major objections 5 minor 80 references
Without retraining, AnchorSteer improves text-to-image faithfulness by anchoring prompt semantics in the initial noise and adding a mid-generation 'Think-Erase-Retouch' correction loop, lifting GenEval SDXL from 55.9 to 63.4.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
AnchorSteer improves text-to-image faithfulness by anchoring initial noise with CLIP/DAS-derived semantics (LP-SDS) and correcting errors during denoising with a VLM-driven Think-Erase-Retouch loop.
T0 review reviewed 2026-08-01 challenge →
load-bearing objection Practical training-free alignment recipe with real benchmark gains, but the anchoring mechanism is oversold and the experimental controls are not yet tight enough. the 3 major comments →
Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Time
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The central claim is that the two sources of compositional failure—an uninformative initial noise and an uncorrected denoising trajectory—can be addressed jointly at inference time by injecting semantics into the initial latent and by making the trajectory self-corrective. The paper advances a specific mechanism: distill CLIP's semantic prior into the diffusion latent space via Latent-Prior Score Distillation Sampling, then diffuse that refined latent forward to produce an initial noise that is both text-aligned and distribution-consistent. Along the trajectory, Think-Erase-Retouch uses a vision-language model to turn misalignment into two actionable signals—a negative prompt for content to
What carries the argument
The load-bearing mechanism is LP-SDS (Latent-Prior Score Distillation Sampling), an optimization objective that distills CLIP visual priors into the diffusion model's latent distribution by following the frozen diffusion model's noise-prediction residual; the paper shows this is equivalent to minimizing a time-weighted KL divergence between the perturbed latent distribution and the model's conditional distribution, pulling the prior onto the diffusion manifold. The second mechanism is the Think-Erase-Retouch steering loop: a VLM decodes the Tweedie-predicted clean image, separates errors into missing (D_pos) and inconsistent (D_neg) prompts, erases via negative-guided DDIM inversion, and ret
Load-bearing premise
The core bet is that after DDPM forward noising to step T=50, the terminal latent still carries enough of the refined semantic prior to steer generation; if diffusion erases that signal, Semantic Anchoring contributes little and the reported gains rest almost entirely on Reflective Steering.
What would settle it
Run AnchorSteer on GenEval with Semantic Anchoring disabled: sample z_T purely as N(0,I) (keeping the same seed and Reflective Steering unchanged). If the overall score does not drop materially below 63.39 (or if it matches the +Steer-only 59.87), the claim that semantic information survives forward diffusion is falsified. A complementary check: measure the distance between the empirical distribution of z_T and N(0,I) or test whether shuffling the latent's spatial layout (as in Table S5) removes the gain.
If this is right
- Semantic Anchoring alone raises structural and attribute metrics (Counting up by about 14 points on SDXL GenEval, Attribute Binding by 2.75), indicating that prompt-aligned initial noise reduces early-stage semantic drift.
- Reflective Steering alone boosts relational reasoning (Two Objects up by 10.6 points on SDXL), showing that mid-generation diagnosis and correction prevent error propagation.
- The two components are complementary: full AnchorSteer beats either component alone on both SDXL and HunyuanDiT, so initialization and trajectory control address different failure modes.
- Gains transfer across architectures—U-Net and DiT backbones both improve—suggesting the method is not tied to one model family.
- The method costs about 68.5 seconds per image on SDXL versus 3.7 seconds for standard sampling, but it also improves a distilled four-step model, pointing to a viable speed-quality trade-off.
Where Pith is reading between the lines
- If the semantic-leakage premise holds more broadly, LP-SDS could become a general prior-injection primitive for other conditioning signals (depth, color palettes, style) that suffer the same CLIP-to-diffusion domain gap.
- The Think-Erase-Retouch loop suggests a modular test for controllability: the same VLM diagnosis could drive different correction policies (e.g., attention-based erasure instead of guided inversion), allowing component-wise isolation of what matters—diagnosis vs. correction.
- Table S5 hints that the anchored initialization's benefit is not a mere distribution-shift artifact: a latent from a mismatched prompt performs no better than Gaussian, while the prompt-matched latent does. A stronger test would vary how much semantic signal survives by starting the noising process from earlier steps.
- Because the VLM runs on decoded intermediate images, the method's ceiling is partly bounded by the VLM's diagnostic accuracy; performance differences across VLMs may track diagnosis quality more than diffusion backbone choice—a testable prediction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes AnchorSteer, a training-free, inference-time framework for improving text-image alignment in text-to-image diffusion models. It consists of two components: Semantic Anchoring, which builds a text-aligned initial noise by extracting a CLIP-based semantic prior via Direct Ascent Synthesis, refining it in latent space with LP-SDS, and then applying DDPM forward diffusion; and Reflective Steering, a VLM-driven Think-Erase-Retouch loop that diagnoses semantic deviations and corrects them through negative-guided inversion and positive-guided re-denoising. Experiments on GenEval and T2I-CompBench++ with SDXL and HunyuanDiT report consistent gains over standard sampling and several baselines, with ablations claiming complementary benefits of the two components. The paper also includes quality/diversity metrics, a user study, runtime analysis, and an extension to distilled models.
Significance. If the reported results are reproducible, AnchorSteer would be a practically useful contribution: it is training-free, architecture-agnostic, and improves compositional faithfulness on both U-Net and DiT backbones. The empirical scope is a strength: the paper evaluates on two benchmarks, compares against multiple training-free and training-based baselines, and includes ablations, a user study, and runtime analysis. I do not see a circular-evaluation problem, since the VLM critic is pretrained and the final outcomes are scored by external benchmarks. However, the mechanism attributed to Semantic Anchoring is not established by the evidence as presented, and the lack of uncertainty quantification and the tuning-on-test details make the headline margins difficult to assess.
major comments (3)
- [Sec. 4.1 (DDPM Forward Diffusion for Noise Initialization)] The claim “z_T still follows N(0,I)” is mathematically incorrect for a fixed z*: the conditional distribution is N(sqrt(alpha_bar_T) z*, (1 - alpha_bar_T) I). This is load-bearing, because the proposed mechanism requires z_T to preserve prompt-specific semantic information while remaining distributionally consistent. With SDXL’s 50-step DDIM schedule, alpha_bar_T is very small, so the mean shift is tiny; the first denoising step amplifies signal and noise by the same factor, so a simple SNR argument does not rescue the claim. The only direct empirical support, Table S5 in Sec. F, is a single run with no error bars and uses z_shu_T as a null, which is not matched: a shuffled latent has different norm/correlation and does not separate prompt-specific leakage from generic structured-initialization effects. Please provide repeated-seed experiments with matched nulls (e.g., z* from another pr
- [Sec. 5.1 and Table S2 (hyperparameter selection)] The hyperparameters N_lp and tau are selected on GenEval (Table S2), and the main GenEval results in Tables 1, 2, and 5 are reported on the same benchmark. This creates a risk of selection-on-test. Several reported differences are small (e.g., T2I-CompBench++ Non-Spatial 0.3139 vs 0.3171), and no standard errors, confidence intervals, or seed-level results are provided anywhere. The claim of “consistent” improvement needs error bars over multiple seeds and a stated validation protocol. This is especially important for the single-run Table S5 and for the +Anchor vs +Steer decomposition in Table 5, which is used to support the complementarity claim.
- [Sec. 5.3 / Table 4 (ImageReward, quality and diversity)] The ImageReward score jumps from 0.4891 to 0.8781 while HPSv2, PickScore, and Aesthetic improve modestly. Such a large gain on one preference model, reported without error bars or qualitative analysis, is surprising and should be verified. In addition, the diversity results in Table S1 are based on 20 prompts per task and 10 images per prompt; this sampling protocol should be stated in the main text, and confidence intervals would help substantiate the diversity claims.
minor comments (5)
- [Eq. (5) and surrounding text] The KL interpretation should be presented as an analogy or heuristic; as written it implies an exact identity that is not established by the SDS derivation, and the paper’s own derivation omits the Jacobian term.
- [Abstract / Sec. 1] There are capitalization and grammar errors: “we proposeAnchorSteer” in the abstract and “First, atnoise initialization” in Sec. 1. Please proofread.
- [Sec. F / Table S5] The semantic-preservation experiment is load-bearing but appears only in the supplementary material. It should either be moved into the main text or at least explicitly referenced from Sec. 4.1 with a one-sentence summary of its limitations.
- [Algorithm S1 / Eqs. (6)-(9)] Both the Erase and Retouch steps use D_neg in the negative-prompt position; please clarify the behavior when D_neg is empty and how the algorithm avoids double-counting erasure during retouch.
- [References] Several references are dated 2026; please verify that they are publicly available or update them. Reproducibility would also be improved by releasing code and exact evaluation scripts.
Circularity Check
No significant circularity: AnchorSteer's improvements are measured on external benchmarks; no component's output is definitionally equal to its input.
full rationale
AnchorSteer's pipeline has three main stages: (1) DAS extracts a CLIP prior from the prompt; (2) LP-SDS refines that prior in VAE latent space with the frozen diffusion model's score, using the standard SDS gradient (Eq. 4) with the U-Net Jacobian omitted; (3) DDPM forward noising produces z_T, and Reflective Steering later inserts a VLM-based Think-Erase-Retouch loop during denoising. None of these stages fits a parameter to GenEval or T2I-CompBench++ labels; the reported gains in Tables 1-3 are computed by external benchmark protocols, and the VLM critic (Qwen-VL) is a pretrained model not fine-tuned on those benchmarks. The use of the same diffusion model in LP-SDS and in the final sampling is a standard self-distillation/initialization technique, not a reduction of the evaluation to the method's own outputs. The assertion in Sec. 4.1 that z_T = sqrt(alpha_T) z* + sqrt(1-alpha_T) eps 'still follows N(0,I)' is not exact for fixed nonzero z*, but this is a correctness/technical concern about signal strength, not a circularity; the semantic-leakage claim is tested empirically in Table S5 rather than assumed in the metric. Self-citations in the reference list (e.g., [24-26,72]) appear only in related-work or motivational contexts and are not load-bearing for the derivation. Therefore the central claim has independent content and no circular step can be exhibited.
Axiom & Free-Parameter Ledger
free parameters (7)
- N_lp (LP-SDS optimization steps) =
400
- inspection stride tau =
1
- erase guidance scale gamma_e =
1.0
- retouch guidance scale gamma_r =
5.0
- DAS optimization hyperparameters (steps, learning rate, initialization) =
not specified
- LP-SDS weighting w(t) and noise schedule =
not specified
- VLM instruction templates =
hand-designed
axioms (8)
- domain assumption DDPM forward diffusion preserves enough semantic information at the terminal timestep to affect generation
- domain assumption CLIP embeddings and DAS produce a semantic prior meaningfully aligned with compositional prompts
- domain assumption VLM can reliably diagnose missing and inconsistent content from intermediate decoded latents
- standard math DDIM inversion is locally reversible enough for erase-retouch edits
- standard math SDS Jacobian omission is a valid approximation for latent optimization
- domain assumption The pretrained diffusion model can render the reinforced positive prompt c+ at the current timestep without retraining
- domain assumption Negative-prompt guidance suppresses erroneous semantics without harming other content
- domain assumption GenEval and T2I-CompBench++ metrics measure the claimed text-image faithfulness
Cite this review
Pith. "Pith review of Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Time." pith.science (2026). https://pith.science/paper/PF3ZK6IV
@misc{pith2026260726647,
author = {Pith},
title = {Pith review of: Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Time},
year = {2026},
howpublished = {\url{https://pith.science/paper/PF3ZK6IV}},
note = {Machine review of arXiv:2607.26647}
}
read the original abstract
While text-to-image diffusion models achieve impressive visual quality, they frequently struggle to maintain precise alignment with complex compositional prompts. An effective strategy is to improve the inference process of diffusion models, thereby better leveraging their pretrained priors to address misalignment. Existing training-free methods can be divided into two categories. The first category focuses on improving the randomly sampled initial noise, either performing costly search over noise pools or manipulating sampled noise without ensuring reliable semantic injection. The second category focuses on improving the denoising trajectory, lacking explicit mechanisms to timely diagnose and correct semantic errors. we propose \textbf{AnchorSteer}, a training-free framework that exerts fine-grained control over \textbf{both initialization} and \textbf{the denoising trajectory}. AnchorSteer consists of two synergistic components: \textbf{Semantic Anchoring} replaces uninformative Gaussian noise with text-aligned initializations via CLIP-based prior extraction and a novel Latent-Prior Score Distillation Sampling (LP-SDS) objective. Specifically, LP-SDS distills CLIP visual priors into the knowledge distribution of diffusion models, mitigating the domain gap between CLIP-based priors and diffusion-based priors. \textbf{Reflective Steering} transforms passive denoising with an active Think--Erase--Retouch loop that enables mid-generation self-correction. It leverages VLM-based diagnosis to detect semantic deviations and performs targeted latent refinement to suppress erroneous content and recover missing attributes. Extensive experiments on GenEval and T2I-CompBench++ demonstrate that AnchorSteer consistently outperforms existing baselines in text--image alignment while preserving high visual quality.
Figures
Reference graph
Works this paper leans on
-
[1]
In: The Fourteenth International Conference on Learning Representations (2026), https://openreview.net/forum?id=xEWooSOgaz
Ahn, D., Kang, J., Lee, S., Min, J., Kim, M., Jang, W., Cho, H., Paul, S., Kim, S., Cha, E., Jin, K.H., Kim, S.: A noise is worth diffusion guidance. In: The Fourteenth International Conference on Learning Representations (2026), https://openreview.net/forum?id=xEWooSOgaz
2026
-
[2]
arXiv preprint arXiv:2309.16609 (2023) 16 X
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al.: Qwen technical report. arXiv preprint arXiv:2309.16609 (2023) 16 X. Wang et al
Pith/arXiv arXiv 2023
-
[3]
In: The Thirteenth International Conference on Learning Representations (2024),https: //openreview.net/forum?id=MKvQH1ekeY
Bai, L., Shao, S., Zhou, Z., Qi, Z., Xu, Z., Xiong, H., Xie, Z.: Zigzag dif- fusion sampling: Diffusion models can self-improve via self-reflection. In: The Thirteenth International Conference on Learning Representations (2024),https: //openreview.net/forum?id=MKvQH1ekeY
2024
-
[4]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Brooks, T., Holynski, A., Efros, A.A.: Instructpix2pix: Learning to follow image editing instructions. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 18392–18402 (2023)
2023
-
[5]
In: Proceedings of the International Conference on Computer Vision (ICCV) (2021)
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the International Conference on Computer Vision (ICCV) (2021)
2021
-
[6]
In: Findings of the Association for Computational Linguistics: EMNLP 2024
Chang, Z., Li, M., Wang, J., Liu, Y., Wang, Q., Liu, Y.: Repairing catastrophic- neglect in text-to-image diffusion models via attention-guided feature enhance- ment. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 11379–11390 (2024)
2024
-
[7]
Chatterjee, A., Stan, G.B.M., Aflalo, E., Paul, S., Ghosh, D., Gokhale, T., Schmidt, L., Hajishirzi, H., Lal, V., Baral, C., et al.: Getting it right: Improving spatial consistencyintext-to-imagemodels.In:EuropeanConferenceonComputerVision. pp. 204–222. Springer (2024)
2024
-
[8]
ACM trans- actions on Graphics (TOG)42(4), 1–10 (2023)
Chefer, H., Alaluf, Y., Vinker, Y., Wolf, L., Cohen-Or, D.: Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM trans- actions on Graphics (TOG)42(4), 1–10 (2023)
2023
-
[9]
In: The Thirteenth International Con- ference on Learning Representations (2025),https://openreview.net/forum?id= E77uvbOTtp
Chung, H., Kim, J., Park, G.Y., Nam, H., Ye, J.C.: CFG++: Manifold-constrained classifier free guidance for diffusion models. In: The Thirteenth International Con- ference on Learning Representations (2025),https://openreview.net/forum?id= E77uvbOTtp
2025
-
[10]
In: Forty-first international conference on machine learning (2024)
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al.: Scaling rectified flow transformers for high-resolution image synthesis. In: Forty-first international conference on machine learning (2024)
2024
-
[11]
In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Everaert, M.N., Fitsios, A., Bocchio, M., Arpa, S., Süsstrunk, S., Achanta, R.: Exploiting the Signal-Leak Bias in Diffusion Models. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 4025–4034 (January 2024)
2024
-
[12]
Feng, H., Ding, Z., Xia, Z., Niklaus, S., Abrevaya, V., Black, M.J., Zhang, X.: Explorativeinbetweeningoftimeandspace.In:EuropeanConferenceonComputer Vision. pp. 378–395. Springer (2024)
2024
-
[13]
Feng, W., He, X., Fu, T.J., Jampani, V., Akula, A.R., Narayana, P., Basu, S., Wang, X.E., Wang, W.Y.: Training-free structured diffusion guidance for composi- tionaltext-to-imagesynthesis.In:TheEleventhInternationalConferenceonLearn- ing Representations (2023),https://openreview.net/forum?id=PUIqjT4rzq7
2023
-
[14]
Fort, S., Whitaker, J.: Direct ascent synthesis: Revealing hidden generative capa- bilities in discriminative models (2025),https://arxiv.org/abs/2502.07753
Pith/arXiv arXiv 2025
-
[15]
Advances in Neural Information Processing Systems36, 52132–52152 (2023)
Ghosh, D., Hajishirzi, H., Schmidt, L.: Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems36, 52132–52152 (2023)
2023
-
[16]
Grimal, P., Borgne, H.L., Ferret, O.: Text-to-image alignment in denoising-based models through step selection (2025),https://arxiv.org/abs/2504.17525
Pith/arXiv arXiv 2025
-
[17]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Guo, X., Liu, J., Cui, M., Li, J., Yang, H., Huang, D.: Initno: Boosting text- to-image diffusion models via initial noise optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9380– 9389 (2024) Anchoring and Steering Diffusion 17
2024
-
[18]
Harrington, A., Koepke, A., Karthik, S., Darrell, T., Efros, A.A.: It’s never too late: Noiseoptimizationforcollapserecoveryintraineddiffusionmodels.In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 43124–43134 (2026)
2026
-
[19]
In: The Eleventh In- ternational Conference on Learning Representations (2023),https://openreview
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-prompt image editing with cross-attention control. In: The Eleventh In- ternational Conference on Learning Representations (2023),https://openreview. net/forum?id=_CDixzkzeyb
2023
-
[20]
Advances in neural information processing systems33, 6840–6851 (2020)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems33, 6840–6851 (2020)
2020
-
[21]
In: NeurIPS 2021 Work- shop on Deep Generative Models and Downstream Applications (2021),https: //openreview.net/forum?id=qw8AKxfYbI
Ho, J., Salimans, T.: Classifier-free diffusion guidance. In: NeurIPS 2021 Work- shop on Deep Generative Models and Downstream Applications (2021),https: //openreview.net/forum?id=qw8AKxfYbI
2021
-
[22]
Huang, K., Duan, C., Sun, K., Xie, E., Li, Z., Liu, X.: T2I-CompBench++: An EnhancedandComprehensiveBenchmarkforCompositionalText-to-ImageGener- ation.IEEETransactionsonPatternAnalysisMachineIntelligence(01),1–17(Jan 5555),https://doi.ieeecomputersociety.org/10.1109/TPAMI.2025.3531907
arXiv 2025
-
[23]
Advances in Neural Information Processing Systems36, 78723–78747 (2023)
Huang, K., Sun, K., Xie, E., Li, Z., Liu, X.: T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems36, 78723–78747 (2023)
2023
-
[24]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Huang, Y., Chen, Y., Ding, L., Zhang, X., Dai, W., Zou, J., Xiong, H., Tian, Q.: Im-zero: Instance-level motion controllable video generation in a zero-shot manner. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 7265–7275 (2025)
2025
-
[25]
In: European Conference on Computer Vision
Huang, Y., Chen, Y., Liu, Y., Zhang, X., Dai, W., Xiong, H., Tian, Q.: Domain- fusion: Generalizing to unseen domains with latent diffusion models. In: European Conference on Computer Vision. pp. 480–498. Springer (2024)
2024
-
[26]
Advances in Neural Information Processing Systems38, 108999–109019 (2026)
Huang, Y., Chen, Y., Zhou, J., Dai, W., Zhang, X., Zou, J., Xiong, H., Tian, Q.: Diffusion-driven progressive target manipulation for source-free domain adap- tation. Advances in Neural Information Processing Systems38, 108999–109019 (2026)
2026
-
[27]
Zenodo (2021)
Ilharco, G., Wortsman, M., Carlini, N., Taori, R., Dave, A., Shankar, V., Namkoong, H., Miller, J., Hajishirzi, H., Farhadi, A., et al.: Openclip. Zenodo (2021)
2021
-
[28]
Advances in Neural Information Processing Systems37, 76177–76209 (2024)
Jiang, D., Song, G., Wu, X., Zhang, R., Shen, D., Zong, Z., Liu, Y., Li, H.: Co- mat: Aligning text-to-image diffusion model with image-to-text concept matching. Advances in Neural Information Processing Systems37, 76177–76209 (2024)
2024
-
[29]
Advances in neural information processing systems35, 26565–26577 (2022)
Karras,T.,Aittala,M.,Aila,T.,Laine,S.:Elucidatingthedesignspaceofdiffusion- based generative models. Advances in neural information processing systems35, 26565–26577 (2022)
2022
-
[30]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition
Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R.C., Schindler, K.: Re- purposing diffusion-based image generators for monocular depth estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 9492–9502 (2024)
2024
-
[31]
In: The Fourteenth International Con- ference on Learning Representations (2026),https://openreview.net/forum?id= 11dzFZ2UM1
Kim, K., Kim, S.: Model already knows the best noise: Bayesian active noise selec- tion via attention in video diffusion model. In: The Fourteenth International Con- ference on Learning Representations (2026),https://openreview.net/forum?id= 11dzFZ2UM1
2026
-
[32]
Advances in neural information processing systems36, 36652–36663 (2023) 18 X
Kirstain,Y.,Polyak,A.,Singer,U.,Matiana,S.,Penna,J.,Levy,O.:Pick-a-pic:An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems36, 36652–36663 (2023) 18 X. Wang et al
2023
-
[33]
In: European Conference on Computer Vision (ECCV)
Koo, G., Yoon, S., Hong, J.W., Yoo, C.D.: Flexiedit: Frequency-aware latent re- finement for enhanced non-rigid editing. In: European Conference on Computer Vision (ECCV). pp. 363–379. Lecture Notes in Computer Science, Springer (2024)
2024
-
[34]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Kumari, N., Zhang, B., Zhang, R., Shechtman, E., Zhu, J.Y.: Multi-concept cus- tomization of text-to-image diffusion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 1931–1941 (2023)
1931
-
[35]
Labs, B.F.: Flux.https://github.com/black-forest-labs/flux(2024)
2024
-
[36]
Li, Z., Zhang, J., Lin, Q., Xiong, J., Long, Y., Deng, X., Zhang, Y., Liu, X., Huang, M., Xiao, Z., Chen, D., He, J., Li, J., Li, W., Zhang, C., Quan, R., Lu, J., Huang, J., Yuan, X., Zheng, X., Li, Y., Zhang, J., Zhang, C., Chen, M., Liu, J., Fang, Z., Wang, W., Xue, J., Tao, Y., Zhu, J., Liu, K., Lin, S., Sun, Y., Li, Y., Wang, D., Chen, M., Hu, Z., Xia...
2024
-
[37]
Lian, L., Li, B., Yala, A., Darrell, T.: LLM-grounded diffusion: Enhancing prompt understandingoftext-to-imagediffusionmodelswithlargelanguagemodels.Trans- actions on Machine Learning Research (2024),https://openreview.net/forum? id=hFALpTb4fR, featured Certification
2024
-
[38]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Liang, Z., Yuan, Y., Gu, S., Chen, B., Hang, T., Cheng, M., Li, J., Zheng, L.: Aesthetic post-training diffusion models from generic preferences with step-by- step preference optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13199–13208 (2025)
2025
-
[39]
In: European conference on computer vision
Liu, N., Li, S., Du, Y., Torralba, A., Tenenbaum, J.B.: Compositional visual gen- eration with composable diffusion models. In: European conference on computer vision. pp. 423–439. Springer (2022)
2022
-
[40]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Liu,Y.,Zhang,Y.,Jaakkola,T.,Chang,S.:Correctingdiffusiongenerationthrough resampling. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8713–8723 (2024)
2024
-
[41]
In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Liu, Y., Zhang, Y., Jaakkola, T., Chang, S.: Correcting diffusion generation through resampling. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8713–8723 (2024).https://doi.org/10.1109/ CVPR52733.2024.00832
arXiv 2024
-
[42]
arXiv preprint arXiv:2501.09732 (2025)
Ma, N., Tong, S., Jia, H., Hu, H., Su, Y.C., Zhang, M., Yang, X., Li, Y., Jaakkola, T., Jia, X., et al.: Inference-time scaling for diffusion models beyond scaling de- noising steps. arXiv preprint arXiv:2501.09732 (2025)
Pith/arXiv arXiv 2025
-
[43]
Transactions on Ma- chineLearningResearch(2024),https://openreview.net/forum?id=a68SUt6zFt, featured Certification
Oquab, M., Darcet, T., Moutakanni, T., Vo, H.V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Mairal, J., Labatut, P., Joulin, A., Bojanowski, P.: DINOv2: Learning robust visual feat...
2024
-
[44]
Peebles,W.,Xie,S.:Scalablediffusionmodelswithtransformers.In:Proceedingsof the IEEE/CVF international conference on computer vision. pp. 4195–4205 (2023)
2023
-
[45]
In: The Twelfth International Conference on Learning Representa- tions (2024),https://openreview.net/forum?id=di52zR8xgf
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: SDXL: Improving latent diffusion models for high-resolution im- age synthesis. In: The Twelfth International Conference on Learning Representa- tions (2024),https://openreview.net/forum?id=di52zR8xgf
2024
-
[46]
In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=FjNys5c7VyY Anchoring and Steering Diffusion 19
Poole, B., Jain, A., Barron, J.T., Mildenhall, B.: Dreamfusion: Text-to-3d using 2d diffusion. In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=FjNys5c7VyY Anchoring and Steering Diffusion 19
2023
-
[47]
Qi, Z., Bai, L., Xiong, H., Xie, Z.: Not all noises are created equally:diffusion noise selection and optimization (2024),https://arxiv.org/abs/2407.14041
Pith/arXiv arXiv 2024
-
[48]
In: International conference on machine learning
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)
2021
-
[49]
Advances in Neural Information Processing Systems36, 3536–3559 (2023)
Rassin, R., Hirsch, E., Glickman, D., Ravfogel, S., Goldberg, Y., Chechik, G.: Lin- guistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. Advances in Neural Information Processing Systems36, 3536–3559 (2023)
2023
-
[50]
In: Proceedings of the Fifth BlackboxNLP workshop on analyzing and interpreting neural networks for NLP
Rassin, R., Ravfogel, S., Goldberg, Y.: Dalle-2 is seeing double: Flaws in word-to- concept mapping in text2image models. In: Proceedings of the Fifth BlackboxNLP workshop on analyzing and interpreting neural networks for NLP. pp. 335–345 (2022)
2022
-
[51]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)
2022
-
[52]
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dream- booth:Finetuningtext-to-imagediffusionmodelsforsubject-drivengeneration.In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 22500–22510 (2023)
2023
-
[53]
In: The Thirteenth International Confer- ence on Learning Representations (2024)
Sadat, S., Hilliges, O., Weber, R.M.: Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In: The Thirteenth International Confer- ence on Learning Representations (2024)
2024
-
[54]
Advances in neural information processing systems35, 36479–36494 (2022)
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al.: Photorealistic text- to-image diffusion models with deep language understanding. Advances in neural information processing systems35, 36479–36494 (2022)
2022
-
[55]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Samuel, D., Ben-Ari, R., Raviv, S., Darshan, N., Chechik, G.: Generating images of rare concepts using pre-trained diffusion models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 4695–4703 (2024)
2024
-
[56]
In: Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2022),https://openreview.net/forum?id=M3Y74vmsMcY
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C.W., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kun- durthy, S.R., Crowson, K., Schmidt, L., Kaczmarczyk, R., Jitsev, J.: LAION-5b: An open large-scale dataset for training next generation image-text models. In: Thirty-sixth Conference on Neural Information ...
2022
-
[57]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Shen, D., Song, G., Xue, Z., Wang, F.Y., Liu, Y.: Rethinking the spatial incon- sistency in classifier-free diffusion guidance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9370–9379 (2024)
2024
-
[58]
Song,J.,Meng,C.,Ermon,S.:Denoisingdiffusionimplicitmodels.In:International Conference on Learning Representations (2021),https://openreview.net/forum? id=St1giarCHLP
2021
-
[59]
In: Interna- tional Conference on Learning Representations (2021),https://openreview.net/ forum?id=PxTIG12RRHS
Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. In: Interna- tional Conference on Learning Representations (2021),https://openreview.net/ forum?id=PxTIG12RRHS
2021
-
[60]
In: Forty-second In- ternational Conference on Machine Learning (2025),https://openreview.net/ forum?id=JpbqiD7n9r 20 X
Tang, Z., Peng, J., Tang, J., Hong, M., Wang, F., Chang, T.H.: Inference-time alignment of diffusion models with direct noise optimization. In: Forty-second In- ternational Conference on Machine Learning (2025),https://openreview.net/ forum?id=JpbqiD7n9r 20 X. Wang et al
2025
-
[61]
Team, Q.: Qwen2.5-vl (January 2025),https://qwenlm.github.io/blog/qwen2. 5-vl/
2025
-
[62]
In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion
Tian, J., Aggarwal, L., Colaco, A., Kira, Z., Gonzalez-Franco, M.: Diffuse attend and segment: Unsupervised zero-shot segmentation using stable diffusion. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. pp. 3554–3563 (2024)
2024
-
[63]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Wallace, B., Dang, M., Rafailov, R., Zhou, L., Lou, A., Purushwalkam, S., Ermon, S., Xiong, C., Joty, S., Naik, N.: Diffusion model alignment using direct preference optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8228–8238 (2024)
2024
-
[64]
arXiv preprint arXiv:2409.12191 (2024)
Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., Fan, Y., Dang, K., Du, M., Ren, X., Men, R., Liu, D., Zhou, C., Zhou, J., Lin, J.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)
Pith/arXiv arXiv 2024
-
[65]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Wang, R., Chen, Z., Chen, C., Ma, J., Lu, H., Lin, X.: Compositional text-to-image synthesis with attention map control of diffusion models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 5544–5552 (2024)
2024
-
[66]
Wu, C., Li, J., Zhou, J., Lin, J., Gao, K., Yan, K., ming Yin, S., Bai, S., Xu, X., Chen, Y., Chen, Y., Tang, Z., Zhang, Z., Wang, Z., Yang, A., Yu, B., Cheng, C., Liu, D., Li, D., Zhang, H., Meng, H., Wei, H., Ni, J., Chen, K., Cao, K., Peng, L., Qu, L., Wu, M., Wang, P., Yu, S., Wen, T., Feng, W., Xu, X., Wang, Y., Zhang, Y., Zhu, Y., Wu, Y., Cai, Y., L...
Pith/arXiv arXiv 2025
-
[67]
In: NeurIPS 2023 Workshop on Regu- latable ML (2023),https://openreview.net/forum?id=8WH2t9F0Ip
Wu, S., Tang, S., Aydore, S., Kearns, M., Roth, A.: Membership inference attack on diffusion models via quantile regression. In: NeurIPS 2023 Workshop on Regu- latable ML (2023),https://openreview.net/forum?id=8WH2t9F0Ip
2023
-
[68]
In: European Conference on Computer Vision (ECCV) (2024)
Wu,T.,Si,C.,Jiang,Y.,Huang,Z.,Liu,Z.:Freeinit:Bridgingtheinitializationgap in video diffusion models. In: European Conference on Computer Vision (ECCV) (2024)
2024
-
[69]
arXiv preprint arXiv:2306.09341 (2023)
Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., Li, H.: Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 (2023)
Pith/arXiv arXiv 2023
-
[70]
In: Proceedingsofthe37thInternationalConferenceonNeuralInformationProcessing Systems
Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: Imagere- ward: learning and evaluating human preferences for text-to-image generation. In: Proceedingsofthe37thInternationalConferenceonNeuralInformationProcessing Systems. pp. 15903–15935 (2023)
2023
-
[71]
In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)
Xu, R., Xi, W., Wang, X., Mao, Y., Cheng, Z.: Stylessp: Sampling startpoint en- hancement for training-free diffusion-based method for style transfer. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)
2025
-
[72]
arXiv preprint arXiv:2511.07756 (2025)
Yan, S., Li, M., Xinliang, B., Yang, J., Zhang, Y., Xiong, G., Lan, Y., Zhang, T., Zhai, W., Zha, Z.J.: Beyond randomness: Understand the order of the noise in diffusion. arXiv preprint arXiv:2511.07756 (2025)
Pith/arXiv arXiv 2025
-
[73]
Yu, J., Wang, Y., Zhao, C., Ghanem, B., Zhang, J.: Freedom: Training-free energy- guidedconditionaldiffusionmodel.In:ProceedingsoftheIEEE/CVFInternational Conference on Computer Vision. pp. 23174–23184 (2023)
2023
-
[74]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Zeng, Q., Song, J., Zheng, H., Jiang, H., Song, M.: D^2-dpm: Dual denoising for quantized diffusion probabilistic models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 9806–9814 (2025).https://doi.org/10. 1609/aaai.v39i9.33063 Anchoring and Steering Diffusion 21
2025
-
[75]
Zhang,G.,Fu,B.,Fan,Q.,Zhang,Q.,Liu,R.,Gu,H.,Zhang,H.,Liu,X.:Compass: Enhancingspatialunderstandingintext-to-imagediffusionmodels.In:Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 15253–15265 (2025)
2025
-
[76]
In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreason- able effectiveness of deep features as a perceptual metric. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 586–595 (2018). https://doi.org/10.1109/CVPR.2018.00068
arXiv 2018
-
[77]
In: International Conference on Computer Vision (2025)
Zhou, Z., Shao, S., Bai, L., Zhang, S., Xu, Z., Han, B., Xie, Z.: Golden noise for diffusion models: A learning framework. In: International Conference on Computer Vision (2025)
2025
-
[78]
Zhuang, C., Hu, Y., Gao, P.: Magnet: We never know how text-to-image diffusion models work, until we learn how vision-language models function. Advances in Neural Information Processing Systems37, 57115–57149 (2024) 22 X. Wang et al. Supplementary Material This supplementary material provides additional details and experimental results to complement the m...
arXiv 2024
-
[79]
As shown in Table S2 (top), the CLIP-only prior (Nlp=0) improves the standard baseline, while apply- ing LP-SDS optimization (Nlp >0) further boosts performance
Effect ofN lp.We varyN lp ∈ {0,200,400,600}. As shown in Table S2 (top), the CLIP-only prior (Nlp=0) improves the standard baseline, while apply- ing LP-SDS optimization (Nlp >0) further boosts performance. This confirms both the effectiveness of the semantic prior and the need to bridge the domain gapbetweenCLIPembeddingsandthe diffusionlatent space.Perf...
-
[80]
and", "or
Effect of Inspection Strideτ.We testτ∈ {1,2,5}. As shown in Ta- ble S2 (bottom), smaller stride (more frequent inspection) consistently improves alignment, particularly on compositional categories such asTwo-object,Count- ing, andPosition, where semantic deviations are most prone to accumulate. Notably, densely applying the correction loop does not compro...
This paper was first reviewed by deepseek-v4-flash on August 1, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.