Pith. sign in

REVIEW 3 major objections 5 minor 68 references

A kernel-entropy guidance signal, linearized into logit space, shifts the fidelity-diversity frontier of text diffusion models and lifts LLaDA-8B pass@32 on HumanEval and MBPP by 8-15 absolute points.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 01:42 UTC pith:DJWNRZI3

load-bearing objection SAKE is a clever training-free diversity guidance with sound core math and promising gains, but the derivation doesn't match the actual masked-diffusion implementation, leaving a load-bearing gap that needs an ablation. the 3 major comments →

arxiv 2608.00024 v1 pith:DJWNRZI3 submitted 2026-07-10 cs.CL cs.AI

Exploring More to Solve More: Boosting Diversity in Text Diffusion Models via Entropy-Based Guidance

Jingwei Zhang , Haoyu Lei , Zijin Feng , Jiacheng Sun , Farzan Farnia This is my paper
classification cs.CL cs.AI
keywords guidancediversityfidelitydiffusiondiscreteentropykernelmethod
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Text diffusion models generate a sentence by starting with blank slots and filling them in over many small steps. A standard way to make them more varied is temperature scaling, which flattens the model's word-choice probabilities blindly and often produces gibberish. This paper proposes a training-free alternative called SAKE that watches the meanings of the words already placed in the sentence and pushes the next word away from whichever words would make the sentence repetitive and crowded.

The key object is a similarity matrix (kernel Gram matrix) over the sentence's token embeddings, blended with a positional kernel, from which the authors compute the order-2 Renyi entropy - a measure of how spread out the tokens are as a whole. Maximizing this entropy produces a repulsive force between similar tokens, and because the force is a weighted sum of embedding differences, multiplying it by the vocabulary embedding matrix turns it directly into a per-word logit adjustment. The result is an adaptive modulator: in repetitive contexts it flattens the distribution to force exploration; when the sentence is already varied the force nearly vanishes and the model's own confidence dominates.

The paper reports Pareto-frontier improvements on synthetic Gaussian mixing, story continuation, and brainstorm tasks, plus higher pass rates on HumanEval, MBPP, and GSM8K with LLaDA-8B. Caveats: the guidance strength is tuned per benchmark by grid search, the headline numbers come without confidence intervals, the claimed linear-time cost is actually quadratic in sequence length when the Gram matrix is computed, and the semantic-geometry assumption behind the kernel is asserted rather than validated.

Core claim

The load-bearing assertion is that a training-free, entropy-based logit guidance improves the fidelity-diversity trade-off of discrete text diffusion and, in particular, that 'applying SAKE to LLaDA-8B-base yields significant performance gains, improving HumanEval pass@32 (41.1->55.8), MBPP pass@32 (48.2->56.1), and GSM8K self-consistency (71.5->75.1)' (Sec. 1). The theoretical core is Eq. 18: psi_div = (2E/sigma^2) * sum_{j!=i} kappa_attn(i,j)^2 * kappa(h_i,h_j)^2 * (h_i - h_j), the linearized gradient of the order-2 Renyi entropy of the kernel Gram matrix. If the paper is correct, DLM inference can be steered toward semantically structured exploration with ~7% throughput cost and no retraining.

Load-bearing premise

The premise that hidden-state Euclidean geometry carries semantic meaning, asserted in Sec. 4.3.2: 'These assumptions hold in modern LLMs (like LLaDA) because LayerNorm/RMSNorm constrains embeddings.' The whole guidance signal (Eq. 18) is a first-order Taylor linearization of a repulsive energy in that geometry, so if embedding distances are not faithful semantic similarities - or if masked-position hidden states are geometrically degenerate - SAKE becomes uncalibrated repulsive jitter and the pass@32 gains in Sec. 5.3 become task-specific accidents. The paper offers no probe, ablation, or baseline isolating the semantic-kernel assumption from the raw repulsion. A second, structurally distinct dependency: the causal chain in Sec. 5.3 from intra-sequence kernel entropy to across-sample pass@k diversity is asserted via the CoT-diversity argument, never derived.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Semantic-Aware Kernel Entropy (SAKE), a training-free guidance method for discrete text diffusion models. SAKE adjusts the sampling logits by adding a signal derived from the gradient of the order-2 Rényi entropy of a kernel Gram matrix built on token representations and relative positions. The authors provide a formal analysis of guidance entropy dynamics (Prop. 4.1) and derive the SAKE signal (Prop. 4.3). They evaluate the method on synthetic tasks, Pareto-frontier benchmarks, and code/math reasoning tasks with LLaDA-8B, reporting improved diversity-fidelity trade-offs and higher pass@32 / self-consistency scores compared to temperature scaling and D-CFG.

Significance. If the mechanism is validated, SAKE would be a lightweight, training-free way to improve multi-sample performance of discrete diffusion language models, with only an estimated 7% throughput reduction (Table 3). The theoretical derivations are transparent and the fixed-temperature comparisons in Table 9 show consistent pass@32 gains (e.g., HumanEval 0.41→0.56 at T=0.2). The idea of using kernel Rényi entropy as a guidance signal is novel and could inspire further work. However, the current manuscript does not adequately validate the core geometry assumption or the step from intra-sequence diversity to across-sample pass@k gains, so the significance is conditional.

major comments (3)
  1. [Sec. 4.3.2 / Algorithm 1] The derivation of SAKE in Prop. 4.3 and Eq. 18 assumes that h_i are the token embeddings of the current tokens at each position, so that the gradient ∇_{h_i}D̃κ can be projected onto the vocabulary embedding matrix E via Eq. 14. However, Algorithm 1 line 3 sets H_t = f_θ(x_{t−1}), i.e., the model's hidden states for the input sequence. In LLaDA-style masked diffusion, most positions of x_{t−1} are [MASK] tokens for most of the reverse process. The hidden states at mask positions are contextualized representations of the mask, not the embeddings of the candidate tokens y. The paper's justification in Sec. 4.3.2 ('LayerNorm/RMSNorm constrains embeddings') does not establish that these hidden states lie in the same semantic geometry as the vocabulary embedding matrix E. This is a load-bearing gap: without alignment, the repulsive term in Eq. 18 may be uncalibrated geometric jitter rather th
  2. [Sec. 5.3] The paper claims that SAKE improves multi-sample reasoning performance (pass@32, self-consistency) by increasing 'diversity-guided chains of thought.' However, the SAKE objective (Eq. 16) maximizes kernel entropy within a single generated sequence, repelling tokens from one another in that sequence. The reported pass@32 metric measures diversity across multiple independently sampled outputs. The causal link between intra-sequence kernel entropy and across-sample output diversity is asserted but never derived or empirically tested. Please provide evidence that SAKE actually increases inter-sample diversity (e.g., measure distinct correct solutions, self-consistency agreement, or pairwise embedding distances across samples) beyond the reported pass@k improvements. Without this, the mechanism behind the downstream gains is unspecified.
  3. [Sec. 5.2 / Sec. 5.3] The paper does not include any ablation that isolates the semantic kernel from a raw positional or purely repulsive baseline. Since SAKE combines a semantic RBF kernel with a positional attention kernel, the observed gains could in principle come from a generic repulsion that jitters the distribution, rather than from semantic awareness. Please add a baseline that uses the same algorithm with (a) only the attention kernel (or a uniform kernel), and (b) random/synthetic 'semantic' distances, to confirm that the semantic kernel is load-bearing for the reported Pareto frontiers and pass@32 improvements.
minor comments (5)
  1. [Appendix A] The proposition numbering is inconsistent: the main text refers to Prop. 4.1 and Prop. 4.3, while the appendix labels them A.1 and A.2. Please unify.
  2. [Eq. 13] The inner product notation ⟨·,·⟩ is introduced without definition; specify the Euclidean inner product in the embedding space.
  3. [Figures 1 and 4] Figure 1 caption says '8-Gaussian-mixture' while Figure 4 caption says '9-Gaussian-mixture'; the figures appear nearly identical. Please clarify and correct the inconsistency.
  4. [Sec. 4.3.2] The phrase 'a single backward pass of a continuous diversity function' is misleading: Algorithm 1 computes the gradient analytically from hidden states, not via automatic differentiation. Suggest rewording to 'a single forward pass followed by a closed-form gradient evaluation.'
  5. [Sec. 5.4] Table 3 reports throughput for prompt lengths up to 1024, but the main benchmark results in Sec. 5.3 likely use shorter prompts; please state the computational cost for the actual benchmark settings, not only the synthetic throughput experiment.

Circularity Check

0 steps flagged

No significant circularity: SAKE's guidance signal is derived in-paper from Renyi entropy of a Gram matrix, and the reported gains are measured on held-out benchmarks, not forced by construction.

full rationale

The derivation chain is self-contained and does not reduce to its inputs. Proposition 4.1 derives the entropy-gradient identity (Eq. 7-8) from the definition of the tilted distribution P_gamma using the covariance identity, with no assumption that the guidance target holds; the proof is given in Appendix A. Proposition 4.3 derives the repulsive-gradient signal (Eq. 17 -> 18) by calculus from the order-2 Renyi entropy of the kernel Gram matrix (Eq. 15), and the density-matrix validity (Hermitian, PSD, unit trace) is proven in the appendix via the Schur product theorem, not imported from a citation. The empirical headline claims (HumanEval pass@32 41.1->55.8, MBPP 48.2->56.1, GSM8K 71.5->75.1) are measured on external benchmarks and are not guaranteed by the construction: Pass@1 slightly decreases (32.9->32.0) and D-CFG fails to improve pass@32, showing the gains are not a forced property of the decoding procedure. The paper's own limitation section explicitly concedes that the relationship between gamma and the decoded distribution is underdeveloped, which is inconsistent with a claim whose conclusion is baked into its premise. The main vulnerability, stated by the paper rather than hidden ('These assumptions hold in modern LLMs (like LLaDA) because LayerNorm/RMSNorm constrains embeddings,' Sec. 4.3.2), is the semantic-geometry assumption on hidden states; this is an explicit, testable premise about model internals, not a circular embedding of the conclusion, though the [MASK]-token mismatch in Algorithm 1 is a genuine correctness risk. The heavy same-group citation presence (Jalali et al. 2025a; Zhang et al. 2024/2025, etc.) is contextual related-work attribution, and the proofs do not rely on it; the kernel-repulsion objective resembles the authors' prior RKE/SPARKE line (Jalali et al. 2025a), which is a novelty-attribute concern rather than a circular reduction, since the paper derives Eq. 18 internally. Tuning gamma and sigma by grid search on the evaluation benchmarks is a potential overfitting concern for the raw numbers, but it is not a fitted parameter being renamed as a prediction, and the reported trade-off (Pass@32 up, Pass@1 down) remains an empirical, falsifiable outcome. No step can be exhibited where a prediction reduces by the paper's own equations or by self-citation to its own inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

No new physical or architectural entities are introduced; the 'quantum density matrix' rho is a mathematical framing of the kernel Gram matrix, not a new object with independent evidential requirements. The central claim rests on five tuned hyperparameters (all disclosed) and on the semantic-geometry assumption that hidden-state Euclidean distances track meaning, which the paper asserts but does not validate. The free-parameter and assumption counts here are the honest measure of what SAKE adds versus what it postulates.

free parameters (5)
  • gamma (guidance scale) = Grid-searched per benchmark (e.g., gamma=0.7 in Table 1; Gaussian sweep gamma in [0.01, 2.0])
    Strength of the SAKE logit perturbation; the headline pass@32/self-consistency numbers are the best over the search grid; results are sensitive to the gamma scale.
  • sigma (semantic kernel bandwidth) = sigma=20 (MDLM), sigma=80 (BD3LM), selected from swept ranges
    Controls which embedding distances count as 'similar'; Gen PPL ranges 10.02-18.33 (MDLM) across the swept sigma range, so the metric is sensitive at the extremes.
  • sigma_attn (attention/positional kernel bandwidth) = sigma_attn=20-40 (MDLM), 10-80 (BD3LM) from sweeps
    Sets the positional receptive field of the repulsion; a vanishing-distance heuristic is given but the value itself is a free choice.
  • top-p trust-region threshold = 0.95, selected via a HumanEval sweep (Table 8)
    Chosen on the evaluation benchmark itself to support the 'standard setting'; pass@32 ranges 35.4-41.2 across top-p, so this choice affects both baseline and method comparisons.
  • sampling temperature T = 0.2-0.7 explored; Table 2 headlined at T=0.2
    The headline gains are reported at T=0.2 where the base model is weakest (pass@32 41.1); at T=0.7 the base reaches 57 and SAKE 64 (Table 9).
axioms (6)
  • standard math Gaussian RBF Gram matrices are PSD and the Hadamard (Schur) product of PSD matrices is PSD.
    Used in the proof of Proposition 4.3 (Appendix A) to assert rho = (1/L) K circle K_attn is a valid density matrix; this is the mathematical core of the entropy objective, and it is correct.
  • domain assumption The diversity potential is smooth and token embeddings are bounded, so the first-order Taylor linearization (Eq. 13) is faithful.
    Invoked in Sec. 4.3.2 with the justification 'LayerNorm/RMSNorm constrains embeddings, and Gaussian kernel in our D-tilde is smooth'; no verification is given that LLaDA's hidden states satisfy the implied condition at guidance scales gamma up to 1.0.
  • domain assumption Euclidean distance between token/hidden-state embeddings measures semantic (dis)similarity, so RBF entropy of the sequence is a semantic diversity measure.
    Central to the 'Semantic-Aware' claim (Sec. 4.3.3); the paper asserts the geometry is meaningful without providing evidence such as embedding-space probes or ablations against non-semantic baselines.
  • domain assumption Maximizing within-sequence kernel entropy transfers to across-sample diversity that improves pass@k and self-consistency.
    The causal chain in Sec. 5.3 from intra-sequence repulsion to a better candidate set for code/math is asserted via the CoT-diversity argument (Sec. 1), not derived; the empirical gains are the only evidence.
  • domain assumption All useful guidance can be expressed as logit-space tilting P_gamma proportional to P_base * exp(gamma * psi) (Eq. 6).
    The framework (Sec. 4.1) assumes softmax re-normalization after adding a guidance signal; temperature and D-CFG fit this form, but sufficiency of the family for diversity control is assumed.
  • domain assumption A top-p filter on the base logits confines the unbounded linearized signal to a plausible-token region.
    The paper admits the linearized objective 'does not plateau' (Sec. 4.3.2) and relies on the top-p trust region to bound extrapolation errors; no analysis quantifies the residual error.

pith-pipeline@v1.3.0-alltime-deepseek · 20387 in / 27990 out tokens · 291199 ms · 2026-08-04T01:42:47.464777+00:00 · methodology

0 comments
read the original abstract

Although diffusion models have revolutionized continuous domains like image synthesis through high quality generations and controllable guidance mechanisms, bringing this controllability to the discrete, sequential nature of text remains an open challenge. Meanwhile, current sampling strategies and guidance methods adjust token likelihoods without capturing the broader semantic landscape, leading to a suboptimal balance between fidelity and diversity. In this work, we introduce a novel training-free Semantic-Aware Kernel Entropy (SAKE) guidance method. Our method computes the order-2 R\'enyi entropy over a kernel Gram matrix that captures both cross-token semantic interactions and relative token positions. By linearizing this objective in the embedding space, we derive a tractable guidance signal that dynamically adjusts the sampling distribution, flattening it to encourage exploration during redundancy and sharpening it for fidelity when diverse. Empirical experiments demonstrate that our approach achieves a superior Pareto frontier between fidelity and diversity, and improves multi-sample performance on reasoning-intensive tasks, such as code and mathematics generation, compared to temperature scaling and discrete guidance baselines.

Figures

Figures reproduced from arXiv: 2608.00024 by Farzan Farnia, Haoyu Lei, Jiacheng Sun, Jingwei Zhang, Zijin Feng.

Figure 1
Figure 1. Figure 1: Comparison of sampling behaviors for a 2D 8-Gaussian-mixture target. Top row: baseline unguided reverse process at different temperatures T ∈ {1, 5, 10, 20}, illustrating a coverage-fidelity trade-off (higher T covers more modes but yields more diffuse, lower-quality samples). Bottom row: Our SAKE with guidance strengths γ ∈ {0.1, 0.2, 0.5, 1.0}, which improves mode coverage while maintaining better precis… view at source ↗
Figure 2
Figure 2. Figure 2: Pareto frontier comparisons across three generative tasks, highlighting the diversity–quality trade-off for our diversity guidance and baseline methods. Top-left: Gaussian mixture synthesis, plotting coverage vs. density. Top-right: arithmetic series generation, plotting uniqueness (unique%) vs. validity. Bottom: Two tasks: story continuation and brainstorm generation, plotting distinct-2 vs. perplexity (l… view at source ↗
Figure 3
Figure 3. Figure 3: Pass@k rates on HumanEval at low (T = 0.2) and high (T = 0.7) temperatures. At T = 0.2, standard Temperature Sampling and D-CFG exhibit signs of mode collapse, while our SAKE method shows a steady improvement from k = 1 to k = 32. Notably, our method at T = 0.2 achieves a Pass@32 score comparable to the baseline at T = 0.7, proving it can extract diverse, high-quality solutions without relying on high temp… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of sampling behaviors for a 2D 9-Gaussian-mixture target. Top row: baseline unguided reverse process at different temperatures T ∈ {1, 5, 10, 20}, illustrating a coverage-fidelity trade-off (higher T covers more modes but yields more diffuse, lower-quality samples). Bottom row: Diversity guidance with strengths γ ∈ {0.1, 0.2, 0.5, 1.0}, which improves mode coverage while maintaining better preci… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 20 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2412.10193 , year=

    Simple guidance mechanisms for discrete diffusion models , author=. arXiv preprint arXiv:2412.10193 , year=

    Pith/arXiv arXiv

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Concrete score matching: Generalized score matching for discrete data , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    arXiv preprint arXiv:2310.16834 , year=

    Discrete diffusion modeling by estimating the ratios of the data distribution , author=. arXiv preprint arXiv:2310.16834 , year=

    Pith/arXiv arXiv

  4. [4]

    Advances in neural information processing systems , volume=

    Diffusion models beat gans on image synthesis , author=. Advances in neural information processing systems , volume=

  5. [5]

    arXiv preprint arXiv:2207.12598 , year=

    Classifier-free diffusion guidance , author=. arXiv preprint arXiv:2207.12598 , year=

    Pith/arXiv arXiv

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    An information-theoretic evaluation of generative models in learning multi-modal distributions , author=. Advances in Neural Information Processing Systems , volume=

  7. [8]

    Advances in neural information processing systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=

  8. [9]

    Advances in neural information processing systems , volume=

    Generative modeling by estimating gradients of the data distribution , author=. Advances in neural information processing systems , volume=

  9. [10]

    arXiv preprint arXiv:2011.13456 , year=

    Score-based generative modeling through stochastic differential equations , author=. arXiv preprint arXiv:2011.13456 , year=

    Pith/arXiv arXiv 2011

  10. [11]

    International conference on machine learning , pages=

    Deep unsupervised learning using nonequilibrium thermodynamics , author=. International conference on machine learning , pages=. 2015 , organization=

    2015

  11. [12]

    Advances in neural information processing systems , volume=

    Elucidating the design space of diffusion-based generative models , author=. Advances in neural information processing systems , volume=

  12. [13]

    Proceedings of the 40th International Conference on Machine Learning , pages=

    Consistency models , author=. Proceedings of the 40th International Conference on Machine Learning , pages=

  13. [14]

    arXiv preprint arXiv:2210.02747 , year=

    Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year=

    Pith/arXiv arXiv

  14. [15]

    arXiv preprint arXiv:2010.02502 , year=

    Denoising diffusion implicit models , author=. arXiv preprint arXiv:2010.02502 , year=

    Pith/arXiv arXiv 2010

  15. [16]

    Advances in neural information processing systems , volume=

    Diffusion-lm improves controllable text generation , author=. Advances in neural information processing systems , volume=

  16. [17]

    arXiv preprint arXiv:2212.10325 , year=

    Seqdiffuseq: Text diffusion with encoder-decoder transformers , author=. arXiv preprint arXiv:2212.10325 , year=

    Pith/arXiv arXiv

  17. [18]

    Advances in neural information processing systems , volume=

    Structured denoising diffusion models in discrete state-spaces , author=. Advances in neural information processing systems , volume=

  18. [19]

    Advances in Neural Information Processing Systems , volume=

    Simple and effective masked diffusion language models , author=. Advances in Neural Information Processing Systems , volume=

  19. [20]

    arXiv preprint arXiv:2406.03736 , year=

    Your absorbing discrete diffusion secretly models the conditional distributions of clean data , author=. arXiv preprint arXiv:2406.03736 , year=

    Pith/arXiv arXiv

  20. [21]

    arXiv preprint arXiv:2502.09992 , year=

    Large language diffusion models , author=. arXiv preprint arXiv:2502.09992 , year=

    Pith/arXiv arXiv

  21. [22]

    arXiv preprint arXiv:2508.15487 , year=

    Dream 7b: Diffusion large language models , author=. arXiv preprint arXiv:2508.15487 , year=

    Pith/arXiv arXiv

  22. [23]

    arXiv preprint arXiv:2203.11171 , year=

    Self-consistency improves chain of thought reasoning in language models , author=. arXiv preprint arXiv:2203.11171 , year=

    Pith/arXiv arXiv

  23. [24]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  24. [25]

    International conference on machine learning , pages=

    Reliable fidelity and diversity metrics for generative models , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  25. [26]

    Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    A diversity-promoting objective function for neural conversation models , author=. Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    2016

  26. [27]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  27. [28]

    Advances in Neural Information Processing Systems , year=

    SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score , author=. Advances in Neural Information Processing Systems , year=

  28. [29]

    High-Resolution Image Synthesis with Latent Diffusion Models , booktitle =

    Robin Rombach and Andreas Blattmann and Dominik Lorenz and Patrick Esser and Bj. High-Resolution Image Synthesis with Latent Diffusion Models , booktitle =. 2022 , pages =

    2022

  29. [30]

    Journal of Machine Learning Research , volume=

    Cascaded diffusion models for high fidelity image generation , author=. Journal of Machine Learning Research , volume=

  30. [31]

    arXiv preprint arXiv:2211.13221 , year=

    Latent video diffusion models for high-fidelity long video generation , author=. arXiv preprint arXiv:2211.13221 , year=

    Pith/arXiv arXiv

  31. [32]

    arXiv preprint arXiv:2311.15127 , year=

    Stable video diffusion: Scaling latent video diffusion models to large datasets , author=. arXiv preprint arXiv:2311.15127 , year=

    Pith/arXiv arXiv

  32. [33]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Align your latents: High-resolution video synthesis with latent diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  33. [34]

    arXiv preprint arXiv:2112.10741 , year =

    Alex Nichol and Prafulla Dhariwal and Aditya Ramesh and Pranav Shyam and Pamela Mishkin and Bob McGrew and Ilya Sutskever and Mark Chen , title =. arXiv preprint arXiv:2112.10741 , year =

    Pith/arXiv arXiv

  34. [35]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Universal guidance for diffusion models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  35. [36]

    Jaakkola , booktitle=

    Gabriele Corso and Yilun Xu and Valentin De Bortoli and Regina Barzilay and Tommi S. Jaakkola , booktitle=

  36. [37]

    European Conference on Computer Vision , pages=

    Improving geo-diversity of generated images with contextualized vendi score guidance , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  37. [38]

    arXiv preprint arXiv:2502.08512 , year=

    Measuring Diversity in Synthetic Datasets , author=. arXiv preprint arXiv:2502.08512 , year=

    Pith/arXiv arXiv

  38. [39]

    Advances in neural information processing systems , volume=

    Assessing generative models via precision and recall , author=. Advances in neural information processing systems , volume=

  39. [40]

    arXiv preprint arXiv:2210.02410 , year =

    Dan Friedman and Adji Bousso Dieng , title =. arXiv preprint arXiv:2210.02410 , year =

    Pith/arXiv arXiv

  40. [41]

    Weber , title=

    Seyedmorteza Sadat and Jakob Buhmann and Derek Bradley and Otmar Hilliges and Romann M. Weber , title=. The Twelfth International Conference on Learning Representations , year=

  41. [42]

    2025 , eprint=

    Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency , author=. 2025 , eprint=

    2025

  42. [43]

    arXiv preprint arXiv:1610.02424 , year=

    Diverse beam search: Decoding diverse solutions from neural sequence models , author=. arXiv preprint arXiv:1610.02424 , year=

    Pith/arXiv arXiv

  43. [44]

    arXiv preprint arXiv:1904.09751 , year=

    The curious case of neural text degeneration , author=. arXiv preprint arXiv:1904.09751 , year=

    Pith/arXiv arXiv 1904

  44. [45]

    arXiv preprint arXiv:2407.01082 , year=

    Turning up the heat: Min-p sampling for creative and coherent llm outputs , author=. arXiv preprint arXiv:2407.01082 , year=

    arXiv

  45. [46]

    arXiv preprint arXiv:2412.19437 , year=

    Deepseek-v3 technical report , author=. arXiv preprint arXiv:2412.19437 , year=

    Pith/arXiv arXiv

  46. [47]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  47. [48]

    2025 , eprint=

    Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models , author=. 2025 , eprint=

    2025

  48. [49]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

    When Kernels Multiply, Clusters Unify: Fusing Embeddings with the Kronecker Product , author =. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

  49. [50]

    IEEE International Symposium on Information Theory (ISIT) , year =

    The Maximum von Neumann Entropy Principle: Theory and Applications in Machine Learning , author =. IEEE International Symposium on Information Theory (ISIT) , year =

  50. [51]

    International Conference on Machine Learning , year=

    KODA: Contrastive Representation Comparison and Alignment for Vision-Language Foundation Models , author=. International Conference on Machine Learning , year=

  51. [52]

    Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative

    Mohammad Jalali and Azim Ospanov and Amin Gohari and Farzan Farnia , booktitle=. Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative. 2026 , url=

    2026

  52. [53]

    Advances in Neural Information Processing Systems , volume=

    Towards a scalable reference-free evaluation of generative models , author=. Advances in Neural Information Processing Systems , volume=

  53. [54]

    International Conference on Machine Learning , pages=

    Towards an Explainable Comparison and Alignment of Feature Embeddings , author=. International Conference on Machine Learning , pages=. 2025 , organization=

    2025

  54. [55]

    International Conference on Machine Learning , pages=

    Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models , author=. International Conference on Machine Learning , pages=. 2025 , organization=

    2025

  55. [56]

    Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , pages =

    A Multi-Armed Bandit Approach to Online Selection and Evaluation of Generative Models , author =. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics , pages =. 2025 , volume =

    2025

  56. [57]

    Proceedings of the 26th International Conference on Machine Learning (ICML) , year=

    PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMs , author=. Proceedings of the 26th International Conference on Machine Learning (ICML) , year=

  57. [58]

    Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-

    Parham Rezaei and Farzan Farnia and Cheuk Ting Li , booktitle=. Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-. 2025 , url=

    2025

  58. [59]

    Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops , pages=

    On the distributed evaluation of generative models , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops , pages=

  59. [60]

    The 41st Conference on Uncertainty in Artificial Intelligence , year=

    Do Vendi Scores Converge with Finite Samples? Truncated Vendi Score for Finite-Sample Convergence Guarantees , author=. The 41st Conference on Uncertainty in Artificial Intelligence , year=

  60. [61]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =

    Ospanov, Azim and Jalali, Mohammad and Farnia, Farzan , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =. 2025 , pages =

    2025

  61. [62]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    An Interpretable Evaluation of Entropy-based Novelty of Generative Models , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , volume =

    2024

  62. [63]

    Unveiling Differences in Generative Models: A Scalable Differential Clustering Approach , author=

  63. [64]

    arXiv preprint arXiv:2505.18901 , year=

    PromptWise: Online Learning for Cost-Aware Prompt Assignment in Generative Models , author=. arXiv preprint arXiv:2505.18901 , year=

    arXiv

  64. [65]

    Forty-third International Conference on Machine Learning , year=

    MMD Guidance: Training-Free Distribution Adaptation for Diffusion Models via Maximum Mean Discrepancy Guidance , author=. Forty-third International Conference on Machine Learning , year=

  65. [66]

    Mixture-Greedy for Online Generative Model Selection: Is

    Bahar Dibaei Nia and Farzan Farnia , booktitle=. Mixture-Greedy for Online Generative Model Selection: Is. 2026 , url=

    2026

  66. [67]

    2026 , url=

    Donya Jafari and Farzan Farnia , booktitle=. 2026 , url=

    2026

  67. [68]

    ICML 2026 Workshop on Combining Theory and Benchmarks: Towards A Virtuous Cycle to Understand and Guarantee Foundation Model Performance , year=

    PromptSplit: Revealing Prompt-Level Disagreement in Generative Models , author=. ICML 2026 Workshop on Combining Theory and Benchmarks: Towards A Virtuous Cycle to Understand and Guarantee Foundation Model Performance , year=

    2026

  68. [69]

    Advances in Neural Information Processing Systems , volume=

    Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models , author=. Advances in Neural Information Processing Systems , volume=