{"id":"c0df48b0-07b5-4d9b-bd36-c3edd9fdfa90","arxiv_id":"2602.09651","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Class-conditional entropy of noisy states peaks at a sharp speciation time in variance-preserving diffusion and can be estimated in trained models to localize semantic decisions during generation.","lead":"This paper shows that the uncertainty a diffusion model has about an image's class spikes at a specific moment during generation, marking when the image's semantic identity is decided. Tracking this entropy signal isolates when different visual features are locked in and how guidance shifts that timing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The empirical estimator is not shown to measure the Section 4 entropy: Appendix B's p(pi)=0.5 turns the partitioned quantity into a Jensen-Shannon divergence, and the unconditional-model proxy alters the complement posterior; the asserted transfer of the speciation-time result is unproven.","rationale":"The reader's weakest-assumption analysis correctly identified the central gap: the empirical estimator deviates from the theoretical quantity in ways that are acknowledged but not derived. My stress-test confirms this is the most load-bearing vulnerability. The theoretical GMM calculation appears internally consistent: the SNR scaling in Eq. 15 and the resulting t_s in Eq. 16 follow from the stated scaling assumptions, and the synthetic curves in Figure 2 support the claimed concentration. The unresolved issue is the bridge to trained models. Appendix B changes the prior to p(pi)=0.5 and uses an unconditional proxy, which converts the quantity into a Jensen-Shannon divergence or an overlap measure evaluated on a mixture that is not the two-component model of Section 4.2. The sentence 'the results from Section 4 still hold' is an assertion without proof, and the paper presents no synthetic validation of Algorithm 1 against the exact H[Z|X_t]. The EDM2-XS experiments further illustrate the gap: Section A.2.3 predicts no sharp transition for EDM/VE processes, yet Figure 1 shows narrow entropy-production peaks on EDM2-XS; this is only a contradiction if the measured quantity is assumed to be the Section 4 entropy, which is exactly what is unproven. I do not see a fatal flaw in the theory itself, and the empirical method may well be useful; but the central claim linking them is conditional on this unverified identification. The reader's CONDITIONAL verdict is therefore appropriate, and my analysis does not move it.","tokens_in":16940,"tokens_out":16144,"duration_ms":159281,"concrete_test":"On the same d-dimensional Gaussian mixture used in Section 4 (equiprobable components, ||mu_i - mu_k||^2/d = delta_ik^2), compute three quantities for d = 256, 1024, 4096, 16384: (a) the exact full class-conditional entropy H[Z|X_t] of Eq. 7; (b) the Algorithm 1 quantity with p(pi)=0.5 and the exact class-complement posterior; (c) the Algorithm 1 quantity with p(pi)=0.5 and the unconditional mixture used as proxy for the complement, as in Section 5.2. Locate the entropy-production peak of each and compare its time to t_s = 1/2 log d + O(1) and to the peak of (a). If (b) or (c) peaks at a time that does not converge to t_s, or if the peak width differs by O(1) in t, the empirical validation does not support the theoretical claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is the asserted identification between Algorithm 1's estimator and the theoretical H[Z|X_t] analyzed in Section 4. The theory derives the speciation-time result (Eq. 16) for a full N-way equiprobable class variable under a Gaussian mixture, where the posterior is exactly the softmax over pairwise log-ratios (Eq. 12). Algorithm 1, however, sets p(pi)=0.5 for every binary partition and, on ImageNet, replaces p(X_t|Z!=i) with the unconditional p(X_t) (Section 5.2). Appendix B explicitly states that this makes the quantity a Jensen-Shannon divergence, and then asserts 'the results from Section 4 still hold' without a derivation. This is not a minor detail: the SNR scaling in Eq. 15 is derived for a pair of Gaussian components, while a class-vs-unconditional comparison involves a mixture of many components with different effective separations. The empirical curves in Figures 1 and 3 are therefore measurements of a different functional, and it remains unproven that their peaks occur at the Section 4 speciation time or even share its scaling. The paper itself flags related limitations (Section 6), but does not connect them to the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes tracking the class-conditional entropy H[Z|X_t] (and its time derivative) along a diffusion trajectory as a signature of semantic commitment. In a high-dimensional Gaussian mixture with a variance-preserving kernel, the authors derive that entropy production concentrates on the speciation time ts = 1/2 log d + O(1), matching the symmetry-breaking instability of Biroli et al. (2024). To make this operational, they define a partitioned class-conditional entropy and an online posterior-tracking estimator (Algorithm 1, following Koulischer et al. 2025a). They apply the method to EDM2-XS on ImageNet and Stable Diffusion 1.5, reporting that entropy production peaks in narrow intermediate noise ranges and that guidance redistributes this production over time. The paper also includes a hierarchical-branching discussion and limitations section.","tokens_in":17379,"tokens_out":8771,"duration_ms":83738,"significance":"If rigorously established, the paper would provide a practically computable information-theoretic diagnostic that connects statistical-physics descriptions of diffusion with observable class-commitment dynamics in large trained models. The theoretical development is parameter-free (no fitted speciation time), and the empirical experiments span two substantial model families. The main risk is that the estimator used in the experiments computes a different functional than the one analyzed in Section 4: Appendix B states that setting p(pi)=0.5 for non-exhaustive partitions converts the quantity into a Jensen-Shannon divergence, and on ImageNet the unconditional model is used as a proxy for the complement posterior. The claimed transfer of the Section 4 speciation-time result to these surrogate quantities is not proven. The transition-width argument in Appendix A is also heuristic, relying on endpoint values rather than explicit bounds. These gaps are substantial but appear addressable within the manuscript's scope, so the paper warrants major revision rather than rejection.","major_comments":[{"comment":"The estimator with p(pi)=0.5 for non-exhaustive partitions makes the quantity a Jensen-Shannon divergence (binary mutual information under a uniform prior), as the paper itself states. Section 4's derivation applies to an N-way equiprobable class variable whose posterior is the softmax over pairwise log-ratios (Eq. 12). No derivation is given for the JS divergence between a single Gaussian component and a multi-component mixture (or between two prompt distributions). Equation (15) is a pairwise SNR; a class-vs-complement comparison involves a mixture with many different effective separations. The sentence 'the results from Section 4 still hold' is an assertion, not a proof. This gap directly affects the claim that the empirical peaks in Figures 1 and 3 occur at the Section 4 speciation time or share its scaling. Please derive the transition for the estimated functional, or explicitly ref","section":"Appendix B ('Choosing the prior'); Algorithm 1; Section 4.3"},{"comment":"Using the unconditional model as a proxy for p(X_t | Z != i) does not define a partition of the class variable, because p(X_t) includes the class i itself. The resulting posterior is proportional to p(x|i)/(p(x|i)+p(x)), a monotone transform of p(i|x), not the binary partition posterior analyzed in Section 4. The paper acknowledges possible bias in Section 6, but does not explain why this proxy would preserve the transition time or its O(1) width. Since the ImageNet experiments are the primary empirical validation of the theory, this proxy needs a theoretical justification (e.g., in a controlled Gaussian setting with known complement) or a direct comparison against an estimator that samples the true complement.","section":"Section 5.2; Appendix B ('Approximating the complement')"},{"comment":"The proof of a sharp transition at u=1 infers a 'spike' in entropy production from endpoint values: the conditional entropy is zero at the beginning and approximately ln(N) at the end. This only shows that the total change is O(log N); it does not bound the width of the transition interval. To establish Eq. (16)'s O(1) time window, one must analyze the entropy production for u = 1 +/- c/log d and show it decays as d -> infinity away from the window. Without such bounds, the claim that the transition has constant width in t (or width O(1/log d) in u) is not proven. Please provide explicit asymptotic estimates or, if only the location is proven, state the width as a conjecture.","section":"Appendix A.2.2; Eq. (16)"},{"comment":"There is a sign inconsistency: Eq. (8) gives  ˙H[Z|X_t] = - (g_t^2/2) E_i Δ_i(t), while Appendix A.1 concludes ˙H = + (g_t^2/2) E_{x,i} ||s_i - s_mix||^2. Since forward-time H[Z|X_t] increases from 0 toward log N, the derivative should be nonnegative, so the minus sign in Eq. (8) appears incorrect. Additionally, the equality between E[||s_i||^2 - ||s_mix||^2] and E[||s_i - s_mix||^2] in Eq. (18) is not generally true for a mixture; the cross term E[(s_i - s_mix)·s_mix] need not vanish. This affects the interpretation of entropy production as a Fisher divergence and needs correction or a qualifying asymptotic statement.","section":"Eq. (8) and Appendix A.1 (Eq. (18))"}],"minor_comments":[{"comment":"Typographical errors: 'termporal' (Figure 1 caption), 'uni modal' (Section 5.2), 'fine rgained' and 'the the snow' (Appendix C.2). A proofreading pass is needed.","section":"Throughout"},{"comment":"The notation p_π^t(·) is used but not defined. Please define the marginal density of X_t under the π-induced mixture and clarify how it relates to p(X_t | π=0) and p(X_t | π=1) in the non-exhaustive case.","section":"Eq. (17)"},{"comment":"Line 3 initializes H_τ ← 1. This appears to assume a 1-bit entropy at the starting noise level. It is unclear whether this is an initialization convention or a computed quantity; please explain.","section":"Algorithm 1"},{"comment":"The text says the entropy can be interpreted as a measure of overlap between marginal distributions of two prompts. The relation of this interpretation to Eq. (17) is informal; a precise statement would help.","section":"Section 5.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is promising, but the gulf between the Section 4 theory (N-way equiprobable mixture) and the Algorithm 1 estimator (uniform-prior binary JS divergence, with an unconditional-model proxy on ImageNet) is the main technical risk. The authors are aware of the proxy issues but do not connect them to the central claim. In addition, the transition-width argument and the sign/Fisher-divergence identity in Appendix A need correction. These are fixable with additional analysis or careful restatement, so I do not recommend rejection, but the revision must address them explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper gives the field something usable: a time-localized entropy signal for when class decisions happen during diffusion sampling, with a parameter-free asymptotic match to the Biroli et al. speciation time. That is a real contribution, and the theoretical core is in good shape. For an equiprobable N-way Gaussian mixture under the VP kernel, the derivation that entropy production concentrates on ts = 1/2 log d is straightforward and, as far as I can tell, correct. The entropy-production formula as a Fisher divergence is a neat and useful identity, and the contrast with VE/EDM (no sharp transition in rescaled time) is honest and worth having.\n\nWhat is genuinely new is the object itself, not just the scaling: tracking H[Z|X_t] and its derivative as a dynamic signature, plus the partitioned variant for hierarchical semantics. That is a practical idea with subfield-level impact. The empirical demonstrations on EDM2-XS and SD1.5 are plausible and qualitatively consistent with the theory, and the guidance-redistribution analysis is a nice bonus.\n\nNow the soft spots, in proportion. The weakest link is the identification between Algorithm 1's estimator and the Section 4 quantity. The theory is for the full class variable with N equiprobable components. The empirical estimator sets p(pi)=0.5 for every binary partition, which Appendix B admits turns the quantity into a Jensen-Shannon divergence, and on ImageNet it replaces the class-complement posterior with the unconditional model. Appendix B asserts the Section 4 results still hold, but no derivation is given. A class-vs-unconditional comparison is not the same as a pairwise Gaussian comparison, and the effective SNR scaling could differ. So the empirical peaks are measurements of a different functional, and the paper does not prove they share the speciation-time scaling. This is a real gap, but not a fatal one: the diagnostic can still work empirically even if the exact theoretical link is unproven, and the paper does flag related limitations in Section 6. The fix is straightforward in principle: a small numerical experiment on Gaussian mixtures using the same estimator, or an actual derivation for the binary partition with p=0.5. Also, there are no error bars anywhere in the figures, and the EDM theory-experiment tension (EDM-style models supposedly have no sharp transition, yet the empirical curves look sharp) is left hanging.\n\nWho is this for? Anyone working on time-localized guidance, semantic emergence, or interpretability of diffusion models. It deserves a serious referee, not a desk reject. My recommendation: send it to review, but the authors should be pushed to close the estimator-theory gap and add some error bars before acceptance.","headline":"Useful diagnostic with a clean Gaussian-mixture result; the empirical estimator drifts from the theory in a way the paper asserts rather than proves.","tokens_in":17726,"tokens_out":1603,"would_cite":true,"duration_ms":17195,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Class-conditional entropy marks the speciation transition in diffusion models.","keywords":["class-conditional entropy","diffusion models","speciation transition","symmetry breaking","entropy production","Gaussian mixture","classifier-free guidance","semantic emergence"],"falsifier":"On a labeled dataset where true class posteriors can be computed by forward-diffusing conditional samples, compare the entropy-production peak from Algorithm 1 against the exact posterior; if the peak's location or width deviates systematically from the theoretical ts = 1/2 log d and O(1) rescaled width—or if a sharp peak appears in an EDM-schedule model where the theory predicts a sqrt(d)-broadened transition—the signature fails.","tokens_in":16902,"feed_emoji":"🎯","tokens_out":6431,"duration_ms":60780,"temperature":0.7,"pith_summary":"Diffusion models resolve semantic identity not gradually but in a narrow time window, and this paper argues that the class-conditional entropy—the remaining uncertainty about a latent class given the noisy sample—drops sharply exactly there. In high-dimensional Gaussian mixtures under the variance-preserving kernel, the entropy production (its time derivative) concentrates at the logarithmic time scale ts = 1/2 log d, the same speciation time predicted by prior statistical-physics analysis. Restricting the entropy to binary partitions of the class space isolates when specific distinctions are decided, from coarse global attributes to fine details. The authors validate the signature on trained image diffusion models and show that guidance shifts semantic commitment earlier. If right, the result turns the phase-transition picture into a practical, estimable diagnostic for time-localized control of sampling.","feed_headline":"Entropy pinpoints when diffusion models commit to a class","feed_subtitle":"A measurable signature ties semantic choice to the speciation time predicted by physics.","key_machinery":"The central objects are the class-conditional entropy H[Z|X_t] and its time derivative, which equals the expected Fisher divergence between class-conditional and unconditional score fields. The dynamics are carried by the pairwise log-posterior ratio between classes: under the Gaussian-mixture model this ratio is Gaussian with mean and variance controlled by the squared inter-class distance divided by the noise variance, summarized by an effective signal-to-noise ratio. Setting that ratio to O(1) at large dimension yields the speciation time ts = 1/2 log d for the variance-preserving kernel. Estimation in trained models uses an online posterior-tracking procedure that accumulates log-likelih","core_discovery":"The central claim is that entropy production, the time derivative of the class-conditional entropy, is a faithful and practical marker of class speciation. For an equiprobable mixture of Gaussians in dimension d, the pairwise log-posterior ratio between classes is Gaussian with mean and variance both scaling as d^{1-u} in the rescaled time u = t/ts, where ts = 1/2 log d for the variance-preserving kernel. Thus the posterior is nearly uniform for u > 1 and nearly a point mass for u < 1, with a transition of O(1) width around u = 1, producing a sharp peak in entropy production at the speciation time. The same calculation shows that variance-exploding and EDM-style kernels do not yield a sharp","pith_inferences":["Because the estimated quantity with a uniform prior is a Jensen–Shannon divergence, a direct test is that entropy-production peaks should coincide with the noise level where a simple linear classifier on noisy inputs attains maximal distinction between the class and its complement.","The class-dependent peak locations suggest per-class schedulers: spending more sampling steps where a given class's entropy production peaks could improve image quality, a prediction testable by comparing generation quality under class-adaptive schedules.","The hierarchical branching picture implies that pairwise partitions drawn from a semantic taxonomy should show peaks ordered by abstraction; on a labeled image dataset this ordering could be tested directly by choosing partitions at different hierarchy levels.","If guidance enforces commitment earlier, the framework suggests that prompt-specific guidance should be applied only after the target attribute's entropy-production window begins; the optimal intervals reported in the paper are consistent with this, though the causal claim is not yet established."],"forward_implications":["Entropy production peaks are a practical, model-agnostic indicator of when a diffusion model commits to a class, enabling noise-level-specific sampling interventions.","Partitioning the entropy resolves semantic decisions by abstraction level: coarse and global attributes commit at higher noise, fine and local attributes later.","Applying guidance within a limited interval shifts entropy production to higher noise, meaning semantic commitment happens earlier; the framework quantifies this redistribution.","The variance-preserving versus variance-exploding distinction implies that the sharp speciation window is tied to the schedule, with EDM-style schedules having a broader transition that matters for scheduler design.","The result unifies the statistical-physics picture of symmetry breaking with an information-theoretic observable that can be computed on real models."],"fun_headline_variants":["Entropy spikes mark when diffusion models pick a class","Class commitment in diffusion revealed by entropy rate","Entropy production flags diffusion's speciation moment","Speciation time in diffusion models read from entropy","Diffusion's class switch is visible in entropy output"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The empirical estimates are treated as the same class-conditional entropy analyzed in Section 4, even though the 0.5 prior makes them Jensen–Shannon divergences and the ImageNet experiments substitute the unconditional model for the class-complement posterior; if these surrogates do not share the theoretical speciation transition, the validation does not support the theory.","fun_headline_variants_meta":{"raw":{"variants":["Entropy spikes mark when diffusion models pick a class","Class commitment in diffusion revealed by entropy rate","Entropy production flags diffusion's speciation moment","Speciation time in diffusion models read from entropy","Diffusion's class switch is visible in entropy output"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000174,"raw_usage":{"total_tokens":1111,"prompt_tokens":728,"completion_tokens":383,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":312}},"tokens_in":472,"tokens_out":383,"duration_ms":4368,"temperature":1.0,"reasoning_tokens":312,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T02:43:16.855546+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a labeled dataset where true class posteriors can be computed by forward-diffusing conditional samples, compare the entropy-production peak from Algorithm 1 against the exact posterior; if the peak's location or width deviates systematically from the theoretical ts = 1/2 log d and O(1) rescaled width—or if a sharp peak appears in an EDM-schedule model where the theory predicts a sqrt(d)-broadened transition—the signature fails.","supporting_citations":[],"review_version":1}