{"id":"2ccb6d2d-9260-44c4-9694-30a14e00f7f8","arxiv_id":"2607.05319","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"SteeringDRL identifies two optimization regimes in diffusion autoencoders and uses gated residual U-Nets with a log SNR curriculum to steer training toward disentangled representations, improving performance across multiple benchmarks.","lead":"This paper finds that diffusion autoencoders can produce equally good images while learning very different internal representations, and traces this to two early-training 'regimes' — one prioritizing image fidelity, the other prioritizing meaningful structure. The authors propose a simple architectural fix (gated skip connections) plus a noise-level curriculum that steers training toward better representations, improving disentanglement benchmarks and reducing seed-to-seed方差.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Regime mechanism is validated only in an artificially regime-visible setting; large-model improvements may arise from standard regularization rather than regime steering.","rationale":"The reader correctly identified the most load-bearing concern: the causal mechanism (two regimes, H1, H2) is validated only in a low-capacity setting where regime separation is artificially maximized, and the extrapolation to larger models relies on performance improvements rather than direct mechanistic evidence. I agree with this assessment.\n\nThe paper is honest about this limitation—it explicitly states that larger models mask regime separation and that the VQ-VAE checkpoint was chosen to make regimes visible. This honesty is commendable. However, the paper still frames H1 and H2 as the causal drivers of the improvements at scale, which is not directly supported. The improvements are real and consistent across benchmarks, but the mechanistic explanation is an extrapolation.\n\nI recommend UNCHANGED rather than a harsher verdict because: (1) the empirical contributions are solid—consistent improvements across multiple datasets and metrics, thorough ablations, honest reporting of negative results (e.g., curriculum hurting standard U-Net); (2) the concern is about the causal interpretation rather than the empirical results themselves; (3) the paper acknowledges the limitation, even if it doesn't fully resolve it. A CONDITIONAL verdict with the caveat about mechanistic extrapolation is appropriate. The concrete test I propose (cross-attention visualization at scale) is feasible and would directly address whether the regime mechanism operates in the models where results are reported.","tokens_in":31979,"tokens_out":3301,"duration_ms":94734,"concrete_test":"Replicate the cross-attention map analysis from Figure 11 on the larger Shapes3D model from Table 1 (the EncDiff U-Net with ~10M+ params). Compare attention maps between (a) the baseline model and (b) the SteeringDRL model, at the same U-Net layer and log SNR level. If the baseline shows diffuse attention and SteeringDRL shows factor-aligned localised attention (as in the 3M model), this supports the regime mechanism at scale. If the attention patterns are similar or differ in a way inconsistent with the regime interpretation, the improvements are likely due to generic regularization rather than regime steering, and the causal framing should be softened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central causal claim is that two optimization regimes exist, driven by skip-connection shortcuts (H1) and early noise-level exposure (H2), and that SteeringDRL steers trajectories between them. However, the only direct evidence for the regime structure and its causal mechanisms comes from a 3M-parameter U-Net with an undertrained 30K-step VQ-VAE, which the authors explicitly selected because regime separation is 'most visible' there (Section E.5). In the larger models used for main results, the authors acknowledge that 'the baseline no longer exhibits regime separation and instead follows the reconstruction regime' (Section 6.1). This means the two-regime structure—the explanatory framework for why H1 and H2 matter—does not manifest in the models where the method's improvements are reported.\n\nThe concern is not merely that regimes are harder to see at scale. It is that the improvements in large models could be explained by simpler, well-known mechanisms without invoking regime dynamics: (1) gated residuals are a form of skip regularization known to improve representation alignment [29, 79, 77]; (2) sigmoid(-λ) loss-weighting is known to improve image fidelity [42]; (3) the log SNR curriculum is a form of curriculum learning. Table 1 shows that the gated residual U-Net alone does not improve DCI (0.832→0.826), and the curriculum alone hurts DCI when applied to the standard U-Net (0.932→0.853). The improvement only appears when all three components combine, which could reflect a generic interaction effect rather than the specific regime-steering mechanism the paper claims.\n\nThe paper does provide trajectory plots for larger models (Figure 3), but these show a single cluster shifting upward—not two regimes. Without direct mechanistic evidence at scale (e.g., cross-attention map alignment as in Figure 11, which was only shown for the 3M model), the claim that H1 and H2 operate through regime selection in large models remains an extrapolation from performance","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper studies optimisation dynamics in diffusion autoencoders for unsupervised representation learning, observing that training trajectories organise around two regimes (reconstruction vs. disentanglement) that are determined early in training. Two mechanisms are proposed: (1) gated residual U-Net blocks replacing skip connections to suppress reconstruction shortcuts (H1), and (2) a log SNR curriculum that gradually widens the noise-level band during training (H2). The combined method, SteeringDRL, is evaluated on attribute disentanglement (Shapes3D, Cars3D, MPI3D) and spatial disentanglement (ClevrTex, PascalVOC), showing improvements over EncDiff and SlotDiffusion baselines with reduced seed variance. The paper provides extensive ablations (Tables 1, 6–11) and trajectory visualisations (Figures 1, 3, 4, 12) to support the regime hypothesis and the individual contributions of each component.","tokens_in":32983,"tokens_out":1601,"duration_ms":203024,"significance":"The paper tackles a genuine question: why diffusion autoencoders with identical objectives and architectures can yield substantially different latent structures. The observation of two optimisation regimes is interesting and the proposed interventions (gated residuals, log SNR curriculum) are simple and well-motivated. The ablation study in Table 1 is thorough, testing each component independently and in combination across 10 seeds. The extension to spatial disentanglement with object-centric learning on both synthetic and real-world data (Tables 3, 4) demonstrates generality beyond the initial setting. The trajectory visualisations provide qualitative support for the regime hypothesis. The method is falsifiable: Figure 4 shows that different curricula produce different trajectories, and Table 11 ablates curriculum hyperparameters. The code release commitment would strengthen reproducibility.","major_comments":[{"comment":"Section 6.1 and Section E.5: The central causal claim—that two optimisation regimes exist and are driven by skip-connection shortcuts (H1) and early noise-level exposure (H2)—is validated only in a 3M-parameter U-Net with an undertrained 30K-step VQ-VAE checkpoint, where regime separation is 'most visible.' The authors acknowledge that 'larger models mask these patterns' (Section 4) and that the baseline 'no longer exhibits regime separation' at the scale used for main results (Section 6.1). This creates a gap between the explanatory framework (regime dynamics) and the models where improvements are reported. The improvements in larger models could potentially be explained by standard regularisation effects of gated residuals and curriculum learning without invoking regime dynamics. The paper should either (a) provide direct trajectory evidence of regime separation at the scale of the Enc","section":null},{"comment":"Table 1: The component-wise ablation reveals that the gated residual U-Net alone does not improve DCI (0.832→0.826) and the curriculum alone applied to the standard U-Net hurts DCI (0.932→0.853). The improvement only appears when all three components (gated residual + sigmoid(−λ) weighting + curriculum) are combined (DCI 0.926). This three-way interaction is load-bearing for the claim that H1 and H2 are the operative mechanisms, yet the paper does not analyse why the components are individually ineffective or counterproductive but jointly beneficial. An alternative explanation is that the gated residual changes the optimisation landscape in a way that makes the curriculum effective for reasons unrelated to regime steering. The paper should discuss this interaction more carefully, ideally with trajectory plots for each two-component combination (gated+weighting, gated+curriculum, standard","section":null},{"comment":"Section 6.1, Figure 3: The claim that SteeringDRL 'steers trajectories toward the disentanglement regime' in larger models is supported by trajectory plots, but the baseline in this setting 'no longer exhibits regime separation and instead follows the reconstruction regime' (Section 6.1). If there is no regime separation at this scale, it is unclear what 'toward the disentanglement regime' means operationally. The paper should clarify whether the trajectory shift in Figure 3 reflects the same regime dynamics observed in Figure 1, or whether it is simply an improvement in disentanglement metrics that is post-hoc labelled as regime steering. A more precise definition of what constitutes evidence of regime steering (as opposed to generic improvement) would strengthen the causal claim.","section":null}],"minor_comments":[{"comment":"Table 2: For MPI3D-toy at N=20, SteeringDRL achieves FactorVAE 0.870, which is lower than EncDiff (0.899) and DyGA (0.930). The text states the method 'remains competitive,' but the gap on FactorVAE is notable. This should be acknowledged more directly.","section":null},{"comment":"Table 5: The curriculum hyperparameters (initial band, widening steps) vary substantially across datasets ([0,10] with 32K steps for attribute disentanglement vs. [−2,2] with 200K steps for OCL). The paper does not explain how these are selected. A brief note on the selection criterion would improve reproducibility.","section":null},{"comment":"Section E.5: The VQ-VAE checkpoint choice (30K vs 150K steps) affects regime visibility, with the 30K checkpoint making regimes 'most visible.' Since the main results use the EncDiff setup (which appears to use a more trained VQ-VAE), the relationship between the analysis checkpoint and the main-results checkpoint should be stated more explicitly.","section":null},{"comment":"Figure 4 caption: 'C5 beats C2 in disentanglement' — the DCI values in Table 11 show C2 (initial band [−5,0]) achieves DCI 0.914 while C5 (initial band [0,10], 32K) achieves 0.926, so the claim is correct but the margin is within one standard deviation. This should be noted.","section":null},{"comment":"Section 3.3, Eq. (6): The derivative dλ/dt appears without explicit derivation. A brief note pointing to the noise schedule definition in Section A.1 would help readers.","section":null},{"comment":"Table 3: The STEERINGDRL rows with 50-step DDIM and 200-step DDPM share the same segmentation metrics (indicated by quotation marks). A footnote or merged rows would be clearer.","section":null},{"comment":"The paper would benefit from a brief discussion of computational overhead introduced by the gated residual U-Net and curriculum, particularly for the 138M-parameter PascalVOC model.","section":null},{"comment":"Section C.1: The FiLM-based amortised encoder is described as reducing parameters by approximately O(N) relative to the split-MLP. A quantitative comparison of parameter counts would be informative, especially since Table 1 shows it matches the split-MLP baseline.","section":null}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the regime mechanism being validated only in an artificially regime-visible setting is legitimate and is the primary reason for the major_revision recommendation. The authors are transparent about this limitation (Sections 4, 6.1, E.5), which is commendable, but the gap between the explanatory framework and the main-results setting needs to be addressed more directly. The paper is not circular—the components are independently motivated by ablations—but the causal story connecting the small-scale regime analysis to large-scale improvements is incomplete. If the authors can provide even partial trajectory evidence at the EncDiff scale, or reframe the contribution as 'practical inductive biases inspired by regime analysis' rather than 'regime steering at scale,' the paper would be substantially strengthened."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The core concern—the gap between the scale at which regime dynamics are most visible and the scale at which we report main results—is well-taken, and we agree the manuscript must do more to bridge it. Below we address each major comment in turn.","responses":[{"response":"The referee correctly identifies a genuine gap in the manuscript: the regime phenomenon is most clearly visible in a low-capacity setting, while the main results use larger models where the baseline no longer exhibits clear two-regime separation. We agree this gap must be addressed more carefully, and we will revise the manuscript accordingly (revision_made = 'partial'). Here is our honest assessment of what we can and cannot do. First, we acknowledge that we cannot fully close the gap with the experiments currently in the paper. We do not have trajectory evidence showing clean two-regime separation at the scale of the EncDiff U-Net used in Table 1. At that scale, the baseline trajectories cluster in what we label the 'reconstruction regime' rather than separating into two distinct regimes. This is a limitation we will state more explicitly. Second, we can partially bridge the gap with evidence already in the paper but not sufficiently highlighted. Figure 4 (and the extended version in Figure 13) shows that, with the gated residual architecture fixed, different curricula produce qualitatively different trajectory shapes—some prioritise early reconstruction, others maintain gradual disentanglement improvement. This is direct evidence that the curriculum steers optimisation dynamics at the larger scale, even though the baseline does not exhibit two-regime separation. The zeroed-skip ablation in Table 6 and Figure 12 provides analogous evidence for H1 at the larger scale: removing skip connections shifts trajectories toward the disentanglement regime, and adding the curriculum on top shifts them further. Third, we agree that an alternative explanation—standard regularisation effects of gated residuals and curriculum—cannot be ruled out with the current evidence. We will re","revision_made":"no","referee_comment":"Section 6.1 and Section E.5: The central causal claim—that two optimisation regimes exist and are driven by skip-connection shortcuts (H1) and early noise-level exposure (H2)—is validated only in a 3M-parameter U-Net with an undertrained 30K-step VQ-VAE checkpoint, where regime separation is 'most visible.' The authors acknowledge that 'larger models mask these patterns' (Section 4) and that the baseline 'no longer exhibits regime separation' at the scale used for main results (Section 6.1). This creates a gap between the explanatory framework (regime dynamics) and the models where improvements are reported. The improvements in larger models could potentially be explained by standard regularisation effects of gated residuals and curriculum learning without invoking regime dynamics. The paper should either (a) provide direct trajectory evidence of regime separation at the scale of the Enc"},{"response":"The referee is correct that the three-way interaction in Table 1 is load-bearing and under-analysed. We will revise the manuscript to discuss this interaction explicitly and add trajectory plots for the two-component combinations (revision_made = 'yes'). Our current understanding of the interaction is as follows. The curriculum alone applied to the standard U-Net hurts DCI (0.932→0.853) because, when skip connections are present, restricting early noise-level exposure delays reconstruction without redirecting the optimisation away from the reconstruction shortcut. The skip pathway allows the decoder to reconstruct from encoder features without relying on the semantic tokens, so the curriculum's effect on noise-level exposure does not translate into improved representation—it simply slows reconstruction. The gated residual alone does not improve DCI (0.832→0.826) because, while it suppresses the skip shortcut, the model still sees the full noise-level band from the start. Without curriculum control, early exposure to reconstruction-favouring noise levels can still bias the model toward the reconstruction regime, even with a weaker shortcut pathway. The combination works because the gated residual removes the high-bandwidth bypass that would otherwise absorb the curriculum's effect, while the curriculum controls the noise-level exposure that drives regime selection. This is consistent with the regime-steering explanation, but the referee is right that it is also consistent with an alternative: the gated residual changes the optimisation landscape in a way that makes the curriculum effective for reasons unrelated to regime dynamics. We cannot fully distinguish between these explanations with the current evidence. However, the zeroed-skip + curriculum ablation in Table 6 is","revision_made":"no","referee_comment":"Table 1: The component-wise ablation reveals that the gated residual U-Net alone does not improve DCI (0.832→0.826) and the curriculum alone applied to the standard U-Net hurts DCI (0.932→0.853). The improvement only appears when all three components (gated residual + sigmoid(−λ) weighting + curriculum) are combined (DCI 0.926). This three-way interaction is load-bearing for the claim that H1 and H2 are the operative mechanisms, yet the paper does not analyse why the components are individually ineffective or counterproductive but jointly beneficial. An alternative explanation is that the gated residual changes the optimisation landscape in a way that makes the curriculum effective for reasons unrelated to regime steering. The paper should discuss this interaction more carefully, ideally with trajectory plots for each two-component combination (gated+weighting, gated+curriculum, standard"},{"response":"The referee raises a valid conceptual point: if there is no two-regime separation at the larger scale, the phrase 'toward the disentanglement regime' is not well-defined. We will revise the manuscript to clarify this (revision_made = 'yes'). Our intended meaning is the following. In the low-capacity setting (Figure 1), the disentanglement regime is characterised by two properties: (1) reconstruction improves more gradually, and (2) disentanglement improves progressively throughout training rather than being deferred. In the larger-scale setting (Figure 3), the baseline follows what we call the reconstruction regime—fast early reconstruction with slower disentanglement improvement. SteeringDRL shifts the trajectory so that reconstruction is slower and disentanglement improves more progressively, which is qualitatively consistent with the disentanglement regime pattern from Figure 1. However, we agree that without two-regime separation at the larger scale, calling this 'regime steering' rather than 'trajectory improvement' involves an extrapolation from the low-capacity setting. We will make this explicit in the revision: the trajectory shift in Figure 3 is qualitatively consistent with the regime dynamics observed in Figure 1, but we cannot confirm it is the same phenomenon. We will also provide a more precise operational definition: evidence of regime steering (as opposed to generic improvement) requires that the trajectory shape changes in a way that matches the regime characteristics—specifically, delayed reconstruction with progressive disentanglement improvement, rather than uniform improvement on both axes. By this definition, Figure 3 shows trajectory shapes consistent with regime steering, but the causal link to the low-capacity regime dynamics remains an infer","revision_made":"no","referee_comment":"Section 6.1, Figure 3: The claim that SteeringDRL 'steers trajectories toward the disentanglement regime' in larger models is supported by trajectory plots, but the baseline in this setting 'no longer exhibits regime separation and instead follows the reconstruction regime' (Section 6.1). If there is no regime separation at this scale, it is unclear what 'toward the disentanglement regime' means operationally. The paper should clarify whether the trajectory shift in Figure 3 reflects the same regime dynamics observed in Figure 1, or whether it is simply an improvement in disentanglement metrics that is post-hoc labelled as regime steering. A more precise definition of what constitutes evidence of regime steering (as opposed to generic improvement) would strengthen the causal claim."}],"tokens_in":32041,"tokens_out":1653,"duration_ms":234233,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Here's the short version: this paper introduces SteeringDRL (gated residual U-Net + log SNR curriculum) for improving disentangled representations in diffusion autoencoders, and it works across multiple benchmarks. The mechanistic explanation — two optimization regimes driven by skip shortcuts and noise-level exposure — is interesting but only directly validated in a small model where the effect is most visible. The method stands on its empirical merits; the causal story is a softer claim that the paper honestly acknowledges is not fully verified at scale.","headline":"Solid empirical method paper with an interesting but under-validated mechanistic story","tokens_in":32827,"tokens_out":758,"would_cite":false,"duration_ms":108284,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Two training tricks steer diffusion autoencoders toward disentangled representations","keywords":[],"falsifier":"If the gated residual and curriculum interventions improve disentanglement in larger models through a mechanism unrelated to the two-regime structure (for example, by regularizing the loss landscape or changing gradient noise), then the paper's causal story about skip shortcuts and noise-level exposure steering regime selection would not hold, even though the practical improvements would remain.","tokens_in":32262,"feed_emoji":"🎚️","tokens_out":846,"duration_ms":101432,"temperature":0.7,"pith_summary":"This paper argues that diffusion autoencoders can produce equally good images while learning qualitatively different latent representations, and that which representation a model arrives at is determined early in training by two mechanisms: skip connections that let the U-Net bypass the semantic pathway (a reconstruction shortcut), and the range of noise levels the model sees in its first steps (which sets the relative pace of image fidelity versus semantic organization). The authors observe that training trajectories cluster into two regimes, a reconstruction regime where image fidelity is prioritized early and disentanglement lags, and a disentanglement regime where both improve gradually. They propose SteeringDRL, which replaces skip connections with gated residuals initialized near zero to suppress the reconstruction shortcut, and introduces a log SNR curriculum that starts training on a narrow band of noise levels and gradually widens it. The combined method improves attribute disentanglement metrics (DCI, FactorVAE, MIG) on Shapes3D, Cars3D, and MPI3D, improves spatial disentanglement (segmentation quality) on ClevrTex and PascalVOC, and reduces variance across random seeds.","feed_headline":"Two tricks steer diffusion autoencoders toward disentangled latent codes","feed_subtitle":"Gated residuals and a noise-level curriculum shift training trajectories from reconstruction shortcuts toward semantic structure, cutting","key_machinery":"Gated residual U-Net blocks with near-zero-initialized gates replacing skip connections; log SNR curriculum that linearly widens the sampled noise-level band from an initial range to the full training range over a fixed number of steps, with an importance correction to maintain the correct loss weighting.","core_discovery":"The central object is the optimization trajectory: a curve of reconstruction quality plotted against representation quality during training. The authors show these trajectories bifurcate into two regimes determined early, and that the regime can be steered by two concrete interventions. The first replaces U-Net skip connections with gated residuals initialized near zero, forcing the decoder to rely on semantic tokens rather than a high-bandwidth pixel bypass. The second is a log SNR curriculum that restricts the noise-level band at the start of training and linearly widens it, controlling when the model encounters reconstruction-favorable noise levels. Together these shift trajectories fromt","pith_inferences":[],"forward_implications":["Diffusion autoencoders with identical objectives and architectures can converge to substantially different latent structures depending on early training dynamics, suggesting that random seed sensitivity in representation learning is partly a trajectory-selection problem rather than purely a capacity or data issue.","The log SNR curriculum is a lightweight intervention that can be applied to existing diffusion training pipelines without architectural changes, potentially improving representation quality in any diffusion model that uses a U-Net with skip connections.","The finding that skip connections act as reconstruction shortcuts that bypass semantic conditioning suggests that architectural choices in diffusion U-Nets have representation-learning consequences that are invisible when only measuring image fidelity.","The regime structure, if it generalizes to transformer-based diffusion architectures where skip pathways are already constrained, may interact differently with the curriculum and could require modified interventions."],"fun_headline_variants":["Steering diffusion autoencoder trajectories toward disentangled latents","Gated residuals and noise curriculum steer diffusion autoencoder latents","Steering diffusion autoencoders away from reconstruction shortcuts","Controlling noise exposure steers diffusion autoencoder trajectories","Diffusion autoencoder training trajectories split into two early regimes"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The two-regime structure and its causal mechanisms are identified and validated primarily in a 3M-parameter U-Net with an undertrained VQ-VAE checkpoint, where regime separation is most visible. The authors note that larger models mask these patterns and that better VQ-VAE training reduces regime separation. The extrapolation that the same mechanisms operate in the larger models used for main results rests on performance improvements rather than direct trajectory evidence at","fun_headline_variants_meta":{"raw":{"variants":["Steering diffusion autoencoder trajectories toward disentangled latents","Gated residuals and noise curriculum steer diffusion autoencoder latents","Steering diffusion autoencoders away from reconstruction shortcuts","Controlling noise exposure steers diffusion autoencoder trajectories","Diffusion autoencoder training trajectories split into two early regimes"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1061,"prompt_tokens":471,"completion_tokens":590,"prompt_tokens_details":null},"tokens_in":471,"tokens_out":590,"duration_ms":26709,"temperature":1.0,"reasoning_tokens":585,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T17:39:36.894722+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the gated residual and curriculum interventions improve disentanglement in larger models through a mechanism unrelated to the two-regime structure (for example, by regularizing the loss landscape or changing gradient noise), then the paper's causal story about skip shortcuts and noise-level exposure steering regime selection would not hold, even though the practical improvements would remain.","supporting_citations":[],"review_version":1}