{"id":"2eb042db-95db-4078-ad31-387d8d8661fb","arxiv_id":"2412.06295","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A PSNR-gated adaptive distillation step for consistency models improves one-step FID on CIFAR-10 and ImageNet 64x64 and transfers to SDXL and SD3-based text-to-image models.","lead":"This paper proposes a curriculum schedule for training consistency models: the teacher model steps further ahead when the student's output is already close, using a PSNR-based discrepancy threshold. The method reports one-step FID of 1.64 on CIFAR-10 and 2.18 on ImageNet 64x64 and shows gains on Stable Diffusion XL and 3.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline FID 1.64 is only reported for CCM+GAN; no static-schedule+GAN control exists, so the KDC schedule's causal contribution to the headline result is unverified.","rationale":"The reader identified teacher drift as the weakest assumption, which is a legitimate risk and is even acknowledged in Section 3.3. However, the more immediate gap in the paper's evidence is the missing GAN-controlled ablation. The final FID numbers are achieved with a composite objective, yet the only evidence for the adaptive schedule's contribution comes from a no-GAN setting where the gain over the best static schedule is small (9.32 vs 9.89) and unreplicated. This is not an external-consensus disagreement or an ad hominem issue; it is an internal experimental-design gap that directly affects the central attribution claim. A concrete fix is straightforward and inexpensive: run the static best schedule with the same GAN loss. If that matches the 1.64 FID, the paper's core contribution is not supported; if it does not, the claim becomes much stronger. The teacher-drift concern is also worth testing, but it is secondary because even a perfect teacher would not rescue the causal claim if the static+GAN baseline matches CCM. Therefore the verdict should remain conditional rather than accept, since the missing control is a necessary condition for the headline claim to be meaningful.","tokens_in":12890,"tokens_out":5605,"duration_ms":63147,"concrete_test":"Retrain the best static schedule from Table 5 (l=0.09, n=3, s=0.03) on CIFAR-10 with the exact same GAN loss, discriminator architecture, hyperparameters, and training budget as the final CCM model, then compare one-step FID to CCM+GAN. Also run the same comparison for CCM and static schedules without the GAN loss to confirm the 0.57 FID gap persists. If static+GAN achieves FID within 0.1 of 1.64, the headline improvement cannot be attributed to the KDC schedule; if CCM+GAN beats static+GAN by a margin exceeding the reported noise in the table, the central claim is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that the KDC-guided adaptive distillation step, rather than some other component, improves one-step generation. However, every headline result (CIFAR-10 FID 1.64, ImageNet 64 FID 2.18, and the SDXL/SD3 tables) is obtained with the full CCM pipeline including GAN loss (Eq. 8-9). The only controlled comparison of the adaptive schedule itself is Table 5, which is explicitly 'without adversarial losses.' In that table, CCM (dynamic, s=0.03) achieves FID 9.32, while the best static strategy (l=0.09, n=3, s=0.03) achieves 9.89—a small gap of 0.57 FID. Nothing in the paper shows whether that gap survives when GAN loss is added. It is entirely possible that the static schedule plus the same GAN objective matches or beats CCM+GAN, in which case the headline 1.64 FID would be attributable to the GAN component and not to the core curriculum mechanism. The paper also does not report error bars or seeds, so we cannot tell whether the 0.57 FID gap is even significant. This is a load-bearing attribution gap: the paper's own ablation sets up the exact control needed, but the control is never run with the final training objective. The teacher-drift concern raised in the reader's verdict is real but secondary: if the static+GAN baseline also matches CCM, the adaptive schedule's benefit is not established regardless of teacher accuracy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes the Curriculum Consistency Model (CCM), a modification of consistency distillation in which the distillation interval is chosen adaptively rather than fixed. At a sampled timestep t, the student prediction is compared with teacher rollouts advanced in small steps of size s; the rollout stops when the PSNR-based Knowledge Discrepancy of the Curriculum (KDC) exceeds a fixed threshold T_KDC, and the student is trained against the resulting target together with an adversarial loss. Experiments report one-step FID scores of 1.64 on CIFAR-10 and 2.18 on ImageNet 64x64, and improvements in CLIP score, FID, and T2I-CompBench over LCM and PCM on SDXL and SD3. Ablations on CIFAR-10 show that the dynamic schedule outperforms static schedules when no GAN loss is used.","tokens_in":13399,"tokens_out":4822,"duration_ms":48977,"significance":"If the reported results hold, the paper would establish a simple and general recipe for stabilizing consistency-distillation difficulty across timesteps, with a rare demonstration on both diffusion-based and flow-matching text-to-image models. The core mechanism is concrete, computationally cheap, and the CIFAR-10 ablations in Table 5 do show a consistent ranking of dynamic over static schedules under the distillation-only objective. The extension to SDXL and SD3 gives the work practical relevance. However, the absence of GAN-matched controls for the schedule, the lack of error bars, and incomplete training details currently prevent the evidence from supporting the headline attribution at the level claimed.","major_comments":[{"comment":"The causal contribution of the KDC schedule to the headline numbers is not established. All headline results in Tables 1–4 are reported for the full objective including adversarial losses, while the only direct comparison between static and dynamic schedules is explicitly run \"without adversarial losses\" in Table 5. In that table, CCM's advantage over the best static strategy is 9.32 vs. 9.89 FID, and no seeds or error bars are reported, so the gap may not be statistically meaningful. A static schedule combined with the same GAN objective could match or beat CCM+GAN, in which case the headline FIDs would be attributable to the adversarial component rather than to the curriculum mechanism. The authors should run the best static strategy (l=0.09, n=3, s=0.03) and at least one other static baseline with the identical GAN loss and training budget, and report FID over multiple seeds.","section":"§4.3, Table 5 vs. Tables 1–3, Eq. (9)"},{"comment":"The adaptive schedule is endogenous to the student being trained, and the threshold T_KDC is a tuned hyperparameter. KDC is computed between the current student fθ and the target model fθ−, so the stopping condition depends on the very model being trained. The threshold T_KDC=60 is used for all datasets and model scales, but the sensitivity analysis in Figure 9 is only shown for CIFAR-10. To support the claim that a single threshold is appropriate, the authors should provide threshold sensitivity or an adaptation mechanism for ImageNet 64x64 and for the SDXL/SD3 experiments, and should state explicitly how T_KDC was selected.","section":"§3.3, Eq. (6), Algorithm 1, Figure 9"},{"comment":"The stopping criterion does not protect against teacher drift. The paper itself notes that the teacher model may make inaccurate predictions due to large timestep sizes in the same section, and because the loop only checks KDC between the student and the teacher-produced target, a low-quality target can still satisfy the threshold and the algorithm will appear to work while training against noisy labels. Please quantify target quality, for example by reporting PSNR between multi-step teacher rollouts and reference trajectory estimates, or by comparing CCM against a variant that caps the distillation interval l or uses a fixed high-quality target schedule.","section":"§3.3 and Algorithm 1"},{"comment":"The large-scale experiments omit several details needed to reproduce or interpret them: the PSNR computation for latent-space models (latent channels, dynamic range, whether computed in latent space or after decoding), the LoRA rank and insertion points, the value and scheduling of λGAN, the ODE solver and discretization for the teacher rollouts, and the exact COCO-2017 5K split used for evaluation. The user study in Figure 5 is also described only as \"subjects were shown generated images and asked for preference,\" with no number of subjects, protocol, or significance test. These omissions make the text-to-image generalization claims difficult to verify.","section":"§4.1 and Appendix A"}],"minor_comments":[{"comment":"The phrase \"to access knowledge discrepancy\" should read \"to assess knowledge discrepancy.\"","section":"§3.2"},{"comment":"The definition of KDC uses (2^n − 1)^2 with n representing bit depth, but the text does not state the range of xest and xtarget; this is especially important for latent-space models, where values are not 8-bit image intensities.","section":"Eq. (6)"},{"comment":"Inside the loop, t and xt are reassigned (lines 10–11), so the subscript t in KDCu_t and in xest becomes ambiguous. Use a separate variable such as t0 for the initial time and keep xest fixed.","section":"Algorithm 1"},{"comment":"The CTM entry appears as \"11.92\"; clarify whether this is NFE=1 with FID=1.92 or another configuration, and add a column separator for readability.","section":"Table 2"},{"comment":"The caption writes l = Pn i=1 si, but for dynamic rows such as l=0.1t and l=1−t, l is a function of t rather than a fixed quantity; state explicitly that these are functional forms.","section":"Table 5"},{"comment":"The text says KDC stability across training iterations is shown in Section 3, but Figure 9 reports only final FID as a function of T_KDC; either add the promised KDC-versus-iteration plot or point to the correct figure.","section":"§3.3 and Figure 9"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely of interest to the consistency-distillation community, and the core idea is simple enough to be worth testing. The main risk is the attribution gap: the only controlled comparison of the adaptive schedule is distillation-only, while the headline results all use the GAN objective. I would ask the editor to require the static+GAN control and at least a few seeds with error bars before further consideration. The novelty relative to iCT and ECM is moderate, and the absence of released code makes the text-to-image results hard to check; these points should be weighed in the decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is simple and worth taking seriously: measure the student–teacher gap in PSNR terms, and when the gap is small, let the teacher iterate further so the student always faces a roughly constant difficulty. That framing is new relative to iCT and ECM, which use fixed decaying schedules, and the extension to flow matching and to SDXL/SD3 is genuinely useful. The paper also does some things right: it shows KDC behavior across datasets and models, includes ablations for static versus dynamic schedules, tests single-step versus multi-step teacher targets, and checks sensitivity to its main threshold. Section 3.3 even acknowledges the teacher-drift risk. That is more than many distillation papers bother to do.\n\nThe soft spot, though, is structural. Every headline result—CIFAR-10 FID 1.64, ImageNet 64 FID 2.18, the SDXL/SD3 tables—comes from the full pipeline with the GAN loss. The only clean comparison of the adaptive schedule itself is Table 5, which is explicitly without adversarial losses, and there the gap over the best static schedule is 0.57 FID. There is no static-schedule-plus-GAN control anywhere. So we cannot tell whether the adaptive schedule, the GAN loss, or their interaction produces the headline numbers. The stress-test note is right: the paper sets up the exact control needed and never runs it. This is not a fatal flaw—the adaptive idea may well survive the test—but it is the load-bearing attribution claim, and it is currently unverified.\n\nOther issues are real but secondary: no error bars or seeds, so even the 0.57 FID gap in Table 5 is not established as significant; T_KDC is tuned on the reported FID (though the 60–70 band looks robust); the user study is one sentence with no protocol; and the large-scale setup omits details like latent-space PSNR computation, LoRA rank, and the adversarial weight. These are the kinds of things a referee would ask for, not reasons to dismiss the work.\n\nOverall, this is a conditional accept in my mind. The paper deserves a serious referee because the idea is novel, the experiments span small and large models, and the ablation design is on the right track. I would ask for the missing static+GAN control, error bars, and a fuller experimental appendix before trusting the headline. For someone working on consistency distillation or few-step generation, this is a useful read despite the unresolved attribution gap.","headline":"A plausible adaptive-schedule idea for consistency distillation, but the headline FID gains are not yet shown to come from the schedule rather than the GAN loss.","tokens_in":13742,"tokens_out":1718,"would_cite":true,"duration_ms":19528,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that consistency distillation underperforms because learning difficulty is uneven across noise levels, and that pacing the teacher's forward jump with a PSNR-based curriculum fixes it, yielding one-step generation that…","keywords":["consistency distillation","curriculum learning","diffusion models","flow matching","few-step sampling","PSNR knowledge discrepancy","text-to-image generation"],"falsifier":"Take a trained CCM, record the pairs $(t, u)$ chosen by the KDC loop, and compare the teacher's multi-step solution $x_u$ against a high-accuracy reference solver with hundreds of steps at the same $(t, u)$. If the error grows sharply as $u-t$ increases, the large-step targets that CCM relies on are not accurate, and the reported gains would come from something other than faithful teacher targets. A second check: corrupt the teacher at $u$ above a cutoff (e.g., freeze its weights or add noise) and see whether the KDC loop still selects large steps and whether FID holds; if it does, the KDC signal is not tracking target quality.","tokens_in":12719,"feed_emoji":"⚡","tokens_out":7787,"duration_ms":65053,"temperature":0.7,"pith_summary":"Consistency distillation trains a student to jump from a noisy input at timestep t to the teacher's output at a later timestep u, but this paper shows the difficulty of that jump is wildly uneven: near pure noise the student and teacher disagree a lot, while near the clean image they agree so much that the student learns little. The authors claim this imbalance, not just discretization error, is why consistency models underperform, and they fix it with a PSNR-based curriculum metric (KDC) that measures the student–teacher gap at each timestep. Whenever the gap is too small, the teacher iterates forward in small steps, extending u until the KDC clears a fixed threshold — \"see further when clear.\" If the claim holds, a single function evaluation can match or beat multi-step baselines: FID 1.64 on CIFAR-10 and 2.18 on ImageNet 64x64, with the same recipe transferring to SDXL and SD3 text-to-image models.","feed_headline":"Adaptive distillation schedule hits FID 1.64 in one step","feed_subtitle":"A PSNR-based difficulty metric paces the teacher's jump, beating multi-step baselines in one function evaluation.","key_machinery":"The load-bearing object is the Knowledge Discrepancy of the Curriculum (KDC), defined as $\\mathrm{KDC}_u^t = 100 - \\mathrm{PSNR}(x_{\\mathrm{est}}, x_{\\mathrm{target}})$, where $x_{\\mathrm{est}}$ is the student's one-step prediction and $x_{\\mathrm{target}}$ is the teacher's target after advancing from $t$ to $u$. CCM wraps KDC in a repeat-until loop (Algo. 1): advance the teacher by a small base step $s$, recompute the target, and keep advancing only while the discrepancy stays below the threshold $T_{\\mathrm{KDC}}$; when it clears the threshold, use that target in the consistency loss $\\mathcal{L}_{\\mathrm{CCM}}$. This adaptive-loop machinery converts the curriculum-difficulty problem into a scheduling problem, and the multi-step iteration is what keeps the teacher's target accurate even when the selected interval $u-t$ is large.","core_discovery":"On the paper's own terms, the central discovery is that the knowledge discrepancy of a consistency-distillation curriculum — quantified as $\\mathrm{KDC} = 100 - \\mathrm{PSNR}(x_{\\mathrm{est}}, x_{\\mathrm{target}})$ — falls monotonically as the timestep moves from noise to image, and that this falling difficulty is harmful. CCM therefore replaces the fixed or monotonically shrinking distillation step $l = u - t$ used by prior consistency models with an adaptive schedule: starting from $t$, the teacher advances in steps of size $s$, recomputing the target each time, until KDC exceeds a threshold $T_{\\mathrm{KDC}}$. The resulting step is small near pure noise and grows when the image is clear, so the student always trains on a curriculum of comparable difficulty. The paper reports that this balanced curriculum, combined with a GAN loss, yields one-step FID 1.64 on CIFAR-10 and 2.18 on ImageNet 64x64, and that on SDXL and SD3 it improves FID, CLIP score, and all six T2I-CompBench categories relative to LCM and PCM.","pith_inferences":["The same curriculum-balancing idea should apply to any self-distillation scheme where a student learns from a frozen or EMA teacher along an ODE — for instance, few-step video or audio generation — provided the discrepancy metric is chosen for that modality.","Because CCM's selected intervals grow exactly when the image is clear, the KDC signal could double as a per-sample difficulty weight for sampling $t$ or for annealing $T_{\\mathrm{KDC}}$, which the paper leaves open.","A testable extension would couple the KDC threshold to per-prompt statistics (e.g., guidance scale or prompt complexity), since text-to-image results show the largest gains on compositional prompts.","If the multi-step teacher iterations are the true source of gains, then a version that precomputes the adaptive $u$ schedule once per training stage and then trains with one-step targets should recover most of the benefit at lower compute."],"forward_implications":["One-step sampling on CIFAR-10 reaches FID 1.64, and on ImageNet 64x64 FID 2.18, competitive with teachers that use dozens or hundreds of function evaluations.","The adaptive schedule transfers to large text-to-image models: on SDXL and SD3, CCM reports lower FID and higher CLIP scores than LCM and PCM at 4 steps.","T2I-CompBench scores improve across all six compositional categories for both SD3 and SDXL, indicating better attribute binding, object relationship, and complex reasoning.","Training converges faster: CCM reaches the same FID as vanilla consistency distillation in about 1.3x fewer iterations.","The method is robust to its main hyperparameters: $T_{\\mathrm{KDC}}$ in the range 60–70 and step size $s$ around 0.03 both give stable gains over CD."],"supporting_citations":[{"why":"Defines consistency models and the consistency-distillation objective that CCM modifies.","marker":"Song et al. (2023)"},{"why":"Introduces latent consistency models, the text-to-image baseline CCM compares against.","marker":"Luo et al. (2023)"},{"why":"iCT, whose fixed power-law step reduction is the schedule CCM contrasts with its adaptive one.","marker":"Song & Dhariwal (2023)"},{"why":"ECM, which identifies the curse of consistency and motivates CCM's step-enlarging strategy.","marker":"Geng et al. (2024)"},{"why":"Provides the noise-to-signal ratio and discretization framework used to reason about KDC across timesteps.","marker":"Karras et al. (2022)"},{"why":"Forms the flow-matching branch of CCM's unified distillation loss.","marker":"Lipman et al. (2023)"},{"why":"PCM is the phased-consistency baseline and the code base CCM is implemented on.","marker":"Wang et al. (2024)"},{"why":"OT-CFM is the flow-matching teacher used for the CIFAR-10 and ImageNet experiments.","marker":"Tong et al. (2023)"},{"why":"SDXL is the large diffusion model where CCM's generalization is tested.","marker":"Podell et al. (2024)"},{"why":"SD3 is the large flow-matching model where CCM's generalization is tested.","marker":"Esser et al. (2024)"}],"fun_headline_variants":["Curriculum that adapts to clarity: one-step FID 1.64","PSNR-paced distillation reaches FID 1.64 in a single step","Adaptive teacher steps equalize difficulty, hit FID 1.64","Balancing distillation difficulty per timestep yields FID 1.64","Clearer images get bigger jumps: one-step FID 1.64"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme assumes the frozen teacher's forward ODE solves stay accurate over the sometimes-large intervals $u-t$ that the KDC loop selects; if the teacher drifts on those long jumps, the student is trained against a degraded target while the KDC threshold still reads \"difficult enough.\"","fun_headline_variants_meta":{"raw":{"variants":["Curriculum that adapts to clarity: one-step FID 1.64","PSNR-paced distillation reaches FID 1.64 in a single step","Adaptive teacher steps equalize difficulty, hit FID 1.64","Balancing distillation difficulty per timestep yields FID 1.64","Clearer images get bigger jumps: one-step FID 1.64"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000302,"raw_usage":{"total_tokens":1788,"prompt_tokens":1042,"completion_tokens":746,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":645}},"tokens_in":658,"tokens_out":746,"duration_ms":7317,"temperature":1.0,"reasoning_tokens":645,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:48:04.445998+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained CCM, record the pairs $(t, u)$ chosen by the KDC loop, and compare the teacher's multi-step solution $x_u$ against a high-accuracy reference solver with hundreds of steps at the same $(t, u)$. If the error grows sharply as $u-t$ increases, the large-step targets that CCM relies on are not accurate, and the reported gains would come from something other than faithful teacher targets. A second check: corrupt the teacher at $u$ above a cutoff (e.g., freeze its weights or add noise) and see whether the KDC loop still selects large steps and whether FID holds; if it does, the KDC signal is not tracking target quality.","supporting_citations":[],"review_version":1}