{"id":"71a0a4b5-4b19-4d27-ad65-216be40f3298","arxiv_id":"2511.19065","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Training MeanFlow by first forming instantaneous velocity and short-gap average velocity, then shifting to long gaps, improves 1-NFE ImageNet FID from 3.43 to 2.87 and speeds training by about 2.5x.","lead":"This paper studies how the two velocity fields in MeanFlow, a few-step generative model, interact during training, and proposes a curriculum that builds instantaneous and small-gap velocities before large-gap ones. The scheme improves one-step ImageNet FID from 3.43 to 2.87 with the same DiT-XL backbone, and reaches baseline quality about 2.5x faster.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Observational basis for 'large-Δt u-loss degrades v' is confounded by removing v-loss during fine-tuning; the curriculum's core rationale may be an artifact.","rationale":"The paper's practical recipe — accelerate v formation and progressively upweight small-Δt u-loss — is empirically supported by ablations on DiT-B, DiT-M, DiT-L, DiT-XL and on FFHQ, and the code is promised. The central risk is not whether the recipe works but whether the stated mechanism is correct. The reader identified TAS and the corruption experiment as weak points; my concern is more specific to Fig. 4, where removing L_v entirely during u-finetuning makes the 'large-Δt degrades v' finding potentially an artifact of unconstrained v drift. This matters because that finding is the main justification for the early suppression of large gaps. If the concern lands, the paper's contribution is still a useful empirical curriculum, but the 'understanding' claim is weakened. A single controlled experiment with a mixed loss can settle it. The reader's conditional verdict remains appropriate: not rejected, but not fully established until the interaction is tested under the joint objective actually used.","tokens_in":18799,"tokens_out":9929,"duration_ms":103773,"concrete_test":"Re-run the Sec. 4.2 experiment (Fig. 4) under the standard MeanFlow joint objective: keep a fixed v-loss fraction (e.g., FM ratio = 0.2 or 0.5, i.e., a nonzero probability of t=r) in every Δt-range condition, matching the actual training setup, and measure 32-NFE FID of v-prediction. If large-Δt conditions no longer degrade FID below the v-pretrained baseline, then O2 is an artifact of removing L_v and the curriculum's protective rationale is unsubstantiated; if degradation persists, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The key evidence for O2 (large-gap average-velocity supervision destabilizes the instantaneous velocity) is Fig. 4, whose protocol in Appendix A sets the sampling ratio of t=r to zero in all Δt-range conditions. Thus the 'large Δt' condition trains only with L_u and no L_v. Because large-Δ L_u does not supervise local velocity, the observed drop in 32-NFE FID may be ordinary drift or catastrophic forgetting of v under a loss that does not constrain it, not an active harmful interaction between the two objectives. The paper's actual curriculum, however, always retains v-loss (or small-gap u-loss, which acts as a proxy for v) alongside large-gap u-loss; hence the harmful regime identified by O2 may never occur in the proposed method. The corruption experiment (Fig. 3) has a related confound: noise injected into L_v enters the shared backbone gradients, so degraded u-learning could reflect shared-parameter gradient noise rather than a prerequisite relation between v quality and u learning. The central 'self-destructive dynamic' narrative, and the justification for suppressing large Δt early, therefore rests on an experimental separation that does not match the joint objective in Eq. 5.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes the interaction between the instantaneous velocity v and the average velocity u in MeanFlow training. Through controlled experiments it claims: (O1) v must be well-formed before u can be learned; (O2) large temporal gaps in u-supervision destabilize v, while small gaps help; (O3) gradient-affinity analysis favors pretraining with small-gap u over pure v-loss before introducing large-gap u. Based on these observations, the authors propose an accelerated MeanFlow training scheme with a progressive weighting β(Δt,s) that emphasizes small gaps early and transitions to uniform weighting, combined with standard diffusion acceleration techniques (MinSNR or DTD). The method reports improved 1-NFE FID on ImageNet 256×256 (2.87 vs. 3.43 with DiT-XL/2), faster convergence, and consistent gains across model sizes and CFG configurations. The paper includes ablations, convergence curves, and an FFHQ appendix.","tokens_in":19097,"tokens_out":4626,"duration_ms":54004,"significance":"If the stated causal story is correct, the paper offers both a practical training recipe and a conceptual framework for MeanFlow-style few-step generative models. The empirical gains are substantial and the ablations (Table 3, Fig. 6, FFHQ appendix) support the practical value of the proposed schedule. The paper also ships code and reproduces trends across architectures and datasets, which strengthens the engineering claim. However, the conceptual claims about learning dynamics — especially O2 and the TAS-based ordering in O3 — rely on experimental designs that do not cleanly separate the hypothesized causal mechanisms from ordinary optimization effects. Since these observations are the stated motivation for the method, the conceptual contribution needs stronger controls before the findings can be accepted as established.","major_comments":[{"comment":"The Δt-range experiment 'Impact of u-learning on v-learning' sets the sampling ratio of t=r to zero in all conditions. Thus the large-Δt condition trains the model with only L_u and no v-loss. The observed degradation in 32-NFE FID for large Δt could result simply from the absence of any instantaneous-velocity supervision (catastrophic forgetting or drift), rather than from an active harmful interaction in which large-gap u-loss interferes with v. The actual training objective (Eqs. 5, 8, 9) always retains a v-loss component (or a small-gap u-loss that acts as a v proxy). Therefore O2 as stated — 'large-gap supervision destabilizes v' — is not established by this protocol. Please add a control that keeps the joint objective intact, for example by fixing the t=r sampling ratio constant while varying only the Δt range, and report 32-NFE FID and v-prediction error in that setting.","section":"§4.2, Fig. 4, Appendix A"},{"comment":"The corruption experiment injects Gaussian noise scaled by k·||v_t(zt|ε)|| into the target of the v-loss while training with L_MF. Because the network is shared, the injected noise corrupts the gradients for all parameters, including those that produce u. The observed degradation in u-learning may therefore be explained by noisy gradients in the shared backbone rather than by a specific causal prerequisite that well-formed v is necessary for learning u. The two-stage pretraining results (Fig. 2) support the benefit of v-pretraining, but they do not isolate the corruption mechanism. Please add a matched control in which the same amount of gradient noise is injected into the u-loss target instead of the v-loss target, or otherwise show that the effect is specific to corrupting v, e.g., by measuring v-prediction accuracy directly.","section":"§4.1, Fig. 3"},{"comment":"The Task Affinity Score (TAS) is proposed as the basis for O3, i.e., that small-Δt pretraining better prepares the model for large-Δt learning than pure v-pretraining. However, TAS is only a cosine similarity computed on 5K sampled points per epoch; the paper does not demonstrate that TAS predicts the final large-Δt performance. No final 1-NFE FID is reported for the models trained with Strategy 1 vs. Strategy 2 before the large-Δt phase. Without such a validation, the ordering prescription rests on an unvalidated proxy. The empirical ablation in Table 3 shows that progressive L_u weighting helps, but it does not isolate the 'small-Δt first versus pure v-first' comparison. Please report the actual large-Δt learning outcome for both initializations, or show a correlation between TAS and final FID across a range of schedules.","section":"§4.3, Fig. 5, Appendix A"}],"minor_comments":[{"comment":"FID values are reported as single-run point estimates. Given the small differences in some comparisons (e.g., Table 3: DTD 10.96 vs. L_u weighting 10.98; Table 4: k=1 10.20 vs. k=2 11.44), the absence of error bars or multiple seeds makes it difficult to judge significance. At minimum, report the variance over 2–3 seeds for the main DiT-B results.","section":"Tables 1–6"},{"comment":"The y-axis label 'FID (NFE=1, ε-pred)' is clear, but the curves for different k values are not annotated beyond the legend. It would help to mark the final FID values directly on the curves, as done in the FFHQ appendix, to ease reading.","section":"Fig. 3"},{"comment":"The notation for the adaptive loss weights (L_adp) is introduced only in Appendix B. Since the main-text equations (8) and (9) use these terms, a one-sentence pointer to the adaptive weighting mechanism (Eq. 7) in the main text would improve readability.","section":"§5, Eq. (8)"},{"comment":"The effect of the schedule parameter k is described as 'linear is best,' but the pattern is non-monotonic (k=1 best, k=0.5 11.16, k=2 11.44, k=3 11.99). A brief discussion of why faster or slower transitions hurt — in terms of the observations in Sec. 4 — would strengthen the analysis.","section":"§6.3, Table 4"}],"recommendation":"major_revision","confidential_remarks":"The practical contribution is solid and likely useful to the community, but the conceptual narrative needs to be repaired before publication. The Fig. 4 protocol is a genuine confound: removing the v-loss entirely in the large-Δt condition cannot support the 'destabilization' claim that motivates suppressing large gaps. The corruption experiment and the TAS proxy raise similar concerns. I would be willing to accept after the authors provide the requested control experiments, or alternatively reframe the paper's contribution as an empirical training schedule without the strong causal claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — quick read.\n\nBottom line: the progressive gap-weighting schedule is a solid empirical contribution. The method works, the ablations are honest, and the paper is worth a serious referee. The mechanistic story around 'large-gap u destroys v' is overstated and rests on a confounded protocol.\n\nWhat's new: the paper gives a clean recipe for MeanFlow training: accelerate v-learning (DTD or MinSNR) and shift the u-loss from small to large time gaps via a linear schedule. It shows v-learning helps u-learning (Fig 2), small-gap u-loss acts as a useful proxy for v (Fig 4), and the improvements hold on ImageNet and FFHQ across DiT sizes. Ablations are honest — each component helps, k=1 is best. That's a practical contribution.\n\nSoft spots: the claim that large-Δt supervision actively degrades v (O2) is not cleanly isolated. In Fig 4, the protocol (Appendix A) sets the sampling ratio of t=r to zero, so the large-Δt condition trains only on L_u with no L_v. The FID drop could be simple drift from removing v-supervision, not an active harmful interaction. Your actual curriculum always retains v or small-gap u alongside large-gap u, so the experiment doesn't reproduce the regime you claim to avoid. Same confound in Fig 3: the corruption experiment injects noise into shared backbone gradients, so degraded u-learning may be gradient noise rather than corrupted v. TAS is a reasonable heuristic but 5K samples per epoch is thin.\n\nAlso, the small-gap-as-proxy insight is partly definitional since u(z,t,t)=v, but that doesn't undercut the schedule. The headline FIDs are single-run point estimates; I'd like error bars, though the consistent direction across model sizes mitigates that.\n\nWho this is for: anyone training MeanFlow or similar two-timescale flow objectives. It's a low-cost improvement that's easy to try. I'd accept it with a request for a cleaner isolation experiment — compare v-loss + large-gap u-loss vs v-loss + small-gap u-loss with matched v-supervision — and error bars on the main numbers.","headline":"Useful training schedule with a plausible but under-isolated mechanism; the small-gap-first weighting works, the 'large-gap corrupts v' claim is weaker than presented.","tokens_in":19559,"tokens_out":4916,"would_cite":true,"duration_ms":50472,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MeanFlow's one-step generation improves when training first establishes instantaneous velocity and small time-gap average velocities, then shifts to large gaps.","keywords":["MeanFlow","instantaneous velocity","average velocity","training curriculum","temporal gap","flow matching","one-step generation","task affinity"],"falsifier":"Train the same DiT backbone with the proposed curriculum but reverse the schedule—supervising large-gap u from the start and adding small gaps later—with the same total compute; if the reversed schedule matches or beats the proposed order on 1-NFE FID, the central 'small-gap-first' claim is wrong. A second check: in the corruption experiment of Figure 3, measure whether the injected noise also changes the u_tgt target in Eq. 3; if u_tgt degrades directly, the observed u-learning failure could stem from a corrupted loss rather than from v being a prerequisite.","tokens_in":18714,"feed_emoji":"🎨","tokens_out":6282,"duration_ms":57211,"temperature":0.7,"pith_summary":"The paper argues that the two velocity fields learned by MeanFlow—instantaneous velocity at a single time and average velocity over a time interval—cannot be trained effectively with a fixed, uniform objective. Controlled experiments show that instantaneous velocity must be well-formed before average velocity can be learned, that large temporal gaps in average-velocity supervision destabilize instantaneous velocity, and that small-gap supervision prepares the model for later large-gap learning. The authors translate this into a curriculum: accelerate instantaneous-velocity formation with existing timestep-sampling and loss-weighting tricks, and progressively shift the average-velocity loss from short to long intervals. The result is faster convergence and better few-step generation at the same model size.","feed_headline":"Reordering training cuts MeanFlow one-step FID to 2.87","feed_subtitle":"Mastering instantaneous velocity and small time gaps first yields faster training and better one-step images.","key_machinery":"The MeanFlow identity, u(z_t, r, t) = (1/(t-r)) ∫_r^t v(z_τ, τ) dτ, which defines average velocity as the temporal integral of instantaneous velocity and underlies the training loss that supervises u against a stop-grad target computed from v; the paper's progressive weighting β(Δt, s) = 1 − s + λs(1 − Δt) that down-weights large temporal gaps early and relaxes to uniform weighting as training progresses, applied after MeanFlow's adaptive loss normalization. The identity makes the dependency of u on v explicit, and the schedule operationalizes the ordering principle.","core_discovery":"MeanFlow's average velocity u—the key to one-step generation—is learned reliably only after the instantaneous velocity v is well formed, and the temporal gap used in u-supervision decides whether u-learning helps or hurts v. Experiments show v-pretraining accelerates u-learning while corrupting v destroys it; small-gap u-supervision builds and refines v, whereas large-gap u-supervision degrades it. Small-gap u-pretraining also yields a better launch point for large-gap u than pure v-pretraining, per task-affinity analysis. The derived curriculum—acceleration for v plus a linear shift of u-weights from small to large gaps—attains 1-NFE FID 2.87 vs 3.43 on ImageNet 256×256 with the same DiT-XL","pith_inferences":["The ordering principle may transfer to other two-time generative models (e.g., consistency models, flow maps, shortcut models) that learn interval-integrated quantities; if so, curriculum schedules based on gap size could improve them as well.","The TAS proxy could be replaced by a direct measurement of optimization dynamics—e.g., how quickly large-gap u-loss decreases as a function of v quality—to make the schedule adaptive rather than fixed to iteration count.","The observed two-way interaction suggests that small-gap u-supervision acts as a regularizer for v; a testable extension is to see whether adding small-gap u-loss alongside v-loss helps even in settings where u is never needed for inference.","If the benefit is caused by v-formation speed rather than the specific schedule shape, then adaptive curricula that pause large-gap supervision until v reaches a threshold FID should match or beat the linear schedule; this is a cheap experiment to run."],"forward_implications":["With the same DiT-XL backbone, the proposed training reaches 1-NFE FID 2.87 on ImageNet 256×256, down from 3.43 for standard MeanFlow, and 2-NFE FID 2.64 vs 2.93.","The method matches the quality of the conventional MeanFlow baseline in about 2.5× fewer training epochs, or with a smaller DiT-L backbone.","The curriculum improves the quality of the learned instantaneous velocity itself, as shown by consistently better multi-step FID (32/64/128 NFE) when u_θ(z_t, t, t) is used as the velocity estimate.","Both components—velocity acceleration (MinSNR or DTD) and progressive L_u weighting—contribute independently, with the combination giving the best results across model sizes.","The improvements are robust to classifier-free-guidance configurations, holding for both small- and large-model CFG settings."],"fun_headline_variants":["Train v first: MeanFlow one-step FID drops to 2.87","Order matters: v-pretraining yields MeanFlow FID 2.87","Small-gap v first: MeanFlow one-step FID improves to 2.87","MeanFlow: train v before u for one-step FID 2.87"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The curriculum's justification rests on the Task Affinity Score—cosine similarity between v-loss and u-loss gradients over 5K samples per epoch—being a reliable predictor of how smoothly large-gap u will later be learned, and on the corruption experiment's assumption that injected noise affects only the v-loss target and not the u_tgt target defined via the same velocity field.","fun_headline_variants_meta":{"raw":{"variants":["Train v first: MeanFlow one-step FID drops to 2.87","Order matters: v-pretraining yields MeanFlow FID 2.87","Small-gap v first: MeanFlow one-step FID improves to 2.87","MeanFlow: train v before u for one-step FID 2.87"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000462,"raw_usage":{"total_tokens":2161,"prompt_tokens":771,"completion_tokens":1390,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":1313}},"tokens_in":515,"tokens_out":1390,"duration_ms":9956,"temperature":1.0,"reasoning_tokens":1313,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:33:35.012019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same DiT backbone with the proposed curriculum but reverse the schedule—supervising large-gap u from the start and adding small gaps later—with the same total compute; if the reversed schedule matches or beats the proposed order on 1-NFE FID, the central 'small-gap-first' claim is wrong. A second check: in the corruption experiment of Figure 3, measure whether the injected noise also changes the u_tgt target in Eq. 3; if u_tgt degrades directly, the observed u-learning failure could stem from a corrupted loss rather than from v being a prerequisite.","supporting_citations":[],"review_version":1}