{"id":"862c9489-4cb3-4f73-811b-8bdd07eac895","arxiv_id":"2412.05780","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A prompt-adaptive model selects the minimum denoising step count that reaches perceptual quality saturation, saving about 5 seconds per image with no detectable quality loss.","lead":"BudgetFusion predicts how much perceptual quality improves with each extra denoising step for a given text prompt, then stops generation at a plateau where more steps add no visible value. It cuts Stable Diffusion inference time by about 60 percent (8.0s to 2.9s per image) while users cannot reliably tell the images apart from full-quality ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 7's step-selection rule is internally inconsistent: under the paper's own monotonicity assumption, the printed `max_t` condition selects the final timestep, so the reported savings cannot come from the stated formula.","rationale":"The paper has genuinely useful components: a large-scale perceptual dataset, an LSTM-based metric predictor with a reasonable ablation, and a user study that at least attempts to validate the perceptual claim. However, the central mechanism—plateau detection—is stated in Eq. 7 in a way that cannot produce the reported step counts under the paper's own monotonicity assumption. The printed condition is an upper-tail inclusion; taking the maximum over t returns the final timestep for any metric where higher is better, and it has an inconsistent meaning for L-SNR and D-SIM, which are lower-is-better. Since t* drives all downstream results (SPI, Table 2, user-study stimuli), the headline 'saves up to five seconds' cannot be attributed to the described algorithm without a corrected formula or released implementation. The reader's concern about the ω weights being tuned on the evaluation metric is real, but it is secondary: the weights are only meaningful once the threshold semantics are fixed. There is also an unresolved numeric inconsistency in Section 4.4, where 35.2% selection of OURS is described as 'lower than 25%.' Given these issues, the CONDITIONAL verdict is appropriate: the authors should correct Eq. 7 (or release the code), re-run the evaluation, and clarify the user-study JND threshold before the savings claim is accepted at face value.","tokens_in":14256,"tokens_out":14788,"duration_ms":160935,"concrete_test":"Ask the authors to release the exact step-selection implementation or re-run Eq. 7 literally on the 1,839 held-out prompts using their trained LSTM predictors. If the literal implementation returns 129 steps for every prompt, the printed formula is definitively wrong; then require a corrected definition (e.g., `min_t` or direction-dependent thresholds) and recompute Table 2, Figure 8, and the SPI comparison. If the literal implementation reproduces the reported t* distribution, the notation must be clarified and the remaining validation issue is the tuning of ω to the evaluation metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.4 defines the per-metric plateau point as t*_m(p) := max_t s.t. m_t(p) ≥ μ({m_t}) + ω_m σ({m_t}) and then sets t*(p) := max_m t*_m(p). The paper also states in the same section that the perceptual metrics are monotone in t. For a metric that increases with t (e.g., I-CLIP, or any correctly oriented score), the set {t : m_t ≥ μ + ωσ} is an upper tail whose maximum is the largest sampled timestep, 129, not the plateau onset. For metrics that decrease with t (raw L-SNR and D-SIM), the same inequality is satisfied on the left tail, so the `max_t` convention has opposite meaning across the three metrics. The reported step suggestions (e.g., t* = 27/33/42 in Figures 5–6, and OURS at 2.89 SPI rather than the 8.00 SPI of a 65-step reference) cannot be produced by the printed equation. Either the inequality should be `≤` for decreasing metrics and `min_t` for increasing metrics, or the plateau rule actually implemented is different from Eq. 7. Because Eq. 7 is the entire step-selection mechanism, this is not a cosmetic typo: the method as written is not reproducible, and the central efficiency claim cannot be cleanly attributed to the stated algorithm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BudgetFusion, a method that predicts, from a text prompt, a minimal number of denoising steps for a text-to-image diffusion model such that additional steps yield only marginal perceptual improvement. The system trains a BiLSTM to map CLIP-encoded text and timestep embeddings to three perceptual metrics (L-SNR, D-SIM, I-CLIP) as functions of the number of inference steps, and then selects a 'plateau' timestep via a threshold rule based on the median and standard deviation of each predicted metric curve. Experiments on Stable Diffusion 2 with COCO captions report improved quality-per-TFLOP efficiency relative to fixed-step baselines and reduced seconds-per-image, supported by a crowdsourced user study comparing BudgetFusion against uniform and reference schedules.","tokens_in":14463,"tokens_out":9404,"duration_ms":84147,"significance":"The paper identifies a promising direction: making diffusion inference compute-adaptive to prompt difficulty based on human perception, rather than using a fixed step count. The construction of a large dataset (882k images), the LSTM-based metric predictor, and the plateau-stopping idea are concrete contributions that could make diffusion deployment more efficient. If the algorithm is correctly specified and the evaluation is made non-circular, the method could be useful in practice. However, the current manuscript has critical issues in the algorithm description and in the interpretation of the user-study data that must be resolved before the central claims are supported.","major_comments":[{"comment":"Under the paper's own monotonicity assumption ('perceptual metrics typically exhibit a monotonic relationship with regard to number of denoising timesteps'), the plateau rule as printed is internally inconsistent. For an increasing metric such as I-CLIP (denoted ↑ in Section 3.2), the set {t : m_t(p) ≥ μ + ωσ} is an upper tail, so taking the maximum over t yields the largest sampled timestep, t_N = 129, for every prompt. This contradicts the reported suggestions t* = 27, 33, 42 in Figures 5–6 and the OURS time savings in Section 4.2. For decreasing metrics, the same inequality defines a lower tail, so the rule has opposite meanings across metrics. The authors must correct the stopping rule (e.g., use min_t for increasing metrics or invert the inequality) and demonstrate that the reported timesteps are actually produced by the stated formula.","section":"Section 3.4, Eq. (7)"},{"comment":"The plateau weights ω_m (ω_L-SNR=0.3, ω_D-SIM=0.2, ω_I-CLIP=0.5) are 'determined using the efficiency measurement as detailed in Section 4.2.' This means they are tuned to maximize the same quality-per-TFLOP metric that Section 4.2 uses as the headline evaluation. The evaluation is therefore circular: the reported efficiency advantage of OURS over UNIFORM and REFERENCE may be largely a consequence of fitting the stopping rule to the benchmark. The authors should specify a separate validation split for weight selection, report the sensitivity of the efficiency results to the ω values, or provide an independent criterion for choosing them.","section":"Section 3.4 and Section 4.2"},{"comment":"The sentence 'participants selected OURS 35.2 ± 8.4% of the time, lower than 25% selection which is commonly used to define the 1 Just-Noticeable-Difference (1 JND) threshold' contains a numerical error: 35.2% is higher than 25%, not lower. If the intended threshold is that 25% selection of OURS corresponds to 75% correct detection of the reference (the standard JND level), then the observed 35.2% actually supports the claim of perceptual similarity, but the text must be corrected and a significance test against the threshold should be reported. As written, the statement is self-contradictory and undermines the 'without compromising perceptual similarity' conclusion.","section":"Section 4.4, user study"},{"comment":"The relative quality gain of OURS over UNIFORM is computed on the same evaluation set used to determine the plateau weights and the average UNIFORM step count. Because the weights are fitted to maximize efficiency on this set, the reported 6.6–8.7% relative gains are likely optimistic. A cross-validation scheme for weight selection and UNIFORM step averaging would make the comparison trustworthy. The authors should also report the variance of these gains across held-out prompt splits.","section":"Section 4.2, Eq. (9)"}],"minor_comments":[{"comment":"The header 'OURSw/o t-encodingt' appears to be a typo; it likely should read 'OURS w/o text-encoding' or 'OURS w/o t-encoding' with a space and correct spelling.","section":"Table 1"},{"comment":"The phrase 'The results are visualized in Section 4.4' should refer to the actual figure number (Figure 9) rather than the section number.","section":"Section 4.4"},{"comment":"The phrase 'with respect toREFERENCE' is missing a space; it should read 'with respect to REFERENCE'.","section":"Section 4.4"},{"comment":"The text states that μ and σ denote the 'median and standard deviation', but Eq. (7) writes them as μ and σ without clarifying that μ is the median; this should be stated explicitly to avoid confusion with the mean.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper has a compelling idea and substantial experimental effort, but the main algorithmic equation is not reproducible as written, and the evaluation is circular in its current form. The user-study numerical error is sloppy and should be corrected. These issues are fixable in a major revision, but the authors need to rigorously clarify the plateau rule, decouple the weight selection from the evaluation metric, and present a corrected statistical interpretation of the user study. I would not recommend acceptance without these changes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about BudgetFusion. First, the core idea is genuinely useful: predict how perceptual quality grows with denoising steps from the text prompt, and stop at the plateau. That is a real contribution on top of AutoDiffusion-style per-sample step selection, and the authors back it with an 882k-image dataset, a user study, and an LSTM that clearly beats the MLP baseline. If it works, a 60% inference cut at constant perceived quality is practically significant.\n\nSecond, the paper as written cannot reproduce its own headline numbers. Equation 7 defines the plateau point as the maximum t such that the metric exceeds the median plus ω times the standard deviation. The paper also says the metrics are monotone in t. For an increasing metric like I-CLIP, that set is an upper tail, so the maximum t is the largest sampled timestep (129), not a plateau. For decreasing metrics like D-SIM, the same inequality holds on the left tail, so max_t has the opposite meaning. The reported suggestions (t*=27,33,42) and the 2.89 SPI result cannot come from that equation. Either the inequality sign or the max/min convention is wrong. This is not a typo: Eq 7 is the entire step-selection mechanism, and the method as written is unreproducible.\n\nThere are smaller issues. The ω_m weights in Eq 7 are chosen to maximize the same efficiency metric used for evaluation, which inflates the savings claim. The user study sentence says 35.2% selection is 'lower than 25%,' which is numerically false and undermines the 1-JND claim. Inception scores have no error bars. AutoDiffusion, the closest adaptive baseline, is cited but not compared. No code or data are released.\n\nMy take: the paper has a real idea and a plausible evaluation, but the load-bearing formula is wrong as written. The authors need to fix Eq 7, show the corrected rule produces the reported t* values, and address the evaluation overfitting. I would send it to peer review, not desk-reject, but only with the expectation of heavy revision. The right audience is the efficient-diffusion and perception-aware rendering community. I would not cite it in its current form.","headline":"Worth reading for the prompt-adaptive plateau idea, but Eq 7 as printed cannot produce the reported step counts; the method needs a corrected formula and fuller evaluation before citing.","tokens_in":15087,"tokens_out":4931,"would_cite":false,"duration_ms":44889,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompt-aware step selection can cut diffusion image generation cost by up to five seconds per image.","keywords":["diffusion models","text-to-image","adaptive inference","perceptual metrics","inference efficiency","LSTM","plateau detection","Stable Diffusion"],"falsifier":"Run BudgetFusion on prompts deliberately outside COCO's distribution (for instance, abstract art, dense text, or unusual object compositions) and have users compare its output against a 65-step reference in a two-alternative forced-choice test; if the selection rate exceeds the one-JND bound, the plateau prediction does not transfer.","tokens_in":13953,"feed_emoji":"⚡","tokens_out":7387,"duration_ms":62895,"temperature":0.7,"pith_summary":"The paper argues that the number of denoising steps a text-to-image diffusion model needs depends on the content of the prompt, so a fixed step count wastes computation on simple prompts and under-generates on complex ones. BudgetFusion learns to predict, from the text alone, how three perceptual quality metrics (pixel-level sharpness, mid-level layout similarity, and semantic alignment) will grow with each additional denoising step. It then picks the step where all three metrics plateau and stops the diffusion process there. The authors report that on Stable Diffusion 2 this saves up to five seconds per image, cutting average inference time by roughly 64%, without a perceptible quality drop in a forced-choice user study.","feed_headline":"Prompt-smart step count cuts image generation by up to 5s","feed_subtitle":"BudgetFusion stops denoising at each prompt's perceptual plateau, matching full generation within one JND.","key_machinery":"The load-bearing object is the per-prompt metric time series predictor: a two-layer BiLSTM (hidden size 512) followed by an MLP and sigmoid, trained with L2 loss on 882,432 Stable Diffusion images generated from 18,384 CLIP-diverse COCO prompts at twelve sampled timesteps. The predictor outputs three curves, one per perceptual scale; the plateau rule (Equation 7) takes the maximum of the three per-metric plateau points, defined by the median plus a metric-specific weight times the standard deviation of the predicted curve. The weights ($\\omega_{L\\text{-}SNR}=0.3$, $\\omega_{D\\text{-}SIM}=0.2$, $\\omega_{I\\text{-}CLIP}=0.5$) are tuned to maximise the same quality-per-TFLOP efficiency metric used in evaluation.","core_discovery":"The central claim is that perceptual quality gain per denoising step is not uniform across prompts: each prompt has its own saturation point beyond which extra denoising steps produce only marginal, imperceptible improvement. BudgetFusion operationalises this by training a bidirectional LSTM that takes a CLIP-embedded prompt and a position-encoded timestep and predicts three metric time series — L-SNR for pixel-level sharpness, DreamSim distance for mid-level layout, and CLIP cosine similarity for semantic alignment. The suggested timestep is the maximum over the three metrics of the first time each metric crosses a median-plus-weighted-standard-deviation threshold. With this rule the model selects timesteps that are on average far lower than the common 65-step reference, matching the reference within one just-noticeable-difference in a crowd-sourced user study while using roughly one third of the compute.","pith_inferences":["The plateau weights are fitted to the same efficiency benchmark used for evaluation, so the reported gains are upper bounds unless those weights generalise to unseen prompt distributions; a sensitivity analysis across prompt categories would bound this.","BudgetFusion's predicted curves could be used to allocate a total latency budget across multiple prompts in a batch rather than stopping each independently, which better matches real serving constraints.","If the perceptual curves are reliable, they could also guide scheduler choice or early-exit criteria in distilled few-step models, where a fixed step count is still the norm."],"forward_implications":["Deployed text-to-image services can run the 0.004-second predictor per prompt and skip roughly two-thirds of denoising steps without users noticing a difference.","The same three-metric plateau rule can be applied to other latent diffusion models, with retraining on that model's generated images.","The method turns inference cost from a fixed hyperparameter into a prompt-dependent decision, so compute is spent where it produces perceivable quality gain.","The quality-per-TFLOP efficiency metric introduced here gives a common yardstick for comparing diffusion acceleration methods."],"supporting_citations":[{"why":"Supplies the base text-to-image model (Stable Diffusion 2) whose denoising process is being optimized.","marker":"[47]"},{"why":"Provides the DreamSim mid-level perceptual metric used as one of the three predicted quality signals.","marker":"[9]"},{"why":"Supplies both the CLIP embedding used to condition the predictor and the image-text alignment metric I-CLIP.","marker":"[45]"},{"why":"Provides the COCO caption corpus from which the 18,384 training prompts are sampled.","marker":"[26]"},{"why":"Defines the Euler scheduler whose fixed start and end timesteps make the quality-versus-step measurements consistent.","marker":"[19]"},{"why":"Introduces the SNR-style pixel-sharpness measure that L-SNR adapts as the pixel-level metric.","marker":"[21]"},{"why":"Supplies the statistical plateau and turning-point detection rule that Equation (7) generalises.","marker":"[8]"}],"fun_headline_variants":["Per-prompt step budgets cut diffusion cost up to 5s","Adaptive denoising stops when humans stop noticing gains","Save 5 seconds per image by matching perceptual limits","BudgetFusion finds each prompt's quality saturation point"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The saving rests on assuming that perceptual quality genuinely saturates at a prompt-dependent step count, and that the plateau weights tuned on COCO with Stable Diffusion 2 transfer to other prompts and base models.","fun_headline_variants_meta":{"raw":{"variants":["Per-prompt step budgets cut diffusion cost up to 5s","Adaptive denoising stops when humans stop noticing gains","Save 5 seconds per image by matching perceptual limits","BudgetFusion finds each prompt's quality saturation point"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000985,"raw_usage":{"total_tokens":4171,"prompt_tokens":927,"completion_tokens":3244,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":3177}},"tokens_in":543,"tokens_out":3244,"duration_ms":24180,"temperature":1.0,"reasoning_tokens":3177,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:21:37.809639+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run BudgetFusion on prompts deliberately outside COCO's distribution (for instance, abstract art, dense text, or unusual object compositions) and have users compare its output against a 65-step reference in a two-alternative forced-choice test; if the selection rate exceeds the one-JND bound, the plateau prediction does not transfer.","supporting_citations":[{"cited_title":"Dream- sim: Learning new dimensions of human visual similarity using synthetic data","cited_arxiv_id":null,"evidence_quote":"Provides the DreamSim mid-level perceptual metric used as one of the three predicted quality signals."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"Provides the COCO caption corpus from which the 18,384 training prompts are sampled."},{"cited_title":"Image reconstruction in snr units: a general method for snr measurement","cited_arxiv_id":null,"evidence_quote":"Introduces the SNR-style pixel-sharpness measure that L-SNR adapts as the pixel-level metric."},{"cited_title":"Microsaccades are triggered by low retinal image slip","cited_arxiv_id":null,"evidence_quote":"Supplies the statistical plateau and turning-point detection rule that Equation (7) generalises."}],"review_version":1}