{"id":"a8d2d36c-8fe7-4eb5-a49c-f5a6c1abbc94","arxiv_id":"2607.15655","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"An adaptive lookahead decoder that gates rollout depth by candidate-score variance and re-triggers exploration from informative branches improves accuracy per decoding step over one-step lookahead on masked diffusion language models.","lead":"AdaLook is a decoding method for masked diffusion language models that adaptively decides how many lookahead steps to take before committing tokens, and can re-launch exploration from promising intermediate states. On math and reasoning benchmarks it reports better accuracy per decoding step than existing one-step lookahead decoding.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Variance-gated rollout depth is the method's core, but τ is never specified (and its role is stated oppositely in §1 vs §4.2); without τ or fixed-depth controls, the reported gains cannot be attributed to adaptivity.","rationale":"The paper's central and only novel mechanism is the variance-gated adaptive rollout continuation. The strongest claim depends on this gate being both correctly specified and responsible for the observed accuracy-efficiency improvement. As written, the gate is specified in contradictory ways (Section 1 vs Section 4.2/Algorithm 1), the critical threshold τ is never given a value, and no ablation or fixed-depth baseline isolates the contribution of adaptivity. Therefore the empirical ordering AdaLook > ETE, while plausible and supported by the reported curves, cannot yet be attributed to the paper's proposed mechanism. This is exactly the weakest assumption identified by the reader, and I agree with the CONDITIONAL verdict: the concern is load-bearing but not falsifying, because the authors may be able to provide τ and sensitivity/ablation results. The Dream-7B appendix is useful independent support for the ordering, but it does not resolve the τ omission or the missing fixed-depth control. The requested concrete test would settle whether the variance gate is doing the claimed work or whether the improvement is simply the result of always or never rolling out deeper.","tokens_in":11303,"tokens_out":7656,"duration_ms":75474,"concrete_test":"Ask authors to (1) disclose τ and the calibration set used to set it; (2) rerun Algorithm 1 on MATH500 with τ=0 and τ=∞ while keeping k=4, Tmax=2, and all other hyperparameters fixed, and plot the accuracy-vs-decoding-steps curves alongside the reported AdaLook curve. If τ=0 matches ETE and τ=∞ matches or beats the reported AdaLook frontier, the variance gate is not the source of the improvement and the 'adaptive' claim is unsupported. If the reported curve sits strictly between τ=0 and τ=∞ and depends smoothly on τ, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's novelty is the variance gate: Algorithm 1 (Step 3) continues rollout only while Var({s_j}) < τ; once variance exceeds τ (or depth hits Tmax) it commits. But Section 1 states the opposite rule ('Only when the variance exceeds a predefined threshold does the decoder continue expanding future decoding trajectories'), an internal contradiction. The τ value is never reported: Section 4.3's fixed-hyperparameter list gives k, α, β, Tmax, cinfo but omits τ, and the calibration subsection never mentions it. Since τ is the only quantity that controls rollout depth, the core 'adaptive' mechanism cannot be reproduced or checked. This is load-bearing for the central claim: without τ, the reported gain over one-step ETE could come from a fixed 2-step rollout with branch expansion, exactly the naive extension the authors say is ineffective (Section 1, L1). No fixed-depth rollout baseline appears in the main curves (Figure 3), so the adaptive contribution is not isolated. A τ=0 run would reduce to ETE; a τ=∞ run would be full-depth rollout; neither control is reported. This does not falsify the empirical ordering, but it means the paper's central conceptual claim—that variance-based adaptation is what improves the trade-off—is unsupported as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaLook, an adaptive multi-step lookahead decoding method for masked diffusion language models. AdaLook extends the one-step Explore-then-Exploit (ETE) framework by (i) continuing rollout while the variance of candidate cumulative scores is below a threshold tau and (ii) dynamically re-triggering or pruning branches based on the ETE confidence-trigger condition. The authors claim that this adaptive rollout and branch-expansion mechanism improves the accuracy-versus-decoding-steps trade-off compared to ETE and Fast-dLLM, and they report results on LLaDA-8B-Instruct and Dream-7B across several benchmarks, with higher maximum accuracy on MATH (43.6% vs. 42.6% for ETE). The manuscript also reports latency measurements and a discussion of code-generation tasks where lookahead provides limited benefit.","tokens_in":11610,"tokens_out":4107,"duration_ms":47021,"significance":"If the empirical ordering AdaLook > ETE > Fast-dLLM holds with the reported calibration discipline, the paper would provide a useful and non-obvious extension of lookahead decoding for masked diffusion LMs. The comparison against external baselines (ETE, Fast-dLLM) on public benchmarks, with two backbones, is a strength, as is the explicit latency analysis and the candid limitation regarding code generation. However, the central mechanism—the variance gate controlled by tau—is not specified, is described with contradictory stopping conditions in Section 1 vs. Section 4.2/Algorithm 1, and is never isolated through fixed-depth or tau-ablation controls. Because the adaptive early-stopping rule is the paper's core contribution, the manuscript is not reproducible as written and the claimed benefit of adaptivity is not yet supported.","major_comments":[{"comment":"The variance-gate rule is described inconsistently. Section 1 says 'Only when the variance exceeds a predefined threshold does the decoder continue expanding future decoding trajectories,' while Section 4.2 states that if variance falls below tau the decoder proceeds with another rollout step, and Algorithm 1 Step 3 continues rollout when Var({s_j}) < tau. These are opposite conditions. Moreover, tau is never assigned a value: the fixed-hyperparameter list in Section 4.3 omits it, and the calibration subsection does not discuss it. Since tau is the sole quantity controlling rollout depth, the core adaptive mechanism cannot be reproduced or checked. Please correct the description, report the value(s) used, and include a sensitivity analysis around tau.","section":"Section 1 vs. Section 4.2, Algorithm 1 Step 3"},{"comment":"The main curves compare AdaLook against one-step ETE and Fast-dLLM, but no fixed-depth rollout baseline is reported. The paper's own motivation is that fixed-depth rollout is ineffective (Figure 1 and L1/L2), and the adaptive gain is claimed to come from avoiding that fixed-depth failure. Without a control that uses the same k, Tmax, and branch-expansion rules but with a fixed depth (e.g., tau=0 reducing to ETE and tau=infinity giving full-depth rollout, or a fixed R=2 baseline), the reported gains cannot be attributed to the variance gate. They could instead arise simply from the extra depth and branch expansion. Please add such controls and, if possible, a tau ablation on at least one benchmark.","section":"Section 5.2, Figure 3"},{"comment":"The variance-based early-stopping signal is not justified in a way that accounts for the cumulative definition of the score. The score in Eq. (1) sums downstream confidence gains over R rollout steps, so as R grows the scores are sums of more nonnegative terms. The variance of such cumulative sums can increase mechanically with R, independent of whether the candidates are becoming more or less discriminative. If the threshold is reached mainly because the accumulated sums grow with rollout depth, the gate does not measure 'whether candidates remain not sufficiently discriminative,' as claimed. Please provide a derivation, normalize the score by R or use per-step variances, and/or show empirically that the variance dynamics actually separate informative from uninformative continuation cases.","section":"Eq. (1), Section 4.2"},{"comment":"The 'optimized' curves scan all combinations of gamma in [0.1,0.4] and calibrated N/C pairs and report 'the best Pareto-frontier results.' This is a selection procedure on the evaluation benchmark unless the Pareto frontier is chosen on a separate calibration set and then evaluated on the benchmark. The manuscript does not state explicitly which points are selected on which data, how many configurations were run, or whether the same selection rule was applied to all methods. While the standard gamma=0.2 setting is less suspect, the optimized setting as described can overstate the achievable trade-off. Please specify the selection protocol, the calibration set, and the number of runs, and report the standard-setting results with error bars or configuration counts.","section":"Section 5.1 and 4.3 'optimized' setting"}],"minor_comments":[{"comment":"Typographical issue: the fixed-hyperparameter list contains a double comma: 'beta=0.01, , Tmax=2.' Please fix.","section":"Section 4.3"},{"comment":"The control flow after the mixed-branch case is ambiguous. Line 35 ('j* <- argmax_j s_j; go to Commit') appears to be outside the for-loop body, but its placement relative to the Case 1/Case 2/Case 3 branches is not clear from indentation. Reformulate the pseudocode with explicit branch labels and ensure each path either commits or continues to the next iteration.","section":"Algorithm 1"},{"comment":"Table 1 reports per-step latency only. To support the practical-efficiency claim, it would help to also report end-to-end wall-clock time for matched accuracy levels (or total latency), since AdaLook may use more batched forward passes even at the same number of reported decoding steps. Without this, the 7% per-step overhead does not directly translate into a statement about total runtime.","section":"Section 5.3, Table 1"},{"comment":"The curves in Figure 3 are dense and the legend/marker scheme is not described. Consider labeling the standard-setting and optimized-setting curves more clearly, and possibly plotting only a subset of configurations for readability.","section":"Figure 3"},{"comment":"The 'Fast Block Diffusion Sampling' baseline is mentioned in the code-generation discussion but is not defined in the experimental setup. Add a one-sentence definition or a pointer to the relevant prior work.","section":"Section 5.4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant problem, and the empirical ordering against external baselines is promising. The main block is reproducibility and attribution: the variance threshold tau is load-bearing but unspecified, and the comparison lacks fixed-depth controls that would isolate the adaptive mechanism. These issues are fixable with additional experiments and reporting, so I recommend major revision rather than rejection. I would also encourage the editor to ensure that any resubmission includes a precise specification of tau and an ablation around it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: AdaLook is a reasonable extension of ETE's one-step lookahead, and the empirical ordering it reports (AdaLook > ETE > Fast-dLLM) looks credible in the standard, non-optimized setting. But the paper has a load-bearing reproducibility gap: the variance threshold τ, which is the entire adaptive mechanism, is never given a value, and Section 1 states the opposite rule from Section 4.2 and Algorithm 1 about when rollout continues. That needs fixing before I'd trust the central claim.\n\nWhat's new: the variance-gated rollout continuation and the three-case branch-expansion state machine. Those are genuinely absent from ETE, Lee et al. 2025, and Fast-dLLM. The method is clearly specified in Algorithm 1, and the appendix with Dream-7B gives a second backbone that supports the ordering. The paper is also honest about its limits: higher per-step latency, no gains on HumanEval. That earns credit.\n\nSoft spots, in order of severity:\n\n1. τ is never specified. Algorithm 1 uses it at the core decision point; Section 4.3's fixed-hyperparameter list omits it; the calibration subsection never mentions it. A reader cannot reproduce the method without knowing whether τ is 0.01 or 10. This is a fixable omission, but it's the difference between a paper and a recipe.\n\n2. Section 1 says 'Only when the variance exceeds a predefined threshold does the decoder continue expanding' — the opposite of Algorithm 1, which continues when variance is below τ. Likely a typo, but it's not a minor one, because it's the mechanism's defining behavior.\n\n3. No fixed-depth rollout baseline in the main curves. The paper motivates the adaptive approach by saying fixed depth fails, but never shows the fixed-depth curve in Figure 3. Without it, the gains could in principle come from simply doing 2-step rollout plus branch expansion, not from adaptivity.\n\n4. The 'optimized' setting scans γ on the test benchmarks. The standard setting (γ=0.2) still shows the ordering, which helps, but the headline 4.5% number is from the optimized curve.\n\nNone of these falsify the empirical claim; they make it under-supported as written. Calibration overlap with MATH500 is also unstated, which matters given the calibration appears to use MATH500.\n\nBottom line: this is a serious method paper that deserves referee time. I'd send it to review with a request for major revision: report τ, fix the contradiction, add the fixed-depth control, and state the calibration source. The core idea is plausible and the standard-setting results are consistent; the paper just needs to close the reproducibility gaps.","headline":"Sensible adaptive lookahead with a real empirical signal, but the central variance threshold is never reported and the paper contradicts itself about how it works.","tokens_in":12205,"tokens_out":2956,"would_cite":true,"duration_ms":29103,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaLook makes lookahead decoding adaptive by rolling out deeper exactly when candidate scores disagree, yielding a better accuracy-efficiency trade-off than one-step lookahead.","keywords":["diffusion language models","masked diffusion","lookahead decoding","adaptive rollout","variance-gated stopping","branch expansion","parallel decoding","accuracy-efficiency trade-off"],"falsifier":"Measure the distribution of Var({s(R)(j)}) at each rollout step on a held-out set like MATH500. If the variance almost always sits above any reasonable tau after one step, AdaLook collapses into one-step lookahead; if it almost always sits below tau up to Tmax, it collapses into fixed two-step rollout. Either pattern would show the variance gate is not the source of the reported trade-off improvement.","tokens_in":11100,"feed_emoji":"⚡","tokens_out":5407,"duration_ms":52813,"temperature":0.7,"pith_summary":"Masked diffusion language models refine masked tokens in parallel, and lookahead decoding can improve the accuracy-efficiency trade-off by exploring future states before committing tokens. The paper claims that one-step lookahead is too shallow and fixed-depth multi-step rollout is wasteful, because the right rollout depth varies across examples and decoding stages. It proposes AdaLook, which after each rollout step computes the variance of candidate scores: low variance means keep looking, high variance means stop and commit. It also re-triggers lookahead from intermediate branches when a rollout state is still uncertain. On math, reasoning, and knowledge benchmarks, AdaLook achieves better accuracy for a given number of decoding steps than one-step lookahead and confidence-aware parallel decoding, with only about 7% higher per-step latency than its one-step counterpart on modern GPUs.","feed_headline":"Adaptive lookahead beats one-step decoding in diffusion language models","feed_subtitle":"Rolling out deeper only when candidate scores diverge lifts accuracy per decoding step on math and reasoning benchmarks.","key_machinery":"The load-bearing mechanism is the variance gate on cumulative candidate scores. AdaLook maintains k=4 candidate trajectories; each is scored by Eq. (1), the log of the explored candidate's own confidence plus the log sum of downstream high-confidence confidence mass unlocked across R rollout steps. Before each additional rollout step the decoder computes Var({s(R)(j)}) across candidates: if it is below threshold tau, the candidates are not yet discriminative and it rolls out one more step; if it exceeds tau, it commits argmax. A second mechanism, dynamic branch expansion, re-evaluates each branch after every forward pass using the original ETE lookahead-trigger condition and handles the thre","core_discovery":"The central claim is that adaptive multi-step lookahead decoding, gated by candidate-score variance, dominates both shallow one-step lookahead and naive fixed-depth rollout for masked diffusion language models. AdaLook keeps k=4 candidate trajectories alive, and after each rollout step computes a cumulative score that adds the log-confidence mass unlocked across all rollout steps (Equation 1). It continues rollout only when the variance of these scores is below a threshold tau, meaning the candidates have not yet separated into a clear winner; once variance exceeds tau, or the maximum depth Tmax=2 is reached, it commits the highest-scoring hypothesis. To handle heterogeneous intermediate sta","pith_inferences":["The paper never reports the value of the variance threshold tau, even though it is the sole new gate that separates AdaLook from a fixed-depth rollout; a reader cannot reproduce the method's stopping behavior without it.","If tau is chosen poorly, AdaLook degenerates: a threshold that is too high makes variance immediately exceed it (so every lookahead stops after one step, reducing to ETE), while too low a threshold keeps rolling to Tmax on every example (reducing to fixed 2-step rollout). The reported gains therefore hinge on a single number the paper leaves unstated.","The same variance-gating idea could be tested in other search/decoding settings—e.g., lookahead in autoregressive sampling or tree search—where candidate scores accumulate over depth and a natural stopping rule is needed.","A testable variant would set tau per decoding stage or per remaining-mask count rather than globally, since early decoding steps may tolerate more rollout than late ones."],"forward_implications":["Fixed-depth multi-step rollout is not the right extension of one-step lookahead; state-dependent depth is where the accuracy-per-step gain comes from.","On harder reasoning benchmarks like MATH and BBH, adaptive lookahead yields larger gains, suggesting deeper exploration matters most when local confidence is unreliable.","At a fixed number of decoding steps, AdaLook reaches higher maximum accuracy than one-step lookahead on MATH, MMLU, and BBH, so the method is not merely spending more computation.","Code generation behaves differently: both one-step and adaptive multi-step lookahead stay close to plain block diffusion sampling, indicating confidence-based lookahead signals are too local for long-range syntactic dependencies.","The extra latency of adaptive multi-step rollout shrinks on stronger GPUs, implying that the decoding-step savings translate into wall-clock gains as hardware batch efficiency improves."],"fun_headline_variants":["AdaLook: adaptive lookahead cuts decoding steps in diffusion LMs","Variance-gated lookahead beats one-step decoding in diffusion LMs","Adaptive depth lookahead improves diffusion LM decoding efficiency","Decode smarter: adaptive lookahead for diffusion language models","AdaLook: variance-driven rollout improves diffusion LM decoding"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that variance among the k=4 candidate rollout scores—after adding up unlocked-confidence mass over a variable number of steps—reliably indicates whether another rollout step will separate good from bad candidates, and that one fixed cut-off value (tau) can serve across examples, decoding stages, and benchmarks; the paper neither derives this threshold nor reports its value.","fun_headline_variants_meta":{"raw":{"variants":["AdaLook: adaptive lookahead cuts decoding steps in diffusion LMs","Variance-gated lookahead beats one-step decoding in diffusion LMs","Adaptive depth lookahead improves diffusion LM decoding efficiency","Decode smarter: adaptive lookahead for diffusion language models","AdaLook: variance-driven rollout improves diffusion LM decoding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000428,"raw_usage":{"total_tokens":2009,"prompt_tokens":710,"completion_tokens":1299,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":454,"completion_tokens_details":{"reasoning_tokens":1211}},"tokens_in":454,"tokens_out":1299,"duration_ms":9518,"temperature":1.0,"reasoning_tokens":1211,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T22:40:35.441384+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the distribution of Var({s(R)(j)}) at each rollout step on a held-out set like MATH500. If the variance almost always sits above any reasonable tau after one step, AdaLook collapses into one-step lookahead; if it almost always sits below tau up to Tmax, it collapses into fixed two-step rollout. Either pattern would show the variance gate is not the source of the reported trade-off improvement.","supporting_citations":[],"review_version":1}