{"id":"babd9a13-037b-4582-95f8-5a4c58645b42","arxiv_id":"2507.08018","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"R3 uses process reward model scores to decide which blocks of a masked diffusion language model's output to remask and regenerate, improving MATH accuracy from 29% to 43% without retraining.","lead":"This paper presents R3, a way to improve text from masked diffusion language models without retraining them: a process reward model scores parts of the output, and low-scoring parts get re-masked and regenerated. On a 127-question math benchmark, the method lifts accuracy from about 29% to 43%, though it does not beat a stronger best-of-N baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing random-remasking control: without it, the claimed PRM-guided improvement is not distinguishable from extra diffusion refinement passes.","rationale":"The reader's weakest assumption is exactly the load-bearing concern I identify: the paper assumes PRM block scores are informative enough to localize errors, yet no experiment isolates the PRM's contribution to remasking from the benefit of simply regenerating tokens. The claim in Section 4.4 that R3's improvement demonstrates 'the benefit of PRM-guided iterative refinement' requires a control where remasking is unguided but compute is matched. Without such a control, the accuracy gain could be explained by the extra diffusion passes, the PRM-based selection among NS candidate refinements, or random chance on 127 questions. The compute accounting issue (Section 4.4 claims 2-4 PRM calls, but Step 5 of Section 3.2 requires NS extra PRM calls per triggered window) is secondary and internal, but it reinforces the need for a careful controlled comparison. I agree with the CONDITIONAL verdict: the paper is promising and requires no retraining, but the central attribution claim is unverified until the random-remask experiment is run. The verdict should remain CONDITIONAL, pending that test.","tokens_in":6252,"tokens_out":4855,"duration_ms":57863,"concrete_test":"Run two controls on the same 127 questions with 10 seeds: (i) R3 with remasking probabilities drawn uniformly at random instead of PR(S_b), preserving the total number of masked tokens per window; (ii) R3 with remasking probabilities anti-correlated with PRM scores (higher score -> more tokens masked). Keep all other steps identical: threshold tau=0.8, NS=5, PRM-based candidate selection, K=8, beta_I=0.8. Compare mean accuracy and bootstrap 95% CI across seeds. If control (i) or (ii) is within one standard error of R3's 42.52%, the process-guided remasking contribution is not supported. Also report the trigger rate and the average masking proportion per window to verify the claimed PRM call counts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PRM-guided remasking drives the 29.13-to-42.52 accuracy gain (Section 4.4). The only PRM-dependent decision is the per-block remasking proportion PR(S_b) (Section 3.2, Step 4 and Appendix B); within a block, tokens are selected at random, so the PRM signal only modulates how many tokens in each block are masked. The paper provides no control in which remasking probabilities are random, constant, or anti-correlated with PRM scores while holding the compute budget fixed (same window triggers, NS=5 refinements, PRM-based candidate selection). If an equal-compute random-remask control achieves similar accuracy, then the improvement is attributable to the additional diffusion refinement passes (self-correction by resampling), not to process guidance. Furthermore, Qwen2.5-Math-PRM-7B is a step-level PRM trained to score reasoning steps with delimiters, but R3 applies it to fixed 32-token blocks that need not align with reasoning steps; no calibration or correlation analysis is reported. Appendix B's exponential mapping is unvalidated: no evidence shows that a lower block score identifies blocks whose remasking improves the final answer. The qualitative example (Figure 1) is a single instance and even shows LLaDA correcting 'b=3' to 'b=2' after remasking, which could occur by chance re-generation at temperature 0.8. Without a random-remask baseline, the foundational assumption that PRM scores localize errors is untested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces R3, an inference-time framework that couples a masked diffusion language model (LLaDA-8B-Instruct) with a process reward model (Qwen2.5-Math-PRM-7B) to improve mathematical reasoning outputs. R3 generates text in fixed-length blocks, periodically scores the latest K blocks with the PRM, remasks a larger proportion of tokens in lower-scoring blocks, proposes several refined window candidates, and selects the best candidate by PRM score. Experiments on a 127-question subset of MATH 500 report accuracy improving from 29.13% (plain diffusion pass@1) to 42.52% with window size K=8, while a block-wise Best-of-N baseline reaches 48.03% using more PRM calls. The authors conclude that PRM-guided process supervision drives the improvement and that R3 is computationally efficient.","tokens_in":6655,"tokens_out":4009,"duration_ms":47536,"significance":"The core idea—using block-level PRM scores to allocate remasking effort during iterative diffusion refinement—is simple, training-free, and potentially useful for any masked diffusion model. The use of final-answer correctness on MATH 500 as the evaluation signal is a strength: the reported gains are not circular with respect to the PRM. However, the central attribution of the gain to process guidance is not yet established, and the paper's efficiency argument contains a counting error. If the missing controls and corrected accounting confirm the claims, this would be a solid and practical contribution for inference-time refinement of diffusion language models.","major_comments":[{"comment":"The central claim that PRM guidance drives the 29.13% to 42.52% improvement is not supported because there is no control that applies the same number and schedule of diffusion refinement passes with random, constant, or anti-correlated remasking probabilities. In the proposed method, the PRM only determines the per-block remasking proportion PR(S_b); within each block, tokens are selected at random. The extra refinement passes and the PRM-based candidate selection (Steps 4–6 of Section 3.2) are shared with any equal-compute remasking control. Please add an ablation that keeps the window trigger schedule, N_S=5, and PRM-based candidate selection fixed, but replaces PR(S_b) with uniform random remasking, and report accuracy, confidence intervals, and PRM call counts. Without this control, the gain cannot be attributed to process guidance rather than to additional resampling.","section":"Section 4.4, Table 1"},{"comment":"The mapping from a PRM block score to a remasking probability, q_b = exp(-alpha_B * S_b), is not validated as an error-localization signal. Qwen2.5-Math-PRM-7B is a step-level PRM, while R3 applies it to fixed 32-token blocks that need not align with reasoning steps; no analysis shows that low-scoring blocks actually contain the errors whose remasking improves the final answer. The qualitative example in Appendix A, Figure 1, is a single stochastic sample at temperature 0.8 and cannot establish this. Please provide a correlation or calibration analysis, such as comparing PRM block scores with known error positions on an annotated subset, or an ablation that remasks high-scoring blocks instead of low-scoring blocks; either would directly test whether the PRM signal localizes errors.","section":"Section 3.2, Appendix B"},{"comment":"The PRM-call accounting is incorrect. The text states that R3 makes only ceil(N_total/K) calls (2 for K=8) in the best case and 2*ceil(N_total/K) calls (4 for K=8) in the worst case, but this counts windows, not block-level PRM evaluations. Even with no refinement triggered, scoring K blocks at each of the ceil(N_total/K) windows requires N_total block-level PRM calls (16 for N_total=16, K=8). If both windows trigger refinement, scoring N_S=5 candidate windows for each window adds 2*5*8=80 block-level calls, for a total of 96, which exceeds the 80 calls used by B-BoN. Please correct the accounting and report measured numbers of block-level PRM evaluations and wall-clock times for all compared methods.","section":"Section 4.4"},{"comment":"The quantitative comparison lacks statistical support and the dataset subset is under-specified. The 127-question subset of MATH 500 is not described, so selection bias cannot be ruled out. The accuracy differences in Table 1 are reported without error bars, confidence intervals, or significance tests; for example, the 42.52% and 48.03% values are based on 54/127 and 61/127 correct, whose difference is within the range expected from binomial sampling noise. Please report bootstrap confidence intervals over questions, perform multiple generation seeds, and describe how the 127 questions were selected. A sensitivity analysis of the key hyperparameters (tau_thresh, beta_I, alpha_B, N_S, and block length) would also help establish that the K=8 result is not a fragile operating point.","section":"Section 4.1, Table 1"}],"minor_comments":[{"comment":"The pseudocode 'W_hat^(s) <- {M_diff(X^(s)_masked|mask for block b) for each block b in window}' is ambiguous: it could mean generating each block independently rather than infilling the whole masked window jointly, which would be inconsistent with Section 3.2's description of refining W_j,masked as a window. Please clarify the exact denoising procedure.","section":"Algorithm 1, line 16"},{"comment":"The text says the original window is retained if no candidate offers improvement, but both the step description and Algorithm 1 select s* = arg max_s M(S^(s)'_W) without comparing against the original window's score. Specify how 'improvement' is defined and whether a candidate worse than the original can ever replace it.","section":"Section 3.2, Step 6"},{"comment":"The relationship between the number of demasking steps (128) and the block generation/refinement process is not specified; in particular, it is unclear whether each block is generated with 128 demasking steps independently or whether the 128 steps are distributed over the whole 512-token sequence. Please clarify.","section":"Section 4.1"},{"comment":"The normalization formula for PR(S_b) uses min and max over the window, but no value for the epsilon constant is given; please report the exact value used in the experiments.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: R3 is a straightforward, training-free inference-time method — use a PRM to decide how many tokens in each 32-token block to remask, then resample with a masked diffusion model. The paper is honestly written, small-scale, and the arithmetic is internally consistent (37/127 = 29.13%, 54/127 = 42.52%). That is not nothing. If the gain holds, it gives a general plug-in for LLaDA-style models.\n\nWhat it does well: the framework is simple, the algorithm is unambiguous, and the comparison to block-wise Best-of-N provides a useful anchor. The authors also state clearly that BoN is more accurate but costs far more PRM calls. The related work correctly positions R3 against ReMDM (remasks without PRM guidance) and d1 (retrains with RL).\n\nWhere it is soft: the evaluation is a single 127-question MATH subset, one base model, one PRM, no error bars, no significance tests. Hyperparameters (tau, beta_I, alpha_B, p_min, K) are not sensitivity-analyzed. That is minor-to-moderate for a short workshop paper.\n\nThe load-bearing gap is the missing random-remasking control. The only PRM-dependent decision is the per-block remasking proportion; token selection within a block is random. Without a control where remasking probabilities are constant, random, or anti-correlated with PRM scores at equal compute, you cannot attribute the 29-to-42 point accuracy gain to process guidance rather than extra refinement passes. The stress-test note is right.\n\nThe efficiency count in Section 4.4 is also wrong. A triggered window costs K initial scores plus NS*K candidate scores, not 2–4 calls total. For K=8 and NS=5, that is 8 + 40 = 48 PRM calls per refined window. The \"best case 2 calls\" ignores candidate scoring. That is a real error, not a nit.\n\nAdditionally, the PRM is a step-level model trained on reasoning steps with delimiters, but R3 applies it to fixed 32-token blocks. No calibration or correlation analysis shows that low block scores correspond to token-level errors. Appendix B's exponential mapping is plausible but unvalidated. These are concerns, not fatal: the reported numbers are not contradictory.\n\nBottom line: this is a workshop-scale contribution with a plausible mechanism and real soft spots. It deserves a serious referee if the authors add a random-remask baseline, fix the compute accounting, and release code/data. Worth reading for anyone working on inference-time scaling for diffusion LMs, but I would not cite it yet.\n\nRecommendation: send to peer review, conditional on revisions.","headline":"A clean, training-free PRM-guided remasking idea whose central gain is unproven: the missing random-remask baseline and the miscounted PRM-call budget are the two things to fix first.","tokens_in":7138,"tokens_out":2442,"would_cite":false,"duration_ms":26358,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A process reward model can guide masked text diffusion at inference time: R3 scores generated blocks, proportionally remasks low-scoring ones, refines them, and raises MATH solve accuracy from 29.13% to 42.52% without retraining.","keywords":["masked diffusion models","process reward models","inference-time refinement","remasking","block diffusion","mathematical reasoning","self-correction","LLaDA"],"falsifier":"Compare R3 against a control that triggers on the same windows but remasks the same total number of tokens uniformly at random within the window, without using block scores. If random-position remasking matches the 42.52% accuracy, the PRM localization is not doing the work; a second test would measure, on a labeled error set from MATH, whether PRM block scores actually correlate with the density of incorrect tokens in that block.","tokens_in":6073,"feed_emoji":"🔁","tokens_out":8807,"duration_ms":88946,"temperature":0.7,"pith_summary":"R3 is an inference-time loop for masked diffusion text generation. For every new 32-token block, a process reward model scores the block; when the weakest block in a window falls below a threshold, R3 remasks a larger fraction of tokens in the low-scoring blocks and asks the diffusion model to fill the masks, then keeps the best rescored candidate. On a 127-question MATH 500 subset, the paper reports that R3 lifts LLaDA-8B solve accuracy from 29.13% to 42.52% with window size eight, while using only 2-4 reward-model calls per sequence against 80 for block-wise best-of-N. The method requires no retraining of the generator or the reward model, so it would turn any pretrained masked diffusion model and any step-level quality signal into a self-correcting text generator.","feed_headline":"Process-guided remasking lifts math accuracy to 42.5%","feed_subtitle":"R3 scores 32-token blocks, remasks weak ones, refines, beating plain diffusion with 2-4 reward calls vs 80.","key_machinery":"The load-bearing object is the remasking probability map from PRM score to mask density. For each block b with score $S_b$, R3 computes $q_b = \\exp(-\\alpha_B S_b)$, normalizes these values across the window to $[p_{\\min}, 1]$, and masks a proportion of tokens $\\rho_b = \\beta_I \\tilde{P}_R(S_b)$ in each block, so low scores produce heavy masking. This map is what focuses the diffusion model's refinement budget on the parts of the text the PRM judges weakest, and the windowed architecture (review every K blocks, trigger at threshold, score $N_S$ candidates) is what keeps the reward-model cost small.","core_discovery":"The central claim is that PRM-guided proportional remasking is a workable self-correction mechanism for block diffusion. In R3, blocks are generated sequentially; every K blocks, the reward model scores each block in the window; if the minimum score is below $\\tau_{\\mathrm{thresh}}$, the window is remasked using a probability that increases as block score decreases, several candidate rewrites are generated and rescored, and the best is spliced back into the sequence. The paper's quantitative evidence is a jump from 29.13% to 42.52% accuracy on a MATH subset at K=8, with accuracy rising as the window grows from 4 to 8, and a 48.03% result for block-wise best-of-N that costs 80 PRM calls per sequence rather than R3's 2-4. The authors read these numbers as showing that targeted, process-guided refinement captures most of the benefit of exhaustive candidate scoring at a fraction of the compute.","pith_inferences":["The paper only tests math reasoning, but the review-remask-refine loop should transfer to code repair, structured generation, or any task where a step-level quality signal exists and the base model can infill masked spans.","The 2-4 versus 80 PRM-call gap implies an unexplored Pareto frontier: a small number of targeted remasking rounds may capture most of the benefit of large candidate sets, and the tradeoff curve is worth measuring directly.","An oracle that always masks exactly the erroneous tokens would set the ceiling for this class of methods; comparing R3 against that oracle would isolate how much headroom remains in the PRM localization step.","The authors' suggested future work of post-training self-correction suggests a natural variant: train the diffusion model to predict its own remasking distribution from its internal states, keeping the R3 structure but dropping the external PRM at deployment."],"forward_implications":["R3 raises LLaDA-8B solve accuracy from 29.13% to 42.52% on the MATH 500 subset with no retraining, so the same loop can be bolted onto existing masked diffusion deployments.","Because accuracy rises with window size from K=4 to K=8, the paper's account implies that broader PRM context improves both error detection and the quality of refinement.","R3 uses 2-4 PRM calls per sequence against 80 for block-wise best-of-N while still reaching 42.52% versus 48.03%, so targeted remasking buys most of the gain of exhaustive sampling much more cheaply.","The framework is model-agnostic on both sides: any masked diffusion model that can infill spans and any step-level reward model can be combined this way, not just the specific pair evaluated here."],"supporting_citations":[{"why":"Supplies LLaDA-8B, the masked diffusion model whose block generation and infilling R3 uses for both initial text and refinement.","marker":"Nie et al., 2025"},{"why":"Supplies Qwen2.5-Math-PRM-7B, the process reward model that scores blocks in R3 and also drives the block-wise best-of-N baseline.","marker":"Zhang et al., 2025"},{"why":"Establishes the remasking primitive, the ability to mask already generated tokens and regenerate them, which R3 steers with PRM scores.","marker":"Wang et al., 2025"},{"why":"Defines block diffusion (BD3-LM), the block-wise generation paradigm that R3's windows operate on.","marker":"Arriola et al., 2025"},{"why":"Provides the process-reward evaluation methodology for mathematical reasoning that motivates using step-level scores for review.","marker":"Zheng et al., 2024"},{"why":"Contrast: the d1 framework post-trains diffusion language models with SFT and RL, against which R3's no-retraining inference-time approach is positioned.","marker":"Zhao et al., 2025"}],"fun_headline_variants":["Block diffusion self-correction: 42.5% MATH with 2-4 PRM calls","R3: remask low-score blocks, refine - MATH hits 42.5%","Self-correcting diffusion via PRM-guided remasking: 42.5% on MATH","Block-wise remasking: 42.5% MATH with 2-4 reward calls vs 80","Training-free self-correction: R3 hits 42.5% MATH"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a PRM score for a 32-token block reflects which tokens inside it are wrong; if block-level scores do not localize token-level errors, the remasking is essentially random rewrites and the reported accuracy gain cannot be attributed to process guidance.","fun_headline_variants_meta":{"raw":{"variants":["Block diffusion self-correction: 42.5% MATH with 2-4 PRM calls","R3: remask low-score blocks, refine - MATH hits 42.5%","Self-correcting diffusion via PRM-guided remasking: 42.5% on MATH","Block-wise remasking: 42.5% MATH with 2-4 reward calls vs 80","Training-free self-correction: R3 hits 42.5% MATH"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002093,"raw_usage":{"total_tokens":8111,"prompt_tokens":895,"completion_tokens":7216,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":7097}},"tokens_in":511,"tokens_out":7216,"duration_ms":57584,"temperature":1.0,"reasoning_tokens":7097,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:25:04.446781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare R3 against a control that triggers on the same windows but remasks the same total number of tokens uniformly at random within the window, without using block scores. If random-position remasking matches the 42.52% accuracy, the PRM localization is not doing the work; a second test would measure, on a labeled error set from MATH, whether PRM block scores actually correlate with the density of incorrect tokens in that block.","supporting_citations":[{"cited_title":"T., Han, J., Yang, Z., Qi, Z., Sahoo, S","cited_arxiv_id":null,"evidence_quote":"Defines block diffusion (BD3-LM), the block-wise generation paradigm that R3's windows operate on."}],"review_version":1}