{"id":"3078ddc6-409a-484f-99ec-ca1e7f248dd5","arxiv_id":"2505.20251","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MCMC search trajectories, pruned to improving transitions, are distilled into an autoregressive model that extrapolates sequence scores beyond both the training data and the search itself.","lead":"The paper trains an autoregressive language model on the improving edits found by a random search over sequences, and shows the model can then generate new sequences that score beyond the search's own best results. This turns expensive Monte Carlo search into a fast, amortized generator for tasks like protein design and controlled text rewriting.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The protein result is not compute-matched: qθ generates 10,000 mutants while MCMC runs only 83–830 proposals, so 'beyond MCMC' may reflect sample budget rather than model quality.","rationale":"The paper has genuine value: the toy example is a clean proof-of-concept, the code is released, and the sentiment task includes oracle evaluation and confidence intervals. My concern is specifically with the headline protein result. The reader's weakest_assumption (guide validity) is reasonable in principle, but the paper's own results weaken it: qθ is evaluated with FoldX, and if the guide s(x) were badly miscalibrated outside [-4,10], qθ would not achieve 46.4% under -7. The guide concern is a conditional risk; the compute asymmetry is an observed property of the experiments. Table 1's 'Iterations' of 3 for qθ is per sample, while MCMC's 83 is the total chain length, and qθ is run 10,000 times. Figure 2's 830-step MCMC is still far fewer final candidates than qθ. A compute-matched comparison could overturn the central claim that qθ extrapolates beyond the sampler. Because this is an empirical gap rather than a formal error, and because the other two tasks and toy example still support the method's usefulness, the appropriate verdict remains conditional: the paper should add a compute-matched MCMC baseline and report qθ's performance as a function of sample count. This does not change the reader's conditional verdict, but it identifies a different and more decisive weak point.","tokens_in":24355,"tokens_out":8587,"duration_ms":86856,"concrete_test":"Run MCMC with the same final-sample budget as qθ: 10,000 independent chains of 83 steps each (or one chain of 830,000 proposals), evaluate every visited or final state with FoldX, and compute the proportions under -5, -6, and -7. Also recompute qθ's threshold fractions restricted to its first 83 and first 830 generated mutants; if MCMC at equal budget reaches or exceeds qθ, or if qθ's advantage shrinks at matched sample counts, the extrapolation claim is largely a sampling-budget artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the unequal sample budget in the protein task. Appendix B.1 states that at inference qθ is prompted with the wildtype and 10,000 mutants are sampled; with fixed-length Δ energy episodes of length 3 (Table 4), this is roughly 30,000 autoregressive passes and 10,000 final sequences. MCMC in Table 1 runs 83 proposals per chain, and Figure 2 extends this to 830 proposals in a single chain. The reported 'Iterations' column counts episode length, not total generation budget, so qθ's '3' hides a 10,000-fold replication. Consequently, the claim in §3.5 that further MCMC 'does not begin to approach' qθ is not established for equal compute: qθ sees roughly two orders of magnitude more final candidates. The guide-scorer assumption in §2.1 is acknowledged by the authors and is partially validated by the oracle/FoldX evaluation; the compute asymmetry is an actual inconsistency in the reported comparison and is more directly testable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for learning extrapolative sequence transformations. The authors run Metropolis-Hastings MCMC with masked-language-model proposals under a guide energy function, subsample state transitions from the resulting chains using several strategies (uniform thinning, first/best, fixed- and variable-length energy improvement), and fine-tune an autoregressive model qθ on episodes of the form x0, s1, x1, ..., xn, optionally conditioning on intermediate scores. At inference, qθ iteratively revises an input sequence. The method is evaluated on a toy binary-sequence task, ACE2 protein stability (with FoldX as oracle), Yelp sentiment extrapolation, and Reddit anonymization. The central evidence is the protein result (Table 1), where the best qθ variant achieves 74.8% of mutants under ddG -5 versus 27.0% for MCMC, and Figure 2, which shows that 830 MCMC steps do not reach qθ's performance. The paper claims that qθ extrapolates as well as or better than MCMC while using far fewer iterations.","tokens_in":24539,"tokens_out":10743,"duration_ms":113052,"significance":"The idea of distilling MCMC exploration into a conditional autoregressive editor is appealing, and the toy example is a clean proof of concept. The paper ships code, evaluates with external oracles (FoldX, the oracle sentiment scorer), and reports extensive ablations of episode-creation strategies, reward types, and MCMC chain lengths, which is commendable. If the protein result survives matched comparisons, the method would offer a practical way to convert expensive stochastic search into a fast generative model for sequence design. The main caveats are experimental: the headline protein comparison is not sample-matched, some baseline numbers are unreproduced, and the best episode strategy is selected after inspecting test metrics. These issues make the current empirical significance uncertain rather than established.","major_comments":[{"comment":"The central protein comparison is not sample-matched. Appendix B.1 states that at inference qθ is prompted with the wildtype sequence and 10,000 mutants are sampled, whereas the MCMC row in Table 1 is based on 83 proposals and Figure 2 extends this to 830 proposals in a single chain. The 'Iterations' column is defined in §3 as the number of revised states in the training episode, not the number of final candidates or a compute measure, so qθ's '3' hides a 10,000-fold replication. Consequently, the claim in §3.5 that further MCMC 'does not begin to approach' qθ, and the abstract's claim of 'significantly higher sample efficiency', are not established for equal sample budget or equal compute. Please add matched comparisons, for example by reporting qθ restricted to 83 and 830 final candidates, running MCMC for a comparable number of proposals (e.g., 10,000), or reporting wall-clock time or FLOPs for both methods.","section":"§3.1, Table 1, Figure 2; Appendix B.1"},{"comment":"The headline numbers in Tables 1–3 are selected as the best of four episode-creation strategies after examining the test metrics. Table 1 reports fixed-length Δ energy because it is best in Table 4; Table 2 reports first/best because it is best in Table 5; Table 3 reports variable-length Δ energy because it is best in Table 6 under the reported metrics. No validation-based selection of the episode strategy is described; Appendix B.1 only addresses selection of the EBM weights. This selection-on-test inflates the reported effect sizes. Please either fix the episode strategy on a validation split before inspecting test results, or report the selection procedure explicitly and quantify its bias, for example by holding out part of the test set for final evaluation.","section":"§3.4, Tables 4–6"},{"comment":"The baseline comparisons are taken from Padmakumar et al. (2023) even though §3.1 states that the authors were unable to reproduce those published results with the released code and that the original procedure exhibits large variance. Comparing qθ against unreproduced numbers weakens the claims that qθ 'significantly outperforms our baselines' in Table 1 and that it outperforms baselines in the sentiment task. Please either provide reproduced baseline numbers under the same evaluation protocol, or clearly mark the comparison as previously published and explain why the numbers are still considered directly comparable.","section":"§3.1, Table 1; §3.2, Table 2"},{"comment":"The anonymization evaluation may be partly circular. The energy function used to generate MCMC training episodes includes a LUAR stylistic-similarity term, and the EER metric is computed with a LUAR-CRUD author embedding. Appendix G.2 notes that the SBERT checkpoint used in the energy function differs from the one used in evaluation, but it does not state whether the LUAR embedding is the same in both places. If the same authorship representation is used both to guide generation and to measure success, the reported EER improvement partly reflects optimization of the evaluation metric. Please clarify whether the LUAR checkpoints differ, and if they are identical, re-evaluate with a different author-embedding model or a held-out variant.","section":"§3.3, Appendix G.2"},{"comment":"The method relies on the assumption that the guide scorer s(x) is directionally informative outside its training support, because training episodes are selected according to improvements in s (for example, the Δ-energy strategies in §2.3). The paper states this limitation in §2.1, and the oracle/FoldX evaluations of final outputs provide indirect evidence that the assumption holds in the tested cases. However, there is no direct check that the selected MCMC transitions are true improvements according to the oracle. Please add a small validation experiment for at least the protein task in which a sample of selected transitions is scored by both the guide and the oracle, and report the correlation or the fraction of concordant pairs in the extrapolation region. This would turn the weakest assumption into a measured quantity.","section":"§2.1–§2.3"}],"minor_comments":[{"comment":"The definition 's(x) = exp PL i ri' contains an undefined symbol 'ri' and an unclear sum; it should presumably be 's_i' with an explicit summation over i. Please correct the notation.","section":"§2, toy example"},{"comment":"Tables 1 and 3 report no confidence intervals or numbers of independent runs, while Table 2 reports 95% confidence intervals over three test sets. Please add variability estimates for all three tasks so that the reader can judge the stability of the reported gains.","section":"§3.1 and §3.3, Tables 1 and 3"},{"comment":"The term 'iterations' is defined as the number of revised states in the training episode, which conflates the number of editing steps with computational cost. Please consider renaming this column (e.g., 'steps') and reporting a separate compute measure such as tokens generated or FLOPs.","section":"§3, 'Iterations' definition"},{"comment":"The sentence 'we prompt with the wildtype sequence and sample 10,000 mutants' should state whether these are independent ancestral samples or include beam search, and how the <stop> symbol is handled in the protein task.","section":"Appendix B.1"},{"comment":"The sentence 'In both cases, we report the better-performing variant with scorer' is ambiguous; please specify which variants were chosen and on what basis they were selected.","section":"§3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a useful framework and a clean toy demonstration, but the main empirical claim currently rests on an unequal sample-budget comparison in the protein task. I believe the authors can address this with additional matched experiments, and the other concerns (selection on test metrics, unreproduced baselines, possible LUAR circularity) are also fixable within the scope of a revision. I would not reject at this stage, but the headline 'beyond MCMC' claim should not be accepted without the matched comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely useful idea—distill MCMC search trajectories into a cheap autoregressive policy—and the toy example is a clean demonstration that this can extrapolate past the sampler. The FoldX evaluation on the real protein task is real external grounding. But the headline comparison against MCMC is not compute-matched, and the paper oversells it.\n\nWhat is new: the episode-selection strategies (Δ-energy-based state selection) and the finding that the distilled model can exceed the sampler's best states. The combination of MLM proposals, MCMC, and fine-tuning an AR model on selected transitions is built from known parts—Goyal, ICE, Decision Transformer—but the packaging and the analysis of episode selection is a contribution.\n\nSoft spots, in order of importance. First, the protein claim. qθ generates 10,000 mutants at inference (Appendix B.1); MCMC in Table 1 runs 83 proposals and Figure 2 extends to 830. So qθ sees roughly twelve times more final candidates than even the extended MCMC. The 'further MCMC does not approach qθ' statement in §3.5 is not established for equal compute. If MCMC had 10,000 proposals, it might well match qθ's tail. That matters because the paper's main selling point is extrapolation beyond the sampler, not just amortized speed. Second, the best episode variant is selected after looking at test metrics (Table 4), which inflates the reported numbers. Third, the protein table has no error bars, and the baselines are taken from a paper the authors say they could not reproduce; that makes the baseline comparisons shaky. Fourth, the guide-scorer assumption in §2.1 is acknowledged but not stress-tested; the FoldX results partially validate it, but a systematic exploration of guide failure modes would make the method more robust.\n\nNone of these sink the method. The efficiency story is real: qθ produces useful samples in far fewer iterations than MCMC, and the anonymization results show it can approximate MCMC's quality at a fraction of the cost. The paper would benefit from a compute-matched comparison (e.g., running MCMC for 10,000 proposals) and a rephrased abstract that claims competitive quality with much faster generation rather than outright superiority.\n\nWho should read it: anyone working on controllable generation, protein design, or amortized search. It's a solid empirical paper. My verdict: deserves serious refereeing, but it needs a major revision to fix the compute mismatch and tone down the claims.","headline":"The core distillation idea is sound and the toy experiment is a genuine proof of concept, but the protein 'beats MCMC' claim rests on an unequal sample budget and post-hoc variant selection.","tokens_in":25086,"tokens_out":3156,"would_cite":true,"duration_ms":31222,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a fine-tuned model can extrapolate beyond the best scores achieved by the MCMC sampler that generated its training data, as shown by protein-stability rates several times higher than the sampler's in the…","keywords":["extrapolation","Markov chain Monte Carlo","Metropolis-Hastings","masked language models","autoregressive sequence generation","protein engineering","sentiment control","text anonymization"],"falsifier":"Run a protein-task variant where the guide is trained only on ddG in $[-4,10]$ but an adversarial validation oracle is constructed so that below $-4$ the sequences the guide ranks as best are actually the least stable; if $q_\\theta$ trained on $\\Delta$-energy episodes then fails to beat MCMC under that oracle, the guide's out-of-range directional signal is the breaking point. Concretely, one would freeze the guide's weights, relabel only the extrapolation region of the evaluation oracle, and compare the FoldX ddG distributions of $q_\\theta$ and MCMC on that oracle.","tokens_in":24129,"feed_emoji":"🧬","tokens_out":10722,"duration_ms":96964,"temperature":0.7,"pith_summary":"The paper argues that the exploratory power of Markov chain Monte Carlo can be distilled into a fast autoregressive model: run MCMC with a masked-language-model proposal, prune each chain to the transitions that most improve a cheap score, and fine-tune a sequence model to reproduce those transitions. The central claim is that the resulting model $q_\\theta$ extrapolates as well as or better than the sampler that produced its training data, while needing only a handful of generation steps. In the ACE2 protein-stability task, $q_\\theta$ generates mutants below ddG $-5$, $-6$, and $-7$ at far higher rates than MCMC itself (74.8%, 61.6%, and 46.4% versus 27.0%, 4.1%, and 0.5%), and running MCMC ten times longer does not close the gap. On sentiment control the model matches most of MCMC's extrapolation rate with one step instead of roughly 500, and on anonymization it keeps most of the semantic similarity with far fewer iterations. If the claim holds, test-time search compute can be converted into training-time data, giving sample-efficient extrapolation in domains where the oracle is expensive to query.","feed_headline":"MCMC-trained model out-extrapolates MCMC itself","feed_subtitle":"Selected sampler transitions teach a fast model to reach stability targets the sampler itself misses.","key_machinery":"The central machinery is a two-phase distillation loop. Phase one builds an energy-based surrogate $\\ln p(x) = s(x) - \\ln Z$ from a cheap guide scorer plus auxiliary terms, and samples it with Metropolis-Hastings whose proposal distribution is a masked language model that masks a random subset of tokens and infills them. Phase two sub-samples the resulting chains into short training episodes — 'first/best', thinning, or $\\Delta$-energy selection — and encodes each episode as a token sequence with special delimiters and observed or predicted scores, then fine-tunes an autoregressive model $q_\\theta$ by teacher forcing to predict each next state given the history. At inference $q_\\theta$ iteratively revises until it emits a stop token, so the model learns a non-autoregressive iterative transformation embedded in an autoregressive decoder. The $\\Delta$-energy episode selection, which keeps transitions with the largest energy improvements, is the variant that gives the largest extrapolation gains in the main protein experiments.","core_discovery":"The discovery is that a model trained on selected MCMC transitions can learn a transformation operator that keeps improving the target property beyond the score range the guide was trained on, and beyond the best states the sampler itself visited. Starting from a state, $q_\\theta$ produces a sequence of revisions, and each revision is trained to imitate a sub-sampled transition that decreased the energy. The paper shows that this operator inverts the usual cost structure: the expensive MCMC search is paid once at training time, and inference is a few autoregressive passes. The strongest evidence is the protein task, where $q_\\theta$'s extrapolation-range success rates exceed every baseline and MCMC's, despite MCMC having been the only source of its training data.","pith_inferences":["A direct consequence the paper leaves implicit is that the method inherits the guide scorer's blind spots: whenever the guide's ranking degrades outside its training support, $q_\\theta$ would be trained toward spurious energy decreases, so the reported successes depend on the guide's gradient remaining informative in the extrapolation region.","The same distillation recipe should transfer to other expensive search settings with cheap scoring signals and pre-trained infilling models, such as small-molecule design or program synthesis, where MCMC-style exploration is already available but inference-time search is costly.","A testable extension is to close the loop between sampler and model: use $q_\\theta$'s own generated states and their oracle scores to re-weight or extend future MCMC episodes, which could push extrapolation further than a single distillation pass.","The approach turns inference-time scaling into training-time scaling, suggesting an unexplored trade-off curve between the MCMC budget spent on episodes and the extrapolation quality of $q_\\theta$; mapping that curve would help practitioners decide how much search compute to amortize."],"forward_implications":["$q_\\theta$ can match or beat MCMC on extrapolation while needing far fewer inference iterations: 3 versus 83 on protein, 1 versus roughly 496 on sentiment, and 4 versus 4498 on anonymization.","On the protein task, running MCMC for ten epochs (830 steps) does not approach $q_\\theta$'s extrapolation performance, so the gain is not simply additional search.","Episode construction matters: fixed-length $\\Delta$-energy selection outperforms uniform thinning on protein extrapolation, first/best is the most effective and fluent variant for sentiment, and variable-length $\\Delta$-energy preserves the most semantics for anonymization.","$q_\\theta$ can be trained to predict scores rather than call the guide at inference, removing the per-step scoring cost with little loss in most configurations.","The pipeline works on extrapolation tasks (protein stability, sentiment) and on an interpolation task (anonymization), so the distillation mechanism is not specific to one domain."],"supporting_citations":[{"why":"It defines the ACE2 stability and sentiment extrapolation tasks, the ICE baseline, and the scorer/oracle setup the paper replicates.","marker":"(Padmakumar et al., 2023)"},{"why":"It demonstrates that masked language models can serve as effective Metropolis-Hastings proposal distributions, which the sampler in this paper relies on.","marker":"(Goyal et al., 2021)"},{"why":"It contributes the ACE2 dataset restricted to ddG between -4 and 10 and the Genhance baseline used for comparison.","marker":"(Chan et al., 2021)"},{"why":"It supplies the ProtBert and Prot-T5 models used as the protein guide scorer and as the proposal distribution for the protein task.","marker":"(Elnaggar et al., 2020)"},{"why":"It provides the FoldX force-field oracle that computes ddG for evaluating the generated mutants.","marker":"(Schymkowitz et al., 2005)"},{"why":"It is the FUDGE classifier-guided decoding baseline for the sentiment control task.","marker":"(Yang & Klein, 2021)"},{"why":"It supplies the reward-to-go sequence-modeling idea that motivates conditioning q_theta on observed or predicted scores during generation.","marker":"(Janner et al., 2021)"},{"why":"Its style-transfer episode generation approach is adapted to build the anonymization training episodes.","marker":"(Khan et al., 2024)"},{"why":"It provides the T5 models used as the sentiment proposal distribution and as the base model fine-tuned into q_theta.","marker":"(Raffel et al., 2020)"}],"fun_headline_variants":["MCMC spawns model that beats MCMC at extrapolation","From Markov chains to faster extrapolation","Learn to extrapolate by imitating MCMC steps","MCMC's own steps train a better extrapolator","Autoregressive model outruns MCMC in extrapolation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the cheap guide scorer $s(x)$, trained only on the observed score range, still points in the right direction for sequences outside that range; if its gradient misleads there, the selected 'improvement' transitions would train $q_\\theta$ toward spurious targets and the claimed extrapolation would not reproduce under the true oracle.","fun_headline_variants_meta":{"raw":{"variants":["MCMC spawns model that beats MCMC at extrapolation","From Markov chains to faster extrapolation","Learn to extrapolate by imitating MCMC steps","MCMC's own steps train a better extrapolator","Autoregressive model outruns MCMC in extrapolation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2949,"prompt_tokens":923,"completion_tokens":2026,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":1948}},"tokens_in":539,"tokens_out":2026,"duration_ms":14620,"temperature":1.0,"reasoning_tokens":1948,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:56:41.138170+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a protein-task variant where the guide is trained only on ddG in $[-4,10]$ but an adversarial validation oracle is constructed so that below $-4$ the sequences the guide ranks as best are actually the least stable; if $q_\\theta$ trained on $\\Delta$-energy episodes then fails to beat MCMC under that oracle, the guide's out-of-range directional signal is the breaking point. Concretely, one would freeze the guide's weights, relabel only the extrapolation region of the evaluation oracle, and compare the FoldX ddG distributions of $q_\\theta$ and MCMC on that oracle.","supporting_citations":[],"review_version":1}