REVIEW 4 major objections 4 minor 17 references
By reframing sequence optimization as planning over executable edit trajectories, STRIDE lifts oracle-scored protein editing success from 42% to 89% and novelty from 47% to 97%, and improves molecule-editing controllability.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 19:03 UTC pith:OFKEG6FJ
load-bearing objection STRIDE's executable edit-trajectory idea is genuinely useful and the molecular results hold up, but the 42-to-89 protein claim is mostly an alignment-to-oracle artifact that the authors half-concede in Section 5. the 4 major comments →
STRIDE: Post-Training LLMs to Reason and Refine Bio-Sequences via Edit Trajectories
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
STRIDE claims that a language model can solve constrained bio-sequence optimization by reasoning in edit space rather than sequence space. The model emits a trace: a ⟨think⟩ block listing atomic edits INSERT/DELETE/REPLACE with positions re-read against the current sequence after each step, followed by the final sequence. Stage I supervises this trace with deterministic shortest-edit scripts from Levenshtein backtracing; Stage II applies group-based policy optimization with a parse-and-execute gate — scripts that fail to reproduce the declared output get zero reward — plus KL regularization to the SFT policy. On TAPE fluorescence under the full action space the authors report 89/100 success
What carries the argument
The central object is the dynamic-indexed executable edit trajectory: a script of INSERT/DELETE/REPLACE actions where every position is a 0-based index into the sequence as it exists at that moment, so the script stays unambiguous as insertions and deletions change lengths. The trajectory is produced deterministically by Levenshtein DP backtracing with fixed tie-breaking, replayed forward to re-derive current-state positions, and serialized as a ⟨think⟩ trace followed by the target sequence. Stage I SFT on these shortest-edit scripts instills a minimal-edit, validity-preserving prior; Stage II applies group-based policy optimization (GRPO, GSPO, CISPO), and a parse-and-execute gate — script
Load-bearing premise
The load-bearing premise is that the fluorescence predictor behind the headline protein numbers — a model trained only on substitution variants — scores insertion- and deletion-containing sequences reliably, because that same predictor writes the synthetic indel training labels, supplies the Stage-II reward, and defines the success metric; the authors themselves note it may score indels less reliably.
What would settle it
Measure STRIDE's top full-action candidates — e.g., 50–100 of the 78 unique improved variants — with an indel-aware fluorescence predictor or a wet-lab GFP assay. If real success and novelty fall well below the oracle-scored 89% and 97% while the replace-only results survive, the central claim collapses; a cheaper computational check is retraining the reward/eval oracle on indel-containing variants and seeing whether the full-action gain persists.
If this is right
- Explicit edit trajectories matter most where the edit space is variable-length: the full-action gain over vanilla SFT (42→89 success) is far larger than the replace-only gain (55→61), so the trajectory interface is what unlocks combinatorial editing.
- Trajectory supervision alone does not buy controllability — STRIDE-SFT scores below direct SFT on molecular strict success (0.579 vs 0.629) — so the subsequent reward-alignment stage carries the optimization gains.
- Policy optimization on explicit scripts gives a better stability profile than policy optimization on direct sequences: STRIDE-GSPO cuts off-target shift rate to 0.755 versus 0.865 for Vanilla GSPO at comparable success, indicating scripts are a better substrate for alignment.
- Index-bookkeeping places a scale requirement on the backbone: roughly 80% per-action index compliance at 14B versus below 60% at 4B, a gap the authors tie directly to the full-action gains.
- If the oracle's indel scores are trustworthy, the full-action results indicate variable-length edits open a much larger and higher-novelty improvement space than substitution-only search, strengthening the case for edit-aware fitness predictors.
Where Pith is reading between the lines
- The consistency gate is the transferable idea: any domain where an LLM's intermediate plan can be executed and checked against a declared output inexpensively — genome editing, retrosynthesis, hardware netlists — could reuse this recipe.
- The paper's own oracle caveat generates a testable prediction: feed the same pipeline an indel-aware fluorescence model or a wet-lab assay and see whether the 42→89% full-action gap shrinks; if it persists, the gain reflects real edit-space biology, and if it collapses, STRIDE was partly aligning to evaluator idiosyncrasies.
- The sub-60% index compliance of the 4B model suggests a cheap extension the paper does not explore: externally maintaining the current sequence's index map in the prompt after each step (tool-style state tracking) could rescue smaller backbones without scaling.
- The 14B-versus-4B index-compliance claim in Section 5 is stated without the supporting table or statistics, so the scale-dependence reading currently rests on an assertion the paper does not display.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes STRIDE, a two-stage post-training framework that trains an LLM to emit executable INSERT/DELETE/REPLACE edit trajectories for discrete biological sequence optimization. Stage I uses Levenshtein backtracing to generate shortest-edit-path demonstrations for supervised fine-tuning; Stage II uses GRPO/GSPO/CISPO to align trajectory generation with task rewards, with a parse-and-execute consistency check. The central claims are (i) on an oracle-based full-action protein fluorescence stress test, STRIDE raises success from 42% to 89% and novelty from 47% to 97% over Vanilla SFT, and (ii) on instruction-conditioned molecular editing, the GSPO-aligned variant improves strict success, validity, and off-target controllability relative to the SFT-only STRIDE model. The paper also reports gains in uniqueness/novelty over discrete diffusion and flow baselines. Section 5 acknowledges that the fluorescence oracle was trained in a replace-only regime and that indel-containing candidates may be scored less reliably.
Significance. If the central results hold, STRIDE would be a valuable contribution: it provides an interpretable, controllable discrete-edit interface for LLM-based sequence optimization, and the deterministic Levenshtein backtracing plus parse-and-execute consistency check are principled safeguards. The open code release is a further strength. However, the headline protein claim is scored by the same learned oracle used for pseudo-labeling and reward, and the full-action improvement is concentrated in the indel regime that the oracle itself is admitted to score unreliably. The molecular experiments provide a useful counterweight, but they do not independently validate the protein headline. The significance therefore depends on whether the authors can establish that the protein gains reflect real fitness improvement rather than reward hacking of a biased oracle.
major comments (4)
- [§3.3, §4.1.1, §4.3] The headline 42→89 claim (Table 3) is a closed evaluation loop: the SaProtHub fluorescence oracle ffl is used to filter synthetic indel pseudo-labels (§4.1.1), to compute the Stage-II reward R_protein (§3.3), and to define Success in evaluation (§4.3). Section 5 concedes that this oracle was trained in a replace-only regime and that indel-containing candidates may be scored less reliably. The replace-only comparison in Table 4 shows only a modest STRIDE advantage (61/100 vs. 55/100), whereas the full-action gain is almost entirely carried by insert/delete edits that the oracle is admitted to score unreliably. This suggests the improvement may reflect aligning the model to the oracle's systematic opinion about indels rather than genuine fluorescence gains. Please (i) explicitly reframe full-action results as exploratory, (ii) add an independent evaluation using a different fluorescence pr
- [§4.3, Tables 3–5] All protein results are reported as point estimates from N=100 candidates generated from a single source sequence, with no error bars, confidence intervals, or multiple random seeds. Given the small margins in the replace-only comparison (61 vs. 55) and the fact that the full-action comparison uses the same oracle for training and evaluation, the reader cannot assess whether the differences are statistically reliable. Please report standard errors or bootstrap intervals, and ideally run the evaluation across multiple seeds and/or multiple source sequences.
- [§5] The claim that Qwen3-14B 'follows the specified indices in ~80% of actions' and that Qwen3-4B drops below 60% is load-bearing for the mechanism: if indices are not faithfully maintained, the improvement cannot be attributed to executable edit trajectories. No supporting table, per-action compliance rate, or parse-and-execute rejection rate is provided. Please report the actual compliance statistics (e.g., fraction of rollouts in which the emitted script executes to the emitted output), broken down by model size and action type, and show that the main results are not driven by rollouts that fail the consistency check.
- [§4.4.2, Tables 2 and 5] The claim that the GSPO-aligned variant improves strict success and controllability is only partially supported. While STRIDE-GSPO improves strict success over STRIDE-SFT (0.579→0.676), it is below both STRIDE-GRPO and STRIDE-CISPO (0.782 and 0.784) on the primary strict-success metric. Per-task, 'More like a drug' collapses from 0.114 to 0.006 strict success (Table 2), and several multi-objective tasks also degrade. The controllability benefit is real (Shift Rate 0.755 vs. 0.983 for SFT), but the manuscript should present GSPO as improving the success–stability trade-off rather than uniformly improving strict success, and should discuss the per-task failures.
minor comments (4)
- [Abstract vs. §1] The acronym expansion is inconsistent: the abstract says 'Sequence Trajectory Refinement via Iterative Discrete Editing' while the introduction says 'SequenceTrajectoryRefinement viaInternalizedDenoisingEmulation' and Figure 1 uses 'Trajectory-Based Atomic Editing'. Please unify.
- [References] There are typographical errors in the reference list, e.g., 'Lcvenshtcin' and 'V olkov' (also in the citation to Arnold & Volkov). Please correct these.
- [Table 2] The caption says '–' marks target properties, but the Table 2 note says 'Shift-related metrics are lower-is-better' while also marking target properties with '–'; please clarify the notation in the caption.
- [§3.1] The paragraph on 'Tokenization and structural prior' mentions 'Regex SMILES tokenizer' but no details or reference for the tokenizer are provided. A brief description or citation would improve reproducibility.
Circularity Check
Protein 'success' is the Stage-II reward itself; full-action gains are a closed oracle loop.
specific steps
-
fitted input called prediction
[Section 3.3 (Reward design) / Section 4.3 (Protein Evaluation Metrics)]
"Protein (fluorescence) optimization. We use a fixed fluorescence oracle ffl(·) to score sequences. ... Ifl = I[ffl(xout) > ffl(xsrc)]. The reward is R_protein = I_edit + I_fl. ... We define the improved set S+ = {xout | ffl(xout) > ffl(xsrc)}. We report: (i) Success: |S+|/N."
The reported protein 'Success' is exactly the improvement indicator I_fl used as the Stage-II reward, with the same oracle ffl and the same inequality. A policy trained to maximize R_protein will, by construction, register as successful on this metric. The headline full-action gain (42/100 → 89/100, Table 3) therefore measures alignment to the evaluator rather than an independent fluorescence prediction. Section 5 concedes the oracle is replace-only and scores indels 'less reliably,' so the largest gains occur precisely where the evaluator is admitted to be weak.
-
fitted input called prediction
[Section 4.1.1 (Synthetic indel augmentation)]
"we sample 1–3 random atomic edits A (INSERT/DELETE/REPLACE) applied to xsrc and assign pseudo labels using a fixed fluorescence predictor (SaProtHub/Model-Fluorescence-650M) (Su et al., 2024). We retain only pseudo-improved samples with Δyi > 0"
The same SaProtHub oracle that defines the Stage-II reward and the evaluation success metric is also used to generate and filter the synthetic indel SFT examples. STRIDE is therefore trained on oracle-produced labels, optimized against the oracle reward, and scored by the oracle — a triply closed loop for the full-action protein results. The pseudo-label filter (Δyi > 0) uses the same improvement inequality as the reward and the metric.
full rationale
The circularity is concentrated in the protein fluorescence claim. Section 3.3 defines the reward as I_fl = I[ffl(xout) > ffl(xsrc)], and Section 4.3 defines evaluation Success with the same oracle and the same inequality. Thus the headline 'success' is the training objective itself, not an independent measurement. Section 4.1.1 further uses the same oracle to create and filter synthetic indel pseudo-labels, so all three stages — SFT data, RL reward, and evaluation — invoke the same learned scorer. The paper's own Section 5 limitation states the oracle was 'trained in a replace-only regime' and that 'indel-containing candidates may be scored less reliably,' which explains why the full-action gap (42→89) is much larger than the replace-only gap (55→61): the model is rewarded for satisfying the oracle's opinion about indels while the oracle is admittedly unreliable there. The molecular results, scored by independent RDKit/property calculators, provide genuine comparative content and prevent the score from being higher. No load-bearing self-citation or imported uniqueness was found; the Levenshtein construction and GRPO/GSPO machinery are standard and independently executable. Overall, the central protein prediction reduces by construction to its own reward/evaluation loop, yielding a partial circularity score of 6.
Axiom & Free-Parameter Ledger
free parameters (6)
- Fluorescence oracle ffl (SaProtHub/Model-Fluorescence-650M) =
pretrained 650M-parameter model
- Edit budget indicator (1 ≤ d ≤ 3) =
1-3 atomic edits
- Strict-success thresholds τLogP=0.5, τQED=0.1, τTPSA=10 Ų, τHBA=τHBD=1 =
as listed in Section 4.3
- Molecular reward scales R_prop ∈ {0,0.5,1}, R_sim ∈ {0,0.5,1}, R_stable ≤0 =
discrete scales; off-target margins described only as "preset margins"
- Unit edit costs in Levenshtein DP =
1, 1, 1
- GRPO/GSPO hyperparameters =
G=8, ε=0.2, KL β=0.001; GSPO ε=3e-4, ε_high=4e-4; LoRA ranks 32/16
axioms (6)
- domain assumption Levenshtein shortest-edit scripts provide transferable process supervision for goal-directed optimization
- domain assumption The fluorescence oracle ffl is an adequate reward/evaluation signal for full-action indel-containing proteins
- domain assumption Qwen3-14B can reliably track dynamic 0-based indices across multi-step edit trajectories
- standard math RDKit parsing/sanitization and property calculators are the ground truth for SMILES validity and molecular properties
- domain assumption Intermediate strings along shortest edit paths need not be chemically or structurally valid; only the final output is evaluated
- domain assumption TAPE train/val splits and MEGA/DrugAssist positive splits are representative and non-leaking
read the original abstract
Discrete biological sequence optimization often requires goal-directed, parser-valid edits to an existing protein or molecule. Diffusion models support iterative refinement but do not expose a controllable discrete-edit interface, while autoregressive LLMs can be myopic when planning constrained edits over multiple steps. We introduce STRIDE (Sequence Trajectory Refinement via Iterative Discrete Editing), a post-training framework that trains an LLM to emit executable INSERT/DELETE/REPLACE trajectories for variable-length refinement. STRIDE first learns Levenshtein-aligned shortest-edit demonstrations, then uses supervised fine-tuning and group-based policy optimization to align trajectories with task rewards while preserving coherent editing. On an oracle-based full-action protein stress test, STRIDE raises success over Vanilla SFT from 42% to 89% and novelty among unique improvements from 47% to 97%. On instruction-conditioned molecular editing, the GSPO-aligned variant improves strict success, controllability, and SMILES validity over the SFT-only STRIDE model (code: https://github.com/daiheng-zhang/STRIDE).
Figures
Reference graph
Works this paper leans on
-
[1]
X., Fusi, N., Amini, A
Alamdari, S., Thakkar, N., Van Den Berg, R., Tenenholtz, N., Strome, B., Moses, A., Lu, A. X., Fusi, N., Amini, A. P., and Yang, K. K. Protein generation with evolu- tionary diffusion: sequence is all you need.BioRxiv, pp. 2023–09,
2023
-
[2]
•Offload:None •Contiguous Gradients:Enabled •Gradient Clipping:Auto D. Off-Target Property Drifts Table 7 provides a per-property breakdown of off-target drifts across the 14 DrugAssist optimization tasks, complementing the aggregate shift metrics reported in Section 4.4.2. We report shift rates for five molecular properties—HBA, HBD, logP, QED, and TPSA—...
arXiv 2070
-
[6]
A., Bastings, J., Poole, B., Berg, R
Hoogeboom, E., Gritsenko, A. A., Bastings, J., Poole, B., Berg, R. v. d., and Salimans, T. Autoregressive diffusion models.arXiv preprint arXiv:2110.02037,
-
[7]
Any-order flexible length masked diffusion.arXiv preprint arXiv:2509.01025,
Kim, J., Cheuk-Kit, L., Domingo-Enrich, C., Du, Y ., Kakade, S., Ngotiaoco, T., Chen, S., and Albergo, M. Any-order flexible length masked diffusion.arXiv preprint arXiv:2509.01025,
-
[11]
P., Kumar, A., Er- mon, S., and Poole, B
Song, Y ., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456,
Pith/arXiv arXiv 2011
-
[12]
Saprothub: Making protein modeling accessible to all biologists.BioRxiv, pp
Su, J., Li, Z., Han, C., Zhou, Y ., He, Y ., Shan, J., Zhou, X., Chang, X., Jiang, S., Ma, D., et al. Saprothub: Making protein modeling accessible to all biologists.BioRxiv, pp. 2024–05,
2024
-
[13]
Wang, R., Zhang, H., Nguyen, T., Feng, S., Pang, H.-W., Yu, X., Xiao, L., and Zhang, P. Z. Pepthink-r1: Llm for interpretable cyclic peptide optimization with cot sft and reinforcement learning.arXiv preprint arXiv:2508.14765,
-
[14]
Diffusion language models are versatile protein learners
Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., and Gu, Q. Diffusion language models are versatile protein learners. arXiv preprint arXiv:2402.18567,
-
[15]
Group sequence policy optimization.arXiv preprint arXiv:2507.18071,
Zheng, C., Liu, S., Li, M., Chen, X.-H., Yu, B., Gao, C., Dang, K., Liu, Y ., Men, R., Yang, A., et al. Group sequence policy optimization.arXiv preprint arXiv:2507.18071,
-
[16]
as a non-autoregressive discrete-flow baseline for variable-length protein sequence generation and editing. Unlike fixed-length masked discrete diffusion, Edit Flows defines a Continuous-Time Markov Chain (CTMC) directly over the space of sequences and parameterizes its generator viaedit operations(IN- SERT/DELETE/REPLACE), naturally supporting length-cha...
2025
-
[1966]
Mol-r1: Towards explicit long-cot reasoning in molecule discovery.arXiv preprint arXiv:2508.08401,
Li, J., Wang, W., Zhang, Q., Li, J., Zhang, D., Zheng, C., Zhang, S., Wei, X., and Li, Q. Mol-r1: Towards explicit long-cot reasoning in molecule discovery.arXiv preprint arXiv:2508.08401,
-
[2016]
Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y ., Wu, Y ., et al. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,
-
[2019]
Havasi, M., Karrer, B., Gat, I., and Chen, R. T. Edit flows: Flow matching with edit operations.arXiv preprint arXiv:2506.09018,
-
[2020]
Chen, A., Li, A., Gong, B., Jiang, B., Fei, B., Yang, B., Shan, B., Yu, C., Wang, C., Zhu, C., et al. Minimax- m1: Scaling test-time compute efficiently with lightning attention.arXiv preprint arXiv:2506.13585,
-
[2021]
D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901,
1901
- [2024]
-
[2025]
Crafting papers on machine learning
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.),Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207–1216, Stan- ford, CA,
2000
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.