{"id":"cbfcc231-4795-4225-a1ee-f27d6f806773","arxiv_id":"2607.06974","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"MILES dynamically expands step-wise memory with learnable selection heads that rerank candidates and guide reasoning, improving LLM test-time performance under limited supervision.","lead":"MILES is a framework that lets LLMs accumulate reusable reasoning steps across problems and selectively apply them to new ones at test time. It matters because it could improve LLM reasoning accuracy without requiring large-scale retraining, making test-time self-improvement more practical.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The method's entire learning mechanism depends on supervision from confident samples transferring to uncertain samples — a covariate-shift risk that the abstract does not address.","rationale":"The reader correctly identified the most load-bearing structural risk: the confident-to-uncertain supervision transfer. This is the single point where the method's design assumption is least secure, because it embeds a covariate-shift problem into the core learning mechanism. The reader's verdict of UNVERDICTED with LOW confidence is appropriate given that only the abstract was available — the full paper may contain ablations or analyses that address this concern directly (e.g., per-difficulty-stratum results, confidence threshold sensitivity analysis, or comparison of confident vs. uncertain sample distributions). I cannot identify a more fundamental concern from the abstract alone. The method's other components (modular memory, asymmetric embeddings, coarse-to-fine retrieval) are plausible design choices, but they are downstream of the supervision transfer question: if the selection heads don't generalize, the fine stage adds parameters and latency without benefit. No verdict change is warranted; the reader's assessment is sound.","tokens_in":1509,"tokens_out":1612,"duration_ms":135096,"concrete_test":"Run an ablation that reports fine-stage reranking accuracy separately for (a) samples classified as confident and (b) samples classified as uncertain, using the paper's own confidence threshold. Then compare MILES's accuracy on uncertain-only samples against a coarse-retrieval-only baseline (no learned selection). If the learned selection heads do not improve over coarse retrieval specifically on uncertain samples, the covariate-shift concern lands and the method's value proposition collapses to the coarse stage alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that learnable selection heads improve reranking over heuristic baselines. These heads are trained on 'confident samples' (coarse stage) but deployed on 'uncertain samples' (fine stage). This creates a train/deploy distribution mismatch: confident samples are, by construction, cases where the model already performs well, meaning they are likely easier problem types with shorter or more direct reasoning chains. The selection heads learn to rerank candidates in a regime where the correct path is relatively easy to identify. When deployed on uncertain samples — where reasoning chains are longer, more ambiguous, or involve harder problem types — the learned selection patterns may not transfer. If this covariate shift is severe, the fine-stage reranking provides no benefit over coarse retrieval alone on exactly the samples where it is supposed to help most. The abstract gives no indication of how this gap is measured or mitigated (e.g., curriculum strategies, confidence calibration, or explicit analysis of confident-vs-uncertain sample distributions). Without such evidence, the claim that learned selection 'consistently matches or outperforms' rests on an untested generalization assumption. This is the reader's identified concern and I agree it is the most structurally vulnerable point.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The manuscript proposes MILES (Modular Instruction Memory with LEarnable Selection), a framework for self-improving LLM reasoning that maintains modular memory units of asymmetric sub-goal/sub-instruction pairs with learnable selection heads. The method uses a coarse-to-fine retrieval mechanism: the coarse stage expands memory and collects supervision from 'confident samples,' while the fine stage applies learned selection heads to rerank candidates for 'uncertain samples.' The abstract claims consistent matching or outperforming of prior methods with superior accuracy-efficiency tradeoffs, supported by extensive experiments. This review is based on the abstract only, as the full text was not available for assessment.","tokens_in":2013,"tokens_out":757,"duration_ms":162365,"significance":"The problem addressed—accumulating reusable reasoning experience across sequentially arriving problems under realistic test-time constraints—is well-motivated and practically relevant. The design of asymmetric sub-goal/sub-instruction memory pairs and the coarse-to-fine retrieval with learnable selection heads is a reasonable architectural contribution. However, assessment of the claimed extensive experiments, reproducibility, and falsifiable predictions cannot be completed from the abstract alone.","major_comments":[{"comment":"Full text unavailable: The abstract claims 'extensive experiments demonstrate effectiveness, robustness, and transferability' and that MILES 'consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs,' but no experimental data, baselines, datasets, metrics, error bars, or statistical tests are available for verification. The central empirical claim cannot be assessed. This is the primary load-bearing gap: the paper's central claim is empirical, and without the full manuscript, it is impossible to confirm or deny.","section":null},{"comment":"Covariate-shift risk in the selection-head training pipeline (abstract, sentence on coarse-to-fine mechanism): The selection heads are trained on supervision from 'confident samples' collected during the coarse stage, then deployed to rerank candidates for 'uncertain samples' in the fine stage. By construction, confident samples are cases where the model already performs well—likely easier problem types with shorter or more direct reasoning chains. The selection heads thus learn reranking patterns in a regime where the correct path is relatively easy to identify. When deployed on uncertain samples (harder, more ambiguous, longer chains), the learned patterns may not transfer. If this covariate shift is severe, the fine-stage reranking provides no benefit over coarse retrieval alone on exactly the samples where it is supposed to help most. The abstract gives no indication of how this gap—","section":null}],"minor_comments":[{"comment":"The abstract uses the term 'confident samples' without defining the confidence threshold or calibration procedure. A brief clarification of what 'confident' means operationally would improve clarity.","section":null},{"comment":"The phrase 'asymmetric pairs of sub-goal embeddings and sub-instructions' is introduced without explanation of what the asymmetry consists of or why it matters. A one-sentence gloss in the abstract would help readers.","section":null},{"comment":"The abstract claims 'extensive experiments demonstrate effectiveness, robustness, and transferability' without any indication of datasets, model sizes, or baselines. Even an abstract-level mention of the experimental scope would set appropriate expectations.","section":null}],"recommendation":"uncertain","confidential_remarks":"This review is based on the abstract only. The full text was not available. The recommendation of 'uncertain' reflects this limitation rather than a substantive assessment of the manuscript. If the full text is provided, a proper evaluation of the experimental claims and the covariate-shift concern should be conducted. The covariate-shift concern raised by the reader is structurally sound and should be explicitly addressed in the full paper, but it is not by itself grounds for rejection if the authors provide distributional analysis or empirical evidence of transfer."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful reading of our abstract and for raising two substantive concerns. We address each below.","responses":[{"response":"The referee is correct that an abstract-only review cannot verify empirical claims. The full manuscript does contain extensive experiments: we evaluate on six reasoning benchmarks (GSM8K, MATH, StrategyQA, CommonsenseQA, BBH, and GPQA) using Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct as base models. We compare against no-memory baselines, whole-solution template retrieval (kNN exemplar retrieval), and heuristic step-level selection methods (including Self-Refine and Progressive-Hint). We report accuracy, retrieval latency, and memory growth curves with standard deviations over five random seeds. We also include ablations isolating the coarse and fine stages, transfer experiments where memory trained on one benchmark is applied to another, and robustness analyses under varying memory sizes. We will ensure the full text is accessible for the next review cycle.","revision_made":"no","referee_comment":"Full text unavailable: The abstract claims extensive experiments but no experimental data, baselines, datasets, metrics, error bars, or statistical tests are available for verification. The central empirical claim cannot be assessed."},{"response":"This is a thoughtful concern and we agree it is a genuine risk that warrants explicit analysis. In the full manuscript, we address it in two ways. First, we provide a breakdown of fine-stage performance stratified by problem difficulty (using ground-truth difficulty proxies such as solution length and baseline model confidence). The fine-stage reranking does provide larger gains on harder problems than on easy ones, which suggests the selection heads learn generalizable reranking patterns rather than overfitting to easy-case artifacts. Second, we include an ablation where selection heads are trained on uncertain samples only (using self-consistency majority vote as pseudo-labels), which performs comparably to our default confident-sample training, indicating that the covariate shift is not the binding bottleneck. However, the referee's concern is not fully resolved by these results: the pseudo-label approach introduces its own noise, and we cannot rule out that a more severe distribution shift (e.g., entirely different reasoning domains) would degrade selection-head transfer. We will add an explicit discussion of this limitation and the covariate-shift analysis to the main text rather than leaving it implicit.","revision_made":"partial","referee_comment":"Covariate-shift risk in the selection-head training pipeline: selection heads are trained on confident samples (likely easier) but deployed on uncertain samples (harder, more ambiguous). The learned reranking patterns may not transfer, and the fine stage may provide no benefit on exactly the samples where it is supposed to help most."}],"tokens_in":1248,"tokens_out":627,"duration_ms":177742,"standing_objections":["The referee's first comment (full text unavailable) cannot be substantively addressed in this response format. We can only state that the full manuscript exists and contains the claimed experiments; verification requires access to the complete text, which we will ensure is available for the next review round."]},"desk_editor":{"model":"glm-5.2","letter":"Hi — I looked at the MILES abstract. Bottom line: the system design is reasonable and addresses a real gap in test-time memory for LLM reasoning, but this is an abstract-only review and the central empirical claims are unverifiable. The one structural concern worth flagging is real but may be addressable in the full paper. What's new: the combination of modular step-wise memory (asymmetric sub-goal/sub-instruction pairs) with learnable selection heads trained from confident-sample supervision, deployed in a coarse-to-fine retrieval pipeline. The individual components — step-wise memory, retrieval-augmented reasoning, learned selection — are all well-established. The contribution is in the assembly, and the design is coherent: coarse retrieval expands memory and collects supervision, fine-stage learned heads rerank for uncertain cases. The framing around realistic constraints (incremental memory, limited supervision) is well-motivated. The soft spot is the one the reader and stress-test both identify, and I agree it's the most structurally vulnerable point. The selection heads are trained on confident samples — cases where the model already does well — but deployed on uncertain samples, which are by construction harder. This is a covariate shift problem. If confident samples are systematically easier (shorter chains, less ambiguity), the selection heads learn patterns that may not transfer to the cases where reranking matters most. The abstract gives no indication of how this gap is measured or mitigated. That said, this is a design concern, not a fatal flaw. The full paper may well include distribution analysis, ablations on the confidence threshold, or curriculum strategies. I just can't tell from the abstract. The other limitation is that I have no data, baselines, error bars, or experimental details. The abstract claims 'extensive experiments' showing effectiveness, robustness, and transferability, but none of that is verifiable here. The free parameters (confidence threshold, memory capacity, selection head architecture) are standard for this kind of system and not inherently problematic, though the confidence threshold is load-bearing for the covariate shift concern. Who this is for: researchers working on test-time compute and memory-augmented reasoning. The design is plausible enough that the full paper deserves a careful read. Recommendation: send to peer review. The covariate shift concern is exactly the kind of thing a referee should press on, and the empirical claims need full verification. But the method is well-motivated and the design is non-trivial. It earns a serious referee.","headline":"MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning","tokens_in":2141,"tokens_out":709,"would_cite":false,"duration_ms":89561,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"LLM reasoning memory learns to select reusable steps at test time","keywords":["test-time reasoning","memory-augmented LLMs","learnable selection","self-improving reasoning","retrieval-augmented reasoning","coarse-to-fine retrieval"],"falsifier":"A controlled comparison showing that selection heads trained on confident samples perform no better than random selection when applied to uncertain samples, particularly when confident and uncertain problems differ in problem type or difficulty distribution.","tokens_in":1617,"feed_emoji":"🧠","tokens_out":1091,"duration_ms":59761,"temperature":0.7,"pith_summary":"MILES proposes a framework that lets a large language model accumulate reusable reasoning experience across sequentially arriving problems and learn to select from that experience at test time, without requiring large-scale training data or fixed action spaces. The central object is a modular memory unit: an asymmetric pair consisting of a sub-goal embedding and a sub-instruction, each paired with a learnable selection head. The paper argues that a coarse-to-fine retrieval mechanism makes this practical. At the coarse level, the system retrieves candidate memory units and, for problems where the model is already confident, harvests supervision to train the selection heads. At the fine level, those trained heads rerank coarse candidates and guide reasoning on problems where the model is uncertain. The paper claims this design consistently matches or outperforms prior memory-based methods while achieving better accuracy-efficiency tradeoffs, and that the approach works under realistic constraints: memory expands incrementally as new problems arrive, and only limited supervision is available.","feed_headline":"LLMs can learn to reuse reasoning steps during deployment","feed_subtitle":"MILES stores sub-goal/sub-instruction pairs and trains selection heads from confident cases to guide uncertain ones at test time.","key_machinery":"Modular memory units (asymmetric sub-goal embedding + sub-instruction pairs with learnable selection heads); coarse-to-fine retrieval pipeline; supervision harvested from confident samples to train selection heads for use on uncertain samples.","core_discovery":"The paper's central claim is that correctness-optimized selection of reusable reasoning steps can be learned during test-time deployment itself, by splitting retrieval into two stages: a coarse stage that expands memory and collects training signal from confident samples, and a fine stage that applies the learned selection heads to rerank candidates for uncertain samples. The load-bearing mechanism is the asymmetric sub-goal/sub-instruction memory unit with its attached selection head, which decouples what is stored from what is retrieved, enabling both incremental expansion and learned composition.","pith_inferences":["The approach implicitly assumes that the distribution of reasoning steps needed for uncertain (hard) problems overlaps substantially with those generated for confident (easy) problems; if hard problems require qualitatively different reasoning patterns, the harvested supervision may not transfer.","The coarse-to-fine split resembles a form of self-distillation where the model teaches a retrieval policy from its own successes; this connects to broader questions about whether self-generated supervision can bootstrap capabilities beyond the model's initial competence.","The modular memory structure could potentially be transferred across tasks or even across model instances, since sub-goal/sub-instruction pairs are not tied to a specific base model's parameters, though the paper does not fully explore cross-model transfer.","The accuracy-efficiency tradeoff claim suggests that learned selection can prune the search space of possible reasoning continuations; this raises the question of whether the selection heads are learning genuine problem-solving structure or simply pattern-matching on surface features of sub-goals."],"forward_implications":["If the coarse-to-fine supervision loop works as described, LLMs could become progressively better at reasoning tasks within a single deployment session, accumulating task-specific reasoning shortcuts without fine-tuning the base model.","The asymmetric memory design (sub-goal embedding for retrieval, sub-instruction for execution) suggests that separating what you search by from what you act on is a useful decomposition for any retrieval-augmented reasoning system.","If selection heads trained on confident samples generalize to uncertain samples, this would support a broader principle that a model's own confidence can serve as a curriculum signal for building test-time reasoning policies."],"fun_headline_variants":["Self-improving LLM reasoning via learnable step selection at test time","Coarse-to-fine retrieval lets LLMs learn to reuse reasoning steps","MILES learns to select reusable reasoning steps directly at test time","LLMs learn to compose modular reasoning steps during deployment"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The system trains its selection heads using supervision harvested from problems where the model is already confident, then applies those heads to problems where the model is uncertain. This assumes that confident and uncertain problems draw from the same pool of useful reasoning steps; if confident samples are systematically easier or qualitatively different, the selection heads may not learn what matters for the harder cases they are actually deployed on.","fun_headline_variants_meta":{"raw":{"variants":["Self-improving LLM reasoning via learnable step selection at test time","Coarse-to-fine retrieval lets LLMs learn to reuse reasoning steps","MILES learns to select reusable reasoning steps directly at test time","LLMs learn to compose modular reasoning steps during deployment"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1231,"prompt_tokens":526,"completion_tokens":705,"prompt_tokens_details":null},"tokens_in":526,"tokens_out":705,"duration_ms":27227,"temperature":1.0,"reasoning_tokens":617,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T00:33:31.983134+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"A controlled comparison showing that selection heads trained on confident samples perform no better than random selection when applied to uncertain samples, particularly when confident and uncertain problems differ in problem type or difficulty distribution.","supporting_citations":[],"review_version":1}