{"id":"9f2cbbf2-f1c1-4532-9d92-2e2d08e27ee5","arxiv_id":"2604.00830","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Evolutionary search over meta-prompt text learns a test-time adaptation policy that outperforms hand-written reflection prompts on Jericho and WebArena-Lite and transfers, in reduced form, to held-out games and web domains.","lead":"This paper makes the rule for improving across repeated attempts - the adaptation policy that tells a language agent how to use its past failures - into something that is learned, by evolving a natural-language instruction on training tasks. The learned instruction beats hand-written reflection prompts on interactive-fiction games and transfers to held-out games and web tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Jericho ID results are selected on the same games used for evaluation: Algorithm 2 picks ϕ* on Dval and Appendix A applies post-hoc z-score selection, while §4.1 reports ID on those same three games—so the headline 0.18→0.41 gain may be a validation artifact.","rationale":"The central claim is that the meta-learned adaptation policy ϕ* is genuinely better than hand-crafted baselines and generalizes. The strongest quantitative evidence is the Jericho ID table (GPT-5: W-AUC 0.18→0.41). But as reported, that comparison is not a clean ID test: Jericho has no separate evaluation split, Algorithm 2 selects ϕ* using W-AUC on Dval, and Appendix A describes a post-hoc z-score selection rule. With only three games and 26 iterations, selection on validation can inflate the chosen expert's score substantially. This is the least secure link in the argument because if the ID gains are selection artifacts, the paper lacks any clean evidence for RQ1, and the OOD gains—while independent—are modest and partially explained by task overlap (§4.3). The reader's weakest assumption was the OOD shift, which is also valid, but the ID leakage is more fundamental: it contaminates the paper's headline result. The concrete fix is to re-run Jericho with a game-level ID holdout. This is a fixable flaw, so I do not move the verdict from the reader's CONDITIONAL; the paper needs this re-analysis before acceptance. Credit is due for the WebArena-Lite train/val/test split and the directionally consistent OOD trends, and the code release is a positive, but the Jericho ID contamination blocks ACCEPT as written.","tokens_in":16904,"tokens_out":7085,"duration_ms":71764,"concrete_test":"Rerun Jericho ID with a task-level split that excludes the validation games from test: e.g., meta-train/validate on Detective and Zork 1, test on Temple (or rotate the held-out ID game and average), and recompute the Table 1 META-TTL vs Naive W-AUC gaps. If the GPT-5 0.41 value collapses toward the Naive baseline when the test game was not used for expert selection, the reported ID gains are a selection artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 says the three Jericho ID games (Detective, Zork 1, Temple) are used \"for meta-training and ID evaluation,\" with no separate ID holdout, while Algorithm 2 selects the deployed ϕ* by maximizing W-AUC over Dval and Appendix A admits the selection rule was a \"post-hoc per-game z-score normalization\" applied after inspecting candidates. The headline ID numbers (e.g., GPT-5 W-AUC 0.18→0.41) therefore measure validation performance on the same task identities used to pick the expert, not an unbiased estimate of ID generalization. Selection on a three-game validation set with 26 optimization iterations can easily inflate W-AUC, so the \"consistently outperforms hand-crafted baselines\" claim (RQ1) is not established by the reported Jericho tables. WebArena-Lite ID is split into train/val/test and is cleaner, but the paper's flagship quantitative result is the contaminated Jericho ID comparison. The OOD results are not subject to this specific leakage, but they are modest and, as §4.3 concedes, concentrated in domains that share structure with training. A clean ID split is thus the minimal condition for the central claim to stand.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Meta-TTL, a bi-level optimization framework for learning a natural-language 'adaptation policy' (a meta-prompt) that governs how a meta-agent rewrites an actor LLM's system prompt between episodes at test time. The inner loop is the standard TTL process; the outer loop uses an LLM proposer and an expert pool to evolve candidate meta-prompts on a training/validation task distribution. The learned meta-prompt is frozen at test time and applied zero-shot. The paper evaluates on Jericho and WebArena-Lite, reporting that Meta-TTL consistently improves W-AUC and average scores over static, Reflexion, Memory Agent, and unoptimized naive meta-agent baselines on both in-distribution (ID) and out-of-distribution (OOD) tasks, with headline gains on Jericho ID (GPT-5: W-AUC 0.18→0.41, avg score 50.4→110.8) and WebArena-Lite ID (up to +0.09 W-AUC). The paper also analyzes emergent properties of the optimized prompts, such as structured output, credit assignment, and conditional fact banks, and includes case studies in appendices.","tokens_in":17153,"tokens_out":3327,"duration_ms":33082,"significance":"If the central claim holds—that effective test-time adaptation is itself learnable and yields transferable strategies—this is a meaningful contribution to the growing literature on test-time learning for LLM agents. The framing of adaptation-policy optimization as bi-level meta-learning is clear and well placed relative to prior work. The paper ships code and provides unusually detailed appendices with full meta-prompts and optimization trajectories, which are valuable for reproducibility and for understanding the mechanism. The OOD evaluation does at least use a frozen phi*, so the OOD results are genuine predictive tests rather than re-fitted on held-out tasks. However, the current evidence has a load-bearing flaw in the Jericho ID evaluation (same games used for selection and reporting), and the OOD setting is weakened by substantial shared structure between 'held-out' and training games. The lack of any variance or uncertainty estimates further limits the strength of the quantitative claims.","major_comments":[{"comment":"The Jericho ID evaluation is not a clean test of RQ1. The same three games (Detective, Zork 1, Temple) are used for meta-training, for the global validation that updates the expert pool (Algorithm 2, lines 10–16), and for the ID evaluation reported in Table 1. Appendix A explicitly describes a post-hoc per-game z-score selection rule applied to candidates evaluated on these same games. The headline Jericho ID gains (e.g., GPT-5 W-AUC 0.18→0.41) therefore measure validation performance on the same task identities used to select phi*, not unbiased ID generalization. With only 26 optimization iterations and a 3-game validation set, selection can substantially inflate the apparent gain. The WebArena-Lite ID evaluation is cleaner (train/val/test split within ID domains), but the paper's flagship quantitative claim rests on the contaminated Jericho result. A clean ID holdout for Jericho, or a","section":"§4.1, Algorithm 2, Appendix A, Table 1"},{"comment":"No variance measures, seeds, or confidence intervals are reported anywhere. Every table is a single point estimate, and several deltas are small (e.g., WebArena-Lite OOD differences of +0.01–0.05 W-AUC on binary-reward tasks), which may be within run-to-run noise. The paper should report multiple seeds or bootstrap confidence intervals for at least the main comparisons; otherwise the 'consistently outperforms' claim (RQ1) is not quantitatively supported.","section":"Tables 1–4 (all experiments)"},{"comment":"The OOD generalization claim is weakened by the similarity between held-out and training tasks. §4.3 concedes that Shopping Admin shares interface and task structure with the ID Shopping domain, and the OOD gains on WebArena-Lite concentrate precisely there. On Jericho, Appendix E.3 shows that the optimized meta-agent, on the OOD game Balances, recommends 'LEARN REZROV', a spell that appears in the Zork I fact bank from the training set. This indicates that parser verbs, spell names, and game syntax are shared across the 'held-out' titles, so title-level holdout does not establish genuine distribution shift. The conclusion that phi* encodes a general adaptation procedure would be considerably stronger with tasks that do not share vocabulary, parser syntax, or UI structures with training, or with a quantitative analysis of shared components.","section":"§4.3, Appendix E.3"},{"comment":"The outer loop depends on the proposer LLM (PROPOSE, line 4) to generate candidate meta-prompts, and the paper never ablates this component. All improvements could be attributed to the proposer's ability to write better prompts from a single session rather than to the evolutionary search mechanism (expert pool, local/global validation) itself. A control condition with random mutation of the meta-prompt (e.g., word-level perturbations with the same number of candidates and validation budget) or a comparison against a simple hill-climbing baseline would be needed to support the claim that the search process, rather than the proposer's prior knowledge, is responsible for the discovered strategies. Without this, the 'learned adaptation policy' framing is partially confounded with the capabilities of the LLM proposer.","section":"Algorithm 2, §3.3, §4.4"}],"minor_comments":[{"comment":"Typo: 'a adaptation policy' should be 'an adaptation policy'.","section":"§3, first sentence"},{"comment":"The table formatting has inconsistent bold/emphasis (e.g., some META-TTL rows are partially bolded, others not). Please make the highlighting uniform and define what bold means in the caption.","section":"Table 1 and Table 2"},{"comment":"The learning curves are single runs with no error bands; adding multiple seeds or shaded confidence intervals would make the 'more stable learning curves' claim (stated in §4.2) actually visible.","section":"Figure 3"},{"comment":"The z-score normalization is described as 'post-hoc' after inspecting candidates. This is a selection-rule choice made on the same validation games; please clarify whether the rule was fixed before seeing the results or chosen after, since this affects the validity of the selection procedure.","section":"Appendix A"},{"comment":"The two sample meta-prompts are useful, but the paper does not state which of the three backbone-specific phi* prompts are used in the main tables for each benchmark. Please clarify the mapping between Table 1/Table 3 and the Appendix B prompts.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The central idea is worthwhile and the paper has strong artifacts (code, prompts, detailed appendices). However, the Jericho ID evaluation is confounded by selection on the same task identities, and the OOD evaluation is weakened by shared structure. These are fixable within the manuscript's scope: add a clean ID holdout (or explicitly relabel the results as validation-selected), report variance, and ideally add a random-mutation baseline. If the authors can provide those, the paper could be a solid contribution. I do not see a fatal error that requires rejection, but the current form overclaims what is established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Briefly: this paper proposes Meta-TTL, a bi-level framework where an outer evolutionary loop searches over natural-language meta-prompts that govern how a meta-agent rewrites the actor's system prompt between episodes. That target—learning the adaptation policy rather than hand-crafting it—is genuinely new relative to Reflexion, EvoTest, and the memory-based TTL line. The framework is clearly specified, and the paper does something real: it runs an evolutionary search with local and global validation, keeps per-task experts, and deploys a frozen text policy zero-shot. The resulting prompt is interpretable and transfers across backbones without weight updates. Credit where due: the OOD evaluation is a genuine out-of-sample prediction—phi* is frozen and not refit on held-out games—and the direction of the OOD gains is consistent across all six Jericho games and all three backbones. That is not nothing.\n\nThe soft spots are real, though. The flagship ID numbers on Jericho are contaminated: Algorithm 2 selects the deployed phi* by maximizing W-AUC over Dval, and Appendix A admits the selection rule itself was chosen post hoc after inspecting candidate scores. Reporting ID evaluation on those same three games (Detective, Zork 1, Temple) means the headline 0.18→0.41 W-AUC gain measures validation performance, not an unbiased estimate of in-distribution generalization. WebArena-Lite has a cleaner train/val/test split, but the gains there are small (often +/−0.01–0.09 W-AUC on binary rewards) and reported without variance, seeds, or confidence intervals. The OOD claim is also weaker than stated: the held-out Jericho games share parser mechanics and even spell vocabulary with the training games (the Balances case study recommends LEARN REZROV, a spell from the Zork I fact bank), and on WebArena the gains concentrate in Shopping Admin, which the authors concede shares structure with ID Shopping. That does not refute the transfer claim, but it does mean the paper overstates how far phi* generalizes.\n\nTwo more minor issues: the abstract promises tau-bench and a prompt-optimization baseline that never appear in the body, and the emergence narrative in Section 4.4 would be more convincing with an ablation that separates the contribution of the seed prompt, proposer instructions, and W-AUC weighting from search-discovered content. None of these are fatal to the central idea. The fix is straightforward: report variance, use a true ID holdout, compare against a real prompt optimizer like OPRO or PromptBreeder, and report per-domain results as they are.\n\nBottom line: this is a serious piece of work with an interesting, well-framed claim and honest OOD effort. The evidence as reported does not yet support 'consistently outperforms hand-crafted baselines' at the strength the abstract suggests. It deserves a real referee—the idea is worth engaging, and the flaws are addressable in revision.","headline":"The core idea—learning the adaptation policy itself rather than hand-crafting it—is genuinely new and worth engaging, but the headline ID numbers are selected on the validation set, so the transfer claim rests on the OOD results, which are real but modest.","tokens_in":17761,"tokens_out":2208,"would_cite":true,"duration_ms":19349,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that how a language agent adapts from experience at test time is itself a learnable capability, and shows that an evolutionarily optimized adaptation prompt beats fixed hand-crafted reflection rules on interactive-fiction","keywords":["test-time learning","meta-learning","adaptation policy","language agents","evolutionary search","prompt optimization","out-of-distribution generalization"],"falsifier":"Run the evolved meta-prompt on a suite of interactive-fiction games that share no parser verbs, room names, or spells with the training games; if the W-AUC gain over the naive meta-agent vanishes, then the claimed generalization is an artifact of task similarity. Alternatively, compare outer-loop proposals against random mutations of equal edit distance—if gains persist, the proposer's reflective ability is not the source of improvement.","tokens_in":16653,"feed_emoji":"🧠","tokens_out":4372,"duration_ms":44107,"temperature":0.7,"pith_summary":"The paper tries to establish that the rule governing how a language agent improves across episodes—the adaptation policy—should be learned from a distribution of tasks rather than designed by hand. It formulates this as a bi-level optimization: an inner loop runs test-time learning episodes, and an outer loop evolves a natural-language meta-prompt that tells a meta-agent how to rewrite the actor's system prompt after each episode. On interactive-fiction and web-navigation benchmarks, the learned meta-prompt outperforms fixed hand-crafted reflection baselines and also improves performance on held-out tasks. If correct, this turns test-time learning itself into an optimizable component of an agent, producing transferable, interpretable adaptation strategies in plain language.","feed_headline":"Evolved adaptation prompts beat hand-crafted self-reflection","feed_subtitle":"Learned by evolutionary search across tasks, the meta-prompt transfers to unseen games and web domains.","key_machinery":"The central object is the natural-language meta-prompt, which fully specifies the adaptation policy: after each episode it tells the meta-agent how to rewrite the actor's system prompt given the trajectory history. The bi-level loop—an inner test-time-learning loop running K episodes and an outer evolutionary search that proposes candidate prompts from rollouts, validates them on per-task experts, and selects a final expert—is the machinery that discovers the optimized meta-prompt. The emergent conditional fact banks and structured six-section output format are what carry the transfer to unseen tasks.","core_discovery":"The central discovery claim is that optimizing the adaptation policy—the meta-prompt governing how a meta-agent rewrites an actor's system prompt between episodes—yields consistently better test-time improvement than fixed hand-crafted policies. Concretely, on Jericho the evolved prompt raises average W-AUC from 0.18 with a naive meta-agent to 0.41 with the strongest tested backbone, and from 0.23 to 0.28 on held-out games; on WebArena-Lite the success-weighted metric rises from 0.55 to 0.63 in the in-distribution setting. The paper further claims that the optimized meta-prompt encodes transferable strategies—explicit credit assignment, grounded fact accumulation, structured output, and cond","pith_inferences":["An implication the paper leaves implicit is that the OOD claim would be strengthened by holding out a genuinely novel domain that shares no parser vocabulary, spell names, or interface structure with training tasks; the current OOD gains concentrate on tasks that resemble training ones.","A control that replaces the proposer's reflective proposals with random mutations of equal edit distance would isolate how much of the improvement comes from the meta-learning selection pressure versus the raw prompt-writing skill of the underlying LLM; the paper does not report such a control.","The conditional fact-bank mechanism resembles retrieval over stored knowledge; a testable extension would compare against an explicitly separate memory store that the meta-agent queries, which the paper does not pursue.","Because local validation only accepts candidates that beat the parent on the same task, the search may be biased toward task-specific overfitting; the paper's z-score expert selection mitigates this but does not eliminate it."],"forward_implications":["Test-time learning becomes a meta-optimization target: any TTL pipeline can be improved by optimizing its adaptation rule instead of hand-tuning reflection prompts.","Because the learned adaptation policy is a text artifact, it can be transferred to different actor models or backbones without retraining model weights.","The learned policies decompose into task-agnostic adaptation strategies plus conditional fact banks, suggesting that interpretable structure can emerge from evolutionary search.","The W-AUC objective and episode-restart setup make sustained improvement the evaluated quantity, so methods that merely stabilize but do not improve are penalized.","The out-of-distribution results imply that at least part of the learned adaptation procedure transfers to structurally similar unseen tasks; the paper attributes this to generalizable strategies."],"fun_headline_variants":["Evolutionary search finds better test-time learning policies","Meta-TTL: Learn adaptation policies, not hand-craft them","Learned adaptation policies transfer across unseen tasks","Optimizing adaptation policy via evolution boosts agents","Test-time learning: evolution beats hand-crafted policies"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper's out-of-distribution claim rests on the assumption that the held-out Jericho games and WebArena's Shopping Admin domain are genuinely out-of-distribution, but those tasks share parser vocabulary, verbs, spell names, and interface structure with training tasks, so the measured OOD transfer may reflect similarity rather than a general adaptation procedure.","fun_headline_variants_meta":{"raw":{"variants":["Evolutionary search finds better test-time learning policies","Meta-TTL: Learn adaptation policies, not hand-craft them","Learned adaptation policies transfer across unseen tasks","Optimizing adaptation policy via evolution boosts agents","Test-time learning: evolution beats hand-crafted policies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1672,"prompt_tokens":776,"completion_tokens":896,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":822}},"tokens_in":520,"tokens_out":896,"duration_ms":9248,"temperature":1.0,"reasoning_tokens":822,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T16:58:53.841711+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the evolved meta-prompt on a suite of interactive-fiction games that share no parser verbs, room names, or spells with the training games; if the W-AUC gain over the naive meta-agent vanishes, then the claimed generalization is an artifact of task similarity. Alternatively, compare outer-loop proposals against random mutations of equal edit distance—if gains persist, the proposer's reflective ability is not the source of improvement.","supporting_citations":[],"review_version":1}