{"id":"37e017c5-1617-4a37-99f9-76c0a6e9b077","arxiv_id":"2505.08459","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An LLM agent that identifies its opponent's strategy and then searches a learned strategy-evaluation network for the best response substantially outperforms prompt-only LLM planners in MicroRTS.","lead":"Large language models can now beat simpler planners in a real-time strategy game by learning which strategies beat which, watching the opponent's moves, and then searching for the best counter-strategy. The system, called SAP, reports win rates competitive with reinforcement-learning agents on MicroRTS without any RL training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'unseen' strategies are an in-distribution random split of the same LLM-generated strategy library, so Eq. (7)'s full-space SEN search is not tested on genuinely novel opponent styles; the central generalization claim is unproven.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing point: the hand-designed six-feature strategy space and the SEN's extrapolation from 30 seen strategies to the full space are the uncertain link in the argument. I agree with that identification. The paper has real strengths: a clear two-stage design, a released code repository, an ablation showing that the SEN contributes a 7.81% improvement and that expert tips contribute 37.5 percentage points, and a direct best-response evaluation against unseen strategies. Those results support the framework's internal coherence. However, the abstract's generalization claim outruns the evidence. The 20 'unseen' strategies are a random holdout from the same LLM generation process and the same feature space, so they cannot establish robustness to opponent behaviors that lie outside Xi. The 85.35% improvement figure is also computed from a round-robin average rather than a fixed external opponent benchmark, but that is a reporting issue that can be fixed with a reanalysis. The deeper scientific question is whether the SEN can select good responses for the full space in Eq. (7); that is exactly what the proposed holdout-feature experiment would test. Since the paper is already CONDITIONAL, my read does not change the verdict; it sharpens the condition under which the central claim should be accepted: the authors should supply an out-of-distribution test that removes entire feature values or uses hand-designed opponents outside Xi. Until then, the claim of robust generalization to novel, unseen strategies remains unverified.","tokens_in":11758,"tokens_out":9702,"duration_ms":100405,"concrete_test":"Retrain the SEN using only strategy pairs drawn from the subset of Xi with Aggression=False and, say, Defense distance in {1,2,3}, then run the Section IV-B2 best-response protocol against opponents whose true strategies have Aggression=True or Defense=4. If the SEN-selected responses against these out-of-support opponents do not significantly outperform a random strategy from the same subset, or if their win rate falls to chance, the SEN is not extrapolating and the 'unseen strategy' generalization claim is unsupported. As a complement, evaluate SAP against hand-constructed opponent policies that deliberately violate one or more axes of Xi, such as alternating between offense and defense across time, measuring both the recognition accuracy of Eq. (9) and the win rate of the resulting best response.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SAP generalizes to novel, unseen strategies rests on the 20 unseen strategies introduced in Section IV-A. These strategies are generated by the same LLM (Eq. 3) within the same hand-coded six-feature space Xi (Section III-B1), and they are a random split from the same 50-strategy library. The held-out set is therefore in-distribution with respect to the feature support, not a test of extrapolation outside Xi. Yet Eq. (7) searches the entire strategy space, so the SEN, trained on only 900 outcome pairs among the 30 seen strategies, is asked to rank strategies whose feature vectors may be far from its training support. The strong result in Section IV-B2 (93.3% best-response wins against unseen strategies) demonstrates interpolation within the LLM's strategy prior, but it does not show that the recognizer (Eq. 9) or the SEN would handle an opponent behavior that falls outside the six hand-designed axes. If a real opponent uses a non-stationary or micro-dependent style outside Xi, the recognizer is forced to map it into Xi and Eq. (7) can return an arbitrary, unvalidated counter-strategy. The framework's exploitation claim is thus load-bearing on the untested assumption that Xi is complete and that the SEN extrapolates reliably beyond the support of its training strategies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Strategy-Augmented Planning (SAP), a two-stage framework for opponent exploitation in adversarial games. In an offline stage, LLMs generate a strategy library inside a hand-coded six-feature strategy space, pairwise strategy battles are simulated, and a small MLP (the Strategy Evaluation Network, SEN) is trained to predict win probabilities between strategy vectors. In the online stage, a trajectory summarizer and an LLM recognizer map the opponent's behavior to a strategy, the SEN is greedily searched over the full strategy space for a best response, and the selected strategy plus expert tips prompt an LLM planner to emit abstract actions. Experiments in MicroRTS on 8x8 and 16x16 maps report that SAP beats the LLM baselines, reaches 97.0% average win rate against the other tested methods, performs competitively with RL agents against scripted AIs, and generalizes to 20 held-out strategies. The paper claims a 85.35% improvement over the Tips-augmented baseline and a 93.3% best-response win rate in the SEN evaluation.","tokens_in":12000,"tokens_out":6242,"duration_ms":60733,"significance":"If the results hold, the contribution is significant: a low-dimensional learned strategy evaluator could replace substantial hand-crafted domain knowledge in grounding LLM planners, and the explicit strategy space makes the opponent-modeling process interpretable. The paper has real strengths: the final win rates are obtained by actually playing games rather than by the SEN's own predictions; the scripted-AI comparison is an external benchmark taken from prior RL work; the code is released; and the ablation of the SEN component (SAP w/o SEN in Table VI) supports the claim that the learned evaluator adds value. However, the current evidence does not yet establish the strong generalization and competitiveness claims, because the unseen strategies are in-distribution with the training library, the statistical basis is minimal, and the full-space search in Eq. (7) is evaluated only inside the support of the hand-coded features.","major_comments":[{"comment":"All win rates are reported as point estimates without episode counts, variance measures, or significance tests. The headline 85.35% improvement and the 92.50% SAP average in Table VI cannot be assessed for reliability; for instance, the 2.22 percentage-point gap between SAP (92.50%) and SAP-EPE (90.28%) in Table VI cannot be distinguished from sampling error unless the per-cell episode count is reported and a paired test is provided. Please report N for every cell, include standard errors or confidence intervals, and run paired tests, including for the 93.3% best-response rate in Fig. 4(b) and for the scripted-AI comparisons in Fig. 3.","section":"IV-A, Table IV, Table VI"},{"comment":"The 'unseen' strategies are a random split of the 50-strategy LLM-generated library, so they are generated by the same Eq. (3) prompt and live inside the same hand-coded six-feature space Xi. This makes the held-out evaluation an in-distribution interpolation test, not a test of generalization to opponent behaviors outside Xi. Because Eq. (7) searches the entire strategy space, the claim of robust generalization to 'novel, unseen strategies' is stronger than the evidence; the authors should add opponents whose feature vectors are deliberately outside the training ranges or that are produced by a different generative procedure, and report recognition accuracy on such opponents.","section":"IV-A and III-C2"},{"comment":"The SEN is trained on 900 outcomes among only 30 seen strategies, but Eq. (7) greedily searches all of Xi. The paper provides no evidence that U extrapolates beyond the support of the training strategy vectors: the confusion matrix in Fig. 4(a) is a split of the same 900-outcome dataset, and Fig. 4(b) evaluates best-response selection when the true opponent strategy is provided, not after the recognizer of Eq. (9). Please report the accuracy of the recognizer itself, test SEN on out-of-distribution strategy pairs, and quantify how many of the maximizing strategies found by Eq. (7) use combinations of feature values that are not represented in the training library.","section":"III-B2, III-C2, IV-B2"},{"comment":"The number of episodes N per strategy pair is never specified, and the treatment of draws is unclear: Eq. (4) defines r_ij as an expected result over N episodes, while later win rates such as Table IV appear to be binary win/loss percentages and Fig. 3 explicitly includes ties. Since the SEN is trained with binary cross-entropy on r_ij, the encoding of draws and the choice of N directly affect the network's training target and the reported win rates; please state N, the tie-handling rule, and the symmetry assumption r_ij = r_ji more precisely.","section":"Eq. (4) and IV-A"}],"minor_comments":[{"comment":"The strategy update interval k is used in Algorithm 2 and mentioned in Section III, but its value in the experiments is never reported; please state k for both the 8x8 and 16x16 experiments and explain how SAP-EPE in Table VI differs from the default k.","section":"IV-A"},{"comment":"The text says the SEN architecture is a 'Multi-Layer Perception'; this should read 'Multi-Layer Perceptron'.","section":"IV-A"},{"comment":"The six strategy features are described in an unlabeled bullet list and referenced from Fig. 1; the list should be numbered or given a section number, and the discrete ranges should be stated in the text as well as in the figure.","section":"III-A and Fig. 1"},{"comment":"The notation is confusing: Eq. (4) writes the expectation with a subscript 'NX R' that is undefined, and Eq. (6) conditions U on r_ij even though r_ij is the target being predicted. Please rewrite these equations with clean conditioning on the two strategy vectors.","section":"Eqs. (4) and (6)"},{"comment":"The Avg column is the mean over all opponent columns except the diagonal, but this is not stated; add a table note and also state the per-cell number of episodes.","section":"Table IV"}],"recommendation":"major_revision","confidential_remarks":"The central framework is plausible and the ablation evidence is encouraging, but the statistical reporting and the in-distribution nature of the unseen-strategy test are load-bearing issues. I would support publication after the authors add episode counts, significance tests, an out-of-distribution opponent evaluation, recognition accuracy, and the missing hyperparameter values."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely worth your time: SAP couples an explicit, low-dimensional semantic strategy space with an offline-trained Strategy Evaluation Network and an LLM-based recognizer that grounds best-response search in concrete plans. That assembly is not in the cited prior work, and it is lightweight in the right way—no RL fine-tuning, just a small MLP on collected game outcomes plus in-context planning. The paper also earns credit for evaluating by actually playing games rather than by the SEN's own predictions, for ablating the SEN and the expert tips, and for releasing code.\n\nWhere it gets soft is the evidence for the headline claim. The '85.35% improvement' comes from a round-robin table that includes SAP beating the baseline TA itself, which inflates the number. Win rates are reported without episode counts, variance, or significance tests, so we cannot tell whether the margins are stable. More importantly, the 'novel, unseen strategies' are a random 20/30 split from the same 50-strategy library generated by the same LLM within the same hand-built six-feature space Xi. That is an in-distribution test of interpolation, not a test of extrapolation to genuinely novel opponent styles. Eq. (7) searches the entire Xi, but the SEN was trained on only 900 outcomes among 30 strategies, so its rankings outside that support are unvalidated. If a real opponent behaves in a way that does not fit the six semantic axes, the recognizer will force it into Xi and the search may return an arbitrary counter-strategy.\n\nThe load-bearing assumptions—that Xi is expressive enough and that the expert tips transfer—are acknowledged only indirectly. The 16x16 results show a noticeable drop, and the conclusion admits the SEN needs retraining per map. These are real limitations, but they are not fatal to the framework. The ablation shows the SEN and tips each matter, and the direction is sensible. What is missing is a proper out-of-distribution test: for instance, perturbing strategy parameters outside the training ranges, adding a seventh feature, or using strategies designed by humans rather than sampled from the same LLM prior.\n\nThis paper is for researchers working on LLM agents in adversarial game environments, especially those who want a cheap alternative to RL-based opponent modeling. It deserves a serious referee, but the revision needs statistical detail and a more careful framing of generalization. I would send it to review with major revision, not desk reject it.","headline":"Useful lightweight opponent-exploitation framework for LLM agents, but the generalization claim is overstated because the 'unseen' strategies are an in-distribution split of the same LLM-generated library.","tokens_in":12583,"tokens_out":1270,"would_cite":true,"duration_ms":13797,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An LLM agent can exploit unseen opponent strategies in an RTS game by outsourcing strategy evaluation to a small network trained offline.","keywords":["opponent modeling","large language models","strategy evaluation network","real-time strategy games","MicroRTS","best response search","LLM planning","opponent exploitation"],"falsifier":"Build an opponent whose style deliberately falls outside the six-feature space—for example, alternating between aggressive and defensive phases within a single episode—and measure SAP's recognition accuracy and win rate. If the best-response search no longer beats the prompting baselines, the generalization claim fails. A cheaper check is to retrain SEN on different 30-strategy subsets and see whether the 93.3% best-response win rate against held-out strategies is stable or collapses.","tokens_in":11492,"feed_emoji":"🎮","tokens_out":8750,"duration_ms":80893,"temperature":0.7,"pith_summary":"The paper argues that a large language model can plan well in an adversarial real-time strategy game it was not trained for, if the model is paired with a small learned strategy evaluator. The two-stage SAP framework first constructs an explicit low-dimensional strategy space and trains a Strategy Evaluation Network on offline battle outcomes, then online recognizes the opponent's strategy and greedily searches the full space for the counter-strategy with the highest predicted win rate. In MicroRTS this raises average win rates by 85.35% over the best prompting baseline and matches the competitiveness of reinforcement learning approaches against strong scripted opponents. The point matters because it separates opponent modeling from domain expertise: the LLM supplies general language and planning, while a cheap trained network supplies the game-specific knowledge.","feed_headline":"A small win-rate network lets LLM planners exploit unseen foes","feed_subtitle":"In MicroRTS, strategy scoring plus online recognition beats prompting baselines and rivals RL agents.","key_machinery":"The engine is the Strategy Evaluation Network (SEN): a three-layer MLP that takes two encoded strategy vectors and outputs a win probability, trained with binary cross-entropy on paired battle results. Around it sit three supports: an LLM-based strategy generator that populates the strategy space with diverse verbal strategies, a trajectory summarizer that compresses raw game logs into feature-aligned summaries, and an online re-planning loop that re-recognizes the opponent every $k$ steps and greedily re-selects the best response via $\\xi^{1,*} = \\arg\\max_{\\xi^1 \\in \\Xi} U(\\xi^1, \\hat{\\xi}^{-1})$. Expert tips bridge the gap between the selected macro-strategy and the concrete abstract-action plan the LLM must emit. The network's low-dimensional input is what lets the whole pipeline be fitted from only 900 battles.","core_discovery":"On the paper's own terms, the central discovery is that opponent exploitation can be decomposed into two tractable tasks: representing strategies by six explicitly defined features, and learning a function $U(\\xi_1,\\xi_{-1})$ that predicts the probability that one strategy beats another. With 900 battle outcomes among 30 seen strategies, the fitted network supports best-response search over the whole strategy space, including 20 strategies never seen in training, and the resulting agent wins 97.00% on average against plan-level and strategy-level baselines on the 8x8 map. The best-response search alone achieves a 93.3% win rate against true unseen opponent strategies. The paper attributes the higher win rate against unseen strategies to the seen set containing stronger strategies, so the key claim is not that unseen opponents are easy but that the evaluator generalizes across the defined space.","pith_inferences":["The hand-crafted six-feature space is the likely ceiling of the approach: if the method is right, learning the strategy embedding itself would widen the range of representable behaviors without redesigning features.","The 16% false-positive rate in the SEN confusion matrix indicates that some predicted winning counter-strategies are actually losing; a hedged fallback that tests the top two candidates for a short window before committing could reduce those losses.","Because the agent re-recognizes only every $k$ steps, it treats the opponent as approximately stationary within that interval; opponents that switch styles mid-episode would stress that assumption and may require shorter intervals or explicit change-point detection.","An equal-strength comparison between seen and unseen strategies would be needed to separate the evaluator's generalization from the paper's observation that the seen strategy set happens to contain stronger strategies."],"forward_implications":["LLM agents can become competitive in unfamiliar real-time games without game-specific training data; the learned evaluator supplies the missing expertise.","The low dimensionality of the strategy representation keeps offline cost small (50 strategies, 900 battles), so adapting the pipeline to a new map mainly means re-running strategy generation and battle collection, not a large reinforcement-learning run.","An opponent catalogue is not required in advance: the defined strategy space plus greedy best-response search extrapolates to styles never seen during training.","Re-planning every $k$ steps makes the agent adaptive within an episode, and the ablation results indicate that shorter intervals strengthen exploitation.","Wherever a strategy can be verbalized in a low-dimensional feature space, the same two-stage design should transfer to other adversarial planning domains, as the paper claims."],"supporting_citations":[{"why":"Provides the MicroRTS environment where strategy battles, SEN training data, and online evaluation are run.","marker":"[3]"},{"why":"Supplies the evaluation setup and the reinforcement-learning baseline used for the comparison with scripted opponents.","marker":"[2]"},{"why":"The prior LLM opponent-modelling work that SAP contrasts with and extends by adding a learned evaluator.","marker":"[12]"},{"why":"The chain-of-summarization LLM planner whose dependence on domain familiarity motivates SAP's external strategy evaluator.","marker":"[11]"},{"why":"Chain-of-thought prompting, one of the plan-level baselines in the main comparison table.","marker":"[18]"},{"why":"The transformer-based policy baseline used in the comparison against rule-based AIs.","marker":"[34]"}],"fun_headline_variants":["Win-rate network unlocks LLM opponent exploitation","Strategy-aware network boosts LLM planning vs foes","Predict strategy outcomes to exploit unseen opponents","LLM planners exploit via strategy win-rate predictions","Small strategy net gives LLMs a planning edge in games"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the six hand-designed strategy features cover every opponent behavior SAP will face, and that a network trained on only 30 strategies and 900 battles predicts best responses across the entire strategy space, including the 20 unseen strategies.","fun_headline_variants_meta":{"raw":{"variants":["Win-rate network unlocks LLM opponent exploitation","Strategy-aware network boosts LLM planning vs foes","Predict strategy outcomes to exploit unseen opponents","LLM planners exploit via strategy win-rate predictions","Small strategy net gives LLMs a planning edge in games"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2659,"prompt_tokens":970,"completion_tokens":1689,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":1619}},"tokens_in":586,"tokens_out":1689,"duration_ms":13722,"temperature":1.0,"reasoning_tokens":1619,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:54:50.273394+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build an opponent whose style deliberately falls outside the six-feature space—for example, alternating between aggressive and defensive phases within a single episode—and measure SAP's recognition accuracy and win rate. If the best-response search no longer beats the prompting baselines, the generalization claim fails. A cheaper check is to retrain SEN on different 30-strategy subsets and see whether the 93.3% best-response win rate against held-out strategies is stable or collapses.","supporting_citations":[{"cited_title":"The combinatorial multi-armed bandit problem and its application to real-time strategy games,","cited_arxiv_id":null,"evidence_quote":"Provides the MicroRTS environment where strategy battles, SEN training data, and online evaluation are run."},{"cited_title":"Transformers as policies for variable action envi- ronments,","cited_arxiv_id":null,"evidence_quote":"The transformer-based policy baseline used in the comparison against rule-based AIs."}],"review_version":1}