{"id":"690dc595-c7a9-49ec-a5c3-1928323ea505","arxiv_id":"2505.17673","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Agents that start with no game knowledge can build a reusable skill library through trial-and-error and visual feedback, then progress further in two complex games than baseline agents given extra hints.","lead":"This paper builds an AI agent that starts with zero game knowledge, sees raw pixels, clicks and types like a human, and learns reusable skills from what changes on screen. The authors show this 'bottom-up' design can make progress in two complex games where standard agents fail, suggesting a path to AI that learns from experience instead of hand-built workflows.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never isolates skill evolution from scaffolding (SAM, MCTS, pretrained LLM), so the central claim that bottom-up learning drives the reported competence is underdetermined.","rationale":"The reader's identified weakness—visual difference as a proxy for semantic reward—is real and is explicitly acknowledged in the paper's Limitations section. It correctly bounds the framework's generality and makes the execution responsive rate partly circular. However, the most load-bearing gap for the central claim is the missing isolation of the skill-evolution mechanism itself. The full system differs from the baselines in multiple ways: SAM-based grounding, MCTS planning, clustering/refinement prompts, and the evolving skill library. The paper's ablations remove one component at a time but never remove the entire skill library while retaining the other scaffolding. Thus the empirical superiority attributed to 'bottom-up skill evolution' could instead stem from the perceptual and planning components, which are not novel to this paradigm. This concern is concrete and testable with a no-evolution control, and it directly affects whether the headline claim is supported. Given the existing CONDITIONAL verdict, this does not move the verdict; it sharpens the specific condition that must be met: show that the evolving skill library, rather than the scaffolding alone, is responsible for the reported competence.","tokens_in":17953,"tokens_out":9779,"duration_ms":84957,"concrete_test":"Run the same bottom-up architecture with the skill library permanently disabled: each step, use the same SAM-segmented observations and MCTS selection, but do not store, reuse, or refine any skill; candidates are generated ad hoc from atomic actions or a randomly initialized library and discarded after execution. Give the full system and this control the same action budget (1,000 steps), LLM backend (GPT-4o), and number of episodes and seeds. If the no-evolution control clears comparable floors or techs within noise, the skill-evolution mechanism is not the driver of the reported competence; if it fails to progress, the concern is resolved in the paper's favor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that agents acquire competence through bottom-up skill evolution. The experiments compare the full system—which includes SAM-based visual grounding, MCTS action selection, clustering/refinement, and a shared skill library—against baselines that receive none of these components, only game-specific text priors. No control removes the skill library while keeping SAM and MCTS. Table 3 ablates MCTS and the visual filter but always inside a system that still accumulates and reuses skills; it cannot show that the evolving library, rather than the scaffolding, produces the 13 floors or 8 techs. Since the paradigm's contribution is the evolution mechanism itself, the missing no-evolution control leaves an alternative explanation: a reactive agent with SAM+MCTS and LLM reasoning might reach comparable progression without any skill abstraction. The comparison is also confounded because baselines lack the same perceptual and planning scaffolding, so the reported superiority does not identify which component is load-bearing. The 'execution responsive rate' is partly circular because skills are admitted only if they cause visual change, but the absent control is the more fundamental gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a bottom-up agent paradigm in which an LLM-powered agent perceives raw screenshots, acts via mouse and keyboard, and incrementally builds a skill library through trial-and-reasoning, without game-specific prompts or privileged APIs. The framework formalizes skill augmentation, MCTS-based invocation, and LLM-guided refinement under an implicit reward based on visual change. Experiments in Slay the Spire and Civilization V report that the bottom-up agent clears 13 floors and completes 50 turns with 8 technologies, outperforming prompted baselines. The authors conclude that agents can acquire competence from scratch through experience, with skill evolution and sharing as the key mechanism.","tokens_in":18151,"tokens_out":5136,"duration_ms":40622,"significance":"If the claims held, this would be a notable step toward experience-driven agents in open-ended environments, complementing top-down workflow designs. The paper deserves credit for using a unified game-agnostic codebase, raw visual inputs, low-level actions, and for releasing code. However, the current evaluation does not establish the central claim: the headline execution metric is partly circular by construction, the experimental design does not isolate skill evolution from the surrounding scaffolding, and single-run results without variance do not support the quantitative superiority claims. The paper's own Section 6 candidly acknowledges several of these issues, including delayed visual signals and reset/evaluation problems; these are not merely future work, because they bear directly on the reported numbers.","major_comments":[{"comment":"The 'Execution Responsive Rate' is circular with the skill-admission filter. Algorithm 1 retains a skill only if 'effect(s(k)) is recognizable,' that is, if it produces observable visual change, and §5.1 defines the metric as the percentage of skill invocations that lead to observable changes in game state. Therefore the reported 98.56% and 92.27% rates are nearly tautological and cannot support the claim that the discovered skills are 'meaningful' or 'highly functional.' Please define an independent metric, such as the contribution of each skill to floors cleared, in-game score, or technologies researched, or evaluate responsiveness on a set of skills that were not filtered by visual-change detection.","section":"§5.1, Table 1; §3.2, Algorithm 1"},{"comment":"The formal objective in Eq. (2) defines Rskill as a sum of Rdiversity, Refficiency, and Rsemantics, but Section 4 states that 'we use only the semantic reward to guide skill evaluation.' The experiments therefore do not optimize or test the formalized objective. Either implement all three terms and report their contributions, or present Eq. (2) as a proposed formulation rather than the objective that is actually evaluated.","section":"§3.1, Eq. (2); §4"},{"comment":"No ablation removes the skill library while keeping the other components. The three ablations disable the visual filter, MCTS, or skill description, but in every condition the agent still accumulates and reuses skills. This leaves open the alternative explanation that the reported progression comes primarily from SAM-based perception, MCTS selection, and LLM reasoning, rather than from bottom-up skill evolution. Add a no-evolution control that keeps the same perception, MCTS, and LLM components but disables all skill storage and reuse.","section":"§5.3, Table 3"},{"comment":"All quantitative comparisons are single-run values with no seed variance or confidence intervals, and the footnotes indicate that some episodes terminated early at steps 301 and 232. Open-ended games are stochastic, so a one-episode difference, such as 13 floors versus 8 floors, cannot support the claim that the bottom-up agent 'outperforms all baselines.' Report multiple independent episodes per condition with mean and standard deviation, and state clearly whether the tabulated numbers are one representative run or an aggregate.","section":"§5.1, Table 1"},{"comment":"The skill-evolution-over-rounds evidence is not statistically grounded. Progression increases from 6 to 8 floors and score from 36 to 53, but the score is non-monotonic across rounds and no variance is reported. With only 100 steps per round and a single trajectory, the observed improvement could reflect stochasticity or simple library growth rather than skill refinement. Run repeated trials and report variance, or soften the claim to an anecdotal demonstration.","section":"§5.2, Table 2"},{"comment":"The baseline comparisons are confounded. The bottom-up agent receives SAM-based segmentation, MCTS, and a shared skill library, while the baselines receive only the text priors shown in Figure 7 and no equivalent perceptual or planning scaffolding. Consequently, the superiority in Table 1 does not identify which component is responsible, and the 'zero prior' framing is misleading because the bottom-up system uses a pretrained segmentation model. Add controlled baselines that share the same low-level interface and scaffolding, with and without skill evolution.","section":"§5.1, Baselines and Figure 7"}],"minor_comments":[{"comment":"In the bottom-up row for Civilization V, there appears to be a stray '1' before '50' in the Progression column; please correct the table formatting.","section":"Table 1"},{"comment":"The abstract claims operation 'without any game-specific prompts or privileged APIs,' but Section 4 introduces SAM for visual grounding; please clarify explicitly whether SAM is considered part of the zero-prior setting or an external perceptual prior.","section":"Abstract and §4"},{"comment":"Section 5.1 says each agent runs for three episodes per environment, but Table 1 reports single progression values with early termination at steps 301 and 232; please explain how the three episodes relate to the reported numbers.","section":"§5.1"},{"comment":"There are several typos, including 'reconized' and 'enviroments' in Section 4 and 'V oyager' in the related-work discussion; a careful copyedit pass is needed.","section":"§4 and Related Work"},{"comment":"Figure 2(b) needs axis labels and a legend; as printed, the visual comparison of game progression is difficult to parse.","section":"Figure 2(b)"},{"comment":"The exploration overhead discussion states that bottom-up agents take about 12 hours versus 6 hours for baselines, while Table 1 reports bottom-up token costs below GPT-4o; please clarify the relationship between wall-clock time and token cost, since these are different quantities.","section":"§6, Exploration Overhead"}],"recommendation":"major_revision","confidential_remarks":"This is a promising proof-of-concept, but the evaluation currently overstates what is demonstrated. The circular metric and the missing no-evolution control are fixable with additional experiments. If the authors can show that progression degrades when skill evolution is disabled, adopt a non-circular responsiveness metric, and report repeated-run statistics, the paper could become suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things before you read it. The system is real and does something genuinely new: an agent that starts with no game-specific prompts, no APIs, no explicit reward, and builds skills from raw screenshots and mouse clicks, running both Slay the Spire and Civilization V off one codebase. And the headline metric is partly circular, so take 98.56% with salt. A novel working system wearing a flawed evaluation — that is the whole paper.\n\nWhat is new: this is the most concrete instantiation I have seen of the Silver–Sutton 'era of experience' idea. Voyager scaffolds learning with game-specific APIs and prompts, and the UI baselines need priors to function. This agent is told only that it is playing an unknown game, and it still clears 13 floors in Slay the Spire and unlocks 8 technologies in Civilization V; baselines with full game priors mostly do worse. Code is released. The authors are also honest about the limits: the Limitations section names the visual-change assumption, the record-and-replay ceiling, the reset problem, and the exploration overhead. That candor earns credit.\n\nSoft spots, in order of realness. First, 'execution responsive rate' is the fraction of invocations producing observable visual change, and skills are admitted only when they produce observable visual change. The number is nearly the filter reporting on itself. The floors, techs, and turns are game-state facts and are not circular, so the core result stands — but the paper leans on the responsive rate rhetorically and should not. Second, the stress-test concern lands: there is no control that keeps SAM, MCTS, and the LLM while disabling the skill library, so the central claim — that evolution, not scaffolding, drives competence — is underdetermined. Table 2's round-over-round improvement (6 to 8 floors as the library grows) is suggestive but correlational; a no-evolution reactive baseline is the missing experiment. Minor, in proportion: no seed variance anywhere on multi-hour runs; Eq. 2 defines a three-term reward while the implementation uses only the semantic term (the paper admits this); and 'zero prior' really means zero game priors — GPT-4o and SAM bring substantial pretrained priors.\n\nWho it is for: anyone working on LLM agents, skill discovery, or open-ended evaluation. The appendix prompts alone are useful if you are setting up zero-prior agent experiments. The citation pattern is fine — Voyager, POET, DIAYN, and the Silver–Sutton essay are the right anchors.\n\nRecommendation: send it to peer review. The problems are fixable — an external non-circular metric, the no-evolution control, multi-seed runs — and the demonstration is one of the few real instantiations of experience-driven bottom-up agents. I would treat it as a conditional-accept candidate, not a desk reject.","headline":"A genuinely new zero-prior agent loop that works in two games, carrying a circular headline metric and an unisolated central claim — worth serious refereeing, not desk rejection.","tokens_in":18729,"tokens_out":6399,"would_cite":true,"duration_ms":48480,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An agent with zero knowledge of a game—no rules, no APIs, no game-specific prompts—learns to play it from raw screenshots and outperforms rivals that were handed full game rules.","keywords":["bottom-up agent design","skill evolution","LLM agents","open-ended environments","implicit reward","experience-driven learning","visual grounding","trial-and-reasoning"],"falsifier":"Run the identical zero-prior agent on a turn-based game where the winning line begins with a preparatory move that changes nothing on screen for several turns—for instance a Slay the Spire-style fight that punishes attacking and rewards building block for an attack that lands later—and inspect which skills enter the library. The framework predicts such preparatory skills will be filtered out as no meaningful action, and if the agent's progression collapses on that floor while a copy of the agent given a second screenshot-difference threshold succeeds, the visual-change reward is the bottleneck the claim must survive.","tokens_in":17710,"feed_emoji":"🎮","tokens_out":9019,"duration_ms":91356,"temperature":0.7,"pith_summary":"This paper tries to establish that LLM-based agents can become competent in unfamiliar, open-ended environments without any human-designed workflow. It proposes a bottom-up design in which an agent starts with an empty skill library, tries out low-level mouse and keyboard actions, keeps only the sequences that visibly change the game screen, and reasons about those changes to build longer, reusable skills. The same game-agnostic codebase, with identical prompts, is dropped into Slay the Spire and Civilization V, and it makes meaningful progress in both. If the claim holds, the main engineering asset in agent systems shifts from task-specific workflows to a shared, evolving library of skills acquired from experience.","feed_headline":"Zero-knowledge agent outplays prompted rivals in two video games","feed_subtitle":"From empty skill library and raw pixels to 13 cleared floors and 50 survived turns.","key_machinery":"The machine is the four-step skill-evolution loop centred on a shared, initially empty skill library $\\mathbb{S}$. Augmentation composes longer action sequences by appending atomic actions to validated shorter skills, keeping a sequence only if it triggers observable environmental change; invocation asks the LLM to propose candidate skills for the current screen and ranks them with Monte Carlo Tree Search; evaluation scores each execution by the visual difference between pre- and post-screenshots, formalised as the semantic reward $R_{\\mathrm{semantics}} = M(p_{\\mathrm{differ}}, \\sigma, x_t, x_{t+1})$; refinement prunes chronically poor skills and rewrites others by LLM reasoning. The formal objective that ties these together is the population-level maximisation of $R_{\\mathrm{skill}} = R_{\\mathrm{diversity}} + R_{\\mathrm{efficiency}} + R_{\\mathrm{semantics}}$ over the library, which turns open-ended play into a continual optimisation problem. Separately, the Segment Anything model converts raw pixels into candidate UI elements, giving the atomic actions a tractable target space.","core_discovery":"The central claim is that competence in an open-ended environment can be bootstrapped entirely from experience, with no priors, through a formalised skill-evolution loop. The environment is modelled as a partially observable Markov decision process whose observations are raw screenshots and whose actions are atomic mouse events, and the evolution of the skill library is written as the population-level objective $\\max_{\\mathbb{S}} \\mathbb{E}[R_{\\mathrm{skill}}(\\sigma,\\mathbb{S},x_t,\\mathcal{T})]$, where $R_{\\mathrm{skill}} = R_{\\mathrm{diversity}} + R_{\\mathrm{efficiency}} + R_{\\mathrm{semantics}}$. The semantic term, the only one used to drive evaluation in the implemented system, is obtained by asking an LLM to compare screenshots before and after a skill executes, treating visual difference as implicit reward. Skills are composed incrementally, retained only when they produce recognisable environmental change, selected at each step by Monte Carlo Tree Search over LLM-proposed candidates, and pruned or rewritten when their rewards are poor. On this basis the agent clears 13 floors of Slay the Spire with a 98.56% execution responsive rate, and completes 50 turns and 8 researched technologies in Civilization V, exceeding every baseline including variants given task-specific priors.","pith_inferences":["The screenshot-difference filter makes the reported execution responsive rate partly tautological: only skills that changed the screen were stored, so a high responsive rate is baked in by construction; a harder test is whether the retained skills are the ones that win games rather than merely move pixels.","The same reward choice biases the library toward greedy, immediately visible behaviours, so the paradigm as implemented is unlikely to discover defensive or setup play; pairing the visual signal with learned long-horizon credit assignment, which the authors list as future work, would directly test this ceiling.","A cheap, decisive experiment would run the codebase in a third environment engineered with delayed-payoff actions and vary the threshold on what counts as a recognisable change, measuring whether library composition and progression track the threshold."],"forward_implications":["Zero-prior agents can produce sustained, meaningful play in complex games: 13 cleared floors and score 81 in Slay the Spire, and 50 turns with 8 researched technologies in Civilization V.","Skill libraries improve with use: across training rounds the library grows through augmentation and pruning while execution responsiveness and in-game score rise, and the paper reports convergence by the fourth round.","MCTS-based skill selection is the linchpin of invocation: ablating it drops progression from several floors to a single floor.","A single unified codebase and prompt set transfers across structurally different games without modification, so deployment effort moves from per-game engineering to a general skill-evolution loop.","The paradigm is built for scale: with a shared library, every deployed agent's discoveries become immediately available to all others, so collective progress accelerates with population size."],"supporting_citations":[{"why":"Supplies the era-of-experience vision the paper instantiates: agents progressing from a stream of experiences rather than curated human data.","marker":"[37]"},{"why":"Voyager, the prior experience-driven skill learner, contrasted as still scaffolded by task-specific prompts and privileged APIs.","marker":"[43]"},{"why":"UI-TARS, the strongest UI-agent baseline, which the bottom-up agent's results are compared against in both games.","marker":"[32]"},{"why":"GPT-4o, the LLM backend used for skill selection, description, and refinement, and also a baseline.","marker":"[28]"},{"why":"Claude 3.7, a second baseline given game priors, whose best Civilization V run of 17 turns the bottom-up agent still exceeds.","marker":"[3]"},{"why":"Segment Anything, used to segment UI elements from raw screenshots so atomic clicks can target meaningful locations.","marker":"[21]"},{"why":"Monte Carlo Tree Search, the selection procedure that ranks candidate skills during invocation and whose removal collapses progression.","marker":"[40]"},{"why":"Makes the case for evaluating agents in open-ended, stream-based environments rather than static benchmarks, motivating the game setup.","marker":"[54]"}],"fun_headline_variants":["Skills evolve from raw pixels to beat game baselines","No priors, no prompts: agent masters games via trial and reason","Bottom-up skill evolution beats top-down prompts in games","From zero knowledge to victory: agent learns two games alone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The design assumes that a visible change on screen is a trustworthy sign that a skill is worth keeping, which means any useful behaviour whose payoff is delayed or visually subtle is never learned and gets pruned away.","fun_headline_variants_meta":{"raw":{"variants":["Skills evolve from raw pixels to beat game baselines","No priors, no prompts: agent masters games via trial and reason","Bottom-up skill evolution beats top-down prompts in games","From zero knowledge to victory: agent learns two games alone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1395,"prompt_tokens":1050,"completion_tokens":345,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":666,"completion_tokens_details":{"reasoning_tokens":277}},"tokens_in":666,"tokens_out":345,"duration_ms":5164,"temperature":1.0,"reasoning_tokens":277,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:42:20.738677+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical zero-prior agent on a turn-based game where the winning line begins with a preparatory move that changes nothing on screen for several turns—for instance a Slay the Spire-style fight that punishes attacking and rewards building block for an attack that lands later—and inspect which skills enter the library. The framework predicts such preparatory skills will be filtered out as no meaningful action, and if the agent's progression collapses on that floor while a copy of the agent given a second screenshot-difference threshold succeeds, the visual-change reward is the bottleneck the claim must survive.","supporting_citations":[{"cited_title":"Welcome to the era of experience","cited_arxiv_id":null,"evidence_quote":"Supplies the era-of-experience vision the paper instantiates: agents progressing from a stream of experiences rather than curated human data."},{"cited_title":"Gpt-4o: Openai’s multimodal flagship model","cited_arxiv_id":null,"evidence_quote":"GPT-4o, the LLM backend used for skill selection, description, and refinement, and also a baseline."},{"cited_title":"Claude 3.7 sonnet and claude code","cited_arxiv_id":null,"evidence_quote":"Claude 3.7, a second baseline given game priors, whose best Civilization V run of 17 turns the bottom-up agent still exceeds."},{"cited_title":"The second half","cited_arxiv_id":null,"evidence_quote":"Makes the case for evaluating agents in open-ended, stream-based environments rather than static benchmarks, motivating the game setup."}],"review_version":1}