Pith.

Robotics · arXiv:2610.13829 v1 · submitted 9 Oct 2026 · 21 references

MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents

Jingwei Song, Zhisheng Chen, Bingfan Zeng, Jing Tang, Yubo Hou, Jinhao Jing, Zhengwei Xie, Linfeng Zhang

The paper says

“A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one world and transfer to later tasks.”

more of the abstract

A saved memory or successful final action does not by itself establish that a reusable capability was acquired; evaluation must separate capability change from carried-over resources, unequal source exposure, and early execution failures. We construct MC-EvoBench as a controlled Minecraft benchmark with a fixed catalog of 19 registered evolution chains, including 12 primary evaluation chains and seven calibration or probe chains, together with three graph-aligned paired probe families for transfer. The benchmark represents each task as a task-state graph, defines stage-level environment verifiers, assigns matched source and target roles, resets the target world, and permits only a declared agent artifact to cross the source-target boundary. For each probe, no-prior, related, unrelated, and shuffled source conditions are evaluated under matched environment-step budgets and five seeds; we report TSR, WGP, and VMP and retain provenance records. The benchmark reveals a consistent direct-capability gap between controller and planner-backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood-species gain hinges on one seed, and all intervals include zero. Stage audits further show that early wood-collection and crafting bottlenecks often leave later capabilities unobserved. MC-EvoBench thus offers a controlled testbed for durable, transferable embodied capabilities. Code and data are available at https://github.com/xzw-ustc/mc-bench-open-source

from the abstract

Pith says

Machine review · deepseek-v4-flash · 12 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A carefully built reset-boundary benchmark whose measured null is real but whose sensitivity is uncalibrated: with no positive control, the headline finding says more about one weak reference artifact than about stored experience as a mechanism. This line is wrong

3 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 21 references say.

The paper says, at Section 4, Tables 3-4; Section 3.3

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.66):

“They do not, however, isolate whether a controlled source interaction causes a later improvement after the target world is reset.”

Pith says: Objection 1 of 3 · major

The benchmark's sensitivity to genuine transfer is never calibrated. No positive-control method known to produce transferable capabilities is run through the reset protocol, and the reference agent's source artifacts are mostly failed episodes and reflection advice, with related source TSR of only 0.100, 0.080, and 0.150 (Table 5).

more of objection 1

Because the tool probes start from a very low execution floor (no-prior target TSR 0.32 and 0.15, Table 3) and most direct-matrix runs stop before the stick stage (Figure 3a, Table F10), the null result in Table 4 cannot be attributed to stored experience as a capability mechanism rather than to the floor hiding early-stage effects or to the single memory implementation's retrieval limits. Findings 2 and 3 should be reframed as null results for this artifact under these instances, with sensitivity calibration flagged as an explicit open requirement.

The paper says, at Section 4.2, Table 2

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

Table 2 reports three-seed averages without error bars or seed-level dispersion, and Table F8 shows that some cells have only two seeds. Several partition differences central to Finding 1, such as the JARVIS-1 controller TSR of 9.9 versus the no-op baseline of 3.5 on core progression, could plausibly be driven by one or two seeds; the paper does not provide the paired per-chain seed data needed to assess this.

more of objection 2

The direct-capability claim should either report per-seed values, show intervals or ranges, or explicitly relegate these differences to descriptive claims.

The paper says, at Section 4.6 and Table D3

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.75):

“All reported confidence intervals include zero, and the intervals are wide because they are estimated from five matched seeds.”

Pith says: Objection 3 of 3 · major

The claim that the small wood-species GATG gain on TSR is driven by a single seed is supported by Table D3 and is stated by the authors. However, the same per-seed inspection shows that the negative tool-probe contrasts are also highly seed-dependent: on the stone-tool probe, the largest related-vs-no-prior deficit (-0.250) comes from seed4 while seed3 shows 0.000 (Table D3).

more of objection 3

With only five seeds, the paired intervals in Table 4 are too wide to distinguish 'no stable transfer' from 'no measurement power.' The paper should state this equivalence explicitly and recommend a minimum seed count before the benchmark is used to audit future RSI methods.

6 minor objections

The paper says, at Abstract and Section 1

Pith says: minor objection 1 of 6

The abstract promises a controlled testbed but would be clearer if it stated up front that the reference study uses a single-agent implementation and that no positive-control transfer instance is included.

The paper says, at Table 1

Pith says: minor objection 2 of 6

The comparison lists 'Experience persistence' and 'Capability transfer' as ticked for Voyager, but Voyager's skill library is not evaluated under a reset-target protocol with matched controls; the table should clarify that these are qualitative dimensions, not measured properties of the cited systems.

show 4 more

The paper says, at Appendix B.1, Table B1

Pith says: minor objection 3 of 6

The 'Calibration and probe' row of Table B1 lists 0 declared tasks and 7 chains with 51 stage occurrences, but the probe families in Appendix F.2 reuse catalog tasks; the table should explain that probe chains reuse task definitions rather than declaring new ones.

The paper says, at Appendix C.5, Equation (C.3)

Pith says: minor objection 4 of 6

The VMP denominator is stated as 'eligible states' with an empty-set fallback to zero; the paper should note that this makes VMP not directly comparable across chains with different eligible-state sets, which is relevant when interpreting the three probe families together.

The paper says, at Appendix E.2

Pith says: minor objection 5 of 6

The retrieval rule adds a 0.15 Jaccard bonus for successful records and uses token-set overlap; the paper should note that for a target instruction such as 'collect acacia log,' the related birch-plan record shares most tokens, so retrieval operates close to exact instruction matching and may not exercise generalization over memory content.

The paper says, at Table 5 caption

Pith says: minor objection 6 of 6

The source-side 'LLM calls' column covers only the JARVIS-1 online extension; the caption would benefit from a footnote that call counts and token usage for the other direct-matrix configurations are reported separately in the appendix, since the current caption implies a broader scope.

What would settle it

Inject a hand-authored oracle artifact in the exact Table G1 schema under the identical paired protocol: for each of the three probe families, create one memory record whose instruction, plan, dispatched plan, and advice describe the correct target stages (Table F11: T48/T49 for wood species; T02, T03, T13, T04, T38 for wooden tool; T02, T03, T13, T04, T05, T10, T10, T62 for stone tool) with species-abstracted advice such as "chop the log named in the current instruction, then craft planks from it." Freeze the artifact, run all four target conditions with the same five agent seeds, world seed 31, stage budgets, and reset rules, and require a GATG whose 95% interval excludes zero on at least the wood-species and wooden-tool probes, plus a positive related-minus-no-prior TSR on both tool probes.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 12 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 1 min 45 s · 32,215 tokens read, 11,910 written
  3. Editor wrote the verdict above 32 s · 33,234 tokens read, 3,187 written
  4. Referee major revision: 3 major objections, 6 minor 29 s · 32,636 tokens read, 3,108 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Two things to know. First, the controlled-reset protocol is the real contribution: only a declared artifact crosses the source-target boundary, the unrelated and shuffled controls are matched on environment steps, and the stage-level metrics keep partial progress visible instead of collapsing everything into final success. I checked the arithmetic — Table 4's paired intervals reproduce correctly from the per-seed TSR records in Table D3, and the claims match the data. Second, the headline null ("stored experience is not acquired capability") is a sound measurement about one specific reference implementation, not about stored experience as a mechanism. The authors know this — sentences b190 and b196 state the scope plainly — but Findings 2 and 3 reach slightly past their own hedge.

What's genuinely new: no cited embodied benchmark isolates transfer this way. Voyager-style skill reuse was never audited against a world reset with physical carryover stripped out and matched related/unrelated/shuffled sources. The failure-stage audit is also useful: most runs die before crafting sticks, which explains why final-success scores are uninformative for these agents. The provenance records, fixed seeds, and frozen catalog make this reproducible in a way most agent papers aren't.

The soft spot is sensitivity calibration, and it's the load-bearing one. No positive-control artifact — a method known to transfer, or a synthetic memory with an injected reusable procedure — has been run through the protocol. That matters because the strongest agents sit near the execution floor (no-prior TSR of 0.15 on the stone-tool probe), and the reference memory stores mostly failed episodes retrieved by token-set Jaccard over instruction text. Under those conditions, the null could reflect probe difficulty or the weakness of this particular artifact rather than a property of stored experience. That is a calibration gap, not a measurement error — the reported facts are what they are, and the authors' scope statements are honest. But it means the benchmark's ability to register real transfer when it occurs is untested.

Minor: Table 2 reports three-seed averages without uncertainty, with a couple of two-seed cells. Not a big deal for the paper's main claim, since the transfer results are all five-seed paired.

This paper deserves a serious referee. The benchmark is a usable contribution to embodied RSI evaluation, and the honest negative result with reproducible provenance is worth publishing. A referee should ask for a positive-control run, or at minimum a re-framing of Findings 2–3 as claims about the reference artifact. Either way it is a revision, not a rejection.

The strongest objection

The load-bearing issue is sensitivity calibration, not the reported numbers. I checked the per-seed records in Table D3 against Tables 3 and 4, including the GATG confidence intervals, and the arithmetic is internally consistent. The interpretive claim, however, goes beyond the measured null. Finding 2 ("Stored experience is not acquired capability") and Finding 3 treat the null as evidence about stored experience as a mechanism, and that inference requires the protocol to register a positive signal when a transfer mechanism is actually present. No ground-truth-positive instance exists in the paper: the strongest agents sit at the execution floor (Table 2), Figure 3a shows most runs stop before the stick stage, the two tool probes start from no-prior TSR of only 0.32 and 0.15 (Table 3), and the only positive GATG appears on the shortest probe and hinges on one seed (Section 4.6). If the longer probes' floor hides early-stage transfer benefits, the null says more about probe difficulty than about experience. Second, the single reference artifact confounds mechanism with implementation: source episodes mostly fail (source TSR 0.100/0.080/0.150, Table 5), so the stored artifact is largely failed attempts plus reflection advice; retrieval is token-set Jaccard over instruction text (Appendix E.2), so the acacia target retrieves the literal birch plan; and Card G1 forbids the planner graph or verifier access. Whether a memory that abstracts over species would register a transfer gain is untested. The authors acknowledge the reference method's scope (sentences b190, b196), so the null is a sound measured fact about this artifact, but the benchmark's ability to detect real transfer, which the headline claim needs, remains uncalibrated.

The central claim has two parts: the benchmark is a controlled testbed, and the reference agent shows no stable transfer. The first part is well supported: the reset protocol (Section 3.3, Appendix D), matched source budgets (Table D2), paired five-seed replicates (Table D3), stage records, and provenance handling are carefully engineered, and the authors hedge the second part explicitly (b178, b189, b190, b196). The strongest reason to keep the verdict CONDITIONAL is that nothing in the paper demonstrates the testbed can register transfer when it exists. The reader named this as the weakest assumption ("No positive-control agent known to transfer was evaluated, so the benchmark's sensitivity to real transfer is uncalibrated"), and I agree; I add that the paper's own failure-stage audit (Finding 5, Figure 3a) makes the mechanism-versus-floor confound concrete: on the two tool probes, even no-prior agents mostly fail before later stages, so the metrics operate near a floor that could suppress genuine early-stage transfer. The wood-species probe, the only one short enough to sit above the floor, is also the only one showing any positive (though seed-dependent) contrast, which is at least consistent with a floor effect masking transfer rather than with experience providing no benefit. I verified the per-seed tables against the aggregate tables and the GATG intervals; they match, and the Table 4 CI column is for GATG, which the per-seed values confirm. Statistical power with five seeds is acknowledged by the authors (b178) and is therefore not the load-bearing issue, and the missing planner names in Table 2 are a rendering defect (the text names DeepSeek and GLM), not an argument-level flaw. Because the concern matches the reader's already-CONDITIONAL verdict and the proposed check is additive, the verdict is UNCHANGED: accept conditionally on adding the oracle positive-control calibration (and ideally per-seed uncertainty for the direct capability matrix, as the reader requested). If the oracle test fails, the correct revision would be to weaken Finding 2 to an implementation-level statement.

The referee's report

If the benchmark is valid and reproducible, it fills a real gap: most Minecraft benchmarks score final-task success only and do not separate capability change from carried-over resources, unequal source exposure, or early execution failure. The paper ships the full catalog, verifiers, protocol, repository location, per-seed records, and machine-readable task definitions, which is a genuine strength for an evaluation-focused contribution. The null transfer result is honestly stated and appropriately scoped to the tested agent, probes, seeds, and budgets. However, the interpretive reach of the null result is limited by the absence of any positive-control artifact that is known to transfer under the reset protocol, so the benchmark's sensitivity to genuine capability transfer is currently uncalibrated. The paper already acknowledges this kind of limitation in general terms, but the missing positive control remains a load-bearing gap for the transfer findings rather than a presentation issue.

Pith's reading of the paper (a machine's summary, not the authors')

MC-EvoBench is built to answer one question: when an agent's performance improves after its own experience, is that a reusable capability or an artifact of carried-over resources, extra source exposure, or execution luck? The paper answers by construction, with a controlled Minecraft benchmark that resets the target world, transfers only a declared memory artifact, and compares related, unrelated, and reversed-order source experience under matched budgets and five seeds, and by measurement, showing that under these controls the reference agent gains no stable transfer: related experience does not improve either tool probe, its small wood-species gain hinges on a single seed, and all 95% confidence intervals include zero. The practical stake is that claims of embodied self-improvement cannot be established by a saved trajectory or a higher final score; they need world-reset, control-matched evidence of changed later behavior. The benchmark also documents that current controllers stall at early wood-collection and crafting stages, so long-horizon scores largely measure early execution bottlenecks rather than later planning or memory.

What carries the argument. The load-bearing object is the paired audit protocol built around a task-state graph $G=(T,S,E)$, where tasks are executable contracts, states are items, tools, locations, or structures, and typed edges ($requires$, $produces$, $consumes$) form strict execution paths while $enables$ and $abstracts$-to edges organize probe design without counting toward progress. Three metrics read off stage evidence: TSR, the fraction of planned stages whose verifier passes; WGP, the weighted fraction of strict execution edges with verified progress; and VMP, the fraction of eligible item or tool milestones reached before a chain stops. The transfer test isolates experience by resetting the target world and inventory and allowing only a frozen artifact to cross, here the JARVIS-1 online extension's episode memory, retrieved by token-set Jaccard similarity with a 0.15 success bonus, up to four records, capped at 256 entries. Four matched target conditions under equal environment-step budgets convert the question "did experience help?" into "did it beat matched controls?", which is the comparison that carries the null result.

Core claim. The paper's central claim is that a saved memory or a successful final action does not constitute an acquired, transferable capability: evidence of experience-driven improvement must be a change in later behavior that survives a full world reset, with only a declared agent artifact crossing the source-target boundary. Operating on that definition, MC-EvoBench evaluates a reference agent, a JARVIS-1-style controller with an online episode-memory extension, on three paired probe families, each with no-prior, related, unrelated, and shuffled source conditions under matched environment-step budgets and five seeds. The result is a null transfer finding: related experience does not reliably beat no prior, and on both tool probes, wooden-shovel and stone-shovel transfer, it lowers Task Success Rate, Weighted Graph Progress, and Verified Milestone Progress, while every 95% confidence interval on the paired contrasts includes zero. The only positive contrast, a small control-adjusted gain on the wood-species probe, is driven by a single seed and disappears without it. A separate direct-capability matrix shows controller-based agents clearly outperforming planner-backed variants, and a stage-level audit shows that most runs stop at early wood and crafting stages, leaving later capabilities unobserved.

The premise it rests on. The reference agent, the JARVIS-1 online extension with its hand-set retrieval settings (Jaccard bonus 0.15, up to four retrieved records, 256-record memory cap) driving a controller that struggles at early crafting, is treated as a representative instance of experience-driven self-improvement; if that artifact is too weak or poorly integrated, the null transfer result says more about the implementation than about stored experience as a mechanism, and with no positive-control agent evaluated, the benchmark's sensitivity to genuine transfer is uncalibrated.

If the paper is right (Pith's inferences, not the authors')

Embodied self-improvement claims should be audited with world-reset, artifact-only, matched-source protocols, because a memory that loads back in is not evidence that a capability was acquired.

On the reference agent, related experience actively hurt both tool probes, with TSR, WGP, and VMP all falling below no-prior after the reset, so more relevant experience cannot be assumed to help.

Matching environment steps does not match computation: related sources used more planner calls and earned more verified progress, yet still transferred nothing, so extra computation alone is not a transfer mechanism.

Early wood-collection and crafting bottlenecks mean most runs never reach later stages, so stage-level records, not final success, are required to see where capability formation is actually blocked.

The wood-species probe's only positive contrast disappearing when one seed is removed shows that five-seed paired estimates sit near the noise floor, and small reported transfer gains from similar setups should not be trusted without per-seed inspection.

Because no positive-control agent known to transfer was run on the same four conditions, the benchmark's sensitivity is uncalibrated: the null result says this memory artifact did not transfer under these settings, not that stored experience cannot transfer, and running a verified skill-reuse agent on the identical probes would settle which reading is right.

The single-seed dependence of the wood-species contrast suggests that at five matched seeds the protocol can only confirm large effects, so users of the benchmark should either raise the seed count or pre-register an effect-size threshold before declaring transfer.

The stage-attrition distribution implies the meaningful transfer signal lives in the short probes, not the long chains, and future comparisons of agents should weight probe results rather than chain averages.

The same four-condition, artifact-only boundary could be applied to other persistence mechanisms such as model updates, skill libraries, or prompt caches, quantifying which artifact types actually change later behavior, which the current reference study does not test.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one world and transfer to later tasks. A saved memory or successful final action does not by itself establish that a reusable capability was acquired; evaluation must separate capability change from carried-over resources, unequal source exposure, and early execution failures. We construct MC-EvoBench as a controlled Minecraft benchmark with a fixed catalog of 19 registered evolution chains, including 12 primary evaluation chains and seven calibration or probe chains, together with three graph-aligned paired probe families for transfer. The benchmark represents each task as a task-state graph, defines stage-level environment verifiers, assigns matched source and target roles, resets the target world, and permits only a declared agent artifact to cross the source-target boundary. For each probe, no-prior, related, unrelated, and shuffled source conditions are evaluated under matched environment-step budgets and five seeds; we report TSR, WGP, and VMP and retain provenance records. The benchmark reveals a consistent direct-capability gap between controller and planner-backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood-species gain hinges on one seed, and all intervals include zero. Stage audits further show that early wood-collection and crafting bottlenecks often leave later capabilities unobserved. MC-EvoBench thus offers a controlled testbed for durable, transferable embodied capabilities. Code and data are available at https://github.com/xzw-ustc/mc-bench-open-source

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-12

The authors' main claim, as the classifier picked it out 0.71

“The benchmark reveals a consistent direct-capability gap between controller and planner-backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood-species gain hinges on one seed, and all intervals include zero.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.76 · human participants 0.03 · controlled comparison 0.88 · randomized assignment 0.11 · states quantitative result 0.81 · compares with prior results 0.70 · mentions code or data availability 0.99

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one world and transfer to later tasks. background 0.98 0.05 0.04 asserted 1.00
A saved memory or successful final action does not by itself establish that a reusable capability was acquired; evaluation must separate capability change from carried-over resources, unequal source exposure, and early execution failures. background 0.49 0.12 0.08 asserted 1.00
We construct MC-EvoBench as a controlled Minecraft benchmark with a fixed catalog of 19 registered evolution chains, including 12 primary evaluation chains and seven calibration or probe chains, together with three graph-aligned paired probe families for transfer. method 0.69 0.95 0.04 asserted 0.99
The benchmark represents each task as a task-state graph, defines stage-level environment verifiers, assigns matched source and target roles, resets the target world, and permits only a declared agent artifact to cross the source-target boundary. method 0.99 0.91 0.03 asserted 0.99
For each probe, no-prior, related, unrelated, and shuffled source conditions are evaluated under matched environment-step budgets and five seeds; we report TSR, WGP, and VMP and retain provenance records. method 1.00 0.95 0.04 asserted 0.94
The benchmark reveals a consistent direct-capability gap between controller and planner-backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood-species gain hinges on one seed, and all intervals include zero. result 1.00 0.95 0.60 asserted 1.00
Stage audits further show that early wood-collection and crafting bottlenecks often leave later capabilities unobserved. result 0.90 0.92 0.07 asserted 0.83
MC-EvoBench thus offers a controlled testbed for durable, transferable embodied capabilities. contribution 0.55 0.60 0.45 asserted 0.99
Code and data are available at https://github.com/xzw-ustc/mc-bench-open-source contribution 0.87 0.53 0.03 asserted 0.98

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 0.93

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-12: 1 request, 7,057 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (3)
Figure 1 of the paper
Figure 1. Evaluation mechanism of MC-EvoBench. A conventional benchmark records only
Figure 2 of the paper
Figure 2. Benchmark design of MC-EvoBench. Panel (a) follows a paired evaluation from source in
Figure 3 of the paper
Figure 3. Failure-stage and per-seed diagnostics. (a) Runs, out of 42 attempts per configuration,

Figures are the authors', reproduced from the paper.

References (21)
  1. [2] Xingze Gao, Chuanrui Hu, Hongda Chen, Pengfei Yao, Zhao Wang, Yi Bai, Zhengwei Wu, Yunyun Han, Xiaofeng Cong, Jie Gui, et al. Evoagentbench: Benchmarking agent self-evolution via ability transfer.arXiv preprint arXiv:2607.05202, on PithPith's reviewarXiv
  2. [7] Chenghao Li, Jun Liu, Songbo Zhang, Huadong Jian, Hao Ni, Lik-Hang Lee, Sung-Ho Bae, Guo- qing Wang, Yang Yang, and Chaoning Zhang. Experience transfer for multimodal llm agents in minecraft game.arXiv preprint arXiv:2604.05533, on PithPith's reviewarXiv
  3. [9] Yulin Luo, Chun-Kai Fan, Menghang Dong, Jiayu Shi, Xiangju Mi, Mengdi Zhao, Bo-Wen Zhang, Cheng Chi, Jiaming Liu, Gaole Dai, et al. Robobench: A comprehensive evaluation benchmark for multimodal large language models as embodied brain.arXiv preprint arXiv:2510.17801, on PithPith's reviewarXiv
  4. [10] Fei Ni, Min Zhang, Pengyi Li, Yifu Yuan, Lingfeng Zhang, Yuecheng Liu, Peilong Han, Longxin Kou, Shaojin Ma, Jinbin Qiao, et al. Embodied arena: A comprehensive, unified, and evolving evaluation platform for embodied ai.arXiv preprint arXiv:2509.15273, on PitharXiv
  5. [11] Junyeong Park, Junmo Cho, and Sungjin Ahn. Mrsteve: Instruction-following agents in minecraft with what-where-when memory. InInternational Conference on Learning Representations, vol- ume 2025, pp. 16922–16960, on Pith
  6. [12] Zhe Ren, Yimeng Chen, Dandan Guo, Guowei Rong, Tonghui Li, RB Xiong, Qingfeng Lan, Wenyi Wang, Li Nanbo, Yibo Yang, et al. Self-improvements in modern agentic systems: A survey. arXiv preprint arXiv:2607.13104, on PithPith's reviewarXiv
  7. [13] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, on PithPith's reviewarXiv
  8. [14] Zhaoyang Wang, Canwen Xu, Boyi Liu, Yite Wang, Siwei Han, Zhewei Yao, Huaxiu Yao, and Yux- iong He. Agent world model: Infinity synthetic environments for agentic reinforcement learning. arXiv preprint arXiv:2602.10090, on PithPith's reviewarXiv
  9. [15] Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, et al. Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models.IEEE Transactions on Pattern Analysis and Machine Inte... on Pith
  10. [16] Zihao Wang, Muyao Li, Kaichen He, Xiangyu Wang, Zhancun Mu, Anji Liu, and Yitao Liang. Openha: A series of open-source hierarchical agentic models in minecraft.arXiv preprint arXiv:2509.13347, on PitharXiv
  11. [17] 11 Preprint Shuhan Xue, Zixin Ding, Yichen Shen, Yinjie Wang, Zhenfei Yin, Yingcheng Wu, Yuxin Chen, Mengdi Wang, and Ling Yang. Past-bench: Benchmarking the foundations of recursive self- improvement in personal agents.arXiv preprint arXiv:2608.04003, on PithPith's reviewarXiv
  12. [18] Junhao Zheng, Xidi Cai, Qiuke Li, Duzhen Zhang, ZhongZhi Li, Yingying Zhang, Le Song, and Qianli Ma. Lifelongagentbench: Evaluating llm agents as lifelong learners.arXiv preprint arXiv:2505.11942, on PithPith's reviewarXiv
  13. [19] Xinyue Zheng, Haowei Lin, Kaichen He, Zihao Wang, Zilong Zheng, and Yitao Liang. Mcu: An evaluation framework for open-ended game agents.arXiv preprint arXiv:2310.08367, on PithPith's reviewarXiv
  14. [31] Fixing the world preserves the declared spawn placement and local resource layout across conditions. The resulting paired estimates characterize variation across agent seeds on these controlled instances. Run labels such asseed0identify records; numericalworld seedandagent see... on Pith
  15. [85] The parser filters subgoals containing “open,” “place,” or “access” followed by a space, rewrites punch/collect/gather to chop, and retains up to six subgoals. ExpeL instead requests a low-level action mapping and can hold that action across a configured number of environment ... on Pith
  16. [2019] William Hill, Ireton Liu, Anita De Mello Koch, Damion Harvey, Nishanth Kumar, George Konidaris, and Steven James. Mineplanner: A benchmark for long-horizon planning in large minecraft worlds.arXiv preprint arXiv:2312.12891, on PithPith's reviewarXiv
  17. [2022] Mingguang Chen, Licheng Wang, and Bo Qu. Recursive self-improvement in ai: From bounded self-refinement to autonomous research loops.arXiv preprint arXiv:2607.07663, on PithPith's reviewarXiv
  18. [2023] 10 Preprint Junchao Huang, Xinting Hu, Boyao Han, Shaoshuai Shi, Zhuotao Tian, Tianyu He, and Li Jiang. Memory forcing: Spatio-temporal memory for consistent scene generation on minecraft.arXiv preprint arXiv:2510.03198, on PitharXiv
  19. [2024] Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Weili Guan, Dongmei Jiang, Yaowei Wang, and Liqiang Nie. Optimus-3: Dual-router aligned mixture-of-experts agent with dual-granularity reasoning-aware policy optimization.arXiv preprint arXiv:2506.10357, 2025a. Zaijing Li, Yuquan... on PitharXiv
  20. [2025] William H Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, and Ruslan Salakhutdinov. Minerl: A large-scale dataset of minecraft demonstrations. arXiv preprint arXiv:1907.13440, on PithPith's reviewarXiv
show 1 more
  1. [2026] Junliang Guo, Yang Ye, Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, and Jiang Bian. Mineworld: a real-time and open-source interactive world model on minecraft.arXiv preprint arXiv:2504.08388, on PithPith's reviewarXiv
Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment