Pith.

Artificial Intelligence · arXiv:2610.36077 v2 · submitted 26 Oct 2026

The Trinity: Capability-Driven Recursive Self-Improvement for Multimodal Agents

Weidong Tang, Ruiheng Wang, Jingxuan Zhang, Hexin Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang

show 13 more authors

Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao, Kaipeng Zhang, Pengfei Zhou, Odin Zhang, Yang You

The paper says

“Multimodal intelligence can be viewed as human-guided co-evolution between worlds and agents through interaction. However, scaling this process faces two bottlenecks: constructing heterogeneous, verifiable environments is costly, and manual Harness iteration cannot keep pace with growing task diversity and complexity.”

more of the abstract

A key insight is that tasks with different modalities and surface forms can share the same underlying capability requirements. We therefore introduce The Trinity, a recursive self-improvement framework coupling world generation with Harness evolution through three roles: Creator, Explorer, and Evolver. Drawing on verified execution trajectories, Evolver shifts the basis of task organization from surface categories to shared requirements, reorganizing the original task tree into a hierarchical capability tree. We introduce Capability-Driven Real2Sim2Real: for heterogeneous tasks at the same node, Creator abstracts their common requirements into simplified, verifiable simulated worlds for capability learning, while evaluation on original tasks measures transfer. Along this capability tree, Explorer progressively evolves reusable capability modules through selective inheritance, forming a Harness tree whose validated updates accumulate across iterations within a fixed topology. The resulting Harnesses specialize in different capabilities, motivating a Capability-Aware Harness Router that uses public task information to select compatible module configurations from the tree. Our evolved Harnesses improve mean performance by 34 points across 28 training benchmarks and 26 points across 18 unseen benchmarks, including two real-world embodied tasks.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 28 Oct 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Capability-driven RSI paper is empirically strong but has a fixed tree ordering issue that needs addressing. This line is wrong

2 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at Section 3.3 / Appendix B / Table 3

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The capability tree is constructed using a single fixed prefix order (c, d, a, b, e, f), claimed to be based on domain breadth. The order determines which capability combinations form nodes, which in turn drives selective inheritance (Section 3.5) and routing (Section 3.6).

more of objection 1

No sensitivity analysis or ablation on alternative orderings is provided. Table 3 shows that the capability tree (Sim) outperforms flat capability-domain groupings by 7.83 points on the Unseen split, but this comparison uses only one ordering. Because the Trie merges common prefixes, different orderings create different node groupings that could affect module evolution and transfer. The paper should test at least one alternative ordering (e.g., reversed, or random) to demonstrate that the benefit stems from the concept of capability combination rather than from the specific ordering chosen. Without this analysis, the central claim that capability-driven organization outperforms task-driven specialization is not fully supported.

The paper says, at Section 4.6 / Table 6

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.96):

“These comparisons evaluate complete Harness–skill configurations and do not isolate the effects of module evolution and skill preparation.”

Pith says: Objection 2 of 2 · major

The embodied transfer experiments evaluate complete Harness–skill configurations and do not isolate the effects of module evolution from skill preparation, as the authors state in the main text. The paper should either perform an ablation that compares evolved modules with Base modules using the same skill preparation, or explicitly quantify the contribution of each component. Since the authors acknowledge the limitation, this is a request for a concrete additional experiment rather than a revelation of a hidden flaw.

4 minor objections

The paper says, at Author list

Pith says: minor objection 1 of 4

Contributors are listed without commas between names in several entries (e.g., 'Weidong Tang1,2 Ruiheng Wang3 Jingxuan Zhang4 Hexin Wang1'). Please add commas or semicolons for clarity.

The paper says, at Page 1, abstract area

Pith says: minor objection 2 of 4

The text at the top of the first page includes the string '/gl⌢beProject Page/githubGitHub' which appears to be a formatting artifact from a template. Please clean this up.

show 2 more

The paper says, at Table 4

Pith says: minor objection 3 of 4

The table caption uses 'Cap.' and 'Mem.' as column headers. Consider spelling these out as 'Cap. Defs.' and 'Memory' or adding a footnote. Also ensure that 'Tok.' is defined in the caption (it is stated as 'total inference tokens (M)' in the following paragraph, but the table itself should be self-contained).

The paper says, at Section 3.4

Pith says: minor objection 4 of 4

The Harness tree construction paragraph refers to 'host-registered bindings' and 'typed input–output assertions' without any example. A brief concrete illustration in the main text (not just in Appendix C) would help readability.

What would settle it

Re-run the Iteration 1 comparison in Table 3 with at least two alternative capability orders: (a,b,c,d,e,f) alphabetical and (f,e,d,c,b,a) reversed, using the same experimental setup (module-only, simulated worlds). Compare the Capability Tree (Sim) Train and Unseen scores to the original. If either alternative order leads to a drop of more than 2 points relative to the original on either split, the ordering choice is a significant factor and the robustness claim weakens.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 28 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 37 s · 31,914 tokens read, 4,341 written
  3. Editor wrote the verdict above 26 s · 32,797 tokens read, 2,505 written
  4. Referee major revision: 2 major objections, 4 minor 32 s · 32,327 tokens read, 3,076 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The Trinity paper tackles a real problem: how to scale agent harness evolution across heterogeneous multimodal tasks. The key insight is to organize tasks by shared capability requirements rather than surface form, building a capability tree and using simulated worlds (Real2Sim2Real) to train reusable modules. The empirical results are broad and convincing: 34-point average gain on 28 training benchmarks, 26 points on 18 unseen, with transfer to two real-robot tasks. The ablation studies (modules vs skills, different organizations, routing variants) are thorough and tell a consistent story.

What is genuinely new is the combination of capability tree induction, simulated world generation for capability learning, selective module inheritance, and a capability-aware router. Individual components exist, but the integrated framework is novel. The paper is also well-written and documents its design choices.

That said, there are notable soft spots. The most significant is the fixed capability tree ordering. The tree is built from training tasks using a fixed prefix order (c, d, a, b, e, f). The authors state the order implies no causal dependencies, but different orderings would create different node groupings, affecting module inheritance and ultimately harness performance. No sensitivity analysis is provided. This is a real concern, as the paper's central claim—capability-driven organization outperforms task-driven specialization—could be sensitive to this single design choice. The stress-test note on this point is on target.

Second, the capability categorization process uses LLM agents and human review. It is not fully automated, and reproducibility is limited given the proprietary backbone. The authors acknowledge benchmark construction challenges but not the categorization pipeline's opacity.

Third, the router evaluation uses an oracle upper bound. The full self-routing configuration comes close (within ~2 points), but the paper does not test degradation under noisier capability estimates or when the tree misses a combination for an unseen task.

Despite these issues, the paper deserves a serious referee. The empirical evidence is strong, the framework is well-motivated, and the claims are largely supported. The main flaw is fixable: test alternative orderings. I would conditionally accept, requiring that sensitivity analysis.

For our reading group, yes—lots of design decisions to debate. I would cite if the ordering issue is resolved. The paper shows clear, honest thinking.

Send to peer review with a request to address the tree ordering sensitivity.

The strongest objection

The capability tree is constructed using a fixed prefix order (c, d, a, b, e, f), claimed to be based on domain breadth (Section 3.3, Appendix B). This order determines which capability combinations form nodes, thereby influencing selective inheritance patterns (Section 3.5) and ultimately the set of available harnesses for routing (Section 3.6). The paper provides no analysis of how sensitive the results are to this ordering. Table 3 shows that the capability tree (Sim) outperforms flat capability-domain groupings by 7.70 points on Train and 7.83 points on Unseen (module-only). However, this comparison uses only one ordering. If an alternative ordering (e.g., reversed or random) leads to significantly different performance, the central claim that capability-driven organization outperforms task-driven specialization may not be robust. The paper asserts the order implies no causal dependencies, but the tree's prefix merging property means that different orders create different node groupings, which could affect module evolution and transfer. Without testing alternative orderings, it is unknown whether the benefit comes from the tree concept or from the specific ordering chosen.

The reader's weakest assumption concerned generalization of the capability tree to unseen tasks. While that is important, the paper already provides positive results on 18 unseen benchmarks, suggesting the tree does generalize. A more subtle but load-bearing assumption is that the fixed prefix order used to build the tree is incidental; if the order actually matters for performance, then the claim that capability-driven organization is superior may be contingent on an arbitrary choice. The paper does not test this, and without such a test the central conclusion is less robust. The concrete test would settle this by showing whether the results hold across different orderings. The verdict remains CONDITIONAL because the paper already has a conditional rating; this test would strengthen the case for acceptance.

The referee's report

If the findings are robust, the work demonstrates a substantial leap in cross-task generalization for agent harnesses, shifting the organizing principle from task-specific optimization to shared capability requirements. The scale of the evaluation (28 training, 18 unseen, two real-robot deployments) and the consistency of the iterative gains (nine additional Train points after the first iteration) make this a potentially important empirical contribution to the field of recursive self-improvement and agentic systems. The separation of harness evolution from model parameter updates isolates the role of control-flow organization in agent performance.

Pith's reading of the paper (a machine's summary, not the authors')

This paper introduces The Trinity, a framework that organizes heterogeneous multimodal tasks by their shared underlying capability requirements rather than by surface task type. It builds a hierarchical capability tree from verified execution trajectories, uses it to generate simplified simulated worlds for training (Capability-Driven Real2Sim2Real), and evolves a tree of specialized reusable harness modules through selective inheritance and local refinement. A capability-aware router then selects the best harness for a given task from public information. The authors claim that this capability-driven approach outperforms task-specific or global harness evolution, achieving a 34-point mean improvement across 28 training benchmarks and a 26-point gain across 18 unseen benchmarks, including two real-world embodied robot tasks, all with a frozen backbone model.

What carries the argument. The key machinery is the hierarchical capability tree, which groups tasks by shared requirements derived from verified trajectories, and the concurrent evolution of two harness trees—one for exploration (Explorer) and one for world creation (Creator)—aligned with the capability tree's prefix structure. Capability-Driven Real2Sim2Real abstracts common requirements into simplified training worlds, while selective inheritance and local refinement accumulate validated module updates across iterations within a fixed topology. A Capability-Aware Harness Router then uses public task information and retrieved execution experience to select the appropriate branch of the Explorer harness tree.

Core claim. The central discovery is that organizing agent harness evolution around shared capability requirements—rather than around individual task categories or a single global harness—enables stronger cross-task transfer and sustained iterative improvement. By first inducing a capability tree from training-task trajectories, then generating simplified, verifiable simulation worlds that isolate each capability, and finally evolving modular harness configurations along the tree's nodes, the framework produces specialized harnesses that generalize to unseen tasks and even to real-world embodied settings. The authors show that the capability tree provides a semantic basis for a router to select compatible harness modules, and they demonstrate that adding or removing capability-relevant modules causally impacts performance.

The premise it rests on. The capability tree is built solely from training-task trajectories and remains fixed; if an unseen task requires a combination of capabilities absent from the training-induced tree, the router may lack an appropriate harness branch, and the system has no mechanism to expand the tree.

If the paper is right (Pith's inferences, not the authors')

If this approach is correct, future agent systems could reduce manual harness engineering by automatically inferring capability requirements from task descriptions and trajectories.

The capability tree could serve as a reusable ontology for transferring harness components across different backbone models or domains.

Capability-aware routing suggests a general paradigm for model selection: map tasks to a capability space rather than to benchmarks, and match configurations accordingly.

The Real2Sim2Real loop demonstrates that simplified simulation worlds, focused on decision logic rather than visual fidelity, can suffice for acquiring transferable capabilities.

The capability tree, once induced from training tasks, implicitly defines a form of task embedding that might be compared to or combined with learned task representations in meta-learning.

The paper's framework stops at the harness level, but the same capability-driven logic could extend to model parameter updates, where different capability modules correspond to different fine-tuning strategies or adapter weights.

The fixed capability tree limits adaptability to genuinely novel capability combinations; a dynamic tree that expands as new tasks are encountered would be a natural extension for continued open-ended learning.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Multimodal intelligence can be viewed as human-guided co-evolution between worlds and agents through interaction. However, scaling this process faces two bottlenecks: constructing heterogeneous, verifiable environments is costly, and manual Harness iteration cannot keep pace with growing task diversity and complexity. A key insight is that tasks with different modalities and surface forms can share the same underlying capability requirements. We therefore introduce The Trinity, a recursive self-improvement framework coupling world generation with Harness evolution through three roles: Creator, Explorer, and Evolver. Drawing on verified execution trajectories, Evolver shifts the basis of task organization from surface categories to shared requirements, reorganizing the original task tree into a hierarchical capability tree. We introduce Capability-Driven Real2Sim2Real: for heterogeneous tasks at the same node, Creator abstracts their common requirements into simplified, verifiable simulated worlds for capability learning, while evaluation on original tasks measures transfer. Along this capability tree, Explorer progressively evolves reusable capability modules through selective inheritance, forming a Harness tree whose validated updates accumulate across iterations within a fixed topology. The resulting Harnesses specialize in different capabilities, motivating a Capability-Aware Harness Router that uses public task information to select compatible module configurations from the tree. Our evolved Harnesses improve mean performance by 34 points across 28 training benchmarks and 26 points across 18 unseen benchmarks, including two real-world embodied tasks.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-28

The authors' main claim, as the classifier picked it out 0.99

“Our evolved Harnesses improve mean performance by 34 points across 28 training benchmarks and 26 points across 18 unseen benchmarks, including two real-world embodied tasks.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.21 · human participants 0.14 · controlled comparison 0.23 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.24 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Multimodal intelligence can be viewed as human-guided co-evolution between worlds and agents through interaction. background 0.99 0.08 0.03 asserted 0.81
However, scaling this process faces two bottlenecks: constructing heterogeneous, verifiable environments is costly, and manual Harness iteration cannot keep pace with growing task diversity and complexity. gap 0.55 0.14 0.03 asserted 1.00
A key insight is that tasks with different modalities and surface forms can share the same underlying capability requirements. background 0.61 0.16 0.07 asserted 0.99
We therefore introduce The Trinity, a recursive self-improvement framework coupling world generation with Harness evolution through three roles: Creator, Explorer, and Evolver. contribution 0.95 0.86 0.10 asserted 1.00
Drawing on verified execution trajectories, Evolver shifts the basis of task organization from surface categories to shared requirements, reorganizing the original task tree into a hierarchical capability tree. method 0.97 0.87 0.09 asserted 1.00
We introduce Capability-Driven Real2Sim2Real: for heterogeneous tasks at the same node, Creator abstracts their common requirements into simplified, verifiable simulated worlds for capability learning, while evaluation on original tasks measures transfer. method 0.86 0.91 0.08 asserted 0.99
Along this capability tree, Explorer progressively evolves reusable capability modules through selective inheritance, forming a Harness tree whose validated updates accumulate across iterations within a fixed topology. method 0.98 0.85 0.06 asserted 1.00
The resulting Harnesses specialize in different capabilities, motivating a Capability-Aware Harness Router that uses public task information to select compatible module configurations from the tree. method 0.80 0.89 0.08 asserted 0.98
Our evolved Harnesses improve mean performance by 34 points across 28 training benchmarks and 26 points across 18 unseen benchmarks, including two real-world embodied tasks. result 1.00 0.96 0.65 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.92

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-28: 1 request, 6,982 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (73)
Figure 1 of the paper
Figure 1. Overview of The Trinity. Creator, Explorer, and Evolver connect multimodal task-space expansion (left) with capability-driven Harness evolution (right) through verified feedback. The lower panels summarize tree construction, world exploration and creation, and capability-aware routing; the charts report mean and benchmark-level Train/Unseen performance across RSI iterations. arXiv:2610.36077v1 [cs.AI] 26 Oct 2026
Figure 2 of the paper
Figure 2. Detailed pipeline of The Trinity. Three stages build task, capability, and Harness trees. Worlds are verified and screened; Harnesses evolve through selective inheritance and local refinement. Routing selects one compatible Harness and stores verified feedback for later tasks. 3.1. Framework Overview The Trinity couples environment generation and Harness evolution with a frozen backbone (
Figure 3 of the paper
Figure 3. Capability-driven world creation at node d. World Creation. Once the capability tree is built, each node is linked to training benchmarks through shared requirements. Although the four tasks in
Figure 4 of the paper
Figure 4. Training cost and Train performance across RSI iterations. Finding 1. Capability-driven organization outperforms task-driven specialization. A capability-centric perspective gives rise to a capability tree, where capability-driven Real2Sim2Real achieves stronger capability transfer and lower inference costs with fewer training environments, demonstrating that less is more. 10
Figure 5 of the paper
Figure 5. Benchmark-level Harness evolution. Left: Train; right: Unseen. Radial bars show Base and Iterations 1–4; *Agent-built from web-searched sources. 4.3. Recursive Self-Improvement Benchmark mean Native Train Base Train 35 50 65 80 0 1 2 3 4 73.2 68.8 Unseen 40 50 60 70 Codex 47.5 DeepSeek 46.2 0 1 2 3 4 64.1 Mean performance Iteration
Figure 6 of the paper
Figure 6. RSI mean performance across benchmarks. Horizontal lines show Codex and DeepSeek reference means. We track the RSI loop (Section 3.5) from Base (It￾eration 0) through Iteration 4 to test whether im￾provements continue after the first update. With the benchmark split, capability requirements, tree topol￾ogy, and backbone fixed, Evolver refines modules from training feedback, Creator renews or reuses curricula, and Explorer collects new trajectories. Evaluation
Figure 7 of the paper
Figure 7. Capability–protocol correlations across six protocols. Red is positive; blue is negative. Capability–Module Correlations. Fig￾ure 7 reports Iteration-4 correlations for 29 modules. Rows abbreviate six shared capabilities: PST, perception and state tracking; HCI, hypothesis and counter￾part inference; CFR, constraint checking and feedback repair; GVT, goal commit￾ment and value trade-off; VEA, video ev￾idence acquisition; and VC, video com￾prehension. Columns denote Perception, Memory, Inference, Planning, Orchestra￾tion, and Loop (Section 2.2), with coeffi￾cients in [−1, 1]. The…
Figure 8 of the paper
Figure 8. Embodied Transfer of Capability-Evolved Harnesses. Real scenes, simulators, and robot trials (SOON E01; CaP-X C-01). The circle marks the revealed cube. Based on their capability requirements, we map the embodied tasks to the 𝑑→𝑎 node of the capability tree and adopt the Harness configurations evolved there. We then use Real2Sim2Real to learn task-specific skills (
Figure 9 of the paper
Figure 9. Task tree. The 44 benchmarks are grouped into five domains and 15 subclasses. Training benchmarks take the colour of their domain and unseen benchmarks a lighter shade of it; SOON and CaP-X show their simulation and real-robot versions; * marks environments rebuilt from published rules. Candidate benchmarks were collected through agent web search over five types of source: research benchmarks released with code or data, public AI competitions, human contests and testing standards, published games and puzzle communities, and professional practice. Because a multimodal agent must…
Figure 10 of the paper
Figure 10. Flat capability tree. Bricks keep the colours of
Figure 11 of the paper
Figure 11. Capability tree. Each node is one combination of capabilities, read along the branch from the root, and each benchmark sits at the node of its combination. Bricks keep the colours of
Figure 12 of the paper
Figure 12. Explorer Harness tree. Every node Harness starts from H0 and combines the modules it selectively inherits with its own local modules. Brick colours mark capability families, and labels give module IDs. C.2. Creator Shared Harness Creator builds the simplified training worlds of each node with a coding harness, created per node and iteration, with a workspace for the world and an evidence directory outside it. Its Base Harness at the first iteration is H0 extended with the pre-RSI modules listed below. Within each iteration, Creator visits the nodes top-down: the children of the…
Figure 13 of the paper
Figure 13. Creator shared Harness. The root is the Base Harness H0 with the pre-RSI modules. Each node shows the modules proposed or revised at that node. The modules of all nodes are merged and deduplicated into one shared library of 25 modules. Brick colours group the library by function. 32
Figure 14 of the paper
Figure 14. AdaPlanBench (train)
Figure 15 of the paper
Figure 15. Among Us (train) 35
Figure 16 of the paper
Figure 16. ARC-AGI-3 (train)
Figure 17 of the paper
Figure 17. BoxingGym (train)
Figure 18 of the paper
Figure 18. Chess Tactics (train)
Figure 19 of the paper
Figure 19. Codenames (train) 36
Figure 20 of the paper
Figure 20. Colonel Blotto (train)
Figure 21 of the paper
Figure 21. DiscoverPhysics (train)
Figure 22 of the paper
Figure 22. Eleusis (train)
Figure 23 of the paper
Figure 23. Euro NCAP AEB (train) 37
Figure 24 of the paper
Figure 24. GLEE Persuasion (train)
Figure 25 of the paper
Figure 25. Master Keying (train)
Figure 26 of the paper
Figure 26. Mastermind (train)
Figure 27 of the paper
Figure 27. Memory Maze (train) 38
Figure 28 of the paper
Figure 28. Micromouse (train)
Figure 29 of the paper
Figure 29. Pallet Rack Inspection (train)
Figure 30 of the paper
Figure 30. PInVerify (train)
Figure 31 of the paper
Figure 31. Pooltool Billiards (train) 39
Figure 32 of the paper
Figure 32. puzz.link / Penpa+ (train)
Figure 33 of the paper
Figure 33. PuzzleScript (train)
Figure 34 of the paper
Figure 34. Saboteur (train)
Figure 35 of the paper
Figure 35. TextQuests (train) 40
Figure 36 of the paper
Figure 36. The Long Dark (train)
Figure 37 of the paper
Figure 37. Video-Index (train)
Figure 38 of the paper
Figure 38. WCA Cube Solving (train)
Figure 39 of the paper
Figure 39. WILT (train) 41
Figure 40 of the paper
Figure 40. ActiView (unseen)
Figure 41 of the paper
Figure 41. ALE-Bench (unseen)
Figure 42 of the paper
Figure 42. BuildArena (unseen)
Figure 43 of the paper
Figure 43. Coup (unseen) 42
Figure 44 of the paper
Figure 44. Crafter (unseen)
Figure 45 of the paper
Figure 45. DiG-bench (unseen)
Figure 46 of the paper
Figure 46. Entity Deduction Arena (unseen)
Figure 47 of the paper
Figure 47. Hanabi (unseen) 43
Figure 48 of the paper
Figure 48. Heads-Up NL Hold’em (unseen)
Figure 49 of the paper
Figure 49. MiniHack Boxoban (unseen)
Figure 50 of the paper
Figure 50. MiniHack Maze / Corridor (unseen)
Figure 51 of the paper
Figure 51. NetHack (unseen) 44
Figure 52 of the paper
Figure 52. Pencil Puzzle Bench (unseen)
Figure 53 of the paper
Figure 53. PHYRE (unseen)
Figure 54 of the paper
Figure 54. SciGym (unseen)
Figure 55 of the paper
Figure 55. Video-MME-v2 (unseen) 45
Figure 56 of the paper
Figure 56. SOON (embodied, simulation)
Figure 57 of the paper
Figure 57. CaP-X (embodied, simulation) E.2. Node Training Levels At every capability node, Evolver designs short training levels from the node’s requirement, and Creator implements and revises them under the Creator Harness of Appendix C.2; each level is a simulation world with a 3D scene, public rules, a fixed action budget, and a scorer, and one level per node and iteration gives 44 levels. Because e and e→f form a separate path for video understanding, Video-Index, the only training benchmark with video input, feeds both nodes. Figures 58–68 show how each node evolves: on the left are…
Figure 58 of the paper
Figure 58. c: Gate Planning & Reachability 47
Figure 59 of the paper
Figure 59. c→d: Value & Goal Commitment 48
Figure 60 of the paper
Figure 60. c→d→a: Execution & Revision Memory 49
Figure 61 of the paper
Figure 61. c→a: Identity, Order & Memory 50
Figure 62 of the paper
Figure 62. d: Evidence-Gated Commitment 51
Figure 63 of the paper
Figure 63. d→a: Exploration & Routing 52
Figure 64 of the paper
Figure 64. d→b: Hypothesis Tracking & Probing 53
Figure 65 of the paper
Figure 65. a: Evidence & Inspection 54
Figure 66 of the paper
Figure 66. b: Signaling & Counterpart Decisions 55
Figure 67 of the paper
Figure 67. e: Video Evidence Acquisition 56
Figure 68 of the paper
Figure 68. e→f: Video Events & Frame Association 57
Figure 69 of the paper
Figure 69. Embodied environments. Real scenes appear above their simulated counterparts, with SOON on the left and CaP-X on the right. Figures 70–73 compare H0 and H(4) under the same task settings. In SOON, the robot must judge the distance to the target desk from head-camera images alone. H0 looks around and reaches out as 58
Figure 70 of the paper
Figure 70. SOON with the Base Harness. Rows show simulation and real-robot views from the fixed office camera, followed by the head camera. The robot walks from the side room along the aisle, looks around, and turns towards the desk beside the fan, but reaches out before the desk is within reach in both settings
Figure 71 of the paper
Figure 71. SOON with the evolved Harness. Rows follow
Figure 72 of the paper
Figure 72. CaP-X with the Base Harness. Rows show head, side, and wrist cameras in simulation, followed by the real robot. In both settings the robot manipulates other cups without inspecting the target cup and incorrectly reports that no cube is present
Figure 73 of the paper
Figure 73. CaP-X with the evolved Harness. Rows follow

Figures are the authors', reproduced from the paper.

Cited by 1 Pith paper

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment