Pith.

Computation and Language · arXiv:2610.20294 v1 · submitted 14 Oct 2026 · 32 references

MMLA-RSI: Recursive Self-Improvement through RTT and RTMA

Junyi Zou, Avrova Donz

The paper says

“An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains.”

more of the abstract

We propose MMLA-RSI, a recursive self-improvement scheme with two implementations in Memory-Mediated Learning Architecture: Reasoning-Time Training (RTT) changes the policy state Φ, while Reasoning-Time Memory Adaptation (RTMA) changes authoritative memory M. Each route also makes its adaptation method an object of improvement. A current numerical updater trains the policy proposing its successor; a current memory manager chooses the records available to the process proposing its successor. An independently admitted successor takes over the same role in the next generation. We give bounded executable rule languages, shared task/proposal interfaces, and one algorithm for either route or their composition. The RTT instance changes scalar schedules in a clipped-momentum update. The RTMA instance chooses NULL or a complete-row write with possible eviction. A fixed interpreter, trusted committer, and independent evaluator bound what can change. State versions, read closures, and conditional properties describe causal execution, accepted transitions, and finite work. Existing numerical-component studies motivate the construction. In four-candidate program selection, retaining LARC feedback updates reduces expected error by 24.65 and 36.65 percentage points for two trained initializations. In chronological continuous-integration replay, feedback binding is positive while retention increases loss. Completed SQL studies improve substantially over an unadapted substrate but do not establish the required additional online benefit. These results characterize their declared components. The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 15 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A careful, honest specification of a bounded RSI construction whose main theorem is conditional on an audit procedure the paper does not actually construct. This line is wrong

2 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 32 references say.

The paper says, at Section 6, Proposition 6.2 and Algorithm 1 step 7

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The admission rule in Eq. (18) invokes audit intervals U_k(eta') and L_k(eta_k), and step 7 of Algorithm 1 says only 'Apply Equation 18 and resource checks,' but no construction of these intervals appears anywhere in the manuscript. No interval family, alpha-allocation scheme, or audit-block size is specified, and the paper's own caveat in Section 6, that reusing an ordinary bootstrap …

more of objection 1

… interval after repeatedly selecting on the same audit data does not establish simultaneous coverage, explicitly disclaims the bootstrap intervals used in its component studies. Because the telescoping bound and the 'at most floor(R(eta_0)/delta) accepted edges' guarantee hold only on the simultaneous-coverage event, the deciding step of the algorithm that the abstract describes as 'one algorithm' is not executable as written. This is a completeness gap in the central deliverable, not a flaw in the conditional mathematics: the premise of Proposition 6.2 is explicit, and the sketched recipe (allocate conditional error probabilities summing to alpha over the candidate-parent comparisons and use fresh audits with valid conditional intervals) is workable, since the generation cap K bounds the number of comparisons by K and hence the number of interval constructions by 2K. The revision should either instantiate a concrete procedure, for example splitting alpha over the 2K intervals and using Hoeffding or permutation intervals on fresh disjoint audit tasks of a stated size, or formally restate the deliverable as parameterized by an external audit oracle satisfying the stated coverage assumption.

The paper says, at Section 6, Eq. (18) and Section 5.8

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

Even granting a valid interval procedure, the manuscript gives no quantitative feasibility analysis for the admission margin delta, which the paper itself lists as a free parameter. The central promise of Proposition 6.2, that accepted chains strictly reduce risk and contain at most floor(R(eta_0)/delta) edges, is vacuous in practice if delta is chosen larger than the interval width that finite …

more of objection 2

… audit blocks can support, since no candidate can then ever be admitted; the paper acknowledges that existence of an admissible successor is an empirical question, but a purely statistical feasibility statement (for example, how interval width scales with audit-block size and task variance, and what delta values are thereby achievable) is within the scope of a construction paper and is currently absent. The revision should add such an analysis, or explicitly label Proposition 6.2 as an oracle-conditional guarantee whose practical reach is untested.

6 minor objections

The paper says, at Section 5.1

Pith says: minor objection 1 of 6

The program is said to be 'the canonical prefix serialization of three scalar expression trees,' while the interpreter 'traverses trees in fixed left-to-right postorder'; please clarify how the prefix serialization relates to the postorder evaluation (for example, parse the serialization into a tree and then evaluate in postorder), since the current wording appears to mix the two orders.

The paper says, at Sections 5.2 and 5.3

Pith says: minor objection 2 of 6

The updater is defined in Section 5.1 through Eval_H(eta; ...), but Eq. (15) invokes U_eta_k directly; one sentence identifying U_eta with the Eval_H-based update would remove the notational gap for the reader.

show 4 more

The paper says, at Algorithm 1, step 7

Pith says: minor objection 3 of 6

The instruction 'Apply Equation 18 and resource checks' forward-references Section 6 without a pointer; adding a cross-reference such as '(defined in Section 6)' would improve readability.

The paper says, at Table 3

Pith says: minor objection 4 of 6

Adjacent rows mix percentage-point differences and half-Brier differences; the caption already warns about units, but a dedicated units column would prevent misreading when the table is cited in later discussion.

The paper says, at Section 2.2

Pith says: minor objection 5 of 6

The displayed TV inequality depends on the fact that the common-random-number coupling is a valid coupling of the two action kernels; one explicit sentence stating that the marginals of (F_Psi'(h,Z), F_Psi(h,Z)) are the two kernels would make the argument self-contained.

The paper says, at Section 6, after Proposition 6.2

Pith says: minor objection 6 of 6

A sentence on how the admission margin delta is intended to be fixed relative to audit-block width and the risk scale would clarify when Eq. (18) is practically satisfiable, given that too large a delta prevents any admission and too small a delta weakens the meaning of each accepted edge.

What would settle it

Complete the audit procedure: specify an interval family (e.g., Hoeffding or empirical-Bernstein intervals at level α/(2K) per risk, computed from paired fresh audit blocks of size n_a), allocate α over the at-most-K candidate–parent comparisons, and require each audit group to be used exactly once. Verify analytically or by simulation that simultaneous coverage holds with probability at least 1−α and that Algorithm 1 step 7 then becomes executable for the RTT, RTMA, and joint domains. Success upgrades Proposition 6.2 from a theorem about an unconstructed object to a guarantee of a concrete procedure; failure means the chain bound remains conditional on an uninstantiated event.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 15 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 2 min 35 s · 28,769 tokens read, 19,610 written
  3. Editor wrote the verdict above 16 s · 29,878 tokens read, 1,468 written
  4. Referee major revision: 2 major objections, 6 minor 2 min 53 s · 29,129 tokens read, 19,905 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Read MMLA-RSI. It is a specification report, and it knows it. What is actually new: two bounded executable rule languages (RTT updater trees, RTMA memory-manager trees), one algorithm covering either route or their composition, and an admission protocol that requires parent participation and fresh independent audit. The recursion idea itself is precedented—STOP, Darwin Gödel Machine—but the disciplined packaging is genuinely useful: fixed interpreter and committer bounds, read closures, explicit witness obligations, and separate accounting of functional use, task benefit, and discovery efficiency.

The best part is the honesty. The paper distinguishes replay disagreement from marginal distribution change, separates functional use from benefit, reports null and negative results (CI retention harms; both SQL online increments fail), and repeatedly says the component studies do not test recursive chains. The propositions are conditional and simple; Props 6.1 and 6.3 hold as stated.

The soft spot is real and central: Prop 6.2. The finite-admission guarantee—at most floor(R(eta0)/delta) accepted edges—rests on simultaneous coverage of audit intervals for every locked candidate and its parent. The paper itself says reusing an ordinary bootstrap interval after repeated selection does not establish this, and its only pointer is to allocate conditional error probabilities and use fresh audits with valid conditional intervals. No concrete interval family, allocation scheme, or audit block size is given. Algorithm 1 step 7 says to apply Equation 18 and resource checks, but Uk and Lk are never constructed. So the central guarantee is a theorem about an unbuilt object. The gap looks fillable—one comparison per generation, finite generations, losses bounded in [0,1], so a Bonferroni allocation over Hoeffding or empirical-Bernstein intervals plausibly works—but until a concrete procedure is specified and its coverage verified, the chain bound is conditional on an uninstantiated assumption.

Minor notes: novelty is moderate, since the recursion is already in STOP and DGM and the paper extends the authors' own MMLA/LARC line; that is fine, the contribution is the bounded instance. The component studies are the authors' own prior reports, but they are fixed-version, disclosed, and explicitly not promoted to chain evidence. Not a flaw.

Who gets value: anyone designing auditable self-improvement experiments or writing a spec for bounded updater/memory search. It deserves a serious referee; the referee's main job is to press for the audit construction or a revised proposition that does not carry that weight. Conditional acceptance is the right verdict. Send it to peer review, not the desk.

The strongest objection

The central claim is that Algorithm 1, with the two bounded rule languages and the independent admission protocol, is a complete recursive construction whose accepted chains are finite and strictly improve task risk. The load-bearing step is Proposition 6.2 (b448, b450): if 'audit intervals cover their risks simultaneously with probability at least 1−α' for every locked candidate and its parent, then each accepted edge reduces R(η) by at least δ, so at most floor(R(η_0)/δ) edges are accepted. Two conditions must hold for this proposition to carry the claim. First, the intervals U_k(η′) and L_k(η_k) in Equation 18 must be constructible from the audit block; Algorithm 1 step 7 says only 'Apply Equation 18 and resource checks,' so as written the decide point is not executable. Second, the coverage event must be a realizable operating characteristic. The paper acknowledges the hard part: 'Reusing an ordinary bootstrap interval after repeatedly selecting on the same audit data does not establish this assumption,' and it explicitly declines to promote its reported bootstrap intervals to sequential admission guarantees. What remains unresolved is the paper's only pointer: 'allocate conditional error probabilities to all candidate–parent comparisons, with their sum at most α, and use fresh audits with valid conditional intervals.' No α-allocation scheme, interval family, or audit-block size is given. The gap is likely fillable — at most one comparison per generation, at most K generations, paired fresh audit tasks, losses bounded in [0,1], so a Bonferroni allocation over Hoeffding or empirical-Bernstein intervals plausibly realizes the event — but until a concrete procedure is specified and its coverage verified, Proposition 6.2 is a theorem about an unconstructed object.

The paper is a specification report, and within its declared scope it is unusually careful. It separates functional use from task benefit, distinguishes replay disagreement from marginal distribution change (Equation 5), reports null and negative component results (CI retention harms; both SQL online increments fail their gates), and states plainly that no beneficial recursive chain has been observed and that the component studies characterize only their declared components (b460). Propositions 6.1–6.3 follow from their stated assumptions; I found no internal inconsistency in the causal-composition or finite-work arguments, and the paper's admission that ordinary bootstrap intervals do not establish simultaneous coverage is an honest delimitation rather than a hidden flaw. The single load-bearing concern is the one the reader identified, sharpened one step further: it is not merely that the coverage assumption is unproved, but that Algorithm 1's step 7 references audit intervals U_k and L_k that the report never defines constructively, so the decide point of the construction is not executable as written. The paper's only pointer — allocate conditional error probabilities and use fresh audits with valid conditional intervals — is a plausible skeleton: at most one candidate–parent comparison per generation, at most K generations, paired fresh audit blocks, and losses in [0,1] make a Bonferroni allocation over Hoeffding or empirical-Bernstein intervals a believable realization. Nothing in the text suggests the coverage event is impossible; the gap is incomplete specification, not demonstrated unsoundness. That is exactly the situation that warrants a CONDITIONAL verdict rather than ACCEPT or REJECT, so my read leaves the reader's verdict unchanged. If a concrete interval construction with verified coverage properties were supplied, the finite-admission guarantee would attach to a fully executable procedure and the verdict could move toward ACCEPT; conversely, if no such construction can be given within the paper's own bounds, the chain bound would be vacuous and the verdict would need to harden. The proposed concrete test — completing the audit construction and verifying its coverage — settles which of these cases holds.

The referee's report

Conditional on the propositions' premises, the core mathematics is sound: the telescoping argument in Proposition 6.2 is valid, Proposition 6.1 is a standard adapted-process induction, and Proposition 6.3 holds by construction. The paper's strengths are the precision of the specification: bounded grammars with serialization caps, deterministic tie-breaking, versioned state with read closures, explicit timing conditions in Eq. (3), a correct distinction between common-random-number replay disagreement and marginal action-kernel difference in Section 2.2, the separation of functional participation from task benefit, and unusually candid reporting of null and negative component results (SParC gain -1.75 points, CoSQL -0.979 points, CI retention G = -0.05337). If the specification is completed as stated, the field receives an auditable, executable template for recursive self-improvement in which accepted chains are finite by construction. What the paper does not deliver is an experiment on the recursive routes, which it explicitly defers, and a concrete realization of the audit-interval component that the admission rule depends on; the latter is the subject of the major comment below.

Pith's reading of the paper (a machine's summary, not the authors')

Recursive self-improvement is usually discussed as a hazard or an aspiration; this paper tries to make it a small, inspectable engineering problem. Its claim is that an agent can improve how it learns within a single unresolved problem through two bounded routes — updating the policy state (RTT) and committing retained experience (RTMA) — and that in each route the current method participates in producing the method that replaces it in the next generation. Around this loop the paper places a fixed interpreter, a trusted committer, an independent evaluator, and an admission rule that forces every accepted successor to lower task risk by at least a fixed amount, which bounds any admitted chain. The component experiments characterize the learning carrier — program selection shows retention cuts expected error by 24.65 and 36.65 percentage points, while the CI replay and SQL studies show feedback binding and task benefit can come apart — and the paper states plainly that no beneficial recursive chain has yet been observed and discovery efficiency is not yet evaluated.

What carries the argument. The construction is carried by the generation loop of Algorithm 1 together with the admission inequality $U_k(\eta')+\delta \le L_k(\eta_k)$ of Proposition 6.2. Within a generation, the parent rule participates in producing its successor: the RTT parent updates the proposal-policy state $\Phi_{\mathrm{prop}}$ through a score-function gradient (Equation 14) before the next candidate is sampled, and the RTMA parent commits the rows in $M_{\mathrm{prop}}$ that the next proposal reads (Equation 17). Boundedness comes from two small rule languages: an RTT program is a triple of scalar expression trees — step size, clipping threshold, momentum — over a fixed clipped-momentum template, and an RTMA program is one expression tree over eight observed features that selects NULL or a complete-row write with eviction; both use fixed finite constants, safe division, finiteness checks, and serialization bounds. Auditable causality comes from the witness obligations of Table 2 and from state versions, read closures, and the timing order $\epsilon_j \le v_j \le u_j < \sigma_{j+1} < \tau$. On the simultaneous-coverage event, the admission rule forces strict risk decrease on every accepted edge, which turns an unbounded-looking self-modification loop into a chain with at most $\lfloor R(\eta_0)/\delta \rfloor$ edges.

Core claim. The paper's central claim is that recursive self-improvement can be specified as a finite, executable, auditable construction rather than a general self-modification ambition. In the RTT route the currently installed numerical updater is literally the optimizer used by the proposal policy that samples and trains successor updaters; in the RTMA route the currently installed memory manager selects the authoritative records that the successor-generating proposal process may read. A successor is locked at most one candidate per generation, audited on fresh paired tasks against its parent, and admitted only if an upper bound on its risk is at least $\delta$ below a lower bound on the parent's risk, after which it takes over the same role in the next generation. The paper proves three conditional properties: causal composition of the whole generation sequence (Proposition 6.1), at most $\lfloor R(\eta_0)/\delta \rfloor$ admitted edges on the fixed task distribution under a simultaneous-coverage audit assumption (Proposition 6.2), and finite total work (Proposition 6.3). The paper is explicit about the boundary of its own contribution: the two routes and their joint realization are specified and the component studies characterize the numerical carrier, but no beneficial recursive chain has been observed and discovery efficiency remains unmeasured.

The premise it rests on. The finite-admission guarantee rests on the assumption that for every locked candidate and its parent, the audit intervals simultaneously cover both risks with probability at least $1-\alpha$; the paper indicates a route to this event through conditional error allocation and fresh audits, but it does not supply the interval procedure itself.

If the paper is right (Pith's inferences, not the authors')

If the construction is sound, an agent can change how it learns within an unresolved problem — its step-size schedule, clipping threshold, momentum, and retained evidence rows — without retraining or altering its pretrained weights.

Any accepted improvement chain is finite and strictly risk-decreasing on the fixed task distribution: at most $\lfloor R(\eta_0)/\delta \rfloor$ edges can be admitted, so unconstrained self-modification is excluded by the audit design rather than by fiat.

A chain becomes an observed instance of recursive improvement only after a second accepted edge, with the first admitted rule verified to participate functionally in producing the second (Definition 5.1); the disabling and replacement interventions of Section 5.9 specify how that participation is tested.

RTT and RTMA compose in a joint instance, and because policy retention and memory retention are treated as separate factors, a failed joint effect cannot be credited to either route alone.

The reported components separate the levels of evidence: useful feedback adaptation exists (program selection), binding is not benefit (CI replay), and the SQL studies' online increments fail their registered gates, so the recursion's value must be established by the audit loop itself, not inferred from the carriers.

The assumption most likely to fail in practice is the simultaneous-coverage audit event: the paper warns that bootstrap intervals reused after repeated selection do not supply it, and although it points to conditional error allocation with fresh audits as a route, it does not construct the intervals; producing a valid sequential procedure is open engineering work.

Because a NULL-selecting initial manager yields no eligible successor under step 6 of Algorithm 1, any future test of the RTMA route must begin with a rule that already performs a non-NULL write; the paper's own all-zero tree example shows the chain simply cannot start otherwise.

Proposition 6.2 invites a calibration experiment the paper does not run: track the empirical risk decrement of every admitted edge on a fixed distribution; a run could satisfy the bound yet show near-zero decrements, which would leave the construction sound but economically pointless.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains. We propose MMLA-RSI, a recursive self-improvement scheme with two implementations in Memory-Mediated Learning Architecture: Reasoning-Time Training (RTT) changes the policy state Φ, while Reasoning-Time Memory Adaptation (RTMA) changes authoritative memory M. Each route also makes its adaptation method an object of improvement. A current numerical updater trains the policy proposing its successor; a current memory manager chooses the records available to the process proposing its successor. An independently admitted successor takes over the same role in the next generation. We give bounded executable rule languages, shared task/proposal interfaces, and one algorithm for either route or their composition. The RTT instance changes scalar schedules in a clipped-momentum update. The RTMA instance chooses NULL or a complete-row write with possible eviction. A fixed interpreter, trusted committer, and independent evaluator bound what can change. State versions, read closures, and conditional properties describe causal execution, accepted transitions, and finite work. Existing numerical-component studies motivate the construction. In four-candidate program selection, retaining LARC feedback updates reduces expected error by 24.65 and 36.65 percentage points for two trained initializations. In chronological continuous-integration replay, feedback binding is positive while retention increases loss. Completed SQL studies improve substantially over an unadapted substrate but do not establish the required additional online benefit. These results characterize their declared components. The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-15

The authors' main claim, as the classifier picked it out 0.84

“The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated.”

How the main claim is stated: asserted 0.34

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.96 · human participants 0.04 · controlled comparison 0.84 · randomized assignment 0.02 · states quantitative result 0.97 · compares with prior results 0.88 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains. background 1.00 0.07 0.07 hedged possibility 0.59
We propose MMLA-RSI, a recursive self-improvement scheme with two implementations in Memory-Mediated Learning Architecture: Reasoning-Time Training (RTT) changes the policy state $\Phi$, while Reasoning-Time Memory Adaptation (RTMA) changes authoritative memory $M$. contribution 0.96 0.75 0.10 asserted 0.96
Each route also makes its adaptation method an object of improvement. method 0.60 0.35 0.05 asserted 1.00
A current numerical updater trains the policy proposing its successor; a current memory manager chooses the records available to the process proposing its successor. method 0.94 0.47 0.04 asserted 0.99
An independently admitted successor takes over the same role in the next generation. method 0.97 0.27 0.06 asserted 1.00
We give bounded executable rule languages, shared task/proposal interfaces, and one algorithm for either route or their composition. method 0.50 0.87 0.05 asserted 1.00
The RTT instance changes scalar schedules in a clipped-momentum update. method 0.99 0.74 0.03 asserted 1.00
The RTMA instance chooses NULL or a complete-row write with possible eviction. method 0.99 0.68 0.03 asserted 0.99
A fixed interpreter, trusted committer, and independent evaluator bound what can change. method 0.95 0.54 0.03 asserted 1.00
State versions, read closures, and conditional properties describe causal execution, accepted transitions, and finite work. method 0.92 0.64 0.04 asserted 0.97
Existing numerical-component studies motivate the construction. background 1.00 0.06 0.02 asserted 0.96
In four-candidate program selection, retaining LARC feedback updates reduces expected error by 24.65 and 36.65 percentage points for two trained initializations. result 1.00 0.81 0.11 asserted 1.00
In chronological continuous-integration replay, feedback binding is positive while retention increases loss. result 1.00 0.82 0.05 asserted 1.00
Completed SQL studies improve substantially over an unadapted substrate but do not establish the required additional online benefit. result 0.57 0.53 0.12 asserted 0.99
These results characterize their declared components. interpretation 0.82 0.50 0.09 asserted 0.96
The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated. limitation 0.56 0.39 0.47 asserted 0.34

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.49

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-15: 1 request, 11,179 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (5)
Figure 1 of the paper
Figure 1. Two state paths, two recursive implementations. RTT changes Φ through a numerical updater; RTMA changes 𝑀 through a memory manager. In each route, the current rule changes the proposal process that produces its successor. Admission installs that successor for the next generation. The routes can operate separately or share an unresolved problem. arXiv:2610.20294v1 [cs.CL] 14 Oct 2026
Figure 2 of the paper
Figure 2. The shared timing condition. The unresolved task persists from attempt to feedback, commit, and a later pinned read. RTT commits a policy-state update; RTMA commits a non-NULL memory successor. Independent future audit outcomes remain outside both paths. The subsequent action kernel averages over a declared source of continuation randomness. To distinguish this object from a replay, write 𝐹Ψ¯ (ℎ, 𝑍) for the executed action, where ℎ fixes the reached task history and arrived inputs, Ψ¯ fixes the non-random read state, and 𝑍 ∼ 𝜈 is the continuation random tape. RNG components of Ξ…
Figure 3 of the paper
Figure 3. Two controlled state paths and their read closure. Purple marks the policy path; teal marks authoritative memory. Context, caches, selectors, random streams, and callbacks are additional dependencies. Each intervention specifies its target and the other paths held fixed. 4.2 Policy retention and feedback binding Let 𝐿 𝑓 𝑘 be expected loss when feedback binding is 𝑓 ∈ {real,sham} and retention is 𝑘 ∈ {keep, reset}. Sham feedback independently applies a uniformly sampled permutation over the declared eligible feedback positions. It retains the feedback values while breaking their…
Figure 4 of the paper
Figure 4. Two realizations of parent participation. The RTT parent updates the numerical proposal state. The RTMA parent manages the authoritative evidence read by the next proposal. Selection and independent audit are shared; an admitted rule returns to its own role in the next generation. Joint runs carry both rules. 5.8 The complete recursive algorithm Algorithm 1 collects the construction into one execution order. Its parameters fix a finite run without fixing experimental hyperparameters in this report. In addition to the language bounds, they include a generation cap 𝐾 ≥ 1, a proposal…
Figure 5 of the paper
Figure 5. Binding is not the same as benefit. Left: program-selection retention and binding effects with original 99% development-group intervals. Right: CI effects in half-Brier units, shown as three-seed means without population intervals. Positive values favor the named effect in both panels; the units and populations differ. B.3 SQL interaction: state changes without endpoint gains The first completed task-level SQL study uses grouped interactions from SParC [23], with 56 development interactions across 28 databases and 75 confirmation interactions across 19 databases. Its primary…

Figures are the authors', reproduced from the paper.

References (32)
  1. [1] Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. arXiv:2306.13649v3, 2024. Version fixed at 2024-01-17 on PithPith's reviewarXiv
  2. [2] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. arXiv:1606.04474v2, 2016. Version fixed at 2016-11-30 on PithPith's reviewarXiv
  3. [3] Glenn W. Brier. Verification of Forecasts Expressed in Terms of Probability.Monthly Weather Review, 78(1):1–3, 1950. https: //doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2. 22 MMLA Technical Report on Pithdoi:10.1175/1520-0493(1950)078
  4. [4] Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, and Quoc V. Le. Symbolic Discovery of Optimization Algorithms. arXiv:2302.06675v4, 2023. Version fixed at 2023-05-08 on PithPith's reviewarXiv
  5. [5] Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution. arXiv:2309.16797v1, 2023. Version fixed at 2023-09-28 on PithPith's reviewarXiv
  6. [6] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400v3, 2017. Version fixed at 2017-07-18 on PithPith's reviewarXiv
  7. [7] Yoav Freund and Robert E. Schapire. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55(1):119–139, 1997. https://doi.org/10.1006/jcss.1997.1504 on Pithdoi:10.1006/jcss.1997.1504
  8. [8] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685v2, 2021. Version fixed at 2021-10-16 on PithPith's reviewarXiv
  9. [9] Louis Kirsch and Jürgen Schmidhuber. Eliminating Meta Optimization Through Self-Referential Meta Learning. arXiv:2212.14392v1, on PithPith's reviewarXiv
  10. [10] Jiahao Lu and Mohan Kankanhalli. Agentic-TTT: Training test-time policy for test-time training. arXiv:2610.12002v1, 2026. Version fixed at 2026-10-08 on PithPith's reviewarXiv
  11. [11] Kevin Lu and Thinking Machines Lab. On-Policy Distillation.Thinking Machines Lab: Connectionism, 2025. October 2025; doi:10.64434/tml.20251026; accessed October 14, 2026 on Pithdoi:10.64434/tml.20251026
  12. [12] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-Refine: Iterativ... on PithPith's reviewarXiv
  13. [13] Alex Nichol, Joshua Achiam, and John Schulman. On First-Order Meta-Learning Algorithms. arXiv:1803.02999v3, 2018. Version fixed at 2018-10-22 on PithPith's reviewarXiv
  14. [14] Juergen Schmidhuber. Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements. arXiv:cs/0309048v5, 2006. Version fixed at 2006-12-17 on PithPith's reviewarXiv
  15. [15] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366v4, 2023. Version fixed at 2023-10-10 on PithPith's reviewarXiv
  16. [16] Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. Learning to (Learn at Test Time): RNNs with Expressive Hidden States. arXiv:2407.04620v4, 2025. Version fixe... on PithPith's reviewarXiv
  17. [17] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test-Time Training with Self-Supervision for Generalization under Distribution Shifts. arXiv:1909.13231v3, 2020. Version fixed at 2020-07-01 on PithPith's reviewarXiv
  18. [18] Yanbo Wang, Jinhua Hao, Yuze Shi, Kun Yuan, and Ming Sun. No Time Like the Present: Agentic Test-Time Training for LLM Agents. arXiv:2607.03441v1, 2026. Version fixed at 2026-07-03 on PithPith's reviewarXiv
  19. [19] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine Learning, 8(3–4):229–256, 1992 on Pith
  20. [20] Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen, Yufan Zhuang, Yoonho Lee, Chengsong Huang, Han Yu, Zhongying CuiZhu, Yifei Ming, Huaxiu Yao, Burak Gokturk, Tomas Pfister, and Chen-Yu Lee. RRSI: Regularized Recursive Self-Improvement of Agent Harnesses. arXiv:2609.24972v3, 2026.... on PithPith's reviewarXiv
show 12 more
  1. [21] Shangjian Yin, Zehao Zhao, Kavosh Asadi, Rui Liu, Yuchen Lu, Shike Mei, Hang Cui, Luke Simon, Zhouxing Shi, and Hamed Firooz. Recursive Self-Improvement via On-Policy Distillation for Reasoning. arXiv:2609.30652v1, 2026. Version fixed at 2026-09-25 on PithPith's reviewarXiv
  2. [22] Tao Yu, Rui Zhang, He Yang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard S... on PithPith's reviewarXiv
  3. [23] Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, and Dragomir Radev. SParC: Cross-Domain Sem... on PithPith's reviewarXiv
  4. [24] Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, Jed McCaleb, Xiaolong Wang, Jan Kautz, Yejin Choi, James Zou, Carlos Guestrin, and Yu Sun. Learning to Discover at Test Time. arXiv:2601.16175v2, 2026. Version fixed at 2026-02-05 on PithPith's reviewarXiv
  5. [25] Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. arXiv:2310.02304v3, 2024. Version fixed at 2024-08-16 on PithPith's reviewarXiv
  6. [26] Guibin Zhang, Haotian Ren, Chong Zhan, Zhenhong Zhou, Junhao Wang, He Zhu, Wangchunshu Zhou, and Shuicheng Yan. MemEvolve: Meta-Evolution of Agent Memory Systems. arXiv:2512.18746v1, 2025. Version fixed at 2025-12-21 on PithPith's reviewarXiv
  7. [27] JennyZhang,ShengranHu,CongLu,RobertLange,andJeffClune. DarwinGödelMachine: Open-EndedEvolutionofSelf-Improving Agents. arXiv:2505.22954v3, 2026. Version fixed at 2026-03-12 on PithPith's reviewarXiv
  8. [28] Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. arXiv:2601.18734v3, 2026. Version fixed at 2026-03-20. 23 MMLA Technical Report on PithPith's reviewarXiv
  9. [29] Junyi Zou and Avrova Donz. LARC: Low-Rank Adaptive Residual Connections for Learning in Frozen Models. arXiv:2609.40063v1, on PithPith's reviewarXiv
  10. [30] Junyi Zou and Avrova Donz. MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation. arXiv:2606.28876v4, 2026. Version fixed at 2026-09-14. 24 on PithPith's reviewarXiv
  11. [2022] Version fixed at 2022-12-29 on Pith
  12. [2026] Version fixed at 2026-09-30 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment