Pith. sign in

REVIEW 5 major objections 6 minor 39 references

Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Macaron-V1 claims a frozen 744B base with four per-turn-routed LoRA specialists is a working Mixture-of-LoRA system: 87.8 on UI4A-Bench and 122/122 base-failure tasks covered by harness search alone.

desk verdict A transparent systems report with a genuinely novel MoL serving design, but the headline numbers rest on lab-built benchmarks that the paper itself admits are in-distribution; worth engaging, not worth taking as independent validation. read the letter →

arxiv 2608.09819 v1 pith:HC3UVCZO submitted 2026-08-10 cs.LG cs.CL

Mind Lab: Vin Bo , Asher Cai , Jingwei Cao , Song Cao , Vic Cao , Amelia Chen , Andrew Chen , Kaijie Chen
show 67 more authors
Cleon Cheng Steven Chiang Kaixuan Fan Hera Feng Huan Feng Arthur Fu Jun Gao Pyke Han Nolan Ho Ori Hong Hailee Hou Piers Hua Charles Huang Miles Jiang Nora Jiang Yuyi Jiang Qiuyu Jin Fancy Kong Kuss Koo Jaron Lee Andrew Lei Alexy Li Dawn Li Lucian Li Ray Li Ricardo Li Smith Li Theo Li Allen Lin Elliot Lin Fan Lin Chen Ling Kairus Liu Kieran Liu Logan Liu Neo Liu Xiang Liu Yuxin Lu Maeve Luo Pony Ma Verity Niu Cole Qiao Guian Qiu Vince Qu Sentry Niko Song Vincent Wang Bo Wu Rio Yang Evelyn Ye Fiona Ye Ina Ye Regis Ye Josh Ying Atlas Zeng Danney Zeng Salmon Zhan Anya Zhang Di Zhang Mia Zhang Sueky Zhang Wei Zhao Ada Zhou Adrian Zhou Yuhua Zhou Juno Zhu Murphy Zhuang
This is my paper · ORCID
classification cs.LGcs.CL
keywords Mixture-of-LoRAcontinuallearningrecursiveself-improvementmodel-harnessco-designgenerativeUIagenticreinforcementlong-contextRLexperientialintelligence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Macaron-V1 is a bid to make continual learning an architectural property rather than a training add-on: a frozen base model carries general ability, specialised LoRA adapters (small plug-in training matrices) carry differentiated behaviour, and a per-turn routing loop chooses which adapter answers. The paper presents Macaron-V1-Venti, a 744B GLM-5.2 base with four such adapters for chat, agent, coding, and GenUI, as a working instance of this Mixture-of-LoRA design, reporting 87.8 on UI4A-Bench, 58.3 on ChatBench, and 64.0 on LivingBench. It also reports that harness-side configuration search alone, with the model frozen, covers all 122 tasks the base fails outright, where the best single configuration covered 11, so many apparent model failures are unelicited behaviours rather than missing skills. The deeper claim is that the harness is a first-class optimisation target: tools, prompts, skills, and UI substrates can be revised, evaluated, and shipped without touching weights, which is what makes the continual-learning story concrete. The paper is explicit that cross-generation compounding and collective-intelligence gains are open questions, so a sympathetic reader should read it as a systems characterisation with two open bets.

What carries the argument

The load-bearing object is the Mixture-of-LoRA (MoL) serving layer: a frozen base, a small registry of specialist LoRA adapters, and a Proxy that treats adapter selection as a first-class per-turn action. The route label is emitted by the chat adapter L0 under a constrained-decoding grammar, so the router is not a separate model but a property of the chat specialist's understanding of the request. Two mechanisms make the loop cheap: the own-view, which rebuilds each specialist's conversation deterministically from an append-only timeline (own turns verbatim, other specialists collapsed to 192-token summaries) and thereby gives emergent per-adapter KV-prefix reuse; and the summary hop, which caps cross-adapter state at 192 tokens. On the learning side, the carried object is the model-harness pair, written $\pi_\varphi(a_t \mid o_{\le t}; \theta, c)$ with $\theta$ the frozen base, $\varphi$ the trainable LoRA parameters, and $c$ a versioned harness configuration: the recursive self-improvement cycle (Discovery, Expansion, Update) alternates configuration search over $c$ with GRPO updates to $\varphi$, and the reported Expansion experiment isolates the $c$-search half. The harness side is carried by three named substrates: UI4A, a component-native generative-UI harness where the model writes ordinary frontend code under runtime-enforced boundaries; the REPL agent harness with executable composition and validated helper reuse; and the Harness Context Protocol, a versioned TOML contract that makes a run reconstructable at the configuration boundary.

What would settle it

Re-run the 46 ChatBench cases and 40 LivingBench scenarios with human judges, or with a judge model from a different family than GLM-5.2, and check whether Macaron-V1-Venti's leads of 58.3 over 55.5 and 64.0 over 63.8 survive; if they collapse, the Personal Intelligence claims are judge-family artifacts. Separately, run the HCP configurations discovered in the 122-task Expansion study on a fresh, unselected slice of TerminalBench-family tasks; if coverage drops toward the 11/122 single-configuration baseline, the 122/122 result reflects adaptive search over a curated failure set rather than a general property of harness search.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that four specialist LoRA adapters on a frozen 744B base, selected per user turn by the chat adapter's own constrained-decoded label, form a workable substitute for a single monolithic post-trained model. The MoL Proxy runs a three-hop loop: L0 routes the request in 24 tokens, the chosen specialist answers from its own conversation view, and a 192-token summary preserves continuity across specialists, with routing accuracy of 99.12% on a 6,448-sample trace and a measured route-plus-summary overhead of about 32% of per-turn latency. The same system reports 87.8 on UI4A-Bench against 75.9 for the strongest published baseline, alongside leading scores on its own Personal Intelligence benchmarks. The second claimed result is that the harness is improvable as well as the model: across 69 jobs and 450 attempts on 122 TerminalBench-family tasks that the frozen GLM-5.2 base fails under the official reward, adaptive configuration search reaches 122/122 cumulative coverage without a single optimizer step. The paper deliberately stops short of claiming that this demonstrates continual learning, defined as compounded gains across model generations, or collective intelligence, defined as complementary gains from independently trained specialists.

Load-bearing premise

The load-bearing premise is that the internal evaluation instruments — ChatBench's judge drawn from the same GLM-5.2 family as the Venti base, LivingBench's simulated sandbox with LLM judges, and UI4A-Bench's scoring policy — measure genuine assistance quality rather than rewarding outputs that resemble the training distribution; the paper itself concedes that these benchmarks are 'less independent of the training process than a frozen external test set'.

Editorial extensions

If this is right

  • New capabilities can ship as adapter registrations on a frozen base, so the base, the specialists, and the harness each move on their own release clock.
  • Configuration search is a first-pass improvement path: 122/122 coverage versus 11/122 for the best single full-set configuration implies that many apparent model failures are unelicited behaviours rather than missing skills.
  • The MoL resident layout stores about 26% of the replicated-base parameter count, a 74% reduction, which is what makes multi-specialist long-context serving feasible on fixed hardware.
  • Routing by the chat adapter itself means routing quality improves for free as the base or the chat specialist improves, at a measured cost of roughly one third of per-turn latency.
  • Because only adapters receive gradients, the base cannot drift as a side effect of specialisation, although routing and harness changes can still alter end-to-end behaviour.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 122/122 coverage generalises beyond the curated failure slice, configuration search becomes the cheap first line of continual improvement, with LoRA updates reserved for behaviours that configuration cannot elicit; the paper's own 13x per-attempt yield ratio between targeted search and full-set sweeps hints at this but is not a controlled comparison.
  • The 192-token summary is an information bottleneck between specialists; a natural test is varying summary length, or replacing summaries with the shared-L0-KV substrate the paper sketches, and measuring cross-specialist task quality.
  • The routing accuracy figure of 99.12% comes from LoRA training data, so a held-out routing audit is the cleanest next check of whether L0's routing generalises.
  • Because the internal benchmarks are judged by LLMs from the same model families as the systems under test, an external human-judge or cross-family-judge calibration would settle whether the Personal Intelligence leads reflect assistance quality or in-distribution mimicry.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents Macaron-V1, an open agent-model family for "experiential intelligence," organized around two goals: adaptation via recursive improvement of versioned model-harness pairs, and collaboration via Mixture-of-LoRA (MoL) composition on a frozen base. The flagship Macaron-V1-Venti pairs a frozen 744B GLM-5.2 base with four LoRA specialists (chat, agent, coding, GenUI) selected per user turn by an L0-emitted route through a Proxy; a 50B Qwen3.6-based variant (Tall) uses the same design. The algorithmic half describes Model-Harness Co-design (UI4A component-native GenUI harness, REPL agent harness, Harness Context Protocol) and a three-stage MindForge recursive self-improvement loop (Discovery, Expansion, Update), with an isolated Expansion experiment covering 122/122 selected base-failure tasks through HCP-carried configuration changes alone. Infrastructure (MinT, LongStraw, sparse-base rollout-mismatch controls) is reported largely from companion studies. Evaluation covers three internal suites (ChatBench, LivingBench, UI4A-Bench) plus twelve general-capability rows against six frontier baselines; headline numbers are 58.3/64.0 on Personal Intelligence and 87.8 on UI4A-Bench. The paper explicitly disclaims cross-generation continual-learning gains and collective-intelligence emergence, and marks imported versus reproduced values throughout.

Significance. If the evidence were fully established, the contribution would be a credible, released instantiation of a modular continual-learning architecture: the MoL separation (frozen base, registered adapters, per-turn routing with per-adapter conversation views and emergent KV reuse) is a clean engineering substrate, and the RSI Expansion study is a falsifiable, well-scoped measurement that many scored base failures can be recovered purely by harness configuration. The paper is exemplary in its scoping: it marks imported values, admits the internal benchmarks are in-distribution (Section 5.1), states the missing overlap audit and judge calibration (Appendix B.1), disclaims routing-quality equivalence (Section 2.3), and explicitly does not claim cross-generation gains (Section 7.2). It ships open weights and a harness, retains versioned UI4A run manifests, and verifies trace identity by sample ID and input hash in the routing study.

major comments (5)
  1. [§5.1, §5.5.1, §B.1, Table 8] The headline support for the abstract's "results validate the current system" is the three internal suites (ChatBench 58.3, LivingBench 64.0, UI4A-Bench 87.8 in Table 8). The manuscript itself states in Section 5.1 that these benchmarks are "less independent of the training process than a frozen external test set," that "evaluation artifacts can in turn supply candidates for later training," and in Appendix B.1 that "This release does not report a frozen data cutoff, an item-level overlap audit against post-training data, human-judge agreement, or cross-judge sensitivity for either benchmark." Because Section 5.5.1 sources the 161 UI4A-Bench cases from "curated examples, de-identified production traffic, and coverage-gap sampling," the 20.7-point UI4A gap against the GLM-5.2 base (87.8 vs 67.1) may measure in-distribution fit of the L3 specialist rather than general UI-generation ability, and the Section 6.4 phrasing that this "support[s] a direct conclusion about clear, accurate, interactive UI generation" is accordingly stronger than the current evidence. The paper should add an item-level overlap audit between UI4A-Bench, ChatBench, and LivingBench items and the training corpora, support the conclusions with a held-out external benchmark and a human-scored sample, or downgrade the claims to in-distribution characterization.
  2. [§B.1, §6.2] The ChatBench row is scored by "a privately deployed GLM-5.2 judge" (Appendix B.1), the same model family as the Macaron-V1-Venti base. Section 6.2 acknowledges that "sharing that model family may favor GLM-derived responses" and that no human or cross-family judge calibration is available, yet the 58.3 score is still presented as a 2.8-point lead over GPT-5.5 (55.5) and a 3.8-point lead over the GLM-5.2 base (54.5). With an unquantified same-family judge, these differences cannot be attributed to conversational quality rather than stylistic mimicry. A scored subset with a second, non-GLM judge or with human raters is needed to bound the effect; otherwise the ChatBench row should be presented as an internal diagnostic rather than a comparative result.
  3. [§2.3 (Table 3), §6.2, §6.4] The evidence base for the headline comparisons is entirely point estimates. ChatBench and LivingBench average three runs per case, but no intervals, bootstrap, or significance tests are reported for any of the twelve rows in Table 8, and Section 6.4 states that scores lack "interval or judge-sensitivity analysis." The paper applies the right caution to the 0.2-point LivingBench gap ("should not be interpreted as established superiority") but not to the 2.8-point ChatBench lead or the UI4A Layer-Score leads. The same statistical weakness affects the routing-quality claim: Table 3 compares five seed-level aggregates per arm with seed identifiers not retained (hence unpaired), and Section 2.3 concedes the comparison "does not establish equivalence," yet Section 2.8 nonetheless lists "no detected reuse-related degradation" among the practical consequences of the design. The central MoL claim that routing does not harm task quality needs a paired, adequately powered comparison with retained seed IDs, and the benchmark leads need interval estimates or a uniformly hedged framing.
  4. [§3.2.5, Table 4] The Expansion experiment is cleanly designed in one respect: the model is frozen, no optimizer step is taken, and every change is an HCP-carried edit, so the 122/122 coverage is a genuine demonstration of configuration-search reach on the selected slice. The manuscript also states the key limitation that this is "a coverage ceiling under adaptive configuration selection, not a held-out estimate of how any single configuration generalizes." What the manuscript does not provide is any way to assess the structure of the set: it states outright that "the retained artifact does not include the per-family task counts," so the reader cannot tell whether 122 tasks from 29 TerminalBench 2.1 families are spread evenly or dominated by a few families, which materially changes the interpretation of full coverage. I ask for the per-family counts (and per-family coverage trajectories if available) to be reported, and for a statement of how many of the 122 tasks are passed by configurations that also pass previously covered tasks, so the reach result can be separated from per-task overfitting of the search.
  5. [Abstract; §6.4; §7.4] The abstract's "Our results validate the current system" and Section 6.4's "direct conclusion about clear, accurate, interactive UI generation" are stronger than the evidence the paper itself describes. Section 7.4 concludes that "the current results document execution checks for parts of this stack and one model snapshot," and Section 7.2 states that the internal suites "target the same Personal Intelligence distribution that informs the RSI loop" and "remain focused in size and scope." Given the admissions in Sections 5.1 and B.1 (in-distribution benchmarks, missing overlap audit, unquantified same-family judge), the scoped phrasing of Section 7.4 is the one the abstract and Section 6.4 should use; otherwise the headline claim exceeds what the reported experiments establish.
minor comments (6)
  1. [Table 2] The confusion-matrix entries run together (e.g., "97719 4 0" should read "977, 19, 4, 0"); add column separators so the per-class counts are legible.
  2. [Fig. 7 / Table 4] The percentage labels on the upper panel of Figure 7 (12.0%, 6.1%, 64.5%, 81.2%) appear to be the pooled pass rates of Table 4, but the caption describes that panel as cumulative coverage; clarify what the labels annotate.
  3. [§1, §7.1] Section 1 states that "MoL demonstrates modular collaboration," which is stronger than Section 7.1's "Neither bet is settled by Macaron-V1"; this release tests only the four shipped specialists, so the Section 1 sentence should be aligned with the scoped statement.
  4. [Table 10] The -52.99-point MME perception drop for Macaron-V1-Tall is reported without discussion; given the text-only training of the adapters, a sentence interpreting this large negative delta (e.g., routing behavior or adapter interference on vision-language inputs) would help the reader.
  5. [§7.2] The data-governance gap is acknowledged but remains material: the paper evaluates on de-identified product conversations and traffic without documenting the de-identification procedure, residual re-identification audit, retention controls, or consent basis. This should either be documented or the applicability of the results to non-product settings should be further qualified.
  6. [Eq. (3)] The final term of Equation (3) is rendered ambiguously; clarify that the score-memory term scales with the summed response length.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the internal benchmarks are in-distribution by the paper's own admission, but this is an external-validity limitation, not a reduction of a predicted result to a fitted input.

full rationale

The paper does not derive its headline results from its own inputs by construction. Its central architectural claims are supported by open-sourced harness code, direct latency and routing measurements, and explicitly scoped execution receipts, rather than by an imported uniqueness theorem or by a fitted parameter renamed as a prediction. The strongest self-admitted risk is the coupling between the internal Personal Intelligence benchmarks and the RSI training loop: Section 5.1 states that 'evaluation artifacts can in turn supply candidates for later training' and that the coupling 'makes the internal benchmarks less independent of the training process than a frozen external test set,' while Section B.1 concedes that no item-level overlap audit is reported and that ChatBench uses a private GLM-5.2 judge from the same family as the Venti base. These are genuine external-validity and contamination risks, but the paper does not present the benchmark scores as derived from the training objective by an equation, nor does it fit a parameter to the evaluation set and then call the resulting score a prediction. The RSI expansion result (122/122 on a selected base-failure set) is explicitly bounded by the paper as measuring configuration-search coverage on a chosen failure slice, not generalization or cross-generation gains. Many citations are to the authors' own companion reports and repositories, but they are not used as a load-bearing uniqueness argument; the key measurements are stated in the paper itself. The result is therefore a systems characterization with acknowledged in-distribution evaluation, not a circular derivation. A score of 2 reflects the density of self-citation and the admitted train/eval coupling, without treating either as a demonstrated circular step.

Assumptions & free parameters 5 free parameters · 6 assumptions · 4 invented entities

The central claims rest on several manually chosen hyperparameters (ranks, token budgets, benchmark weights, training recipe) and on the validity of lab-authored LLM-judged benchmarks. The paper discloses most of these dependencies, but they still mean the headline results are conditional on design choices and evaluation instruments that have not been independently audited.

free parameters (5)
  • LoRA rank, alpha, and target modules = Venti: r=16, alpha=32; Tall: r=64, alpha=128 (Appendix B.6)
    Chosen by hand per base and not ablated; all four adapters share them, so every benchmark result is conditional on these values.
  • Routing decode budget and summary cap = 24 tokens for route decode; 192 tokens for summary (Section 2.3)
    System design choices that determine routing overhead and how much cross-adapter context is preserved; no sweep is reported.
  • LivingBench score weighting = 0.7 need fulfillment + 0.3 process quality (Section 5.4.3)
    A fixed deterministic combination that directly shapes the headline Personal Intelligence numbers.
  • UI4A-Bench Layer Score weights = Engineering 8%, Task 18%, Visual 38%, Interaction 20%, Constraint 16% (Section 6.2)
    Fixed weights chosen by the benchmark authors before aggregation; different weights would change the Final Score.
  • Training hyperparameters = AdamW lr=5e-6, batch 4 (L3 batch 2), epochs 4 or 1, warmup 0.1 (Appendix B.6)
    One shared recipe with no ablations, so model behavior cannot be separated from these choices.
assumptions (6)
  • domain assumption Frozen-base plus LoRA composition preserves base capabilities and avoids cross-task interference.
    Section 2.1 admits the release lacks the budget-matched single-LoRA comparison needed to quantify interference; this is treated as design motivation rather than an empirical finding.
  • domain assumption Per-turn routing to exactly one specialist is adequate for user requests, with multi-intent turns handled by later conversation segmentation.
    Section 2.8 explicitly calls the one-intent-per-turn assumption a deliberate simplification, but the headline routing and quality results depend on it.
  • domain assumption LLM-mediated judges provide valid measurements of conversation, simulation, and UI quality.
    ChatBench, LivingBench, and UI4A-Bench all use LLM judges; Appendix B.1 notes no human agreement or cross-family judge calibration is available.
  • ad hoc to paper The 122-task base-failure set is a suitable probe of configuration-search reach.
    Section 3.2.5 states the tasks were selected because the frozen base fails all of them, and the 122/122 result is an adaptive-selection ceiling, not a held-out generalization estimate.
  • domain assumption The harness used in evaluation (REPL, UI4A, Claude Code for coding rows) is close enough to production serving for results to transfer.
    Section 3.1 describes train-serve harness sharing as a goal, but Section 3.2.2 notes that matching configuration does not guarantee behaviorally identical executions.
  • standard math GRPO with LoRA adapters and LongStraw response-only replay compute valid policy-gradient updates.
    The paper cites GRPO and the LongStraw companion report; Section 4.2 explicitly limits 'exact' to the stated response-only transaction, not parameter-wise equivalence to full-sequence backpropagation.
invented entities (4)
  • MoL Proxy with own-view conversation reconstruction independent evidence
    purpose: Select one LoRA per user turn, rebuild each adapter's context from summaries, and enable KV reuse across turns.
    The serving harness is open-sourced, so the mechanism can be run externally, but routing-quality evidence is based on a training-data trace and small internal comparisons.
  • UI4A Action contract (Origin, State, Execution, Visibility)
    purpose: Let the model write ordinary frontend code under runtime-enforced boundaries for generative UI.
    The main quality evidence comes from the lab-authored UI4A-Bench; no independent outside-lab evaluation is available.
  • Harness Context Protocol (HCP) independent evidence
    purpose: Versioned TOML contract for runtime and tool configuration, used as the portable boundary in the RSI loop.
    A public SDK and RFC are linked, allowing external inspection, though the protocol's benefit to training is not independently measured.
  • MindForge RSI control plane
    purpose: Orchestrate problem banks, evaluation runs, trajectory selection, training jobs, and model-harness version lineage.
    Described as the lifecycle backend, but no public release or external validation is provided in this report.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA." pith.science (2026). https://pith.science/paper/HC3UVCZO

@misc{pith2026260809819,
  author       = {Pith},
  title        = {Pith review of: Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HC3UVCZO}},
  note         = {Machine review of arXiv:2608.09819}
}
read the original abstract

Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals. Adaptation is pursued through recursive improvement of versioned model-harness pairs, where experience from one configuration is evaluated under an external contract and used to construct its successor. Collaboration is pursued via the Mixture-of-LoRA (MoL) architecture that freezes a base model, composes specialist LoRA adapters, and selects one LoRA per user turn. The flagship Macaron-V1-Venti combines a 744B GLM-5.2 base with four LoRAs for chat, agent, coding, and GenUI; the Qwen3.6-based Macaron-V1-Tall (50B) uses the same design for local deployment. This report presents Macaron-V1 as a co-designed system spanning architecture, algorithms, and infrastructure. The MoL architecture supports continual learning through extensible LoRA specialists. The algorithm combines Model-Harness Co-design and recursive self-improvement loop, including the UI4A component-native GenUI harness, a stateful action substrate, versioned HCP contract, and the agentic RL framework MindForge. The supporting infrastructure includes the post-training platform MinT, the long-context RL method LongStraw, and stability techniques for sparse MoE and DSA base models. We evaluate Macaron-V1 on Personal Intelligence, GenUI, and general capability benchmarks against frontier baselines. Our results validate the current system, while compounding gains from continual learning and collective intelligence remain open questions.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 11 canonical work pages

  1. [4]

    Effective LoRA adapter routing using task representations.arXiv preprint arXiv:2601.21795,

    Akash Dhasade, Anne-Marie Kermarrec, Igor Pavlovic, Diana Petrescu, Rafael Pires, Mathis Randl, and Martijn de Vos. Effective LoRA adapter routing using task representations.arXiv preprint arXiv:2601.21795,

    arXiv

  2. [5]

    URLhttps://arxiv.org/abs/2601.21795

    doi: 10.48550/arXiv.2601.21795. URLhttps://arxiv.org/abs/2601.21795. Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. Length-controlled AlpacaEval: A simple way to debias automatic evaluators.arXiv preprint arXiv:2404.04475,

    work page doi:10.48550/arxiv.2601.21795

  3. [6]

    Scaling laws for reward model overoptimization.arXiv preprint arXiv:2210.10760,

    Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization.arXiv preprint arXiv:2210.10760,

    arXiv

  4. [8]

    When does continual learning require learning.arXiv preprint arXiv:2607.07847,

    Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, and Yutong Bai. When does continual learning require learning.arXiv preprint arXiv:2607.07847,

    arXiv

  5. [9]

    When Does Continual Learning Require Learning

    doi: 10.48550/arXiv.2607.07847. URLhttps://arxiv.org/abs/2607.07847. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations (ICLR),

    work page Pith review arXiv doi:10.48550/arxiv.2607.07847

  6. [10]

    EvolveRouter: Co-evolving routing and prompt for multi-agent question answering.arXiv preprint arXiv:2604.05149,

    Jiatan Huang, Zheyuan Zhang, Kaiwen Shi, Yanfang Ye, and Chuxu Zhang. EvolveRouter: Co-evolving routing and prompt for multi-agent question answering.arXiv preprint arXiv:2604.05149,

    arXiv

  7. [11]

    URLhttps://arxiv.org/abs/2604.05149

    doi: 10.48550/arXiv.2604.05149. URLhttps://arxiv.org/abs/2604.05149. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? InInternational Conference on Learning Representations (ICLR),

    arXiv doi:10.48550/arxiv.2604.05149

  8. [12]

    EXG: Self-evolving agents with experience graphs.arXiv preprint arXiv:2605.17721,

    Yuxin Jin, Siyuan Zhang, Hanchen Wang, Lu Qin, Ying Zhang, and Wenjie Zhang. EXG: Self-evolving agents with experience graphs.arXiv preprint arXiv:2605.17721,

    arXiv

Show all 39 references
  1. [13]

    URLhttps: //arxiv.org/abs/2605.17721

    doi: 10.48550/arXiv.2605.17721. URLhttps: //arxiv.org/abs/2605.17721. Seth Karten, Joel Zhang, Tersoo Upaa Jr., Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, and Kiran Vodrahalli. Continual harness: Online adaptation for self-improving foundation agents.arXiv preprint arXi...

    doi arXiv

  2. [14]

    URLhttps://arxiv.org/abs/2605.09998

    doi: 10.48550/arXiv.2605.09998. URLhttps://arxiv.org/abs/2605.09998. Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, et al. Dynabench: Rethinking benchmarking in NLP.arXiv prep...

    doi arXiv

  3. [15]

    Macaron-A2UI: A model for generative UI in per- sonal agents

    Fancy Kong, Congjie Zheng, Murphy Zhuang, Rio Yang, Sueky Zhang, Arthur Fu, Gene Jin, Song Cao, Kai- jie Chen, Andrew Chen, Pony Ma, and Mind Lab. Macaron-A2UI: A model for generative UI in per- sonal agents. Mind Lab: A Lab for Experiential Intelligence, 2026.https://macaron....

    2026

  4. [17]

    URLhttps://arxiv.org/abs/2605.12357

    doi: 10.48550/arXiv.2605.12357. URLhttps://arxiv.org/abs/2605.12357. Yaniv Leviathan et al. Generative UI: LLMs are effective UI generators. Technical report, Google Research,

    doi arXiv

  5. [19]

    Every step evolves: Scaling reinforcement learning for trillion-scale thinking model.arXiv preprint arXiv:2510.18855,

    Ling Team. Every step evolves: Scaling reinforcement learning for trillion-scale thinking model.arXiv preprint arXiv:2510.18855,

  6. [20]

    Zhichen Liu, Ruihan Sun, Hengjie Yang, Zipeng Wu, Zhaohan Chen, Xiaofan Zhang, and Yang Xu

    Introduces IcePop token-level discrepancy masking. Zhichen Liu, Ruihan Sun, Hengjie Yang, Zipeng Wu, Zhaohan Chen, Xiaofan Zhang, and Yang Xu. LiveMem: Maintaining memory state continuity in long-running LLM inference.arXiv preprint arXiv:2608.02515,

    arXiv

  7. [21]

    URLhttps://arxiv.org/abs/2608.02515

    doi: 10.48550/arXiv.2608.02515. URLhttps://arxiv.org/abs/2608.02515. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations (ICLR),

    doi arXiv

  8. [22]

    Wenhan Ma, Hailin Zhang, Liang Zhao, Yifan Song, Yudong Wang, Zhifang Sui, and Fuli Luo

    https://macaron.im/mindlab/research/mint-rl-infrastructure-for-experiential-intelligence. Wenhan Ma, Hailin Zhang, Liang Zhao, Yifan Song, Yudong Wang, Zhifang Sui, and Fuli Luo. Stabilizing MoE reinforcement learning by aligning training and inference routers.arXiv preprint a...

  9. [23]

    Categorizing variants of Goodhart’s law.arXiv preprint arXiv:1803.04585,

    David Manheim and Scott Garrabrant. Categorizing variants of Goodhart’s law.arXiv preprint arXiv:1803.04585,

    arXiv

  10. [24]

    Kimi K2: Open agentic intelligence.arXiv preprint arXiv:2507.20534,

    Moonshot AI. Kimi K2: Open agentic intelligence.arXiv preprint arXiv:2507.20534,

    arXiv

  11. [25]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Qwen Team. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    arXiv

  12. [26]

    Qwen3.6.https://qwen.ai, 2026a

    Qwen Team. Qwen3.6.https://qwen.ai, 2026a. Qwen Team. Qwen3.7: The agent frontier.https://qwen.ai/blog?id=qwen3.7, 2026b. Inioluwa Deborah Raji, Emily M. Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. AI and the everything in the whole wide world benchmark.arXiv p...

    arXiv

  13. [27]

    Shrey Shah and Justin Wagle

    https://thinkingmachines.ai/blog/lora/. Shrey Shah and Justin Wagle. MoLoRA: Composable specialization via per-token adapter routing.arXiv preprint arXiv:2603.15965,

  14. [28]

    URLhttps://arxiv.org/abs/2603.15965

    doi: 10.48550/arXiv.2603.15965. URLhttps://arxiv.org/abs/2603.15965. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models...

    doi

  15. [29]

    Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E

    Introduces GRPO. Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E. Gonzalez, and Ion Stoica. S-LoRA: Serving thousands of concurrent LoRA adapters.arXiv preprint arXiv:2311.03285,

    arXiv

  16. [30]

    Evaluating AI systems under contextual uncertainty

    Stanford RegLab. Evaluating AI systems under contextual uncertainty. Stanford HAI Working Paper, 2025.https: //reglab.stanford.edu. SWE Atlas Authors. SWE Atlas QnA: A question-answering benchmark over software engineering repositories.https: //labs.scale.com/leaderboard/sweatlas-qna,

    2025

  17. [31]

    LaMDA: Language models for dialog applications.arXiv preprint arXiv:2201.08239,

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. LaMDA: Language models for dialog applications.arXiv preprint arXiv:2201.08239,

    arXiv

  18. [32]

    Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D. Nguyen. Multi-agent collaboration mechanisms: A survey of LLMs.arXiv preprint arXiv:2501.06322,

    arXiv

  19. [33]

    URLhttps://arxiv.org/abs/2501.06322

    doi: 10.48550/ arXiv.2501.06322. URLhttps://arxiv.org/abs/2501.06322. Bing Wang, Ximing Li, Changchun Li, Jinjin Chi, Gang Niu, and Masashi Sugiyama. Decomposing the basic abil- ities of large language models: Mitigating cross-task interference in multi-task instruct-tuning.ar...

    doi arXiv

  20. [34]

    URLhttps://arxiv.org/abs/2605.05676

    doi: 10.48550/arXiv.2605.05676. URLhttps://arxiv.org/abs/2605.05676. Accepted at ICML

    doi arXiv

  21. [35]

    Executable code actions elicit better LLM agents.arXiv preprint arXiv:2402.01030,

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better LLM agents.arXiv preprint arXiv:2402.01030,

    arXiv

  22. [36]

    Efficient LLM agents with REPL harnesses: Executable composition and validated reuse

    Bo Wu, Di Zhang, Jun Gao, Murphy Zhuang, Kieran Liu, Andrew Chen, Pony Ma, and Mind Lab. Efficient LLM agents with REPL harnesses: Executable composition and validated reuse. Mind Lab: A Lab for Experiential Intelligence, 2026.https://macaron.im/mindlab/research/ efficient-llm...

    2026

  23. [37]

    Agentic context engineer- ing: Evolvingcontextsforself-improvinglanguagemodels

    Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic context engineer- ing: Evolvingcontextsforself-improvinglanguagemodels. InInternatio...

    arXiv

  24. [38]

    Zhipu AI. GLM-5.2. Hugging Face Model Hub, 2026.https://huggingface.co/zai-org/GLM-5.2. Changhai Zhou, Yuhua Zhou, Shiyang Zhang, Yibin Wang, and Zekai Liu. Dynamic operator optimization for efficient multi-tenant LoRA model serving.Proceedings of the AAAI Conference on Artifi...

    2026 arXiv

  25. [39]

    URLhttps://arxiv.org/abs/2503

    doi: 10.48550/arXiv.2503.01935. URLhttps://arxiv.org/abs/2503. 01935. Murphy Zhuang, Rio Yang, Hao Fu, Fan Lin, Hera Feng, Xiang Liu, Qian Qiao, Theo Li, Sueky Zhang, Andrew Chen, Pony Ma, and Mind Lab. UI4A: A component-native harness for generative UI. Mind Lab: A Lab for Ex...

    doi arXiv

  26. [2021]

    Continual learning in large language models: Methods, challenges, and opportunities.arXiv preprint arXiv:2603.12658, 2026a

    Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, and Xuemin Lin. Continual learning in large language models: Methods, challenges, and opportunities.arXiv preprint arXiv:2603.12658, 2026a. doi: 10.48550/arXiv.2603.12658. URLhttps://arxiv.org/abs/2603.12658. Jiaqi Chen, Yanzh...

    doi arXiv

  27. [2022]

    GLM-5: From vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,

    39 GLM-5 Team. GLM-5: From vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,

    arXiv

  28. [2023]

    Recursive harness self- improvement.arXiv preprint arXiv:2607.15524, 2026a

    Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, and Yujin Tang. Recursive harness self- improvement.arXiv preprint arXiv:2607.15524, 2026a. doi: 10.48550/arXiv.2607.15524. URLhttps://arxiv.org/ abs/2607.15524. Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwoo...

    doi arXiv

  29. [2024]

    Recursive self-improvement in AI: From bounded self-refinement to autonomous research loops.arXiv preprint arXiv:2607.07663, 2026b

    Mingguang Chen, Licheng Wang, and Bo Qu. Recursive self-improvement in AI: From bounded self-refinement to autonomous research loops.arXiv preprint arXiv:2607.07663, 2026b. doi: 10.48550/arXiv.2607.07663. URL https://arxiv.org/abs/2607.07663. Yuxin Chen, Yi Zhang, Zhengzhou Ca...

    doi arXiv

  30. [2025]

    TiC-LM: A web-scale benchmark for time-continual LLM pretraining

    Jeffrey Li, Mohammadreza Armandpour, Seyed Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemula- palli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, and Fartash Faghri. TiC-LM: A web-scale benchmark for time-continual LLM pretraining. InProceedings ...

    doi arXiv

  31. [2026]

    Bowman and George E

    Samuel R. Bowman and George E. Dahl. What will it take to fix benchmarking in natural language understanding? arXiv preprint arXiv:2104.02145,

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.