Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

A 35B agent matches trillion-parameter models on long-horizon tasks by scaling training trajectories and unifying specialized teachers, not by growing parameters.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 16:57 UTC pith:GHUP7TQ7

load-bearing objection Solid three-stage agent recipe with real multi-domain data work; the “35B matches 1T” headline is oversold by tool-asymmetric science evals and selective score framing. the 4 major comments →

arxiv 2606.30616 v2 pith:GHUP7TQ7 submitted 2026-06-29 cs.CL

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Lei Bai , Zongsheng Cao , Yang Chen , Zhiyao Cui , Shangheng Du , Yue Fan , Shiyang Feng , Zijie Guo
show 45 more authors
Haonan He Liang He Xiaohan He Shuyue Hu Yusong Hu Songtao Huang Yichen Jiang Hao Li Xin Li Dahua Lin Weihao Lin Fenghua Ling Dongrui Liu Zhuo Liu Wenjie Lou Runmin Ma Chunjiang Mu Haoyang Peng Tianshuo Peng Jinxin Shi Luohe Shi Boyuan Sun Zelin Tan Shengji Tang Yan Teng Qianyi Wang Xiaosong Wang Yiming Wu Yi Xie Xiangchao Yan Jingqi Ye Peng Ye Fangchen Yu Jiakang Yuan Bihao Zhan Bo Zhang Chen Zhang Shufei Zhang Shuaiyu Zhang Wenlong Zhang Yiqun Zhang Junpeng Zhao Zhijie Zhong Bowen Zhou Yuhao Zhou
This is my paper
classification cs.CL
keywords agentic modelslong-horizon agentson-policy distillationknowledge-action graphmixture-of-expertstool usemulti-teacher distillationagent training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that long-horizon agent skill can be scaled without trillion-parameter models. The authors train Agents-A1, a 35B mixture-of-experts agent, on multi-domain trajectories that average about 45K tokens and record external knowledge, actions, observations, and verifier outcomes. After broad supervised fine-tuning, they train domain teachers for search, science, instruction following, tool calling, and related skills, then fold those teachers into one student with multi-teacher domain-routed on-policy distillation and salient vocabulary alignment. They report that this 35B student leads or matches much larger models on several public long-horizon benchmarks, including search, science olympiad and research suites, instruction following, and molecular binding tasks. A sympathetic reader would care because the work offers a concrete recipe for deployable agents that stay small while still handling multi-step, tool-using, verifiable work.

Core claim

Agents-A1 shows that a 35B MoE agent can reach or match trillion-parameter systems on long-horizon agent benchmarks by scaling the agent horizon—long multi-domain trajectories plus heterogeneous abilities—rather than parameters. Using a three-stage recipe of full-domain supervised fine-tuning, specialized domain teachers, and multi-teacher domain-routed on-policy distillation with salient vocabulary alignment, the single student unifies six domains and reports leading or competitive scores on SEAL-0, IFBench, HiPhO, FrontierScience-Olympiad, MolBench-Bind, and related suites against 1T-class baselines.

What carries the argument

The long-horizon knowledge-action graph (KAG), which links domain corpus, actions, observations, and verifiers into process-level trajectories, plus multi-teacher domain-routed on-policy distillation with salient vocabulary alignment: each student rollout is supervised only by its domain teacher on a compact top-k teacher support under a domain-normalized reverse-KL loss so specialized policies transfer without mixing incompatible token-level signals.

Load-bearing premise

The headline that a 35B agent matches trillion-parameter models assumes the authors' long training trajectories and mixed evaluation setups transfer fairly to public benchmarks, including comparisons of their tool-using agent against differently configured large models.

What would settle it

Independently re-run Agents-A1 and the cited 1T baselines on SEAL-0, HiPhO, FrontierScience-Olympiad, IFBench, and BrowseComp under identical tool access, judges, turn limits, and context windows; if the 35B lead vanishes under matched protocols, the central claim fails as stated.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A deployable ~35B agent can approach 1T-model scores on multi-step search, scientific research, and instruction following when trained on long verified trajectories.
  • Full-domain supervised fine-tuning alone leaves conflicts among reasoning styles; multi-teacher on-policy distillation is needed to restore specialized strengths in one policy.
  • Process-level supervision that records actions, observations, and verifier outcomes is more useful for long-horizon agents than final-answer-only data.
  • Search, tool use, and scientific reasoning can mutually support each other when unified rather than trained as isolated skills.
  • The same three-stage path—broad SFT, domain teachers, domain-routed distillation—is offered as a practical alternative to pure parameter scaling for long interactive tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If horizon scaling generalizes, teams may shift investment from ever-larger base models toward verifiable multi-turn environments and teacher consolidation for agent products.
  • The KAG-plus-routed-distillation pattern could extend to other compositional stacks, such as multi-app workflow agents, beyond the six domains studied.
  • Matched-protocol re-evaluations that fix tool access and judges are the cleanest public test of whether the 35B-vs-1T headline is infrastructure-driven or evaluation-driven.
  • The atomic habits the authors flag next—planning before reasoning, reflection before acting, long-context summarization—may become the bottleneck once trajectory length is no longer the limiter.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents Agents-A1, a 35B MoE agent trained to approach trillion-parameter performance by scaling the agent horizon rather than model size. The authors build a long-horizon knowledge-action infrastructure (KAG) that links external knowledge, actions, observations, and verifier outcomes into trajectories averaging ~45K tokens, then train with a three-stage recipe: full-domain SFT on multi-domain agentic data, domain-level teacher training (SFT/RL per domain), and multi-teacher domain-routed on-policy distillation with salient vocabulary alignment (SVA; Eqs. 4–6) to unify six heterogeneous domains into one student. Empirically, Agents-A1 is reported to lead or match ~1T models (Kimi-K2.6, DeepSeek-V4-pro, GPT-5.5) on several long-horizon agent benchmarks (e.g., SEAL-0 56.4, IFBench 80.6, HiPhO 46.4, FS-O 79.0, MolBench-Bind 56.8 in the abstract), with staged ablations (Tables 4–9) and long-horizon case studies (12-hour MLE optimization; Earth-science closed loop).

Significance. If the central claim holds under fair, same-harness comparison, this is a practically important systems result for the agent community: it argues that long-horizon trajectory infrastructure plus multi-teacher on-policy distillation can substitute for extreme parameter scale on interactive agent tasks, and it ships a concrete recipe (KAG construction, domain teachers, domain-routed SVA OPD) that others can try to reproduce. Strengths include broad multi-benchmark coverage, staged ablations from base → SFT → teachers → OPD, explicit honesty that OPD does not always beat specialized teachers and that engineering (SciCode, MLE-Bench-Lite) still lags 1T models, and qualitative long-horizon applications. The SVA objective and domain-normalized multi-teacher aggregation are clear technical contributions relative to sampled-token OPD. The significance of the headline “35B reaches 1T” depends critically on evaluation protocol parity, which is currently uneven on the science suites that support several of the abstract’s “leading” claims.

major comments (4)
  1. §5.1 and Table 9 / Fig. 1: For HiPhO, FrontierScience-Olympiad (FS-O), and FrontierScience-Research (FS-R)—three of the five suites where the abstract claims leading results versus 1T models—comparison models are evaluated tool-free while Agents-A1 is trained and run with search/visit/code/scholar. The paper frames this as avoiding baseline degradation, but the reported leads (HiPhO 46.4, FS-O 79.0, FS-R 40.0) then confound horizon-scaling + OPD with asymmetric tool access at test time. Without same-harness numbers for Kimi-K2.6 / DeepSeek-V4-pro / GPT-5.5 under the tool-augmented protocol (or tool-free Agents-A1 under the baselines’ protocol), those science leads do not establish parameter-parity as stated. This is load-bearing for the abstract’s central claim and should be fixed or the claim reframed with explicit protocol caveats and matched ablations.
  2. Abstract vs Table 9: The abstract states Agents-A1 achieves “leading results … on … MolBench-Bind (56.8)” relative to 1T models, but Table 9 reports GPT-5.5 at 62.2 (and MatTools GPT-5.5 68.8 vs Agents-A1 47.1). “Leading” is therefore incorrect for MolBench-Bind under the paper’s own table. The abstract and Fig. 1 framing should be corrected to match Table 9, and the selective “trillion-parameter-level” language should be qualified where engineering and some scientific-agentic tasks still trail large closed models.
  3. §4.2.1, §4.2.3–4.2.4 and §5.1: Teacher RL and trajectory filtering rely on LLM judges, rule-based verifiers, and rubric models that are closely related to evaluation judges (e.g., search correctness LLM judge; IF rule validators; tool rubrics). Mild contamination / stack-favoring risk is not quantified. At minimum, report sensitivity to judge model choice, hold out a judge-independent subset, or show that gains persist under fully independent official judges not used in training. Without this, part of the “matches 1T” gap closure could be reward–eval alignment rather than genuine horizon scaling.
  4. Table 4 / §5.2.1 and Table 9: Full-domain SFT already hurts some domains (instruction following, HLE, τ2-Bench reproduction), and OPD is presented as resolving pattern conflict—but OPD still does not consistently match domain teachers (explicitly noted). The paper needs a clearer quantitative transfer analysis: per-domain teacher→student gap after OPD, and whether domain-normalized L_MT-SVA (Eq. 6) is necessary versus unweighted multi-teacher or single-teacher OPD. Without that ablation, the load-bearing claim that multi-teacher domain-routed SVA is what unifies heterogeneous abilities remains under-supported relative to “more long-horizon SFT + RL.”
minor comments (7)
  1. Title/abstract “trillion-parameter performance” is stronger than the body, which already notes lag on SciCode (44.3) and MLE-Bench-Lite (43.9 vs GPT-5.5 72.7). Soften abstract wording to “competitive with / matching on selected long-horizon agent suites.”
  2. §5.1 τ2-Bench: Official Qwen3.5-35B-A3B 81.2 vs reproduced ~32–33 is a large discrepancy; expand the discussion of codebase/environment differences so readers can interpret Table 8’s jump to 82.5.
  3. Eqs. (4)–(5): Specify the default top-k for SVA and report coverage ρ statistics; free parameter k is listed but not ablated in the main text.
  4. Figure 1 vs Table 9: Align all numbers (e.g., BrowseComp, GAIA, MolBench) and mark tool-use conditions so the radar/bar figure does not overstate parity.
  5. §2.2 Definition 1 and §3: KAG is well motivated; a small worked example linking one (s_t,a_t,o_t,v_t) chain from wiki search or MLE tree to the serialized training message would help reproducibility.
  6. Minor typos/consistency: “Salient Vocabulary Alignmen” (missing t) in contributions; “DeepSeek-V4-pro(Max)” vs “DSV4-Pro (Max)”; arXiv date line “29 Jun 2026” is fine if intentional but check metadata.
  7. §6 Future work on atomic abilities (planning, reflection, summarization) is appropriate; consider stating whether any of these were measured as intermediate metrics during the 12-hour MLE run (Fig. 4).

Circularity Check

1 steps flagged

Empirical systems paper: headline 35B-vs-1T scores are external benchmark measurements, not quantities forced by the training loss or by self-cited uniqueness.

specific steps
  1. self citation load bearing [§2.2.1 Definition 1 / citation [22]; also §3.2 [26], §4.2.4 [32]]
    "Motivated by previous work [22], we define the KAG as follows. Definition 1 (Knowledge-action graph). ... Trajectories are generated in an agentic harness that grows a tree of executable solution nodes, following prior solution-search workflows such as MLEvolve [26]. ... By data reuse as described above and applying a PAPO-style advantage to enhance GRPO [32]"

    Infrastructure and RL shaping pieces are justified partly by same-team prior work (Agents-K1, MLEvolve, PAPO). This is ordinary method reuse, not a uniqueness theorem that forces the headline performance claim; the 35B-vs-1T result still rests on external benchmark numbers. Flagged only as minor non-load-bearing self-citation (score contribution ~1), not as a reduction of the central claim to its inputs.

full rationale

Agents-A1 is a training-and-evaluation systems paper, not a closed-form derivation. The central claim (a 35B MoE agent reaches or matches ~1T models on long-horizon agent suites) is supported by reported pass rates on public leaderboards (SEAL-0, IFBench, HiPhO, FS-O, BrowseComp, SciCode, etc.), which are not defined by the SFT/RL/OPD objectives. The knowledge-action graph, domain teachers, and domain-routed SVA OPD are constructive training machinery; their success is tested externally rather than being true by construction. Self-citations (Agents-K1 for KAG motivation, MLEvolve for MLE tree search, PAPO for asymmetric advantage) supply method components and do not import a uniqueness theorem that forbids alternatives or forces the 35B=1T result. Mild contamination-style risks (LLM judges in teacher RL resembling eval judges; self-play KAG using the same tool stack as test) are evaluation-fairness / generalization concerns, not circular reductions of a claimed first-principles prediction to its inputs. Protocol asymmetry on science suites (tool-equipped Agents-A1 vs tool-free 1T baselines) is a load-bearing fairness issue for the science half of the headline, but again is not circularity of derivation. Score 1 only for routine non-load-bearing self-citation of infrastructure; no step reduces Eq. X to Eq. Y by definition or renames a fit as a prediction of the same quantity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 4 invented entities

Load-bearing content is empirical systems engineering: base model Qwen3.5-35B-A3B, constructed multi-domain trajectories, domain teachers, and OPD hyperparameters. No physical free constants. Free parameters are training/design knobs that materially affect reported scores. Axioms are standard ML practice plus domain assumptions about verifiers and judges. Invented entities are methodological constructs (KAG, SVA, domain-routed OPD), not physical objects; they lack independent evidence outside this training stack.

free parameters (6)
  • SVA top-k teacher support size k
    Controls the truncated reverse-KL support in Eq. 4; not fixed by theory and directly shapes distillation fidelity vs coverage ρ.
  • SFT learning rate 1e-5, batch 16, max seq 131072, 1 epoch
    Full-domain SFT hyperparameters (Table 3) that set the student initialization quality before teachers and OPD.
  • Search RL: GRPO clip [0.2,0.28], KL 0.001, entropy 0.0001, LR 1e-6, 8 rollouts, efficiency-penalty horizon K
    Hand-chosen RL knobs and efficiency/repetition penalties that define the search teacher policy used for distillation.
  • Tool RL process-advantage weight λ_neg=0.5 and hard-task set size 64 with heavy reuse
    Asymmetric PAPO-style advantage and extreme data reuse (Eqs. 7–9) strongly shape tool-calling teacher gains.
  • Domain-normalized multi-teacher aggregation over active domains D_B
    Equal domain weighting in Eq. 6 is a design choice that trades off specialty vs balance; different weights would change student scores.
  • Trajectory length / turn budgets (avg 45K tokens; up to 300 tool calls; 256K context in search collection)
    Horizon caps and filtering rules determine what supervision enters training and thus the central 'scale the horizon' claim.
axioms (5)
  • domain assumption GRPO-style group-relative policy optimization with the stated rewards improves long-horizon tool policies over SFT alone.
    Used throughout §4.2 teacher training; standard in recent LLM RL but not proved for these multi-turn agent settings.
  • domain assumption LLM judges and automatic verifiers provide sufficiently accurate correctness/process signals for RL and trajectory filtering.
    Search correctness reward, tool rubrics, and many data filters depend on judges/verifiers (§3–4).
  • ad hoc to paper Hard domain routing (one teacher per sample) plus SVA truncated reverse KL is a valid transfer objective for unifying heterogeneous agent policies.
    Core of §2.3; motivated by cited OPD analyses but specific multi-teacher domain-normalized form is paper-specific.
  • domain assumption Public long-horizon benchmarks (GAIA, BrowseComp, SEAL-0, SciCode, HLE, HiPhO, FS-O/R, IFBench, etc.) are adequate proxies for 'trillion-parameter-level' agent competence.
    Underpins the abstract and §5 claim comparison to Kimi-K2.6 / DeepSeek-V4-pro / GPT-5.5.
  • standard math Standard cross-entropy SFT on response tokens and MoE base Qwen3.5-35B-A3B are valid starting points for agent specialization.
    Ordinary supervised learning setup in §4.1; not novel but load-bearing.
invented entities (4)
  • Knowledge-action graph (KAG) G_d=(C_d,A_d,O_d,V_d) no independent evidence
    purpose: Unify evidence, actions, observations, and verifiers into process-level long-horizon supervision across domains.
    Definition 1 and §2.2 introduce KAG as the paper's core data abstraction; utility is shown only inside this training pipeline.
  • Salient Vocabulary Alignment (SVA) no independent evidence
    purpose: Replace single-token OPD with truncated reverse KL on teacher top-k tokens to stabilize multi-teacher distillation.
    Eqs. 4–5; a new training objective component without external validation beyond this student.
  • Domain-routed multi-teacher on-policy distillation objective L_MT-SVA no independent evidence
    purpose: Aggregate heterogeneous teacher signals without soft mixing or high-frequency domain domination.
    Eq. 6 and §4.3; the unification mechanism for the six-domain student.
  • Agents-A1 35B MoE student policy no independent evidence
    purpose: Single deployable agent claimed to reach 1T-level long-horizon performance.
    The artifact embodying the central claim; evidence is the reported benchmark table, not an external independent measurement.

pith-pipeline@v1.1.0-grok45 · 27680 in / 4660 out tokens · 52836 ms · 2026-07-14T16:57:20.429696+00:00 · methodology

0 comments
read the original abstract

We introduce Agents-A1, a 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling the agent horizon. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities. To support this goal, we build a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories with an average length of 45K tokens. Based on this, we train Agents-A1 with a three-stage recipe. First, we perform full-domain supervised fine-tuning to align the base model with broad agentic behaviors. Second, we train domain-level teacher models to capture specialized expertise in each domain. Third, we propose a multi-teacher domain-routed on-policy distillation with salient vocabulary alignment to improve knowledge transfer efficiency across different domains, unifying six heterogeneous domains into one deployable student model. Agents-A1 achieves strong and broad performance for long-horizon agent benchmarks. Compared with 1T-parameter model such as Kimi-K2.6 and DeepSeek-V4-pro, Agents-A1 achieves leading results on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8), and remains highly competitive on SciCode (44.3), HLE (47.6) and BrowseComp (75.5). We hope this work provides the community with a practical path for scaling the horizon using a 35B agent that can reach or match the performance of 1T models on long-horizon tasks.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation

    cs.CL 2026-07 conditional novelty 6.0

    In a controllable skill-graph gym, world-model CoT, a little long-horizon data, and high-quality trajectories acquire planning; OPD shapes patterns more reliably than GRPO; MOPD integrates only shared compatible patterns.

Reference graph

Works this paper leans on

44 extracted references · 18 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Kimi.Kimi K2.6: Advancing Open-Source Coding.https://www.kimi.com/blog/kimi-k2-

  2. [2]

    https://openai.com/index/introducing- gpt- 5- 5

    OpenAI.Introducing GPT-5.5. https://openai.com/index/introducing- gpt- 5- 5 . 2026

    2026

  3. [3]

    Anthropic.Introducing Claude Opus 4.6.https://www.anthropic.com/news/claude- opus-4-6. 2026

    2026

  4. [4]

    Gemini 3 Pro - Google DeepMind.url: https://deepmind.google/models/gemini/ pro/

  5. [5]

    DeepSeek-AI.DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence. 2026

    2026

  6. [6]

    Qwen.Qwen3.5: Towards Native Multimodal Agents.https://qwen.ai/blog?id=qwen3.5. 2026

    2026

  7. [7]

    MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

    Jun Shern Chan et al. “MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering”. In:arXiv preprint arXiv:2410.07095(2024)

    Pith/arXiv arXiv 2024

  8. [8]

    OpenAI.FrontierScience: Evaluating AI’s Ability To Perform Expert-level Scientific Tasks.https: //openai.com/index/frontierscience/. 2026

    2026

  9. [9]

    Humanity’s last exam

    Long Phan et al. “Humanity’s last exam”. In:arXiv preprint arXiv:2501.14249(2025)

    Pith/arXiv arXiv 2025

  10. [10]

    Browsecomp: A simple yet challenging benchmark for browsing agents

    Jason Wei et al. “Browsecomp: A simple yet challenging benchmark for browsing agents”. In: arXiv preprint arXiv:2504.12516(2025)

    Pith/arXiv arXiv 2025

  11. [11]

    Glm-5: from vibe coding to agentic engineering

    Aohan Zeng et al. “Glm-5: from vibe coding to agentic engineering”. In:arXiv preprint arXiv:2602.15763(2026)

    Pith/arXiv arXiv 2026

  12. [12]

    Towards Long-horizon Agentic Multimodal Search

    Yifan Du et al. “Towards Long-horizon Agentic Multimodal Search”. In:arXiv preprint arXiv:2604.12890(2026)

    Pith/arXiv arXiv 2026

  13. [13]

    Zhipu AI.GLM-5.2: Built for Long-Horizon Tasks.https://z.ai/blog/glm-5.2. 2026

    2026

  14. [14]

    Kimi K2. 5: Visual Agentic Intelligence

    Kimi Team et al. “Kimi K2. 5: Visual Agentic Intelligence”. In:arXiv preprint arXiv:2602.02276 (2026)

    Pith/arXiv arXiv 2026

  15. [15]

    Scaling long-horizon llm agent via context-folding

    Weiwei Sun et al. “Scaling long-horizon llm agent via context-folding”. In:arXiv preprint arXiv:2510.11967(2025)

    arXiv 2025

  16. [16]

    Cwm: An open-weights llm for research on code generation with world models

    Jade Copet et al. “Cwm: An open-weights llm for research on code generation with world models”. In:arXiv preprint arXiv:2510.02387(2025)

    arXiv 2025

  17. [17]

    SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models

    Thinh Pham et al. “SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models”. In:arXiv preprint arXiv:2506.01062(2025)

    Pith/arXiv arXiv 2025

  18. [18]

    Generalizing verifiable instruction following

    Valentina Pyatkin et al. “Generalizing verifiable instruction following”. In:Advances in Neural Information Processing Systems38 (2026)

    2026

  19. [19]

    HiPhO: How Far Are (M) LLMs from Humans in the Latest High School Physics Olympiad Benchmark?

    Fangchen Yu et al. “HiPhO: How Far Are (M) LLMs from Humans in the Latest High School Physics Olympiad Benchmark?” In:arXiv preprint arXiv:2509.07894(2025)

    arXiv 2025

  20. [20]

    MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization

    Lisheng Zhang et al. “MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization”. In:arXiv preprint arXiv:2604.21937 (2026). 26

    Pith/arXiv arXiv 2026

  21. [21]

    SciCode: A Research Coding Benchmark Curated by Scientists

    Minyang Tian et al. “SciCode: A Research Coding Benchmark Curated by Scientists”. In: Advances in Neural Information Processing Systems. Ed. by A. Globerson et al. Vol. 37. Cur- ran Associates, Inc., 2024, pp. 30624–30650.doi: 10 . 52202 / 079017 - 0963.url: https : / / proceedings . neurips . cc / paper _ files / paper / 2024 / file / 36850592258c8c41cec...

    2024

  22. [22]

    Agents-K1: Towards Agent-native Knowledge Orchestration

    Zongsheng Cao et al. “Agents-K1: Towards Agent-native Knowledge Orchestration”. In:arXiv preprint arXiv:2606.13669(2026)

    Pith/arXiv arXiv 2026

  23. [23]

    On-Policy Distillation

    Kevin Lu and Thinking Machines Lab. “On-Policy Distillation”. In:Thinking Machines Lab: Con- nectionism(2025). https://thinkingmachines.ai/blog/on-policy-distillation.doi: 10.64434/ tml.20251026

    2025

  24. [24]

    Revisiting on-policy distillation: Empirical failure modes and simple fixes

    Yuqian Fu et al. “Revisiting on-policy distillation: Empirical failure modes and simple fixes”. In:arXiv preprint arXiv:2603.25562(2026)

    Pith/arXiv arXiv 2026

  25. [25]

    MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering

    Rushi Qiang et al. “MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering”. In:arXiv preprint arXiv:2505.07782(2025)

    Pith/arXiv arXiv 2025

  26. [26]

    MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

    Shangheng Du et al. “MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery”. In:arXiv preprint arXiv:2606.06473(2026)

    Pith/arXiv arXiv 2026

  27. [27]

    GitHub repository

    NVIDIA.NeMo Gym: An Open Source Library for Scaling Reinforcement Learning Environments for LLM.https://github.com/NVIDIA-NeMo/Gym. GitHub repository. 2025

    2025

  28. [28]

    WildChat: 1M ChatGPT Interaction Logs in the Wild

    Wenting Zhao et al. “WildChat: 1M ChatGPT Interaction Logs in the Wild”. In:The Twelfth International Conference on Learning Representations. 2024.url:https://openreview. net/forum?id=Bl8u7ZRlbM

    2024

  29. [29]

    Victor Barres et al.𝜏2-Bench: Evaluating Conversational Agents in a Dual-Control Environment

  30. [30]

    arXiv:2506.07982 [cs.AI].url:https://arxiv.org/abs/2506.07982

    Pith/arXiv arXiv

  31. [31]

    VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real- world Applications

    Wei He et al. “VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real- world Applications”. In:arXiv preprint arXiv:2509.26490(2025)

    arXiv 2025

  32. [32]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao et al. “Deepseekmath: Pushing the limits of mathematical reasoning in open language models”. In:arXiv preprint arXiv:2402.03300(2024)

    Pith/arXiv arXiv 2024

  33. [33]

    ZelinTanetal.PAPO:StabilizingRubricIntegrationTrainingviaDecoupledAdvantageNormaliza- tion. 2026. arXiv:2603.26535 [cs.AI].url:https://arxiv.org/abs/2603.26535

    Pith/arXiv arXiv 2026

  34. [34]

    Gaia: a benchmark for general ai assistants

    Grégoire Mialon et al. “Gaia: a benchmark for general ai assistants”. In:International Conference on Learning Representations. Vol. 2024. 2024, pp. 9025–9049

    2024

  35. [35]

    xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations

    Kaiyuan Chen et al. “xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations”. In:arXiv preprint arXiv:2506.13651(2025)

    Pith/arXiv arXiv 2025

  36. [36]

    Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks

    Yushi Bai et al. “Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks”. In:Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025, pp. 3639–3664

    2025

  37. [37]

    Instruction-Following Evaluation for Large Language Models

    Jeffrey Zhou et al. “Instruction-Following Evaluation for Large Language Models”. In:arXiv preprint arXiv:2311.07911(2023)

    Pith/arXiv arXiv 2023

  38. [38]

    Deepseek-v3. 2: Pushing the frontier of open large language models

    Aixin Liu et al. “Deepseek-v3. 2: Pushing the frontier of open large language models”. In:arXiv preprint arXiv:2512.02556(2025)

    Pith/arXiv arXiv 2025

  39. [39]

    Siyu Liu et al.MatTools: Benchmarking Large Language Models for Materials Science Tools. 2025. arXiv: 2505 . 10852 [cond-mat.mtrl-sci].url: https : / / arxiv . org / abs / 2505 . 10852. 27

    2025

  40. [40]

    2013.url: https://sabiod.lis-lab.fr/icml2013/challenge_description.html (visited on 06/17/2026)

    ICML 2013 Workshop on Machine Learning for Bioacoustics.Challenge Description. 2013.url: https://sabiod.lis-lab.fr/icml2013/challenge_description.html (visited on 06/17/2026)

    2013

  41. [41]

    The International Best Track Archive for Climate Stewardship (IB- TrACS): Unifying tropical cyclone best track data

    Kenneth R. Knapp et al. “The International Best Track Archive for Climate Stewardship (IB- TrACS): Unifying tropical cyclone best track data”. In:Bulletin of the American Meteorological Society(2010).doi:10.1175/2009BAMS2755.1

    doi:10.1175/2009bams2755.1 2010

  42. [42]

    International Best Track Archive for Climate Stewardship (IBTrACS) Project

    J. Gahtan et al. “International Best Track Archive for Climate Stewardship (IBTrACS) Project”. In:NOAANationalCentersforEnvironmentalInformation(2024).doi: 10.25921/82ty-9e16

    doi:10.25921/82ty-9e16 2024

  43. [43]

    2008.url:https: //www.metoc.navy.mil/jtwc/products/atcr/2008atcr.pdf

    Joint Typhoon Warning Center.Annual Tropical Cyclone Report 2008. 2008.url:https: //www.metoc.navy.mil/jtwc/products/atcr/2008atcr.pdf

    2008

  44. [44]

    Updated 2025-09-23

    NOAANationalCentersforEnvironmentalInformation.IBTrACSv04r01ColumnDocumentation. Updated 2025-09-23. 2025.url: https : / / www . ncei . noaa . gov / sites / default / files/2025-09/IBTrACS_v04r01_column_documentation.pdf. 28 A. Appendix A.1. Contributions and Acknowledgments Knowledge-Action Infrastructure:Zongsheng Cao†, Bihao Zhan, Zhijie Zhong Full-dom...

    2025