Pith. sign in

REVIEW 4 major objections 5 minor 24 references

Distilling a large language model's agent skills into a small student can fail when the student's early mistakes pull the trajectory outside the teacher's comfort zone; DASH-OPD prevents this by switching between student and teacher based o

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 14:02 UTC pith:UAGSU34P

load-bearing objection DASH-OPD's switching framework is clean and the ALFWorld results are consistent, but the z-score normalization in Eq. (7) can blind the controller to sustained absolute discrepancy — a real mismatch with the stated goal — and the evaluation lacks ablations and robustness checks to back the causal claim. the 4 major comments →

arxiv 2607.29078 v1 pith:UAGSU34P submitted 2026-07-31 cs.LG

DASH-OPD: Discrepancy-Aware Switching with Hysteresis for On-Policy Distillation

Yuchen Xia , Qianguo Sun , Chao Song , Junlong Wu , Yiyan Qi , Yunjian Xu This is my paper
classification cs.LG
keywords on-policy distillationmulti-turn agentsdiscrepancy-aware switchinghysteresisdrift evidencerecovery evidenceALFWorldteacher-student intervention
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Distilling a large language model's agent skills into a small student usually trains the student on its own rollouts, but when the student makes an early mistake, the rest of the trajectory leaves the teacher's comfort zone and the teacher's advice loses value. DASH-OPD tries to solve this by watching, at every turn, how surprised the teacher is by the student's action tokens, and vice versa. Those surprise signals are accumulated over several turns into drift evidence (how much support the student owes) and recovery evidence (how much return credit the student has earned). When drift evidence crosses a threshold, the teacher takes over; when recovery evidence crosses a lower threshold, control returns to the student. On a text-based household-agent benchmark, this adaptive, hysteresis-protected switching beats fixed or random curriculum schedules on success rate while using 35.4% fewer teacher turns and 77.1% fewer switches. The paper itself cautions that surprise is an imperfect proxy for risk.

Core claim

DASH-OPD's central claim is that the right unit for deciding teacher intervention is not training progress but the trajectory-level disagreement between the two models. It defines a mean log-probability ratio over action tokens for each turn, in both directions, and turns these into two running evidence counters. A student-to-teacher ratio grows drift evidence; a teacher-to-student ratio grows recovery evidence. The controller switches teacher support on when drift evidence exceeds tau_on and returns control when recovery evidence exceeds tau_off; thresholds (3,1) keep the switching hysteretic. On ALFWorld, this yields the best success rate at both student scales, with the 4B student surpass

What carries the argument

The switching controller is built on two evidence counters, drift evidence D_t and recovery evidence R_t, updated by normalized, clipped versions of the per-turn mean log-probability ratio between student and teacher on action tokens. This ratio estimates reverse KL when the student acts and forward KL when the teacher acts. The evidence accumulates over turns (Eqs. 8-9), so a switch requires sustained signals rather than a single fluctuation; this hysteresis is what prevents flapping. Fixed thresholds tau_on=3, tau_off=1, a minimum teacher-segment length, a teacher budget, and a stagnation detector complete the controller.

Load-bearing premise

The load-bearing premise is that how much the large teacher dislikes the small student's action choices reflects when the small student truly needs help; the paper itself says this is imperfect, because both models can be wrong together and harmless differences can set off the alarm.

What would settle it

A reader could log every switch decision during ALFWorld training and compare it against an oracle that labels each turn as 'needs teacher' or 'does not' (for example, whether the next teacher turn actually improves the probability of task success). If DASH-OPD's switches are frequently wrong — triggering teacher help on trajectories that succeed anyway, or staying silent when the trajectory is unrecoverable — the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Teacher support is deployed only where it is needed: DASH-OPD reduces teacher-generated turns by 35.4% and executor switches by 77.1% relative to the guided baseline while improving success rate.
  • The 4B student trained with DASH-OPD reaches 41.24% overall success on the full evaluation set, surpassing the 30B zero-shot teacher (40.88%).
  • Deployment cost falls: DASH-OPD generates fewer response tokens per trajectory than every OPD baseline at both student scales, so a given success rate is reached at lower inference cost.
  • Post-return student persistence is high (91% of returns last at least three turns for 4B), showing the hysteresis mechanism prevents oscillation.
  • The same controller configuration works at two student scales, suggesting the thresholds are not tightly coupled to a particular student size.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the discrepancy proxy transfers, the same evidence-switching structure could apply to other long-horizon interactive tasks such as web navigation or tool use; the paper tests only a text-based household environment.
  • The fixed thresholds are an obvious next tuning target: one could lower them early in training and raise them later, or make them depend on trajectory difficulty; the paper leaves this unexplored.
  • Because the paper is a work in progress with single runs, exact numbers (35.4%, 77.1%, success rates) should be read as provisional until replication and variance reporting.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DASH-OPD, a method for multi-turn on-policy distillation that switches between a student and a frozen teacher based on an adaptive, hysteretic discrepancy signal. At each turn the non-executing model scores the action tokens under the same history, yielding a mean log-probability ratio (Eqs. 5–6). These directional signals are z-scored against trajectory-local statistics and accumulated into drift and recovery evidence (Eqs. 8–9); switching occurs when evidence crosses hand-set thresholds, with additional safeguards including a stagnation flag, a teacher budget, and a minimum teacher-segment length. On ALFWorld, the authors report that DASH-OPD achieves the highest success rate on every split and student scale, while using fewer teacher turns and fewer executor switches than Guided-OPD. The manuscript is explicitly labeled a work in progress, and code, training logs, and checkpoints are promised but not yet released.

Significance. If the reported results are robust, DASH-OPD is a meaningful step forward for agentic on-policy distillation: it offers a concrete, interpretable hysteresis mechanism for bidirectional teacher–student switching, and the reported Pareto dominance over baseline methods on deployment cost and success rate would be practically valuable. The algorithmic exposition is clear, and the directional separation of drift and recovery signals is a sensible design that goes beyond fixed curriculum schedules. However, the current evidence is not yet strong enough to establish the central claim. The evaluation rests on a single seed, a single attempt per task, no confidence intervals, hand-set thresholds without sensitivity analysis, and no released code. More seriously, the normalization in Eq. (7) appears mismatched with the claimed goal of detecting departure from the teacher's familiar domain, because it responds to changes in discrepancy rather than to its absolute level.

major comments (4)
  1. [§4.3, Eq. (7)] The z-score normalization in Eq. (7) is a load-bearing concern. Each signal is normalized by its trajectory-local running mean and variance. If a student trajectory begins already far from the teacher and remains consistently discrepant, the normalized drift signal is approximately zero and drift evidence never exceeds τ_on; the controller cannot detect that the trajectory has left the teacher's familiar domain unless the discrepancy is still increasing. This is not merely the 'imperfect proxy' limitation stated in §6 — it is an internal mismatch between the stated goal (detect departure) and the implemented signal (detect change). The paper provides no ablation that isolates the discrepancy evidence from the stagnation flag g_t, warm-start initialization (Eq. 12), the teacher budget q, or the minimum segment length ℓ_min, so the reported gains cannot be confidently attributed to the dis
  2. [§5.1–§5.2, Table 1] The central empirical claim is based on a single evaluation seed and one attempt per task ('seed 42', 'Each task is attempted once'). On 274 tasks, the standard error of a success rate near 32.85% is about 2.8 percentage points; the headline improvement of +5.84 pp over TCOD is therefore only about 2 SE, and the +3.28 pp improvement over Guided-OPD at 4B is about 1.2 SE. Without multiple independent evaluation seeds, confidence intervals, or per-task variance, the 'highest success rate across all splits and student scales' claim is not statistically grounded. The teacher-turn and switch-count reductions in §5.2 similarly lack any measure of run-to-run variability.
  3. [§4.4, §5.2] There is no ablation study separating the contribution of the discrepancy-based switching from the other controller components: the stagnation flag g_t, the linear warm-start in Eq. (12), the budget q, the minimum segment length ℓ_min, and the threshold behavior of Eqs. (10)–(11). The reported efficiency gains (35.4% fewer teacher turns, 77.1% fewer switches) could in principle be driven by these auxiliary heuristics rather than by the discrepancy signal. Given that Eq. (7) may be blind to sustained absolute discrepancy, an ablation is essential to support the paper's central attribution.
  4. [§5.1, Abstract] The manuscript is labeled 'work in progress' and states that code, training logs, and model checkpoints 'will be released later.' No code or checkpoints are currently available, and the empirical results are not reproducible from the paper alone. For a paper whose contribution is empirical and whose method depends on several hand-set hyperparameters (τ_on=3, τ_off=1, ℓ_min=2, K=3, q=1, ρ=1), releasing the exact training and evaluation pipeline is necessary for the claims to be verified.
minor comments (5)
  1. [§4.2, Eq. (7)] Notation is inconsistent: the executor is denoted m_t in Eq. (4) but m_t^t in Eq. (7). Please unify. Also, the phrase 'thet-th turn' in §4.5 contains a typo.
  2. [§5.2, Table 1] The table mentions an 'ALFWorld hard set' that is excluded from the Overall column but is never defined in the text. Either define it or remove the mention.
  3. [§4.4, Eq. (10)] The stagnation flag g_t is described only informally ('an action or an observation repeats for K consecutive student turns'). The precise condition — which tokens count, whether the repetition is exact or approximate, and how it interacts with the evidence update — should be specified in the pseudocode or text.
  4. [§5.1] The evaluation uses two splits (IID and OOD) and 274 tasks, but all tasks are evaluated with a single seed and a single temperature. Reporting the number of runs, standard deviations, and any task-level bootstraps would substantially strengthen the empirical claims.
  5. [Figure 3] The 'teacher reference' star is useful, but the text claims 'strictly Pareto-dominates every distilled baseline.' For the 1.7B student the token-count savings relative to Guided-OPD are 89.9 tokens, while for the 4B student they are 12.1 tokens; please clarify whether these differences are above the noise level of token generation.

Circularity Check

0 steps flagged

No circularity: the switching mechanism is empirically evaluated, not derived from the reported success-rate numbers.

full rationale

DASH-OPD's contribution is an adaptive switching algorithm whose inputs—the mean log-probability ratios in Eqs. (5)–(6)—are computed directly from student and teacher model probabilities, and whose outputs (executor choices) are evaluated against an external ALFWorld success-rate benchmark. The switching thresholds (τ_on=3, τ_off=1) and other controller hyperparameters are fixed constants; the paper does not fit them to the reported SR values, and no equation identifies the success rate with the discrepancy signal. The central claim is therefore an empirical result, not a derivation whose conclusion is contained in its premises. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation: the method explicitly follows Guided-OPD's loss formulation with an external citation, and the discrepancy signal is not a renamed known result. The z-score normalization in Eq. (7) and the paper's own limitation that 'student-teacher discrepancy is an imperfect proxy for risk' are potential design or generalization concerns, but they do not make the reported gains true by construction. Thus no circularity is present.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The central claim rests on (i) the assumption that the action-token log-probability ratio is a reliable proxy for the student leaving the teacher's familiar domain, and (ii) a set of hand-chosen controller hyperparameters. The paper itself acknowledges (i) is imperfect in Section 6. No new entities are introduced; the drift/recovery evidence variables are algorithmic state.

free parameters (7)
  • tau_on = 3
    Drift-evidence threshold for teacher support (Eq. 10). Hand-set, same for both student scales, with no sensitivity analysis.
  • tau_off = 1
    Recovery-evidence threshold for returning control to the student (Eq. 11). Hand-set, no sensitivity analysis.
  • lmin = 2
    Minimum teacher-segment length before control can return (Eq. 11). Hand-set.
  • K = 3
    Stagnation flag g_t activates after K repeated actions/observations (Eq. 10). Hand-set.
  • rho = 1
    Evidence retention factor in Eqs. (8)-(9); rho=1 means no forgetting of historical evidence.
  • vmin, dmax, kappa_S, kappa_T = 1e-6, unbounded, 0, 0
    Normalization variance floor, signal bound, and slack parameters; set to simplified values (no bound, no slack).
  • q = 1
    Teacher budget fraction; Hmax = floor(q*Tmax) = 30 turns, so the full horizon is available for teacher support.
axioms (5)
  • domain assumption Mean log-probability ratio over action tokens estimates teacher-student divergence and indicates need for teacher support
    Eqs. (5)-(6) define the signal; Section 6 concedes 'Student-teacher discrepancy is an imperfect proxy for risk: shared errors may remain undetected, whereas benign differences may trigger intervention.'
  • domain assumption Only action tokens (mask A_t) are used for discrepancy; reasoning-token divergence is irrelevant
    Eqs. (5)-(6) restrict the discrepancy to parsed action content; the paper gives no evidence that reasoning-token divergence is unimportant for trajectory drift.
  • domain assumption CUSUM-style accumulation with fixed thresholds detects persistent drift without excessive switching
    Equations (7)-(9) implement a CUSUM-like controller; no sensitivity analysis or theoretical guarantees are provided for the threshold choices.
  • domain assumption The frozen teacher policy remains a valid reference for the student throughout training
    The 4B student's success rate exceeds the teacher reference, suggesting the teacher may not always be the right oracle for 'familiar domain'.
  • domain assumption Standard reverse-KL/forward-KL distillation objective (Eq. 13) is appropriate for OPD
    Adopted from Guided-OPD [10]; not independently justified in this paper.

pith-pipeline@v1.3.0-daily-deepseek · 9343 in / 13207 out tokens · 128076 ms · 2026-08-03T14:02:20.714291+00:00 · methodology

0 comments
read the original abstract

On-policy distillation (OPD) trains student models on their own rollouts to reduce exposure bias. However, in multi-turn agent scenarios, early student errors can lead a trajectory away from the teacher's familiar domain. Existing curriculum learning methods regulate how much teacher support is used according to training progress, but cannot determine when it is needed. In light of this, we propose DASH-OPD, Discrepancy-Aware Switching with Hysteresis for OPD, a new agentic OPD method that can switch executors adaptively and bidirectionally. On each turn, DASH-OPD calculates a mean log-probability ratio between the two executors over action tokens as their discrepancy. Student-to-teacher ratios on student turns form drift signals, while teacher-to-student ratios on teacher turns form recovery signals. These signals are normalized and accumulated over multiple turns into drift and recovery evidence. DASH-OPD switches executors when the evidence exceeds its corresponding switching threshold. This multi turn accumulation makes the switching hysteretic, preventing high-frequency switches caused by transient fluctuations. On ALFWorld, DASH-OPD outperforms all the baselines and demonstrates superior training and deployment efficiency. This paper is a work in progress. Code, training logs, and model checkpoints will be released later.

Figures

Figures reproduced from arXiv: 2607.29078 by Chao Song, Junlong Wu, Qianguo Sun, Yiyan Qi, Yuchen Xia, Yunjian Xu.

Figure 1
Figure 1. Figure 1: Existing rollout curricula use fixed or random schedules for teacher intervention. Large language model (LLM) agents solve long￾horizon tasks by interleaving reasoning, ac￾tions, and environment feedback [1, 2, 3], but their strongest capabilities often reside in mod￾els with excessively high deployment costs [4]. Knowledge distillation [5, 6, 7] can transfer such capabilities to smaller student models. On… view at source ↗
Figure 2
Figure 2. Figure 2: DASH-OPD accumulates teacher–student discrepancy into drift and recovery evidence, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Deployment efficiency on the full ALFWorld validation set. Overall SR is plotted against [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Teacher intervention profiles for the Qwen3-4B student. We plot the cumulative number of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Persistence after returning control to the student during DASH-OPD training. Left: the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 7 linked inside Pith

  1. [1]

    ReAct: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=WE_vluYUL-X

    2023

  2. [2]

    ScienceWorld: Is your agent smarter than a 5th grader?

    R. Wang, P. Jansen, M.-A. Côté, and P. Ammanabrolu, “ScienceWorld: Is your agent smarter than a 5th grader?” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2022, pp. 11 279–11 298. [Online]. Available: https://aclanthology.org/2022.emnlp-main.775/

    2022

  3. [3]

    WebShop: Towards scalable real-world web interaction with grounded language agents,

    S. Yao, H. Chen, J. Yang, and K. Narasimhan, “WebShop: Towards scalable real-world web interaction with grounded language agents,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 20 744–20 757. [Online]. Available: https://proceedings.neurips.cc/ paper_files/paper/2022/hash/82ad13ec01f9fe44c01cb91814fd7b8c-Abstract-Conference.html

    2022

  4. [4]

    FrugalGPT: How to use large language models while reducing cost and improving performance,

    L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,” Transactions on Machine Learning Research, 2024. [Online]. Available: https://openreview.net/forum?id=cSimKw5p6R

    2024

  5. [5]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015. [Online]. Available: https://arxiv.org/abs/1503.02531

    Pith/arXiv arXiv 2015

  6. [6]

    Sequence-level knowledge distillation,

    Y . Kim and A. M. Rush, “Sequence-level knowledge distillation,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, 2016, pp. 1317–1327. [Online]. Available: https://aclanthology.org/D16-1139/

    2016

  7. [7]

    MiniLLM: Knowledge distillation of large language models,

    Y . Gu, L. Dong, F. Wei, and M. Huang, “MiniLLM: Knowledge distillation of large language models,” in International Conference on Learning Representa- tions, 2024. [Online]. Available: https://proceedings.iclr.cc/paper_files/paper/2024/hash/ 8ac015d409635f196f9e3e9dcfb9a94e-Abstract-Conference.html

    2024

  8. [8]

    On-policy distillation of language models: Learning from self-generated mistakes,

    R. Agarwal, N. Vieillard, Y . Zhou, P. Stanczyk, S. R. Garea, M. Geist, and O. Bachem, “On-policy distillation of language models: Learning from self-generated mistakes,” in International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=3zKtaqxLhW

    2024

  9. [10]

    On-policy distillation with curriculum turn-level guidance for multi-turn agents,

    G. Li, M. Zheng, M. Song, R. Liu, T. Yang, J. Sun, Q. Zhong, H. Guo, J. Fang, D. Zhang, and J. Wang, “On-policy distillation with curriculum turn-level guidance for multi-turn agents,” arXiv preprint arXiv:2606.15912, 2026. [Online]. Available: https://arxiv.org/abs/2606.15912

    arXiv 2026

  10. [11]

    ThriftyDAgger: Budget-aware novelty and risk gating for interactive imitation learning,

    R. Hoque, A. Balakrishna, E. Novoseller, A. Wilcox, D. S. Brown, and K. Goldberg, “ThriftyDAgger: Budget-aware novelty and risk gating for interactive imitation learning,” in Proceedings of the 5th Conference on Robot Learning , ser. Proceedings of Machine Learning Research, vol. 164. PMLR, 2022, pp. 598–608. [Online]. Available: https://proceedings.mlr.p...

    2022

  11. [12]

    Robot-gated interactive imitation learning with adaptive intervention mechanism,

    H. Cai, Z. Peng, and B. Zhou, “Robot-gated interactive imitation learning with adaptive intervention mechanism,” in Proceedings of the 42nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 267. PMLR, 2025, pp. 6243–6256. [Online]. Available: https://proceedings.mlr.press/v267/cai25e.html

    2025

  12. [13]

    ALFWorld: Aligning text and embodied environments for interactive learning,

    M. Shridhar, X. Yuan, M.-A. Côté, Y . Bisk, A. Trischler, and M. Hausknecht, “ALFWorld: Aligning text and embodied environments for interactive learning,” in International Conference on Learning Representations, 2021. [Online]. Available: https://arxiv.org/abs/2010.03768

    Pith/arXiv arXiv 2021

  13. [14]

    Curriculum learning,

    Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” inProceedings of the 26th Annual International Conference on Machine Learning . ACM, 2009, pp. 41–48. [Online]. Available: https://icml.cc/Conferences/2009/papers/119.pdf

    2009

  14. [15]

    TurnOPD: Making on-policy distillation turn-aware for efficient long-horizon agent training,

    Y . Zhou, K. Zheng, H. Li, D. Peng, C. Xu, and J. Chen, “TurnOPD: Making on-policy distillation turn-aware for efficient long-horizon agent training,” arXiv preprint arXiv:2607.05804, 2026. [Online]. Available: https://arxiv.org/abs/2607.05804

    Pith/arXiv arXiv 2026

  15. [16]

    SWITCH: Studying with teacher for knowledge distillation of large language models,

    J. Koo, Y . Hwang, Y . Kim, T. Kang, H. Bae, and K. Jung, “SWITCH: Studying with teacher for knowledge distillation of large language models,” in Findings of the Association for Computational Linguistics: NAACL 2025. Association for Computational Linguistics, 2025, pp. 3733–3746. [Online]. Available: https://aclanthology.org/2025.findings-naacl.206/

    2025

  16. [17]

    AdaSwitch: Balancing exploration and guidance in knowledge distillation via adaptive switching,

    J. Peng, M. Wang, H. Cai, Y . Li, K. Zhang, S. Wang, D. Yin, and X. Zhao, “AdaSwitch: Balancing exploration and guidance in knowledge distillation via adaptive switching,”arXiv preprint arXiv:2510.07842, 2025. [Online]. Available: https://arxiv.org/abs/2510.07842

    arXiv 2025

  17. [18]

    Are full rollouts necessary for on-policy distillation?

    Y . Zhang, J. Chai, Y . Fu, S. Tu, X. Wang, W. Lin, G. Yin, Q. Zhang, Y . Zhu, and D. Zhao, “Are full rollouts necessary for on-policy distillation?” arXiv preprint arXiv:2605.31490 , 2026. [Online]. Available: https://arxiv.org/abs/2605.31490

    Pith/arXiv arXiv 2026

  18. [19]

    Less is more: Early stopping rollout for on-policy distillation,

    Z. Zhou, J. Li, H. Tang, Y . N. Wu, and D. Terzopoulos, “Less is more: Early stopping rollout for on-policy distillation,” arXiv preprint arXiv:2605.27028, 2026. [Online]. Available: https://arxiv.org/abs/2605.27028

    Pith/arXiv arXiv 2026

  19. [20]

    AgentBench: Evaluating LLMs as agents,

    X. Liu, H. Yu, H. Zhang, Y . Xu, X. Lei, H. Lai, Y . Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y . Su, H. Sun, M. Huang, Y . Dong, and J. Tang, “AgentBench: Evaluating LLMs as agents,” inInternational Conference on Learning Representations, 2024. [Online]. Available: https://proceedings.iclr.cc/paper_fil...

    2024

  20. [21]

    On information and sufficiency,

    S. Kullback and R. A. Leibler, “On information and sufficiency,” The Annals of Mathematical Statistics , vol. 22, no. 1, pp. 79–86, 1951. [Online]. Available: https://projecteuclid.org/euclid.aoms/1177729694

    arXiv 1951

  21. [22]

    Note on a method for calculating corrected sums of squares and products,

    B. P. Welford, “Note on a method for calculating corrected sums of squares and products,” Technometrics, vol. 4, no. 3, pp. 419–420, 1962

    1962

  22. [23]

    Continuous inspection schemes,

    E. S. Page, “Continuous inspection schemes,” Biometrika, vol. 41, no. 1/2, pp. 100–115, 1954

    1954

  23. [24]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al., “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025. [Online]. Available: https://arxiv.org/abs/2505.09388 10

    Pith/arXiv arXiv 2025

  24. [2026]

    Available: https://arxiv.org/abs/2604.24005

    [Online]. Available: https://arxiv.org/abs/2604.24005

    Pith/arXiv arXiv