Pith. sign in

REVIEW 4 major objections 4 minor 26 references

PRADA removes the process reward model from the online path, distilling its quality judgments into a lightweight edge policy and a threshold-based scheduler, and claims that most of the LLM's accuracy survives at a fraction of the latency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 15:13 UTC pith:A2WLWH22

load-bearing objection A smart two-stage architecture with a plausible but unverified reward assumption; the optimality theorem is overstated and the simulations need error bars and calibration analysis. the 4 major comments →

arxiv 2607.18244 v1 pith:A2WLWH22 submitted 2026-04-29 cs.NI cs.ITmath.IT

Accelerating Heterogeneous Agent Collaboration in Dynamic Edge Networks

Tianji He , Yulin Shao , Fen Hou This is my paper
classification cs.NI cs.ITmath.IT
keywords heterogeneous agent collaborationprocess reward modeledge LLM inferenceoffline distillationLagrangian schedulingthreshold policyquality-latency trade-offdynamic edge networks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the process reward model (PRM) used to judge reasoning quality can be taken out of the online loop entirely. In the proposed PRADA framework, a compact network trained offline by the PRM screens each reasoning step at the edge, and a server-side Lagrangian scheduler resolves contention for parallel capacity and bandwidth. The paper proves that a first-stage 'stay local' decision remains optimal even after communication and queuing penalties are added, and that the optimal server policy has a threshold structure. Simulations on three reasoning benchmarks show most of the large model's accuracy is retained while end-to-end latency drops substantially, and that performance saturates past critical levels of server parallelism and bandwidth.

Core claim

The central claim is that the PRM's role can be changed from an online evaluator to an offline supervisor: its step-level score differences become the reward for training a small binary screening policy, and at run time only that policy and a threshold scheduler act. Under this design, context is uploaded only for nominated steps, the candidate set shrinks by an order of magnitude, and the PRM's latency and memory costs disappear from the online path. Proposition 1 shows that if the learned value function judges 'stay local' at least as good as 'offload' under the coarse reward that ignores communication and queuing, it remains optimal when those penalties are added; hence the offline-traine

What carries the argument

The load-bearing object is the process reward model (PRM), a neural network that scores how likely a partial reasoning state is to lead to a correct final answer. PRADA uses it only in training: the reward for each step is PRM(next state) - PRM(current state) minus a scaled computation cost, and this reward trains a few-hundred-thousand-parameter edge policy. The other central piece is the Lagrangian relaxation of the server's resource-constrained allocation problem; it yields two per-request quantities — the net advantage of immediate execution over local, and the net advantage of queueing over local — and the optimal action is a threshold comparison against the Lagrange multipliers, with b

Load-bearing premise

The load-bearing premise is that the process reward model's per-step score differences are a faithful measure of final reasoning correctness on the deployment benchmarks; if that reward is miscalibrated or distribution-shifted, the distilled policy's offloading choices would not track true accuracy and the accuracy-retention claim would collapse.

What would settle it

Measure, on the deployment benchmarks (gsm8k, gaokao2023en, mmlu_stem), whether the PRM's step-level score increment PRM(x_{k+1}) - PRM(x_k) is positively correlated with the probability that the trajectory reaches a correct final answer. If the correlation is weak or negative on any of these distributions, the reward signal used to train the screening policy is not faithful, and the central accuracy-retention claim would lack support.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The online inference path no longer contains a PRM forward pass, eliminating the latency and memory overhead that made PRM-guided collaboration impractical in multi-user systems.
  • Because a Stage-1 decision to stay local is provably safe, the edge screening policy and the server scheduler can be designed and tuned independently.
  • The threshold structure of the optimal scheduler means the server can act with simple per-request comparisons instead of solving a mixed-integer program every slot.
  • Provisioning guidance: critical values of server parallelism and bandwidth mark where gains saturate; beyond them the bottleneck shifts from queuing to computation or from communication to contention, so resources can be provisioned jointly and moderately.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The offline-teacher pattern could extend beyond PRMs: any expensive auxiliary model that supervises a lightweight orchestrator (verifiers, search heuristics, or other reward models) might be confined to training in resource-constrained multi-agent systems.
  • The saturation phenomenon suggests a practical provisioning heuristic: add parallel capacity and bandwidth until the accuracy-latency frontier stops moving; the simulations indicate the knee is identifiable without knowing the task mix.
  • A stress test of the framework's foundation would be to deliberately miscalibrate the PRM reward during training and observe how much accuracy retention degrades; this would isolate whether the PRM's score differences, rather than some other property of the policy, are carrying the result.
  • The reward is defined on consecutive-step PRM score differences and assumes a Markovian, additive structure; an extension could relax this to non-Markovian rewards or learned advantage functions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes PRADA, a two-stage framework for heterogeneous collaboration between edge SLMs and a server LLM in dynamic multi-user networks. Stage 1 trains a lightweight binary screening policy offline using a process reward model (PRM) as a dense reward teacher, eliminating online PRM inference. Stage 2 is a centralized server scheduler that assigns final actions (local, queue, or immediate execution) using a Lagrangian relaxation and a claimed threshold-structured optimal policy. The authors prove a monotonicity preservation result for local decisions (Proposition 1) and a threshold optimality theorem for the scheduler (Theorem 2). Simulations on gsm8k, gaokao2023en, and mmlu_stem report that PRADA retains most of the LLM accuracy while substantially reducing latency, and they identify threshold effects for server concurrency and bandwidth.

Significance. If the central claims are correct, PRADA is a practically relevant contribution: it removes the PRM from the online loop, provides a unified FLOP-level latency model, and gives structural insights into provisioning computation and communication resources. The idea of using a large teacher model offline to supervise a tiny online policy is appealing and broadly applicable. The paper also contains analytic results (Propositions 1 and 2) that aim to justify the two-stage decoupling. However, the empirical accuracy-retention claim rests on an unvalidated reward signal, and the theoretical optimality statements are stronger than what the proofs actually establish. With the requested calibration analysis and a proper duality-gap discussion, the contribution could be solid.

major comments (4)
  1. [§V-A1, Eqs. (20)–(21)] The PRM score-difference reward is the sole semantic quality signal for training π_θ, but it is trained only on math500-test and evaluated on gsm8k, gaokao2023en, and mmlu_stem with no calibration evidence. The central accuracy-retention claim depends on these deltas tracking final correctness out-of-distribution. Please provide a correlation/AUC analysis between PRM deltas and final correctness on the evaluation benchmarks, or an oracle-routing comparison. Without this, Stage 1's screening may discard steps that should be offloaded, and Stage 2 cannot recover them.
  2. [§IV-B-4, Appendix B, Theorem 2] Theorem 2 claims the threshold policy maximizes 'the original constrained objective' (28), but the proof only shows optimality for fixed Lagrange multipliers λ_s and µ. There is no argument establishing zero duality gap for this mixed-integer program, and complementary slackness in Case III is invoked without a convexity/duality proof. The statement should be qualified as optimality of the Lagrangian relaxation, or a duality-gap bound must be supplied. This is load-bearing because the scheduler's optimality is a headline theoretical result.
  3. [§IV-A-3, Proposition 1, Appendix A] Proposition 1 is a monotonicity statement: if the coarse Q-function prefers the local action, then the true Q-function also prefers it. It assumes the learned Q^{π_θ} exactly satisfies Q(·,1) ≤ Q(·,0). In practice, these are finite-capacity PPO approximations, and the proof does not show that training on the coarse reward produces the required ordering. Moreover, Eq. (35) subtracts only the immediate TC and TQ penalties; the true global action-value function also includes coupling effects on other users and future queue states, which are not captured by this additive correction. Please clarify the assumptions under which the equality in (35) actually holds.
  4. [§V-B, Table II, Figs. 5–8] All numerical results are reported as point estimates without error bars, confidence intervals, or a statement about the number of runs. Table II shows PRADA at 90.3% vs π_θ at 93.9% on gsm8k, but with no variance it is impossible to tell whether the retention is signal or noise. In addition, Fig. 5a shows the random scheduler outperforming PRADA at small M; this deserves a principled explanation rather than a post-hoc justification, because it weakens the claim that the learned two-stage design is superior to random baseline in the resource-limited regime.
minor comments (4)
  1. [§I-C] Typo: 'inclduing' should be 'including'.
  2. [§IV-B-4, Eqs. (31)–(32)] The two quantities w_{i,t} and w̄_{i,t} are visually very similar; use a more distinct notation (e.g., w^imm and w^queue) to avoid confusion, especially in Theorem 2.
  3. [§V-D, Figs. 7–8] The x-axis labels '0.1 0.1 0.1 0.5 0.5 0.5 1 1 1 2 2 2 ...' are duplicated and unclear. Use a standard monotone axis (e.g., 0.1, 0.5, 1, 2, 4, 6).
  4. [§V-A] The description says 'the complete set of training hyperparameters is listed in Table I,' but Table I only lists five parameters; please include the remaining values (e.g., batch size, trajectory buffer size, discount factor) or state that they are standard.

Circularity Check

0 steps flagged

No significant circularity: PRADA's accuracy retention is measured against external benchmarks, and the theoretical claims are algebraic consequences of explicit definitions.

full rationale

The derivation chain is self-contained rather than circular. The Stage-1 policy is trained to maximize the coarse return in Eq. (20), whose per-step reward is the PRM score difference PRM(x_{k+1}) - PRM(x_k) minus a computational cost (Eq. 21); however, the paper's central accuracy-retention claim is not derived from this reward. It is an empirical claim evaluated on gsm8k, gaokao2023en, and mmlu_stem against all-SLM, all-LLM, and RSD baselines (Table II, Figures 4, 5, 8). Thus there is no fitted parameter renamed as a prediction and no equation-level reduction of the accuracy result to the training reward. Proposition 1 is a valid monotonicity statement, not circularity. The paper defines the 'true' action-value eQ_{i,t} in Eq. (27) explicitly as the coarse Q-value minus nonnegative latency penalties, and then proves that Q(1) <= Q(0) implies eQ(1) <= eQ(0) because the penalties only reduce the offload action's value. This is algebra on the paper's own definition; the conclusion is not assumed in the premise. Whether eQ faithfully represents the actual global MDP objective is a modeling assumption and a correctness risk, but it is not a circular step. Theorem 2 is derived from a Lagrangian relaxation and the threshold structure follows from per-request comparisons of the Lagrangian terms. The step 'and thus the original constrained objective' relies on an unproven zero duality gap for the discrete constraints; this is a rigor/optimality-gap concern, not circularity. The self-citations in the paper (refs. [6], [18], [19], [22]) appear only in background and related-work sections and are not load-bearing for the PRADA framework. No uniqueness theorem from the authors' prior work is invoked, and no core assumption is justified solely by a self-citation. The most serious weakness, identified in the skeptic analysis, is the lack of calibration evidence that Skywork PRM score differences transfer from math500-test to the evaluation distributions; if the PRM deltas do not track final correctness, the learned screening policy would be unfaithful. This is an external-validity and correctness threat, not a circularity: the paper does not define accuracy in terms of PRM scores, and the empirical comparisons are independent of the reward definition. Therefore the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The ledger shows the framework imports the PRM and latency model as external components; the only free numbers are the unlisted β, γ_th, and FlopsSLM, plus the structural assumptions about reward validity and Lagrangian duality.

free parameters (3)
  • β (quality-latency trade-off parameter)
    Appears in objective (6) and rewards (20)-(21); controls the operating point on the accuracy-latency frontier, but its value in the reported experiments is not listed, leaving the central trade-off numbers unreproducible.
  • γ_th (transmission SNR threshold)
    Determines whether an offload request succeeds (Fig. 1, Sec. II-B-1); no value provided in Table III.
  • FlopsSLM (edge device peak FLOPs/s)
    Converts local SLM FLOPs to time in Eq. (18); not given in Table III alongside FlopsLLM.
axioms (5)
  • domain assumption User arrivals follow a Poisson process with rate λ
    Eq. (1), modeling the dynamic multi-user workload; not validated against real traffic traces.
  • domain assumption PRM score differences are a valid reward for final reasoning quality
    Eqs. (20)-(21) define the coarse return entirely through PRM scores; the accuracy results depend on this transfer, but no calibration evidence is provided for the deployment benchmarks.
  • domain assumption Transformer inference cost is accurately captured by Eqs. (14)-(16)
    FLOPs formulas for prefill/decode are used to compute all computation delays; they ignore memory-bound effects, batching, and dynamic shapes.
  • ad hoc to paper Lagrangian dual maximization equals the original constrained scheduling optimum
    Theorem 2 claims optimality for the original objective, but the proof only establishes optimality for fixed Lagrange multipliers; no strong-duality or integrality-gap argument is given (Sec. IV-B-4).
  • ad hoc to paper Exact Q-values satisfy the Proposition 1 condition
    Proposition 1's guarantee assumes the coarse Q-function ordering is exact; in practice πθ and Vφ are finite-capacity approximations trained with PPO (Sec. IV-A-2), so the guarantee is conditional.

pith-pipeline@v1.3.0-alltime-deepseek · 22037 in / 16667 out tokens · 169364 ms · 2026-08-02T15:13:38.741668+00:00 · methodology

0 comments
read the original abstract

Deploying large language models (LLMs) at the network edge is hindered by their enormous cost, yet the reasoning quality they provide remains indispensable. Heterogeneous collaboration between edge small models and a server LLM has emerged as a promising direction, but existing methods fail under the dynamic conditions of multi-user contention, autoregressive generation, and time-varying resources. This paper puts forward a process reward model (PRM)-aided two-stage decoupled acceleration (PRADA) framework, which is built on a fundamental change of perspective: instead of querying a PRM online, which cripples multi-user systems with prohibitive latency, we use the PRM solely as an offline teacher. Its reasoning-quality intuition is fully distilled into a lightweight policy that screen each step locally, without any context upload, while a Lagrangian scheduler at the server resolves resource contention through a threshold-structured policy. Across diverse reasoning benchmarks, PRADA retains the vast majority of the LLM's accuracy while substantially reducing end-to-end latency. The results further reveal threshold effects for both server parallel capacity and total bandwidth: performance saturates beyond critical resource levels, after which the system bottleneck shifts from queuing to computation or from communication to contention. These structural findings provide actionable guidance for joint provisioning of computation and communication resources without requiring per-benchmark tuning.

Figures

Figures reproduced from arXiv: 2607.18244 by Fen Hou, Tianji He, Yulin Shao.

Figure 1
Figure 1. Figure 1: System model of dynamic heterogeneous agent collab [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the interplay between slots and steps. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the prefill and decode phases in model [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy and computational cost of the policy network [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance of PRADA under different server parallel [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Processing, communication, and queuing delays as functions of the server LLM parallel capacity [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Processing, communication, and queuing delays as functions of total bandwidth [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance of PRADA under different bandwidth [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 7 linked inside Pith

  1. [1]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language models are few-shot learners,”Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020

    1901

  2. [2]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in Neural Information Processing Systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  3. [3]

    Retrieval- augmented generation for knowledge-intensive NLP tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive NLP tasks,”Advances in Neural Information Processing Systems, vol. 33, pp. 9459–9474, 2020

    2020

  4. [4]

    Toolformer: Language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dess `ı, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,”Advances in Neural Infor- mation Processing Systems, vol. 36, pp. 68 539–68 551, 2023

    2023

  5. [5]

    Mobile edge intelligence for large language models: A contemporary survey,

    G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, and K. Huang, “Mobile edge intelligence for large language models: A contemporary survey,”IEEE Communications Surveys & Tutorials, vol. 27, no. 6, pp. 3820–3860, 2025

    2025

  6. [6]

    Federated edge learning with misaligned over-the-air computation,

    Y . Shao, D. G ¨und¨uz, and S. C. Liew, “Federated edge learning with misaligned over-the-air computation,”IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 3951–3964, 2021

    2021

  7. [7]

    Knowledge-augmented reasoning distillation for small language models in knowledge-intensive tasks,

    M. Kang, S. Lee, J. Baek, K. Kawaguchi, and S. J. Hwang, “Knowledge-augmented reasoning distillation for small language models in knowledge-intensive tasks,”Advances in Neural Information Process- ing Systems, vol. 36, pp. 48 573–48 602, 2023

    2023

  8. [8]

    DDK: Distilling domain knowledge for efficient large language models,

    J. Liu, C. Zhang, J. Guo, Y . Zhang, H. Que, K. Deng, Z. Bai, J. Liu, G. Zhang, J. Wanget al., “DDK: Distilling domain knowledge for efficient large language models,”Advances in Neural Information Processing Systems, vol. 37, pp. 98 297–98 319, 2024

    2024

  9. [9]

    Jiuzhang3.0: Efficiently improving mathe- matical reasoning by training small data synthesis models,

    K. Zhou, B. Zhang, J. Wang, Z. Chen, W. X. Zhao, J. Sha, Z. Sheng, S. Wang, and J.-R. Wen, “Jiuzhang3.0: Efficiently improving mathe- matical reasoning by training small data synthesis models,”Advances in Neural Information Processing Systems, vol. 37, pp. 1854–1889, 2024

    2024

  10. [10]

    Demystifying small language models for edge deployment,

    Z. Lu, X. Li, D. Cai, R. Yi, F. Liu, W. Liu, J. Luan, X. Zhang, N. D. Lane, and M. Xu, “Demystifying small language models for edge deployment,” inProceedings of the Association for Computational Linguistics, 2025, pp. 14 747–14 764

    2025

  11. [11]

    RouteLLM: Learning to route LLMs from pref- erence data,

    I. Ong, A. Almahairi, V . Wu, W.-L. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica, “RouteLLM: Learning to route LLMs from pref- erence data,” inInternational Conference on Learning Representations, 2025

    2025

  12. [12]

    FrugalGPT: How to use large language models while reducing cost and improving performance,

    L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,”arXiv preprint arXiv:2305.05176, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    Hybrid SLM and LLM for edge-cloud collaborative inference,

    Z. Hao, H. Jiang, S. Jiang, J. Ren, and T. Cao, “Hybrid SLM and LLM for edge-cloud collaborative inference,” inMobiSys, 2024, pp. 36–41

    2024

  14. [14]

    Uncertainty-aware hybrid inference with on-device small and remote large language models,

    S. Oh, J. Kim, J. Park, S.-W. Ko, T. Q. S. Quek, and S.-L. Kim, “Uncertainty-aware hybrid inference with on-device small and remote large language models,” inIEEE International Conference on Machine Learning for Communication and Networking, 2025

    2025

  15. [15]

    CITER: Collaborative inference for efficient large language model decoding with token-level routing,

    W. Zheng, Y . Chen, W. Zhang, S. Kundu, Y . Li, Z. Liu, E. P. Xing, H. Wang, and H. Yao, “CITER: Collaborative inference for efficient large language model decoding with token-level routing,”arXiv preprint arXiv:2502.01976, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Reward-guided speculative decoding for efficient LLM reasoning,

    B. Liao, Y . Xu, H. Dong, J. Li, C. Monz, S. Savarese, D. Sahoo, and C. Xiong, “Reward-guided speculative decoding for efficient LLM reasoning,” inInternational Conference on Machine Learning, 2025

    2025

  17. [17]

    G-boost: Boost- ing private slms with general llms,

    Y . Fan, Y . Mao, L. Lai, Y . Zhang, Z. Qian, and Y . Gao, “G-boost: Boost- ing private slms with general llms,”arXiv preprint arXiv:2503.10367, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    LLMind: Or- chestrating AI and IoT with LLM for complex task execution,

    H. Cui, Y . Du, Q. Yang, Y . Shao, and S. C. Liew, “LLMind: Or- chestrating AI and IoT with LLM for complex task execution,”IEEE Communications Magazine, vol. 63, no. 4, pp. 214–220, 2024

    2024

  19. [19]

    Cayley graph optimization for scalable multi-agent communication topologies,

    J. Luo and Y . Shao, “Cayley graph optimization for scalable multi-agent communication topologies,”arXiv preprint arXiv:2604.09703, 2026

    Pith/arXiv arXiv 2026

  20. [20]

    Multi-agent actor-critic for mixed cooperative-competitive environ- ments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environ- ments,”Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  21. [21]

    Multiagent cooperation and competition with deep reinforcement learning,

    A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru, J. Aru, and R. Vicente, “Multiagent cooperation and competition with deep reinforcement learning,”PloS one, vol. 12, no. 4, p. e0172395, 2017

    2017

  22. [22]

    A theory of semantic communication,

    Y . Shao, Q. Cao, and D. G¨und¨uz, “A theory of semantic communication,” IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 12 211– 12 228, 2024

    2024

  23. [23]

    Scaling large language model-based multi- agent collaboration,

    C. Qian, Z. Xie, Y . Wang, W. Liu, K. Zhu, H. Xia, Y . Dang, Z. Du, W. Chen, C. Yanget al., “Scaling large language model-based multi- agent collaboration,”arXiv preprint arXiv:2406.07155, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  25. [25]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement,

    A. Yang, B. Zhang, B. Hui, B. Gao, B. Yu, C. Li, D. Liu, J. Tu, J. Zhou, J. Lin, K. Lu, M. Xue, R. Lin, T. Liu, X. Ren, and Z. Zhang, “Qwen2.5-math technical report: Toward mathematical expert model via self-improvement,”arXiv preprint arXiv:2409.12122, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Skywork-o1 open series,

    J. He, T. Wei, R. Yan, J. Liu, C. Wang, Y . Gan, S. Tu, C. Y . Liu, L. Zeng, X. Wang, B. Wang, Y . Li, F. Zhang, J. Xu, B. An, Y . Liu, and Y . Zhou, “Skywork-o1 open series,” 2024. [Online]. Available: https://doi.org/10.5281/zenodo.16998085

    doi:10.5281/zenodo.16998085 2024