Pith. sign in

REVIEW 4 major objections 7 minor 45 references

Training a language model on many inference controllers at once beats specializing it to one workflow.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 13:03 UTC pith:R6YHAUDC

load-bearing objection Clear framing of multi-controller post-training with real over-specialization evidence; the win over strong single-controller baselines is real in direction but thin and single-seed. the 4 major comments →

arxiv 2607.23771 v1 pith:R6YHAUDC submitted 2026-07-26 cs.AI

Training Language Models to Cooperate with Inference-Time Controllers

Moumita Choudhury , Vanshaj Khattar , Jing Liu , Toshiaki Koike-Akino , Ankush Chakrabarty , Shlomo Zilberstein , Ye Wang This is my paper
classification cs.AI
keywords controller-aware post-traininginference-time controllersmulti-task RLturn-level GRPOmodule compositionLLM reasoningcross-controller generalization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern LLM systems do not just answer once; they run structured multi-call procedures—chain-of-thought, self-refine, debate, step-back abstraction, and the like—that the paper calls controllers. Most post-training still optimizes the model for a single fixed pattern, so the model overfits that workflow and transfers poorly when the same local behaviors appear in a new arrangement. CALM treats post-training as multi-task reinforcement learning over a family of controllers, each built by composing reusable modules (roles with fixed prompts and output schemas). A shared policy is trained with turn-level group-relative policy optimization, optionally with module-level loss weighting. On math reasoning, multi-controller training outperforms single-controller baselines on training controllers, held-out recombinations of known modules, broader controller shifts with new roles, and harder out-of-distribution sets. The practical point is that the base model can be taught to cooperate with a distribution of inference workflows rather than one script.

Core claim

Controller-aware multi-task post-training over module-composed controllers produces a shared policy that generalizes better across held-out controller compositions and broader controller shifts than single-controller optimization. Empirically, CALM variants beat the strongest single-controller baselines on GSM8K averages and stay competitive or better under compositional and shift tiers, with the flat multi-controller mix showing the most consistent gains on MATH500 and AMC 2023.

What carries the argument

CALM: multi-task turn-level GRPO over controller-induced MDPs, where each controller is an executable composition of reusable modules; optional module-masked losses (flat mix, uniform weights, single-role-per-step, adaptive gradient-norm balancing) redistribute updates across module types.

Load-bearing premise

A single end-of-trajectory correct/incorrect reward, shared as the same advantage on every turn, is enough for the model to learn reusable module skills instead of controller-specific shortcuts—even though multi-turn training is unstable.

What would settle it

Train the same base model with CALM versus the best single-controller run on the paper’s six training controllers, then measure accuracy on the held-out compositional controllers (H1–H3) and shift controllers (H4–H6) on GSM8K and on MATH500/AMC 2023; if multi-controller training does not beat single-controller averages on those held-out tiers, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Deployments that switch among CoT, refine, debate, and verification pipelines can share one post-trained policy instead of a specialist per workflow.
  • Automatic controller search can pair with a base model already trained to execute shared modules in novel compositions.
  • Module-level training choices should be matched to the expected test controller mix: adaptive weighting helps familiar recombinations; flat mixing is safer under large structural shift.
  • Single-controller fine-tuning is a measurable over-specialization risk when evaluation uses a family of inference procedures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If module interfaces stay stable, expanding the training controller inventory may transfer to controllers discovered after training without further gradient updates.
  • Credit-assignment fixes beyond terminal rewards (turn-level or module-level shaping) could unlock longer multi-agent controllers where CALM currently hits instability.
  • The same compositional multi-task framing may apply outside math to tool use and planning pipelines that reuse critique, plan, and act roles.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces CALM (Controller-Aware Language Models), a post-training framework in which a single shared policy is trained by RL (turn-level GRPO, following ReMA) over a distribution of inference-time controllers (CoT, Self-Refine, Debate, Step-back, Quality-Diversity, Role-Assignment) rather than a single fixed workflow. Controllers are formalized as compositions of reusable modules (COT, CRITIC, ABSTRACTOR, DEBATER, JUDGE, ROUTER), and the authors study four variants that differ in how module-level turns are weighted in the loss (MIXED, UNIFORM, SINGLE, ADAPTIVE gradient-norm balancing). Experiments train Llama-3.2-3B-Instruct with LoRA on 2,048 GSM8K problems and evaluate on the six training controllers, three held-out compositional controllers, three controller-shift controllers with new module types, and OOD math (MATH500, AMC 2023). The main findings: single-controller training over-specializes (often catastrophically, e.g., Debate 71.87% on itself vs 40.86% on CoT); all four CALM variants exceed the best single-controller baseline on the training-controller average (71.49–73.72% vs 71.11%); and CALM variants are competitive or better under compositional and shift tiers, with CALM-MIXED most consistent on OOD data. The authors are candid that multi-turn GRPO is unstable and that they do not solve this.

Significance. The training–deployment mismatch the paper targets is real and increasingly important as inference-time orchestration (agent scaffolds, automated controller search such as ADAS/Archon) outpaces post-training practice. To my knowledge this is the first systematic study of post-training over a distribution of inference-time controllers, and the formulation as multi-task RL over controller-induced MDPs with a module-level loss decomposition is clean and well-connected to the multi-task/modular RL literature. Reproducibility is a genuine strength: full executable Python implementations of all controllers (Appendix B), a complete hyperparameter table (Table 5), system-prompt formats, and qualitative trajectories (Appendix E) are shipped, and evaluation covers held-out compositions, module-vocabulary shift, and OOD datasets rather than only in-distribution accuracy. The over-specialization result for single-controller training is convincingly demonstrated and useful by itself. If the comparative claims survive proper uncertainty quantification, the paper would be a solid and timely contribution to the inference-time-compute/post-training literature.

major comments (4)
  1. [Tables 2–3, §6.2–6.3] No uncertainty estimates anywhere, and every configuration appears to be a single training run (Table 5 lists one hyperparameter set; no seeds reported). The headline margins are small relative to both sources of noise. On GSM8K (n≈1,319), a binomial 95% CI around p≈0.72 is roughly ±2.4 points from test-set sampling alone — exceeding the training-tier margin of CALM-ADAPTIVE over Step-back (73.72 vs 71.11) and the controller-shift margin of CALM-MIXED over CoT (75.69 vs 74.91), before accounting for RL seed variance, which at 3B scale with LoRA is typically several points (and which the paper itself documents as severe enough to collapse B2, App. D). On AMC 2023 the 2.5-point increments imply n=40, so the reported ~4-point separations correspond to 1–2 problems. The directional claim (single-controller over-specialization) is convincingly supported by the catastrophic off-controller drop
  2. [§6.2, Appendix D] Asymmetric checkpoint protocol: for Self-Refine (B2) the paper reports the best validation checkpoint (step 156) because training collapses, while all other rows appear to report final checkpoints after 192 steps. Best-checkpoint selection on validation introduces selection bias into Table 2, and it is applied only to one baseline, making the B2 column hard to interpret (the Self-Refine baseline average of 36.30% is so low that it functions mainly as a strawman in the training-tier average). Please (i) state explicitly which checkpoint is used for every row of Table 2, (ii) clarify whether the validation set used for B2 selection is disjoint from all reported test sets, and (iii) consider reporting final-checkpoint numbers for B2 alongside, or selecting checkpoints for all runs by the same validation rule.
  3. [§6.3, Table 3 (AMC 2023)] Internal contradiction in the results text. The paper states 'CALM-MIXED leads across all three tiers with averages of 20.00%, 26.25%, and 23.75%', but Table 3 shows CALM-UNIFORM's controller-shift average is 26.25%, strictly above CALM-MIXED's 23.75% — the very next sentence concedes this. Similarly on MATH500, CoT beats CALM-MIXED on B1 (45.0 vs 42.4) and H4 (39.8 vs 39.0), so 'leads' claims should be made per-tier with the actual numbers. Related: §6.3 describes MATH500/AMC as 'out-of-distribution evaluations for the MATH Level 3–5 training data', but the training data is GSM8K (§6.1) — this sentence appears to be a copy-paste error and should be corrected. These are small fixes but they sit in the paragraph carrying the paper's strongest OOD claim.
  4. [§4.2, Eq. (5) and Eq. (9); §1 hypothesis] The central hypothesis is that multi-controller training works because the policy learns reusable modules that transfer to novel compositions. The experimental evidence is indirect: the advantage in Eq. (9) is trajectory-level and identical for all turns, and the module-masked decomposition in Eq. (5) explicitly 'does not alter the credit assignment' — so nothing in the method or the evaluation directly measures module-level competence (e.g., is the CRITIC module actually better in CALM models? The B2-column gains are suggestive but conflate critic quality with robustness to critique-style context). Furthermore, CALM-MIXED vs single-controller baselines conflates controller-awareness with training-distribution diversity; a data-matched control (e.g., single-controller training with 6× data, or multi-controller rollouts with a non-controller-aware aggregate) would isolate the mechanism. I
minor comments (7)
  1. [§4.1 vs Appendix A] Eq. (4) in §4.1 and Eq. (13) in Appendix A are verbatim duplicates (including the surrounding paragraph). Consolidate to a single statement with a forward reference.
  2. [Table 2] Several rows appear misaligned in the rendered table (e.g., the Step-back row shows '75.89 68.31 75.59' in positions that suggest cell slippage around the B5 column; the CALM rows have adjacent cells run together). Please proof the final table carefully; with margins this small, a transposed digit changes conclusions.
  3. [§6.1 / Table 1] H2 and H3 are author-designed ('we design manually'), and all module labels/interfaces are author-constructed. This is fine, but the compositional-tier claims would benefit from one sentence acknowledging that the held-out compositions are the authors' own constructions rather than independently proposed controllers, since this bears on how 'held-out' they are.
  4. [Appendix D, Figure 2] Figure 2 shows training curves only for single-controller baselines. Given that training instability is a recurring theme, curves for the four CALM variants (and an indication of whether checkpoint selection was also used for them) would materially help the reader.
  5. [§6.1] The format-reward magnitudes (−0.5/−0.2/−0.1) are unablated free parameters; a brief sensitivity note or a justification for the asymmetry would be useful. Similarly, uniform controller sampling (c ~ Uniform(C_train)) is a design choice that advantages rare modules; worth one sentence of discussion since CALM-ADAPTIVE's shift-tier drop is attributed to overfitting the training module distribution.
  6. [Appendix B, Listings 4/8/9] Minor presentation: the H3 controller is named 'H5__Debate___Refine' in Listing 9 (confusing given H5 is a different controller), and the Step-back instruction contains 'invovled'. Since these prompts are part of the released artifact, please clean up.
  7. [§6.3] Please state the evaluation protocol for H1 (self-consistency with 5 samples at temperature 0.8) versus the greedy decoding stated in Appendix B.2 — whether sampled evaluation introduces run-to-run variance in Table 2/3 H1 numbers should be made explicit.

Circularity Check

0 steps flagged

No significant circularity: empirical multi-task RL evaluated on external ground-truth answers and held-out controllers.

full rationale

CALM frames post-training as multi-task RL over controller-induced MDPs and reports accuracy under binary terminal rewards against dataset ground truth (GSM8K, MATH500, AMC 2023). The training objective (Eq. 3 / L_CALM) averages expected task reward over a declared family of training controllers; evaluation uses held-out controller compositions, new module types, and OOD math sets. Success is not defined in terms of a quantity fitted from the same measurements that are later called predictions. Module labels and controller programs are author-specified compositions drawn from cited methods, but that is experimental design, not a definitional loop: the reported metric remains answer correctness, not recovery of the controller inventory or of fitted module weights. Citations to GRPO, ReMA, ADAS, and multi-task RL supply methods and baselines; none function as a load-bearing uniqueness theorem that forces the empirical claim. Self-Refine checkpoint selection and multi-turn instability affect reliability of comparisons, not circularity of the derivation. No step reduces a claimed prediction to its inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

The central claim rests on standard RL-for-LLMs assumptions, a compositional controller model, and several engineering choices (uniform controller sampling, trajectory-level advantages, format rewards, LoRA on a 3B instruct model). No new physical entities; free parameters are ordinary RL/training knobs rather than science constants fitted to prove a law.

free parameters (5)
  • KL coefficient β = 5e-4
    Fixed at 5e-4 in Table 5; shapes how far the policy may move from the reference and can change multi-turn stability and transfer.
  • clip ε / group size G = ε=0.2, G=8
    GRPO clip 0.2 and group size 8 are chosen hyperparameters that define the advantage estimator used for all claims.
  • format reward magnitudes = -0.5/-0.2/-0.1
    Per-turn penalties (−0.5 parse, −0.2 missing fields, −0.1 extra fields) are hand-set and affect every module interface during training.
  • CALM-ADAPTIVE EMA/balancing knobs = ρ=0.9, α=1.0, clip[0.1,10]
    ρ=0.9, α=1.0, weight clip [0.1,10], δ=1e-6 control module gradient reweighting and thus which variant wins which tier.
  • training controller set C_train and uniform sampling = 6 controllers, uniform
    Choice and equal weighting of B1–B6 define the multi-task distribution; the paper notes the true test controller distribution is unavailable.
axioms (6)
  • domain assumption Decoding under a controller is a multi-turn MDP with terminal binary correctness reward and shared trajectory-level advantage across turns.
    Sec. 2–3 and Eq. 9; credit assignment does not distinguish turn quality beyond the final answer.
  • domain assumption Controllers are compositions of a shared module vocabulary identified by role instruction and output schema.
    Sec. 3 and Table 4; this underwrites the recombination hypothesis for held-out compositions.
  • domain assumption Turn-level importance-ratio aggregation (ReMA-style) is an appropriate multi-turn GRPO extension for variable-length controllers.
    Sec. 2 and 4.1; adopted rather than re-derived.
  • standard math Standard clipped importance-weighted policy gradient / GRPO math and KL regularization to a reference policy.
    Eqs. 1–2, 10–14; conventional RLHF/GRPO machinery.
  • domain assumption GSM8K/MATH/AMC answer extraction correctness is a valid proxy for controller-aware reasoning quality.
    Sec. 6.1; all main tables use binary final-answer accuracy on math word problems.
  • ad hoc to paper When multi-turn training is unstable, reporting a best validation checkpoint (Self-Refine step 156) is acceptable for comparison.
    Sec. 6.2; affects the single-controller baseline landscape used to claim CALM gains.
invented entities (2)
  • CALM multi-controller objective and module-masked GRPO variants (MIXED/UNIFORM/SINGLE/ADAPTIVE) no independent evidence
    purpose: Train one shared policy over a distribution of controller-induced protocols with optional module-level loss decomposition.
    Core methodological object of the paper; not a physical entity, but an invented training construct whose value is only evidenced by the reported runs.
  • Module vocabulary M (COT, CRITIC, ABSTRACTOR, DEBATER, JUDGE, ROUTER, plus shift-only roles) independent evidence
    purpose: Provide reusable local turn interfaces so controllers can be treated as compositions and losses can be module-masked.
    Author-defined grouping of prompts/schemas (Table 4); generalization claims depend on this taxonomy matching real controller structure.

pith-pipeline@v1.2.0-grok45-kimik3 · 27546 in / 3772 out tokens · 79471 ms · 2026-07-30T13:03:13.985328+00:00 · methodology

0 comments
read the original abstract

Large language model (LLM) performance increasingly depends not only on the base model, but also on the inference-time controller used to organize reasoning. Existing post-training methods, however, typically optimize for a single fixed interaction pattern, despite real deployments relying on diverse controllers such as Chain-of-Thought, self-consistency, debate, planning, and verification pipelines. This creates a training--deployment mismatch and limits transfer to new workflows. We introduce CALM (Controller-Aware Language Models), a post-training framework that explicitly places controllers in the training loop. We formulate controller-aware post-training as multi-task reinforcement learning over controller-induced interaction protocols, where controllers are compositions of reusable local reasoning modules. This structure also induces a module-level decomposition of mixed-controller training under a turn-level GRPO objective, enabling a systematic study of controller and module-aware training strategies. We evaluate CALM on held-out controller compositions and broader controller shifts, showing that controller-aware post-training improves generalization across inference-time workflows beyond single-controller optimization.

Figures

Figures reproduced from arXiv: 2607.23771 by Ankush Chakrabarty, Jing Liu, Moumita Choudhury, Shlomo Zilberstein, Toshiaki Koike-Akino, Vanshaj Khattar, Ye Wang.

Figure 1
Figure 1. Figure 1: Overview of CALM. (a) Training controllers are compositions of reusable modules (COT, CRITIC, ABSTRACTOR, DEBATER, JUDGE, etc.) drawn from established inference-time methods (Hu et al., 2025). Each controller consists of diverse set of roles and could follow a number of turns with or without memory. (b) Single-controller post-training baseline, where a fixed controller generates trajectories. We use a mult… view at source ↗
Figure 2
Figure 2. Figure 2: Training curves for different training configurations for single-controller training (B1–B6) 20 [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 11 linked inside Pith

  1. [1]

    ICLR , year=

    Automated Design of Agentic Systems , author=. ICLR , year=

  2. [2]

    arXiv preprint arXiv:2409.15254 , year=

    Archon: An architecture search framework for inference-time techniques , author=. arXiv preprint arXiv:2409.15254 , year=

    Pith/arXiv arXiv

  3. [3]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  4. [4]

    Journal of Machine Learning Research , volume=

    Palm: Scaling language modeling with pathways , author=. Journal of Machine Learning Research , volume=

  5. [5]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    Pith/arXiv arXiv

  6. [6]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  7. [7]

    ICLR , year=

    Self-consistency improves chain of thought reasoning in language models , author=. ICLR , year=

  8. [8]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  9. [9]

    arXiv preprint arXiv:2310.06117 , year=

    Take a step back: Evoking reasoning via abstraction in large language models , author=. arXiv preprint arXiv:2310.06117 , year=

    Pith/arXiv arXiv

  10. [10]

    Advances in Neural Information Processing Systems , volume=

    Self-refine: Iterative refinement with self-feedback , author=. Advances in Neural Information Processing Systems , volume=

  11. [11]

    Advances in Neural Information Processing Systems , volume=

    Reflexion: Language agents with verbal reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=

  12. [12]

    Advances in Neural Information Processing Systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in Neural Information Processing Systems , volume=

  13. [13]

    The eleventh international conference on learning representations , year=

    React: Synergizing reasoning and acting in language models , author=. The eleventh international conference on learning representations , year=

  14. [14]

    Forty-first International Conference on Machine Learning , year=

    Improving factuality and reasoning in language models through multiagent debate , author=. Forty-first International Conference on Machine Learning , year=

  15. [15]

    Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

    Generative agents: Interactive simulacra of human behavior , author=. Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

  16. [16]

    Frontiers of Computer Science , volume=

    A survey on large language model based autonomous agents , author=. Frontiers of Computer Science , volume=. 2024 , publisher=

    2024

  17. [17]

    The Thirteenth International Conference on Learning Representations , year=

    Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  18. [18]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2501.03262 , year=

    Reinforce++: A simple and efficient approach for aligning large language models , author=. arXiv preprint arXiv:2501.03262 , year=

    Pith/arXiv arXiv

  20. [20]

    arXiv preprint arXiv:2503.09501 , year=

    Rema: Learning to meta-think for llms with multi-agent reinforcement learning , author=. arXiv preprint arXiv:2503.09501 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2504.20073 , year=

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning , author=. arXiv preprint arXiv:2504.20073 , year=

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2410.09671 , year=

    Openr: An open source framework for advanced reasoning with large language models , author=. arXiv preprint arXiv:2410.09671 , year=

    Pith/arXiv arXiv

  23. [23]

    First Workshop on Multi-Turn Interactions in Large Language Models , year=

    Improved Multi-Agent Collaboration with Multi-Turn Reinforcement Learning , author=. First Workshop on Multi-Turn Interactions in Large Language Models , year=

  24. [24]

    Advances in Neural Information Processing Systems , year=

    Experience Replay for Continual Learning , author=. Advances in Neural Information Processing Systems , year=

  25. [25]

    Advances in Neural Information Processing Systems , year=

    Gradient Episodic Memory for Continual Learning , author=. Advances in Neural Information Processing Systems , year=

  26. [26]

    arXiv preprint arXiv:1707.0183 , year=

    Teacher-Student Curriculum Learning , author=. arXiv preprint arXiv:1707.0183 , year=

    arXiv

  27. [27]

    International Conference on Machine Learning , year=

    Automated Curriculum Learning for Neural Networks , author=. International Conference on Machine Learning , year=

  28. [28]

    International Conference on Machine Learning , year=

    Prioritized Level Replay , author=. International Conference on Machine Learning , year=

  29. [29]

    International conference on machine learning , pages=

    Model-agnostic meta-learning for fast adaptation of deep networks , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  30. [30]

    Machine Learning , volume =

    Multitask Learning , author =. Machine Learning , volume =. 1997 , publisher =

    1997

  31. [31]

    Advances in Neural Information Processing Systems , volume =

    Distral: Robust Multitask Reinforcement Learning , author =. Advances in Neural Information Processing Systems , volume =

  32. [32]

    Advances in Neural Information Processing Systems , volume =

    Gradient Surgery for Multi-Task Learning , author =. Advances in Neural Information Processing Systems , volume =

  33. [33]

    International Conference on Machine Learning , pages =

    Action Robust Reinforcement Learning and Applications in Continuous Control , author =. International Conference on Machine Learning , pages =. 2019 , organization =

    2019

  34. [34]

    International Conference on Machine Learning , pages =

    Generalization to New Actions in Reinforcement Learning , author =. International Conference on Machine Learning , pages =. 2021 , organization =

    2021

  35. [35]

    2017 IEEE international conference on robotics and automation (ICRA) , pages=

    Learning modular neural network policies for multi-task and multi-robot transfer , author=. 2017 IEEE international conference on robotics and automation (ICRA) , pages=. 2017 , organization=

    2017

  36. [36]

    International conference on machine learning , pages=

    Modular multitask reinforcement learning with policy sketches , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  37. [37]

    International Conference on Learning Representations , year =

    Diversity is All You Need: Learning Skills without a Reward Function , author =. International Conference on Learning Representations , year =

  38. [38]

    International Conference on Machine Learning , pages=

    Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  39. [39]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  40. [40]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    Pith/arXiv arXiv

  42. [42]

    The twelfth international conference on learning representations , year=

    Let's verify step by step , author=. The twelfth international conference on learning representations , year=

  43. [43]

    2024 , howpublished =

    Llama 3.2 3B Instruct , author =. 2024 , howpublished =

    2024

  44. [44]

    2024 , journal =

    HybridFlow: A Flexible and Efficient RLHF Framework , author =. 2024 , journal =

    2024

  45. [45]

    International conference on machine learning , pages=

    Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks , author=. International conference on machine learning , pages=. 2018 , organization=

    2018