Pith. sign in

REVIEW 3 major objections 6 minor 300 references

At a stationary point, regret-trained attention exactly implements smoothed fictitious play.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:42 UTC pith:2BCEDVT3

load-bearing objection A solid stationary-point analysis of regret-trained attention on the simplex, with a genuinely new swap-regret loss; the swap-regret deployment story is more heuristic than the theorems suggest. the 3 major comments →

arxiv 2607.23333 v1 pith:2BCEDVT3 submitted 2026-07-25 cs.LG cs.AI

Training with (Swap) Regret Loss in a Single-Layer Self-Attention Model: A Case Study on the Probability Simplex

Chanwoo Park , Asuman Ozdaglar This is my paper
classification cs.LG cs.AI MSC 68T0791A2691A10
keywords regret lossself-attentionsmoothed fictitious playswap regretcorrelated equilibriumcoarse correlated equilibriumprobability simplexfixed-point layer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that a minimal one-layer attention model trained with a loss defined by decision-theoretic regret does not merely imitate no-regret algorithms: at a stationary point of the loss, its forward pass is exactly the update of smoothed fictitious play, with learning rate Θ~(1/√T). A new swap-regret loss is introduced for a multi-head variant, turning each head into an external-regret learner and a fixed-point layer into a no-swap-regret algorithm. If correct, this gives a differentiable mechanism by which attention-based agents inherit equilibrium guarantees: external regret leads to coarse correlated equilibrium, swap regret to correlated equilibrium.

Core claim

The paper shows that the squared regret loss has a first-order stationary point at parameters V=-kI, a=0, v_c=v·1, with k=Θ~(1/√T). At that configuration, the softmax attention output is entropy-regularized FTRL, i.e., smoothed fictitious play, on the probability simplex. For the swap-regret variant, the stationary point implements the classical reduction from external to swap regret: each attention head runs smoothed fictitious play on the scaled loss x_{i,t}=π_{t,i}ℓ_t, and the fixed point of the resulting column-stochastic matrix becomes the next policy. The paper proves stationarity of the loss under i.i.d. Gaussian loss vectors and gives empirical evidence that random-initialization tra

What carries the argument

The key objects are the Softmax operator and its Jacobian J_sm(z)=Diag(π)-ππᵀ, whose shift-invariance and zero row/column sums force the bias directions to vanish; permutation symmetry of the Gaussian loss distribution reduces the parameter gradient to a scalar derivative along the ray V=-kI; and the fixed-point layer, whose differentiability follows from Perron-Frobenius theory, converts d parallel external-regret heads into a single no-swap-regret policy.

Load-bearing premise

The load-bearing premise is that training on i.i.d. Gaussian block inputs reaches and preserves the stationary configuration when the model is deployed on the recurrent, policy-dependent input distribution, and that training dynamics actually arrive at the stationary point—something the paper does not prove.

What would settle it

Train the swap-regret architecture under the paper's Gaussian objective, then deploy it on the recurrent fixed-point input distribution and compare the forward pass to the classical no-swap-regret update; if the per-head outputs diverge from smoothed fictitious play on x_{i,t}=π_{t,i}ℓ_t, or the maximum CE gap stops decreasing, the stationarity transfer claim fails. Equivalently, evaluate the gradient of the swap-regret loss at the claimed stationary configuration under bounded non-Gaussian losses; a nonzero gradient would show the stationary point is specific to Gaussian training.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim is correct, a single-layer softmax attention model trained with regret loss and deployed by all players in a finite game makes the time-averaged joint play converge to a coarse correlated equilibrium.
  • The multi-head fixed-point architecture trained with swap-regret loss makes the time-averaged joint play converge to a correlated equilibrium.
  • These guarantees arise without supervised traces of the algorithms: the regret loss directly shapes the attention weights into the desired mechanism.
  • The stationary stepsize k=Θ~(1/√T) matches the learning rate needed for no-regret guarantees, so the induced dynamics are not merely interpretable but also provably adaptive.
  • The result gives the first precise stationary-point characterization for attention with a non-linear operator, replacing post-hoc projection arguments with end-to-end optimization through the Softmax.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the proofs assume i.i.d. standard Gaussian loss vectors; if the stationary point persists only under Gaussian inputs, the equilibrium conclusions depend on a train-to-deployment transfer that the paper bridges only by a dimension-calibration heuristic for the swap-regret architecture.
  • Editorial inference: the paper does not prove convergence of training dynamics, so reaching the stationary point is an assumption; if gradient flow converges elsewhere, the equilibrium guarantees do not apply.
  • Editorial inference: a direct test is to train on bounded, non-Gaussian losses and check whether V=-kI remains stationary; a nonzero gradient would show the mechanism is an artifact of Gaussian training.
  • Editorial inference: the fixed-point layer could be replaced by alternative linear-map reductions; if the fixed point is essential, this exposes a computational trade-off for transformer-based no-swap-regret learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies regret-loss training of minimal self-attention architectures on probability-simplex policies. Theorem 1 claims that a single-layer softmax attention model trained with squared external regret over i.i.d. Gaussian loss vectors has a stationary point at V = -kI, a = 0, v_c = v1 with k = Θ~(1/√T); at this configuration the forward pass is exactly smoothed fictitious play / entropy-regularized FTRL. Corollary 1 then states that deploying this architecture in a repeated game yields convergence to a coarse correlated equilibrium. The paper further introduces a multi-head fixed-point architecture and a swap-regret loss, and Theorem 2 claims a stationary point whose forward pass implements the Blum–Mansour reduction, with each head running smoothed fictitious play; Corollary 2 states that deployment converges to a correlated equilibrium. The proofs use Stein's lemma, permutation symmetries, softmax Jacobian bounds, and Gaussian supremum estimates, with the detailed arguments in Appendices A–D. Section 5 explicitly concedes that no convergence of training dynamics is proved.

Significance. If the stationary-point characterizations are correct, this is a valuable mechanistic contribution: it shows that a decision-theoretic regret objective can, at least at a first-order stationary point, reproduce classical no-regret algorithms in a differentiable attention layer without supervised traces. The proof strategy is nontrivial and largely self-contained, and the paper gives finite-horizon experiments that are consistent with the identified FTRL family. The main value is therefore the conditional statement 'at the identified parameter configuration, the forward pass implements smoothed fictitious play / Blum–Mansour.' The paper's broader claims about training and deployment, however, go beyond what is proved, and the swap-regret deployment uses an ad-hoc calibration that is not part of Theorem 2. These gaps are load-bearing for the abstract and corollaries.

major comments (3)
  1. [Section 5; Corollaries 1–2] The corollaries state that a player who 'deploys the trained architecture' converges to a CCE/CE. But Theorems 1–2 only establish existence of a stationary point of the population regret loss under Gaussian training inputs. Section 5 explicitly says the paper does not prove 'global optimality or convergence of training dynamics.' Thus, as written, the corollaries do not follow from the theorems. This is not a minor caveat: the title and abstract promise that training with regret loss steers attention toward equilibrium behavior. The paper either needs a convergence result (even under simplifying assumptions) or must be reframed as a conditional statement: if the parameters equal the characterized stationary configuration, then deployment has the stated equilibrium property. The empirical FTRL-matching experiments in §3.2 are suggestive but not a proof.
  2. [§4.1, Theorem 2, Eqs. (9)–(10)] The swap-regret deployment guarantee is not established. Theorem 2's stationarity analysis is for the Gaussian block objective (10), while deployment feeds x_{i,t} = π_{t,i} ℓ_t through the fixed-point recurrence (9). I do not think the train/deploy distribution mismatch by itself invalidates the forward-pass identification, because the identified configuration is defined for arbitrary input blocks; no 'transfer of stationarity' is needed for the forward pass. The real gaps are (i) no proof that training reaches the stationary configuration, and (ii) the deployment calibration k_head = d·k_train in §4.1 is introduced without derivation. If d is not treated as a constant, k_head = d·k_train scales as d/√T, which is not the O(1/√T) regime on which the entropy-FTRL no-regret bound depends. Consequently Corollary 2, as stated, does not follow from Theorem 2. Either prove the calibration or w
  3. [Appendix A, Theorems 3–4; notation in §1] The claimed stationary-point scale Θ~(1/√T) is underspecified. Theorem 3 gives L'(k)<0 only for k ≤ c/√(Td), while Theorem 4 gives L'(k)>0 only for k = Θ(1/√T · polylog(d)); in fact, for k = C/√T with constant C, the negative comparator term in (24) dominates for large d, so positivity requires a √log d factor. Thus the statement 'if k = Θ~(1/√T)' is either false or relies on a nonstandard convention in which the tilde hides polylog(d) rather than only polylog(T). The notation in §1 defines Θ~ only via 'polylogarithmic factors' without specifying the variable. This matters because the no-regret stepsize in Theorem 1 and Corollary 1 is exactly the quantity whose scale is being characterized. Please state the convention precisely and adjust the theorem statements accordingly.
minor comments (6)
  1. [Abstract] The phrase 'Blum–Mansour no-pass implementation algorithm' appears to be a typo; it should read 'no-swap-regret algorithm' or 'Blum–Mansour implementation.'
  2. [Theorems 1–2 statements] The wording 'is a stationary point ... if k = Θ~(1/√T)' suggests every k in that range works. The proofs only establish existence of at least one such k via a sign change. Rephrase as 'there exists k = Θ~(1/√T) for which the configuration is stationary.'
  3. [Section 5] The limitations listed in Section 5 are appropriately candid, but they should be reflected in the abstract and corollary statements. As written, the abstract promises equilibrium behavior from training, which is stronger than the stationary-point result actually proved.
  4. [Notation, §1] Please define explicitly whether Θ~ hides polylog(T), polylog(d), or both. The current definition is ambiguous and the ambiguity affects the substance of Theorem 1 and Theorem 2.
  5. [§4.1] The empirical section would benefit from reporting the actual value of d·k_train used in the deployment experiments and comparing it with the horizon-T no-regret regime, to make the calibration less opaque.
  6. [Appendix D, Eq. (60)] The identity Φ(k) = d E[R(k)^2] + d(d-1)(E[R(k)])^2 is correct, but Proposition 5 is essential and could be stated more prominently in the main text, since it is what reduces the swap-regret derivative to the single-head external-regret derivative.

Circularity Check

0 steps flagged

No significant circularity: the stationary points are derived from first-order conditions of the squared-regret objective, with no fitted parameter forcing the FTRL/Blum–Mansour match.

full rationale

The central derivation is self-contained. Theorem 1 computes the gradient of E[Regret^2] at V=-kI, a=0, v_c=v1 and proves the scalar derivative L'(k) changes sign between k=O~(1/√(Td)) and k=Θ~(1/√T), so a root k* of the claimed scale exists; nothing is fitted to make the forward pass equal smoothed fictitious play, and no supervised trace of the algorithm is used. The forward-pass identity Softmax(-k Σ ℓ_s) = entropy-regularized FTRL is read off from the architecture after the stationarity analysis, not imposed as an ansatz. Theorem 2 likewise derives the stationary configuration for the swap objective: the proof reduces the d-head objective to d independent single-head external-regret losses, giving Φ'(k) = d·L'_ext(k) (Eq. 61), and then reuses the same Gaussian sign-change argument. The classical Blum–Mansour reduction is cited and also proved in Appendix C, so the fixed-point layer's role is independently supported. Self-citations to Park et al. (2025b, 2025a) provide context, the regret-loss framework, and a conjecture; they are not load-bearing for the proofs, which are carried out with Stein's lemma, softmax Jacobian bounds, and Gaussian supremum estimates. The paper explicitly states in Section 5 that it does not prove convergence of training dynamics and that the proof relies on full-information Gaussian training losses; these are scope/transfer limitations affecting the strength of Corollaries 1-2 as statements about trained models, not circular reductions. The empirical FTRL comparison in Section 3.2 fits a scalar η and measures closeness rather than presenting a fitted parameter as a prediction, and the k_head = d·k_train calibration in Section 4.1 is an ad hoc bridge, not a step that makes the theorem true by definition. No equation reduces to its own input, and no load-bearing claim rests solely on an author self-citation.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The central claim rests on the Gaussian training distribution, standard softmax/Stein-type tools, and an unproven transfer assumption from i.i.d. Gaussian block training to recurrent deployment. No new physical entities are introduced; the swap-regret loss is the paper's contribution and is supported by the theorems and experiments, so it is not listed as an invented entity.

free parameters (3)
  • k (attention/stepsize scale) = Θ~(1/√T) (existence proven; exact value not specified)
    The scalar k in the stationary configuration V=-kI is not fitted to data; it is the root of L'(k) whose existence is proven in the interval (c/√(Td), C/√T). The theorem gives scaling but no closed form.
  • k_head = d·k_train (deployment calibration) = d·k_train
    Ad-hoc dimension-only calibration introduced in §4.1 to account for the difference between training inputs (i.i.d. Gaussian blocks) and deployment inputs (π_t-scaled losses). No derivation is given; it is load-bearing for the CE evaluation and for the claim that deployment matches Blum–Mansour.
  • fitted FTRL stepsize η in §3.2 = 0.4456±0.0046 (d=3, T=20)
    A scalar fitted to trained model outputs to measure closeness to entropy-FTRL; used as empirical evidence for Theorem 1, not part of the theoretical derivation.
axioms (6)
  • domain assumption Full-information Gaussian loss sequence ℓ_t∼N(0,I_d) i.i.d. during training
    The permutation-symmetry and Stein's-lemma arguments in Appendices A and D rely on this distribution; the stationary-point theorems are proven for Gaussian losses, not for arbitrary bounded loss sequences.
  • standard math Standard Softmax properties: shift-invariance, Jacobian PSD, permutation equivariance
    Used throughout to reduce gradients to multiples of the zero-sum projector and to establish pointwise vanishing of certain gradient blocks. These are exact properties, not approximations.
  • standard math Stein's lemma for Gaussian vectors
    Used in Fact 3 and Lemma 14 to compute derivatives of expectations under Gaussian losses; standard tool.
  • standard math Gaussian supremum moment and MGF bounds (Lemmas 8, 12)
    Used to control remainder terms in the sign estimates of L'(k); bounds are stated with absolute constants whose values are not explicitly computed.
  • ad hoc to paper Transfer of stationary configuration from training distribution to deployment distribution (including Fixed-Point layer)
    Assumed in Corollaries 1–2 and §4.1; not proven. The paper only provides an empirical calibration k_head = d·k_train.
  • standard math Perron–Frobenius/implicit-function theorem for differentiability of the fixed-point map
    Remark 1 uses Perron–Frobenius and the implicit function theorem to justify differentiability for strictly positive column-stochastic matrices; standard.

pith-pipeline@v1.3.0-alltime-deepseek · 37064 in / 18670 out tokens · 174676 ms · 2026-07-31T23:42:46.278243+00:00 · methodology

0 comments
read the original abstract

We revisit the regret loss framework introduced in Park et al. (2025), which uses decision-theoretic regret as a direct loss function for training models to make better decisions, through the lens of probability-simplex policies. Our first result shows that a single-layer self-attention model trained with regret loss admits a stationary point whose forward-pass exactly matches smoothed fictitious play with the appropriate stepsize that ensures no-regret behavior-i.e., for any given policy input, the model outputs the same update that smoothed fictitious play would produce. In parallel, we also newly introduce a swap-regret loss function, which extends the regret-loss framework beyond external regret and enables models to directly optimize for swap-deviation robustness. We further show that this swap-regret loss admits a stationary point whose forward pass implements the corresponding swap-regret update induced by classical Blum-Mansour no-pass implementation algorithm, with each head implementing an external-regret update via smoothed fictitious play. Together, these results show that regret-trained attention can realize differentiable mechanisms whose deployment induces equilibrium behavior in games: external-regret dynamics lead to coarse correlated equilibrium, while swap-regret dynamics lead to correlated equilibrium. Thus, regret-based objectives steer minimal attention architectures toward online-learning dynamics with game-theoretic guarantees, without supervised traces of those algorithms.

Figures

Figures reproduced from arXiv: 2607.23333 by Asuman Ozdaglar, Chanwoo Park.

Figure 1
Figure 1. Figure 1: Estimated stationary stepsize k ⋆ on the probability simplex as a function of (d, T ) for the single-head softmax model (left) and the swap-regret architecture (right). Each marker aggregates 50 independent runs with 800 Monte Carlo samples per run; shaded bands indicate ±1 standard deviation. Corollary 1. Consider a repeated game where each player deploys the trained single-layer architecture from Theorem… view at source ↗
Figure 2
Figure 2. Figure 2: Random-initialization regret-loss training at [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Multi-head linear self-attention architecture for swap-regret minimization. Each head [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Equilibrium gaps of time-averaged joint mixed play in weighted rock–paper–scissors. Left: [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 85 linked inside Pith

  1. [1]

    Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003

    A general class of no-regret learning algorithms and game-theoretic equilibria , author=. Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003. Proceedings , pages=. 2003 , organization=

    2003

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Computational equivalence of fixed points and no regret algorithms, and convergence to equilibria , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    2013 , publisher=

    Matrix analysis , author=. 2013 , publisher=

    2013

  4. [4]

    The annals of Statistics , pages=

    Estimation of the mean of a multivariate normal distribution , author=. The annals of Statistics , pages=. 1981 , publisher=

    1981

  5. [5]

    The Annals of Mathematical Statistics , volume=

    Values of Mills' ratio of area to bounding ordinate and of the normal probability integral for large values of the argument , author=. The Annals of Mathematical Statistics , volume=. 1941 , publisher=

    1941

  6. [6]

    Games and Economic Behavior , volume=

    Learning correlated equilibria in games with compact sets of strategies , author=. Games and Economic Behavior , volume=. 2007 , publisher=

    2007

  7. [7]

    2024 , eprint=

    Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality , author=. 2024 , eprint=

    2024

  8. [8]

    2024 , eprint=

    Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context , author=. 2024 , eprint=

    2024

  9. [9]

    Advances in Neural Information Processing Systems , volume=

    Efficient phi-regret minimization in extensive-form games via online mirror descent , author=. Advances in Neural Information Processing Systems , volume=

  10. [10]

    Proceedings of the 40th International Conference on Machine Learning , pages =

    Near-Optimal Phi-Regret Learning in Extensive-Form Games , author =. Proceedings of the 40th International Conference on Machine Learning , pages =. 2023 , editor =

    2023

  11. [11]

    International Conference on Machine Learning , pages=

    Constrained phi-equilibria , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  12. [12]

    arXiv preprint arXiv:2505.17277 , year=

    Comparator-Adaptive Phi-Regret: Improved Bounds, Simpler Algorithms, and Applications to Games , author=. arXiv preprint arXiv:2505.17277 , year=

    arXiv

  13. [13]

    arXiv preprint arXiv:2402.00396 , year=

    Efficient exploration for llms , author=. arXiv preprint arXiv:2402.00396 , year=

    Pith/arXiv arXiv

  14. [14]

    Exploration-driven policy optimization in

    Du, Yihan and Winnicki, Anna and Dalal, Gal and Mannor, Shie and Srikant, R , journal=. Exploration-driven policy optimization in

  15. [15]

    Proceedings of the 25th international conference on Machine learning , pages=

    No-regret learning in convex games , author=. Proceedings of the 25th international conference on Machine learning , pages=

  16. [16]

    International conference on machine learning , pages=

    Linear transformers are secretly fast weight programmers , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  17. [17]

    Exploratory Preference Optimization: Harnessing Implicit

    Xie, Tengyang and Foster, Dylan J and Krishnamurthy, Akshay and Rosset, Corby and Awadallah, Ahmed Hassan and Rakhlin, Alexander , booktitle=. Exploratory Preference Optimization: Harnessing Implicit

  18. [18]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  19. [19]

    arXiv preprint arXiv:2412.16720 , year=

    Openai o1 system card , author=. arXiv preprint arXiv:2412.16720 , year=

    Pith/arXiv arXiv

  20. [20]

    Advances in neural information processing systems , volume=

    Decision transformer: Reinforcement learning via sequence modeling , author=. Advances in neural information processing systems , volume=

  21. [21]

    arXiv preprint arXiv:2310.06770 , year=

    Swe-bench: Can language models resolve real-world github issues? , author=. arXiv preprint arXiv:2310.06770 , year=

    Pith/arXiv arXiv

  22. [22]

    agentic ai: A conceptual taxonomy, applications and challenges , author=

    Ai agents vs. agentic ai: A conceptual taxonomy, applications and challenges , author=. arXiv preprint arXiv:2505.10468 , year=

    arXiv

  23. [23]

    arXiv preprint arXiv:2502.01118 , year=

    Large language model-enhanced multi-armed bandits , author=. arXiv preprint arXiv:2502.01118 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2406.05064 , year=

    Pretraining decision transformers with reward prediction for in-context multi-task structured bandit learning , author=. arXiv preprint arXiv:2406.05064 , year=

    arXiv

  25. [25]

    arXiv e-prints , pages=

    Monea, Giovanni and Bosselut, Antoine and Brantley, Kiant. arXiv e-prints , pages=

  26. [26]

    International Conference on Learning Representations , year=

    SmartPlay: A Benchmark for LLMs as Intelligent Agents , author=. International Conference on Learning Representations , year=

  27. [27]

    arXiv preprint arXiv:2406.00244 , year=

    Controlling large language model agents with entropic activation steering , author=. arXiv preprint arXiv:2406.00244 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2404.02649 , year=

    On the importance of uncertainty in decision-making with large language models , author=. arXiv preprint arXiv:2404.02649 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2312.03801 , year=

    Generalization to new sequential decision making tasks with in-context learning , author=. arXiv preprint arXiv:2312.03801 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2505.09901 , year=

    Comparing Exploration-Exploitation Strategies of LLMs and Humans: Insights from Standard Multi-armed Bandit Tasks , author=. arXiv preprint arXiv:2505.09901 , year=

    Pith/arXiv arXiv

  31. [31]

    Beyond numeric awards: In-context dueling bandits with

    Xia, Fanzeng and Liu, Hao and Yue, Yisong and Li, Tongxin , journal=. Beyond numeric awards: In-context dueling bandits with

  32. [32]

    arXiv preprint arXiv:2405.16376 , year=

    Stride: A tool-assisted llm agent framework for strategic and interactive decision-making , author=. arXiv preprint arXiv:2405.16376 , year=

    Pith/arXiv arXiv

  33. [33]

    Nature , volume=

    A graph placement methodology for fast chip design , author=. Nature , volume=. 2021 , publisher=

    2021

  34. [34]

    Nature medicine , volume=

    The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care , author=. Nature medicine , volume=. 2018 , publisher=

    2018

  35. [35]

    The International Journal of Robotics Research , volume=

    Learning dexterous in-hand manipulation , author=. The International Journal of Robotics Research , volume=. 2020 , publisher=

    2020

  36. [36]

    2022 , publisher=

    Algorithms for reinforcement learning , author=. 2022 , publisher=

    2022

  37. [37]

    1999 , publisher=

    Coordination games , author=. 1999 , publisher=

    1999

  38. [38]

    MIT Press Books , volume=

    A general theory of equilibrium selection in games , author=. MIT Press Books , volume=. 1988 , publisher=

    1988

  39. [39]

    arXiv preprint arXiv:2112.11471 , year=

    Towards a science of human-ai decision making: a survey of empirical studies , author=. arXiv preprint arXiv:2112.11471 , year=

    Pith/arXiv arXiv

  40. [40]

    Journal of Machine Learning Research , volume=

    Using confidence bounds for exploitation-exploration trade-offs , author=. Journal of Machine Learning Research , volume=

  41. [41]

    https://docs.aws.amazon.com/bedrock/ , year=

    Amazon Bedrock Documentation , author=. https://docs.aws.amazon.com/bedrock/ , year=

  42. [42]

    https://ai.google.dev/gemini-api/docs/model-tuning , year=

    Fine-tuning with the Gemini API , author=. https://ai.google.dev/gemini-api/docs/model-tuning , year=

  43. [43]

    https://docs.mistral.ai/guides/finetuning/ , year=

    Fine-tuning Mistral Models , author=. https://docs.mistral.ai/guides/finetuning/ , year=

  44. [44]

    arXiv preprint arXiv:2408.00118 , year=

    Gemma 2: Improving open language models at a practical size , author=. arXiv preprint arXiv:2408.00118 , year=

    Pith/arXiv arXiv

  45. [45]

    arXiv preprint arXiv:2201.10005 , year=

    Text and code embeddings by contrastive pre-training , author=. arXiv preprint arXiv:2201.10005 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2403.05530 , year=

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=

    Pith/arXiv arXiv

  47. [47]

    European Journal of Information Systems , volume=

    Design principles for artificial intelligence-augmented decision making: An action design research study , author=. European Journal of Information Systems , volume=. 2025 , publisher=

    2025

  48. [48]

    1966 , publisher=

    Behavior of sequential predictors of binary sequences , author=. 1966 , publisher=

    1966

  49. [49]

    Proceedings of the third Annual Workshop on Computational Learning Theory , pages=

    Aggregating strategies , author=. Proceedings of the third Annual Workshop on Computational Learning Theory , pages=

  50. [50]

    arXiv preprint arXiv:2502.18439 , year=

    Maporl: Multi-agent post-co-training for collaborative large language models with reinforcement learning , author=. arXiv preprint arXiv:2502.18439 , year=

    Pith/arXiv arXiv

  51. [51]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  52. [52]

    Autonomous Agents and Multi-Agent Systems , volume=

    A survey and critique of multiagent deep reinforcement learning , author=. Autonomous Agents and Multi-Agent Systems , volume=. 2019 , publisher=

    2019

  53. [53]

    ICLR , year=

    Multiagent Finetuning of Language Models , author=. ICLR , year=

  54. [54]

    ICLR , year=

    Decoupled weight decay regularization , author=. ICLR , year=

  55. [55]

    Games and Economic Behavior , volume=

    Quantal response methods for equilibrium selection in 2 2 coordination games , author=. Games and Economic Behavior , volume=. 2016 , publisher=

    2016

  56. [56]

    arXiv preprint arXiv:2502.04780 , year=

    SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning , author=. arXiv preprint arXiv:2502.04780 , year=

    Pith/arXiv arXiv

  57. [57]

    arXiv preprint arXiv:2211.14275 , year=

    Solving math word problems with process-and outcome-based feedback , author=. arXiv preprint arXiv:2211.14275 , year=

    Pith/arXiv arXiv

  58. [58]

    ICLR , year=

    The curious case of neural text degeneration , author=. ICLR , year=

  59. [59]

    arXiv preprint arXiv:1606.06565 , year=

    Concrete problems in AI safety , author=. arXiv preprint arXiv:1606.06565 , year=

    Pith/arXiv arXiv

  60. [60]

    Advances in neural information processing systems , volume=

    Inverse reward design , author=. Advances in neural information processing systems , volume=

  61. [61]

    arXiv preprint arXiv:2305.20050 , year=

    Let's verify step by step , author=. arXiv preprint arXiv:2305.20050 , year=

    Pith/arXiv arXiv

  62. [62]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  63. [63]

    2025 , journal=

    When One LLM Drools, Multi-LLM Collaboration Rules , author=. 2025 , journal=

    2025

  64. [64]

    Games and Economic Behavior , volume=

    A new theory of equilibrium selection for games with complete information , author=. Games and Economic Behavior , volume=. 1995 , publisher=

    1995

  65. [65]

    ICLR , year=

    Training language models to self-correct via reinforcement learning , author=. ICLR , year=

  66. [66]

    NAACL , year=

    Teaching models to balance resisting and accepting persuasion , author=. NAACL , year=

  67. [67]

    arXiv preprint arXiv:1709.06011 , year=

    Guided deep reinforcement learning for swarm systems , author=. arXiv preprint arXiv:1709.06011 , year=

    Pith/arXiv arXiv

  68. [68]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    Pith/arXiv arXiv

  69. [69]

    Advances in Neural Information Processing Systems , volume=

    Qlora: Efficient finetuning of quantized llms , author=. Advances in Neural Information Processing Systems , volume=

  70. [70]

    5 technical report , author=

    Qwen2. 5 technical report , author=. arXiv preprint arXiv:2412.15115 , year=

    Pith/arXiv arXiv

  71. [71]

    arXiv preprint arXiv:2310.15421 , year=

    FANToM: A benchmark for stress-testing machine theory of mind in interactions , author=. arXiv preprint arXiv:2310.15421 , year=

    Pith/arXiv arXiv

  72. [72]

    arXiv preprint arXiv:2312.09241 , year=

    Tinygsm: achieving> 80\ author=. arXiv preprint arXiv:2312.09241 , year=

    Pith/arXiv arXiv

  73. [73]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  74. [74]

    International Conference on Machine Learning , pages=

    Principled reinforcement learning with human feedback from pairwise or k-wise comparisons , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  75. [75]

    NeurIPS , year=

    MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making , author=. NeurIPS , year=

  76. [76]

    arXiv preprint arXiv:1909.08593 , year=

    Fine-tuning language models from human preferences , author=. arXiv preprint arXiv:1909.08593 , year=

    Pith/arXiv arXiv 1909

  77. [77]

    NAACL Findings , year=

    OVM,Outcome-supervised Value Models for Planning in Mathematical Reasoning , author=. NAACL Findings , year=

  78. [78]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  79. [79]

    ACL , year=

    Adversarial NLI: A new benchmark for natural language understanding , author=. ACL , year=

  80. [80]

    arXiv preprint arXiv:2402.05133 , year=

    Personalized Language Modeling from Personalized Human Feedback , author=. arXiv preprint arXiv:2402.05133 , year=

    Pith/arXiv arXiv

Showing first 80 references.