Pith. sign in

REVIEW 4 major objections 4 minor 7 cited by

Multi-task GRPO claims it can lift worst-task accuracy by 16–28% over standard GRPO and 6% over DAPO while keeping average accuracy competitive.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:11 UTC pith:JV7MQWMU

load-bearing objection Plausible practical recipe for multi-task GRPO, but the 'improvement' signal is a weak reed and the empirical claims rest on single runs. the 4 major comments →

arxiv 2602.05547 v2 pith:JV7MQWMU submitted 2026-02-05 cs.CL cs.AIcs.LG

Multi-Task GRPO: Reliable LLM Reasoning Across Tasks

Shyam Sundhar Ramesh , Xiaotong Ji , Matthieu Zimmer , Sangwoong Yoon , Zhiyong Wang , Haitham Bou Ammar , Aurelien Lucchi , Ilija Bogunovic This is my paper
classification cs.CL cs.AIcs.LG
keywords multi-task reinforcement learningGRPOLLM reasoningworst-task accuracytask reweightingzero-gradient promptsratio-preserving samplingdistributionally robust optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that the standard recipe for applying GRPO to multiple reasoning tasks at once—averaging per-task losses and sampling uniformly—quietly abandons the weakest tasks. The authors propose Multi-Task GRPO, which does two things: it reweights tasks each step using both each task's reward and how much that task improved over the previous update, and it uses a ratio-preserving sampler so the intended task proportions survive the zero-gradient filtering that GRPO performs. In experiments post-training a 3B model on three and nine reasoning tasks (Countdown, Zebra, ARC), the method reports 16–28% absolute gains in worst-task accuracy over standard GRPO, 6% over DAPO, similar average accuracy, and reaches a given worst-task threshold in half the steps. A sympathetic reader would care because it offers a concrete path to LLM post-training that optimizes reliability across skills rather than average benchmark scores.

Core claim

MT-GRPO claims that robustness in multi-task RL post-training can be achieved by decoupling task reweighting from the GRPO loss, which the authors show is unusable for comparing tasks because it evaluates to zero both when a task's rollouts are all wrong and all correct. Instead, task weights are updated from true task rewards J_k plus an improvement signal I_k = J_GRPO,k(θ_{t+1}) − J_GRPO,k(θ_t) (Subroutine 1), which upweights tasks that are both weak and improving slowly, while a ratio-preserving sampler oversamples high filtering-rate tasks and resamples until post-filtered batch counts match a Multinomial(B,z) draw. The paper's experiments claim this yields consistent worst-task accuracy

What carries the argument

The load-bearing objects are (1) the improvement-aware weight update IWU (Subroutine 1), which uses the per-task improvement I_k^{(t)} = J_GRPO,k(θ_{t+1}) − J_GRPO,k(θ_t) added to λ J_k(θ_t) as the signal for softmax logits ξ, and (2) the Ratio-Preserving Sampler (Algorithm 2), which draws desired post-filtered counts from Multinomial(B, z), inflates sampling by estimated filtering rate (capped at M_acc), and resamples by deficiency until counts are met or budget exhausted. The sampler is what translates the adapted weights into realized gradient proportions.

Load-bearing premise

The load-bearing premise is that the per-step improvement signal I_k^{(t)}—the difference of GRPO objectives before and after an update—accurately tells how much each task is learning; if a failing task's GRPO objective sits at zero (or is filtered away), it can look identical to a task that is already solved.

What would settle it

Take two tasks and set one to always produce all-correct rollouts and the other all-wrong rollouts (or filter both out). Both have J_GRPO = 0, so IWU sees no difference and cannot upweight the failing task; a run would show worst-task accuracy stagnating. Or, without such extreme settings, compare I_k^{(t)} to held-out task accuracy changes across 100 steps; if the correlation is near zero, the reweighting is not the cause of the reported gains.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single λ controls the trade-off between worst-task robustness and average accuracy, so practitioners can dial how much to protect weak tasks.
  • Post-filtering zero-gradient prompts no longer silently skew training toward easy, low-filtering tasks.
  • Worst-task accuracy targets can be reached in roughly half the steps in the 3-task setting, reducing compute for reliable multi-task models.
  • The approach extends to 9 tasks across difficulty levels, suggesting it scales beyond a few curated tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test the case where a task is genuinely stuck at all-correct or all-wrong rollouts; in that regime I_k is zero by construction, so the reweighting would treat a completely failing task the same as a perfectly solved one — an easy ablation would reveal whether this matters.
  • The improvement signal is estimated from a single policy step; a natural extension is to use a moving average of I_k or holdout rewards to reduce noise before deciding to upweight a task.
  • The ratio-preserving sampler is a general fix for any GRPO-style method that filters zero-advantage prompts, so it could be dropped into single-task training to control which prompts survive, not just multi-task.
  • If worst-task accuracy is the right reliability metric, the same reweighting loop could be applied to other RL post-training objectives (PPO, RLOO) wherever the per-task loss has similar zero-ambiguity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Multi-Task GRPO (MT-GRPO), a post-training algorithm for multi-task LLM reasoning. MT-GRPO combines improvement-aware task reweighting (Subroutine 1) with a ratio-preserving sampler (Algorithm 2) to address two issues in multi-task GRPO: average-reward optimization neglecting worst-task performance, and uneven zero-gradient rates causing realized batch proportions to deviate from intended task weights. The authors evaluate on Qwen-2.5-3B in a 3-task setting (Countdown, Zebra, ARC) and a 9-task setting (easy/medium/hard variants), reporting consistent worst-task accuracy improvements over GRPO, SEC-GRPO, DAPO, and SEC-DAPO, with a claimed 16–28% absolute worst-task improvement over GRPO and 6% over DAPO, while maintaining competitive average accuracy. The paper also provides a theoretical derivation of the objective from a constrained formulation and appendices with ablations.

Significance. If the empirical claims hold, MT-GRPO offers a practical, lightweight modification to GRPO for improving robustness across heterogeneous reasoning tasks. The ratio-preserving sampler directly targets a real structural problem in multi-task GRPO (zero-gradient filtering skewing task proportions), and the paper provides code. However, the central 'improvement-aware' mechanism rests on an unreliable signal: Eq. (8) inherits the zero-ambiguity of the GRPO loss that the paper itself flags in §3.1, and all headline results appear to be single runs with no error bars. The method is plausible and potentially useful, but the evidence as presented is not yet convincing enough for publication without substantial revision.

major comments (4)
  1. [§3.1, §3.2, Eq. (8)] The paper's own §3.1 argues that the GRPO loss is unreliable for task reweighting because it evaluates to zero both when all rollouts are incorrect and when all are correct. Yet Eq. (8) defines the improvement signal I_k^{(t)} as the difference of J_GRPO,k between consecutive iterates, and Subroutine 1 uses this signal to reweight tasks. With the KL coefficient set to 0 (Appendix D) and group-mean-centered advantages, J_GRPO,k(θ_t) is zero by construction, so I_k^{(t)} reduces to J_GRPO,k(θ_{t+1}) — a one-step importance-weighted surrogate, not a measurement of task-wise learning progress. For any prompt with constant rewards across rollouts, I_k contributes zero whether the task is perfectly solved or completely stuck. Thus the 'improvement-aware' reweighting does not actually measure improvement as claimed. The λ J_k(θ_t) term can partly compensate, but then the mechanism is reward-bas
  2. [Appendix D, Subroutine 1] The improvement signal I_k^{(t)} is clipped to [0.1, 0.2] for stability, giving a maximum dynamic range of 0.1. For the λ values used (0.2, 0.25, 0.9, 1.2) and J_k in [0,1], the reward term λ J_k(θ_t) dominates s_k^{(t)} in magnitude. The ablation in Appendix E compares Subroutine 1 (improvement-aware) with Subroutine 2 (regularized reward-only), but these subroutines differ in multiple respects: presence of I_k, softmax-centering structure, and the regularization hyperparameter η. This is not a controlled ablation of the improvement term. To support the claim that I_k is doing causal work, the authors should run a controlled comparison (same update structure with and without I_k, and with I_k unclipped or rescaled) across multiple seeds.
  3. [§6, Figures 4–11] All reported accuracies appear to be single runs with no standard errors or confidence intervals. The headline claim of '6% absolute improvement over DAPO' (abstract, §6.2) is a difference between two point estimates. GRPO training on LLMs is seed-sensitive, and the 6% gap may be within run-to-run noise. Because the central claim is a comparative empirical claim ('consistently outperforms baselines'), the paper should report at least 3 seeds with mean and standard deviation (or another appropriate uncertainty measure) for the main results. This is load-bearing; without it, the robustness gains are not established.
  4. [Abstract, §6.2, Figure 7] The abstract and §6.2 state that MT-GRPO maintains 'competitive average accuracy' while improving worst-task accuracy. In the 9-task experiment, MT-GRPO with λ=1.2 has average accuracy 61.7 versus DAPO's 68.7, a drop of 7 points. This may be an acceptable trade-off, but 'competitive' is misleading for that setting. The paper should qualify this claim and discuss how the trade-off scales with λ, perhaps by reporting the best λ for each notion of competitive average accuracy.
minor comments (4)
  1. [Eq. (11)] Eq. (11) contains a redundant repetition: 'A(x, y_i) = A(x, y_i) = ...'. Please clean up the notation.
  2. [Figure 4/7 captions] The multi-panel captions are dense and the bars are not all individually labeled in the text. Adding explicit numeric accuracy values in the caption or body text would improve readability.
  3. [§B.1] In the derivation of Ω(z), the notation around the double sum Σ_{k,j} μ_{kj} is used inconsistently ('Σ_{k=1}^K Σ_{j=1}^K' vs 'Σ_{k,j}'). Please standardize to avoid confusion.
  4. [§6.1, Figure 5] The claim '50% fewer training steps' should specify the baseline and threshold explicitly. It is stated vaguely in the text and the figure mixes methods; please clarify.

Circularity Check

0 steps flagged

No significant circularity: MT-GRPO's derivation is self-contained; the Eq. (8) improvement-signal concern is a non-circular correctness issue.

full rationale

MT-GRPO's derivation chain is self-contained. The robust objective (Eqs. 4–5) is obtained by a standard Lagrangian relaxation derived in Sec. B.1; the alternating updates (Eqs. 6–7) follow from softmax-parameterized gradient descent on the inner objective; and the improvement-aware update in Subroutine 1 is derived in Sec. B.3 from a first-order Taylor expansion of a per-step minimax objective, not assumed as a conclusion. The empirical evaluation uses external ReasoningGym datasets and benchmarks against GRPO/DAPO baselines that do not use MT-GRPO's task weights; no parameter is fitted to the reported worst-task accuracies and then relabeled as a prediction. There are no load-bearing self-citations: the only borrowed conceptual ingredient, the task-level improvement signal, is cited to Famo (Liu et al. [11]), whose authors do not overlap with this paper. The one concern worth flagging is non-circular: Sec. 3.1 warns that J_GRPO is zero-ambiguous for cross-task comparison, and Eq. (8) defines I_k as a difference of J_GRPO values; with the KL coefficient set to 0 (Appendix D) and mean-centered group advantages, J_GRPO,k(θ_t) is zero by construction, so I_k collapses to a one-step surrogate inheriting the same ambiguity. This is a signal-quality / correctness risk for the mechanism, not a circular reduction: the central claim is not forced by its inputs, the comparisons are external, and the paper's reported gains stand or fall on reproducibility rather than on a definitional equivalence.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central algorithm depends on several tuned hyperparameters (λ, clipping bounds, IWU optimizer settings, RPSAMPLER factors) and one domain-specific assumption that the GRPO-loss improvement signal is meaningful despite the paper's own critique of J_GRPO as a task-comparison signal. No new physical entities are introduced; the algorithmic components are not independent entities.

free parameters (4)
  • λ trade-off parameter = 0.2, 0.25 (Exp 1); 0.1, 0.3, 0.9, 1.2 (Exp 2)
    Controls the balance between worst-task reward and improvement in Subroutine 1; the reported gains vary with λ and there is no principled selection rule.
  • Improvement clipping bounds = [0.1, 0.2]
    Section D: improvements I_k^{(t)} are “clipped to {0.1,0.2} for stability”—an ad hoc chosen range that directly affects weight updates.
  • IWU optimizer hyperparameters = lr 0.025, weight decay 1e-5 (Exp 1); lr 1e-4 (Exp 2)
    Gradient descent on task-weight logits in Subroutine 1 uses these values; no sensitivity analysis provided.
  • RPSAMPLER hyperparameters = M_os=3, N_rs=10 (Exp 1) / 2 (Exp 2), M_acc=5
    Oversampling factor, maximum resampling rounds, and maximum acceptance inflation factor are hand-set and affect how faithfully task weights are realized.
axioms (4)
  • standard math Lagrangian relaxation and strong duality hold for the constrained multi-task objective (Eq 4–5, B.1, B.3).
    The max–min reformulation and the closed form of Ω(z) rely on convexity and compactness of the simplex; standard but unverified in this setting.
  • ad hoc to paper The GRPO loss difference I_k^{(t)} is a meaningful improvement signal despite being zero for all-correct and all-wrong rollouts.
    Invoked in Eq (8) and Subroutine 1, even though §3.1 argues J_GRPO is unreliable for task comparison because of this exact ambiguity.
  • domain assumption All tasks use the same reward protocol (1.0 correct, 0.1 incorrect-format, 0 otherwise), making task rewards comparable without normalization.
    Stated in Section D; if reward scales differed, a single λ would bias reweighting toward high-variance tasks.
  • domain assumption Zero-gradient filtering removes only non-informative prompts and does not bias the per-task advantage estimates used for policy updates.
    The RPSAMPLER correctness argument depends on this; the paper does not analyze bias introduced by the filter-resample loop.

pith-pipeline@v1.3.0-alltime-deepseek · 25761 in / 12989 out tokens · 133427 ms · 2026-08-03T04:11:22.719980+00:00 · methodology

0 comments
read the original abstract

RL-based post-training with GRPO is widely used to improve large language models on individual reasoning tasks. However, real-world deployment requires reliable performance across diverse tasks. A straightforward multi-task adaptation of GRPO often leads to imbalanced outcomes, with some tasks dominating optimization while others stagnate. Moreover, tasks can vary widely in how frequently prompts yield zero advantages (and thus zero gradients), which further distorts their effective contribution to the optimization signal. To address these issues, we propose a novel Multi-Task GRPO (MT-GRPO) algorithm that (i) dynamically adapts task weights to explicitly optimize worst-task performance and promote balanced progress across tasks, and (ii) introduces a ratio-preserving sampler to ensure task-wise policy gradients reflect the adapted weights. Experiments on both 3-task and 9-task settings show that MT-GRPO consistently outperforms baselines in worst-task accuracy. In particular, MT-GRPO achieves 16-28% and 6% absolute improvement on worst-task performance over standard GRPO and DAPO, respectively, while maintaining competitive average accuracy. Moreover, MT-GRPO requires 50% fewer training steps to reach 50% worst-task accuracy in the 3-task setting, demonstrating substantially improved efficiency in achieving reliable performance across tasks.

Figures

Figures reproduced from arXiv: 2602.05547 by Aurelien Lucchi, Haitham Bou Ammar, Ilija Bogunovic, Matthieu Zimmer, Sangwoong Yoon, Shyam Sundhar Ramesh, Xiaotong Ji, Zhiyong Wang.

Figure 1
Figure 1. Figure 1: GRPO assigns uniform task weights and samples without regard to task difficulty or zero-gradient rates. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: In strict worst-task optimization (ε = 0), task weights rapidly collapse to the current worst task and oscillate as the worst task shifts, resulting in near-zero weighting of Countdown. Issues with strict worst-task reward maximization: While the alternating updates in Equation (6) correctly optimize the minimax objective in Equation (5) for ε = 0, they can lead to degenerate dynamics in which training is … view at source ↗
Figure 3
Figure 3. Figure 3: Ratios of zero-gradient prompts across tasks [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Experiment 1: MT-GRPO substantially outperforms all baselines in terms of worst-task accuracy [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: MT-GRPO reaches target worst-task accuracy [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Experiment 1: Top plots: Task-wise accuracies. Bottom plots: MT-GRPO reallocates weights toward the under-performing tasks. In contrast, baselines continue to prioritize high-performing Countdown, leading to weaker performance on Zebra or ARC and only marginal gains on Countdown. ARC is typically underrepresented in training batches relative to its task weight (middle vs right). RP SAMPLER ensures alignmen… view at source ↗
Figure 7
Figure 7. Figure 7: Experiment 2: Top plots: Increasing λ improves worst-task accuracy of MT-GRPO but reduces average accuracy, highlighting a trade-off controlled by λ. λ = 0.1 yields the highest average per-task relative change, showing stronger gains on weaker tasks. Bottom plots: For smaller λ (0.1, 0.3), MT-GRPO prioritizes slower-improving tasks, yielding larger gains on hard tasks while sacrificing easy ones. For large… view at source ↗
Figure 8
Figure 8. Figure 8: Experiment 1 (3-tasks): Left: Comparison between MT-GRPO with and without RP SAMPLER (RPS). In the absence of RPS, MT-GRPO increases the weight assigned to ARC in an attempt to compensate for the mismatch between target weights and effective batch representation. However, the effective ARC representation still remains lower than with RPS, highlighting the benefit of RP SAMPLER . Right: Comparison between M… view at source ↗
Figure 9
Figure 9. Figure 9: Left: Task weights assigned to Zebra across methods in Experiment 1. (See [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Experiment 1 (3 tasks): Comparison of improvement-aware updates (Subroutine 1) and regularized [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Experiment 2 (9 tasks): Comparison between improvement-aware updates (Subroutine 1) and [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR

    cs.AI 2026-06 unverdicted novelty 7.0

    TAC is a bandit curriculum for multi-domain RLVR that prioritizes domains whose gradient updates align with and benefit other domains, yielding up to 2.8-point macro accuracy gains over learnability-only baselines on ...

  2. Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR

    cs.AI 2026-06 unverdicted novelty 7.0

    TAC is a new online curriculum for multi-domain RLVR that estimates cross-domain transferability from gradient geometry to achieve up to 2.8 point macro-averaged accuracy gains over learnability-only baselines on six-...

  3. Harmony in Diversity: Multi-domain Contrastive Policy Optimization for Large Reasoning Models

    cs.CL 2026-05 unverdicted novelty 7.0

    MCPO applies contrastive learning to GRPO-style RL by treating cross-domain correct rollouts as positives and incorrect ones as negatives to improve multi-domain reasoning performance in LRMs.

  4. World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments

    cs.AI 2026-07 unverdicted novelty 6.0

    MedAgentBench-v3 shows capability ceilings and format-knowledge barriers limit pure RL to 18.2% while rule-based SFT reaches 34.1% on clinical protocol tasks.

  5. Target Policy Optimization

    cs.LG 2026-04 unverdicted novelty 6.0

    TPO constructs a target distribution q proportional to the old policy times exp(utility) and trains the policy to match it via cross-entropy, matching or beating PPO and GRPO especially under sparse rewards.

  6. CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflicts

    cs.LG 2026-05 unverdicted novelty 5.0

    CARE-RL combines PA-GRM for task-adaptive rewards on open-ended tasks and DACSP for modulating RL updates using historical capability directions, reporting higher total average scores than baselines on Qwen models.

  7. M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Models

    cs.AI 2026-05 unverdicted novelty 5.0

    M2A uses null-space model merging to combine mathematical and agentic reasoning in LLMs, raising SWE-Bench Verified performance from 44.0% to 51.2% on Qwen3-8B without retraining.

Reference graph

Works this paper leans on

88 extracted references · 44 linked inside Pith · cited by 6 Pith papers

  1. [1]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Self-evolving curriculum for llm reasoning.arXiv preprint arXiv:2505.14970, 2025

    Xiaoyin Chen, Jiarui Lu, Minsu Kim, Dinghuai Zhang, Jian Tang, Alexandre Piché, Nicolas Gontier, Yoshua Bengio, and Ehsan Kamalloo. Self-evolving curriculum for llm reasoning.arXiv preprint arXiv:2505.14970, 2025

    arXiv 2025

  5. [5]

    Nemotron-crossthink: Scaling self-learning beyond math reasoning.arXiv preprint arXiv:2504.13941, 2025

    Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, et al. Nemotron-crossthink: Scaling self-learning beyond math reasoning.arXiv preprint arXiv:2504.13941, 2025

    arXiv 2025

  6. [6]

    Understanding and improving information transfer in multi-task learning.arXiv preprint arXiv:2005.00944, 2020

    Sen Wu, Hongyang R Zhang, and Christopher Ré. Understanding and improving information transfer in multi-task learning.arXiv preprint arXiv:2005.00944, 2020

    Pith/arXiv arXiv 2005

  7. [7]

    Gradient surgery for multi-task learning.Advances in neural information processing systems, 33:5824–5836, 2020

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning.Advances in neural information processing systems, 33:5824–5836, 2020. Multi-Task GRPO 12

    2020

  8. [8]

    Stochastic gradient methods for distributionally robust optimiza- tion with f-divergences.Advances in neural information processing systems, 29, 2016

    Hongseok Namkoong and John C Duchi. Stochastic gradient methods for distributionally robust optimiza- tion with f-divergences.Advances in neural information processing systems, 29, 2016

    2016

  9. [9]

    Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.arXiv preprint arXiv:1911.08731, 2019

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.arXiv preprint arXiv:1911.08731, 2019

    Pith/arXiv arXiv 1911

  10. [10]

    Multiple-gradient descent algorithm (mgda) for multiobjective optimization

    Jean-Antoine Désidéri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5-6):313–318, 2012

    2012

  11. [11]

    Famo: Fast adaptive multitask optimization.Advances in Neural Information Processing Systems, 36:57226–57243, 2023

    Bo Liu, Yihao Feng, Peter Stone, and Qiang Liu. Famo: Fast adaptive multitask optimization.Advances in Neural Information Processing Systems, 36:57226–57243, 2023

    2023

  12. [12]

    Distributionally robust language modeling.arXiv preprint arXiv:1909.02060, 2019

    Yonatan Oren, Shiori Sagawa, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust language modeling.arXiv preprint arXiv:1909.02060, 2019

    Pith/arXiv arXiv 1909

  13. [13]

    Doremi: Optimizing data mixtures speeds up language model pretraining

    Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V Le, Tengyu Ma, and Adams Wei Yu. Doremi: Optimizing data mixtures speeds up language model pretraining. arXiv preprint arXiv:2305.10429, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Regmix: Data mixture as regression for language model pre-training.arXiv preprint arXiv:2407.01492, 2024

    Qian Liu, Xiaosen Zheng, Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang, and Min Lin. Regmix: Data mixture as regression for language model pre-training.arXiv preprint arXiv:2407.01492, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    Task-adaptive pretrained language models via clustered-importance sampling.arXiv preprint arXiv:2410.03735, 2024

    David Grangier, Simin Fan, Skyler Seto, and Pierre Ablin. Task-adaptive pretrained language models via clustered-importance sampling.arXiv preprint arXiv:2410.03735, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Nemotron-climb: Clustering-based iterative data mixture bootstrapping for language model pre-training.arXiv preprint arXiv:2504.13161, 2025

    Shizhe Diao, Yu Yang, Yonggan Fu, Xin Dong, Dan Su, Markus Kliegl, Zijia Chen, Peter Belcak, Yoshi Suhara, Hongxu Yin, Mostofa Patwary, Yingyan, Lin, Jan Kautz, and Pavlo Molchanov. Nemotron-climb: Clustering-based iterative data mixture bootstrapping for language model pre-training.arXiv preprint arXiv:2504.13161, 2025. URLhttps://arxiv.org/abs/2504.13161

    Pith/arXiv arXiv 2025

  17. [17]

    Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms.arXiv preprint arXiv:2402.14740, 2024

    Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms.arXiv preprint arXiv:2402.14740, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  19. [19]

    Vineppo: Refining credit assignment in rl training of llms.arXiv preprint arXiv:2410.01679, 2024

    Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. Vineppo: Refining credit assignment in rl training of llms.arXiv preprint arXiv:2410.01679, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Policy gradient methods for reinforcement learning with function approximation.Advances in neural information processing systems, 12, 1999

    Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation.Advances in neural information processing systems, 12, 1999

    1999

  21. [21]

    L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Open- reasoner-zero: An open source approach to scaling reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Heung-Yeung Shum, and Xiangyu Zhang. Open- reasoner-zero: An open source approach to scaling reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025. URLhttps://arxiv.org/abs/2503.24290

    Pith/arXiv arXiv 2025

  23. [23]

    Learning to reason at the frontier of learnability.arXiv preprint arXiv:2502.12272, 2025

    Thomas Foster, Anya Sims, Johannes Forkel, Mattie Fellows, and Jakob Foerster. Learning to reason at the frontier of learnability.arXiv preprint arXiv:2502.12272, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Reasoning gym: Reasoning environments for reinforcement learning with verifiable rewards.arXiv preprint arXiv:2505.24760, 2025

    Zafir Stojanovski, Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, and Andreas Köpf. Reasoning gym: Reasoning environments for reinforcement learning with verifiable rewards.arXiv preprint arXiv:2505.24760, 2025. URLhttps://arxiv.org/abs/2505.24760

    arXiv 2025

  25. [25]

    Multi-task learning as a bargaining game

    Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Gal Chechik, and Ethan Fetaya. Multi-task learning as a bargaining game. InInternational Conference on Machine Learning, pages 16109–16128. PMLR, 2022. Multi-Task GRPO 13

    2022

  26. [26]

    Robust empirical optimization is almost the same as mean–variance optimization.Operations research letters, 46(4):448–452, 2018

    Jun-ya Gotoh, Michael Jong Kim, and Andrew EB Lim. Robust empirical optimization is almost the same as mean–variance optimization.Operations research letters, 46(4):448–452, 2018

    2018

  27. [27]

    Non-convex distributionally robust optimization: Non-asymptotic analysis.Advances in Neural Information Processing Systems, 34:2771–2782, 2021

    Jikai Jin, Bohang Zhang, Haiyang Wang, and Liwei Wang. Non-convex distributionally robust optimization: Non-asymptotic analysis.Advances in Neural Information Processing Systems, 34:2771–2782, 2021

    2021

  28. [28]

    Kernel distributionally robust optimization: Generalized duality theorem and stochastic approximation

    Jia-Jie Zhu, Wittawat Jitkrittum, Moritz Diehl, and Bernhard Schölkopf. Kernel distributionally robust optimization: Generalized duality theorem and stochastic approximation. InInternational Conference on Artificial Intelligence and Statistics, pages 280–288. PMLR, 2021

    2021

  29. [29]

    Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Doge: Domain reweighting with generalization estimation

    Simin Fan, Matteo Pagliardini, and Martin Jaggi. Doge: Domain reweighting with generalization estimation. arXiv preprint arXiv:2310.15393, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Grape: Optimize data mixture for group robust multi-target adaptive pretraining.arXiv preprint arXiv:2505.20380, 2025

    Simin Fan, Maria Ios Glarou, and Martin Jaggi. Grape: Optimize data mixture for group robust multi-target adaptive pretraining.arXiv preprint arXiv:2505.20380, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Multi-task learning as multi-objective optimization

    Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. InAdvances in Neural Information Processing Systems, volume 31, 2018

    2018

  33. [33]

    Gradnorm: Gradient normal- ization for adaptive loss balancing in deep multitask networks

    Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normal- ization for adaptive loss balancing in deep multitask networks. InInternational conference on machine learning, pages 794–803. PMLR, 2018

    2018

  34. [34]

    Conflict-averse gradient descent for multi-task learning.Advances in Neural Information Processing Systems, 34:18878–18890, 2021

    Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning.Advances in Neural Information Processing Systems, 34:18878–18890, 2021

    2021

  35. [35]

    Just pick a sign: Optimizing deep multitask models with gradient sign dropout.Advances in Neural Information Processing Systems, 33:2039–2050, 2020

    Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout.Advances in Neural Information Processing Systems, 33:2039–2050, 2020

    2039

  36. [36]

    In defense of the unitary scalarization for deep multi-task learning.Advances in Neural Information Processing Systems, 35:12169–12183, 2022

    Vitaly Kurin, Alessandro De Palma, Ilya Kostrikov, Shimon Whiteson, and Pawan K Mudigonda. In defense of the unitary scalarization for deep multi-task learning.Advances in Neural Information Processing Systems, 35:12169–12183, 2022

    2022

  37. [37]

    Auto-lambda: Disentangling dynamic task relationships.arXiv preprint arXiv:2202.03091, 2022

    Shikun Liu, Stephen James, Andrew J Davison, and Edward Johns. Auto-lambda: Disentangling dynamic task relationships.arXiv preprint arXiv:2202.03091, 2022

    Pith/arXiv arXiv 2022

  38. [38]

    Gradient deconfliction via orthogonal projections onto subspaces for multi-task learning

    Shijie Zhu, Hui Zhao, Tianshu Wu, Pengjie Wang, Hongbo Deng, Jian Xu, and Bo Zheng. Gradient deconfliction via orthogonal projections onto subspaces for multi-task learning. InProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, pages 204–212, 2025

    2025

  39. [39]

    Towards impartial multi-task learning

    Liyang Liu, Yi Li, Zhanghui Kuang, Jing-Hao Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. InInternational conference on learning representations, 2021

    2021

  40. [40]

    Rotograd: Gradient homogenization in multitask learning.arXiv preprint arXiv:2103.02631, 2021

    Adrián Javaloy and Isabel Valera. Rotograd: Gradient homogenization in multitask learning.arXiv preprint arXiv:2103.02631, 2021

    Pith/arXiv arXiv 2021

  41. [41]

    Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models.arXiv preprint arXiv:2010.05874, 2020

    Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models.arXiv preprint arXiv:2010.05874, 2020

    Pith/arXiv arXiv 2010

  42. [42]

    Multi-task learning using uncertainty to weigh losses for scene geometry and semantics

    Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 7482–7491, 2018

    2018

  43. [43]

    Revisiting scalarization in multi-task learning: A theoretical perspective.Advances in Neural Information Processing Systems, 36: 48510–48533, 2023

    Yuzheng Hu, Ruicheng Xian, Qilong Wu, Qiuling Fan, Lang Yin, and Han Zhao. Revisiting scalarization in multi-task learning: A theoretical perspective.Advances in Neural Information Processing Systems, 36: 48510–48533, 2023. Multi-Task GRPO 14

    2023

  44. [44]

    On the convergence of stochastic multi-objective gradient manipulation

    Zheyuan Zhou, Guojun Li, Xuanyuan Zhang, Zhe Chen, and Yang You. On the convergence of stochastic multi-objective gradient manipulation. InInternational Conference on Machine Learning, pages 27192– 27214. PMLR, 2022

    2022

  45. [45]

    Direction-oriented multi-objective learning: Simple and provable stochastic algorithms.Advances in Neural Information Processing Systems, 36:4509–4533, 2023

    Peiyao Xiao, Hao Ban, and Kaiyi Ji. Direction-oriented multi-objective learning: Simple and provable stochastic algorithms.Advances in Neural Information Processing Systems, 36:4509–4533, 2023

    2023

  46. [46]

    Mftcoder: Boosting code llms with multitask fine-tuning

    Bingchang Liu, Chaoyu Chen, Zi Gong, Cong Liao, Huan Wang, Zhichao Lei, Ming Liang, Dajun Chen, Min Shen, Hailian Zhou, et al. Mftcoder: Boosting code llms with multitask fine-tuning. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5430–5441, 2024

    2024

  47. [47]

    Multi-task imitation learning for linear dynamical systems

    Thomas T Zhang, Katie Kang, Bruce D Lee, Claire Tomlin, Sergey Levine, Stephen Tu, and Nikolai Matni. Multi-task imitation learning for linear dynamical systems. InLearning for Dynamics and Control Conference, pages 586–599. PMLR, 2023

    2023

  48. [48]

    Bora: Bayesian hierarchical low-rank adaption for multi-task large language models.arXiv preprint arXiv:2407.15857, 2024

    Simen Eide and Arnoldo Frigessi. Bora: Bayesian hierarchical low-rank adaption for multi-task large language models.arXiv preprint arXiv:2407.15857, 2024

    Pith/arXiv arXiv 2024

  49. [49]

    Coba: convergence balancer for multitask finetuning of large language models.arXiv preprint arXiv:2410.06741, 2024

    Zi Gong, Hang Yu, Cong Liao, Bingchang Liu, Chaoyu Chen, and Jianguo Li. Coba: convergence balancer for multitask finetuning of large language models.arXiv preprint arXiv:2410.06741, 2024

    Pith/arXiv arXiv 2024

  50. [50]

    Conditional language policy: A general framework for steerable multi-objective finetuning

    Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alekh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupta, Kumar Avinava Dubey, et al. Conditional language policy: A general framework for steerable multi-objective finetuning. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 2153–2186, 2024

    2024

  51. [51]

    Mixture-of-loras: An efficient multitask tuning for large language models.arXiv preprint arXiv:2403.03432, 2024

    Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. Mixture-of-loras: An efficient multitask tuning for large language models.arXiv preprint arXiv:2403.03432, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Optimizing multi- task learning for enhanced performance in large language models

    Zhen Qi, Jiajing Chen, Shuo Wang, Bingying Liu, Hongye Zheng, and Chihang Wang. Optimizing multi- task learning for enhanced performance in large language models. In2024 4th International Conference on Electronic Information Engineering and Computer Communication (EIECC), pages 1179–1183. IEEE, 2024

    2024

  53. [53]

    Mixing it up: The cocktail effect of multi-task fine-tuning on llm performance–a case study in finance

    Meni Brief, Oded Ovadia, Gil Shenderovitz, Noga Ben Yoash, Rachel Lemberg, and Eitam Sheetrit. Mixing it up: The cocktail effect of multi-task fine-tuning on llm performance–a case study in finance. arXiv preprint arXiv:2410.01109, 2024

    Pith/arXiv arXiv 2024

  54. [54]

    Aurora: Activating chinese chat capability for mixtral-8x7b sparse mixture-of-experts through instruction-tuning.arXiv preprint arXiv:2312.14557, 2023

    Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Yaofei Duan, Kunyan Cai, Han Ma, Jiaxi Cui, Jian Li, Patrick Cheong-Iao Pang, Yapeng Wang, et al. Aurora: Activating chinese chat capability for mixtral-8x7b sparse mixture-of-experts through instruction-tuning.arXiv preprint arXiv:2312.14557, 2023

    Pith/arXiv arXiv 2023

  55. [55]

    Dynamic data mixing maximizes instruction tuning for mixture-of-experts

    Tong Zhu, Daize Dong, Xiaoye Qu, Jiacheng Ruan, Wenliang Chen, and Yu Cheng. Dynamic data mixing maximizes instruction tuning for mixture-of-experts. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1663–1677, 2025

    2025

  56. [56]

    Deep rein- forcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep rein- forcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    2017

  57. [57]

    Fine-tuning language models from human preferences.arXiv preprint arXiv:1909.08593, 2019

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences.arXiv preprint arXiv:1909.08593, 2019

    Pith/arXiv arXiv 1909

  58. [58]

    Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022

    Pith/arXiv arXiv 2022

  59. [59]

    Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35:27730–27744, 2022. Multi-Task GRPO 15

    2022

  60. [60]

    Star: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  61. [61]

    Tulu 3: Pushing frontiers in open language model post-training.arXiv preprint arXiv:2411.15124, 2024

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training.arXiv preprint arXiv:2411.15124, 2024

    Pith/arXiv arXiv 2024

  62. [62]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118, 2025

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118, 2025

    Pith/arXiv arXiv 2025

  63. [63]

    Minimax-m1: Scaling test-time compute efficiently with lightning attention

    Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, Cheng Zhu, et al. Minimax-m1: Scaling test-time compute efficiently with lightning attention. arXiv preprint arXiv:2506.13585, 2025

    Pith/arXiv arXiv 2025

  64. [64]

    Group sequence policy optimization.arXiv preprint arXiv:2507.18071, 2025

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization.arXiv preprint arXiv:2507.18071, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    Pith/arXiv arXiv 2025

  66. [66]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Pith/arXiv arXiv 2025

  68. [68]

    Multi-reward grpo for stable and prosodic single- codebook tts llms at scale.arXiv preprint arXiv:2511.21270, 2025

    Yicheng Zhong, Peiji Yang, and Zhisheng Wang. Multi-reward grpo for stable and prosodic single- codebook tts llms at scale.arXiv preprint arXiv:2511.21270, 2025

    arXiv 2025

  69. [69]

    Gdpo: Group reward-decoupled normalization policy optimization for multi-reward rl optimization.arXiv preprint arXiv:2601.05242, 2026

    Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Peter Belcak, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, et al. Gdpo: Group reward-decoupled normalization policy optimization for multi-reward rl optimization.arXiv preprint arXiv:2601.05242, 2026

    Pith/arXiv arXiv 2026

  70. [70]

    Learning to optimize multi-objective alignment through dynamic reward weighting.arXiv preprint arXiv:2509.11452, 2025

    Yining Lu, Zilong Wang, Shiyang Li, Xin Liu, Changlong Yu, Qingyu Yin, Zhan Shi, Zixuan Zhang, and Meng Jiang. Learning to optimize multi-objective alignment through dynamic reward weighting.arXiv preprint arXiv:2509.11452, 2025

    arXiv 2025

  71. [71]

    Modomodo: Multi-domain data mixtures for multimodal llm reinforcement learning.arXiv preprint arXiv:2505.24871, 2025

    Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, and Jiacheng Zhu. Modomodo: Multi-domain data mixtures for multimodal llm reinforcement learning.arXiv preprint arXiv:2505.24871, 2025

    Pith/arXiv arXiv 2025

  72. [72]

    Medgrpo: Multi-task reinforcement learning for heterogeneous medical video understanding.arXiv preprint arXiv:2512.06581, 2025

    Yuhao Su, Anwesa Choudhuri, Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Meng Zheng, Yuhan Shen, Arun Innanje, Terrence Chen, Ehsan Elhamifar, et al. Medgrpo: Multi-task reinforcement learning for heterogeneous medical video understanding.arXiv preprint arXiv:2512.06581, 2025

    Pith/arXiv arXiv 2025

  73. [73]

    Tempr1: Improving temporal understanding of mllms via temporal-aware multi-task reinforcement learning.arXiv preprint arXiv:2512.03963, 2025

    Tao Wu, Li Yang, Gen Zhan, Yabin Zhang, Yiting Liao, Junlin Li, Deliang Fu, Li Zhang, and Limin Wang. Tempr1: Improving temporal understanding of mllms via temporal-aware multi-task reinforcement learning.arXiv preprint arXiv:2512.03963, 2025

    Pith/arXiv arXiv 2025

  74. [74]

    Mt 3: Scaling mllm-based text image machine translation via multi-task reinforcement learning.arXiv preprint arXiv:2505.19714, 2025

    Zhaopeng Feng, Yupu Liang, Shaosheng Cao, Jiayuan Su, Jiahan Ren, Zhe Xu, Yao Hu, Wenxuan Huang, Jian Wu, and Zuozhu Liu. Mt 3: Scaling mllm-based text image machine translation via multi-task reinforcement learning.arXiv preprint arXiv:2505.19714, 2025. URL https://arxiv.org/abs/2505. 19714

    Pith/arXiv arXiv 2025

  75. [75]

    Reasoning curriculum: Bootstrapping broad llm reasoning from math.arXiv preprint arXiv:2510.26143, 2025

    Bo Pang, Deqian Kong, Silvio Savarese, Caiming Xiong, and Yingbo Zhou. Reasoning curriculum: Bootstrapping broad llm reasoning from math.arXiv preprint arXiv:2510.26143, 2025

    arXiv 2025

  76. [76]

    Omni-thinker: Scaling multi-task rl in llms with hybrid reward and task scheduling.arXiv preprint arXiv:2507.14783, 2025

    Derek Li, Jiaming Zhou, Leo Maxime Brunswic, Abbas Ghaddar, Qianyi Sun, Liheng Ma, Yu Luo, Dong Li, Mark Coates, Jianye Hao, et al. Omni-thinker: Scaling multi-task rl in llms with hybrid reward and task scheduling.arXiv preprint arXiv:2507.14783, 2025. Multi-Task GRPO 16

    arXiv 2025

  77. [77]

    Position: Llms need a bayesian meta-reasoning framework for more robust and generalizable reasoning

    Hanqi Yan, Linhai Zhang, Jiazheng Li, Zhenyi Shen, and Yulan He. Position: Llms need a bayesian meta-reasoning framework for more robust and generalizable reasoning. In2025 International Conference on Machine Learning: ICML25, 2025

    2025

  78. [78]

    Imbalanced gradients in rl post-training of multi-task llms.arXiv preprint arXiv:2510.19178, 2025

    Runzhe Wu, Ankur Samanta, Ayush Jain, Scott Fujimoto, Jeongyeol Kwon, Ben Kretzu, Youliang Yu, Kaveh Hassani, Boris Vidolov, and Yonathan Efroni. Imbalanced gradients in rl post-training of multi-task llms.arXiv preprint arXiv:2510.19178, 2025

    arXiv 2025

  79. [79]

    Dump: Automated distribution-level curriculum learning for rl-based llm post-training.arXiv preprint arXiv:2504.09710, 2025

    Zhenting Wang, Guofeng Cui, Kun Wan, and Wentian Zhao. Dump: Automated distribution-level curriculum learning for rl-based llm post-training.arXiv preprint arXiv:2504.09710, 2025

    arXiv 2025

  80. [80]

    Learning like humans: Advancing llm reasoning capabilities via adaptive difficulty curriculum learning and expert-guided self-reformulation

    Enci Zhang, Xingang Yan, Wei Lin, Tianxiang Zhang, and Qianchun Lu. Learning like humans: Advancing llm reasoning capabilities via adaptive difficulty curriculum learning and expert-guided self-reformulation. arXiv preprint arXiv:2505.08364, 2025

    arXiv 2025

Showing first 80 references.