Pith. sign in

REVIEW 3 major objections 4 minor 100 references

Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AudioRubrics claims that self-evolving, audio-grounded rubric rewards let a 7B audio-language model beat every comparable baseline on three audio-reasoning benchmarks.

desk verdict The audio-domain adaptation is real and well-executed, but the rubric judge sees ground-truth correctness before scoring, so the central 'process-level supervision' claim is not yet established. read the letter →

arxiv 2608.02831 v1 pith:CUTSL4EW submitted 2026-08-03 cs.SD cs.CL

Fangxu Yu , Tao Feng , Dehai Min , Zinan Lin , Weijia Xu , Michael Xu , Philip S. Yu , Ge Liu
show 1 more author
Tianyi Zhou
This is my paper · ORCID
classification cs.SDcs.CL
keywords audioreasoningreinforcementlearningevolvingrubricsprocessrewardslargeaudio-languagemodelsGRPOaudio-groundedevaluationrewarddesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AudioRubrics claims that the reward signal for training audio-language models can be made far more informative by replacing a single outcome check with a set of audio-grounded rubrics—binary criteria with weights—that are regenerated from the model's own rollouts as training proceeds. The paper argues that outcome-only reinforcement learning lets the model reach correct answers without genuinely listening, while fixed hand-written process criteria saturate once the policy passes them and never adapt to the question at hand. It reports that a 7B model trained this way outperforms all comparable open and training-based baselines on MMAU, MMAR, and MMSU, and that the advantage grows with the capability of the rubric generator and judge. A careful reader would care because it suggests a concrete recipe for keeping RL supervision dense and non-saturating in a sensory domain.

What carries the argument

Rubric-as-reward: a per-sample set of binary criteria $r_k$ with weights $w_k$ summing to 1, where a judge model $\Phi$ that hears the raw audio returns $b_{k,i}\in\{0,1\}$ for whether rollout $i$ satisfies criterion $k$, producing the rubric reward $R^{\mathrm{rub}}_i=\sum_{k\in\mathcal{K}} w_k b_{k,i}$. The distinctive machinery is the evolution step: $\Phi$ sees the group of $G=8$ rollouts, proposes up to $N_{\mathrm{new}}=3$ new positive or negative rubrics that separate strong from weak answers, prunes rubrics with zero variance across the group, keeps the top $M=5$ most discriminative, and reweights them. The final per-rollout reward is $R_i=R^{\mathrm{out}}_i+\gamma R^{\mathrm{rub}}_i+\delta R^{\mathrm{over}}_i$, where $R^{\mathrm{out}}$ is accuracy plus format and $R^{\mathrm{over}}=1-|o_i|/L$ penalizes overthinking; GRPO then optimizes the policy.

What would settle it

Retrain AudioRubrics with the same judge and rubric loop but feed the judge only a text transcript, not the raw waveform; if the reported MMAU/MMAR/MMSU accuracies (78.00/65.80/65.86) do not drop, the audio-grounding story is falsified. Alternatively, on a held-out set of rollouts, compare judge verdicts on the rubrics against human expert verdicts: near-chance agreement would show the reward signal is noise.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that self-evolving, audio-grounded rubric rewards make a 7B audio-language model reach state-of-the-art results among similar-size models: 78.00% on MMAU Test-mini, 65.80% on MMAR, and 65.86% on MMSU average, against 75.20%, 62.20%, and 63.14% for the GRPO baseline trained with outcome rewards alone. The rubrics are initialized per question from the raw waveform, then at each RL step an audio-capable judge both scores the current rollouts and elicits new discriminative criteria, dropping any criterion all rollouts either pass or fail, and re-weighting the survivors. The paper also shows that the reward signal is what drives the gain: ablating evolution, static rubrics, or the length penalty each lowers accuracy, and swapping the judge for a weaker model drops performance below the vanilla GRPO baseline.

Load-bearing premise

The load-bearing premise is that one external judge model (Gemini-3.1-Pro) can be trusted, throughout training, to hear the audio, generate meaningful rubrics, and grade every rollout correctly; the entire process reward flows through this judge without verification.

Editorial extensions

If this is right

  • If the central claim holds, outcome-only RLVR for audio reasoning can be improved by process supervision without hand-designing reward functions per question: the rubrics are generated automatically from the audio and the rollouts.
  • The reported ablation quantifies each component: evolving rubrics add roughly two points over static rubrics on MMAU (76.20 vs 78.00 full), and removing the length penalty costs about one point, so the gains are not an artifact of a single term.
  • The judge-capability dependence means practitioners adopting the method should expect gains only when the judge/generator is strong; with a weaker judge the method can underperform plain GRPO.
  • The convergence to a stable reasoning length—contrasting with GRPO's collapse and the unbounded growth without the penalty—suggests the reward design also controls inference cost at deployment.
  • The improvement transfers to the perception-oriented MMSU split (66.14 vs 60.16 for CESAR, the best comparable baseline), which the paper reads as evidence that the model is actually listening rather than pattern-matching text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests the same evolving-rubric loop could be dropped onto other sensory domains—video, or audio-plus-video—wherever a strong multimodal judge can ground criteria in the raw input; the audio-specific taxonomy is a design choice, not a prerequisite.
  • A natural extension the paper does not test is ensembling several judges or training a small reward model on judge verdicts, which would reduce the single-judge failure mode visible in its Figure 3(b).
  • The unexplained constraint in the static prompt that the reference answer must satisfy less than half the rubric weight probably biases early rubrics upward in difficulty; varying that threshold could change early-training dynamics and is worth a targeted ablation.
  • Because the variance filter keeps only rubrics that discriminate among just 8 rollouts, some kept criteria may separate by chance early in training; increasing the group size or applying a statistical significance threshold could sharpen the reward signal.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes AudioRubrics, a reinforcement-learning framework for audio reasoning in which a frozen audio-capable model (Gemini-3.1-Pro) first generates per-question rubrics from the raw waveform and then, at each GRPO step, proposes new rubrics from the current policy rollouts, judges all rubrics on those rollouts, discards nondiscriminative criteria, and reweights the survivors. The resulting rubric score is combined with accuracy/format rewards and a length penalty. Experiments on MMAU, MMAR, and MMSU with a Qwen2.5-Omni-7B base report accuracy above all similar-size baselines, with ablations attributing the gains to RL training, static rubrics, rubric evolution, and the overthinking penalty, plus an analysis of response-length stability and rubric evolution over training.

Significance. If the effects are real and the mechanism is as claimed, the paper offers a practical way to supply dense, adaptive process supervision for audio reasoning without hand-designing criteria, and the qualitative examples of rubric co-evolution are instructive. The paper's strengths include evaluation across three benchmarks with a wide baseline set, transparent appendix prompts, an ablation table, sensitivity tables for reward weights, and a training-length analysis showing that the full method avoids both collapse and explosion. However, the central attribution of the gains to audio-grounded process supervision is not yet supported by the evidence presented.

major comments (3)
  1. [§3.2, Figure 11, Algorithm 1] The judge call that elicits new rubrics and returns binary judgments also receives the ground-truth answer, each rollout's extracted final answer with a CORRECT/WRONG label, and a per-rollout correctness summary. Because the variance filter then keeps rubrics whose verdicts best separate the rollouts, the retained rubric reward can act as a soft proxy for final-answer correctness rather than as an independent audio-grounded process signal. Table 6 only ablates the presence of the rubric reward, not the judge's access to outcome labels; a blinded control in which the judge scores the same rubrics without any correctness information, or in a separate label-free call, is needed to support the paper's central claim that the gains come from evolving, audio-grounded process supervision. As written, the 'audio-grounded process reward' interpretation is confounded with outcome leakage.
  2. [§4.3, Tables 4–5, Table 7] The final hyperparameters γ=0.5 and δ=0.15 are selected from sensitivity sweeps run on the same three evaluation benchmarks used for the headline results, and Table 6 reports single-run accuracies without error bars or multiple seeds. Since several reported gains are small (e.g., 65.86 vs 65.44 on MMSU in the ablation), the ranking could be affected by seed noise, and tuning on the test benchmarks can inflate absolute numbers. Please report selection on a held-out split and means/standard deviations over at least three seeds, or otherwise justify that the reported differences exceed run-to-run variance.
  3. [Figure 9 and §3.2] The static rubric generation prompt imposes an unexplained constraint that the reference answer's satisfied rubric weight be less than 0.5. This is not described in the method section, and since the reference answer is the ground-truth response y* used in Eq. (5), the constraint may bias the initial rubrics toward criteria that the correct answer does not satisfy. Please state the motivation for this constraint, report its effect, and test sensitivity to the threshold.
minor comments (4)
  1. [§2.1, Eq. (1)] As printed, Eq. (1) defines R(o) as a sum over all G rollouts of importance-weighted rewards, then subtracts the group mean and divides by the group standard deviation; this is not the GRPO objective, and the per-response reward and advantage notation is inconsistent. Please rewrite so that r(o_i) is the per-rollout reward and A_i = (r(o_i) - mean)/std.
  2. [Figure 3(a)] The horizontal axis is labeled only 'values' with tick marks 0, 0.1, 0.3, 0.5, 0.7, 0.9; it should be labeled γ, and the caption should state whether the y-axis is averaged over the three benchmarks or refers to a selected one.
  3. [§4.5] The 'ratio of newly evolved rubrics adopted' is not defined; please state whether it is the fraction of kept rubrics coming from R_new after the variance filter and the top-M selection.
  4. [Table 2] The column headers for MMSU and MMAU appear to be misaligned, with the 'Seman./Phono./Para./Avg' group repeated without a clear separator between the two benchmarks; please reformat the table.

Circularity Check

1 steps flagged · score 4.0 of 10

Process reward is generated with the ground-truth answer and per-rollout correctness in the judge prompt, so the claimed outcome-independent process supervision is partially confounded by construction.

  1. other [Section 3.2, Eq. (7); Appendix D, Figure 11]
    "The final rubric reward is: Rrub_i = Σ_{k∈K} w_k b_{k,i} (7). ... ## Ground-truth final answer {gt_letter} ... --- T1 [final answer: {ans} -> CORRECT / WRONG (chose X)] --- ... ## Per-rollout correctness summary T1: CORRECT ... TN: WRONG (chose B)"

    Eq. (7) defines R_rub from the binary judgments b_{k,i}, which are returned by the judge call in Figure 11. That call is explicitly given the ground-truth answer, per-rollout correctness labels, and a correctness summary. Therefore R_rub is a function of final-answer correctness by construction, even though the paper presents it as process-level supervision 'beyond merely checking the final answer'. The variance filter then keeps rubrics whose verdicts vary most across rollouts; because the judge already knows which rollouts are correct, the retained rubrics are likely to track correctness, allowing R_rub to act as a soft correctness classifier. The prompt's statement that correctness is 'context only' is a request, not a blinding control.

full rationale

The main claim is that AudioRubrics improves audio reasoning by supplying evolving, audio-grounded process rewards that go beyond final-answer correctness. The load-bearing step is Eq. (7), where R_rub is built from judge verdicts b_{k,i}; those verdicts are produced by the prompt in Fig. 11, which provides the ground-truth answer, per-rollout correctness labels, and a correctness summary. Thus R_rub is a function of outcome correctness by construction. Because the variance filter keeps rubrics whose verdicts separate rollouts, and the judge already knows which rollouts are correct, the retained rubrics can simply track correctness, making the process reward a soft outcome classifier. The prompt's 'correctness is context only' instruction is a request, not a blinding control, so the central mechanism claim is unproven. This is a genuine partial circularity/confound, but not a full definitional equivalence: the benchmarks are external, the RL objective is standard GRPO, and the ablations (static vs evolving rubrics, length penalty) are informative. No load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling-by-citation were found. The static-rubric generation constraint (reference_score < 0.5) is arbitrary but not circular. Overall score 4.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or ontological entities. Its load-bearing burden is carried by free hyperparameters and by the assumed reliability of an external judge model, both listed above.

free parameters (8)
  • gamma (rubric reward weight) = 0.5
    Tuned via sensitivity analysis on the evaluation benchmarks (Table 4); peak at 0.5.
  • delta (overthinking penalty weight) = 0.15
    Tuned via sensitivity analysis on the evaluation benchmarks (Table 5); peak at 0.15.
  • L (target reasoning length) = 256
    Hand-set reference length that normalizes the overthinking penalty; no sensitivity analysis shown.
  • alpha (accuracy weight) = 0.9
    Hand-set weight for the accuracy reward.
  • beta (format weight) = 0.1
    Hand-set weight for the format reward.
  • M (top discriminative rubrics kept) = 5
    Keeps at most five discriminative rubrics per group; hand-set.
  • N_new (new rubrics per group) = 3
    Elicits up to three new rubrics per step; hand-set.
  • reference_score constraint = must be < 0.5
    Prompt-level constraint in Figure 9 forcing the ground-truth reference answer to satisfy less than half the rubric weight; ad hoc and unexplained.
assumptions (4)
  • domain assumption The fixed five-category rubric taxonomy (auditory evidence grounding, cross-cue verification, reasoning clarity and flow, reasoning focus and efficiency, domain-specific audio techniques) is a sufficient basis for evaluating audio reasoning quality.
    The taxonomy is fixed in advance (Section 3.2, Eq. 5) and used to generate all initial rubrics; no evidence is given that five categories cover all relevant aspects.
  • domain assumption Gemini-3.1-Pro can accurately listen to raw audio and generate valid, non-redundant, audio-grounded rubrics and binary judgments.
    The entire reward depends on this external judge; Figure 3 shows that with a weaker judge (GPT-audio-1.5) the method falls below the GRPO baseline, so judge capability is critical.
  • domain assumption The AVQA dataset, with 'video' replaced by 'audio' in questions, is a valid training distribution for audio reasoning.
    Section 4.1 states training data is constructed this way, following R1-AQA; the transformation may introduce cues that differ from native audio reasoning tasks.
  • domain assumption Ground-truth answers in MMAU, MMAR, and MMSU are correct and unambiguous.
    The accuracy reward is computed as exact match against y*; any annotation errors would propagate into the reward.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning." pith.science (2026). https://pith.science/paper/CUTSL4EW

@misc{pith2026260802831,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CUTSL4EW}},
  note         = {Machine review of arXiv:2608.02831}
}
read the original abstract

Audio reasoning is essential for machine understanding of the acoustic world. Reinforcement learning with verifiable rewards can elicit such reasoning, yet existing reward designs are complementary in their limitations: outcome-based rewards supervise only the final answer and let the model reach it without attending to the audio, whereas process-based rewards score the reasoning itself but rely on coarse, hand-crafted, and fixed criteria that neither adapt to each question nor stay grounded in the acoustic evidence. Moreover, questions differ in what they demand, with some hinging on perception and others on multi-step reasoning, and any static criterion weakens as the policy improves. Supervising the reasoning process with fine-grained, audio-grounded, and adaptive rewards is therefore crucial, yet challenging since such rewards are impractical to design by hand for every sample. To this end, we introduce AudioRubrics, a reinforcement learning framework that supervises audio reasoning with self-evolving, audio-grounded rubric rewards. AudioRubrics synthesizes per-sample rubrics from the raw waveform and, conditioned on the model's own rollouts, regenerates and reweights criteria per group, supplying a continuous learning signal that keeps targeting the current policy's weaknesses as static criteria saturate. Comprehensive evaluations across three audio reasoning benchmarks reveal that AudioRubrics substantially outperforms a wide range of open-source and training-based baselines. Furthermore, our analysis shows that the gains scale with the capability of the rubric generator and judge, and AudioRubrics converges to a stable reasoning length that avoids both degenerate collapse and unbounded growth. The improvement in audio perception further demonstrates the effectiveness of anchoring supervision in the acoustic evidence. Our project page is available at https://audiorubrics.github.io.

Figures

Figures reproduced from arXiv: 2608.02831 by the authors.

Figure 1
Figure 1. Motivation of the proposed AUDIORUBRICS. The goal is to progressively enhance the reasoning ability of LALMs by rewarding not only the correct outcome but also the reasoning process with evolving rubrics, which continually distill new criteria from the model’s own rollouts to push the reasoning from superficial guesses toward deep, faithful, and evidence-grounded explanations. 2 PRELIMINARY 2.1 GROUP RELATIVE POLICY… view at source ↗
Figure 2
Figure 2. Overview of AUDIORUBRICS. A rubric generator initializes audio-grounded rubrics for each question. At each step, the policy LALM samples rollouts, and the generator elicits refined rubrics from the rollouts and judges every rubric on them; non-discriminative rubrics are discarded. The resulting rubric score is combined with the outcome reward for GRPO optimization. 3 AUDIORUBRICS: RL WITH EVOLVING RUBRICS AS REWARDS… view at source ↗
Figure 3
Figure 3. (a) Average performance across all benchmarks with varying rubric reward weight [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: The ratio of newly evolved rubrics adopted during training. To understand how the rubric set adapts during training, we track the ratio of newly evolved rubrics that are adopted for the reward at each step. As shown in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Qualitative case study. The GRPO baseline latches onto the surface topic (Superman) and [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 7
Figure 7. Figure 7: Average response length dur￾ing training under three reward configu￾rations. Vanilla GRPO collapses toward near-zero length, removing the length penalty leads to unbounded growth, and our full method maintains a stable length throughout [PITH_FULL_IMAGE:figures/full_f…
Figure 6
Figure 6. Figure 6: Qualitative case study of rubric co-evolution. Two static rubrics are retained for the easy [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 8
Figure 8. Figure 8: Inference prompt used for all benchmark evaluation (greedy decoding, temperature [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Prompt for generating the initial (static) per-question rubrics. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Full system prompt that drives per-step rubric evolution (Call 1). [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Call 1: jointly generate new candidate rubrics and judge all rubrics against the 8 rollouts. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Call 2: assign normalized importance weights to the top- [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Prompt for judging whether a response satisfies each rubric. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

100 extracted references · 13 canonical work pages

  1. [1]

    2023 , url =

    Benjamin Elizalde and Soham Deshmukh and Mahmoud Al Ismail and Huaming Wang , title =. 2023 , url =. doi:10.1109/ICASSP49357.2023.10095889 , timestamp =

    arXiv 2023

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Checklists are better than reward models for aligning language models , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    arXiv preprint arXiv:2410.13720 , year=

    Movie gen: A cast of media foundation models , author=. arXiv preprint arXiv:2410.13720 , year=

    arXiv

  4. [4]

    arXiv preprint arXiv:2605.09272 , year=

    Towards Conversational Medical AI with Eyes, Ears and a Voice , author=. arXiv preprint arXiv:2605.09272 , year=

    arXiv

  5. [5]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Slam-omni: Timbre-controllable voice interaction system with single-stage training , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  6. [6]

    International Conference on Learning Representations , volume=

    Listen, think, and understand , author=. International Conference on Learning Representations , volume=

  7. [7]

    arXiv preprint arXiv:2605.12474 , year=

    Reward hacking in rubric-based reinforcement learning , author=. arXiv preprint arXiv:2605.12474 , year=

    arXiv

  8. [8]

    arXiv preprint arXiv:2602.05125 , year=

    Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks , author=. arXiv preprint arXiv:2602.05125 , year=

    arXiv

Show all 100 references
  1. [9]

    arXiv preprint arXiv:2605.20164 , year=

    Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR , author=. arXiv preprint arXiv:2605.20164 , year=

    arXiv

  2. [10]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    arXiv

  3. [11]

    arXiv preprint arXiv:2607.26246 , year=

    Weak-to-Strong On-Policy Distillation , author=. arXiv preprint arXiv:2607.26246 , year=

    arXiv

  4. [12]

    arXiv preprint arXiv:2602.21628 , year=

    Rucl: Stratified rubric-based curriculum learning for multimodal large language model reasoning , author=. arXiv preprint arXiv:2602.21628 , year=

  5. [13]

    arXiv preprint arXiv:2602.00846 , year=

    Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis , author=. arXiv preprint arXiv:2602.00846 , year=

    arXiv

  6. [14]

    arXiv preprint arXiv:2603.16600 , year=

    Rationale matters: Learning transferable rubrics via proxy-guided critique for VLM reward models , author=. arXiv preprint arXiv:2603.16600 , year=

  7. [15]

    arXiv preprint arXiv:2604.13029 , year=

    Visual preference optimization with rubric rewards , author=. arXiv preprint arXiv:2604.13029 , year=

    arXiv

  8. [16]

    arXiv preprint arXiv:2510.14738 , year=

    AutoRubric-R1V: Rubric-Based Generative Rewards for Faithful Multimodal Reasoning , author=. arXiv preprint arXiv:2510.14738 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2602.04649 , year=

    Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models , author=. arXiv preprint arXiv:2602.04649 , year=

  10. [18]

    arXiv preprint arXiv:2602.01511 , year=

    Alternating reinforcement learning for rubric-based reward modeling in non-verifiable llm post-training , author=. arXiv preprint arXiv:2602.01511 , year=

  11. [19]

    arXiv preprint arXiv:2510.07284 , year=

    Online rubrics elicitation from pairwise comparisons , author=. arXiv preprint arXiv:2510.07284 , year=

  12. [20]

    arXiv preprint arXiv:2602.10885 , year=

    Reinforcing chain-of-thought reasoning with self-evolving rubrics , author=. arXiv preprint arXiv:2602.10885 , year=

  13. [21]

    arXiv preprint arXiv:2511.19399 , year=

    Dr tulu: Reinforcement learning with evolving rubrics for deep research , author=. arXiv preprint arXiv:2511.19399 , year=

    arXiv

  14. [22]

    arXiv preprint arXiv:2508.16949 , year=

    Breaking the exploration bottleneck: Rubric-scaffolded reinforcement learning for general llm reasoning , author=. arXiv preprint arXiv:2508.16949 , year=

    arXiv

  15. [23]

    arXiv e-prints , pages=

    Auto-rubric: Learning to extract generalizable criteria for reward modeling , author=. arXiv e-prints , pages=

  16. [24]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    CARMO: Dynamic Criteria Generation for Context Aware Reward Modelling , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  17. [25]

    arXiv preprint arXiv:2510.07743 , year=

    Openrubrics: Towards scalable synthetic rubric generation for reward modeling and llm alignment , author=. arXiv preprint arXiv:2510.07743 , year=

  18. [26]

    International Conference on Learning Representations , volume=

    Prometheus: Inducing fine-grained evaluation capability in language models , author=. International Conference on Learning Representations , volume=

  19. [27]

    Advances in Neural Information Processing Systems , volume=

    Rule based rewards for language model safety , author=. Advances in Neural Information Processing Systems , volume=

  20. [28]

    arXiv preprint arXiv:2512.20061 , year=

    Scaling Reinforcement Learning for Content Moderation with Large Language Models , author=. arXiv preprint arXiv:2512.20061 , year=

  21. [29]

    2025 , note =

    Gemini 3.1 Pro , howpublished =. 2025 , note =

    2025

  22. [30]

    arXiv preprint arXiv:2407.10759 , year=

    Qwen2-audio technical report , author=. arXiv preprint arXiv:2407.10759 , year=

    arXiv

  23. [31]

    arXiv preprint arXiv:2512.23808 , year=

    MiMo-Audio: Audio Language Models are Few-Shot Learners , author=. arXiv preprint arXiv:2512.23808 , year=

  24. [32]

    GitHub Repository , howpublished =

    Zhao, Shuaijiang and Guo, Tingwei and Wen, Cheng and Xiang, Bajian and Zou, Wei , title =. GitHub Repository , howpublished =. 2025 , publisher =

    2025

  25. [33]

    Advances in Neural Information Processing Systems , volume=

    Audio flamingo 3: Advancing audio intelligence with fully open large audio language models , author=. Advances in Neural Information Processing Systems , volume=

  26. [34]

    arXiv preprint arXiv:2412.08905 , year=

    Phi-4 technical report , author=. arXiv preprint arXiv:2412.08905 , year=

    arXiv

  27. [35]

    arXiv preprint arXiv:2507.16632 , year=

    Step-audio 2 technical report , author=. arXiv preprint arXiv:2507.16632 , year=

    arXiv

  28. [36]

    2025 , eprint=

    Qwen2.5-Omni Technical Report , author=. 2025 , eprint=

    2025

  29. [37]

    arXiv preprint arXiv:2504.18425 , year=

    Kimi-audio technical report , author=. arXiv preprint arXiv:2504.18425 , year=

    arXiv

  30. [38]

    KimiTeam and Ding Ding and Zeqian Ju and Yichong Leng and Songxiang Liu and Tong Liu and Zeyu Shang and Kai Shen and Wei Song and Xu Tan and Heyi Tang and Zhengtao Wang and Chu Wei and Yifei Xin and Xinran Xu and Jianwei Yu and Yutao Zhang and Xinyu Zhou and Y. Charles and Jun...

    2025 doi arXiv

  31. [39]

    2023 , url =

    Bharathan Balaji and Venkata Sai Gargeya Vunnava and Nina Domingo and Shikhar Gupta and Harsh Gupta and Geoffrey Guest and Aravind Srinivasan , title =. 2023 , url =. doi:10.1145/3616385 , timestamp =

    2023 doi

  32. [40]

    Sakshi and Jaehyeon Kim and Wei Ping and Rafael Valle and Dinesh Manocha and Bryan Catanzaro , title =

    Sreyan Ghosh and Zhifeng Kong and Sonal Kumar and S. Sakshi and Jaehyeon Kim and Wei Ping and Rafael Valle and Dinesh Manocha and Bryan Catanzaro , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2503.03983 , eprinttype =. 2503.03983 , timestamp =

    2025 doi arXiv

  33. [41]

    Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models , journal =

    Arushi Goel and Sreyan Ghosh and Jaehyeon Kim and Sonal Kumar and Zhifeng Kong and Sang. Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models , journal =. 2025 , url =. doi:10.48550/ARXIV.2507.08128 , eprinttype =. 2507.08128 , timestamp =

    2025 doi arXiv

  34. [42]

    Proceedings of the 30th ACM International Conference on Multimedia , pages=

    AVQA: A Dataset for Audio-Visual Question Answering on Videos , author=. Proceedings of the 30th ACM International Conference on Multimedia , pages=

  35. [43]

    Goucher and Adam Perelman and Aditya Ramesh and Aidan Clark and AJ Ostrow and Akila Welihinda and Alan Hayes and Alec Radford and Aleksander Madry and Alex Baker

    Aaron Hurst and Adam Lerer and Adam P. Goucher and Adam Perelman and Aditya Ramesh and Aidan Clark and AJ Ostrow and Akila Welihinda and Alan Hayes and Alec Radford and Aleksander Madry and Alex Baker. GPT-4o System Card , journal =. 2024 , url =. doi:10.48550/ARXIV.2410.21276...

    2024 doi arXiv

  36. [44]

    CoRR , volume =

    OpenAI , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2303.08774 , eprinttype =. 2303.08774 , timestamp =

    2023 doi arXiv

  37. [45]

    Wav2CLIP: Learning Robust Audio Representations from Clip , booktitle =

    Ho. Wav2CLIP: Learning Robust Audio Representations from Clip , booktitle =. 2022 , url =. doi:10.1109/ICASSP43922.2022.9747669 , timestamp =

    2022

  38. [46]

    Pengi: An Audio Language Model for Audio Tasks , booktitle =

    Soham Deshmukh and Benjamin Elizalde and Rita Singh and Huaming Wang , editor =. Pengi: An Audio Language Model for Audio Tasks , booktitle =. 2023 , url =

    2023

  39. [47]

    Liu and Leonid Karlinsky and James R

    Yuan Gong and Hongyin Luo and Alexander H. Liu and Leonid Karlinsky and James R. Glass , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  40. [48]

    Optimal Transport for Treatment Effect Estimation , booktitle =

    Hao Wang and Jiajun Fan and Zhichao Chen and Haoxuan Li and Weiming Liu and Tianqiao Liu and Quanyu Dai and Yichao Wang and Zhenhua Dong and Ruiming Tang , editor =. Optimal Transport for Treatment Effect Estimation , booktitle =. 2023 , url =

    2023

  41. [49]

    CoRR , volume =

    Jiajun Fan , title =. CoRR , volume =. 2021 , url =. 2112.04145 , timestamp =

    2021 arXiv

  42. [50]

    Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection , booktitle =

    Jiajun Fan and Yuzheng Zhuang and Yuecheng Liu and Jianye Hao and Bin Wang and Jiangcheng Zhu and Hao Wang and Shu. Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection , booktitle =. 2023 , url =

    2023

  43. [51]

    Generalized Data Distribution Iteration , booktitle =

    Jiajun Fan and Changnan Xiao , editor =. Generalized Data Distribution Iteration , booktitle =. 2022 , url =

    2022

  44. [52]

    CoRR , volume =

    Jiajun Fan and Changnan Xiao and Yue Huang , title =. CoRR , volume =. 2021 , url =. 2106.06232 , timestamp =

    2021 arXiv

  45. [53]

    CoRR , volume =

    Ye Li and Yuan Meng and Zewen Sun and Kangye Ji and Chen Tang and Jiajun Fan and Xinzhu Ma and Shutao Xia and Zhi Wang and Wenwu Zhu , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2506.12723 , eprinttype =. 2506.12723 , timestamp =

    2025 doi

  46. [54]

    CoRR , volume =

    Hao Wang and Jianxun Lian and Mingqi Wu and Haoxuan Li and Jiajun Fan and Wanyue Xu and Chaozhuo Li and Xing Xie , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2308.02925 , eprinttype =. 2308.02925 , timestamp =

    2023 doi arXiv

  47. [55]

    CoRR , volume =

    Changnan Xiao and Haosen Shi and Jiajun Fan and Shihong Deng , title =. CoRR , volume =. 2021 , url =. 2106.00707 , timestamp =

    2021 arXiv

  48. [56]

    CoRR , volume =

    Hao Wang and Zhichao Chen and Jiajun Fan and Yuxin Huang and Weiming Liu and Xinggao Liu , title =. CoRR , volume =. 2022 , url =. doi:10.48550/ARXIV.2210.11039 , eprinttype =. 2210.11039 , timestamp =

    2022 doi

  49. [57]

    CoRR , volume =

    Jiajun Fan and He Ba and Xian Guo and Jianye Hao , title =. CoRR , volume =. 2020 , url =. 2011.06752 , timestamp =

    2020 arXiv

  50. [58]

    The Thirteenth International Conference on Learning Representations,

    Jiajun Fan and Shuaike Shen and Chaoran Cheng and Yuxin Chen and Chumeng Liang and Ge Liu , title =. The Thirteenth International Conference on Learning Representations,. 2025 , url =

    2025

  51. [59]

    CoRR , volume =

    Ziwen Wang and Jiajun Fan and Ruihan Guo and Thao Nguyen and Heng Ji and Ge Liu , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2506.07459 , eprinttype =. 2506.07459 , timestamp =

    2025 doi

  52. [60]

    CoRR , volume =

    Ziwen Wang and Jiajun Fan and Thao Nguyen and Heng Ji and Ge Liu , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2506.07413 , eprinttype =. 2506.07413 , timestamp =

    2025 doi

  53. [61]

    CoRR , volume =

    John Schulman and Filip Wolski and Prafulla Dhariwal and Alec Radford and Oleg Klimov , title =. CoRR , volume =. 2017 , url =. 1707.06347 , timestamp =

    2017 arXiv

  54. [62]

    CoRR , volume =

    Changnan Xiao and Haosen Shi and Jiajun Fan and Shihong Deng , title =. CoRR , volume =. 2021 , url =. 2105.03923 , timestamp =

    2021 arXiv

  55. [63]

    CoRR , volume =

    Ye Li and Chen Tang and Yuan Meng and Jiajun Fan and Zenghao Chai and Xinzhu Ma and Zhi Wang and Wenwu Zhu , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2407.05010 , eprinttype =. 2407.05010 , timestamp =

    2024 doi arXiv

  56. [64]

    The Twelfth International Conference on Learning Representations,

    Changli Tang and Wenyi Yu and Guangzhi Sun and Xianzhao Chen and Tian Tan and Wei Li and Lu Lu and Zejun Ma and Chao Zhang , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  57. [65]

    Forty-first International Conference on Machine Learning,

    Zhifeng Kong and Arushi Goel and Rohan Badlani and Wei Ping and Rafael Valle and Bryan Catanzaro , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  58. [66]

    CoRR , volume =

    Yunfei Chu and Jin Xu and Xiaohuan Zhou and Qian Yang and Shiliang Zhang and Zhijie Yan and Chang Zhou and Jingren Zhou , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2311.07919 , eprinttype =. 2311.07919 , timestamp =

    2023 doi arXiv

  59. [67]

    CoRR , volume =

    Yunfei Chu and Jin Xu and Qian Yang and Haojie Wei and Xipin Wei and Zhifang Guo and Yichong Leng and Yuanjun Lv and Jinzheng He and Junyang Lin and Chang Zhou and Jingren Zhou , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2407.10759 , eprinttype =. 2407.10759 ,...

    2024 doi arXiv

  60. [68]

    CoRR , volume =

    Jin Xu and Zhifang Guo and Jinzheng He and Hangrui Hu and Ting He and Shuai Bai and Keqin Chen and Jialin Wang and Yang Fan and Kai Dang and Bin Zhang and Xiong Wang and Yunfei Chu and Junyang Lin , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2503.20215 , eprint...

    2025 doi arXiv

  61. [69]

    2025 , eprint=

    Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models , author=. 2025 , eprint=

    2025

  62. [70]

    Chi and Quoc V

    Jason Wei and Xuezhi Wang and Dale Schuurmans and Maarten Bosma and Brian Ichter and Fei Xia and Ed H. Chi and Quoc V. Le and Denny Zhou , editor =. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , booktitle =. 2022 , url =

    2022

  63. [71]

    CoRR , volume =

    Yan Wang and Yawen Zeng and Jingsheng Zheng and Xiaofen Xing and Jin Xu and Xiangmin Xu , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2407.05355 , eprinttype =. 2407.05355 , timestamp =

    2024 doi arXiv

  64. [72]

    CoRR , volume =

    Ziyang Ma and Zhuo Chen and Yuping Wang and Eng Siong Chng and Xie Chen , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2501.07246 , eprinttype =. 2501.07246 , timestamp =

    2025 doi arXiv

  65. [74]

    arXiv preprint arXiv:2505.09439 , year=

    Omni-r1: Do you really need audio to fine-tune your audio llm? , author=. arXiv preprint arXiv:2505.09439 , year=

  66. [75]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Audio-reasoner: Improving reasoning capability in large audio language models , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  67. [76]

    CoRR , volume =

    Zhifei Xie and Mingbao Lin and Zihang Liu and Pengcheng Wu and Shuicheng Yan and Chunyan Miao , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2503.02318 , eprinttype =. 2503.02318 , timestamp =

    2025 doi

  68. [77]

    OpenAI o1 System Card , journal =

    Aaron Jaech and Adam Kalai and Adam Lerer and Adam Richardson and Ahmed El. OpenAI o1 System Card , journal =. 2024 , url =. doi:10.48550/ARXIV.2412.16720 , eprinttype =. 2412.16720 , timestamp =

    2024 doi arXiv

  69. [78]

    2025 , eprint=

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities , author=. 2025 , eprint=

    2025

  70. [79]

    arXiv preprint arXiv:2406.05673 , year=

    Flow of reasoning: Training llms for divergent reasoning with minimal examples , author=. arXiv preprint arXiv:2406.05673 , year=

    arXiv

  71. [80]

    arXiv preprint arXiv:2601.06559 , year=

    ArrowGEV: Grounding Events in Video via Learning the Arrow of Time , author=. arXiv preprint arXiv:2601.06559 , year=

    arXiv

  72. [81]

    arXiv preprint arXiv:2601.04171 , year=

    Agentic Rubrics as Contextual Verifiers for SWE Agents , author=. arXiv preprint arXiv:2601.04171 , year=

  73. [82]

    Advances in Neural Information Processing Systems , volume=

    Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=

  74. [83]

    arXiv preprint arXiv:2511.12344 , year=

    Reward and Guidance through Rubrics: Promoting Exploration to Improve Multi-Domain Reasoning , author=. arXiv preprint arXiv:2511.12344 , year=

  75. [84]

    arXiv preprint arXiv:2507.17746 , year=

    Rubrics as rewards: Reinforcement learning beyond verifiable domains , author=. arXiv preprint arXiv:2507.17746 , year=

    arXiv

  76. [85]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , journal =

    DeepSeek. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , journal =. 2025 , url =. doi:10.48550/ARXIV.2501.12948 , eprinttype =. 2501.12948 , timestamp =

    2025 doi arXiv

  77. [86]

    Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2402.03300 , eprinttype =. 2402.03300 , timestamp =

    2024 doi arXiv

  78. [87]

    arXiv preprint arXiv:2503.11197 , year=

    Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering , author=. arXiv preprint arXiv:2503.11197 , year=

    arXiv

  79. [88]

    arXiv preprint arXiv:2410.19168 , year=

    Mmau: A massive multi-task audio understanding and reasoning benchmark , author=. arXiv preprint arXiv:2410.19168 , year=

    arXiv

  80. [89]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Audio-Thinker: Guiding Large Audio Language Model When and How to Think via Reinforcement Learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  81. [90]

    arXiv preprint arXiv:2604.12527 , year=

    Audio-Cogito: Towards Deep Audio Reasoning in Large Audio Language Models , author=. arXiv preprint arXiv:2604.12527 , year=

    arXiv

  82. [91]

    arXiv preprint arXiv:2510.11454 , year=

    Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning , author=. arXiv preprint arXiv:2510.11454 , year=

  83. [92]

    arXiv preprint arXiv:2602.13685 , year=

    AuTAgent: A Reinforcement Learning Framework for Tool-Augmented Audio Reasoning , author=. arXiv preprint arXiv:2602.13685 , year=

  84. [93]

    arXiv preprint arXiv:2602.10439 , year=

    AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning , author=. arXiv preprint arXiv:2602.10439 , year=

  85. [94]

    arXiv preprint arXiv:2511.15848 , year=

    Step-Audio-R1 Technical Report , author=. arXiv preprint arXiv:2511.15848 , year=

  86. [95]

    arXiv preprint arXiv:2510.20867 , year=

    Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards , author=. arXiv preprint arXiv:2510.20867 , year=

  87. [96]

    ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Wav2clip: Learning robust audio representations from clip , author=. ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2022 , organization=

    2022

  88. [97]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  89. [98]

    ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Clap learning audio concepts from natural language supervision , author=. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2023 , organization=

    2023

  90. [99]

    arXiv preprint arXiv:2503.02318 , year=

    Audio-reasoner: Improving reasoning capability in large audio language models , author=. arXiv preprint arXiv:2503.02318 , year=

  91. [100]

    arXiv preprint arXiv:2506.04779 , year=

    Mmsu: A massive multi-task spoken language understanding and reasoning benchmark , author=. arXiv preprint arXiv:2506.04779 , year=

    arXiv

  92. [101]

    arXiv preprint arXiv:2505.13032 , year=

    Mmar: A challenging benchmark for deep reasoning in speech, audio, music, and their mix , author=. arXiv preprint arXiv:2505.13032 , year=

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.