Pith. sign in

REVIEW 3 major objections 6 minor 51 references

CoRT claims that tokenwise log-likelihood contrasts from counterfactual replay—rescoring the same sampled response after removing rubric criteria—serve as a proxy for rubric dependence and can redistribute the signed GRPO advantage across t

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 01:47 UTC pith:MXGWO3DY

load-bearing objection CoRT is a cheap, clean token-level credit-weighting trick for GRPO, but its own controls undercut the rubric-specific rationale and the headline gain rests on best-checkpoint selection without seeds. the 3 major comments →

arxiv 2607.25659 v1 pith:MXGWO3DY submitted 2026-07-28 cs.AI

CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization

Bo-Wen Zhang , Junwei He , Wen Wang , Song-Lin Lv , Wentao Ma , Rongyi Lin , Shuhan Zhong , Lan-Zhe Guo This is my paper
classification cs.AI
keywords token-level credit assignmentcounterfactual replayGRPOrubric-based reinforcement learninginstruction followingpolicy optimizationlog-likelihood contrastreward redistribution
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether token-level credit can be derived from the policy itself, without training a separate relevance model, in GRPO-style reinforcement learning where a single scalar reward per response is broadcast uniformly to all generated tokens. CoRT's answer is yes: replay the same sampled response under a prompt with the rubric criteria removed, record the per-token change in log probability, and treat that change as a measure of how much each token depends on the rubric. These contrasts are converted into bounded, response-normalized weights that redistribute the original group-relative advantage, so rubric-dependent tokens receive more of the update while the response-level reward and its direction are preserved. Across instruction-following benchmarks and reward granularities, the method improves over matched response-level GRPO in most comparisons, with an average gain of 4.4 percentage points, and remains competitive with a learned token-relevance baseline while avoiding a separate training stage.

Core claim

The central claim is that the counterfactual replay contrast Δ_{i,t} = log π(y_t | x⁺, y_<t) − log π(y_t | x⁻, y_<t), computed by rescoring the same generated token sequence after deleting the rubric criterion list from the prompt, is a usable proxy for rubric dependence. CoRT maps these unbounded contrasts through a sigmoid replay-margin transform, normalizes within each response so that token weights average to one, and multiplies the original GRPO advantage A_i by these weights to obtain a token-level shaped advantage bA_{i,t} = sg(w_{i,t}) A_i. The signed, group-relative direction of the update is preserved; only its distribution across tokens changes. The paper argues this improves cred

What carries the argument

The central object is the counterfactual replay contrast: the difference between the log-likelihood the frozen policy assigns to each generated token under the full rubric prompt and under a criteria-free prompt. Three transformations carry the argument: (1) a sigmoid replay-margin transform that bounds and centers the unbounded contrast, yielding scores in (−1/2, 1/2) when the contrast equals a learned centering offset; (2) response-mean normalization, which guarantees the mean token weight and the mean shaped advantage per response stay equal to the original GRPO advantage; and (3) a cubic Hermite SmoothStep schedule that ramps weight strength from 0 to 1, keeping early updates close to GR

Load-bearing premise

The load-bearing premise is that deleting the instruction list from the prompt removes only the rubric criteria and leaves all other task-relevant context unchanged; if the original prompt already implies the criteria, or if prompt-length and format differences alone drive the likelihood contrasts, then the computed weights do not measure rubric dependence and the redistribution loses its rationale.

What would settle it

Construct a prompt set where the same rubric criteria also appear in the original instruction text, so that the criteria-free replay differs only by duplicated content. If CoRT still shows a large gain over GRPO, the signal is not rubric-specific; if the gain disappears, the method depends on a clean criteria-free replay. Alternatively, measure CoRT weights when the criteria-free prompt is replaced by a length-matched, format-matched list of semantically unrelated criteria: true rubric-dependent tokens should show no systematic contrast, and training should behave like unweighted GRPO.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Token-level credit can be obtained purely from policy-internal likelihood contrasts under a prompt intervention, removing the need for a separate token-relevance learning stage.
  • CoRT preserves the response-level reward and group-relative normalization, so it can be layered onto GRPO-style algorithms without changing the rollout, verifier, reward, or clipping structure.
  • The method improves over matched response-level GRPO across model families and reward granularities, with an average gain of 4.4 percentage points and gains on most comparisons.
  • Response normalization and scheduled activation are necessary for stability: ablations without them show scale drift, length clipping, and gradient spikes.
  • CoRT remains competitive with a learned token-relevance baseline while avoiding the extra data-generation and training cost of that baseline.
  • The method is compatible with different policy optimization objectives, as demonstrated with DAPO and GSPO, suggesting replay-derived token credit is a general credit-allocation layer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the contrast is computed along observed trajectories, CoRT's effectiveness hinges on the criteria-free prompt being a clean counterfactual; in settings where the original prompt already encodes the same criteria, the signal weakens—a limitation the paper itself acknowledges—so gains may shrink when rubrics are redundant with the prompt.
  • The same replay-contrast mechanism could serve as a lightweight attribution tool beyond training, identifying which spans of a generation the policy actually conditions on without external labels, which may be useful for interpretability of rubric-following behavior.
  • A testable extension: apply CoRT to sparse-reward reasoning or long-horizon agentic tasks and compare whether counterfactual-replay weights remain predictive of human-judged rubric dependence; the paper's criterion-wise ablation suggests local, visible criteria work best, while global or absence-style criteria may require a different intervention, such as removing a specific criterion rather than

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes CoRT, a token-level credit-weighting method for rubric-conditioned GRPO. CoRT rescues the same sampled response under the original rubric prompt (x+) and a criteria-free prompt (x- = x), computes tokenwise log-likelihood contrasts, maps them to bounded response-normalized weights, and redistributes the response-level GRPO advantage across tokens. The authors claim that CoRT improves over response-level GRPO in most comparisons (average gain 4.4 pp) and is competitive with the learned token-relevance baseline RTT, without training an auxiliary scorer. Experiments cover Qwen3-4B, Qwen2.5-7B, and Qwen3-14B under CSR and AON rewards, plus integration with DAPO and GSPO.

Significance. If the central premise held—that the criteria-free replay contrast isolates dependence on rubric content—CoRT would be a simple and attractive alternative to learned token-level credit assignment. The paper also honestly reports fixed-step-500 results and training diagnostics, which is commendable. However, the load-bearing premise is directly challenged by the paper's own matched-context controls (Table 8), which show that the contrast is nearly identical for neutral filler and shuffled criteria. Moreover, the main empirical claim is based on best-within-window checkpoint selection, and the baseline rows are taken from a prior paper rather than rerun in the same pipeline. These issues do not make the idea incoherent, but they require substantial additional evidence before the claimed 4.4 pp gain can be attributed to rubric-guided credit allocation.

major comments (3)
  1. [Experimental Setup / Table 1 and Table 4] The paper defines x^- = x, i.e., the criteria-free prompt is obtained by removing the entire instruction list. The matched-context controls in Table 8 are therefore critical: for the same fixed response, the average cue-token contrast is 6.11 for criteria-free, 6.67 for length-matched neutral filler, and 5.76 for shuffled HIR criteria, while other tokens sit near 1.0. This near-equality means the contrast is driven by the presence/absence of an appended instruction block and its length, not by the specific rubric criteria. This undermines the central claim that Δ_{i,t} is a proxy for rubric dependence. A training run with a neutral-filler or shuffled-criteria replay control is required; if it reproduces the 4.4 pp gain, the method's rationale as a rubric-guided credit allocator collapses, and the gain could be an artifact of reweighting tokens sensitive to any appended block. The criteri
  2. [Table 1 caption] The main tables report the best validation score within the first 500 trainer steps, while Table 4 reports fixed step-500 results. The fixed step-500 results include negative deltas relative to GRPO (e.g., Qwen2.5-7B AON IFBench prompt -2.31, instruction -1.91), and the overall pattern of gains is weaker. Best-within-window selection can overstate improvement and is sensitive to checkpoint noise. The paper should report fixed-step results as the primary comparison, or better, provide multiple seeds with error bars. Without this, the claim of improvement in the 'vast majority of comparisons' is not reliably supported.
  3. [Table 1 caption] All baseline rows (Instruct, SFT, DPO, GRPO, RTT) are taken from RTT (Xu et al. 2026), while CoRT rows are new runs by the authors. Differences in training code, random seeds, and evaluation infrastructure can easily produce differences of the magnitude reported (several points on IFEval/IFBench). For a controlled comparison, the GRPO and RTT baselines need to be rerun in the same pipeline, or at least multiple seeds of both should be provided. Otherwise the central empirical comparison is not matched.
minor comments (6)
  1. [Conclusion] The abstract states an 'average gain of 4.4 percentage points' but does not specify the set of comparisons (which models, rewards, and benchmarks) over which the average is taken. Please clarify.
  2. [Figure 3] The conclusion concedes that the signal 'can be weaker when criteria are redundant with the original prompt.' This is precisely the case where x^- = x fails to isolate rubric content, and it is in tension with the main claim. The authors should either reconcile this or explicitly restrict the method's applicability.
  3. [Algorithm 1] The diagnostic figure lacks y-axis labels in the PDF text extraction; please ensure all panels have clear axis labels in the camera-ready version.
  4. [Related Work] The notation sg(·) is used for stop-gradient without definition in the main text; define it near the policy objective.
  5. [Table 8] The paper cites several 2026 arXiv preprints that may not be widely accessible; consider noting their status as preprints in the references.
  6. [Table 8] The matched-context controls are based on a single generated response. Report variance across multiple responses and prompts, or acknowledge the n=1 limitation explicitly in the main text.

Circularity Check

0 steps flagged

No significant circularity: CoRT's replay contrast is an explicit proxy and its empirical gains are not forced by the weight construction.

full rationale

CoRT's derivation is not circular. The replay contrast Δ_{i,t} = ℓ⁺_{i,t} − ℓ⁻_{i,t} is an operational proxy defined from the policy's own likelihoods, and the paper explicitly labels it a proxy rather than a calibrated measure: "We use this contrast as a proxy for rubric dependence, rather than as a calibrated measure of token importance." No theorem is claimed that would make 'rubric dependence' and the contrast equivalent by construction. The normalization identities — mean weight 1 and mean shaped advantage equal to the original advantage — are explicitly stated design constraints, not fitted predictions: "By construction, 1/T_i Σ_t w_{i,t} = 1 and 1/T_i Σ_t w_{i,t}A_i = A_i." The headline empirical claim, that CoRT improves over response-level GRPO, is established by external benchmarks under a shared evaluation protocol and is not an analytic consequence of the weight construction. Hyperparameters b, τ, η, and the ramp length are hand-set, not fit to the test results. The paper's own limitation statement — signal is weaker when criteria are redundant with the original prompt — is an empirical caveat, not a circular reduction. Baseline rows are cited from RTT and the CoRT runs use the same evaluation protocol; no load-bearing argument reduces to a self-citation. Overall, no specific step equates an output to an input by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The method rests on the assumption that criteria-free replay isolates rubric dependence, on a heuristic contrast-to-weight mapping, and on an unproven proxy relationship. No new physical or mathematical entity is introduced. Hyperparameters b, τ, η, K, and k0 are hand-set, and their values matter for training stability.

free parameters (5)
  • η (weighting strength) = 0.5
    Controls the overall strength of replay weighting; hand-set, not fitted to data.
  • b (replay contrast centering) = 0 (zero-centered)
    Centers the sigmoid transform; a non-zero b could systematically bias which tokens are upweighted.
  • τ (sigmoid temperature) = 1.0 (unit temperature)
    Sets the sharpness of the contrast-to-weight mapping.
  • K (SmoothStep ramp length) = 100 trainer steps
    Controls how gradually replay weighting is introduced; the paper shows removing the ramp causes late-training instability.
  • k0 (warmup offset) = not reported
    Warmup offset in the SmoothStep schedule; the paper does not state its value.
axioms (5)
  • domain assumption The criteria-free prompt x^- = x is a matched counterfactual that removes only rubric criteria.
    If the original prompt already contains redundant cues or if prompt length/format differences drive the contrast, the weights do not isolate rubric dependence. The paper's matched-context controls are only single examples.
  • ad hoc to paper Tokenwise log-likelihood contrast Δ_{i,t} is a valid proxy for rubric dependence.
    This is the central heuristic of CoRT; no proof is given, and the paper itself notes the signal is weak when criteria are redundant or reward is noisy.
  • standard math GRPO with clipped surrogate and group-relative advantages is a sound base objective.
    The paper builds on the standard PPO-style clipped surrogate and group-relative normalization, which are accepted background results.
  • domain assumption The frozen scoring policy θbar accurately represents the current policy for replay scoring.
    Contrasts are computed with a frozen policy per batch; policy drift during training may change what the contrast measures.
  • domain assumption Rubric verifier rewards (CSR/AON) are accurate enough to train on.
    Rewards are rule-based on HIR, but AON is sparse and the paper does not analyze verifier noise.

pith-pipeline@v1.3.0-alltime-deepseek · 15783 in / 12587 out tokens · 123794 ms · 2026-08-01T01:47:52.075881+00:00 · methodology

0 comments
read the original abstract

Rubric-based reinforcement learning enriches language model training by evaluating model outputs against explicit criteria. Yet in GRPO-style pipelines, these structured judgments are reduced to a scalar response-level reward and converted into a response-level advantage, which is broadcast uniformly to all generated tokens. This leaves no explicit mechanism for allocating credit within a response, even when different criteria are grounded in different spans, formatting decisions, or semantic choices. We propose CoRT, a token-level credit weighting method for rubric-conditioned GRPO. Instead of training an auxiliary token scoring model, CoRT uses counterfactual replay to rescore the same sampled response under the original rubric-conditioned prompt and a matched criteria-free prompt. The resulting tokenwise log-likelihood contrasts serve as a proxy for dependence on the rubric context. CoRT maps these contrasts to bounded, response-normalized weights and uses them to redistribute the signed GRPO advantage across tokens, without introducing an auxiliary scorer or changing the response-level reward. Experiments across instruction-tuned models and reward granularities show that CoRT improves over matched response-level GRPO in the vast majority of comparisons, with an average gain of 4.4 percentage points. The method remains competitive with learned token-level credit baselines while avoiding a separate relevance-learning stage. These results suggest that policy-internal counterfactual likelihood contrasts provide an effective training signal for within-response credit allocation while retaining the simplicity and stability of GRPO.

Figures

Figures reproduced from arXiv: 2607.25659 by Bo-Wen Zhang, Junwei He, Lan-Zhe Guo, Rongyi Lin, Shuhan Zhong, Song-Lin Lv, Wentao Ma, Wen Wang.

Figure 1
Figure 1. Figure 1: Real replay example. The same response is rescored [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of CoRT. Response level rubric rewards are first converted into a scalar GRPO advantage, which is normally [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Post-500 continuation for Qwen2.5-7B CSR inte [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qwen3-4B CSR failure-mode diagnostics for a [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 17 linked inside Pith

  1. [1]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  2. [2]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    Contextcite: Attributing model generation to context , author=. Advances in Neural Information Processing Systems , volume=

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=

  5. [5]

    Advances in Neural Information Processing Systems , volume =

    Policy Gradient Methods for Reinforcement Learning with Function Approximation , author =. Advances in Neural Information Processing Systems , volume =

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Lima: Less is more for alignment , author=. Advances in Neural Information Processing Systems , volume=

  7. [7]

    Advances in Neural Information Processing Systems , volume=

    Hindsight Experience Replay , author=. Advances in Neural Information Processing Systems , volume=

  8. [8]

    Advances in Neural Information Processing Systems , volume=

    Counterfactual Data Augmentation using Locally Factored Dynamics , author=. Advances in Neural Information Processing Systems , volume=

  9. [9]

    Advances in Neural Information Processing Systems , volume=

    Generalizing verifiable instruction following , author=. Advances in Neural Information Processing Systems , volume=

  10. [10]

    NeurIPS 2025 Workshop on Efficient Reasoning , year=

    Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains , author=. NeurIPS 2025 Workshop on Efficient Reasoning , year=

    2025

  11. [11]

    International Conference on Learning Representations , volume=

    Quantifying the plausibility of context reliance in neural machine translation , author=. International Conference on Learning Representations , volume=

  12. [12]

    International Conference on Learning Representations , volume=

    Let's verify step by step , author=. International Conference on Learning Representations , volume=

  13. [13]

    International Conference on Learning Representations , volume=

    WizardLM: Empowering large pre-trained language models to follow complex instructions , author=. International Conference on Learning Representations , volume=

  14. [14]

    International Conference on Learning Representations , volume=

    Muffin: Curating multi-faceted instructions for improving instruction following , author=. International Conference on Learning Representations , volume=

  15. [15]

    Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques , pages =

    An Image Synthesizer , author =. Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques , pages =

  16. [16]

    Machine Learning , volume =

    Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , author =. Machine Learning , volume =

  17. [17]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  18. [18]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Infobench: Evaluating instruction following ability in large language models , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  19. [19]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Verif: Verification engineering for reinforcement learning in instruction following , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  20. [20]

    From Generation to Judgment: Opportunities and Challenges of

    Dawei Li and Bohan Jiang and Liangjie Huang and Alimohammad Beigi and Chengshuai Zhao and Zhen Tan and Amrita Bhattacharjee and Yuxuan Jiang and Canyu Chen and Tianhao Wu and others , booktitle =. From Generation to Judgment: Opportunities and Challenges of

  21. [21]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    From complex to simple: Enhancing multi-constraint complex instruction following ability of large language models , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  22. [22]

    Computational Linguistics , volume=

    Large language model instruction following: A survey of progresses and challenges , author=. Computational Linguistics , volume=

  23. [23]

    arXiv preprint arXiv:2212.08073 , year=

    Constitutional ai: Harmlessness from ai feedback , author=. arXiv preprint arXiv:2212.08073 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2404.02823 , year=

    Conifer: Improving complex constrained instruction-following ability of large language models , author=. arXiv preprint arXiv:2404.02823 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2605.21726 , year=

    Probabilistic Attribution For Large Language Models , author=. arXiv preprint arXiv:2605.21726 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2512.23457 , year=

    Replay failures as successes: Sample-efficient reinforcement learning for instruction following , author=. arXiv preprint arXiv:2512.23457 , year=

    arXiv

  27. [27]

    arXiv preprint arXiv:2511.10507 , year=

    Advancedif: Rubric-based benchmarking and reinforcement learning for advancing llm instruction following , author=. arXiv preprint arXiv:2511.10507 , year=

    arXiv

  28. [28]

    arXiv preprint arXiv:2509.19199 , year=

    Agentic reinforcement learning with implicit step rewards , author=. arXiv preprint arXiv:2509.19199 , year=

    arXiv

  29. [29]

    arXiv preprint arXiv:2602.16165 , year=

    Hiper: Hierarchical reinforcement learning with explicit credit assignment for large language model agents , author=. arXiv preprint arXiv:2602.16165 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2603.08754 , year=

    Hindsight credit assignment for long-horizon llm agents , author=. arXiv preprint arXiv:2603.08754 , year=

    arXiv

  31. [31]

    arXiv preprint arXiv:2601.08430 , year=

    RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generation , author=. arXiv preprint arXiv:2601.08430 , year=

    arXiv

  32. [32]

    arXiv preprint arXiv:2604.02795 , year=

    Xu, Tianze and Zheng, Yanzhao and Lu, Pengrui and Ye, Lyumanshan and Wu, Yong and Zhang, Zhentao and Yu, Yuanqiang and Ma, Chao and Zhu, Jihuai and Liu, Pengfei and Dong, Baohua and Zhu, Hangcheng and Huang, Ruohui and Yu, Gang , title=. arXiv preprint arXiv:2604.02795 , year=

    Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2508.16949 , year=

    Breaking the exploration bottleneck: Rubric-scaffolded reinforcement learning for general llm reasoning , author=. arXiv preprint arXiv:2508.16949 , year=

    arXiv

  34. [34]

    arXiv preprint arXiv:2604.03128 , year=

    Self-distilled rlvr , author=. arXiv preprint arXiv:2604.03128 , year=

    Pith/arXiv arXiv

  35. [35]

    Wang, Jiaxuan and Ouyang, Xuan and Chen, Zhiyu and Hu, Yulan and Pan, Zheng and Li, Xin and Guo, Lan-Zhe , journal=

  36. [36]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  37. [37]

    arXiv preprint arXiv:2311.07911 , year=

    Instruction-following evaluation for large language models , author=. arXiv preprint arXiv:2311.07911 , year=

    Pith/arXiv arXiv

  38. [38]

    arXiv preprint arXiv:2505.07591 , year=

    A multi-dimensional constraint framework for evaluating and improving instruction following in large language models , author=. arXiv preprint arXiv:2505.07591 , year=

    Pith/arXiv arXiv

  39. [39]

    arXiv preprint arXiv:2507.18071 , year=

    Group sequence policy optimization , author=. arXiv preprint arXiv:2507.18071 , year=

    Pith/arXiv arXiv

  40. [40]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  42. [42]

    arXiv preprint arXiv:2412.15115 , year=

    Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year=

    Pith/arXiv arXiv

  43. [43]

    arXiv preprint arXiv:2601.05242 , year=

    Gdpo: Group reward-decoupled normalization policy optimization for multi-reward rl optimization , author=. arXiv preprint arXiv:2601.05242 , year=

    Pith/arXiv arXiv

  44. [44]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  45. [45]

    arXiv preprint arXiv:1711.00399 , year=

    Counterfactual explanations without opening the black box: Automated decisions and the GDPR , author=. arXiv preprint arXiv:1711.00399 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2510.00194 , year=

    GRPO- lambda : Credit Assignment improves LLM Reasoning , author=. arXiv preprint arXiv:2510.00194 , year=

    arXiv

  47. [47]

    arXiv preprint arXiv:2412.16720 , year=

    Openai o1 system card , author=. arXiv preprint arXiv:2412.16720 , year=

    Pith/arXiv arXiv

  48. [48]

    Nature , volume=

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , author=. Nature , volume=. 2025 , publisher=

    2025

  49. [49]

    Judea Pearl , title =

  50. [50]

    Proceedings of the Thirty-Eighth

    Gu, Zihui and Sun, Xingwu and Lian, Fengzong and Kang, Zhanhui and Xu, Chengzhong and Fan, Ju , title=. Proceedings of the Thirty-Eighth

  51. [51]

    Proceedings of the Thirty-Eighth

    Chen, Yihan and Xu, Benfeng and Wang, Quan and Liu, Yi and Mao, Zhendong , title=. Proceedings of the Thirty-Eighth