Pith. sign in

REVIEW 7 cited by

Prefix Grouper: Efficient GRPO Training through Shared-Prefix Forward

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.05433 v1 pith:C3NWHLP7 submitted 2025-06-05 cs.LG cs.AI

Zikang Liu , Tongtian Yue , Yepeng Tang , Longteng Guo , Junxian Cai , Qingbin Liu , Xi Chen , Jing Liu This is my paper
classification cs.LGcs.AI
keywords prefixgrpogroupertrainingcomputationalforwardgrouppolicy
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Group Relative Policy Optimization (GRPO) enhances policy learning by computing gradients from relative comparisons among candidate outputs that share a common input prefix. Despite its effectiveness, GRPO introduces substantial computational overhead when processing long shared prefixes, which must be redundantly encoded for each group member. This inefficiency becomes a major scalability bottleneck in long-context learning scenarios. We propose Prefix Grouper, an efficient GRPO training algorithm that eliminates redundant prefix computation via a Shared-Prefix Forward strategy. In particular, by restructuring self-attention into two parts, our method enables the shared prefix to be encoded only once, while preserving full differentiability and compatibility with end-to-end training. We provide both theoretical and empirical evidence that Prefix Grouper is training-equivalent to standard GRPO: it yields identical forward outputs and backward gradients, ensuring that the optimization dynamics and final policy performance remain unchanged. Empirically, our experiments confirm that Prefix Grouper achieves consistent results while significantly reducing the computational cost of training, particularly in long-prefix scenarios. The proposed method is fully plug-and-play: it is compatible with existing GRPO-based architectures and can be seamlessly integrated into current training pipelines as a drop-in replacement, requiring no structural modifications and only minimal changes to input construction and attention computation. Prefix Grouper enables the use of larger group sizes under the same computational budget, thereby improving the scalability of GRPO to more complex tasks and larger models. Code is now available at https://github.com/johncaged/PrefixGrouper

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts

    cs.LG 2026-05 conditional novelty 8.0 of 10

    DualKV is a new FlashAttention variant that shares prompt KV across multiple rollouts in RL training, delivering 1.63-3.82x speedups on 8B-30B models while remaining mathematically identical to standard attention.

  2. Faster Synchronous On-Policy RL via Straggler-Aware Group Sizing

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    SAGC dynamically adjusts group sizes in synchronous GRPO and DAPO via online constrained optimization to cut stragglers, improve wall-clock speed, and maintain or improve rewards and downstream reasoning performance.

  3. Schedule-Level Shared-Prefix Reuse for LLM RL Training

    cs.DC 2026-05 unverdicted novelty 6.0 of 10

    Schedule-level shared-prefix reuse decouples prefix and suffix passes in GRPO training to compute shared prefixes once, delivering up to 4.395x speedup and 59.1% HBM reduction while preserving numerical equivalence.

  4. DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    DualKV eliminates redundant prompt replication in RL training attention kernels via fused dual-KV CUDA operations and token repacking, delivering 1.63-3.82x policy-update speedups while remaining mathematically equiva...

  5. Tree Training: Accelerating Agentic LLMs Training via Shared Prefix Reuse

    cs.LG 2025-11 unverdicted novelty 6.0 of 10

    Tree Training serializes tree trajectories via DFS and uses redundancy-free partitioning to compute weighted per-token losses exactly once per token, achieving up to 6.2x training speedup on dense and MoE models.

  6. LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Real-World Multilingual VQA

    cs.CV 2025-09 unverdicted novelty 6.0 of 10

    LaV-CoT introduces a multi-stage visual CoT pipeline and GRPO training with language-consistency rewards, delivering up to 9.5% accuracy gains on multilingual VQA benchmarks over similar-sized open models.

  7. DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts

    cs.LG 2026-05 conditional novelty 5.0 of 10

    DualKV removes the N-fold recomputation of shared prompt tokens in RL policy updates via an exact two-region FlashAttention-style kernel, measuring 1.6–3.8x policy-update speedups.

Pith tools