Pith. sign in

REVIEW 4 cited by

Hyperspherical Normalization for Scalable Deep Reinforcement Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.15280 v2 pith:XG557ZLO submitted 2025-02-21 cs.LG

Hojoon Lee , Youngdo Lee , Takuma Seno , Donghu Kim , Peter Stone , Jaegul Choo This is my paper
classification cs.LG
keywords learningsimbav2hypersphericalmodelnormalizationoptimizationperformancereinforcement
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Scaling up the model size and computation has brought consistent performance improvements in supervised learning. However, this lesson often fails to apply to reinforcement learning (RL) because training the model on non-stationary data easily leads to overfitting and unstable optimization. In response, we introduce SimbaV2, a novel RL architecture designed to stabilize optimization by (i) constraining the growth of weight and feature norm by hyperspherical normalization; and (ii) using a distributional value estimation with reward scaling to maintain stable gradients under varying reward magnitudes. Using the soft actor-critic as a base algorithm, SimbaV2 scales up effectively with larger models and greater compute, achieving state-of-the-art performance on 57 continuous control tasks across 4 domains. The code is available at https://dojeon-ai.github.io/SimbaV2.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping

    eess.SY 2026-07 conditional novelty 6.0 of 10

    FAOC maps RL actions into a state-dependent feasible parameter set, guaranteeing optimal-control feasibility and improving RL-MPC performance on table tennis.

  2. Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Splitting weight matrices into a fixed-norm direction and learnable per-row/column magnitudes improves LLM training over AdamW/Muon, removes weight decay and warmup, and transfers the optimal LR across width.

  3. Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments

    cs.LG 2026-03 conditional novelty 5.0 of 10

    PPO plateaus can be avoided by increasing the number of parallel environments, which reduces both the outer-loop step size and update noise; scaling to 1M environments sustained improvement to 1T transitions.

  4. Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies

    cs.LG 2025-08 conditional novelty 4.0 of 10

    A survey that categorizes deep reinforcement learning scaling strategies into data, network, and training budget dimensions and outlines challenges for scaling DRL systems.

Pith tools