A unified measurement pipeline on 27 LLM checkpoints shows activation maxima spanning four orders of magnitude, with MoE models 14-23x lower than matched dense models and residual streams carrying the global max in most cases.
Qwen2.5 technical report
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2representative citing papers
Template collapse is a distinct failure mode in agentic RL invisible to entropy; mutual information proxies diagnose it better and SNR-aware filtering using reward variance improves input-dependent reasoning and task performance across planning, math, navigation, and code tasks.
citing papers explorer
-
Measuring Maximum Activations in Open Large Language Models
A unified measurement pipeline on 27 LLM checkpoints shows activation maxima spanning four orders of magnitude, with MoE models 14-23x lower than matched dense models and residual streams carrying the global max in most cases.
-
RAGEN-2: Reasoning Collapse in Agentic RL
Template collapse is a distinct failure mode in agentic RL invisible to entropy; mutual information proxies diagnose it better and SNR-aware filtering using reward variance improves input-dependent reasoning and task performance across planning, math, navigation, and code tasks.