TABALIGN pairs a diffusion language model planner emitting binary cell masks with a trained attention verifier, raising average accuracy 15.76 points over strong baselines on eight table benchmarks while speeding execution 44.64%.
Attention illuminates llm reasoning: The preplan-and-anchor rhythm enables fine-grained policy optimization
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
abstract
The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps. This work positions attention as a privileged substrate that renders the internal logic of LLMs legible, not merely as a byproduct of computation, but as a mechanistic blueprint of reasoning itself. We first distinguish attention heads between locally and globally focused information processing and reveal that locally focused heads produce a sawtooth pattern near the diagonal indicating phrasal chunks, while globally focused heads expose tokens that exert broad downstream influence over future tokens. We formalize these with two metrics: 1) Windowed Average Attention Distance, which measures the extent of backward attention within a clipped window; 2) Future Attention Influence, which quantifies a token's global importance as the average attention it receives from subsequent tokens. Taken together, these signals reveal a recurring preplan-and-anchor mechanism, where the model first performs a long-range contextual reference to generate an introductory token, which is immediately followed by or coincides with a semantic anchor token that organizes subsequent reasoning. Leveraging these insights, we introduce three novel RL strategies that dynamically perform targeted credit assignment to critical nodes (preplan tokens, anchor tokens, and their temporal coupling) and show consistent performance gains across various reasoning tasks. By aligning optimization with the model's intrinsic reasoning rhythm, we aim to transform opaque optimization into an actionable structure-aware process, hoping to offer a potential step toward more transparent and effective optimization of LLM reasoning.
citation-role summary
background 1
citation-polarity summary
years
2026 8verdicts
UNVERDICTED 8roles
background 1polarities
background 1representative citing papers
RLSD mixes self-distillation for token-level policy difference magnitudes with RLVR for reliable update directions from response correctness to reach higher convergence and better training stability.
RiVER applies calibrated ranking rewards from execution scores to train LLMs on score-based tasks without ground-truth, producing gains on both heuristic contests and exact-solution coding benchmarks.
AGDO improves dLLM reasoning performance by determining denoising order and emphasizing tokens based on attention-derived dependencies rather than random masking.
GRAIL reweights token advantages via gradient saliency in RLVR, outperforming GRPO by 3.60% accuracy and 3.05% Pass@3 on five LLM families.
SERL selectively reweights learning using task success and environment feedback to reach 90.0% success on ALFWorld and 80.1% on WebShop, outperforming RL and distillation baselines.
VIMPO derives a policy-implied value function from optimality conditions for critic-free RL in LLMs and shows gains over GRPO on math benchmarks.
Early Stopping Rollout limits on-policy distillation to initial tokens to counter teacher decay, outperforming full rollouts in accuracy, efficiency, and stability while revealing cascading alignment effects.
citing papers explorer
-
From Table to Cell: Attention for Better Reasoning with TABALIGN
TABALIGN pairs a diffusion language model planner emitting binary cell masks with a trained attention verifier, raising average accuracy 15.76 points over strong baselines on eight table benchmarks while speeding execution 44.64%.
-
Self-Distilled RLVR
RLSD mixes self-distillation for token-level policy difference magnitudes with RLVR for reliable update directions from response correctness to reach higher convergence and better training stability.
-
Reinforcement Learning without Ground-Truth Solutions can Improve LLMs
RiVER applies calibrated ranking rewards from execution scores to train LLMs on score-based tasks without ground-truth, producing gains on both heuristic contests and exact-solution coding benchmarks.
-
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models
AGDO improves dLLM reasoning performance by determining denoising order and emphasizing tokens based on attention-derived dependencies rather than random masking.
-
GRAIL: Gradient-Reweighted Advantages for Reinforcement Learning with Verifiable Rewards
GRAIL reweights token advantages via gradient saliency in RLVR, outperforming GRPO by 3.60% accuracy and 3.05% Pass@3 on five LLM families.
-
What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
SERL selectively reweights learning using task success and environment feedback to reach 90.0% success on ALFWorld and 80.1% on WebShop, outperforming RL and distillation baselines.
-
VIMPO: Value-Implicit Policy Optimization for LLMs
VIMPO derives a policy-implied value function from optimality conditions for critic-free RL in LLMs and shows gains over GRPO on math benchmarks.
-
Less is More: Early Stopping Rollout for On-Policy Distillation
Early Stopping Rollout limits on-policy distillation to initial tokens to counter teacher decay, outperforming full rollouts in accuracy, efficiency, and stability while revealing cascading alignment effects.