LoopUS converts pretrained LLMs into looped latent refinement models via block decomposition, selective gating, random deep supervision, and confidence-based early exiting to improve reasoning performance.
Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Prediction
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
Autoregressive models (ARMs) currently constitute the dominant paradigm for large language models (LLMs). Energy-based models (EBMs) represent another class of models, which have historically been less prevalent in LLM development, yet naturally characterize the optimal policy in post-training alignment. In this paper, we provide a unified view of these two model classes. Taking the chain rule of probability as a starting point, we establish an explicit bijection between ARMs and EBMs in function space, which we show to correspond to a special case of the soft Bellman equation in maximum entropy reinforcement learning. Building upon this bijection, we derive the equivalence between supervised learning of ARMs and EBMs. Furthermore, we analyze the distillation of EBMs into ARMs by providing theoretical error bounds. Our results provide insights into the ability of ARMs to plan ahead, despite being based on the next-token prediction paradigm.
citation-role summary
background 1
citation-polarity summary
years
2026 4roles
background 1polarities
background 1representative citing papers
Derives that attention-induced non-Markovian dynamics yield a nonlocal Onsager-Machlup action whose short-memory expansion recovers the local action of a companion paper.
Three-layer ontology injection significantly improves enterprise agent metric accuracy and role consistency, especially in low-coverage domains such as Vietnamese regulated industries.
citing papers explorer
-
LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models
LoopUS converts pretrained LLMs into looped latent refinement models via block decomposition, selective gating, random deep supervision, and confidence-based early exiting to improve reasoning performance.
-
Path-Measure Dynamics of Attention-Driven World Models: A Nonlocal Onsager--Machlup Approach
Derives that attention-induced non-Markovian dynamics yield a nonlocal Onsager-Machlup action whose short-memory expansion recovers the local action of a companion paper.
-
Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agents
Three-layer ontology injection significantly improves enterprise agent metric accuracy and role consistency, especially in low-coverage domains such as Vietnamese regulated industries.
- Constitutional On-Policy Safe Distillation