Finite-state tools add only log|M| bits to finite-precision recurrent controllers, while a single tape tool yields Turing completeness with O(log|Q|+log|Γ|) bits, realized exactly by one-layer selective SSMs.
On the Expressive Power and Limitations of Multi-Layer SSMs
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We study the expressive power and limitations of multi-layer state-space models (SSMs). First, we show that multi-layer SSMs face fundamental limitations in compositional tasks, revealing an inherent gap between SSMs and streaming models. Then, we examine the role of chain-of-thought (CoT), showing that offline CoT does not fundamentally increase the expressiveness, while online CoT can substantially increase its power. Indeed, with online CoT, multi-layer SSMs become equivalent in power to streaming algorithms. Finally, we investigate the tradeoff between width and precision, showing that these resources are not interchangeable in the base model, but admit a clean equivalence once online CoT is allowed. Overall, our results offer a unified perspective on how depth, finite precision, and CoT shape the power and limits of SSMs.
fields
cs.FL 1years
2026 1verdicts
ACCEPT 1representative citing papers
citing papers explorer
-
When Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models?
Finite-state tools add only log|M| bits to finite-precision recurrent controllers, while a single tape tool yields Turing completeness with O(log|Q|+log|Γ|) bits, realized exactly by one-layer selective SSMs.