Pith Number

pith:WJCRQIER

pith:2021:WJCRQIERNWMDPNCSCWVDCZKC6K

not attested not anchored not stored refs resolved

Efficiently Modeling Long Sequences with Structured State Spaces

Albert Gu, Christopher R\'e, Karan Goel

S4 uses a low-rank correction to the state matrix A so state space models can be computed efficiently via Cauchy kernels while retaining long-range power.

arxiv:2111.00396 v3 · 2021-10-31 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{WJCRQIERNWMDPNCSCWVDCZKC6K}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

S4 achieves SoTA on every task from the Long Range Arena benchmark, including solving the challenging Path-X task of length 16k that all prior work fails on, while being as efficient as all competitors.

C2weakest assumption

The low-rank correction to the state matrix A permits stable diagonalization and that the resulting Cauchy kernel computation fully preserves the theoretical long-range modeling strengths of the underlying SSM without introducing approximation errors that degrade performance on real data.

C3one line summary

S4 is an efficient state space sequence model that captures long-range dependencies via structured parameterization of the SSM, achieving state-of-the-art results on the Long Range Arena and other benchmarks while being faster than Transformers for generation.

References

52 extracted · 52 resolved · 5 Pith anchors

[1] Unitary evolution recurrent neural networks 2016

[2] Adaptive Input Representations for Neural Language Modeling 2018 · arXiv:1809.10853

[3] An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling 2018 · arXiv:1803.01271

[4] Trellis networks for sequence modeling 2019

[5] Dilated recurrent neural networks 2017

Formal links

3 machine-checked theorem links

Cited by

171 papers in Pith

FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention

State Space Models Meet Remote Sensing: A Survey

Efficient Remote Sensing Instance Segmentation with Linear-Time State Space Distilled Visual Foundation Models

ZONOS2 Technical Report

MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction

Receipt and verification

First computed	2026-07-05T04:46:12.497941Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

b2451820916d9837b45215aa316542f2b0501ff691f06de49e8eec152c5b4dc7

Aliases

arxiv: 2111.00396 · arxiv_version: 2111.00396v3 · doi: 10.48550/arxiv.2111.00396 · pith_short_12: WJCRQIERNWMD · pith_short_16: WJCRQIERNWMDPNCS · pith_short_8: WJCRQIER

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/WJCRQIERNWMDPNCSCWVDCZKC6K \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: b2451820916d9837b45215aa316542f2b0501ff691f06de49e8eec152c5b4dc7

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "2de9430fa455ce79b6f7bcefb0568c063b3e18a85b032f11dfbd29592d4f6c6e",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2021-10-31T03:32:18Z",
    "title_canon_sha256": "005efae14363ecad4d4a31266a29e0e4193008ccf62dff1cb1b8ee24ae6dda10"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2111.00396",
    "kind": "arxiv",
    "version": 3
  }
}