Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Neural networks leverage nominally quantum and post-quantum representations

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Neural networks pretrained on next-token prediction spontaneously discover quantum and post-quantum belief geometries.

desk verdict A solid, reproducible demonstration that standard next-token models linearly encode quantum and post-quantum belief geometries, but the headline overstates the evidence by claiming inference-time Bayesian updating. read the letter →

arxiv 2507.07432 v2 pith:VGEWA5ZY submitted 2025-07-10 cs.LG quant-ph

Paul M. Riechers , Thomas J. Elliott , Adam S. Shai This is my paper
classification cs.LGquant-ph
keywords beliefstategeometrynext-tokenpredictiongeneralizedhiddenMarkovmodelsquantumstochasticprocessespost-quantummemoryBlochsphereneuralnetworkinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that deep neural networks trained in the standard way, predicting the next token from context, do not merely memorize sequences but learn an internal geometry of beliefs about the future. This geometry is the one produced by Bayesian filtering over the minimal generative model of the training data, and for some classical stochastic processes that minimal model is quantum or post-quantum rather than classical. Concretely, the paper claims a fixed affine map sends network activations to the generalized Bloch vector or density matrix that Bayesian updates over a qudit memory would produce. If true, standard pretraining yields a linear encoding of belief geometry that no finite-state classical model can realize exactly, and this encoding is nearly independent of network architecture.

What carries the argument

The carrying object is the generalized hidden Markov model (GHMM), a linear-algebraic description of stochastic processes in which a token sequence applies linear operators to a latent vector, with classical HMMs, quantum instruments, and generalized probabilistic theories as special cases. Belief states are history-induced predictive vectors, and their geometry lives in the convex hull of the theory's pure states. The paper's test is an affine probe: after training, a learned linear map from concatenated activations to the generalized Bloch vector or density matrix of the minimal generator should reproduce the Bayesian belief-update trajectory. The observed linearity is what lets neural activations realize non-orthogonal states that classical discrete memory cannot.

What would settle it

Build a finite-state hidden Markov model that generates the Bloch Walk process at the paper's parameter values (alpha = 1, beta = sqrt(51)) and verify that it reproduces the process's token probabilities; if such a model exists, the middle-row experiment would show only a classical representation.

Watch

Extended reading notes

Core claim

The central claim is that next-token-prediction pretraining causes transformers, RNNs, LSTMs, and GRUs to linearly represent the minimal belief geometry of their training process. For the classical Mess3 process the activations map to a fractal subset of a 2-simplex; for the newly introduced Bloch Walk process they map to a 2D slice of the Bloch sphere, the belief states of a single-qubit generator; and for the Moon process they map to curved manifolds of a 3-dimensional post-quantum generalized probabilistic theory. The paper presents weighted linear regression as evidence: a single affine map from concatenated layer activations to ground-truth belief states achieves low root mean square error for the minimal generator, while fits to classical Markov-order-3 approximations and to randomly initialized networks are worse, and pairwise cosine similarities between predicted and true belief states have R-squared near 1.

Load-bearing premise

The claim that the Bloch Walk process has no finite-state classical (HMM) representation; the paper cites the general phenomenon but supplies no proof for this specific process, so if a finite HMM exists the quantum demonstration would collapse to a classical one.

Editorial extensions

If this is right

  • Pretrained sequence models can be read out as Bayesian filters: a fixed affine map on activations gives the latent belief state of the minimal generator.
  • The same pretraining objective that produces language models also produces compact post-classical world models, so world-model claims about such networks should include quantum and post-quantum possibilities.
  • Because the representation is linear and architecture-independent, the belief geometry can be predicted from the data-generating process itself rather than from implementation details.
  • Quantum-advantage claims need a caveat: classical networks with real-valued activations can emulate post-classical belief updates up to floating-point precision, although the exponential Hilbert-space dimension of large quantum circuits still blocks practical replacement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would train on data with unknown minimal generators and use the affine-probe error gap to detect whether a post-classical representation has been found.
  • The same linear-encoding claim suggests that for any finite-context sequence model, the geometry of next-token distributions in activation space is governed by the training process's GHMM, which could let practitioners estimate the minimal generator from activations alone.
  • The authors note that biological neural networks are not investigated; if the mechanism is the continuous real-valued activation space itself, the phenomenon may extend beyond artificial networks.
  • Applying the probe to large pretrained language models on natural language is a direct next test: if natural data has compact generative structure, residual-stream activations should show similar post-classical geometries rather than purely classical ones.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper claims that deep neural networks trained by standard next-token prediction spontaneously learn to represent the Bayesian belief geometry of the minimal generative model of their training data, including for stochastic processes whose minimal generators are quantum (Bloch Walk, FRDN) or post-quantum (Moon). The evidence consists of training transformer, LSTM, GRU, and RNN models on four processes and fitting a fixed affine map from concatenated layer activations to the exact belief states of the minimal generator, as defined through a generalized hidden Markov model framework. The paper reports low in-sample RMSE for the minimal-generator target, high pairwise cosine-similarity agreement, a random-initialization control, and an architecture sweep; it also includes public code and model checkpoints.

Significance. If the central claim holds, the result is significant: it would show that ordinary next-token pretraining yields linear encodings of minimal predictive-state geometries that go beyond finite classical hidden Markov models, and that this phenomenon is architecture-independent. The paper has real strengths: the target belief geometries for Mess3, FRDN, and Moon are defined externally from the process theory and prior literature, the random-initialization control is appropriate, the architecture sweep is broad, and the public code and checkpoint release support reproducibility. The main gap is that the probe evidence is entirely in-sample, so the current experiments establish linear decodability of the anchor contexts rather than an inference-time representational claim on unseen contexts; additionally, the non-classicality of the newly introduced Bloch Walk process is asserted rather than proved for the specific parameter values used.

major comments (3)
  1. [Appendix F, Eqs. (F3)-(F9)] The affine probe is fitted and evaluated on the same anchor set A, defined as all sequences up to the context window: the 10-fold cross-validation in Section F.2.c is used only to select the regularization parameter r, after which the final model is retrained on all data and the reported RMSE is computed on A via Eqs. (F8)-(F9). Consequently, the RMSE and cosine-similarity R2 values in Figs. 2 and 4 quantify in-sample linear decodability, not the claim in the abstract and Section VIII that the network performs iterative Bayesian updates 'during inference' on newly observed contexts. Please add a genuine held-out evaluation: fit L on one subset of contexts and report RMSE/R2 on disjoint contexts, ideally including sequences longer than the training context window or held-out token combinations; the random-initialization control does not address this issue.
  2. [Section IV.B and Appendix D.2] The claim that the Bloch Walk process 'has no finite HMM representation' is load-bearing for the quantum content of the middle-row demonstration, but it is asserted without proof and the cited Monras-Winter result is a general theorem, not a demonstration for this specific four-token process defined by Eqs. (D4)-(D7). If the process actually admits a finite HMM for the chosen parameters (alpha=1, beta=sqrt(51)), the middle-row experiment would only show a classical representation. Please provide a direct proof, or a precise citation establishing non-HMM realizability for the exact process and parameter regime used, or explicitly soften the claim to 'nominally quantum' throughout.
  3. [Figures 2 and 5-8; Section IV.C] The comparison between the fit to the minimal generator and the fit to the Markov-order-3 baseline is not controlled for target dimensionality. For the Bloch Walk process the quantum target lives in a 2-dimensional affine subspace, while the Markov-order-3 belief simplex has 64 dimensions, so raw RMSE values from Eqs. (F8)-(F9) on targets of different dimensions are not directly comparable; the same concern applies to the FRDN and Moon comparisons. Please report a dimension- or scale-normalized error (for example, RMSE divided by the root-mean-square norm of the target belief vectors, or per-dimension RMSE) and show that the stated preference for the minimal generator survives this normalization.
minor comments (6)
  1. [Abstract] There is a typo in 'these representation' in the abstract; it should read 'these representations'.
  2. [Table I and Figs. 2, 5-8] All experiments use a single random seed (seed=42), and the reported RMSE values have no error bars; please either add multiple seeds and report variability, or explicitly state that the results are single-seed demonstrations.
  3. [Section III, Eq. (4)] The formatting of Eq. (4) is broken: the implication line ends with 'c, c\' in R' without a period or connection to the preceding expression; please clean up the notation.
  4. [Appendix D.2.b] There is a typo in 'subseqeunt' in the text preceding Eq. (D11); it should be 'subsequent'.
  5. [Title and Section I.A] The term 'post-quantum' is used in a generalized-probabilistic-theory sense, which may be confused with post-quantum cryptography; please define the acronym GPT and explicitly distinguish the two meanings at first use.
  6. [Appendix H, Table IV] The hyperlinks in Table IV appear as placeholders ('[run]', 'config', 'HF'); since the reproducibility claim depends on these resources, the published version should contain the actual URLs.

Circularity Check

1 steps flagged · score 6.0 of 10

The central representation claim rests on an affine probe whose reported error is the in-sample training objective; no held-out contexts are evaluated.

  1. fitted input called prediction [Appendix F, Eqs. (F1)-(F9); Sec. IV.A; Figs. 2 and 4 captions]
    "We assemble a dataset from anchor sequences A = {w_n}^|A|_{n=1} ⊂ X^*. In our analysis, we take the set of anchor points to be all sequences generated by the process up to the context window length of the network. ... MSE = ∑_i p_i ||η(w_i) − η̂(w_i)||^2_2 (F8) RMSE = √MSE (F9)."

    The affine map L is the weighted least-squares solution L* = (P^{1/2}A)^+ P^{1/2}Γ (F4), and the reported RMSE (F8-F9) and the cosine-similarity R^2 in Fig. 4 are evaluated on the same anchor set A that defines the regression. The paper presents these training residuals as evidence that activations represent the belief geometry and that networks perform Bayesian updates as more context is observed (Sec. VIII), but no held-out contexts (e.g., sequences longer than the context window) are probed. Since the probe has about 256 input dimensions and only 2-3 target dimensions, a low in-sample RMSE is a minimized objective value, not a predictive test; the random-init control rules out pure capacity artifacts but does not address the train/eval overlap.

full rationale

The core benchmark geometries for Mess3, FRDN, and Moon are externally defined by process transition matrices from the literature, so the target is not extracted from the network or defined by the authors' prior results. However, the central evidence for representing those geometries is an affine probe whose parameters are fit on the full anchor set and whose reported error is the training error on that same set; cross-validation is used only to select the regularization parameter. The self-citations (Refs. [1], [2], [15]) are not load-bearing for the new quantum/post-quantum experiments, which are self-contained. The Bloch Walk no-finite-HMM assertion is an unproven import from Monràs and Winter for a new process; this is a correctness risk, not circularity, and does not change the score. Overall, one central prediction reduces to in-sample fit, giving partial circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The probe-based evidence is fit to the target geometry on the same contexts used for evaluation, so the central representation is partly a fitting statement rather than a prediction. The new Bloch Walk process's claimed non-classicality is an unproved ad hoc assumption, while the FRDN and Moon non-classicality come from the cited literature. The training and baseline choices contain several hand-picked constants, including process parameters, context length, and the Markov order.

free parameters (5)
  • Affine probe (L, b) = Estimated by regularized weighted least squares with regularization selected by 10-fold cross-validation
    The central evidence is a linear map fit from network activations to ground-truth belief vectors on the same contexts used for evaluation. The reported RMSE and cosine similarities are in-sample, so the existence of the map is a fitting result, not an out-of-sample prediction.
  • Markov-order-3 baseline = Order 3, giving 64 belief-state dimensions
    The baseline dimension was chosen to match the hidden-state dimensionality of the networks. RMSE is compared across different target dimensions without normalization, which can bias results toward lower-dimensional targets.
  • Process parameters = Mess3 alpha=0.85, x=0.05; Bloch Walk alpha=1, beta=sqrt(51); FRDN alpha=2000, lambda=0.49; Moon alpha=e, beta=0.5
    Hand-picked parameter values define the stochastic processes used for training. The results are not shown to be robust to variation in these parameters.
  • Training hyperparameters = 20,000 epochs, Adam learning rate 1e-4, batch size 128 or 16 for some transformers, seed 42
    All experiments use a single random seed and one run per architecture-process pair, so there are no error bars or significance tests over random initializations.
  • Context window length = 8 tokens
    Anchors include all sequences up to length 8, and evaluation is performed on that same set. The probe is not tested on longer or held-out contexts.
assumptions (5)
  • ad hoc to paper The Bloch Walk process has no finite-state HMM representation
    Asserted in Sec. IV.B with a general citation to Monras-Winter, but no proof or specific argument is given for this newly introduced process. The quantum minimality of the process is load-bearing for the quantum representation claim.
  • domain assumption The Moon process has no finite-dimensional quantum generator
    Taken directly from Fanizza et al. as an external theorem; the paper does not re-derive it.
  • domain assumption Floating-point neural network activations provide an effectively real-valued vector space
    Introduced in Sec. I.B with the caveat up to machine precision. This assumption underlies the claim that networks can emulate post-classical geometries, but it creates tension with the abstract's absolute no finite classical circuit statement.
  • standard math Predictive vectors of a GHMM determine future conditional probabilities linearly
    Used throughout Secs. II and III. This is standard linear algebra for GHMMs and is derived in the paper.
  • ad hoc to paper Next-token cross-entropy training induces a world model whose sufficient statistics are the predictive vectors
    This is the hypothesis under test, assumed in the probing design. The paper provides correlational evidence that a linear decoder can recover the geometry, but no causal evidence that the network's predictions are computed from these beliefs.
invented entities (1)
  • Bloch Walk process
    purpose: A classical stochastic process claimed to require a single qubit of quantum memory but no finite HMM, used as the primary test of quantum belief geometry in neural networks.
    Newly introduced in this paper. The no-finite-HMM property is asserted but not proven, so the quantum characterization lacks independent support until a proof or external verification is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural networks leverage nominally quantum and post-quantum representations." pith.science (2026). https://pith.science/paper/VGEWA5ZY

@misc{pith2026250707432,
  author       = {Pith},
  title        = {Pith review of: Neural networks leverage nominally quantum and post-quantum representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VGEWA5ZY}},
  note         = {Machine review of arXiv:2507.07432}
}
read the original abstract

We show that deep neural networks, including transformers and RNNs, pretrained as usual on next-token prediction, intrinsically discover and represent beliefs over 'quantum' and 'post-quantum' low-dimensional generative models of their training data -- as if performing iterative Bayesian updates over the latent state of this world model during inference as they observe more context. Notably, neural nets easily find these representation whereas there is no finite classical circuit that would do the job. The corresponding geometric relationships among neural activations induced by different input sequences are found to be largely independent of neural-network architecture. Each point in this geometry corresponds to a history-induced probability density over all possible futures, and the relative displacement of these points reflects the difference in mechanism and magnitude for how these distinct pasts affect the future.

Figures

Figures reproduced from arXiv: 2507.07432 by the authors.

Figure 1
Figure 1. FIG. 1 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. shows that neural activations of pretrained neural networks directly instantiate the belief geometry associated with Bayesian updates to the latent states of classical, quantum, and post-quantum world models, moving up to the more generalized category whenever it allows a representation in lower dimensions. Moreover, these representations are universal across different types of neural network architecture—whether RN… view at source ↗
Figure 3
Figure 3. FIG. 3 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: FIG. 4 [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5 [PITH_FULL_IMAGE:figures/full_fig_p025_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6 [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]
Figure 7
Figure 7. Figure 7: FIG. 7 [PITH_FULL_IMAGE:figures/full_fig_p027_7.png]
Figure 8
Figure 8. Figure 8: FIG. 8 [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 2
Figure 2. Figure 2: Fig2.py [PITH_FULL_IMAGE:figures/full_fig_p029_2.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PreScience: A Dataset and Benchmark for Scientific Forecasting

    cs.AI 2026-02 conditional novelty 6.0 of 10

    A new benchmark tests whether AI can forecast future scientific papers; frontier LLMs score ~5.6/10 on matching real abstracts, and simulated corpora are measurably less diverse and novel than human science.

  2. Identifiability and minimality bounds of quantum and post-quantum models of classical stochastic processes

    quant-ph 2025-09 conditional novelty 6.0 of 10

    Quantum hidden Markov models are shown to be identifiable via a finite set of word probabilities, and the minimal quantum memory dimension is proved to be at least the square root of the minimal generalized hidden Mar...

Reference graph

Works this paper leans on

45 extracted references · 43 canonical work pages · cited by 2 Pith papers

  1. [1]

    In this case, our GHMM is constructed from the initial vector⟨⟨η(∅)| = ℧(ρ0) and from the transition operatorsT (x) =∑ yK⊤ x,y⊗K† x,y

    (Transposed) Liouville-space representation We obtain the transpose of the standard Liouville-space representation [13] via the linear invertible ket-flipper map℧, such that℧(c|α⟩⟨β|) =c|α⟩⊤⊗⟨β| =⟨α|∗⊗c⟨β| for allc∈ C and for any two vectors in the Hilbert space|α⟩,|β⟩∈H , where⟨α| =|α⟩† and (·)∗ denotes complex conjugation. In this case, our GHMM is cons...

    work page

  2. [2]

    Generalized Bloch representation of density matrices It is well known that the state of a qubitρ can be expressed via its Bloch vector⃗ a: ρ =I/2 +⃗ a·⃗ σ/2 , (B1) where⃗ σ= (σx,σy,σz) is the vector of Pauli matrices. For a quantum system of arbitrary finite dimension—i.e., a qudit ρ acting on ad-dimensional vector spaceVd—we achieve something similar via...

    work page

  3. [3]

    However, because of the measurement, the CPTP map has|X| subchannels, each a trace-non-preserving superoperator Ax on the density matrix

    Generalized Bloch representation of subchannels If we marginalize over measurements, the memory goes through a completely positive and trace preserving (CPTP) map. However, because of the measurement, the CPTP map has|X| subchannels, each a trace-non-preserving superoperator Ax on the density matrix. By linearity, each of these subchannels can be fully de...

    work page

  4. [4]

    The process is defined by two parameters,α and x, with dependent quantitiesβ = (1−α)/2 and y = 1− 2x

    Classical: Mess3 process The Mess3 process [1, 2, 16] has three hidden statesS ={1, 2, 3}, and three observable tokensX ={a,b,c}. The process is defined by two parameters,α and x, with dependent quantitiesβ = (1−α)/2 and y = 1− 2x. For experiments we usedx = 0.05,α = 0.85. The labeled transition matrices are: T (a) =   αy βx βx αx βy βx αx βx βy   (D1...

    work page

  5. [5]

    Quantum: Bloch Walk process Here we introduce the Bloch Walk process, a probability density over sequences of tokensX ={0, 1, 2, 3}, which can be generated with a single qubit of quantum memory but has no finite HMM representation. We find that neural networks trained on this classical stochastic process (whether RNNs or transformers) linearly represent t...

    work page

  6. [6]

    Quantum: FRDN Despite not having any finite HMM generator, the FRDN process can be generated by a single qutrit—a quantum system with a three-dimensional Hilbert space [6]. We find that neural networks trained on this classical stochastic process intrinsically learn the finite-dimensional quantum generative mechanism, and represent Bayesian updates over t...

    work page 2000

  7. [7]

    Following Ref

    Post-quantum: Moon process A minimal example of a post-quantum process—a classical stochastic process with a finite-dimensional GHMM generator, yet no finite HMM and no finite-dimensional quantum generator—has a simple three-dimensional representation, and an observable alphabet of three symbolsX ={a,b,c}. Following Ref. [6], the linear maps generating th...

    work page

  8. [8]

    Experimental Design We conduct a comprehensive evaluation of four neural network architectures on four distinct stochastic processes, resulting in 16 experimental configurations. The architectures include transformers, LSTMs, GRUs, and vanilla RNNs, while the processes consist of Mess3 (classical), FRDN (quantum), Bloch Walk (quantum), and the Moon Proces...

    work page

Show all 45 references
  1. [9]

    The model uses multi-head attention with 4 heads (dimension 16 per head), 64-dimensional embeddings, and a 256-dimensional feed-forward network with ReLU activation

    Model Architectures For the Transformer architecture, we employ a 4-layer model implemented using the TransformerLens framework [22]. The model uses multi-head attention with 4 heads (dimension 16 per head), 64-dimensional embeddings, and a 256-dimensional feed-forward network...

  2. [10]

    All experiments use consistent random seeding (seed=42) for reproducibility

    Training Methodology Training data is generated from each stochastic process with the following parameters (see Appendix D for process definitions): • Mess3: a = 0.85, x = 0.05 • Bloch Walk: α = 1, β = √ 51 • FRDN: α = 2000, λ = 0.49 • Moon Process: α =e, β = 1/2 Each training...

    2000

  3. [11]

    Training is distributed across multiple GPUs, with specific GPU assignments managed through a parallel execution framework

    Implementation Details All experiments are implemented in PyTorch 2.0 with CUDA acceleration, using FP32 precision throughout. Training is distributed across multiple GPUs, with specific GPU assignments managed through a parallel execution framework. To ensure reproducibility,...

  4. [12]

    General Approach Our main analysis quantifies whether neural network activations encode belief states through an affine transformation of their internal activations. Given a neural activation vector⃗ aw∈ Rd (with e.g., activations from a particular position and a single layerd...

  5. [13]

    Deduplication Before regression, we identify and aggregate duplicate token prefixes

    Implementation details a. Deduplication Before regression, we identify and aggregate duplicate token prefixes. For each unique prefix, we retain the activation vector from its first occurrence, and then we sum the probabilities across all occurrences of the same prefix. This d...

  6. [14]

    Partition data into training (90%) and validation (10%) sets

  7. [15]

    Fit the weighted regression on the training set

  8. [16]

    Evaluate weighted error on the validation set:∑ ipi∥η(wi)− ˆη(wi)∥2 The r minimizing average validation error across folds is selected for the final model trained on all data. d. Evaluation Metrics To quantify how well the belief states were represented in network activations ...

  9. [17]

    Ground truth belief statesη(wi)

  10. [18]

    This provides a complementary view to the distance-based metrics, focusing on angular rather than Euclidean-distance relationships

    Predicted belief statesˆη(wi) from the regression By comparing these geometric relationships, we evaluate whether the linear probe preserves the relative orientations between belief vectors. This provides a complementary view to the distance-based metrics, focusing on angular ...

  11. [19]

    This control undergoes the same regression analysis, allowing us to quantify how much structure arises from training versus architecture

    Control Experiments To verify that the learned representations are not artifacts of the architecture alone, we compare against networks with randomly initialized weights (i.e., before any training (backpropagation) has happened). This control undergoes the same regression anal...

  12. [20]

    Please see the README.md for instructions on how to recreate all training and figure generation in this manuscript

    Code Repository The code is publicly available at https://github.com/adamimos/epsilon-transformers/tree/quantum-public. Please see the README.md for instructions on how to recreate all training and figure generation in this manuscript. The repository contains: • Training scrip...

  13. [21]

    Huggingface Dataset The complete dataset of trained model checkpoints and pre-computed analysis results is publicly available at SimplexAI/quantum-representations. The dataset contains: • 16 trained neural network models(4 architectures× 4 processes) • Pre-computed belief stat...

  14. [22]

    Note that losses in these files/links are reported normalized to the minimal possible loss given the process the network is trained on

    Training Details and Data For completeness, we include links to the exact commits of the codebase used during training of each experiment in this manuscript, as well as links to Weights & Biases for that training run, the training config, and the saved model checkpoints, in Ta...

  15. [23]

    Shai, Sarah E

    Adam S. Shai, Sarah E. Marzen, Lucas Teixeira, Alexander Gietelink Oldenziel, and Paul M. Riechers. Transformers represent belief state geometry in their residual stream.NeurIPS, arXiv:2405.15943, 2024. 1, 7, 8, 12, 16

    2024 arXiv

  16. [24]

    Riechers, Daniel Filan, and Adam S

    Mateusz Piotrowski, Paul M. Riechers, Daniel Filan, and Adam S. Shai. Constrained belief updates explain geometric structures in transformer representations.ICML, 2025. 1, 7, 13, 16

    2025

  17. [25]

    P. M. Riechers. Transforming metastable memories: The nonequilibrium thermodynamics of computation. In D. Wolpert, C. Kempes, P. Stadler, and J. Grochow, editors,The Energetics of Computing in Life and Machines, pages 353–380. SFI Press, 2019. 2

    2019

  18. [26]

    Cambridge university press, 2010

    Michael A Nielsen and Isaac L Chuang.Quantum computation and quantum information. Cambridge university press, 2010. 3

    2010

  19. [27]

    General probabilistic theories: An introduction.Physics Reports, 1033:1–64, 2023

    Martin Plávala. General probabilistic theories: An introduction.Physics Reports, 1033:1–64, 2023. 3

    2023

  20. [28]

    Quantum theory in finite dimension cannot explain every general process with finite memory.Communications in Mathematical Physics, 405(2):50, 2024

    Marco Fanizza, Josep Lumbreras, and Andreas Winter. Quantum theory in finite dimension cannot explain every general process with finite memory.Communications in Mathematical Physics, 405(2):50, 2024. 3, 5, 8, 19, 20

    2024

  21. [29]

    Quantum learning of classical stochastic processes: The completely positive realization problem

    Alex Monràs and Andreas Winter. Quantum learning of classical stochastic processes: The completely positive realization problem. Journal of Mathematical Physics, 57(1):015219, 01 2016. 3, 5, 6, 8 30 TABLE IV. Training resources for all models. Each hyperlink provides access to...

    2016

  22. [30]

    D. R. Upper. Theory and Algorithms for Hidden Markov Models and Generalized Hidden Markov Models. PhD thesis, University of California, Berkeley, 1997. Published by University Microfilms Intl, Ann Arbor, Michigan. 5, 19

    1997

  23. [31]

    A canonical form for weighted automata and applications to approximate minimization

    Borja Balle, Prakash Panangaden, and Doina Precup. A canonical form for weighted automata and applications to approximate minimization. In2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science, pages 701–712. IEEE, 2015. 5

    2015

  24. [32]

    P. M. Riechers and J. P. Crutchfield. Spectral simplicity of apparent complexity, Part I: The nondiagonalizable metadynamics of prediction. Chaos, 28:033115, 2018. 5

    2018

  25. [33]

    Quantum adaptive agents with efficient long-term memories

    Thomas J Elliott, Mile Gu, Andrew JP Garner, and Jayne Thompson. Quantum adaptive agents with efficient long-term memories. Physical Review X, 12(1):011007, 2022. 6

    2022

  26. [34]

    Quantum generation of stochastic processes: spectral invariants and memory bounds

    Magdalini Zonnios, Alexander Boyd, and Felix Binder. Quantum generation of stochastic processes: spectral invariants and memory bounds. New Journal of Physics, 2025. 6

    2025

  27. [35]

    Fundamentals of quantum mechanics in Liouville space.European Journal of Physics, 41(6):063002,

    Jerryman A Gyamfi. Fundamentals of quantum mechanics in Liouville space.European Journal of Physics, 41(6):063002,

  28. [36]

    Riechers, Chaitanya Gupta, Artemy Kolchinsky, and Mile Gu

    Paul M. Riechers, Chaitanya Gupta, Artemy Kolchinsky, and Mile Gu. Thermodynamically ideal quantum state inputs to any device.PRX Quantum, 5:030318, Jul 2024. 6, 14

    2024

  29. [37]

    Riechers and Thomas J

    Paul M. Riechers and Thomas J. Elliott. Identifiability and minimality bounds of quantum and post-quantum models of classical stochastic processes.arXiv, 2025. 6, 19

    2025

  30. [38]

    S. E. Marzen and J. P. Crutchfield. Nearly maximally predictive features and their dimensions.Phys. Rev. E, 95(5):051301(R),

  31. [39]

    Position: The platonic representation hypothesis

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The platonic representation hypothesis. In Forty-first International Conference on Machine Learning, 2024. 12

    2024

  32. [40]

    J. P. Crutchfield, C. J. Ellison, J. R. Mahoney, and R. G. James. Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation.CHAOS, 20(3):037105,

  33. [42]

    RNNs represent belief state geometry in their hidden states

    Keenan Pepper. RNNs represent belief state geometry in their hidden states. https://apartresearch.com/project/rnns- represent-belief-state-geometry-in-hidden-state, June 2024. Research submission to the Computational Mechanics Hackathon research sprint co-hosted by Apart, PIBB...

    2024

  34. [43]

    Riechers, Henry R

    Paul M. Riechers, Henry R. Bigelow, Eric A. Alt, and Adam S. Shai. Next-token pretraining implies in-context learning. arXiv:2505.18373, 2025. 13

    2025 arXiv

  35. [44]

    Jakóbczyk and M

    L. Jakóbczyk and M. Siennicki. Geometry of Bloch vectors in two-qubit system.Physics Letters A, 286(6):383–390, 2001. 14

    2001

  36. [45]

    Transformerlens.https://github.com/TransformerLensOrg/TransformerLens, 2022

    Neel Nanda and Joseph Bloom. Transformerlens.https://github.com/TransformerLensOrg/TransformerLens, 2022. 20

    2022

  37. [2010]

    Santa Fe Institute Working Paper 10-08-015; arxiv.org:1007.5354 [cond-mat.stat-mech]. 12

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.