Pith. sign in

REVIEW 3 major objections 5 minor 79 references

This paper claims that neural volatility forecasters trained only on squared returns internally reconstruct the next latent log-volatility state, and that in long-cycle regimes this reconstruction is an explicit linear projection followed b

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 02:20 UTC pith:CPRJH4NP

load-bearing objection Useful benchmark and a clear stage-wise finding, but the central two-stage claim rests on an extreme persistence regime where the latent state is almost directly observable; at ρ=0.9999 the effect mostly disappears. the 3 major comments →

arxiv 2607.25459 v1 pith:CPRJH4NP submitted 2026-07-28 cs.LG cs.AIq-fin.ST

Emergent Latent-State Computation under Stochastic Volatility

Xiaoyu Huang , Lulu Wang This is my paper
classification cs.LG cs.AIq-fin.ST
keywords stochastic volatilitymechanistic interpretabilitylatent-state representationlinear probingTransformerstime-series forecastingpartial observabilityvolatility filtering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether a neural network that only ever sees asset returns nevertheless builds an internal model of the hidden volatility process that generated them. In a controlled multivariate stochastic-volatility simulation, the authors train Transformers and MLPs to predict next-step squared returns and then probe their hidden layers for the ground-truth next latent log-volatility state. They find a consistent two-stage computation: intermediate representations encode substantial information about the next latent state, and the output head converts that representation into a squared-return forecast. In long-cycle regimes, they isolate the first stage as a simple learned filter—a linear projection followed by ℓ2 normalization—and show by causal perturbation that this stage carries the latent-state signal. If the claim holds, it means mechanistic interpretability can be extended from deterministic language-model tasks to noisy, partially observed latent dynamics, and that at least in simulation, volatility forecasters are doing implicit state filtering rather than shallow curve fitting.

Core claim

The paper's central discovery is that, in the simulated multivariate stochastic-volatility setting, sequence models trained to forecast squared returns internally solve a latent-state inference problem first and map the inferred state to the forecast only afterwards. Linear ridge probes recover the next latent log-volatility h_{T+1} from the final hidden representation with median R² around 0.9 across architectures, activation functions, and both MSE and negative-log-likelihood losses. Under MSE training, the learned output head is less aligned with h_{T+1} than the representation itself is, and replacing the head with a probe-aligned readout consistently improves mean absolute error; the pa

What carries the argument

The central object is the explicit latent-state filter in Equation (3): a learned linear embedding x_i W_in^T, ℓ2 normalization, and a probe readout B. For long volatility cycles, this composition alone extracts most of the decodable information about h_{T+1}, and causal perturbation of the post-normalization representation produces the largest downstream collapse in both decodability and forecast quality. The supporting measurement device is the linear ridge probe, which gives a stage-by-stage map of where h_{T+1} becomes linearly recoverable.

Load-bearing premise

The central claim rests on making the hidden volatility process almost perfectly persistent (the largest eigenvalue of the transition matrix is 0.999999), so the next latent state is nearly identical to the current one and can be estimated almost directly from the last squared return; the paper's own persistence sweep shows probe R² drops from about 0.9 to 0.29 when the largest eigenvalue is reduced to 0.9999.

What would settle it

Re-run the identical probe and intervention suite at a spectral radius of 0.9999, or on data where the last return is made uninformative about the next latent state; if the two-stage decomposition and the explicit long-cycle filter still yield high decodability and causally relevant post-ℓ2 representations, the mechanism is genuine filtering, whereas a collapse would show the headline result is an artifact of near-frozen dynamics.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Forecast quality in these models is governed mainly by how faithfully the hidden representation tracks the latent volatility state; the final mapping is secondary.
  • In long-cycle regimes the forecast becomes directly inspectable: it is a readout of a normalized linear projection of the return window, so the implicit volatility estimate can be computed by hand.
  • Part of the performance loss under noisy MSE training is recoverable by realigning the output head to the latent-state direction, with consistent MAE gains across volatility periods.
  • The architectural stage that carries the latent state is not fixed: it depends on the period of the latent dynamics, so claims about where a model computes must be conditioned on the data-generating process.
  • Under Gaussian negative-log-likelihood training, the model has a formal incentive to predict exactly exp(h_{T+1}), and the probe results confirm that it does.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the near-unit-root simulation design may be inflating the probe R² values; at spectral radius 0.999999, h_{T+1} ≈ h_T, so the 'filter' could largely be a copy of the current state. A decisive test is to rerun the probe suite with shorter-memory latent dynamics and see whether the two-stage structure survives.
  • Our inference: the explicit filter suggests a practical interpretability check for real volatility data—train a small model on daily returns and compare the learned embedding direction to the log of recent squared returns. If the filter is general, the dominant direction should track the most recent squared return.
  • Our inference: the probe–readout gap points to a cheap training intervention—adding a latent-state alignment term to the loss may stabilize forecasts under MSE without architectural changes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies mechanistic interpretability of small Transformers and MLPs trained to forecast next-step squared returns in a simulated multivariate stochastic volatility (MSV) model. The latent log-volatility state h_t is known to the researcher but hidden from the model. The authors report a two-stage computation: hidden representations encode substantial information about h_{T+1}, and the output head maps this representation to squared-return forecasts. They further report stage-wise emergence of latent-state decodability in Transformers, depending on the volatility period, and, in long-cycle regimes, an explicit latent-state filter consisting of a learned linear projection followed by ℓ2 normalization (Eq. 3). The evidence comes from ridge-probe R², learned-readout R², output-head replacement, stage-wise probing, and activation ablations. The central claim is that the model performs an implicit filtering computation whose latent-state estimate is localizable and explicit in long-cycle regimes.

Significance. If the central claim held, the paper would provide a useful controlled benchmark for mechanistic interpretability under partial observability, with a known ground-truth latent state, a clean identification of architectural stage, and a concrete filter form. The manuscript has real strengths: the setting is well motivated, the latent state is indeed observable to the researcher, the code is released, and the combination of probing with causal ablations is appropriate. However, the current evidence is consistent with a much simpler explanation: at the main calibration ρ=0.999999, the latent process is so persistent that h_{T+1} is almost directly readable from the observed squared returns, so the high probe R² values may not reflect a learned filtering computation at all. Because this concern is quantitative and is supported by the paper's own Table 3, the significance of the findings is not yet established.

major comments (3)
  1. [Section 2, Section G, Table 3] The main specification sets ρ=0.999999 with ση=0.02. Then the stationary variance of h_t is ση²/(1−ρ²)≈200, while Var(log ε²)=4.93 for the observation log r_t²=h_t+log ε_t². Thus h_T, and hence h_{T+1}≈h_T, is almost directly observable from the last squared return; a trivial last-observation linear predictor attains R²≈200/(200+4.93)≈0.98. The paper's own Table 3 shows that at ρ=0.9999, where the stationary variance is only ≈2, the probe R² falls to 0.286—almost exactly the trivial predictor value ≈0.29—and at ρ=0.999999 the probe R² is 0.901, below the trivial baseline. This indicates that the high decodability in Table 1 may simply reflect observability of a near-unit-root state, not a learned filtering computation. Please add a last-observation baseline (or a full-window log-squared-return baseline) and evaluate at ρ values where filtering is nontrivial. Without such controls the cen
  2. [Section 3.3, Eq. (3)] The 'explicit latent-state filter' in Eq. (3) is not a fully learned model mechanism. The term B is a ridge-probe readout fit to h_{T+1} after training; the only learned component in the filter is W_in followed by ℓ2 normalization. Unless B is derived from the model weights, Eq. (3) describes a post-hoc diagnostic applied to the normalized embedding, not a filter implemented by the network. The ablation in Table 4 shows that W_in and normalization are jointly necessary for probe decodability, but it does not show that the model computes ĥ. Please either derive B from the model or revise the claim to 'the normalized embedding is linearly readable' rather than 'the model implements an explicit latent-state filter.'
  3. [Section H] The causal perturbation removes the component of the post-ℓ2-norm representation aligned with the ridge-probe direction. Because this representation is the input to attention and all subsequent layers, ablating any direction at this stage is likely to degrade downstream computation, even if the model does not use the probed direction. The paper does not report a control ablation in a random direction or with a probe trained on an untrained model. Thus the conclusion that the latent-state direction is 'causally relevant' is not uniquely supported. Please add such controls; otherwise the intervention evidence is consistent with a generic sensitivity of early representations.
minor comments (5)
  1. [Section 2] The sentence 'Full architectural details for both models are provided in Section A' appears twice in succession. Please remove the duplicate.
  2. [Section 3.3] Typo: 'Further details are descried' should be 'described'.
  3. [Eq. (3)] The underbrace notation in Eq. (3) is confusing: it places B outside the ℓ2-normalized term but B is described as part of the filter. Please clarify that B is a probe readout, not a model parameter.
  4. [References] Jacquier et al. (2002a) and Jacquier et al. (2002b) refer to the same paper with the same title and journal; please merge them into a single reference.
  5. [Section G / Table 3] Table 3 reports only mean R² without IQRs, while other tables report median [IQR]. For consistency and to assess variability, please report the same summary statistics.

Circularity Check

1 steps flagged

The 'explicit latent-state filter' in Eq. (3) is a ridge probe fitted to h_{T+1} presented as an emergent mechanism; the two-stage evidence itself is independent, while the near-unit-root calibration is a confounding risk rather than circularity.

specific steps
  1. fitted input called prediction [Section 3.3, Eq. (3); Section 3.1 'Linear probing']
    "Thus, most latent-state information is extracted by a single nonlinear normalization step applied after a learned linear projection: ĥ_{i,t+1} = x_i W_in^T / ||x_i W_in^T||_2 B, where W_in is the learned linear embedding and B is the probe readout. We refer to this composition as an explicit latent-state filter. ... We fit a linear ridge-probe [Alain and Bengio, 2016] from each asset representation z_i to the corresponding MSV model's latent state h_{i,T+1}."

    B is the ridge probe fitted to the target h_{i,T+1} on the same data, so the output of Eq. (3) is by construction the best linear estimate of that target from the ℓ²-normalized embedding. Its high R² (~0.9) measures the probe's fit, not a component in the trained model: the model's forward pass contains W_in and the output head, but no B. Presenting this fitted composition as 'the Transformer's latent-state computation simplifies into an explicit filter' renames a diagnostic readout as an emergent mechanism. The Section H causal perturbation also measures decodability through this same fitted probe, so it inherits the same fitted-readout construction and cannot separate representation quality from probe fit.

full rationale

The core two-stage claim is not circular: models are trained only on squared-return targets and the probe target h_{T+1} is an external ground truth from the simulator, so high linear decodability is meaningful evidence about the trained representation. The stage-wise emergence results (Section 3.2) are also properties of the trained network, not of the probe readout. The one genuinely circular construction is Eq. (3) / H3: the 'explicit latent-state filter' includes the ridge probe B that was fit to h_{T+1}, so the claim that the model simplifies to this filter is partly a renaming of the probe's least-squares fit. I do not treat the near-unit-root calibration (rho=0.999999, sigma_eta=0.02) as circularity; it is a serious robustness/confounding concern, and the paper's own Table 3 (R^2 drops to 0.286 at rho=0.9999) shows the main evidence is brittle, but this is a calibration-sensitivity problem rather than a definitional reduction. There is no load-bearing self-citation chain; Huang et al. (2025) is mentioned only as related perspective. Overall: one central sub-claim is partly forced by a fitted diagnostic, while the main empirical decomposition remains independent.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claims rest on the simulation's persistence regime (ρ=0.999999, ση=0.02), the hand-set cyclic construction of A, the probe-based definition of 'encoding', and the conflation of MSE-optimal forecasts with the true latent state. No new physical or mathematical entities are introduced; the 'explicit latent filter' is a description of existing weights plus a fitted probe, not an invented entity.

free parameters (4)
  • spectral radius rho = 0.999999
    Scales the spillover matrix A; controls persistence and thus how recoverable the latent state is. Table 3 shows probe R2 drops from 0.90 to 0.29 when rho is 0.9999, making this the single most load-bearing simulation choice.
  • innovation standard deviation sigma_eta = 0.02
    Sets the noise level of the latent process; with rho near 1, the latent increments are tiny, making h nearly constant over the observation window.
  • volatility period omega = 21,30,90,178,365,730
    Controls the cyclic structure of the spillover matrix; the stage-wise emergence claim is explicitly conditional on omega.
  • ridge probe readout B = fit to data
    The 'explicit latent filter' in Eq. (3) includes a ridge-probe readout B fit to h_{T+1} from the normalized embedding; the filter description is therefore partly a fitted diagnostic, not a parameter-free derivation.
axioms (4)
  • domain assumption Observation model r_t = exp(h_t/2) ⊙ ε_t, ε_t ~ N(0, Σ_ε) (Eq. 1).
    The benchmark's ground-truth generative process; all probe targets and forecast targets are defined with respect to this assumed model.
  • ad hoc to paper Latent state follows h_{t+1} = μ + A(h_t − μ) + η_t with A = Q D(ω) Q^T and spectral radius ρ = 0.999999 (Eq. 2).
    The block-rotation construction for cyclic spillovers is introduced for this study; the near-unit-root scaling is a hand-set regime that the main claims depend on.
  • domain assumption Linear probe R² measures the information the model's computation 'uses' or 'encodes'.
    The two-stage and stage-wise claims are based primarily on ridge-probe decodability; the paper acknowledges in Section J that probes alone do not establish a circuit-level mechanism.
  • domain assumption Under MSE training, the optimal forecast E[r²_{T+1} | past] is treated as functionally equivalent to the true latent state exp(h_{T+1}).
    The NLL optimum equals exp(h_{T+1}) (Theorem C), but MSE's optimum is a filtered estimate; the paper's framing slides between these in the readout-misalignment analysis.

pith-pipeline@v1.3.0-alltime-deepseek · 12101 in / 15213 out tokens · 152166 ms · 2026-08-01T02:20:35.740561+00:00 · methodology

0 comments
read the original abstract

Mechanistic interpretability has largely focused on language models and deterministic toy tasks. Much less is known about how sequence models internally represent latent stochastic dynamics under noisy, partially observed observations. We study this question in a controlled multivariate stochastic volatility setting, where models observe only returns while the ground-truth latent volatility state is known to the researcher. This setting provides a useful benchmark for mechanistic interpretability under partial observability: the latent state is hidden from the model but directly available for evaluation. Across architectures, losses, and output heads, we find evidence for a two-stage computation. Hidden representations encode substantial information about the next latent volatility state, and the output head maps this representation to squared return forecasts. Furthermore, in Transformers, latent-state decodability emerges at identifiable architectural stages whose location depends on the volatility period. In long-cycle regimes, this computation simplifies into an explicit latent-state filter consisting of a learned linear projection followed by $\ell^2$ normalization. Output-head replacement further shows that part of the degradation under noisy MSE training arises from readout misalignment rather than representation failure. These results suggest that stochastic volatility models provide a useful benchmark for mechanistic interpretability under noisy latent dynamics and partial observability.

Figures

Figures reproduced from arXiv: 2607.25459 by Lulu Wang, Xiaoyu Huang.

Figure 1
Figure 1. Figure 1: Stage-wise emergence and causal relevance of latent-state decodability in the Transformer. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Probe R2 and linear head R2 for decoding hT +1 from the final representation under MSE training, shown across volatility periods for the Transformer and MLP (median over n = 10 seeds). In both architectures, probe R2 remains above head R2 across all periods, with smaller in long-cycle regimes. The R2 in Transformer are consistently higher. E Less Structured Emergence in the MLP In the Transformer, the stag… view at source ↗
Figure 3
Figure 3. Figure 3: Stage-wise ridge probe R2 for decoding hT +1 from intermediate MLP representations across volatility periods. Unlike the Transformer, the MLP exhibits a smoother and less localized rise in decodability, with weaker dependence on volatility period. We also consider an MLP control with an ℓ 2 -normalization layer inserted after the initial linear embedding, mirroring the corresponding preprocessing step in t… view at source ↗
Figure 4
Figure 4. Figure 4: Stage-wise ridge probe R2 for decoding hT +1 in an MLP with ℓ 2 normalization after the initial linear layer. F Stage-Wise Results for Two-Layer Transformers We also examine whether a deeper Transformer exhibits a different stage-wise emergence pattern. Fig￾ure 5 shows that the main decodability transitions still occur in the first layer. In our setting, increasing depth from one layer to two does not subs… view at source ↗
Figure 5
Figure 5. Figure 5: Stage-wise ridge probe R2 for decoding hT +1 from intermediate representations in a two-layer Transformer across volatility periods. The main gains in decodability occur before or within the first layer, while the second layer yields only limited additional improvement. G Impact of Signal-to-Noise Ratio The spectral radius ρ of the state-transition matrix A controls the persistence of the latent process ht… view at source ↗
Figure 6
Figure 6. Figure 6: Representative latent trajectories for different spectral radii [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Forecast error change (∆MSE and ∆MAE, ablated minus baseline) as a function of ablation strength α at ω=365, one curve per intervention stage (mean ± std, n=10 seeds). Intervening at after ℓ 2 -norm causes the largest degradation; later stages other than positional embedding have negligible effect. I Related Work Probing and causal interventions. Our stage-wise analysis builds on linear probing methods, wh… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 14 linked inside Pith

  1. [1]

    arXiv preprint arXiv:1606.08415 , year=

    Gaussian error linear units (gelus) , author=. arXiv preprint arXiv:1606.08415 , year=

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:2501.16496 , year=

    Open problems in mechanistic interpretability , author=. arXiv preprint arXiv:2501.16496 , year=

    Pith/arXiv arXiv

  3. [3]

    arXiv preprint arXiv:2211.00593 , year=

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small , author=. arXiv preprint arXiv:2211.00593 , year=

    Pith/arXiv arXiv

  4. [4]

    2023 , journal=

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author=. 2023 , journal=

    2023

  5. [5]

    arXiv preprint arXiv:2301.05217 , year=

    Progress measures for grokking via mechanistic interpretability , author=. arXiv preprint arXiv:2301.05217 , year=

    Pith/arXiv arXiv

  6. [6]

    International Conference on Machine Learning , pages=

    A toy model of universality: Reverse engineering how networks learn group operations , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  7. [7]

    arXiv preprint arXiv:2511.12421 , year=

    From Black Box to Bijection: Interpreting Machine Learning to Build a Zeta Map Algorithm , author=. arXiv preprint arXiv:2511.12421 , year=

    arXiv

  8. [8]

    Advances in neural information processing systems , volume=

    The clock and the pizza: Two stories in mechanistic explanation of neural networks , author=. Advances in neural information processing systems , volume=

  9. [9]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  10. [10]

    Scandinavian Journal of Statistics , pages=

    Statistical analysis of time series: Some recent developments [with discussion and reply] , author=. Scandinavian Journal of Statistics , pages=. 1981 , publisher=

    1981

  11. [11]

    Time series models , pages=

    Statistical aspects of ARCH and stochastic volatility , author=. Time series models , pages=. 1996 , publisher=

    1996

  12. [12]

    1986 , publisher=

    Modelling financial time series , author=. 1986 , publisher=

    1986

  13. [13]

    Journal of econometrics , volume=

    Generalized autoregressive conditional heteroskedasticity , author=. Journal of econometrics , volume=. 1986 , publisher=

    1986

  14. [14]

    Econometrica: Journal of the econometric society , pages=

    Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , author=. Econometrica: Journal of the econometric society , pages=. 1982 , publisher=

    1982

  15. [15]

    1996 , booktitle =

    5 Stochastic volatility , series =. 1996 , booktitle =

    1996

  16. [16]

    The review of economic studies , volume=

    Stochastic volatility: likelihood inference and comparison with ARCH models , author=. The review of economic studies , volume=. 1998 , publisher=

    1998

  17. [17]

    Journal of Business & Economic Statistics , volume=

    Bayesian analysis of stochastic volatility models , author=. Journal of Business & Economic Statistics , volume=. 2002 , publisher=

    2002

  18. [18]

    arXiv preprint arXiv:2202.07125 , year=

    Transformers in time series: A survey , author=. arXiv preprint arXiv:2202.07125 , year=

    Pith/arXiv arXiv

  19. [19]

    Computer Science Review , volume=

    A survey of transformer networks for time series forecasting , author=. Computer Science Review , volume=. 2026 , publisher=

    2026

  20. [20]

    International journal of forecasting , volume=

    Temporal fusion transformers for interpretable multi-horizon time series forecasting , author=. International journal of forecasting , volume=. 2021 , publisher=

    2021

  21. [21]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Informer: Beyond efficient transformer for long sequence time-series forecasting , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  22. [22]

    Journal of financial economics , volume=

    Common risk factors in the returns on stocks and bonds , author=. Journal of financial economics , volume=. 1993 , publisher=

    1993

  23. [23]

    Econometrica , volume=

    Modeling and forecasting realized volatility , author=. Econometrica , volume=. 2003 , publisher=

    2003

  24. [24]

    Journal of Financial Econometrics , volume=

    Identification of global and local shocks in international financial markets via general dynamic factor models , author=. Journal of Financial Econometrics , volume=. 2019 , publisher=

    2019

  25. [25]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Master: Market-guided stock transformer for stock price forecasting , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  26. [26]

    arXiv preprint arXiv:2304.04912 , year=

    Financial time series forecasting using CNN and transformer , author=. arXiv preprint arXiv:2304.04912 , year=

    Pith/arXiv arXiv

  27. [27]

    Proceedings of the web conference 2020 , pages=

    Html: Hierarchical transformer-based multi-task learning for volatility prediction , author=. Proceedings of the web conference 2020 , pages=

    2020

  28. [28]

    Journal of financial econometrics , volume=

    Persistence and kurtosis in GARCH and stochastic volatility models , author=. Journal of financial econometrics , volume=. 2004 , publisher=

    2004

  29. [29]

    Journal of Economic surveys , volume=

    Estimation methods for stochastic volatility models: a survey , author=. Journal of Economic surveys , volume=. 2004 , publisher=

    2004

  30. [30]

    Journal of Econometrics , volume=

    Analysis of high dimensional multivariate stochastic volatility models , author=. Journal of Econometrics , volume=. 2006 , publisher=

    2006

  31. [31]

    2012 , publisher=

    Time series analysis by state space methods , author=. 2012 , publisher=

    2012

  32. [32]

    Applications in Economics and Finance , volume=

    State-space models , author=. Applications in Economics and Finance , volume=. 2013 , publisher=

    2013

  33. [33]

    Bayesian Inference of State Space Models: Kalman Filtering and Beyond , pages=

    The state space model in finance , author=. Bayesian Inference of State Space Models: Kalman Filtering and Beyond , pages=. 2021 , publisher=

    2021

  34. [34]

    The Review of Financial Studies , volume=

    Empirical asset pricing via machine learning , author=. The Review of Financial Studies , volume=. 2020 , publisher=

    2020

  35. [35]

    Management Science , volume=

    Deep learning in asset pricing , author=. Management Science , volume=. 2024 , publisher=

    2024

  36. [36]

    European journal of operational research , volume=

    Deep learning with long short-term memory networks for financial market predictions , author=. European journal of operational research , volume=. 2018 , publisher=

    2018

  37. [37]

    Advances in neural information processing systems , volume=

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting , author=. Advances in neural information processing systems , volume=

  38. [38]

    Advances in neural information processing systems , volume=

    Non-stationary transformers: Exploring the stationarity in time series forecasting , author=. Advances in neural information processing systems , volume=

  39. [39]

    arXiv preprint arXiv:2505.24650 , year=

    Beyond the black box: Interpretability of llms in finance , author=. arXiv preprint arXiv:2505.24650 , year=

    Pith/arXiv arXiv

  40. [40]

    Attention is not not explanation , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=

    2019

  41. [41]

    Distill , year=

    Zoom In: An Introduction to Circuits , author=. Distill , year=

  42. [42]

    Transformer Circuits Thread , year=

    A Mathematical Framework for Transformer Circuits , author=. Transformer Circuits Thread , year=

  43. [43]

    Journal of Financial Econometrics , volume=

    A machine learning approach to volatility forecasting , author=. Journal of Financial Econometrics , volume=. 2023 , publisher=

    2023

  44. [44]

    arXiv preprint arXiv:1511.05121 , year=

    Deep Kalman Filters , author=. arXiv preprint arXiv:1511.05121 , year=

    Pith/arXiv arXiv

  45. [45]

    ICLR , year=

    Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data , author=. ICLR , year=

  46. [46]

    NeurIPS , year=

    Sequential Neural Models with Stochastic Layers , author=. NeurIPS , year=

  47. [47]

    NeurIPS , year=

    Deep State Space Models for Time Series Forecasting , author=. NeurIPS , year=

  48. [48]

    Journal of Financial Econometrics , volume=

    Volatility forecasting with machine learning and intraday commonality , author=. Journal of Financial Econometrics , volume=. 2024 , publisher=

    2024

  49. [49]

    ACL , year=

    What Does BERT Look at? An Analysis of BERT's Attention , author=. ACL , year=

  50. [50]

    ACL Workshop on BlackboxNLP , year=

    A Multiscale Visualization of Attention in the Transformer Model , author=. ACL Workshop on BlackboxNLP , year=

  51. [51]

    Transactions of the Association for Computational Linguistics , volume=

    A Primer in BERTology: What We Know About How BERT Works , author=. Transactions of the Association for Computational Linguistics , volume=

  52. [52]

    arXiv preprint arXiv:2502.10357 , year=

    Learning euler factors of elliptic curves , author=. arXiv preprint arXiv:2502.10357 , year=

    Pith/arXiv arXiv

  53. [53]

    International Joint Conference on Artificial Intelligence , year=

    A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction , author=. International Joint Conference on Artificial Intelligence , year=

  54. [54]

    arXiv preprint arXiv:1605.06432 , year=

    Deep variational bayes filters: Unsupervised learning of state space models from raw data , author=. arXiv preprint arXiv:1605.06432 , year=

    Pith/arXiv arXiv

  55. [55]

    Nature communications , volume=

    Deep learning for universal linear embeddings of nonlinear dynamics , author=. Nature communications , volume=. 2018 , publisher=

    2018

  56. [56]

    International Conference on Machine Learning , pages=

    Transformers learn in-context by gradient descent , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  57. [57]

    International conference on machine learning , pages=

    Transformers as algorithms: Generalization and stability in in-context learning , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  58. [58]

    Advances in Neural Information Processing Systems , volume=

    Transformers are uninterpretable with myopic methods: a case study with bounded dyck grammars , author=. Advances in Neural Information Processing Systems , volume=

  59. [59]

    Attention is not explanation , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

  60. [60]

    The Review of Economic Studies , volume=

    Multivariate stochastic variance models , author=. The Review of Economic Studies , volume=. 1994 , publisher=

    1994

  61. [61]

    Handbook of financial time series , pages=

    Multivariate stochastic volatility , author=. Handbook of financial time series , pages=. 2009 , publisher=

    2009

  62. [62]

    2020 , howpublished=

    Zoom In: An Introduction to Circuits , author=. 2020 , howpublished=

    2020

  63. [63]

    2021 , howpublished=

    A Mathematical Framework for Transformer Circuits , author=. 2021 , howpublished=

    2021

  64. [64]

    2022 , howpublished=

    In-context Learning and Induction Heads , author=. 2022 , howpublished=

    2022

  65. [65]

    arXiv preprint arXiv:1610.01644 , year=

    Understanding Intermediate Layers Using Linear Classifier Probes , author=. arXiv preprint arXiv:1610.01644 , year=

    Pith/arXiv arXiv

  66. [66]

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing , year=

    Designing and Interpreting Probes with Control Tasks , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing , year=

    2019

  67. [67]

    Computational Linguistics , volume=

    Probing Classifiers: Promises, Shortcomings, and Advances , author=. Computational Linguistics , volume=

  68. [68]

    2022 , howpublished=

    Causal Scrubbing: A Method for Rigorously Testing Interpretability Hypotheses , author=. 2022 , howpublished=

    2022

  69. [69]

    Proceedings of Machine Learning Research , year=

    Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations , author=. Proceedings of Machine Learning Research , year=

  70. [70]

    Journal of Machine Learning Research , year=

    Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability , author=. Journal of Machine Learning Research , year=

  71. [71]

    arXiv preprint arXiv:2309.16042 , year=

    Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , author=. arXiv preprint arXiv:2309.16042 , year=

    Pith/arXiv arXiv

  72. [72]

    International Conference on Learning Representations , year=

    A Time Series is Worth 64 Words: Long-term Forecasting with Transformers , author=. International Conference on Learning Representations , year=

  73. [73]

    Journal of Computational and Graphical Statistics , volume=

    Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , author=. Journal of Computational and Graphical Statistics , volume=. 1996 , publisher=

    1996

  74. [74]

    Advances in neural information processing systems , volume=

    Locating and editing factual associations in gpt , author=. Advances in neural information processing systems , volume=

  75. [75]

    arXiv preprint arXiv:2404.15255 , year=

    How to use and interpret activation patching , author=. arXiv preprint arXiv:2404.15255 , year=

    Pith/arXiv arXiv

  76. [76]

    arXiv preprint arXiv:2306.03819 , year=

    LEACE: Perfect Linear Concept Erasure in Closed Form , author=. arXiv preprint arXiv:2306.03819 , year=

    Pith/arXiv arXiv

  77. [77]

    Advances in neural information processing systems , volume=

    Investigating gender bias in language models using causal mediation analysis , author=. Advances in neural information processing systems , volume=

  78. [78]

    Proceedings of 1994 ieee international conference on neural networks (ICNN'94) , volume=

    Estimating the mean and variance of the target probability distribution , author=. Proceedings of 1994 ieee international conference on neural networks (ICNN'94) , volume=. 1994 , organization=

    1994

  79. [79]

    International Conference on Learning Representations , volume=

    Can transformers do enumerative geometry? , author=. International Conference on Learning Representations , volume=