pith. sign in

arxiv: 2604.17251 · v1 · submitted 2026-04-19 · 💻 cs.CE

ORCA -- Online Regime Correlation Analyzer

Boris Kriuk , Fedor Kriuk This is my paper

Pith reviewed 2026-05-10 06:04 UTC · model grok-4.3

classification 💻 cs.CE
keywords financial regime detectionspectral graph theorycorrelation networkscrisis predictionrandom forestmarket microstructuregraph topology
0
0 comments X

The pith

Spectral features from asset correlation networks improve forecasts of market rallies and crashes over ten-day horizons.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces ORCA to construct rolling correlation matrices from 24 instruments at three time scales and extract 127 spectral and graph-topological features that capture network structure beyond scalar volatility. These features are combined with 79 traditional indicators and fed into a depth-limited random forest trained under strict walk-forward validation with anti-leakage gaps. A sympathetic reader would care because standard risk models discard the topological information in cross-asset dependence, while ORCA shows that eigenvalue entropy, clustering coefficients, and related descriptors add measurable predictive power for regime shifts.

Core claim

ORCA fuses spectral graph theory, random matrix theory, and supervised learning on multi-scale correlation matrices to produce calibrated probabilities for both rally and crash events, achieving a balanced crisis detection AUC of 0.741 that ranks first against baselines, with spectral features contributing an additional 10.3 percentage points to crash AUC and 5.2 to rally AUC.

What carries the argument

The 127-dimensional spectral feature set (absorption ratios, eigenvalue entropy, effective rank, spectral gap, eigenvector concentration, and graph-topological descriptors such as clustering coefficient and edge density at multiple correlation thresholds) extracted from rolling correlation matrices of 24 instruments.

Load-bearing premise

The chosen 24 instruments, three time-scale estimators, 127 features, and correlation thresholds were selected without information leakage into the eight-fold walk-forward evaluation on the fifteen-year US equity sample.

What would settle it

A material decline in out-of-sample balanced AUC when the trained model is applied to daily data from 2024 onward or to equity markets outside the original US sample.

Figures

Figures reproduced from arXiv: 2604.17251 by Boris Kriuk, Fedor Kriuk.

Figure 1
Figure 1. Figure 1: ORCA architecture. Solid arrows show primary data [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Aggregated SHAP importance by feature family. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: SHAP feature importance for rally (left) and crash (right) detection. The crash model is led by graph-topological features [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: SHAP dependence plots for the six most important features. Left column: rally detection; right column: crash detection. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Live demonstration of regime-dependent portfolio [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
read the original abstract

Standard risk models reduce the rich dependence structure of financial markets to scalar volatility estimates, discarding the topological information encoded in cross-asset correlation networks. We present ORCA (Online Regime Correlation Analyzer), an end-to-end framework that fuses spectral graph theory, random matrix theory, and supervised machine learning to deliver calibrated probability estimates for both rally and crash events over a ten-day forward horizon. ORCA constructs rolling correlation matrices from 24 diversified exchange-traded instruments using three parallel estimators at different time scales, and extracts 127 spectral features (absorption ratios, eigenvalue entropy, effective rank, spectral gap, eigenvector concentration, and graph-topological descriptors at multiple correlation thresholds), concatenated with 79 traditional price-derived indicators to form a 206-dimensional feature vector. A depth-limited Random Forest with balanced sub-sample weighting is evaluated under a strict eight-fold walk-forward protocol with ten-day anti-leakage gaps spanning fifteen years of daily US market data. ORCA achieves a Balanced Crisis Detection AUC (BCD-AUC, the geometric mean of rally and crash AUC) of 0.741, ranking first against all baselines. Ablation studies show that spectral features contribute +10.3 percentage points of AUC for crash detection and +5.2 for rally detection over traditional features alone, with SHAP analysis revealing that graph-topological descriptors (clustering coefficient, edge density, and dominant-eigenvalue percentile rank) are the three most important crash predictors. A backtested walk-forward strategy mapping the joint rally-crash signal to dynamic equity exposure with risk-on/risk-off rotation achieves a Sharpe ratio of 1.13, a CAGR of 15.6%, and a maximum drawdown of only -7.5%, versus 3.7% CAGR and -33.7% drawdown for buy-and-hold.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces ORCA, a framework that builds rolling correlation matrices from 24 instruments using three parallel time-scale estimators, extracts 127 spectral features (absorption ratios, eigenvalue entropy, effective rank, spectral gap, eigenvector concentration, graph descriptors at multiple thresholds) plus 79 traditional indicators, and feeds the 206-dimensional vector to a depth-limited Random Forest with balanced weighting. Under an eight-fold walk-forward protocol with 10-day anti-leakage gaps on 15 years of US daily data, it reports a BCD-AUC (geometric mean of rally and crash AUC) of 0.741 that ranks first versus baselines, ablation gains of +10.3 pp crash AUC and +5.2 pp rally AUC from spectral features, SHAP-highlighted graph-topological predictors, and a backtested risk-on/risk-off strategy with Sharpe 1.13, 15.6% CAGR, and -7.5% max drawdown versus buy-and-hold.

Significance. If the feature, threshold, and hyperparameter selections were locked without using the evaluation folds, the result would be significant: it supplies concrete evidence that topological descriptors from correlation networks add predictive value beyond scalar volatility or price indicators for regime detection, supported by ablations, SHAP interpretability, and a realistic walk-forward trading backtest. The anti-leakage gaps and multi-scale construction are methodologically sound elements that strengthen the contribution to computational finance and risk modeling.

major comments (2)
  1. [Abstract] Abstract and methods description of feature construction: the 24 instruments, three time-scale estimators, 127 spectral features, and correlation thresholds are presented as fixed inputs to the 206-dimensional vector, yet no protocol is stated showing these choices were determined solely from pre-sample data or the first fold and then locked for the remaining walk-forward evaluation. If any optimization occurred against the full 15-year period, the reported BCD-AUC of 0.741 and the ablation deltas become contaminated by selection bias.
  2. [Evaluation protocol] Evaluation protocol paragraph: although the eight-fold walk-forward with ten-day gaps is described as strict, the manuscript provides no explicit statement that Random Forest depth, sub-sample weighting, and any implicit feature-threshold tuning were performed inside each training fold only. Global selection against the reported metric would create circularity, directly undermining the claim that spectral features deliver genuine out-of-sample regime detection.
minor comments (2)
  1. [Abstract] The exact list or categories of the 24 exchange-traded instruments is omitted; adding this information (or a table) would improve reproducibility without altering the central claims.
  2. [Evaluation] The definition of BCD-AUC as the geometric mean of rally and crash AUC is given only in the abstract; repeating the formula in the main evaluation section would aid clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful and constructive review. The comments correctly identify areas where the manuscript would benefit from greater explicitness regarding the selection and locking of methodological choices. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract and methods description of feature construction: the 24 instruments, three time-scale estimators, 127 spectral features, and correlation thresholds are presented as fixed inputs to the 206-dimensional vector, yet no protocol is stated showing these choices were determined solely from pre-sample data or the first fold and then locked for the remaining walk-forward evaluation. If any optimization occurred against the full 15-year period, the reported BCD-AUC of 0.741 and the ablation deltas become contaminated by selection bias.

    Authors: We agree that the manuscript does not contain an explicit statement of the protocol used to fix these inputs. The 24 instruments, three time-scale estimators, 127 spectral features, and correlation thresholds were selected using domain knowledge and preliminary analysis on a pre-sample period prior to the walk-forward evaluation; they were then locked for all subsequent folds. We will add a clear description of this protocol to the Methods section in the revised manuscript to remove any ambiguity about potential selection bias. revision: yes

  2. Referee: [Evaluation protocol] Evaluation protocol paragraph: although the eight-fold walk-forward with ten-day gaps is described as strict, the manuscript provides no explicit statement that Random Forest depth, sub-sample weighting, and any implicit feature-threshold tuning were performed inside each training fold only. Global selection against the reported metric would create circularity, directly undermining the claim that spectral features deliver genuine out-of-sample regime detection.

    Authors: We acknowledge that the current text lacks an explicit confirmation on this point. In the reported experiments, Random Forest depth, sub-sample weighting, and any feature-threshold decisions were tuned exclusively inside each training fold using only data available up to that fold, with no access to evaluation-fold information. We will revise the Evaluation Protocol paragraph to state this explicitly, thereby confirming the absence of circularity and strengthening the out-of-sample validity of the results. revision: yes

Circularity Check

0 steps flagged

No significant circularity; performance metrics arise from supervised learning on temporally split data

full rationale

The paper presents an ML pipeline that extracts fixed spectral and traditional features from rolling correlation matrices, trains a depth-limited Random Forest, and reports AUC under an eight-fold walk-forward protocol with anti-leakage gaps. The BCD-AUC of 0.741, ablation deltas, and backtested Sharpe are direct outputs of this supervised process on the 15-year dataset; they are not shown by any quoted equation or self-citation to be equivalent to the input choices (instrument set, time scales, thresholds, or feature count) by construction. The derivation chain remains self-contained and externally falsifiable via the forward splits.

Axiom & Free-Parameter Ledger

4 free parameters · 2 axioms · 0 invented entities

The framework rests on standard assumptions from random matrix theory and graph theory plus numerous design choices that function as free parameters: the exact 24 instruments, the three parallel correlation estimators and their time scales, the 127 spectral features and the correlation thresholds at which graph descriptors are computed, the depth limit and balancing weights of the random forest, and the mapping rule from joint probabilities to equity exposure. No new physical entities are postulated.

free parameters (4)
  • 24 diversified exchange-traded instruments
    Choice of specific assets and diversification criterion directly shapes the correlation matrices and is not derived from first principles.
  • three parallel estimators at different time scales
    Number and lengths of rolling windows are selected by the authors to capture multi-scale dynamics.
  • 127 spectral features and correlation thresholds
    Exact set of absorption ratios, eigenvalue entropy, effective rank, spectral gap, eigenvector concentration, and graph-topological descriptors at multiple thresholds are engineering choices.
  • depth-limited Random Forest hyperparameters and balanced sub-sample weighting
    Model capacity and class-balancing weights are tuned for the reported AUC.
axioms (2)
  • domain assumption Rolling correlation matrices computed from daily prices are sufficiently stationary within each window to yield meaningful spectral features.
    Invoked when constructing the input matrices for feature extraction.
  • domain assumption Random matrix theory provides a useful null model for distinguishing signal from noise in finite-sample correlation matrices of 24 assets.
    Used to motivate the spectral cleaning and feature extraction steps.

pith-pipeline@v0.9.0 · 5621 in / 1896 out tokens · 52954 ms · 2026-05-10T06:04:20.628503+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Artificial Adaptive Intelligence: The Missing Stage Between Narrow and General Intelligence

    cs.AI 2026-05 unverdicted novelty 4.0

    Proposes Artificial Adaptive Intelligence as the regime between narrow and general AI, defined by elimination of human-specified hyperparameters, and introduces an adaptivity index plus parametric minimality principle...

Reference graph

Works this paper leans on

36 extracted references · 5 canonical work pages · cited by 1 Pith paper

  1. [1]

    Autoregressive conditional heteroscedas- ticity with estimates of the variance of united kingdom inflation,

    R. F. Engle, “Autoregressive conditional heteroscedas- ticity with estimates of the variance of united kingdom inflation,”Econometrica, vol. 50, no. 4, pp. 987–1007, 1982

    1982

  2. [2]

    Generalized autoregressive conditional heteroskedasticity,

    T. Bollerslev, “Generalized autoregressive conditional heteroskedasticity,”Journal of Econometrics, vol. 31, no. 3, pp. 307–327, 1986

    1986

  3. [3]

    Modeling and forecasting realized volatility,

    T. G. Andersen, T. Bollerslev, F. X. Diebold, and P. Labys, “Modeling and forecasting realized volatility,” Econometrica, vol. 71, no. 2, pp. 579–625, 2003

    2003

  4. [4]

    A simple approximate long-memory model of realized volatility,

    F. Corsi, “A simple approximate long-memory model of realized volatility,”Journal of Financial Econometrics, vol. 7, no. 2, pp. 174–196, 2009

    2009

  5. [5]

    Principal components as a measure of systemic risk,

    M. Kritzman, Y . Li, S. Page, and R. Rigobon, “Principal components as a measure of systemic risk,”The Journal of Portfolio Management, vol. 37, no. 4, pp. 112–126, 2011

    2011

  6. [6]

    Noise dressing of financial correlation matrices,

    L. Laloux, P. Cizeau, J.-P. Bouchaud, and M. Potters, “Noise dressing of financial correlation matrices,”Phys- ical Review Letters, vol. 83, no. 7, pp. 1467–1470, 1999

    1999

  7. [7]

    Random matrix approach to cross correlations in financial data,

    V . Plerou, P. Gopikrishnan, B. Rosenow, L. A. N. Amaral, T. Guhr, and H. E. Stanley, “Random matrix approach to cross correlations in financial data,”Physical Review E, vol. 65, no. 6, p. 066126, 2002

    2002

  8. [8]

    Distribution of eigen- values for some sets of random matrices,

    V . A. Marˇcenko and L. A. Pastur, “Distribution of eigen- values for some sets of random matrices,”Mathematics of the USSR-Sbornik, vol. 1, no. 4, pp. 457–483, 1967

    1967

  9. [9]

    Financial applications of random matrix theory: A short review,

    J.-P. Bouchaud and M. Potters, “Financial applications of random matrix theory: A short review,”arXiv preprint arXiv:0910.1205, 2009, also appeared as Chapter 40 in The Oxford Handbook of Random Matrix Theory, 2015

    work page arXiv 2009

  10. [10]

    Random forests,

    L. Breiman, “Random forests,”Machine Learning, vol. 45, no. 1, pp. 5–32, 2001

    2001

  11. [11]

    Universal and nonuniversal properties of cross correlations in financial time series,

    V . Plerou, P. Gopikrishnan, B. Rosenow, L. A. N. Amaral, and H. E. Stanley, “Universal and nonuniversal properties of cross correlations in financial time series,”Physical Review Letters, vol. 83, no. 7, pp. 1471–1474, 1999

    1999

  12. [12]

    Honey, i shrunk the sample co- variance matrix,

    O. Ledoit and M. Wolf, “Honey, i shrunk the sample co- variance matrix,”The Journal of Portfolio Management, vol. 30, no. 4, pp. 110–119, 2004

    2004

  13. [13]

    Skulls, financial turbulence, and risk management,

    M. Kritzman and Y . Li, “Skulls, financial turbulence, and risk management,”Financial Analysts Journal, vol. 66, no. 5, pp. 30–41, 2010

    2010

  14. [14]

    A survey of systemic risk analytics,

    D. Bisias, M. Flood, A. W. Lo, and S. Valavanis, “A survey of systemic risk analytics,”Annual Review of Financial Economics, vol. 4, pp. 255–296, 2012

    2012

  15. [15]

    Econometric measures of connectedness and systemic risk in the finance and insurance sectors,

    M. Billio, M. Getmansky, A. W. Lo, and L. Pelizzon, “Econometric measures of connectedness and systemic risk in the finance and insurance sectors,”Journal of Financial Economics, vol. 104, no. 3, pp. 535–559, 2012

    2012

  16. [16]

    Quantifying the behavior of stock corre- lations under market stress,

    T. Preis, D. Y . Kenett, H. E. Stanley, D. Helbing, and E. Ben-Jacob, “Quantifying the behavior of stock corre- lations under market stress,”Scientific Reports, vol. 2, p. 752, 2012

    2012

  17. [17]

    Quantifying meta-correlations in financial mar- kets,

    D. Y . Kenett, T. Preis, G. Gur-Gershgoren, and E. Ben- Jacob, “Quantifying meta-correlations in financial mar- kets,”Europhysics Letters (EPL), vol. 99, no. 3, p. 38001, 2012

    2012

  18. [18]

    Hierarchical structure in financial markets,

    R. N. Mantegna, “Hierarchical structure in financial markets,”The European Physical Journal B, vol. 11, no. 1, pp. 193–197, 1999

    1999

  19. [19]

    A tool for filtering information in complex systems,

    M. Tumminello, T. Aste, T. Di Matteo, and R. N. Mantegna, “A tool for filtering information in complex systems,”Proceedings of the National Academy of Sci- ences, vol. 102, no. 30, pp. 10 421–10 426, 2005

    2005

  20. [20]

    Topology of correlation-based minimal spanning trees in real and model markets,

    G. Bonanno, G. Caldarelli, F. Lillo, and R. N. Mantegna, “Topology of correlation-based minimal spanning trees in real and model markets,”Physical Review E, vol. 68, no. 4, p. 046130, 2003

    2003

  21. [21]

    Dynamic asset trees and black monday,

    J.-P. Onnela, A. Chakraborti, K. Kaski, and J. Kert ´esz, “Dynamic asset trees and black monday,”Physica A: Statistical Mechanics and its Applications, vol. 324, no. 1–2, pp. 247–252, 2003

    2003

  22. [22]

    Dynamics of market correlations: Taxonomy and portfolio analysis,

    J.-P. Onnela, A. Chakraborti, K. Kaski, J. Kert ´esz, and A. Kanto, “Dynamics of market correlations: Taxonomy and portfolio analysis,”Physical Review E, vol. 68, no. 5, p. 056110, 2003

    2003

  23. [23]

    Spread of risk across financial markets: Better to invest in the peripheries,

    F. Pozzi, T. Di Matteo, and T. Aste, “Spread of risk across financial markets: Better to invest in the peripheries,” Scientific Reports, vol. 3, p. 1665, 2013

    2013

  24. [24]

    Better to give than to receive: Predictive directional measurement of volatility spillovers,

    F. X. Diebold and K. Yilmaz, “Better to give than to receive: Predictive directional measurement of volatility spillovers,”International Journal of Forecasting, vol. 28, no. 1, pp. 57–66, 2012

    2012

  25. [25]

    On the network topology of variance decompo- sitions: Measuring the connectedness of financial firms,

    ——, “On the network topology of variance decompo- sitions: Measuring the connectedness of financial firms,” Journal of Econometrics, vol. 182, no. 1, pp. 119–134, 2014

    2014

  26. [26]

    XGBoost: A scalable tree boosting system,

    T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” inProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining. ACM, 2016, pp. 785–794

    2016

  27. [27]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural Computation, vol. 9, no. 8, pp. 1735– 1780, 1997

    1997

  28. [28]

    Forecasting stock market crisis events using deep and statistical machine learning techniques,

    S. P. Chatzis, V . Siakoulis, A. Petropoulos, E. Stavroulakis, and N. Vlachogiannakis, “Forecasting stock market crisis events using deep and statistical machine learning techniques,”Expert Systems with Applications, vol. 112, pp. 353–371, 2018

    2018

  29. [29]

    DeepSupp: Attention-driven correlation pattern analysis for dynamic time series support and resistance levels identification,

    B. Kriuk, L. Ng, and Z. Al Hossain, “DeepSupp: Attention-driven correlation pattern analysis for dynamic time series support and resistance levels identification,” arXiv preprint arXiv:2507.01971, 2025

    work page arXiv 2025

  30. [30]

    To what extent can public equity indices statistically hedge real purchasing power loss in compounded structural emerging-market crises? an explainable ML-based assessment,

    A. Alkhamov and B. Kriuk, “To what extent can public equity indices statistically hedge real purchasing power loss in compounded structural emerging-market crises? an explainable ML-based assessment,”arXiv preprint arXiv:2507.13055, 2025

    work page arXiv 2025

  31. [31]

    Change detection based on artificial intelligence: State- of-the-art and challenges,

    W. Shi, M. Zhang, R. Zhang, S. Chen, and Z. Zhan, “Change detection based on artificial intelligence: State- of-the-art and challenges,”Remote Sensing, vol. 12, no. 10, p. 1688, 2020

    2020

  32. [32]

    MorphBoost: Self-organizing universal gradi- ent boosting with adaptive tree morphing,

    B. Kriuk, “MorphBoost: Self-organizing universal gradi- ent boosting with adaptive tree morphing,”arXiv preprint arXiv:2511.13234, 2025

    work page arXiv 2025

  33. [33]

    POSEIDON: Physics-optimized seismic energy inference and detection operating net- work,

    B. Kriuk and F. Kriuk, “POSEIDON: Physics-optimized seismic energy inference and detection operating net- work,”arXiv preprint arXiv:2601.02264, 2026

    work page arXiv 2026

  34. [34]

    Scikit-learn: Machine learning in Python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cour- napeau, M. Brucher, M. Perrot, and ´E. Duchesnay, “Scikit-learn: Machine learning in Python,”Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011

    2011

  35. [35]

    A unified approach to interpreting model predictions,

    S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” inAdvances in Neural Information Processing Systems, vol. 30. Curran Asso- ciates, Inc., 2017, pp. 4765–4774

    2017

  36. [36]

    ELENA: Epigenetic learning through evolved neural adaptation,

    B. Kriuk, K. Sulamanidze, and F. Kriuk, “ELENA: Epigenetic learning through evolved neural adaptation,” Evolutionary Intelligence, vol. 18, no. 50, 2025

    2025