Pith. sign in

REVIEW 4 major objections 4 minor 25 references

Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read On high-frequency crypto order books, smoothing and feature engineering add more predictive accuracy than neural depth — simple models match or beat deep networks by 1-2%.

desk verdict A clear benchmark undone by a centered Savitzky–Golay filter that leaks the future into the features; the main claim is unsupported until the filter is made causal. read the letter →

arxiv 2506.05764 v2 pith:HFFVHI3O submitted 2025-06-06 cs.LG q-fin.TR

classification cs.LGq-fin.TR
keywords limitorderbookscryptocurrencymicrostructuremid-pricemovementpredictionSavitzky-GolaysmoothingKalmanfilteringXGBoostDeepLOBfeatureengineeringvsmodelcomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the predictive success of deep learning on limit-order-book data comes from architectural depth or from how the inputs are prepared. Benchmarking six models — logistic regression, XGBoost, CatBoost, a CNN+LSTM, a CNN+XGBoost hybrid, and DeepLOB — on 100 ms BTC/USDT snapshots with raw, Kalman-smoothed, and Savitzky-Golay-smoothed inputs, it finds that smoothing improves every model while no deep architecture consistently beats the simple ones. After Savitzky-Golay denoising, XGBoost and logistic regression match or exceed the neural networks by 1-2% in accuracy and F1, with far lower training and inference cost. If the claim holds, the practical lesson is that on noisy high-frequency data, effort spent on filtering, feature engineering, and label construction buys more accuracy than stacking hidden layers, and latency-sensitive strategies can use interpretable models without sacrificing edge.

What carries the argument

The load-bearing mechanism is the Savitzky-Golay filter: a local cubic polynomial is fit by least squares over a 21-point centered window ($j=-10,\dots,10$), so the smoothed value at time $t$ is $\hat v_t = \sum_{j=-10}^{10} c_j v_{t+j}$ with fixed convolution weights $c_j$; the paper applies this smoothing to each LOB feature series before scaling. It is the single intervention that lifts accuracy for every model in the benchmark, which is why the model comparison is really a comparison at fixed input quality. The supporting machinery is the shared feature set of the simpler models — first-level and five-level order imbalance, weighted mid-price change, and cumulative book depth — together with the evaluation design that varies label scheme (binary vs ternary), horizon (100 ms, 500 ms, 1 s), and denoising filter (raw, Kalman, Savitzky-Golay).

What would settle it

Re-run the binary benchmark at the 500 ms horizon with a causal Savitzky-Golay filter — fit the cubic over $\{v_{t-20},\ldots,v_t\}$ or shift the smoothed series ten steps later — and retrain with walk-forward splits so no test window shares a smoothing window with training data. If XGBoost and logistic regression still edge out DeepLOB and CNN+LSTM by 1-2% and the smoothed inputs still beat the raw ones, the central claim holds; if the model gap or the smoothing gain collapses, the reported accuracy came from lookahead.

Watch

Extended reading notes

Core claim

The paper's central claim is that, for short-horizon prediction of mid-price moves (the midpoint of the best bid and ask) on cryptocurrency limit order books, data preprocessing and hyperparameter tuning contribute more to accuracy than architectural depth. On a benchmark of six models over BTC/USDT snapshots, no deep network consistently beats the baselines: after Savitzky-Golay smoothing, XGBoost and logistic regression marginally outperform DeepLOB and the CNN+LSTM variants by 1-2% in accuracy and F1 while training in a fraction of the time, and the same smoothing improves every model, whereas Kalman filtering with fixed noise covariances can degrade performance below the raw baseline. The paper also documents two secondary findings: using all 40 book levels rather than 5 raises binary accuracy from about 0.58 to 0.715 but cuts the usable sample by more than half, and feeding ten consecutive snapshots instead of one adds roughly 2% accuracy for both XGBoost and logistic regression at several times the training cost. The paper is explicit that these results come from a single trading day of offline experiments, leaving cross-day robustness and live deployment open.

Load-bearing premise

The accuracy gains rest on a smoothing filter that, as written, uses future snapshots to clean each data point, and the paper does not show it was applied causally or that the train/test split prevented that future information from leaking into the test set.

Editorial extensions

If this is right

  • Model complexity is a second-order factor: adding a third convolutional block to a CNN+LSTM pipeline (the DeepLOB configuration) produces no consistent accuracy gain over simpler variants, so the paper's recipe of filter, tune, and simple model is the benchmark to beat.
  • Denoising choice dominates model choice: Savitzky-Golay smoothing improves accuracy across the board, while Kalman filtering with fixed noise variances can fall below the raw-data baseline, meaning preprocessing choices deserve as much tuning attention as hyperparameters.
  • There is a quantifiable coverage-accuracy trade-off: 40-level books reach 0.715 binary accuracy but cover only 5,442 usable test snapshots, while 10- and 5-level books raise coverage to more than 18,000 and drop accuracy to about 0.58, so system builders must choose an operating point.
  • Sequence context is a modest 2% gain: feeding ten consecutive 100 ms snapshots instead of one raises F1 by about 2% for both XGBoost and logistic regression, at several times the training time but still only minutes on a laptop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own equations suggest an upper bound rather than a guaranteed edge: because the smoothing window is centered, a reader cannot rule out that part of the measured gain comes from features that peek at future snapshots, so the numbers should be treated as best-case until a causal filter is tested.
  • If the flat model ordering generalizes, published deep-learning gains on public LOB benchmarks deserve re-examination against a tuned, denoised tree baseline; some reported 'architectural' improvements may be preprocessing effects any model would pick up.
  • A nearby question the paper leaves open is whether preprocessing dominance transfers to event-driven LOB data and thinner books, where noise and imbalance signals have different shapes; the answer would decide whether the finding is about crypto LOBs specifically or about noisy high-frequency data generally.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper benchmarks six classifiers—logistic regression, XGBoost, CatBoost, CNN+LSTM, CNN+XGBoost, and DeepLOB—on BTC/USDT limit order book snapshots from Bybit sampled at 100 ms. It compares raw data, Kalman-filtered data, and Savitzky–Golay-smoothed data under binary and ternary labeling schemes at 100 ms, 500 ms, and 1000 ms horizons. The central claim is that with data preprocessing and hyperparameter tuning, simpler models (XGBoost, logistic regression) match or exceed deeper neural networks, with faster inference and greater interpretability. The results are presented in four tables reporting F1 scores or accuracy across configurations, plus runtime comparisons.

Significance. If the result holds under a leakage-free protocol, the paper makes a practically useful point: for high-frequency LOB forecasting, input engineering and hyperparameter choice may dominate architectural depth, with implications for latency and interpretability in deployment. The paper uses publicly available data, defines hand-crafted features explicitly, includes multiple model families, and reports training runtimes, which are strengths. However, the empirical evidence is undermined by a likely lookahead bias in the Savitzky–Golay preprocessing, and the claims are made without error bars, significance tests, or multi-day validation. The paper also ships no code, limiting reproducibility of the preprocessing pipeline.

major comments (4)
  1. [§4.2.1, Eq. (7)] The Savitzky–Golay filter as defined is centered: the smoothed value at time t is a weighted sum of v_{t+j} for j = -10,...,+10, so it uses up to 1 second of future observations. The manuscript never states that the filter is applied one-sidedly, that the smoothed series is shifted by the window half-width before modelling, or that filtering is performed separately within train and test blocks. If the full 100 ms series is filtered before the 80/20 split, every test feature contains information from after the feature timestamp; for the 500 ms and 1000 ms horizons this is direct lookahead. This could explain the pattern in Tables 1 and 2, where Savitzky–Golay is the only filter that consistently improves every model while the causal Kalman filter often degrades accuracy. The authors must rerun the experiments under a clearly causal protocol—for example, a one-sided filter, a shift by the half-window, or filtering applied independently to training and test periods with no future values—and report whether the SG gains survive. As written, the central conclusion that preprocessing/denoising drives performance is not supported.
  2. [§3, §5.1–5.2, Tables 1–2] The experiments use a single trading day (2025-01-30) and report no confidence intervals, repeated runs, or significance tests. The abstract's claim that simpler models outperform complex neural networks by 1–2% is therefore not statistically grounded; differences of this size across models may be within sampling noise, especially since the test sets are not held constant across configurations (support ranges from 5,442 to 18,336). The authors should add error bars via bootstrapping or repeated train/test splits, and ideally evaluate on additional trading days, to support the global claim.
  3. [Tables 1 and 2] Prediction horizon and LOB depth are varied simultaneously: the 100 ms rows use a 5-level LOB while the 500 ms and 1000 ms rows use a 40-level LOB. Consequently, the apparent improvement with longer horizons is confounded with the change in input depth, and no clean horizon effect can be extracted. The same confound affects the comparison of filtering methods across rows, because the test sets differ in size and composition. The experimental design should hold depth fixed when varying horizon, or otherwise report a factorial design.
  4. [§5.3, Table 3] The depth comparison for XGBoost is not controlled: the model is tested on 5,442 snapshots at 40 levels but on 18,336 snapshots at 5 levels. The drop in accuracy from 0.715 to 0.580 may reflect the different test sets (different time periods and noise characteristics) rather than LOB depth alone. The conclusion that 'shallower LOB increases coverage but reduces accuracy' requires a matched test set or a per-support analysis.
minor comments (4)
  1. [§2.4] There are repeated typos ('Inspired y Sirignano' should be 'Inspired by Sirignano', 'bais' should be 'bias', 'cuased' should be 'caused'). The phrase 'Efficient Market Theorem' should be 'Efficient Market Hypothesis'. The sentence implying that efficiency predicts exactly 50% accuracy conflates market efficiency with unpredictability under a specific loss function.
  2. [§5 and Table 3] The text says the models are evaluated with F1 score, but the tables are titled 'Classification ... accuracy' and Section 5.3 uses the word 'accuracy' for the metric that appears to be F1. The metric used in each table should be stated consistently.
  3. [§5.4] The subsection heading contains a typo: 'Logistic Regregression' should be 'Logistic Regression'.
  4. [§4.2 and §5] The Kalman filter parameters Q and R, the Savitzky–Golay window length and polynomial degree, and the ternary epsilon threshold are all described as tuned on a small sample, but the actual values and tuning procedure are not reported. Providing these details, along with the train/validation split and any code repository, would substantiate the 'hyperparameter tuning' claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's conclusions are empirical benchmark observations, not derivations that reduce to their own inputs.

full rationale

This is an empirical benchmarking study, not a derivation chain. The central claim — that with preprocessing and hyperparameter tuning, simpler models like XGBoost and logistic regression match or exceed deeper networks (Abstract; Conclusion) — is supported by out-of-sample accuracy and F1 results in Tables 1, 2, and 4. The models are trained on an 80/20 split and evaluated on held-out test rows, so the reported accuracies are not defined in terms of the fitted parameters. The handcrafted features in Eqs. (1)–(4) are fixed economic quantities, not fit to the labels. The Savitzky–Golay filter in Eq. (7) uses pre-computed convolution weights from a polynomial least-squares fit on the input series itself; it does not use label information and is not a fitted parameter renamed as a prediction. The Kalman filter in Eqs. (8)–(10) is likewise a preprocessing transform with fixed or grid-searched noise parameters, and the paper explicitly notes its limited tuning. There is no load-bearing self-citation chain: the paper is single-authored and its references are to external prior work (e.g., Savitzky & Golay 1964; DeepLOB; FI-2010), not to a prior result by the same author that is being recycled as a forced premise. The only notable validity concern is that Eq. (7) defines a centered Savitzky–Golay window with j ranging from −10 to +10, meaning each smoothed value uses 10 future snapshots; if filtering is applied before the train/test split without a causal shift, test features could contain lookahead information. That is a data-leakage and real-time-deployment concern, not circularity: the conclusion does not become equivalent to its inputs by construction, and no equation or fitted parameter makes the reported comparison tautological. Accordingly, the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new objects, forces, or postulates. All free parameters are standard hyperparameters or filter settings, though many are not reported. The main hidden assumption is the causal usability of the Savitzky-Golay filter.

free parameters (5)
  • ternary epsilon threshold = not reported
    Section 2.4 describes tuning epsilon to achieve roughly equal class frequencies for ternary classification; this threshold is fitted to the data and directly affects the labels and results.
  • Kalman filter noise variances Q and R = not reported
    Section 4.2.2 assumes a random walk with process and observation noise variances Q and R. The paper says these were tuned by a limited grid search, but the final values are not given.
  • Savitzky-Golay window length and polynomial degree = window 21, degree 3
    Section 4.2.1 sets a cubic polynomial and a window of 21 points by hand, balancing accuracy and compute. This is a manual choice that affects the smoothed features.
  • inverse-frequency class weights = not reported
    The paper applies inverse-frequency weighting to handle class imbalance, but does not report the exact weights or the class frequencies used.
  • XGBoost hyperparameters = not reported
    Section 4.1 says grid search over n_estimators and learning rate, but the grid and chosen values are not reported.
assumptions (4)
  • domain assumption A random walk plus Gaussian noise model is appropriate for LOB features.
    The Kalman filter in Eq. (8) assumes this state-space model, and the paper does not validate it against the actual feature dynamics.
  • ad hoc to paper Centered Savitzky-Golay smoothing can be used without introducing lookahead leakage, or that lookahead is acceptable.
    Eq. (7) uses future observations in the smoothing window, but the paper does not discuss causality or leakage. This assumption is load-bearing for the reported accuracy gains.
  • domain assumption A single trading day of Bybit BTC/USDT data is representative of LOB microstructure dynamics.
    The paper draws conclusions from 2025-01-30 only, and the conclusion itself acknowledges that more days are needed.
  • domain assumption Discarding snapshots with missing depth levels does not bias the evaluation.
    Section 5.3 and 5.4 reduce the test set substantially when using 40 levels, and the paper treats the remaining snapshots as representative without a robustness analysis of the missing data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer." pith.science (2026). https://pith.science/paper/HFFVHI3O

@misc{pith2026250605764,
  author       = {Pith},
  title        = {Pith review of: Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HFFVHI3O}},
  note         = {Machine review of arXiv:2506.05764}
}
read the original abstract

Cryptocurrency price dynamics are driven largely by microstructural supply demand imbalances in the limit order book (LOB), yet the highly noisy nature of LOB data complicates the signal extraction process. Prior research has demonstrated that deep-learning architectures can yield promising predictive performance on pre-processed equity and futures LOB data, but they often treat model complexity as an unqualified virtue. In this paper, we aim to examine whether adding extra hidden layers or parameters to "blackbox ish" neural networks genuinely enhances short term price forecasting, or if gains are primarily attributable to data preprocessing and feature engineering. We benchmark a spectrum of models from interpretable baselines, logistic regression, XGBoost to deep architectures (DeepLOB, Conv1D+LSTM) on BTC/USDT LOB snapshots sampled at 100 ms to multi second intervals using publicly available Bybit data. We introduce two data filtering pipelines (Kalman, Savitzky Golay) and evaluate both binary (up/down) and ternary (up/flat/down) labeling schemes. Our analysis compares models on out of sample accuracy, latency, and robustness to noise. Results reveal that, with data preprocessing and hyperparameter tuning, simpler models can match and even exceed the performance of more complex networks, offering faster inference and greater interpretability.

Figures

Figures reproduced from arXiv: 2506.05764 by the authors.

Figure 1
Figure 1. Comparison of the raw and cumulative limit order books. Aggregating quantities across levels [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. CNN + CatBoost: Pass embedding to CatBoost then performs ordered boosting [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. DeepLOB: Three CNN layers processed by an LSTM [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Simpler CNN + LSTM: One CNN layer processed by an LSTM [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: XGBoost Only architecture: LOB input is passed through multiple rounds of gradient-boosted [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: CNN + XGBoost architecture: a feature extractor network learns embeddings from limit order [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Savitzky–Golay and Kalman Filters on LOB [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 18 canonical work pages

  1. [1]

    P., & Roberts, S

    Zhang, J., Bennett, K. P., & Roberts, S. (2022).Axial-LOB: High-Frequency Trading with Axial Atten- tion. UCL Discovery.https://discovery.ucl.ac.uk/id/eprint/10172667/

  2. [2]

    A., Barboza, F., Silva, L

    Bileki, G. A., Barboza, F., Silva, L. H. C., & Bonato, V. (2022). Order book mid-price movement inference by CatBoost classifier from convolutional feature maps.Applied Soft Computing, 116, 108274. doi:10.1016/j.asoc.2021.108274

  3. [3]

    Berti, L., & Kasneci, G. (2025). TLOB: A Novel Transformer Model with Dual Attention for Stock Price Trend Prediction with Limit Order Book Data.arXiv preprint arXiv:2502.15757.https://doi. org/10.48550/arXiv.2502.15757

  4. [4]

    N., & Zhang, Y

    Kercheval, A. N., & Zhang, Y. (2015). Modelling high-frequency limit order book dynamics with support vector machines.Quantitative Finance, 15(8), 1315–1329.https://doi.org/10.1080/14697688.2015. 1032546

  5. [5]

    Ntakaris, A., Magris, M., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2018). Benchmark dataset for mid-price forecasting of limit order book data with machine learning methods.Journal of Forecasting, 37.https://doi.org/10.1002/for.2543

  6. [6]

    Zhang, Z., Zohren, S., & Roberts, S. (2019). DeepLOB: Deep Convolutional Neural Networks for Limit Order Books.IEEE Transactions on Signal Processing, 67(11), 3001–3012.https://doi.org/10.1109/ TSP.2019.2907260

  7. [7]

    Sirignano, J., & Cont, R. (2019). Universal features of price formation in financial markets: perspectives from deep learning.Quantitative Finance

  8. [8]

    Briola, A., Bartolucci, S., & Aste, T. (2024). Deep Limit Order Book Forecasting.arXiv:2403.09267 [q-fin.TR].https://doi.org/10.48550/arXiv.2403.09267

Show all 25 references
  1. [9]

    Briola, A., Turiel, J., & Aste, T. (2020). Deep Learning modeling of Limit Order Book: a comparative perspective.arXiv:2007.07319 [q-fin.TR].https://doi.org/10.48550/arXiv.2007.07319

  2. [10]

    Zheng, B., Moulines, E., & Abergel, F. (2012). Price Jump Prediction in Limit Order Book. arXiv:1204.1381 [q-fin.TR].https://doi.org/10.48550/arXiv.1204.1381

  3. [11]

    Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2020). Using Deep Learning for price prediction by exploiting stationary limit order book features.Applied Soft Computing, 93, 106401. ISSN 1568-4946. doi:10.1016/j.asoc.2020.106401

  4. [12]

    (2022, April 1)

    Divakar, V., & Bhattacharyya, R. (2022, April 1). Optimizing Order Types in Indian Futures Market. SSRN Scholarly Paper 4072374.http://dx.doi.org/10.2139/ssrn.4072374

  5. [13]

    Wu, Y., Mahfouz, M., Magazzeni, D., & Veloso, M. (2021). How Robust Are Limit Order Book Represen- tations Under Data Perturbation?arXiv:2110.04752 [q-fin.TR].https://doi.org/10.48550/arXiv. 2110.04752

  6. [14]

    Hultin, H., Hult, H., Prouti` ere, A., Samama, S., & Tarighati, A. (2023). A generative model of a limit order book using recurrent neural networks.Quantitative Finance, 23(6), 931–958. doi:10.1080/14697688.2023.2205583

  7. [15]

    Zheng, Z., Li, G., Ouyang, D., Liang, D., & Shao, J. (2024). Limit Order Book Event Stream Prediction with Diffusion Model.arXiv:2412.09631 [q-fin.ST].https://doi.org/10.48550/arXiv.2412.09631

  8. [16]

    Su, Y., Sun, Z., Li, J., & Yuan, X. (2021). The Price Impact of Generalized Order Flow Imbalance. arXiv:2112.02947 [q-fin.TR].https://doi.org/10.48550/arXiv.2112.02947 11

  9. [17]

    Chordia, T., Subrahmanyam, A., & Roll, R. W. (2001, November13). Order Imbalance, Liquidity, and Market Returns. SSRN Scholarly Paper 282759.https://ssrn.com/abstract=282759

  10. [18]

    D., & Howison, S

    Xu, K., Gould, M. D., & Howison, S. D. (2019, October 30). Multi-Level Order-Flow Imbalance in a Limit Order Book. SSRN Scholarly Paper 3479741.http://dx.doi.org/10.2139/ssrn.3479741

  11. [19]

    Dezhkam, A., & Manzuri, M. T. (2023). Forecasting stock market for an efficient portfolio by combining XGBoost and Hilbert–Huang transform.Engineering Applications of Artificial Intelligence, 118, 105626. doi:10.1016/j.engappai.2022.105626

  12. [20]

    Marsza lek, A., & Burczy´ nski, T. (2024). Modeling of limit order book data with ordered fuzzy numbers. Applied Soft Computing, 158, 111555.https://doi.org/10.1016/j.asoc.2024.111555

  13. [21]

    Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2018). Using Deep Learning for Price Prediction by Exploiting Stationary Limit Order Book Features.arXiv:1810.09965 [cs.LG].https://doi.org/10.48550/arXiv.1810.09965

  14. [22]

    Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2017). Forecasting stock prices from the limit order book using convolutional neural networks. InProceedings of the IEEE

  15. [23]

    Dietterich, T. G. (2000). Ensemble methods in machine learning. InMultiple Classifier Systems(pp. 1–15)

  16. [24]

    M., & Menkveld, A

    Hendershott, T., Jones, C. M., & Menkveld, A. J. (2011). Does algorithmic trading improve liquidity? Journal of Finance, 66(1), 1–33

  17. [25]

    Savitzky, A., & Golay, M. J. E. (1964). Smoothing and differentiation of data by simplified least squares procedures.Analytical Chemistry, 36(8), 1627–1639. 12

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.