REVIEW 4 major objections 4 minor 25 references
Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read On high-frequency crypto order books, smoothing and feature engineering add more predictive accuracy than neural depth — simple models match or beat deep networks by 1-2%.
desk verdict A clear benchmark undone by a centered Savitzky–Golay filter that leaks the future into the features; the main claim is unsupported until the filter is made causal. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the Savitzky-Golay filter: a local cubic polynomial is fit by least squares over a 21-point centered window ($j=-10,\dots,10$), so the smoothed value at time $t$ is $\hat v_t = \sum_{j=-10}^{10} c_j v_{t+j}$ with fixed convolution weights $c_j$; the paper applies this smoothing to each LOB feature series before scaling. It is the single intervention that lifts accuracy for every model in the benchmark, which is why the model comparison is really a comparison at fixed input quality. The supporting machinery is the shared feature set of the simpler models — first-level and five-level order imbalance, weighted mid-price change, and cumulative book depth — together with the evaluation design that varies label scheme (binary vs ternary), horizon (100 ms, 500 ms, 1 s), and denoising filter (raw, Kalman, Savitzky-Golay).
What would settle it
Re-run the binary benchmark at the 500 ms horizon with a causal Savitzky-Golay filter — fit the cubic over $\{v_{t-20},\ldots,v_t\}$ or shift the smoothed series ten steps later — and retrain with walk-forward splits so no test window shares a smoothing window with training data. If XGBoost and logistic regression still edge out DeepLOB and CNN+LSTM by 1-2% and the smoothed inputs still beat the raw ones, the central claim holds; if the model gap or the smoothing gain collapses, the reported accuracy came from lookahead.
Extended reading notes
Core claim
The paper's central claim is that, for short-horizon prediction of mid-price moves (the midpoint of the best bid and ask) on cryptocurrency limit order books, data preprocessing and hyperparameter tuning contribute more to accuracy than architectural depth. On a benchmark of six models over BTC/USDT snapshots, no deep network consistently beats the baselines: after Savitzky-Golay smoothing, XGBoost and logistic regression marginally outperform DeepLOB and the CNN+LSTM variants by 1-2% in accuracy and F1 while training in a fraction of the time, and the same smoothing improves every model, whereas Kalman filtering with fixed noise covariances can degrade performance below the raw baseline. The paper also documents two secondary findings: using all 40 book levels rather than 5 raises binary accuracy from about 0.58 to 0.715 but cuts the usable sample by more than half, and feeding ten consecutive snapshots instead of one adds roughly 2% accuracy for both XGBoost and logistic regression at several times the training cost. The paper is explicit that these results come from a single trading day of offline experiments, leaving cross-day robustness and live deployment open.
Load-bearing premise
The accuracy gains rest on a smoothing filter that, as written, uses future snapshots to clean each data point, and the paper does not show it was applied causally or that the train/test split prevented that future information from leaking into the test set.
Editorial extensions
If this is right
- Model complexity is a second-order factor: adding a third convolutional block to a CNN+LSTM pipeline (the DeepLOB configuration) produces no consistent accuracy gain over simpler variants, so the paper's recipe of filter, tune, and simple model is the benchmark to beat.
- Denoising choice dominates model choice: Savitzky-Golay smoothing improves accuracy across the board, while Kalman filtering with fixed noise variances can fall below the raw-data baseline, meaning preprocessing choices deserve as much tuning attention as hyperparameters.
- There is a quantifiable coverage-accuracy trade-off: 40-level books reach 0.715 binary accuracy but cover only 5,442 usable test snapshots, while 10- and 5-level books raise coverage to more than 18,000 and drop accuracy to about 0.58, so system builders must choose an operating point.
- Sequence context is a modest 2% gain: feeding ten consecutive 100 ms snapshots instead of one raises F1 by about 2% for both XGBoost and logistic regression, at several times the training time but still only minutes on a laptop.
Reading between the lines
- The paper's own equations suggest an upper bound rather than a guaranteed edge: because the smoothing window is centered, a reader cannot rule out that part of the measured gain comes from features that peek at future snapshots, so the numbers should be treated as best-case until a causal filter is tested.
- If the flat model ordering generalizes, published deep-learning gains on public LOB benchmarks deserve re-examination against a tuned, denoised tree baseline; some reported 'architectural' improvements may be preprocessing effects any model would pick up.
- A nearby question the paper leaves open is whether preprocessing dominance transfers to event-driven LOB data and thinner books, where noise and imbalance signals have different shapes; the answer would decide whether the finding is about crypto LOBs specifically or about noisy high-frequency data generally.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper benchmarks six classifiers—logistic regression, XGBoost, CatBoost, CNN+LSTM, CNN+XGBoost, and DeepLOB—on BTC/USDT limit order book snapshots from Bybit sampled at 100 ms. It compares raw data, Kalman-filtered data, and Savitzky–Golay-smoothed data under binary and ternary labeling schemes at 100 ms, 500 ms, and 1000 ms horizons. The central claim is that with data preprocessing and hyperparameter tuning, simpler models (XGBoost, logistic regression) match or exceed deeper neural networks, with faster inference and greater interpretability. The results are presented in four tables reporting F1 scores or accuracy across configurations, plus runtime comparisons.
Significance. If the result holds under a leakage-free protocol, the paper makes a practically useful point: for high-frequency LOB forecasting, input engineering and hyperparameter choice may dominate architectural depth, with implications for latency and interpretability in deployment. The paper uses publicly available data, defines hand-crafted features explicitly, includes multiple model families, and reports training runtimes, which are strengths. However, the empirical evidence is undermined by a likely lookahead bias in the Savitzky–Golay preprocessing, and the claims are made without error bars, significance tests, or multi-day validation. The paper also ships no code, limiting reproducibility of the preprocessing pipeline.
major comments (4)
- [§4.2.1, Eq. (7)] The Savitzky–Golay filter as defined is centered: the smoothed value at time t is a weighted sum of v_{t+j} for j = -10,...,+10, so it uses up to 1 second of future observations. The manuscript never states that the filter is applied one-sidedly, that the smoothed series is shifted by the window half-width before modelling, or that filtering is performed separately within train and test blocks. If the full 100 ms series is filtered before the 80/20 split, every test feature contains information from after the feature timestamp; for the 500 ms and 1000 ms horizons this is direct lookahead. This could explain the pattern in Tables 1 and 2, where Savitzky–Golay is the only filter that consistently improves every model while the causal Kalman filter often degrades accuracy. The authors must rerun the experiments under a clearly causal protocol—for example, a one-sided filter, a shift by the half-window, or filtering applied independently to training and test periods with no future values—and report whether the SG gains survive. As written, the central conclusion that preprocessing/denoising drives performance is not supported.
- [§3, §5.1–5.2, Tables 1–2] The experiments use a single trading day (2025-01-30) and report no confidence intervals, repeated runs, or significance tests. The abstract's claim that simpler models outperform complex neural networks by 1–2% is therefore not statistically grounded; differences of this size across models may be within sampling noise, especially since the test sets are not held constant across configurations (support ranges from 5,442 to 18,336). The authors should add error bars via bootstrapping or repeated train/test splits, and ideally evaluate on additional trading days, to support the global claim.
- [Tables 1 and 2] Prediction horizon and LOB depth are varied simultaneously: the 100 ms rows use a 5-level LOB while the 500 ms and 1000 ms rows use a 40-level LOB. Consequently, the apparent improvement with longer horizons is confounded with the change in input depth, and no clean horizon effect can be extracted. The same confound affects the comparison of filtering methods across rows, because the test sets differ in size and composition. The experimental design should hold depth fixed when varying horizon, or otherwise report a factorial design.
- [§5.3, Table 3] The depth comparison for XGBoost is not controlled: the model is tested on 5,442 snapshots at 40 levels but on 18,336 snapshots at 5 levels. The drop in accuracy from 0.715 to 0.580 may reflect the different test sets (different time periods and noise characteristics) rather than LOB depth alone. The conclusion that 'shallower LOB increases coverage but reduces accuracy' requires a matched test set or a per-support analysis.
minor comments (4)
- [§2.4] There are repeated typos ('Inspired y Sirignano' should be 'Inspired by Sirignano', 'bais' should be 'bias', 'cuased' should be 'caused'). The phrase 'Efficient Market Theorem' should be 'Efficient Market Hypothesis'. The sentence implying that efficiency predicts exactly 50% accuracy conflates market efficiency with unpredictability under a specific loss function.
- [§5 and Table 3] The text says the models are evaluated with F1 score, but the tables are titled 'Classification ... accuracy' and Section 5.3 uses the word 'accuracy' for the metric that appears to be F1. The metric used in each table should be stated consistently.
- [§5.4] The subsection heading contains a typo: 'Logistic Regregression' should be 'Logistic Regression'.
- [§4.2 and §5] The Kalman filter parameters Q and R, the Savitzky–Golay window length and polynomial degree, and the ternary epsilon threshold are all described as tuned on a small sample, but the actual values and tuning procedure are not reported. Providing these details, along with the train/validation split and any code repository, would substantiate the 'hyperparameter tuning' claim.
Circularity Check
No circularity found: the paper's conclusions are empirical benchmark observations, not derivations that reduce to their own inputs.
full rationale
This is an empirical benchmarking study, not a derivation chain. The central claim — that with preprocessing and hyperparameter tuning, simpler models like XGBoost and logistic regression match or exceed deeper networks (Abstract; Conclusion) — is supported by out-of-sample accuracy and F1 results in Tables 1, 2, and 4. The models are trained on an 80/20 split and evaluated on held-out test rows, so the reported accuracies are not defined in terms of the fitted parameters. The handcrafted features in Eqs. (1)–(4) are fixed economic quantities, not fit to the labels. The Savitzky–Golay filter in Eq. (7) uses pre-computed convolution weights from a polynomial least-squares fit on the input series itself; it does not use label information and is not a fitted parameter renamed as a prediction. The Kalman filter in Eqs. (8)–(10) is likewise a preprocessing transform with fixed or grid-searched noise parameters, and the paper explicitly notes its limited tuning. There is no load-bearing self-citation chain: the paper is single-authored and its references are to external prior work (e.g., Savitzky & Golay 1964; DeepLOB; FI-2010), not to a prior result by the same author that is being recycled as a forced premise. The only notable validity concern is that Eq. (7) defines a centered Savitzky–Golay window with j ranging from −10 to +10, meaning each smoothed value uses 10 future snapshots; if filtering is applied before the train/test split without a causal shift, test features could contain lookahead information. That is a data-leakage and real-time-deployment concern, not circularity: the conclusion does not become equivalent to its inputs by construction, and no equation or fitted parameter makes the reported comparison tautological. Accordingly, the circularity score is 0.
Assumptions & free parameters
free parameters (5)
- ternary epsilon threshold =
not reported
- Kalman filter noise variances Q and R =
not reported
- Savitzky-Golay window length and polynomial degree =
window 21, degree 3
- inverse-frequency class weights =
not reported
- XGBoost hyperparameters =
not reported
assumptions (4)
- domain assumption A random walk plus Gaussian noise model is appropriate for LOB features.
- ad hoc to paper Centered Savitzky-Golay smoothing can be used without introducing lookahead leakage, or that lookahead is acceptable.
- domain assumption A single trading day of Bybit BTC/USDT data is representative of LOB microstructure dynamics.
- domain assumption Discarding snapshots with missing depth levels does not bias the evaluation.
Cite this review
Pith. "Pith review of Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer." pith.science (2026). https://pith.science/paper/HFFVHI3O
@misc{pith2026250605764,
author = {Pith},
title = {Pith review of: Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer},
year = {2026},
howpublished = {\url{https://pith.science/paper/HFFVHI3O}},
note = {Machine review of arXiv:2506.05764}
}
read the original abstract
Cryptocurrency price dynamics are driven largely by microstructural supply demand imbalances in the limit order book (LOB), yet the highly noisy nature of LOB data complicates the signal extraction process. Prior research has demonstrated that deep-learning architectures can yield promising predictive performance on pre-processed equity and futures LOB data, but they often treat model complexity as an unqualified virtue. In this paper, we aim to examine whether adding extra hidden layers or parameters to "blackbox ish" neural networks genuinely enhances short term price forecasting, or if gains are primarily attributable to data preprocessing and feature engineering. We benchmark a spectrum of models from interpretable baselines, logistic regression, XGBoost to deep architectures (DeepLOB, Conv1D+LSTM) on BTC/USDT LOB snapshots sampled at 100 ms to multi second intervals using publicly available Bybit data. We introduce two data filtering pipelines (Kalman, Savitzky Golay) and evaluate both binary (up/down) and ternary (up/flat/down) labeling schemes. Our analysis compares models on out of sample accuracy, latency, and robustness to noise. Results reveal that, with data preprocessing and hyperparameter tuning, simpler models can match and even exceed the performance of more complex networks, offering faster inference and greater interpretability.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Zhang, J., Bennett, K. P., & Roberts, S. (2022).Axial-LOB: High-Frequency Trading with Axial Atten- tion. UCL Discovery.https://discovery.ucl.ac.uk/id/eprint/10172667/
-
[2]
Bileki, G. A., Barboza, F., Silva, L. H. C., & Bonato, V. (2022). Order book mid-price movement inference by CatBoost classifier from convolutional feature maps.Applied Soft Computing, 116, 108274. doi:10.1016/j.asoc.2021.108274
arXiv 2022
-
[3]
Berti, L., & Kasneci, G. (2025). TLOB: A Novel Transformer Model with Dual Attention for Stock Price Trend Prediction with Limit Order Book Data.arXiv preprint arXiv:2502.15757.https://doi. org/10.48550/arXiv.2502.15757
-
[4]
Kercheval, A. N., & Zhang, Y. (2015). Modelling high-frequency limit order book dynamics with support vector machines.Quantitative Finance, 15(8), 1315–1329.https://doi.org/10.1080/14697688.2015. 1032546
arXiv 2015
-
[5]
Ntakaris, A., Magris, M., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2018). Benchmark dataset for mid-price forecasting of limit order book data with machine learning methods.Journal of Forecasting, 37.https://doi.org/10.1002/for.2543
doi:10.1002/for.2543 2018
- [6]
-
[7]
Sirignano, J., & Cont, R. (2019). Universal features of price formation in financial markets: perspectives from deep learning.Quantitative Finance
work page 2019
-
[8]
Briola, A., Bartolucci, S., & Aste, T. (2024). Deep Limit Order Book Forecasting.arXiv:2403.09267 [q-fin.TR].https://doi.org/10.48550/arXiv.2403.09267
Show all 25 references
- [9]
- [10]
-
[11]
Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2020). Using Deep Learning for price prediction by exploiting stationary limit order book features.Applied Soft Computing, 93, 106401. ISSN 1568-4946. doi:10.1016/j.asoc.2020.106401
2020
-
[12]
(2022, April 1)
Divakar, V., & Bhattacharyya, R. (2022, April 1). Optimizing Order Types in Indian Futures Market. SSRN Scholarly Paper 4072374.http://dx.doi.org/10.2139/ssrn.4072374
2022 doi
- [13]
-
[14]
Hultin, H., Hult, H., Prouti` ere, A., Samama, S., & Tarighati, A. (2023). A generative model of a limit order book using recurrent neural networks.Quantitative Finance, 23(6), 931–958. doi:10.1080/14697688.2023.2205583
2023
- [15]
- [16]
-
[17]
Chordia, T., Subrahmanyam, A., & Roll, R. W. (2001, November13). Order Imbalance, Liquidity, and Market Returns. SSRN Scholarly Paper 282759.https://ssrn.com/abstract=282759
2001
-
[18]
D., & Howison, S
Xu, K., Gould, M. D., & Howison, S. D. (2019, October 30). Multi-Level Order-Flow Imbalance in a Limit Order Book. SSRN Scholarly Paper 3479741.http://dx.doi.org/10.2139/ssrn.3479741
2019 doi
-
[19]
Dezhkam, A., & Manzuri, M. T. (2023). Forecasting stock market for an efficient portfolio by combining XGBoost and Hilbert–Huang transform.Engineering Applications of Artificial Intelligence, 118, 105626. doi:10.1016/j.engappai.2022.105626
2023
-
[20]
Marsza lek, A., & Burczy´ nski, T. (2024). Modeling of limit order book data with ordered fuzzy numbers. Applied Soft Computing, 158, 111555.https://doi.org/10.1016/j.asoc.2024.111555
2024
- [21]
-
[22]
Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2017). Forecasting stock prices from the limit order book using convolutional neural networks. InProceedings of the IEEE
2017
-
[23]
Dietterich, T. G. (2000). Ensemble methods in machine learning. InMultiple Classifier Systems(pp. 1–15)
2000
-
[24]
M., & Menkveld, A
Hendershott, T., Jones, C. M., & Menkveld, A. J. (2011). Does algorithmic trading improve liquidity? Journal of Finance, 66(1), 1–33
2011
-
[25]
Savitzky, A., & Golay, M. J. E. (1964). Smoothing and differentiation of data by simplified least squares procedures.Analytical Chemistry, 36(8), 1627–1639. 12
1964
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.