{"id":"ff49c0d8-4f96-4ca4-8958-42bfee3d9542","arxiv_id":"2412.19372","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"ALPE, an online reinforcement-learning regressor, is reported to beat batch ML models for mid-price forecasting, but the evaluation likely leaks the target into the inputs.","lead":"This paper forecasts stock mid-prices from limit order book data using an online reinforcement-learning agent, ALPE, and reports lower errors than batch machine-learning baselines across 100 U.S. stocks. The central claim depends on whether the predicted price is unknown at prediction time, and the paper's own equations raise doubts by including the target in the inputs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-lag protocol feeds the target p_t into ALPE via Extended feature u2 and the Eq. (17) exploration action, so the reported error advantage does not yet establish predictive skill.","rationale":"I read the paper as attempting to demonstrate a real-time mid-price forecasting advantage for ALPE. The strongest claim is that ALPE consistently achieves the lowest RMSE/RRMSE across 100 stocks and six feature sets. The reader's weakest assumption identifies the most load-bearing condition: the target must not be available to the model at the same timestamp. I checked the manuscript and found the same internal inconsistency: Eq. (1) defines p_t from current bid/ask; Table 1's Extended feature u2 is algebraically identical to p_t; the protocol is described as zero-lag and ALPE's window is one current LOB state. If the recorded target is p_t, the comparison is contaminated. This is not a disagreement with consensus; it is an internal inconsistency in the paper's own definitions. I also note Eq. (17) injects p_t into ALPE's exploration action and Eq. (24) makes the target depend on the current prediction; both are separate contamination routes, but the u2 input leak is the cleanest to test. A second, smaller inconsistency is the Amazon Extended-GD RRMSE quoted in the conclusion (2.484E-04) versus Table 3 (2.944E-04). There is no independent support (code, data, or formal verification) that would allow a reader to resolve these ambiguities. The concrete test above would settle whether the concern lands: removing u2 and p_t from the exploration action while re-estimating one representative result is a minimal, decisive check. I therefore keep the reader's REJECT verdict and recommend no change.","tokens_in":107998,"tokens_out":5841,"duration_ms":59975,"concrete_test":"Inspect the evaluation pipeline to determine whether the target used in RMSE/RRMSE is p_t or p_{t+1}. If it is p_t (as Eqs. 1, 17, and 19 indicate), rerun the Amazon Exte experiment with u2 removed from Table 1 and with the exploration branch of Eq. (17) changed from p_t + α^e_t to α^e_t, holding all other hyperparameters fixed. If the ALPE Exte RMSE rises from 2.527E-02 to the same order as competitor rows (≈1E-01), the reported advantage is leakage, not forecasting skill. If the true target is instead p_{t+1}, repeat with the target properly shifted and report whether the claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical superiority of ALPE. For that claim to hold, the RMSE/RRMSE values must measure out-of-sample forecasting error for a future or contemporaneous mid-price that is not an input. The paper's equations are inconsistent with that condition. Eq. (1) defines p_t=(P^ask_t+P^bid_t)/2; Table 1 defines Extended feature u2=(P^ask_1+P^bid_1)/2, i.e., p_t; and §3.1 describes an event-by-event, zero-lag protocol with ALPE using only the current LOB state. If the recorded target is p_t, then the Extended inputs contain the target and the reported advantage is leakage rather than forecasting. Independently, Eq. (17) sets the exploration action to a_t = p_t + α^e_t, injecting the true current mid-price into ALPE's action whenever exploration fires, which can depress RMSE even under the Simple feature set. Eq. (24) also defines the training target using the current network output, creating a self-referential regression target. The conclusion's Amazon Extended-GD RRMSE (2.484E-04) does not match Table 3 (2.944E-04), which further undermines confidence in the reported numbers. The absence of released code/data means the target alignment cannot be checked from the manuscript alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ALPE, a reinforcement-learning agent for event-by-event mid-price forecasting using Level 1 limit order book data from 100 S&P 500 stocks over September-November 2022. The agent is compared against Naive, ARIMA, MLP, CNN, LSTM, GRU, and RBFNN across six feature sets using RMSE and RRMSE, and the paper claims that ALPE consistently achieves the lowest forecasting error. The central claim is not supported by the stated evaluation design: the target mid-price p_t appears directly as an input feature in the Extended set, the exploration action in Eq. (17) injects p_t into the agent's output, and the training target in Eq. (24) is a function of the network's own current output. These issues mean the reported error numbers do not measure predictive skill under the zero-lag protocol described in the manuscript.","tokens_in":108268,"tokens_out":3772,"duration_ms":40824,"significance":"If the forecasting claim were valid, the paper would provide a useful large-scale comparison of an adaptive, batch-free mid-price forecaster across 100 stocks, with a potentially useful normalized error metric in RRMSE. The GD/MDI feature-importance pipeline and the event-by-event protocol are also relevant to the HFT forecasting literature. However, the paper provides no code or data, and the load-bearing evaluation is compromised by direct target leakage through the features and the action construction. The manuscript therefore does not currently establish that ALPE has predictive skill, and the quantitative claims in the abstract, Section 4, and Section 5 are not interpretable as forecasting performance as stated.","major_comments":[{"comment":"The target variable is included in the model input. Equation (1) defines p_t = (P^ask_t + P^bid_t)/2, and Table 1 defines the Extended feature u2 = (P^ask_1 + P^bid_1)/2, which is exactly p_t at the same event. Under the zero-lag, event-by-event protocol described in §3.1, where the state at event t is used to predict the mid-price at event t, the Extended feature sets give every model direct access to the target. The reported RMSE and RRMSE values therefore measure target reconstruction and leakage, not forecasting error. This invalidates the central performance claim as stated.","section":"§3.1, §3.2, Table 1"},{"comment":"The exploration action injects the contemporaneous target into the agent's output. Equation (17) sets a_t = p_t + α^e_t whenever exploration fires, so ALPE's reported output is partly or wholly constructed from the true mid-price at the same event. This leakage affects even the Simple feature sets and gives ALPE an advantage that has nothing to do with learned forecasting ability. The reward function in Eq. (19) also evaluates the action against p_t, reinforcing the same contemporaneous dependence.","section":"§3.3.1, Eq. (17)"},{"comment":"The training target is self-referential. Equation (24) defines f_{π,target} as an expression that includes f_π(s_t, a_t; θ_ALPE), the current network output, and Eq. (25) minimizes the squared difference between the network output and this target. Because the target moves with the estimator, the loss does not define a fixed external regression target and can be made small by construction. This undermines the claim that the network learns a policy or value function from the reward signal.","section":"§3.3.3, Eqs. (24)-(25)"},{"comment":"The headline quantitative claim in the conclusion is inconsistent with the reported table. Section 5 states that for Amazon under Extended GD, ALPE achieved an RRMSE of 2.484E-04, but Table 3 reports an RRMSE of 2.944E-04 for ALPE under Exte GD. This discrepancy in the paper's central example, together with the target-leakage issues above, makes it impossible to take the numerical superiority claims at face value without access to code and data.","section":"§5 vs. Table 3"}],"minor_comments":[{"comment":"The text refers to 'our proposed DQR model,' but the model introduced in the paper is ALPE; this appears to be a leftover from an earlier draft.","section":"§3.1"},{"comment":"For Amazon Simple, the GRU row reports an RRMSE of 9.206E-03 alongside an RMSE of 1.048E-01; this is an order of magnitude larger than the other RRMSE values and is inconsistent with the stated relationship RRMSE = RMSE/p_t, suggesting a typographical error.","section":"Table 3"},{"comment":"The ticker 'OALPEY' in Table A.9 appears to be a typo for ORLY, and 'L VS' in Table 2 and Table A.6 appears to be a typo for LVS.","section":"Table A.9 and Table 2"},{"comment":"There are numerous typographical and formatting errors, including 'definied' in Eq. (2), 'kernalised' in §3.2, 'V AR' in Section 2, and inconsistent use of 'Extensive' versus 'Extended.' These should be corrected in any revision.","section":"Throughout"},{"comment":"The notation around the action is confusing: α_t, α^e_t, and the action bounds a_min, a_max are introduced without making clear whether α_t is a parameter or an action variable, and Eq. (17) mixes the exploration perturbation with the learned policy output. Clarifying this notation would help, but it does not resolve the target-leakage problem.","section":"Eq. (17) and surrounding text"}],"recommendation":"reject","confidential_remarks":"For the editor: the central evaluation protocol is compromised as written, and the concerns are internal to the manuscript's own equations rather than a matter of disagreement with prevailing consensus. The paper also provides no code or data, so the target-alignment issue cannot be checked or fixed by reviewers. A resubmission would need to redefine the forecasting target as a genuinely future mid-price, remove p_t from the feature vector and the action construction, replace the self-referential training target with an external loss, and provide code or data enabling verification. As it stands, the reported results do not establish predictive skill."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the core claim — that ALPE consistently beats ARIMA, MLP, CNN, LSTM, GRU, and RBFNN across 100 S&P 500 stocks — is not supported by the experiments as described. Second, the reason is not tuning or benchmarking noise; it is that the target mid-price is built into the model's own inputs. Eq. (1) defines p_t as the average of the best bid and ask, and Table 1 defines the Extended feature u2 as exactly that average. Under the stated zero-lag, event-by-event protocol, a model that sees u2 is reading the answer. Eq. (17) makes it worse: whenever exploration fires, the action is p_t plus noise, so the reward function can be gamed even with the Simple features. The self-referential training target in Eq. (24) — the target depends on the current network output — is a third circularity. These are not minor blemishes; they invalidate the reported RMSE/RRMSE numbers as measures of forecasting skill.\n\nThere is real work here that deserves credit. The 100-stock benchmark, the six input configurations, the rolling-window comparison, and the Friedman/Conover statistical testing are all a step beyond the prior RBFNN paper. The RRMSE normalization idea is reasonable for cross-stock comparison. And the paper is honest about wanting a batch-free, online policy, which is a legitimate practical goal. But the central empirical contribution is built on a protocol that leaks the target, and the conclusion's Amazon RRMSE for Extended-GD (2.484E-04) does not match Table 3 (2.944E-04), which does not inspire confidence that the numbers were checked.\n\nI also note there is no code or data release, so the target alignment cannot be independently verified from the manuscript. That matters especially when the paper's own equations contradict the claims.\n\nWho is this for? A reader who wants a cautionary example of target leakage in high-frequency forecasting, or someone doing a literature review on RL for LOB prediction. It is not a reliable source for a practical forecasting result. If the authors re-run with a next-event mid-price target, remove u2, fix Eq. (17) and Eq. (24), and release code, the underlying benchmark could become useful. As it stands, the paper deserves a serious referee only to document the leakage and send it back. My recommendation: do not desk-reject silently, but do not publish the empirical claim as written.","headline":"The Extended feature set feeds the target mid-price straight into the model, so the headline forecasting advantage is leakage, not skill.","tokens_in":108799,"tokens_out":1583,"would_cite":false,"duration_ms":20692,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ALPE, a batch-free reinforcement learning agent, is claimed to achieve the lowest mid-price forecasting error across 100 S&P 500 stocks.","keywords":["reinforcement learning","high-frequency trading","mid-price forecasting","limit order book","S&P 500 stocks","Adaptive Learning Policy Engine","RRMSE"],"falsifier":"Re-run ALPE with the mid-price feature $u_2$ removed from all feature sets, or with every feature lagged one event so the target is $p_{t+1}$; if a trivial model that outputs $u_2$ matches ALPE's reported RMSE, then the zero-lag protocol leaks the answer.","tokens_in":107758,"feed_emoji":"📈","tokens_out":9641,"duration_ms":83373,"temperature":0.7,"pith_summary":"The paper claims that ALPE, a reinforcement learning agent built for batch-free, event-by-event operation, forecasts the limit order book mid-price more accurately than a range of statistical and deep learning benchmarks. The evaluation uses Level 1 NASDAQ data for 100 S&P 500 stocks over September to November 2022, with six input feature sets and two error metrics, RMSE and the newly proposed RRMSE. The reported result is that ALPE achieves the lowest forecasting error in every setting, and that its advantage over ARIMA, Naive, MLP, CNN, LSTM, GRU, and RBFNN is statistically significant in most pairwise comparisons. The paper's case for caring is that a model requiring only the current LOB state, with no batch retraining, is better suited to the non-stationary, rapidly moving conditions of high-frequency trading.","feed_headline":"ALPE agent beats eight models at mid-price forecasting","feed_subtitle":"Paper reports lowest RMSE and RRMSE on 100 S&P 500 stocks across six feature sets.","key_machinery":"The load-bearing mechanism is the ALPE agent: a Markov decision process over LOB states with continuous actions bounded in $[-0.1, 0.14]$, a reward $R_t = -|a_t - p_t|\\,(1 - \\epsilon_t)$, a discount factor $\\gamma = 0$, and an epsilon-greedy policy with adaptive decay $\\epsilon_{t+1} = \\max(0.0001, \\epsilon_t \\cdot 0.9996)$. The exploitation network is an MLP with eight hidden layers of 64 ReLU units, batch normalization, and a single linear output, trained for two epochs with the Adam optimizer. Feature importance transforms (MDI and GD) and the RRMSE metric are used to build input variants and to compare errors across stocks.","core_discovery":"The central discovery reported is that a model-free, value-based RL agent with an eight-hidden-layer MLP policy network, adaptive epsilon decay, and a reward that penalizes distance from the true mid-price can outperform batch-trained alternatives on mid-price forecasting. The authors report consistent lowest RMSE and RRMSE scores across 100 stocks, all six feature sets, and all benchmark models, and they support this with a Friedman test followed by Conover post-hoc pairwise comparisons. The paper interprets this as evidence that minimal-batch, zero-lag reinforcement learning adapts to market shifts in real time and provides a practical route to low-latency forecasting.","pith_inferences":["We infer that the reported errors may largely measure how faithfully ALPE copies the mid-price feature $u_2$ from its input, since the target at event $t$ is also present as a feature; removing $u_2$ or shifting the target to $p_{t+1}$ would test whether any genuine predictive skill remains.","RRMSE could serve as a broadly useful normalized error metric for comparing forecasting models across assets of different price scales, but its value in this paper is entangled with the same target-leak issue because the denominator is the predicted quantity.","A stock-specific or volatility-dependent epsilon decay schedule is a natural extension; the paper itself notes the fixed decay factor is not ideal for low-volume stocks, and our reading suggests this is where ALPE's gains are least consistent.","The event-driven, one-state protocol could be reframed as a one-event-ahead forecasting benchmark, which would connect this RL setup to the large body of order-flow and mid-price movement prediction tasks."],"forward_implications":["If ALPE's reported advantage is real, online reinforcement learning with a one-state look-back can beat batch-trained deep networks, which would make low-latency adaptive forecasting feasible in practice.","The adaptive epsilon-decay schedule, fixed across all stocks, offers a single, tuning-light exploration-exploitation balance that the paper argues is sufficient across a 100-stock universe.","RRMSE, by normalizing RMSE by the event mid-price, gives a stock-independent error scale that the paper shows is especially informative for lower-volume names.","A two-epoch training budget on a single LOB state means the policy network can be updated per event, a property the paper links to the approximately 10-event stationary windows observed in LOB data.","The statistically significant pairwise wins over most baselines, as reported, indicate the error reductions are unlikely to be chance fluctuations."],"supporting_citations":[{"why":"Provides the RBFNN baseline and the MDI and GD feature importance methods used to construct the six input feature sets.","marker":"[1]"},{"why":"Supplies the feature engineering and the 10-event rolling window protocol adopted for the batch-trained competitor models.","marker":"[38]"},{"why":"The gradient descent feature importance procedure used to create the GD-adjusted input features.","marker":"[39]"},{"why":"The Adam optimizer used to train the ALPE policy network with a minimal two-epoch budget.","marker":"[40]"}],"fun_headline_variants":["ALPE RL agent outperforms eight models at mid-price forecasting","Adaptive RL engine beats batch models on 100 S&P 500 stocks","Zero-lag RL policy engine tops ML/DL in mid-price prediction","Model-free RL agent wins mid-price forecasting on NASDAQ data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The protocol assumes the model's input at event $t$ does not contain the mid-price $p_t$ that is being predicted, but the Extended feature set includes $u_2 = (P^{ask}_t + P^{bid}_t)/2$, which is exactly $p_t$, so the reported errors depend on that assumption being satisfied.","fun_headline_variants_meta":{"raw":{"variants":["ALPE RL agent outperforms eight models at mid-price forecasting","Adaptive RL engine beats batch models on 100 S&P 500 stocks","Zero-lag RL policy engine tops ML/DL in mid-price prediction","Model-free RL agent wins mid-price forecasting on NASDAQ data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000557,"raw_usage":{"total_tokens":2589,"prompt_tokens":825,"completion_tokens":1764,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":1688}},"tokens_in":441,"tokens_out":1764,"duration_ms":12562,"temperature":1.0,"reasoning_tokens":1688,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:39:40.333193+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run ALPE with the mid-price feature $u_2$ removed from all feature sets, or with every feature lagged one event so the target is $p_{t+1}$; if a trivial model that outputs $u_2$ matches ALPE's reported RMSE, then the zero-lag protocol leaks the answer.","supporting_citations":[{"cited_title":"Online High-Frequency Trading Stock Forecasting with Automated Feature Clustering and Radial Basis Function Neural Networks","cited_arxiv_id":"2412.16160","evidence_quote":"Provides the RBFNN baseline and the MDI and GD feature importance methods used to construct the six input feature sets."},{"cited_title":"Ntakaris, G","cited_arxiv_id":null,"evidence_quote":"Supplies the feature engineering and the 10-event rolling window protocol adopted for the batch-trained competitor models."},{"cited_title":"Optimum Output Long Short-Term Memory Cell for High-Frequency Trading Forecasting","cited_arxiv_id":"2304.09840","evidence_quote":"The gradient descent feature importance procedure used to create the GD-adjusted input features."}],"review_version":1}