{"id":"54800a7e-a2b4-401b-a659-0cef71ac9514","arxiv_id":"1908.01112","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Mid-LSTM, a hybrid LSTM/HMM/linear-regression model, is claimed to improve midterm S&P 500 price forecasts by 2-4% and to raise portfolio Sharpe ratios, but the evidence has selection and labeling problems.","lead":"The paper proposes Mid-LSTM, a neural network that combines LSTM, a hidden Markov model, and linear regression to predict stock prices 30 to 60 days ahead. The authors report small accuracy gains on S&P 500 data, but the headline 120.16% annual return appears to come from a non-annualized table, and no code or error bars are provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The midterm evaluation uses recursive multi-step prediction, but the LSTM is trained only one-step-ahead; no closed-loop training or error-accumulation analysis is provided, so the central empirical claim is unsupported.","rationale":"The reader's weakest assumption identifies the same gap, and I agree it is the most load-bearing concern. The paper's value is empirical: a midterm forecasting tool with better accuracy and lower risk. That value depends on the model operating in the same regime as the evaluation. The described training objective never exposes the LSTM to its own predictions, so the claimed anomaly-circumvention property is not demonstrated for the actual use case. This is a missing-evidence objection, not a claim that the model is false; a teacher-forcing diagnostic would settle it. Secondary but real problems reinforce rejection: Eq. (5) presents Mid-ARMA as CAPM-based but no derivation from Eq. (3) is given, and the correlation coefficient replaces the market beta without justification; the HC-subset comparisons in Tables 1-2 compare the 50 high-correlation stocks against all-stock baselines, so the reported 12% and 4% margins are not matched; and no error bars or code are provided for the 451-run averages. These are not needed for the verdict because the recursive mismatch alone leaves the central claim unverified. I therefore keep the reader's REJECT verdict; per the rubric, verdict_should_be is UNCHANGED.","tokens_in":13020,"tokens_out":9795,"duration_ms":91374,"concrete_test":"On the same S&P 500 split, run the trained Mid-LSTM over the six 60-day testing windows in two modes: (a) recursive, exactly as described in Section 4.1; and (b) teacher-forced, replacing each predicted input price, volume, and market value with the true observed value before the next step. Compute MPA over days 30-60 and the portfolio metrics of Tables 3-4 for both modes. If recursive MPA is materially lower than teacher-forced MPA (e.g., more than the claimed 2-4% margin), the reported midterm advantage is an artifact of unmodelled error accumulation and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's own Section 1 states that the main difficulty of midterm prediction is that 'the latter predicted prices are based on the previous predicted prices' and errors accumulate. But the training setup in Sections 3.2.2 and 4.1 is one-step: training outputs have length 1, and the L2 loss in Eq. (10) uses actual lagged values as inputs. Evaluation, by contrast, is full-sequence recursive: Section 4.1 feeds predicted values back into the window for 30-60 days. No closed-loop training, scheduled sampling, teacher-forcing diagnostic, or error-growth analysis is provided. The reported MPA gains and portfolio returns in Tables 1-4 are produced entirely in this recursive regime, so every downstream component (HMM states, correlation coefficient, linear-regression refinement) operates on LSTM outputs whose stability is unexamined. This is the load-bearing gap: if the one-step model diverges under feedback, the 2-4% accuracy claim and the 120.16% return are not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Mid-LSTM, a hybrid predictor for 30–60-day S&P 500 stock prices. It combines an ARMA-type linear structure with LSTM outputs, hidden Markov model (HMM) states relating price and volume, a CAPM-inspired market term, and a final linear regression refinement. Experiments on 451 stocks over six rolling windows report mean prediction accuracy (MPA) 0.9308 for Mid-LSTM versus 0.9258 for LSTM, trend accuracy 0.846, and portfolio backtests with average Sharpe ratios up to 2.99 and a claimed 120.16% annual return.","tokens_in":13344,"tokens_out":8024,"duration_ms":79754,"significance":"If substantiated, Mid-LSTM would be a practically useful midterm forecasting tool and an instructive example of injecting financial structure into recurrent forecasting. The paper has concrete strengths: it evaluates on held-out rolling windows, compares against standard baselines, reports both price accuracy and trend accuracy, and provides enough architectural detail to reimplement. However, the empirical support for the headline claims is currently weak. The main accuracy gain over LSTM is about half a percentage point with no uncertainty quantification; the headline return figure is not the annualized number in the paper's own tables; and the subset result that produces the largest gains is not matched with baseline methods on the same subset.","major_comments":[{"comment":"The LSTM is trained one step ahead using real lagged values as inputs (training output length 1 in Section 4.1), but the midterm evaluation is a recursive full-sequence forecast in which predicted values are fed back into the window for 30–60 days. The paper itself identifies cumulative error as the main difficulty of midterm prediction, yet no closed-loop training, scheduled sampling, teacher-forcing diagnostic, or error-growth analysis is provided. All of the reported MPA, TA, and portfolio results in Tables 1–4 are produced in this recursive regime, so the central empirical claim is unsupported unless the stability of the one-step model under feedback is demonstrated.","section":"§3.2.2, §4.1, Eq. (10)"},{"comment":"The abstract's claim of a '2–4% improvement in prediction accuracy' is not consistent with the reported numbers. Table 1 gives mean MPA 0.9308 for Mid-LSTM versus 0.9258 for LSTM, an absolute difference of 0.0050 and roughly a 0.5% relative improvement; the gain over linear regression is similar. No confidence intervals, standard errors, or repeated-seed results are reported for any MPA or TA number, so the 0.005 advantage cannot currently be distinguished from noise. The authors should state exactly which metric supports the 2–4% claim and add uncertainty quantification.","section":"Abstract, Tables 1–2"},{"comment":"The abstract's claim of 'up to 120.16% annual return' is contradicted by Table 4, where the largest annualized return for Mid-LSTM is 42.95% (R-2) and the average annualized return is 26.51%. The value 120.16 appears in Table 3 as the period cumulative return for R-6, not an annualized return. The abstract should either report the annualized figures from Table 4 or explicitly describe 120.16% as a window-specific cumulative return.","section":"Abstract, Tables 3–4"},{"comment":"The strongest accuracy results are reported for the 50-stock subset 'highly correlated with the market' (Mid-LSTM HC: MPA 0.9637, TA 0.9200), but no baseline method is evaluated on that same subset. Because the subset is selected using training-data market correlation, the comparison to full-sample baselines is not matched, and the improvement could be an artifact of selecting easier-to-predict stocks. The authors should provide linear, ridge, random forest, and LSTM results on the same 50 stocks.","section":"§4.4, Tables 1–2"},{"comment":"The min-max normalization in Eq. (19) appears to be computed with the minimum and maximum over the full observed series, which would use future information when normalizing training data and could leak test-period information into the model. The paper should specify whether the normalization statistics are computed on each rolling training window only, and if the current implementation uses full-sample min/max, the experiments should be rerun with causal normalization.","section":"§3.2.1, Eq. (19)"}],"minor_comments":[{"comment":"The data description is inconsistent: the text says the sample runs from 01/02/2009 to 12/24/2018, but the stated training period starts 11/09/2007 and the figure shows 2007 dates. Please correct the date ranges.","section":"§4.1"},{"comment":"The 'CAPM-based' term in Eq. (5) is asserted rather than derived from Eq. (3); it replaces expected returns with the market index level and beta with a correlation coefficient. Since the coefficients are fitted by regression, the model may be better described as a flexible linear regression with a market interaction term rather than a CAPM-constrained model.","section":"§2.2, Eq. (5)"},{"comment":"The number of training iterations is fixed at 10 with no convergence criterion, validation-based early stopping, or discussion of stability across initializations. Please provide training curves or a stopping rule.","section":"§3.2.2"},{"comment":"The four HMM states are predefined qualitatively (e.g., 'large trading volume, high stock price'), so it is unclear what the HMM learns. If the states are manually labeled rather than inferred, the 'hidden state' claim should be softened.","section":"§3.1.2, §3.2.2"},{"comment":"Several typos and display artifacts need correction: 'Afterwords' should be 'Afterwards', 'The first it to use' should be 'The first is to use', and Figure 5's subplot labels are not legible in the present PDF.","section":"Figures and typography"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a KDD workshop paper; for a full archival venue the empirical standard needs to be raised substantially. The recursive-evaluation mismatch and the abstract/table inconsistencies are fixable, but the authors will need to rerun experiments with closed-loop training or provide error-accumulation diagnostics, add uncertainty quantification, and correct the headline numbers before the central claims are supportable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the Mid-LSTM idea is a legitimate, if incremental, combination of LSTM, HMM, and a CAPM-flavored linear regression. That part is fine. What sinks the paper is the evaluation: they train the LSTM one step ahead (training output length 1, Section 4.1) but then evaluate it by feeding its own predictions back for 30–60 days. They even identify cumulative error as the central difficulty in the introduction, yet provide no closed-loop training, no scheduled sampling, no error-growth analysis. The MPA and Sharpe numbers all come from that recursive regime, so the main claim is unsupported.\n\nA few other soft spots, in rough order of severity:\n\n- The abstract says \"up to 120.16% annual return,\" but Table 4, which reports annualized returns, peaks at 42.95%. The 120.16 in Table 3 is a single-window cumulative return, not annualized. That's an apples-to-oranges headline.\n- The high-correlation (HC) subset results (Mid-LSTM HC: 0.9637 MPA, 0.92 TA) compare against all-stock baselines, not against baselines run on the same 50 stocks. Without matched baselines, the \"especially for highly correlated stocks\" claim could just be an artifact of easier-to-predict stocks.\n- Equation (5) is asserted, not derived. The coefficients are fitted on training data. That doesn't make it circular—the accuracy evaluation is on held-out test windows—but the \"CAPM-based\" framing is decorative, not structural.\n- No error bars, no code, no data release. For a paper whose entire value is empirical, that's a real gap.\n\nCredit where due: the paper is clearly written, the rolling-window setup is described, and the idea of using HMM to extract a volume–price state and feeding that into a linear refinement is not something I've seen before. The trend-accuracy metric is a reasonable addition. But the load-bearing empirical claim doesn't survive scrutiny.\n\nWho's this for? Someone working on hybrid classical/ML pipelines for midterm equity forecasting might get ideas from the architecture. But as a claimed result, it needs major revision: run matched baselines on the HC subset, add error-accumulation diagnostics or switch to closed-loop training, fix the return claim, and release code/data.\n\nMy recommendation: if this lands on your desk, don't desk-reject out of hand—send it to a referee who can ask for those specific revisions. It's a serious paper with a fixable flaw, not a throwaway.","headline":"Useful architecture idea, but the empirical evaluation is undermined by a train-on-one-step/evaluate-recursively mismatch and an inflated return claim.","tokens_in":13778,"tokens_out":2037,"would_cite":false,"duration_ms":19512,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid deep network, Mid-LSTM, claims to sidestep market anomalies and lift midterm stock forecasts by 2–4%.","keywords":["midterm stock prediction","LSTM","hidden Markov model","capital asset pricing model","anomaly circumvention","portfolio allocation","S&P 500","ARMA"],"falsifier":"The cleanest check is a walk-forward retraining experiment: after each observed month, retrain Mid-LSTM on all data up to that point and predict the next 30–60 days; then compare the rolling mean prediction accuracy and the resulting portfolio Sharpe ratio with the fixed-split numbers reported in the paper. If the accuracy advantage over linear regression and plain LSTM shrinks to noise, or if the recursive 30–60 day predictions are no better than point-by-point forecasts that use true prices, the central claim is falsified.","tokens_in":12855,"feed_emoji":"📈","tokens_out":8475,"duration_ms":77063,"temperature":0.7,"pith_summary":"Midterm stock-price prediction (30–60 days ahead) is hard because errors compound when predicted prices are fed back into the model; this paper argues that the root cause is that ordinary networks cannot avoid anomalies, and proposes a hybrid network, Mid-LSTM, designed to circumvent them. Mid-LSTM chains three components: an LSTM predicts stock price, market index, and volume; a hidden Markov model extracts four hidden states linking price and volume; and a linear regression layer combines these with a CAPM-style market-correlation term to refine the forecast. The paper reports that this pipeline improves mean prediction accuracy over linear regression, ridge regression, random forest, and plain LSTM by 2–4%, and that portfolios built on its predictions earn up to 120.16% annualized return with an average Sharpe ratio of 2.99 on S&P 500 stocks. The central claim, in the authors' terms, is that anomaly circumvention is what makes midterm forecasting accurate enough for value investing.","feed_headline":"LSTM that dodges market anomalies lifts midterm forecasts 2-4%","feed_subtitle":"Hybrid network adds hidden market states and volume patterns; portfolio test posts up to 120% annual return.","key_machinery":"The load-bearing object is the Mid-ARMA model of equation (5), which expresses the predicted price as a weighted sum of an ARMA forecast, a CAPM-inspired market term $\\rho(\\lambda M_t^A + \\eta)$, and a hidden-state term $\\gamma S_t^A$; Mid-LSTM is the neural realization of that equation. The LSTM produces the component forecasts $X_t^A$, $M_t^A$, $V_t^A$; the HMM (with $K=4$ states corresponding to high/low volume crossed with high/low price) produces $S_t^A$; and the linear regression layer learns the scalar weights $\\alpha,\\eta,\\lambda,\\gamma,c$ by minimizing a mean-square loss over the 30–60 day prediction horizon. The mechanism's job is to let the model absorb anomalous market and volume movements into the market and hidden-state terms so they do not propagate through the recursively generated price sequence.","core_discovery":"On its own terms, the paper's discovery is that a model which deliberately routes market-wide shocks away from the price path can sustain recursive multi-step forecasting. The proposed Mid-LSTM implements the Mid-ARMA equation, $\\hat{X}_t = \\alpha X_t^A + \\lambda\\rho M_t^A + \\eta\\rho + \\gamma S_t^A + c$, where $X_t^A$ is an ARMA forecast, $M_t^A$ is an ARMA forecast of the market index, $\\rho$ is the correlation between stock and market, and $S_t^A$ is a hidden-state variable extracted from the joint behavior of price and volume via a four-state HMM. The LSTM supplies $X_t^A$, $M_t^A$, and $V_t^A$; the linear regression learns the weights. The authors report that on 451 S&P 500 stocks over six testing windows, this design raises mean prediction accuracy to 0.9308 versus 0.9258 for plain LSTM and 0.9253 for linear regression, raises trend accuracy to 0.8460 (0.9200 for the 50 most market-correlated stocks), and in the second asset set yields an average annualized return of 26.51% and an average Sharpe ratio of 2.99.","pith_inferences":["A natural test the paper leaves implicit is whether Mid-LSTM's advantage grows with the severity of market anomalies; restricting the test set to high-volatility periods (e.g., the 2018 Q4 sell-off) and comparing error growth against calm periods would directly probe the anomaly-circumvention mechanism.","The HMM's K=4 states are fixed in advance; a variant that learns the number of states from data could reveal whether the four volume-price regimes are the right granularity or whether more states improve anomaly detection.","The reported 120.16% annualized return comes from a single six-window testing span; averaging over many non-overlapping out-of-sample windows would tell whether the edge is systematic or period-specific.","If the one-step LSTM actually diverges under recursive prediction, one could add a feedback-correction layer (e.g., retraining on predicted sequences) and check whether the accuracy gain persists; this is an extension the paper does not consider."],"forward_implications":["If the claim holds, value investors can use the model's 30–60 day forecasts as a risk filter: stocks highly correlated with the market get the largest accuracy benefit, so capital can be concentrated there.","The linear-regression weights give analysts an interpretable decomposition of each price forecast into ARMA, market, and volume-hidden-state contributions, which can flag when a prediction is being driven by an anomaly.","The trend-accuracy results imply the model is better used for directional decisions (up/down over a month) than for exact price levels, since trend accuracy reaches 0.92 for market-correlated stocks.","Because the framework is horizon-agnostic, the same Mid-ARMA/Mid-LSTM recipe can be retargeted to other horizons by changing the window length and the number of HMM states."],"supporting_citations":[{"why":"Supplies the ARMA time-series model that Mid-ARMA extends.","marker":"[2]"},{"why":"Supplies the CAPM relationship between stock and market that motivates the market term in Mid-ARMA.","marker":"[13]"},{"why":"Supplies the LSTM memory-cell architecture used by the prediction network.","marker":"[8]"},{"why":"Supplies the hidden Markov model formulation used to extract hidden states from price and volume.","marker":"[16]"},{"why":"Establishes that LSTM can detect anomalies in time series, motivating the anomaly-circumvention claim.","marker":"[12]"},{"why":"Reports the empirical volume-price associations that justify the four hidden states in the HMM.","marker":"[22]"}],"fun_headline_variants":["Anomaly-avoiding LSTM lifts midterm forecasts 2-4%","Mid-LSTM skips anomalies, lifts accuracy and returns","Hybrid LSTM dodges shocks, gains 2-4% accuracy","Stock LSTM routes around anomalies for midterm gains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a network trained on one-step-ahead targets keeps producing accurate outputs when its own predictions are fed back into the window for 30 to 60 days, with no error accumulation—an assumption the paper asserts but does not test.","fun_headline_variants_meta":{"raw":{"variants":["Anomaly-avoiding LSTM lifts midterm forecasts 2-4%","Mid-LSTM skips anomalies, lifts accuracy and returns","Hybrid LSTM dodges shocks, gains 2-4% accuracy","Stock LSTM routes around anomalies for midterm gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1349,"prompt_tokens":1021,"completion_tokens":328,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":252}},"tokens_in":637,"tokens_out":328,"duration_ms":3832,"temperature":1.0,"reasoning_tokens":252,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:23:41.377925+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The cleanest check is a walk-forward retraining experiment: after each observed month, retrain Mid-LSTM on all data up to that point and predict the next 30–60 days; then compare the rolling mean prediction accuracy and the resulting portfolio Sharpe ratio with the fixed-split numbers reported in the paper. If the accuracy advantage over linear regression and plain LSTM shrinks to noise, or if the recursive 30–60 day predictions are no better than point-by-point forecasts that use true prices, the central claim is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the CAPM relationship between stock and market that motivates the market term in Mid-ARMA."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the hidden Markov model formulation used to extract hidden states from price and volume."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that LSTM can detect anomalies in time series, motivating the anomaly-circumvention claim."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports the empirical volume-price associations that justify the four hidden states in the HMM."}],"review_version":1}