{"id":"b424b982-7d4b-4d06-820e-1d9157b84b7a","arxiv_id":"2507.01964","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"The paper reports 93 to 98 percent 'accuracy' for an LSTM on Nigerian stocks, but the metric is undefined and the result most plausibly reflects autocorrelation of price levels rather than skill in predicting returns.","lead":"An LSTM neural network was trained on 21 years of Nigerian stock prices and is reported to predict future prices with 93 to 98 percent accuracy. The accuracy metric is never defined, no naive baseline or code is provided, and several error values in the results tables are internally inconsistent, so the headline claim cannot be verified as stated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 93-98% 'accuracy' is never defined and is contradicted by impossible error rows (WAPCO MAE>RMSE; AIICO RMSE not sqrt(MSE)), so the central return-prediction claim has no coherent empirical basis.","rationale":"The reader's weakest assumption is exactly where the paper breaks. I agree that the undefined 'accuracy' metric is the load-bearing element, and the internal numerical contradictions make the problem worse than a mere labeling gap. If 'accuracy' were, for example, R-squared on price levels, a naive last-value predictor would often score above 0.9 on persistent series, so the 93-98% figures would not imply forecasting skill for returns. The abstract explicitly claims prediction of 'future stock market prices and returns with over 90% accuracy', so the metric must be return-relevant; the paper never shows that. The impossible MAE/RMSE pairs are not minor typos because the inconsistencies appear within rows (WAPCO: MAE=4.474, RMSE=0.7401; AIICO: RMSE=0.544, MSE=0.00285), and BERGER's RMSE differs between Table 3 and Table 4 for the same LSTM result. These problems indicate the numbers were not produced by one reproducible evaluation. The lack of a documented chronological split is a further risk, but even with a chronological split the metric problem would remain. No code, data, or baseline is provided, so there is no independent way to verify the claim. The appropriate disposition remains rejection; the paper would need a defined metric, corrected tables, a chronological split, and a persistence baseline before the central claim could be conditionally credited. I therefore see no reason to alter the reader's verdict.","tokens_in":7452,"tokens_out":3669,"duration_ms":40763,"concrete_test":"Request the exact formula for 'Accuracy' together with the code or data, or independently reimplement the stated LSTM configuration (units=40, input shape (55,1), dropout=0.2, batch size=35, epochs=50) on a chronological 80/20 split, and recompute every row of Table 3. The claim is settled only if each row satisfies MAE <= RMSE and RMSE = sqrt(MSE), 'Accuracy' is explicitly a return-direction or return-level skill metric, and the LSTM beats a persistence baseline that predicts the previous close on the test set; otherwise the reported accuracy cannot support the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that the number labelled 'Accuracy' in Tables 3 and 4 measures genuine out-of-sample skill at predicting stock returns. Section 3 ('Evaluations Metrics Utilised') lists Accuracy, MAE, MSE, and RMSE but never defines Accuracy, and all results are reported on predicted closing prices rather than returns. For a price series near a random walk, a persistence model that predicts the previous close can score very high on level-based fit metrics such as R-squared or normalized error while having zero skill for returns, so an undefined level-based 'accuracy' cannot support the abstract's claim of predicting returns with over 90% accuracy. The supporting error columns are internally inconsistent: WAPCO has MAE=4.474 and RMSE=0.7401, but MAE cannot exceed RMSE for the same error vector; AIICO has RMSE=0.544 with MSE=0.00285, although sqrt(0.00285) is about 0.053; and BERGER's LSTM RMSE is 0.2776 in Table 3 but 0.2710 in Table 4 for the same model. These contradictions imply that the metrics are not a coherent output of the claimed evaluation, so the 0.93-0.98 accuracy values are uninterpretable. In addition, the 80/20 split in Section 3 is never stated to be chronological; without that, training/test leakage cannot be excluded. The central claim therefore rests on an undefined quantity that is contradicted by the paper's own numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an LSTM model to forecast Nigerian equity stock returns, using daily price data from 2001 to 2022 for eight stocks listed on the Nigerian Stock Exchange. The model is trained on 80% of the data and tested on 20%, and performance is reported in terms of accuracy, MAE, MSE, and RMSE, with comparisons against ANN and CNN. The central claim, stated in the abstract and conclusion, is that the LSTM predicts future prices and returns with over 90% accuracy, averaging 97%. However, the paper never defines the 'accuracy' metric, and the reported error metrics in Tables 3 and 4 contain internal contradictions.","tokens_in":7714,"tokens_out":4470,"duration_ms":40584,"significance":"If the claimed results were valid, the paper would provide strong evidence that LSTM models can forecast equity returns in an African emerging market with 93-98% accuracy, which would be a notable empirical contribution. The paper deserves credit for using a real multi-decade dataset and for benchmarking three neural architectures. However, the central claim is not supported as stated: 'accuracy' is undefined, the metric values are internally inconsistent, and the evaluation protocol is not specified well enough to rule out leakage. Because the headline result rests on these uninterpretable numbers, the paper cannot be considered a reliable contribution in its current form.","major_comments":[{"comment":"The metric 'Accuracy' is listed among the evaluation metrics but no formula or definition is given anywhere in the paper. The abstract and conclusion use this quantity to assert that the model predicts returns with over 90% accuracy (97% average), yet for a regression task on price levels, 'accuracy' is not a standard metric. Without a definition, the reader cannot tell whether it measures return-direction correctness, R-squared, or some normalized error; under the most plausible level-based definitions, a persistence forecast would also score very high, so the headline claim is uninterpretable.","section":"Section 3, 'Evaluations Metrics Utilised'"},{"comment":"The WAPCO row reports MAE=4.474 and RMSE=0.7401. For any fixed error vector, MAE cannot exceed RMSE (by the RMS-AM inequality), so these two numbers cannot come from the same set of prediction errors. Similarly, the AIICO row gives MSE=0.00285 and RMSE=0.544, but sqrt(0.00285) is approximately 0.053, not 0.544. These internal contradictions indicate that the metric columns are not a coherent output of the claimed evaluation.","section":"Table 3"},{"comment":"The same model and stock appear with different metric values across the two tables: BERGER LSTM RMSE is 0.2776 in Table 3 but 0.2710 in Table 4, and WAPCO accuracy is 0.9739 in Table 3 but 0.97739 in Table 4. Since Table 4 is supposed to report the same LSTM results as Table 3, these discrepancies suggest that the numbers were produced by different runs or were transcribed inconsistently, further undermining the reliability of the reported 93-98% accuracy range.","section":"Tables 3 and 4"},{"comment":"The paper states that 80% of the dataset was used for training and 20% for testing, but it never specifies whether the split is chronological. For a time-series forecasting problem, a random split can allow future information to enter the training set, and the MinMaxScaler is described as being applied before the split, which risks using test-set statistics in scaling. Without a chronological split and leakage-free preprocessing, the out-of-sample nature of the reported errors cannot be verified.","section":"Section 3, dataset split"}],"minor_comments":[{"comment":"The phrase 'Mean Squared Error (MAE)' should read 'Mean Squared Error (MSE)'; the same heading also misspells 'Evaluations' as 'Evaluations'.","section":"Section 3, 'Evaluations Metrics Utilised'"},{"comment":"The sentence referring to 'the first row of Table 2' should instead refer to Table 3, since Table 2 contains a sample of raw stock data, not model predictions.","section":"Section 4, interpretation paragraph"},{"comment":"Stock names are misspelled in the table header and rows: 'GTBAN' should be 'GTBank', 'VAPC' should be 'WAPCO', and the CNN column header is missing a closing parenthesis.","section":"Table 4"},{"comment":"The phrase 'Recurrent Neutral Network' should be 'Recurrent Neural Network', and 'Hochreiter and Schmid Huber' should be 'Hochreiter and Schmidhuber'.","section":"Literature Review"},{"comment":"The text 'a unit value of 40, a total input shape of (55,1)' should use the plural 'units', and the relationship between the input shape, lookback window, and the number of units is not explained.","section":"Section 3, model architecture"},{"comment":"Reference [10] is a Towards Data Science blog post, not a peer-reviewed source; it should be clearly marked as a tutorial or replaced with a primary reference.","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper appears to have been published in the Journal of Advances in Mathematics and Computer Science, but as an arXiv submission it is assessed on its own merits. In my view, the undefined accuracy metric and the internally inconsistent tables are load-bearing flaws that cannot be fixed within the scope of a revision; a complete re-analysis with a defined metric and a leakage-free chronological split would be required. I do not see a path to acceptance for the current manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. This is a standard LSTM-on-prices paper applied to eight Nigerian equities, and the headline 93-98% accuracy claim does not survive contact with the paper's own tables. 'Accuracy' is never defined, and the error columns in Table 3 contradict basic arithmetic.\n\nWhat it does well: the dataset is real — eight NSE-listed stocks with over 20 years of daily data across banking, insurance, manufacturing, and telecom. The workflow is the standard LSTM pipeline (MinMaxScaler, 80/20 split, Keras/TensorFlow, unit=40, dropout=0.2, 50 epochs), and they benchmark LSTM against ANN and CNN, a reasonable structure. The literature review is honest and cites prior Nigerian-market ML studies, so the citation pattern is not a problem.\n\nThe soft spots are load-bearing, not cosmetic. Section 3 lists the metrics as 'Accuracy, Mean Absolute Error (MAE), Mean Squared Error (MAE), and Root Mean Squared Error.' The typo is minor; the missing formula for Accuracy is not. The only quantitative gloss is the conclusion's '97% predictive accuracy and 3% lost errors.' The plausible readings — R² or 1 minus a normalized price-level error — are exactly the numbers that land near 0.95-0.98 for an autocorrelated price series even when the model has zero skill on returns. All metrics are computed on closing prices, not on returns, which is what the abstract claims to predict. No persistence or random-walk baseline appears anywhere.\n\nThe numbers themselves are impossible. WAPCO shows MAE=4.474 and RMSE=0.7401; for a single error vector, MAE cannot exceed RMSE. AIICO shows RMSE=0.544 with MSE=0.00285, whose square root is about 0.053. BERGER's LSTM RMSE is 0.2776 in Table 3 and 0.2710 in Table 4. The 80/20 split is never documented as chronological, and there is no code or data to check against. These contradictions sit directly under the central claim, so the empirical result is uninterpretable as written.\n\nThe question is legitimate — whether deep learning adds genuine out-of-sample skill for an understudied emerging market like Nigeria is worth a careful study. This is not that study. The fair use of this paper is as a cautionary example for a reading group on how evaluation metrics go wrong in financial ML. Nobody should cite the 93-98% numbers as evidence of LSTM skill.\n\nRecommendation: desk-reject. If a venue wants to give the authors a path, the fix list is well-defined — define accuracy, add a persistence baseline, make the split explicitly chronological, correct the tables, and share code. But the submission in front of us has no coherent empirical basis.","headline":"A standard LSTM-application paper whose 93-98% accuracy claim is uninterpretable: the metric is undefined and the error tables are internally impossible.","tokens_in":8330,"tokens_out":8547,"would_cite":false,"duration_ms":82451,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims an LSTM trained on 20 years of cleaned Nigerian Stock Exchange data predicts the tested equities with accuracy between 93% and 99%.","keywords":["LSTM","stock return forecasting","Nigerian Stock Exchange","deep learning","recurrent neural networks","equity price prediction","performance metrics","financial time series"],"falsifier":"Take the LSTM's predicted closing prices for the held-out 20% of any tested stock, form one-day-ahead return predictions, and compare them with the naive forecast that tomorrow's close equals today's close. If the naive forecast has equal or lower RMSE on price levels, or if the LSTM's sign accuracy on returns is near 50%, the central 'over 90% accuracy' claim is refuted.","tokens_in":7169,"feed_emoji":"📈","tokens_out":6770,"duration_ms":66545,"temperature":0.7,"pith_summary":"The paper tries to establish that a long short-term memory (LSTM) network, trained on cleaned and normalized Nigerian Stock Exchange price data from 2001 to 2022, can forecast future equity prices and returns with accuracy above 90%, averaging about 97% across the eight stocks tested. If true, a single deep-learning architecture using only daily price and volume features would give investors a dependable signal in a market where conventional statistical forecasts have had limited success. The authors also claim the LSTM matches or beats an artificial neural network and a convolutional neural network on most of the same equities. The stakes are practical: a model like this could inform buy, sell, and hold decisions in an emerging market, and it would extend the case for LSTM-based financial time-series forecasting beyond developed markets.","feed_headline":"LSTM model hits 97% accuracy on Nigerian equities","feed_subtitle":"Two decades of cleaned Nigerian exchange data: the LSTM outperforms ANN and CNN on most of eight stocks.","key_machinery":"The load-bearing object is the LSTM recurrent network, whose cell state acts as an internal memory that the forget, input, and output gates update. Each gate is a sigmoid layer with a point-wise multiplication operator that decides what to remove, add, or emit from the memory. The paper's specific configuration — 40 units, an input window of 55 prior observations, dropout 0.2, batch size 35, and 50 epochs — maps normalized opening, high, low, close, and volume values to a predicted closing price. The gates matter because they allow the network to carry relevant price patterns across time steps instead of treating each day in isolation, which the authors argue is what lets it handle the non-linear nature of equity series. Supporting machinery includes MinMaxScaler normalization to a 0-1 range and reshaping the data into the 3D matrix format the LSTM consumes.","core_discovery":"The central discovery claimed is that the LSTM forecasts closing prices for eight Nigerian equities with accuracy scores between 0.9346 and 0.98533, with the accompanying MAE, MSE, and RMSE values interpreted as low loss errors. The specific network uses 40 units, an input shape of 55 time steps by 1 feature, a dropout of 0.2, a batch size of 35, and 50 epochs; it is trained on 80% of each stock's cleaned data and tested on the remaining 20%. From the eight-stock table the authors report an average predictive accuracy of 97% and 3% lost errors. They conclude that LSTM models are useful for financial time-series problems provided the training dataset is reliable, and they recommend hybrid LSTM-CNN models as future work to reduce the risk of relying on a single model.","pith_inferences":["The paper does not state whether the 80/20 split respects chronological order; if the split is random, the model could see future patterns during training, so the accuracy claim is only meaningful under a time-ordered split.","The reported accuracy is computed against predicted price levels, not against the sign or magnitude of one-day returns; a naive forecast that repeats yesterday's close can score nearly as well on price levels even with no predictive skill for returns.","A direct extension would measure the LSTM's return-direction accuracy against a 'repeat last close' benchmark on the same test window, which would settle whether the model forecasts movements or merely tracks the series.","If the accuracy holds on a time-ordered split with return-based metrics, the same pipeline could be tested across other African exchanges, where thin trading and data gaps would stress the cleaning and normalization steps."],"forward_implications":["Trained on cleaned NSE data covering 2001-2022, the LSTM predicts the tested equities' closing prices with reported accuracy between 93% and 99%, which the authors translate into a 97% average and 3% error.","Against an artificial neural network and a convolutional neural network on the same stocks, the LSTM has higher reported accuracy on most of the eight names, so the recurrent memory structure is claimed to fit these price series better.","Because the model needs only daily price and volume features plus standard hyperparameter values, the method is portable to other Nigerian equities with at least 20 years of consistent data.","Predicted closing prices can be converted into return signals, so an investor who trusts the model would have a concrete input to trading decisions rather than relying on technical or fundamental judgment alone.","The authors' recommended hybrid LSTM-CNN extension follows directly: combining the sequence memory of LSTM with the feature extraction of CNN would target the single-model risk the study acknowledges."],"supporting_citations":[{"why":"Defines the LSTM architecture the paper adopts and is the source for its gate and cell-state description.","marker":"[2]"},{"why":"Reported that an online RNN/LSTM model outperformed batch-processing models on Indian market stock data, a baseline the paper cites for LSTM accuracy.","marker":"[5]"},{"why":"Applied LSTM to Brazilian stock prices and found it predicted future movement accurately, the closest prior evidence the paper builds on.","marker":"[7]"},{"why":"Provided the LSTM architecture figure and reported successful hybrid technical-analysis and machine-learning stock prediction.","marker":"[8]"},{"why":"Applied a recurrent neural network to NSE stock prices and reported accurate prediction of future price returns on Nigerian equities.","marker":"[9]"}],"fun_headline_variants":["LSTM forecasts Nigerian stocks with 97% accuracy","Nigerian equity returns predicted by LSTM at 97%","LSTM beats ANN and CNN on Nigerian equity forecasts","Deep learning LSTM hits 97% on NSE stock predictions","LSTM model achieves 97% accuracy for Nigerian equities"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim collapses if the reported accuracy measures how closely the model tracks past closing price levels rather than how well it predicts future returns, because a model that simply repeats the last observed price can score very high on such a metric without forecasting skill.","fun_headline_variants_meta":{"raw":{"variants":["LSTM forecasts Nigerian stocks with 97% accuracy","Nigerian equity returns predicted by LSTM at 97%","LSTM beats ANN and CNN on Nigerian equity forecasts","Deep learning LSTM hits 97% on NSE stock predictions","LSTM model achieves 97% accuracy for Nigerian equities"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2499,"prompt_tokens":891,"completion_tokens":1608,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":1525}},"tokens_in":507,"tokens_out":1608,"duration_ms":11670,"temperature":1.0,"reasoning_tokens":1525,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:38:51.084221+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the LSTM's predicted closing prices for the held-out 20% of any tested stock, form one-day-ahead return predictions, and compare them with the naive forecast that tomorrow's close equals today's close. If the naive forecast has equal or lower RMSE on price levels, or if the LSTM's sign accuracy on returns is near 50%, the central 'over 90% accuracy' claim is refuted.","supporting_citations":[],"review_version":1}