{"id":"49ee4fa9-4f70-415c-86e5-7ce2b1e7c9d6","arxiv_id":"1908.03571","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A tuned LSTM with random forest feature selection and zero-crossing-based window sizes reports 54.05% lower RMSE than a standard LSTM on a single industrial boiler dataset, though the baseline is not fairly specified.","lead":"This paper presents a modified LSTM for predicting industrial flow, adding random forest feature selection and period-based windowing. The authors report a 54.05% RMSE improvement over a standard LSTM on one boiler dataset, but the comparison and evidence have serious gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 54.05% improvement over \"original LSTM\" is unsupported because Table VII's baseline hyperparameters are never specified while the proposed method's 9.13 RMSE was obtained after test-set selection over five variants.","rationale":"I read the paper in good faith: the proposed pipeline is a heuristically assembled LSTM variant with random-forest feature selection, period detection, data reshaping, and a five-run model selection loop. For its central empirical claim to hold, the comparison against the \"original LSTM\" baseline must be apples-to-apples. That condition is not met as reported: the baseline hyperparameters are absent, while the proposed method's hyperparameters were tuned on the same evaluation metric. This is precisely the reader's weakest assumption. My independent check — re-running the baseline with matched hyperparameters and evaluating the selected model on an untouched partition — would settle the concern. I do not see any reason to move the reader's verdict: the evidence as presented does not support the 54.05% relative improvement, so the REJECT verdict remains appropriate. I agree with the reader's identification of the baseline-comparison fairness as the load-bearing issue; the additional test-set-selection bias in Algorithm 3 reinforces the same conclusion rather than introducing a separate one.","tokens_in":9572,"tokens_out":4979,"duration_ms":49170,"concrete_test":"Rerun Experiment IV with the original LSTM using exactly the proposed method's tuned hyperparameters — sequence length 500, 100 hidden-layer nodes, 50 iterations, the same Adam settings, and the same 2/3-1/3 holdout split on the same Harbin boiler dataset — and record the test-set RMSE. If that RMSE is close to 9.13 rather than 19.87, the 54.05% improvement is a hyperparameter artifact. In the same experimental setup, hold out a third partition that is not used for the five-period selection in Algorithm 3, and evaluate the selected model on it; if its RMSE is substantially above 9.13, the reported number is optimistically biased.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that \"the LSTM algorithm with multivariate tuning is the best prediction algorithm whose RMSE is 9.13. However, that of the original LSTM is 19.87, and thereby the LSTM based on multivariate tuning is optimized by 54.05%\" (Section VI.D). This comparison is load-bearing, and the paper does not establish its fairness. The hyperparameters of the \"original LSTM\" baseline in Experiment IV / Table VII are never given, whereas the proposed method's settings — sequence length 500, 100 hidden-layer nodes, 50 iterations — were explicitly tuned in Section VI.C by minimizing RMSE across Tables III, IV, and V. An untuned or default LSTM could plausibly yield 19.87, while the same LSTM architecture with the tuned hyperparameters might produce an RMSE much closer to 9.13. If so, the claimed 54.05% improvement is attributable to hyperparameter optimization, not to the multivariate tuning modules.\n\nA second, compounding issue is that Algorithm 3 fits five LSTM models for the five smallest detected periods and returns the one with minimal RMSE (\"if m > M.RSME then m <- M.RSME\"). If this RMSE is evaluated on the test partition, the reported 9.13 is the minimum of five test-set estimates, i.e., selection on the test set. The paper's claim in Section VI.C that convergence of training and test loss curves \"proves that the model is not overfitting\" does not address this selection bias and is not a valid statistical argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an LSTM-based prediction method for industrial flow data, termed 'LSTM based on multivariate tuning.' The method consists of three modules: a data conversion module that uses random forest importance to reduce dimensionality and converts time series into supervised sequences with a window size n; an LSTM modeling module; and a tuning module that estimates candidate periods from zero-crossings of a normalized target and iterates over the five smallest detected periods, selecting the model with the smallest RMSE. The central claim is that on a 70-dimensional industrial boiler dataset with over 400,000 records, the proposed method achieves an RMSE of 9.13 versus 19.87 for a 'traditional LSTM,' a claimed 54.05% improvement. The paper also provides time and space complexity analyses and describes the experimental tuning of sequence length, hidden-layer nodes, and number of iterations.","tokens_in":9872,"tokens_out":10393,"duration_ms":95579,"significance":"Should the empirical claim hold, the proposed method would be a practically relevant contribution to industrial time-series prediction, as it addresses periodicity and multivariate dependency, and the complexity analysis is a useful addition. The paper also gives concrete algorithms and a real-world evaluation. However, the evaluation as presented does not support the central claim: the baseline comparison is not matched, the reported 'optimal' parameters are contradicted by the paper's own tuning table, the model-selection procedure is biased, and no error bars or repeated runs are provided. The internal inconsistencies in the period-detection and pseudocode would need to be resolved before the method is reproducible. The contribution is therefore not yet established at the level expected for a journal publication.","major_comments":[{"comment":"The comparison between the proposed method and 'original LSTM' in Table VII is not a controlled experiment. The hyperparameters of the LSTM baseline are never specified, whereas the proposed method's sequence length, hidden-layer size, and iteration count were explicitly tuned in Section VI.C. Table III shows that setting the sequence length to 500 with hidden size 50 already yields an RMSE of 11.49, nearly halving the reported baseline of 19.87, so the claimed 54.05% improvement could be an artifact of hyperparameter selection rather than of the multivariate-tuning modules. The authors should compare against an LSTM with the same tuned hyperparameters (e.g., 500/100/50) and report error bars over multiple random seeds.","section":"VI.D / Table VII"},{"comment":"Table IV reports that Experiment 10 (sequence length 500, 200 hidden-layer nodes, 50 iterations) achieves RMSE 8.69, which is lower than the RMSE of 9.13 for Experiment 9 with 100 hidden-layer nodes. Nevertheless, the text states that 100 hidden nodes 'can achieve a better experimental result' and later designates 500/100/50 as the optimal parameters. This contradicts the paper's stated objective of minimizing RMSE and renders the tuning process and the headline 9.13 result internally inconsistent; the authors need to explain why the lower-RMSE configuration was not used.","section":"VI.C / Table IV"},{"comment":"Algorithm 3 selects the model with minimal RMSE among the five smallest detected periods. If the RMSE used in this comparison is computed on the test partition, then the reported 9.13 is the result of fitting to the test set via model selection, and the claim in Section IV.D that 'there is no overfitting' from considering only five periods does not address this selection bias. The loss-curve argument in Section VI.C is also not a statistical test of overfitting and does not justify using the selected minimum without a correction (e.g., nested validation or a model-selection test).","section":"IV.D / Algorithm 3"},{"comment":"The period-detection procedure is internally inconsistent: the text says the predictive values are normalized into the (0,1) interval, but Algorithm 2 then detects periods by counting steps until the sign of y changes (y > 0 versus else). If all values lie in (0,1), every y is positive, count never resets, and stepSet remains empty, so Algorithm 3 would fail at line 3. The normalization must be redefined (e.g., zero-mean standardization) or the detection logic must be changed; as written, the proposed algorithm is not implementable.","section":"IV.D / Algorithm 2"},{"comment":"In Algorithm 3, the variable dataT is computed once before the loop (line 3) using stepSet[count] with count=0, and is never recomputed inside the while loop. Consequently, the loop trains an LSTM on the same transformed data for all five iterations instead of using each of the five period candidates, contradicting the textual description in Section IV.D and Section VI.C. This is a load-bearing discrepancy in the algorithm's presentation and must be corrected.","section":"V.A / Algorithm 3"},{"comment":"All reported RMSE values in Tables III–VII are single numbers with no standard deviations, confidence intervals, or number of repeated runs, despite the statement in Section VI.B that 'the experiment reduces the experimental error by multiple measurements and averaging.' Because LSTM training is stochastic, the claimed 54.05% improvement cannot be assessed without such information. At minimum, the authors should provide the mean and standard deviation over several independent runs for each configuration.","section":"VI.B-C"}],"minor_comments":[{"comment":"The acronym RMSE is consistently misspelled 'RSME' throughout the manuscript, including equations and tables.","section":"Throughout"},{"comment":"Algorithm 1 line 15 contains the typo 'new row siez' and the reshape operation is not specified precisely.","section":"IV.B / Algorithm 1"},{"comment":"Section IV.D says the first five periods are 'taken as the number of iterations'; they are actually candidate window sizes n, not iteration counts.","section":"IV.D"},{"comment":"Several sentences are ungrammatical, e.g., 'the number of iterations will be It is tentatively set to 50 times' in Section VI.C; a thorough language edit is needed.","section":"VI.C"},{"comment":"Figure and table numbering is inconsistent; for example, Section VI.C refers to 'Figure 6' while the optimal-prediction plot is labeled 'Figure 8'.","section":"VI.C"}],"recommendation":"reject","confidential_remarks":"This manuscript appears to be an early-stage preprint. The central empirical claim is not supported by the current experimental design, and the paper contains multiple internal inconsistencies (tuning, algorithm pseudocode, normalization) that would require substantial reworking. I would not encourage an invited revision; a fresh, properly controlled evaluation with matched baselines and error bars would be needed before the contribution can be assessed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a reject for me, but not because the idea is crazy. The proposed LSTM pipeline — random-forest feature selection, zero-crossing period detection, and candidate window sizes — is a sensible engineering approach for multivariate industrial time series. The trouble is the evaluation: the headline 54.05% improvement is built on an unspecified baseline, test-set selection, and no error bars.\n\nWhat's actually new: the specific combination is not in the cited literature, and it's a reasonable integration of standard pieces. The complexity analysis is explicit, and the authors identify real challenges — opaque periodicity and multivariate dependence. If I were an engineer with this boiler dataset, I might try their pipeline. There is value in that.\n\nBut the soft spots are load-bearing. The \"original LSTM\" baseline in Table VII is never given hyperparameters, while the proposed method's settings were tuned on test RMSE in Tables III–V. An untuned LSTM could easily give 19.87; a tuned LSTM might come close to 9.13. So the claimed 54.05% improvement is unsupported. Algorithm 3 also fits five LSTM models for the five smallest detected periods and returns the one with lowest RMSE; if that RMSE is computed on the test partition, it is selection on the test set. The paper's statement that training/test loss curves \"prove\" no overfitting does not address this. There are internal contradictions too: Table IV shows 200 hidden nodes gives RMSE 8.69, yet the text says 100 is better; Table III shows sequence length changing RMSE from 22.83 to 11.49, yet the text says it has small influence. That suggests the narrative was written after the results.\n\nWho this is for: practitioners wanting a quick, interpretable way to add periodicity and feature selection to LSTM. As a paper, it needs a matched baseline, repeated runs with error bars, and evaluation on more than one dataset. I wouldn't desk-reject the topic, but I also wouldn't accept this version.\n\nRecommendation: send it to peer review only if you expect a referee to demand a redone experiment section. The underlying idea is fine, but the central claim is currently unsubstantiated.","headline":"The method is a reasonable engineering combination, but the 54.05% improvement claim is unsupported by an unfair baseline and test-set selection.","tokens_in":10386,"tokens_out":4669,"would_cite":false,"duration_ms":49661,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that adding multivariate tuning to LSTM cuts flow-prediction RMSE by 54.05% on an industrial boiler dataset.","keywords":["LSTM","time series prediction","flow prediction","industrial data","multivariate tuning","periodic analysis","root mean square error","random forest feature selection"],"falsifier":"Retrain the original LSTM using the same hyperparameters the proposed method was tuned to—sequence length 500, 100 hidden nodes, and 50 iterations—on the same train/test split; if its RMSE falls to roughly 9-10, the 54.05% improvement is primarily hyperparameter tuning and not the multivariate or periodic modules.","tokens_in":9339,"feed_emoji":"🏭","tokens_out":4154,"duration_ms":39547,"temperature":0.7,"pith_summary":"This paper tries to establish that flow prediction on industrial time-series data improves substantially when a long short-term memory network is combined with a multivariate tuning pipeline. On a real boiler dataset with 70 sensor dimensions, the proposed method reports a root-mean-square error of 9.13, compared with 19.87 for a plain LSTM, a 54.05% reduction, and it also beats random forest, backpropagation, and CNN baselines. The authors argue that industrial data's hidden periodicity and cross-variable dependencies are the reasons LSTM alone underperforms, and that their tuning modules address exactly those two features. If the claim holds, factories could forecast output or flow more reliably and detect equipment problems earlier with a modest algorithmic change.","feed_headline":"Multivariate tuning cuts LSTM flow-prediction error by 54%","feed_subtitle":"On a real industrial boiler dataset, the tuned network's RMSE drops to 9.13 from plain LSTM's 19.87.","key_machinery":"The load-bearing object is the tuning module's period detection: the target series is normalized to the (0,1) interval, the positions where the sign of the value flips are recorded, the distances between flips define candidate period lengths, and the five smallest such lengths are used as window sizes n in the data reshaping step. This turns an opaque, hard-to-guess hyperparameter—how many past rows to feed the LSTM—into a quantity read off the data itself. The same module then trains an LSTM for each candidate period and selects the model with the minimum RMSE, which the paper treats as an approximate optimum with reduced overfitting risk.","core_discovery":"The central discovery is an algorithmic recipe: convert multivariate time series into supervised sequences by selecting the top variables via random-forest importance (covering 95% of cumulative importance) and reshaping n consecutive rows into one training row; train a single-layer LSTM on those sequences; then, instead of guessing a window size, detect the signal's period by recording zero-crossings of the normalized target variable, take the five shortest period values, and iterate training over those window sizes, keeping the model with the lowest RMSE. On the industrial boiler dataset this pipeline reaches RMSE 9.13, and the paper attributes the gain to the periodic window selection plus the multivariate transformation. The authors also argue the chosen model is not overfitting because the training and test loss curves converge.","pith_inferences":["A matched-hyperparameter comparison would likely shrink the gap: plain LSTM is known to be sensitive to look-back length and hidden size, so the head-to-head claim needs the baseline tuned to the same budget to be credible.","The period-detection rule assumes the normalized target alternates sign; for strictly positive flows that never cross zero, the algorithm would record no flips, so a practical extension would need a detrending or baseline-subtraction step.","The same modular recipe—feature selection, window-from-periodicity, iterative RMSE selection—could be lifted onto other sequence models such as GRU or temporal convolutional networks; the paper does not test this, but nothing in the modules is LSTM-specific beyond the network itself.","The paper's 'no overfitting' argument relies on loss-curve convergence and on averaging over five periods; a stronger test would be a true held-out time block from a later date, since the random split leaks neighboring timestamps across train and test."],"forward_implications":["If the reported RMSE reduction holds under matched baselines, adopting the tuning pipeline should improve flow forecasting in settings where LSTM is already deployed, without changing the core network architecture.","The zero-crossing period detector gives a data-driven rule for choosing the look-back window, replacing manual selection and plausibly transferring to other periodic industrial signals such as pressure or temperature.","The random-forest feature selection step shrinks 70 sensor dimensions to a 95%-importance subset, so the method should also reduce training cost and model complexity relative to training on all dimensions.","Selecting the best of five candidate window sizes by validation RMSE provides a built-in guard against picking a poor window, though the 'first five smallest periods' rule is an arbitrary cap."],"supporting_citations":[{"why":"Supplies the LSTM architecture that the proposed method builds on and the original-LSTM baseline it is compared against.","marker":"[8]"},{"why":"The Adam optimizer used in training and in the iterative tuning module; the paper's complexity analysis assumes it.","marker":"[13]"},{"why":"Establishes the recurrent-neural-network background and the long-term dependency problem the paper says LSTM addresses.","marker":"[7]"}],"fun_headline_variants":["Periodic windows boost LSTM flow prediction by 54%","LSTM flow predictor improved 54% via periodic window tuning","Tuned LSTM slashes flow prediction error by 54%","Multivariate LSTM with periodic windows cuts error 54%","Zero-crossing windows lift LSTM flow accuracy 54%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison rests on the assumption that the 'original LSTM' baseline was run with a fair, comparable hyperparameter configuration, but the paper never reports that baseline's window size, hidden nodes, or iteration count.","fun_headline_variants_meta":{"raw":{"variants":["Periodic windows boost LSTM flow prediction by 54%","LSTM flow predictor improved 54% via periodic window tuning","Tuned LSTM slashes flow prediction error by 54%","Multivariate LSTM with periodic windows cuts error 54%","Zero-crossing windows lift LSTM flow accuracy 54%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000271,"raw_usage":{"total_tokens":1545,"prompt_tokens":780,"completion_tokens":765,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":396,"completion_tokens_details":{"reasoning_tokens":678}},"tokens_in":396,"tokens_out":765,"duration_ms":7425,"temperature":1.0,"reasoning_tokens":678,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:12:05.836725+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the original LSTM using the same hyperparameters the proposed method was tuned to—sequence length 500, 100 hidden nodes, and 50 iterations—on the same train/test split; if its RMSE falls to roughly 9-10, the 54.05% improvement is primarily hyperparameter tuning and not the multivariate or periodic modules.","supporting_citations":[{"cited_title":"Long short-term memory neural network for traffic speed prediction using remote microwave sensor data,","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM architecture that the proposed method builds on and the original-LSTM baseline it is compared against."},{"cited_title":"Time series prediction with recurrent neural networks trained by a hybrid pso–ea algorithm,","cited_arxiv_id":null,"evidence_quote":"Establishes the recurrent-neural-network background and the long-term dependency problem the paper says LSTM addresses."}],"review_version":1}