{"id":"34e95a88-4bfc-4000-841d-085a49f6844e","arxiv_id":"2412.15998","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A hybrid CNN-LSTM model fitted to CMAPSS data reports best R-squared but with contradictory RMSE/R2 values and no evidence of novelty.","lead":"A hybrid CNN-LSTM model is applied to predict aircraft engine remaining useful life on the NASA CMAPSS data, with a claimed R-squared of 0.86. The claim is undermined by internally inconsistent metrics and a false 'first attempt' novelty statement.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 4's RMSE/R^2 pairs are mathematically impossible on a single test set; the claim that CNN-LSTM achieves the best R^2 is therefore unsupported.","rationale":"The reader's weakest assumption is also the most load-bearing one: all compared models are evaluated on the same CMAPSS test set with the same target definition. The paper's own equations (1) and (2) imply R^2 = 1 - RMSE^2 / Var(y) on a fixed test set, so RMSE and R^2 are not free parameters. Table 4's entries are mutually incompatible with this identity by up to two orders of magnitude, and the rankings by RMSE and R^2 conflict (MLP has the lowest RMSE but near-lowest R^2; Random Forest has the second-lowest RMSE but the lowest R^2). Because the paper does not specify the CMAPSS subset, test split, or target variance, the discrepancy cannot be dismissed as a typo in a single row; it invalidates the entire comparison. The central claim of the paper is exactly that CNN-LSTM achieves the highest accuracy and superior R^2, so this mathematical inconsistency is sufficient to reject the paper. The reader's additional concerns about novelty and data leakage through test-based preprocessing choices are also plausible, but they are secondary. No new concern is needed; the rejection stands, and the proposed concrete test would settle the matter definitively in any revision.","tokens_in":11433,"tokens_out":4069,"duration_ms":32045,"concrete_test":"Recompute from Table 4 the implied target variance V_implied = RMSE^2 / (1 - R^2) for every row; if all models were scored on the same fixed test set, V_implied must be a single shared constant. Then run a single script on the actual CMAPSS test set (stating the subset and the piecewise-RUL cap, e.g., 130) to compute the true target variance, and check whether any reported (RMSE, R^2) pair is consistent with it. The discrepancy of two to three orders of magnitude among rows (e.g., Random Forest V ≈ 77 vs. Linear Regression V ≈ 3,453) settles the issue. Also report per-model test indices; if they differ, the comparison is not apples-to-apples.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim that the proposed CNN-LSTM hybrid achieves the best R^2 on CMAPSS rests on the assumption that all models are evaluated on the same test set with the same target values. That assumption is falsified by the paper's own Table 4. Section 3.4 defines RMSE (Eq. 1) and R^2 (Eq. 2); for a fixed test set with target variance V, the identity R^2 = 1 - RMSE^2 / V must hold for every model. Table 4 violates this massively: the implied V from each row is not a constant, e.g. Linear Regression implies V ≈ 3,453, Random Forest implies V ≈ 77, MLP implies V ≈ 42, and CNN-LSTM implies V ≈ 1,271 (computed as RMSE^2 / (1 - R^2)). The contradictions also appear as rank reversals: MLP has the lowest RMSE (4.51) but an R^2 of only 0.52, while Random Forest has the second-lowest RMSE (6.68) but the worst R^2 (0.42). Since R^2 is a decreasing function of RMSE for fixed V, such ordering is impossible unless models were scored on different test splits or different target definitions. The paper never states the CMAPSS subset (FD001-FD004), the number of test samples, or the actual target variance, so the discrepancy cannot be resolved. Therefore the superiority claim for CNN-LSTM, based on R^2, is not supported by the reported evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a hybrid CNN-LSTM architecture for Remaining Useful Life (RUL) estimation on the NASA CMAPSS turbofan engine dataset. The pipeline includes EMA smoothing, normalization, PCA-based feature reduction, and a piecewise-linear RUL target capped at 130 cycles. The proposed model is compared against Linear Regression, Random Forest, XGBoost, MLP, and LSTM using RMSE and R². The central claim is that the CNN-LSTM hybrid achieves the best R² score among the compared methods. The paper reports a table of results (Table 4) and concludes that the proposed model outperforms all baselines.","tokens_in":11912,"tokens_out":4047,"duration_ms":34831,"significance":"If the empirical claim were sound, the manuscript would provide a useful, if incremental, demonstration of a known hybrid architecture applied to a standard prognostics benchmark. The topic is relevant to predictive maintenance and the CMAPSS dataset is a community standard. However, the core evidence is invalid: the reported RMSE and R² values in Table 4 are mathematically inconsistent for a single test set, and the preprocessing choices appear to have been selected using test-set information. The paper also omits reproducibility-critical details for the proposed model. The significance of the contribution, as presented, is therefore not established.","major_comments":[{"comment":"The RMSE and R² pairs in Table 4 are mutually inconsistent for a fixed test set. As defined in Section 3.4, Eqs. (1)-(2), R² = 1 - RMSE²/V, where V is the variance of the true test targets; V must be the same for every model evaluated on the same test set. The table reports, for example, MLP RMSE 4.51 with R² 0.52, Random Forest RMSE 6.68 with R² 0.42, and CNN-LSTM RMSE 13.34 with R² 0.86. These imply V ≈ 42, ≈ 77, and ≈ 1,271, respectively, which is impossible. The rank reversals are also impossible: for a fixed V, a lower RMSE must correspond to a higher R², yet MLP has the lowest RMSE but a lower R² than CNN-LSTM. Because the paper never states which CMAPSS subset (FD001-FD004) was used, the number of test samples, or the target variance, the discrepancy cannot be resolved. The central claim that CNN-LSTM achieves the best R² is therefore unsupported by the reported evidence.","section":"Section 5, Table 4"},{"comment":"The selection of the EMA smoothing parameter and the normalization method is described as choosing the configuration with the 'best evaluation result' (Section 4.3.2) or one that 'visually outperformed other configurations' (Section 4.3.1). The manuscript does not explicitly confine this selection to training or validation data. If the test set was used to choose these preprocessing choices, then the reported test metrics are optimistically biased and the comparison is not an honest evaluation of generalization. The paper must state that all preprocessing hyperparameters were selected via cross-validation on the training folds only, and report the candidate values considered and the selection criterion.","section":"Sections 4.3.1 and 4.3.2"},{"comment":"The proposed CNN-LSTM model is not specified at the level needed to reproduce the experiments or to interpret the comparison. Although hyperparameter tuning tables are provided for Random Forest, XGBoost, and MLP (Tables 1-3), the CNN-LSTM description gives only the number of convolution filters (64), sequence length (30), and the piecewise RUL cap (130). The number of LSTM layers and units, dropout, batch size, epochs, optimizer, learning rate, and the number of independent runs are omitted. Since the paper's central claim is an empirical superiority claim, these details are load-bearing for the validity of the comparison.","section":"Section 5"}],"minor_comments":[{"comment":"The abstract states that the comparison includes CNN, but Table 4 has no standalone CNN row; the list of compared methods in Section 5 also omits CNN. Please align the abstract with the actual experiments.","section":"Abstract and Section 5"},{"comment":"The paper says the dataset contains '26 numerical features' but later states that PCA reduces the dimension 'from 24 to 12.' Clarify the number of input features and how the 24-dimensional representation is obtained from the raw CMAPSS columns.","section":"Section 4.1 and Section 4.4.1"},{"comment":"The 'layer_sizes' entries in the MLP hyperparameter table, such as '8.16.64.32.8', appear to be strings rather than tuples of integers; formatting them as, e.g., (8, 16, 64, 32, 8) would avoid ambiguity.","section":"Table 3"},{"comment":"The claim that this is 'the first attempt to adopt this approach for RUL estimation in prognostics' is not substantiated by the related-work section, which reviews only pure CNN and pure LSTM approaches and does not survey existing CNN-LSTM hybrids for RUL. Either provide a systematic literature search or soften the novelty claim.","section":"Section 2"},{"comment":"The reference list has formatting inconsistencies, such as inconsistent use of journal names, page ranges, and author name formats (compare entries [4], [5], and [6]). Please unify the style and check for missing page numbers or DOIs.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript is a short conference-style paper with an internally inconsistent central evaluation. The Table 4 metrics cannot all be correct for a single test set, which suggests that the experiments were either run on different splits or the metrics were miscalculated. This is not a presentation issue but a fundamental invalidation of the reported comparison. The paper also lacks reproducibility details for the proposed model and does not state the CMAPSS subset used. Even though the topic is relevant, the manuscript would need a complete re-run of the experiments with transparent evaluation protocols before it could be considered further."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe reader's take is right, and the stress-test note holds up. Table 4 is internally impossible. For a fixed test set, R² = 1 − RMSE²/Var(y), so the implied variance changes wildly across rows: Linear Regression implies V≈3454, Random Forest ≈77, MLP ≈42, CNN-LSTM ≈1271. This isn't rounding error; the models were evaluated on different targets or different test splits. The paper never states which CMAPSS subset (FD001-FD004) was used, so the discrepancy can't be resolved. The central claim that CNN-LSTM achieves the best R² is unsupported.\n\nWhat the paper does well: it is clearly organized, uses a public benchmark, defines the metrics, and documents a reasonable preprocessing pipeline (EMA smoothing, piecewise RUL cap, PCA, SelectKBest). The visualizations are fine. But none of this is new, and the novelty claim is false. CNN-LSTM for RUL estimation predates this work (e.g., Hinchi and Tkiouat 2018); the authors cite related LSTM and CNN papers but missed the hybrid prior work.\n\nOther soft spots, in proportion: preprocessing choices (EMA alpha, normalization method, PCA dimension, RUL cap) were selected based on test-set evaluation, which leaks test information into model selection. The paper also omits code, architecture hyperparameters (LSTM layers, units, batch size, epochs), and the exact train/test split. Those omissions alone would make replication hard.\n\nThis paper is not worth refereeing. The main quantitative result is self-contradictory, and the contribution is not original. I would not bring it to a reading group and would not cite it. If the authors fixed Table 4 and the evaluation protocol, they might have a modest empirical data point, but as written the conclusions do not follow.\n\nRecommendation: reject without peer review.","headline":"Table 4's RMSE/R² pairs are mutually inconsistent, and the claimed novelty doesn't hold up; the paper's central comparison is invalid.","tokens_in":12323,"tokens_out":2319,"would_cite":false,"duration_ms":19004,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a hybrid CNN-LSTM model—one-dimensional convolutions feeding LSTM layers—estimates remaining useful life on the CMAPSS turbo-fan dataset with the best R² among the compared methods.","keywords":["Remaining Useful Life","CNN-LSTM hybrid","predictive maintenance","CMAPSS dataset","deep learning","time series forecasting","RUL estimation","multivariate sensor data"],"falsifier":"Compute the variance $\\operatorname{Var}(y)$ of the true RUL labels on the CMAPSS test set and check each reported model against $R^2 = 1 - \\frac{\\mathrm{RMSE}^2}{\\operatorname{Var}(y)}$; any pair that violates this identity shows the scores were not produced on the same test set and target, so the ranking would not follow.","tokens_in":11215,"feed_emoji":"🔧","tokens_out":10832,"duration_ms":88987,"temperature":0.7,"pith_summary":"The paper sets out to establish that remaining useful life—how many operating cycles a machine part still has before failure—is predicted more accurately by a hybrid CNN-LSTM model than by conventional regression, standalone CNN, or standalone LSTM models. The hybrid applies convolutional filters to each time step's sensor readings to extract local features, then lets LSTM layers track how those features evolve over the sequence. On the CMAPSS turbo-fan run-to-failure dataset, the authors report that their CNN-LSTM model achieves an $R^2$ of 0.86, the highest among the methods they compare, with an RMSE of 13.34. If that result holds, predictive-maintenance systems could schedule interventions from noisy multivariate sensor histories with more confidence, since the model is designed to exploit both spatial patterns and long-term temporal dependencies.","feed_headline":"Hybrid CNN-LSTM tops the field at predicting engine failure","feed_subtitle":"On the CMAPSS jet-engine benchmark, the combined architecture reports the best R² among compared regression and deep-learning models.","key_machinery":"The load-bearing mechanism is the CNN-LSTM stack: a 1D convolutional layer with 64 filters and ReLU activation applies kernels across the sensor channels at each time step, a non-overlapping max-pooling layer compresses those feature maps, and the pooled sequence is passed to LSTM layers that maintain cell state across time steps, followed by dense layers that regress to RUL. The argument also depends on the piecewise-linear target construction with a 130-cycle cap and on preprocessing that includes exponential moving average smoothing, standardization, PCA to 12 components, and Select K Best feature ranking. Together these pieces let the model learn local multivariate patterns and long-range temporal trends in one pass instead of relying on fixed sliding-window features.","core_discovery":"On its own terms, the paper's discovery is that a deliberately simple combination—one 1D convolutional layer with 64 filters and ReLU activation, a max-pooling layer, LSTM layers over time, and a dense regression head trained with squared error—captures the degradation signal in multivariate sensor data better than any of the alternatives tested. The authors construct a piecewise-linear RUL target capped at 130 cycles, smooth the raw signals with exponential moving averages, standardize them, reduce the retained sensors with PCA, and add the first principal component as a feature. Against this preprocessed CMAPSS data they compare Linear Regression, Random Forest, XGBoost, MLP, CNN, and LSTM, and report that CNN-LSTM attains $R^2=0.86$, ahead of LSTM's $0.75$ and the standalone CNN's $0.79$. The paper frames this as the first use of the CNN-LSTM hybrid for RUL estimation in prognostics, arguing that the CNN stage supplies non-handcrafted local features and the LSTM stage supplies the sequence memory that pure CNN sliding-window features miss.","pith_inferences":["An ablation that removes the CNN stage or the LSTM stage on the same preprocessed inputs would show whether the reported gain comes from the combination itself or from the shared preprocessing and feature engineering.","The same stack could transfer to other degradation signals such as bearing vibration, battery capacity fade, or structural health monitoring, where local impulse features and long-term drift both matter.","Testing on the multi-operating-condition subsets of CMAPSS would tell whether the hybrid advantage persists when the data contain several regimes, since the paper reports one aggregate result."],"forward_implications":["If the reported ranking is correct, hybrid CNN-LSTM is a stronger baseline than CNN-only or LSTM-only for RUL prediction on run-to-failure fleets.","The preprocessing recipe—EMA smoothing, standardization, PCA, and Select K Best ranking—can be reused on other multivariate degradation datasets.","Sequence-aware RUL forecasts would let maintenance planners act on predicted remaining cycles rather than fixed threshold alarms, reducing unplanned downtime.","Because the paper reports a single aggregate result on one benchmark, the architecture's advantage still needs confirmation on other run-to-failure datasets before it generalizes.","The authors' own conclusion that the model is computationally heavy implies practical embedded deployment will require a lighter variant."],"supporting_citations":[{"why":"Supplies the piecewise-linear RUL target construction with the 130-cycle cap used in the experiments.","marker":"[3]"},{"why":"Provides the deep CNN regression baseline whose architecture is extended with the LSTM stage.","marker":"[4]"},{"why":"Introduces the LSTM network for RUL estimation that the hybrid builds on.","marker":"[5]"},{"why":"Defines the LSTM cell with input, forget, and output gates used as the sequence-learning module.","marker":"[23]"},{"why":"Supplies the CMAPSS run-to-failure simulation dataset that all experiments use.","marker":"[26]"},{"why":"Provides the turbofan degradation benchmark dataset for the evaluation.","marker":"[31]"},{"why":"A multi-scale deep CNN RUL model on the same dataset, representing the state of the art to be beaten.","marker":"[32]"},{"why":"Describes MLP with PCA and feature selection for RUL estimation, the preprocessing lineage the paper follows.","marker":"[33]"}],"fun_headline_variants":["CNN-LSTM hybrid beats rivals for machine-life prediction","First CNN-LSTM for RUL estimation tops CMAPSS benchmark","CNN-LSTM hybrid nets best R² in engine-life prediction","Hybrid CNN-LSTM sets accuracy record on CMAPSS RUL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes all models were evaluated on the same CMAPSS test set with the same remaining-useful-life target, so the RMSE and R² values can be ranked directly.","fun_headline_variants_meta":{"raw":{"variants":["CNN-LSTM hybrid beats rivals for machine-life prediction","First CNN-LSTM for RUL estimation tops CMAPSS benchmark","CNN-LSTM hybrid nets best R² in engine-life prediction","Hybrid CNN-LSTM sets accuracy record on CMAPSS RUL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000472,"raw_usage":{"total_tokens":2379,"prompt_tokens":1009,"completion_tokens":1370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1298}},"tokens_in":625,"tokens_out":1370,"duration_ms":9374,"temperature":1.0,"reasoning_tokens":1298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:53:10.196645+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the variance $\\operatorname{Var}(y)$ of the true RUL labels on the CMAPSS test set and check each reported model against $R^2 = 1 - \\frac{\\mathrm{RMSE}^2}{\\operatorname{Var}(y)}$; any pair that violates this identity shows the scores were not produced on the same test set and target, so the ranking would not follow.","supporting_citations":[{"cited_title":"An artificial neural network method for remaining useful life prediction of equipment subject to condition monitoring,","cited_arxiv_id":null,"evidence_quote":"Provides the turbofan degradation benchmark dataset for the evaluation."},{"cited_title":"CNN have great potential to identify the various salient patterns of sensor signals","cited_arxiv_id":null,"evidence_quote":"Supplies the piecewise-linear RUL target construction with the 130-cycle cap used in the experiments."},{"cited_title":"Remaining Cycles,","cited_arxiv_id":null,"evidence_quote":"Provides the deep CNN regression baseline whose architecture is extended with the LSTM stage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the LSTM network for RUL estimation that the hybrid builds on."},{"cited_title":"Predicting Bearings Degradation Stages for Predictive Maintenance in the Pharmaceutical Industry","cited_arxiv_id":null,"evidence_quote":"Defines the LSTM cell with input, forget, and output gates used as the sequence-learning module."},{"cited_title":"A similarity-based prognostics approach for remaining useful life estimation of engineered systems","cited_arxiv_id":null,"evidence_quote":"Supplies the CMAPSS run-to-failure simulation dataset that all experiments use."},{"cited_title":"Long short-term memory,","cited_arxiv_id":null,"evidence_quote":"A multi-scale deep CNN RUL model on the same dataset, representing the state of the art to be beaten."},{"cited_title":"Recurrent neural networks and robust time series prediction","cited_arxiv_id":null,"evidence_quote":"Describes MLP with PCA and feature selection for RUL estimation, the preprocessing lineage the paper follows."}],"review_version":1}