{"id":"e6edd704-a451-4ea8-8be2-89d26cd0ed33","arxiv_id":"2501.17865","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A benchmark of eight ML models for gas turbine CO and NOx prediction finds no consistent winner, despite a conclusion favoring KNN.","lead":"This paper compares eight machine learning models for predicting CO and NOx emissions from gas turbine data. The claimed winner, KNN, is not consistently best according to the paper's own tables, and no data or code is provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 contradicts the conclusion: for CO, GRU has lower MSE/RMSE (0.00036/0.01896) than KNN (0.00041/0.02033), so the claim that KNN 'consistently outperformed other models' is not supported by the paper's own results.","rationale":"The reader's weakest assumption was that the unreported train/test split makes the KNN ranking unstable. That is a real reproducibility concern, and the paper provides no code, data, or split details. However, I found a more direct and load-bearing problem: the paper's own Table 1 and Section 4.3 contradict the headline conclusion. For CO, GRU has lower MSE and RMSE than KNN, so KNN is not 'consistently outperforming' across the reported metrics. The conclusion only holds if one arbitrarily prioritizes MAE and MAPE over MSE and RMSE for the CO target, and that choice is not argued. This is an internal inconsistency, not a disagreement with external consensus, so it does not require any outside benchmark to land. The missing split is secondary: even with a perfect split description, the reported numbers already fail to support the central claim. Given no independent reproducibility artifacts and a routine application of standard models to a private dataset, the README's REJECT verdict remains appropriate. I set verdict_should_be to UNCHANGED because my concern reinforces rather than changes the reader's rejection.","tokens_in":7181,"tokens_out":3872,"duration_ms":39226,"concrete_test":"Recompute the CO test-set MSE and RMSE for GRU and KNN from the original test predictions, which requires the authors to release the code, data, or exact split. If the values match Table 1, GRU is better on squared error and the conclusion must be revised to a per-metric statement; if the table was transcribed incorrectly, a corrected comparison may restore the claim. Either way, the current text does not support 'KNN consistently outperformed other models.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 states that KNN is superior and 'consistently outperformed other models' for both CO and NOx. The paper's own Table 1 does not support this. For NOx, KNN is indeed best on all four metrics (MSE 0.00030, RMSE 0.01741, MAE 0.00864, MAPE 1.75548). For CO, however, KNN has MSE 0.00041, RMSE 0.02033, MAE 0.01020, MAPE 6.90689, while GRU has MSE 0.00036, RMSE 0.01896, MAE 0.01062, MAPE 8.34426. Thus on CO, GRU wins the squared-error metrics by roughly 14% in MSE and 7% in RMSE, while KNN wins only MAE and MAPE. Section 4.3 explicitly acknowledges this ('GRU achieves the lowest MSE and RMSE'), so the conclusion overgeneralizes. Because MSE/RMSE and MAE/MAPE disagree on CO, the central claim depends on silently treating MAE/MAPE as decisive without justification; if squared error is the relevant criterion for PEMS, the claim fails outright. The unreported train/test split and missing code make it impossible to check stability, but the internal contradiction is decisive on its own.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a comparative evaluation of eight machine learning models (Linear Regression, SVM, Decision Trees, XGBoost, MLP, LSTM, GRU, and KNN) for predicting CO and NOx emissions from gas turbines, using a proprietary 113,675-row steady-state dataset. The authors describe preprocessing, per-model hyperparameters, and report normalized MSE, RMSE, MAE, and MAPE on a test set. The abstract and conclusions claim that KNN consistently outperformed all other models and is therefore most suitable for predictive emission monitoring systems (PEMS).","tokens_in":7535,"tokens_out":3140,"duration_ms":31378,"significance":"If the claimed ranking were rigorously established, the paper could be a useful applied benchmark for PEMS model selection. The use of a real industrial dataset is a potential strength, and the comparison across both traditional and deep learning models is appropriate in scope. However, the significance is severely limited by the fact that the paper's central claim is contradicted by its own reported results for CO, and by the absence of any reproducible experimental protocol (no train/test split, no error bars, no statistical tests, no code or data release). As it stands, the paper does not provide a defensible basis for preferring KNN over the other models, so its practical and scientific contribution is marginal.","major_comments":[{"comment":"The conclusion that \"KNN consistently outperformed other models\" is contradicted by Table 1 for CO emissions. For CO, GRU achieves lower MSE (0.00036 vs 0.00041) and lower RMSE (0.01896 vs 0.02033) than KNN, while KNN only wins on MAE (0.01020 vs 0.01062) and MAPE (6.90689% vs 8.34426%). Section 4.3 itself acknowledges that GRU achieves the lowest MSE and RMSE for CO. Therefore the central claim is not supported by the paper's own data; the authors would need to either weaken the conclusion to a per-metric, per-pollutant statement or provide a principled justification for why MAE/MAPE should be decisive over squared-error metrics.","section":"Section 5 (Conclusion) and Table 1"},{"comment":"The experimental protocol is underspecified to the point of unreproducibility. The paper never states how the 113,675 rows were split into training, validation, and test sets, whether the split respects temporal order (which matters for time-series data), what window size was used for the LSTM and GRU inputs, whether the target variables were standardized, or how the reported 'normalized' metrics were computed. Without these details, the ranking in Table 1 cannot be independently checked, and the results could change under a different split or preprocessing choice.","section":"Section 4.1 and 4.2 (Data Collection and Training Setup)"},{"comment":"The paper reports a single run per model with no confidence intervals, standard deviations, or statistical significance tests. Several metrics are very close: for NOx, KNN's MSE (0.00030) and GRU's MSE (0.00032) differ by about 6%, which may well be within run-to-run noise. The claim that KNN 'consistently' outperforms is therefore not statistically established. The authors should report multiple runs with different seeds, error bars, or at least a significance test (e.g., paired bootstrap or Wilcoxon) to support the ranking.","section":"Section 4.3 (Results and Discussion)"}],"minor_comments":[{"comment":"There are numerous formatting and typographical issues: 'T raining Setup' in Section 4.2, 'T able 1' in Section 4.3, LaTeX artifacts such as '$num_epochs' and '$learning_rate' in the GRU paragraph, and an incomplete sentence in the Decision Trees paragraph where the model name is truncated to 'G'.","section":"Throughout"},{"comment":"Figure 2 is described as a 'histogram plot' but the subplots are bar charts of metric values per model; the caption should be corrected and the y-axis labels are missing.","section":"Figure 2"},{"comment":"The description of KNN says it is used for 'classification problems,' but in this paper it is used as a regressor; this should be corrected for clarity.","section":"Section 3 (Methodology)"},{"comment":"The dataset is described as containing 13 numerical variables, but the paper does not list them or cite a public source; if the dataset is proprietary, this should be stated explicitly, while a full feature list would help readers assess the modeling setup.","section":"Section 4.1"},{"comment":"The MAPE formula is undefined when yi = 0; since emission measurements can be near zero, the paper should discuss how such cases were handled in the normalized target space.","section":"Equation (2)"}],"recommendation":"reject","confidential_remarks":"The paper has a clear internal contradiction: the abstract and conclusion claim KNN consistently outperforms all models, but Table 1 shows GRU winning on squared-error metrics for CO. This is not a mere presentational slip; it invalidates the paper's main claim. Combined with the absence of any train/test split description, error bars, or statistical tests, and the lack of code/data availability, I do not see a reasonable revision path that would preserve the paper's claimed contribution. The journal may wish to consider whether a purely descriptive benchmark without reproducibility meets its standards."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper before citing it as evidence that KNN is the best model for gas turbine emission prediction. It is not. The paper's own Table 1 shows GRU has lower CO MSE (0.00036 vs 0.00041) and RMSE (0.01896 vs 0.02033) than KNN; KNN only wins on MAE and MAPE for CO. The authors even acknowledge this in Section 4.3, then the conclusion says KNN 'consistently outperformed other models.' That overgeneralization is the load-bearing flaw. It is not a deep methodological problem, but it is a real claim-vs-evidence mismatch.\n\nWhat the paper does well: it is an honest, applied comparison of eight off-the-shelf models on a real industrial dataset (113,675 steady-state rows from a Baker Hughes turbine). Hyperparameters are reported in enough detail to reproduce the models on your own data, and the related work is sensible, citing the public UCI benchmark and more recent deep forest work. That gives the paper value as a practitioner's data point, not as a research advance.\n\nThe soft spots are the usual ones for a private-data benchmark: no train/test split description, no error bars, no statistical tests, and no code or data released. So the model ranking is not independently verifiable, and the paper gives no sense of whether the differences between KNN, GRU, and XGBoost are meaningful or noise. You should also be careful that the CO conclusion depends on preferring MAE/MAPE over squared error without justification. If squared error matters for your PEMS, GRU wins on CO.\n\nWho is this for? Engineers working on PEMS for similar turbines who want a quick sanity check on model families. It is not for researchers looking for new methods or reproducible benchmarks. A serious referee could send this back with a request to fix the conclusion and report the experiment setup; it is not a desk-reject caliber paper, but it is not a strong accept either.","headline":"A competent industrial case study whose own Table 1 contradicts its headline claim that KNN consistently beats everything.","tokens_in":8011,"tokens_out":1355,"would_cite":false,"duration_ms":15280,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that K-nearest neighbors predicts CO and NOx emissions from a steady-state gas-turbine dataset more accurately than seven alternative models, making it the strongest candidate for integration into a Predictive Emission…","keywords":["gas turbine emissions","CO and NOx prediction","predictive emission monitoring system","machine learning comparison","K-nearest neighbors","LSTM and GRU","XGBoost","steady-state operation"],"falsifier":"Run all eight models on the same 113,675 rows with several random and chronological train/test splits, keeping the stated preprocessing and hyperparameters: if the NOx MAPE distributions of KNN and GRU overlap substantially across splits, the paper's superiority claim is not stable.","tokens_in":7040,"feed_emoji":"💨","tokens_out":9589,"duration_ms":83159,"temperature":0.7,"pith_summary":"The paper tries to establish that machine learning can predict carbon monoxide and nitrogen oxide emissions from gas turbines accurately enough to support predictive emission monitoring, and that one algorithm, K-nearest neighbors, does the job best. The evidence is a comparison of eight models on 113,675 steady-state operating records collected between August 2019 and April 2021, judged by normalized MSE, RMSE, MAE, and MAPE on a held-out test set. A sympathetic reader would care because a model that forecasts emissions from existing process sensors could lower the cost of continuous emissions monitoring and allow operators to adjust combustion in real time for regulatory compliance. The paper also claims that non-linear models decisively beat linear ones, with plain linear regression failing badly on CO.","feed_headline":"KNN wins eight-model race to predict turbine CO and NOx","feed_subtitle":"On 113,675 steady-state turbine readings, KNN edges GRU and XGBoost — a low-cost path to real-time emissions monitoring.","key_machinery":"The argument is carried by a side-by-side evaluation protocol: all models train on the same 113,675-row steady-state dataset, each with tailored preprocessing (standardization for most models, raw features for trees and XGBoost, a sliding window for LSTM) and hyperparameters tuned to minimize validation MSE, and are then ranked by normalized MSE, RMSE, MAE, and MAPE on the test set. Within that protocol the decisive mechanism is KNN's distance-weighted averaging over its four closest operational neighbors: it predicts an emission value by finding past operating points similar to the current one, rather than fitting a global formula, which suits the complex, non-linear emission patterns the paper finds in the data.","core_discovery":"On its own terms, the paper's discovery is that the K-nearest neighbors regressor, using four distance-weighted neighbors, posts the best overall accuracy on this dataset: the lowest MSE, RMSE, MAE, and MAPE for NOx (0.00030, 0.01741, 0.00864, and 1.76%) and the lowest MAE and MAPE for CO (0.01020 and 6.91%). One nuance in the table is that GRU has the lowest CO MSE and RMSE, so KNN's claim to the top rests mainly on NOx plus the CO error metrics that penalize typical rather than large errors. The authors read the pattern as consistent superiority and conclude KNN is highly suitable for integration into a Predictive Emission Monitoring System, with GRU, LSTM, and XGBoost also performing well and linear models, especially linear regression at 66.3% MAPE on CO, judged inadequate.","pith_inferences":["Because the paper reports a single train/test split whose construction is never described, the small margins between KNN and GRU (about 0.2 percentage points in NOx MAPE) could invert under resampling; testing across several splits is a direct next step.","The dataset covers only steady-state full-load operation, so the ranking says nothing about startup, shutdown, or transient regimes, where emission spikes are largest; extending the comparison to those regimes is the most consequential test.","GRU's win on CO MSE and RMSE while KNN wins on CO MAE and MAPE shows the 'best model' verdict depends on which error metric regulators weight, so a deployment choice would need a metric aligned with regulatory limits.","The recurrent models were given only 100 training epochs at fixed learning rates, so the comparison may under-represent what LSTM and GRU could reach with proper tuning."],"forward_implications":["KNN is accurate enough on both pollutants to be a leading candidate for integration into Predictive Emission Monitoring Systems.","Non-linear models as a class dominate linear ones on this data, with Linear Regression's 66.3% MAPE on CO marking it as unsuited to the task.","GRU, LSTM, and XGBoost are credible alternatives, with GRU attaining the lowest CO MSE and RMSE.","Accurate ML-based prediction allows operators to shift from after-the-fact compliance checks to real-time emission adjustments.","The tabulated metrics provide a benchmark of eight models on a common dataset for future PEMS research."],"supporting_citations":[{"why":"Supplies the gas-turbine emission dataset (the benchmark introduced there) that the comparison trains and tests on.","marker":"[8]"},{"why":"Introduces the Predictive Emission Monitoring System concept that defines the paper's deployment target.","marker":"[1]"},{"why":"Recent computational-intelligence PEMS study that the paper groups with the methods its eight-model comparison extends.","marker":"[9]"},{"why":"Provides the gradient-boosting emission model that the paper's XGBoost configuration builds on and compares against.","marker":"[10]"},{"why":"Recent deep-forest ensemble pipeline for CO and NOx prediction that marks the current state of the line of work being compared.","marker":"[13]"}],"fun_headline_variants":["KNN beats GRU and XGBoost to predict turbine NOx and CO","Nearest neighbors win emission forecast contest on 113k readings","KNN regressor outdoes nets for gas turbine CO and NOx","Simple KNN tops advanced ML for turbine emission prediction","KNN's four-neighbor model wins turbine emission race"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire ranking depends on a single train/test split that the paper never describes, so the reported margin, about two tenths of a percentage point in NOx MAPE, might not survive a different split.","fun_headline_variants_meta":{"raw":{"variants":["KNN beats GRU and XGBoost to predict turbine NOx and CO","Nearest neighbors win emission forecast contest on 113k readings","KNN regressor outdoes nets for gas turbine CO and NOx","Simple KNN tops advanced ML for turbine emission prediction","KNN's four-neighbor model wins turbine emission race"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000677,"raw_usage":{"total_tokens":3083,"prompt_tokens":957,"completion_tokens":2126,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":2036}},"tokens_in":573,"tokens_out":2126,"duration_ms":16067,"temperature":1.0,"reasoning_tokens":2036,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:35:26.699575+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run all eight models on the same 113,675 rows with several random and chronological train/test splits, keeping the stated preprocessing and hyperparameters: if the NOx MAPE distributions of KNN and GRU overlap substantially across splits, the paper's superiority claim is not stable.","supporting_citations":[{"cited_title":"& Others Predicting CO and NOxemissions from gas turbines: novel data and abenchmark PEMS..TURKISH J","cited_arxiv_id":null,"evidence_quote":"Supplies the gas-turbine emission dataset (the benchmark introduced there) that the comparison trains and tests on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Predictive Emission Monitoring System concept that defines the paper's deployment target."},{"cited_title":"Environmental Pollution Prediction of NOx by Process Analysis and Predictive Modelling in Natural Gas Turbine Power Plants","cited_arxiv_id":"2011.08978","evidence_quote":"Recent computational-intelligence PEMS study that the paper groups with the methods its eight-model comparison extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the gradient-boosting emission model that the paper's XGBoost configuration builds on and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Recent deep-forest ensemble pipeline for CO and NOx prediction that marks the current state of the line of work being compared."}],"review_version":1}