{"id":"c2a18d20-38e3-4170-bc9c-f6cca4de6e0d","arxiv_id":"2506.06285","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The proposed fuzzy regressors for photovoltaic forecasting are interpretable and easy to configure, but the empirical claim that they outperform machine learning and deep learning baselines is contradicted by the reported results.","lead":"This paper introduces several fuzzy inference models for day-ahead solar power forecasting, including a new Mamdani regressor, genetic-algorithm feature selection variants, and random ensemble versions. The reported advantage over machine learning and deep learning baselines is not supported by the paper's own tables.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's superiority claim is contradicted by the paper's own Tables 3–6: on every dataset the best reported NRMSE belongs to a baseline model, not to GEN-NTSK or RF-NTSK, and no validation split, seeds, or error bars are given.","rationale":"The reader's REJECT verdict is correct. The reader's weakest assumption focused on the missing validation split and the possibility of tuning on test data; that is a serious procedural concern. My stress-test identifies a more direct, load-bearing problem: even if the evaluation protocol were clean, the reported numbers themselves contradict the central claim. In every one of the four datasets, the best NRMSE is achieved by a baseline model, not by the proposed models highlighted in the abstract. The differences between the proposed models and the best baselines are typically small, and without repeated runs or error bars no superiority claim can be drawn from them. There is also no description of how the training and test sets were split or how random components of the ensemble models were seeded, which further undermines the comparisons. I therefore agree with the rejection, but my emphasis is on the internal contradiction between the abstract and the tables rather than solely on the unspecified tuning protocol. The model idea may still have value, but the paper as written does not establish its headline claim.","tokens_in":16289,"tokens_out":3884,"duration_ms":41578,"concrete_test":"Reproduce Tables 3–6 on the public DKASC Alice Springs and Yulara datasets using the nfisis package with a fixed 60/20/20 split, tuning hyperparameters on the validation fold only, and repeat each configuration with 10 seeds, reporting mean and standard deviation. Then compare the best proposed model in each table against the best baseline in that table. If a baseline remains best on all four series, or if the proposed model's advantage is within one standard deviation, the abstract's claim of superior performance is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that GEN-NTSK and RF-NTSK 'achieve superior performance' and 'often outperform' traditional ML and DL—is not merely unsupported; it is contradicted by the paper's own tables. Taking the reported NRMSE at face value: Alice 1A best is eTS 0.20639 vs GEN-NTSK(wRLS) 0.20711; Alice 38 best is exTS 0.20443 vs RF-NTSK 0.21087; Yulara 1 best is CNN 0.16568 vs RF-NTSK 0.18351; Yulara 5 best is ePL-KRLS-DISCO 0.20501 vs GEN-NTSK(wRLS) 0.20625. Thus a non-proposed baseline has the lowest error on every dataset. Section 4.1 states only that hyperparameters are 'optimized through a grid search to achieve the lowest possible error' without specifying a validation split, and Tables 3–6 report single point estimates with no seeds, repeats, or error bars. For daily PV series of roughly 730 points, differences of 0.001–0.02 NRMSE are likely within run-to-run noise, so even the reported ordering among close methods is not established. The abstract's 'superior performance' claim therefore fails at face value, independent of whether the grid search was contaminated by test data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new Mamdani fuzzy regressor (NMR) and extends the authors' earlier New Takagi-Sugeno-Kang (NTSK) model with genetic-algorithm feature selection and ensemble variants (GEN-NMR, GEN-NTSK, R-NMR, R-NTSK, RF-NTSK). The models are described mathematically, and an experiment is reported on four photovoltaic power datasets (Alice 1A, Alice 38, Yulara 1, Yulara 5) comparing NRMSE, NDEI, and MAPE against a wide range of classical, deep-learning, and evolving-fuzzy baselines. The abstract and conclusion claim that the proposed genetic and ensemble fuzzy models, especially GEN-NTSK and RF-NTSK, achieve superior performance and often outperform traditional ML and DL models, while maintaining interpretability. The paper also reports rule-based interpretation via a table of extracted rules and provides a Python library (nfisis).","tokens_in":16573,"tokens_out":4377,"duration_ms":40815,"significance":"If the reported superiority were established, the paper would offer a practical interpretable alternative to deep learning for PV forecasting. The model formulations are simple and the release of the nfisis library is a useful contribution. However, the central empirical claim is not supported by the data presented in the paper and is in fact contradicted by the authors' own tables: on every dataset a baseline model yields the lowest NRMSE. Because the experimental protocol lacks an explicit validation split and reports only single point estimates without seeds or error bars, the comparative performance claims cannot be regarded as reliable. The contribution therefore reduces to a modest algorithmic variation whose claimed advantage is unsubstantiated.","major_comments":[{"comment":"The central claim that GEN-NTSK and RF-NTSK 'achieve superior performance' and 'often outperform' traditional ML/DL models is contradicted by the paper's own results. In Table 3 (Alice 1A) the best NRMSE is eTS (0.20639) versus GEN-NTSK(wRLS) 0.20711; in Table 4 (Alice 38) exTS achieves 0.20443 versus RF-NTSK 0.21087; in Table 5 (Yulara 1) CNN achieves 0.16568 versus RF-NTSK 0.18351; in Table 6 (Yulara 5) ePL-KRLS-DISCO achieves 0.20501 versus GEN-NTSK(wRLS) 0.20625. Thus no proposed model obtains the best NRMSE on any dataset, and the abstract's superiority claim fails at face value.","section":"Abstract; Tables 3–6"},{"comment":"The hyperparameter optimization is described as a 'grid search to achieve the lowest possible error' (Section 4.1) without stating that a validation split was used; this leaves open the possibility that test data were used for model selection. The tables report single point estimates with no seeds, repeated runs, or error bars. With daily PV series of roughly 730 points, differences of 0.001–0.02 NRMSE are likely within run-to-run noise, so even the ordering among close methods is not established. The absence of a specified evaluation protocol invalidates the comparative performance claims.","section":"Section 4.1, Tables 3–6"},{"comment":"The Discussion section (Section 4.5) acknowledges that CNN and LS-SVM achieved the lowest errors among DL and classical models, and the Conclusion restricts the claim to 'GEN-NTSK and RF-NTSK obtained the lowest error among the proposed models.' This is inconsistent with the abstract's claim of outperforming ML/DL models. The paper should either remove the superiority claim or provide evidence that the proposed models are competitive when evaluated with a proper validation protocol.","section":"Section 4.3/4.5 and Conclusion"},{"comment":"Equation (3) assigns samples to rule index floor((y_k - y_underline)/IS), which yields 0 when y_k equals the minimum y_underline; since rules are indexed 1 to Rmax, this is an off-by-one error that would misassign the minimal-target sample. The authors should correct the indexing or clarify the boundary condition. While this is a localized technical issue, it affects the formal definition of the proposed NMR model.","section":"Section 3.1.1, Eqs. (1)–(3)"}],"minor_comments":[{"comment":"The sentence 'Finally, Section 5 concludes de paper and proposes future works' contains a typo ('de' should be 'the'), and the sentence 'The models are applied to renewable energy datasets' is repeated verbatim a few lines later.","section":"Section 1"},{"comment":"The description of R-NMR/R-NTSK ensemble construction is incomplete: 'the best-performing model from this subset is added to the ensemble' does not specify which data (training or validation) are used to judge the best performer, nor how diversity is maintained across iterations.","section":"Section 3.2"},{"comment":"The term 'NFISiS' is used both as the model family name and as the name of the released library (nfisis); the authors should clarify whether NFISiS refers to the set of models or specifically to the software package.","section":"Abstract and Section 1"},{"comment":"The reference for 'Regression Tree' cites only 'Breiman et al.' without a year or full publication details; this should be completed to match the other references.","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper's main empirical claim is contradicted by its own reported tables, and the experimental protocol is too incomplete to support the comparative conclusions. The central contribution is not salvageable in the present form: a proper revision would require new experiments with a clearly specified validation split, repeated runs, and error bars, and the outcome of such experiments is uncertain. I therefore recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a paper where the abstract promises more than the tables deliver. The NMR construction is a clean, simple idea: extend the authors' earlier NTSK by replacing polynomial consequents with constant rule means, with rules defined by equally spaced target intervals. That is a legitimate contribution, and the paper also ships a Python library (nfisis), which is real and useful for anyone wanting to try these models. The GA wrapper and ensemble variants are standard but competently described.\n\nThe soft spot is the empirical claim. The abstract says GEN-NTSK and RF-NTSK 'achieve superior performance' and 'often outperform' ML/DL models. But in Tables 3-6, the best NRMSE on every dataset belongs to a baseline: eTS on Alice 1A, exTS on Alice 38, CNN on Yulara 1, ePL-KRLS-DISCO on Yulara 5. The proposed models never win on NRMSE. On MAPE they win once (R-NTSK on Yulara 5). So the headline claim is contradicted by the paper's own numbers, independent of any methodological issue.\n\nThe evaluation protocol also is thin. Hyperparameters are 'optimized through a grid search to achieve the lowest possible error' with no mention of a validation split; no seeds, repeats, or error bars. With daily PV series of about 730 points, NRMSE differences of 0.001-0.02 are easily within run-to-run noise. So even the ordering between close methods is not established. That is a load-bearing problem, not a minor one.\n\nThere is also a small notational slip in Equation (1): the interval size should be (max-min)/Rmax, and what is written is ambiguous with the bar notation. But that is minor; the surrounding text clarifies it.\n\nWho is this for? Someone interested in simple, interpretable fuzzy rule construction might take the NMR idea and give it a proper evaluation. The paper is not ready as is, but it deserves a serious referee because the model idea is well-defined and the code is available. My recommendation: send it to review, but be prepared for a major revision or rejection unless the authors rerun the experiments with a proper validation protocol and drop the superiority claim that their own tables contradict.","headline":"Clear model idea, shipped code, but the empirical claim is contradicted by the paper's own tables; evaluation protocol is too weak to support superiority.","tokens_in":17100,"tokens_out":1932,"would_cite":false,"duration_ms":18719,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that genetic and ensemble fuzzy inference systems, especially GEN-NTSK and RF-NTSK, match or beat deep learning on photovoltaic forecasting while keeping rules interpretable.","keywords":["fuzzy inference systems","time series forecasting","Takagi-Sugeno-Kang models","Mamdani regressor","genetic algorithm feature selection","ensemble learning","photovoltaic power forecasting","interpretability"],"falsifier":"Re-run the four photovoltaic series with a fixed train/validation/test split, tune every model only on the validation set, and report errors over several random seeds; if GEN-NTSK and RF-NTSK no longer sit near the top of the NRMSE, NDEI, and MAPE tables, the comparative claim fails. A second check is whether the GA-selected feature subsets and final rule counts stay stable across seeds, since unstable selection would undercut the interpretability benefit.","tokens_in":16064,"feed_emoji":"☀️","tokens_out":8875,"duration_ms":80516,"temperature":0.7,"pith_summary":"The paper tries to establish that a family of data-driven fuzzy inference systems can give renewable-energy forecasters an interpretable alternative to deep learning without sacrificing accuracy. It extends the New Takagi-Sugeno-Kang (NTSK) model into a Mamdani-style regressor whose rules are cut from the target variable's own range, and then wraps that regressor in a genetic-algorithm feature selector and in ensembles. On four photovoltaic time series from two plants, the paper reports that the genetic and ensemble variants, especially GEN-NTSK and RF-NTSK, often post lower errors than classical machine-learning and deep-learning baselines while using just a handful of rules. If the claim holds, solar plant operators would get forecast accuracy comparable to deep learning from rule sets they can read, trained in far less time.","feed_headline":"Fuzzy models rival deep learning for solar forecasting","feed_subtitle":"Genetic and ensemble fuzzy models beat neural nets on four photovoltaic datasets while staying interpretable.","key_machinery":"The load-bearing mechanism is the rule-generation scheme shared by NTSK and the new NMR: the target variable's observed range is split into $R_{\\max}$ equally spaced intervals, each training sample is assigned to a rule by the interval its target value falls in, and Gaussian membership functions are fit to the antecedent attributes within each rule. The consequent of a rule is this target interval itself, so a rule reads as a statement of the form 'when the inputs resemble this cluster, the next-step power falls in this range'. The genetic wrapper evaluates candidate feature subsets by the fuzzy model's error, and the ensemble variants R-NMR, R-NTSK, and RF-NTSK average models built on randomized attribute subsets; RF-NTSK combines the random forest and R-NTSK outputs with weights inversely proportional to their training errors.","core_discovery":"The central claim is that the proposed fuzzy models, in particular Genetic New Takagi-Sugeno-Kang (GEN-NTSK) and Random Forest New Takagi-Sugeno-Kang (RF-NTSK), achieve error metrics that match or beat both traditional machine learning and deep learning on the tested photovoltaic series while producing a simpler, rule-based structure. The paper reports that on the Alice Springs and Yulara datasets, GEN-NTSK (wRLS) and R-NTSK/RF-NTSK frequently appear among the lowest normalized root-mean-square errors, and the best fuzzy models often tie or exceed the best non-fuzzy baselines, which are LS-SVM among classical models and CNN or GRU among deep models. The rule-design mechanism, splitting the target's range into equally spaced intervals, assigning samples to rules by where their target falls, and fitting Gaussian fuzzy sets from those partitions, gives the user direct control over the number of rules and makes each consequent interpretable as a target-variation interval rather than a polynomial.","pith_inferences":["Beyond the paper: the interval-based rule generation is not solar-specific, so the same models should transfer to other one-step-ahead forecasting targets such as wind power, electricity load, or prices; this is directly testable with the released library.","Beyond the paper: because the paper reports no variance across runs, a repeated-seed benchmark would reveal whether GEN-NTSK's edge is stable or partly tuning luck.","Beyond the paper: the un-evaluated NMR classifier sketched in the paper, one rule per class, could give an interpretable fuzzy classifier, but its accuracy is unknown."],"forward_implications":["A practitioner who needs an interpretable solar-forecasting model can choose GEN-NTSK or RF-NTSK and expect errors near the best deep-learning results on daily photovoltaic series.","Because the number of rules is a user-set hyperparameter, the same implementation can produce anything from a one-rule baseline to a nineteen-rule model, making the accuracy-interpretability tradeoff explicit.","RF-NTSK's error-weighted blend of random forest and R-NTSK gives a reusable recipe for combining a tree ensemble with a fuzzy regressor.","The genetic wrapper can shrink the input set, as in the reported Alice 1A case where six of twelve attributes were selected, which the paper links to better interpretability and reduced overfitting risk."],"supporting_citations":[{"why":"Proposes the NTSK model that this paper extends, supplying the target-variation rule-generation mechanism.","marker":"Alves et al. [2024]"},{"why":"Documents the TSK limitations and the prior NTSK crisp/interval forecasting design that motivates the new models.","marker":"Sa Teles Rocha Alves et al. [2024]"},{"why":"Defines the Mamdani inference architecture that the new NMR reworks into a data-driven regressor.","marker":"Mamdani [1974]"},{"why":"Defines the TSK rule form whose limitations the paper addresses with NTSK.","marker":"Takagi and Sugeno [1985]"},{"why":"Provides eTS, the evolving fuzzy baseline that obtains the lowest error in one of the comparison tables.","marker":"Angelov and Filev [2004]"},{"why":"Provides LSTM, the deep-learning baseline the proposed models claim to beat.","marker":"Hochreiter and Schmidhuber [1997]"},{"why":"Provides SVM, the classical baseline with the lowest errors among non-fuzzy models in several tables.","marker":"Cortes and Vapnik [1995]"},{"why":"Provides the random forest component used inside RF-NTSK.","marker":"Ho [1995]"}],"fun_headline_variants":["Genetic fuzzy model beats deep nets on solar data","Interpretable fuzzy rules rival neural nets in solar forecast","Genetic fuzzy model wins solar forecasting race","Rule-based fuzzy system outdoes deep learning on PV data","Fuzzy ensembles beat neural nets for solar power forecast"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central performance claims rest on the assumption that the grid search used to set hyper-parameters was run on training or validation data only and never on the test portion that produced the reported error tables; the paper does not state the data split, the random seeds, or the number of repeated runs.","fun_headline_variants_meta":{"raw":{"variants":["Genetic fuzzy model beats deep nets on solar data","Interpretable fuzzy rules rival neural nets in solar forecast","Genetic fuzzy model wins solar forecasting race","Rule-based fuzzy system outdoes deep learning on PV data","Fuzzy ensembles beat neural nets for solar power forecast"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000532,"raw_usage":{"total_tokens":2589,"prompt_tokens":1002,"completion_tokens":1587,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":1514}},"tokens_in":618,"tokens_out":1587,"duration_ms":10370,"temperature":1.0,"reasoning_tokens":1514,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:34:16.945484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the four photovoltaic series with a fixed train/validation/test split, tune every model only on the validation set, and report errors over several random seeds; if GEN-NTSK and RF-NTSK no longer sit near the top of the NRMSE, NDEI, and MAPE tables, the comparative claim fails. A second check is whether the GA-selected feature subsets and final rule counts stay stable across seeds, since unstable selection would undercut the interpretability benefit.","supporting_citations":[],"review_version":1}