{"id":"1fa170d9-3ec7-4917-9d5c-6820873a11fb","arxiv_id":"2412.17853","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A KAN-based doubly residual N-BEATS model with adversarial domain adaptation is claimed to improve zero-shot day-ahead electricity price forecasts on Nord Pool by 13% over N-BEATS and 24% over KAN.","lead":"Researchers built a neural network that combines Kolmogorov-Arnold layers with an N-BEATS-style residual structure and adversarial training to forecast day-ahead electricity prices in markets it never saw during training. The model beat two standard baselines on the Nord Pool market in a zero-shot test, but the paper ships no code, no detailed architecture, and no significance testing.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zero-shot claim rests on unsupported hyperparameter tuning and an internal inconsistency: the loss notation in Eq. 1 treats regression as classification, so the reported gains cannot be attributed to the proposed mechanism.","rationale":"I agree with the reader's central concern that the zero-shot protocol is unverified and likely compromised by hyperparameter tuning on overlapping calendar time. That alone supports rejection on reproducibility grounds. My analysis also identifies a second load-bearing issue the reader mentioned but did not fully develop: the paper's mathematical formulation (Eq. 1) defines the risk as classification error P(F(X) != Y) for a regression task, making the theoretical framing internally inconsistent. The domain-invariance claim is central to the paper's stated contribution but is never empirically verified; no experiment shows the domain classifier is confused, no ablation removes the adversarial branch, and no baseline error bars are given. Because the central claim depends on attribution of the improvement to the proposed KAN-plus-adversarial mechanism, and because the evaluation lacks both a clean zero-shot protocol and statistical support, the verdict remains REJECT. I would not move to a less severe verdict without code, named hyperparameters, a corrected theoretical formulation, and a significance test on the reported gains. I partially agree with the reader because they focused on leakage while I see the attribution problem as equally load-bearing; both concerns point to the same conclusion.","tokens_in":6020,"tokens_out":1464,"duration_ms":12879,"concrete_test":"Re-run the reported experiment under a strictly separated protocol: (1) Hold out all of 2018 for all markets (Nord Pool test, PJM validation) before any hyperparameter optimization; (2) optimize hyperparameters only on data up to 31 Dec 2016 (FR/BE) and PJM up to 31 Dec 2017; (3) retrain the proposed model and both baselines on this protocol; (4) report MAE/SMAPE for at least 5 random seeds per model with full result distributions. If the proposed model no longer beats N-BEATS/KAN by a statistically significant margin on the held-out 2018 Nord Pool period, the central zero-shot claim fails. Additionally, remove the adversarial branch (or freeze the gradient reversal layer) and rerun to test whether the reported improvement comes from domain adaptation or from other architectural choices.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that a KAN-based doubly residual N-BEATS with adversarial domain adaptation achieves lower MAE/SMAPE than KAN and N-BEATS in zero-shot forecasting of Nord Pool prices. For this to hold, (a) the evaluation must be genuinely zero-shot, and (b) the reported improvements must be attributable to the proposed architecture and adversarial training. Both conditions are insecure.\n\nFirst, §4 states hyperparameters were optimized with Bayesian optimization (TPE) but never says which data were used for selection. The PJM training set extends to 24 Dec 2018, overlapping the Nord Pool test period (1 Jan 2018 – 24 Dec 2018). If the same calendar span informed hyperparameter selection, information about the target period can leak into model choice, breaking the zero-shot protocol. The paper provides no code, no tuned hyperparameter values, no data split specification for the tuning set, and no ablation.\n\nSecond, the formal framework is internally inconsistent. Appendix A.1 defines the objective as minimizing risk R_Mt(F) = P(F(X) != Y) (Eq. 1), but forecasting is a regression problem with continuous Y, not classification. Classification risk is undefined for real-valued prices. This is not a stylistic slip: the entire adversarial mechanism is justified by domain-invariant representations, yet no experiment verifies that the domain classifier is actually confused. Table 1 reports only point-forecast errors; the domain-invariance claim is never tested, and no baselines with or without the adversarial branch are shown. Therefore the reported 13%/24% improvements cannot be attributed to the proposed mechanism rather than to implementation details or hyperparameters.\n\nThird, Table 1 gives error bars (±0.10, ±0.12, etc.) only for the proposed model, not for KAN or N-BEATS. Without variances for all models and significance tests, the improvements are not statistically supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a zero-shot time series forecasting model for day-ahead electricity prices that combines Kolmogorov-Arnold Networks (KANs) with a doubly residual N-BEATS-like architecture and an adversarial domain adaptation mechanism using a gradient reversal layer. The model is trained on two of three source markets (PJM, EPEX France, EPEX Belgium) and tested on the unseen Nord Pool market. The authors report MAE and SMAPE comparisons against standard KAN and N-BEATS baselines, claiming approximately 13% and 24% improvements in accuracy. The theoretical framework is described in Appendix A, with additional appendices on zero-shot learning and evaluation metrics.","tokens_in":6276,"tokens_out":8861,"duration_ms":74146,"significance":"If the results were properly validated, the paper would make a useful contribution: combining KANs with the N-BEATS residual stacking is a natural and relatively new idea, and demonstrating zero-shot transfer across electricity markets would have practical value for forecasting in markets with scarce historical data. The paper also correctly identifies the need for domain-invariant representations. However, the significance is severely weakened by the current form of the manuscript: the formal framework contains a fundamental error, the empirical evaluation is missing essential methodological detail, and the main performance claim is not supported by appropriate statistics. The reader cannot currently verify the central zero-shot claim, so the contributions remain at the level of an interesting proposal rather than a validated result.","major_comments":[{"comment":"Equation (1) defines the target risk as R_Mt(F) = P(F(X) ≠ Y), which is a classification error (0-1 loss). For the regression problem at hand, where Y is a continuous electricity price, the event F(X) ≠ Y essentially always occurs, making this quantity identically 1 and uninformative. The formal framework should instead minimize an expected loss E_{(x,y)∼P_Mt}[L(F(x), y)] for a suitable regression loss L (e.g., MAE or SMAPE). This is not a stylistic slip: the adversarial training justification in the same appendix is expressed in terms of this mis-specified risk, and as written it does not describe the actual optimization objective used in the experiments.","section":"Appendix A.1"},{"comment":"The zero-shot protocol is not verifiable. The paper states that hyperparameters 'are optimized using a Bayesian optimization method' with a tree-structured Parzen estimator, but it never states which data were used for selection. The PJM training set extends to 24 December 2018, which is the same calendar period as the Nord Pool test set (1 January 2018 to 24 December 2018). If any part of the tuning procedure used data from that overlapping period, information about the target market could have influenced model selection, breaking the zero-shot claim. The authors must specify the exact data split used for hyperparameter optimization, report the tuned hyperparameter values, and confirm that no target-market data or same-period source data were used for model selection.","section":"Section 4"},{"comment":"The baseline KAN and N-BEATS results are reported as single point values with no error bars, while only the proposed model is shown with ± intervals. The number of independent runs is not stated, and no statistical significance tests are performed. Consequently, the claimed improvement of 'around 13% and 24%' is not supported by the reported evidence. The authors should report means and standard deviations over multiple seeds for all models, state the number of runs, and apply an appropriate paired test (e.g., Diebold-Mariano or bootstrap) to the forecast errors to demonstrate that the differences are not due to noise.","section":"Table 1"},{"comment":"The central claim that the adversarial framework 'learns market-invariant representations' is not empirically verified. There is no experiment showing that the domain classifier is actually confused, no comparison to a non-adversarial version of the same backbone, and no ablation varying the gradient reversal strength or architecture. Without such experiments, the observed error differences cannot be attributed to the adversarial mechanism; they could be due entirely to the KAN-based residual backbone. The authors should add an ablation and report the domain classifier accuracy on held-out source data to demonstrate that invariance is achieved.","section":"Section 2 and Appendix A.1"},{"comment":"The experimental protocol is under-specified. The sentence 'The values are averaged over different models, each time with a different market as the secondary market' is ambiguous: for each primary market there are two remaining source markets, so it is unclear whether the reported numbers are averages over two models (one per secondary market) or over a different number of configurations, and whether the secondary markets were used individually or jointly. The authors must clarify the number of models averaged, the exact training configurations, and how the reported error bars were computed.","section":"Section 4"}],"minor_comments":[{"comment":"The entire appendix on 'Zero Shot Learning with attributes' discusses class-attribute matrices and seen classes, which has no connection to the time series forecasting problem addressed in this paper. It appears to be leftover material from a different manuscript and should be removed or substantially rewritten to discuss domain adaptation for regression.","section":"Appendix B"},{"comment":"In Equation (7), there is a typo: 'l(h(x, y' should be 'l(h(x,y))'. The covariate shift derivation is also not used elsewhere in the paper; if it is kept, it should be explicitly connected to the proposed method.","section":"Appendix B.1"},{"comment":"The phrase 'We observe an improvement of around 13% and 24% in accuracy' is imprecise: the percentages should be computed directly from the table values, the baselines should be named, and 'accuracy' is a misnomer for regression error metrics.","section":"Section 4"},{"comment":"The caption reads 'The next day forecast presents a comparison' and is awkwardly worded. More importantly, a single qualitative example is not evidence of systematic performance; the quantitative results in Table 1 are the relevant evidence.","section":"Figure 2"},{"comment":"Equation (2) uses the notation 'N Ni,j(yb i,j−1, Xb−1)' without defining Xb−1 or clarifying the indexing of the backcast input. All symbols should be defined to make the architecture reproducible.","section":"Appendix A.2"},{"comment":"The phrase 'This addition not only enriches the model's representational capacity' is vague; the abstract would be clearer if it stated the architecture components explicitly.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a workshop-style manuscript that has been submitted to a journal without significant expansion. The unrelated nature of Appendix B, the undefined notation in Eq. (2), and the ambiguous description of the averaging in Table 1 suggest that the manuscript was assembled quickly. The authors should be asked to address the major comments thoroughly, especially the zero-shot protocol and the error bars; if they cannot provide the necessary experimental evidence, the paper should not be accepted. There is also no code repository, which would be important for reproducibility in a journal setting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's central claim — that a KAN-based doubly residual N-BEATS with adversarial domain adaptation beats plain KAN and N-BEATS on zero-shot Nord Pool forecasting — is plausible but not supported by the experiments as reported. The architecture combination is genuinely new (I don't know of prior work mixing KAN layers, N-BEATS residual stacks, and a gradient reversal layer for energy price forecasting), and the problem is a real one: day-ahead markets with scarce data need off-the-shelf forecasters. The authors also deserve credit for testing on an unseen market with three different primary markets, and for keeping the model description reasonably clear.\n\nWhere it falls down is the evaluation. The PJM training set runs through 24 Dec 2018, which is the same calendar window as the Nord Pool test period. That's a temporal overlap that could let the model learn time-specific effects from another market, so the 'zero-shot' label is questionable. Worse, the paper never says what data the Bayesian optimization used for hyperparameter selection. If any of that touched the test window, the reported gains are not zero-shot at all. The baselines have no error bars, so the 13%/24% improvements could easily be noise. There's also a formal slip: Appendix A.1 defines risk as P(F(X) != Y), which is a classification risk, but forecasting is regression. That's not a cosmetic typo — it undermines the theoretical framing. And despite the paper's claim that the adversarial branch learns market-invariant representations, there is no ablation or domain-classifier confusion measurement to show the branch actually does anything. The gains could come from the KAN backbone alone or from hyperparameters.\n\nNone of these are fatal to the idea. The combination is worth exploring, and a clean evaluation would be useful. But as it stands, the evidence doesn't establish the claims. I'd tell the authors to fix the temporal split, specify the tuning protocol, add error bars and significance tests, run an ablation of the adversarial branch, and correct the formalism. That version would deserve a serious referee.\n\nThis paper is for researchers working on cross-market forecasting or KAN applications. It's not ready for publication in its current form, but the kernel is there. I would not cite it in the next year, and I would not bring it to a reading group as a model of experimental practice. For peer review: I'd desk reject now, but with an invitation to resubmit after a proper revision.","headline":"Plausible architecture, unconvincing evidence: the zero-shot gains aren't established because the evaluation has temporal leakage and missing statistics.","tokens_in":6920,"tokens_out":4091,"would_cite":false,"duration_ms":34777,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes that a doubly residual N-BEATS network built on Kolmogorov-Arnold Networks and trained with adversarial domain adaptation can forecast day-ahead electricity prices for an unseen market, reporting 13% and 24% lower error…","keywords":["zero-shot forecasting","electricity price forecasting","Kolmogorov-Arnold networks","N-BEATS","domain adaptation","adversarial training","spline activations","day-ahead electricity market"],"falsifier":"Inspect the Bayesian optimization logs and training configuration: if any Nord Pool 2018 prices were passed to hyperparameter search, or if the reported model was chosen after looking at Nord Pool test errors, the zero-shot claim collapses. A cleaner test is to truncate every training market before 2018, fix hyperparameters using only pre-2018 data, and rerun; if the 13% and 24% margins vanish, the central claim fails.","tokens_in":5773,"feed_emoji":"⚡","tokens_out":7586,"duration_ms":61825,"temperature":0.7,"pith_summary":"This paper tries to establish that a forecasting model can transfer across electricity markets without ever seeing the target market. The model stacks Kolmogorov-Arnold Networks, which use learnable univariate spline functions on their edges, inside a doubly residual N-BEATS structure, and trains the whole system adversarially on two source markets so that the learned features become market-invariant. Tested by forecasting Nord Pool day-ahead prices for all of 2018 with no Nord Pool data in training, the model reports lower mean absolute error and symmetric MAPE than standard KAN and N-BEATS baselines, by about 24% and 13%. If this holds, a trader entering a new market could generate usable day-ahead forecasts immediately, without years of local price history.","feed_headline":"Zero-shot price forecasts beat KAN and N-BEATS by 13-24%","feed_subtitle":"Adversarial KAN-based N-BEATS model forecasts unseen Nord Pool prices with no local training data.","key_machinery":"The load-bearing object is a doubly residual KAN stack: a sequence of blocks, each a Kolmogorov-Arnold Network whose edge operations are learnable univariate B-spline functions, wired together with backcast residuals and hierarchical forecast aggregation in the style of N-BEATS. Around this backbone, a gradient reversal layer implements the adversarial game between the price forecaster and a domain classifier, forcing the early representation to hide which source market produced a sample while later layers keep market-specific detail. This combined structure is what the paper claims generates domain-invariant features, and it is the mechanism by which the zero-shot transfer is supposed to happen.","core_discovery":"The central claim is that replacing the MLP blocks of N-BEATS with Kolmogorov-Arnold Networks and adding a gradient-reversal domain classifier to the training objective yields representations that transfer to a market excluded from training. In the reported experiments, the proposed model reaches a mean absolute error around 2.51 and a symmetric MAPE around 0.086 on the Nord Pool 2018 test year, regardless of which of the three source markets is treated as primary; the best N-BEATS and KAN baselines are higher on both metrics. The authors state this is an improvement of roughly 13% over N-BEATS and 24% over KAN, and that the adversarial framework learns market-invariant representations.","pith_inferences":["Ablating the gradient reversal layer while keeping the KAN backbone would show whether the domain-adversarial term, rather than the spline architecture alone, causes the zero-shot gain; the paper does not report that comparison.","The current model ingests only the price series, so a testable extension is to add exogenous drivers such as temperature or load forecasts to the KAN inputs, which the spline activations can accommodate without a change in the adversarial objective.","The paper trains with one primary and one secondary market per run; a natural extension is to feed several secondary markets through the same domain classifier, since the objective only requires a domain label per sample."],"forward_implications":["A market operator opening in a new region could issue day-ahead forecasts on day one, using only price histories from established markets.","The reported 13% and 24% error reductions imply that, among the architectures tested, the KAN-based adversarial backbone is the better default for cross-market zero-shot forecasting.","The zero-shot errors sit in the same order of magnitude as published supervised results with NBEATSx, so the approach could substitute for local training when target-market data are unavailable.","Because the KAN layers learn smooth B-spline curves, forecasts come with a visual account of the functional shapes the model uses, which is not available from opaque large models."],"supporting_citations":[{"why":"Supplies the doubly residual N-BEATS architecture of backcast residuals and hierarchical forecast aggregation that the proposed backbone adapts.","marker":"[2]"},{"why":"Provides the Kolmogorov-Arnold representation theorem and the KAN design with learnable univariate spline functions on edges.","marker":"[3]"},{"why":"Supplies the gradient reversal layer used to train the forecasting model adversarially against the domain classifier.","marker":"[4]"},{"why":"Defines the standard KAN architecture used as a time-series baseline in the zero-shot comparison.","marker":"[6]"},{"why":"Provides published NBEATSx electricity-price forecasting results used to state that the proposed zero-shot errors are in the same order of magnitude.","marker":"[7]"},{"why":"Describes the Bayesian optimization procedure used to select hyperparameters, which is central to assessing whether the target market influenced model selection.","marker":"[5]"},{"why":"Motivates domain adaptation for time series and the temporal-dynamics considerations the framework must address.","marker":"[1]"}],"fun_headline_variants":["Zero-shot energy forecasts beat KAN and N-BEATS by 13-24%","Adversarial KAN-N-BEATS model nails unseen market prices","Cross-market zero-shot forecasting with KAN networks","KAN-based model adapts to new markets with zero training","Energy price forecasting that transfers without retraining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the target market's data never entering training or model selection; the paper does not specify which data the Bayesian optimizer saw, and the PJM training set runs through the same calendar year as the Nord Pool test period.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot energy forecasts beat KAN and N-BEATS by 13-24%","Adversarial KAN-N-BEATS model nails unseen market prices","Cross-market zero-shot forecasting with KAN networks","KAN-based model adapts to new markets with zero training","Energy price forecasting that transfers without retraining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000712,"raw_usage":{"total_tokens":3188,"prompt_tokens":914,"completion_tokens":2274,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":2187}},"tokens_in":530,"tokens_out":2274,"duration_ms":16740,"temperature":1.0,"reasoning_tokens":2187,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:27:45.721149+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the Bayesian optimization logs and training configuration: if any Nord Pool 2018 prices were passed to hyperparameter search, or if the reported model was chosen after looking at Nord Pool test errors, the zero-shot claim collapses. A cleaner test is to truncate every training market before 2018, fix hyperparameters using only pre-2018 data, and rerun; if the 13% and 24% margins vanish, the central claim fails.","supporting_citations":[{"cited_title":">8q /R3ω 8'N;> H sEAmo h>rey.G]`ڦ' 崋pɸ/.wʶm sڦ,","cited_arxiv_id":null,"evidence_quote":"Provides the Kolmogorov-Arnold representation theorem and the KAN design with learnable univariate spline functions on edges."},{"cited_title":"Time series forecasting model based on domain adaptation and shared attention","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient reversal layer used to train the forecasting model adversarially against the domain classifier."},{"cited_title":"Domain-adversarial training of neural networks","cited_arxiv_id":null,"evidence_quote":"Provides published NBEATSx electricity-price forecasting results used to state that the proposed zero-shot errors are in the same order of magnitude."}],"review_version":1}