{"id":"033952a5-660f-4516-bd5f-4084e590d7e4","arxiv_id":"2412.09889","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LeakySineLU, a piecewise-scaled Snake-style activation with a periodic derivative, achieves the best average rank over 112 UCR time series datasets in MLP and FCN classifiers.","lead":"This paper introduces LeakySineLU, a new mathematical rule for neural networks that mixes sine-squared oscillations with a straight line, and tests it on 112 time series classification datasets. It reports the best average ranking in two standard network setups, but the improvements over existing rules are often not statistically significant.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MLP win/loss/draw counts all sum to 127 while the paper claims 112 UCR datasets; if MLP rankings used a different dataset set, the central ranking claim is not supported as stated.","rationale":"I chose the dataset-count inconsistency over the reader's weakest assumption (fixed schedule, no repeated seeds) because it is a concrete, checkable error in the reported results themselves. The FCN counts matching 112 while every MLP count sums to 127 is strong evidence that the MLP experiment did not use the stated 112 datasets, which directly affects the validity of the central ranking claim. The absence of repeated-seed variance is a robustness concern, but it would not invalidate the numerical claim if the computation were correct. The counting error, by contrast, suggests the computation may have been performed on a different benchmark set. The fix is straightforward (recompute on the correct subset), so the appropriate verdict remains conditional rather than outright rejection; the reader already demanded count corrections, so my concern does not move the verdict.","tokens_in":10986,"tokens_out":8825,"duration_ms":89776,"concrete_test":"Clone https://github.com/jose-gilberto/leakysinelu, inspect the raw accuracy outputs for MLP and FCN, and count the number of datasets (rows) in each result table. If the MLP table has 127 rows and the FCN table has 112 rows, identify the dataset list used for the MLP experiments, filter it to the 112 equal-length UCR datasets (no 'Vary' as time series length), and recompute the average ranks and the critical-difference diagram for MLP. Check whether LeakySineLU remains in first position with average rank 4.2402. Also verify that every win/loss/draw triple sums to 112 after correction. If LeakySineLU loses first place, the headline claim fails for MLP; if it remains first, the paper should still be corrected to report the actual dataset count.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 and the abstract state that experiments use '112 equal-length subset' datasets from UCR. However, every pairwise win/loss/draw count reported for the MLP architecture sums to 127, not 112. In Fig. 4, LeakySineLU vs ReLU is 67/3/57, vs Snake is 75/3/49, vs PReLU is 68/3/56; in Fig. 6, all five comparisons also sum to 127. In contrast, the FCN counts (Fig. 7 and Fig. 9) consistently sum to 112. This systematic discrepancy indicates that the MLP ranking analysis was run on a different number of datasets (likely 127) from the FCN analysis, or that the reported counts are erroneous. Because the MLP critical-difference diagram (Fig. 5) and the abstract's 'best average ranking in all comparative scenarios' rest on those rankings, the empirical basis of the headline claim is internally inconsistent. This is not a stylistic issue: if the MLP rank order was computed on a 127-dataset set that includes datasets excluded from the stated 112 equal-length subset, the claim as written is false and the FCN result alone cannot rescue it.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LeakySineLU, a piecewise activation defined as sigma(x) = sin^2(x) + x for x > 0 and (sin^2(x) + x)/2 otherwise, designed to combine unboundedness, non-linearity, monotonicity, and a semi-periodic derivative. It evaluates this activation against ReLU, PReLU, ELU, GeLU, SiLU, Snake, TanH, Sigmoid, and Sine in an MLP and an FCN on 112 equal-length UCR time-series classification datasets. The central claim is that LeakySineLU obtains the best average ranking in all comparative scenarios, supported by critical-difference diagrams, pairwise win/draw/loss counts, and multi-comparison matrices.","tokens_in":11186,"tokens_out":13572,"duration_ms":121890,"significance":"If the empirical claim holds, the paper offers a simple, drop-in activation that improves rank-based aggregate performance for time-series classification in simple MLP and FCN models, and its property checklist is a useful organizing device. The study is broad in benchmark coverage, uses standard non-parametric statistical post-hoc analysis, and makes the code available. There is no evident circularity: LeakySineLU is hand-designed from a property checklist and evaluated on external benchmarks, with no constants fitted to the UCR results. The significance is limited by the small mean-accuracy differences and the lack of repeated-seed variance; the contribution is incremental rather than a major advance, and the claims need to be calibrated to the actual evidence.","major_comments":[{"comment":"The MLP pairwise win/draw/loss counts are inconsistent with the stated 112-dataset benchmark. In Fig. 4, LeakySineLU vs ReLU is 67/3/57 (sum 127), vs Snake is 75/3/49 (sum 127), and vs PReLU is 68/3/56 (sum 127); every comparison in the MLP MCM of Fig. 6 also sums to 127, whereas the FCN counts in Figs. 7 and 9 sum to 112. Since the captions and text state that the experiments use 112 equal-length datasets, the MLP ranking in Fig. 5 and the MCM in Fig. 6 may have been computed on a different dataset set. This is a load-bearing inconsistency for the abstract claim of best average ranking in all comparative scenarios. The authors must either rerun the MLP analysis on the declared 112 equal-length subset or re-scope the claim to the actual dataset set used; the FCN results alone cannot support the MLP claim.","section":"§5, Figs. 4 and 6"},{"comment":"The headline ranking result is accompanied by lower mean accuracy and mostly non-significant pairwise differences in both architectures. In the MLP MCM, LeakySineLU has mean accuracy 0.7081 against ReLU 0.7098 and PReLU 0.7097, with Wilcoxon p-values 0.8267 and 0.9042; in the FCN MCM it has 0.8006 against ReLU 0.8025, with p = 0.8583. No repeated-seed variance is reported, so the rank ordering may be unstable. The paper should either temper the claim from best to conditionally competitive, or add confidence intervals and repeated seeds and show that the average-rank advantage is not an artifact of a single run. A majority of wins in pairwise counts is not sufficient evidence of superiority when the mean accuracy is lower and the test is not significant.","section":"§5, Figs. 6 and 9"},{"comment":"The benchmark is restricted to the 112 equal-length UCR datasets and to one fixed training schedule per architecture (Adadelta with lr = 1.0 and 1000 epochs for MLP; Adam with lr = 0.001 and 2000 epochs for FCN), while the abstract and conclusion speak of time series classification without this qualification. The paper should either narrow the claims to equal-length classification under this fixed protocol, or add evidence that the ranking is stable under variable-length series and different hyperparameters. Otherwise the generalization claim is not supported by the reported experiments.","section":"§4.1, footnote 1"}],"minor_comments":[{"comment":"The otherwise branch of the definition is ambiguous; as printed, sin2(x)+x over 2 could be read as sin^2(x) + x/2 rather than (sin^2(x)+x)/2. Please add parentheses.","section":"§3.2, Eq. (5)"},{"comment":"Because sin^2(+∞) is undefined, the limit argument should be written using inequalities, for example sin^2(x) ≥ 0, so x + sin^2(x) → +∞ and (x + sin^2(x))/2 → -∞.","section":"§3.3, Eqs. (7)-(8)"},{"comment":"Definition 3 requires σ′(x+T) = σ′(x) for all x, but the derivative of LeakySineLU is undefined at x = 0 and the one-sided limits differ; please state that the periodicity holds on each branch of the derivative.","section":"§2, Definition 3"},{"comment":"The sub-derivative definition in Section 2 is stated for convex functions, but LeakySineLU is not convex; the treatment of the discontinuity at x = 0 should instead use one-sided derivatives or a clearly defined generalized subgradient.","section":"§3.5"},{"comment":"The Fourier-series motivation leading to Eq. (17) is not rigorous: cosine and sine basis terms with different arguments cannot be collapsed into a single sin(X) applied to an input matrix without specifying phase and argument-matching conditions.","section":"§3.6"},{"comment":"The sentence introducing the ReLU family is garbled: it says ReLU, ELU as an exponential linear unit (PReLU) for its learnable parameters, which conflates ELU and PReLU and introduces LeakyReLU elsewhere without defining it.","section":"§3.1"},{"comment":"The Friedman test is invoked following [5], but no Friedman statistic or p-value is reported; please include it or clarify that only the Wilcoxon signed-rank test with Holm correction is used.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The MLP dataset-count discrepancy is the key gate. If the corrected rerun changes the MLP ranking, the abstract and conclusions must be revised accordingly. The contribution is incremental but potentially publishable if the claims are recalibrated to the evidence. I would ask for the rerun before recommending acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: LeakySineLU is a minor variant of Snake—for positive inputs it is exactly Snake with a=1, and for negative inputs it is the same expression halved. The paper's real contribution is empirical: a property checklist for time-series activations and a benchmark on 112 equal-length UCR datasets across MLP and FCN, with release of code. The FCN ranking (best average rank 3.0491) is plausible and internally consistent. But the MLP pairwise win/loss/draw counts all sum to 127, not 112: 67+3+57, 75+3+49, 68+3+56, and the MCM rows do the same. That is a systematic discrepancy, not a typo, and it is load-bearing because the abstract's \"best average ranking in all comparative scenarios\" rests on the MLP ranking as much as the FCN one. Unless the authors can show the MLP analysis actually used a 127-dataset set, the headline claim is false as written. This alone should trigger a major revision. The other soft spots are real but smaller. Mean accuracy is slightly below ReLU in both architectures (MLP 0.7081 vs 0.7098; FCN 0.8006 vs 0.8025), so the \"best\" claim is rank-based, and most pairwise differences are not statistically significant after Holm correction. There is no repeated-seed variance reported, one fixed training schedule per architecture, and variable-length datasets are excluded. Those limitations deserve mention in the paper, and they temper the practical significance. What the paper does well: the property analysis is genuinely useful for thinking about why activations might matter for time series, the experimental protocol follows standard practice from the time-series deep learning literature (Friedman test, critical difference diagrams, MCM), and the code is public. The writing is honest about statistical equivalence for many comparisons, which I credit. The novelty is modest, so this is not a breakthrough. But it is a legitimate empirical question—does a semi-periodic activation help on time series classification?—and the paper gives a reproducible, if currently inconsistent, partial answer. I would send this to a serious referee. It is not a desk reject. But it needs the MLP dataset count corrected, variance reported, and the abstract toned down to match the actual effect sizes.","headline":"A modest Snake variant whose FCN results are plausible but whose MLP pairwise counts sum to 127, not 112—so the central claim is currently not supported as written.","tokens_in":11777,"tokens_out":2018,"would_cite":false,"duration_ms":22055,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that LeakySineLU, a semi-periodic activation whose derivative is periodic, is the best-ranked activation for time series classification on 112 equal-length UCR datasets in both an MLP and an FCN.","keywords":["LeakySineLU","activation function","time series classification","semi-periodic activation","periodic derivative","deep learning","UCR archive"],"falsifier":"Run the same MLP and FCN comparison with per-activation hyperparameter search and include the variable-length UCR datasets, reporting accuracy over multiple random seeds; if LeakySineLU no longer holds the best average rank in either architecture, or if its rank advantage over ReLU and PReLU falls within seed noise, the paper's central claim fails.","tokens_in":10750,"feed_emoji":"📈","tokens_out":5124,"duration_ms":46658,"temperature":0.7,"pith_summary":"The paper argues that activation functions for time series neural networks have been under-studied, and that the key desirable properties—unboundedness, non-linearity, monotonicity, and a periodic derivative—can be combined in one function. It proposes LeakySineLU, defined piecewise as $\\sin^2(x)+x$ on positive inputs and half that on negative inputs, and evaluates it against nine standard activations on 112 equal-length UCR classification datasets. In both a simple MLP and a fully convolutional network, LeakySineLU achieves the best average rank among all compared activations, though its mean accuracy is often close to or slightly below ReLU and PReLU. The paper's central claim is that this semi-periodic activation is a stronger default choice for time series classification than commonly used alternatives.","feed_headline":"LeakySineLU ranks first on 112 time-series datasets","feed_subtitle":"A semi-periodic activation with a periodic derivative beats nine standard activations in average rank on the UCR benchmark.","key_machinery":"The central object is the LeakySineLU activation, $\\sigma(x)=\\sin^2(x)+x$ for $x>0$ and $\\sigma(x)=(\\sin^2(x)+x)/2$ otherwise. Its derivative is $\\sigma'(x)=\\sin(2x)+1$ on the positive side and $\\sigma'(x)=(\\sin(2x)+1)/2$ on the negative side, so the derivative is periodic with period $\\pi$ on each side while the function itself is unbounded and monotonic. The discontinuity at $x=0$ is handled with a sub-derivative, exactly as ReLU and PReLU handle theirs. This combination—unbounded, monotonic, non-linear, and with a periodic derivative—is what the paper argues lets a network keep negative-valued observations (no dying ReLU) while still expressing periodic patterns.","core_discovery":"On 112 equal-length datasets from the UCR archive, the paper reports that LeakySineLU achieves the best average rank among ten activation functions in both architectures tested: average rank 4.2402 in the MLP and 3.0491 in the FCN. In pairwise one-versus-one comparisons it wins a majority of datasets against ReLU, PReLU, and Snake, and in the FCN it is statistically different from ELU and Snake under the Wilcoxon test with Holm correction. Notably, in the MLP its mean accuracy (0.7081) is slightly below ReLU (0.7098) and PReLU (0.7097), so the rank advantage is not the same as a mean-accuracy advantage. The paper interprets the results as evidence that a semi-periodic activation captures periodic structure in time series better than monotonic or purely periodic alternatives, while avoiding the vanishing-gradient and dying-ReLU failure modes of bounded and zero-slope activations.","pith_inferences":["Because the evaluation used one fixed training schedule for every activation, a testable extension is to repeat the comparison with per-activation hyperparameter tuning and with variable-length UCR datasets included; the rank ordering could change under either condition.","The gap between LeakySineLU's best average rank and its slightly lower mean accuracy in the MLP suggests the rank advantage may come from avoiding worst-case failures on particular datasets, so the activation could be most valuable where other activations collapse.","The factor of 1/2 on the negative branch is an arbitrary design choice; testing other scales would reveal whether the exact constant matters for the reported advantage.","The same periodic-derivative principle could be tested on other time series tasks such as forecasting and extrinsic regression, where seasonal structure is even more central."],"forward_implications":["LeakySineLU is a drop-in activation with no extra learnable parameters, applicable to both dense and convolutional time series classifiers.","On equal-length UCR classification, it ranks first among ten activations in both the MLP and FCN settings by average rank.","It preserves negative-valued observations instead of zeroing them, addressing the dying-ReLU information loss while remaining unbounded and non-linear.","Its derivative is periodic in both the positive and negative domains, giving networks a mechanism to represent periodic structure, consistent with the paper's Fourier-series motivation.","In pairwise comparisons, it wins a majority of datasets against ReLU, PReLU, and Snake in both architectures."],"supporting_citations":[{"why":"Supplies the 112 equal-length UCR time series datasets that form the entire experimental benchmark.","marker":"[3]"},{"why":"Provides the MLP and FCN architecture templates and the statistical protocol (Friedman test, Wilcoxon signed-rank test, critical difference diagrams) used for ranking.","marker":"[5]"},{"why":"Supplies the Multi-Comparison Matrix method used as a stability check on the average-ranking results.","marker":"[10]"},{"why":"Motivates the need for periodic behavior by showing neural networks fail to learn periodic functions, and supplies the Universal Extrapolation Theorem used in the Fourier-series derivation.","marker":"[31]"},{"why":"Provides the discussion of dying ReLU and learnable negative slopes that motivates the negative-branch behavior of LeakySineLU.","marker":"[8]"},{"why":"Serves as the sine-as-activation baseline and as motivation for using periodic functions in deep networks.","marker":"[16]"},{"why":"Supplies the time-series classification from scratch architectures that the paper adapts into its MLP and FCN models.","marker":"[24]"},{"why":"Supplies the comparative study of periodic versus monotonic activation functions that motivates the semi-periodic design.","marker":"[23]"}],"fun_headline_variants":["LeakySineLU achieves best average rank on 112 UCR datasets","Semi-periodic activation leads on 112 time-series benchmarks","New activation LeakySineLU ranks first among 10 on 112 sets","LeakySineLU outranks 9 rivals on 112 time-series datasets","Periodic derivative helps LeakySineLU rank top on 112 datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire comparison uses one fixed training schedule (optimizer, learning rate, and epoch count) applied identically to every activation, and it covers only equal-length UCR datasets, so if that schedule happens to suit LeakySineLU or variable-length series are where periodic derivatives matter, the best-rank result will not generalize.","fun_headline_variants_meta":{"raw":{"variants":["LeakySineLU achieves best average rank on 112 UCR datasets","Semi-periodic activation leads on 112 time-series benchmarks","New activation LeakySineLU ranks first among 10 on 112 sets","LeakySineLU outranks 9 rivals on 112 time-series datasets","Periodic derivative helps LeakySineLU rank top on 112 datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00045,"raw_usage":{"total_tokens":2214,"prompt_tokens":837,"completion_tokens":1377,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":1294}},"tokens_in":453,"tokens_out":1377,"duration_ms":12158,"temperature":1.0,"reasoning_tokens":1294,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:36:34.409849+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same MLP and FCN comparison with per-activation hyperparameter search and include the variable-length UCR datasets, reporting accuracy over multiple random seeds; if LeakySineLU no longer holds the best average rank in either architecture, or if its rank advantage over ReLU and PReLU falls within seed noise, the paper's central claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 112 equal-length UCR time series datasets that form the entire experimental benchmark."},{"cited_title":"In: 2015 IEEE International Conference on Computer Vision (ICCV)","cited_arxiv_id":null,"evidence_quote":"Provides the discussion of dying ReLU and learnable negative slopes that motivates the negative-branch behavior of LeakySineLU."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as the sine-as-activation baseline and as motivation for using periodic functions in deep networks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the time-series classification from scratch architectures that the paper adapts into its MLP and FCN models."},{"cited_title":"In: 1999 Ninth International Conference on Artificial Neural Networks ICANN 99","cited_arxiv_id":null,"evidence_quote":"Supplies the comparative study of periodic versus monotonic activation functions that motivates the semi-periodic design."}],"review_version":1}