Pith. sign in

REVIEW 4 major objections 5 minor 24 references

SG-Blend: Learning an Interpolation Between Improved Swish and GELU for Robust Neural Representations

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes SG-Blend, a learnable blend of a symmetrized Swish and GELU, and claims it beats ReLU, Swish, GELU, and Mish on every benchmark tested.

desk verdict A simple learnable blend of shifted Swish and GELU shows consistent small gains, but single-run results and a 100x BLEU typo undermine the 'consistently outperforms' claim. read the letter →

arxiv 2505.23942 v1 pith:VNUBI7J4 submitted 2025-05-29 cs.LG cs.AI

Gaurav Sarkar , Jay Gala , Subarna Tripathi This is my paper · ORCID
classification cs.LGcs.AI
keywords SG-BlendSSwishSwishGELUlearnableactivationfunctionimageclassificationnaturallanguageprocessing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SG-Blend, a smooth activation function that learns to interpolate between a symmetrized version of Swish (which the paper calls SSwish) and GELU using per-layer parameters ($\alpha,\beta,\gamma$). Its central claim is that this adaptive blend consistently outperforms ReLU, Swish, GELU, and Mish on the reported benchmarks: CIFAR-10 with ResNet18 and ResNet50, IMDB sentiment with BERT, and a WMT14 English-German translation subset, with the largest reported gain in the translation metric. If the claim holds, SG-Blend offers a drop-in replacement that could let practitioners avoid choosing activations by task or architecture, at negligible computational cost. The paper also contributes SSwish itself, which in its additional experiments matches or slightly beats Swish.

What carries the argument

The load-bearing object is the parameterized convex combination in Eq. (5), whose two components are the new SSwish and the established GELU. SSwish, defined as $x\sigma(\beta x)-\gamma$, augments Swish with a learnable sigmoid steepness $\beta$ and a learnable vertical shift $\gamma$; the shift allows the network to center its activation statistics, which the paper hypothesizes improves gradient symmetry. The blend weight $\alpha$ is also learned, making the activation shape adaptive per layer. The smoothness of both components and the derivative identity in Eq. (6) — the SG-Blend gradient is the same convex combination of the component gradients — are what let the network tune its nonlinearity while preserving differentiable, stable gradient flow.

What would settle it

A multi-seed replication would settle the claim: run each reported configuration (ResNet18 and ResNet50 on CIFAR-10, BERT on IMDB, and the 50k-pair transformer on WMT14) at least five times with identical hyperparameters, and check whether the SG-Blend minus baseline margins consistently stay away from zero. The paper itself names the untested generative-model regime, so a second check would be the same comparison on a generation task.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that SG-Blend — the learnable convex combination $\mathrm{SG\text{-}Blend}_{\alpha,\beta,\gamma}(x) = \alpha\cdot\mathrm{SSwish}_{\beta,\gamma}(x) + (1-\alpha)\cdot\mathrm{GELU}(x)$ with $\mathrm{SSwish}_{\beta,\gamma}(x) = x\sigma(\beta x)-\gamma$ — is a more universally robust activation than its fixed-shape predecessors. The parameters $\alpha\in[0,1]$, $\beta\in[0.1,10]$, and $\gamma$ are learned per layer by backpropagation, so each layer can specialize toward SSwish's controlled non-monotonicity or GELU's smooth probabilistic profile. Because both components are smooth, SG-Blend is $C^\infty$ and its derivative is a smooth weighted sum, which the paper ties to stable gradient flow. Empirically, the paper reports best-in-table results in every configuration it tests, including 93.23% top-1 accuracy on CIFAR-10 with ResNet18 and 89.56% validation accuracy on IMDB with BERT.

Load-bearing premise

The load-bearing premise is that the reported accuracy and BLEU differences between SG-Blend and the baselines are genuine generalization differences rather than run-to-run noise, because each configuration was evaluated once with early stopping and no random seeds or error bars.

Editorial extensions

If this is right

  • SG-Blend can be inserted into existing ResNet and Transformer models as a drop-in replacement for ReLU, Swish, GELU, or Mish without changing the architecture and with negligible extra compute.
  • Per-layer learning of $\alpha$ lets a single network settle on SSwish-like nonlinearity in some layers and GELU-like smoothness in others, which is what the paper identifies as the source of its adaptability.
  • The largest reported gain is in translation: on the WMT14 subset SG-Blend scores 0.5735 BLEU versus 0.5172 for GELU and 0.5001 for Swish.
  • SSwish alone already matches or exceeds Swish in the paper's additional experiments, so the symmetry modification contributes independently of the GELU blend.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: tracking the learned per-layer values of $\alpha$ after training would show whether vision layers converge toward SSwish and language layers toward GELU, turning the paper's motivating hypothesis into a directly testable prediction.
  • Editorial extension: because each configuration was evaluated once, the same protocol with multiple random seeds would show whether margins like the 0.36% CIFAR-10 ResNet18 gap persist; this is the minimal experiment that would separate the blend's effect from training noise.
  • Editorial extension: the convex-combination recipe generalizes to other activation pairs, so SG-Blend can be read as an instance of a broader learnable-activation family rather than a single function.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes SSwish (Eq. 3), a vertically shifted Swish with learnable parameters beta and gamma, and SG-Blend (Eq. 5), a convex combination of SSwish and GELU with a learnable interpolation weight alpha. It reports image classification on CIFAR-10 with ResNet18/50, sentiment analysis on IMDB with a model described as BERT, and machine translation on a 50,000-pair WMT14 En-De subset with a small Transformer, claiming consistent gains over ReLU, Swish, GELU, and Mish with negligible overhead. An additional experimental section reports SSwish-only comparisons. The central claim is empirical: the adaptive interpolation is alleged to generalize better across vision and NLP tasks and architectures.

Significance. SG-Blend is a clearly specified and easy-to-implement activation function, and the anonymous code release plus multi-modality evaluation are positive features. The contribution is incremental, however: a learnable convex combination of two known activations with a shift parameter is not a large conceptual advance, and the reported gains are small. The decisive question is whether the single-run differences in Tables 3-5 represent true generalization improvement. Because the paper reports no seeds, error bars, or statistical comparisons, the significance currently rests on an unverified empirical claim. If multi-seed experiments confirm stable gains, the paper would be a useful practical addition to the activation-function literature.

major comments (4)
  1. [Section 3.1, Tables 3-5] Every configuration is evaluated once, with early stopping and no seeds, error bars, or significance tests (Section 3.1). The margins are small: +0.36 percentage points vs Swish on ResNet18 CIFAR-10, +0.68 vs ReLU on ResNet50, +0.08 vs Mish on IMDB, and +0.0563 BLEU on the WMT subset. These differences are within plausible run-to-run variation for a single T4 run, especially since no baseline hyperparameter search is reported and all activations share the same training schedule. This evidence cannot support the abstract's and Section 3.4's claim that SG-Blend 'consistently outperforms' strong baselines. The authors should rerun with multiple seeds per configuration, report means and standard deviations, and where appropriate apply paired or matched comparisons. The paper's own Limitations paragraph notes potential overfitting risk from the three learnable parameters, which reinforces the need for variance reporting.
  2. [Abstract, Section 1, Table 5] The abstract and Section 1 claim 'upto 5.63 in BLEU score,' but Table 5 reports SG-Blend's BLEU as 0.5735 versus GELU's 0.5172, a difference of only +0.0563. The factor-of-100 discrepancy must be resolved, and the BLEU scale (0-1 versus 0-100) must be stated explicitly. As written, this is a factual inconsistency in the headline empirical claim, and it makes the WMT result difficult to interpret.
  3. [Section 3.3.1 and Section 6.3] The main IMDB experiment is described as using 'a standard Keras implementation of a BERT model' and 'BERT base' in Sections 3.1 and 3.3.1, but Section 6.3 describes a 'lightweight 2-layer BERT-style transformer' with model dimension 64 and a training time of about 55 seconds. If these are the same model, the text substantially overstates the scale of the language model; if they are different models, the paper must state which one produced Table 4. The claim that the results demonstrate effectiveness in 'large-scale language models' is not supported without this clarification.
  4. [Section 2.2, Eq. (3)] SSwish is repeatedly called a 'symmetric' or 'first-order symmetric' variant of Swish, but Eq. (3) is simply a vertical shift: SSwish(x) = Swish(x) - gamma. A vertical shift does not make the function odd or symmetric about the origin, and the first derivative in Eq. (4) is neither even nor odd. The statement that 'by incorporating learnable slope and bias parameters, SSwish is designed to enforce symmetry in the activation's response' is therefore incorrect as stated. If the intended property is only that the learnable offset can center the mean activation, the paper should say so and drop the symmetry terminology; otherwise, the authors should provide a definition that actually enforces symmetry.
minor comments (5)
  1. [Section 2.3] The text says GELU is 'unbounded below but approaches 0 slowly as x→−∞'; this is internally contradictory because the standard GELU is bounded below, with an infimum of about -0.17 near x = -1.
  2. [Section 3.3.2] The WMT14 evaluation uses a 50,000-pair subset with a 10% validation split from the same subset; the paper should explicitly state that these BLEU scores are not comparable to standard full-dataset WMT benchmarks and should report the exact evaluation script, since BLEU implementations can differ by an order of magnitude.
  3. [Section 5] The claim of 'negligible computational overhead' for SG-Blend is not directly measured; training time is reported only for SSwish versus Swish in Tables 6-7, not for SG-Blend versus the baselines in Tables 3-5.
  4. [Tables 6-7] The additional experiments in Section 6 use different setups (a 2-layer Transformer and a custom CNN) from Section 3, and the relationship between these SSwish-only results and the main SG-Blend experiments is not clearly stated; consider labeling them explicitly as preliminary ablations.
  5. [Section 3.3.1] The statement that ReLU, Swish, GELU, Mish, and SG-Blend all produce 0.00% dead neurons in the BERT model is surprising and needs a brief explanation of how dead neurons were counted, especially for ReLU.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: SG-Blend is a proposed interpolation family evaluated on external benchmarks, with no load-bearing self-citation or fitted-prediction loop.

full rationale

The paper makes no claim of deriving SG-Blend from first principles; Eq. (5) simply defines SG-Blend as alpha*SSwish(x) + (1-alpha)*GELU(x), and Eq. (3) defines SSwish as x*sigmoid(beta*x) - gamma. The learnable parameters alpha, beta, and gamma are optimized by backpropagation, which is standard for learnable activations and does not constitute fitting a parameter to a dataset and then predicting the same dataset. The empirical claims are evaluated against external benchmarks (CIFAR-10, IMDB, WMT14 subset) and against standard baselines (ReLU, Swish, GELU, Mish), so the performance comparisons are not equivalent to the definition of the activation by construction. The Section 6 SSwish experiments are additional external evaluations used as motivation, not as a self-citation or as an imported uniqueness theorem. There is no load-bearing self-citation: all cited prior work is external to the authors. The main caveats are statistical reliability concerns (single runs, no error bars or seeds, and an abstract/table discrepancy of 5.63 vs. 0.0563 BLEU) and the authors' own stated limitations about generative tasks and potential overfitting; these affect confidence in the empirical claim but are not circularity. Therefore, the derivation chain is self-contained and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim relies on three learned parameters (alpha, beta, gamma) and on the implicit assumption that single-run benchmark comparisons are meaningful. The 'symmetry' motivation for gamma is not supported by the math, since a constant shift does not make SSwish even.

free parameters (3)
  • alpha
    Learnable blend weight in Eq. (5), initialized at 0.5 per layer. The final learned values are not reported, so it is unknown whether the network actually blends or collapses to one component.
  • beta
    Learnable scaling of the sigmoid in SSwish (Eq. 3), initialized at 1.0 and constrained to [0.1, 10]. Controls sharpness.
  • gamma
    Learnable vertical shift in SSwish (Eq. 3), initialized at 0. The paper claims this enforces symmetry, but a constant shift does not create even symmetry.
assumptions (3)
  • ad hoc to paper SSwish's vertical shift gamma improves symmetry and gradient flow.
    The paper asserts in Section 2.2 that gamma 'promotes symmetry in activation statistics' and leads to stable training, but provides no derivation or measurement. A vertical shift does not make the function symmetric about the origin.
  • domain assumption Single-run results with early stopping are representative of generalization.
    Section 3.1 describes one run per configuration, and Tables 3 to 5 report no variance. The claim of consistent improvement relies on these runs not being affected by random seed or checkpoint selection.
  • domain assumption Baselines were tuned comparably to SG-Blend.
    The paper uses standard hyperparameters for all models but does not state that the baselines received the same hyperparameter tuning or early-stopping criteria as SG-Blend, so the reported gains could reflect under-tuned baselines.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SG-Blend: Learning an Interpolation Between Improved Swish and GELU for Robust Neural Representations." pith.science (2026). https://pith.science/paper/VNUBI7J4

@misc{pith2026250523942,
  author       = {Pith},
  title        = {Pith review of: SG-Blend: Learning an Interpolation Between Improved Swish and GELU for Robust Neural Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VNUBI7J4}},
  note         = {Machine review of arXiv:2505.23942}
}
read the original abstract

The design of activation functions remains a pivotal component in optimizing deep neural networks. While prevailing choices like Swish and GELU demonstrate considerable efficacy, they often exhibit domain-specific optima. This work introduces SG-Blend, a novel activation function that blends our proposed SSwish, a first-order symmetric variant of Swish and the established GELU through dynamic interpolation. By adaptively blending these constituent functions via learnable parameters, SG-Blend aims to harness their complementary strengths: SSwish's controlled non-monotonicity and symmetry, and GELU's smooth, probabilistic profile, to achieve a more universally robust balance between model expressivity and gradient stability. We conduct comprehensive empirical evaluations across diverse modalities and architectures, showing performance improvements across all considered natural language and computer vision tasks and models. These results, achieved with negligible computational overhead, underscore SG-Blend's potential as a versatile, drop-in replacement that consistently outperforms strong contemporary baselines. The code is available at https://anonymous.4open.science/r/SGBlend-6CBC.

Figures

Figures reproduced from arXiv: 2505.23942 by the authors.

Figure 1
Figure 1. First-order derivative of SG-Blend, Swish and GELU. interpretation and smooth gradient profile, has become the preferred choice for Transformer (Vaswani et al. [2017]) architectures. Nevertheless, its performance might be suboptimal in certain vision tasks where a more pronounced nonlinearity could be beneficial. This task-specific efficacy underscores a fundamental problem: the lack of a single activation function … view at source ↗
Figure 2
Figure 2. The Symmetric Swish (SSwish) activation function for various [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. First derivative of Symmetric Swish (SSwish) for various [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The SG-Blend activation function shape for fixed β, γ and varying α. α = 1 recovers SSwish, α = 0 recovers GELU, and intermediate values provide a smooth blend. Learning Dynamics and Adaptability: The key advantage of SG-Blend is its adaptability. By learning α, the ne…
Figure 5
Figure 5. Figure 5: The SG-Blend activation function shape for fixed α and varying β and γ. Properties of SG-Blend: • Smoothness: As a convex combination of two smooth functions (SSwish and GELU), SG-Blend is also infinitely differentiable (C∞). • Boundedness: Like SSwish and GELU, SG-Ble…
Figure 6
Figure 6. Figure 6: First derivative of SG-Blend activation function shape for varying α, β and γ. 2.4 Implementation Details We implement SSwish and SG-Blend as custom activation layers compatible with standard deep learning frameworks such as PyTorch (Paszke et al. [2019]) and TensorFlo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 12 canonical work pages

  1. [1]

    The generalized sigmoid activation function: Competitive supervised learning

    Sridhar Narayan. The generalized sigmoid activation function: Competitive supervised learning. Information Sciences, 99 0 (1): 0 69--82, 1997. ISSN 0020-0255. doi:https://doi.org/10.1016/S0020-0255(96)00200-9. URL https://www.sciencedirect.com/science/article/pii/S0020025596002009

    doi:10.1016/s0020-0255(96)00200-9 1997

  2. [2]

    LeCun, L \'e on Bottou, Genevieve B

    Yann A. LeCun, L \'e on Bottou, Genevieve B. Orr, and Klaus-Robert M \"u ller. Efficient BackProp, pages 9--48. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-642-35289-8. doi:10.1007/978-3-642-35289-8_3. URL https://doi.org/10.1007/978-3-642-35289-8_3

    doi:10.1007/978-3-642-35289-8_3 2012

  3. [3]

    Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted Boltzmann machines . International Conference on Machine Learning, pages 807--814, 6 2010. URL https://icml.cc/Conferences/2010/papers/432.pdf

    work page 2010

  4. [4]

    Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for activation functions . International Conference on Learning Representations, 2 2018. URL https://openreview.net/pdf?id=SkBYYyZRZ

    work page 2018

  5. [5]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv: Learning, 2016

    work page 2016

  6. [6]

    E fficient N et: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc Le. E fficient N et: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105--6114. PMLR, 09--15 Jun 2019. URL https://proceedings.mlr.pre...

    work page 2019

  7. [7]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008, 2017. URL http://arxiv.org/abs/1706.03762

    arXiv 2017

  8. [8]

    Learning Multiple Layers of Features from Tiny Images , 1 2009

    Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images , 1 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

    work page 2009

Show all 24 references
  1. [9]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142--150...

    2011

  2. [10]

    Findings of the 2014 workshop on statistical machine translation

    Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Ale s Tamchyna. Findings of the 2014 workshop on statistical machine translation. In Pr...

    2014

  3. [11]

    Deep Residual Learning for Image Recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition . In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR '16, pages 770--778. IEEE, June 2016. doi:10.1109/CVPR.2016.90. URL http://ieeexplore.i...

    2016

  4. [12]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter ...

    2019

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  6. [14]

    A Robustly Optimized BERT Pre-training Approach with Post-training

    Zhuang Liu, Wayne Lin, Ya Shi, and Jun Zhao. A Robustly Optimized BERT Pre-training Approach with Post-training . 1 2021. 10.1007/978-3-030-84186-7\ _ 31 . URL https://doi.org/10.1007/978-3-030-84186-7_31

    2021 doi

  7. [15]

    Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, ...

    2019 arXiv

  8. [16]

    Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng

    Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Wa...

    2016

  9. [17]

    Mish: A self regularized non-monotonic neural activation function

    Diganta Misra. Mish: A self regularized non-monotonic neural activation function. arXiv preprint arXiv:1908.08681, 4: 0 2, 2019

    1908 arXiv

  10. [18]

    An overview of gradient descent optimization algorithms

    Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747, 2016

    2016 arXiv

  11. [19]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Dekang Lin, Yuji Matsumoto, and Rada Mihalcea, editors, Proceedings of the 49th Annual Meeting of the Association for Computational ...

    2011

  12. [20]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014. URL https://api.semanticscholar.org/CorpusID:6628106

    2014 arXiv

  13. [21]

    Andrew L. Maas. Rectifier nonlinearities improve neural network acoustic models. 2013. URL https://api.semanticscholar.org/CorpusID:16489696

    2013

  14. [22]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV '15, page 1026–1034, USA, 2015. IEEE Com...

    2015 doi

  15. [23]

    Activation functions in deep learning: A comprehensive survey and benchmark

    Shiv Ram Dubey, Satish Kumar Singh, and Bidyut Baran Chaudhuri. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503: 0 92--108, 2022. ISSN 0925-2312. doi:https://doi.org/10.1016/j.neucom.2022.06.111. URL https://www.sciencedirect.co...

    2022 doi

  16. [24]

    Adaptive parametric activation

    Konstantinos Panagiotis Alexandridis, Jiankang Deng, Anh Nguyen, and Shan Luo. Adaptive parametric activation. In European Conference on Computer Vision, pages 455--476. Springer, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.