Pith. sign in

REVIEW 4 cited by

A Survey on Activation Functions and their relation with Xavier and He Normal Initialization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.06632 v1 pith:WNXHY2RD submitted 2020-03-18 cs.NE cs.LG

Leonid Datta This is my paper · ORCID
classification cs.NEcs.LG
keywords activationinitializationfunctionfunctionsimportantnormalsurveyweight
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In artificial neural network, the activation function and the weight initialization method play important roles in training and performance of a neural network. The question arises is what properties of a function are important/necessary for being a well-performing activation function. Also, the most widely used weight initialization methods - Xavier and He normal initialization have fundamental connection with activation function. This survey discusses the important/necessary properties of activation function and the most widely used activation functions (sigmoid, tanh, ReLU, LReLU and PReLU). This survey also explores the relationship between these activation functions and the two weight initialization methods - Xavier and He normal initialization.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Extreme Value Theory-based Distributed Interference Prediction for 6G Industrial Sub-networks

    eess.SP 2025-07 reject novelty 6.0 of 10

    A hybrid transformer, extreme value theory, and conformal prediction framework claims calibrated interference tail forecasts with coverage guarantees for 6G sub-networks.

  2. Exploring the Feasibility of Deep Learning Techniques for Accurate Gender Classification from Eye Images

    cs.CV 2025-07 conditional novelty 4.0 of 10

    A custom CNN trained on periocular eye images reaches 99% reported gender classification accuracy on CVBL and 96% on a Kaggle eye dataset.

  3. Motion Control in Multi-Rotor Aerial Robots Using Deep Reinforcement Learning

    cs.RO 2025-02 conditional novelty 4.0 of 10

    In a Simulink drone simulator, TD3 with curriculum learning and acceleration feedback achieved lower positional error and higher success than DDPG and than TD3 without curriculum.

  4. A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture

    cs.NE 2025-06 reject novelty 2.0 of 10

    A hybrid PSO-SGD optimizer is tested on small feedforward networks, with the paper claiming large MSE reductions that the presented tables only partially support.

Pith tools