Pith. sign in

REVIEW 5 major objections 6 minor 43 references

A Training Framework for Optimal and Stable Training of Polynomial Neural Networks

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Polynomial neural networks can be trained stably at degrees up to 22, and reach near-ReLU accuracy on seven datasets, through a boundary loss plus selective gradient clipping.

desk verdict A plausible, well-scoped empirical framework for stable PNN training, but the degree-22 headline overreaches and the evaluation needs error bars and code before the strong claims can be taken at face value. read the letter →

arxiv 2505.11589 v1 pith:S23WYCUH submitted 2025-05-16 cs.LG

Forsad Al Hossain , Tauhidur Rahman This is my paper · ORCID
classification cs.LG
keywords polynomialneuralnetworkshomomorphicencryptionboundarylossselectivegradientclippingtrainingstabilityactivationfunctionsbatchnormalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Polynomial neural networks replace ReLU with polynomial activations, which makes them compatible with homomorphic encryption but hard to train once the degree rises. This paper claims that two techniques together remove that difficulty: a Boundary Loss that exponentially penalizes any activation input leaving a preset safe range, and Selective Gradient Clipping that clips large gradients everywhere except BatchNorm parameters. With both in place, the paper reports stable training for polynomial degrees 2, 4, and 8 across seven image, audio, and activity-recognition datasets, with accuracy close to ReLU baselines, and stable degree-22 training on Speech Commands. If true, this means privacy-preserving inference can use genuine polynomial activations instead of approximating ReLU under encryption.

What carries the argument

The central object is the composite training loss, $L_{\text{total}} = L_{\text{CE}} + \lambda \sum_i L_i$, where $L_i$ is the Boundary Loss on layer $i$'s activation inputs: $L_i = \frac{1}{|X_i|} \sum_{j=1}^{|X_i|} (e^{\max(|X_{i,j}| - \alpha B, 0)} - 1)$, with $B$ the fitting interval, $\alpha \in (0,1]$ a slack factor, and $\lambda$ the penalty weight. The companion mechanism is Selective Gradient Clipping: gradients of non-BatchNorm parameters are divided by $\max(1, \|\nabla\theta_{\text{non-BN}}\|_2 / c)$ while BatchNorm gradients pass unchanged. The argument is carried by the interaction between these two: the exponential term keeps activation inputs inside the polynomial's reliable domain, clipping absorbs the resulting large gradients, and exempting BatchNorm preserves the running statistics the network depends on.

What would settle it

On CIFAR-10 with degree-8 activations, set $B=35$, $\alpha=0.5$, and $\lambda=1000$, and record the maximum $|X_{i,j}|$ for every layer at every step of the first epoch. If that maximum grows toward or beyond $B$ despite the boundary loss, the presumed balance fails; likewise, if the full method and the no-clipping baseline attain equal accuracy, the claimed synergy between boundary loss and selective clipping is not load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that PNN training instability has two manageable causes: inputs to the polynomial activations drift outside the interval where the fitted polynomial approximates the target activation, and boundary-penalty gradients damage BatchNorm's running statistics. The paper's Boundary Loss, $L_i = \frac{1}{|X_i|} \sum_j (e^{\max(|X_{i,j}| - \alpha B, 0)} - 1)$, added to the classification loss with weight $\lambda = 1000$, pushes activation inputs back inside $[-\alpha B, \alpha B]$, while Selective Gradient Clipping rescales by norm only the non-BatchNorm parameters, leaving BatchNorm's scale and shift parameters untouched. The paper argues these mechanisms are synergistic: the loss prevents polynomial outputs from exploding, and the clipping prevents the loss's own steep gradients from destabilizing optimization. On its own terms, the discovery is that high-degree polynomial networks, not just low-degree approximations, are trainable and can approach ReLU accuracy, easing the path to encrypted inference.

Load-bearing premise

The whole scheme rests on the assumption that an exponential penalty on large activation inputs will keep those inputs bounded from the first training steps, before the highest-degree term of the polynomial can diverge; if the penalty is too weak relative to the classification loss, no amount of later clipping can recover the run.

Editorial extensions

If this is right

  • High-degree polynomial activations can be used in deep, encryption-compatible architectures without sacrificing accuracy, so encrypted inference no longer depends on low-degree approximations that limit expressivity.
  • A single trained PNN can serve both plaintext and encrypted inference, since its layers are expressible by additions, multiplications, and affine transforms that fold batch normalization into adjacent linear layers.
  • BatchNorm parameters should be excluded from any gradient-clipping scheme when training PNNs; clipping them is reported to cause training collapse even at degree 2.
  • Choosing $B$ near the lower end of the stable training range is a practical recipe that balances accuracy and stability, with the safe range growing as polynomial degree increases.
  • On the seven datasets tested, degree-8 PNNs match or nearly match ReLU baselines, indicating polynomial activations are not inherently a capacity bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The boundary-loss idea is a general cure for training any activation with unbounded output, such as Taylor or spline activations: penalize out-of-fit-range inputs during early training rather than redesigning the activation; the paper does not test this transfer.
  • Because the degree-22 result appears on only one dataset and the paper gives no convergence proof, the practical ceiling is still unknown; a natural next test is reproducing the method on deeper convolutional architectures and larger benchmarks.
  • The method's sensitivity to $B$ suggests a testable refinement: start with a wide boundary for stability and shrink it as activations settle, which could remove the manual degree-dependent tuning the paper currently recommends.
  • If BatchNorm-free architectures are used, the selective-clipping component may become unnecessary; the paper's ablations imply the benefit of clipping comes mostly from protecting BatchNorm statistics, but this simplification is not evaluated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a training framework for Polynomial Neural Networks (PNNs) aimed at homomorphic-encryption-compatible inference. The two proposed components are a Boundary Loss that exponentially penalizes activation inputs outside a predefined range, and Selective Gradient Clipping that clips gradients of non-BatchNorm parameters while leaving BatchNorm parameters unclipped. The authors evaluate the framework on seven image, audio, and human-activity datasets using ResNet-18 and MLP architectures with polynomial degrees 2 and 8, report ablations on CIFAR-10 and CIFAR-100, study the effect of the fitting boundary B and of excluding BatchNorm from clipping, and report one encrypted-inference result on a 5-layer MLP. The abstract claims stable training up to polynomial degree 22, but the only supporting evidence is a single undocumented sentence in Section 5.2.

Significance. If the results hold, the framework is a simple and potentially practical contribution to HE-friendly neural network training: it makes no exotic assumptions, uses only standard layers, and openly reports all hyperparameters (B, alpha, lambda, c), which is useful for reproducibility. The main strengths are the breadth of the seven-dataset evaluation, the explicit ablation of the two proposed components, and the honest admission in Section 7.1 that the validation is empirical. However, the significance is currently dampened by three gaps: the headline degree-22 claim is not backed by any reported experiment; Table 1 reports single accuracies without error bars; and there is no quantitative comparison against prior PNN stabilization methods. These gaps must be addressed before the central claims about 'stable training with high-degree polynomials' and 'standard methods typically fail' can be accepted as demonstrated.

major comments (5)
  1. [Abstract and Section 5.2] The claim that PNNs 'exhibit stable training and strong performance with polynomial degrees up to 22' rests on a single undocumented sentence in Section 5.2 ('we also successfully trained polynomial degrees as high as 22 for the Speech Commands dataset'), with no accuracy, architecture, B, alpha, lambda, clipping threshold, number of seeds, or baseline comparison reported, and Table 1 reports only degrees 2 and 8. The Conclusion (Section 8) itself restricts the validated claim to 'degrees up to 8 across a diverse set of benchmarks,' so the abstract and contributions overstate the evidence. This is load-bearing because the abstract uses degree 22 as a headline capability; the sentence should either be removed from the abstract and contributions or replaced with a fully documented experimental result.
  2. [Table 1 and Figures 3-5] Table 1 reports a single accuracy number per condition with no standard deviation, seed count, or number of runs, which makes it impossible to assess whether the reported parity with ReLU baselines is significant or stable; the same issue affects Figures 3-5, and the only statistical statement in Section 5.1 ('p < 0.01, two-sided Z-test') does not describe the number of trials or what is counted as a success. The paper should report mean and variance over at least several seeds for the central results, and for the ablation comparisons it should state the success-rate definition and trial counts.
  3. [Section 2.2 and Section 5] No quantitative comparison is made to existing PNN stabilization methods cited in Section 2.2 (e.g., Goyal et al., Aboulatta et al., Gottemukkula, and AESPA), so the claim that 'standard methods typically fail' is not supported by any baseline experiment. At minimum, the paper should train one or two prior stabilization baselines under the same architectures and report their accuracy and divergence rates, or alternatively soften the claim to say that the proposed method succeeds without such comparisons.
  4. [Equations (1)-(2) and Section 7.1] The framework's stability mechanism depends on the exponential boundary penalty being strong enough to dominate the classification loss for inputs just above alpha*B, but for high degree d the post-activation magnitude can grow like (alpha*B)^d, and no analysis of this balance is provided; Section 5.2 itself concedes that setting B too low can lead to an exploding boundary loss. The paper's own limitation note (Section 7.1) states the validation is empirical; given that the central claim is stability, the authors should either provide a formal stability argument or at least report failure rates across a grid of B values and initialization seeds to quantify the claimed stability.
  5. [Section 6] The HE evaluation is limited to one 5-layer MLP on UCI-HAR with an inference time of 177.33 seconds per sample; no HE evaluation of the ResNet-18 architectures used for the image and audio tasks is reported, so the abstract's statement that the framework 'confirm[s] the HE-compatibility of the trained models' is demonstrated only for a simple case. The authors should either scope the HE claim to the evaluated 5-layer MLP or add an HE evaluation for at least one convolutional model.
minor comments (6)
  1. [Table 1] The PNN Accuracy column appears to contain a formatting problem: for MNIST the row lists '0.994' and '0.9944' in the same cell, which is confusing; the table layout should be clarified.
  2. [Section 5.2] There is a typo: 'differnt' should be 'different'. Also, Figure 4's caption says 'Each dot represents a training run' but the number of runs per configuration is not stated.
  3. [Section 5.1] The Z-test sentence should specify the number of runs, the definition of a successful run, and whether the test compares convergence success or final accuracy, so that the reader can interpret the p-value.
  4. [Section 3.1.1] The number m of sampling points used for polynomial fitting is not reported; this matters because the fit quality depends on m and the sampling range.
  5. [Figure 2 caption] The caption states that the activation function was fitted for the range [-35,35], but the text says B varies by degree and dataset; please clarify that this is one example configuration.
  6. [References] The reference formatting is inconsistent, for example the first reference cites 'HIP' rather than the full act name, and several entries use 'et al.' in nonstandard ways; this should be normalized to the journal style.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; all reported accuracy results are empirical and no prediction reduces to a fitted input.

full rationale

The paper's central claims are empirical training results, not derivations from a self-referential mechanism. The proposed Boundary Loss in Eq. (1) and Selective Gradient Clipping in Eq. (3) are training regularizers applied before evaluation, and the accuracies reported in Table 1 are measured on held-out test data; no reported quantity is algebraically equal to a fitted parameter of the loss. The polynomial coefficients are least-squares fits to ReLU over [-B,B], but that approximation does not by itself determine the reported test accuracy, and the paper's contribution is precisely the empirical claim that the proposed training techniques make such PNNs trainable. Hyperparameters such as B, alpha, lambda, and c are openly reported and tuned, not disguised as derived constants or as predictions. There are no load-bearing self-citations or imported uniqueness theorems; citations to prior work are to external authors and do not carry the argument. The abstract's degree-22 claim rests on one undocumented sentence in Section 5.2, and the conclusion later limits the claim to degree 8; however, that is an evidentiary/overclaim issue rather than a circularity issue. The Limitations section explicitly states that the development and validation are primarily empirical, which is consistent with the absence of a derivation chain that could reduce to its own inputs. Therefore no circular step can be exhibited, and the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on hand-tuned hyperparameters (B, alpha, lambda, c) and on untested behavioral assumptions about the interaction among the boundary penalty, polynomial coefficients, and BatchNorm. No new physical or mathematical entities are introduced.

free parameters (4)
  • Polynomial fitting boundary B = 13-14 for degree 2, 20-30 for degree 4, 35 for degree 8
    Defines the interval [-B, B] for least-squares fitting and the boundary loss threshold. It is chosen per degree and dataset based on stability and test accuracy in Section 5.2.
  • Boundary threshold multiplier alpha = 1.0 for degree 2, 0.75 for degree 4, 0.5 for degrees 8 and above
    Scales B in the boundary loss L_i. It is selected per degree without a principled rule, as described in Section 3.2.2.
  • Boundary loss penalty lambda = 1000
    Weight combining the cross-entropy loss and all boundary losses. It is fixed across all settings, with robustness reported over 1 to 1000 in Section 5.2.
  • Gradient clipping threshold c = 0.5 to 1.0
    Norm threshold for selective gradient clipping. The paper reports robustness over this range but does not state the exact value used for each experiment.
assumptions (4)
  • domain assumption A least-squares polynomial fit to ReLU over [-B, B] remains a good approximation once activation inputs are constrained to that range.
    Section 3.1.1 and Eq. (1) assume that constraining pre-activations preserves accuracy; no error bound relating approximation error to classification loss is provided.
  • domain assumption The exponential boundary loss can dominate the classification loss near the boundary and steer training before divergence occurs.
    The composite loss in Eq. (2) is a weighted sum, but no proof shows the intended dynamics. Section 5.2 notes that low B causes exploding boundary loss, showing the assumption can fail.
  • domain assumption Leaving BatchNorm parameters unclipped preserves the statistics needed for training stability.
    Section 5.2.1 reports that clipping BN parameters causes drift and collapse, but the paper does not analyze why or under what conditions this holds.
  • domain assumption CKKS homomorphic inference preserves the trained model's accuracy sufficiently for deployment.
    Section 6 reports one 5-layer MLP encrypted inference at 94.7% accuracy, but does not bound approximation error from rescaling, scaling factors, or multiplicative depth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Training Framework for Optimal and Stable Training of Polynomial Neural Networks." pith.science (2026). https://pith.science/paper/S23WYCUH

@misc{pith2026250511589,
  author       = {Pith},
  title        = {Pith review of: A Training Framework for Optimal and Stable Training of Polynomial Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S23WYCUH}},
  note         = {Machine review of arXiv:2505.11589}
}
read the original abstract

By replacing standard non-linearities with polynomial activations, Polynomial Neural Networks (PNNs) are pivotal for applications such as privacy-preserving inference via Homomorphic Encryption (HE). However, training PNNs effectively presents a significant challenge: low-degree polynomials can limit model expressivity, while higher-degree polynomials, crucial for capturing complex functions, often suffer from numerical instability and gradient explosion. We introduce a robust and versatile training framework featuring two synergistic innovations: 1) a novel Boundary Loss that exponentially penalizes activation inputs outside a predefined stable range, and 2) Selective Gradient Clipping that effectively tames gradient magnitudes while preserving essential Batch Normalization statistics. We demonstrate our framework's broad efficacy by training PNNs within deep architectures composed of HE-compatible layers (e.g., linear layers, average pooling, batch normalization, as used in ResNet variants) across diverse image, audio, and human activity recognition datasets. These models consistently achieve high accuracy with low-degree polynomial activations (such as degree 2) and, critically, exhibit stable training and strong performance with polynomial degrees up to 22, where standard methods typically fail or suffer severe degradation. Furthermore, the performance of these PNNs achieves a remarkable parity, closely approaching that of their original ReLU-based counterparts. Extensive ablation studies validate the contributions of our techniques and guide hyperparameter selection. We confirm the HE-compatibility of the trained models, advancing the practical deployment of accurate, stable, and secure deep learning inference.

Figures

Figures reproduced from arXiv: 2505.11589 by the authors.

Figure 1
Figure 1. Example of a neural network layer structure for our use cases. It only consists of layers [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example of fitted polynomial activation functions for polynomial degrees of 2, 4, and 8. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Evaluation of training stability and accuracy via ablation study on CIFAR-10 (top) and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Impact of polynomial fitting boundary B on test accuracy for CIFAR-10 (left) and UCI￾HAR (right) datasets for polynomial degrees 2, 4, 8, and 12. Each dot represents a training run with varied parameters (e.g., dropout, gradient clipping, λ, fit samples). Small B leads…
Figure 5
Figure 5. Figure 5: Effect of including (purple bars) versus excluding (red bars; our proposed Selective Gradient [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 26 canonical work pages

  1. [1]

    https://www.hhs.gov/hipaa/, 1996

    Centers for medicare & medicaid services, the health insurance portability and accountability act of 1996 (hipaa). https://www.hhs.gov/hipaa/, 1996. Accessed: 2024-05-05

    work page 1996

  2. [2]

    http://data.europa.eu/eli/reg/2016/679/oj, 2016

    Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec (general data protection regulation). http://data.europa.eu/eli/reg/2016/679/oj, 2016

    work page 2016

  3. [3]

    Stabilizing Inputs to Approximated Nonlinear Functions for Inference with Homomorphic Encryption in Deep Neural Networks

    Moustafa Aboulatta, Matthias Ossadnik, and Seyed-Ahmad Ahmadi. Stabilizing inputs to approximated nonlinear functions for inference with homomorphic encryption in deep neural networks. ArXiv, abs/1902.01870, 2019. URL https://api.semanticscholar.org/CorpusID:59606154

    work page Pith review arXiv 1902

  4. [4]

    Ali, Jinhyun So, and Amir Salman Avestimehr

    Ramy E. Ali, Jinhyun So, and Amir Salman Avestimehr. On polynomial approximations for privacy-preserving and verifiable relu networks. ArXiv, abs/2011.05530, 2020. URL https://api.semanticscholar.org/CorpusID:226299960

    arXiv 2011

  5. [5]

    OpenFHE : Open-source fully homomorphic encryption library

    Ahmad Al Badawi, Andreea Alexandru, Jack Bates, Flavio Bergamaschi, David Bruce Cousins, Saroja Erabelli, Nicholas Genise, Shai Halevi, Hamish Hunt, Andrey Kim, Yongwoo Lee, Zeyu Liu, Daniele Micciancio, Carlo Pascoe, Yuriy Polyakov, Ian Quah, Saraswathy R.V., Kurt Rohloff, Jonathan Saylor, Dmitriy Suponitsky, Matthew Triplett, Vinod Vaikuntanathan, and V...

    work page 2022

  6. [6]

    A methodology for training homomorphic encryption friendly neural networks

    Moran Baruch, Nir Drucker, Lev Greenberg, and Guy Moshkowich. A methodology for training homomorphic encryption friendly neural networks. In ACNS Workshops, 2021. URL https://api.semanticscholar.org/CorpusID:252545815

    work page 2021

  7. [7]

    Sensitive tuning of large scale cnns for e2e secure prediction using homomorphic encryption

    Moran Baruch, Nir Drucker, Gilad Ezov, Eyal Kushnir, Jenny Lerner, Omri Soceanu, and Itamar Zimerman. Sensitive tuning of large scale cnns for e2e secure prediction using homomorphic encryption. ArXiv, abs/2304.14836, 2023. URL https://api.semanticscholar.org/CorpusID:258417852

    arXiv 2023

  8. [8]

    nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data

    Fabian Boemer, Anamaria Costache, Rosario Cammarota, and Casimir Wierzynski. ngraph-he2: A high-throughput framework for neural network inference on encrypted data. 0 (2019/947), 2019. URL https://eprint.iacr.org/2019/947. Publication info: Published elsewhere. https://arxiv.org/pdf/1908.04172.pdf

    work page Pith review arXiv 2019

Show all 43 references
  1. [9]

    Low latency privacy preserving inference

    Alon Brutzkus, Oren Elisha, and Ran Gilad-Bachrach. Low latency privacy preserving inference. In International Conference on Machine Learning, 2018. URL https://api.semanticscholar.org/CorpusID:57189196

    2018

  2. [10]

    Capture-24: A large dataset of wrist-worn activity tracker data collected in the wild for human activity recognition

    Shing Chan, Hang Yuan, Catherine Tong, Aidan Acquah, Abram Schonfeldt, Jonathan Gershuny, and Aiden Doherty. Capture-24: A large dataset of wrist-worn activity tracker data collected in the wild for human activity recognition. Scientific Data, 11, 2024. URL https://api.semanti...

    2024

  3. [11]

    Homomorphic encryption for arithmetic of approximate numbers

    Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Application of Cryptology and Information Security, 2017. URL https://api.semanticscholar.org/CorpusID:3164123

    2017

  4. [12]

    Chrysos, Stylianos Moschoglou, Giorgos Bouritsas, Yannis Panagakis, Jiankang Deng, and Stefanos Zafeiriou

    Grigorios G. Chrysos, Stylianos Moschoglou, Giorgos Bouritsas, Yannis Panagakis, Jiankang Deng, and Stefanos Zafeiriou. P–nets: Deep polynomial neural networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7323--7333, 2020. URL https://api....

    2020

  5. [13]

    The mnist database of handwritten digit images for machine learning research

    Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29 0 (6): 0 141--142, 2012

    2012

  6. [14]

    Cryptonets: applying neural networks to encrypted data with high throughput and accuracy

    Nathan Dowlin, Ran Gilad-Bachrach, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. Cryptonets: applying neural networks to encrypted data with high throughput and accuracy. In Proceedings of the 33rd International Conference on International Conference on Machin...

  7. [15]

    Scalable interpretability via polynomials

    Abhimanyu Dubey, Filip Radenovic, and Dhruv Mahajan. Scalable interpretability via polynomials. CoRR, abs/2205.14108, 2022. doi:10.48550/ARXIV.2205.14108. URL https://doi.org/10.48550/arXiv.2205.14108

    2022 doi arXiv

  8. [16]

    A new remez-type algorithm for best polynomial approximation

    Nadaniela Egidi, Lorella Fatone, and Luciano Misici. A new remez-type algorithm for best polynomial approximation. In Yaroslav D. Sergeyev and Dmitri E. Kvasov, editors, Numerical Computations: Theory and Algorithms, pages 56--69, Cham, 2020. Springer International Publishing....

    2020

  9. [17]

    Interpretable polynomial neural ordinary differential equations

    Colby Fronk and Linda Petzold. Interpretable polynomial neural ordinary differential equations. Chaos, 33 4, 2022. URL https://api.semanticscholar.org/CorpusID:251467938

    2022

  10. [18]

    Polynomial activation functions

    Vikas Gottemukkula. Polynomial activation functions. ICLR 2020 Conference Withdrawn Submission, 2020. URL https://openreview.net/forum?id=rkxsgkHKvH

    2020

  11. [19]

    Improved polynomial neural networks with normalised activations

    Mohit Goyal, Rajan Goyal, and Brejesh Lall. Improved polynomial neural networks with normalised activations. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1--8, 2020. doi:10.1109/IJCNN48605.2020.9207535

    2020

  12. [20]

    Zhang, Shaoqing Ren, and Jian Sun

    Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. 2015 IEEE International Conference on Computer Vision (ICCV), pages 1026--1034, 2015. URL https://api.semanticscholar.org/CorpusID:13740328

    2015

  13. [21]

    Cryptodl: Deep neural networks over encrypted data

    Ehsan Hesamifard, Hassan Takabi, and Mehdi Ghasemi. Cryptodl: Deep neural networks over encrypted data. ArXiv, abs/1711.05189, 2017. URL https://api.semanticscholar.org/CorpusID:9181081

    2017 arXiv

  14. [22]

    Stinchcombe, and Halbert L

    Kurt Hornik, Maxwell B. Stinchcombe, and Halbert L. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2: 0 359--366, 1989. URL https://api.semanticscholar.org/CorpusID:2757547

    1989

  15. [23]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ArXiv, abs/1502.03167, 2015. URL https://api.semanticscholar.org/CorpusID:5808102

    2015 arXiv

  16. [24]

    Highly accurate cnn inference using approximate activation functions over homomorphic encryption

    Takumi Ishiyama, Takuya Suzuki, and Hayato Yamana. Highly accurate cnn inference using approximate activation functions over homomorphic encryption. 2020 IEEE International Conference on Big Data (Big Data), pages 3989--3995, 2020. URL https://api.semanticscholar.org/CorpusID:...

    2020

  17. [25]

    GAZELLE : A low latency framework for secure neural network inference

    Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. GAZELLE : A low latency framework for secure neural network inference. page 1651–1669, 2018. ISBN 978-1-939133-04-5. URL https://www.usenix.org/conference/usenixsecurity18/presentation/juvekar

    2018

  18. [26]

    Universal Approximation with Deep Narrow Networks

    Patrick Kidger and Terry Lyons. Universal Approximation with Deep Narrow Networks . In Jacob Abernethy and Shivani Agarwal, editors, Proceedings of Thirty Third Conference on Learning Theory, volume 125 of Proceedings of Machine Learning Research, pages 2306--2327. PMLR, 09--1...

    2020

  19. [27]

    On the expressive power of deep polynomial neural networks

    Joe Kileel, Matthew Trager, and Joan Bruna. On the expressive power of deep polynomial neural networks. In Neural Information Processing Systems, 2019. URL https://api.semanticscholar.org/CorpusID:168169693

    2019

  20. [28]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. URL https://api.semanticscholar.org/CorpusID:18268744

    2009

  21. [29]

    Cifar-100 (canadian institute for advanced research)

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-100 (canadian institute for advanced research). URL http://www.cs.toronto.edu/ kriz/cifar.html

  22. [30]

    Precise approximation of convolutional neural networks for homomorphically encrypted data

    Junghyun Lee, Eunsang Lee, Joon-Woo Lee, Yongjune Kim, Young-Sik Kim, and Jong-Seon No. Precise approximation of convolutional neural networks for homomorphically encrypted data. IEEE Access, 11: 0 62062--62076, 2021. URL https://api.semanticscholar.org/CorpusID:235166812

    2021

  23. [31]

    Optimized layerwise approximation for efficient private inference on fully homomorphic encryption, 2024

    Junghyun Lee, Eunsang Lee, Young-Sik Kim, Yongwoo Lee, Joon-Woo Lee, Yongjune Kim, and Jong-Seon No. Optimized layerwise approximation for efficient private inference on fully homomorphic encryption, 2024. URL https://arxiv.org/abs/2310.10349

    2024

  24. [32]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2017. URL https://api.semanticscholar.org/CorpusID:53592270

    2017

  25. [33]

    P. Montel. Sur les polynômes d’approximation. Bulletin de la Soci&\#233;t&\#233; math&\#233;matique de France, 2: 0 151–192, 1918. ISSN 0037-9484, 2102-622X. doi:10.24033/bsmf.989

    1918 doi

  26. [34]

    Trefethen

    Ricardo Pach \'o n and Lloyd N. Trefethen. Barycentric-remez algorithms for best polynomial approximation in the chebfun system. BIT Numerical Mathematics, 49: 0 721--741, 2009. URL https://api.semanticscholar.org/CorpusID:12404969

    2009

  27. [35]

    AESPA: accuracy preserving low-degree polynomial activation for fast private inference

    Jaiyoung Park, Michael Jaemin Kim, Wonkyung Jung, and Jung Ho Ahn. AESPA: accuracy preserving low-degree polynomial activation for fast private inference. CoRR, abs/2201.06699, 2022. URL https://arxiv.org/abs/2201.06699

    2022 arXiv

  28. [36]

    Self-learning activation functions to increase accuracy of privacy-preserving convolutional neural networks with homomorphic encryption

    Bernardo Pulido-Gaytan and Andrei Tchernykh. Self-learning activation functions to increase accuracy of privacy-preserving convolutional neural networks with homomorphic encryption. PLOS ONE, 19 0 (7): 0 1--31, 07 2024. doi:10.1371/journal.pone.0306420. URL https://doi.org/10....

    2024 doi

  29. [37]

    Human Activity Recognition Using Smartphones

    Jorge Reyes-Ortiz, Davide Anguita, Alessandro Ghio, Luca Oneto, and Xavier Parra. Human Activity Recognition Using Smartphones . UCI Machine Learning Repository, 2013. DOI : https://doi.org/10.24432/C54S4K

    2013 doi

  30. [38]

    Speech commands: A dataset for limited-vocabulary speech recognition, 2018

    Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition, 2018. URL https://arxiv.org/abs/1804.03209

    2018 arXiv

  31. [39]

    Ppolynets: Achieving high prediction accuracy and efficiency with parametric polynomial activations

    Wei Wu, Jian Liu, Huimei Wang, Fengyi Tang, and Ming Xian. Ppolynets: Achieving high prediction accuracy and efficiency with parametric polynomial activations. IEEE Access, 6: 0 72814--72823, 2018. doi:10.1109/ACCESS.2018.2882407

    2018

  32. [40]

    Extrapolation of polynomial nets and their generalization guarantees, 2022

    Yongtao Wu. Extrapolation of polynomial nets and their generalization guarantees, 2022

    2022

  33. [41]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    2017

  34. [42]

    Polynomial activation neural networks: Modeling, stability analysis and coverage bp-training

    Jun Zhou, Huimin Qian, Xinbiao Lu, Zhaoxia Duan, Haoqian Huang, and Zhen Shao. Polynomial activation neural networks: Modeling, stability analysis and coverage bp-training. Neurocomput., 359 0 (C): 0 227–240, sep 2019. ISSN 0925-2312. doi:10.1016/j.neucom.2019.06.004. URL http...

    2019 doi

  35. [43]

    Converting transformers to polynomial form for secure inference over homomorphic encryption, 2023

    Itamar Zimerman, Moran Baruch, Nir Drucker, Gilad Ezov, Omri Soceanu, and Lior Wolf. Converting transformers to polynomial form for secure inference over homomorphic encryption, 2023

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.