Pith. sign in

REVIEW 4 major objections 4 minor 40 references

Neural Networks Learn Distance Metrics

T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that neural networks naturally favor distance-based representations—smaller activations mean closer to a learned prototype—and shows that small architectural changes that force intensity-style outputs can collapse…

desk verdict A clean empirical datapoint about an RBF-style layer, but the headline claim about distance bias is not established; the ReLU2 failure is a dead-ReLU artifact. read the letter →

arxiv 2502.02103 v1 pith:34FIE7EV submitted 2025-02-04 cs.LG cs.AIstat.ML

Alan Oursland This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords distance-basedrepresentationsintensity-basedMahalanobisdistanceReLUdeadneuronsprototypelearningneuralnetworkinterpretabilityMNISTOffsetL2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that neural networks do not primarily encode features by activation magnitude (intensity), but by distance to learned prototypes: smaller activations signal proximity. It tests this by building six simple MNIST classifiers whose final layer is architecturally forced into one mode or the other, using a second activation for non-negativity and a negation layer to flip a distance code into an intensity code. The key empirical pattern is that ReLU2, forced into intensity mode, collapses to 56% accuracy with mass node death, while ReLU2-Neg, which can internally use distance, recovers to 96%; Abs2 stays at baseline but Abs2-Neg degrades. The paper then proposes OffsetL2, a layer that computes a weighted L2 distance to a learned per-class prototype, which reaches 97.6% with very low variance and closes the gap between normal and negated variants. If correct, the results imply that activation values in trained networks should be read geometrically, as distances, and that architecture design should accommodate distance-based representations rather than only intensity-based ones.

What carries the argument

The carrying object is the distinction between distance and intensity representations, formalized through Mahalanobis distance. A distance representation is modeled as $f(x) = |W(x-\mu)|_p$: a linear layer learns the eigenvector and scaling transform, the bias learns the prototype offset, and an Abs or ReLU pair encodes absolute distance. CrossEntropyLoss is the constraint that forces intensity output—high activation for the correct class—and a negation layer is the mechanism that flips a positive distance code into the required negative intensity code. The geometric picture is that each output hyperplane intersects the latent space at learned prototypes; a distance-mode hyperplane passes near the optimal class center $z_c$, while an intensity-mode hyperplane passes near anti-centers $z_{\neg c}$. OffsetL2, $y_i = \|\alpha_i \odot (x - \mu_i)\|_2$, carries the argument's final validation by making a single learned prototype per class explicit.

What would settle it

Train a ReLU2-Neg network on MNIST, identify each class prototype by intersecting the first-layer hyperplanes, and measure the correlation between an input's distance to the class-c prototype and the class-c logit. The paper's claim predicts that smaller distances correspond to larger (less negative) or near-zero logits in a consistent ordering across all ten classes; if the logits instead vary with intensity-style separation and show no distance ordering, the distance-representation reading is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that ReLU-based architectures exhibit a natural bias toward distance-based learning. Cross-entropy loss with a non-negative final layer forces an intensity readout—large logits for the correct class—so a second ReLU creates an intensity-constrained network; that network fails catastrophically because minimizing activations for the 90% of examples that are not the target class drives nearly all pre-activations negative and kills 33% of output nodes outright. Adding a negation layer converts a positive distance code into a negative intensity code, restoring near-baseline performance. Abs networks behave differently because absolute value never kills nodes, but Abs2-Neg underperforms, which the paper traces to a geometric constraint: a hyperplane must pass through one optimal prototype point plus 127 less-optimal points, whereas Abs2 can choose among combinatorial anti-prototype configurations. The paper's final validation is OffsetL2, which explicitly models the distance to a single learned prototype per class and achieves the best accuracy and stability, supporting the claim that distance-based representations are the natural mode of neural computation.

Load-bearing premise

The argument stands on the claim that the architecture constraints really force one representational type into the final layer: cross-entropy plus a non-negative activation forces intensity, and a negation layer converts distance into intensity—if the ReLU2 collapse is instead just the standard dying-ReLU optimization problem, the central distance-versus-intensity distinction loses its footing.

Editorial extensions

If this is right

  • In a trained network, a small activation of a node is evidence that the input is near that node's learned prototype, not evidence that the feature is absent or weak.
  • Putting a second ReLU immediately before logits—an intensity-forcing arrangement—can cause catastrophic dying-neuron collapse; a negation layer ahead of the logits can recover performance by allowing distance-based internal codes.
  • Abs-based layers are more robust under intensity constraints because no nodes die, but their distance-constrained variant pays a geometric price in high-dimensional prototype selection.
  • Explicit prototype-distance layers like OffsetL2 outperform implicit hyperplane-based prototype discovery on MNIST and sharply reduce run-to-run variance, with standard deviation down to 0.07%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the distance-bias claim generalizes, widely used practices that implicitly penalize or suppress activations—such as weight decay, dropout, or activation-based pruning—may be operating as prototype-distance regularizers and should be reinterpreted accordingly.
  • Editorial inference: the dead-node mechanism predicts that any clustered dataset with many non-target classes per decision, not just MNIST, should show a ReLU2-style collapse under intensity constraints; running the same six architectures on CIFAR-10 or a synthetic Gaussian-mixture task would test the scope.
  • Editorial inference: the OffsetL2 success suggests a direct architectural recipe—replace the final linear layer with a learned per-class Mahalanobis layer—that could be tested on larger-scale classification and in the hidden layers of deeper networks, not just the two-layer setting studied here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript hypothesizes that neural networks have a natural bias toward distance-based representations, in which small activations indicate proximity to learned prototypes, and contrasts this with intensity-based representations in which large activations indicate feature presence. To test this, it trains simple two-layer MNIST models with six primary architectures (Abs, ReLU, Abs2, Abs2-Neg, ReLU2, ReLU2-Neg) and reports that ReLU2 collapses while ReLU2-Neg recovers, and that Abs2 retains baseline accuracy while Abs2-Neg degrades. It then introduces a geometric framework based on Mahalanobis distance and prototype/anti-prototype centers, and proposes the OffsetL2 layer, which explicitly computes a weighted L2 distance to learned reference points. The paper claims that the experiments validate the framework and that the OffsetL2 results support a distance-based view of neural representations.

Significance. If the central claim were established, the paper would offer a substantive interpretive shift for deep learning: reading activations as distances rather than intensities would affect architecture design and interpretability. The manuscript also includes a useful empirical artifact: a simple, reproducible two-layer setup, source code, and an explicit OffsetL2 architecture with strong MNIST accuracy and low variance. However, the evidence presented does not support the central claim. The ReLU2/ReLU2-Neg contrast is confounded by well-known dead-ReLU optimization behavior; the representational type is never measured directly; and the OffsetL2 validation is partly circular because that layer computes a distance by construction. The statistical reporting also contains inconsistencies that prevent the results from being interpreted as stated.

major comments (4)
  1. [§5.2–5.3 and §6.2] The ReLU2 versus ReLU2-Neg contrast does not isolate distance-based versus intensity-based representations. With CrossEntropyLoss and no final bias, ReLU2 must make the correct-class preactivation positive after the final ReLU, while ReLU2-Neg can place the correct class on the ReLU zero plateau and drive non-target classes positive. The dead-node statistics reported in §6.2 (33.0% permanently inactive, 53.5% active on less than 5% of inputs) are the standard dead-ReLU gradient-flow failure, and the discussion explicitly invokes this known mechanism. The experiment therefore cannot discriminate the paper's distance-bias hypothesis from ordinary optimization pathology.
  2. [§6.1–6.3] The representational type is assigned by the sign convention the authors impose, not measured in the trained networks. No analysis shows that ReLU2-Neg's output-layer geometry is aligned with class-conditional prototype centers, nor that the logits correlate with a Mahalanobis-distance-like quantity. The Abs2-Neg performance drop is explained post hoc with the unmeasured entities zc and z¬c, so the explanation has no predictive content as tested. A direct probe of the learned representation is needed before the central claim can be accepted.
  3. [§6.4 and Table 5] The OffsetL2 validation is partly by construction: the layer explicitly computes ||αi ⊙ (x − µi)||2, so its strong performance is expected for an RBF-like layer and cannot demonstrate that ordinary networks naturally prefer distance representations. In addition, the comparison mixes training conventions: the baselines and primary variants are trained for 5,000 epochs, while the OffsetL2 models are trained for 50,000 epochs, making the accuracy gains in Table 5 relative to Tables 2–3 uninterpretable as evidence of architectural advantage.
  4. [§5.2–5.3 and §6.2] There are multiple statistical reporting inconsistencies. The same t-statistic (t(38) = -17.33) is reported for both ReLU2's degradation and ReLU2-Neg's recovery; the comparison of ReLU2-Neg to baseline ReLU reports t(38) = -12.78 with p < 0.001 yet describes the two as 'statistically comparable'; and §6.2 gives ReLU2 accuracy as 47.20% ± 12.00% while Table 2 reports 56.31 ± 19.31. These discrepancies must be resolved before the empirical claims can be evaluated.
minor comments (4)
  1. [§6.1] Figure 1 is referenced in the text and has a caption, but no figure appears in the manuscript.
  2. [Tables 3 and 5] Model naming is inconsistent: 'ReLU2 Neg' and 'Abs2 Neg Bias' appear in Table 3, while 'ReLU2-Neg' and 'Abs2-Neg' are used elsewhere; the hyphenation and bias notation should be uniform.
  3. [Abstract and §6.4] The abstract emphasizes six architectural variants, but §6.4 introduces four additional OffsetL2-based models; the paper should clarify the relationship between the primary six and the later models.
  4. [§5.3] The text contains 'p ¡ 0.001' instead of 'p < 0.001', and one t-test comparison appears to be copied from the previous paragraph.

Circularity Check

2 steps flagged · score 6.0 of 10

Key validation steps reduce to the paper's own sign conventions: the ReLU2-Neg 'distance' success is defined by the Neg layer, and OffsetL2's success is a self-consistency check of a hand-built distance layer.

  1. self definitional [Section 3.5 (Neg Layers), Section 4.2 (Model Design), applied in Section 5.3 (Distance Learning Models)]
    "Models with the negation learn a positive distance representation which is converted by the negation layer into a negative intensity representation. ... ReLU2-Neg exhibited a remarkable recovery from ReLU2’s catastrophic failure ... This recovery supports our hypothesis that neural networks may be biased towards learning distance-based representations, with the Neg transformation enabling ReLU2-Neg to leverage this bias effectively."

    The paper labels ReLU2-Neg as 'distance-based' by construction: Neg is defined as converting a positive distance representation into a negative intensity one. For a successful ReLU2-Neg classifier, the target class ends up with the smallest ReLU activation because after negation the largest logit is the one closest to zero; this is the same 'minimal activation' pattern the paper equates with distance. The recovery is also fully consistent with the standard dying-ReLU asymmetry documented in Section 6.2 (33.0% permanently inactive and 53.5% active for less than 5% of inputs in ReLU2). The experiment therefore tests a sign convention and an optimization pathology, not an independently observed representational preference.

  2. other [Section 6.4 'Validation Through Additional Experiments']
    "Our theory about the Abs2-Neg performance drop suggests that a layer designed to explicitly represent the distance to a single optimal point might correct the performance difference. ... we propose a layer called OffsetL2 that computes the weighted L2 distance from a single learned reference point µ ... These findings validate our theoretical framework: by explicitly modeling geometric constraints through direct distance calculations, OffsetL2 not only improves accuracy but also stabilizes training."

    OffsetL2 is built to instantiate exactly the framework's distance claim: weighted L2 from a learned prototype per class. Its good MNIST accuracy is therefore a consistency check of its own defining equation, not an external test of whether ordinary networks spontaneously learn distance metrics. The paper itself says the layer is 'conceptually similar to' RBF networks, so the result is a known class of method under a new name. Using this success as 'validation' of the distance-bias hypothesis is self-confirming rather than a falsifiable prediction.

full rationale

The paper contains real, self-contained experiments (baseline ReLU/Abs, Abs2, bias-inclusion checks) and does not invoke a uniqueness theorem or rely on a machine-unverified self-citation chain to forbid alternatives. However, the central inference is partly circular. The ReLU2/ReLU2-Neg contrast is presented as evidence of a distance bias, but the 'distance' label is assigned by the Neg sign convention rather than by measuring the learned representation; any successful ReLU2-Neg classifier tends to place the target class at minimal activation because of the loss and sign flip, so the architecture's success is built into the imposed output order. The dead-node statistics in Section 6.2 provide a standard optimization explanation that is never experimentally distinguished from the distance-bias story. Similarly, OffsetL2 is defined as a distance layer and then its strong performance is reported as validating the distance framework, which is self-confirming and also a known RBF result under a new name. Self-citations to Oursland (2024a,b) supply the Mahalanobis interpretation, but I do not count them as an independent circular step because no load-bearing argument reduces solely to them. Overall, the paper has independent empirical content, but its headline claim is supported partly by construction rather than by falsifiable prediction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 2 invented entities

The ledger shows that the central claim depends on an assumed semantics for activations, on unmeasured prototype constructs, and on learned OffsetL2 parameters whose values are fitted to MNIST rather than derived. The framework is largely carried over from the author's prior preprints, and the new layer's success is partly guaranteed by its explicit distance computation.

free parameters (5)
  • Hidden layer width = 128
    Chosen by hand for all models; no sweep is reported, and capacity affects whether ReLU2 collapses as described.
  • SGD learning rate = 0.001
    Fixed without tuning; the model comparisons assume this rate is fair to all architectures.
  • Training epochs = 5,000 then 50,000
    The 50,000 epoch schedule was adopted after observing non-convergence at 5,000 epochs, a post hoc change that affects the OffsetL2 comparisons.
  • OffsetL2 reference points mu_i = Learned on MNIST training data
    Per-class prototype vectors fit by gradient descent; the claim of a single optimal prototype rests on these fitted values.
  • OffsetL2 scaling vectors alpha_i = Learned on MNIST training data
    Per-dimension weights fit by gradient descent; they modulate the distance but are not predicted by the theory.
assumptions (6)
  • domain assumption CrossEntropyLoss, activation positivity, and a negation layer force the final layer to use either intensity or distance representations
    Section 4.2 states this without proof; the mapping between architecture and representational type is never verified by probing the networks.
  • standard math Eigendecomposition and norm algebra in Eqs. (2) and (3) are standard linear algebra
    Listed for completeness; this part is not a weakness.
  • domain assumption Mahalanobis eigendecomposition describes what linear plus Abs or ReLU layers compute
    Section 3.3 asserts this based on Oursland 2024a; no derivation or experiment in this paper independently establishes it.
  • domain assumption Two-layer MLPs on MNIST represent neural networks broadly enough for claims about natural representational bias
    The title and Section 7 generalize from shallow MNIST models to neural networks in general.
  • ad hoc to paper MNIST class labels are valid proxies for latent prototypes zc and z-not-c
    Section 6.1 defines ideal centers from class clusters and uses them to explain the Abs2 and Abs2-Neg gap; these centers are not directly observed.
  • domain assumption Abs(x)=ReLU(x)+ReLU(-x) means ReLU encodes distance
    Section 3.2 uses this identity to reinterpret ReLU as distance, but the identity holds for any input and does not by itself establish what networks learn.
invented entities (2)
  • Prototype center zc and anti-center z-not-c
    purpose: Ideal latent points that the second-layer hyperplane is claimed to intersect to produce distance or intensity representations
    Section 6.1 introduces these constructs to explain Abs2-Neg underperformance; no direct measurement or independent falsifiable prediction is provided.
  • OffsetL2 layer
    purpose: Explicit weighted L2 distance from a learned reference point, used as a final classification layer
    Section 6.4 introduces the layer; its empirical support consists only of the in-paper MNIST runs, and it is acknowledged to be conceptually similar to RBF networks, so independent evidence outside this paper is absent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Networks Learn Distance Metrics." pith.science (2026). https://pith.science/paper/34FIE7EV

@misc{pith2026250202103,
  author       = {Pith},
  title        = {Pith review of: Neural Networks Learn Distance Metrics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/34FIE7EV}},
  note         = {Machine review of arXiv:2502.02103}
}
read the original abstract

Neural networks may naturally favor distance-based representations, where smaller activations indicate closer proximity to learned prototypes. This contrasts with intensity-based approaches, which rely on activation magnitudes. To test this hypothesis, we conducted experiments with six MNIST architectural variants constrained to learn either distance or intensity representations. Our results reveal that the underlying representation affects model performance. We develop a novel geometric framework that explains these findings and introduce OffsetL2, a new architecture based on Mahalanobis distance equations, to further validate this framework. This work highlights the importance of considering distance-based learning in neural network design.

Figures

Figures reproduced from arXiv: 2502.02103 by the authors.

Figure 1
Figure 1. Class distributions in the latent space show overlapping clusters with varying statistical [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 26 canonical work pages

  1. [1]

    Boolean functions and artificial neural networks

    Martin Anthony. Boolean functions and artificial neural networks. Technical Report LSE-CDAM-2003-01, Centre for Discrete and Applicable Mathematics, London School of Economics, January 2003. URL http://www.maths.lse.ac.uk/cdam/. A version to appear in Boolean Functions: Volume II, edited by Yves Crama and Peter Hammer

    work page 2003

  2. [2]

    Representation learning: A review and new perspectives

    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35 0 (8): 0 1798--1828, 2013

    2013

  3. [3]

    Signature verification using a siamese time delay neural network

    Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard S "a ckinger, and Roopak Shah. Signature verification using a siamese time delay neural network. In Advances in Neural Information Processing Systems, pages 737--744, 1994

    work page 1994

  4. [4]

    Radial basis functions, multi-variable functional interpolation and adaptive networks

    David S Broomhead and David Lowe. Radial basis functions, multi-variable functional interpolation and adaptive networks. Royal Signals and Radar Establishment Malvern (United Kingdom), 1988

    work page 1988

  5. [5]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597--1607. PMLR, 2020

    2020

  6. [6]

    Girshick, Larry Zitnick, and Dhruv Batra

    Michael Cogswell, Faruk Ahmed, Ross B. Girshick, Larry Zitnick, and Dhruv Batra. Reducing overfitting in deep networks by decorrelating representations. arXiv preprint arXiv:1511.06068, 2016

    arXiv 2016

  7. [7]

    The mnist database of handwritten digit images for machine learning research [best of the web]

    Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29 0 (6): 0 141--142, 2012

    work page 2012

  8. [8]

    High-dimensional data analysis: The curses and blessings of dimensionality

    David L Donoho. High-dimensional data analysis: The curses and blessings of dimensionality. AMS math challenges lecture, 1 0 (2000): 0 32, 2000

    work page 2000

Show all 40 references
  1. [9]

    Visualizing higher-layer features of a deep network

    Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network. University of Montreal, 1341 0 (3): 0 1, 2009

    2009

  2. [10]

    Deep sparse rectifier neural networks

    Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 315--323, 2011

    2011

  3. [11]

    Deep Learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016

    2016

  4. [12]

    Qualitatively characterizing neural network optimization problems

    Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. In International Conference on Learning Representations (ICLR), 2015

    2015

  5. [13]

    Dimensionality reduction by learning an invariant mapping

    Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), volume 2, pages 1521--1528. IEEE, 2006

    2006

  6. [14]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 1026--1034, 2015

    2015

  7. [15]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729--9738, 2020

    2020

  8. [16]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Cl \'e ment Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018

    2018

  9. [17]

    Learning vector quantization

    Teuvo Kohonen. Learning vector quantization. In Michael A Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 537--540. MIT Press, Cambridge, MA, 1995

    1995

  10. [18]

    Gradient-based learning applied to document recognition

    Yann LeCun, L \'e on Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

    1998

  11. [19]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521 0 (7553): 0 436--444, 2015

    2015

  12. [20]

    Wide neural networks of any depth evolve as linear models under gradient descent

    Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019

    2019

  13. [21]

    Visualizing the loss landscape of neural nets

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Advances in neural information processing systems, volume 31, 2018

    2018

  14. [22]

    The mythos of model interpretability

    Zachary C Lipton. The mythos of model interpretability. Queue, 16 0 (3): 0 31--57, 2018

    2018

  15. [23]

    On the generalized distance in statistics

    Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Proceedings of the National Institute of Sciences of India, 2 0 (1): 0 49--55, 1936

    1936

  16. [24]

    A logical calculus of the ideas immanent in nervous activity

    Warren S McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5 0 (4): 0 115--133, 1943

    1943

  17. [25]

    The mahalanobis distance and its applications in discriminant analysis

    Geoffrey J McLachlan. The mahalanobis distance and its applications in discriminant analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 11 0 (2): 0 e1452, 2019

    2019

  18. [26]

    Mish: A self regularized non-monotonic neural activation function

    Diganta Misra. Mish: A self regularized non-monotonic neural activation function. arXiv preprint arXiv:1908.08681, 2019

    1908 arXiv

  19. [27]

    Methods for interpreting and understanding deep neural networks

    Gr \'e goire Montavon, Wojciech Samek, and Klaus-Robert M \"u ller. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73: 0 1--15, 2018

    2018

  20. [28]

    Fast learning in networks of locally-tuned processing units

    John Moody and Christian J Darken. Fast learning in networks of locally-tuned processing units. Neural Computation, 1 0 (2): 0 281--294, 1989

    1989

  21. [29]

    Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML), pages 807--814, 2010

    2010

  22. [30]

    Feature visualization

    Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2 0 (11): 0 e7, 2017

    2017

  23. [31]

    Interpreting neural networks through mahalanobis distance

    Alan Oursland. Interpreting neural networks through mahalanobis distance. arXiv preprint arXiv:2410.19352, 2024 a

    2024 arXiv

  24. [32]

    Neural networks use distance metrics

    Alan Oursland. Neural networks use distance metrics. arXiv preprint arXiv:2411.17932, 2024 b

    2024 arXiv

  25. [33]

    Universal approximation using radial-basis-function networks

    Jooyoung Park and Irwin W Sandberg. Universal approximation using radial-basis-function networks. In Proceedings of the International Conference on Neural Networks, pages 246--257. MIT Press, 1991

    1991

  26. [34]

    The effectiveness of data augmentation in image classification using deep learning

    Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621, 2017

    2017 arXiv

  27. [35]

    Introduction to a general theory of elementary propositions

    Emil L Post. Introduction to a general theory of elementary propositions. American Journal of Mathematics, 43 0 (3): 0 163--185, 1921

    1921

  28. [36]

    Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for activation functions. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=Hk4_qw5K-. arXiv preprint arXiv:1710.05941

    2018 arXiv

  29. [37]

    The perceptron: a probabilistic model for information storage and organization in the brain

    Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65 0 (6): 0 386, 1958

    1958

  30. [38]

    Prototypical networks for few-shot learning

    Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in neural information processing systems, volume 30, 2017

    2017

  31. [39]

    Distance metric learning for large margin nearest neighbor classification

    Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research, 10 0 (Feb): 0 207--244, 2009

    2009

  32. [40]

    Xing, Andrew Y

    Eric P. Xing, Andrew Y. Ng, Michael I. Jordan, and Stuart J. Russell. Distance metric learning, with application to clustering with side-information. In Advances in Neural Information Processing Systems (NeurIPS), pages 505--512, 2002

    2002

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.