Pith. sign in

REVIEW 3 cited by

Fix your classifier: the marginal value of training the last weight layer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1801.04540 v2 pith:W4JDCGDD submitted 2018-01-14 cs.LG cs.CVstat.ML

Elad Hoffer , Itay Hubara , Daniel Soudry This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords classifiermodelsclassificationneuralnumbertasksusedvalue
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Neural networks are commonly used as models for classification for a wide variety of tasks. Typically, a learned affine transformation is placed at the end of such models, yielding a per-class value used for classification. This classifier can have a vast number of parameters, which grows linearly with the number of possible classes, thus requiring increasingly more resources. In this work we argue that this classifier can be fixed, up to a global scale constant, with little or no loss of accuracy for most tasks, allowing memory and computational benefits. Moreover, we show that by initializing the classifier with a Hadamard matrix we can speed up inference as well. We discuss the implications for current understanding of neural network models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GIF: Generative Inspiration for Face Recognition at Scale

    cs.CV 2025-05 conditional novelty 6.0 of 10

    GIF trains face recognition by predicting structured integer codes per identity, cutting classifier cost from linear to logarithmic in the number of identities while improving IJB-B/IJB-C accuracy over efficient-train...

  2. LCA: Loss Change Allocation for Neural Network Training

    cs.LG 2019-09 conditional novelty 6.0 of 10

    A per-parameter decomposition of training loss change shows that learning is noisy, with only about half of parameters helping per step, some layers hurting overall, and learning spikes synchronized across layers.

  3. LightMC: A Dynamic and Efficient Multiclass Decomposition Algorithm

    cs.LG 2019-08 conditional novelty 6.0 of 10

    LightMC learns a multiclass ECOC decomposition dynamically by treating the decoder as a softmax layer and updating the coding matrix with per-class gradient averages.

Pith tools