Softmax temperature controls the rank of learned representations: high temperature induces rank-deficit bias, compresses features, hurts OOD generalization, and improves OOD detection.
A winner-take-all circuit with controllable soft max property.Advances in neural information processing systems, 12, 1999
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization
Softmax temperature controls the rank of learned representations: high temperature induces rank-deficit bias, compresses features, hurts OOD generalization, and improves OOD detection.