Language models show good calibration when asked to estimate the probability that their own answers are correct, with performance improving as models get larger.
Hybrid Models with Deep and Invertible Features
2 Pith papers cite this work, alongside 34 external citations. Polarity classification is still indexing.
2
Pith papers citing it
34
external citations · Pith
abstract
We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of the features, and p(targets | features), the predictive distribution, can be computed exactly in a single feed-forward pass. We show that our hybrid model, despite the invertibility constraints, achieves similar accuracy to purely predictive models. Moreover the generative component remains a good model of the input features despite the hybrid optimization objective. This offers additional capabilities such as detection of out-of-distribution inputs and enabling semi-supervised learning. The availability of the exact joint density p(targets, features) also allows us to compute many quantities readily, making our hybrid model a useful building block for downstream applications of probabilistic deep learning.
verdicts
UNVERDICTED 2representative citing papers
NRM enables OoD detection by joint latent likelihood, assigning lower values to SVHN than CIFAR-10 (unlike VAEs/flows) and consistent across other OoD sets.
citing papers explorer
-
Language Models (Mostly) Know What They Know
Language models show good calibration when asked to estimate the probability that their own answers are correct, with performance improving as models get larger.
-
Out-of-Distribution Detection Using Neural Rendering Generative Models
NRM enables OoD detection by joint latent likelihood, assigning lower values to SVHN than CIFAR-10 (unlike VAEs/flows) and consistent across other OoD sets.