GPT-3 can learn to express well-calibrated uncertainty about its answers using natural language phrases rather than logits.
hub
Deep Anomaly Detection with Outlier Exposure
21 Pith papers cite this work, alongside 402 external citations. Polarity classification is still indexing.
21
Pith papers citing it
402
external citations · Pith
abstract
It is important to detect anomalous inputs when deploying machine learning systems. The use of larger and more complex inputs in deep learning magnifies the difficulty of distinguishing between anomalous and in-distribution examples. At the same time, diverse image and text data are available in enormous quantities. We propose leveraging these data to improve deep anomaly detection by training anomaly detectors against an auxiliary dataset of outliers, an approach we call Outlier Exposure (OE). This enables anomaly detectors to generalize and detect unseen anomalies. In extensive experiments on natural language processing and small- and large-scale vision tasks, we find that Outlier Exposure significantly improves detection performance. We also observe that cutting-edge generative models trained on CIFAR-10 may assign higher likelihoods to SVHN images than to CIFAR-10 images; we use OE to mitigate this issue. We also analyze the flexibility and robustness of Outlier Exposure, and identify characteristics of the auxiliary dataset that improve performance.
hub tools
citation-role summary
method 1
citation-polarity summary
roles
method 1polarities
use method 1representative citing papers
DISC extracts multi-statistic trajectories from diffusion denoising to both detect and classify types of distributional shifts in OOD data.
DOSER detects OOD actions via diffusion-model denoising error and applies selective regularization based on predicted transitions, proving gamma-contraction with performance bounds and outperforming priors on offline RL benchmarks.
Sparse autoencoders on ViT class tokens reveal stable Class Activation Profiles for in-distribution data, enabling OOD detection via divergence from core energy profiles.
ASTRANet combines a redshift-free spectral classifier, a 16-score anomaly detector, and conformal prediction to identify and calibrate uncertainty for out-of-taxonomy astronomical transients.
Debiased negative mining via Monte-Carlo sampling from ID labels and unlabeled wild data improves OOD detection with VLMs and achieves new state-of-the-art results.
GAMR introduces geometric-aware manifold regularization via virtual outlier synthesis to enhance intra-class compactness and inter-class separation, improving robustness to noisy labels beyond passive sample filtering.
ROAST selectively trains anomaly detectors on less vulnerable patient data with targeted outlier exposure, boosting recall by 16.2% in black-box settings and reducing training time by 88.3%.
LoFT uses parameter-efficient fine-tuning of foundation models for long-tailed semi-supervised learning, supported by proofs that this reduces hypothesis complexity to minimize balanced posterior error and compresses outlier acceptance regions, with LoFT-OW handling open-world OOD cases.
QueST replaces local point tracking with persistent semantic queries that globally attend to spatio-temporal features and apply 3D grounding to suppress drift, cutting absolute point error by 67.7% versus TAP-Net on long articulated sequences.
Scaling pretrained representations improves label-free OOD detection on frozen backbones, causing performance gaps between global and local detectors to vanish across vision and language tasks.
Language models show good calibration when asked to estimate the probability that their own answers are correct, with performance improving as models get larger.
Develops an RL-augmented gradient descent optimizer for dynamic OOD detection together with a temporal error decomposition framework comparing it to standard GD.
TaskFusion combines AGF feature mapping, cross-task augmentation, and distilled replay for continual anomaly detection on heterogeneous tabular data, reporting gains over baselines on 21 datasets.
DFDNet disentangles content from style via dual modules to boost fine-grained OOD detection performance on multiple datasets.
HRP decouples annotation reliability (alpha) and pseudo-label reliability (beta) via bilevel meta-learning and routes them to distinct objectives in reliability-aware Mixup and contrastive learning for improved noisy-label robustness.
TER-DAgger uses force-prediction mismatches to trigger human corrections and residual-policy training, lifting precision-insertion success from 40.0% to 77.2% on average.
NRM enables OoD detection by joint latent likelihood, assigning lower values to SVHN than CIFAR-10 (unlike VAEs/flows) and consistent across other OoD sets.
Sparse autoencoders show OOD prompts increase fallacious concept activation in transformers, offering a mechanistic measure of shift and a path to robust fine-tuning.
Energy-based fine-tuning outperforms other OOD detection methods on the real-world Plant Pathology 2021 dataset, improving detection over softmax while maintaining in-distribution accuracy.
DMDSC adapts class-specific margins dynamically by label frequency in deep simplex classifiers to improve open-set recognition on imbalanced medical image datasets.
citing papers explorer
-
Teaching Models to Express Their Uncertainty in Words
GPT-3 can learn to express well-calibrated uncertainty about its answers using natural language phrases rather than logits.
-
Beyond Binary Out-of-Distribution Detection: Characterizing Distributional Shifts with Multi-Statistic Diffusion Trajectories
DISC extracts multi-statistic trajectories from diffusion denoising to both detect and classify types of distributional shifts in OOD data.
-
Beyond Penalization: Diffusion-based Out-of-Distribution Detection and Selective Regularization in Offline Reinforcement Learning
DOSER detects OOD actions via diffusion-model denoising error and applies selective regularization based on predicted transitions, proving gamma-contraction with performance bounds and outperforming priors on offline RL benchmarks.
-
Sparsity as a Key: Unlocking New Insights from Latent Structures for Out-of-Distribution Detection
Sparse autoencoders on ViT class tokens reveal stable Class Activation Profiles for in-distribution data, enabling OOD detection via divergence from core energy profiles.
-
Catching Disguised Transients with ASTRANet: Anomaly-Aware Spectroscopic Classification and Conformal Calibration
ASTRANet combines a redshift-free spectral classifier, a 16-score anomaly detector, and conformal prediction to identify and calibrate uncertainty for out-of-taxonomy astronomical transients.
-
Debiased Negative Mining Improves Out-of-distribution Detection with Pre-trained Vision-Language Models
Debiased negative mining via Monte-Carlo sampling from ID labels and unlabeled wild data improves OOD detection with VLMs and achieves new state-of-the-art results.
-
GAMR: Geometric-Aware Manifold Regularization with Virtual Outlier Synthesis for Learning with Noisy Labels
GAMR introduces geometric-aware manifold regularization via virtual outlier synthesis to enhance intra-class compactness and inter-class separation, improving robustness to noisy labels beyond passive sample filtering.
-
ROAST: Risk-aware Outlier-exposure for Adversarial Selective Training of Anomaly Detectors Against Evasion Attacks
ROAST selectively trains anomaly detectors on less vulnerable patient data with targeted outlier exposure, boosting recall by 16.2% in black-box settings and reducing training time by 88.3%.
-
LoFT: Parameter-Efficient Fine-Tuning for Long-tailed Semi-Supervised Learning in Open-World Scenarios
LoFT uses parameter-efficient fine-tuning of foundation models for long-tailed semi-supervised learning, supported by proofs that this reduces hypothesis complexity to minimize balanced posterior error and compresses outlier acceptance regions, with LoFT-OW handling open-world OOD cases.
-
QueST: Persistent Queries as Semantic Monitors for Drift Suppression in Long-Horizon Tracking
QueST replaces local point tracking with persistent semantic queries that globally attend to spatio-temporal features and apply 3D grounding to suppress drift, cutting absolute point error by 67.7% versus TAP-Net on long articulated sequences.
-
Scaling Pretrained Representations Enables Label-Free Out-of-Distribution Detection Without Fine-Tuning
Scaling pretrained representations improves label-free OOD detection on frozen backbones, causing performance gaps between global and local detectors to vanish across vision and language tasks.
-
Language Models (Mostly) Know What They Know
Language models show good calibration when asked to estimate the probability that their own answers are correct, with performance improving as models get larger.
-
Theoretical Grounding of Out-Of-Distribution Detection With Reinforcement Learning Optimizer
Develops an RL-augmented gradient descent optimizer for dynamic OOD detection together with a temporal error decomposition framework comparing it to standard GD.
-
TaskFusion: Continual Anomaly Detection for Heterogeneous Tabular Data
TaskFusion combines AGF feature mapping, cross-task augmentation, and distilled replay for continual anomaly detection on heterogeneous tabular data, reporting gains over baselines on 21 datasets.
-
Dual Feature Decoupling for Fine-Grained OOD Detection
DFDNet disentangles content from style via dual modules to boost fine-grained OOD detection performance on multiple datasets.
-
Holistic Reliability Propagation: Decoupling Annotation and Prediction for Robust Noisy-Label
HRP decouples annotation reliability (alpha) and pseudo-label reliability (beta) via bilevel meta-learning and routes them to distinct objectives in reliability-aware Mixup and contrastive learning for improved noisy-label robustness.
-
Force-Aware Residual DAgger via Trajectory Editing for Precision Insertion with Impedance Control
TER-DAgger uses force-prediction mismatches to trigger human corrections and residual-policy training, lifting precision-insertion success from 40.0% to 77.2% on average.
-
Out-of-Distribution Detection Using Neural Rendering Generative Models
NRM enables OoD detection by joint latent likelihood, assigning lower values to SVHN than CIFAR-10 (unlike VAEs/flows) and consistent across other OoD sets.
-
At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
Sparse autoencoders show OOD prompts increase fallacious concept activation in transformers, offering a mechanistic measure of shift and a path to robust fine-tuning.
-
Beyond Toy Benchmarks: A Systematic Evaluation of OOD Detection Methods For Plant Pathology Classification
Energy-based fine-tuning outperforms other OOD detection methods on the real-world Plant Pathology 2021 dataset, improving detection over softmax while maintaining in-distribution accuracy.
-
DMDSC: A Dynamic-Margin Deep Simplex Classifier for Open-Set Recognition on Medical Image Datasets
DMDSC adapts class-specific margins dynamically by label frequency in deep simplex classifiers to improve open-set recognition on imbalanced medical image datasets.