Pith. sign in

REVIEW 4 cited by

Distilling Model Failures as Directions in Latent Space

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2206.14754 v2 pith:ANLOQ4OX submitted 2022-06-29 cs.LG

Saachi Jain , Hannah Lawrence , Ankur Moitra , Aleksander Madry This is my paper · ORCID
classification cs.LG
keywords modelfailuremodesautomaticallychallengingdirectionsdistillingframework
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Existing methods for isolating hard subpopulations and spurious correlations in datasets often require human intervention. This can make these methods labor-intensive and dataset-specific. To address these shortcomings, we present a scalable method for automatically distilling a model's failure modes. Specifically, we harness linear classifiers to identify consistent error patterns, and, in turn, induce a natural representation of these failure modes as directions within the feature space. We demonstrate that this framework allows us to discover and automatically caption challenging subpopulations within the training dataset. Moreover, by combining our framework with off-the-shelf diffusion models, we can generate images that are especially challenging for the analyzed model, and thus can be used to perform synthetic data augmentation that helps remedy the model's failure modes. Code available at https://github.com/MadryLab/failure-directions

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 11 citations worldwide. Full citation record

  1. GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A generate-and-verify pipeline using LLMs and grounded VLMs discovers fine-grained error slices in detection/segmentation models, with a new FeSD benchmark reporting Precision@10 of 0.73 versus 0.31 for adapted baselines.

  2. HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging

    cs.CV 2025-01 conditional novelty 6.0 of 10

    HiBug2 discovers error slices in vision models via structured GPT-generated attributes, efficient enumeration, and prediction of unseen failure patterns, improving model repair over prior methods.

  3. Dataset Augmentation by Mixing Visual Concepts

    cs.CV 2024-12 conditional novelty 5.0 of 10

    MVC fine-tunes Stable Diffusion with mixed CLIP caption embeddings to produce in-domain synthetic images, improving classifier accuracy on several benchmarks.

  4. Explainability for Vision Foundation Models: A Survey

    cs.CV 2025-01 conditional novelty 4.0 of 10

    A structured review of 122 papers on explainability for vision foundation models, with a taxonomy and the finding that quantitative evaluation is rare (36%).

Pith tools