{"paper":{"title":"Benchmarking Neural Network Robustness to Common Corruptions and Perturbations","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Benchmarks for common corruptions show negligible relative robustness gains from AlexNet to ResNet.","cross_cats":["cs.CV","stat.ML"],"primary_cat":"cs.LG","authors_text":"Dan Hendrycks, Thomas Dietterich","submitted_at":"2019-03-28T20:56:37Z","abstract_excerpt":"In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations. We find that there are negligible changes in relative cor"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the 15 chosen corruptions and the perturbations in ImageNet-P sufficiently represent the common real-world image degradations classifiers will encounter in deployment.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Introduces ImageNet-C and ImageNet-P benchmarks revealing negligible robustness gains from AlexNet to ResNet models on common corruptions and perturbations, plus methods to improve them.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Benchmarks for common corruptions show negligible relative robustness gains from AlexNet to ResNet.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"2add343f8cb6aa176e77282b7b65934478f30024412f40ba0b287f94d2e00d28"},"source":{"id":"1903.12261","kind":"arxiv","version":1},"verdict":{"id":"02eddbd9-de5c-46c2-a10f-a550a4419adb","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-13T04:56:52.450015Z","strongest_claim":"We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers.","one_line_summary":"Introduces ImageNet-C and ImageNet-P benchmarks revealing negligible robustness gains from AlexNet to ResNet models on common corruptions and perturbations, plus methods to improve them.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the 15 chosen corruptions and the perturbations in ImageNet-P sufficiently represent the common real-world image degradations classifiers will encounter in deployment.","pith_extraction_headline":"Benchmarks for common corruptions show negligible relative robustness gains from AlexNet to ResNet."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/1903.12261/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":62,"sample":[{"doi":"","year":2013,"title":"Applying convolutional neural networks concepts to hybrid nn-hmm model for speech recognition","work_id":"9436cef1-b1e3-41f2-831c-b5680a61be5d","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"Why do deep convolutional networks generalize so poorly to small image transformations?","work_id":"3ce65db5-2a7c-48ae-a1c1-26dd55576ac3","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2016,"title":"Measuring neural net robustness with constraints","work_id":"48868be1-0205-4925-9d59-d619235c9923","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2005,"title":"A non-local algorithm for image denoising","work_id":"caeab37b-9ce3-4300-9889-1cb903bc6416","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2016,"title":"Defensive distillation is not robust to adversarial examples","work_id":"a2850dac-02e8-4333-a395-d7b82c936372","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":62,"snapshot_sha256":"1b9f66ac54b27bef1bce4097562af091645b8e072824f97e1744a0a7ab377004","internal_anchors":0},"formal_canon":{"evidence_count":1,"snapshot_sha256":"37c09687df2154df9245b9158884898af498502b8fe540225f3717a3e0d78471"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}