Newer 'relaxed' differential privacy analyses yield tighter upper-bound guarantees without changing actual privacy loss, and attack success can be translated into a lower bound on that loss.
Cold Case: The Lost MNIST Digits
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Although the popular MNIST dataset [LeCun et al., 1994] is derived from the NIST database [Grother and Hanaoka, 1995], the precise processing steps for this derivation have been lost to time. We propose a reconstruction that is accurate enough to serve as a replacement for the MNIST dataset, with insignificant changes in accuracy. We trace each MNIST digit to its NIST source and its rich metadata such as writer identifier, partition identifier, etc. We also reconstruct the complete MNIST test set with 60,000 samples instead of the usual 10,000. Since the balance 50,000 were never distributed, they enable us to investigate the impact of twenty-five years of MNIST experiments on the reported testing performances. Our results unambiguously confirm the trends observed by Recht et al. [2018, 2019]: although the misclassification rates are slightly off, classifier ordering and model selection remain broadly reliable. We attribute this phenomenon to the pairing benefits of comparing classifiers on the same digits.
fields
cs.LG 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
That which we call private
Newer 'relaxed' differential privacy analyses yield tighter upper-bound guarantees without changing actual privacy loss, and attack success can be translated into a lower bound on that loss.