REVIEW 5 major objections 4 minor 1 cited by
White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
T0 review · 5 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Under a Gibbs-posterior model of training, the optimal membership inference attack depends on the model only through the loss, so white-box access to parameters gives no asymptotic advantage over black-box loss access.
desk verdict The paper's main theoretical claim—that optimal membership inference depends only on the loss—is invalidated by a marginalization error in Theorem 1; the empirical attacks are still worth a look. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Gibbs posterior over parameters, $P(\theta\mid z_1,\dots,z_n,m_1,\dots,m_n)\propto \exp\left(-\frac{1}{T}\sum_{i=1}^n m_i \ell(\theta,z_i)\right)$, combined with the score function of Theorem 2: $s(z_1,\theta,p_T)=\frac{1}{T}(\tau_{p_T}(z_1)-\ell(\theta,z_1))$. Here $\tau_{p_T}(z_1)$ is the typical loss a model drawn from the posterior would assign to $z_1$ if $z_1$ were not in the training set, so the score is a calibrated loss comparison that turns membership inference into a one-dimensional decision. This machinery makes the loss the only model-dependent quantity, and its tractable approximations (MALT, MAST, MATT) are what the paper evaluates experimentally.
What would settle it
Train a fixed architecture on a fixed dataset many times with different random seeds or batch orders, holding out one fixed sample $z_1$ in half the runs; for each trained parameter vector $\theta$, record its loss $\ell(\theta,z_1)$ and whether $z_1$ was in the training set. Estimate the empirical likelihood ratio $P(\theta\mid z_1\in\text{train})/P(\theta\mid z_1\notin\text{train})$. If there exist two parameter vectors with the same loss that yield substantially different likelihood ratios, then the loss is not a sufficient statistic for membership and Theorem 2's black-box/white-box equivalence fails for that training procedure.
Extended reading notes
Core claim
Theorem 2 shows that under the Gibbs posterior of Equation (2), the Bayes-optimal membership inference score for a sample $z_1$ is $s(z_1,\theta,p_T)=\frac{1}{T}(\tau_{p_T}(z_1)-\ell(\theta,z_1))$, where $\tau_{p_T}(z_1)$ is the soft-minimum (softmin) of the loss over typical parameters that have never seen $z_1$. Because the trained parameters $\theta$ enter this score only through the loss $\ell(\theta,z_1)$, the paper concludes that, asymptotically, white-box access to internal parameters adds no information over black-box access to the loss. The paper further argues that existing membership inference methods are coarser approximations of this optimal score: shadow models estimate the expectation over training sets, and the simple 0-1 attack is a threshold on the loss that ignores per-sample calibration.
Load-bearing premise
The training algorithm must actually produce parameters according to the Gibbs posterior $P(\theta\mid\text{data})\propto \exp(-\frac{1}{T}\sum_i m_i \ell(\theta,z_i))$; if real SGD or Bayesian inference does not sample from this distribution, then the conclusion that the loss is the only informative signal and that white-box access is useless does not transfer to actual models.
Editorial extensions
If this is right
- Defenders should treat loss values as the primary privacy signal: mechanisms that shrink the gap between training and validation losses, such as data augmentation and regularization, directly reduce membership inference accuracy, as the paper's ImageNet experiments confirm.
- Any white-box attack that exploits gradients or hidden activations can at best match, not beat, a tuned loss-threshold attack in the asymptotic regime defined by the Gibbs posterior.
- The paper's MALT rule, which flags any sample whose loss falls below a global threshold, outperforms shadow models on CIFAR-10 with a small convolutional network and on ImageNet with ResNet-101 and VGG-16 in the reported settings.
- Under $\epsilon$-differential privacy, the posterior membership probability is bounded by $\lambda + \epsilon/4$, giving a concrete operational meaning to $\epsilon$: even a value as small as $0.01$ is a strong membership-privacy guarantee.
- The three proposed attacks form a hierarchy of approximations, with MATT using a Taylor/influence-function expansion and outperforming the threshold-based MALT on a logistic regression problem.
Reading between the lines
- If the loss is a sufficient statistic for membership, then membership attacks become largely architecture-agnostic: the same loss-threshold recipe should transfer across model families, and practitioners may not need to train per-architecture shadow models.
- A natural controlled test of the theory is to train models with stochastic gradient Langevin dynamics, which provably samples the Gibbs posterior; in that setting the loss-only optimality should hold exactly and parameter-based white-box attacks should show no advantage.
- At finite training-set sizes the expectation over $T$ in Theorem 2 is only approximated, so parameter-dependent statistics (such as influence functions or gradient norms) may still add information in small-$n$ regimes; the paper's asymptotic claim leaves this possibility open.
- The per-sample threshold $\tau(z_1)$ used by MAST could be estimated more cheaply than training dozens of shadow models, for example by using a single model with multiple dropout draws or a small ensemble, which would make the attack more practical at scale.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Bayes-optimal framework for membership inference under the Gibbs posterior assumption P(θ|z1,...,zn,m) ∝ exp(−(1/T)Σ m_i ℓ(θ,z_i)). Its main theoretical claim, Theorem 2, is that the optimal membership score depends on the model parameters only through the loss ℓ(θ,z1), implying that black-box and white-box attacks are asymptotically equivalent. The paper then derives three practical attack families (MALT, MAST, MATT) as approximations of this score and reports strong empirical results on logistic regression, a small CNN on CIFAR-10, and VGG-16/ResNet-101 on ImageNet, outperforming the 0-1 baseline and shadow-model attacks in several settings.
Significance. If the theoretical claim were correct, the paper would establish a clean and surprising equivalence between white-box and black-box membership inference, with direct implications for privacy accounting and attack design. The proposed threshold-based attacks (especially MALT) are simple, reproducible in spirit, and the empirical comparisons on ImageNet with data augmentation are informative and useful. However, the central theoretical result is not correct as stated: the proof of Theorem 1 marginalizes over the auxiliary training information T under the wrong measure, and a concrete counterexample under the paper's own Eq. (2) shows that exact membership posteriors can differ for two parameters with identical loss. Because the loss-only optimality and the white-box/black-box equivalence are load-bearing for the paper's conceptual contribution, the significance of the manuscript as a theoretical statement is not established.
major comments (5)
- [Section 3.3, Theorem 1 proof, Eqs. (5)-(6)] The law of total expectation is applied incorrectly. In M(θ,z1) = E_T[P(m1=1|θ,z1,T)], the expectation must be taken over the conditional distribution P(T|θ,z1), because θ is drawn from P(θ|z1,T,m1) and hence is statistically dependent on T. The proof instead averages the likelihood ratio under the marginal distribution of T, which is not the posterior membership probability. This error propagates to Theorem 2 and invalidates the claimed loss-only optimality.
- [Section 3.3, Theorem 2] The loss-only claim is falsified by a finite counterexample under Eq. (2). Take Θ={a,b,c}, fixed z1,z2, T=1, λ=1/2, and losses ℓ(a,z1)=0, ℓ(b,z1)=0, ℓ(c,z1)=10; ℓ(a,z2)=10, ℓ(b,z2)=0, ℓ(c,z2)=0. Exact enumeration gives P(m1=1|θ=a,z1)=0.600 and P(m1=1|θ=b,z1)=0.643, although ℓ(a,z1)=ℓ(b,z1)=0. Theorem 2's formula assigns the same value (≈0.633) to both cases, matching neither exact posterior. Thus the proposed attack is not Bayes optimal under Eq. (2), and the conclusion that white-box access provides no benefit does not follow.
- [Section 4.1, Eqs. (14) and (29)] The definitions of τ are inconsistent. Eq. (14) defines τ_p(z1) = −T log ∫ e^{−ℓ(t,z1)/T} p(t) dt, while Eq. (29) defines τ(z1) = log ∫ e^{−ℓ(t,z1)/T} p(t) dt, dropping the factor −T. The score in Eq. (30) is then s = −ℓ + τ, which differs from the Theorem 2 score (τ_p − ℓ)/T by both a sign and a scale. The Gaussian derivation in Eqs. (33)-(34) and (46) treats τ as approximately the negative log-integral, not the log-integral. This inconsistency affects all subsequent MALT/MAST thresholds and should be corrected before the empirical scores can be linked to Theorem 2.
- [Section 6.2, Table 1] The logistic-regression comparison reports 'the peak accuracy obtained for the best threshold.' If this threshold is selected using the test set or the same samples being evaluated, the reported attack accuracies are optimistic and the comparison between MALT and MATT is not a fair estimate of deployable attack performance. Section 5.2 says the threshold is estimated on simulated sets, but the table caption and Section 5.2 should specify unambiguously whether the reported accuracy is the oracle best threshold or a threshold chosen on separate data.
- [Section 3.1, Eq. (1)-(2)] Even setting aside the marginalization error, the paper's optimality result is conditional on the strong assumption that real training procedures sample parameters from the Gibbs posterior in Eq. (2). For SGD-trained deep networks this is not established, so the transfer of Theorem 2's conclusions to the ImageNet experiments is an extrapolation rather than a consequence of the theorem. This is not by itself a reason to reject, but it should be stated more carefully if the theoretical claims are revised.
minor comments (4)
- [Figure 1 caption] The caption says 'Comparison of MALT and MALT' but the text and figure compare MALT and MAST; the caption should be corrected.
- [Section 5.2] The sentence 'We observed that there is almost no difference between chosing the threshold on the set to be tested and cross-validating it' is surprising given the known sensitivity of threshold-based attacks; please clarify which set is used for the reported numbers and whether this observation was tested on all datasets or only in the logistic-regression setting.
- [Abstract and Section 1] The phrase 'optimal strategy' is used before the assumptions are stated; consider saying 'Bayes optimal under the Gibbs-posterior assumption' to avoid overstatement.
- [Section 4.1, Eq. (29)] The notation τ(z1) clashes with the earlier τ_p(z1) and with the constant τ in MALT; using distinct symbols for the calibration term, the soft-min term, and the constant threshold would improve readability.
Circularity Check
No circularity: the loss-only optimality result is an explicit algebraic consequence of the stated Gibbs-posterior assumption, and the practical attack thresholds are estimated from independent shadow/held-out data.
full rationale
The derivation chain is self-contained and does not re-import its conclusion as an input. Equation (2) explicitly assumes the trained parameters follow a Gibbs posterior proportional to exp(-(1/T) Σ m_i ℓ(θ,z_i)); Theorem 2 then computes the Bayes-optimal membership ratio for that model. The cancellation of the nuisance posterior p_T(θ) in the likelihood ratio (Eqs. 16-19) is a direct algebraic identity, so the 'depends only on the loss' property is a conditional theorem under the stated assumption rather than a fitted quantity renamed as a prediction. The practical methods estimate thresholds (constant τ for MALT, per-sample τ(z1) for MAST, and the Taylor score for MATT) from shadow models, held-out validation, or influence-function approximations with known membership status, and then apply them to a target model; no parameter fitted to the target membership is reported as a prediction. The paper's citations of Shokri et al. and Yeom et al. are used to compare or reinterpret existing attacks, not as load-bearing self-citations, and no uniqueness theorem from the authors' own prior work is invoked. The supplied three-point counterexample targets the law-of-total-expectation step in Theorem 1, arguing the expectation should be over P(T|θ,z1) rather than the prior P(T); that is a correctness or measure-theoretic concern about the proof, not a circularity. Under the hard rules, correctness objections are outside the circularity score, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- lambda (membership prior) =
0.5 in experiments
- Temperature T =
1 for Bayesian posterior, small for SGD
- MALT threshold tau =
Chosen on validation/test data; best threshold reported in Table 1
- MAST per-sample calibration tau(z1) =
Estimated from 30 shadow models in Section 6.3
assumptions (5)
- domain assumption Posterior over parameters has the Gibbs form P(theta|z,m) proportional to exp(-1/T sum m_i l(theta,z_i))
- domain assumption Membership indicators m_i are independent Bernoulli(lambda)
- domain assumption The prior on theta is uniform on a bounded set or folded into the loss
- ad hoc to paper Mean-field assumption for MAST: p_T(t) does not depend on T
- ad hoc to paper Laplace approximation for MATT: posterior is Gaussian with Hessian H, and H is roughly unchanged when z1 is removed
Cite this review
Pith. "Pith review of White-box vs Black-box: Bayes Optimal Strategies for Membership Inference." pith.science (2026). https://pith.science/paper/CDTVS64Z
@misc{pith2026190811229,
author = {Pith},
title = {Pith review of: White-box vs Black-box: Bayes Optimal Strategies for Membership Inference},
year = {2026},
howpublished = {\url{https://pith.science/paper/CDTVS64Z}},
note = {Machine review of arXiv:1908.11229}
}
read the original abstract
Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depend on the loss function, and thus black-box attacks are as good as white-box attacks. As the optimal strategy is not tractable, we provide approximations of it leading to several inference methods, and show that existing membership inference methods are coarser approximations of this optimal strategy. Our membership attacks outperform the state of the art in various settings, ranging from a simple logistic regression to more complex architectures and datasets, such as ResNet-101 and Imagenet.
Figures
Forward citations
Cited by 1 Pith paper
-
How much do language models memorize?
A compression-based measurement puts GPT-style model memorization capacity at roughly 3.6 bits per parameter, with membership inference success following a sigmoid in the dataset-to-capacity ratio.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[3]
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In CCS, 2016
work page 2016
-
[4]
Giuseppe Ateniese, Luigi V Mancini, Angelo Spognardi, Antonio Villani, Domenico Vitali, and Giovanni Felici. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. IJSN, 2015
work page 2015
-
[5]
Algorithmic stability for adaptive data analysis
Raef Bassily, Kobbi Nissim, Adam Smith, Thomas Steinke, Uri Stemmer, and Jonathan Ullman. Algorithmic stability for adaptive data analysis. In STOC, 2016
work page 2016
-
[6]
Battista Biggio, Igino Corona, Blaine Nelson, Benjamin I. P. Rubinstein, Davide Maiorca, Giorgio Fumera, Giorgio Giacinto, and Fabio Roli. Security Evaluation of Support Vector Machines in Adversarial Environments. Springer International Publishing, 2014
work page 2014
-
[7]
Unsupervised learning by predicting noise
Piotr Bojanowski and Armand Joulin. Unsupervised learning by predicting noise. In ICML, 2017
work page 2017
-
[8]
The secret sharer: Measuring unintended neural network memorization & extracting secrets
Nicholas Carlini, Chang Liu, Jernej Kos, \'U lfar Erlingsson, and Dawn Song. The secret sharer: Measuring unintended neural network memorization & extracting secrets. arXiv preprint arXiv:1802.08232, 2018
arXiv 2018
Show all 31 references
-
[9]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009
2009
-
[10]
Discriminative unsupervised feature learning with convolutional neural networks
Alexey Dosovitskiy, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox. Discriminative unsupervised feature learning with convolutional neural networks. In NIPS, 2014
2014
-
[11]
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006
2006
-
[12]
Robust traceability from trace amounts
Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. Robust traceability from trace amounts. In Proceedings of the Symposium on the Foundations of Computer Science, 2015
2015
-
[13]
Logan: evaluating privacy leakage of generative models using generative adversarial networks
Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. Logan: evaluating privacy leakage of generative models using generative adversarial networks. arXiv preprint arXiv:1705.07663, 2017
2017 arXiv
-
[14]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016
2016
-
[15]
Understanding black-box predictions via influence functions
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In ICML, 2017
2017
-
[16]
Kanwal, Tegan Maharaj, Emmanuel Bengio, Asja Fischer, Aaron Courville, Simon Lacoste-Julien, and Yoshua Bengio
David Krueger, Nicolas Ballas, Stanislaw Jastrzebski, Devansh Arpit, Maxinder S. Kanwal, Tegan Maharaj, Emmanuel Bengio, Asja Fischer, Aaron Courville, Simon Lacoste-Julien, and Yoshua Bengio. A closer look at memorization in deep networks. In ICML, 2017
2017
-
[17]
Kullback
S. Kullback. Information Theory And Statistics . Dover Publications, 1997
1997
-
[18]
Understanding membership inferences on well-generalized learning models
Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen. Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04889, 2018
2018 arXiv
-
[19]
B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. SIAM J. Control Optim., 1992
1992
-
[20]
Learning in a large function space: Privacy-preserving mechanisms for SVM learning
Benjamin IP Rubinstein, Peter L Bartlett, Ling Huang, and Nina Taft. Learning in a large function space: Privacy-preserving mechanisms for SVM learning. arXiv:0911.5708, 2009
2009 arXiv
-
[21]
Berg, and Li Fei-Fei
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge . IJCV, 2015
2015
-
[22]
Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models
Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In NCSS, 2019
2019
-
[23]
Jordan, and Eran Halperin
Sriram Sankararaman, Guillaume Obozinski, Michael I. Jordan, and Eran Halperin. Genomic privacy and limits of individual detection in a pool. Nature Genetics, 2009
2009
-
[24]
Membership inference attacks against machine learning models
Reza Shokri, Marco Stronati, and Vitaly Shmatikov. Membership inference attacks against machine learning models. IEEE Symp. Security and Privacy, 2017
2017
-
[25]
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2014
2014
-
[26]
A. W. van der Vaart. Asymptotic statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1998
1998
-
[27]
Privacy for free: Posterior sampling and stochastic gradient monte carlo
Yu-Xiang Wang, Stephen Fienberg, and Alex Smola. Privacy for free: Posterior sampling and stochastic gradient monte carlo. In ICML, 2015
2015
-
[28]
On-average KL -privacy and its equivalence to generalization for max-entropy mechanisms
Yu-Xiang Wang, Jing Lei, and Stephen E Fienberg. On-average KL -privacy and its equivalence to generalization for max-entropy mechanisms. In PSD. Springer, 2016
2016
-
[29]
Bayesian learning via stochastic gradient langevin dynamics
Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In ICML, 2011
2011
-
[30]
Privacy risk in machine learning: Analyzing the connection to overfitting
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In CSF, 2018
2018
-
[31]
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In ICLR, 2017
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.