{"id":"f0dac6a2-53aa-4880-95b4-1e36634a4432","arxiv_id":"1908.11229","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Under a Gibbs posterior assumption, the Bayes-optimal membership inference attack depends only on the model's loss, and simple loss-threshold attacks outperform shadow models.","lead":"Under a standard Bayesian model of training, this paper derives the mathematically optimal way to tell whether an image was in a model's training set, and shows that it depends only on the model's loss on that image. This means black-box access (querying the model) is as powerful as white-box access (seeing all internal parameters), and simple loss-based attacks beat previous shadow-model attacks.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2 marginalizes T under its prior, not the posterior T|θ,z1; a three-point counterexample with equal losses gives different exact membership posteriors, falsifying the loss-only/white-box-equals-black-box claim even under Eq. (2).","rationale":"The reader's weakest assumption concerns whether real SGD or Bayesian training produces the Gibbs posterior of Eq. (2); that is a legitimate external-validity concern. However, the more serious problem is internal: even granting Eq. (2), the proof of Theorem 1/2 does not compute the Bayes posterior P(m1|θ,z1). The law of total expectation requires averaging over T conditional on θ,z1, because θ is generated from a distribution that depends on T and m1. The paper averages over the prior P(T), which drops the information θ carries about the other memberships. Under the prior, the integrand is loss-only; under the correct posterior, the averaging measure depends on the full parameter θ. The three-point counterexample makes this concrete: θ=a and θ=b have identical loss on z1 but exact posteriors 0.600 and 0.643, while the paper's formula yields the same value 0.633 for both. Therefore the claimed 'optimal' attack is not optimal, and there is a white-box advantage (distinguishing a from b) even in the paper's own idealized model. This is the single load-bearing concern because it directly invalidates the main theoretical contribution and the justification for the practical attacks' optimality. It changes the verdict from conditional acceptance to rejection of the central theoretical claim, even though the empirical methods might still be useful heuristics.","tokens_in":13039,"tokens_out":22752,"duration_ms":198621,"concrete_test":"Write a short script that enumerates the n=2, three-parameter counterexample above. For each θ∈{a,b,c}, compute the exact posterior P(m1=1|θ,z1) by summing over m2∈{0,1} using Eq. (2) with the specified losses. Then compute Theorem 2's expression by averaging σ(log(P(θ|m1=1,z1,m2)/P(θ|m1=0,z1,m2))) over the prior on m2. Check whether two θ with equal l(θ,z1) have different exact posteriors, and whether Theorem 2's value matches either exact posterior. If the answer is yes, the theorem is falsified as stated.","verdict_should_be":"REJECT","load_bearing_attack":"The proof of Theorem 1 (Eqs. 5-6) writes M(θ,z1)=E_T[P(m1=1|θ,z1,T)] as the law of total expectation, but T={z2..zn,m2..mn} is not independent of the observed θ: θ is drawn from P(θ|z1,T,m1), so the correct marginalization is over T|θ,z1, not the prior P(T). Consequently, Theorem 2's score s(z1,θ,p_T) is averaged under the wrong measure. The exact Bayes likelihood ratio is R(θ)=E_T[ e^{-l(θ,z1)/T} p_T(θ)/Z_T(z1) ] / E_T[p_T(θ)], which generally depends on θ beyond l(θ,z1). Concrete counterexample: take Θ={a,b,c}, fixed z1,z2, T=1, λ=1/2, and losses l(a,z1)=0, l(b,z1)=0, l(c,z1)=10; l(a,z2)=10, l(b,z2)=0, l(c,z2)=0. Exact enumeration gives P(m1=1|θ=a,z1)=0.600 and P(m1=1|θ=b,z1)=0.643, although l(a,z1)=l(b,z1)=0. Theorem 2's formula gives the same value for both (≈0.633), matching neither exact posterior. Thus the proposed attack is not Bayes optimal, and the white-box/no-benefit conclusion fails even under the paper's own Eq. (2).","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Bayes-optimal framework for membership inference under the Gibbs posterior assumption P(θ|z1,...,zn,m) ∝ exp(−(1/T)Σ m_i ℓ(θ,z_i)). Its main theoretical claim, Theorem 2, is that the optimal membership score depends on the model parameters only through the loss ℓ(θ,z1), implying that black-box and white-box attacks are asymptotically equivalent. The paper then derives three practical attack families (MALT, MAST, MATT) as approximations of this score and reports strong empirical results on logistic regression, a small CNN on CIFAR-10, and VGG-16/ResNet-101 on ImageNet, outperforming the 0-1 baseline and shadow-model attacks in several settings.","tokens_in":13401,"tokens_out":5266,"duration_ms":51510,"significance":"If the theoretical claim were correct, the paper would establish a clean and surprising equivalence between white-box and black-box membership inference, with direct implications for privacy accounting and attack design. The proposed threshold-based attacks (especially MALT) are simple, reproducible in spirit, and the empirical comparisons on ImageNet with data augmentation are informative and useful. However, the central theoretical result is not correct as stated: the proof of Theorem 1 marginalizes over the auxiliary training information T under the wrong measure, and a concrete counterexample under the paper's own Eq. (2) shows that exact membership posteriors can differ for two parameters with identical loss. Because the loss-only optimality and the white-box/black-box equivalence are load-bearing for the paper's conceptual contribution, the significance of the manuscript as a theoretical statement is not established.","major_comments":[{"comment":"The law of total expectation is applied incorrectly. In M(θ,z1) = E_T[P(m1=1|θ,z1,T)], the expectation must be taken over the conditional distribution P(T|θ,z1), because θ is drawn from P(θ|z1,T,m1) and hence is statistically dependent on T. The proof instead averages the likelihood ratio under the marginal distribution of T, which is not the posterior membership probability. This error propagates to Theorem 2 and invalidates the claimed loss-only optimality.","section":"Section 3.3, Theorem 1 proof, Eqs. (5)-(6)"},{"comment":"The loss-only claim is falsified by a finite counterexample under Eq. (2). Take Θ={a,b,c}, fixed z1,z2, T=1, λ=1/2, and losses ℓ(a,z1)=0, ℓ(b,z1)=0, ℓ(c,z1)=10; ℓ(a,z2)=10, ℓ(b,z2)=0, ℓ(c,z2)=0. Exact enumeration gives P(m1=1|θ=a,z1)=0.600 and P(m1=1|θ=b,z1)=0.643, although ℓ(a,z1)=ℓ(b,z1)=0. Theorem 2's formula assigns the same value (≈0.633) to both cases, matching neither exact posterior. Thus the proposed attack is not Bayes optimal under Eq. (2), and the conclusion that white-box access provides no benefit does not follow.","section":"Section 3.3, Theorem 2"},{"comment":"The definitions of τ are inconsistent. Eq. (14) defines τ_p(z1) = −T log ∫ e^{−ℓ(t,z1)/T} p(t) dt, while Eq. (29) defines τ(z1) = log ∫ e^{−ℓ(t,z1)/T} p(t) dt, dropping the factor −T. The score in Eq. (30) is then s = −ℓ + τ, which differs from the Theorem 2 score (τ_p − ℓ)/T by both a sign and a scale. The Gaussian derivation in Eqs. (33)-(34) and (46) treats τ as approximately the negative log-integral, not the log-integral. This inconsistency affects all subsequent MALT/MAST thresholds and should be corrected before the empirical scores can be linked to Theorem 2.","section":"Section 4.1, Eqs. (14) and (29)"},{"comment":"The logistic-regression comparison reports 'the peak accuracy obtained for the best threshold.' If this threshold is selected using the test set or the same samples being evaluated, the reported attack accuracies are optimistic and the comparison between MALT and MATT is not a fair estimate of deployable attack performance. Section 5.2 says the threshold is estimated on simulated sets, but the table caption and Section 5.2 should specify unambiguously whether the reported accuracy is the oracle best threshold or a threshold chosen on separate data.","section":"Section 6.2, Table 1"},{"comment":"Even setting aside the marginalization error, the paper's optimality result is conditional on the strong assumption that real training procedures sample parameters from the Gibbs posterior in Eq. (2). For SGD-trained deep networks this is not established, so the transfer of Theorem 2's conclusions to the ImageNet experiments is an extrapolation rather than a consequence of the theorem. This is not by itself a reason to reject, but it should be stated more carefully if the theoretical claims are revised.","section":"Section 3.1, Eq. (1)-(2)"}],"minor_comments":[{"comment":"The caption says 'Comparison of MALT and MALT' but the text and figure compare MALT and MAST; the caption should be corrected.","section":"Figure 1 caption"},{"comment":"The sentence 'We observed that there is almost no difference between chosing the threshold on the set to be tested and cross-validating it' is surprising given the known sensitivity of threshold-based attacks; please clarify which set is used for the reported numbers and whether this observation was tested on all datasets or only in the logistic-regression setting.","section":"Section 5.2"},{"comment":"The phrase 'optimal strategy' is used before the assumptions are stated; consider saying 'Bayes optimal under the Gibbs-posterior assumption' to avoid overstatement.","section":"Abstract and Section 1"},{"comment":"The notation τ(z1) clashes with the earlier τ_p(z1) and with the constant τ in MALT; using distinct symbols for the calibration term, the soft-min term, and the constant threshold would improve readability.","section":"Section 4.1, Eq. (29)"}],"recommendation":"reject","confidential_remarks":"The counterexample in the stress-test note is correct and directly refutes the paper's central theorem under its own assumptions. Because the loss-only property and the white-box/black-box equivalence are the paper's main theoretical contribution, this is not a local fixable issue: a corrected derivation would have to average over P(T|θ,z1), and that posterior depends on θ beyond ℓ(θ,z1), so the headline conclusion is unlikely to survive in the form presented. The empirical attack results may still have value as heuristics, but as submitted the manuscript's central claim is unsupported. I would not invite a minor revision; a major revision would require replacing or substantially weakening the main theorem and re-evaluating what the paper claims to prove."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know this paper has a load-bearing flaw in its central theorem. The proof of Theorem 1 uses the law of total expectation as M(θ,z1)=E_T[P(m1=1|θ,z1,T)], but T is not independent of θ given z1. The correct marginalization is over T|θ,z1. A simple three-parameter counterexample with two parameters having the same loss gives different exact membership posteriors (0.600 vs 0.643), so the loss-only claim is false even under the paper's own Gibbs model. I checked the counterexample and it is valid. This means the \"black-box equals white-box\" conclusion, as stated, does not follow.\n\nThat said, the paper is not without merit. The Gibbs-posterior setup is a clean way to formalize training randomness, and the three attack approximations (MALT, MAST, MATT) are sensible. The observation that shadow models are essentially Monte Carlo estimates of the calibration term is nice. MALT is basically Yeom et al.'s loss-threshold attack, but the per-sample threshold idea (MAST) and the Taylor variant (MATT) are original. The ImageNet experiments are also useful: MALT gets ~90% accuracy without augmentation, which is a strong practical result.\n\nThe soft spots beyond the theorem: Eq. (29) is inconsistent with Eq. (14) in sign and scaling of τ; the logistic-regression results in Table 1 report \"peak accuracy\" with the threshold chosen on the data, which inflates the numbers; the shadow-model comparison in Table 2 borrows numbers from other papers rather than a controlled rerun; and the citation list omits Nasr et al. (2018) on white-box attacks. None of these would be fatal on their own, but combined they make the empirical claims softer than the text suggests.\n\nWho should read this? Anyone working on membership inference attacks will find the practical recipe useful, and the error is a good teaching example of why marginalizing over latent training variables has to be done with care. But the theory should not be cited as establishing loss-only optimality.\n\nMy recommendation: send it to peer review—the error is subtle and the empirical part deserves scrutiny—but a careful referee will need to insist on fixing or reframing the theoretical claim. As it stands, the central conclusion is unsupported.","headline":"The paper's main theoretical claim—that optimal membership inference depends only on the loss—is invalidated by a marginalization error in Theorem 1; the empirical attacks are still worth a look.","tokens_in":13942,"tokens_out":9892,"would_cite":false,"duration_ms":84055,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Under a Gibbs-posterior model of training, the optimal membership inference attack depends on the model only through the loss, so white-box access to parameters gives no asymptotic advantage over black-box loss access.","keywords":["membership inference","Bayes optimal attack","white-box vs black-box","loss function","Gibbs posterior","shadow models","differential privacy"],"falsifier":"Train a fixed architecture on a fixed dataset many times with different random seeds or batch orders, holding out one fixed sample $z_1$ in half the runs; for each trained parameter vector $\\theta$, record its loss $\\ell(\\theta,z_1)$ and whether $z_1$ was in the training set. Estimate the empirical likelihood ratio $P(\\theta\\mid z_1\\in\\text{train})/P(\\theta\\mid z_1\\notin\\text{train})$. If there exist two parameter vectors with the same loss that yield substantially different likelihood ratios, then the loss is not a sufficient statistic for membership and Theorem 2's black-box/white-box equivalence fails for that training procedure.","tokens_in":1911,"feed_emoji":"🎯","tokens_out":2495,"duration_ms":79605,"temperature":0.7,"pith_summary":"This paper asks when an attacker can tell whether a given sample was used to train a model, and derives the Bayes-optimal answer under a simple probabilistic model of training: parameters are sampled proportionally to the exponential of the negative summed training loss divided by a temperature. The central result is that the optimal membership score for a sample is its loss under the trained model compared to a per-sample threshold, so the model matters only through the loss. Consequently the paper concludes that the white-box setting provides no benefit compared to black-box membership inference, asymptotically. The paper then turns this rule into three practical attacks (MALT, MAST, MATT) and reports that they outperform shadow-model baselines on logistic regression, CIFAR-10, and ImageNet. If the model of training holds, the practical takeaway is that membership privacy is governed by loss values rather than by the accessibility of model internals.","feed_headline":"The best membership attack needs only the loss function","feed_subtitle":"White-box access to model parameters adds no edge over black-box loss, theory says.","key_machinery":"The load-bearing object is the Gibbs posterior over parameters, $P(\\theta\\mid z_1,\\dots,z_n,m_1,\\dots,m_n)\\propto \\exp\\left(-\\frac{1}{T}\\sum_{i=1}^n m_i \\ell(\\theta,z_i)\\right)$, combined with the score function of Theorem 2: $s(z_1,\\theta,p_T)=\\frac{1}{T}(\\tau_{p_T}(z_1)-\\ell(\\theta,z_1))$. Here $\\tau_{p_T}(z_1)$ is the typical loss a model drawn from the posterior would assign to $z_1$ if $z_1$ were not in the training set, so the score is a calibrated loss comparison that turns membership inference into a one-dimensional decision. This machinery makes the loss the only model-dependent quantity, and its tractable approximations (MALT, MAST, MATT) are what the paper evaluates experimentally.","core_discovery":"Theorem 2 shows that under the Gibbs posterior of Equation (2), the Bayes-optimal membership inference score for a sample $z_1$ is $s(z_1,\\theta,p_T)=\\frac{1}{T}(\\tau_{p_T}(z_1)-\\ell(\\theta,z_1))$, where $\\tau_{p_T}(z_1)$ is the soft-minimum (softmin) of the loss over typical parameters that have never seen $z_1$. Because the trained parameters $\\theta$ enter this score only through the loss $\\ell(\\theta,z_1)$, the paper concludes that, asymptotically, white-box access to internal parameters adds no information over black-box access to the loss. The paper further argues that existing membership inference methods are coarser approximations of this optimal score: shadow models estimate the expectation over training sets, and the simple 0-1 attack is a threshold on the loss that ignores per-sample calibration.","pith_inferences":["If the loss is a sufficient statistic for membership, then membership attacks become largely architecture-agnostic: the same loss-threshold recipe should transfer across model families, and practitioners may not need to train per-architecture shadow models.","A natural controlled test of the theory is to train models with stochastic gradient Langevin dynamics, which provably samples the Gibbs posterior; in that setting the loss-only optimality should hold exactly and parameter-based white-box attacks should show no advantage.","At finite training-set sizes the expectation over $T$ in Theorem 2 is only approximated, so parameter-dependent statistics (such as influence functions or gradient norms) may still add information in small-$n$ regimes; the paper's asymptotic claim leaves this possibility open.","The per-sample threshold $\\tau(z_1)$ used by MAST could be estimated more cheaply than training dozens of shadow models, for example by using a single model with multiple dropout draws or a small ensemble, which would make the attack more practical at scale."],"forward_implications":["Defenders should treat loss values as the primary privacy signal: mechanisms that shrink the gap between training and validation losses, such as data augmentation and regularization, directly reduce membership inference accuracy, as the paper's ImageNet experiments confirm.","Any white-box attack that exploits gradients or hidden activations can at best match, not beat, a tuned loss-threshold attack in the asymptotic regime defined by the Gibbs posterior.","The paper's MALT rule, which flags any sample whose loss falls below a global threshold, outperforms shadow models on CIFAR-10 with a small convolutional network and on ImageNet with ResNet-101 and VGG-16 in the reported settings.","Under $\\epsilon$-differential privacy, the posterior membership probability is bounded by $\\lambda + \\epsilon/4$, giving a concrete operational meaning to $\\epsilon$: even a value as small as $0.01$ is a strong membership-privacy guarantee.","The three proposed attacks form a hierarchy of approximations, with MATT using a Taylor/influence-function expansion and outperforming the threshold-based MALT on a logistic regression problem."],"supporting_citations":[{"why":"Defines the shadow-model framework that is the black-box state-of-the-art baseline the paper must beat, and is later reinterpreted as a Monte-Carlo estimation of the expectation over training sets.","marker":"Shokri et al., 2017"},{"why":"Introduces the 0-1 and loss-based attacks and the overfitting-to-privacy connection, and provides a baseline that the paper shows to be a coarser approximation of the optimal score.","marker":"Yeom et al., 2018"},{"why":"Motivates the Gibbs-posterior assumption in Equation (1) by showing that stochastic gradient Langevin dynamics samples from such a posterior.","marker":"Welling & Teh, 2011"},{"why":"Supplies the differential privacy definition that the paper adapts into its membership-privacy guarantees and the $\\epsilon$-dependent bounds in Properties 1 and 2.","marker":"Dwork et al., 2006"},{"why":"Provides the asymptotic statistics used to justify the Laplace/Gaussian approximation of the posterior that underlies the MATT attack.","marker":"van der Vaart, 1998"},{"why":"Supplies the influence-function Taylor expansion for how the optimal parameters shift when a sample is removed, which is the core of the MATT score.","marker":"Koh & Liang, 2017"}],"fun_headline_variants":["Best membership attack: loss only, no weights needed","White-box edge vanishes in optimal membership test","Optimal membership inference ignores model internals","Loss-based attack matches white-box access","Why black-box beats white-box for membership"],"cache_read_input_tokens":16000,"weakest_assumption_plain":"The training algorithm must actually produce parameters according to the Gibbs posterior $P(\\theta\\mid\\text{data})\\propto \\exp(-\\frac{1}{T}\\sum_i m_i \\ell(\\theta,z_i))$; if real SGD or Bayesian inference does not sample from this distribution, then the conclusion that the loss is the only informative signal and that white-box access is useless does not transfer to actual models.","fun_headline_variants_meta":{"raw":{"variants":["Best membership attack: loss only, no weights needed","White-box edge vanishes in optimal membership test","Optimal membership inference ignores model internals","Loss-based attack matches white-box access","Why black-box beats white-box for membership"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000464,"raw_usage":{"total_tokens":2267,"prompt_tokens":842,"completion_tokens":1425,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":1358}},"tokens_in":458,"tokens_out":1425,"duration_ms":8754,"temperature":1.0,"reasoning_tokens":1358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:21:58.239975+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a fixed architecture on a fixed dataset many times with different random seeds or batch orders, holding out one fixed sample $z_1$ in half the runs; for each trained parameter vector $\\theta$, record its loss $\\ell(\\theta,z_1)$ and whether $z_1$ was in the training set. Estimate the empirical likelihood ratio $P(\\theta\\mid z_1\\in\\text{train})/P(\\theta\\mid z_1\\notin\\text{train})$. If there exist two parameter vectors with the same loss that yield substantially different likelihood ratios, then the loss is not a sufficient statistic for membership and Theorem 2's black-box/white-box equivalence fails for that training procedure.","supporting_citations":[{"cited_title":"Membership inference attacks against machine learning models","cited_arxiv_id":null,"evidence_quote":"Defines the shadow-model framework that is the black-box state-of-the-art baseline the paper must beat, and is later reinterpreted as a Monte-Carlo estimation of the expectation over training sets."},{"cited_title":"Privacy risk in machine learning: Analyzing the connection to overfitting","cited_arxiv_id":null,"evidence_quote":"Introduces the 0-1 and loss-based attacks and the overfitting-to-privacy connection, and provides a baseline that the paper shows to be a coarser approximation of the optimal score."},{"cited_title":"Bayesian learning via stochastic gradient langevin dynamics","cited_arxiv_id":null,"evidence_quote":"Motivates the Gibbs-posterior assumption in Equation (1) by showing that stochastic gradient Langevin dynamics samples from such a posterior."}],"review_version":1}