Pith. sign in

REVIEW 4 major objections 4 minor 34 references

Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read The number of adversarial attack iterations reveals whether a sample was in the model's training set.

desk verdict Genuinely new feature, but the paper's own motivation contradicts the mechanism and no sensitivity analysis backs up the claim of reliability; worth a serious referee but not acceptance as-is. read the letter →

arxiv 2506.02711 v1 pith:Q7ESIA24 submitted 2025-06-03 cs.CR

classification cs.CR
keywords membershipinferenceattackadversarialexamplesiterationcountprivacyleakageblack-boxwhite-boxlabel-onlymetric-based
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a membership inference attack that reads a sample's training-set status not from the model's confidence or loss, but from how many iterations an adversarial attack needs to fool the model on that sample. The claim is that member samples take more attack iterations than non-members, so a single threshold on the iteration count separates the two populations in black-box and white-box settings. Because the attack needs no training data and no shadow models, it is far cheaper than distribution-based membership inference. Across three image datasets and four model architectures, the paper reports that this iteration feature matches or beats confidence-based, boundary-distance, and loss-based baselines, and in some white-box cases wins by large margins.

What carries the argument

The machinery is the iteration counter attached to three existing adversarial attacks: SimBA for score-based black-box access, HopSkipJumpAttack for label-only black-box access, and PGD for white-box access. IMIA runs the chosen attack, records the number of iterations until it produces an adversarial example, and applies a threshold to that count. The single feature carries the entire argument: it must separate members from non-members across attack algorithms, model architectures, and datasets without needing training data or shadow models.

What would settle it

A control experiment that matches member and non-member samples on prediction confidence and distance to the decision boundary, or repeats the attacks across many random seeds, would show whether iteration counts carry independent membership signal or merely re-encode sample difficulty and attack randomness.

Watch

Extended reading notes

Core claim

The central discovery is the iteration gap: when generating adversarial examples with SimBA, HopSkipJumpAttack, or PGD, member samples consistently require more iterations than non-member samples, even though the paper describes members as lying closer to the decision boundary. IMIA formalizes this into a metric-based attack: run the appropriate adversarial strategy, count the iterations until a successful adversarial example is found, and classify the sample as a member when the count exceeds a threshold. The paper presents this as a universal, lightweight alternative to soft-output metrics, because the same rule works when only hard labels are available, and it argues that the iteration count exposes a privacy leak that confidence and distance features only partially capture.

Load-bearing premise

The attack assumes that the extra iterations members require are caused by membership itself, not by sample difficulty, attack randomness, or the particular hyperparameters chosen for the adversarial algorithm.

Editorial extensions

If this is right

  • Membership inference can be conducted with a single scalar (attack iteration count) without training data or shadow models.
  • The same attack rule transfers across access levels by swapping the adversarial algorithm: score-based black-box, label-only black-box, and white-box.
  • Models that expose only hard labels remain vulnerable to a metric-based membership inference attack, which soft-output baselines cannot cover.
  • An adversary can run the attack on a single target sample, making it usable when only one record's membership is at stake.
  • The reported white-box gains imply that gradient-based attack trajectories carry membership signal beyond the loss baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If iteration counts separate members only because they proxy confidence or boundary distance, IMIA would be a roundabout re-expression of existing metrics; a direct test would compare members and non-members matched on those quantities.
  • The same iteration-count feature may transfer to other domains such as text or tabular data, where gradient-based and decision-based adversarial attacks already exist.
  • One testable extension is that defenses which shrink the iteration gap, such as adversarial training or output smoothing, would lower IMIA's AUROC and could serve as privacy defenses.
  • The choice of query budget and threshold are themselves attack parameters; future work could treat them as tunable variables that trade attack cost against inference accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes IMIA, a membership inference attack that uses the number of iterations required by an adversarial attack (SimBA, HopSkipJumpAttack, or PGD) to generate an adversarial example from a target sample as a membership signal. Experiments on CIFAR10, CIFAR100, and STL10 with four architectures compare IMIA against score-based baselines (softmax response, prediction entropy, modified entropy), a label-only boundary-distance baseline, and the white-box loss baseline. The authors report AUROC and accuracy with standard deviations, claiming that the iteration count is a reliable and universal membership feature across black-box, label-only, and white-box settings. No code is provided.

Significance. If established, the idea of using adversarial-iteration counts as a membership signal is interesting and potentially lightweight, since IMIA avoids shadow models and training data. The experimental coverage is broad: three datasets, four architectures, and three attack settings, with standard deviations reported. However, the central claim of reliability is not yet supported: the motivation contains an internal contradiction, the threshold-selection protocol is missing, the iteration counter is defined by hand-tuned and uncensored attack hyperparameters, and the method underperforms baselines in a nontrivial number of configurations. The contribution is incremental over existing boundary-distance and confidence-based attacks unless these issues are resolved.

major comments (4)
  1. [Section 3.1] The motivating statement says that 'member samples, being closer to the decision boundary, generally require more iterations to generate adversarial examples than non-member samples.' This is internally contradictory: samples closer to the boundary should require fewer iterations to cross it, and the boundary-distance baseline of Choquette-Choo et al. (reference [12]) treats a larger boundary distance as the member signal. If members are actually farther from the boundary, then IMIA's iteration count is essentially a proxy for boundary distance or confidence; if members are closer, the expected sign of the iteration effect is reversed. The paper must reconcile this statement with the empirical iteration histogram and with the boundary-distance baseline.
  2. [Tables 1-3] The inference-accuracy results in Tables 1, 2, and 3 depend on the threshold τ in Algorithm 1, but the manuscript never specifies how τ is selected for the reported accuracy numbers. Without a protocol (for example, choosing τ on a separate calibration set or reporting the maximum attainable accuracy on the evaluation set), the accuracy comparisons are not reproducible and may be optimistically biased. Please specify the exact threshold-selection rule used for every table.
  3. [Appendix A and Section 3.2] The iteration count is defined by attack hyperparameters that are not varied, so the observed signal may be an artifact of the configuration. For SimBA (Table 4), max_iters=300 with an unbounded L∞ bound and random basis order means that samples never flipping within 300 iterations are censored, mixing genuine convergence with budget exhaustion. For HSJA (Table 5), the paper sets num_iterations=100 but does not state which counter is actually measured as the 'number of iterations.' For PGD (Sec. 3.2), 50 steps with α=0.001 and a random start mean the first successful iteration is heavily influenced by initialization and step size. Without a sensitivity analysis over epsilon, max_iters, step size, and random seeds, the reported AUROC gains may not be stable. Provide such an analysis or justify why the ranking is invariant.
  4. [Tables 1-2] The claim that the iteration count is a 'reliable feature' is weakened by the results themselves. In Table 1, IMIA is below Modified Entropy in AUROC for CIFAR100-VGG (67.36 vs 69.61), STL10-ResNeXt (69.25 vs 70.84), STL10-VGG (61.32 vs 62.68), and CIFAR10-ResNeXt (73.34 vs 73.54), among others; in Table 2, IMIA underperforms the boundary baseline in several comparisons, e.g., CIFAR100-ResNet AUROC 85.95 vs 86.12 and CIFAR10-ResNeXt AUROC 69.77 vs 70.43. The paper should either temper the universality claim or analyze the conditions under which the iteration feature helps versus hurts.
minor comments (4)
  1. [Section 3.2] In the SimBA optimization formulation, the notation 'd' and 'M' is used without definition, and 'queries ≤ M' is vague; please define these quantities precisely.
  2. [Figure 1] Figure 1(a) is described as a scatter diagram of boundary distance versus iteration count, but the text does not interpret what the reader should conclude from it; Figure 1(b) uses 2k samples per class, which differs from the 3k samples used in the evaluation sets, so the figure should clarify this discrepancy.
  3. [Section 4.3] The sentence 'in the white-box setting, We choose' has an erroneous capital 'W' after a comma; please correct this and similar grammatical issues throughout the manuscript.
  4. [General] No code or public release is mentioned; given the demonstrated hyperparameter sensitivity of the iteration counter, providing code or detailed per-seed results would materially aid reproducibility and verification.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: IMIA's iteration-count feature is measured from standard adversarial attacks and evaluated with swept thresholds; the central claim does not reduce to its own inputs.

full rationale

The derivation chain is self-contained. In Algorithm 1 the feature is defined as the output of an external adversarial attack: 'τt ← S(fθ, (xt, yt))', and the attack strategies (SimBA, HSJA, PGD) are pre-existing methods whose iteration counts are not defined in terms of membership. The decision rule in Algorithm 1 is a threshold comparison, 'return 1(τt ≥ τ)', and the paper reports AUROC and accuracy over thresholds on balanced member/non-member sets; no parameter is fitted to membership labels in order to construct the iteration-count feature. The Section 3.1 claim that 'member samples, being closer to the decision boundary, generally require more iterations to generate adversarial examples than non-member samples' is an empirical histogram observation, not a definitional consequence of the paper's equations, and the later evaluation sets are separate from the motivating figure. There are no load-bearing self-citations: the cited attack papers and MIA baselines are external works. The closest concern is Appendix A's statement that the SimBA perturbation size was 'set a small value for observing the number of iterations during the adversarial samples’ generation process'; this is hyperparameter selection rather than an equation-level reduction of the predicted quantity to an input. The internally strained motivation (members 'closer to the boundary' yet needing 'more iterations') and the absence of sensitivity analysis are correctness and robustness limitations, not circularity, because no predicted result is identical by construction to a fitted input or to a self-citation chain.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The method introduces no new entities or defensive mechanisms. It relies on existing adversarial attacks and the empirical premise that iteration counts track membership. The main hand-chosen components are the attack hyperparameters and the selection of iteration count as the feature.

free parameters (3)
  • SimBA perturbation size epsilon = 0.05
    Chosen by hand to be 'small' so that iteration counts during adversarial generation are observable (Appendix A, Table 4). The discriminative power of the iteration count depends on this value.
  • HSJA number of iterations = 100
    Fixed iteration budget for HopSkipJumpAttack; it bounds the recorded iteration count and affects the membership signal (Appendix A, Table 5).
  • PGD step budget and step size = 50 steps, alpha=0.001, epsilon=3/255
    White-box attack configuration determines the number of iterations recorded by IMIA (Appendix A).
assumptions (2)
  • domain assumption The number of adversarial iterations is a stable, membership-correlated quantity under the chosen attack configurations.
    Sections 3.1 and 3.2 assume the iteration count is informative; no stability or theoretical analysis is provided.
  • domain assumption Members and non-members are comparable in input difficulty, so iteration differences reflect membership rather than sample hardness or attack randomness.
    The evaluation does not control for input difficulty or repeated attack runs, yet attributes iteration differences to membership.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack." pith.science (2026). https://pith.science/paper/Q7ESIA24

@misc{pith2026250602711,
  author       = {Pith},
  title        = {Pith review of: Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q7ESIA24}},
  note         = {Machine review of arXiv:2506.02711}
}
read the original abstract

Membership inference attack (MIA) has become one of the most widely used and effective methods for evaluating the privacy risks of machine learning models. These attacks aim to determine whether a specific sample is part of the model's training set by analyzing the model's output. While traditional membership inference attacks focus on leveraging the model's posterior output, such as confidence on the target sample, we propose IMIA, a novel attack strategy that utilizes the process of generating adversarial samples to infer membership. We propose to infer the member properties of the target sample using the number of iterations required to generate its adversarial sample. We conduct experiments across multiple models and datasets, and our results demonstrate that the number of iterations for generating an adversarial sample is a reliable feature for membership inference, achieving strong performance both in black-box and white-box attack scenarios. This work provides a new perspective for evaluating model privacy and highlights the potential of adversarial example-based features for privacy leakage assessment.

Figures

Figures reproduced from arXiv: 2506.02711 by the authors.

Figure 1
Figure 1. (a) Scatter diagram showing the relationship between the distance from samples to the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Diagrammatic sketch for IMIA to conduct MIA. 3.2 Methodology Given the target model and the target images, the adversary can choose an adversarial strategy S in SimBA [17], HopSkipJumpAttack [10] and PGD [18] based on different MIA settings to generate adversarial samples and measure the number of iterations during this process. Score-based black-box attacks. Adversary can obtain the full probability output of the t… view at source ↗
Figure 3
Figure 3. ROC curve on MIA for the combination of different models on CIFAR100, CIFAR10 and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 18 canonical work pages

  1. [12]

    Label- only membership inference attacks

    Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label- only membership inference attacks. In International conference on machine learning , pages 1964–1974. PMLR, 2021

  2. [1]

    Superdriverai: Towards design and implementation for end-to-end learning-based autonomous driving

    Shunsuke Aoki, Issei Yamamoto, Daiki Shiotsuka, Yuichi Inoue, Kento Tokuhiro, and Keita Miwa. Superdriverai: Towards design and implementation for end-to-end learning-based autonomous driving. In 2023 IEEE V ehicular Networking Conference (VNC), pages 195–198. IEEE, 2023

  3. [2]

    Scalable membership inference attacks via quantile regression

    Martin Bertran, Shuai Tang, Aaron Roth, Michael Kearns, Jamie H Morgenstern, and Steven Z Wu. Scalable membership inference attacks via quantile regression. Advances in Neural Information Processing Systems, 36, 2024

  4. [3]

    Towards evaluating the robustness of neural networks

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , pages 39–57. Ieee, 2017

  5. [4]

    The secret sharer: Evaluating and testing unintended memorization in neural networks

    Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19) , pages 267–284, 2019

  6. [5]

    Extracting training data from large language models

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-V oss, Kather- ine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2633–2650, 2021

  7. [6]

    Membership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 2022

  8. [7]

    Quantifying memorization across neural language models

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646, 2022

Show all 34 references
  1. [8]

    Forecasting stock market crisis events using deep and statistical machine learning techniques

    Sotirios P Chatzis, Vassilis Siakoulis, Anastasios Petropoulos, Evangelos Stavroulakis, and Nikos Vlachogiannakis. Forecasting stock market crisis events using deep and statistical machine learning techniques. Expert systems with applications , 112:353–371, 2018

  2. [9]

    Chameleon: Increasing label-only membership leakage with adaptive poisoning

    Harsh Chaudhari, Giorgio Severi, Alina Oprea, and Jonathan Ullman. Chameleon: Increasing label-only membership leakage with adaptive poisoning. arXiv preprint arXiv:2310.03838 , 2023

  3. [10]

    Hopskipjumpattack: A query-efficient decision-based attack

    Jianbo Chen, Michael I Jordan, and Martin J Wainwright. Hopskipjumpattack: A query-efficient decision-based attack. In 2020 ieee symposium on security and privacy (sp) , pages 1277–1294. IEEE, 2020

  4. [11]

    Amplifying membership exposure via data poisoning

    Yufei Chen, Chao Shen, Yun Shen, Cong Wang, and Yang Zhang. Amplifying membership exposure via data poisoning. Advances in Neural Information Processing Systems , 35:29830– 29844, 2022

  5. [13]

    Privacy side channels in machine learning systems

    Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher A Choquette-Choo, Matthew Jagielski, Milad Nasr, Eric Wallace, and Florian Tramèr. Privacy side channels in machine learning systems. In 33rd USENIX Security Symposium (USENIX Security 24) , pages 6861–6848, 2024. 10

  6. [14]

    Leveraging adversarial examples to quantify membership information leakage

    Ganesh Del Grosso, Hamid Jalalzai, Georg Pichler, Catuscia Palamidessi, and Pablo Piantanida. Leveraging adversarial examples to quantify membership information leakage. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10399–10409, 2022

  7. [15]

    Risk assessment for hospital readmissions: Insights from machine learning algorithms

    Rohit R Dixit. Risk assessment for hospital readmissions: Insights from machine learning algorithms. Sage Science Review of Applied Machine Learning , 4(2):1–15, 2021

  8. [16]

    Explaining and harnessing adversar- ial examples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014

  9. [17]

    Simple black-box adversarial attacks

    Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. Simple black-box adversarial attacks. In International conference on machine learning , pages 2484–2493. PMLR, 2019

  10. [18]

    Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Aleksander Madry. Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

  11. [19]

    Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning

    Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP) , pages 739–753. IEEE, 2019

  12. [20]

    Scalable extraction of training data from (production) language models

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ip- polito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311...

  13. [21]

    Recite, reconstruct, recollect: Memorization in lms as a multifaceted phenomenon

    USVSN Sai Prashanth, Alvin Deng, Kyle O’Brien, Jyothir SV , Mohammad Aflah Khan, Jaydeep Borkar, Christopher A Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, et al. Recite, reconstruct, recollect: Memorization in lms as a multifaceted phenomenon. arXiv preprint a...

  14. [22]

    Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models

    Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246, 2018

  15. [23]

    Machine learning as an early warning system to predict financial crisis

    Aristeidis Samitas, Elias Kampouris, and Dimitris Kenourgios. Machine learning as an early warning system to predict financial crisis. International Review of Financial Analysis , 71: 101507, 2020

  16. [24]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017

  17. [25]

    Systematic evaluation of privacy risks of machine learning models

    Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2615–2632, 2021

  18. [26]

    Privacy risks of securing machine learning models against adversarial examples

    Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pages 241–257, 2019

  19. [27]

    Understanding practical membership privacy of deep learning

    Marlon Tobaben, Gauri Pradhan, Yuan He, Joonas Jälkö, and Antti Honkela. Understanding practical membership privacy of deep learning. arXiv preprint arXiv:2402.06674, 2024

  20. [28]

    The space of transferable adversarial examples

    Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv:1704.03453, 2017

  21. [29]

    Truth serum: Poisoning machine learning models to reveal their se- crets

    Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth serum: Poisoning machine learning models to reveal their se- crets. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security...

  22. [30]

    On the importance of difficulty calibration in membership inference attacks

    Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440, 2021. 11

  23. [31]

    You only query once: An efficient label-only membership inference attack

    Yutong Wu, Han Qiu, Shangwei Guo, Jiwei Li, and Tianwei Zhang. You only query once: An efficient label-only membership inference attack. In The Twelfth International Conference on Learning Representations

  24. [32]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018

  25. [33]

    Machine learning-based vehicle intention trajectory recognition and prediction for autonomous driving

    Hanyi Yu, Shuning Huo, Mengran Zhu, Yulu Gong, and Yafei Xiang. Machine learning-based vehicle intention trajectory recognition and prediction for autonomous driving. In 2024 7th International Conference on Advanced Algorithms and Control Engineering (ICAACE) , pages 771–775. ...

  26. [34]

    Low-cost high-power membership inference attacks

    Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-power membership inference attacks. arXiv preprint arXiv:2312.03262, 2023. 12 A Implementation details for each adversarial strategy In Table 4, we show the implementation details for the Simple Black-box Attack (...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.