Pith. sign in

REVIEW 2 major objections 5 minor 72 references

Covert Attacks on Machine Learning Training in Passively Secure MPC

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An active adversary in a passively secure MPC training run can add errors to secret-shared multiplications and, without being detected, poison the model, amplify membership inference, and reconstruct nearly exact training data.

desk verdict First credible demonstration that passive MPC training can be actively attacked via additive errors; results are plausible but the fixed-point transfer needs to be shown, not asserted. read the letter →

arxiv 2505.17092 v1 pith:GDTIBILG submitted 2025-05-21 cs.CR cs.LG

Matthew Jagielski , Daniel Escudero , Rahul Rachuri , Peter Scholl This is my paper · ORCID
classification cs.CRcs.LG MSC 68P2594A60
keywords securemultipartycomputationpassivesecurityactiveadversaryadditiveattacksmachinelearningpoisoningmembershipinferencetrainingdatareconstructiongradientmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Secure multiparty computation lets parties train on pooled data while keeping inputs secret; 'passive' security assumes corrupt parties follow the protocol, and it is often argued that active cheating is unlikely because it could be detected and hurt reputations. This paper claims that an active party in a passively secure MPC training protocol can, with essentially zero risk of detection, both corrupt the learned model and break its privacy guarantees. The attack surface is the ability to add a chosen error to each secret-shared multiplication, and the paper shows how these errors, applied to comparisons, sigmoids, softmaxes, and gradient multiplications, compose into gradient zeroing, gradient shifting, and gradient scaling. These produce backdoor and targeted poisoning, disproportionate accuracy loss for a targeted data contributor, large membership-inference amplification, and near-exact training-input reconstruction. The experiments run in a floating-point plaintext simulator of MPC, which is the paper's key idealization; if the results carry over to real fixed-point arithmetic, the reputation-based case for passive security in ML training collapses.

What carries the argument

The load-bearing mechanism is the additive attack on a secret-shared multiplication, formalized in an arithmetic black-box functionality where the adversary may supply an error $\epsilon$ after each product. The model-update step is linear and multiplication-free, so the attacks concentrate on gradient computations, which contain the protocol's only relevant multiplications. On top of this primitive, the paper layers low-level activation manipulations (flipping secure-comparison outputs, offsetting inputs by large constants, and tampering with reciprocal and exponentiation subroutines) and high-level gradient strategies (zeroing, shifting, scaling) that turn isolated errors into whole-model behavior. These layers are what let the adversary act without seeing secret-shared data or weights.

What would settle it

Run the attacks in an actual fixed-point MPC training protocol over $\mathbb{Z}_M$ with precision $f$, using the same datasets and the same error magnitudes, and compare the resulting model behaviors to the floating-point simulations. A decisive observation would be whether injected offsets like a shift of 10000 or a scaling of 10000 survive the truncation step after each multiplication and avoid wraparound modulo $M$; if the effective additive errors change materially, attack success rates and accuracy should be re-measured under the fixed-point arithmetic the paper abstracts away.

Watch

Extended reading notes

Core claim

The paper's central claim is that passively secure MPC training protocols contain a systematic exploitable gap: an active adversary who can place an additive error on every multiplication can implement strong adversarial-ML attacks without ever seeing the secret-shared data or weights. On the low level, the adversary flips outputs of secure comparisons, forces ReLU, piecewise-linear sigmoid, direct sigmoid, and softmax into chosen outputs, and tampers with reciprocal and exponentiation subroutines. On the high level, these primitives become gradient zeroing (removing an example's contribution), gradient shifting (adding a chosen vector to gradients, via parameter transfer or neuron override), and gradient scaling (multiplying gradients by a large scalar supplied by the loss derivative). The paper demonstrates near-perfect backdoor and targeted attacks on logistic regression and small neural networks with little accuracy loss, fairness attacks that reduce accuracy disproportionately for a targeted subpopulation, membership-inference true-positive-rate gains of up to about 40x at low false-positive rates, and reconstruction of almost exact training images (mean absolute error 0.017 of pixel range for logistic regression and 0.036 for neural networks), with the reconstruction attack destroying model utility.

Load-bearing premise

The load-bearing premise is that attacks measured in a floating-point plaintext simulator match what happens in real MPC training, which uses fixed-point arithmetic over a finite ring with truncation and possible wraparound; if large additive errors are truncated away or wrap the modulus, the attacks could fail or become detectable.

Editorial extensions

If this is right

  • If the paper is right, passively secure MPC training offers no meaningful protection against a malicious party; post-hoc accuracy checks and reputation arguments do not catch the attacks.
  • Computing sigmoid and softmax with active security is a cheap mitigation for the gradient-scaling family, since those activations are computed once per example and take a small fraction of training time in the direct-softmax implementations the paper targets.
  • Gradient-shifting attacks (parameter transfer for linear models, neuron override for small neural networks) plant backdoors and targeted misclassifications with near-100% success and at most a few percent accuracy loss on the evaluated datasets.
  • Gradient scaling can amplify membership inference by converting any example into an apparent outlier, raising true-positive rate at low false-positive rates by factors of roughly 2 to 40 on the tested models.
  • The same gradient-scaling mechanism, pushed to strength 10000 on the final batch, reconstructs a nearly exact training image; this is not stealthy, since accuracy drops to random-guess level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the attacks should be evaluated on dishonest-majority protocols too: the paper notes its manipulations apply to any linear secret-sharing scheme, so additive sharing over $\mathbb{Z}_M$ is a direct test bed.
  • I infer that a practical middle-ground defense for gradient shifting could be randomized spot-checking of multiplications: if the adversary injects $p$ errors and a fraction $\alpha$ of multiplications is checked, detection probability is $1-(1-\alpha)^p$, and the paper's attack uses so many errors that even tiny $\alpha$ would catch it; no such protocol is constructed.
  • I infer that moving from floating-point simulation to fixed-point arithmetic is likely to change the attacks quantitatively rather than eliminate them: moderate additive errors may be trimmed by truncation, while the large shifts used for forcing activations risk modulo wraparound, so an attacker would need to tune error magnitudes to the modulus.
  • I infer that gradient-scaling privacy attacks will transfer more readily than gradient-shifting poisoning attacks to larger models, because scaling exploits the loss-derivative multiplication common to all gradient computations, whereas shifting relies on gradients transferable across nonconvex networks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies active (malicious) adversaries in passively secure MPC-based machine learning training. The authors introduce low-level attacks on secure comparisons and activation functions (ReLU, piecewise and direct sigmoid/softmax), then compose them into three high-level gradient manipulation strategies: gradient zeroing, shifting, and scaling. They instantiate these into concrete attacks: backdoor and targeted poisoning via parameter transfer and neuron override, membership-inference amplification, training-data reconstruction, fairness degradation, and data-poisoning amplification. Experiments are run in a plaintext floating-point JAX simulator that mimics MPC-friendly operations. The paper argues that these attacks are simple, effective, and in most cases cause little to no model accuracy degradation, thereby challenging the view that passive security is sufficient for MPC training if active adversaries face reputation risk.

Significance. If the attacks transfer to real MPC training protocols, this is a significant result for the MPC and PPML communities: it provides concrete evidence that passively secure training protocols are vulnerable to active adversaries in ways that are both easy to mount and hard to detect by accuracy-based checks. The paper clearly organizes the attack surface (input modification, activation modification, combined modification) and proposes practical, cheap mitigations such as using actively secure sigmoid/softmax. The mathematical analysis of activation-function manipulations is careful under the arithmetic black-box model, and the authors promise open-source code, which aids reproducibility. However, the significance is conditional on the experimental simulator faithfully representing fixed-point MPC arithmetic, which is the paper's main unresolved gap.

major comments (2)
  1. [§2.2, §A.1 (Figure 4), §A.4, Limitations]
  2. [§5.4 (Figure 7), §A.1]
minor comments (5)
  1. [§5.3, Table 5, Abstract]
  2. [§5.2, §5.3, §5.6]
  3. [§A.3 vs §5]
  4. [§2.3, Figure 4]
  5. [Table 1]

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attacks are constructed from stated MPC arithmetic operations and validated against external datasets; the fixed-point assumption is explicit and does not reduce the derivation to its inputs.

full rationale

The paper's derivation chain is constructive rather than circular. It starts from the arithmetic black box in Appendix A.1 (Figure 4), where an adversary can inject an additive error after each multiplication. From this, Section 3 derives low-level manipulations of comparisons, ReLU, sigmoid, and softmax; Section 4 composes these into gradient zeroing, shifting, and scaling; Section 5 instantiates poisoning, membership inference, reconstruction, and fairness attacks. Each step follows from the protocol's stated operations, not from the conclusion being derived. The attack evaluations use standard external datasets (FMNIST, Census, Purchase, Texas) and held-out triggered or test examples, so the reported success rates are empirical demonstrations rather than quantities forced by fitting. Attack hyperparameters are tuned on the target data, but the paper's central claim is feasibility of the attacks, not a calibrated quantitative prediction; this is standard attack engineering, not circularity. The membership inference experiment measures deviation on the gradient-scaling target class, yet this is an explicit attack/evaluation design choice and the resulting TPRs are far from trivial (e.g., 2.3% to 76% at 0.1% FPR), so success is not guaranteed by construction. Self-citations, such as [44] for mixed-protocol MPC primitives and [49], [59], [60] for neural network and MI background, appear as context or inspiration and are not load-bearing for the paper's main claims. Finally, the acknowledged gap between floating-point simulation and fixed-point Z_M arithmetic is an external validity assumption, not a circular reduction: it concerns whether the attacks transfer to real MPC implementations, but does not make any derivation equivalent to its own inputs. No qualifying circularity pattern is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper's central feasibility claim rests on the F_MPC additive-error model, on the transferability of floating-point simulations to real fixed-point MPC, and on specific activation-function implementations. The attack hyperparameters listed above are fitted to make the demonstrations succeed, but they are not used to define the central claim.

free parameters (5)
  • Reconstruction gradient scaling strength C = 10000
    Used to overwrite model weights with a scaled gradient so that first-layer weights reveal the input example (Section 5.4).
  • MI gradient scaling strength = not reported; tuned
    Selected by "a small amount of tuning" to keep accuracy drop low while amplifying membership inference (Section 5.3).
  • Poisoning amplification softmax shift and number of poisoned examples = softmax shift -2 to -5; 2 to 10 flipped examples per target
    Tuned per dataset/model in Section 5.6 to produce weak unamplified attacks that scaling then strengthens.
  • Neuron override constants (delta, mu, c) = not fully specified
    Handcrafted in Section 5.2 using trigger direction and data mean; c is a hyperparameter trading off activation rates.
  • Availability shift gradient size = small (not specified)
    Used in Section 5.7 to achieve accuracy degradation without full collapse.
assumptions (4)
  • domain assumption An active adversary in passively secure MPC can inject an arbitrary additive error after each multiplication (the F_MPC arithmetic black box).
    This models the attack surface of honest-majority linear secret-sharing protocols, as described in Section 2.2 and Appendix A.1. The paper relies on this to construct all attacks.
  • domain assumption Fixed-point arithmetic over a finite ring Z_M with no wraparound, and floating-point simulation is a faithful proxy for it.
    Section 2.2 and Limitations state the paper works over floating point and assumes no significant impact from fixed-point approximation, citing [22,23]. This is load-bearing for the experimental validation.
  • domain assumption Passive protocols compute activation functions (comparison, ReLU, piecewise sigmoid, direct sigmoid/softmax) using the specific compositions described in Section 3.
    The attacks target these exact realizations (e.g., SecureML piecewise sigmoid, Keller-Sun exponentials), not the whole space of possible implementations.
  • standard math Linear secret-sharing schemes with robust reconstruction are used, and the adversary cannot read secret-shared values.
    Background from Section 2.2 and Appendix A.1; this determines that manipulations must be data-independent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Covert Attacks on Machine Learning Training in Passively Secure MPC." pith.science (2026). https://pith.science/paper/GDTIBILG

@misc{pith2026250517092,
  author       = {Pith},
  title        = {Pith review of: Covert Attacks on Machine Learning Training in Passively Secure MPC},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDTIBILG}},
  note         = {Machine review of arXiv:2505.17092}
}
read the original abstract

Secure multiparty computation (MPC) allows data owners to train machine learning models on combined data while keeping the underlying training data private. The MPC threat model either considers an adversary who passively corrupts some parties without affecting their overall behavior, or an adversary who actively modifies the behavior of corrupt parties. It has been argued that in some settings, active security is not a major concern, partly because of the potential risk of reputation loss if a party is detected cheating. In this work we show explicit, simple, and effective attacks that an active adversary can run on existing passively secure MPC training protocols, while keeping essentially zero risk of the attack being detected. The attacks we show can compromise both the integrity and privacy of the model, including attacks reconstructing exact training data. Our results challenge the belief that a threat model that does not include malicious behavior by the involved parties may be reasonable in the context of PPML, motivating the use of actively secure protocols for training.

Figures

Figures reproduced from arXiv: 2505.17092 by the authors.

Figure 1
Figure 1. MPC logistic regression gradient Here, ΠMatMul performs matrix multi￾plication, Πsigmoid implements the sig￾moid function in MPC, and ΠElemMul performs elementwise multiplication. This algorithm can be easily extended to k-class classification for k > 2, by replacing w with a d×k matrix, the bi￾ases by a k-dimensional vector, and re￾placing Πsigmoid with Πsoftmax, a mul￾ticlass extension of sigmoid that we will desc… view at source ↗
Figure 2
Figure 2. Gradient scaling leads to successful reconstruction attacks on logistic regression. The top [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Gradient scaling attacks permit successful reconstruction attacks on neural networks. The [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: MPC arithmetic black box for fixed-point arithmetic modulo [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Parameter Transfer Attack from Section 5.1 [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Neuron Override Attack from Section 5.2 Scaling for Reconstruction Attacks Input: Attacker protocol inputs: Attacker target class (one hot vector) y, scaling strength C. Secret shared protocol inputs: Secret shared batch XB, YB, secret shared model weights θi = (wi , b…
Figure 7
Figure 7. Figure 7: Gradient Scaling Attack from Section 5.4 [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 60 canonical work pages

  1. [1]

    SecureML: A system for scalable privacy-preserving machine learning,

    P. Mohassel and Y . Zhang, “SecureML: A system for scalable privacy-preserving machine learning,” in2017 IEEE Symposium on Security and Privacy. IEEE Computer Society Press, 13 May 2017, pp. 19–38

    work page 2017

  2. [2]

    SecureNN: 3-party secure computation for neural network training,

    S. Wagh, D. Gupta, and N. Chandran, “SecureNN: 3-party secure computation for neural network training,”PoPETs, no. 3, pp. 26–49, Jul. 2019

    work page 2019

  3. [3]

    Falcon: Honest- majority maliciously secure framework for private deep learning,

    S. Wagh, S. Tople, F. Benhamouda, E. Kushilevitz, P. Mittal, and T. Rabin, “Falcon: Honest- majority maliciously secure framework for private deep learning,”PoPETs, no. 1, pp. 188–208, Jan. 2021

    work page 2021

  4. [4]

    Secure quantized training for deep learning,

    M. Keller and K. Sun, “Secure quantized training for deep learning,” Cryptology ePrint Archive, Report 2022/933, 2022. [Online]. Available: https://eprint.iacr.org/2022/933

    work page 2022

  5. [5]

    Cryptographically private support vector machines,

    S. Laur, H. Lipmaa, and T. Mielik ¨ainen, “Cryptographically private support vector machines,” Cryptology ePrint Archive, Report 2006/198, 2006. [Online]. Available: https://eprint.iacr.org/2006/198

    work page 2006

  6. [6]

    ABY - A framework for efficient mixed-protocol secure two-party computation,

    D. Demmler, T. Schneider, and M. Zohner, “ABY - A framework for efficient mixed-protocol secure two-party computation,” inNDSS 2015. The Internet Society, Feb. 2015

    work page 2015

  7. [7]

    Cerebro: A platform for multi-party cryptographic collaborative learning,

    W. Zheng, R. Deng, W. Chen, R. A. Popa, A. Panda, and I. Stoica, “Cerebro: A platform for multi-party cryptographic collaborative learning,” inUSENIX Security 2021. USENIX Association, Aug. 2021, pp. 2723–2740

    work page 2021

  8. [8]

    Manticore: Efficient framework for scalable secure multiparty computation protocols,

    S. Carpov, K. Deforth, N. Gama, M. Georgieva, D. Jetchev, J. Katz, I. Leontiadis, M. Mohammadi, A. Sae-Tang, and M. Vuille, “Manticore: Efficient framework for scalable secure multiparty computation protocols,” Cryptology ePrint Archive, Report 2021/200, 2021. [Online]. Available: https://eprint.iacr.org/2021/200

    work page 2021

Show all 72 references
  1. [9]

    Private collaborative neural network learning,

    M. Chase, R. Gilad-Bachrach, K. Laine, K. Lauter, and P. Rindal, “Private collaborative neural network learning,” Cryptology ePrint Archive, Report 2017/762, 2017. [Online]. Available: https://eprint.iacr.org/2017/762

    2017

  2. [10]

    BOLT: Privacy-preserving, accurate and efficient inference for transformers,

    Q. Pang, J. Zhu, H. M¨ollering, W. Zheng, and T. Schneider, “BOLT: Privacy-preserving, accurate and efficient inference for transformers,” in2024 IEEE Symposium on Security and Privacy. IEEE Computer Society Press, May 2024, pp. 4753–4771

    2024

  3. [11]

    Iron: Private inference on transformers,

    M. Hao, H. Li, H. Chen, P. Xing, G. Xu, and T. Zhang, “Iron: Private inference on transformers,” inAdvances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 15 718–15 ...

    2022

  4. [12]

    Sharemind: A framework for fast privacy-preserving computations,

    D. Bogdanov, S. Laur, and J. Willemson, “Sharemind: A framework for fast privacy-preserving computations,” inESORICS 2008, ser. LNCS. Springer, Berlin, Heidelberg, Oct. 2008, pp. 192–206

    2008

  5. [13]

    “Syft,” 2023, https://github.com/OpenMined/PySyft

    2023

  6. [14]

    Private machine learning in tensorflow using secure computation,

    M. Dahl, J. Mancuso, Y . Dupis, B. Decoste, M. Giraud, I. Livingstone, J. Patriquin, and G. Uhma, “Private machine learning in tensorflow using secure computation,”CoRR, vol. abs/1810.08130, 2018. [Online]. Available: http://arxiv.org/abs/1810.08130

    2018 arXiv

  7. [15]

    Mpyc: Multiparty computation in python,

    B. Schoenmakers, “Mpyc: Multiparty computation in python,” 2023, https://github.com/lschoe/ mpyc

    2023

  8. [17]

    CryptGPU: Fast privacy-preserving machine learning on the GPU,

    S. Tan, B. Knott, Y . Tian, and D. J. Wu, “CryptGPU: Fast privacy-preserving machine learning on the GPU,” in2021 IEEE Symposium on Security and Privacy. IEEE Computer Society Press, May 2021, pp. 1021–1038

    2021

  9. [18]

    Flash boys 2.0: Frontrunning in decentralized exchanges, miner extractable value, and consensus instability,

    P. Daian, S. Goldfeder, T. Kell, Y . Li, X. Zhao, I. Bentov, L. Breidenbach, and A. Juels, “Flash boys 2.0: Frontrunning in decentralized exchanges, miner extractable value, and consensus instability,” in2020 IEEE Symposium on Security and Privacy. IEEE Computer Society Press,...

    2020

  10. [19]

    Auditing differentially private machine learning: How private is private sgd?

    M. Jagielski, J. Ullman, and A. Oprea, “Auditing differentially private machine learning: How private is private sgd?”Advances in Neural Information Processing Systems, vol. 33, pp. 22 205–22 216, 2020. 14

    2020

  11. [20]

    Adversary instantiation: Lower bounds for differentially private machine learning,

    M. Nasr, S. Songi, A. Thakurta, N. Papernot, and N. Carlin, “Adversary instantiation: Lower bounds for differentially private machine learning,” in2021 IEEE Symposium on security and privacy (SP). IEEE, 2021, pp. 866–882

    2021

  12. [21]

    Security against covert adversaries: Efficient protocols for realistic adversaries,

    Y . Aumann and Y . Lindell, “Security against covert adversaries: Efficient protocols for realistic adversaries,” inTCC 2007, ser. LNCS. Springer, Berlin, Heidelberg, Feb. 2007, pp. 137–156

    2007

  13. [22]

    CrypTFlow: Secure TensorFlow inference,

    N. Kumar, M. Rathee, N. Chandran, D. Gupta, A. Rastogi, and R. Sharma, “CrypTFlow: Secure TensorFlow inference,” in2020 IEEE Symposium on Security and Privacy. IEEE Computer Society Press, May 2020, pp. 336–353

    2020

  14. [23]

    Truncation untangled: Scaling fixed-point arithmetic for privacy-preserving machine learning to large models and datasets,

    C. Harth-Kitzerow, A. Suresh, and G. Carle, “Truncation untangled: Scaling fixed-point arithmetic for privacy-preserving machine learning to large models and datasets,” Cryptology ePrint Archive, Paper 2024/1953, 2024. [Online]. Available: https://eprint.iacr.org/2024/1953

    2024

  15. [24]

    Circuits resilient to additive attacks with applications to secure computation,

    D. Genkin, Y . Ishai, M. Prabhakaran, A. Sahai, and E. Tromer, “Circuits resilient to additive attacks with applications to secure computation,” in46th ACM STOC. ACM Press, May / Jun. 2014, pp. 495–504

    2014

  16. [25]

    Exploiting machine learning to subvert your spam filter

    B. Nelson, M. Barreno, F. J. Chi, A. D. Joseph, B. I. Rubinstein, U. Saini, C. Sutton, J. D. Tygar, and K. Xia, “Exploiting machine learning to subvert your spam filter.”LEET, vol. 8, no. 1-9, pp. 16–17, 2008

    2008

  17. [26]

    How to backdoor federated learning,

    E. Bagdasaryan, A. Veit, Y . Hua, D. Estrin, and V . Shmatikov, “How to backdoor federated learning,” inInternational conference on artificial intelligence and statistics. PMLR, 2020, pp. 2938–2948

    2020

  18. [27]

    Poisoning attacks against support vector machines,

    B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks against support vector machines,”arXiv preprint arXiv:1206.6389, 2012

    2012 arXiv

  19. [28]

    Manipulating ma- chine learning: Poisoning attacks and countermeasures for regression learning,

    M. Jagielski, A. Oprea, B. Biggio, C. Liu, C. Nita-Rotaru, and B. Li, “Manipulating ma- chine learning: Poisoning attacks and countermeasures for regression learning,” in2018 IEEE symposium on security and privacy (SP). IEEE, 2018, pp. 19–35

    2018

  20. [29]

    Badnets: Identifying vulnerabilities in the machine learning model supply chain,

    T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnerabilities in the machine learning model supply chain,”arXiv preprint arXiv:1708.06733, 2017

    2017 arXiv

  21. [30]

    Targeted backdoor attacks on deep learning systems using data poisoning,

    X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Targeted backdoor attacks on deep learning systems using data poisoning,”arXiv preprint arXiv:1712.05526, 2017

    2017 arXiv

  22. [31]

    When does machine learning FAIL? Generalized transferability for evasion and poisoning attacks,

    O. Suciu, R. Marginean, Y . Kaya, H. Daum´e, III, and T. Dumitras, “When does machine learning FAIL? Generalized transferability for evasion and poisoning attacks,” inUSENIX Security 2018. USENIX Association, Aug. 2018, pp. 1299–1316

    2018

  23. [32]

    Poison frogs! targeted clean-label poisoning attacks on neural networks,

    A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer, T. Dumitras, and T. Goldstein, “Poison frogs! targeted clean-label poisoning attacks on neural networks,”Advances in neural information processing systems, vol. 31, 2018

    2018

  24. [33]

    Subpopulation data poisoning at- tacks,

    M. Jagielski, G. Severi, N. Pousette Harger, and A. Oprea, “Subpopulation data poisoning at- tacks,” inProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021, pp. 3104–3122

    2021

  25. [34]

    Membership inference attacks against machine learning models,

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” in2017 IEEE Symposium on Security and Privacy. IEEE Computer Society Press, May 2017, pp. 3–18

    2017

  26. [35]

    Privacy risk in machine learning: Analyzing the connection to overfitting,

    S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” inCSF 2018 Computer Security Foundations Symposium. IEEE Computer Society Press, 2018, pp. 268–282

    2018

  27. [36]

    Membership inference attacks from first principles,

    N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 1897–1914

    2022

  28. [37]

    Enhanced membership inference attacks against machine learning models,

    J. Ye, A. Maddi, S. K. Murakonda, V . Bindschaedler, and R. Shokri, “Enhanced membership inference attacks against machine learning models,” inProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2022, pp. 3093–3106

    2022

  29. [38]

    Extracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingssonet al., “Extracting training data from large language models,” in30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 2633–2650. 15

    2021

  30. [39]

    Reconstructing training data with informed adversaries,

    B. Balle, G. Cherubin, and J. Hayes, “Reconstructing training data with informed adversaries,” in2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 1138–1156

    2022

  31. [40]

    A note on interpreting canary exposure,

    M. Jagielski, “A note on interpreting canary exposure,”arXiv preprint arXiv:2306.00133, 2023

    2023 arXiv

  32. [41]

    Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy,

    G. Kaissis, J. Hayes, A. Ziller, and D. Rueckert, “Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy,”arXiv preprint arXiv:2307.03928, 2023

    2023 arXiv

  33. [42]

    Muse: Secure inference resilient to malicious clients,

    R. Lehmkuhl, P. Mishra, A. Srinivasan, and R. A. Popa, “Muse: Secure inference resilient to malicious clients,” inUSENIX Security 2021. USENIX Association, Aug. 2021, pp. 2201–2218

    2021

  34. [43]

    SafeNet: Mitigating data poisoning attacks on private machine learning,

    H. Chaudhari, M. Jagielski, and A. Oprea, “SafeNet: Mitigating data poisoning attacks on private machine learning,” Cryptology ePrint Archive, Report 2022/663, 2022. [Online]. Available: https://eprint.iacr.org/2022/663

    2022

  35. [44]

    Improved primitives for MPC over mixed arithmetic-binary circuits,

    D. Escudero, S. Ghosh, M. Keller, R. Rachuri, and P. Scholl, “Improved primitives for MPC over mixed arithmetic-binary circuits,” inCRYPTO 2020, Part II, ser. LNCS. Springer, Cham, Aug. 2020, pp. 823–852

    2020

  36. [45]

    ABY3: A mixed protocol framework for machine learning,

    P. Mohassel and P. Rindal, “ABY3: A mixed protocol framework for machine learning,” in ACM CCS 2018. ACM Press, Oct. 2018, pp. 35–52

    2018

  37. [46]

    Benchmarking privacy preserving scientific operations,

    A. Aly and N. P. Smart, “Benchmarking privacy preserving scientific operations,” Cryptology ePrint Archive, Report 2019/354, 2019. [Online]. Available: https://eprint.iacr.org/2019/354

    2019

  38. [47]

    Property inference attacks on fully connected neural networks using permutation invariant representations,

    K. Ganju, Q. Wang, W. Yang, C. A. Gunter, and N. Borisov, “Property inference attacks on fully connected neural networks using permutation invariant representations,” inProceedings of the 2018 ACM SIGSAC conference on computer and communications security, 2018, pp. 619–633

    2018

  39. [48]

    Reverse-engineering deep ReLU networks,

    D. Rolnick and K. Kording, “Reverse-engineering deep ReLU networks,” inProceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, H. D. III and A. Singh, Eds., vol. 119. PMLR, 13–18 Jul 2020, pp. 8178–8187. [Online]. Av...

    2020

  40. [49]

    High accuracy and high fidelity extraction of neural networks,

    M. Jagielski, N. Carlini, D. Berthelot, A. Kurakin, and N. Papernot, “High accuracy and high fidelity extraction of neural networks,” in29th USENIX security symposium (USENIX Security 20), 2020, pp. 1345–1362

    2020

  41. [50]

    JAX: composable transformations of Python+NumPy programs,

    J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, “JAX: composable transformations of Python+NumPy programs,” 2018. [Online]. Available: http://github.com/google/jax

    2018

  42. [51]

    Analyzing federated learning through an adversarial lens,

    A. N. Bhagoji, S. Chakraborty, P. Mittal, and S. Calo, “Analyzing federated learning through an adversarial lens,” inInternational Conference on Machine Learning. PMLR, 2019, pp. 634–643

    2019

  43. [52]

    Editing models with task arithmetic,

    G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,”arXiv preprint arXiv:2212.04089, 2022

    2022 arXiv

  44. [53]

    Handcrafted backdoors in deep neural networks,

    S. Hong, N. Carlini, and A. Kurakin, “Handcrafted backdoors in deep neural networks,”Ad- vances in Neural Information Processing Systems, vol. 35, pp. 8068–8080, 2022

    2022

  45. [54]

    When the curious abandon honesty: Federated learning is not private,

    F. Boenisch, A. Dziedzic, R. Schuster, A. S. Shamsabadi, I. Shumailov, and N. Papernot, “When the curious abandon honesty: Federated learning is not private,” in2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P). IEEE, 2023, pp. 175–199

    2023

  46. [55]

    To be forgotten or to be fair: Unveiling fairness implications of machine unlearning methods,

    D. Zhang, S. Pan, T. Hoang, Z. Xing, M. Staples, X. Xu, L. Yao, Q. Lu, and L. Zhu, “To be forgotten or to be fair: Unveiling fairness implications of machine unlearning methods,”arXiv preprint arXiv:2302.03350, 2023

    2023 arXiv

  47. [56]

    Poisoning attacks on algorithmic fairness,

    D. Solans, B. Biggio, and C. Castillo, “Poisoning attacks on algorithmic fairness,” inJoint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2020, pp. 162–177

    2020

  48. [57]

    On adversarial bias and the robustness of fair machine learning,

    H. Chang, T. D. Nguyen, S. K. Murakonda, E. Kazemi, and R. Shokri, “On adversarial bias and the robustness of fair machine learning,”arXiv preprint arXiv:2006.08669, 2020

    2006 arXiv

  49. [58]

    Amplifying membership exposure via data poisoning,

    Y . Chen, C. Shen, Y . Shen, C. Wang, and Y . Zhang, “Amplifying membership exposure via data poisoning,”Advances in Neural Information Processing Systems, vol. 35, pp. 29 830–29 844, 2022. 16

    2022

  50. [59]

    Truth serum: Poisoning machine learning models to reveal their secrets,

    F. Tram`er, R. Shokri, A. San Joaquin, H. Le, M. Jagielski, S. Hong, and N. Carlini, “Truth serum: Poisoning machine learning models to reveal their secrets,” inProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2022, pp. 2779–2792

    2022

  51. [60]

    The privacy onion effect: Memorization is relative,

    N. Carlini, M. Jagielski, C. Zhang, N. Papernot, A. Terzis, and F. Tramer, “The privacy onion effect: Memorization is relative,”Advances in Neural Information Processing Systems, vol. 35, pp. 13 263–13 276, 2022

    2022

  52. [61]

    Cryptography secure against related-key attacks and tampering,

    M. Bellare, D. Cash, and R. Miller, “Cryptography secure against related-key attacks and tampering,” inASIACRYPT 2011, ser. LNCS. Springer, Berlin, Heidelberg, Dec. 2011, pp. 486–503

    2011

  53. [62]

    Related-key security for pseudorandom functions beyond the linear barrier,

    M. Abdalla, F. Benhamouda, A. Passel `egue, and K. G. Paterson, “Related-key security for pseudorandom functions beyond the linear barrier,” inCRYPTO 2014, Part I, ser. LNCS. Springer, Berlin, Heidelberg, Aug. 2014, pp. 77–94

    2014

  54. [63]

    Learning representations by back- propagating errors,

    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back- propagating errors,”nature, vol. 323, no. 6088, pp. 533–536, 1986. A Instantiating the Attacks in MPC A.1 MPC Over Fixed-Point Arithmetic An MPC protocol is often modelled abstractly as a sec...

    1986

  55. [65]

    Protocol:

    Locally computeG w =∇ w(θP , XP , YP ). Protocol:

  56. [67]

    Compute probabilityP= Π sigmoid(D)

  57. [69]

    Figure 5: Parameter Transfer Attack from Section 5.1 Neuron Override Input:Trigger δ, data mean µ, mean scaling α, target neuron index j

    Finally, compute∇ wℓ= Π ElemMul(F, XB)+Gw and∇ bℓ=F. Figure 5: Parameter Transfer Attack from Section 5.1 Neuron Override Input:Trigger δ, data mean µ, mean scaling α, target neuron index j. Secret shared batch XB, YB, secret shared model weightsθ i = (W 0 i , B0 i , W1 i , B1...

  58. [70]

    Figure 6: Neuron Override Attack from Section 5.2 Scaling for Reconstruction Attacks Input:Attacker protocol inputs: Attacker target class (one hot vector) y, scaling strength C

    Following standard backpropagation, adversary shifts hidden layer derivative ∇W 1 ℓ−G W 1 and first layer derivative∇ W 0 ℓ−G W 0. Figure 6: Neuron Override Attack from Section 5.2 Scaling for Reconstruction Attacks Input:Attacker protocol inputs: Attacker target class (one ho...

  59. [71]

    Compute predictionD= Π MatMul(XB, wi) +b i

  60. [72]

    Compute probabilityP= Π softmax(D)+C·y

  61. [73]

    Compute scalar loss derivativeF=P−Y B

  62. [74]

    After Protocol (Reconstruction):

    Finally, compute∇ wℓ= Π ElemMul(F , XB)and∇ bℓ=F. After Protocol (Reconstruction):

  63. [75]

    flip” parameters will flip the results of comparisons made in the MPC sigmoid. To flip only one element of a batch, the corresponding “flip

    Attack returnsw y, a reconstructed input. Figure 7: Gradient Scaling Attack from Section 5.4 allows us to be faithful to the MPC-friendly approximations while simultaneously using Jax to autodifferentiate and accelerate our code on GPU. 19 In the implementation of each functio...

  64. [2021]

    Available: https://arxiv.org/abs/2109.00984

    [Online]. Available: https://arxiv.org/abs/2109.00984

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.