pith. sign in

arxiv: 2302.05545 · v4 · submitted 2023-02-10 · 💻 cs.CR · cs.IT· cs.LG· math.IT

Privacy Against Agnostic Inference Attacks in Vertical Federated Learning

Morteza Varasteh This is my paper

Pith reviewed 2026-05-24 09:15 UTC · model grok-4.3

classification 💻 cs.CR cs.ITcs.LGmath.IT
keywords vertical federated learninginference attackprivacy preserving schemesagnostic attacklogistic regressionactive partypassive partymodel distortion
0
0 comments X

The pith

The active party in vertical federated learning can perform agnostic inference attacks on the passive party's samples by using an independently trained model on its own features.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

In vertical federated learning, two parties train a model together, with one holding labels and the other holding separate features. The paper demonstrates that the label-holding active party can infer information about the passive party's features by training its own model without access to the joint model scores. This agnostic attack applies to both training data and new prediction samples. The authors also propose adjustable privacy schemes that distort the passive party's model parameters to reduce the attack's effectiveness while preserving overall model utility.

Core claim

The active party can carry out inference attacks on both training and prediction phase samples by acquiring an ML model independently trained on the training samples available to them. This type of inference attack does not require the active party to be aware of the score of a specific sample. Privacy-preserving schemes are proposed that systematically distort the VFL parameters corresponding to the passive party's features, with an adjustable level that trades off privacy and interpretability.

What carries the argument

The agnostic inference attack, carried out via an independently trained model by the active party on its available features, combined with adjustable parameter distortion as a countermeasure.

If this is right

  • The attack succeeds without needing the joint model's confidence scores for specific samples.
  • Using observed confidence scores from the prediction phase can improve the attack performance.
  • The proposed distortion schemes allow tuning the balance between passive party privacy and active party interpretability.
  • Experimental results confirm the attack works and the countermeasures reduce it while keeping utility.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the attack generalizes beyond logistic regression, similar risks may exist in other VFL models.
  • Parties may need to negotiate distortion levels based on their specific privacy and utility needs.
  • Future defenses could focus on preventing the active party from training effective independent models.

Load-bearing premise

The active party's independently trained model can still capture relevant patterns about the samples even without the passive party's features.

What would settle it

An experiment showing that the inference accuracy of the active party's independent model is equivalent to random chance on the passive party's feature values.

Figures

Figures reproduced from arXiv: 2302.05545 by Morteza Varasteh.

Figure 1
Figure 1. Figure 1: Example of vertical federated learning to evaluate whether to approve a user’s credit card application by incorporating more features from a Fintech company. The bank holds features of ‘age’ and ‘income’ while the Fintech company holds features of ‘deposit’ and ‘average online shopping times’. Only the bank owns the label information in the training dataset and testing dataset, i.e., the ground truth that … view at source ↗
Figure 2
Figure 2. Figure 2: Empirical mean, variance, and mean of the means of features. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of MSE per feature obtained from half estimation, agnostic inference [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: MSE per feature obtained from agnostic inference attack via [PITH_FULL_IMAGE:figures/full_fig_p032_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: MSE per feature resulted from agnostic inference attack via [PITH_FULL_IMAGE:figures/full_fig_p034_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Heat map of covariance matrix for the datasets in Table I [PITH_FULL_IMAGE:figures/full_fig_p035_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: MSE per feature resulted from agnostic inference attack via [PITH_FULL_IMAGE:figures/full_fig_p035_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Accuracy of AM versus active features for the datasets in Table I [PITH_FULL_IMAGE:figures/full_fig_p036_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Illustration of PI trade-off for case i and case iv for the datasets bank and adult [PITH_FULL_IMAGE:figures/full_fig_p037_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Illustration of PI trade-off for case iii for the datasets bank, adult, satellite and [PITH_FULL_IMAGE:figures/full_fig_p038_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Illustration of PI trade-off for case i for the datasets satellite and pendigits. [PITH_FULL_IMAGE:figures/full_fig_p039_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: MSE resulted from inference attack when PPSs (with different interpretability levels) [PITH_FULL_IMAGE:figures/full_fig_p040_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: MSE per feature of inference attack versus different values of [PITH_FULL_IMAGE:figures/full_fig_p040_13.png] view at source ↗
read the original abstract

A novel form of inference attack in vertical federated learning (VFL) is proposed, where two parties collaborate in training a machine learning (ML) model. Logistic regression is considered for the VFL model. One party, referred to as the active party, possesses the ground truth labels of the samples in the training phase, while the other, referred to as the passive party, only shares a separate set of features corresponding to these samples. It is shown that the active party can carry out inference attacks on both training and prediction phase samples by acquiring an ML model independently trained on the training samples available to them. This type of inference attack does not require the active party to be aware of the score of a specific sample, hence it is referred to as an agnostic inference attack. It is shown that utilizing the observed confidence scores during the prediction phase, before the time of the attack, can improve the performance of the active party's autonomous ML model, and thus improve the quality of the agnostic inference attack. As a countermeasure, privacy-preserving schemes (PPSs) are proposed. While the proposed schemes preserve the utility of the VFL model, they systematically distort the VFL parameters corresponding to the passive party's features. The level of the distortion imposed on the passive party's parameters is adjustable, giving rise to a trade-off between privacy of the passive party and interpretabiliy of the VFL outcomes by the active party. The distortion level of the passive party's parameters could be chosen carefully according to the privacy and interpretabiliy concerns of the passive and active parties, respectively, with the hope of keeping both parties (partially) satisfied. Finally, experimental results demonstrate the effectiveness of the proposed attack and the PPSs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that in vertical federated learning using logistic regression, an active party holding labels can mount agnostic inference attacks on both training and prediction-phase samples by training an independent model solely on its own features and labels. It further proposes adjustable privacy-preserving schemes that distort the passive party's parameters to reduce attack effectiveness while preserving VFL utility, with a tunable trade-off between passive-party privacy and active-party interpretability. Experiments are presented to show attack success and PPS effectiveness.

Significance. If the attack premise and defense efficacy hold under standard VFL conditions, the work identifies a new inference vector that does not require score access and supplies practical, tunable countermeasures. This would be relevant for VFL deployments where feature asymmetry is the norm.

major comments (2)
  1. [§4] §4 (attack construction): the agnostic attack requires that an independent model trained on active-party features alone recovers enough signal to infer labels or properties despite the systematic absence of passive features. No condition, bound, or ablation is supplied for the regime in which passive features carry substantial predictive power—the usual justification for VFL—leaving the central effectiveness claim without support when the autonomous model is weak.
  2. [§6] §6 (experimental evaluation): the reported attack results omit baselines for conventional inference attacks, data-exclusion rules, and error analysis or statistical significance tests. Without these, it is not possible to determine whether the observed attack performance is attributable to the agnostic mechanism or to experimental artifacts, directly affecting assessment of the central claim.
minor comments (2)
  1. [Abstract] Abstract: 'interpretabiliy' is misspelled.
  2. [§3] Notation for the independent model and the VFL model should be distinguished more clearly to avoid reader confusion between the two training processes.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the major comments below and commit to revisions that strengthen the manuscript without altering its core claims.

read point-by-point responses

  1. Referee: [§4] §4 (attack construction): the agnostic attack requires that an independent model trained on active-party features alone recovers enough signal to infer labels or properties despite the systematic absence of passive features. No condition, bound, or ablation is supplied for the regime in which passive features carry substantial predictive power—the usual justification for VFL—leaving the central effectiveness claim without support when the autonomous model is weak.

    Authors: We agree that attack effectiveness depends on signal in the active party's features. The manuscript demonstrates the attack in realistic VFL partitions where the active party achieves non-trivial independent accuracy, consistent with the paper's focus on agnostic attacks that require no score access. To address the concern, we will add a dedicated subsection in §4 with discussion of applicability conditions, including when passive features dominate, plus ablations varying feature predictive power. revision: yes

  2. Referee: [§6] §6 (experimental evaluation): the reported attack results omit baselines for conventional inference attacks, data-exclusion rules, and error analysis or statistical significance tests. Without these, it is not possible to determine whether the observed attack performance is attributable to the agnostic mechanism or to experimental artifacts, directly affecting assessment of the central claim.

    Authors: We accept that additional experimental rigor is needed. The revised version will incorporate baselines against conventional inference attacks, explicit data-exclusion rules, error bars, and statistical significance tests (e.g., paired t-tests with p-values) on attack success rates to isolate the agnostic mechanism's contribution. revision: yes

Circularity Check

0 steps flagged

No significant circularity; attack and defenses defined independently of VFL outputs

full rationale

The paper's central construction defines the agnostic inference attack via an autonomous ML model trained solely on the active party's own features and labels (explicitly independent of VFL training). PPSs are then introduced as adjustable distortions applied to passive-party parameters after VFL training. No equation reduces a claimed prediction or uniqueness result to a fitted parameter or self-citation by construction. No self-citation is invoked as load-bearing for the attack premise or defense. The derivation chain remains self-contained against external benchmarks and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is applied and experimental; it introduces no free parameters fitted to data, no new axioms beyond standard ML assumptions, and no invented entities such as new particles or forces.

pith-pipeline@v0.9.0 · 5841 in / 1195 out tokens · 34536 ms · 2026-05-24T09:15:30.385648+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages · 2 internal anchors

  1. [1]

    Federated learning with differential privacy: Algorithms and performance analysis,

    K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. S. Quek, and H. V . Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454–3469, 2020

    work page 2020

  2. [2]

    Communication-Efficient Learning of Deep Networks from Decentralized Data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in Proceedings of the 20th International Conference on Artificial Intelligence 42 and Statistics , ser. Proceedings of Machine Learning Research, A. Singh and J. Zhu, Eds., vol. 54. PMLR, 20–22 Apr 2017, pp. 1273–1282...

    work page 2017

  3. [3]

    Decentralized federated learning for electronic health records,

    S. Lu, Y . Zhang, and Y . Wang, “Decentralized federated learning for electronic health records,” in 2020 54th Annual Conference on Information Sciences and Systems (CISS) , 2020, pp. 1–5

    work page 2020

  4. [4]

    Privacy-preserving federated brain tumour segmentation,

    W. Li, F. Milletari, D. Xu, N. Rieke, J. Hancox, W. Zhu, M. Baust, Y . Cheng, S. Ourselin, M. J. Cardoso, and A. Feng, “Privacy-preserving federated brain tumour segmentation,” CoRR, vol. abs/1910.00962, 2019. [Online]. Available: http://arxiv.org/abs/1910.00962

    work page arXiv 1910

  5. [5]

    Federated Learning for Emoji Prediction in a Mobile Keyboard

    F. Beaufays, K. Rao, R. Mathews, and S. Ramaswamy, “Federated learning for emoji prediction in a mobile keyboard,” 2019. [Online]. Available: https://arxiv.org/abs/1906.04329

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  6. [6]

    Federated Learning for Mobile Keyboard Prediction

    A. Hard, K. Rao, R. Mathews, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage, “Federated learning for mobile keyboard prediction,” CoRR, vol. abs/1811.03604, 2018. [Online]. Available: http://arxiv.org/abs/1811.03604

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  7. [7]

    A quasi-newton method based vertical federated learning framework for logistic regression,

    K. Yang, T. Fan, T. Chen, Y . Shi, and Q. Yang, “A quasi-newton method based vertical federated learning framework for logistic regression,” CoRR, vol. abs/1912.00513, 2019. [Online]. Available: http://arxiv.org/abs/1912.00513

    work page arXiv 1912

  8. [8]

    A trusted recommendation scheme for privacy protection based on federated learning,

    Y . Wang, Y . Tian, X. Yin, and X. Hei, “A trusted recommendation scheme for privacy protection based on federated learning,” CCF Transactions on Networking , vol. 3, pp. 218–228, 12 2020

    work page 2020

  9. [9]

    Toward resource-efficient federated learning in mobile edge computing,

    R. Yu and P. Li, “Toward resource-efficient federated learning in mobile edge computing,” IEEE Network , vol. 35, no. 1, pp. 148–155, 2021

    work page 2021

  10. [10]

    Federated learning for privacy-preserving ai,

    Y . Cheng, Y . Liu, T. Chen, and Q. Yang, “Federated learning for privacy-preserving ai,” Commun. ACM , vol. 63, no. 12, p. 33–36, nov 2020. [Online]. Available: https://doi.org/10.1145/3387107

    work page doi:10.1145/3387107 2020

  11. [11]

    Feature inference attack on model predictions in vertical federated learning,

    X. Luo, Y . Wu, X. Xiao, and B. C. Ooi, “Feature inference attack on model predictions in vertical federated learning,” CoRR, vol. abs/2010.10152, 2020. [Online]. Available: https://arxiv.org/abs/2010.10152

    work page arXiv 2010

  12. [12]

    Privacy against inference attacks in vertical federated learning,

    B. Rassouli, M. Varasteh, and D. Gunduz, “Privacy against inference attacks in vertical federated learning,” 2022. [Online]. Available: https://arxiv.org/abs/2207.11788

    work page arXiv 2022

  13. [13]

    Protocols for secure computations,

    A. C. Yao, “Protocols for secure computations,” in 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982), 1982, pp. 160–164

    work page 1982

  14. [14]

    A generalisation, a simplification and some applications of paillier’s probabilistic public- key system,

    I. Damg ˚ard and M. Jurik, “A generalisation, a simplification and some applications of paillier’s probabilistic public- key system,” in Public Key Cryptography , K. Kim, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2001, pp. 119–136

    work page 2001

  15. [15]

    UCI machine learning repository,

    D. Dua and C. Graff, “UCI machine learning repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml

    work page 2017

  16. [16]

    Data-Snooping Biases in Tests of Financial Asset Pricing Models,

    A. W. Lo and A. C. MacKinlay, “Data-Snooping Biases in Tests of Financial Asset Pricing Models,” The Review of Financial Studies, vol. 3, no. 3, pp. 431–467, 04 2015. [Online]. Available: https://doi.org/10.1093/rfs/3.3.431

    work page doi:10.1093/rfs/3.3.431 2015

  17. [17]

    [Online]

    github, “Github,” 2020. [Online]. Available: https://github.com/mrtzvrst

    work page 2020

  18. [18]

    Comprehensive analysis of privacy leakage in vertical federated learning during prediction,

    X. Jiang, X. Zhou, and J. Grossklags, “Comprehensive analysis of privacy leakage in vertical federated learning during prediction,” Proceedings on Privacy Enhancing Technologies , vol. 2022, pp. 263–281, 04 2022

    work page 2022

  19. [19]

    A generalized inverse for matrices,

    R. Penrose, “A generalized inverse for matrices,” Mathematical Proceedings of the Cambridge Philosophical Society , vol. 51, no. 3, p. 406–413, 1955

    work page 1955

  20. [20]

    The geometry of algorithms with orthogonality constraints,

    A. Edelman, T. A. Arias, and S. T. Smith, “The geometry of algorithms with orthogonality constraints,” SIAM Journal on Matrix Analysis and Applications , vol. 20, no. 2, pp. 303–353, 1998. [Online]. Available: https://doi.org/10.1137/S0895479895290954

    work page doi:10.1137/s0895479895290954 1998

  21. [21]

    A penalty-free infeasible approach for a class of nonsmooth optimization problems over the stiefel manifold,

    N. Xiao, X. Liu, and Y .-x. Yuan, “A penalty-free infeasible approach for a class of nonsmooth optimization problems over the stiefel manifold,” 2021. [Online]. Available: https://arxiv.org/abs/2103.03514

    work page arXiv 2021

  22. [22]

    A class of smooth exact penalty function methods for optimization problems 43 with orthogonality constraints,

    N. Xiao, X. Liu, and Y . xiang Yuan, “A class of smooth exact penalty function methods for optimization problems 43 with orthogonality constraints,” Optimization Methods and Software , vol. 37, no. 4, pp. 1205–1241, 2022. [Online]. Available: https://doi.org/10.1080/10556788.2020.1852236

    work page doi:10.1080/10556788.2020.1852236 2022

  23. [23]

    Note on the generalized inverse of a matrix product,

    T. N. E. Greville, “Note on the generalized inverse of a matrix product,” SIAM Review, vol. 8, no. 4, pp. 518–521,

    work page

  24. [24]

    Available: https://doi.org/10.1137/1008107

    [Online]. Available: https://doi.org/10.1137/1008107

    work page doi:10.1137/1008107

  25. [25]

    The matrix cookbook,

    K. B. Petersen and M. S. Pedersen, “The matrix cookbook,” Oct. 2008, version 20081110. [Online]. Available: http://www2.imm.dtu.dk/pubdb/p.php?3274

    work page 2008