Pith. sign in

REVIEW 5 major objections 6 minor 19 references

Evaluating Query Efficiency and Accuracy of Transfer Learning-based Model Extraction Attack in Federated Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Starting from a pretrained model makes copying a federated model cheaper and more accurate than training from scratch.

desk verdict A useful empirical comparison of pretrained vs from-scratch surrogates for model extraction in FL, but the missing controls and favorable assumptions keep the central query-efficiency claim from being established. read the letter →

arxiv 2505.23791 v1 pith:IJVWOQGF submitted 2025-05-25 cs.CR cs.AIcs.LG

Sayyed Farid Ahamed , Sandip Roy , Soumya Banerjee , Marc Vucovich , Kevin Choi , Abdul Rahman , Alison Hu , Edward Bowen
show 1 more author
Sachin Shetty
This is my paper · ORCID
classification cs.CRcs.AIcs.LG
keywords modelextractionattackfederatedlearningtransferqueryefficiencyfidelityMLaaSsurrogateimageclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning is supposed to protect client data, but the model it produces is exposed through an API, and this paper asks how easily that model can be copied. It establishes that a model extraction attack against a federated-learning victim succeeds more cheaply when the attacker starts from a pretrained model and fine-tunes it on the victim's queried predictions. Across two neural-network architectures, three image datasets, and federated setups with five or ten clients, accuracy and fidelity of the extracted model rise with query budget, and the pretrained start beats from-scratch training at every budget tested. The concrete result is that on CIFAR-10 a pretrained ResNet surrogate reaches 73.65–73.89% accuracy with 20,000 queries, while the best from-scratch result at 25,000 queries is 68.94%, so a public pretrained checkpoint meaningfully lowers the cost of theft.

What carries the argument

The load-bearing object is the extracted surrogate model $M_e$ built from input-output pairs $(q_i, M_v(q_i))$ collected by querying the victim API, and the mechanism that changes the result is transfer learning: instead of random initialization, $M_e$ starts from a pretrained checkpoint and is fine-tuned on the collected pairs. Algorithm 1 formalizes this: query the victim, store the prediction vectors, then either train from scratch or load pretrained weights and fine-tune. The evaluation metrics—accuracy, fidelity (the fraction of test inputs where $M_v$ and $M_e$ agree), and KL divergence between the two models' output distributions—are what make the improvement visible. The pretrained initialization acts as a prior that reduces the number of queries needed to approximate the victim's decision boundary.

What would settle it

Run the same transfer-learning attack with a query set from a different domain, or with an API that returns only the predicted class label instead of all confidence scores, while keeping the victim model and training procedure fixed; if fine-tuned pretrained surrogates no longer beat from-scratch training at small budgets, the reported advantage is specific to in-distribution soft-label queries.

Watch

Extended reading notes

Core claim

The paper's central claim is that a model extraction attacker targeting a federated-learning victim model gets a measurably better copy, for the same query budget, by fine-tuning a pretrained network than by training an identical network from scratch. In the experiments, the pretrained ResNet surrogate on CIFAR-10 reaches 73.65–73.89% accuracy at 20,000 queries, beating the best from-scratch 25,000-query result of 68.94%, and reaches 76.12% at 25,000 queries, close to the victim baseline of 76.52%. The same ordering holds for fidelity and for FashionMNIST, with the transfer-learning advantage largest at small query sets; the paper reports about 12 percentage points higher accuracy at 10,000 queries on CIFAR-10. The paper frames this as a demonstration that pretrained parameters, not just query volume, determine attack success.

Load-bearing premise

The measured gains assume the attacker's query images come from the same distribution as the victim's private training data and that the API returns a full list of confidence scores for every class, not just the predicted label.

Editorial extensions

If this is right

  • Pay-per-query MLaaS defenses are weaker than they appear when attackers can start from a public pretrained checkpoint.
  • On CIFAR-10, a pretrained ResNet surrogate at 20,000 queries reaches 73.65–73.89% accuracy, beating the best from-scratch 25,000-query result of 68.94%, and at 25,000 queries it nearly matches the victim's 76.52% baseline.
  • Across all three datasets, extraction accuracy and fidelity improve with query set size for from-scratch surrogates, so query budget is a primary control on attack quality.
  • The transfer-learning gain is concentrated at small budgets: 10,000-query pretrained surrogates run about 12 percentage points ahead of their from-scratch counterparts on CIFAR-10.
  • Switching from 5 to 10 federated clients shifts extraction accuracy by only a few points, so attack success is driven more by query budget and initialization than by the federated topology.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's query set is drawn from the same data distribution as the victim's training data, a real attacker holding only out-of-distribution queries might see smaller absolute accuracy and a smaller transfer-learning gap; this is a natural stress test the paper does not run.
  • The algorithm stores the victim's full prediction vector per query; if the API returned only the predicted class label instead of all confidence scores, the information per query would drop and the fine-tuning advantage could shrink. A hard-label variant would isolate how much of the result depends on soft labels.
  • Public pretrained checkpoints exist for far larger and non-image models; the same fine-tuning recipe could be tried against language or multimodal APIs, though the paper's evidence is limited to two architectures and three image datasets.
  • Defenders could treat 'attacker has a good pretrained prior' as a design assumption; query-pattern detection and output perturbation would need to be evaluated against fine-tuned surrogates, not just from-scratch ones.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper studies model extraction (ME) attacks against federated-learning (FL) victim models, comparing surrogates trained from scratch with surrogates initialized from pretrained models and then fine-tuned on victim query-response pairs. Experiments are conducted with NVFlare across CIFAR-10, MNIST, and FashionMNIST, using basic CNN and ResNet architectures, with 5 and 10 FL clients and query budgets from 5k to 25k. The main reported results are that extraction accuracy and fidelity increase with query set size, and that transfer-learning initialization yields higher accuracy and fidelity than from-scratch training, especially at smaller query budgets.

Significance. If the transfer-learning advantage holds up, the paper would make a useful empirical contribution to the model-extraction literature by showing that attackers with access to public pretrained models can extract a functional copy of an FL-deployed victim with fewer API queries. The use of an actual FL platform (NVFlare), multiple datasets, two architectures, and three complementary metrics (accuracy, fidelity, KL divergence) is a strength, and the paper is commendably explicit about its algorithm and threat-model assumptions. However, the central query-efficiency claim is not yet established because the comparisons do not isolate the information obtained from victim queries from the information already present in the pretrained model, and several experimental details (single runs, soft-label ambiguity, in-distribution query construction) currently prevent a robust conclusion.

major comments (5)
  1. [V-C, Tables II–III] The central claim that transfer learning improves query efficiency is not yet isolated from the pretraining prior. The paper reports, for example, 73.65% accuracy for the pretrained CIFAR-10 surrogate at 20k queries versus 66.03% from scratch, but it does not report a no-fine-tuning control (the pretrained model evaluated directly on D*) or a control fine-tuned on the same query images with ground-truth labels instead of victim outputs. Without these controls, the apparent gain could be largely inherited from the pretrained model's existing task knowledge rather than from querying M_v, which would undercut the security implication that the attacker gains from the victim queries. The provenance of the pretrained models is also not stated, so the size of this prior cannot be assessed.
  2. [Sections II and IV] The threat model in Section II grants the attacker only 'an unlabeled reference dataset D' with minimal knowledge of the victim's training distribution, but Section IV states that the training set is evenly split with one half used to train M_v and the other half used to generate the query dataset. This is a same-distribution assumption that is much stronger than the stated threat model; it likely inflates both absolute extraction accuracy and the observed transfer-learning advantage. The authors should either state this as an explicit limitation or rerun the evaluation with out-of-distribution or otherwise unlabeled reference data to match the stated attacker capabilities.
  3. [Algorithm 1, line 5; Section II] Algorithm 1 records p_i <- M_v(q_i), which the use of KL divergence as a metric suggests are full prediction vectors, while Section II only says the API 'returns the prediction M_v(x)'. Soft-label responses give the attacker strictly more information than hard labels and can inflate the measured accuracy and fidelity of the extracted model. The paper must clarify whether the API returns soft or hard labels and, ideally, evaluate both settings to show the robustness of the reported gains.
  4. [Tables I–III] All reported numbers appear to come from a single run per configuration; no seeds, error bars, or variance measures are provided. The transfer-learning advantage at small query budgets is often quantified by differences of a few percentage points (e.g., Table II, N=10, 20k: 73.65% versus 66.03%), and without repeated runs it is not possible to determine whether the claimed advantage is statistically meaningful or within run-to-run noise. Please report means and standard deviations over multiple independent runs and specify the random seeds and other randomness controls.
  5. [Section V-C, Table I] The text states that the highest recorded extraction accuracy is about 76.12%, 'closely matching the baseline accuracy of 76.52%', but Table I reports the centralized ResNet victim accuracy on CIFAR-10 as 76.22%, not 76.52%. The 76.12% value matches 76.22% almost exactly, while 76.52% appears nowhere in the tables. This internal inconsistency must be corrected, and any quantitative 'closely matching' claim should be tied to the actual baseline value reported in the experiments.
minor comments (6)
  1. [Table I caption] The caption contains a typo: 'atatck' should be 'attack'.
  2. [Section I] The phrase 'Section Section VI' should be 'Section VI'.
  3. [Tables II and III] The column labeled N is not defined in the text or captions; it appears to denote the number of FL clients (with N=0 likely meaning centralized), but this should be stated explicitly.
  4. [Section IV] KL divergence is used as a metric but is never defined by an equation; please add the formula and specify the base of the logarithm used in the reported values.
  5. [Section IV] The pretrained models' provenance is not given (e.g., ImageNet weights, pretraining code or library, and any preprocessing applied). This information is needed for reproducibility and for interpreting the transfer-learning results.
  6. [Figures 3 and 4] Figures 3 and 4 are referenced but not described in the text; the captions should state what each curve represents and the exact configurations shown.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical measurements against a held-out test set, with no fitted parameter renamed as prediction and no load-bearing self-citation chain.

full rationale

The paper is an experimental evaluation, not a derivation. Its central claim—that fine-tuning a pretrained ResNet on query-response pairs yields higher accuracy and fidelity than training from scratch, especially with small query sets—is supported by measured results in Tables II and III against a fixed test set. There is no equation in which a predicted quantity is defined in terms of the same quantity, no parameter fitted to a subset and then reported as a prediction, and no uniqueness theorem or theoretical result imported from the authors' prior work to force the conclusion. The self-citations to transfer-learning methodology (refs. [18], [19]) are background references and are not load-bearing. The in-distribution query set and soft-label API are favorable assumptions that could limit real-world applicability, and the absence of a ground-truth-label or no-query control is a legitimate experimental-design concern, but these are correctness/validity issues rather than circularity. All reported numbers are independent measurements of attack performance, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No mathematical derivation is attempted; the reported numbers are measurements. The load-bearing assumptions are about attacker data access, API response format, and the availability of a suitable pretrained model.

assumptions (3)
  • domain assumption Query inputs are drawn from the same distribution as the victim's training data (the unused half of the training set).
    Section IV, Dataset paragraph: 'The training set is evenly split, with one half used to train M_v and the other half designated for generating the query dataset.' This is much stronger than the threat model's 'unlabeled reference dataset' and drives the high extraction numbers.
  • domain assumption The API returns full softmax prediction vectors, not just top-1 labels.
    Algorithm 1 lines 4-5 record p_i <- M_v(q_i) as the training signal for the surrogate. The paper never specifies whether this is a probability vector or a hard label; soft labels give the attacker strictly more information and make fine-tuning more effective.
  • domain assumption Pretrained ResNet weights used for the transfer-learning surrogate exist and are appropriate across all three datasets.
    Section V-C says 'we employ a pre-trained ResNet model' but gives no source, architecture variant, or pretraining dataset. The result depends on this artifact being available and transferable to CIFAR-10, MNIST, and FashionMNIST.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Query Efficiency and Accuracy of Transfer Learning-based Model Extraction Attack in Federated Learning." pith.science (2026). https://pith.science/paper/IJVWOQGF

@misc{pith2026250523791,
  author       = {Pith},
  title        = {Pith review of: Evaluating Query Efficiency and Accuracy of Transfer Learning-based Model Extraction Attack in Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IJVWOQGF}},
  note         = {Machine review of arXiv:2505.23791}
}
read the original abstract

Federated Learning (FL) is a collaborative learning framework designed to protect client data, yet it remains highly vulnerable to Intellectual Property (IP) threats. Model extraction (ME) attacks pose a significant risk to Machine Learning as a Service (MLaaS) platforms, enabling attackers to replicate confidential models by querying black-box (without internal insight) APIs. Despite FL's privacy-preserving goals, its distributed nature makes it particularly susceptible to such attacks. This paper examines the vulnerability of FL-based victim models to two types of model extraction attacks. For various federated clients built under the NVFlare platform, we implemented ME attacks across two deep learning architectures and three image datasets. We evaluate the proposed ME attack performance using various metrics, including accuracy, fidelity, and KL divergence. The experiments show that for different FL clients, the accuracy and fidelity of the extracted model are closely related to the size of the attack query set. Additionally, we explore a transfer learning based approach where pretrained models serve as the starting point for the extraction process. The results indicate that the accuracy and fidelity of the fine-tuned pretrained extraction models are notably higher, particularly with smaller query sets, highlighting potential advantages for attackers.

Figures

Figures reproduced from arXiv: 2505.23791 by the authors.

Figure 1
Figure 1. ME attack executed through predictive API in a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Framework of the proposed TL-based ME attack executed in an FL environment. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. ME attack accuracy with ResNet pre-trained model on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: ME attack fidelity with ResNet pre-trained model on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 12 canonical work pages

  1. [1]

    Federated learning of deep networks using model averaging,

    H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,”arXiv preprint arXiv:1602.05629, vol. 2, no. 2, 2016

    arXiv 2016

  2. [2]

    Anomaly detection via federated learning,

    M. Vucovich, A. Tarcar, P. Rebelo, N. Gade, R. Porwal, A. Rahman, C. Redino, K. Choi, D. Nandakumar, R. Schilleret al., “Anomaly detection via federated learning,”arXiv preprint arXiv:2210.06614, 2022

    arXiv 2022

  3. [3]

    De-pois: An attack- agnostic defense against data poisoning attacks,

    J. Chen, X. Zhang, R. Zhang, C. Wang, and L. Liu, “De-pois: An attack- agnostic defense against data poisoning attacks,”IEEE Transactions on Information Forensics and Security, vol. 16, pp. 3412–3425, 2021

    work page 2021

  4. [4]

    A novel smartphone-based human activity recognition approach using convolutional autoencoder long short-term memory network,

    D. Thakur, S. Roy, S. Biswas, E. S. Ho, S. Chattopadhyay, and S. Shetty, “A novel smartphone-based human activity recognition approach using convolutional autoencoder long short-term memory network,” in2023 IEEE 24th International Conference on Information Reuse and Integra- tion for Data Science (IRI). IEEE, 2023, pp. 146–153

    2023

  5. [5]

    Source inference attacks in federated learning,

    H. Hu, Z. Salcic, L. Sun, G. Dobbie, and X. Zhang, “Source inference attacks in federated learning,” in2021 IEEE International Conference on Data Mining (ICDM). IEEE, 2021, pp. 1102–1107

    work page 2021

  6. [6]

    Mia-bad: An approach for enhancing membership inference attack and its mit- igation with federated learning,

    S. Banerjee, S. Roy, S. F. Ahamed, D. Quinn, M. Vucovich, D. Nan- dakumar, K. Choi, A. Rahman, E. Bowen, and S. Shetty, “Mia-bad: An approach for enhancing membership inference attack and its mit- igation with federated learning,” in2024 International Conference on Computing, Networking and Communications (ICNC). IEEE, 2024, pp. 635–640

    work page 2024

  7. [7]

    On safeguarding privacy and security in the framework of federated learning,

    C. Ma, J. Li, M. Ding, H. H. Yang, F. Shu, T. Q. Quek, and H. V . Poor, “On safeguarding privacy and security in the framework of federated learning,”IEEE network, vol. 34, no. 4, pp. 242–248, 2020

    work page 2020

  8. [8]

    Model extraction attacks revis- ited,

    J. Liang, R. Pang, C. Li, and T. Wang, “Model extraction attacks revis- ited,” inProceedings of the 19th ACM Asia Conference on Computer and Communications Security, 2024, pp. 1231–1245

    work page 2024

Show all 19 references
  1. [9]

    Inversenet: Augmenting model extraction attacks with training data inversion

    X. Gong, Y . Chen, W. Yang, G. Mei, and Q. Wang, “Inversenet: Augmenting model extraction attacks with training data inversion.” in IJCAI, 2021, pp. 2439–2447

    2021

  2. [10]

    Model extraction warning in mlaas paradigm,

    M. Kesarwani, B. Mukhoty, V . Arya, and S. Mehta, “Model extraction warning in mlaas paradigm,” inProceedings of the 34th Annual Com- puter Security Applications Conference, 2018, pp. 371–380

    2018

  3. [11]

    Model extraction attacks on split federated learning,

    J. Li, A. S. Rakin, X. Chen, L. Yang, Z. He, D. Fan, and C. Chakrabarti, “Model extraction attacks on split federated learning,”arXiv preprint arXiv:2303.08581, 2023

    2023 arXiv

  4. [12]

    Blockchain-based lightweight authentication protocol for iot-enabled smart agriculture,

    A. Vangala, S. Roy, and A. K. Das, “Blockchain-based lightweight authentication protocol for iot-enabled smart agriculture,” in2022 In- ternational Conference on Cyber-Physical Social Intelligence (ICCSI). IEEE, 2022, pp. 110–115

    2022

  5. [13]

    Stealing machine learning models via prediction{APIs},

    F. Tram `er, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction{APIs},” in25th USENIX security symposium (USENIX Security 16), 2016, pp. 601–618

    2016

  6. [14]

    High accuracy and high fidelity extraction of neural networks,

    M. Jagielski, N. Carlini, D. Berthelot, A. Kurakin, and N. Papernot, “High accuracy and high fidelity extraction of neural networks,” in29th USENIX security symposium (USENIX Security 20), 2020, pp. 1345– 1362

    2020

  7. [15]

    Securing age-of- information (aoi)-enabled 5g smart warehouse using access control scheme,

    A. K. Das, S. Roy, E. Bandara, and S. Shetty, “Securing age-of- information (aoi)-enabled 5g smart warehouse using access control scheme,”IEEE Internet of Things Journal, vol. 10, no. 2, pp. 1358– 1375, 2022

    2022

  8. [16]

    Data-free model extraction,

    J.-B. Truong, P. Maini, R. J. Walls, and N. Papernot, “Data-free model extraction,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4771–4780

    2021

  9. [17]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” inArtificial intelligence and statistics. PMLR, 2017, pp. 1273– 1282

    2017

  10. [18]

    Attl: An automated targeted transfer learning with deep neural networks,

    S. F. Ahamed, P. Aggarwal, S. Shetty, E. Lanus, and L. J. Freeman, “Attl: An automated targeted transfer learning with deep neural networks,” in 2021 IEEE Global Communications Conference (GLOBECOM). IEEE, 2021, pp. 1–7

    2021

  11. [19]

    Targeted transfer learning: Leveraging optimal transport for enhanced knowledge transfer,

    S. F. Ahamed, K. A. Islam, and S. Shetty, “Targeted transfer learning: Leveraging optimal transport for enhanced knowledge transfer,” in2024 International Conference on Computing, Networking and Communica- tions (ICNC). IEEE, 2024, pp. 506–510

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.