REVIEW 4 major objections 4 minor 24 references
ZKP-FedEval: Verifiable and Privacy-Preserving Federated Evaluation using Zero-Knowledge Proofs
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read ZKP-FedEval claims that federated evaluation can become verifiable and privacy-preserving by having each client generate a zero-knowledge proof that its local loss is below a threshold, without revealing the loss value.
desk verdict The protocol design is sensible, but the implemented circuit does not verify the loss, so the paper's central soundness claim is unsupported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the threshold-verification arithmetic circuit written in Circom and instantiated with Groth16. It is meant to encode the relation that the hashed weights match the public input and that the fixed-point loss computed over the client's data is below $T$, with the round nonce mixed in to prevent replay. The circuit turns the private witness (model weights and data) into a public, succinct proof; Groth16's constant-size proofs and fast verification are what make the protocol scalable. The caveat carried by the implementation is that the actual circuit, per the paper's Section VII-F, checks only the threshold condition and not the full forward-pass computation.
What would settle it
Inspect the published Circom circuit and attempt to build a witness that sets the final loss register below $T$ while skipping the convolution and pooling constraints; if the Groth16 verifier accepts such a proof (or if those constraints are absent), the claimed binding between proof and true loss is refuted.
Extended reading notes
Core claim
ZKP-FedEval's central claim is that the evaluation phase of federated learning can be secured by replacing raw loss reports with zero-knowledge proofs of a threshold predicate. Concretely, the server publishes the global model $W_g$, a loss threshold $T$, and a round nonce $N_{\mathrm{round}}$; each client computes its local loss $L_i$ and, when $L_i < T$, produces a Groth16 proof $\pi_i$ for a Circom circuit whose public inputs are the hash of the model, the threshold, and the nonce. The server verifies $\pi_i$ and counts the client as valid, learning only that the loss is below threshold. The paper claims this arrangement is zero-knowledge, sound against clients who would falsely claim compliance, and resistant to replay through the nonce, and reports experiments showing per-proof verification time and proof size are essentially constant while total server time grows linearly with the number of clients. The authors state that the circuit simulates the forward pass and loss computation, though they also acknowledge in the limitations that the implemented circuit only enforces the threshold condition rather than the full computation.
Load-bearing premise
The soundness guarantee holds only if the zero-knowledge circuit re-derives the loss from the model and data; the paper's own experiments admit that the implemented circuit checks only the threshold condition, not the full loss computation, so this binding premise is currently unverified.
Editorial extensions
If this is right
- Federated evaluation can be reduced to a single bit per client: whether local loss is below the threshold, which weakens metric-based inference attacks.
- Server-side verification cost per proof is constant and total cost scales linearly with the number of valid proofs, making the protocol predictable for larger FL deployments.
- A threshold acts as a quality gate: clients whose loss is too high are filtered out without the server knowing their exact loss.
- The trade-off between inclusiveness and signal is controlled by $T$; for a well-trained model a low threshold yields 100% valid proofs and reveals little information, while a poorly matched model separates good and bad clients.
Reading between the lines
- If the circuit is extended to verify the full forward pass, the honest-protocol measurements will likely grow with model depth; the reported 0.4–0.5 s for a small CNN suggests that deep architectures would need circuit-level optimizations or transparent-setup SNARKs with different trade-offs.
- The single-bit protocol could be generalized to a small set of threshold bands, trading a few bits of leakage for a more informative evaluation signal while still hiding the exact loss.
- The protocol's privacy guarantee covers the evaluation metric only; data poisoning and gradient leakage remain outside its scope, so it should be combined with, not substituted for, robust aggregation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes ZKP-FedEval, a federated-learning evaluation protocol in which clients prove in zero knowledge that their local loss on a private dataset is below a public threshold T, without revealing the loss value. The protocol uses a Circom circuit instantiated with Groth16; the server verifies each proof and aggregates the binary outcomes. Experiments on MNIST and HAR report proof-generation time, verification time, proof/communication size, and validation rates. The central advertised property is soundness: a client cannot produce a valid proof unless its computed loss satisfies the threshold. I found that the paper itself contains a direct contradiction on this point: the implementation uses only a threshold-comparison circuit, not a circuit that recomputes the forward pass and loss, so the main verifiability claim is unsupported.
Significance. If the full circuit were implemented, the protocol would offer a useful and reasonably lightweight way to obtain a privacy-preserving, verifiable binary signal about client-side model quality in FL; the self-contained prototype and the discussion of trusted-setup trade-offs are helpful. However, the delivered artifact verifies only an unconstrained threshold comparison. As a result, the soundness guarantee that distinguishes this work from simple client self-reporting is not established, and the performance measurements cannot be attributed to the CNN/MLP evaluation workload described in the protocol. The contribution in its current form is a benchmark of a toy comparison circuit rather than of verifiable federated evaluation.
major comments (4)
- [§IV-C, §VI, §VII-F] The protocol description and the implementation are directly contradictory. §IV-C states that the circuit 'simulates the forward pass of the chosen model' and computes the loss for an input sample, and Algorithm 1, line 20, says the witness is computed using the CNN/MLP forward pass and loss function. In contrast, §VI says the ZKP implementation used 'a simple loss threshold comparison circuit,' and §VII-F explicitly admits that 'the current circuit only verifies the threshold condition rather than the full forward pass and loss computation.' If the circuit relation only enforces L_i < T for an unconstrained private value L_i, then a malicious client can submit any value below T without computing the model's loss on its data. The proof then asserts nothing about the client's actual evaluation result, and the soundness guarantee stated in §VIII does not follow.
- [§VII (Table II, Figures 2–5)] The experimental measurements are taken on the threshold-comparison circuit, not on a circuit that verifies the CNN/MLP forward pass and loss. The reported 0.4–0.5 s proof-generation time for MNIST and 0.12–0.13 s for HAR therefore do not characterize the cost of the protocol as described in §IV and Algorithm 1. A circuit that enforces the full forward pass and loss computation would have substantially more constraints and, in general, different proof-generation times; the paper provides no data for that circuit, so the scalability and overhead conclusions are unsupported.
- [§VIII and §IV-C] Even setting aside the threshold-only implementation, the protocol does not specify how the client's private dataset is bound to the proof. The public inputs listed in Algorithm 1 are H_Wg, T, and N_round; there is no commitment, hash, or authenticated identifier for the client's dataset D_i. Consequently, a proof that the witness data yields loss below T does not establish that the client's true local dataset satisfies the threshold, since the prover is free to choose any witness data satisfying the arithmetic constraints. A formal relation definition or a data-commitment mechanism is needed for the claimed verifiability.
- [§VII-B, Figure 3, Table II] The reported proof sizes are inconsistent with the claim that they are Groth16 proofs. Figure 3 and Table II report average proof sizes of about 0.79 KiB for MNIST and 0.26 KiB for HAR, but a Groth16 proof consists of three group elements and has a size determined only by the curve, not by the circuit or dataset. The same circuit instantiation should yield the same proof size for both datasets (up to public-input encoding differences, which should be stated). The paper should specify what exactly is included in the 'proof size' (e.g., JSON encoding, public inputs, hashes) or correct the reported values.
minor comments (4)
- [§VI, Table I] Section VI describes the MNIST model as a '2-layer CNN with channels evolving from 32 to 64,' while Table I lists Conv2D(1,16) and Conv2D(16,32) with an FC(32×7×7,10) layer; the two descriptions should be reconciled.
- [§VI, Table I] Section VI describes the HAR model as a '1D CNN with temporal convolutions (9 input channels → 64 → 128),' but Table I lists only fully connected layers; please clarify the actual architecture.
- [§VI, §VII] The paper states that both IID and basic non-IID data distributions were tested, but §VII reports only aggregate results; at minimum, a sentence explaining whether non-IID affected validation rates and timings should be added.
- [References] References [2] and [5] are self-citations with arXiv identifiers dated after the main text; please verify the dates/versions and ensure they are appropriate.
Circularity Check
No significant circularity; the central claims inherit from Groth16 soundness, and the admitted circuit/forward-pass gap is a correctness/incompleteness issue, not a circular derivation.
full rationale
The paper is a system-design paper, not a derivation of a first-principles result, so most circularity patterns do not apply. The claimed soundness property in Section VIII ('A client cannot generate a valid proof unless the computed loss meets the threshold condition') is exactly the intended semantics of the ZKP circuit relation, and the paper invokes Groth16's standard soundness rather than deriving that property from its own assumptions. The self-citations ([2], [5], [16]) are contextual and not load-bearing: they support well-known background claims about poisoning attacks, differential privacy, and ZKP use in IoT. The main defect flagged in the manuscript is a correctness gap, not circularity: Section IV-C states the circuit 'simulates the forward pass of the chosen model' and 'computes the loss,' while Section VII-F admits 'The current circuit only verifies the threshold condition rather than the full forward pass and loss computation.' This contradiction makes the Section VIII verifiability claim unsupported for the delivered artifact, but an unsupported or internally inconsistent claim is not equivalent to its inputs by construction. The observed monotonic increase of validation rate with threshold (Section VII-C) is a direct consequence of Algorithm 1's 'if Li < T then Generate ZKP' guard, so it is definitional, but it is a minor illustrative result rather than a load-bearing predicted outcome. No uniqueness theorem, imported ansatz, or renamed known result from the authors' prior work appears. Score 2 reflects the minor, non-load-bearing self-citations and the definitional threshold observation, without finding the central derivation circular.
Assumptions & free parameters
free parameters (2)
- loss threshold T =
0.5, 1.0, 1.5
- fixed-point precision =
6 decimal places
assumptions (4)
- standard math Groth16 zk-SNARK is complete and sound.
- domain assumption The Circom circuit encodes the full forward pass and loss computation.
- domain assumption Client datasets are disjoint and held privately.
- domain assumption The server is honest and correctly broadcasts Wg, T, Nround.
Cite this review
Pith. "Pith review of ZKP-FedEval: Verifiable and Privacy-Preserving Federated Evaluation using Zero-Knowledge Proofs." pith.science (2026). https://pith.science/paper/AJ23FL32
@misc{pith2026250711649,
author = {Pith},
title = {Pith review of: ZKP-FedEval: Verifiable and Privacy-Preserving Federated Evaluation using Zero-Knowledge Proofs},
year = {2026},
howpublished = {\url{https://pith.science/paper/AJ23FL32}},
note = {Machine review of arXiv:2507.11649}
}
read the original abstract
Federated Learning (FL) enables collaborative model training on decentralized data without exposing raw data. However, the evaluation phase in FL may leak sensitive information through shared performance metrics. In this paper, we propose a novel protocol that incorporates Zero-Knowledge Proofs (ZKPs) to enable privacy-preserving and verifiable evaluation for FL. Instead of revealing raw loss values, clients generate a succinct proof asserting that their local loss is below a predefined threshold. Our approach is implemented without reliance on external APIs, using self-contained modules for federated learning simulation, ZKP circuit design, and experimental evaluation on both the MNIST and Human Activity Recognition (HAR) datasets. We focus on a threshold-based proof for a simple Convolutional Neural Network (CNN) model (for MNIST) and a multi-layer perceptron (MLP) model (for HAR), and evaluate the approach in terms of computational overhead, communication cost, and verifiability.
Figures
Reference graph
Works this paper leans on
-
[1]
Communication-Efficient Learning of Deep Networks from Decentralized Data,
H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” Jan. 2023, arXiv:1602.05629 [cs]. [Online]. Available: http://arxiv.org/abs/1602.05629
arXiv 2023
-
[2]
A Bayesian Incentive Mechanism for Poison-Resilient Federated Learning
D. Commey, R. A. Sarpong, G. S. Klogo, W. Bagyl-Bac, and G. V . Crosby, “A Bayesian Incentive Mechanism for Poison-Resilient Federated Learning,” Jul. 2025, arXiv:2507.12439 [cs]. [Online]. Available: http://arxiv.org/abs/2507.12439
work page Pith review arXiv 2025
-
[3]
M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning,” Dec. 2018. [Online]. Available: https://arxiv.org/abs/1812.00910v2
arXiv 2018
-
[4]
Exploiting Unintended Feature Leakage in Collaborative Learning,
L. Melis, C. Song, E. De Cristofaro, and V . Shmatikov, “Exploiting Unintended Feature Leakage in Collaborative Learning,” in 2019 IEEE Symposium on Security and Privacy (SP) , May 2019, pp. 691–706, iSSN: 2375-1207. [Online]. Available: https://ieeexplore.ieee. org/document/8835269
-
[5]
Securing Health Data on the Blockchain: A Differential Privacy and Federated Learning Framework,
D. Commey, S. Hounsinou, and G. V . Crosby, “Securing Health Data on the Blockchain: A Differential Privacy and Federated Learning Framework,” May 2024, arXiv:2405.11580 [cs]. [Online]. Available: http://arxiv.org/abs/2405.11580
arXiv 2024
-
[6]
Practical Secure Aggregation for Privacy-Preserving Machine Learning,
K. Bonawitz, V . Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical Secure Aggregation for Privacy-Preserving Machine Learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . Dallas Texas USA: ACM, Oct. 2017, pp. 1175–1191. [Online]. Available: https://dl.acm...
arXiv 2017
-
[7]
The knowledge complexity of interactive proof-systems,
MIT, S. Goldwasser, S. Micali, MIT, C. Rackoff, and University of Toronto, “The knowledge complexity of interactive proof-systems,” in Providing Sound Foundations for Cryptography: On the Work of Shafi Goldwasser and Silvio Micali , Weizmann Institute of Science and O. Goldreich, Eds. Association for Computing Machinery, Oct. 2019. [Online]. Available: ht...
work page 2019
-
[8]
Circom: A circuit description language for building zero-knowledge applications,
M. Bell ´es-Mu˜noz, M. Isabel, J. L. Mu ˜noz-Tapia, A. Rubio, and J. Baylina, “Circom: A circuit description language for building zero-knowledge applications,” IEEE Transactions on Dependable and Secure Computing , vol. 20, no. 6, pp. 4733–4751, 2022, publisher: IEEE. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/ 10002421/
work page 2022
Show all 24 references
-
[9]
On the Size of Pairing-Based Non-interactive Arguments,
J. Groth, “On the Size of Pairing-Based Non-interactive Arguments,” in Advances in Cryptology – EUROCRYPT 2016 , M. Fischlin and J.-S. Coron, Eds. Berlin, Heidelberg: Springer, 2016, pp. 305–326
2016
-
[10]
Gradient-based learning applied to document recognition,
Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998, publisher: Ieee. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/726791/
1998
-
[11]
The algorithmic foundations of differential privacy,
C. Dwork and A. Roth, “The algorithmic foundations of differential privacy,” Foundations and Trends® in Theoretical Computer Science , vol. 9, no. 3–4, pp. 211–407, 2014, publisher: Now Publishers, Inc. [Online]. Available: https://www.nowpublishers.com/article/Details/ TCS-042
2014
-
[12]
SafetyNets,
Z. Ghodsi, View Profile, T. Gu, View Profile, S. Garg, and View Profile, “SafetyNets,” Proceedings of the 31st International Conference on Neural Information Processing Systems , pp. 4675–4684, Dec. 2017. [Online]. Available: https://dlnext.acm.org/doi/abs/10.5555/ 3294996.3295220
2017
-
[13]
{GAZELLE}: A Low Latency Framework for Secure Neural Network Inference,
C. Juvekar, V . Vaikuntanathan, and A. Chandrakasan, “ {GAZELLE}: A Low Latency Framework for Secure Neural Network Inference,” 2018, pp. 1651–1669. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity18/presentation/juvekar
2018
-
[14]
VerifyNet: Secure and Verifiable Federated Learning,
G. Xu, H. Li, S. Liu, K. Yang, and X. Lin, “VerifyNet: Secure and Verifiable Federated Learning,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 911–926, 2020. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8765347
2020
-
[15]
Auditable and verifiable federated learning based on blockchain-enabled decentralization,
A. P. Kalapaaking, I. Khalil, X. Yi, K.-Y . Lam, G.-B. Huang, and N. Wang, “Auditable and verifiable federated learning based on blockchain-enabled decentralization,” IEEE Transactions on Neural Networks and Learning Systems , 2024, publisher: IEEE. [Online]. Available: https:...
2024
-
[16]
Securing Blockchain- based IoT Systems with Physical Unclonable Functions and Zero- Knowledge Proofs,
D. Commey, S. Hounsinou, and G. V . Crosby, “Securing Blockchain- based IoT Systems with Physical Unclonable Functions and Zero- Knowledge Proofs,” in 2024 IEEE 49th Conference on Local Computer Networks (LCN) , Oct. 2024, pp. 1–7, iSSN: 2832-1421. [Online]. Available: https:/...
2024
-
[17]
Communication-efficient learning of deep networks from decentralized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273–1282. [Online]. Available: https://proceedings.mlr.press/ v54/mcmahan17...
2017
-
[18]
iden3/snarkjs,
J. Baylina, “iden3/snarkjs,” 2020
2020
-
[19]
A public domain dataset for human activity recognition using smartphones
D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, “A public domain dataset for human activity recognition using smartphones.” in Esann, vol. 3, 2013, pp. 3–4, issue: 1. [Online]. Available: https: //www.esann.org/sites/default/files/proceedings/legacy/es2013-84.pdf
2013
-
[20]
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,
T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,” Sep. 2019, arXiv:1909.06335 [cs]. [Online]. Available: http://arxiv.org/ abs/1909.06335
2019 arXiv
-
[21]
Advances and open problems in federated learning,
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, and R. Cummings, “Advances and open problems in federated learning,” Foundations and trends® in machine learning , vol. 14, no. 1–2, pp. 1–210, 2021, publisher: Now P...
2021
-
[22]
Deep leakage from gradients,
L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” Advances in neural information processing systems , vol. 32,
-
[23]
Poseidon: A new hash function for {Zero-Knowledge} proof systems,
L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, and M. Schofnegger, “Poseidon: A new hash function for {Zero-Knowledge} proof systems,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 519–535. [Online]. Available: https://www.usenix.org/conference/ usenix...
2021
-
[2019]
Available: https://proceedings.neurips.cc/paper/2019/ hash/60a6c4002cc7b29142def8871531281a-Abstract.html
[Online]. Available: https://proceedings.neurips.cc/paper/2019/ hash/60a6c4002cc7b29142def8871531281a-Abstract.html
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.