Pith. sign in

REVIEW 4 major objections 4 minor 24 references

ZKP-FedEval: Verifiable and Privacy-Preserving Federated Evaluation using Zero-Knowledge Proofs

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read ZKP-FedEval claims that federated evaluation can become verifiable and privacy-preserving by having each client generate a zero-knowledge proof that its local loss is below a threshold, without revealing the loss value.

desk verdict The protocol design is sensible, but the implemented circuit does not verify the loss, so the paper's central soundness claim is unsupported. read the letter →

arxiv 2507.11649 v2 pith:AJ23FL32 submitted 2025-07-15 cs.LG cs.DCcs.NI

classification cs.LGcs.DCcs.NI
keywords FederatedLearningZero-KnowledgeProofszk-SNARKsGroth16Privacy-PreservingEvaluationVerifiableComputationLossThresholdCircom
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning distributes training but often relies on clients sharing evaluation metrics, which can leak information about their private data. This paper proposes ZKP-FedEval, a protocol in which each client computes its local loss and, if the loss is below a server-set threshold, generates a zero-knowledge proof of that fact. The server verifies the proof and aggregates only the binary outcome, so the exact loss is never disclosed yet the server can check the threshold claim. The authors implement the proof system with the Circom language and the Groth16 zk-SNARK, and measure constant proof size and linear verification scaling on MNIST and HAR datasets. The intended consequence is a federated evaluation phase that is both private and sound, at the cost of a single bit of information per client.

What carries the argument

The load-bearing object is the threshold-verification arithmetic circuit written in Circom and instantiated with Groth16. It is meant to encode the relation that the hashed weights match the public input and that the fixed-point loss computed over the client's data is below $T$, with the round nonce mixed in to prevent replay. The circuit turns the private witness (model weights and data) into a public, succinct proof; Groth16's constant-size proofs and fast verification are what make the protocol scalable. The caveat carried by the implementation is that the actual circuit, per the paper's Section VII-F, checks only the threshold condition and not the full forward-pass computation.

What would settle it

Inspect the published Circom circuit and attempt to build a witness that sets the final loss register below $T$ while skipping the convolution and pooling constraints; if the Groth16 verifier accepts such a proof (or if those constraints are absent), the claimed binding between proof and true loss is refuted.

Watch

Extended reading notes

Core claim

ZKP-FedEval's central claim is that the evaluation phase of federated learning can be secured by replacing raw loss reports with zero-knowledge proofs of a threshold predicate. Concretely, the server publishes the global model $W_g$, a loss threshold $T$, and a round nonce $N_{\mathrm{round}}$; each client computes its local loss $L_i$ and, when $L_i < T$, produces a Groth16 proof $\pi_i$ for a Circom circuit whose public inputs are the hash of the model, the threshold, and the nonce. The server verifies $\pi_i$ and counts the client as valid, learning only that the loss is below threshold. The paper claims this arrangement is zero-knowledge, sound against clients who would falsely claim compliance, and resistant to replay through the nonce, and reports experiments showing per-proof verification time and proof size are essentially constant while total server time grows linearly with the number of clients. The authors state that the circuit simulates the forward pass and loss computation, though they also acknowledge in the limitations that the implemented circuit only enforces the threshold condition rather than the full computation.

Load-bearing premise

The soundness guarantee holds only if the zero-knowledge circuit re-derives the loss from the model and data; the paper's own experiments admit that the implemented circuit checks only the threshold condition, not the full loss computation, so this binding premise is currently unverified.

Editorial extensions

If this is right

  • Federated evaluation can be reduced to a single bit per client: whether local loss is below the threshold, which weakens metric-based inference attacks.
  • Server-side verification cost per proof is constant and total cost scales linearly with the number of valid proofs, making the protocol predictable for larger FL deployments.
  • A threshold acts as a quality gate: clients whose loss is too high are filtered out without the server knowing their exact loss.
  • The trade-off between inclusiveness and signal is controlled by $T$; for a well-trained model a low threshold yields 100% valid proofs and reveals little information, while a poorly matched model separates good and bad clients.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the circuit is extended to verify the full forward pass, the honest-protocol measurements will likely grow with model depth; the reported 0.4–0.5 s for a small CNN suggests that deep architectures would need circuit-level optimizations or transparent-setup SNARKs with different trade-offs.
  • The single-bit protocol could be generalized to a small set of threshold bands, trading a few bits of leakage for a more informative evaluation signal while still hiding the exact loss.
  • The protocol's privacy guarantee covers the evaluation metric only; data poisoning and gradient leakage remain outside its scope, so it should be combined with, not substituted for, robust aggregation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes ZKP-FedEval, a federated-learning evaluation protocol in which clients prove in zero knowledge that their local loss on a private dataset is below a public threshold T, without revealing the loss value. The protocol uses a Circom circuit instantiated with Groth16; the server verifies each proof and aggregates the binary outcomes. Experiments on MNIST and HAR report proof-generation time, verification time, proof/communication size, and validation rates. The central advertised property is soundness: a client cannot produce a valid proof unless its computed loss satisfies the threshold. I found that the paper itself contains a direct contradiction on this point: the implementation uses only a threshold-comparison circuit, not a circuit that recomputes the forward pass and loss, so the main verifiability claim is unsupported.

Significance. If the full circuit were implemented, the protocol would offer a useful and reasonably lightweight way to obtain a privacy-preserving, verifiable binary signal about client-side model quality in FL; the self-contained prototype and the discussion of trusted-setup trade-offs are helpful. However, the delivered artifact verifies only an unconstrained threshold comparison. As a result, the soundness guarantee that distinguishes this work from simple client self-reporting is not established, and the performance measurements cannot be attributed to the CNN/MLP evaluation workload described in the protocol. The contribution in its current form is a benchmark of a toy comparison circuit rather than of verifiable federated evaluation.

major comments (4)
  1. [§IV-C, §VI, §VII-F] The protocol description and the implementation are directly contradictory. §IV-C states that the circuit 'simulates the forward pass of the chosen model' and computes the loss for an input sample, and Algorithm 1, line 20, says the witness is computed using the CNN/MLP forward pass and loss function. In contrast, §VI says the ZKP implementation used 'a simple loss threshold comparison circuit,' and §VII-F explicitly admits that 'the current circuit only verifies the threshold condition rather than the full forward pass and loss computation.' If the circuit relation only enforces L_i < T for an unconstrained private value L_i, then a malicious client can submit any value below T without computing the model's loss on its data. The proof then asserts nothing about the client's actual evaluation result, and the soundness guarantee stated in §VIII does not follow.
  2. [§VII (Table II, Figures 2–5)] The experimental measurements are taken on the threshold-comparison circuit, not on a circuit that verifies the CNN/MLP forward pass and loss. The reported 0.4–0.5 s proof-generation time for MNIST and 0.12–0.13 s for HAR therefore do not characterize the cost of the protocol as described in §IV and Algorithm 1. A circuit that enforces the full forward pass and loss computation would have substantially more constraints and, in general, different proof-generation times; the paper provides no data for that circuit, so the scalability and overhead conclusions are unsupported.
  3. [§VIII and §IV-C] Even setting aside the threshold-only implementation, the protocol does not specify how the client's private dataset is bound to the proof. The public inputs listed in Algorithm 1 are H_Wg, T, and N_round; there is no commitment, hash, or authenticated identifier for the client's dataset D_i. Consequently, a proof that the witness data yields loss below T does not establish that the client's true local dataset satisfies the threshold, since the prover is free to choose any witness data satisfying the arithmetic constraints. A formal relation definition or a data-commitment mechanism is needed for the claimed verifiability.
  4. [§VII-B, Figure 3, Table II] The reported proof sizes are inconsistent with the claim that they are Groth16 proofs. Figure 3 and Table II report average proof sizes of about 0.79 KiB for MNIST and 0.26 KiB for HAR, but a Groth16 proof consists of three group elements and has a size determined only by the curve, not by the circuit or dataset. The same circuit instantiation should yield the same proof size for both datasets (up to public-input encoding differences, which should be stated). The paper should specify what exactly is included in the 'proof size' (e.g., JSON encoding, public inputs, hashes) or correct the reported values.
minor comments (4)
  1. [§VI, Table I] Section VI describes the MNIST model as a '2-layer CNN with channels evolving from 32 to 64,' while Table I lists Conv2D(1,16) and Conv2D(16,32) with an FC(32×7×7,10) layer; the two descriptions should be reconciled.
  2. [§VI, Table I] Section VI describes the HAR model as a '1D CNN with temporal convolutions (9 input channels → 64 → 128),' but Table I lists only fully connected layers; please clarify the actual architecture.
  3. [§VI, §VII] The paper states that both IID and basic non-IID data distributions were tested, but §VII reports only aggregate results; at minimum, a sentence explaining whether non-IID affected validation rates and timings should be added.
  4. [References] References [2] and [5] are self-citations with arXiv identifiers dated after the main text; please verify the dates/versions and ensure they are appropriate.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the central claims inherit from Groth16 soundness, and the admitted circuit/forward-pass gap is a correctness/incompleteness issue, not a circular derivation.

full rationale

The paper is a system-design paper, not a derivation of a first-principles result, so most circularity patterns do not apply. The claimed soundness property in Section VIII ('A client cannot generate a valid proof unless the computed loss meets the threshold condition') is exactly the intended semantics of the ZKP circuit relation, and the paper invokes Groth16's standard soundness rather than deriving that property from its own assumptions. The self-citations ([2], [5], [16]) are contextual and not load-bearing: they support well-known background claims about poisoning attacks, differential privacy, and ZKP use in IoT. The main defect flagged in the manuscript is a correctness gap, not circularity: Section IV-C states the circuit 'simulates the forward pass of the chosen model' and 'computes the loss,' while Section VII-F admits 'The current circuit only verifies the threshold condition rather than the full forward pass and loss computation.' This contradiction makes the Section VIII verifiability claim unsupported for the delivered artifact, but an unsupported or internally inconsistent claim is not equivalent to its inputs by construction. The observed monotonic increase of validation rate with threshold (Section VII-C) is a direct consequence of Algorithm 1's 'if Li < T then Generate ZKP' guard, so it is definitional, but it is a minor illustrative result rather than a load-bearing predicted outcome. No uniqueness theorem, imported ansatz, or renamed known result from the authors' prior work appears. Score 2 reflects the minor, non-load-bearing self-citations and the definitional threshold observation, without finding the central derivation circular.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the circuit actually recomputing the loss; this is not delivered by the prototype, so the paper's soundness claim is unsupported. The only free parameters are the threshold and fixed-point precision, neither fitted.

free parameters (2)
  • loss threshold T = 0.5, 1.0, 1.5
    Hand-chosen protocol parameter; the entire proof statement is 'loss < T'. The validation rate results are conditional on these values and no tuning procedure is given.
  • fixed-point precision = 6 decimal places
    Fixed-point representation with 6 decimal places chosen for the circuit; affects proof generation time and accuracy, no sensitivity analysis.
assumptions (4)
  • standard math Groth16 zk-SNARK is complete and sound.
    Relied on for the claimed soundness and zero-knowledge properties; cited from [9].
  • domain assumption The Circom circuit encodes the full forward pass and loss computation.
    Central to the protocol's privacy and verifiability claim, but contradicted by Section VII-F which says the implemented circuit only verifies the threshold condition.
  • domain assumption Client datasets are disjoint and held privately.
    Standard FL assumption; needed so that a loss proof is meaningful.
  • domain assumption The server is honest and correctly broadcasts Wg, T, Nround.
    Protocol assumes a trusted server for parameter distribution; no malicious-server analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ZKP-FedEval: Verifiable and Privacy-Preserving Federated Evaluation using Zero-Knowledge Proofs." pith.science (2026). https://pith.science/paper/AJ23FL32

@misc{pith2026250711649,
  author       = {Pith},
  title        = {Pith review of: ZKP-FedEval: Verifiable and Privacy-Preserving Federated Evaluation using Zero-Knowledge Proofs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AJ23FL32}},
  note         = {Machine review of arXiv:2507.11649}
}
read the original abstract

Federated Learning (FL) enables collaborative model training on decentralized data without exposing raw data. However, the evaluation phase in FL may leak sensitive information through shared performance metrics. In this paper, we propose a novel protocol that incorporates Zero-Knowledge Proofs (ZKPs) to enable privacy-preserving and verifiable evaluation for FL. Instead of revealing raw loss values, clients generate a succinct proof asserting that their local loss is below a predefined threshold. Our approach is implemented without reliance on external APIs, using self-contained modules for federated learning simulation, ZKP circuit design, and experimental evaluation on both the MNIST and Human Activity Recognition (HAR) datasets. We focus on a threshold-based proof for a simple Convolutional Neural Network (CNN) model (for MNIST) and a multi-layer perceptron (MLP) model (for HAR), and evaluate the approach in terms of computational overhead, communication cost, and verifiability.

Figures

Figures reproduced from arXiv: 2507.11649 by the authors.

Figure 1
Figure 1. ZKP-FedEval System Architecture: The server distributes the model, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Communication cost (total upload size) as a function of the number [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 2
Figure 2. Average client-side proof generation time and server-side verification [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Validation rate (percentage of clients generating valid proofs) for [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Total server verification time as the number of clients increases, [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 18 canonical work pages

  1. [1]

    Communication-Efficient Learning of Deep Networks from Decentralized Data,

    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” Jan. 2023, arXiv:1602.05629 [cs]. [Online]. Available: http://arxiv.org/abs/1602.05629

  2. [2]

    A Bayesian Incentive Mechanism for Poison-Resilient Federated Learning

    D. Commey, R. A. Sarpong, G. S. Klogo, W. Bagyl-Bac, and G. V . Crosby, “A Bayesian Incentive Mechanism for Poison-Resilient Federated Learning,” Jul. 2025, arXiv:2507.12439 [cs]. [Online]. Available: http://arxiv.org/abs/2507.12439

  3. [3]

    Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning,

    M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning,” Dec. 2018. [Online]. Available: https://arxiv.org/abs/1812.00910v2

  4. [4]

    Exploiting Unintended Feature Leakage in Collaborative Learning,

    L. Melis, C. Song, E. De Cristofaro, and V . Shmatikov, “Exploiting Unintended Feature Leakage in Collaborative Learning,” in 2019 IEEE Symposium on Security and Privacy (SP) , May 2019, pp. 691–706, iSSN: 2375-1207. [Online]. Available: https://ieeexplore.ieee. org/document/8835269

  5. [5]

    Securing Health Data on the Blockchain: A Differential Privacy and Federated Learning Framework,

    D. Commey, S. Hounsinou, and G. V . Crosby, “Securing Health Data on the Blockchain: A Differential Privacy and Federated Learning Framework,” May 2024, arXiv:2405.11580 [cs]. [Online]. Available: http://arxiv.org/abs/2405.11580

  6. [6]

    Practical Secure Aggregation for Privacy-Preserving Machine Learning,

    K. Bonawitz, V . Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical Secure Aggregation for Privacy-Preserving Machine Learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . Dallas Texas USA: ACM, Oct. 2017, pp. 1175–1191. [Online]. Available: https://dl.acm...

  7. [7]

    The knowledge complexity of interactive proof-systems,

    MIT, S. Goldwasser, S. Micali, MIT, C. Rackoff, and University of Toronto, “The knowledge complexity of interactive proof-systems,” in Providing Sound Foundations for Cryptography: On the Work of Shafi Goldwasser and Silvio Micali , Weizmann Institute of Science and O. Goldreich, Eds. Association for Computing Machinery, Oct. 2019. [Online]. Available: ht...

  8. [8]

    Circom: A circuit description language for building zero-knowledge applications,

    M. Bell ´es-Mu˜noz, M. Isabel, J. L. Mu ˜noz-Tapia, A. Rubio, and J. Baylina, “Circom: A circuit description language for building zero-knowledge applications,” IEEE Transactions on Dependable and Secure Computing , vol. 20, no. 6, pp. 4733–4751, 2022, publisher: IEEE. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/ 10002421/

Show all 24 references
  1. [9]

    On the Size of Pairing-Based Non-interactive Arguments,

    J. Groth, “On the Size of Pairing-Based Non-interactive Arguments,” in Advances in Cryptology – EUROCRYPT 2016 , M. Fischlin and J.-S. Coron, Eds. Berlin, Heidelberg: Springer, 2016, pp. 305–326

  2. [10]

    Gradient-based learning applied to document recognition,

    Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998, publisher: Ieee. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/726791/

  3. [11]

    The algorithmic foundations of differential privacy,

    C. Dwork and A. Roth, “The algorithmic foundations of differential privacy,” Foundations and Trends® in Theoretical Computer Science , vol. 9, no. 3–4, pp. 211–407, 2014, publisher: Now Publishers, Inc. [Online]. Available: https://www.nowpublishers.com/article/Details/ TCS-042

  4. [12]

    SafetyNets,

    Z. Ghodsi, View Profile, T. Gu, View Profile, S. Garg, and View Profile, “SafetyNets,” Proceedings of the 31st International Conference on Neural Information Processing Systems , pp. 4675–4684, Dec. 2017. [Online]. Available: https://dlnext.acm.org/doi/abs/10.5555/ 3294996.3295220

  5. [13]

    {GAZELLE}: A Low Latency Framework for Secure Neural Network Inference,

    C. Juvekar, V . Vaikuntanathan, and A. Chandrakasan, “ {GAZELLE}: A Low Latency Framework for Secure Neural Network Inference,” 2018, pp. 1651–1669. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity18/presentation/juvekar

  6. [14]

    VerifyNet: Secure and Verifiable Federated Learning,

    G. Xu, H. Li, S. Liu, K. Yang, and X. Lin, “VerifyNet: Secure and Verifiable Federated Learning,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 911–926, 2020. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8765347

  7. [15]

    Auditable and verifiable federated learning based on blockchain-enabled decentralization,

    A. P. Kalapaaking, I. Khalil, X. Yi, K.-Y . Lam, G.-B. Huang, and N. Wang, “Auditable and verifiable federated learning based on blockchain-enabled decentralization,” IEEE Transactions on Neural Networks and Learning Systems , 2024, publisher: IEEE. [Online]. Available: https:...

  8. [16]

    Securing Blockchain- based IoT Systems with Physical Unclonable Functions and Zero- Knowledge Proofs,

    D. Commey, S. Hounsinou, and G. V . Crosby, “Securing Blockchain- based IoT Systems with Physical Unclonable Functions and Zero- Knowledge Proofs,” in 2024 IEEE 49th Conference on Local Computer Networks (LCN) , Oct. 2024, pp. 1–7, iSSN: 2832-1421. [Online]. Available: https:/...

  9. [17]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273–1282. [Online]. Available: https://proceedings.mlr.press/ v54/mcmahan17...

  10. [18]

    iden3/snarkjs,

    J. Baylina, “iden3/snarkjs,” 2020

  11. [19]

    A public domain dataset for human activity recognition using smartphones

    D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, “A public domain dataset for human activity recognition using smartphones.” in Esann, vol. 3, 2013, pp. 3–4, issue: 1. [Online]. Available: https: //www.esann.org/sites/default/files/proceedings/legacy/es2013-84.pdf

  12. [20]

    Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,

    T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,” Sep. 2019, arXiv:1909.06335 [cs]. [Online]. Available: http://arxiv.org/ abs/1909.06335

  13. [21]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, and R. Cummings, “Advances and open problems in federated learning,” Foundations and trends® in machine learning , vol. 14, no. 1–2, pp. 1–210, 2021, publisher: Now P...

  14. [22]

    Deep leakage from gradients,

    L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” Advances in neural information processing systems , vol. 32,

  15. [23]

    Poseidon: A new hash function for {Zero-Knowledge} proof systems,

    L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, and M. Schofnegger, “Poseidon: A new hash function for {Zero-Knowledge} proof systems,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 519–535. [Online]. Available: https://www.usenix.org/conference/ usenix...

  16. [2019]

    Available: https://proceedings.neurips.cc/paper/2019/ hash/60a6c4002cc7b29142def8871531281a-Abstract.html

    [Online]. Available: https://proceedings.neurips.cc/paper/2019/ hash/60a6c4002cc7b29142def8871531281a-Abstract.html

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.