REVIEW 5 major objections 5 minor 24 references
A chunked zero-knowledge circuit attests fair-lending metrics on 32,768 rows with 32 verified proofs.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-05 00:45 UTC pith:NBQYO27P
load-bearing objection A credible, reproducible chunked-ZK demonstration on real HMDA data; the model-binding trust anchor is the load-bearing gap. the 5 major comments →
ZK-SR117: A Chunked Zero-Knowledge Attestation Design for Aggregated Fair-Lending Metrics, with a Control Mapping toward Full SR 11-7 Coverage
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that aggregated fairness statistics can be attested at supervisory scale by split-and-aggregate: commit the full batch, split it into K independent chunks of fixed size, prove each chunk's per-group or per-bin counts with the same pre-calibrated circuit, and compute the final statistic in the clear from the K published count vectors. The paper argues this is the only one of the three tested designs to reach 32,768 rows: the flat single-ReduceSum circuit suffers accumulator overflow past a few thousand rows (74% error at N=2048, more than 600% at N=8192 on synthetic batches; confirmed severe on real data), and the numerically exact tree-reduction circuit did not finish se
What carries the argument
The chunked attestation circuit: K independent small zkSNARK circuits of fixed size n=1,024, each proving per-group counts (demographic parity) or per-bin counts (30 values for 10-bin ECE) that are additive across chunks, with the final statistic computed in the clear from the published counts. Calibration and setup are done once (4.3-4.4 seconds) and reused across all chunks, which isolates the arithmetic to a single fixed batch size and keeps compile time independent of total N; this is the mechanism that avoids the flat-sum overflow and the tree-reduction compile wall.
Load-bearing premise
The examiner must trust that the proving and verifying keys it holds were honestly generated from the bank's real deployed model weights, because the implementation provides no independently checkable commitment to those weights.
What would settle it
Compile the tree-reduction circuit at N=2048 on a proving backend that does not use the toolchain's settings-calibration search; if it completes in seconds, then the chunked design's status as the only design that reached this scale is a property of the toolchain, not of the circuit structure.
If this is right
- An examiner can verify a 32,768-row fairness attestation in about 40 milliseconds per proof (parallelizable), replacing the numerical re-computation step of a document-exchange review.
- Any metric with an additive per-group or per-bin decomposition—calibration error, population-stability index, group-count fairness gaps—can be attested on the same chunked architecture; rank-based metrics such as AUC and KS require a different strategy.
- Model-version binding is empirically confirmed: a proof from one trained model fails against a different model's verifying key, so stale attestations can be rejected after retraining.
- The one-sided hypothesis-test framing with simulated Type I/II error rates gives regulators concrete N-sizing guidance: detecting a gap only 0.02 above threshold at 18.8% subgroup prevalence needs on the order of 32,768 rows, not 1,024.
- Annually, proof artifacts for all FDIC-insured institutions at ten controls per quarter project to roughly 4.5 TB, which the paper estimates is within registry storage capacity.
Where Pith is reading between the lines
- If the chunked pattern generalizes as the paper argues, the same split-and-aggregate architecture could carry attestations in other regulated domains—credit, insurance, hiring—wherever the metric is additive; that is a projection, not a result in the paper.
- The obvious next engineering step, flagged by the paper as future work, is the standalone commitment to model weights; without it, the entire attestation rests on trust in the key-generation process, a trust assumption the paper itself spells out in Section 4.2.
- A cross-backend replication of the tree-reduction compile-time wall would settle whether the chunked design's advantage is structural or specific to the proving toolchain used; the paper explicitly leaves this open, so the scaling claim may be narrower than it appears.
- The paper's own bootstrap mismatches (uniform resampling vs the actual stratified sampling) could be corrected by stratified resampling; recomputing the two confidence intervals that way is a small, concrete follow-up that would either confirm or tighten the reported error analysis.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ZK-SR117, a chunked zero-knowledge attestation design for aggregated fair-lending statistics under U.S. bank supervision (SR 11-7 / OCC 2011-12). The central technical contribution is a circuit architecture that splits a large committed validation batch (N=32,768) into K=32 chunks of n=1,024, proves per-chunk aggregate counts with EZKL-based zkSNARKs, and computes the final statistic (demographic-parity gap, then expected calibration error) in the clear from the verified chunk outputs. The authors report 32/32 proofs verifying, per-chunk proving times under 4 seconds for the demographic-parity gap and about 14.7 seconds for ECE, and small gaps between attested values and same-sample plaintext values (0.00037 for ECE). They also characterize two alternatives (flat ReduceSum and tree reduction), document a real HMDA sentinel-code outlier that broke proving, and introduce a nine-row mapping from SR 11-7 control language to zero-knowledge statements. The paper is explicit that only two controls, one model class, and one task are implemented end-to-end; the nonce protocol, corpus commitment, and additional controls are design proposals. The artifact is publicly available with a reproducible pipeline.
Significance. If the result is taken at its stated scope—a publicly reproducible feasibility study of chunked aggregated-statistic attestation on real HMDA data—it is a genuinely useful contribution. The implementation is unusually transparent, the circuit-precision check against same-sample plaintext ECE is methodologically sound, and the authors clearly separate implemented results from design proposals. The two-control demonstration and the documented failure modes (flat-sum overflow, tree-reduction compile wall, sentinel-code outlier) are concrete and valuable for the zkML-for-regulation community. However, the paper's broader supervisory claim is not yet supported: the implemented artifact does not provide an independently checkable commitment to the deployed model weights, and the anti-cherry-picking nonce/sampling proof is not implemented. These are load-bearing gaps for the stated purpose of supervisory attestation, and they require either additional implementation work or a substantially more limited framing.
major comments (5)
- The implemented artifact binds a proof only to the EZKL proving/verifying key pair, which is generated from whatever weights the bank supplies at settings time. The paper states this directly: verification 'does not give the examiner an independent commitment-opening check against a weight vector the bank discloses separately.' The §7.3 cross-verification experiment only shows that proofs from different weights do not verify under each other's keys; it does not give the examiner any way to check that the key pair was honestly derived from the deployed model. Since the central claim is an attestation of the deployed model's fair-lending behavior, this is a load-bearing trust-anchor gap. The authors should either implement a standalone KZG/Pedersen commitment to θ with an opening check in the circuit (as §9 lists) or explicitly limit the paper's claim to attestation of a computation perfor
- The nonce-controlled sampling protocol is not part of the implemented artifact. The experimental results use a fixed published nonce string, and the 'sampling-proof folded into the circuit statement' is listed under 'Specified in the protocol, not yet implemented.' Consequently, the demonstrated system cannot actually prevent a bank from selecting rows in light of the nonce, and the verifier has no circuit-enforced check that the committed rows are the nonce-selected ones. The abstract's phrase 'committed, nonce-sampled batches' therefore overstates what is implemented. Either implement the sampling proof and corpus commitment, or describe the empirical evaluation as a non-adversarial demonstration of the circuit-scaling mechanism only.
- There is a numerical inconsistency in the flat-sum baseline. Table 3 reports relative error of 74% at N=2048 and >600% at N=8192, while Section 7.2 (immediately below the table) reports real-HMDA mean absolute percent errors of 28.7% at N=2048, 4.4% at N=4096, and 97.9% at N=8192. The table caption says all three designs were 'tested directly on real HMDA-trained model weights and real HMDA batches,' which makes the discrepancy confusing. The authors should clarify which numbers come from synthetic sweeps and which from real data, correct the table to match the real-data figures, and ensure the non-monotonic N=4096 result is represented consistently. This is a reporting error that undermines the baseline comparison as currently presented.
- The bootstrap confidence intervals used to interpret the demographic-parity result are built by uniform resampling, while the attestation uses stratified, nonce-controlled sampling with fixed subgroup proportions. The authors acknowledge this mismatch in §7.5.3 and even note that the ECE value falls just outside a similarly constructed uniform CI, but they still use the uniform CI in §7.3 to support the claim that the 0.00289 deviation is normal sampling variance. This is a like-for-like comparison problem, not a mere wording issue. The paper should recompute the bootstrap intervals under stratified resampling matching the actual strata sizes, or present the same-sample plaintext value (as done for ECE) as the primary ground truth for circuit precision.
- The percentile-clip preprocessing rule ([0.5th, 99.5th]) was introduced after the outlier-driven proving failure, and applying it changed the full-holdout ground-truth gap from 0.0911 to 0.0653. The paper states that sensitivity to this choice is deferred to a companion note. Since the headline error and the attested result are both post-clip, the choice of clip bounds is a data-adaptive modeling decision that could affect the attested statistic. At minimum, the paper should report a sensitivity analysis over alternative percentile bounds (e.g., [1st, 99th], [0.1st, 99.9th]) for the final demographic-parity and ECE attestations, or explicitly state that the demonstrated claim is conditional on this specific published preprocessing spec and that its fairness implications are unaudited.
minor comments (5)
- The abstract says the protocol includes 'a nonce-based sampling protocol resisting bank-side cherry-picking,' but the implementation uses a fixed nonce string and the full nonce protocol is design work. Consider wording such as 'a nonce-based sampling protocol is proposed' to avoid overstating the implemented scope.
- The phrase 'per-chunk hashing' is invoked as the binding mechanism for the dataset, but no algorithm or public-input format is specified. The reproducibility artifact may clarify this, but the paper should describe what an external verifier actually receives and checks to confirm the row set.
- The ECE definition says 'standard equal-width ECE binning' but does not specify the variable being binned (presumably predicted probabilities), the bin edges, or how ties are handled. Please make the definition self-contained.
- The compile-time ranges for tree reduction are reported as '90–280s,' which is ambiguous: it could mean a range across machines or a timeout. Please report precise values per machine and N.
- The T×k leakage bound is already carefully caveated as an upper bound. Consider moving the caveat into the main limitation discussion, since the correlated-bits issue may be relevant to a real deployment.
Circularity Check
No circular derivation: attested statistics are computed outputs checked against independent plaintext/ground-truth values; the weight-binding gap and post-hoc preprocessing are acknowledged limitations, not built-in equivalences.
full rationale
The claimed derivation chain is a systems implementation: the chunked circuit proves per-chunk counts, the final gap/ECE is computed in the clear from those counts, and the results are validated against (a) full-holdout ground truth and (b) plaintext computation on the same committed rows. None of these outputs is defined as a transformation of the validation target. Section 7.5.3 explicitly computes plaintext ECE on the exact 32,768-row stratified sample (0.33397) and compares it to the attested 0.33360; this is an independent check of circuit precision, not a fitted prediction. The two concerns raised by the reader are real but are not circularity. First, the preprocessing spec in Section 7.3 was introduced after the outlier-driven failure and 'changed the full-holdout ground truth itself (from 0.0911 to 0.0653)'; this is a post-hoc evaluation-protocol decision that weakens pre-registration/external validity—a point the paper itself flags in Section 8 with a deferred sensitivity analysis—but there is no equation or fitted parameter whose value is reused as the attested statistic. Second, Section 4.2 admits that EZKL 'bakes the model weights into the circuit at settings-generation time' and that verification 'does not give the examiner an independent commitment-opening check'; this is an explicitly scoped missing component (Table 1), not a claim that the proof's object is defined circularly—the paper states a standalone KZG commitment is future work. No load-bearing self-citation exists: reference [1] is by the first author but is not used to justify the scaling or soundness results, which are argued from measured EZKL behavior and standard zkSNARK properties. The flat-sum/tree-reduction comparison is an empirical benchmark performed in-paper on real HMDA batches, not an imported uniqueness claim. I therefore find no step where a 'prediction' reduces by construction to its inputs.
Axiom & Free-Parameter Ledger
free parameters (7)
- Percentile clip bounds [0.5th, 99.5th] per feature =
0.5th and 99.5th percentiles of each of the 4 features on the training split
- Attestation scale: N=32,768, K=32 chunks of n=1,024 =
32,768 rows; 32 chunks; 1,024 per chunk
- ECE bin count B=10 =
10 equal-width bins
- HMDA slice filter =
conventional, first-lien, home-purchase, principal-residence originations and denials (409,905 rows)
- Model weights (4-feature logistic regression, Model A) =
5 parameters incl. intercept; AUC 0.6966 pre-fix; coefficients not tabulated
- EZKL settings and SRS (logrows=17) =
logrows=17, production Powers-of-Tau SRS
- Illustrative control thresholds (DP and ECE) =
0.05 / 0.05
axioms (7)
- domain assumption Knowledge soundness of the EZKL/Halo2-PLONK zkSNARK with negligible per-proof knowledge error
- ad hoc to paper The EZKL proving/verifying key pair implicitly binds the attested circuit to the specific model weights
- ad hoc to paper Percentile clipping does not materially distort the attested fairness statistic
- domain assumption Stratified nonce-controlled sampling with strata shares fixed to true held-out prevalence yields the stated sampling-variance behavior
- ad hoc to paper The fixed published nonce string behaves like the beacon-derived, examiner-signed nonce
- domain assumption Beacon unpredictability and commitment binding for the proposed nonce protocol
- domain assumption The HMDA slice defines a suitable population for fair-lending attestation
invented entities (4)
-
Supervisory attestation registry
no independent evidence
-
Beacon-derived examiner-signed nonce construction
no independent evidence
-
Corpus-commitment extension (Com(corpus) on a fixed filing schedule)
no independent evidence
-
Table 2 Row 9 preprocessing specification (feature-wise percentile clip as a supervisory data-quality control)
independent evidence
Cite this review
Pith. "Pith review of ZK-SR117: A Chunked Zero-Knowledge Attestation Design for Aggregated Fair-Lending Metrics, with a Control Mapping toward Full SR 11-7 Coverage." pith.science (2026). https://pith.science/paper/NBQYO27P
@misc{pith2026260802664,
author = {Pith},
title = {Pith review of: ZK-SR117: A Chunked Zero-Knowledge Attestation Design for Aggregated Fair-Lending Metrics, with a Control Mapping toward Full SR 11-7 Coverage},
year = {2026},
howpublished = {\url{https://pith.science/paper/NBQYO27P}},
note = {Machine review of arXiv:2608.02664}
}
read the original abstract
Deploying ML models in regulated decision-making (credit underwriting, fraud detection, loan approval) requires demonstrating fairness and robustness to auditors without exposing model weights or customer data. We address this attestation problem for U.S. bank supervision under SR 11-7 and OCC 2011-12 guidance. We present a chunked zero-knowledge circuit design that attests an aggregated fairness statistic - the demographic-parity gap - on committed, nonce-sampled batches of real 2022 HMDA mortgage data, and demonstrate it end-to-end: 32,768 rows, 32 independently verified zkSNARK proofs, aggregated attested gap within 0.0029 of the true held-out value, per-chunk proving under 4 seconds. We also demonstrate extensibility by attesting a second control on the identical architecture - expected calibration error at 10 bins - with all 32 chunks verified, per-chunk proving at about 14.7 seconds, and attested ECE within 0.00037 of the plaintext value on the same committed rows. We compare this design against two alternatives - a flat summation circuit, which overflows past a few thousand rows, and a tree-reduction circuit, numerically exact but intractable to compile - and find the chunked design is the only one that reached this scale. We discovered and root-caused a genuine data-quality failure (a sentinel-code outlier distorting circuit proving and the fairness statistic itself) and resolved it with a published preprocessing specification. We also propose a fuller mapping from SR 11-7 and OCC 2011-12 control language to zero-knowledge statements (nine control elements spanning soundness, calibration, robustness, and drift), a nonce-based sampling protocol resisting bank-side cherry-picking, and a threat model for deployment, proposed as design work, not implemented results. Two controls, one model class, one task are demonstrated end-to-end; the rest is scoped and left as future work.
Reference graph
Works this paper leans on
-
[1]
M. N. Uddin, “A Regulatory Governance Framework for AI-Driven Financial Fraud Detection in U.S. Banking: Integrating OCC, SR 11-7, CFPB, and FinCEN Compliance Requirements for Model Development, Validation, and Monitoring Lifecycles,” 2026
work page 2026
-
[2]
Model Validation Practice in Banking: A Structured Approach,
A. Sudjianto and X. Zhang, “Model Validation Practice in Banking: A Structured Approach,” arXiv, 2024
work page 2024
-
[3]
Model Risk Management and Validation Frameworks for Machine Learning Models in Banking,
E. Nketiah et al., “Model Risk Management and Validation Frameworks for Machine Learning Models in Banking,” International Journal For Multidisciplinary Research, 2026
work page 2026
-
[4]
Modifying Model Risk Management Practice in the Era of AI/ML,
S. Brotcke, “Modifying Model Risk Management Practice in the Era of AI/ML,” Journal of Risk Management in Financial Institutions, 2020
work page 2020
-
[5]
L. Scaramuzza et al., “ ‘Show Me You Comply. . . Without Showing Me Anything’: Zero- Knowledge Software Auditing for AI-Enabled Systems,” arXiv, 2025. 27
work page 2025
-
[6]
Trustless Audits without Revealing Data or Models,
S. Waiwitlikhit et al., “Trustless Audits without Revealing Data or Models,” International Conference on Machine Learning, 2024
work page 2024
-
[7]
Verifiable Evaluations of Machine Learning Models Using zkSNARKs,
J. South et al., “Verifiable Evaluations of Machine Learning Models Using zkSNARKs,” arXiv, 2024
work page 2024
-
[8]
FairZK: A Scalable System to Prove Machine Learning Fairness in Zero- Knowledge,
K. Zhang et al., “FairZK: A Scalable System to Prove Machine Learning Fairness in Zero- Knowledge,” IEEE Symposium on Security and Privacy, 2025
work page 2025
-
[9]
FairProof: Confidential and Certifiable Fairness for Neural Networks,
C. Yadav et al., “FairProof: Confidential and Certifiable Fairness for Neural Networks,” In- ternational Conference on Machine Learning, 2024
work page 2024
-
[10]
ZEN: An Optimizing Compiler for Verifiable, Zero-Knowledge Neural Network Inferences,
B. Feng et al., “ZEN: An Optimizing Compiler for Verifiable, Zero-Knowledge Neural Network Inferences,” 2021
work page 2021
-
[11]
ZKML: An Optimizing System for ML Inference in Zero-Knowledge Proofs,
T. Chen et al., “ZKML: An Optimizing System for ML Inference in Zero-Knowledge Proofs,” European Conference on Computer Systems, 2024
work page 2024
-
[12]
Artemis: Efficient Commit-and-Prove SNARKs for zkML,
N. Lycklama et al., “Artemis: Efficient Commit-and-Prove SNARKs for zkML,” arXiv, 2024
work page 2024
-
[13]
ZKTorch: Compiling ML Inference to Zero-Knowledge Proofs via Parallel Proof Accumulation,
T. Chen et al., “ZKTorch: Compiling ML Inference to Zero-Knowledge Proofs via Parallel Proof Accumulation,” arXiv, 2025
work page 2025
-
[14]
Jolt Atlas: Verifiable Inference via Lookup Arguments in Zero Knowledge,
B. Benno et al., “Jolt Atlas: Verifiable Inference via Lookup Arguments in Zero Knowledge,” arXiv, 2026
work page 2026
-
[15]
Benchmarking CNN Components in EZKL: A Layer-Level Analysis for EVM- Compatible Deployment,
F. Akor et al., “Benchmarking CNN Components in EZKL: A Layer-Level Analysis for EVM- Compatible Deployment,” Digital Signal Processing and Signal Processing Education Work- shop, 2026
work page 2026
-
[16]
SR 11-7: Guidance on Model Risk Man- agement,
Board of Governors of the Federal Reserve System, “SR 11-7: Guidance on Model Risk Man- agement,” April 4, 2011
work page 2011
-
[17]
OCC Bulletin 2011-12: Sound Practices for Model Risk Management,
Office of the Comptroller of the Currency, “OCC Bulletin 2011-12: Sound Practices for Model Risk Management,” April 4, 2011
work page 2011
-
[18]
Home Mortgage Disclosure (Regulation C),
Consumer Financial Protection Bureau, “Home Mortgage Disclosure (Regulation C),” 12 CFR Part 1003
-
[19]
Consumer Financial Protection Bureau, “Circular 2022-03: Adverse Action Notification Re- quirements in Connection with Credit Decisions Based on Complex Algorithms,” 2022
work page 2022
-
[20]
A Distributed Randomness Beacon Daemon,
drand (League of Entropy), “A Distributed Randomness Beacon Daemon,”https://drand. love
-
[21]
National Institute of Standards and Technology, “NIST Randomness Beacon,”https:// beacon.nist.gov
-
[22]
K. Gurkan, K. Wei Jie, et al., “Perpetual Powers of Tau,” Privacy & Scaling Explorations, https://github.com/privacy-scaling-explorations/perpetualpowersoftau
-
[23]
W. G. Cochran, “Sampling Techniques,” 3rd ed., John Wiley & Sons, 1977
work page 1977
-
[24]
Interagency Fair Lending Examination Procedures,
Federal Financial Institutions Examination Council, “Interagency Fair Lending Examination Procedures,” 2009 (updated periodically). 28 Appendix: Sample SR 11-7 Attestation Package This appendix illustrates Contribution 4 (Section 1) concretely: what a bank would actually submit to an OCC or Federal Reserve examiner under the protocol proposed in this pape...
work page 2009
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.