{"id":"0b4e0193-c6a4-4440-b88a-8fc5bf6a4b8b","arxiv_id":"2509.00332","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CryptoFace performs face verification entirely on encrypted data with FHE, using a patch-based network that cuts inference latency to about 23 minutes per verification.","lead":"This paper introduces CryptoFace, a face recognition system that runs the entire pipeline (feature extraction, matching, thresholding) on encrypted images using fully homomorphic encryption. It is the first such end-to-end system, and reports a 7x speedup over prior encrypted-inference models while keeping accuracy close to cleartext face recognition.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The unproven claim in Sec. 3.1 that a client cannot infer server reference features from match/no-match outputs is the key weakness: it is exactly the oracle used by the template-recovery attack [2] the paper cites.","rationale":"The paper's central claim has two parts: an architectural/performance result and a security guarantee. The architecture part is credible: the patch-based design, depth reduction, single bootstrapping, and released code support the latency/accuracy numbers. The security part, however, depends entirely on the informal assertion that releasing only a final scalar protects the reference template. That assertion is the weakest point because it is (a) unproven, (b) contradicted by the paper's own citation [2] showing that threshold-comparison oracles enable template recovery, and (c) not rescued by the semi-honest assumption, since a semi-honest client may still choose arbitrary probe images and run the protocol repeatedly. The reader's weakest_assumption identifies this same issue, and I agree: the verdict should remain conditional. The paper should either prove a formal statement (e.g., simulation-based security for the single-execution functionality, with explicit out-of-scope for oracle attacks), add query limits or a different comparison mechanism, or soften the 'cannot infer' claim. Without one of these, the headline security claim is not established. I do not see a separate concern of comparable weight: the baselines and evaluation are imperfect but do not threaten the core architectural claim.","tokens_in":15033,"tokens_out":6980,"duration_ms":91389,"concrete_test":"Using the released CryptoFace implementation, mount the Bassit et al. [2] attack against the full protocol: enroll a random LFW reference, then as the client submit many crafted probes (e.g., adversarially optimized face images or latent-code searches through CryptoFaceNet) for the claimed identity and record the decrypted match bit. Stop when a recovered template achieves high cosine similarity (say >0.9) with the enrolled feature or when non-mated probes are accepted. Report the required number of queries and success rate. If feasible (e.g., <10^4 queries), the Section 3.1 claim is falsified; if infeasible, identify the property (noise, threshold, feature dimensionality) that blocks the attack and document it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central security claim rests on the Section 3.1 assertion: since the client receives 'only the matching result, a positive or negative scalar,' an adversarial client 'cannot infer the encrypted features stored on the server.' This inference is not valid. The protocol hands the client a decryptable accept/reject bit for every chosen probe, so after the first run the client has a match oracle. The paper itself cites Bassit et al. [2], a template-recovery attack on homomorphically encrypted biometric systems with unprotected threshold comparison; CryptoFace's returned comparison bit gives the adversary the same query interface. No simulator, query-complexity bound, or formal security definition is provided, and the semi-honest model does not prevent a corrupted client from choosing crafted probe images across many executions. If the intended guarantee is only that intermediate features are never released, the wording 'cannot infer' is too strong and the privacy claim needs to be restated as, e.g., security under a bounded number of queries or a clear statement that oracle-based template recovery is out of scope. As written, the headline 'robust and provable security' is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents CryptoFace, an end-to-end encrypted face recognition system built on CKKS fully homomorphic encryption (FHE). All stages of the pipeline—feature extraction, feature matching, and score thresholding—are performed on encrypted data, and the client receives only an encrypted accept/reject bit. The core architecture, CryptoFaceNet, is a mixture of shallow patch-based CNNs that reduces multiplicative depth to a single bootstrapping operation and is evaluated in parallel, giving near-resolution-independent latency. A degree-2 polynomial is used to approximate the L2 normalization, and the matching threshold is estimated on the cleartext training folds. Experiments on standard face benchmarks (LFW, AgeDB, CALFW, CPLFW, CFP-FP, and subsets of IJB-B/IJB-C) report a 7x speedup over MPCNN and AutoFHE baselines adapted for face recognition, with accuracy improvements up to +8.8 percentage points. The code is released.","tokens_in":15333,"tokens_out":4270,"duration_ms":51416,"significance":"If the results hold, CryptoFace is a useful engineering contribution: it demonstrates a practical FHE-based face recognition pipeline that avoids decryption at the server, provides a detailed latency breakdown, and releases source code. The patch-based parallel design and the depth-optimized convolution block are sensible and likely to be adopted by practitioners. However, the paper's headline security guarantee is not established. The threat-model argument in Sec. 3.1 dismisses oracle-based template recovery without proof, and the baseline comparison is partly handicapped by a transferred AutoFHE configuration. These issues do not invalidate the empirical findings, but they do affect the strength of the central claims.","major_comments":[{"comment":"The assertion that an adversarial client 'cannot infer the encrypted features stored on the server' is unsupported. The protocol gives the client an oracle: after decrypting the returned bit, the client can submit arbitrary crafted probe images and observe match/no-match results. The paper itself cites Bassit et al. [2], a template recovery attack against homomorphically encrypted biometric systems with unprotected threshold comparison—exactly the interface CryptoFace exposes. No simulator, query-complexity bound, or formal security definition is provided. This is load-bearing for the 'robust and provable security' claim in the abstract and conclusion. Please either provide a proof bounding template recovery from binary oracle access, or restate the guarantee as security against a semi-honest server and against passive leakage of intermediate features, explicitly excluding active oracle-","section":"Sec. 3.1, Threat Model"},{"comment":"AutoFHE is evaluated using a search result obtained on CIFAR and transferred to the face dataset without re-running the search. This is disclosed, but it likely handicaps the baseline: AutoFHE's architecture is not adapted to 64x64 face inputs or to the face recognition task, whereas CryptoFaceNet is designed specifically for this setting. As a result, the claimed '+6.73%' accuracy and '2.9x' speedup over AutoFHE are not a like-for-like comparison. Please either run AutoFHE's search on a face dataset (or a close proxy), or clearly label the comparison as 'out-of-the-box transfer' and temper the claims accordingly.","section":"Sec. 4, Baselines"},{"comment":"The 10-fold cross-validation results are reported as point estimates only, with no variance or significance testing. Given the small accuracy gaps between methods (e.g., MPCNN ResNet44 at 89.64% vs. CryptoFaceNet4 at 89.42% for 64x64), it is unclear whether the observed differences are statistically meaningful. Please include fold-level standard deviations or a paired significance test.","section":"Table 1, Sec. 4.1"}],"minor_comments":[{"comment":"In the definition of Score, the second term should be y2/||y2||, not y1/||y1||. As written, the expression is identically zero and the subsequent simplification to 2 - 2 y1·y2/(||y1||||y2||) is inconsistent.","section":"Sec. 3.1, Eq. (3)"},{"comment":"The labels CryptoFaceNet4/9/16 are introduced after the table but can be confusing at first reading; please define the naming convention (number of patches) explicitly before the table or in a footnote.","section":"Sec. 4.1, Table 1"},{"comment":"The comparison 'CryptoFace speeds up inference by 5.4x and increases FR performance by +3.82%' should state the explicit baseline (MPCNN ResNet32) in the same sentence for clarity.","section":"Sec. 4.1, first paragraph after Table 1"},{"comment":"The bottom panel reports log2 approximation error, but the caption does not state the base of the logarithm. Please add this detail.","section":"Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The security issue is not a matter of disagreement with consensus; it is an internally unsupported claim that should be fixed before publication. I recommend requiring the authors to either prove a security guarantee against oracle-based template recovery or substantially weaken the threat-model statement. The editors may also wish to ask the authors to moderate the 'first end-to-end' and 'provable security' phrasing in the abstract, since prior encrypted face matching works (e.g., [3], [15]) already provide server-side encrypted comparison, though not the full pipeline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is the first real end-to-end encrypted face recognition pipeline under FHE, and the architecture work is genuine. But the paper's central privacy claim—that an adversarial client cannot infer the server's reference features from the match bit—is asserted, not proven, and sits in direct tension with the template-recovery attack the authors themselves cite.\n\nWhat's actually new: prior secure FR systems (Boddeti 2018, HERS) encrypt only extracted features. CryptoFace encrypts the raw image and runs feature extraction, matching, and thresholding entirely in the encrypted domain. The engineering choices are sensible: the mixture of shallow patch CNNs cuts multiplicative depth, the depth-optimized shifted AESPA block saves levels, and the distribution-aware low-degree polynomial for L2 normalization is a practical solution. The latency results—7x faster than MPCNN, 2.9x faster than the transferred AutoFHE, and near-resolution-independent scaling—are plausible and backed by the tables. Code is released, which counts for a lot.\n\nThe soft spots are proportionate. The big one is the security claim. In Section 3.1, the paper argues that since the client receives only a positive/negative scalar, an adversarial client cannot infer the server's encrypted features. That does not follow. The client gets a decryptable accept/reject bit for each chosen probe—that is a match oracle. Bassit et al. [2], which the paper cites, shows exactly this oracle enables template recovery. No simulator, query-complexity bound, or formal security definition is provided. If the intended guarantee is only that intermediate features are never released, the claim should be restated. As written, \"robust and provable security\" is unsupported.\n\nSecondary issues: Table 1 reports 10-fold cross-validation averages but no error bars, so the accuracy gaps against the baselines are hard to assess. The AutoFHE baseline is transferred from a CIFAR search, which likely understates what AutoFHE could do if searched on this task. The polynomial coefficients and matching threshold are estimated on cleartext portions of the test data—standard practice for 10-fold CV, so not a serious flaw. And at ~1,400 seconds per verification, \"practical\" really means \"non-interactive.\"\n\nWho this is for: people working on privacy-preserving biometrics or FHE inference who want a concrete, reproducible first step for end-to-end encrypted FR. It deserves serious refereeing, not desk rejection, but the security section needs a rewrite and ideally a bounded-query or simulator-based argument before the headline claim can be taken as stated.","headline":"First credible end-to-end encrypted face recognition under FHE, with a load-bearing security overclaim that needs fixing before the headline holds.","tokens_in":15785,"tokens_out":1896,"would_cite":true,"duration_ms":22849,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CryptoFace is the first face recognition system to keep the whole pipeline—extraction, matching, and decision—encrypted, and it reports a 7x speedup with accuracy gains over prior encrypted networks.","keywords":["face recognition","fully homomorphic encryption","encrypted inference","privacy-preserving biometrics","patch-based convolutional networks","polynomial activations","CKKS","cosine similarity"],"falsifier":"Run the cited template-recovery attack against the CryptoFace match oracle: let an adversarial client craft probe faces, receive the decrypted positive/negative result, and attempt to recover the server's reference feature. Recovery of the template, or of a face image whose matching behavior matches the reference, would refute the paper's end-to-end privacy claim.","tokens_in":14934,"feed_emoji":"🔐","tokens_out":9284,"duration_ms":100931,"temperature":0.7,"pith_summary":"The paper claims to build the first face recognition system that runs the entire verification pipeline—feature extraction, similarity matching, and score thresholding—under fully homomorphic encryption, so a server never sees a plaintext face, feature, or match decision. The central design is CryptoFaceNet, a mixture of shallow patch convolutional networks: each patch of a face is processed by a small network with low multiplicative depth, the patch networks are evaluated in parallel, and their outputs are fused by ciphertext additions, which keeps latency near constant as resolution grows. Against state-of-the-art FHE neural networks adapted for face recognition, the paper reports a 7.2x speedup over the strongest baseline and up to 8.8 percentage points higher verification accuracy, with a single bootstrapping operation per inference. The authors position CryptoFace as a practical route to face recognition with robust, post-quantum security guarantees.","feed_headline":"Face recognition now runs end-to-end encrypted, 7x faster","feed_subtitle":"CryptoFace keeps faces, features, and match results encrypted end to end—7x faster and up to +8.8% more accurate.","key_machinery":"The load-bearing mechanism is CryptoFaceNet, a mixture of shallow patch convolutional networks (PCNNs). A face is divided into non-overlapping patches, each processed by a small CNN whose low multiplicative depth allows encrypted evaluation with only one bootstrapping operation; the patch CNNs run in parallel, and their local features are fused by a decomposed vector-matrix product that reduces to ciphertext additions. Two supporting pieces carry the rest of the argument: a depth-optimal shifted AESPA block, a convolutional block whose Hermite polynomial activation is merged into the preceding convolution to save two multiplicative levels; and a distribution-aware degree-2 polynomial p(t)=β2","core_discovery":"CryptoFace's central claim is that all three stages of face recognition—feature extraction, feature matching, and threshold comparison—can be performed in the encrypted domain with no decryption at any point, under the CKKS fully homomorphic encryption scheme. The client encrypts a probe face and sends it to the server, which extracts an encrypted feature with CryptoFaceNet, matches it against an encrypted reference feature in the server's database, computes the cosine-similarity score via a distribution-aware degree-2 polynomial approximation of the inverse square root, compares the score to a threshold, and returns only an encrypted positive/negative result that the client decrypts. The pa","pith_inferences":["The end-to-end privacy guarantee reduces to the behavior of the match/no-match oracle: if an adversarial client can query the server with crafted probes and observe the decrypted binary result, the cited template-recovery attack suggests the reference feature may be recoverable; the paper does not test this.","The near-resolution-independent latency figure assumes enough parallel compute for all patch networks at once; on hardware with limited parallelism, doubling resolution should raise latency, so the constant-latency behavior is deployment-dependent.","The polynomial ℓ2 approximation and the decision threshold are calibrated on the distribution of squared feature norms in the test data; using CryptoFace on a much harder or out-of-distribution face set would likely require recalibrating the control points and threshold.","The same patch-mixture recipe could transfer to other encrypted high-resolution vision tasks such as iris or gait recognition, since the FHE constraints—low multiplicative depth, parallel patch evaluation, polynomial activations—are generic."],"forward_implications":["A server can offer face verification without ever seeing raw face images or feature vectors in plaintext, shifting the privacy boundary from the feature extractor to the client's secret key.","The same architecture handles higher-resolution inputs with negligible latency increase, so secure face recognition can use more informative images rather than being limited to tiny crops.","Reducing the pipeline to a single bootstrapping operation cuts both latency and RAM footprint relative to prior FHE CNNs, which need multiple bootstrappings.","Because matching and thresholding happen under encryption, the system supports one-to-many identification as well as one-to-one verification, with rank-1 accuracy of 92.19% on a 1:128 retrieval task."],"supporting_citations":[{"why":"Supplies multiplexed parallel convolution and repeated packing methods, and the strongest FHE-CNN baseline (ResNet44) CryptoFace claims a 7.2x speedup over.","marker":"[28]"},{"why":"Baseline FHE network whose search-based mixed-degree activations and AESPA implementation CryptoFace builds on and compares against.","marker":"[1]"},{"why":"Supplies the low-degree Hermite polynomial activation and basis-wise normalization that the CryptoFaceNet block adapts.","marker":"[39]"},{"why":"Provides the ArcFace loss and cosine-similarity verification protocol that CryptoFaceNet is trained with and evaluated on.","marker":"[13]"},{"why":"Supplies the ViT-style patch splitting that CryptoFaceNet's mixture of PCNNs adopts.","marker":"[14]"},{"why":"Defines the CKKS scheme with its packed slots, levels, and rescaling, which determines the cost model and level budget for all encrypted operations.","marker":"[7]"},{"why":"Supplies the template-recovery attack on homomorphically encrypted biometrics with unprotected threshold comparison; the threat CryptoFace's end-to-end design is meant to avoid and the main challenge to the client-privacy assumption.","marker":"[2]"},{"why":"Supplies the prior two-party encrypted face-matching protocol and threat model that CryptoFace extends from feature-only encryption to end-to-end encryption.","marker":"[3]"},{"why":"The SEAL library in which the system is implemented, modified with bootstrapping as in the baselines.","marker":"[41]"}],"fun_headline_variants":["Encrypted face recognition: no decryption, 7x faster","Face matching without ever decrypting – CryptoFace does it","CryptoFace: fully homomorphic face recognition, faster and accurate","End-to-end encrypted face ID: 7x speedup, +8.8% accuracy"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The entire privacy guarantee rests on the claim that an adversarial client who receives only an encrypted match/no-match result cannot infer the server's reference features; the paper asserts this without proof, and the template-recovery attack it cites suggests the binary oracle may leak enough to reconstruct the template.","fun_headline_variants_meta":{"raw":{"variants":["Encrypted face recognition: no decryption, 7x faster","Face matching without ever decrypting – CryptoFace does it","CryptoFace: fully homomorphic face recognition, faster and accurate","End-to-end encrypted face ID: 7x speedup, +8.8% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1181,"prompt_tokens":693,"completion_tokens":488,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":437,"completion_tokens_details":{"reasoning_tokens":409}},"tokens_in":437,"tokens_out":488,"duration_ms":5213,"temperature":1.0,"reasoning_tokens":409,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:41:33.082056+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the cited template-recovery attack against the CryptoFace match oracle: let an adversarial client craft probe faces, receive the decrypted positive/negative result, and attempt to recover the server's reference feature. Recovery of the template, or of a face image whose matching behavior matches the reference, would refute the paper's end-to-end privacy claim.","supporting_citations":[{"cited_title":"Low-complexity deep convolutional neural networks on fully homomorphic encryption using multiplexed parallel convolutions","cited_arxiv_id":null,"evidence_quote":"Supplies multiplexed parallel convolution and repeated packing methods, and the strongest FHE-CNN baseline (ResNet44) CryptoFace claims a 7.2x speedup over."},{"cited_title":"AutoFHE: Automated adaption of CNNs for efficient evaluation over FHE","cited_arxiv_id":null,"evidence_quote":"Baseline FHE network whose search-based mixed-degree activations and AESPA implementation CryptoFace builds on and compares against."},{"cited_title":"AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference","cited_arxiv_id":"2201.06699","evidence_quote":"Supplies the low-degree Hermite polynomial activation and basis-wise normalization that the CryptoFaceNet block adapts."},{"cited_title":"ArcFace: Additive angular mar- gin loss for deep face recognition.IEEE Transactions on Pat- tern Analysis and Machine Intelligence , 44(10):5962–5979,","cited_arxiv_id":null,"evidence_quote":"Provides the ArcFace loss and cosine-similarity verification protocol that CryptoFaceNet is trained with and evaluated on."},{"cited_title":"An image is worth 16x16 words: Trans- formers for image recognition at scale","cited_arxiv_id":null,"evidence_quote":"Supplies the ViT-style patch splitting that CryptoFaceNet's mixture of PCNNs adopts."},{"cited_title":"Homomorphic encryption for arithmetic of approx- imate numbers","cited_arxiv_id":null,"evidence_quote":"Defines the CKKS scheme with its packed slots, levels, and rescaling, which determines the cost model and level budget for all encrypted operations."},{"cited_title":"Template recovery at- tack on homomorphically encrypted biometric recognition systems with unprotected threshold comparison","cited_arxiv_id":null,"evidence_quote":"Supplies the template-recovery attack on homomorphically encrypted biometrics with unprotected threshold comparison; the threat CryptoFace's end-to-end design is meant to avoid and the main challenge to the client-privacy assumption."},{"cited_title":"Secure face matching using fully homomorphic encryption","cited_arxiv_id":null,"evidence_quote":"Supplies the prior two-party encrypted face-matching protocol and threat model that CryptoFace extends from feature-only encryption to end-to-end encryption."},{"cited_title":"Microsoft SEAL (3.6)","cited_arxiv_id":null,"evidence_quote":"The SEAL library in which the system is implemented, modified with bootstrapping as in the baselines."}],"review_version":1}