{"id":"75ef3754-861c-48b5-a78e-2ec6cbf9218a","arxiv_id":"1908.04172","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"nGraph-HE2 combines CKKS optimizations and a client-aided activation protocol to run standard pre-trained networks on encrypted data, achieving the first MobileNetV2 evaluation on ImageNet and 1,998 images/s on CryptoNets.","lead":"This paper presents nGraph-HE2, a compiler framework that runs neural network inference on encrypted data using the CKKS homomorphic encryption scheme and a client-aided protocol for non-linear activations. It reports the first encrypted inference on an ImageNet-scale model (MobileNetV2) and a state-of-the-art throughput of about 2,000 images per second on the smaller CryptoNets benchmark.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Client-aided activations expose exact pre-activation values to the client, so model confidentiality is not provided and the privacy-preserving claim is conditional on model leakage being out of threat model.","rationale":"The paper is a solid systems contribution: the CKKS encoding and arithmetic optimizations are concrete, measured, and backed by open-source code; the accuracy degradation on MobileNetV2 is tiny and the throughput numbers are internally consistent. The reader's CONDITIONAL verdict is well supported. The strongest objection is not to the arithmetic but to the scope of the privacy claim. The client-aided protocol is transparently described and explicitly labeled a placeholder, but the abstract and introduction frame the result as privacy-preserving inference / homomorphic evaluation of standard networks. Since every non-polynomial activation is computed by the client on decrypted values, the server's model is exposed to the client. This is a real limitation for any deployment where the model owner's weights are sensitive, which is a standard assumption in ML-as-a-service privacy work. My concrete test would demonstrate the exact weight recovery; in fact, the protocol design already guarantees it, so the test would serve as confirmation rather than discovery. I therefore agree with the reader's weakest_assumption and recommend keeping the CONDITIONAL verdict, with the condition being that model confidentiality is explicitly out of scope or replaced by a secure two-party primitive (e.g., garbled circuits) as the paper suggests. No change to the verdict is needed beyond what the reader already stated.","tokens_in":19324,"tokens_out":14308,"duration_ms":146932,"concrete_test":"Run the released nGraph-HE2 code in the client-aided mode with a malicious client that records the decrypted pre-activation vectors for a chosen set of inputs on a single FC layer (or the first ReLU boundary of MobileNetV2). Solve Y = W X for W using least squares over enough linearly independent queries. If the recovered W matches the server's weights to within CKKS noise (e.g., relative error < 1e-3), model confidentiality is demonstrably absent, confirming that the privacy claim holds only under a threat model that excludes model secrecy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 4.1 the server sends every pre-activation ciphertext to the client, who decrypts to pt, applies ReLU/MaxPool in plaintext, and re-encrypts. This gives the client exact intermediate values for every non-polynomial layer. Because the client controls the encrypted inputs, it can choose inputs adaptively and solve the linear equations Wx = pt for the weight matrices of the FC/conv layers, recovering the server's model up to CKKS noise. The paper explicitly acknowledges this ('this approach can leak information about the model to the client') and calls the client-aided step 'a placeholder for more secure implementations.' Thus the load-bearing assumption is that model confidentiality is out of scope, or that the client is trusted not to exploit the leakage. The abstract's broad phrasing 'privacy-preserving inference' and the introduction's 'privacy-preserving execution of a full production-level deep neural network' go beyond that assumption. If the intended threat model is only data privacy (server never sees user data), the protocol satisfies it; but the central claim as stated is under-specified and, in a model-confidentiality setting, unsupported. This is the weakest point because all headline results (CryptoNets-ReLU, MobileNetV2) depend on the client-aided activations; removing them would remove the ability to use native ReLU and thus the claimed generality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents nGraph-HE2, a framework for privacy-preserving neural network inference on encrypted data built on Intel nGraph and Microsoft SEAL's CKKS implementation. The contributions are: a scalar-encoding optimization that reduces CKKS encoding to O(L) memory/runtime in the batch-axis packing setting; complex packing that packs two real values into one complex slot to double throughput when no ciphertext-ciphertext multiplication is needed; optimized ciphertext-plaintext addition and multiplication; lazy rescaling and depth-aware encoding graph-level optimizations; and a client-aided two-party protocol for non-polynomial activations such as ReLU and MaxPool. The evaluation reports a 3.4x-87.9x scalar-encoding speedup, 2.6x-4.2x ciphertext-plaintext addition speedup, a 2.6x multiplication speedup, an 8x CryptoNets runtime improvement from lazy rescaling, 1,998 images/s on CryptoNets, and the first encrypted ImageNet inference on MobileNetV2 with 60.4%/82.7% top-1/top-5 accuracy and 381 ms/image amortized runtime.","tokens_in":19521,"tokens_out":5778,"duration_ms":61047,"significance":"If the results hold, this is a substantial systems contribution to practical homomorphic inference: the CKKS-specific optimizations are non-trivial, clearly described, and empirically evaluated; the scalar-encoding savings have a supporting proof in Appendix A.3; and the MobileNetV2 result appears to be the first encrypted inference on ImageNet, a concrete milestone for the field. The paper also ships open-source code, which supports reproducibility. The main caveat is that the client-aided protocol reveals intermediate plaintext values to the client, so the broad 'privacy-preserving inference' claim is only valid under a data-privacy threat model, not a model-confidentiality threat model. This limitation is acknowledged in Section 4.1, but it is not reflected in the abstract or introduction, and every headline native-activation result depends on this protocol.","major_comments":[{"comment":"The client-aided activation protocol sends every pre-activation ciphertext to the client, who decrypts it, applies the non-polynomial function in plaintext, and re-encrypts the result. The client therefore obtains exact intermediate values for every non-polynomial layer, and because the client controls the encrypted inputs, it can in principle recover the server's weight matrices by adaptive queries. The paper explicitly says 'this approach can leak information about the model to the client' and calls it 'a placeholder for more secure implementations,' yet the abstract claims 'privacy-preserving inference' and the introduction claims 'privacy-preserving execution of a full production-level deep neural network' without stating the model-confidentiality exemption. Since CryptoNets-ReLU and all MobileNetV2 results in Table 8 depend on this protocol, the central privacy claim must be re-scoped to a data-privacy-only threat model, or the protocol must be replaced by a secure activation mechanism; otherwise the broad claim is unsupported.","section":"Section 4.1; Abstract; Section 1"},{"comment":"Lazy rescaling is asserted to preserve correctness without a formal noise-budget analysis, and the only supporting evidence is the CryptoNets accuracy of 98.95% reported in Table 5. Because lazy rescaling is one of the three headline contributions and yields an 8x speedup, the paper should either provide a correctness/noise argument or validate the optimization more broadly across architectures and parameter sets; otherwise the claim that it is a generally applicable graph-level optimization is not established.","section":"Section 3.3.1; Table 5"},{"comment":"The throughput comparison in Table 7 mixes nGraph-HE2 measurements with numbers from prior publications (LoLa, Faster CryptoNets, Gazelle, etc.) without stating the hardware, SEAL version, or thread counts used for the prior results. Since the 'state-of-the-art throughput' claim is a central result, the paper should either report same-platform baselines for the comparison methods or explicitly caveat that the comparison is cross-hardware and therefore indicative rather than controlled.","section":"Table 7; Section 4.3.1"}],"minor_comments":[{"comment":"The abstract says 'the largest network to date,' but Section 4.3.2 more precisely says 'the largest network whose linear layers have been homomorphically evaluated'; the qualifier should be used in the abstract and conclusion as well, because the client-aided activation computation means the non-linear layers are not evaluated purely homomorphically.","section":"Section 4.3.2; Abstract"},{"comment":"Reference [12], 'Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference,' is missing a year and venue; please provide the complete citation.","section":"Reference [12]"},{"comment":"In Algorithm 6, the comment 'Compute [z·q]2^64' should refer to the Barrett ratio r (i.e., [z·r]2^64), since q is the modulus and r is the precomputed ratio; as written the comment is inconsistent with the code.","section":"Section 3.2.2, Algorithm 6"},{"comment":"The footnote discussing the correction term for complex multiplication contains an unclear expression 'af + bдi'; this appears to be a typographical error and should be rewritten or removed.","section":"Section 3.1.2, footnote 3"},{"comment":"The notation 'R∗qL' for the ciphertext space appears to be a formatting artifact for R_{q_L}; please clarify the notation so it is consistent with the rest of the paper.","section":"Section 2.2.3"}],"recommendation":"major_revision","confidential_remarks":"The technical core of the paper is solid and the empirical results are impressive, but the manuscript's central privacy claim is broader than what the client-aided protocol actually provides. I would encourage the editor to treat the threat-model scoping as a mandatory revision, not merely a wording change, because the headline claims are what most readers will take away. If the authors re-scope to data privacy and strengthen the lazy-rescaling correctness discussion, the paper would be a strong contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this one. It is a real systems paper: the authors extend their nGraph-HE compiler with CKKS optimizations and produce the first homomorphic evaluation of MobileNetV2 on ImageNet, at 60.4%/82.7% top-1/top-5 accuracy and 381 ms/image amortized. They also push CryptoNets throughput to 1,998 images/s, and 2,959 images/s with a client-aided ReLU variant. Those numbers are the headline contribution, and they appear to hold up.\n\nWhat is genuinely new is mostly integration rather than cryptographic invention. Complex packing, lazy rescaling, and depth-aware encoding all exist in prior cited work, and the paper credits that work. The contribution is combining them in a batch-axis-packing graph compiler and demonstrating end-to-end results, including the first ImageNet-scale encrypted evaluation. The low-level measurements (3x-88x scalar encoding, 2.6x-4.2x ciphertext-plaintext addition, 2.6x multiplication, 8x from lazy rescaling) are internally consistent with the algorithms and tables. Accuracy degradation is tiny, and the open-source release supports reproducibility. The citation pattern is honest.\n\nThe main soft spot is the client-aided activation protocol. In Section 4.1, the server sends every pre-activation ciphertext to the client, who decrypts, applies ReLU or MaxPool in plaintext, and re-encrypts. That gives the client exact intermediate values, and a client that controls its own inputs can in principle solve for the server's weight matrices up to CKKS noise. The paper acknowledges this and calls the protocol a placeholder for garbled circuits or SGX. That is honest, but the abstract and introduction still describe the result as \"privacy-preserving inference\" without qualifying the threat model. If the goal is only data privacy—the server never sees user data—the protocol satisfies it. If model confidentiality matters, the central claim is not supported. This needs to move from a buried caveat to the framing.\n\nTwo smaller issues. Lazy rescaling is empirically validated on CryptoNets but lacks a formal noise or scale analysis; for a systems paper that is a minor gap, not a flaw. And Table 7 mixes hardware and batching choices from earlier papers, so the \"state-of-the-art throughput\" claim is suggestive rather than a controlled head-to-head comparison. It is probably true, but it is not proven by the table.\n\nThis deserves a serious referee and likely acceptance after a revision that fixes the threat-model language. The math and measurements are sound; the caveat is real, recognized by the authors, and needs to be prominent from the start. I would cite this if I were working on HE inference.","headline":"Real systems contribution with the first encrypted ImageNet inference, but the client-aided activation protocol openly trades away model confidentiality and should be front-and-center in the threat model.","tokens_in":742,"tokens_out":1989,"would_cite":true,"duration_ms":41951,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Standard pre-trained neural networks can be evaluated on encrypted data without retraining, preserving accuracy, with reported throughputs of 1,998 images/s on CryptoNets and 381 ms/image on ImageNet-scale MobileNetV2.","keywords":["homomorphic encryption","neural network inference","CKKS","client-aided computation","graph compiler","batch-axis packing","complex packing","MobileNetV2"],"falsifier":"Run the MobileNetV2-0.35-224 configuration on the same hardware and batch size, and compare encrypted inference against plaintext logits over the 50,000-image ImageNet validation set: if the encrypted top-1 accuracy differs from 60.4% by more than about 0.01% or the amortized runtime exceeds 381 ms/image by more than trial variance, the central claim does not reproduce.","tokens_in":19056,"feed_emoji":"🔐","tokens_out":7905,"duration_ms":80465,"temperature":0.7,"pith_summary":"This paper claims that standard, pre-trained neural networks with their native ReLU activations and real-number arithmetic can be evaluated on encrypted data at practical throughput, without retraining or redesigning the model. The route is a graph compiler that treats homomorphic encryption as a hardware target and couples the CKKS approximate homomorphic scheme with a client-aided two-party step for non-polynomial activations. The framework reports 1,998 images/s on the CryptoNets benchmark and the first encrypted inference on ImageNet using MobileNetV2, at 60.4% top-1 accuracy and 381 ms/image amortized runtime. If true, privacy-preserving inference becomes a compiler-level feature for existing models rather than a cryptographic redesign of each network.","feed_headline":"Encrypted ImageNet inference runs in 381 ms per image","feed_subtitle":"A CKKS-plus-client protocol runs standard MobileNetV2 without retraining, losing under 0.01% accuracy.","key_machinery":"The load-bearing mechanism is the CKKS approximate homomorphic scheme used with batch-axis packing, where one ciphertext holds a batch of real inputs and every weight is a scalar plaintext reused across slots. Two optimizations carry most of the performance: complex packing, which stores two real values as a complex pair in one slot and thereby doubles the number of inputs per ciphertext in networks with no ciphertext-ciphertext multiplications; and lazy rescaling, which postpones or drops the noise-reduction rescale so that it happens once per layer instead of once per multiplication. Depth-aware encoding further reduces cost by encoding plaintext weights at the coefficient-modulus level of the ciphertext they will meet, just-in-time. The non-polynomial activations are handled by a client-aided protocol in which the server hands intermediate ciphertexts to the client, the client decrypts, applies ReLU or MaxPool, and returns a fresh encryption; this also resets the noise budget, letting the effective multiplicative depth be the depth between activation layers rather than the whole network depth.","core_discovery":"nGraph-HE2 extends its predecessor so that standard, pre-trained networks can be evaluated homomorphically without changing their activations or number types. The framework's claim is that, by combining the CKKS approximate homomorphic scheme with a client-aided two-party protocol for non-polynomial functions, the compiler can run real networks on encrypted inputs at usable throughput: 1,998 images/s on CryptoNets and the first homomorphic evaluation of MobileNetV2 on ImageNet, with 60.4%/82.7% top-1/top-5 accuracy and 381 ms/image amortized runtime. The same stack reports a 3x-88x speedup in scalar encoding, 2.6x-4.2x in ciphertext-plaintext addition, and 2.6x in ciphertext-plaintext multiplication in the batch-axis packing setting. The accuracy loss on MobileNetV2 is around 0.01%, which is why the paper positions the approach as preserving model accuracy rather than trading it away.","pith_inferences":["If a deployment cannot trust the client with intermediate activations, the same system could swap the client-aided ReLU for a secure two-party or hardware-protected implementation without touching the linear layers, and the throughput gains would likely carry over.","Complex packing is not limited to these networks: any real-valued computation graph over encrypted data that uses only addition, subtraction, and multiplication by plaintext constants can run two values per slot, so the trick may apply to other private-computation workloads.","The reported communication overhead in the LAN setting suggests a large remaining gain from compressing ciphertexts or overlapping communication, independent of further homomorphic-encryption mathematics.","Scaling to larger batches or deeper networks will hit the client-aided round trips: each ReLU or MaxPool boundary costs a client interaction, so latency, not compute, may become the bottleneck for deeper models."],"forward_implications":["Existing production models can be deployed for encrypted inference without retraining or swapping activations, so original model accuracy is preserved.","Batch-axis packing plus complex packing puts twice as many images into each ciphertext whenever a network has no ciphertext-ciphertext multiplications, directly doubling throughput.","Client-aided activation refreshes the noise budget at each non-polynomial layer, reducing the required multiplicative depth from total network depth to the depth between nonlinearities and allowing much smaller encryption parameters.","At the reported amortized throughput, encrypted inference at the ImageNet scale becomes a feasible batch workload, with latency dominated by server-client communication rather than homomorphic arithmetic.","Accuracy loss from encryption is around 0.01% on MobileNetV2, so deployment decisions no longer have to trade model accuracy for privacy."],"supporting_citations":[{"why":"Prior version of the same compiler; establishes the graph-compiler-for-HE approach and the batch-axis packing baseline this work extends.","marker":"[6]"},{"why":"Full-RNS CKKS construction that supplies the modular representation the arithmetic optimizations exploit.","marker":"[13]"},{"why":"Original CKKS scheme; source of real-number arithmetic, plaintext packing, and rescaling.","marker":"[14]"},{"why":"CryptoNets network; benchmark of choice and the 98.95% accuracy target on MNIST.","marker":"[24]"},{"why":"Gazelle, a hybrid HE-garbled-circuit framework; comparison baseline and the secure alternative discussed for non-polynomial activations.","marker":"[27]"},{"why":"MobileNetV2 architecture on which the ImageNet evaluation is run.","marker":"[38]"},{"why":"Underlying HE library whose encoding and arithmetic routines are optimized in this work.","marker":"[39]"}],"fun_headline_variants":["Encrypted MobileNetV2 runs in 381 ms per image","nGraph-HE2: homomorphic inference on pre-trained models","First encrypted ImageNet inference: MobileNetV2 at 82.7% top-5","CKKS-based framework runs standard neural nets on encrypted data","Encrypted inference at 1,998 images/s on CryptoNets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The client-aided activation step assumes the client is allowed to see intermediate pre-activation values, so the model can leak to the client; if a deployment requires the model weights to remain secret from the client, the privacy guarantee is weakened.","fun_headline_variants_meta":{"raw":{"variants":["Encrypted MobileNetV2 runs in 381 ms per image","nGraph-HE2: homomorphic inference on pre-trained models","First encrypted ImageNet inference: MobileNetV2 at 82.7% top-5","CKKS-based framework runs standard neural nets on encrypted data","Encrypted inference at 1,998 images/s on CryptoNets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000271,"raw_usage":{"total_tokens":1696,"prompt_tokens":1077,"completion_tokens":619,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":523}},"tokens_in":693,"tokens_out":619,"duration_ms":6087,"temperature":1.0,"reasoning_tokens":523,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:49:34.789793+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the MobileNetV2-0.35-224 configuration on the same hardware and batch size, and compare encrypted inference against plaintext logits over the 50,000-image ImageNet validation set: if the encrypted top-1 accuracy differs from 60.4% by more than about 0.01% or the amortized runtime exceeds 381 ms/image by more than trial variance, the central claim does not reproduce.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior version of the same compiler; establishes the graph-compiler-for-HE approach and the batch-axis packing baseline this work extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Original CKKS scheme; source of real-number arithmetic, plaintext packing, and rescaling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CryptoNets network; benchmark of choice and the 98.95% accuracy target on MNIST."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gazelle, a hybrid HE-garbled-circuit framework; comparison baseline and the secure alternative discussed for non-polynomial activations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MobileNetV2 architecture on which the ImageNet evaluation is run."}],"review_version":1}