{"id":"b4e9bbfe-2a40-452f-92a8-bd9cfd9e3093","arxiv_id":"2411.15707","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Nimbus speeds up secure two-party Transformer inference using a client-side outer-product protocol for linear layers and distribution-aware low-degree polynomial approximations for non-linear layers.","lead":"Nimbus is a new protocol for private two-party inference of Transformer models, claiming 2.7 to 4.7 times faster BERTbase inference than the previous state of the art with almost no accuracy loss. It moves the heavy encrypted matrix multiplication to the client side and uses data-aware low-degree polynomials for GELU and Softmax, lowering the cost of privacy-preserving NLP.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed speedups rest on unspecified RLWE parameters; the COP protocol's m=3072 scalar-poly accumulation has unverified noise growth, so decryption correctness and actual communication cost are not established.","rationale":"The reader identified the same primary concern: the protocol's correctness and security rely on the reused RLWE scheme having an adequate noise budget, yet the paper supplies no concrete ciphertext modulus or noise analysis. This is the most load-bearing issue because if the noise budget is insufficient, decryption fails and the entire efficiency comparison collapses; if a larger modulus is required, the communication and computation numbers change materially. The protocol idea itself is coherent, and the code link provides a path to verification, so the concern is addressable rather than fatal. The distribution-aware approximation is a secondary concern because the paper evaluates on the target GLUE distribution and tunes breakpoints per depth, making the risk empirical rather than structural. The security proof is incomplete but plausibly patchable with a standard semi-honest simulation argument. Therefore the conditional verdict is appropriate: the paper should provide concrete HE parameters or a full proof, and ideally a runnable artifact, before the efficiency claims can be fully accepted.","tokens_in":29358,"tokens_out":10402,"duration_ms":99006,"concrete_test":"Extract the actual HE parameters (polynomial degree N, ciphertext modulus q, plaintext modulus, noise distribution) from the released repository (https://github.com/secretflow/spu, Nimbus-related commit). Compute the worst-case noise bound after the m=3072 accumulation in Algorithm 1 using the fixed-point shares (scale s=18) and verify that total noise < q/2 for the claimed security level. Additionally, run an end-to-end decryption-correctness test on at least 10^6 random inputs for the largest layer at the claimed settings and report any decryption failures. If the parameters cannot be extracted or failures occur, the reported speedups are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim (2.7x-4.7x end-to-end speedup) presupposes that the reused BumbleBee RLWE scheme can be instantiated with N=8192, plaintext ring Z_{2^64} for linear layers, and a modulus q that both meets the target security level and keeps decryption correct after the client's homomorphic evaluation. In Algorithm 1 (Appendix C.1), each output row is computed as c[alpha] = sum_{beta in [m]} (x_{alpha,beta} ⊗ [[W_beta]]), with m up to 3072 for Linear_h2. Each scalar-poly multiplication multiplies the ciphertext noise by the client's share x_{alpha,beta}, which in Z_{2^64} can have absolute value as large as 2^63; the subsequent addition of ~2^12 such terms amplifies the noise further. For standard RLWE, the resulting noise is on the order of 2^75 times the initial noise, which would require a ciphertext modulus far larger than the 64-bit plaintext ring and could substantially inflate ciphertext sizes and communication. The paper gives no concrete modulus q, no noise budget analysis, no parameter selection, and no description of any digit decomposition. The security proof in Appendix C.1 is also only a sketch and appears to invoke circuit privacy against the client's own view, which is not the standard notion needed here; this does not by itself invalidate security, but it leaves the proof incomplete. Without resolving the noise-budget question, the reported latencies and communication numbers are unverified, and the core correctness of the protocol is conditional.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Nimbus, a secure two-party inference framework for Transformers. For linear layers, Nimbus introduces a client-side outer-product (COP) protocol with row-wise encoding that moves encrypted weights to the client during a one-time setup, eliminating input communication online and producing compact output ciphertexts. For non-linear layers, the paper fits low-degree piecewise polynomials to GELU and exp using the observed activation distribution, and uses a smaller ring (Z_{2^32}) with a fused truncation/upcast conversion. The paper reports end-to-end speedups of 2.7x-4.7x over BumbleBee for BERTbase at sequence length 128, and an average accuracy loss of 0.08% on the GLUE benchmark (with fine-tuning). It also reports a 60% reduction in communication compared with BumbleBee.","tokens_in":1863,"tokens_out":2416,"duration_ms":63721,"significance":"If the efficiency numbers hold with concrete, secure parameters, the work is a substantial advance for private Transformer inference: the COP protocol questions the standard server-side inner-product paradigm, the distribution-aware approximation is a pragmatic way to reduce non-linear layer costs, and the open-source implementation is a valuable asset for the community. The paper includes detailed complexity analysis and an amortized setup-cost discussion in the appendix. However, the significance is conditional on resolving the missing ciphertext-parameter/noise analysis and on correcting the overstatements in the abstract and introduction regarding accuracy loss and speedup.","major_comments":[{"comment":"The central efficiency claim rests on the reused RLWE-based HE scheme being instantiated with N=8192, plaintext ring Z_{2^64} for the linear layers, and a ciphertext modulus q that simultaneously meets the target security level and keeps decryption correct after up to m=3072 scalar-polynomial multiplications and additions per output ciphertext (m up to 3072 for Linear_h2). The paper provides no concrete modulus q, no noise-budget analysis, no digit decomposition, and no parameter selection. Without this, the reported latencies and communication totals are not verifiable, and the protocol's correctness (decryption without failure) is not established. This is a load-bearing gap that must be addressed before the efficiency claims can be accepted.","section":"Section 5 / Appendix C.1 / Algorithm 1"},{"comment":"The security proof is only a brief sketch and appears to conflate the client's and server's views. For the corrupted-client case, the relevant property is that the encrypted weights hide W under the HE scheme (standard IND-CPA security), not the circuit-privacy property of the HE scheme, which concerns hiding the function from the decryptor. For the corrupted-server case, the server receives a ciphertext encrypting W*X_c - R; the uniform mask R makes the plaintext uniform, but the simulation must also account for the ciphertext noise distribution, which the sketch does not do. A full simulation-based proof with explicit hybrids is needed.","section":"Appendix C.1, Security Proof of the Matrix Multiplication Protocol"},{"comment":"The abstract states that 'the average accuracy loss of our approach is 0.08% compared to the non-2PC inference without privacy.' This number is only for Nimbus†, which is fine-tuned to the approximation; without fine-tuning, the average loss is 0.6% (Section 5.1). The abstract and introduction omit this qualification. Moreover, because the fine-tuning adapts the model to the approximate activation functions, the 0.08% figure is not an independent measure of approximation fidelity but rather a measure of the model's ability to compensate for the approximation.","section":"Abstract / Section 5.1 / Table 2"},{"comment":"The claimed '2.7x-4.7x end-to-end speedup' excludes the one-time setup cost of transmitting the encrypted weights (180 MB to 720 MB per linear layer per block) and is reported for sequence length 128. Appendix F.1 reports that for sequence length 1 the speedup is only 1.2x-2.1x, and Appendix F.3 shows that the setup cost is amortized only after about 3 queries. The abstract and introduction should qualify the speedup claim by stating the sequence-length regime and by clarifying that the reported online latency excludes the one-time setup.","section":"Abstract / Section 5.2 / Appendix F.1 / F.3"},{"comment":"The distribution-aware polynomials are fitted to the activation distribution of the same BERTbase model on the same GLUE benchmark used for evaluation, and the best accuracy is obtained after fine-tuning the model to the approximate activations. This creates a mild circularity: the reported 0.08% loss is not a pure measure of how well the approximation generalizes to unseen data or to other models. The paper should report results on a genuinely held-out distribution or explicitly discuss this limitation and its potential impact on the accuracy claims.","section":"Section 4.2 / Table 2 and Appendix C.2"}],"minor_comments":[{"comment":"The notation for rings, e.g., Z2ℓ and AN,2ℓ, is inconsistent with the standard Z_{2^ell} and A_{N,2^ell}; the typesetting makes the exponent and subscript difficult to distinguish. Please use consistent and unambiguous notation throughout.","section":"Section 2.2"},{"comment":"The exponential approximation is written as exp(x) ≈ 0 for x < T_exp, which is intentionally an approximation in a low-probability region. Please state explicitly that this is an approximation for that region rather than presenting an exact equality, to avoid confusion.","section":"Section 4.2, Equation (4)"},{"comment":"The correctness derivation for the truncation-upcast fusion contains a misplaced '=' in the second line and a parenthetical reference ('The third line holds since...') that does not match the displayed equation. The relationship between the generic parameter k in the proof and the concrete B2A width ell' - ell + s should also be clarified.","section":"Appendix E, Equation (8)"},{"comment":"The text reports '4.8×~5.9× faster than Iron and 2.7×~4.7× faster than BumbleBee' but the WAN total latencies (40.96/8.52=4.81 and 23.00/8.52=2.70) give a range of 2.7x-4.8x over BumbleBee; the 4.7x upper bound is slightly inconsistent with the plotted numbers.","section":"Section 5.2, Figure 6"},{"comment":"The text says the non-linear communication is '3× less' than BumbleBee, but the ratio for Softmax is 354.26/115.35 ≈ 3.07 and for GELU is 185.13/53.22 ≈ 3.48. Please use the exact ratios or state them as approximately 3x.","section":"Section 5.3, Table 3"}],"recommendation":"major_revision","confidential_remarks":"This is a solid systems/MPC paper with a potentially impactful protocol change and a useful open-source implementation. The two main blockers are (1) the complete absence of concrete RLWE parameters and noise analysis for the COP protocol, which makes the headline efficiency numbers unverifiable, and (2) the abstract's overstatements about accuracy loss and speedup, which omit the fine-tuning requirement, the sequence-length restriction, and the one-time setup cost. The security proof also needs to be made rigorous. These issues are fixable within the scope of a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one. It has two genuinely new pieces for semi-honest two-party Transformer inference. First, the COP protocol flips the usual server-side inner product into a client-side outer product with row-wise encoding, which kills input ciphertext communication and packs output ciphertexts densely. Second, the distribution-aware approximation fits low-degree piecewise polynomials to actual activation histograms, which lets them drop to a 32-bit ring for GELU/Softmax and cuts rounds roughly in half. The efficiency numbers for BERTbase at sequence length 128 look internally consistent, and the appendix gives a genuinely useful breakdown across hidden sizes, sequence lengths, and LAN/WAN.\n\nThe soft spots are real but addressable. The abstract's 0.08% accuracy loss is the fine-tuned number; without fine-tuning the average loss is 0.6%. The 2.7-4.7x end-to-end speedup is for seq 128; at seq 1 it shrinks to 1.2-2.1x, and the one-time setup cost (180-720MB of encrypted weights per layer) only amortizes after tens of queries. All of this is in the appendix, but the abstract overstates the scope. Second, the security proof is explicitly a sketch. Appendix C.1 says 'brief proof idea' and leans on circuit privacy of the underlying BumbleBee HE scheme, and the paper gives no concrete ciphertext modulus or noise budget. The stress-test worry about m=3072 scalar-poly accumulations is legitimate: multiplying by 64-bit shares and summing thousands of terms will blow up noise unless q is chosen generously. That doesn't mean the protocol is broken - BumbleBee already multiplies by large ring elements - but the paper should report parameters and a full proof or a precise reference, otherwise the efficiency claims are not reproducible.\n\nThe mild circularity the reader flags is real too: the polynomials are fit on the same BERTbase model and evaluated on GLUE. It's not fatal, but the authors should say the 0.08% number is a matched-domain result.\n\nWho is this for? Anyone building or benchmarking private Transformer inference. It deserves a serious referee; the system is well-built, the code link exists, and the baseline comparisons are fair. I'd send it to review with a request for a scoped abstract, concrete HE parameters, and a completed security proof.","headline":"Nimbus is a real step forward in practical 2PC Transformer inference, but the abstract oversells the headline numbers and the crypto parameters need to be spelled out.","tokens_in":30306,"tokens_out":3261,"would_cite":true,"duration_ms":30293,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Nimbus claims secure Transformer inference can be 2.7x-4.7x faster with 0.08% accuracy loss.","keywords":["secure two-party inference","homomorphic encryption","Transformer inference","GELU approximation","Softmax approximation","matrix multiplication","privacy-preserving machine learning","BERT"],"falsifier":"A concrete falsification: instantiate the COP protocol with a stated $(N,q)$ parameter set and compute the accumulated noise after $m=3072$ scalar-polynomial multiplications on a single ciphertext row; if decryption fails, or the modulus $q$ needed to avoid failure forces ciphertexts larger than the paper's reported communication numbers, the central efficiency claim collapses at that security level.","tokens_in":29089,"feed_emoji":"🔐","tokens_out":12203,"duration_ms":95348,"temperature":0.7,"pith_summary":"This paper tries to establish that secure two-party inference for Transformer models can be made practical by attacking both bottlenecks at once: replacing inner-product-style homomorphic matrix multiplication with a client-side outer-product protocol, and replacing high-degree polynomial approximations of GELU and Softmax with low-degree approximations tuned to the model's measured activation distribution. If the claims hold, a client holding private data can query a proprietary Transformer with nearly the plaintext model's accuracy at a fraction of the previous latency. The concrete reported payoff is that BERTbase inference speeds up by 2.7x to 4.7x over the previous state of the art across network settings, with about 60% less communication and an average accuracy loss of 0.08% after light fine-tuning. The trade-off is that the client becomes the computational heavy lifter and is assumed to be a server-grade machine.","feed_headline":"Private BERT inference runs up to 4.7x faster","feed_subtitle":"Client-side matrix multiplication plus low-degree GELU/Softmax approximations yield 0.08% accuracy loss.","key_machinery":"The central object is the client-side outer product (COP) protocol with row-wise encoding, together with distribution-aware polynomial fitting for non-linear functions. In COP, each row of the weight matrix is encoded into a polynomial and encrypted once at setup; the client multiplies each plaintext activation share as a scalar against the ciphertext polynomials and accumulates the partial products, so an entire output row is assembled from outer-product contributions, and a right-shift operation packs several output ciphertexts into one dense polynomial. For non-linear layers, the load-bearing mechanism is a fitting procedure that minimizes $\\int_l^h p(x)[f(x)-f'(x)]^2\\,dx$ under the empirical density $p(x)$ of the activations, which lets GELU be approximated by a quadratic on a narrow interval and exponential by a cubic on one interval plus a linear tail, enabling the smaller ring $\\mathbb{Z}_{2^{32}}$ and a free fused truncation-upcast back to $\\mathbb{Z}_{2^{64}}$.","core_discovery":"The paper's central claim is that the two bottlenecks in secure two-party Transformer inference—homomorphic matrix multiplication in linear layers and secure evaluation of GELU and Softmax—are both removable by rebalancing work and using input statistics. Nimbus has the server encrypt its static weights once in a setup phase; the client then computes each output row through an outer product: scalar multiplications of its plaintext activation shares against the encrypted weight polynomials, accumulated and packed by right shifts into dense ciphertexts. For non-linear layers, Nimbus fits GELU and the exponential in Softmax with low-degree piecewise polynomials whose breakpoints are per-layer tuned by minimizing expected squared error under the empirically measured activation density, and a fused truncation-and-upcast protocol converts between $\\mathbb{Z}_{2^{32}}$ and $\\mathbb{Z}_{2^{64}}$ rings at no extra communication. The paper reports that on BERTbase this yields 2.7x to 4.7x faster end-to-end secure inference than the state-of-the-art protocol, roughly 60% lower communication, and average accuracy within 0.08% of plaintext inference after light fine-tuning.","pith_inferences":["A consequence the paper leaves implicit is that the client-side design moves most of the homomorphic work onto the client, so the reported speedups assume a server-class client; a thin client would see smaller gains, and the paper itself notes the client should have comparable CPU and RAM to the server.","A testable extension is evaluating the distribution-aware approximations under domain shift, since the polynomials are fitted to 512 training tokens and their accuracy on out-of-distribution or adversarial activations is not measured.","The outer-product-with-static-weights idea should carry over to other static-weight architectures such as GPT-style decoders and vision transformers, and to the prefill phase of generative inference; the paper evaluates only BERTbase.","A full parameter set with a noise-budget accounting is needed to reproduce the communication/computation trade-off at a stated security level; without it the speedups are relative to a particular implementation rather than a certified property."],"forward_implications":["If the central claims are correct, secure two-party BERTbase inference becomes 2.7x to 4.7x faster than the previous state-of-the-art protocol across LAN and WAN settings while staying within 0.08% of plaintext accuracy after light fine-tuning.","The one-time cost of sending encrypted weights is amortized: the method overtakes the state-of-the-art baseline after roughly three queries and reaches its maximal speedup around forty queries.","Linear layers improve by 2.9x to 12.5x and non-linear layers by 2.9x to 4.0x over the state-of-the-art protocol, with linear-layer gains largest in LAN settings where computation dominates latency.","Communication per Transformer block falls to about 60% of the state of the art, with linear-layer rounds halved and Softmax and GELU rounds reduced.","For sequence lengths 32 and 128, end-to-end speedups of 1.9x to 7.6x are reported across hidden sizes, though the gain drops to 1.2x to 2.1x for sequence length 1 (the generation phase)."],"supporting_citations":[{"why":"supplies the state-of-the-art two-party Transformer inference baseline and the RLWE-based additive HE scheme that Nimbus builds on","marker":"[26]"},{"why":"defines the server-side inner product (SIP) protocol and window encoding that Nimbus's COP protocol replaces","marker":"[14]"},{"why":"introduces the piecewise polynomial approximation for GELU and exponential that Nimbus's distribution-aware fitting improves upon","marker":"[5]"},{"why":"provides the prior two-party DNN inference framework with NTT-based homomorphic matrix multiplication that the COP complexity analysis is compared against","marker":"[17]"},{"why":"gives the RLWE-based additive homomorphic encryption scheme that the adopted HE scheme is built upon","marker":"[33]"},{"why":"supplies the implementation framework and sub-protocols (comparison, B2A, wrap) used in the evaluation and baselines","marker":"[28]"},{"why":"is the alternative private Transformer inference baseline with SIMD encoding and integer-only approximations compared in the experiments","marker":"[29]"},{"why":"provides the Ferret OT protocol used as the oblivious transfer instantiation for non-linear sub-protocols","marker":"[41]"},{"why":"documents the regularity of activation distributions that motivates the distribution-aware approximation approach","marker":"[24]"}],"fun_headline_variants":["Secure BERT inference: 4.7x faster with Nimbus","Nimbus cuts secure Transformer inference time by up to 4.7x","Faster private BERT: outer-product tricks and polynomial GELU","Nimbus: 4.7x faster two-party BERT inference, 0.08% accuracy loss","Reinventing secure BERT: 4.7x speedup via outer products and low-degree fits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results lean on two load-bearing premises: the lattice-based additive homomorphic encryption built on the ring learning-with-errors (RLWE) problem provides circuit privacy and enough noise budget to sustain up to thousands of scalar-polynomial multiplications per output row without decryption failure or parameter blow-up, and the activation distribution estimated from 512 training tokens is stable enough across layers and test data to make the fixed low-degree polynomial approximations accurate.","fun_headline_variants_meta":{"raw":{"variants":["Secure BERT inference: 4.7x faster with Nimbus","Nimbus cuts secure Transformer inference time by up to 4.7x","Faster private BERT: outer-product tricks and polynomial GELU","Nimbus: 4.7x faster two-party BERT inference, 0.08% accuracy loss","Reinventing secure BERT: 4.7x speedup via outer products and low-degree fits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000226,"raw_usage":{"total_tokens":1526,"prompt_tokens":1062,"completion_tokens":464,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":349}},"tokens_in":678,"tokens_out":464,"duration_ms":4047,"temperature":1.0,"reasoning_tokens":349,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:59:22.962873+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsification: instantiate the COP protocol with a stated $(N,q)$ parameter set and compute the accumulated noise after $m=3072$ scalar-polynomial multiplications on a single ciphertext row; if decryption fails, or the modulus $q$ needed to avoid failure forces ciphertexts larger than the paper's reported communication numbers, the central efficiency claim collapses at that security level.","supporting_citations":[{"cited_title":"Bumblebee: Secure two-party inference framework for large transformers","cited_arxiv_id":null,"evidence_quote":"supplies the state-of-the-art two-party Transformer inference baseline and the RLWE-based additive HE scheme that Nimbus builds on"},{"cited_title":"Iron: Private inference on transformers","cited_arxiv_id":null,"evidence_quote":"defines the server-side inner product (SIP) protocol and window encoding that Nimbus's COP protocol replaces"},{"cited_title":"Cheetah: Lean and fast secure {two-party} deep neural network inference","cited_arxiv_id":null,"evidence_quote":"provides the prior two-party DNN inference framework with NTT-based homomorphic matrix multiplication that the COP complexity analysis is compared against"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"gives the RLWE-based additive homomorphic encryption scheme that the adopted HE scheme is built upon"},{"cited_title":"{SecretFlow-SPU}: A performant and {User- Friendly} framework for {Privacy-Preserving} machine learning","cited_arxiv_id":null,"evidence_quote":"supplies the implementation framework and sub-protocols (comparison, B2A, wrap) used in the evaluation and baselines"},{"cited_title":"Bolt: Privacy- preserving, accurate and efficient inference for transformers","cited_arxiv_id":null,"evidence_quote":"is the alternative private Transformer inference baseline with SIMD encoding and integer-only approximations compared in the experiments"},{"cited_title":"Ferret: Fast extension for correlated ot with small communication","cited_arxiv_id":null,"evidence_quote":"provides the Ferret OT protocol used as the oblivious transfer instantiation for non-linear sub-protocols"}],"review_version":1}