Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation

Bingsheng Zhang; Tianpei Lu; Yuntian Chen; Zhanyong Tang; Zheng Wang; Zhiying Shi

REVIEW 5 major objections 5 minor 1 cited by

Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that mixing homomorphic encryption and secret sharing at sub-operator granularity makes private transformer inference much faster and cheaper to communicate.

desk verdict Promising fine-grained HE/SS protocols, but the headline speedups are an aggregation of microbenchmarks, not a real end-to-end system, and accuracy is plaintext-only. read the letter →

arxiv 2412.16537 v2 pith:UU5IZBL4 submitted 2024-12-21 cs.CR

Yuntian Chen , Zhanyong Tang , Tianpei Lu , Bingsheng Zhang , Zhiying Shi , Zheng Wang This is my paper · ORCID

classification cs.CR

keywords securetwo-partycomputationhomomorphicencryptionsecretsharingprivatetransformerinferencepiecewisepolynomialapproximationBFVBERTGLUE

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FASTLMPI is trying to show that the standard recipe for private transformer inference — homomorphic encryption for linear layers and secret sharing for non-linear ones — leaves large savings on the table. The paper's wager is that switching between the two primitives at a finer granularity, inside each operator rather than between operators, removes the two dominant costs: HE's expensive rotations and SS's heavy interaction. It contributes four two-party protocols for matrix multiplication, SoftMax, LayerNorm, and GeLU, plus a piecewise-fitting rule that places segment boundaries at inflection points. If these claims hold, private BERTbase inference would run about 2.8 times faster on a LAN than the BOLT baseline and move 72.2% less data.

What carries the argument

The load-bearing mechanism is fine-grained collaborative computation: in each protocol, values move between prime-field BFV ciphertexts ($\mathbb{Z}_p$) and ring secret shares ($\mathbb{Z}_{2^k}$), so each sub-step runs in whichever primitive is cheaper. In $\Pi_{\text{matmul}}$, Alice sends SIMD-encrypted rows of her matrix and Bob returns the encrypted product minus a random mask, converting an HE rotation into one communication round and yielding additive shares for the next layer. In $\Pi_{\text{softmax}}$, the expensive exponential is computed over ring shares with an existing subprotocol, then packed into a SIMD ciphertext for the denominator sum and modular inverse. The fitting recipe selects piecewise segment endpoints at inflection points, where the second derivative vanishes, and near-zero third-derivative points, producing lower-degree polynomials with smaller endpoint error.

What would settle it

Run the complete FASTLMPI BERTbase inference on the described LAN and WAN testbeds and compare wall-clock time and bytes sent against the summed microbenchmarks; a total substantially above the reported 131.36 s, 679.90 s, or 1479.82 s, or communication above 20.49 GB, would invalidate the aggregation. Separately, evaluate MRPC, RTE, and SST-2 accuracy through the full secure pipeline; if F1 and accuracy drop below the reported 89.1, 54.5, and 90.7, the plaintext-accuracy assumption fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the cost profile of private transformer inference is governed by where the boundary between HE and SS is drawn, not by the choice of either primitive alone. In FASTLMPI, matrix multiplication avoids all ciphertext rotations: Alice sends SIMD-encrypted copies of her input, Bob multiplies them with his matrix in plaintext and subtracts a random mask, and the result is an additive secret share that feeds the next linear layer. Non-linear operators run the other way, borrowing HE to collapse many secret-sharing interactions into a small number of SIMD ciphertext operations; SoftMax in particular is reported to become 14x faster and 260x cheaper in communication than in BOLT. For GeLU, a five-piece polynomial with breakpoints at the inflection points $\pm\sqrt{2}$ and at $\pm 5.075$ reaches accuracy comparable to degree-6 and degree-7 fits while using at most degree-4 polynomials. The authors report end-to-end BERTbase runtime of 131.36 s on a LAN versus 369.19 s for BOLT, with communication falling from 72.21 GB to 20.49 GB, and GLUE accuracy close to the plaintext baseline without retraining.

Load-bearing premise

The end-to-end speedups rest on the assumption that adding microbenchmark timings reproduces real end-to-end performance, and the accuracy numbers rest on the assumption that plaintext approximation error carries over unchanged to the encrypted fixed-point pipeline.

Editorial extensions

If this is right

End-to-end BERTbase inference is reported to drop from 369.19 s to 131.36 s on a LAN, from 1502.12 s to 679.90 s on a 400 Mbps/10 ms WAN, and from 3784.39 s to 1479.82 s on a 200 Mbps/40 ms WAN.
Total communication falls from 72.21 GB to 20.49 GB, a 72.2% reduction, with the Secure SoftMax protocol alone contributing a roughly 260-fold cut.
The GeLU approximation matches the accuracy of degree-6 and degree-7 fits in prior systems while using lower-degree polynomials, and the same inflection-point recipe fits Sigmoid, Tanh, and Mish more accurately than existing piecewise fits.
Because the four protocols cover the operators shared by decoder-only transformers, the paper argues the framework extends to GPT and BERTlarge without new cryptographic machinery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the paper estimates end-to-end time by summing microbenchmarks in the style of NEXUS, the headline speedups should be checked by a full end-to-end secure execution; field conversions and subprotocol rounds may not be fully captured by the per-operator tables.
The inflection-point segmentation rule is a general numerical recipe: any differentiable activation whose second and third derivatives vanish at identifiable points should be fit better with lower-degree polynomials, independent of the cryptography.
The paper's WAN/LAN comparison implies a tunable design space: deployments with slow networks should lean on HE-style non-interactive steps, while fast local networks should favor SS, and a cost model could pick the mix automatically.
The accuracy evaluation uses plaintext approximation of seg5GeLU, so the untested claim is whether encrypted fixed-point arithmetic preserves those F1 and accuracy numbers.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea here is worth a look: instead of the usual split (HE for linear, SS for nonlinear), the paper mixes both primitives inside each operator. The proposed protocols for matmul, Softmax, LayerNorm, and GeLU are a legitimate extension of BOLT and BumbleBee, and the inflection-point-based segmentation for piecewise polynomial approximation is a neat, cheap heuristic that does appear to lower MAE at equal or lower polynomial degree. The microbenchmarks, if taken at face value, show large per-operator gains. That part is real and useful.

But the headline claims do not hold up as stated. Section IV-C openly says the end-to-end numbers are the aggregation of microbenchmarks, not a measured full-protocol run. That is load-bearing: the pipeline includes Zp-to-Z2k conversions, relinearization inside BFV.Square/HMult, and the rounds and communication of SIRNN and CrypTFlow2 subprotocols. None of those are itemized in the totals, so the 54–64% runtime reduction and 72.2% communication reduction versus BOLT are unsupported. The accuracy evaluation is also plaintext-only; the secure fixed-point pipeline is never measured, and the one reported RTE accuracy (54.5) is far below BOLT's 65.0, which the authors chalk up to model retraining. That is not a fair comparison.

There are also internal inconsistencies that a careful referee would flag: the text claims all homomorphic operations are ciphertext-plaintext multiplications, but the GeLU and LayerNorm protocols use BFV.Square and BFV.HMult on two ciphertexts; the degree of the GeLU polynomials is described as both 3 and 4 in different places; and there is no dedicated security proof for GeLU—the LayerNorm proof just says "same as SoftMax" without the necessary simulator details. The segmentation constants (5.075, 6.48, 4.60) are picked by eyeballing where derivatives get small, and the MAE numbers are in-sample fits. These are fixable, but they make the current version too shaky to accept.

Who is this for? Researchers in private inference who want new protocol components and a plausible approximation trick. They will get ideas from the protocol constructions, but they should not take the end-to-end claims at face value. I would send this to peer review—the novelty is sufficient to warrant referee time—but the verdict should be a major revision, or reject, with the specific demand for a real end-to-end implementation and accuracy measured through the secure pipeline. Without that, the central claims are just arithmetic over optimistic parts.

For your reading group: worth a maybe, mainly to discuss the microbenchmark versus system-evaluation problem. I would not cite it in my own work until the claims are backed by a measured system.

Referee Report

5 major / 5 minor

Summary. The paper presents FASTLMPI, a two-party secure computation framework for private transformer inference, built around four new protocols (Πmatmul, Πsoftmax, Πln, Πgelu) that combine homomorphic encryption (BFV) and additive secret sharing at a finer granularity than prior work, plus a piecewise polynomial approximation method for differentiable non-linear functions such as GeLU, Sigmoid, Tanh, and Mish. The central claims are that these protocols outperform the BOLT baseline by 40×, 14×, 2×, and 2.2× respectively, and that the resulting end-to-end BERTbase inference achieves a 54–64% runtime reduction and a 72.2% communication reduction over BOLT. The evaluation consists of microbenchmarks for individual operators, a plaintext accuracy comparison on GLUE tasks, and an end-to-end table that is explicitly constructed as a sum of microbenchmark timings rather than a full-protocol execution.

Significance. If the claimed end-to-end speedups were verified by a real full-protocol execution and the accuracy were confirmed in the secure fixed-point pipeline, FASTLMPI would represent a valuable step forward in private transformer inference. The core idea of mixing HE and SS inside operators to remove rotations and lower communication is plausible and worth exploring. The paper also ships an anonymous implementation, which aids reproducibility. However, the absence of an end-to-end measurement, the plaintext-only accuracy evaluation, and the in-sample nature of the approximation error analysis mean that the paper's headline quantitative claims are not substantiated as they stand.

major comments (5)

[Section IV-C, Table VI] The end-to-end performance claim (54–64% runtime reduction and 72.2% communication reduction versus BOLT) is not supported by a full-protocol measurement. Section IV-C states that the end-to-end performance is 'the aggregation of the microbenchmarks.' The protocols invoke subprotocols whose costs are not itemized in the aggregate—concretely, Πsoftmax calls ΠrExp from SIRNN, Πgelu calls ΠLT and ΠB2A from CrypTFlow2, Πln calls Πinvsqrt from SCI, and all protocols require Zp ↔ Z2k conversions and truncation after multiplications. The tables also reuse BOLT's BSGS routine for three of the four feed-forward linear layers without a separate cost breakdown. Without executing the full pipeline and counting these conversion and inter-operator costs, the headline speedup and communication numbers are unverified.
[Section IV-C.1, Table V] Accuracy is evaluated in plaintext only, not through the secure fixed-point protocols. Table V compares floating-point plaintext, a BERT baseline, BOLT, and FASTLMPI, but the FASTLMPI row uses the seg5GeLU approximation in plaintext arithmetic. The actual pipeline uses fixed-point representation with s=12, k=37, modular arithmetic over Zp and Z2k, and the approximate subprotocols ΠrExp and Πinvsqrt, all of which can introduce errors not captured by a plaintext simulation. The reported RTE accuracy of 54.5 versus BOLT's 65.0 already indicates a substantial gap, and the paper's note that this may be due to BOLT retraining does not establish that the secure FASTLMPI pipeline preserves accuracy.
[Section IV-B.3, Fig. 7] The MAE evaluation of seg5GeLU is in-sample and therefore not a valid measure of approximation quality. The polynomials are fitted with numpy.polyfit, and the MAE is then computed on 10,000 points spanning the same range (-6 to 6) used for the fit. This measures training error, not generalization error; a comparison with BOLT or Bumblebee would require evaluation on a held-out set or an independent grid. Likewise, the choice of segmentation endpoints such as ±5.075 is justified by inspecting where derivatives become small, which is an ad hoc heuristic; the paper does not demonstrate that these endpoints generalize across input distributions.
[Section VII versus Sections III-C and III-E] The conclusion claims that 'all homomorphic operations are transformed into ciphertext-plaintext multiplications,' but this is contradicted by the protocol specifications. In Πln, the implementation uses BFV.Square to compute [ [eA²] ]B, which is a ciphertext-ciphertext operation, and in Πgelu it uses BFV.HMult([ [X²] ]A, [ [X] ]A) to obtain [ [X³] ]A. These are not ciphertext-plaintext multiplications. This inconsistency undermines the stated design rationale and should be corrected or qualified.
[Section VIII, Theorems 1–3] The security proofs are incomplete and contain apparent errors. Theorem 3 (LayerNorm) is dismissed with a single sentence: 'The proof process is the same as that of SoftMax.' The proof of Theorem 2 (SoftMax) describes the corrupted-B case as having B receive [ [⟨Y⟩B] ]B, but the protocol specification has A send [ [⟨Y⟩B] ]B to B; the simulator's action of 'receiving' this from the corrupted party does not match the protocol flow. A security paper needs complete, correct simulation-based proofs for each protocol, especially when the protocols combine HE and SS with subprotocol conversions.

minor comments (5)

[Throughout] The manuscript contains many typos and inconsistent notations: 'LayrNorm' for LayerNorm, 'DESCIRPTION' in the Section III heading, 'Ret-BOLT' in Table V, 'Modeify' and 'Sid' in the security proofs, and inconsistent use of ∧ versus ♢ for XOR.
[Section IX-B, Eq. (34)] The definition of F1(x) for the Sigmoid approximation appears to be missing an x² term: '0.4998102695 + 0.2527736008x − 0.00869807952 − 0.0127621849x³' has a constant term where a quadratic term should likely appear, making the polynomial degree inconsistent with the text.
[Section III-A, Eq. (14)] Equation (14) uses the symbol ⊠ for a homomorphic multiplication in two terms while the surrounding text and Fig. 3 use ⊡; this should be unified for readability.
[Section II-A] The notation section states that XOR is denoted by ∧, but Eq. (28) and the Πgelu text use the diamond symbol ♢ for XOR; the notation should be consistent.
[Section IV-B.2] The GeLU comparison labels BOLT's GeLU as 'the version without preprocessing,' but the main comparison in Table IV and the abstract treats this as the standard BOLT baseline; the distinction should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the protocols are compositions of external primitives benchmarked against external baselines, and the in-sample polynomial fitting is explicitly reported as fitting, not as an independent prediction.

full rationale

FASTLMPI's claimed derivations are not circular. The four protocols are specified as compositions of external primitives (BFV/SEAL, Pi_rExp from SIRNN, Pi_LT and Pi_B2A from CrypTFlow2, Pi_invsqrt from SCI) and are benchmarked against external baselines (BOLT, IRON, FITFC, SCIOT); no parameter is fitted to the headline 54-64% runtime reduction or 72.2% communication reduction. The piecewise GeLU/Sigmoid/Tanh/Mish polynomials are obtained with numpy.polyfit on the target function, but the paper does not present the resulting MAE as an independent prediction; it reports in-sample approximation error and then separately evaluates model accuracy on GLUE in plaintext. The statement in Section IV-C that 'the end-to-end performance is the aggregation of the microbenchmarks' is an explicit accounting rule, not a hidden identification of a fitted parameter with a predicted outcome. It may understate costs if conversions, key switching, or subprotocol rounds are omitted, but that is a completeness and verification limitation, not a circular derivation. No load-bearing self-citation or imported uniqueness claim is used to force the central results. Accordingly, no step satisfies the quoted-reduction standard required to establish circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several hand-picked segmentation constants and fitted polynomial coefficients, plus the security and correctness of external subprotocols. No new physical or cryptographic entities are introduced. The main free parameters are the segmentation endpoints and polynomial coefficients, which are tuned to the target functions and then evaluated on the same data range.

free parameters (6)

GeLU outer segmentation points ±5.075 = ±5.075
Chosen by observing that the second and third derivatives of GeLU become small near these values. Not derived from an equation; a hand-picked threshold.
Sigmoid outer segmentation point 6.48 = 6.48
Selected as the point where y'' < 1e-5. An empirical cutoff.
Tanh outer segmentation point 4.60 = 4.60
Selected as the point where y'' < 1e-5. An empirical cutoff.
Mish segmentation points = -2.2563763963607935, 1.4905711794854284
Zero points of Mish's second derivative obtained by dichotomy, not closed-form.
Polynomial coefficients for all fitted activations = Listed in Appendix IX
Coefficients are fitted via numpy.polyfit on the target function; they are not independent of the accuracy evaluation.
Epsilon robustness constant = 1e-5
Added to the GeLU approximation to avoid zero values; chosen by hand.

assumptions (5)

standard math BFV scheme is semantically secure under the RLWE assumption.
Invoked for all homomorphic operations; a standard cryptographic assumption.
domain assumption Semi-honest, static adversary model with simulation-based security.
The paper only claims security against semi-honest adversaries; malicious security is out of scope.
domain assumption Fixed-point encoding with s=12 bits and ring size k=37 does not overflow in conversions between Zp and Z2k.
Section II-D.3 states that overflow probability is small if x is small or the ring is large; the paper assumes this holds for the chosen parameters.
domain assumption Subprotocols from SIRNN (ΠrExp, Πinvsqrt) and CrypTFlow2 (ΠLT, ΠB2A) are secure and correct.
FASTLMPI composes these external protocols without re-proving their security. The security of the composition is assumed.
standard math numpy.polyfit least-squares fitting produces appropriate polynomial approximations for each segment.
The approximation accuracy is evaluated over the same domain used for fitting.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation." pith.science (2026). https://pith.science/paper/UU5IZBL4

@misc{pith2026241216537,
  author       = {Pith},
  title        = {Pith review of: Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UU5IZBL4}},
  note         = {Machine review of arXiv:2412.16537}
}

read the original abstract

Homomorphic encryption (HE) and secret sharing (SS) enable computations on encrypted data, providing significant privacy benefits for large transformer-based models (TBM) in sensitive sectors like medicine and finance. However, private TBM inference incurs significant costs due to the coarse-grained application of HE and SS. We present FASTLMPI, a new approach to accelerate private TBM inference through fine-grained computation optimization. Specifically, through the fine-grained co-design of homomorphic encryption and secret sharing, FASTLMPI achieves efficient protocols for matrix multiplication, SoftMax, LayerNorm, and GeLU. In addition, FASTLMPI introduces a precise segmented approximation technique for differentiable non-linear, improving its fitting accuracy while maintaining a low polynomial degree. Compared to solution BOLT (S&P'24), FASTLMPI shows a remarkable 54% to 64% decrease in runtime and an impressive 72.2% reduction in communication costs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Breaking the Layer Barrier: Remodeling Private Transformer Inference with Hybrid CKKS and MPC
cs.CR 2025-08 conditional novelty 7.0 of 10

BLB combines operator-level fusion with a new secure CKKS-to-MPC conversion, reporting up to 21x less communication and 13x lower GPU latency than BOLT for private Transformer inference.

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.