{"id":"5c6d311d-6300-4575-be45-6a66c62fa0cd","arxiv_id":"2412.10652","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"CENTAUR speeds up privacy-preserving Transformer inference by permuting model weights and secret-sharing the input, at the cost of replacing provable privacy with empirical attack resistance.","lead":"CENTAUR combines random permutation with secure multi-party computation (SMPC) to speed up privacy-preserving Transformer inference while keeping model accuracy unchanged, reporting 5 to 30 times faster inference than existing SMPC-based frameworks. The catch is that the privacy guarantee is empirical, not mathematical, because some intermediate results are exposed to the cloud in permuted plaintext form.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Public model checkpoints let the cloud recover the permutation from transmitted weights, so the claimed DRA-resistance is falsifiable in the paper's own BERT/GPT-2 setting.","rationale":"The reader's weakest assumption is that the distance-correlation proof in Appendix B is misapplied to deterministic weights. That is a valid concern, but an even more concrete and experimentally decisive problem exists: the permutation that is supposed to protect all intermediate results is recoverable from the permuted parameters alone whenever the underlying model weights are public, which is exactly the case for the BERT and GPT-2 benchmarks used in the paper. Since P1 receives every permuted weight matrix in the clear, and since the original weights are available for download, column matching recovers π and π2 to high accuracy. With π known, the permuted intermediates that the paper claims are private become plaintext intermediates, and the paper's own control experiments show that those plaintext intermediates leak 60–100% ROUGE-L under SIP, EIA, and BRE. This is not a mere gap in a proof; it is a concrete attack against the experimental configuration the paper reports. The Appendix B issue is real but secondary: even a correct distance-correlation inequality would not address the public-weight recovery attack, because the permutation is not secret from an adversary who can compare Θ′ with publicly known weights. For proprietary, non-public model weights the attack would not apply directly, but the paper neither states that restriction in its central claim nor evaluates a proprietary model; its empirical privacy case is built on public checkpoints. The distance-correlation argument also remains unsound for deterministic weights, and the paper itself disclaims theoretical security. Taken together, the central 'impossible trinity' claim is not supported: the efficiency and accuracy results may be reproducible, but the privacy leg fails in the demonstrated setting. The appropriate remedy would be to either restrict the privacy claim to weights that are not publicly available and demonstrate security under that explicit assumption, or to redesign the protocol so that permutation secrecy does not depend on the weights being unknown. As submitted, the paper should be rejected; a revised version with a corrected threat model and security evaluation could be reconsidered.","tokens_in":24898,"tokens_out":10620,"duration_ms":110412,"concrete_test":"Reproduce the CENTAUR initialization for BERT-large/QNLI with a randomly generated π. Give the adversarial P1 only the transmitted Θ′ (WQπ, WKπ, WOπ, etc.). Download the official BERT-large checkpoint, convert to the same fixed-point representation used in CrypTen, and recover π by maximizing column-wise correlation between Θ′ and the public weights. Measure the exact permutation recovery rate. If recovery is above 99%, take the protected intermediates O4π, O5π2, O6π from Table 1, un-permute with the recovered π and π2, and rerun SIP, EIA, and BRE. If ROUGE-L rises from the reported ~1–6% to >30–60%, the central privacy claim is falsified in the paper's own benchmark setting.","verdict_should_be":"REJECT","load_bearing_attack":"The privacy of inference data rests entirely on the secrecy of the permutation matrices, but P1 receives, in the clear, all permuted model parameters (Section 4.2): WQπ, WKπ, WVπ, WOπ, W1π2, W2π2, and LayerNorm vectors γπ, βπ. The same π is reused across the attention projections and the same π2 across the FFN matrices. For the BERT and GPT-2 checkpoints used in the experiments, the original weights are publicly downloadable. A semi-honest P1 can therefore recover π and π2 by simple column matching: each column of WQπ is exactly (up to fixed-point quantization) a column of the public WQ, so correlating columns identifies the permutation; even with quantization, nearest-neighbor matching succeeds. Once π is known, every feature-permuted intermediate that reaches the nonlinear layers in the clear—Xπ before GeLU/LayerNorm, O4π, O5π2, O6π—can be un-permuted before applying SIP/EIA/BRE, which the paper shows recover 60–100% ROUGE-L from plaintext intermediates (Table 1, 'W/O'). Appendix C's cracking analysis considers only attacks on intermediate activations and never considers the much easier attack from Θ′ itself. Separately, Appendix B's distance-correlation argument (Eq. 5) averages over random WA, while CENTAUR uses fixed deterministic weights, so the claimed bound does not transfer. The net effect is that the privacy leg of the 'impossible trinity' is neither proven nor empirically established in the very setting the paper evaluates.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CENTAUR, a three-party framework for privacy-preserving Transformer inference that combines random permutation of model parameters with secret sharing of inference data. The central claim is that it resolves the \"impossible trinity\": it allegedly protects both model parameters and user data, achieves plaintext-equivalent accuracy, and is 5.0–30.4 times faster than SMPC-based PPTI baselines on BERT and GPT-2. The efficiency claim is supported by detailed runtime and communication breakdowns across LAN/WAN settings, and the performance claim is supported by exact parity with plaintext inference on GLUE and Wikitext benchmarks. The privacy claim rests on the assertion that randomly permuted intermediate results and permuted model parameters do not leak user inference data, supported by a distance-correlation argument in Appendix B and by data-reconstruction attack experiments in Section 5.2 and Appendix C. The paper's own Limitations section disclaims theoretical security and states that privacy is supported only by empirical attack evaluations.","tokens_in":25205,"tokens_out":4844,"duration_ms":46411,"significance":"If the privacy claim were sound, the efficiency and performance results would be a significant practical contribution: the runtime reductions are large, the performance parity is exact, and the extension to LLaMA-7B shows generality. The efficiency and performance experiments are the paper's genuine strengths, and the DRA evaluation is thoughtfully designed as a benchmark suite. However, the privacy leg of the central claim is not established, and in the paper's own experimental setting it is demonstrably false: the cloud party P1 receives all permuted model parameters in the clear, and for the public BERT/GPT-2 checkpoints used in the experiments, those parameters permit recovery of the permutation. Since privacy is one of the three pillars of the claimed contribution, the paper's central result fails in the setting it evaluates. The significance of the remaining efficiency/performance results does not compensate for the unsupported core claim.","major_comments":[{"comment":"The cloud platform P1 receives, in the clear, all permuted model parameters WQπ, WKπ, WVπ, WOπ, W1π2, W2π2, and LayerNorm vectors γπ, βπ. The experiments use publicly downloadable BERT and GPT-2 checkpoints. A semi-honest P1 can therefore recover the secret permutation matrices π and π2 by simple column matching: each column of WQπ is, up to fixed-point quantization, a column of the public WQ, so correlating columns identifies the permutation. Once π is known, every permuted intermediate result sent to P1 in the clear—Xπ before GeLU/LayerNorm, O4π, O5π2, O6π—can be un-permuted, reducing the attack to the \"W/O\" rows of Table 1, which recover 60–100% ROUGE-L on plaintext intermediates. Appendix C's cracking analysis considers only attacks on intermediate activations and never considers key recovery from Θ′ itself. This is a load-bearing omission: the privacy claim in Section 5.2 is invalid in the exact setting the paper evaluates.","section":"Section 4.2 and Appendix C"},{"comment":"The distance-correlation argument applies an inequality from Zheng et al. (2022) that involves an expectation over random weight matrices WA. CENTAUR uses fixed, deterministic pre-trained Transformer weights, so the expectation does not transfer. Even if the inequality did transfer, a bound on distance correlation against a one-dimensional projection is not a demonstrated proxy for resistance to optimization-based and learning-based data reconstruction attacks. The paper's own Limitations section concedes that CENTAUR \"cannot directly achieve theoretical security,\" so the privacy guarantee reduces entirely to the empirical attack experiments; those experiments are undermined by the key-recovery attack described above. This step, not a mere presentation issue, is load-bearing for the stated privacy contribution.","section":"Appendix B, Eq. (5)"},{"comment":"The empirical security evaluation compares attacks on permuted intermediates against attacks on random matrices and concludes that CENTAUR's intermediates are \"comparable to random inputs.\" This conclusion presupposes that the attacker does not know the permutation. In the deployment model of Section 4.2, however, P1's own input includes Θ′, and for the public checkpoints used in the experiments the permutation is recoverable from Θ′. The \"Rand\" control is therefore not the appropriate baseline for the stated threat model. The paper's answer to Q1 is consequently not established for a semi-honest cloud platform, which is precisely the adversary CENTAUR is designed to protect against.","section":"Section 5.2 and Table 1"}],"minor_comments":[{"comment":"Algorithms 4 and 5 are captioned \"Privacy-preserving Embedding (ΠEmbedding)\" and \"Privacy-preserving Adaptation (ΠAdaptation)\", but the text in Section 4.3.2 calls them ΠPPEmbedding and ΠPPAdaptation. Please unify the notation.","section":"Appendix A"},{"comment":"Table 1 reports \"ROUGE-L F1 Score (%)\" while Table 3's caption says \"RougeL-F%\"; Section 5.2 also states that ROUGE-L ranges from 0 to 1 but reports values such as 100.00. Please standardize the metric notation and units across captions, tables, and text.","section":"Tables 1 and 3"},{"comment":"The bias terms are written inconsistently: Eq. (3) displays +BOπ, +B1π2, and +B2π, while the footnote says biases can be secretly shared and added with ΠAdd. Clarify whether biases are permuted, shared, or both, because this affects the correctness of the permuted-state computations.","section":"Section 4.3.1, Eq. (3)"},{"comment":"The paper does not state whether source code will be released. Given that the privacy claim is contested, releasing the implementation and attack-evaluation code would materially improve reproducibility and would allow independent verification of the efficiency and attack results.","section":"General"}],"recommendation":"reject","confidential_remarks":"The efficiency and performance experiments are solid and likely publishable as a systems contribution, but the privacy pillar of the \"impossible trinity\" fails in the paper's own experimental setting: a semi-honest P1 holding Θ′ can recover the permutation from public checkpoints, and Appendix B provides no transferable theoretical bound. Fixing this would require a different protocol (e.g., secret-shared weights), which would remove the headline efficiency gain, or a substantially narrowed claim that excludes public models. Both are outside the scope of a minor revision, so I recommend rejection, although I would encourage the authors to resubmit a version focused on the efficiency/performance results with an honest, restricted privacy statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The efficiency and performance work is real, but the privacy claim collapses under the simplest attack. CENTAUR permutes the weights and sends them to P1. For BERT/GPT-2 the original weights are public, so P1 can recover the permutation by column matching—each column of WQπ is a column of WQ. Once π is known, every permuted intermediate that reaches the nonlinear layers in the clear can be un-permuted and fed to SIP/EIA/BRE, which the paper shows recover 60–100% ROUGE-L from plaintext intermediates. The paper's cracking analysis never considers this attack; it only tries to crack permutations from intermediate activations. So the 'privacy' leg of the trinity is not empirically established in the exact setting the paper evaluates.\n\nWhat's genuinely new: converting linear layers to communication-free plaintext-times-secret-share multiplications, and computing Softmax/GeLU/LayerNorm on permuted states, with only two rounds of communication per nonlinearity. The speedups (5–30x over PUMA/MPCFormer/SecFormer) are plausible and the accuracy matches plaintext exactly. That is a meaningful engineering contribution, and the LLaMA extension is a nice bonus.\n\nSoft spots beyond the public-weights attack: Appendix B's distance-correlation argument averages over random weight matrices, but CENTAUR uses fixed deterministic weights, so the bound doesn't transfer. The paper's own Limitations section concedes there is no theoretical security. Missing code and error bars are minor and addressable. The novelty relative to Zheng et al. 2022 is incremental, but the Transformer-specific protocol is new.\n\nWho should read this: anyone working on practical PPTI. The threat model needs to be rewritten—either assume the model parameters are private (which CENTAUR doesn't protect) or add a proper formal argument for why public weights don't help the attacker. As written, the privacy claim is not credible for public checkpoints.\n\nIt deserves serious peer review: the efficiency result is strong and the privacy failure is an instructive, publishable lesson. But acceptance would require major revision, likely reframing the contribution as 'efficient transformer inference with a weaker, empirical privacy notion in a restricted threat model,' and honestly reporting the public-weights attack.","headline":"Real efficiency gains and exact accuracy, but the privacy guarantee is broken by the public-checkpoint attack, so the 'trinity' claim does not hold.","tokens_in":25725,"tokens_out":3385,"would_cite":false,"duration_ms":28868,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CENTAUR claims that a hybrid of random permutation and secure multi-party computation bridges the \"impossible trinity\" of privacy, efficiency, and performance in Transformer inference.","keywords":["privacy-preserving inference","Transformer","random permutation","secret sharing","secure multi-party computation","data reconstruction attack","hybrid privacy mechanism"],"falsifier":"Run SIP, embedding-inversion, and BRE attacks on the permuted intermediate results of the first transformer block of BERT-large or GPT-2-large and compare with the paper's random-input control: if any attack recovers more than about 10% ROUGE-L F1 on the permuted state, the central privacy claim fails.","tokens_in":24694,"feed_emoji":"🔐","tokens_out":9694,"duration_ms":76954,"temperature":0.7,"pith_summary":"The paper argues that the \"impossible trinity\" of privacy, efficiency, and performance in privacy-preserving Transformer inference can be bridged by a hybrid scheme: keep model parameters randomly permuted so the cloud never sees the true weights, keep user data in secret shares, and only convert to permuted plaintext for nonlinear operations. The central claim is that intermediate activations in this permuted state resist state-of-the-art data reconstruction attacks about as well as random noise, while inference runs 5.0–30.4 times faster than existing secure-MPC frameworks and matches plaintext accuracy without retraining. If correct, this gives cloud model-as-a-service deployments a practical middle path that hides both proprietary weights and user inputs without sacrificing accuracy or speed.","feed_headline":"Hybrid scheme runs private Transformer inference 30x faster","feed_subtitle":"Random permutations plus secret sharing keep accuracy at plaintext level while resisting data-reconstruction attacks.","key_machinery":"The central object is the hybrid state machine: model parameters live as permuted plaintext matrices, while inference data and intermediate values live as secret shares, with a small set of protocols ($\\Pi$ScalMul for plaintext–share multiplication, $\\Pi$MatMul for shared matrix multiplication, $\\Pi$PPP for moving between states, and $\\Pi$PPSM, $\\Pi$PPGeLU, $\\Pi$PPLN for nonlinearities) controlling the transitions. Two mathematical facts carry the argument: multiplying a matrix by a permutation $\\pi$ permutes its rows or columns and is orthogonal, so linear layers can absorb the permutation at zero communication cost; and any elementwise nonlinearity $f$ satisfies $f(X\\pi)=f(X)\\pi$, so nonlinear layers can be computed on reconstructed permuted values. The distance-correlation inequality Eq. (5) is invoked to bound leakage from permuted intermediate results by the leakage of a one-dimensional projection.","core_discovery":"On the paper's own terms, the discovery is that the conflict between SMPC's strong privacy and high cost and permutation-based methods' low cost and weak privacy is not inherent: a carefully choreographed alternation between secret-shared and randomly-permuted states lets each operation run in its cheapest secure form. Linear layers become communication-free multiplications between permuted plaintext weights and shared data; nonlinearities like softmax, GeLU, and LayerNorm run directly on permuted plaintext because permutation commutes with elementwise functions; and a privacy-preserving permutation protocol restores correctness where the permutation would otherwise cancel out. The paper claims this yields plaintext-identical accuracy on BERT and GPT-2, 5–30x faster inference than SMPC baselines, and empirical resistance to SIP, embedding-inversion, and BRE attacks on all four permuted intermediate targets.","pith_inferences":["A natural stress test beyond the paper is a known-plaintext attack on the first embedding layer: if an adversary controls one query whose true embedding is known, the effective permutation key space may shrink enough to make partial recovery of the permutation feasible.","The same permutation/secret-sharing alternation should generalize to other mostly-linear architectures with permutation-equivariant nonlinearities, such as state-space models or certain diffusion backbones, though the paper only demonstrates Transformer variants.","The efficiency numbers assume the cloud is semi-honest and does not collude with the model developer; under collusion the scheme degrades to permutation-only security, so the no-collusion threat model is load-bearing for the deployment story.","Because Eq. (5) is applied to deterministic Transformer weights, a re-derivation for fixed matrices would either substantiate or undermine the theoretical privacy bound; the empirical attack results alone cannot arbitrate."],"forward_implications":["Cloud deployment of BERT- and GPT-2-class models can serve inference with plaintext accuracy and no retraining or distillation, closing the performance gap that forced prior schemes to approximate softmax or GeLU.","In wide-area networks the speed advantage grows to 6.3–30.4x for BERT-large and 5.8–28.4x for GPT-2-large, because the scheme cuts communication volume by roughly 2.4–37.6x.","The approach transfers to newer architectures without redesign: RMSNorm and SwiGLU are also permutation-equivariant, and the paper reports LLaMA-7B inference under 3 seconds per generated token at 0.39 GB communication in its LAN setup.","The privacy guarantee is empirical, not provable: it holds against the specific SIP, embedding-inversion, and BRE attack families tested, and the paper explicitly disavows theoretical security in its limitations section."],"supporting_citations":[{"why":"It supplies the distance-correlation inequality Eq. (5) that bounds leakage of permuted intermediate results by a one-dimensional projection.","marker":"Zheng et al. 2022"},{"why":"It defines the permutation-based PPTI baseline CENTAUR builds on and whose exposed intermediate results create the privacy gap.","marker":"Yuan et al. 2023"},{"why":"It provides Beaver triples, the correlated randomness underlying the secret-shared multiplications.","marker":"Beaver 1992"},{"why":"It provides the universal-composability theorem used to compose the share-state subprotocols into a secure protocol.","marker":"Canetti 2001"},{"why":"It supplies the MPCFormer baseline that replaces nonlinearities with quadratics, showing the performance cost CENTAUR avoids and giving a comparison point for efficiency.","marker":"Li et al. 2023"},{"why":"It supplies the PUMA baseline with optimized SMPC protocols for nonlinear layers, used as the strongest efficiency comparison.","marker":"Dong et al. 2023"},{"why":"It supplies the SecFormer baseline that approximates softmax with quadratics, used as a comparison for accuracy and speed.","marker":"Luo et al. 2024"},{"why":"It provides the Embedding Inversion Attack, one of the three DRA families used to show that plaintext intermediate results leak and permuted ones do not.","marker":"Song and Raghunathan 2020"},{"why":"It supplies the SIP and BRE attack methods, the other two DRA families in the empirical privacy evaluation.","marker":"Chen et al. 2024"}],"fun_headline_variants":["Hybrid secret-sharing and permutation runs private transformers 30x faster","Alternating shuffles and secret sharing makes private inference plaintext-fast","CENTAUR: 30x faster private transformer inference via hybrid SMPC-permutation","Secret-shared and permuted states shatter the impossible trinity","Choreographed shuffles and secret sharing give plaintext-accuracy private inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an intermediate result in a randomly permuted state, which the cloud sees in plaintext, provides meaningful privacy for the user's inference data; the paper's own limitations section concedes this is not theoretical security, and the theoretical argument rests on a distance-correlation bound originally derived for random matrices being applied to fixed Transformer weights.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid secret-sharing and permutation runs private transformers 30x faster","Alternating shuffles and secret sharing makes private inference plaintext-fast","CENTAUR: 30x faster private transformer inference via hybrid SMPC-permutation","Secret-shared and permuted states shatter the impossible trinity","Choreographed shuffles and secret sharing give plaintext-accuracy private inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000747,"raw_usage":{"total_tokens":3315,"prompt_tokens":918,"completion_tokens":2397,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":2299}},"tokens_in":534,"tokens_out":2397,"duration_ms":14561,"temperature":1.0,"reasoning_tokens":2299,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:44:24.523992+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SIP, embedding-inversion, and BRE attacks on the permuted intermediate results of the first transformer block of BERT-large or GPT-2-large and compare with the paper's random-input control: if any attack recovers more than about 10% ROUGE-L F1 on the permuted state, the central privacy claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the distance-correlation inequality Eq. (5) that bounds leakage of permuted intermediate results by a one-dimensional projection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides Beaver triples, the correlated randomness underlying the secret-shared multiplications."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the universal-composability theorem used to compose the share-state subprotocols into a secure protocol."}],"review_version":1}