REVIEW 3 major objections
Layer-parallel SNLP cuts encrypted Transformer bootstrap depth by about 2.65x with only modest perplexity loss.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-11 13:02 UTC pith:U7LX5WSE
load-bearing objection Useful FHE-depth application of prior SNLP: simulation shows lower sequential nonlinear depth and less error amplification, with the main caveat being omitted real-CKKS softmax circuits rather than a broken argument. the 3 major comments →
Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
SNLP reduces the sequential nonlinear depth of encrypted Transformer inference from L stages to (L-N)+K stages while preserving usable language-model quality. On a 0.5B IDN-trained model under degree-12 Chebyshev approximation, the n24-K4 configuration lowers symbolic bootstrap count from 53 to 20 (2.65x) with +1.2% perplexity degradation and lower error amplification (1.36x vs 1.42x sequential); the same lower-amplification pattern holds across all eight tested models.
What carries the argument
NFE (Nonlinear Forward Evaluations) = (L-N)+K, together with SNLP's Identity-Newton (purely additive) or HC-Newton (small linear stream mixing) corrections. Because the corrections add essentially zero multiplicative depth, NFE becomes a faithful proxy for bootstrap count and lets the method trade sequential nonlinear depth for a small number of parallel solver iterations.
Load-bearing premise
The plaintext Chebyshev simulation (with simplified RMSNorm noise and omitted comparison-circuit costs) still ranks sequential versus SNLP error and bootstrap cost the same way real CKKS encryption would.
What would settle it
Implement both sequential and SNLP n24-K4 paths under actual CKKS for the same 0.5B model and check whether measured bootstrap counts and relative perplexity amplification still favor SNLP by roughly the reported margins.
If this is right
- Encrypted inference can target a depth-quality frontier distinct from the ordinary GPU wall-clock frontier, sometimes preferring more solver iterations if they cut sequential nonlinear depth.
- Architectures whose residual-stream mixing is purely linear (e.g., mHC) already reduce nonlinear FHE depth and pair naturally with SNLP.
- SNLP-aware training is required to reach the low-NFE regime; models without it stay near the sequential depth.
- Because softmax dominates local approximation error, SNLP gains remain available even if better polynomial or gated-attention replacements improve the blocks themselves.
Where Pith is reading between the lines
- The same depth-reduction idea could be applied to other deep residual stacks whose expensive nonlinearities are sequential (e.g., encrypted vision or speech transformers).
- If larger models continue the observed trend of lower error amplification, SNLP's relative advantage may become easier to harvest at production scale.
- FHE training objectives that directly penalize sequential nonlinear depth rather than wall-clock time could further shift the usable (N,K) frontier.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies whether Structured Newton Layer Parallelism (SNLP) can reduce the sequential nonlinear depth of encrypted Transformer inference under FHE. Rather than redesigning per-block operators (softmax, RMSNorm, activations), SNLP rewrites the inter-layer graph so that a prefix is sequential and a suffix is solved over K Newton-style iterations with cheap structured corrections (IDN/HCN). The authors introduce NFE = (L−N)+K as a proxy for sequential nonlinear stages, show in a symbolic CKKS model that NFE tracks bootstrap count within ~1–2%, and measure error accumulation via Chebyshev polynomial replacements in plaintext PyTorch across 8 Nanochat models (0.5B–3B, four architecture families). Headline result: on a 0.5B IDN-trained model at degree 12, SNLP n24-K4 cuts symbolic bootstraps 53→20 (2.65×) with reported +1.2% PPL degradation and lower error amplification (1.36× vs 1.42×); SNLP has lower amplification than sequential on all tested models. Ablations attribute most error to softmax; simulated CKKS arithmetic noise is negligible.
Significance. If the relative depth and error claims hold under more realistic FHE accounting, this is a useful complementary axis for encrypted Transformer inference: prior FHE-ML work optimizes per-block polynomial approximations and circuit depth, while SNLP targets the L-fold sequential composition of those blocks. The NFE metric, the explicit separation of FHE-optimal vs wall-clock-optimal (N,K) points, the observation that linear residual-stream mixing (mHC) is inherently FHE-friendly, and the controlled ablations (softmax-dominated error; negligible CKKS noise) are concrete contributions. Strengths include a clear symbolic cost model (Table 1), multi-model evaluation (Table 5), degree and noise sweeps (Tables 3–4), Pareto (N,K) analysis (Figure 1), and an honest Limitations section. The work is simulation-based rather than a full CKKS implementation, so significance is as a structural study of inter-layer depth, not as a deployable encrypted system.
major comments (3)
- §4.2 and §6 acknowledge that softmax max-subtraction (comparison) and renormalization (reciprocal) are omitted from NFE and from the simulated error, yet the abstract and Table 1 still report a 2.65× symbolic bootstrap reduction. The manuscript asserts that identical per-block approximations preserve the sequential-vs-SNLP ranking, but does not give an explicit depth accounting showing that these omitted circuits attach once per nonlinear stage and therefore scale with NFE = (L−N)+K rather than with total block evaluations K·N. Because Table 2 shows softmax dominates the error budget, this is the load-bearing place where relative bootstrap and amplification rankings could shift under real CKKS. Please add a short critical-path argument (and, if possible, a sensitivity experiment that injects a fixed extra mult-depth or reciprocal approximation per nonlinear stage) so the 53→20 claim is n
- Abstract, §5 (NFE vs PPL), and Figure 1 caption report “+1.2% perplexity degradation” for n24-K4 alongside lower error amplification (1.36× vs 1.42× in Table 3 / Table 5). Lower amplification implies lower HE-PPL for SNLP than for sequential HE, so +1.2% cannot simultaneously mean degradation relative to sequential under the same HE approximation. Clarify the exact reference: (i) SNLP exact-arith vs sequential exact after IDN training, (ii) SNLP-HE vs sequential-HE, or (iii) SNLP-HE vs sequential exact. Reconcile the number with Table 5 (exact PPL 14.07, amps 1.419 / 1.355) and state it uniformly in abstract, bullets, and figure caption. This is load-bearing for the headline quality claim.
- Table 5 and Figure 1 show that the aggressive n24-K4 operating point (NFE 12, 2.65× bootstraps) is available only for IDN/HCN-regularized models; baselines are limited to n8-K4 (1.14×). The abstract correctly names the “0.5B IDN-trained model” for the headline, but the broader claim that SNLP reduces encrypted nonlinear depth should more sharply separate (a) structural depth reduction conditional on SNLP-aware training from (b) the milder, training-free configurations. Without that separation, readers may over-generalize the 2.65× figure to untuned checkpoints. A short subsection or table column making “training required for NFE≤16” explicit would fix this.
Circularity Check
No significant circularity: bootstrap counts restyle NFE by a linear cost model, but the load-bearing PPL/amplification claims are independent measurements under fixed approximations.
specific steps
-
self definitional
[§4.1 FHE Cost Model and NFE; Table 1]
"We define NFE(Nonlinear Forward Evaluations) as the number of sequential nonlinear stages in the computation graph: NFE_seq = L, NFE_SNLP = (L−N) + K ... With 15 usable CKKS levels before bootstrapping, the number of bootstraps scales linearly with NFE. In this symbolic model, NFE tracks the bootstrap count within 1–2% across all tested configurations (Table 1)."
Bootstrap counts in the symbolic model are defined to scale linearly with NFE (IDN correction contributes zero multiplicative depth by design). Reporting that NFE tracks bootstraps at 0.99–1.02×, and headline reductions such as 53→20 (2.65×) for n24-K4, therefore largely restate the NFE reduction under that linear map rather than an independent FHE measurement. This is a minor definitional restyling of the depth claim; it does not force the separate empirical PPL or amplification results.
full rationale
The paper’s central scientific content is empirical comparison of error accumulation (PPL_HE / PPL_exact and absolute PPL under degree-12 Chebyshev replacements) for sequential vs SNLP forward paths across 8 models. Amplification is defined against exact-operation PPL and is not fitted to force SNLP superiority; both paths share the same per-block polynomial code, so relative differences come from computation-graph structure. Dependence on Han et al. (2026) for the SNLP construction and SNLP-aware training is ordinary prior-work citation, not a uniqueness theorem or ansatz smuggled in as external fact, and does not make the FHE measurements tautological. The only mild definitional loop is that symbolic bootstrap counts are computed from NFE under the assumption that bootstraps scale linearly with NFE (IDN adds zero multiplicative depth), so “NFE tracks bootstraps within 1–2%” and “53→20 bootstraps” largely restate the NFE reduction. That does not collapse the quality claims. Score 1 reflects that minor restyling only.
Axiom & Free-Parameter Ledger
free parameters (5)
- Chebyshev polynomial degree d =
12 (default); tested {8,10,12,14}
- CKKS usable levels before bootstrap =
15
- SNLP schedule (N, K) =
headline N=24, K=4 (among sweep)
- Softmax Chebyshev interval =
[-20, 0]
- RMSNorm Goldschmidt residual noise scale =
10^{-(n+1)} per n iterations
axioms (4)
- ad hoc to paper NFE = (L−N)+K is a faithful proxy for sequential nonlinear FHE stages and tracks bootstrap count within ~1–2%.
- domain assumption IDN correction has zero FHE multiplicative depth (ciphertext additions and plaintext scalar multiplies only).
- ad hoc to paper Relative sequential vs SNLP ranking is preserved when both paths use identical per-block polynomial approximations, even if absolute FHE costs omit comparison/reciprocal circuits.
- domain assumption CKKS approximate arithmetic and polynomial approximation of exp/rsqrt/sigmoid/tanh are the right model of encrypted Transformer blocks.
invented entities (2)
-
NFE (Nonlinear Forward Evaluations)
no independent evidence
-
Symbolic CKKS cost model linking NFE to bootstrap count
no independent evidence
read the original abstract
Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks. We study whether Structured Newton Layer Parallelism (SNLP) can make this inter-layer composition more FHE-friendly: each Transformer block still requires polynomial approximations for operations such as softmax and RMSNorm, but SNLP reduces the layerwise sequential nonlinear depth from L stages to a small number of solver iterations plus linear structured corrections. Using a simulation framework based on Chebyshev polynomial approximations, we measure error accumulation under sequential versus SNLP inference across 8 models and 4 architecture families. On a 0.5B IDN-trained model, SNLP reduces symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation, while lowering error amplification (1.36x vs. 1.42x). Across all tested models, SNLP has lower amplification than sequential inference. Ablations show that softmax approximation dominates the error budget and CKKS arithmetic noise is negligible in our setting, suggesting that SNLP is complementary to block-level FHE-friendly operator design rather than a replacement for it.
Figures
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.