{"id":"760be854-59f2-4268-8c67-7da707c6946e","arxiv_id":"2607.04819","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SNLP reduces encrypted Transformer nonlinear depth from L sequential stages to (L−N)+K, cutting symbolic bootstraps ~2.65× with lower error amplification than sequential inference.","lead":"Layer-parallel Newton-style inference can cut the sequential nonlinear depth of encrypted Transformers, reducing estimated FHE bootstraps by about 2.65× with only small perplexity loss. That matters because bootstrapping, not matrix multiplies, is the main cost of private Transformer inference.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"Headline 2.65× bootstrap and amplification claims rest on a symbolic NFE/critical-path model plus poly simulation that may not preserve cost or error ranking under real CKKS.","rationale":"The reader’s weakest assumption is exactly the load-bearing hinge: the HE-approximation simulation and symbolic 15-level CKKS model must preserve sequential-vs-SNLP cost and error ranking for the central claim to hold outside the proxy. I refine that concern along two concrete failure modes the paper leaves open—(1) critical-path NFE counting vs sequentialized total work over N suffix ciphertexts per iteration, and (2) omitted softmax comparison/reciprocal and real Goldschmidt depth that may scale with total nonlinear evaluations rather than NFE—but I do not find an internal contradiction or a stronger unstated flaw. Within the simulation, ablations (softmax dominance, negligible additive CKKS noise, mHC linearity, Pareto need for IDN training) are coherent, and amplification is fairly defined against each method’s own exact PPL while absolute HE PPL is also reported. No machine-checked proofs or released CKKS artifacts exist, so CONDITIONAL with high confidence remains the right call; the appropriate next evidence is still open code plus at least one real CKKS microbenchmark. Verdict stays UNCHANGED.","tokens_in":11690,"tokens_out":725,"duration_ms":50645,"concrete_test":"Microbenchmark in OpenFHE/HElib: one block’s degree-12 Chebyshev softmax + RMSNorm/rsqrt under (a) 4 sequential blocks vs (b) SNLP-style S=1 prefix + N=3 suffix, K=2, additive IDN correction. Record actual bootstrap count on the critical path and output L2/PPL proxy error. If SNLP bootstrap reduction falls below ~1.5× or HE error amplification ranking reverses vs the PyTorch simulation, the headline claim does not transfer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim quotes 20 vs 53 bootstraps and lower HE amplification (1.36× vs 1.42×) with encrypted PPL within 1.2%. Both numbers come from proxies, not encrypted runs (§4.1–4.2; Table 1; Limitations). Cost: NFE=(L−N)+K and bootstrap counts assume (i) IDN corrections add zero multiplicative depth, (ii) the N suffix blocks in each iteration contribute only one sequential nonlinear stage (critical-path parallelism), and (iii) 15 usable CKKS levels make bootstraps track NFE within 1–2%. If a real HE stack evaluates the N independent suffix ciphertexts one-by-one, critical-path work becomes ~ (L−N)+K·N and the 2.65× shrinks or vanishes. Error: Chebyshev softmax on clamped [−20,0] with max-subtraction, RMSNorm as residual noise (not Goldschmidt), and no comparison/reciprocal circuits means omitted depth and distribution shift under finite-K SNLP are unmeasured. Softmax dominates the simulated budget (Table 2), so any real circuit whose cost scales with total softmax evaluations (K·N) rather than NFE could reverse both the cost ranking and the amplification advantage. The paper scopes this honestly, but the headline still presents bootstrap counts and HE PPL as if they were FHE results.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper studies whether Structured Newton Layer Parallelism (SNLP) can reduce the sequential nonlinear depth of encrypted Transformer inference under FHE. Rather than redesigning per-block operators, SNLP replaces L sequential nonlinear block evaluations with a short sequential prefix plus K parallel suffix iterations and linear structured corrections (IDN/HCN). Using a Chebyshev-polynomial simulation of FHE-unfriendly ops and a symbolic CKKS cost model, the authors define NFE = (L−N)+K as a bootstrap proxy and compare sequential vs SNLP error accumulation across 8 Nanochat models in 4 architecture families. Headline result: on a 0.5B IDN-trained model at degree-12, SNLP n24-K4 cuts symbolic bootstraps 53→20 (2.65×) with encrypted PPL within 1.2% of sequential encrypted PPL and lower amplification (1.36× vs 1.42×); SNLP amplification is lower on all tested models. Ablations attribute most error to softmax approximation and find CKKS arithmetic noise negligible.","tokens_in":12039,"tokens_out":1380,"duration_ms":22949,"significance":"If the relative depth and error-accumulation claims hold under real FHE stacks, this is a useful complementary axis for private Transformer inference: inter-layer graph restructuring rather than only better local polynomial approximations. Strengths include multi-model/multi-architecture evaluation, a clear NFE–bootstrap tracking check (Table 1), per-operation ablations isolating softmax (Table 2), degree and noise sweeps (Tables 3–4), Pareto analysis of NFE vs PPL (Figure 1), and per-layer error traces (Figure 2). The paper is also explicit that SNLP does not remove the need for FHE-friendly block design and that FHE-optimal (N,K) can differ from GPU wall-clock optima. The main contribution is therefore a well-scoped structural hypothesis with reproducible simulation evidence, not a full end-to-end encrypted system.","major_comments":[{"comment":"§4.1 and Table 1: NFE and the 20 vs 53 bootstrap counts rest on a critical-path multiplicative-depth model (15 usable levels; IDN adds zero mult depth; each SNLP iteration contributes one nonlinear stage for the whole suffix). This is plausible for modulus consumption along a dependency chain, but the manuscript should state more carefully what is and is not counted: total ciphertext multiplications, number of ciphertexts that must be bootstrapped when N suffix states are live, packing/rotation overhead for holding N parallel streams, and whether a practical HE runtime that serializes independent block evaluations would still see bootstrap count track NFE rather than ~(L−N)+K·N. Without that clarification, the headline 2.65× bootstrap reduction can be read as measured FHE cost rather than symbolic critical-path depth.","section":null},{"comment":"§4.2 and Limitations: the HE simulation omits comparison circuits for softmax max-subtraction and reciprocal approximation for normalization, and replaces RMSNorm Goldschmidt with controlled residual noise. The relative sequential-vs-SNLP comparison is fair only if these omitted circuits do not change the sequential depth ranking or interact with finite-K SNLP distribution shift. Please either (i) fold a symbolic depth cost for comparison/reciprocal into NFE for both paths and re-report Table 1/5, or (ii) give a short argument/bound that these costs are identical per nonlinear stage and therefore cancel in the NFE ratio, and note any residual risk if SNLP’s parallel suffix changes activation ranges relative to the calibrated Chebyshev intervals.","section":null},{"comment":"Table 5 and Figure 1: the strong low-NFE operating points (n24-K4, 2.65×) require SNLP-aware training; baselines are limited to ~1.14× NFE reduction with <1 pp amplification gain. The abstract’s headline numbers are for the IDN-trained 0.5B model. Please make this co-design requirement first-class in the abstract and introduction (one sentence), so readers do not infer that SNLP is a drop-in inference transform for arbitrary pretrained Transformers under FHE.","section":null}],"minor_comments":[{"comment":"Abstract vs §5: amplification is reported as 1.36× vs 1.42× in the abstract and 1.356× vs 1.419× in Table 5; keep one consistent rounding and state the eval token budget (200k) next to the headline.","section":null},{"comment":"Eq. (4): define N and K at first use in the main text before NFE_SNLP; currently N is introduced in prose after the equation in §4.1.","section":null},{"comment":"Table 2 caption notes absolute PPL differs from Table 5 due to a smaller eval set; consider reporting the same 200k-token protocol for the ablation or adding a one-row check that relative degradations match at 200k.","section":null},{"comment":"Figure 2 uses n24-K1 while the headline config is n24-K4; a short note that K=1 is chosen to isolate single-iteration parallel error (or an extra K=4 panel) would avoid confusion.","section":null},{"comment":"Related work: a brief pointer to how NFE relates to multiplicative depth accounting in THE-X / Iron / BOLT / THOR would help FHE readers map the metric to prior circuit-depth tables.","section":null},{"comment":"Typos/style: “bootsraps” does not appear, but “FHE-optimal ≠ wallclock-optimal” in §6 is fine; ensure consistent “mHC” vs “mHC-Newton/HCN” naming between §3.2 and Table 5.","section":null}],"recommendation":"minor_revision","confidential_remarks":"Fit is reasonable for an ML systems / private inference venue that accepts simulation-first FHE studies, provided the symbolic bootstrap language stays qualified. The work depends on the authors’ concurrent SNLP paper; novelty here is the FHE-depth framing and error-accumulation study, which should be judged on that axis rather than as a new parallel solver. I would not require a full CKKS implementation for acceptance if the cost-model and omitted-circuit points above are tightened."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a careful systems paper that takes the authors’ prior SNLP solver and asks a useful FHE question—does changing the inter-layer graph reduce sequential nonlinear depth without making polynomial approximation error worse? Inside their simulation, the answer is yes, and the evidence is broader than a single toy model.\n\nWhat is actually new is not SNLP itself (that is Han et al. 2026). It is the NFE framing, the symbolic CKKS bootstrap accounting, and the head-to-head error-amplification study under shared Chebyshev approximations across 8 models and 4 families. The ablations earn their keep: softmax dominates the error budget, CKKS arithmetic noise is negligible at the precisions they test, mHC looks inherently friendlier because the extra mixing is linear, and FHE-optimal (N,K) is not the same as GPU wall-clock optimal. They also show SNLP-aware training is required for the aggressive n24-K4 regime; baselines only tolerate mild configs. That is honest and useful.\n\nThe soft spot is real but scoped. There are no actual CKKS runs. Bootstrap counts (20 vs 53) come from NFE plus a 15-level symbolic model, and the HE path uses clamped Chebyshev softmax with max-subtraction, residual noise for RMSNorm, and no comparison/reciprocal circuits. Absolute cost numbers are estimates. I am less worried than the stress-test about critical-path collapse: if the N suffix blocks are independent within an iteration, multiplicative depth still tracks (L−N)+K, not K·N. Total work still scales with K·N, and omitted circuits could change absolute depth, but the relative amplification claim under identical per-block approximations is on firmer ground than the headline bootstrap ratio. The paper’s own limitations section mostly says this.\n\nMath and citations look fine for a systems/ML preprint; no internal contradiction. This is for people working on private LLM inference or FHE-friendly architectures, not a general ML audience. I would send it to peer review. Engage if you care about encrypted Transformers; treat the 2.65× as a depth-proxy result until someone ships real HE microbenchmarks or code.","headline":"Solid FHE-systems application of prior SNLP: real multi-model evidence that layer-parallel structure can cut sequential nonlinear depth and not worsen poly-error accumulation, but the 2.65× bootstrap headline is still a symbolic proxy, not encrypted runs.","tokens_in":12696,"tokens_out":574,"would_cite":true,"duration_ms":14885,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Structured Newton Layer Parallelism cuts sequential nonlinear depth in encrypted Transformer inference, using far fewer bootstraps while accumulating less polynomial error.","keywords":["fully homomorphic encryption","Transformer inference","layer parallelism","SNLP","polynomial approximation","CKKS","bootstrapping","error amplification"],"falsifier":"Run actual CKKS (or a circuit that includes max-subtraction comparisons and real rsqrt) on the same 0.5B IDN model at degree-12 softmax: if SNLP n24-K4 no longer shows lower error amplification or roughly 2.65× fewer bootstraps than sequential, the claim fails.","tokens_in":12536,"feed_emoji":"🔐","tokens_out":957,"duration_ms":19787,"temperature":0.7,"pith_summary":"Encrypted Transformer inference is expensive because each layer still needs polynomial approximations of nonlinear ops such as softmax, and those stages stack in sequence, burning modulus and forcing frequent bootstraps. This paper argues that Structured Newton Layer Parallelism changes the inter-layer graph rather than the local operators: a prefix of layers runs as usual, then many suffix layers are evaluated in parallel over a small number of Newton-style iterations and corrected with cheap linear updates that add little or no multiplicative depth. In a Chebyshev-polynomial simulation across eight models, that structure lowers error amplification relative to ordinary sequential inference and, on a 0.5B IDN-trained model, cuts symbolic bootstraps from 53 to 20 while keeping encrypted perplexity within about 1.2% of the sequential encrypted baseline. Softmax approximation remains the main local error source; arithmetic noise is negligible. A sympathetic reader cares because bootstrap cost dominates FHE latency, so any method that shortens sequential nonlinear depth without wrecking quality is a practical lever, complementary to better per-block approximations.","feed_headline":"Layer-parallel inference cuts encrypted Transformer bootstraps 2.65×","feed_subtitle":"Parallel Newton-style layer solves keep encrypted perplexity within 1.2% while lowering error buildup.","key_machinery":"Structured Newton Layer Parallelism (SNLP): solve the residual equation for the layer hidden-state trace by evaluating N suffix blocks in parallel for K iterations, then applying a structured Newton correction (Identity Newton is purely additive with zero FHE multiplicative depth; HC Newton uses a small linear mixing matrix). NFE = (L-N)+K tracks bootstrap count in the symbolic CKKS model.","core_discovery":"SNLP reduces layerwise sequential nonlinear depth from L stages to (L-N)+K stages plus linear structured corrections. Under degree-12 Chebyshev simulation, this consistently lowers error amplification versus sequential inference across eight models in four architecture families, and on a 0.5B IDN-trained model yields 2.65\times fewer symbolic bootstraps (20 vs 53) with encrypted perplexity within 1.2% of sequential encrypted PPL.","pith_inferences":["Pairing SNLP with softmax-light or linear-attention blocks could stack the depth savings, since softmax already dominates the simulated error budget.","Training that optimizes the depth–quality frontier rather than GPU wall-clock time may unlock more aggressive encrypted operating points.","The same residual-equation view of depth may apply to other deep residual stacks under FHE, not only Transformers.","Once comparison circuits for stable softmax are counted, absolute bootstrap numbers will rise, but relative ranking should still favor structures whose corrections stay linear."],"forward_implications":["Encrypted inference can trade a few extra solver iterations for a large cut in sequential nonlinear depth and bootstrap count.","SNLP-aware training is required to reach the low-NFE operating region with usable quality.","Models with linear residual-stream mixing (mHC-style) are inherently more FHE-friendly because those connections add no nonlinear depth.","FHE-optimal SNLP settings can differ from wall-clock-optimal ones: higher K can still be cheaper when cost is dominated by depth.","SNLP complements, rather than replaces, better local polynomial approximations of softmax and related ops."],"fun_headline_variants":["SNLP cuts encrypted Transformer bootstraps 2.65x to 20","Layer-parallel Newton drops FHE bootstraps 53 to 20","Parallel layer solves trim encrypted nonlinear depth 2.65x","SNLP lowers FHE error amp and bootstraps on 0.5B model","Encrypted Transformer bootstraps fall 2.65x via SNLP"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The Chebyshev-and-symbolic-bootstrap simulation ranks sequential versus layer-parallel inference the same way real encrypted arithmetic would.","fun_headline_variants_meta":{"raw":{"variants":["SNLP cuts encrypted Transformer bootstraps 2.65x to 20","Layer-parallel Newton drops FHE bootstraps 53 to 20","Parallel layer solves trim encrypted nonlinear depth 2.65x","SNLP lowers FHE error amp and bootstraps on 0.5B model","Encrypted Transformer bootstraps fall 2.65x via SNLP"]},"model":"grok-4.5","effort":"low","cost_usd":0.004734,"raw_usage":{"total_tokens":1349,"prompt_tokens":792,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":47340000,"prompt_tokens_details":{"text_tokens":792,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":473,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":792,"tokens_out":84,"duration_ms":4805,"temperature":1.0,"reasoning_tokens":473,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T16:18:47.181084+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run actual CKKS (or a circuit that includes max-subtraction comparisons and real rsqrt) on the same 0.5B IDN model at degree-12 softmax: if SNLP n24-K4 no longer shows lower error amplification or roughly 2.65× fewer bootstraps than sequential, the claim fails.","supporting_citations":[],"review_version":2}