Pith. sign in

REVIEW 3 major objections 115 references

Layer-parallel SNLP cuts encrypted Transformer bootstrap depth by about 2.65x with only modest perplexity loss.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 13:02 UTC pith:U7LX5WSE

load-bearing objection Useful FHE-depth application of prior SNLP: simulation shows lower sequential nonlinear depth and less error amplification, with the main caveat being omitted real-CKKS softmax circuits rather than a broken argument. the 3 major comments →

arxiv 2607.04819 v1 pith:U7LX5WSE submitted 2026-07-06 cs.LG cs.CR

Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers

Ligong Han , Kai Xu , Hao Wang , Ruijiang Gao , Akash Srivastava This is my paper
classification cs.LG cs.CR
keywords fully homomorphic encryptionCKKSTransformer inferencelayer parallelismSNLPpolynomial approximationbootstrap reductionnonlinear depth
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Encrypted Transformer inference is expensive mainly because each layer's nonlinear pieces (softmax, norms, activations) must be approximated by polynomials and then stacked L times in sequence, burning modulus levels and forcing frequent bootstraps. This paper asks whether Structured Newton Layer Parallelism (SNLP) can change that inter-layer graph: evaluate many suffix layers in parallel over a few solver iterations and stitch them with cheap linear corrections instead of composing L full nonlinear stages. In a Chebyshev-polynomial simulation across eight models, SNLP consistently accumulates less approximation error than sequential inference. On a 0.5B IDN-trained model the headline configuration cuts symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation and lower error amplification. Softmax remains the dominant local error source, so SNLP is presented as complementary to better block-level FHE operators, not a substitute for them.

Core claim

SNLP reduces the sequential nonlinear depth of encrypted Transformer inference from L stages to (L-N)+K stages while preserving usable language-model quality. On a 0.5B IDN-trained model under degree-12 Chebyshev approximation, the n24-K4 configuration lowers symbolic bootstrap count from 53 to 20 (2.65x) with +1.2% perplexity degradation and lower error amplification (1.36x vs 1.42x sequential); the same lower-amplification pattern holds across all eight tested models.

What carries the argument

NFE (Nonlinear Forward Evaluations) = (L-N)+K, together with SNLP's Identity-Newton (purely additive) or HC-Newton (small linear stream mixing) corrections. Because the corrections add essentially zero multiplicative depth, NFE becomes a faithful proxy for bootstrap count and lets the method trade sequential nonlinear depth for a small number of parallel solver iterations.

Load-bearing premise

The plaintext Chebyshev simulation (with simplified RMSNorm noise and omitted comparison-circuit costs) still ranks sequential versus SNLP error and bootstrap cost the same way real CKKS encryption would.

What would settle it

Implement both sequential and SNLP n24-K4 paths under actual CKKS for the same 0.5B model and check whether measured bootstrap counts and relative perplexity amplification still favor SNLP by roughly the reported margins.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Encrypted inference can target a depth-quality frontier distinct from the ordinary GPU wall-clock frontier, sometimes preferring more solver iterations if they cut sequential nonlinear depth.
  • Architectures whose residual-stream mixing is purely linear (e.g., mHC) already reduce nonlinear FHE depth and pair naturally with SNLP.
  • SNLP-aware training is required to reach the low-NFE regime; models without it stay near the sequential depth.
  • Because softmax dominates local approximation error, SNLP gains remain available even if better polynomial or gated-attention replacements improve the blocks themselves.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same depth-reduction idea could be applied to other deep residual stacks whose expensive nonlinearities are sequential (e.g., encrypted vision or speech transformers).
  • If larger models continue the observed trend of lower error amplification, SNLP's relative advantage may become easier to harvest at production scale.
  • FHE training objectives that directly penalize sequential nonlinear depth rather than wall-clock time could further shift the usable (N,K) frontier.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper studies whether Structured Newton Layer Parallelism (SNLP) can reduce the sequential nonlinear depth of encrypted Transformer inference under FHE. Rather than redesigning per-block operators (softmax, RMSNorm, activations), SNLP rewrites the inter-layer graph so that a prefix is sequential and a suffix is solved over K Newton-style iterations with cheap structured corrections (IDN/HCN). The authors introduce NFE = (L−N)+K as a proxy for sequential nonlinear stages, show in a symbolic CKKS model that NFE tracks bootstrap count within ~1–2%, and measure error accumulation via Chebyshev polynomial replacements in plaintext PyTorch across 8 Nanochat models (0.5B–3B, four architecture families). Headline result: on a 0.5B IDN-trained model at degree 12, SNLP n24-K4 cuts symbolic bootstraps 53→20 (2.65×) with reported +1.2% PPL degradation and lower error amplification (1.36× vs 1.42×); SNLP has lower amplification than sequential on all tested models. Ablations attribute most error to softmax; simulated CKKS arithmetic noise is negligible.

Significance. If the relative depth and error claims hold under more realistic FHE accounting, this is a useful complementary axis for encrypted Transformer inference: prior FHE-ML work optimizes per-block polynomial approximations and circuit depth, while SNLP targets the L-fold sequential composition of those blocks. The NFE metric, the explicit separation of FHE-optimal vs wall-clock-optimal (N,K) points, the observation that linear residual-stream mixing (mHC) is inherently FHE-friendly, and the controlled ablations (softmax-dominated error; negligible CKKS noise) are concrete contributions. Strengths include a clear symbolic cost model (Table 1), multi-model evaluation (Table 5), degree and noise sweeps (Tables 3–4), Pareto (N,K) analysis (Figure 1), and an honest Limitations section. The work is simulation-based rather than a full CKKS implementation, so significance is as a structural study of inter-layer depth, not as a deployable encrypted system.

major comments (3)
  1. §4.2 and §6 acknowledge that softmax max-subtraction (comparison) and renormalization (reciprocal) are omitted from NFE and from the simulated error, yet the abstract and Table 1 still report a 2.65× symbolic bootstrap reduction. The manuscript asserts that identical per-block approximations preserve the sequential-vs-SNLP ranking, but does not give an explicit depth accounting showing that these omitted circuits attach once per nonlinear stage and therefore scale with NFE = (L−N)+K rather than with total block evaluations K·N. Because Table 2 shows softmax dominates the error budget, this is the load-bearing place where relative bootstrap and amplification rankings could shift under real CKKS. Please add a short critical-path argument (and, if possible, a sensitivity experiment that injects a fixed extra mult-depth or reciprocal approximation per nonlinear stage) so the 53→20 claim is n
  2. Abstract, §5 (NFE vs PPL), and Figure 1 caption report “+1.2% perplexity degradation” for n24-K4 alongside lower error amplification (1.36× vs 1.42× in Table 3 / Table 5). Lower amplification implies lower HE-PPL for SNLP than for sequential HE, so +1.2% cannot simultaneously mean degradation relative to sequential under the same HE approximation. Clarify the exact reference: (i) SNLP exact-arith vs sequential exact after IDN training, (ii) SNLP-HE vs sequential-HE, or (iii) SNLP-HE vs sequential exact. Reconcile the number with Table 5 (exact PPL 14.07, amps 1.419 / 1.355) and state it uniformly in abstract, bullets, and figure caption. This is load-bearing for the headline quality claim.
  3. Table 5 and Figure 1 show that the aggressive n24-K4 operating point (NFE 12, 2.65× bootstraps) is available only for IDN/HCN-regularized models; baselines are limited to n8-K4 (1.14×). The abstract correctly names the “0.5B IDN-trained model” for the headline, but the broader claim that SNLP reduces encrypted nonlinear depth should more sharply separate (a) structural depth reduction conditional on SNLP-aware training from (b) the milder, training-free configurations. Without that separation, readers may over-generalize the 2.65× figure to untuned checkpoints. A short subsection or table column making “training required for NFE≤16” explicit would fix this.

Circularity Check

1 steps flagged

No significant circularity: bootstrap counts restyle NFE by a linear cost model, but the load-bearing PPL/amplification claims are independent measurements under fixed approximations.

specific steps
  1. self definitional [§4.1 FHE Cost Model and NFE; Table 1]
    "We define NFE(Nonlinear Forward Evaluations) as the number of sequential nonlinear stages in the computation graph: NFE_seq = L, NFE_SNLP = (L−N) + K ... With 15 usable CKKS levels before bootstrapping, the number of bootstraps scales linearly with NFE. In this symbolic model, NFE tracks the bootstrap count within 1–2% across all tested configurations (Table 1)."

    Bootstrap counts in the symbolic model are defined to scale linearly with NFE (IDN correction contributes zero multiplicative depth by design). Reporting that NFE tracks bootstraps at 0.99–1.02×, and headline reductions such as 53→20 (2.65×) for n24-K4, therefore largely restate the NFE reduction under that linear map rather than an independent FHE measurement. This is a minor definitional restyling of the depth claim; it does not force the separate empirical PPL or amplification results.

full rationale

The paper’s central scientific content is empirical comparison of error accumulation (PPL_HE / PPL_exact and absolute PPL under degree-12 Chebyshev replacements) for sequential vs SNLP forward paths across 8 models. Amplification is defined against exact-operation PPL and is not fitted to force SNLP superiority; both paths share the same per-block polynomial code, so relative differences come from computation-graph structure. Dependence on Han et al. (2026) for the SNLP construction and SNLP-aware training is ordinary prior-work citation, not a uniqueness theorem or ansatz smuggled in as external fact, and does not make the FHE measurements tautological. The only mild definitional loop is that symbolic bootstrap counts are computed from NFE under the assumption that bootstraps scale linearly with NFE (IDN adds zero multiplicative depth), so “NFE tracks bootstraps within 1–2%” and “53→20 bootstraps” largely restate the NFE reduction. That does not collapse the quality claims. Score 1 reflects that minor restyling only.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central claim rests on a simulation-and-symbolic-cost stack rather than a closed-form theorem. Load-bearing choices include the NFE definition, the 15-level CKKS budget, Chebyshev degree and fitting intervals, simplified RMSNorm/ReLU2 modeling, and reliance on SNLP-aware pretrained models. No new physical entities are postulated; the invented constructs are metrics and cost models for the FHE comparison.

free parameters (5)
  • Chebyshev polynomial degree d = 12 (default); tested {8,10,12,14}
    Default d=12 chosen after degree sweep; controls accuracy–depth tradeoff and drives reported PPL/amplification numbers.
  • CKKS usable levels before bootstrap = 15
    Symbolic bootstrap counts assume 15 usable levels; scales NFE to bootstrap estimates in Table 1.
  • SNLP schedule (N, K) = headline N=24, K=4 (among sweep)
    Headline n24-K4 and other (N,K) points are selected configurations; NFE and quality depend on these hand-chosen operating points.
  • Softmax Chebyshev interval = [-20, 0]
    Fitting interval [−20,0] calibrated from validation percentiles; clamps inputs and sets approximation domain.
  • RMSNorm Goldschmidt residual noise scale = 10^{-(n+1)} per n iterations
    rsqrt residual modeled as multiplicative noise 10^−(n+1) rather than full encrypted iteration; affects absolute error budget.
axioms (4)
  • ad hoc to paper NFE = (L−N)+K is a faithful proxy for sequential nonlinear FHE stages and tracks bootstrap count within ~1–2%.
    Introduced in §4.1 and validated only inside the authors’ symbolic CKKS model (Table 1).
  • domain assumption IDN correction has zero FHE multiplicative depth (ciphertext additions and plaintext scalar multiplies only).
    Stated in §3.2 and §4.1; standard for additive plaintext-scaled CKKS ops, load-bearing for depth savings.
  • ad hoc to paper Relative sequential vs SNLP ranking is preserved when both paths use identical per-block polynomial approximations, even if absolute FHE costs omit comparison/reciprocal circuits.
    Explicit methodological claim in §4.2 and Limitations §6.
  • domain assumption CKKS approximate arithmetic and polynomial approximation of exp/rsqrt/sigmoid/tanh are the right model of encrypted Transformer blocks.
    Background §3.1 following standard FHE-ML practice (THE-X, Iron, BOLT, THOR).
invented entities (2)
  • NFE (Nonlinear Forward Evaluations) no independent evidence
    purpose: Scalar metric equating sequential nonlinear stages under sequential vs SNLP graphs for FHE cost comparison.
    Defined in abstract/§4.1 as NFE_seq=L and NFE_SNLP=(L−N)+K; not a prior standard FHE metric.
  • Symbolic CKKS cost model linking NFE to bootstrap count no independent evidence
    purpose: Estimate bootstraps and multiplicative depth without running full encryption.
    §4.1 and Table 1; internal model with assumed levels and per-block depth ~4d+7.

pith-pipeline@v1.1.0-grok45 · 15568 in / 3609 out tokens · 31830 ms · 2026-07-11T13:02:05.935357+00:00 · methodology

0 comments
read the original abstract

Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks. We study whether Structured Newton Layer Parallelism (SNLP) can make this inter-layer composition more FHE-friendly: each Transformer block still requires polynomial approximations for operations such as softmax and RMSNorm, but SNLP reduces the layerwise sequential nonlinear depth from L stages to a small number of solver iterations plus linear structured corrections. Using a simulation framework based on Chebyshev polynomial approximations, we measure error accumulation under sequential versus SNLP inference across 8 models and 4 architecture families. On a 0.5B IDN-trained model, SNLP reduces symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation, while lowering error amplification (1.36x vs. 1.42x). Across all tested models, SNLP has lower amplification than sequential inference. Ablations show that softmax approximation dominates the error budget and CKKS arithmetic noise is negligible in our setting, suggesting that SNLP is complementary to block-level FHE-friendly operator design rather than a replacement for it.

Figures

Figures reproduced from arXiv: 2607.04819 by Akash Srivastava, Hao Wang, Kai Xu, Ligong Han, Ruijiang Gao.

Figure 1
Figure 1. Figure 1: NFE vs. PPL under degree-12 HE approximation. Each point is one [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Per-layer relative error ∥hHE − hexact∥/∥hexact∥ under degree-12 polynomial ap￾proximation (0.5B IDN model, N = 24, K = 1). SNLP uses 3.5× fewer bootstraps (15 vs. 53) yet achieves comparable per-layer error to sequential inference. Prefix layers (0–7) are shared. In the parallel layers (8–31, shaded), SNLP error stays at or below sequential despite evaluating all suffix blocks in parallel with only one Ne… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

115 extracted references · 40 linked inside Pith

  1. [1]

    Advances in Neural Information Processing Systems , volume=

    Simple and effective masked diffusion language models , author=. Advances in Neural Information Processing Systems , volume=

  2. [2]

    Advances in neural information processing systems , volume=

    Simplified and generalized masked diffusion for discrete data , author=. Advances in neural information processing systems , volume=

  3. [3]

    arXiv preprint arXiv:2503.01103 , year=

    Direct discriminative optimization: Your likelihood-based visual generative model is secretly a gan discriminator , author=. arXiv preprint arXiv:2503.01103 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2507.15897 , year=

    Redi: Rectified discrete flow , author=. arXiv preprint arXiv:2507.15897 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2202.00512 , year=

    Progressive distillation for fast sampling of diffusion models , author=. arXiv preprint arXiv:2202.00512 , year=

    Pith/arXiv arXiv

  6. [6]

    International Conference on Machine Learning , pages=

    Consistency Models , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  7. [7]

    arXiv preprint arXiv:2509.24526 , year=

    Cmt: Mid-training for efficient learning of consistency, mean flow, and flow map models , author=. arXiv preprint arXiv:2509.24526 , year=

    arXiv

  8. [8]

    arXiv preprint arXiv:2511.23342 , year=

    Flow Straighter and Faster: Efficient One-Step Generative Modeling via MeanFlow on Rectified Trajectories , author=. arXiv preprint arXiv:2511.23342 , year=

    arXiv

  9. [9]

    arXiv preprint arXiv:2505.13447 , year=

    Mean flows for one-step generative modeling , author=. arXiv preprint arXiv:2505.13447 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2406.07507 , volume=

    Flow map matching , author=. arXiv preprint arXiv:2406.07507 , volume=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2506.17298 , year=

    Mercury: Ultra-Fast Language Models Based on Diffusion , author=. arXiv preprint arXiv:2506.17298 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2508.02193 , year=

    Seed diffusion: A large-scale diffusion language model with high-speed inference , author=. arXiv preprint arXiv:2508.02193 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2502.09992 , year=

    Large language diffusion models , author=. arXiv preprint arXiv:2502.09992 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2509.26488 , year=

    dparallel: Learnable parallel decoding for dllms , author=. arXiv preprint arXiv:2509.26488 , year=

    arXiv

  15. [15]

    arXiv preprint arXiv:2112.07804 , year=

    Tackling the generative learning trilemma with denoising diffusion GANs , author=. arXiv preprint arXiv:2112.07804 , year=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2510.06303 , year=

    Sdar: A synergistic diffusion-autoregression paradigm for scalable sequence generation , author=. arXiv preprint arXiv:2510.06303 , year=

    arXiv

  17. [17]

    arXiv preprint arXiv:2505.22618 , year=

    Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding , author=. arXiv preprint arXiv:2505.22618 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2310.14189 , year=

    Improved techniques for training consistency models , author=. arXiv preprint arXiv:2310.14189 , year=

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2410.21035 , year=

    Beyond autoregression: Fast llms via self-distillation through time , author=. arXiv preprint arXiv:2410.21035 , year=

    Pith/arXiv arXiv

  20. [20]

    ACM computing surveys , volume=

    Diffusion models: A comprehensive survey of methods and applications , author=. ACM computing surveys , volume=. 2023 , publisher=

    2023

  21. [21]

    The Twelfth International Conference on Learning Representations , year=

    Let's verify step by step , author=. The Twelfth International Conference on Learning Representations , year=

  22. [22]

    arXiv preprint arXiv:2412.01152 , year=

    Intellect-1 technical report , author=. arXiv preprint arXiv:2412.01152 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2508.09192 , year=

    Diffusion llms can do faster-than-ar inference via discrete diffusion forcing , author=. arXiv preprint arXiv:2508.09192 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2108.07732 , year=

    Program synthesis with large language models , author=. arXiv preprint arXiv:2108.07732 , year=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2107.03374 , year=

    Evaluating large language models trained on code , author=. arXiv preprint arXiv:2107.03374 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2503.09573 , year=

    Block diffusion: Interpolating between autoregressive and diffusion language models , author=. arXiv preprint arXiv:2503.09573 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2410.21357 , year=

    Energy-based diffusion language models for text generation , author=. arXiv preprint arXiv:2410.21357 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2509.06949 , year=

    Revolutionizing reinforcement learning framework for diffusion large language models , author=. arXiv preprint arXiv:2509.06949 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2601.07568 , year=

    d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation , author=. arXiv preprint arXiv:2601.07568 , year=

    arXiv

  32. [32]

    arXiv preprint arXiv:2508.10875 , year=

    A survey on diffusion language models , author=. arXiv preprint arXiv:2508.10875 , year=

    Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2505.15781 , year=

    dkv-cache: The cache for diffusion language models , author=. arXiv preprint arXiv:2505.15781 , year=

    Pith/arXiv arXiv

  34. [34]

    arXiv preprint arXiv:2509.26328 , year=

    Fast-dllm v2: Efficient block-diffusion llm , author=. arXiv preprint arXiv:2509.26328 , year=

    arXiv

  35. [35]

    arXiv preprint arXiv:2504.19720 , year=

    Taming the Titans: A Survey of Efficient LLM Inference Serving , author=. arXiv preprint arXiv:2504.19720 , year=

    Pith/arXiv arXiv

  36. [36]

    ACM Computing Surveys , volume=

    Towards efficient generative large language model serving: A survey from algorithms to systems , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  37. [37]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Llm in a flash: Efficient large language model inference with limited memory , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  38. [38]

    arXiv preprint arXiv:2508.15487 , year=

    Dream 7b: Diffusion large language models , author=. arXiv preprint arXiv:2508.15487 , year=

    Pith/arXiv arXiv

  39. [39]

    The Thirteenth International Conference on Learning Representations , year=

    Simple Guidance Mechanisms for Discrete Diffusion Models , author=. The Thirteenth International Conference on Learning Representations , year=

  40. [40]

    Advances in neural information processing systems , volume=

    Semi-implicit denoising diffusion models (siddms) , author=. Advances in neural information processing systems , volume=

  41. [41]

    arXiv preprint arXiv:2505.21467 , year=

    Accelerating diffusion language model inference via efficient kv caching and guided diffusion , author=. arXiv preprint arXiv:2505.21467 , year=

    arXiv

  42. [42]

    arXiv preprint arXiv:2506.06295 , year=

    dllm-cache: Accelerating diffusion large language models with adaptive caching , author=. arXiv preprint arXiv:2506.06295 , year=

    Pith/arXiv arXiv

  43. [43]

    arXiv preprint arXiv:2511.19269 , year=

    CDLM: Consistency Diffusion Language Models For Faster Sampling , author=. arXiv preprint arXiv:2511.19269 , year=

    arXiv

  44. [44]

    arXiv preprint arXiv:2510.23606 , year=

    Variational Masked Diffusion Models , author=. arXiv preprint arXiv:2510.23606 , year=

    arXiv

  45. [45]

    arXiv preprint arXiv:2511.19473 , year=

    WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning , author=. arXiv preprint arXiv:2511.19473 , year=

    arXiv

  46. [46]

    Transactions of Mathematics and Its Applications , volume=

    Geometric aspects of data-processing of Markov chains , author=. Transactions of Mathematics and Its Applications , volume=. 2024 , publisher=

    2024

  47. [47]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Di [M] o: Distilling masked diffusion models into one-step generator , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  48. [48]

    Communications of the ACM , volume=

    Generative adversarial networks , author=. Communications of the ACM , volume=. 2020 , publisher=

    2020

  49. [49]

    arXiv preprint arXiv:2209.03003 , year=

    Flow straight and fast: Learning to generate and transfer data with rectified flow , author=. arXiv preprint arXiv:2209.03003 , year=

    Pith/arXiv arXiv

  50. [50]

    Advances in neural information processing systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=

  51. [51]

    arXiv preprint arXiv:2511.19942 , year=

    Differential smoothing mitigates sharpening and improves llm reasoning , author=. arXiv preprint arXiv:2511.19942 , year=

    arXiv

  52. [52]

    arXiv preprint arXiv:2510.08141 , year=

    Arbitrary Entropy Policy Optimization Breaks The Exploration Bottleneck of Reinforcement Learning , author=. arXiv preprint arXiv:2510.08141 , year=

    Pith/arXiv arXiv

  53. [53]

    Entropy-preserving reinforcement learning , author=

  54. [54]

    Thinking Machines Lab: Connectionism , year =

    Kevin Lu and Thinking Machines Lab , title =. Thinking Machines Lab: Connectionism , year =

  55. [55]

    The Twelfth International Conference on Learning Representations , year=

    Parallelizing non-linear sequential models over the sequence length , author=. The Twelfth International Conference on Learning Representations , year=

  56. [56]

    Advances in Neural Information Processing Systems , volume=

    Towards Scalable and Stable Parallelization of Nonlinear RNNs , author=. Advances in Neural Information Processing Systems , volume=

  57. [57]

    Advances in Neural Information Processing Systems , volume=

    Parallelizing MCMC Across the Sequence Length , author=. Advances in Neural Information Processing Systems , volume=

  58. [58]

    2025 , publisher =

    Andrej Karpathy , title =. 2025 , publisher =

    2025

  59. [59]

    arXiv preprint arXiv:2604.21254 , year=

    Hyperloop Transformers , author=. arXiv preprint arXiv:2604.21254 , year=

    Pith/arXiv arXiv

  60. [60]

    Advances in Neural Information Processing Systems , volume=

    Deep equilibrium models , author=. Advances in Neural Information Processing Systems , volume=

  61. [61]

    Advances in Neural Information Processing Systems , volume=

    Scaling up test-time compute with latent reasoning: A recurrent depth approach , author=. Advances in Neural Information Processing Systems , volume=

  62. [62]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Value Residual Learning , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  63. [63]

    2024 , url =

    Keller Jordan and Jeremy Bernstein and Brendan Rappazzo and Boza Vlado and You Jiacheng and Franz Cesista and Braden Koszarsky , title =. 2024 , url =

    2024

  64. [64]

    arXiv preprint arXiv:2512.24880 , year=

    mHC: Manifold-Constrained Hyper-Connections , author=. arXiv preprint arXiv:2512.24880 , year=

    Pith/arXiv arXiv

  65. [65]

    The Thirteenth International Conference on Learning Representations , year=

    Hyper-Connections , author=. The Thirteenth International Conference on Learning Representations , year=

  66. [66]

    International Conference on Machine Learning , pages=

    Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving , author=. International Conference on Machine Learning , pages=. 2021 , organization=

    2021

  67. [67]

    Synthesis of Parallel Algorithms , pages=

    Prefix sums and their applications , author=. Synthesis of Parallel Algorithms , pages=. 1990 , publisher=

    1990

  68. [68]

    International Conference on Learning Representations , year=

    Parallelizing Linear Recurrent Neural Nets Over Sequence Length , author=. International Conference on Learning Representations , year=

  69. [69]

    First Conference on Language Modeling , year=

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces , author=. First Conference on Language Modeling , year=

  70. [70]

    2026 , archiveprefix =

    Attention Residuals , author =. 2026 , archiveprefix =. 2603.15031 , primaryclass =

    Pith/arXiv arXiv 2026

  71. [71]

    Qwen2.5: A Party of Foundation Models , url =

    Qwen Team , month =. Qwen2.5: A Party of Foundation Models , url =

  72. [72]

    arXiv preprint arXiv:2401.02385 , year=

    TinyLlama: An Open-Source Small Language Model , author=. arXiv preprint arXiv:2401.02385 , year=

    Pith/arXiv arXiv

  73. [73]

    arXiv preprint arXiv:2503.19786 , year=

    Gemma 3 Technical Report , author=. arXiv preprint arXiv:2503.19786 , year=

    Pith/arXiv arXiv

  74. [74]

    arXiv preprint arXiv:2506.20639 , year=

    DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation , author=. arXiv preprint arXiv:2506.20639 , year=

    Pith/arXiv arXiv

  75. [75]

    Advances in Neural Information Processing Systems , volume=

    Parallelizing Linear Transformers with the Delta Rule over Sequence Length , author=. Advances in Neural Information Processing Systems , volume=

  76. [76]

    International Conference on Learning Representations , year=

    Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding , author=. International Conference on Learning Representations , year=

  77. [77]

    Advances in Neural Information Processing Systems , volume=

    Deeppcr: Parallelizing sequential operations in neural networks , author=. Advances in Neural Information Processing Systems , volume=

  78. [78]

    Applied numerical mathematics , volume=

    An estimator for the diagonal of a matrix , author=. Applied numerical mathematics , volume=. 2007 , publisher=

    2007

  79. [79]

    2023 , eprint=

    C-Pack: Packaged Resources To Advance General Chinese Embedding , author=. 2023 , eprint=

    2023

  80. [80]

    A stochastic estimator of the trace of the influence matrix for

    Hutchinson, Michael F , journal=. A stochastic estimator of the trace of the influence matrix for

Showing first 80 references.