Pith. sign in

REVIEW 3 major objections 5 minor 14 references

Masked Gated Linear Unit

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that the gate and value projections of a Gated Linear Unit can be folded into one shared weight matrix partitioned by learnable binary masks, and that the resulting SwiMGLU layer matches or beats SwiGLU accuracy while…

desk verdict Neat architectural trick for cutting GLU memory reads, but the headline speed and accuracy numbers rest on naive baselines and unequal training budgets. read the letter →

arxiv 2506.23225 v1 pith:S36NX22C submitted 2025-06-29 cs.LG cs.CL

Yukito Tajima , Nakamasa Inoue , Yusuke Sekikawa , Ikuro Sato , Rio Yokota This is my paper · ORCID
classification cs.LGcs.CL
keywords MaskedGatedLinearUnitMixtureofElement-wiseGatingSwiGLUbinarymaskmemorybandwidthreductionLLMinferenceFlashMGLUkernelstraight-throughestimator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Masked Gated Linear Units (MGLUs) claim to remove the main memory bottleneck of GLU-based feed-forward layers in LLMs: instead of reading two full FP16 weight matrices per token for the gate and value streams, a single matrix $W$ is partitioned by learnable binary masks into complementary subspaces, so one FP16 read plus $n_m$ mask bits serves both streams. The paper introduces the Mixture of Element-wise Gating (MoEG) architecture for choosing those masks, and FlashMGLU, a fused CUDA kernel that loads every weight once and evaluates all mask bits on chip. In pretraining and downstream evaluations at 159M and 1.08B scale, the Swish-activated variant SwiMGLU with four masks matches or exceeds the SwiGLU baseline on average accuracy while reducing projection-layer memory loads by up to 47%. If the claim holds, generative LLM inference, which is dominated by weight transfer from HBM to SRAM, could become substantially cheaper per token without degrading model quality. The paper argues that learnable complementary masks -- not sparsity alone -- are what recover the expressivity of two full-rank projections.

What carries the argument

The load-bearing object is the complementary binary mask pair $\{M_i,\bar M_i\}$ acting element-wise on a shared weight matrix $W$; it carves two subspaces out of one matrix so that gate and value streams cost one weight read instead of two. MoEG is the architecture that assembles $n_m$ such mask pairs into a mixture, and FlashMGLU is the kernel that exploits their complementarity: it packs all $n_m$ mask bits into one integer per weight, loads weight and mask together in a coalesced transaction, computes the bare product $x_kW_{row,k}$ once, then adds or does not add that product to each mask's gate and value accumulators according to the packed bits. The straight-through estimator supplies gradients to the binary masks during training. Together these pieces make the per-token memory load $(16+n_m)hd$ bits instead of $32hd$ bits, which is the quantity that carries the inference-speed argument.

What would settle it

Run FlashMGLU against an optimized GLU kernel that loads both weight matrices with the same coalesced FP16 access pattern on the same GPU: if the measured speed-up does not approach the 32/(16+n_m) byte-bound (about 1.9x for n_m=1) and instead drops toward 1.0x at n_m=8 on H100, the claim that mask-based weight sharing is what delivers the inference gain is falsified. A second check is whether increasing n_m from 1 to 8 keeps latency nearly flat once register pressure is controlled; if latency scales linearly with mask count even with the fused kernel, the packing mechanism is not doing the work claimed.

Watch

Extended reading notes

Core claim

The core discovery is that the multiplicative gate-value interaction of a GLU does not require two separate matrices. A single weight matrix $W\in\mathbb{R}^{h\times d}$ can be element-wise divided by binary masks $M_i$ into gate and value subspaces, with the gate stream reading $M_i\odot W$ and the value stream reading $\bar M_i\odot W$, and summing over $n_m$ mask pairs recovers the gating interaction. The paper shows empirically that the resulting SwiMGLU layer with $n_m=4$ matches or surpasses SwiGLU on average zero-shot and two-shot accuracy across six tasks at both tested scales, while its per-token memory load falls from $32hd$ bits to $(16+n_m)hd$ bits. The same complementarity is what lets FlashMGLU compute the gate and value streams in a single pass: the kernel loads each weight once together with packed mask bits, accumulates gated and ungated partial sums in registers, and writes only the outputs, yielding up to 19.66x speedup over a naive PyTorch MGLU and a 34% latency reduction over a standard GLU on an RTX 5090.

Load-bearing premise

The load-bearing assumption is that single-token generative inference is strictly memory-bound, so that cutting per-token weight reads from two FP16 matrices to one FP16 matrix plus mask bits yields a proportional wall-clock speedup rather than being swamped by extra compute or kernel overhead.

Editorial extensions

If this is right

  • Generative decode would load one FP16 matrix plus $n_m$ mask bits per FFN token instead of two FP16 matrices, so FFN weight traffic drops by up to 47% at $n_m=1$.
  • SwiMGLU with four masks is claimed to match or beat SwiGLU average accuracy on ARC Easy, ARC Challenge, HellaSwag, PiQA, SciQ, and Winogrande, while using fewer weight parameters.
  • FlashMGLU latency is nearly independent of mask count up to $n_m=8$, so adding routes adds accuracy without proportionally adding inference cost.
  • The advantage persists at larger intermediate sizes ($h=4096$, $d=14336$), where the kernel is still about 18x faster than a naive PyTorch MGLU.
  • Training cost grows with $n_m$, so the paper recommends $n_m=4$ as the quality/efficiency sweet spot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the memory-bound premise holds, the same mask-partition idea should combine with low-precision weights: a 4-bit shared matrix plus mask bits would cut loads further, but the masks would have to be trained jointly with the quantization grid.
  • The Top-K routing experiment in the appendix points to an input-dependent middle ground in which only the two most active mask routes are evaluated; testing that variant at the 1B scale, where the current results are less complete, would show whether sparsity can be traded for accuracy without losing the memory benefit.
  • The reported speed-up is against a naive nn.Linear GLU baseline, so the kernel-level contribution is not yet separated from the architecture-level contribution; an optimized GLU kernel comparison would isolate whether the 34% advantage comes from reading one matrix instead of two or from the specific CUDA implementation.
  • Because learned masks allocate gate capacity per layer, the method implies a depth-dependent gating budget for FFNs; a direct test would be comparing the learned per-layer gate ratios against a fixed 50/50 split to see where the accuracy gain actually comes from.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Masked Gated Linear Units (MGLUs), a family of GLU-like layers that replace the separate gate and value weight matrices of a standard GLU with a single shared weight matrix partitioned by learnable binary masks. The Mixture of Element-wise Gating (MoEG) architecture and the FlashMGLU CUDA kernel are introduced, with the kernel combining mask bits and loading each weight once. The authors report up to 19.7x speedups over a naïve PyTorch MGLU, 47% lower memory traffic and 34% faster inference than standard GLUs, and LLM pre-training experiments on 159M and 1.08B models claiming that SwiMGLU matches or surpasses SwiGLU accuracy on downstream benchmarks.

Significance. If the central claims were fully supported, the paper would offer a practical way to reduce memory bandwidth during single-token generative inference of LLMs, which is a real bottleneck. The architectural idea of folding gate and value streams into one masked matrix is clearly explained, the kernel algorithm is concrete and reproducible from the pseudocode, and the authors are transparent about training-time FLOP overhead. However, the accuracy claim currently rests on trainings with substantially different compute budgets, and the headline efficiency numbers are measured against a naïve nn.Linear GLU baseline and an idealized bit-counting formula rather than the actual byte-addressable memory traffic of the implemented kernel. These issues are load-bearing for the paper's main narrative and require targeted experimental and analytical fixes.

major comments (3)
  1. [§5.2, Tables 3–6] The claim that SwiMGLU 'matches or surpasses' SwiGLU accuracy is not established under equal training budget. Table 6 shows that small SwiMGLU with nm=4 uses 33 GPU hours versus 22 for SwiGLU, and large SwiMGLU uses 1376 GPU hours versus 768 for SwiGLU. Section 4.1 further states that training FLOPs are (6+8nm)hd versus 18hd for SwiGLU, i.e., roughly 2.1x more FLOPs per token for nm=4. The accuracy gaps in Tables 3 and 4 (e.g., 56.85 vs 56.00 zero-shot for the large model) are therefore confounded by 1.5–1.8x more training compute. An iso-compute comparison, or at minimum a matched-iteration comparison with reported throughput, is needed before 'surpassing' can be attributed to the masked architecture rather than to additional compute.
  2. [§4.1, Table 2; Algorithm 1; Appendix E Algorithm 3] The memory-load formula of (16+nm)hd bits per token assumes that masks are stored and fetched at bit granularity, but the implemented FlashMGLU kernel packs mask bits into one 8-bit integer per element and loads it as a byte. For nm<=8, the actual mask traffic is therefore 8hd bits, not nmhd bits. With nm=1 the real per-token load is 24hd bits versus 32hd bits for SwiGLU, a 25% reduction, not the 47% claimed in Section 4.1 and the Abstract. The '47% more memory-efficient' figure is an ideal bit-counting bound that the byte-oriented kernel does not achieve. The authors should either use bit-packed memory loads (with a concrete mechanism) or revise the memory-efficiency claims to reflect byte-addressable HBM traffic.
  3. [Appendix E, Tables 18–19 and §5.2] The paper promises in Section 5.2 a 'full head-to-head comparison between FlashMGLU and a highly tuned standard GLU kernel' in Appendix E, but Tables 18–19 actually compare against a 'standard PyTorch GLU baseline', i.e., the naive nn.Linear implementation. This is not a tuned GLU kernel that reads its two matrices with coalesced loads and register reuse. The weakness is visible in the data: on an H100 at nm=8 the advantage over this baseline collapses to 1.00x, and on the RTX 5090 it is 1.15x. A tuned two-matvec GLU kernel would establish how much of the reported 1.51x (RTX 5090, nm=1) or '34% faster' speedup survives. Without such a comparison, the headline '34% faster than standard GLUs' is an overstatement rather than a validated performance claim.
minor comments (5)
  1. [Abstract and §5.2] The specific '34% faster' figure is not tied to any single configuration in Tables 18–19; reported speedups vary from 1.00x to 1.51x depending on device, nm, and problem size. Please state the exact configuration for each headline number.
  2. [Throughout] There are typos and inconsistencies: 'Efficeint' in Section 2, 'Trainig' in Appendix B, 'glboal' in Section 1, and 'MoGE' in Section 5.3 (should be 'MoEG').
  3. [Appendix C] The text describes the fraction of 'rows' in W devoted to the gate pathway, but the masks are element-wise (h x d) rather than row-wise; please clarify whether the statistic is per-row, per-column, or per-element.
  4. [Section 4] The sentence 'reduces the number of global memory reads from nm × 4 to 1' is imprecise, since the kernel also reads the packed mask bytes; it should say 'to 1 weight load plus 1 mask load per tile'.
  5. [Tables 3–4] No confidence intervals or multiple seeds are reported for the downstream accuracy numbers; differences of 0.3–0.9 percentage points between SwiGLU and SwiMGLU could be within run-to-run variance.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the accuracy and speed claims rest on external baselines and measured hardware, and the sole self-citation is non-load-bearing training tooling.

full rationale

The paper's central claims decompose into (a) an architectural/memory statement, (b) measured kernel latency, and (c) downstream accuracy. For (a), Eqs. (2)-(3) define MGLU from a single shared weight matrix and complementary binary masks; the Section 4.1 memory-load calculation — one FP16 matrix plus nm mask bits, i.e. (16 + nm)hd vs 32hd for GLU — is an algebraic consequence of that definition, not a prediction fitted from data. For (b), Tables 16-19 report wall-clock CUDA/Triton latencies against PyTorch baselines on RTX 5090 and H100 GPUs, which are external hardware measurements. For (c), Tables 3-4 compare against SwiGLU (Shazeer, 2020) using the LM Evaluation Harness, and no fitted constant is defined in terms of the reported average accuracy. The only in-house reference is Fujii et al. (2024) llm-recipes, cited in Appendix A.1: 'All experiments reported in this paper are implemented based on the llm-recipes framework (Fujii et al., 2024).' Rio Yokota is an author of that citation, so it is a self-citation, but it is a training harness and not a premise of the accuracy or efficiency conclusions. The unequal training budgets in Table 6 (small SwiGLU 22 GPU-hours vs SwiMGLU nm=4 33; large SwiGLU 768 vs SwiMGLU nm=4 1376) are a legitimate experimental confound for the 'surpassing' wording, but confounding is not circularity: it does not make the result true by definition. No step reduces an equation to its own input, no fitted parameter is renamed as a prediction, and no prior result is imported as a forced uniqueness theorem. Score 2 reflects only the minor non-load-bearing self-citation; the derivation is otherwise self-contained.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a few domain assumptions about memory-bound inference and binary mask packing, plus standard linear algebra. The main free parameter is the mask count n_m, which is tuned empirically. The learned mask logits themselves are model parameters, not constants fit to the result.

free parameters (4)
  • number of masks n_m = 4 (chosen as optimal trade-off)
    Ablations in Section 5.3 and Appendix B.4 show accuracy saturates around n_m=4-8, while memory savings vanish at n_m=16, making n_m a manually tuned hyperparameter.
  • mask logit initialization scale = 0.01
    Algorithm 2 initializes mask logits as 0.01*randn; the scale is not swept and affects early learning dynamics.
  • binarization threshold for straight-through estimator = 0
    Algorithm 2 binarizes logits with (soft_mask > 0); the threshold is fixed and not analyzed.
  • kernel splitk factor = unstated
    Algorithms 1 and 3 partition the K dimension into splitk chunks; this affects kernel efficiency but not model accuracy.
assumptions (5)
  • domain assumption Inference at batch size 1 is memory-bound: per-token latency is dominated by reading weights from HBM to SRAM.
    Invoked in Section 1 and Section 4 to justify why reducing bytes read yields proportional speedups.
  • domain assumption Binary masks can be packed into 8-bit words and loaded with negligible bandwidth overhead relative to the weight matrix.
    Assumed in FlashMGLU design (Section 4) to make the (16+n_m) h d load bound realistic.
  • domain assumption The straight-through estimator supplies usable gradients for the binary masks during training.
    Training in Section 3.2 and Algorithm 2 relies on STE; no proof that this optimization surface is well-behaved.
  • domain assumption Weights and activations are FP16, and masks are stored in 1-bit form at inference.
    Section 4.1 computes memory loads under this precision assumption.
  • standard math Standard properties of matrix multiplication and Hadamard products hold as used in Eqs. (1)-(3).
    Baseline arithmetic for GLU and MGLU definitions.
invented entities (1)
  • Mixture of Element-wise Gating (MoEG) with complementary binary masks independent evidence
    purpose: Replace two full-rank gate/value projection matrices with one shared matrix plus binary masks to cut inference memory reads.
    The architecture is fully specified by Eqs. (2)-(3) and tested in downstream-accuracy and latency experiments, giving external falsifiable handles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Masked Gated Linear Unit." pith.science (2026). https://pith.science/paper/S36NX22C

@misc{pith2026250623225,
  author       = {Pith},
  title        = {Pith review of: Masked Gated Linear Unit},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S36NX22C}},
  note         = {Machine review of arXiv:2506.23225}
}
abstract

Gated Linear Units (GLUs) have become essential components in the feed-forward networks of state-of-the-art Large Language Models (LLMs). However, they require twice as many memory reads compared to feed-forward layers without gating, due to the use of separate weight matrices for the gate and value streams. To address this bottleneck, we introduce Masked Gated Linear Units (MGLUs), a novel family of GLUs with an efficient kernel implementation. The core contribution of MGLUs include: (1) the Mixture of Element-wise Gating (MoEG) architecture that learns multiple binary masks, each determining gate or value assignments at the element level on a single shared weight matrix resulting in reduced memory transfer, and (2) FlashMGLU, a hardware-friendly kernel that yields up to a 19.7 $\times$ inference-time speed-up over a naive PyTorch MGLU and is 47% more memory-efficient and 34% faster than standard GLUs despite added architectural complexity on an RTX5090 GPU. In LLM experiments, the Swish-activated variant SwiMGLU preserves its memory advantages while matching - or even surpassing - the downstream accuracy of the SwiGLU baseline.

Figures

Figures reproduced from arXiv: 2506.23225 by the authors.

Figure 1
Figure 1. Comparison of FFNs. (a) Two-layer FFN using GELU. (b) SwiGLU FFN with a gating [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Mixture of Element-wise Gating (MoEG). (a) All gate and value projection matrices are computed from the shared weight matrix W. (b) The MoEG-based SwiMGLU architecture with m routes, each of which leverages element-wise gating. to learn disentangled feature gating and value transformations as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Diagram of how FlashMGLU forward pass is performed during generative inference. When the weight W is split into two blocks in the K dimension (splitk = 2), the input vector x is also partitioned into two. We pre-compute the unmasked matrix-vector operation, than selectively add-up the sum according to the mask values avoiding excessive memory reads of weight matrices. Algorithm 1 FlashMGLU forward pass: Split-K Matr… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Latency comparison Latency [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison of learning curves for different FFN architectures. The top and bottom rows illustrate the changes in training loss / training perplexity of small and large models respectively. The left columns compare existing methods against SwiMGLU, and the right columns…
Figure 6
Figure 6. Figure 6: Comparison of downstream task scores across different FFN architectures of large models. In all metrics, the proposed method, SwiMGLU nm = 4 achieves the best performance. larger intermediate sizes (h = 4096, d = 14336) used in larger Llama-3.2 8B models, our optimized…
Figure 7
Figure 7. Figure 7: Training loss and perplexity of learned vs. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Trainig Curves of small MGLU variants with different activation functions. Left: training loss; right: validation perplexity [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Diagram of a Top-1 SwiMGLU block. We illustrate one token being routed across four mask experts, where the router independently routes each token. The Top-1 SwiMGLU layer returns the output of the selected experts multiplied by the router gate value. B.2 Top-K Routing …
Figure 10
Figure 10. Figure 10: Learning curves of small SwiMGLU models under different Top-K routing strategies. Left: training loss and validation perplexity for K ∈ {1, 2, 4}; right: Top-2 routing compared with the non-routed SwiMGLU baseline [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Training curves of learned vs. fixed masks in small SwiMGLU models. Lef: nm = 1; Right: nm = 2 [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Training curves of small SwiMGLU models across different mask count [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Training loss and perplexity for mask-ablation variants. All ablation variants—No Gate Mask, No Value Mask, and No Masks—converge to higher loss and perplexity compared to the fully masked MGLU baseline, highlighting the necessity of maintaining complementary mask￾def…
Figure 14
Figure 14. Figure 14: Layer-wise gate allocation for learned masks. Algorithm 2 PyTorch-Style Implementation of MGLU (nm = 1). class MGLU(nn.Linear): def __init__(self, in_features, out_features): super(MGLU, self).__init__(in_features, out_features, False) self.register_parameter( "mask",…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

  1. [1]

    Table 16: MGLU latency and speed-ups

    / split_k; int start = chunk * chunk_size; int end = min(start + chunk_size, N); int row_off = row * N; int idx = start + threadIdx.x * 2; int stride = blockDim.x * 2; float total = 0.0f; float msum[N_MASKS * 2] = {0.0f, 0.0f, ...}; for (; idx + 1 < end; idx += stride) { __half2 a2 = *reinterpret_cast<const __half2*>(&A[row_off + idx]); __half2 x2 = *rein...

    work page 1990

  2. [3]

    Gemma 3 technical report

    Gemma Team. Gemma 3 technical report. arXiv preprint arXiv:arXiv:2503.19786,

    arXiv

  3. [6]

    Table 5: Model architecture of small and large variants. Model Size h d Context Length #Heads #Layers small 768 3072 1024 24 12 large 2048 8192 4096 32 16 Table 6: Number of weight and mask parameters, estimated storage size, and training GPU hours for each model configuration. Scale Model nm #Weights #Masks Size (MB) GPU Hours small GELU – 113M 0 215 18 ...

    work page 2021

  4. [8]

    2SSP: A two-stage framework for structured pruning of llms

    Fabrizio Sandri, Elia Cunegatti, and Giovanni Iacca. 2SSP: A two-stage framework for structured pruning of llms. arXiv preprint arXiv2501.17771,

    arXiv

  5. [9]

    Dauphin, Angela Fan, Michael Auli, and David Grangier

    Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In Proc. International Conference on Machine Learning (ICML), 2017b. 11 Yoshua Bengio, Nicholas L’eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1...

    arXiv

  6. [10]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457,

    arXiv

  7. [11]

    Kazuki Fujii, Taishi Nakamura, and Rio Yokota

    doi: 10.5281/zenodo.12608602. Kazuki Fujii, Taishi Nakamura, and Rio Yokota. llm-recipes, May

    doi:10.5281/zenodo.12608602

  8. [2010]

    Gaussian error linear units (gelus)

    10 Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415,

    arXiv

Show all 14 references
  1. [2016]

    Dauphin, Angela Fan, Michael Auli, and David Grangier

    Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In Proc. International Conference on Machine Learning (ICML), pages 933–941, 2017a. Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202,

    2002 arXiv

  2. [2020]

    Prajit Ramachandran, Barret Zoph, and Quoc V . Le. Searching for activation functions.arXiv preprint arXiv:1710.05941,

    arXiv

  3. [2021]

    A.1 Model Architecture and Training Configuration Table 5 summarizes the architectural configurations shared across all experiments

    12 A Experimental Details Here we provide more details about the model architecture, training configurations and resources used in our experiments. A.1 Model Architecture and Training Configuration Table 5 summarizes the architectural configurations shared across all experimen...

    2024

  4. [2022]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    arXiv

  5. [2024]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  6. [2025]

    Mahoney, and Kurt Keutzer

    Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W. Mahoney, and Kurt Keutzer. Ai and memory wall. arXiv preprint arXiv2403.14123,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.