Pith. sign in

REVIEW 3 major objections 3 minor 30 references

CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models

T0 review · 3 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CurveFP co-designs a low-precision number format with its product arithmetic, claiming that 7-bit log-curve codebooks can match or beat FP8 for language-model inference and training.

desk verdict Carefully built block-scaled LNS variant with an honest matched-experiment suite; the core training claim rests on a 128M run, but the math is correct and the paper deserves a serious referee. read the letter →

arxiv 2608.10010 v2 pith:5FX2UV5U submitted 2026-08-08 cs.LG

Ye Qiao This is my paper · ORCID
classification cs.LG
keywords CurveFPlow-precisionarithmeticlogarithmicnumbersystemLLMquantizationclosed-productalgebrablockscalingrationalradixlanguagemodelpretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CurveFP claims that low-precision number formats for language models can be designed together with the arithmetic that multiplies them, rather than optimizing scalar accuracy first and then paying for a conventional multiplier. It arranges quantized magnitudes on several interleaved logarithmic curves under a shared power-of-two block scale, so every nonzero product becomes an exact combination of sign, integer-index addition, exponent addition, and a carry. The paper instantiates this idea as CurveFP8 for training and CurveFP7 for seven-bit inference, and reports that CurveFP7 beats tensorwise FP8 perplexity on four 7B–9B models with one fewer element bit, that CurveFP8 lowers error in all 36 paired training-GEMM comparisons, and that in three matched 3B-token pretraining triplets CurveFP8 reaches mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8, with a lower format penalty in every seed. A preliminary 4×4 spatial tile closes at 500 MHz with one fewer product register and 4.6% less area than the FP8 tile. The paper presents these results as support for a co-designed representation-and-arithmetic family while explicitly leaving system-level efficiency and larger-scale training to future study.

What carries the argument

The load-bearing object is the CurveFP codebook: a block of values sharing one power-of-two scale $2^{a_b}$, each nonzero element storing a sign, an exponent $e$, and a curve index $k\in\{0,\dots,K-1\}$, with magnitude $2^{a_b}r^{e+k/K}$ where $r=2^{p/q}$ is a reduced rational radix and $K=2^C$. The uniform curve index is what carries the argument: multiplication becomes sign XOR, $(k_x+k_w)\bmod K$, and $e_x+e_w+\lfloor(k_x+k_w)/K\rfloor$, so product formation needs no general multiplier and no codebook lookup. The radix $r$ then fixes the phase count $H=qK/\gcd(p,qK)$ of Equation 4, which schedules accumulation as a sum over $H$ fixed phase weights of binary-shifted integer counts, making the hardware cost of a precision choice explicit.

What would settle it

On a model whose activations contain per-block outliers larger than the chosen power-of-two scale, measure CurveFP8 versus FP8 perplexity at block sizes 64 and 512; if the fraction of values saturating the CurveFP endpoint grows enough to push its perplexity more than 0.02 above FP8's, the block-log-uniform premise is the limiting factor.

Watch

Extended reading notes

Core claim

On its own terms, CurveFP's discovery is that uniform spacing of the curve index makes the product of any two representable values exactly representable in the same codebook: signs XOR, curve indices add modulo $K$ with a carry into the exponent, exponents add, and shared power-of-two scales add. Because magnitudes sit on a rational-radix lattice $r^{e+k/K}=2^{p(eK+k)/(qK)}$, the number of distinct fractional phases a dot product must accumulate is $H=qK/\gcd(p,qK)$, so every range–resolution choice carries an explicit accumulation contract. Instantiated as CurveFP8 E4C3/E5C2 and CurveFP7 E3C3, this structure reaches FP8-class pretraining parity—mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8 across three matched 3B-token runs—and CurveFP7 at seven bits beats tensorwise FP8 perplexity on four 7B–9B models while staying within 1.32% of native quality. It also reports a preliminary routed tile in which CurveFP8 closes at 500 MHz with one fewer product register and 4.6% less area than the timing-closing FP8 tile.

Load-bearing premise

The load-bearing premise is that, within each block, magnitudes are approximately log-uniform, so a single power-of-two scale plus the fixed log lattice can represent activations and weights accurately; because CurveFP has no per-element exponent, any outlier that the block scale cannot absorb is saturated or coarsely quantized.

Editorial extensions

If this is right

  • Training GEMMs can drop the general mantissa multiplier: with the same role split as FP8 (E4C3 forward, E5C2 gradients), every nonzero product is a few integer additions, and the product path closes timing at 500 MHz without an extra register in the preliminary tile.
  • Seven-bit inference becomes a practical operating point: at block size 64, CurveFP7 beats tensorwise FP8 perplexity on all four tested 7B–9B models and stays within 1.32% of native quality, and at block size 512 it crosses INT8.
  • The phase law $H=qK/\gcd(p,qK)$ gives designers an explicit quality–complexity dial: a denser radix such as $r=2^{3/4}$ improves perplexity by 0.0055–0.0259 but requires 32 phases instead of 8.
  • Matching training and runtime arithmetic nearly halves the out-of-distribution penalty for CurveFP8, and the CurveFP8-trained checkpoint transfers to seven-bit runtime, so one numeric family can span training and deployment without retraining.
  • The paper's hardware evidence is limited to a feasibility tile: area savings of 4.6% at equal peak throughput come with memory, interconnect, scale transport, and workload effects explicitly excluded from both designs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if within-block log-uniformity holds for larger models, CurveFP's phase schedule could combine with outlier-aware scale selection (per-row or per-token scales) to absorb activation outliers without a conventional multiplier; the paper tests only fixed block sizes 64–512.
  • Beyond the paper: the phase law suggests a hardware experiment the paper does not run—fix tile area and sweep radix from $p/q=1$ to $3/4$, and measure whether the added phase-reduction logic erodes the 4.6% area advantage as $H$ grows from 8 to 32.
  • Beyond the paper: the codebook is parameter-free apart from E, C, p/q, and block scale, so the same closed-product algebra could be applied to KV-cache or attention-score tensors, whose distributions may not be log-uniform; that remains untested.
  • Beyond the paper: because the paper's training evidence is one 128.3M-parameter architecture at 3B tokens, a direct extension is a matched FP8-versus-CurveFP8 run at 7B+ scale, where activation distributions may deviate further from log-uniform blocks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. CurveFP is a block-scaled number format family in which each element stores a sign, an E-bit exponent, and a C-bit curve index under a shared power-of-two scale and a rational radix r=2^{p/q}. The paper shows that multiplication is exactly closed in the code domain (sign XOR, curve-index addition with carry, exponent addition) and derives H=qK/gcd(p,qK) phase classes for accumulation. It instantiates E4C3/E5C2 for training and E3C3 for 7-bit inference, and reports: FP8-class post-training perplexity on four 7B-9B models; lower NMSE in all 36 paired training-GEMM windows; three matched 3B-token pretraining runs at 128.3M parameters showing mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8; downstream transfer parity; and a 4x4 Nangate45 tile that closes at 500 MHz with one fewer product register and about 4.6% smaller area than FP8.

Significance. If the results hold, CurveFP is a genuinely useful co-design: it replaces general product-forming multipliers with small integer additions and sign/carry logic, makes the accumulation cost explicit through the phase law, and appears to reach FP8-class accuracy with a seven-bit inference format. The paper is unusually careful in its statistical hedging: it matches seeds, token order, and schedules across training modes, reports uncertainties, and states parity rather than superiority. It also ships machine-checked hardware verification and an artifact-based reproducibility plan. The main weaknesses are that from-scratch training evidence is limited to a single 128.3M-parameter model trained on 3B tokens, and the per-block scaling configuration in the training protocol is under-specified; these gaps weaken but do not invalidate the central 'FP8-class training' claim.

major comments (3)
  1. [§4.5, §B.2, Eq. (6)] The matched pretraining protocol never states the block size G or the scale-exponent width S used by CurveFP8's per-block power-of-two scales, even though Eq. (6) makes the storage rate depend on S/G and Figure 2b shows inference quality depends strongly on G. The same section also does not state the scale granularity used for the FP8 baseline; if FP8 used per-tensor scales while CurveFP8 used per-block scales, the comparison in Table 4 confounds datatype with scaling policy. Please report these parameters for both formats.
  2. [§3.1, Eq. (1)] For E4C3, Eq. (1) gives each block an element field that spans only a fixed magnitude ratio of 2^{15.875}, about 6×10^4. Since the scale exponent a_b is shared across the block, ceil-absmax scaling shifts this window but cannot widen it; any block whose max/min ratio exceeds this window will force small elements to the smallest positive magnitude or to saturation, with relative errors well above 10% for affected values. The paper does not report outlier statistics, block-level max/min ratios, or sensitivity of the Table 4 results to this failure mode, so the central parity claim rests on an unverified assumption that LLM activations and weights are approximately log-uniform within each block. Please add this analysis or explicitly qualify the claim.
  3. [§4.5, §6] The only from-scratch training evidence is a 128.3M-parameter model trained on 3B tokens; the 7B-9B results are post-training quantization only. Since activation and gradient outliers typically become more severe as model scale grows, the conclusion that CurveFP8 provides an 'FP8-class training path' is not established at the scale where FP8 is most relevant. Either extend the training study to a larger model or narrow the claim in the abstract and conclusion to the tested scale.
minor comments (3)
  1. [§3.1, Table 1] The sentence 'We reserve magnitude index zero as a sentinel and omit the smallest nonzero exponent/curve combination, leaving 2^{E+C}-1 positive magnitudes' is internally inconsistent: reserving a sentinel and additionally omitting the smallest nonzero combination would leave 2^{E+C}-2 positive levels, not 2^{E+C}-1. Please clarify the codebook construction.
  2. [§1, §5, Table 6] The area reductions in the introduction (4.4% and 3.9%) differ from those in the abstract, Section 5, and Table 6 (4.6% and 4.1%); please make the numbers consistent.
  3. [§4.4, Table 3] Table 3 reports only aggregate NMSE ratios per model and GEMM role; if the text claims all 36 paired comparisons are wins, the individual paired values should be included in the supplementary material for verification.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: core quality results are measured against external FP8/BF16/INT8 baselines, and the closed-product and phase-count properties follow from the stated definitions.

full rationale

The paper's central claims are not circular. CurveFP's closed-product rule and phase count are derived from the defining codebook in Section 3: Equation 1 defines the rational-radix element code, Equation 2 defines the index update, and Equation 4 derives H = qK/gcd(p, qK) from the additive subgroup generated by p in Z_{qK}. These are definitional algebraic facts, not empirical discoveries dressed as predictions, and the paper explicitly presents product closure as a designed property rather than a fitted outcome. The main empirical comparisons are all against external baselines: CurveFP7 is compared with FP8 and INT8 on four public models (Section 4.2), CurveFP8 is compared with FP8 on paired training-GEMM windows from Pythia and Llama (Section 4.4), and the matched pretraining study compares CurveFP8, FP8, and BF16 under shared initialization, tokens, optimizer, and schedule (Section 4.5). The scale selection procedure (reconstruction-MSE search for static operands, ceil-absmax for dynamic operands) is standard quantization calibration and is applied to both CurveFP and the baselines; it does not encode the reported perplexity or error outcomes. Self-citations to TeLLMe, FASQ, and APEX-Q appear only as related work and are not load-bearing premises for any derivation or empirical claim. No fitted parameter is renamed as a prediction, and no uniqueness or impossibility result is imported from the authors' prior work. Any concern about the bounded within-block dynamic range of the format is a correctness or generalization risk, not circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The design rests on two empirical assumptions (log-uniform value distribution within blocks; convergence of quantized training) that are tested only on the specific models in the paper, plus standard math. The free parameters (block size, scale width, radix) are hand-chosen design choices, not fitted to data.

free parameters (3)
  • Block size G = 64, 128, 256, 512 (swept; G512 used for principal inference results)
    The trade-off between scale metadata overhead and local range adaptation is chosen by the authors and directly affects accuracy (Figure 2b).
  • Scale exponent width S = not explicitly specified
    Effective bit rate R = 1+E+C+S/G depends on S, but the paper never states S for the principal experiments, making the true per-value bit cost imprecise.
  • Radix p/q = 1/1 default; ablated at 8/9 and 3/4
    The rational radix tunes range versus resolution and changes the phase count; the default is a design choice, not derived.
assumptions (3)
  • standard math Properties of exponentiation and modular arithmetic used to derive the phase count (Eq. 4).
    Standard group theory in Z_{qK}; no independent empirical support needed.
  • domain assumption Within each block, LLM activations and weights are approximately log-uniform so a single power-of-two scale plus the log lattice gives FP8-class accuracy.
    This is the empirical premise tested on four models for inference and one small model for training; it is not guaranteed for other architectures or scales.
  • domain assumption Training with quantized forward operands and output gradients under AdamW converges to the same quality as FP8 training on the tested setup.
    The 3B-token experiments support this for a 128M model, but it is an assumption for larger models.
invented entities (1)
  • CurveFP format family (E4C3, E5C2, E3C3 variants) independent evidence
    purpose: A block-scaled logarithmic datatype with closed product arithmetic for LLM training and inference.
    The format is the paper's designed contribution, directly validated by the reported perplexity and hardware measurements; it is not an ad hoc entity invoked to explain an unrelated result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models." pith.science (2026). https://pith.science/paper/5FX2UV5U

@misc{pith2026260810010,
  author       = {Pith},
  title        = {Pith review of: CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5FX2UV5U}},
  note         = {Machine review of arXiv:2608.10010}
}
read the original abstract

Low-precision formats usually optimize scalar fidelity while inheriting conventional product arithmetic. We introduce CurveFP, a block-scaled family that distributes magnitudes across interleaved logarithmic curves. Uniform curve indices make every nonzero product an exact sign and integer-index update, while a rational radix exposes the finite phase schedule required for accumulation. We instantiate the algebra as CurveFP8 E4C3/E5C2 for training and CurveFP7 E3C3 for compact inference. On four 7B-9B models, CurveFP7 beats tensorwise FP8 perplexity with one fewer element bit and stays within 1.32% of native quality. CurveFP8 lowers error in all 36 paired training-GEMM comparisons. Across three matched 3B-token pretraining triplets, it reaches mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8 and has a lower format penalty in every seed. Downstream evaluation shows transfer parity and a consistent WikiText-103 gain. In a preliminary 4x4 Nangate45 spatial accelerator tile, CurveFP8 uses one fewer product register and 4.6% less area than timing-closing FP8 at 500 MHz. These results support CurveFP as a numerical and arithmetic co-design, while leaving system-level efficiency to future study.

Figures

Figures reproduced from arXiv: 2608.10010 by the authors.

Figure 1
Figure 1. CURVEFP co-designs representation and multiplication. Power-of-two scales supply operand￾block range, while sign, exponent, and curve fields encode each element. Uniform curve indices make products exactly closed under XOR and integer addition. Rational radix produces H fixed phase classes; products route to signed integer bins that are combined using fixed phase weights. This mapping defines the inference and train… view at source ↗
Figure 2
Figure 2. CURVEFP7 enters the FP8 quality regime and remains stable as scale groups widen. (a) Perplexity increase over each native model. G64 CURVEFP7 beats tensorwise FP8 on all four architectures and wins against rowwise FP8 on Llama and Qwen3.5 while trailing it on Qwen3 and Falcon-H1. (b) A representative complete granularity sweep on Falcon-H1, reporting actual WikiText-2 perplexity from G128 through rowwise and tensorw… view at source ↗
Figure 2
Figure 2. Seven-bit CURVEFP7 reaches the FP8 quality regime. Left: perplexity increase over native on four models. Right: the Falcon-H1 granularity sweep, where CURVEFP7 crosses INT8 at G512. stated otherwise, we quantize every linear layer except the tied vocabulary head and keep normalization and nonlinear operations in BF16/FP16. Every comparison shares data, tokenizer, model weights or initialization, and evaluation code;… view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: CURVEFP8 and FP8 track BF16 across three seeds. Both panels run all checkpoints in BF16, removing inference quantization to compare learned weights. Lines show three-seed means at 30 matched checkpoints; bands span the seed-wise minimum and maximum. Right: each quantiz…
Figure 4
Figure 4. Figure 4: Matching training and runtime arithmetic reduces mean format-transfer penalty. Cells are three-seed means relative to BF16 inference on the same checkpoint. OOD values average WikiText-103 and PG-19 log-PPL changes in milli-NLL; task values are accuracy-point changes. …
Figure 4
Figure 4. Figure 4: Matching training and runtime reduces format-transfer loss. Left: OOD penalty for BF16- trained and matched-trained checkpoints. Right: zero- and five-shot changes for CURVEFP8 and CURVEFP7 on the CURVEFP8-trained checkpoint [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Complete three-seed format-transfer matrix. Each row fixes the training arithmetic and each column [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 2 canonical work pages

  1. [1]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, and Torsten Hoefler

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, and Torsten Hoefler. QuaRot: Outlier-free 4-bit inference in rotated LLMs.arXiv preprint arXiv:2404.00456,

    arXiv

  2. [2]

    Selected RTL and final routed netlists pass 64 seeded variable-K matrices spanning 1,010 compute cycles and 1,024 checked outputs per datatype

    Cells Setup WS Setup TNS Hold WS pJ/MAC CurveFP8 P0 0 49,282.9 37,999+0.0640+0.1044.78 CurveFP8 P1 1 49,540.1 33,868+0.4260+0.1044.78 FP8 P0 0 53,245.5 41,077−0.045−1.223 +0.105– FP8 P1 1 51,646.8 35,716+0.3020+0.1064.75 C.2 Verification, tools, and power method All 256 E4M3FN byte patterns were checked against PyTorch 2.6, and all 16,384 CurveFP E4 magni...

    work page 2026

  3. [3]

    (pt) ↑ BF16 FP8 CurveFP8CurveFP7 Runtime format 0 -0.25 -0.24 -0.45 0 -0.08 -0.07 -0.02 0 -0.03 -0.13 +0.01 (c) Five-shot Δacc

    14 BF16 FP8 CurveFP8CurveFP7 Runtime format BF16 FP8 CurveFP8Training arithmetic 0 +6.94 +7.02 +7.40 0 +3.48 +4.66 +4.94 0 +5.56 +4.01 +4.27 (a) OOD ΔNLL (milli) ↓ BF16 FP8 CurveFP8CurveFP7 Runtime format 0 -0.03 -0.14 -0.17 0 -0.06 +0.10 +0.19 0 -0.15 -0.02 -0.16 (b) Zero-shot Δacc. (pt) ↑ BF16 FP8 CurveFP8CurveFP7 Runtime format 0 -0.25 -0.24 -0.45 0 -0...

    work page 2024

  4. [7]

    Reena Elangovan, Charbel Sakr, Anand Raghunathan, and Brucek Khailany

    URL https://arxiv.org/ abs/1804.01526. Reena Elangovan, Charbel Sakr, Anand Raghunathan, and Brucek Khailany. LO-BCQ: Block clustered quantization for 4-bit (W4A4) LLM inference.Transactions on Machine Learning Research,

    arXiv

  5. [9]

    URLhttps://arxiv.org/abs/2407.21783

    doi: 10.48550/arXiv.2407.21783. URLhttps://arxiv.org/abs/2407.21783. Ioannis Kouretas and Vassilis Paliouras. Logarithmic number system for deep learning. In2018 7th International Conference on Modern Circuits and Systems Technologies, pp. 1–4,

    arXiv doi:10.48550/arxiv.2407.21783

  6. [10]

    URLhttps://doi.org/10.1109/MOCAST.2018.8376572

    doi: 10.1109/ MOCAST.2018.8376572. URLhttps://doi.org/10.1109/MOCAST.2018.8376572. 11 Yuhang Li, Xin Dong, and Wei Wang. Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks. InInternational Conference on Learning Representations,

    arXiv 2018

  7. [11]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort

    URL https://arxiv.org/abs/1909.13144. Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. SpinQuant: LLM quantization with learned rotations. InInternational Conference on Learning Representations,

    arXiv 1909

  8. [12]

    org/abs/2405.16406

    URL https://arxiv. org/abs/2405.16406. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843,

    arXiv

Show all 30 references
  1. [14]

    URL https://arxiv.org/abs/2209

    doi: 10.48550/arXiv.2209.05433. URL https://arxiv.org/abs/2209. 05433. Daisuke Miyashita, Edward H. Lee, and Boris Murmann. Convolutional neural networks using logarithmic data representation.arXiv preprint arXiv:1603.01025,

    doi arXiv

  2. [15]

    URL https://arxiv.org/abs/1603.01025

    doi: 10.48550/arXiv.1603.01025. URL https://arxiv.org/abs/1603.01025. Denis Paperno, Germ´an Kruszewski, Angeliki Lazaridou, Ngoc-Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fern´andez. The LAMBADA dataset: Word prediction requiring a...

    doi arXiv

  3. [16]

    URL https://aclanthology.org/P16-1144/

    doi: 10.18653/v1/P16-1144. URL https://aclanthology.org/P16-1144/. Guilherme Penedo, Hynek Kydl´ıˇcek, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro von Werra, and Thomas Wolf. The FineWeb datasets: Decanting the web for the finest text data at scal...

    arXiv

  4. [17]

    URL https: //arxiv.org/abs/2406.17557

    doi: 10.48550/arXiv.2406.17557. URL https: //arxiv.org/abs/2406.17557. Ye Qiao, Zhiheng Chen, Yifan Zhang, Yian Wang, and Sitao Huang. TeLLMe: An efficient end-to-end ternary LLM prefill and decode accelerator with table-lookup matmul on edge FPGAs. InProceedings of the 2026 A...

    doi arXiv

  5. [20]

    URL https://arxiv.org/ abs/2310.10537

    doi: 10.48550/arXiv.2310.10537. URL https://arxiv.org/ abs/2310.10537. 12 Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. WinoGrande: An adversarial winograd schema challenge at scale.arXiv preprint arXiv:1907.10641,

    doi arXiv

  6. [21]

    doi: 10.48550/arXiv.1907. 10641. URLhttps://arxiv.org/abs/1907.10641. Semyon Savkin, Eitan Porat, Or Ordentlich, and Yury Polyanskiy. NestQuant: Nested lattice quantization for matrix products and LLMs. InProceedings of the 42nd International Conference on Machine Learning, vo...

    1907 doi arXiv

  7. [22]

    URL https: //doi.org/10.1609/aaai.v40i48.42293

    doi: 10.1609/aaai.v40i48.42293. URL https: //doi.org/10.1609/aaai.v40i48.42293. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. BLiMP: The benchmark of linguistic minimal pairs for english.Transactions of the Associati...

    doi

  8. [23]

    URL https://doi.org/ 10.1162/tacl_a_00321

    doi: 10.1162/tacl a 00321. URL https://doi.org/ 10.1162/tacl_a_00321. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. InProceedings of the 40th International...

    doi

  9. [24]

    An Yang et al

    URLhttps://proceedings.mlr.press/v202/xiao23c.html. An Yang et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    arXiv

  10. [25]

    doi: 10.48550/arXiv.2505. 09388. URLhttps://arxiv.org/abs/2505.09388. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? InProceedings of the 57th Annual Meeting of the Association for Computational Lin...

    doi arXiv

  11. [26]

    URLhttps://aclanthology.org/P19-1472/

    doi: 10.18653/v1/P19-1472. URLhttps://aclanthology.org/P19-1472/. Jiawei Zhao, Steve Dai, Rangharajan Venkatesan, Brian Zimmer, Mustafa Ali, Ming-Yu Liu, Brucek Khailany, William J. Dally, and Anima Anandkumar. LNS-Madam: Low-precision training in logarithmic number system usi...

    doi

  12. [27]

    URLhttps://doi.org/10.1109/TC.2022.3202747

    doi: 10.1109/TC.2022.3202747. URLhttps://doi.org/10.1109/TC.2022.3202747. Jingwei Zuo et al. Falcon-H1: A family of hybrid-head language models redefining efficiency and perfor- mance.arXiv preprint arXiv:2507.22448,

    2022

  13. [28]

    13 A Algorithmic Details A.1 Closed-product addressing and accumulator bounds Let g= gcd(p, qK), nx =e xK+k x, nw =e wK+k w, and u=p(n x +n w)

    URLhttps://arxiv.org/abs/2507.22448. 13 A Algorithmic Details A.1 Closed-product addressing and accumulator bounds Let g= gcd(p, qK), nx =e xK+k x, nw =e wK+k w, and u=p(n x +n w). Each nonzero product contributes δ= (−1)sx⊕sw , h= umodqK g , t=a x +a w + u qK (7) to integer b...

    arXiv

  14. [2016]

    URL https://arxiv

    doi: 10.48550/arXiv.1609.07843. URL https://arxiv. org/abs/1609.07843. Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenth- waite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, Naveen Mellempudi, Stuart Ober- man, Moham...

    doi arXiv

  15. [2018]

    URL https://arxiv.org/abs/1803

    doi: 10.48550/arXiv.1803.05457. URL https://arxiv.org/abs/1803. 05457. Mario Drumond, Tao Lin, Martin Jaggi, and Babak Falsafi. Training DNNs with hybrid block floating point. InAdvances in Neural Information Processing Systems, volume 31,

    doi arXiv

  16. [2019]

    doi: 10.48550/arXiv.1911. 05507. URLhttps://arxiv.org/abs/1911.05507. Bita Darvish Rouhani, Ritchie Zhao, Ankit More, et al. Microscaling data formats for deep learning.arXiv preprint arXiv:2310.10537,

    1911 doi arXiv

  17. [2020]

    Maxime Christ, Florent de Dinechin, and Fr´ed´eric P´etrot

    URLhttps://arxiv.org/abs/1911.11641. Maxime Christ, Florent de Dinechin, and Fr´ed´eric P´etrot. Low-precision logarithmic arithmetic for neural network accelerators. In2022 IEEE 33rd International Conference on Application-Specific Systems, Architectures and Processors, pp. 72–79,

    1911 arXiv

  18. [2022]

    URL https: //doi.org/10.1109/ASAP54787.2022.00021

    doi: 10.1109/ASAP54787.2022.00021. URL https: //doi.org/10.1109/ASAP54787.2022.00021. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try ARC, the AI2 reasoning challenge.a...

    2022

  19. [2023]

    URL https://arxiv.org/abs/2304.01373

    doi: 10.48550/arXiv.2304.01373. URL https://arxiv.org/abs/2304.01373. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. PIQA: Reasoning about physical commonsense in natural language. InProceedings of the AAAI Conference on Artificial Intelligence,

    doi arXiv

  20. [2024]

    URL https://arxiv.org/abs/2404

    doi: 10.48550/arXiv.2404.00456. URL https://arxiv.org/abs/2404. 00456. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, a...

    doi arXiv

  21. [2025]

    URLhttps://arxiv.org/abs/2502.05376

    doi: 10.48550/arXiv.2502.05376. URLhttps://arxiv.org/abs/2502.05376. Aaron Grattafiori et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    doi

  22. [2026]

    URL https://qwen.ai/ blog?id=qwen3.5. Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, and Timothy P. Lillicrap. Compressive transformers for long-range sequence modelling.arXiv preprint arXiv:1911.05507,

    1911 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.