{"id":"91000339-4edf-4d32-966b-75cb881e0853","arxiv_id":"2608.10010","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A block-scaled logarithmic format with exact integer-index products matches FP8-class perplexity on 7B-9B language models with seven bits and shows parity with FP8 in 3B-token pretraining.","lead":"CurveFP is a new low-precision number format that makes multiplication a simple addition on small integer codes while using a shared scale per block. It matches FP8 accuracy for language-model training and inference in tests, using seven bits instead of eight for inference and simplifying the hardware product path.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CurveFP8's bounded within-block dynamic range is the load-bearing risk: a block outlier forces small values to underflow to the smallest positive, and the 128M-parameter training evidence does not stress this regime.","rationale":"The central claim is that CurveFP co-designs representation and arithmetic to reach FP8-class LLM quality. The arithmetic half is mathematically solid: product indices close exactly and Eq. 4-5 correctly derive the phase count. The empirical half rests on the representation's fit to real activation and weight distributions. The weakest link is that CurveFP8's per-element exponent field is only 4 bits and all elements in a block share one scale, so within-block dynamic range is capped at roughly 2^16; a single outlier can force coarse quantization of the rest of the block. This is the same broad area the reader flagged, but the reader's stated mechanism ('no per-element exponent') is factually wrong, so I mark partial agreement. The concern is load-bearing because the only training-scale evidence is a 128M model, where outlier stress is far weaker than in 7B and larger models; post-training WikiText-2 results are a narrow probe. The concrete test directly measures the failure mode on a public 7B model without retraining. I do not propose changing the reader's CONDITIONAL verdict: the math and small-scale evidence are honest, and the missing pieces (scale granularity, larger-scale training, code) are exactly what a conditional acceptance should require.","tokens_in":13926,"tokens_out":21915,"duration_ms":211898,"concrete_test":"Run a quantization audit on Llama-3-8B or Qwen3-8B: extract activation tensors from every linear layer over 64-token windows, then for G in {64,128,256,512} compute CurveFP8 E4C3 with ceil-absmax block scaling and FP8 E4M3, using the same scales and calibration as the paper's training path. For each block, count (i) elements that saturate or underflow to the smallest positive, and (ii) elements with relative error greater than 10%. If any layer's median high-error fraction under CurveFP8 exceeds the FP8 fraction by more than 2x, the log-uniform block assumption fails at 7B scale and the 128M training result does not transfer; if the fractions are comparable, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"CurveFP8 E4C3 (Eq. 1) gives each element a 4-bit exponent e in [-8,7] and a 3-bit curve index; with a shared power-of-two block scale, the element field spans a within-block magnitude ratio of about 2^15.875 (~60,000:1). Since all G elements in a block share a_b, any block whose max/min ratio exceeds this range forces either endpoint saturation or mapping of small values to the smallest positive magnitude; affected small values can incur relative errors well above 10%. The log-uniform spacing gives constant relative error for in-range values, so the format's accuracy depends on activations and weights being approximately log-uniform within each block. The central training claim, FP8-class quality, is demonstrated only on a 128.3M-parameter model trained for 3B tokens. Small models exhibit less extreme activation and gradient outliers than 7B-9B models, so the block-range stress that would expose this failure mode is not exercised. The 7B-9B inference results are post-training quantization on WikiText-2 with static scale search (reconstruction-MSE), which may not reflect the dynamic ceil-absmax scaling used in training, nor worst-case blocks. The reader's version of this concern says CurveFP has no per-element exponent; that is incorrect (Eq. 1 includes an E-bit exponent e). The real issue is the fixed range of that exponent combined with the shared scale: the shared scale shifts the window but cannot widen it. Section B.2 also omits the scale granularity (G or tensorwise) used in the Table 4 pretraining runs, so the format's actual operating regime is underspecified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"CurveFP is a block-scaled number format family in which each element stores a sign, an E-bit exponent, and a C-bit curve index under a shared power-of-two scale and a rational radix r=2^{p/q}. The paper shows that multiplication is exactly closed in the code domain (sign XOR, curve-index addition with carry, exponent addition) and derives H=qK/gcd(p,qK) phase classes for accumulation. It instantiates E4C3/E5C2 for training and E3C3 for 7-bit inference, and reports: FP8-class post-training perplexity on four 7B-9B models; lower NMSE in all 36 paired training-GEMM windows; three matched 3B-token pretraining runs at 128.3M parameters showing mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8; downstream transfer parity; and a 4x4 Nangate45 tile that closes at 500 MHz with one fewer product register and about 4.6% smaller area than FP8.","tokens_in":14236,"tokens_out":12129,"duration_ms":122276,"significance":"If the results hold, CurveFP is a genuinely useful co-design: it replaces general product-forming multipliers with small integer additions and sign/carry logic, makes the accumulation cost explicit through the phase law, and appears to reach FP8-class accuracy with a seven-bit inference format. The paper is unusually careful in its statistical hedging: it matches seeds, token order, and schedules across training modes, reports uncertainties, and states parity rather than superiority. It also ships machine-checked hardware verification and an artifact-based reproducibility plan. The main weaknesses are that from-scratch training evidence is limited to a single 128.3M-parameter model trained on 3B tokens, and the per-block scaling configuration in the training protocol is under-specified; these gaps weaken but do not invalidate the central 'FP8-class training' claim.","major_comments":[{"comment":"The matched pretraining protocol never states the block size G or the scale-exponent width S used by CurveFP8's per-block power-of-two scales, even though Eq. (6) makes the storage rate depend on S/G and Figure 2b shows inference quality depends strongly on G. The same section also does not state the scale granularity used for the FP8 baseline; if FP8 used per-tensor scales while CurveFP8 used per-block scales, the comparison in Table 4 confounds datatype with scaling policy. Please report these parameters for both formats.","section":"§4.5, §B.2, Eq. (6)"},{"comment":"For E4C3, Eq. (1) gives each block an element field that spans only a fixed magnitude ratio of 2^{15.875}, about 6×10^4. Since the scale exponent a_b is shared across the block, ceil-absmax scaling shifts this window but cannot widen it; any block whose max/min ratio exceeds this window will force small elements to the smallest positive magnitude or to saturation, with relative errors well above 10% for affected values. The paper does not report outlier statistics, block-level max/min ratios, or sensitivity of the Table 4 results to this failure mode, so the central parity claim rests on an unverified assumption that LLM activations and weights are approximately log-uniform within each block. Please add this analysis or explicitly qualify the claim.","section":"§3.1, Eq. (1)"},{"comment":"The only from-scratch training evidence is a 128.3M-parameter model trained on 3B tokens; the 7B-9B results are post-training quantization only. Since activation and gradient outliers typically become more severe as model scale grows, the conclusion that CurveFP8 provides an 'FP8-class training path' is not established at the scale where FP8 is most relevant. Either extend the training study to a larger model or narrow the claim in the abstract and conclusion to the tested scale.","section":"§4.5, §6"}],"minor_comments":[{"comment":"The sentence 'We reserve magnitude index zero as a sentinel and omit the smallest nonzero exponent/curve combination, leaving 2^{E+C}-1 positive magnitudes' is internally inconsistent: reserving a sentinel and additionally omitting the smallest nonzero combination would leave 2^{E+C}-2 positive levels, not 2^{E+C}-1. Please clarify the codebook construction.","section":"§3.1, Table 1"},{"comment":"The area reductions in the introduction (4.4% and 3.9%) differ from those in the abstract, Section 5, and Table 6 (4.6% and 4.1%); please make the numbers consistent.","section":"§1, §5, Table 6"},{"comment":"Table 3 reports only aggregate NMSE ratios per model and GEMM role; if the text claims all 36 paired comparisons are wins, the individual paired values should be included in the supplementary material for verification.","section":"§4.4, Table 3"}],"recommendation":"major_revision","confidential_remarks":"This is a promising paper with clean derivations and carefully hedged experiments. The main blocking issues are the under-specified per-block scaling configuration in the training protocol and the limited model scale of the from-scratch training study; both are fixable in revision. I do not see a fundamental error in the closed-product or phase-count derivations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the mathematical core is a rational-radix block-scaled LNS lattice, and the paper admits as much in Section 2; the novelty is the datatype interface — uniform curve indices that make products exact sign and index updates, plus the phase-count law H = qK/gcd(p,qK) that ties radix choice to accumulation cost. That law is elementary group theory, but it is correct and it is used, not window dressing. Second, the empirical work is unusually disciplined: matched seeds, token order, and schedules for the 3B-token pretraining triplets, uncertainty reported, parity stated as parity, and the hardware microbenchmark scoped as a feasibility check with explicit exclusions. That is real credit earned.\n\nWhat is genuinely new is the specific CurveFP7/CurveFP8 format pair and the evidence that a seven-bit closed-product format can sit in the FP8 quality regime. The 36 paired GEMM diagnostics all favor CurveFP8, and the three-seed pretraining shows a consistent, if tiny, format-penalty advantage over FP8. The downstream transfer matrix is thorough, and the paper resists overclaiming. The hardware numbers are preliminary and honestly labeled; the 4x4 tile tells you something about product datapath shape, not system efficiency.\n\nNow the soft spots, in proportion. The load-bearing risk is the bounded within-block dynamic range: E4C3's element field spans roughly 2^15.9, and because the block scale shifts but cannot widen the window, any block with a wide max/min ratio will force small values to underflow to the smallest positive magnitude. The stress-test note is right that the reader's claim about \"no per-element exponent\" is wrong — Eq. 1 includes a 4-bit exponent — and that the real issue is the fixed exponent range under a shared scale. That risk is not exercised by the 128.3M-parameter training runs; small models have milder outliers than 7B-9B models. The 7B-9B evidence is post-training quantization on WikiText-2 with MSE scale search, which does not fully replicate dynamic ceil-absmax training conditions. Also, Section B.2 omits the scale granularity for the Table 4 pretraining runs, and the scale-exponent width S in Eq. 6 is never pinned down for the principal formats. Minor, but underspecified.\n\nIf the code ships and the block-range behavior is tested at a larger training scale, this moves from conditional to solid. The algebra is sound, the experiments are honest, and the related work is placed correctly. I would send it to review: the referees should probe the within-block range story and ask for the missing protocol details, but the paper is coherent and worth that time.","headline":"Carefully built block-scaled LNS variant with an honest matched-experiment suite; the core training claim rests on a 128M run, but the math is correct and the paper deserves a serious referee.","tokens_in":14816,"tokens_out":1193,"would_cite":true,"duration_ms":16159,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CurveFP co-designs a low-precision number format with its product arithmetic, claiming that 7-bit log-curve codebooks can match or beat FP8 for language-model inference and training.","keywords":["CurveFP","low-precision arithmetic","logarithmic number system","LLM quantization","closed-product algebra","block scaling","rational radix","language model pretraining"],"falsifier":"On a model whose activations contain per-block outliers larger than the chosen power-of-two scale, measure CurveFP8 versus FP8 perplexity at block sizes 64 and 512; if the fraction of values saturating the CurveFP endpoint grows enough to push its perplexity more than 0.02 above FP8's, the block-log-uniform premise is the limiting factor.","tokens_in":13688,"feed_emoji":"🧮","tokens_out":9785,"duration_ms":91888,"temperature":0.7,"pith_summary":"CurveFP claims that low-precision number formats for language models can be designed together with the arithmetic that multiplies them, rather than optimizing scalar accuracy first and then paying for a conventional multiplier. It arranges quantized magnitudes on several interleaved logarithmic curves under a shared power-of-two block scale, so every nonzero product becomes an exact combination of sign, integer-index addition, exponent addition, and a carry. The paper instantiates this idea as CurveFP8 for training and CurveFP7 for seven-bit inference, and reports that CurveFP7 beats tensorwise FP8 perplexity on four 7B–9B models with one fewer element bit, that CurveFP8 lowers error in all 36 paired training-GEMM comparisons, and that in three matched 3B-token pretraining triplets CurveFP8 reaches mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8, with a lower format penalty in every seed. A preliminary 4×4 spatial tile closes at 500 MHz with one fewer product register and 4.6% less area than the FP8 tile. The paper presents these results as support for a co-designed representation-and-arithmetic family while explicitly leaving system-level efficiency and larger-scale training to future study.","feed_headline":"7-bit CurveFP beats FP8 on LLM perplexity","feed_subtitle":"A closed-product log lattice matches FP8 training and beats it at 7-bit inference.","key_machinery":"The load-bearing object is the CurveFP codebook: a block of values sharing one power-of-two scale $2^{a_b}$, each nonzero element storing a sign, an exponent $e$, and a curve index $k\\in\\{0,\\dots,K-1\\}$, with magnitude $2^{a_b}r^{e+k/K}$ where $r=2^{p/q}$ is a reduced rational radix and $K=2^C$. The uniform curve index is what carries the argument: multiplication becomes sign XOR, $(k_x+k_w)\\bmod K$, and $e_x+e_w+\\lfloor(k_x+k_w)/K\\rfloor$, so product formation needs no general multiplier and no codebook lookup. The radix $r$ then fixes the phase count $H=qK/\\gcd(p,qK)$ of Equation 4, which schedules accumulation as a sum over $H$ fixed phase weights of binary-shifted integer counts, making the hardware cost of a precision choice explicit.","core_discovery":"On its own terms, CurveFP's discovery is that uniform spacing of the curve index makes the product of any two representable values exactly representable in the same codebook: signs XOR, curve indices add modulo $K$ with a carry into the exponent, exponents add, and shared power-of-two scales add. Because magnitudes sit on a rational-radix lattice $r^{e+k/K}=2^{p(eK+k)/(qK)}$, the number of distinct fractional phases a dot product must accumulate is $H=qK/\\gcd(p,qK)$, so every range–resolution choice carries an explicit accumulation contract. Instantiated as CurveFP8 E4C3/E5C2 and CurveFP7 E3C3, this structure reaches FP8-class pretraining parity—mean BF16-inference perplexity 22.5366 versus 22.5407 for FP8 across three matched 3B-token runs—and CurveFP7 at seven bits beats tensorwise FP8 perplexity on four 7B–9B models while staying within 1.32% of native quality. It also reports a preliminary routed tile in which CurveFP8 closes at 500 MHz with one fewer product register and 4.6% less area than the timing-closing FP8 tile.","pith_inferences":["Beyond the paper: if within-block log-uniformity holds for larger models, CurveFP's phase schedule could combine with outlier-aware scale selection (per-row or per-token scales) to absorb activation outliers without a conventional multiplier; the paper tests only fixed block sizes 64–512.","Beyond the paper: the phase law suggests a hardware experiment the paper does not run—fix tile area and sweep radix from $p/q=1$ to $3/4$, and measure whether the added phase-reduction logic erodes the 4.6% area advantage as $H$ grows from 8 to 32.","Beyond the paper: the codebook is parameter-free apart from E, C, p/q, and block scale, so the same closed-product algebra could be applied to KV-cache or attention-score tensors, whose distributions may not be log-uniform; that remains untested.","Beyond the paper: because the paper's training evidence is one 128.3M-parameter architecture at 3B tokens, a direct extension is a matched FP8-versus-CurveFP8 run at 7B+ scale, where activation distributions may deviate further from log-uniform blocks."],"forward_implications":["Training GEMMs can drop the general mantissa multiplier: with the same role split as FP8 (E4C3 forward, E5C2 gradients), every nonzero product is a few integer additions, and the product path closes timing at 500 MHz without an extra register in the preliminary tile.","Seven-bit inference becomes a practical operating point: at block size 64, CurveFP7 beats tensorwise FP8 perplexity on all four tested 7B–9B models and stays within 1.32% of native quality, and at block size 512 it crosses INT8.","The phase law $H=qK/\\gcd(p,qK)$ gives designers an explicit quality–complexity dial: a denser radix such as $r=2^{3/4}$ improves perplexity by 0.0055–0.0259 but requires 32 phases instead of 8.","Matching training and runtime arithmetic nearly halves the out-of-distribution penalty for CurveFP8, and the CurveFP8-trained checkpoint transfers to seven-bit runtime, so one numeric family can span training and deployment without retraining.","The paper's hardware evidence is limited to a feasibility tile: area savings of 4.6% at equal peak throughput come with memory, interconnect, scale transport, and workload effects explicitly excluded from both designs."],"supporting_citations":[{"why":"Defines the FP8 E4M3/E5M2 formats that serve as CurveFP's principal baseline and the role split CurveFP8 mirrors.","marker":"(Micikevicius et al., 2022)"},{"why":"Introduces per-block shared scales, the mechanism CurveFP adapts as its block-scaling contract.","marker":"(Rouhani et al., 2023)"},{"why":"Establishes logarithmic data representation whose additive product algebra CurveFP's index-domain products extend.","marker":"(Miyashita et al., 2016)"},{"why":"Supplies the outlier-migration baseline that CurveFP claims to render unnecessary via log-lattice density.","marker":"(Xiao et al., 2023)"},{"why":"Provides a rotation-based outlier-removal method representing the preprocessing class CurveFP contrasts with.","marker":"(Ashkboos et al., 2024)"},{"why":"Provides the model whose activations and gradients give the paired training-GEMM diagnostics.","marker":"(Biderman et al., 2023)"},{"why":"Supplies the Llama checkpoints used in the post-training inference comparisons and GEMM windows.","marker":"(Grattafiori et al., 2024)"},{"why":"Supplies the pretraining corpus used for the three matched 3B-token runs.","marker":"(Penedo et al., 2024)"}],"fun_headline_variants":["Every nonzero product exact on CurveFP's log lattice","CurveFP7: 7-bit inference beats FP8 perplexity","CurveFP8 lowers error in all 36 training GEMMs","CurveFP co-designs format and product arithmetic","Closed-product log lattice reaches FP8-class training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, within each block, magnitudes are approximately log-uniform, so a single power-of-two scale plus the fixed log lattice can represent activations and weights accurately; because CurveFP has no per-element exponent, any outlier that the block scale cannot absorb is saturated or coarsely quantized.","fun_headline_variants_meta":{"raw":{"variants":["Every nonzero product exact on CurveFP's log lattice","CurveFP7: 7-bit inference beats FP8 perplexity","CurveFP8 lowers error in all 36 training GEMMs","CurveFP co-designs format and product arithmetic","Closed-product log lattice reaches FP8-class training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001923,"raw_usage":{"total_tokens":7578,"prompt_tokens":1047,"completion_tokens":6531,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":6448}},"tokens_in":663,"tokens_out":6531,"duration_ms":45638,"temperature":1.0,"reasoning_tokens":6448,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:38:01.790569+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a model whose activations contain per-block outliers larger than the chosen power-of-two scale, measure CurveFP8 versus FP8 perplexity at block sizes 64 and 512; if the fraction of values saturating the CurveFP endpoint grows enough to push its perplexity more than 0.02 above FP8's, the block-log-uniform premise is the limiting factor.","supporting_citations":[],"review_version":2}