{"id":"92f6af26-2488-452a-87c6-9272079b4402","arxiv_id":"2509.22536","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A hybrid-granularity FP8 training recipe achieves near-lossless accuracy versus BF16 on reasoning benchmarks with up to 22% faster training and 14% lower memory.","lead":"This paper presents an end-to-end FP8 training recipe for large language models that combines block-wise weight quantization with token-wise activation quantization and UE8M0 scaling factors. It reports that FP8 training matches BF16 accuracy on reasoning benchmarks while reducing training time by up to 22% and peak memory by up to 14%.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FP8 losslessness at 7B rests on an untested transfer from 1.5B; the flagship InfiR2-7B-FP8 lacks a BF16 twin under the same 160B-token recipe.","rationale":"The paper's strongest evidence for losslessness is the 1.5B head-to-head comparison in Section 5.3, which is a real strength: it uses the identical recipe for both precisions and shows overlapping loss curves. However, the flagship contribution is the 7B model, and the losslessness claim there is supported only by indirect evidence: an SFT-only comparison on a different base model (Qwen2.5-Math-7B, Table 4) and a performance table (Table 2) that compares against a base model without the same continual-pretraining recipe. The reader's weakest assumption correctly identifies this gap. Additionally, the stated 1–2 point noise bound is contradicted by the 2.46-point GPQA gap in Table 5, which is a concrete sign that the bound is not universally applicable. This does not invalidate the paper, but it raises the risk that the central claim may not transfer to the scale and data mixture used for the flagship model. A BF16 twin run at 7B with the full recipe would settle the question, but it is expensive—hence a conditional verdict is appropriate. The efficiency numbers (Table 6) are concrete and independently useful, but they do not address the accuracy-transfer concern.","tokens_in":11437,"tokens_out":4071,"duration_ms":30660,"concrete_test":"Train Qwen2.5-7B-base with the same recipe as InfiR2-7B-FP8 (160B-token continual pre-training, then two-stage SFT with InfiAlign-SFT-72k/165k, same hyperparameters) in BF16, and evaluate on AIME24/25, GPQA, and LiveCodeBench v5. If the FP8 model lies within the stated 1–2 point noise bound of this BF16 twin on all benchmarks, the losslessness claim holds at scale; if not, the central claim is falsified for the flagship model. Also report whether the GPQA gap reproduces.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central 'essentially lossless' claim is directly validated only at the 1.5B scale (Section 5.3, Table 5), where FP8 and BF16 are compared under the identical 160B-token continual pre-training and SFT recipe. Loss curves overlap, but GPQA differs by 2.46 points (31.94 vs 29.48), exceeding the stated 1–2 point noise bound. The flagship InfiR2-7B-FP8 (Table 2) has no BF16-trained counterpart with the same recipe; the only 7B evidence is SFT-only on Qwen2.5-Math-7B (Table 4), which does not exercise the full continual-pretraining dynamics that could amplify FP8 quantization error over 160B tokens. If scale or data mixture changes how FP8 rounding accumulates, the losslessness claim is unverified exactly where it matters most—the 7B launch model. The paper's assertion that FP8 training is 'essentially lossless' is therefore conditional on an untested transfer assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents InfiR2, an end-to-end FP8 training recipe for continual pretraining and supervised fine-tuning of reasoning-oriented LLMs. The method combines block-wise weight quantization, token-wise activation quantization, E4M3 for forward/backward passes, UE8M0 power-of-two scaling factors, and FP32 master weights/gradients/optimizer states. Experiments include 160B-token continual pretraining of Qwen2.5-1.5B/7B models followed by two-stage SFT, plus a controlled BF16 vs FP8 comparison on the 1.5B model (Table 5, Figure 4) and SFT-only comparisons on Qwen2.5-Math models (Table 4). The paper reports training-time reductions up to 22%, peak-memory reductions up to 14%, and throughput gains up to 19% (Table 6), and claims FP8 training is 'essentially lossless' relative to BF16. The flagship artifact is InfiR2-7B-FP8, released with code and checkpoints.","tokens_in":11745,"tokens_out":4101,"duration_ms":33277,"significance":"If validated, the paper makes a practical contribution: a reusable, open FP8 recipe for large-scale continual pretraining and SFT, with a head-to-head 1.5B full-pipeline comparison showing overlapping loss curves and broadly comparable benchmark scores. The release of code, training logs, and checkpoints is a concrete strength that lowers the barrier to reproducing and extending FP8 training. The main scientific claim, however, is a losslessness claim, and that claim is currently supported directly only at the 1.5B scale; the 7B flagship lacks a BF16 counterpart under the same recipe. Given the field's reliance on benchmark deltas, the 2.46-point GPQA gap in the key validation table also needs statistical grounding. These issues do not invalidate the recipe, but they should be fixed before the paper's central claim can be accepted as stated.","major_comments":[{"comment":"The central 'essentially lossless' claim rests on this single full-pipeline BF16 vs FP8 comparison at the 1.5B scale. On GPQA the FP8 model scores 29.48 versus 31.94 for BF16, a 2.46-point gap, which is larger than the '1-2 point margin attributable to evaluation noise' invoked in Sections 5.2.3 and 6. The paper provides no confidence intervals, standard errors, or multiple-seed runs for any benchmark, so the reader cannot tell whether this gap is noise or systematic. Please either report statistical uncertainty or soften the claim to 'typically within 1-2 points, with one exception' and discuss the direction of the discrepancy.","section":"Section 5.3, Table 5"},{"comment":"The flagship InfiR2-7B-FP8 has no BF16-trained counterpart under the same 160B-token continual-pretraining recipe. Figure 3 shows only FP8 loss curves at 7B, while Table 4 is an SFT-only comparison starting from Qwen2.5-Math models and does not exercise the accumulation of FP8 quantization errors over 160B tokens of continual pretraining. Thus the losslessness claim is unverified at the scale where the paper's main model sits. This is the key load-bearing gap. I recommend either providing a 7B BF16 run under the same recipe (even with a reduced token budget) or explicitly limiting the losslessness claim to the 1.5B scale and reframing the 7B results as a demonstration of recipe stability rather than losslessness.","section":"Section 5.1.2-5.1.3, Table 2, Figure 3"},{"comment":"The efficiency claims are reported as 'up to' maxima, and the underlying measurement conditions are incomplete. In two of the four configurations the FP8 forward pass is slower than BF16 (1.5B/32k: 875 vs 841 ms; 7B/8k: 2300 vs 1760 ms), and the peak-memory ratio at 7B/8k is only 0.95. The paper should state the hardware (GPU model), software stack (kernel library, CUDA version), number of repeats, and whether reported times are medians or means. Without this, the reader cannot assess how representative the headline 22%/14%/19% gains are, and the claim of a 'comprehensive recipe' is difficult to reproduce.","section":"Section 5.4, Table 6"}],"minor_comments":[{"comment":"The continual-pretraining configuration lists only batch size, learning rates, weight decay, and context length; it omits optimizer, scheduler, warm-up, gradient clipping, and precision layout of optimizer states. The SFT table omits the optimizer and scheduler details as well. Please provide the full configuration for reproducibility.","section":"Tables 1 and 3"},{"comment":"The variant 'FP8 w. FP32 scale' in Table 4 is not defined in the setup. Please clarify how it differs from the standard FP8 variant (e.g., whether UE8M0 scaling is replaced by ordinary FP32 scaling).","section":"Section 5.2.1"},{"comment":"The term 'Decay' in the legend is not explained. I assume it denotes a learning-rate decay phase; please define it in the caption or text.","section":"Figures 3 and 4"},{"comment":"The throughput ratio for the 1.5B/8k configuration is listed as 1.10x while the total-time ratio is 0.78x. The relationship between these metrics is not fully transparent; please state how throughput is computed and why it does not exactly track total time.","section":"Table 6"},{"comment":"There are numerous typographical errors and malformed references: 'Minimun Learning Rate' (Table 1), 'performe' (Section 5.2.1), 'Daviditat B. de la Torre' and 'annd Fan-Kai Lin' in the Nemotron reference, and inconsistent URL formatting. A careful proofreading pass is needed.","section":"General / References"},{"comment":"The LLM usage statement is transparent and useful; I suggest including it in the main text or at least keeping it in the version of record.","section":"Appendix A.1"}],"recommendation":"major_revision","confidential_remarks":"The central claim is a strong losslessness statement, but the evidence at the flagship scale is incomplete: the 7B model has no BF16 twin under the same recipe, and the one full-pipeline comparison at 1.5B contains a GPQA gap larger than the stated noise bound. In my view, the editor should require either a 7B BF16 comparison or a revised, scoped claim before accepting. The paper's value as an open recipe is clear, and I do not see evidence of circularity or inappropriate fitting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things before reading this. First, it's an engineering recipe paper, not a scientific breakthrough, but the recipe is concrete and mostly well-validated at 1.5B. Second, the central claim—that FP8 training is essentially lossless—is directly supported only at 1.5B for the full continual pre-training + SFT pipeline. The 7B flagship has no BF16 twin under the same 160B-token recipe, so that part is an extrapolation.\n\nWhat the paper does well: it assembles known components (block-wise weight quantization, token-wise activation quantization, UE8M0 scaling) into a specific hybrid that is genuinely new as a combination. The 1.5B validation is solid: matched loss curves over 160B tokens and a benchmark table where FP8 stays within a few points of BF16. The SFT-only comparison at 7B (Table 4) also shows parity, which partially offsets the missing full-pipeline comparison. Efficiency numbers are modest and honest—7–22% time reduction, not the theoretical 2x—and the paper presents them as practical gains.\n\nWhere the soft spots are: the GPQA gap in Table 5 is 2.46 points (31.94 vs 29.48), larger than the paper's stated 1–2 point noise bound. That inconsistency matters because the noise bound is used to define \"lossless.\" It doesn't kill the paper, but the bound is hand-wavy. The bigger issue is the 7B transfer: the only 7B losslessness evidence is SFT-only, which doesn't exercise the long-horizon continual pre-training dynamics where FP8 error could accumulate. The paper also promises code and checkpoints but doesn't ship them, which limits reproducibility now, though that's common for preprints under review.\n\nBottom line: this deserves a serious referee. It's a useful recipe paper that the community will cite if the missing baseline is addressed or the claim is softened to \"lossless at 1.5B full pipeline and 7B SFT.\" I'd send it to peer review with a request for the 7B BF16 continual pre-training run, or a more carefully scoped claim.","headline":"A practical FP8 recipe worth refereeing, but the 'essentially lossless' claim is fully verified only at 1.5B; the 7B flagship lacks a matched BF16 continual pre-training baseline.","tokens_in":12211,"tokens_out":1973,"would_cite":true,"duration_ms":16070,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid-granularity FP8 recipe makes 8-bit training match BF16 accuracy on reasoning benchmarks.","keywords":["FP8 training","low-precision LLM training","hybrid granularity quantization","UE8M0 scaling factor","continual pre-training","supervised fine-tuning","reasoning benchmarks","training efficiency"],"falsifier":"Train the identical 7B model and data mixture in BF16 under the same recipe and compare loss curves and benchmark scores with the published FP8 run; if the gap exceeds the 1–2 point evaluation-noise band, or the loss curves diverge, the essentially lossless claim fails. A cheaper check: re-run the 1.5B comparison with several seeds to see whether the observed FP8-vs-BF16 deltas are stable or just noise.","tokens_in":11379,"feed_emoji":"⚡","tokens_out":4810,"duration_ms":37516,"temperature":0.7,"pith_summary":"The paper sets out to show that training large language models in FP8, not just BF16, can be a practical default: it claims an end-to-end recipe for continual pre-training and supervised fine-tuning that keeps model quality within evaluation noise of the BF16 baseline while cutting training time, memory, and compute. The recipe's core is hybrid-granularity quantization — block-wise scales for weights, token-wise scales for activations — combined with an exponent-only scaling-factor format that rounds up to powers of two. The paper validates the claim on a 160B-token continual pre-training run plus two-stage SFT, reporting nearly identical loss curves and benchmark scores within 1–2 points, alongside up to 22% faster training, 14% lower peak memory, and 19% higher throughput. If correct, the work turns FP8 from a promising research direction into a drop-in replacement that makes large-scale reasoning-model training cheaper and more accessible.","feed_headline":"FP8 training matches BF16 accuracy on reasoning models","feed_subtitle":"A hybrid-granularity recipe cuts training time up to 22% and memory 14% with no benchmark loss.","key_machinery":"The load-bearing object is a hybrid-granularity quantization scheme that splits FP8 tensors into two regimes: weights are quantized per-block (small sub-matrices, each with its own scaling factor), while activations are quantized per-token (a distinct scale per token position), because activations have wider dynamic ranges and strong outliers. Scaling factors are stored in a UE8M0 (E8M0) format that represents only an exponent and rounds each scale up to the nearest power of two; this guarantees the quantized tensor stays within the FP8 range and stabilizes training. Master weights, weight gradients, and optimizer states remain in FP32, so small updates accumulate without rounding loss. The","core_discovery":"On the paper's own terms, the central discovery is that FP8 training can be essentially lossless when quantization granularity is matched to tensor type: weights get block-wise quantization, activations get finer token-wise quantization, and the scaling factor uses an E8M0 exponent-only format that rounds upward to powers of two so the FP8 range is never overpushed. Throughout a 160B-token continual pre-training of a 1.5B model, FP8 and BF16 loss curves overlap almost exactly, and after supervised fine-tuning the models score within a 1–2 point band — attributed to evaluation noise — on reasoning benchmarks. The same recipe applied to a larger 7B model yields a reasoning model that substanti","pith_inferences":["The 1.5B head-to-head validates losslessness; the 7B model has no BF16 twin under the same recipe, so the strongest version of the claim — lossless at scale — remains an extrapolation.","The reported speedups (up to 22%) are well below FP8's theoretical 2x; the gap suggests quantization and communication overheads dominate, and a tuned kernel implementation could push efficiency higher.","FP8's occasional accuracy gains on the smaller model could come from quantization acting as a regularizer; a control experiment with multiple seeds would separate that effect from noise.","Memory savings would be most valuable in long-context or MoE training, where activation memory dominates; the recipe is a natural starting point for those regimes."],"forward_implications":["FP8 can be used for the full training pipeline — continual pre-training and two-stage SFT — without a benchmark-quality penalty.","Training time drops by up to 22%, peak memory by up to 14%, and throughput rises by up to 19% relative to BF16.","The near-identical loss curves imply FP8 training preserves learning dynamics, not just final scores.","The recipe's efficiency gains allow larger batch sizes or longer contexts on the same hardware.","Models trained with this recipe on a 7B base show large reasoning gains over their starting checkpoint."],"fun_headline_variants":["FP8 training: lossless accuracy, 22% faster","Hybrid FP8 recipe matches BF16, cuts memory 14%","FP8 reasoning models: lossless, 19% higher throughput","160B-token FP8 training: BF16 parity, 22% time cut"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The losslessness claim rests on a 1.5B model where FP8 and BF16 were directly compared; if the same equivalence does not hold for the 7B model — which was trained only in FP8 — the central claim is unproven for the flagship result.","fun_headline_variants_meta":{"raw":{"variants":["FP8 training: lossless accuracy, 22% faster","Hybrid FP8 recipe matches BF16, cuts memory 14%","FP8 reasoning models: lossless, 19% higher throughput","160B-token FP8 training: BF16 parity, 22% time cut"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000645,"raw_usage":{"total_tokens":2794,"prompt_tokens":733,"completion_tokens":2061,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":1982}},"tokens_in":477,"tokens_out":2061,"duration_ms":12433,"temperature":1.0,"reasoning_tokens":1982,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T14:48:54.515254+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the identical 7B model and data mixture in BF16 under the same recipe and compare loss curves and benchmark scores with the published FP8 run; if the gap exceeds the 1–2 point evaluation-noise band, or the loss curves diverge, the essentially lossless claim fails. A cheaper check: re-run the 1.5B comparison with several seeds to see whether the observed FP8-vs-BF16 deltas are stable or just noise.","supporting_citations":[],"review_version":1}