{"id":"0739c461-d2c0-4058-9dbd-89123b9e5305","arxiv_id":"2502.00026","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dynamic block floating-point format with pivot-focus and adaptive grouping, plus a hierarchical lookup table, lets attention Softmax run in integer-only hardware with negligible accuracy loss.","lead":"DBFP is a block floating-point variant that uses a median pivot and adaptive exponent grouping to make attention's Softmax operation accurate in low-precision integers. The authors pair it with a hierarchical lookup table and an FPGA/ASIC engine, reporting up to 74% GPU Softmax speedup and 10x throughput over prior designs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 10x FOM advantage over SOTA designs is an artifact of comparing different input widths; normalizing N removes the claimed margin.","rationale":"The paper's accuracy results are credible: DBFP with DH-LUT maintains near-floating-point accuracy across LLaMA and vision models, which directly supports the 'negligible accuracy loss' part of the central claim. The performance part, however, has two components: a 74% GPU softmax speedup and a 10x FOM improvement over SOTA. The GPU speedup is reported without a described baseline, which is a reproducibility gap, but the FOM comparison is more clearly load-bearing and more concretely flawed. The FOM metric in Eq. 11 includes N, the number of inputs, and Table 3 compares designs with wildly different N values (1-16 vs 1024). Since the paper explicitly uses FOM to justify the 10x low-overhead claim, the validity of that claim depends entirely on whether FOM is a legitimate cross-design metric. It is not, as N is a workload parameter rather than a design efficiency parameter. The reader's weakest_assumption identifies exactly this issue. My concrete test would settle it: normalize N out of the metric or scale baselines to the same input width. If the advantage disappears, the abstract's performance claim is overstated; if it survives, the design is genuinely superior. Given that the accuracy contribution and the hardware implementation are still valuable, conditional acceptance remains the right verdict, so I do not change the reader's judgment.","tokens_in":13648,"tokens_out":6083,"duration_ms":59309,"concrete_test":"Recompute FOM for all baselines in Table 3 using a normalized metric that removes the input-count factor, e.g., FOM' = Fmax × W / (LUT + FF), or alternatively scale each baseline design to N = 1024 by replicating its datapath and re-measuring resource usage and Fmax. If the 10x FOM advantage over ISCAS'23 shrinks to below 3x or reverses, the headline hardware-performance claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation 11 defines FOM = Fmax × N × W / (LUT + FF). In Table 3, the proposed design is evaluated with N = 1024 inputs, while every prior accelerator processes only N = 1 to 16 inputs. Because N appears linearly in the numerator, a design that simply supports a wider input vector gains a large FOM advantage without any improvement in per-input efficiency. The paper's statement that Ours delivers a 10x FOM improvement over ISCAS'23 therefore conflates workload width with architectural quality. If a prior 8-input design were scaled to 1024 inputs by instantiating 128 parallel datapaths, its resources would also scale roughly linearly, so the normalized FOM (per input) would remain about the same, and the 10x claim would not be reproduced. The central 'low overhead performance improvement over SOTA' claim in the abstract rests on this unnormalized metric.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DB-Attn, a hardware-software co-design framework for attention in LLM inference. It proposes DBFP, a block floating-point variant with a median-pivot alignment and an adaptive grouping strategy; DH-LUT, a hierarchical lookup-table method for computing Softmax in a BFP-like format; and an RTL-level engine evaluated on FPGA and ASIC. The accuracy experiments on LLaMA, LLaMA2, LLaMA3, ViT, Swin, and DETR show that DBFP plus DH-LUT matches FP16/FP32 accuracy much more closely than vanilla BFP or FP8, and the paper reports a 74% GPU Softmax speedup and a 10x FOM improvement over prior softmax accelerators.","tokens_in":13861,"tokens_out":4008,"duration_ms":45783,"significance":"If the central claims hold, the paper makes a useful contribution to narrow-precision LLM inference by showing that attention nonlinearities, especially Softmax, can be approximated with a block floating-point format and integer-friendly lookup tables without meaningful accuracy loss. The accuracy study is broad, covering both language and vision models with external benchmarks, and the hardware work includes an RTL implementation with FPGA and 28nm ASIC synthesis, which is a strength. However, the headline hardware comparison is confounded by an input-width mismatch in the FOM metric, and the integer-only derivation in Eq. (9) is not reconciled with the multi-group DBFP definition, so the hardware and algorithmic significance claims need substantial revision.","major_comments":[{"comment":"The FOM comparison in Table 3 is not normalized by input count, and this invalidates the stated 10x advantage over ISCAS'23. Eq. (11) defines FOM = Fmax x N x W / (LUT + FF), yet the prior accelerators process N = 1 to 16 inputs while Ours processes N = 1024. Since N appears linearly in the numerator, comparing these numbers conflates supported input width with architectural quality. Per input, ISCAS'23 gives 49.056/8 ≈ 6.13, while Ours gives 509.563/1024 ≈ 0.50, so the claimed margin reverses under normalization. The abstract's '10x low overhead performance improvement over SOTA designs' therefore rests on an unnormalized metric and needs to be replaced by a normalized comparison, or the claim must be reformulated as a bandwidth-scaling capability.","section":"Hardware Implement Evaluation, Eq. (11), Table 3"},{"comment":"Eq. (9) claims integer-only Softmax by factoring out a shared exponent s from both numerator and denominator, but this is inconsistent with the adaptive grouping definition of DBFP given earlier. In the Methodology section, DBFP partitions a set X into k subsets, each with its own unique shared exponent s_j. For a Softmax denominator that sums over elements from multiple subgroups, the terms are 2^{s_j} e_int_j with different s_j, so a common factor does not exist and the cancellation shown in Eq. (9) does not hold globally. The paper needs to state explicitly whether each Softmax row is a single DBFP block with one shared exponent, or, if multiple groups are used, how cross-group exponent alignment is performed before the denominator sum. Without this clarification, the central 'integer-only computation' property is not established.","section":"DB-Attn Algorithm Design, Eq. (9)"},{"comment":"The claimed '74% GPU speedup on Softmax of LLaMA' and 'at least 30%' speedups are not reproducible because the baseline is undefined. The Implementations section says a custom CUDA operator emulates DBFP formats and replaces the Attention layer, but it does not specify whether the baseline is a PyTorch Softmax, a cuDNN kernel, a FlashAttention variant, or a custom FP16/FP32 CUDA kernel, nor does it give sequence lengths, batch sizes, or timing methodology. Since the speedup is a headline result in the abstract, the authors must report a precise baseline definition and the measurement conditions.","section":"DBFP GPU Run-time Analysis and Implementations"},{"comment":"The constrained optimization in Eqs. (2)-(6) is not tied to the implemented algorithm or hardware. The paper states that alternating minimization yields an optimal set S, but no implementation step, pseudocode, or ablation shows that the membership variables, the hyperparameter beta, or the empty-set outlier term are actually used in the DBFP conversion or in the DH-LUT construction. As written, the theoretical analysis appears disconnected from the empirical pivot-focus and grouping choices, so the claim of an 'optimal solution' is unsupported. The authors should either show how the optimization maps to the implemented grouping or explicitly present Eqs. (2)-(6) as motivation rather than as a solved design procedure.","section":"Theoretical Analysis, Eqs. (2)-(6)"}],"minor_comments":[{"comment":"The table header 'Methods NUM Format' is ambiguous; please use a separate column for the number of inputs N and a second column for the numeric format, and label units consistently.","section":"Table 3"},{"comment":"The caption 'Pipeline's balanced proportion under input sequences length growth' is incomplete, and the label 'Redunction' appears to be a typo for 'Reduction'; please correct both.","section":"Fig. 5"},{"comment":"Please explain the NaN entries for the FP8 e4m3 rows; if the format cannot represent the required dynamic range, the table should say so explicitly rather than leaving the entries unexplained.","section":"Table 1"},{"comment":"The phrase 'for the first time, completes nonlinear operations in a BFP-like format' is a strong novelty claim and should be softened or supported with a more precise comparison to existing integer-only or BFP-based nonlinear-operation work.","section":"DB-Attn Algorithm Design"},{"comment":"The notation N_gamma, p_gamma_i, and L_E is introduced quickly; please define each symbol in one place and state the range of the shared exponent explicitly.","section":"Optimization of Softmax, Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The accuracy results and the RTL implementation are the strongest parts of the paper, and I see no indication of fabrication. The main obstacle is the FOM normalization issue in Table 3, which affects the headline '10x' claim, together with the unresolved question of how Eq. (9) interacts with multi-group DBFP. Both are fixable with additional experiments and rewriting, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First: the accuracy results are the strongest part. The median-pivot BFP with adaptive grouping plus a hierarchical LUT keeps LLaMA and vision models within noise of FP16 across many tasks, while vanilla BFP collapses. That is a credible, reproducible-looking result and the main reason to read the paper.\n\nWhat is actually new: applying BFP to nonlinear ops via a shared exponent that cancels in softmax division (Eq. 9). That is a neat bit-level trick, not circular, and it enables integer-only softmax. The DH-LUT with non-uniform intervals is sensible and shows a Pareto trade-off between LUT bits and accuracy. The RTL engine is real work—they synthesize on 28nm and report area, power, and throughput.\n\nThe soft spot is the hardware comparison. Table 3 defines FOM = Fmax × N × W / (LUT+FF) and then compares their design at N=1024 against prior designs at N=1–16. Because N appears linearly, a wider input vector mechanically inflates FOM regardless of per-input efficiency. The stress-test is right: scale a prior design to 1024 inputs and its FOM rises by the same factor. So the 10x claim over ISCAS'23 is not apples-to-apples. Re-plot with N normalized, or reframe as bandwidth capability rather than per-resource efficiency.\n\nTwo smaller issues. The clustering derivation in Eqs. 2–6 is elaborate but not tied to the actual grouping or pivot used in the implementation; it reads as a formalization after the fact. The GPU speedup baseline (74% on LLaMA softmax) is not precisely defined. And no code or artifacts are released, so reproducing the accuracy numbers takes effort.\n\nWho this is for: people working on integer-only transformer accelerators or BFP variants. It is an incremental but useful engineering contribution, not a new scientific principle. I would send it to a serious referee if it were submitted, with a request to fix the FOM normalization and clarify the GPU baseline. The accuracy core is solid enough to warrant revision rather than rejection.","headline":"Solid accuracy results and a neat bit-level trick for integer-only softmax, but the 10x FOM gain is an artifact of comparing different input widths.","tokens_in":14367,"tokens_out":3437,"would_cite":true,"duration_ms":31558,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A median-pivot format runs LLM Softmax on integer hardware","keywords":["block floating point","LLM inference","Softmax acceleration","hardware-software co-design","lookup table","attention","quantization","FPGA/ASIC accelerator"],"falsifier":"Recompute Eq. 11 with every design normalized to the same input count, for example by measuring each accelerator's throughput per input at $N=1024$; if the proposed engine's FOM advantage over ISCAS'23 falls below 10x, the headline comparison is an artifact of workload size rather than architectural speed.","tokens_in":1704,"feed_emoji":"⚡","tokens_out":1931,"duration_ms":87326,"temperature":0.7,"pith_summary":"This paper tries to establish that block floating point (BFP), a format in which one exponent is shared across a block of numbers, can be extended from linear operations like matrix multiplication to nonlinear operations like Softmax without losing model accuracy. It introduces DBFP, which aligns each block to the median exponent rather than the maximum and adaptively groups numbers with similar magnitudes so outliers do not distort the whole block. On top of that it builds DH-LUT, a hierarchical lookup table that evaluates the exponential in DBFP using integer arithmetic, and an RTL engine that pipelines max, exponent, shared-exponent subtraction, and division. If the paper is right, attention, the bottleneck of long-context LLM inference, can run on cheap integer-only hardware with accuracy close to floating point.","feed_headline":"A median-pivot format runs LLM Softmax on integer hardware","feed_subtitle":"Attention's exp, sum, and divide become lookup and shift operations, keeping LLaMA accuracy while cutting Softmax latency by 74%.","key_machinery":"The load-bearing object is DBFP, a block floating-point format whose shared exponent is chosen by a pivot-focus policy, meaning median instead of maximum alignment, and whose grouping is adaptive rather than fixed-size. The paper formalizes the grouping as an optimization over membership weights and shared exponents, solved by alternating updates, with an empty-set weight that keeps outliers from dragging the pivot. The companion mechanism is DH-LUT, a non-uniform hierarchical lookup table indexed by shared exponent and high mantissa bits, which evaluates $e^{x_i - x_{\\max}}$ in DBFP so that Softmax becomes a ratio of integer table lookups. This machinery does two jobs: it keeps conversion error small enough to preserve model accuracy, and it turns exponent alignment, addition, and division into single-cycle or shift-based integer operations that the hardware pipeline exploits.","core_discovery":"The paper claims that BFP failed on nonlinear operations for three reasons: outlier sensitivity, coarse exponent granularity, and hardware complexity, and that all three can be addressed by letting the shared exponent follow the data distribution. In DBFP, each block uses a representative value such as the median as its alignment pivot, and blocks are adaptively partitioned so that elements with similar magnitudes share an exponent; the grouping choice is formalized as minimizing $\\sum_{i} \\sum_{S_j} \\mu_{ij}^{\\beta} d_{ij}^2$ with an empty-set term that absorbs outliers. With DBFP, the Softmax exponential can be evaluated by a two-dimensional hierarchical lookup table, and the whole attention path, including matmul, Softmax, and division, can be expressed as integer operations because shared exponents factor out. The paper reports that replacing only the attention layer in LLaMA, ViT, Swin, and DETR keeps zero-shot accuracy and perplexity essentially unchanged, while a CUDA Softmax emulation cuts LLaMA Softmax latency by 74% and the FPGA engine reports a 10x figure-of-merit gain over prior Softmax accelerators.","pith_inferences":["My inference: the 10x figure-of-merit advantage over prior Softmax accelerators is not yet established as a fair architectural comparison, because Eq. 11 counts input count N directly and the proposed design is evaluated at N=1024 while the cited designs process N=1 to 16; normalizing to equal N would likely shrink the gap substantially.","My inference: the median-pivot and adaptive-grouping strategies shift the accuracy burden to a per-block preprocessing step that must select the pivot and partition, so the hardware should also account for data-dependent sorting or grouping cost, which the reported pipeline latency may undercount.","My inference: the 74% GPU speedup is measured with a custom CUDA operator that emulates DBFP, not with the RTL engine, so the software and hardware numbers are separate claims; a fair comparison would report the RTL engine's latency on the same LLaMA Softmax workload.","A testable extension of the same idea would be to apply DBFP to the denominator pass of attention and to GELU in the feed-forward network, then measure whether the integer-only claim extends to a full transformer layer rather than the attention path alone."],"forward_implications":["If DBFP is correct, nonlinear layers no longer force a separate floating-point datapath; the same integer engine used for BFP matmul can perform Softmax, with shared exponents cascading from linear to nonlinear stages without format conversion.","Long-context inference, where attention cost grows quadratically with sequence length, would see the largest relative gain because Softmax's exponential and division become lookup and shift operations instead of multi-cycle transcendental computations.","The DH-LUT approach is described as general, so GELU, LayerNorm, and other elementwise nonlinearities in Transformers could be accelerated the same way by fitting their value ranges into a DBFP-indexed lookup table.","The paper's own results imply that narrow-precision, integer-only LLM inference is a realistic target on FPGA and ASIC, since the attention path is the part that previously forced floating-point hardware."],"supporting_citations":[{"why":"It establishes the block floating-point format, one shared exponent per block, that DBFP modifies with pivot-focus and adaptive grouping.","marker":"Drumond et al. 2018"},{"why":"It shows that BFP-style shared-exponent formats can accelerate narrow-precision inference at scale, the context the paper extends to nonlinear operations.","marker":"Darvish Rouhani et al. 2020"},{"why":"It documents Softmax as a major fraction of transformer inference time, motivating the attention-focused hardware-software co-design.","marker":"Stevens et al. 2021"},{"why":"It provides Hyft, an adaptive-numeric-format Softmax accelerator that the FPGA comparison uses as a state-of-the-art baseline.","marker":"Xia and Zhang 2023"},{"why":"It provides the TCAS-I'22 high-precision base-2 Softmax architecture, one of the baselines in the FPGA figure-of-merit comparison.","marker":"Zhang et al. 2023"},{"why":"It provides the ISCAS'23 hardware-efficient Softmax approximation, the strongest prior baseline the paper claims to beat by 10x.","marker":"Koca, Do, and Chang 2023"},{"why":"It supplies the approximate floating-point divider technique that the DBFP divider unit turns into a single-cycle shift-add operation.","marker":"Jha et al. 2020"}],"fun_headline_variants":["DBFP turns LLM attention into integer lookups","Adaptive BFP: 74% faster LLaMA Softmax","Median-pivot BFP puts Softmax on integer hardware","Lookup-table Softmax: 10x gain over SOTA","Pivot-focus grouping makes Attention integer-only"],"cache_read_input_tokens":16640,"weakest_assumption_plain":"The load-bearing premise is that the figure of merit $\\mathrm{FOM}=F_{\\max}NW/(\\mathrm{LUT}+\\mathrm{FF})$ is a fair comparison across designs with different input counts $N$, because the paper's 10x advantage comes from evaluating its engine at $N=1024$ while the prior accelerators process $N=1$ to $16$.","fun_headline_variants_meta":{"raw":{"variants":["DBFP turns LLM attention into integer lookups","Adaptive BFP: 74% faster LLaMA Softmax","Median-pivot BFP puts Softmax on integer hardware","Lookup-table Softmax: 10x gain over SOTA","Pivot-focus grouping makes Attention integer-only"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":3160,"prompt_tokens":1016,"completion_tokens":2144,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":2061}},"tokens_in":632,"tokens_out":2144,"duration_ms":17555,"temperature":1.0,"reasoning_tokens":2061,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:17:21.581179+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute Eq. 11 with every design normalized to the same input count, for example by measuring each accelerator's throughput per input at $N=1024$; if the proposed engine's FOM advantage over ISCAS'23 falls below 10x, the headline comparison is an artifact of workload size rather than architectural speed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It establishes the block floating-point format, one shared exponent per block, that DBFP modifies with pivot-focus and adaptive grouping."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It shows that BFP-style shared-exponent formats can accelerate narrow-precision inference at scale, the context the paper extends to nonlinear operations."},{"cited_title":"R.; Venkatesan, R.; Dai, S.; Khailany, B.; and Raghunathan, A","cited_arxiv_id":null,"evidence_quote":"It documents Softmax as a major fraction of transformer inference time, motivating the attention-focused hardware-software co-design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the TCAS-I'22 high-precision base-2 Softmax architecture, one of the baselines in the FPGA figure-of-merit comparison."},{"cited_title":"A.; Do, A","cited_arxiv_id":null,"evidence_quote":"It provides the ISCAS'23 hardware-efficient Softmax approximation, the strongest prior baseline the paper claims to beat by 10x."},{"cited_title":"K.; Prasad, K.; Srivastava, V","cited_arxiv_id":null,"evidence_quote":"It supplies the approximate floating-point divider technique that the DBFP divider unit turns into a single-cycle shift-add operation."}],"review_version":1}