{"id":"3f18fb37-aeae-455a-868e-f29e2f90e2f0","arxiv_id":"1908.02386","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"16-bit posits train MNIST and Fashion-MNIST feedforward networks with less accuracy loss than 16-bit floating point, and 5 to 8 bit posits yield better inference accuracy and energy-delay tradeoffs than float or fixed point.","lead":"Researchers built Cheetah, a software and hardware framework for testing low-precision number formats in neural networks that run on edge devices. They report that 16-bit posits train small networks with less accuracy loss than 16-bit floats, and that 5 to 8 bit posits improve the speed-versus-energy tradeoff for inference.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training comparison is against a vanilla FP16 baseline without loss scaling or master weights; the claimed posit advantage likely reflects FP16's narrow exponent range and is internally inconsistent with the paper's <1% claim.","rationale":"The reader's weakest assumption is exactly the load-bearing point: the FP16 training baseline in Table V is not representative of practical FP16 training, and the posit advantage could disappear once loss scaling, FP32 master weights, or BFLOAT16 is used. This matters because the paper's most distinctive contribution is the training claim; the inference and EDP results have some independent support from the FPGA synthesis and internal consistency, but the training claim is based on two small datasets, one network type, no error bars, and an unreported posit es configuration. The paper's own 'less than 1% accuracy degradation' assertion is contradicted by its Table V, which further weakens confidence. I do not see a reason to move the reader's CONDITIONAL verdict: the issue is addressable with additional experiments, but it must be addressed before the central claim can be accepted.","tokens_in":18229,"tokens_out":6088,"duration_ms":63881,"concrete_test":"Re-run the four-layer FC experiments of Table V on MNIST and Fashion-MNIST with identical architecture, data order, and hyperparameters, reporting mean and standard deviation over 10 seeds, for: (a) IEEE FP16 with FP32 master weights, loss scaling, and stochastic rounding following Micikevicius et al. [34]; (b) BFLOAT16 following Kalamkar et al. [37]; and (c) Posit-16 with the exact es used in the paper. If (a) or (b) matches Posit-16 accuracy within about 0.5 percentage points, the claimed posit training advantage is a baseline artifact rather than a format advantage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central training claim (Section V-C, Table V) rests on a comparison between 16-bit posits and a deliberately bare 16-bit float pipeline: no loss scaling, no FP32 master weight copy, no stochastic rounding, and no reported mixed-precision accumulation. Under those conditions Float-16 collapses to 90.646% on MNIST and 81.725% on Fashion-MNIST, roughly 7.4 and 7.4 percentage points below Float-32, which is far worse than established FP16 training results (e.g., Micikevicius et al. [34] maintain near-FP32 accuracy on much harder tasks). Posit-16's advantage over this baseline may therefore be an artifact of FP16's narrow 5-bit exponent range versus the wider dynamic range of the chosen posit16 configuration; Table V does not report es for Posit-16, even though posit range and precision depend strongly on es (Eq. 3). The paper's own text claims 'less than 1% accuracy degradation' for 16-bit posits, but Table V shows Posit-16 is 1.60 percentage points below Posit-32 on MNIST and 1.86 percentage points below on Fashion-MNIST, an internal inconsistency. Until a fair FP16 baseline is run and the posit configuration is specified, the headline claim that 16-bit posits outperform 16-bit floating point in DNN training is unsupported as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Cheetah, a hardware/software co-design framework for low-precision DNN training and inference that supports posit, floating-point, and fixed-point formats under rounding and linear quantization. The framework couples software emulation in Keras/TensorFlow with an FPGA soft-core implementation of exact multiply-and-accumulate (EMAC) operations. The authors evaluate inference on MNIST, Fashion-MNIST, and CIFAR-10 at 5–8 bit widths and training on MNIST and Fashion-MNIST at 16 and 32 bits. The two headline claims are that 16-bit posits outperform 16-bit floating point in DNN training, and that 5–8-bit posits improve the accuracy-versus-energy-delay-product trade-off over float and fixed-point for inference. The paper also reports on the EMAC FPGA implementation and compares resource usage, latency, and energy-delay product across formats.","tokens_in":18515,"tokens_out":4347,"duration_ms":48149,"significance":"If substantiated, the training result would be a notable contribution: it would show a simple round-to-nearest posit pipeline matching or exceeding FP16 training without loss scaling or master-weight copies, which would be attractive for edge training. The inference study is also useful in that it compares formats at equal bit-widths, uses a consistent EMAC hardware baseline, and reports a concrete FPGA implementation. The distortion-rate analysis (Fig. 4) is a reasonable way to connect format properties to DNN parameter statistics. The framework itself, with support for multiple formats and quantization schemes, is a useful artifact for the community. However, the central training claim rests on a comparison against a vanilla FP16 baseline that lacks established mixed-precision techniques, and the paper contains a direct numerical inconsistency in the magnitude of the claimed accuracy degradation. As a result, the headline result is not yet established by the evidence presented.","major_comments":[{"comment":"The claim that '16-bit posits outperform 16-bit floating point in DNN training' is not supported as stated because the FP16 baseline is a bare pipeline with no loss scaling, no FP32 master-weight copies, no stochastic rounding, and no mixed-precision accumulation. The cited prior work on FP16 training (e.g., [14], [34]) uses these techniques specifically to prevent the accuracy collapse that Table V shows for Float-16 (90.646% on MNIST and 81.725% on Fashion-MNIST, roughly 7.4 points below Float-32). Without a fair FP16 baseline, the observed gap may reflect FP16's limited 5-bit exponent range rather than any advantage of the posit format. Please rerun the comparison with a state-of-the-art mixed-precision FP16 baseline, or explicitly restrict and justify the claim to a 'vanilla FP16 without mixed-precision techniques' comparison.","section":"Section V-C, Table V"},{"comment":"The text states that 16-bit posits show 'less than 1% accuracy degradation' without exhaustively analyzing DNN training parameters. Table V, however, shows Posit-16 at 96.535% versus Posit-32 at 98.131% on MNIST (a 1.60 percentage-point drop) and at 87.400% versus 89.263% on Fashion-MNIST (a 1.86 percentage-point drop). This is an internal inconsistency between the text and the reported numbers, and it must be corrected. In addition, although Table V reports averages over 10 independent runs, no variance, standard deviation, or significance test is given, so the claimed 'slightly better accuracy' of Posit-16 over Float-16 cannot be assessed statistically.","section":"Section V-C, final paragraph and Table V"},{"comment":"The table reports accuracy for 'Posit-16' without specifying the posit exponent size es. Since a posit's value, dynamic range, and precision depend critically on es (Eq. 3), and since the paper elsewhere shows that es ∈ {0,1,2} materially changes accuracy (Table IV), the reported Posit-16 numbers are not reproducible or comparable to Float-16 without knowing which es was used. Please report the exact posit configuration and the analogous float exponent-bit width for every training run.","section":"Section V-C, Table V and Section II-B"},{"comment":"The inference comparison in Table III reports 'respective best results' for each format, with posit es ∈ {0,1,2} and floating-point exponent width we ∈ {3,4}. This is a form of selecting the best exponent configuration per format, but the table does not indicate which es or we was chosen for each bit-width and dataset. If the best configuration is selected per cell on the test set, the comparison is optimistic for both formats and should be justified or replaced with a fixed configuration policy; at minimum, the selected configurations should be listed so that the reader can assess whether the comparison is balanced.","section":"Section V-A, Table III"},{"comment":"The description of the training pipeline is incomplete in ways that are load-bearing for the training claim. The paper says networks are trained 'using only the specified numerical format,' but it does not state how gradients, weight updates, and biases are quantized, whether the EMAC/quire accumulator is used during training as well as inference, how the learning rate is represented, or what optimizer and hyperparameters are used. Without this information, the FP16 and Posit-16 results in Table V cannot be reproduced, and it is unclear whether the comparison isolates the numerical format or conflates it with pipeline choices.","section":"Section V-C and Section IV-A"}],"minor_comments":[{"comment":"The sentence 'However, the scope of these studies is limited, as highlighted below' is followed by item 1: 'None of the previous works explore the propriety of the posit numerical format'; 'propriety' should be 'appropriateness' or 'suitability'.","section":"Section I, Introduction"},{"comment":"The legends in Figures 8 and 9 contain garbled tokens such as '/uni00000029/uni0000004c/...' instead of readable format labels. These figures are otherwise central to the EDP comparison and need to be regenerated with correct text.","section":"Figures 8 and 9"},{"comment":"Table VI lists 'Courbariaux et al. [45]' as a separate row, but reference [45] appears to be the same arXiv paper as reference [32] (both cite arXiv:1412.7024) with different titles. This duplication should be cleaned up, and the table's 'DNN library' entry for Wang et al. ('Home Suite') should be clarified.","section":"Table VI and References"},{"comment":"The text says that β is selected from {1, 2, 4, 8} to provide 'a wide coverage (~82%) of the dynamic range of each numerical format, as shown in Table I,' but Table I reports coverage percentages for the numerical formats themselves, not for the β values. The connection between the β grid and the coverage percentages is not explained and should be made explicit.","section":"Section IV-A and Table I"},{"comment":"The abstract states that Cheetah 'supports mixed-precision floating point and fixed-point numerical formats,' but the training experiments reported here use only a single 16- or 32-bit format per network with no mixed-precision configuration. Please either add such experiments or soften the abstract to describe the framework's capability rather than claiming demonstrated mixed-precision training.","section":"Abstract and Section IV"}],"recommendation":"major_revision","confidential_remarks":"The paper's strongest contribution is the framework and the inference-side comparison, which are concrete and worth publishing after revision. The training claim, however, is the paper's headline and is currently supported only by an arguably unfair FP16 baseline and internally inconsistent accuracy statements. The authors should be asked to either rerun with a strong FP16 baseline or substantially narrow their claim, and to report variance and exact numerical configurations. I do not see grounds for rejection, since the issues are fixable with additional experiments and careful reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Cheetah is a genuine engineering contribution, but the headline result—16-bit posits beat 16-bit floats for DNN training—is not supported as stated. The paper is the first in its cited set to compare posit and float for training at equal bit-width, and the framework is useful: it supports multiple formats, quantization modes, and a parameterized FPGA exact-MAC (EMAC) with a quire, and it reports EDP/latency tradeoffs. The same-bit-width comparison and the quantization-approach comparison are the right way to ask the question, and the small-dataset inference tables (Tables III/IV) are consistent with earlier Deep Positron work showing posits help at 5–8 bits.\n\nThe soft spots are mostly about the training comparison. Table V's FP16 baseline is bare-bones: no loss scaling, no FP32 master-weight copy, no stochastic rounding, and no mixed-precision accumulation. That is why Float-16 lands ~7.4 points below Float-32 on both datasets, far worse than published FP16 training (e.g., Micikevicius et al.). Comparing Posit-16 to that baseline is not a fair test; the posit advantage could be mostly dynamic range. The paper's own text says 16-bit posits show 'less than 1% accuracy degradation,' but Table V shows Posit-16 is 1.6 points below Posit-32 on MNIST and 1.86 on Fashion-MNIST. That is an internal inconsistency and should be fixed. Also, Table V omits the es configuration for Posit-16, reports no variance despite ten runs, and does not give error bars.\n\nThe inference and hardware sections are in better shape. Choosing the best exponent width per format is a defensible design-space selection, but it should be labeled as such; it is not a head-to-head format comparison across fixed formats. The hardware results are only for a single EMAC soft core, not a full accelerator, and no synthesis artifacts are provided, so the EDP numbers can't be independently verified. These are addressable.\n\nFor whom does this paper work? People designing low-precision inference hardware on FPGAs and people tracking posit arithmetic. The training claim, if it survives a fair baseline, would be significant for on-device learning, but right now it doesn't.\n\nRecommendation: send it out for review, not desk reject. It deserves referee time, but the referees should ask for a proper FP16 mixed-precision baseline, the missing es and variance, and ideally code or synthesis projects. I wouldn't cite the training claim as it stands.","headline":"Useful engineering framework and the first same-bit-width posit-vs-float training comparison, but the headline training claim rests on a vanilla FP16 baseline and an internal inconsistency that need fixing before I would trust it.","tokens_in":19064,"tokens_out":3121,"would_cite":false,"duration_ms":34239,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The Cheetah framework shows 16-bit posits train small DNNs better than 16-bit floats, and 5–8-bit posits win the accuracy-versus-energy tradeoff for edge inference.","keywords":["posit arithmetic","low-precision training","edge inference","energy-delay product","quantization","exact multiply-accumulate","mixed precision","deep neural networks"],"falsifier":"Retrain the same 4-layer feedforward networks on MNIST and Fashion-MNIST with a standard mixed-precision FP16 schedule (32-bit weight updates, loss scaling, or stochastic rounding) and compare test accuracy with the 16-bit posit runs; if FP16 then reaches within about 0.1–0.5 percentage points of 32-bit float while 16-bit posit stays about 6 points lower, the training-advantage claim would be falsified.","tokens_in":18038,"feed_emoji":"🧮","tokens_out":5876,"duration_ms":59255,"temperature":0.7,"pith_summary":"Cheetah is a hardware/software co-design framework for low-precision deep networks aimed at edge devices. It emulates training and inference in three numerical formats—posit, floating point, and fixed-point—under two quantization schemes, and pairs them with an exact multiply-accumulate (EMAC) soft core to measure energy-delay product. The paper's central claim is that the posit format is not only competitive but superior at low precision: 16-bit posits train small feedforward networks more accurately than 16-bit floats, and 5–8-bit posits give better inference accuracy per energy-delay cost than floating point or fixed-point. A sympathetic reader would care because, if true, edge hardware could run smaller and cheaper arithmetic without sacrificing accuracy.","feed_headline":"16-bit posits beat 16-bit floats for DNN training","feed_subtitle":"Cheetah also shows 5–8-bit posits win the accuracy-versus-energy tradeoff on edge inference.","key_machinery":"The central object is the posit number format, a Type III unum whose tapered precision allocates more fraction bits to values near 1, with a regime field providing wide dynamic range. The argument is carried by pairing this format with the EMAC—an exact multiply-accumulate soft core that multiplies posits without intermediate rounding and accumulates into a wide fixed-point register (the quire) before a single convergent rounding step. Cheetah combines these with rounding or linear quantization and searches over bit-width and exponent/regime settings to find the point where accuracy and energy-delay product meet a design constraint.","core_discovery":"The central claim is that posits outperform IEEE-style floating point and fixed-point at matched low bit widths for both training and inference on small image-classification tasks. In the paper's experiments, 16-bit posit training reaches 96.535% and 87.400% on MNIST and Fashion-MNIST, compared with 90.646% and 81.725% for 16-bit float; at 5 bits, posit inference beats float by 28.5 percentage points and fixed-point by 31.62 on CIFAR-10. The authors attribute this to two mechanisms: posit's tapered precision concentrates representation accuracy near 1, matching DNN weight and activation distributions, and the EMAC core defers rounding until after all products are accumulated.","pith_inferences":["The 16-bit posit training advantage may depend on the comparison margin: a stronger FP16 baseline using 32-bit accumulation and loss scaling could close the gap, and testing that baseline on the same networks would separate format benefit from baseline weakness.","The energy-delay results are measured on a Virtex-7 FPGA soft core; an ASIC implementation could shift the comparison, especially because posit encode/decode is resource-heavy while avoiding subnormals saves latency.","Because posit's tapered precision is matched to weight distributions in small networks, deeper networks with wider dynamic ranges (such as ImageNet-scale CNNs) may need more exponent bits, and the framework's search over es is a natural way to test this."],"forward_implications":["Small edge DNNs can be trained end-to-end at 16-bit posit precision with simple round-to-nearest arithmetic, without loss scaling or stochastic rounding, and still land within about 1% of 32-bit float accuracy.","Inference at 5–8 bits with posits, especially with linear quantization and exponent width es=0 or 1, keeps accuracy close to full precision while reducing EMAC energy-delay product.","Linear quantization with shift lets designers replace general multipliers with constant-power-of-two scalings, lowering hardware cost and enabling power-of-two scaling.","Cheetah-style search over bit-width and posit exponent settings can pick a configuration from a target accuracy and energy-delay constraint rather than relying on ad hoc choices."],"supporting_citations":[{"why":"Defines the posit format and its tapered precision property that the paper uses to explain accuracy gains.","marker":"[25]"},{"why":"Proposes the posit exact multiply-accumulate (EMAC) core that Cheetah adopts for hardware evaluation.","marker":"[20]"},{"why":"Establishes the performance-efficiency trade-off methodology for low-precision DNN formats that Cheetah extends to training.","marker":"[21]"},{"why":"Describes the mixed-precision FP16 training recipe (32-bit weight updates, loss scaling) against which Cheetah's 16-bit posit training is implicitly compared.","marker":"[34]"},{"why":"Prior 8-bit float training work that Cheetah cites as requiring stochastic rounding or parameter analysis, in contrast to its simple round-to-nearest posits.","marker":"[14]"},{"why":"Earlier study of posit for DNN weight storage that this paper extends to full training and compute.","marker":"[19]"},{"why":"Supplies the Kulisch accumulator concept used by the EMAC to defer rounding until after accumulation.","marker":"[42]"}],"fun_headline_variants":["16-bit posit training beats float on small DNNs","5-8 bit posit inference wins on accuracy and energy","Cheetah: posits outperform floats and fixed-point at low bits","Posit DNNs: better training and inference than IEEE formats","Low-precision posits edge out float and fixed-point for DNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the FP16 baseline in Table V is a fair comparator for 16-bit training; it uses no mixed-precision accumulation, loss scaling, or stochastic rounding, so if a stronger FP16 recipe were used the claimed posit advantage could shrink.","fun_headline_variants_meta":{"raw":{"variants":["16-bit posit training beats float on small DNNs","5-8 bit posit inference wins on accuracy and energy","Cheetah: posits outperform floats and fixed-point at low bits","Posit DNNs: better training and inference than IEEE formats","Low-precision posits edge out float and fixed-point for DNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000318,"raw_usage":{"total_tokens":1763,"prompt_tokens":880,"completion_tokens":883,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":793}},"tokens_in":496,"tokens_out":883,"duration_ms":9408,"temperature":1.0,"reasoning_tokens":793,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:45:49.054305+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same 4-layer feedforward networks on MNIST and Fashion-MNIST with a standard mixed-precision FP16 schedule (32-bit weight updates, loss scaling, or stochastic rounding) and compare test accuracy with the 16-bit posit runs; if FP16 then reaches within about 0.1–0.5 percentage points of 32-bit float while 16-bit posit stays about 6 points lower, the training-advantage claim would be falsified.","supporting_citations":[{"cited_title":"Beating floating point at its own game: Posit arithmetic,","cited_arxiv_id":null,"evidence_quote":"Defines the posit format and its tapered precision property that the paper uses to explain accuracy gains."},{"cited_title":"Performance-efficiency trade-off of low-precision numerical formats in deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Establishes the performance-efficiency trade-off methodology for low-precision DNN formats that Cheetah extends to training."},{"cited_title":"Mixed precision training,","cited_arxiv_id":null,"evidence_quote":"Describes the mixed-precision FP16 training recipe (32-bit weight updates, loss scaling) against which Cheetah's 16-bit posit training is implicitly compared."},{"cited_title":"Training deep neural networks with 8-bit floating point numbers,","cited_arxiv_id":null,"evidence_quote":"Prior 8-bit float training work that Cheetah cites as requiring stochastic rounding or parameter analysis, in contrast to its simple round-to-nearest posits."},{"cited_title":"Deep learning infer- ence on embedded devices: Fixed-point vs posit,","cited_arxiv_id":null,"evidence_quote":"Earlier study of posit for DNN weight storage that this paper extends to full training and compute."},{"cited_title":"Kulisch, Computer arithmetic and validity: theory, implementation, and applications , 1st ed., ser","cited_arxiv_id":null,"evidence_quote":"Supplies the Kulisch accumulator concept used by the EMAC to defer rounding until after accumulation."}],"review_version":1}