{"id":"343dad20-3aac-49c9-ab58-aa2ce33d03ab","arxiv_id":"2510.10676","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"4-bit quantization of NLLB-200 plus a custom FPGA accelerator (NLPE) yields claimed 4.1x smaller and 4.2x faster multilingual translation, but translation quality is never measured.","lead":"This paper compresses the NLLB-200 translation model to 4-bit precision and pairs it with a custom FPGA compute engine, claiming 4.1x smaller models, 4.2x faster inference, and lower resource use than prior accelerators. It is pitched as a low-cost multilingual translator for Indian languages, but it reports no measured translation-quality metric.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No translation-quality metric (BLEU/chrF/COMET) is reported, so the central claim that FP4 NLLB-200 preserves translation capability is unsupported; the 0.56 GB FP4 footprint also conflicts with ~0.3 GB for 600M params at 4 bits.","rationale":"I agree with the reader's REJECT verdict, but I see the absence of any translation-quality metric as the single most load-bearing concern, slightly ahead of the cross-platform hardware comparison. The abstract and conclusion explicitly promise that FP4 quantization gives large size and speed gains 'without compromising translation capability'; that clause is a factual assertion about output quality, and the paper contains no measurement of it anywhere—not even a single BLEU number. The text itself admits this: Section III says 'satisfactory results' and Section III/IV says a comprehensive error analysis 'would highlight' issues, acknowledging one was not performed. If the FP4 model produces degraded or unusable translations, even entirely correct size/speed numbers do not deliver a deployable translation system. The hardware comparison (Table IV) is also problematic—different FPGAs, models, precisions, flows, and a missing OPU GOPS entry—but that is a criticism of the efficiency claims; the quality gap cuts at the usefulness of the entire system. A standard BLEU/chrF evaluation is cheap, concrete, and directly settles the point. The FP4 footprint arithmetic (0.56 GB vs. 0.3 GB expected) is a second, easy-to-check inconsistency that further increases correctness risk. Therefore the reader's REJECT stands; a revision with accuracy tables, reproducible benchmarks, and corrected size accounting could be re-reviewable on the merits.","tokens_in":14486,"tokens_out":6612,"duration_ms":58705,"concrete_test":"Use the released GitHub code to evaluate the FP4/INT4 NLLB-200 600M against the original FP32/BF16 model on FLORES-200 for the claimed Indic-to-English/English-to-Indic pairs, reporting BLEU/chrF (and COMET if feasible), with a pre-registered non-inferiority margin (e.g., within 2 BLEU points). This directly settles whether 'without compromising translation capability' holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The manuscript's central promise—that FP4/INT4 quantization of NLLB-200 600M yields a 4.1x smaller footprint and 4.2x faster inference 'without compromising translation capability' (Abstract; Section IV)—has no supporting quality measurement anywhere. Section III reports only 'satisfactory results' on 1000 queries per language; no BLEU, chrF, COMET, or human evaluation appears. Figure 10 plots only size, time, and tokens/s, not translation quality. The conclusion asserts 'without compromising translation capability' with no accuracy data. The paper even concedes at the end of Section III: 'A comprehensive error analysis would highlight the exact translation performance...' — i.e., the analysis is absent. Since a translation system whose output quality is unmeasured cannot be said to preserve capability, the headline software result is not established even if the RTL and quantization pipeline are real. Compounding this, the 0.56 GB FP4 footprint is arithmetically inconsistent with 600M parameters at 4 bits (about 0.3 GB), suggesting the size-reduction claim needs re-examination (e.g., actual per-tensor bit widths or inclusion of non-weight data).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Bhasha-Rupantarika, an algorithm-hardware co-design flow for multilingual NMT on edge FPGAs. The software side applies BitsAndBytes-based post-training quantization and QLoRA fine-tuning to a distilled NLLB-200 600M model, reporting a 0.56 GB FP4 footprint, 4.1x model-size reduction, 4.2x inference speedup, and 66 tokens/s throughput. The hardware side proposes the NLPE accelerator with an RMMEC-based SIMD MAC unit and a CORDIC-based FASST activation vector array, synthesized on a ZCU104, with claimed 1.96x LUT and 1.65x FF reductions and 2.2x/4.6x throughput gains over OPU/HPTA. The stated goal is to enable low-resource multilingual translation, particularly between Indian and international languages.","tokens_in":14761,"tokens_out":6770,"duration_ms":58301,"significance":"The problem is relevant and timely: quantized multilingual NMT with a custom accelerator could be useful for edge deployment in resource-limited regions. Strengths include the public code/dataset link, the RTL implementation in SystemVerilog, the use of an iso-functional emulator for co-verification, and the comparison against an FP32 baseline for the quantization study. However, the central claims are not currently established: translation quality is never measured, the FP4 footprint number is internally inconsistent, and the hardware efficiency claims rest on non-apples-to-apples comparisons. If these issues are corrected, the work could make a meaningful contribution; as submitted, the evidence does not support the conclusions.","major_comments":[{"comment":"No translation-quality metric is reported. Section III states only that PTQ was evaluated on 1000 queries per language and produced 'satisfactory results'; no BLEU, chrF, COMET, or human evaluation is provided, and Figure 10 plots only model size, translation time, and tokens/s. The Conclusion then asserts 'without compromising translation capability.' The paper itself concedes in Section III that 'a comprehensive error analysis would highlight the exact translation performance...', acknowledging the absence. Since the headline 4.1x size reduction and 4.2x speedup are only meaningful if output quality is preserved, this missing evidence is load-bearing. The authors should add quantitative quality metrics for the FP32 baseline and each quantized precision, ideally broken down by language pair.","section":"Section III / Abstract / Conclusion"},{"comment":"The reported FP4 footprint of 0.56 GB is arithmetically inconsistent with a 600M-parameter model at 4 bits, which should occupy roughly 0.3 GB; 0.56 GB implies about 7.5 bits per parameter. If the number includes block-wise scale/zero-point overhead, embedding tables, or non-weight tensors, that breakdown must be provided. As written, the 'FP4' label and the 4.1x model-size reduction claim are not self-consistent and require either correction or a clear accounting of the extra storage.","section":"Section III, Fig. 10"},{"comment":"The headline hardware improvements are derived from cross-platform, cross-model comparisons. NLPE is evaluated for NLLB on a ZCU104, while Table IV compares it with NPE (BERT on Zynq Z-7100), HPTA (Swin-T on ZCU102), Q-BERT (MNLI on ZCU102), TPDS'22 (NMT on VCU118), and others. These differ in board, model, precision, and synthesis flow, so a smaller LUT count or higher GOPS figure can reflect different workload sizes rather than design superiority. The claimed 1.96x LUT reduction, 1.65x FF reduction, and 2.2x/4.6x throughput gains are therefore not established. The authors should either compare on the same model/FPGA or provide a principled normalization that accounts for model size, precision, and hardware resources.","section":"Section III, Table IV"},{"comment":"The paper does not demonstrate end-to-end execution of NLLB on the NLPE. Section III describes synthesis and verification of the MAC and NAF units and reports a board-level resource table, but Figure 10's tokens/s appears to come from a software/GPU inference run, and the 684.48 GOPS entry in Table IV is unexplained. No full-transformer mapping, memory-bandwidth analysis, or measured end-to-end latency for the NLLB model on the FPGA is provided. The claim that the accelerator 'deploys' the quantized NLLB model is therefore not substantiated. Please separate unit-level results from system-level projections and provide an actual end-to-end FPGA demonstration or a carefully validated performance model.","section":"Section III / Section II-B"}],"minor_comments":[{"comment":"The text alternates between 'FP4' and 'INT4' for the distilled NLLB variant, and BitsAndBytes 4-bit quantization is typically NF4 rather than FP4. Clarify the exact format used and use consistent terminology throughout.","section":"Section II-A"},{"comment":"The caption refers to 'NVU' while the text refers to 'NMV'; unify the naming for the nonlinear vector array.","section":"Fig. 7"},{"comment":"The column heading 'Throughput GOPS' is ambiguous; some rows have a dash for BRAM/throughput. Add units, define GOPS, and explain how the 684.48 GOPS figure is derived for the NLPE.","section":"Table IV"},{"comment":"Citation numbering is inconsistent, e.g., OPU is cited as both [8] and [21] in the same paragraph. Check all references for duplicate or mismatched numbering.","section":"References"},{"comment":"The text says 'Python 3.0,' which is likely outdated; use 'Python 3.x.' Also, the role of Qkeras in a BitsAndBytes/QLoRA pipeline is unclear and should be explained or removed.","section":"Section III"},{"comment":"The figure lacks a legend or color labels; it is difficult to tell which series corresponds to model size, translation time, and tokens per second. Add axis clarity and a legend.","section":"Fig. 10"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has substantial strengths in the hardware design and public reproducibility artifacts, but the missing translation-quality metric and the unfair cross-platform comparison are serious. I believe these are addressable with additional experiments and a careful rewrite, so I am not recommending rejection. The heavy reliance on the authors' own prior works (Refs. 31, 36, 41) in the comparisons and the self-admitted absence of error analysis should be carefully checked by the editor. The paper may fit an architecture/systems venue rather than a general ML venue, depending on the corrected scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zubair, quick read of the Bhasha-Rupantarika submission. The core idea is a real one: take the distilled NLLB-200 600M model, quantize it to 4-bit, and run it on a custom FPGA engine. That integrated package - quantized model plus the NLPE accelerator with the RMMEC multiplier blocks and CORDIC-based activation array - is new. The RTL is concrete, they report synthesis numbers, and the code/data are supposedly public. That is enough to take the paper seriously.\n\nWhat it does not do is validate the thing that matters: translation quality. The abstract claims FP4 quantization works 'without compromising translation capability,' but the paper contains no BLEU, chrF, COMET, or human evaluation. Section III says 'satisfactory results' on 1000 queries per language, and Figure 10 only plots size and throughput. The conclusion even concedes 'a comprehensive error analysis would highlight the exact translation performance' - which is another way of saying the analysis is missing. For a translation paper, that is not a minor omission; it is the load-bearing wall.\n\nThere is also an arithmetic red flag: 600M parameters at 4 bits is about 0.3 GB, yet the paper reports a 0.56 GB FP4 footprint. The multiplier is off by almost 2x, and the paper never explains what else is in that number. It could be embeddings, but the claim requires a breakdown.\n\nThe hardware story is mixed. The component-level MAC/NAF comparisons in Tables II and III look reasonably controlled. But the system-level Table IV compares NLPE on a ZCU104 running NLLB against OPU on a Z-7100 running BERT, HPTA on a ZCU102 running Swin-T, and so on. Different boards, different models, different precisions, different synthesis flows. A smaller LUT count under those conditions can just mean a smaller workload. The 2.2x/4.6x throughput gains do not survive that comparison.\n\nWho is this for? Anyone working on edge MT accelerators will find the RTL details and the sub-octet quantization data interesting, but they should treat the system-level claims as unverified. I would send it to peer review if only to force the authors to add accuracy numbers and fix the arithmetic, but I would not accept it in its current form.","headline":"Concrete FPGA accelerator for 4-bit NLLB-200, but missing translation-quality metrics and uneven hardware comparisons undermine the central claims.","tokens_in":15395,"tokens_out":3763,"would_cite":false,"duration_ms":30674,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FP4 quantization of a distilled 200-language translation model, paired with a custom FPGA accelerator, delivers a 4.1x smaller footprint, 4.2x faster inference, and 66 tokens/s throughput — without losing translation quality, according to t","keywords":["multilingual neural machine translation","model quantization","FP4 precision","FPGA accelerator","algorithm-hardware co-design","NLLB-200","Indic languages","transformer inference"],"falsifier":"Run the proposed NLPE and the two prior accelerators from the paper's comparison on the same FPGA board with the same NLLB-200 INT4/FP4 workload and the same synthesis flow, and see whether the 1.96x LUT reduction and 2.2x/4.6x throughput gains persist. Separately, compute BLEU or COMET scores for FP4 versus FP32 on the same query set to test the 'without compromising translation capability' claim.","tokens_in":14237,"feed_emoji":"🌐","tokens_out":8141,"duration_ms":68951,"temperature":0.7,"pith_summary":"Bhasha-Rupantarika is an algorithm-hardware co-design for running multilingual translation on resource-limited devices. The paper claims that a distilled 200-language translation model, quantized to FP4, drops to 0.56 GB (4.1x smaller than FP32), infers 4.2x faster, and reaches 66 tokens/s (a 4.8x improvement), all without compromising translation ability. To back that up, it introduces an FPGA accelerator (NLPE) with a mixed-precision SIMD multiply-accumulate array and a CORDIC-based activation unit; on a ZCU104 board the design uses 1.96x fewer LUTs and 1.65x fewer FFs than prior NLP accelerators, with 2.2x and 4.6x higher throughput. If these results hold, a single bidirectional model can translate between Indian and international languages on IoT-class hardware, a step toward real-time translation in rural areas.","feed_headline":"FP4 translator hits 66 tokens/s on a small FPGA accelerator","feed_subtitle":"A 0.56 GB multilingual model plus co-designed hardware targets real-time Indic-to-world translation on IoT-class devices.","key_machinery":"The load-bearing mechanism is the NLPE accelerator, built around two reusable units: a SIMD multiply-accumulate engine whose 4-bit RMMEC blocks can be configured as multipliers or exponent comparators, allowing INT4, FP4, FP8, and BF16 operations in the same datapath; and a CORDIC-based FASST vector array that delivers SoftMax, sigmoid, tanh, and related nonlinearities at FP8/BF16 precision. An output-stationary systolic MAC array keeps partial sums in place while weights and activations stream through, and all units are pipelined to overlap computation with memory traffic. On the algorithm side, the claim rests on quantizing a distilled encoder-decoder transformer (NLLB-200, 600M parameters","core_discovery":"The paper's central claim is that sub-octet quantization is what makes a 200-language neural translation model deployable on edge hardware, and that a co-designed accelerator lets the savings appear in real silicon. Concretely, the distilled NLLB-200 model (a 200-language transformer), post-training quantized to 4-bit floating point with small trainable adapters, keeps translations usable while cutting the footprint to 0.56 GB and enabling 66 tokens/s on the proposed NLPE engine. The accelerator's SIMD MAC array uses reconfigurable 4-bit RMMEC blocks that switch between multiplier and exponent-comparator roles, and its CORDIC-based FASST unit computes SoftMax, sigmoid, tanh, and related nonl","pith_inferences":["Because the reported hardware gains compare a small translation model on one board against larger models on other boards, a controlled replay on identical workloads is needed before treating the resource ratios as general.","If independent translation-quality scores confirm the FP4 result, 4-bit floating point could become a default edge precision for multilingual sequence-to-sequence models, extending beyond Indian languages.","The paper's pipeline sketches speech-to-text and text-to-speech around the translator; an integrated voice-to-voice system on the same FPGA is the natural next experiment, with end-to-end latency as the success metric.","The authors' scaling hypothesis suggests testing the same RMMEC/FASST datapath at larger model sizes; a positive result would extend the co-design from edge translation to cloud-scale chatbots."],"forward_implications":["A 200-language translation model can fit in 0.56 GB, small enough for edge and IoT storage budgets.","At 66 tokens/s, interactive translation on a low-power FPGA becomes realistic rather than batch-only.","A single bidirectional model replaces separate Indian-to-English, English-to-foreign, and reverse models, cutting total model count and memory overhead.","The same accelerator datapath can be reconfigured across precisions and activation types, so future transformer variants can deploy without redesigning the MAC or nonlinear units.","The reported resource savings leave headroom on the same FPGA class for larger or additional workloads."],"fun_headline_variants":["4-bit translator: 4.1x smaller, 4.8x faster on edge FPGA","200-language FP4 model fits in 0.56 GB for edge deployment","Co-design cuts NMT size 4.1x, speeds inference 4.2x on FPGA","FP4 quantization: 66 tokens/s multilingual NMT on IoT FPGA","Edge NMT: sub-octet quantization shrinks model, boosts speed"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the hardware comparison is fair: the new design is measured on one board and model while the compared accelerators use different boards and models, so the claimed resource and throughput advantages could partly reflect an easier workload rather than a better design.","fun_headline_variants_meta":{"raw":{"variants":["4-bit translator: 4.1x smaller, 4.8x faster on edge FPGA","200-language FP4 model fits in 0.56 GB for edge deployment","Co-design cuts NMT size 4.1x, speeds inference 4.2x on FPGA","FP4 quantization: 66 tokens/s multilingual NMT on IoT FPGA","Edge NMT: sub-octet quantization shrinks model, boosts speed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000838,"raw_usage":{"total_tokens":3519,"prompt_tokens":804,"completion_tokens":2715,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":2604}},"tokens_in":548,"tokens_out":2715,"duration_ms":17139,"temperature":1.0,"reasoning_tokens":2604,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T10:16:11.999538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the proposed NLPE and the two prior accelerators from the paper's comparison on the same FPGA board with the same NLLB-200 INT4/FP4 workload and the same synthesis flow, and see whether the 1.96x LUT reduction and 2.2x/4.6x throughput gains persist. Separately, compute BLEU or COMET scores for FP4 versus FP32 on the same query set to test the 'without compromising translation capability' claim.","supporting_citations":[],"review_version":1}