{"id":"58097761-f806-4924-ae55-d52d0a5742f6","arxiv_id":"2505.09407","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A hybrid quantum-classical encoder-decoder is reported to translate four languages with 82% accuracy, but the paper's evaluation is too unreliable to support the claim.","lead":"This paper presents QEDACVC, a quantum encoder-decoder architecture for multilingual translation that combines quantum convolution, pooling, attention, and variational circuits, trained on a small OPUS subset for English, French, German, and Hindi. The paper claims 82% accuracy and high BLEU scores, which would be a striking result if the evidence and comparisons were credible.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) counts wrong predictions as true negatives, so the reported accuracies are nonstandard and the 82% headline is unsupported; the central empirical claim depends entirely on this metric.","rationale":"The reader's weakest assumption identifies the same load-bearing problem: Section 4's accuracy metric is nonstandard and the BLEU numbers are implausible. My reading confirms this is not merely a presentation issue. Eq. (3) explicitly labels incorrect model predictions as true negatives, so the reported accuracies are not comparable to standard classification accuracy and cannot support the claimed comparison with classical models. The paper also contains internal inconsistencies (Algorithm Step 5 says 100 epochs and lr 1e-5; Section 4 says 50 epochs and lr 2e-7) and provides no code or data, so the numbers in Table 3 are the sole evidence for the central claim. Under the standard definition of true negatives, the metric is not a valid accuracy; under a corrected metric there is no evidence the reported values survive. Therefore the reader's REJECT verdict is appropriate and my stress-test does not change it. I do not see a reason to soften the verdict based on the manuscript text; the nonstandard metric and implausible BLEU scores are sufficient and load-bearing. The concrete check I propose would settle the question definitively by re-evaluating on the same data with a conventional metric and a trivial baseline sanity check.","tokens_in":9423,"tokens_out":4805,"duration_ms":50437,"concrete_test":"Request the released code, exact OPUS subsets, and decoding script, then recompute the Section 4 evaluation using standard per-token accuracy (correct predicted tokens / total reference tokens) and sacreBLEU. Also run Eq. (3) on a baseline that predicts a random token at each position: a valid accuracy metric should give chance-level performance (~1/V), not the ~0.25 this definition yields. If the English accuracy drops materially or the baseline is not at chance, the headline claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive weakness is in Section 4, Eq. (3). The paper defines true negative (β) as \"Number of incorrect responses to predicted tokens\"; in standard classification a true negative is a correct rejection, not an error. Counting model errors as successes makes the accuracy numerator depend on the mistakes themselves, and the definition is not meaningful for sequence-to-sequence translation, where there is no well-defined set of \"negative tokens\" per output position. Table 3 then compares these Eq. (3) accuracies (81.8/71.9/72.8/58.5) with GRU/LSTM/BERT/GPT/T5 accuracies, but unless the baselines used the identical nonstandard definition the comparison is apples-to-oranges. The abstract's headline \"82%\" is only the English row, and the BLEU scores (89.2 English, 80.1 German, 79.1 French, 81.9 Hindi) are far above typical high-quality MT scores and cannot be checked because no code, data split, or decoding details are provided. The conclusion that QEDACVC \"performs comparatively well\" and \"defies parameter scaling\" rests entirely on these numbers, so the metric flaw is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes QEDACVC, a quantum encoder-decoder architecture that combines quantum convolutional, pooling, attention, and variational circuit components, and applies it to multilingual machine translation on English, German, French, and Hindi subsets of the OPUS corpus. The authors report 82% accuracy for English and BLEU scores of 89.2 (English), 80.1 (German), 79.1 (French), and 81.9 (Hindi) with a model of 550 total parameters, and compare these results against GRU, LSTM, BERT, GPT, and T5 baselines. They also present an ablation study that varies quantum convolution and attention components. The central claim is that QEDACVC performs comparatively well against classical state-of-the-art models while using far fewer parameters.","tokens_in":9642,"tokens_out":4863,"duration_ms":48672,"significance":"If the reported results were valid, a 550-parameter quantum model achieving near-state-of-the-art multilingual translation would be a striking contribution to quantum natural language processing, especially given the scarcity of work on quantum machine translation. The paper also demonstrates a genuine attempt to assemble quantum circuit components for encoder-decoder translation and includes an ablation study that identifies the contribution of individual components. However, the evaluation is deeply flawed: the accuracy metric is nonstandard to the point of being meaningless, the BLEU scores are implausible, the experimental configuration is internally inconsistent, and the baselines are not described. These issues are load-bearing, because the paper's headline accuracy and all comparative conclusions rest on this evaluation. The manuscript would need a complete re-run of experiments with standard metrics, reproducible code, and a rigorously described baseline protocol before its claims could be assessed.","major_comments":[{"comment":"The accuracy metric in Eq. (3) is invalid. The paper defines true negative (β) as \"Number of incorrect responses to predicted tokens\"; in standard classification, a true negative is a correct rejection, not an error. Counting model errors as successes inflates the accuracy numerator by the model's own mistakes and makes higher accuracy partially a reward for incorrect outputs. Moreover, sequence-to-sequence translation has no well-defined set of negative tokens at each output position, so the four-way confusion matrix cannot be applied meaningfully. Because Table 3 uses these accuracies to compare QEDACVC against classical baselines without stating that the baselines used the identical nonstandard definition, all accuracy comparisons in the paper are invalid. This is the core evaluation claim and cannot be fixed by textual revision; the experiments must be re-run with a standard metric.","section":"§4, Eq. (3)"},{"comment":"The reported BLEU scores—89.2 for English, 80.1 for German, 79.1 for French, and 81.9 for Hindi—are far above typical results in machine translation, especially for a model trained on only 10,000 sentence pairs and containing only 550 parameters. The paper provides no code, no exact data split, no tokenization details, no beam-search width or decoding strategy, and no description of the BLEU implementation used. Without these, the BLEU numbers are unreproducible and not credible. The conclusion that QEDACVC \"performs comparatively well\" against BERT, GPT, and T5 depends entirely on these unverifiable scores.","section":"§4, Table 3 and Fig. 11"},{"comment":"The hyperparameters are stated inconsistently. Algorithm 1, Step 5 specifies dropout rate 0.02, epochs 100, learning rate 1e-5, batch size 8, and sequence length 64; Section 4, Training states dropout rate 0.02, epochs 50, learning rate 2e-7, batch size 8, and maximum sequence length 64. These discrepancies mean that the reported results are not tied to a single, reproducible configuration and undermine the validity of every training curve and evaluation number in the paper.","section":"§3, Algorithm 1, Step 5 vs. §4, Training"},{"comment":"The validation-loss column contains repeated identical values across different language configurations at the same epoch (for example, 0.215 at epoch 10 for English and Hindi, 0.167 at epoch 15, and 0.157 at epoch 50). Such exact repetitions are implausible for independent training runs on different languages and suggest that the reported values may be corrupted or copied. Additionally, the training narrative claims stabilization around epochs 32, 32, 43, and 35 for English, German, French, and Hindi, respectively, but Table 2 lists only epochs in increments of five, so these claimed stabilization points do not correspond to any logged row, making the training description internally inconsistent.","section":"Table 2"},{"comment":"The paper does not describe how the GRU, LSTM, BERT, GPT, and T5 baselines were implemented, trained, or evaluated. There is no information about their data splits, hyperparameters, training setup, tokenization, decoding, or which implementation of the accuracy and BLEU metrics was used. Without this information, the comparison in Table 3 and Fig. 11 is not meaningful; it is unclear whether the baselines were evaluated under conditions even roughly comparable to QEDACVC or whether they used the same nonstandard accuracy definition.","section":"§4, baselines"},{"comment":"The abstract and the highlights claim an accuracy of 82%, but Table 3 shows that this value applies only to English; the corresponding accuracies are 71.9% for German, 72.8% for French, and 58.5% for Hindi. Reporting only the best language as the headline 'multilingual translation' accuracy is misleading and does not represent a multilingual average. The paper should state per-language results and an aggregate measure if one is intended.","section":"Abstract and §1"}],"minor_comments":[{"comment":"Reference [22] is cited in the text as \"Myles Doyle et al. (2020)\" but the reference list entry for [22] is \"Mina A, Vahid S, Seyed M, et al., Application of Quantum Natural Language Processing for Language Translation,\" which is an Abbaszade et al. work with different authors and year. The citation is mismatched and needs correction.","section":"§2 and §8"},{"comment":"The text dates Google Neural Machine Translation System as (2022), but the reference list entry is the 2016 arXiv paper (arXiv:1609.08144). The year should be corrected.","section":"§2, reference [24]"},{"comment":"The sample inferences in Fig. 10(a-c) are labeled with language pairs, but the figure does not show source sentences, reference translations, or the target-language output in a way that allows assessment. The captions should make clear what is being translated and what the expected output is.","section":"§4, Fig. 10"},{"comment":"Equation (2) defines attention with \"M\" as the softmax activation, but the description says \"M, A is the ask, B is the key, and C is the value,\" where \"ask\" appears to be a typo for \"query.\" The notation should be clarified and the typo fixed.","section":"§4, Eq. (2) and surrounding text"},{"comment":"Table 5 is titled \"Model Vs. Parameter-Based Performance Impact\" and the column is called \"Parameter Optimization Mode,\" but the rows O1-O5 describe ablated model configurations, not parameters being optimized. Renaming the column to \"Configuration\" would better match the content.","section":"§5, Tables 5 and 6"},{"comment":"The sentence \"Training QEDACVC is trained for a zero-shot multilingual setup\" is unclear, because the model is explicitly trained on all four language pairs. The use of \"zero-shot\" does not match the standard terminology for multilingual machine translation.","section":"§4, Training"},{"comment":"No error bars, standard deviations, or repeated-run statistics are reported for any accuracy or BLEU number. Given the small batch size (8) and limited training data, reporting variance across seeds is necessary to support the claimed comparisons.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"I agree with the reader's assessment. The evaluation problems are not cosmetic: the accuracy definition in Eq. (3) is mathematically incoherent for translation, the BLEU scores are implausible and unreproducible, and the experimental configuration is internally inconsistent. These are load-bearing errors that would require a complete re-run of the experiments with standard metrics, code release, and properly described baselines. The manuscript's central claim cannot be salvaged by textual revision. The reference list also contains several citation mismatches that should be corrected in any future version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — a quick read of arXiv:2505.09407. What is actually new: the architecture, QEDACVC, is a concrete combination of quantum convolutional layers, pooling, attention, and a variational circuit in an encoder-decoder for multilingual machine translation, with only 550 trainable parameters. That combination is not in the earlier quantum-NLP work I know, and it is a reasonable thing to explore. The paper also makes a good-faith effort to survey related classical and quantum translation systems, and the limitations paragraph is candid about hardware and data constraints. I want to credit that.\n\nThe soft spot is not the idea; it is the evaluation. Eq. (3) in Section 4 defines a true negative as 'number of incorrect responses to predicted tokens.' That is backwards: a true negative should be a correct rejection. Under the paper's definition, model mistakes are added to the numerator, and for sequence-to-sequence output there is no meaningful 'negative token' per position. So the reported accuracy is not a standard classification accuracy, and the comparison in Table 3 with GRU/LSTM/BERT/GPT/T5 is apples-to-oranges unless those baselines used the same nonstandard definition—which is not stated. The headline 82% is also only the English row; German, French, and Hindi sit at 71.9, 72.8, and 58.5.\n\nThe BLEU numbers are the other red flag. 89.2 BLEU on a 10,000-pair training set is far beyond what even strong classical systems achieve on standard benchmarks. Unless there is a very small or near-duplicate test set, or a nonstandard tokenizer or decoding procedure, these numbers are not credible. No code, no exact OPUS subsets, no split details, and no decoding details are given, so the values cannot be checked. The hyperparameters are also inconsistent: Algorithm 1 says 100 epochs and learning rate 1e-5; Section 4 says 50 epochs and 2e-7. That kind of internal contradiction undermines trust. The baseline models are not described either—same data? same tokenizer? trained from scratch? fine-tuned?—and there are no error bars or multiple seeds.\n\nThe ablation study is a nice idea in principle, but it inherits the same broken accuracy metric, so it does not rescue the empirical story. The claim of 'defying parameter scaling' depends entirely on the validity of these metrics.\n\nWho is this for? Someone surveying quantum NLP components might skim the architecture section. But no one should rely on the reported results. If the authors supply code and data, correct the metric, run standard BLEU, and document baselines, this could become a useful paper. As submitted, I would not send it to referees; the load-bearing numbers are not defensible. Desk reject, with an invitation to resubmit after a complete re-evaluation.","headline":"A novel quantum encoder-decoder architecture buried under an unsound evaluation: the headline accuracy rests on a misdefined metric and implausible BLEU scores.","tokens_in":10153,"tokens_out":4161,"would_cite":false,"duration_ms":42560,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a 550-parameter quantum encoder-decoder, QEDACVC, performs multilingual translation on OPUS data at accuracy levels comparable to classical transformer models.","keywords":["Quantum Computing","Multilingual Machine Translation","Quantum Encoder-Decoder","Quantum Attention","Quantum Convolution","Variational Quantum Circuits","OPUS Corpus","BLEU Score"],"falsifier":"Take the trained QEDACVC checkpoints and re-score the same test predictions with standard token-level accuracy (only exact target matches count) and with a standard BLEU implementation against the reference sentences. If the true accuracy is near 20–40% rather than 82%, or the BLEU scores fall below typical neural MT numbers for a 10,000-pair training set, the central claim is falsified. Alternatively, replicate training from the OPUS subsets and check whether any model with 550 parameters reaches BLEU >80; a published small-data translation benchmark at that scale would contradict known data requirements.","tokens_in":9183,"feed_emoji":"⚛️","tokens_out":6657,"duration_ms":62208,"temperature":0.7,"pith_summary":"QEDACVC is proposed as a fully quantum alternative to the classical multilingual translation stack. The paper argues that an encoder-decoder built entirely from parameterized quantum circuits—convolution, pooling, attention, and a variational final layer—can translate among English, German, French, and Hindi using OPUS corpus subsets, with 550 total parameters and an accuracy of 82% plus BLEU scores in the 79–89 range. If this is right, it would show that the expensive attention-based neural machinery behind current translation services can be replaced by a tiny quantum circuit without losing translation quality. The paper also reports ablation results indicating that both the quantum convolution and quantum attention layers contribute to the accuracy gain. The headline result depends on the paper's accuracy definition, set out in Equation (3), which the author uses alongside BLEU as the evaluation protocol.","feed_headline":"A 550-parameter quantum circuit claims 82% translation accuracy","feed_subtitle":"Quantum convolution, pooling, attention, and variational layers translate four languages from 10,000 pairs.","key_machinery":"The load-bearing object is the 8-qubit quantum circuit stack assembled as an encoder-decoder. Quantum convolutional layers apply pairs of parameterized U3 gates with Ising interactions to extract features; quantum pooling layers reduce dimension by measuring half the qubits; a quantum attention circuit replaces the transformer's scaled dot-product attention; a quantum variational circuit—Hadamard, RY rotations, CNOT shifts, and Z measurement—serves as the decoder's fully connected head. These circuit blocks are the entire model: there is no classical transformer underneath, and the 550 trainable parameters all live in the rotation gates. The argument hangs on these circuits' ability to learn translation patterns from token streams, evaluated by the paper's accuracy formula and BLEU.","core_discovery":"Central claim: QEDACVC, a quantum encoder-decoder with attention-based convolutional variational circuits, is a viable multilingual machine translation architecture. The model encodes text through four pairs of quantum convolutional and pooling layers on an 8-qubit circuit, applies a quantum attention circuit, and decodes through quantum convolutional, pooling, and attention layers ending in a variational circuit that acts as a fully connected layer. Trained on 10,000 English–German–French–Hindi sentence pairs from OPUS, it is reported to reach 81.8% accuracy and 89.2 BLEU for English, 71.9/80.1 for German, 72.8/79.1 for French, and 58.5/81.9 for Hindi, outperforming GRU and LSTM by large margins and matching BERT, GPT, and T5 in the paper's comparison tables. The paper further claims that with 550 parameters and eight layers, the architecture is orders of magnitude smaller than classical baselines while maintaining comparable accuracy. Ablation runs O1–O5 show accuracy increasing monotonically as quantum convolution and quantum attention layers are added, which the paper reads as evidence that each quantum component carries real modeling weight.","pith_inferences":["A BLEU score above 80 on a 10,000-pair training set is far beyond what classical neural MT typically achieves; if the numbers are reproducible, they would suggest the evaluation compares against a very small or memorized test set, or that the BLEU implementation is not the standard reference-based metric.","Equation (3)'s true-negative convention likely inflates accuracy substantially: in a vocabulary of tens of thousands of tokens, most positions the model does not predict are 'incorrect responses to predicted tokens,' so a model predicting almost nothing could still score high.","A direct test of the quantum components' contribution would be to replace each quantum layer with a random or identity circuit; if accuracy stays high, the ablations measure parameter count rather than circuit structure.","If the framework scales, the natural next experiment is to compare QEDACVC against a classical transformer of similar parameter count on the same 10,000-pair split, which the paper does not report."],"forward_implications":["A quantum encoder-decoder with only 550 parameters can run multilingual translation for four languages at accuracy levels the paper reports as comparable to BERT, GPT, and T5.","Adding quantum convolution layers and a quantum attention layer each improves accuracy across all four languages in the ablation study.","The architecture's dependence on open quantum simulators and free quantum hardware suggests translation models of this kind can be trained without large compute clusters.","Because the loss in Equation (1) blends encoder and decoder token probabilities, the paper implies the quantum circuit stack can be trained end-to-end by gradient-based optimization.","The reported metric gap between English (81.8) and Hindi (58.5) is attributed to dataset constraints, implying that larger or better-balanced OPUS subsets would raise lower-resource-language performance."],"supporting_citations":[{"why":"Supplies the OPUS multilingual parallel corpus subsets used for training, validation, and testing.","marker":"[14]"},{"why":"Defines the attention mechanism that the quantum attention layer is designed to replace.","marker":"[9]"},{"why":"Introduces quanvolutional layers, the basis of the quantum convolution component.","marker":"[10]"},{"why":"Provides quantum pooling techniques used for the 50% dimensionality reduction.","marker":"[11]"},{"why":"Motivates the variational circuit layer's parameter optimization via quantum natural gradient.","marker":"[12]"},{"why":"Supplies the quantum self-attention circuit concept adapted into QEDACVC's attention layer.","marker":"[13]"},{"why":"Cited as the source of the accuracy metric in Equation (3) used for the headline 82%.","marker":"[28]"},{"why":"Defines the BLEU metric used for translation quality scores in Tables 2 and 3.","marker":"[29]"}],"fun_headline_variants":["550-parameter quantum circuit hits 82% translation accuracy","Quantum translation with just 550 parameters matches BERT, GPT","Tiny quantum model translates four languages at 82% accuracy","Quantum encoder-decoder rivals classical LLMs on translation","82% accuracy: quantum circuit beats GRU, LSTM in translation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the paper's accuracy formula, which treats incorrect responses to predicted tokens as true negatives, and its BLEU implementation measure translation quality in the standard way; if either is nonstandard, the headline results collapse.","fun_headline_variants_meta":{"raw":{"variants":["550-parameter quantum circuit hits 82% translation accuracy","Quantum translation with just 550 parameters matches BERT, GPT","Tiny quantum model translates four languages at 82% accuracy","Quantum encoder-decoder rivals classical LLMs on translation","82% accuracy: quantum circuit beats GRU, LSTM in translation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000756,"raw_usage":{"total_tokens":3381,"prompt_tokens":985,"completion_tokens":2396,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":2311}},"tokens_in":601,"tokens_out":2396,"duration_ms":16823,"temperature":1.0,"reasoning_tokens":2311,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:31:51.853579+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained QEDACVC checkpoints and re-score the same test predictions with standard token-level accuracy (only exact target matches count) and with a standard BLEU implementation against the reference sentences. If the true accuracy is near 20–40% rather than 82%, or the BLEU scores fall below typical neural MT numbers for a 10,000-pair training set, the central claim is falsified. Alternatively, replicate training from the OPUS subsets and check whether any model with 550 parameters reaches BLEU >80; a published small-data translation benchmark at that scale would contradict known data requirements.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the BLEU metric used for translation quality scores in Tables 2 and 3."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the OPUS multilingual parallel corpus subsets used for training, validation, and testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the attention mechanism that the quantum attention layer is designed to replace."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces quanvolutional layers, the basis of the quantum convolution component."},{"cited_title":", JAX: composable transformations of Python and NumPy programs (2018) http://github.com/jax-ml/jax","cited_arxiv_id":null,"evidence_quote":"Cited as the source of the accuracy metric in Equation (3) used for the headline 82%."}],"review_version":1}