{"id":"860ef76d-88e8-4ae0-9512-e19e40db7f0c","arxiv_id":"2508.21332","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A benchmark of QASA, QRWKV, and QKSAN against Transformer and MLP on five tiny datasets, with results that contradict the paper's own tables.","lead":"This paper compares three quantum-inspired text generation models against a Transformer and an MLP on five tiny toy datasets. It claims quantum models achieve better vocabulary diversity and zero repetition, but the reported numbers are internally inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported BLEU-1=1.000 is contradicted by the paper's own Table VIII: QRWKV's generated sentence shares at most one unigram with the reference, so the central competitive-BLEU claim cannot stand.","rationale":"The reader's weakest assumption—that the evaluation metrics were computed and reported correctly—is exactly where the paper breaks. I independently checked the two headline numbers. First, Table VIII's QRWKV output shares only one unigram with its reference, so the reported BLEU-1 of 1.000 in Table III is contradicted by the paper's own example; this is not a subtle normalization issue. Second, Distinct-1 = 1.000 on the Haiku dataset is impossible as a corpus-level ratio given the stated vocab size and total token count. These are internal arithmetic contradictions, not disagreements with external consensus. A single verifiable re-computation of these two metrics settles the matter. Since the paper provides no code or data and its own example contradicts its tables, the REJECT verdict is justified; the central claim of quantum models being competitive in text generation is unsupported by the reported evidence.","tokens_in":11662,"tokens_out":2795,"duration_ms":29469,"concrete_test":"Recompute BLEU-1 for the QRWKV Simple Sentences row in Table VIII using a standard implementation (e.g., NLTK corpus_bleu/sentence_bleu with whitespace tokenization): generated '<UNK>a woman sits by the river' versus reference 'birds fly in the sky'. If BLEU-1 is not 1.000 (expected ≈0.0–0.14), then Table III's 1.000 is incorrect and the central competitive-BLEU claim lacks support. As a second check, recompute corpus-level Distinct-1 from Table I: for Haiku, 20 samples × 17 words ≈ 340 tokens and vocab size 89, so Distinct-1 ≤ 89/340 ≈ 0.26; any reported value of 1.000 is arithmetically impossible.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is empirical: QKSAN's BLEU-1 of 0.2800 and QRWKV's perfect Distinct-1/BLEU-1 are offered as evidence that quantum-inspired models are competitive. All support comes from Tables II–VII, so the load-bearing assumption is that those metrics were computed correctly. The paper's own example falsifies that assumption. Table VIII lists QRWKV's best Simple Sentences generation as '<UNK>a woman sits by the river' for the reference 'birds fly in the sky'. Standard unigram matching gives precision 1/7 ≈ 0.14 (only 'the' overlaps), not BLEU-1 = 1.000 as reported in Table III. Even allowing unusual tokenization, the reported score is not achievable. Likewise, corpus-level Distinct-1 = 1.000 on Haiku is arithmetically impossible: Table I gives 20 samples × 17 words ≈ 340 total tokens with vocab size 89, so Distinct-1 ≤ 89/340 ≈ 0.26. If Distinct-1 were averaged per sentence rather than over the corpus, the paper never states this, and the reported 1.000 would still require every generated sentence to be repetition-free, which is not established. No code, data, or generation hyperparameters are provided to audit these numbers. Because the headline conclusions rest entirely on these tables, the internal contradiction is decisive.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a comparative study of three hybrid quantum-classical text-generation models (QASA, QKSAN, QRWKV) against Transformer and MLP baselines across five small datasets (simple sentences, short stories, quantum phrases, haiku, proverbs). It evaluates perplexity, BLEU-1/2, Distinct-1, repetition rate, and fluency, and claims that quantum-inspired models are competitive in specific scenarios: QKSAN achieves BLEU-1 0.2800 with zero repetition, and QRWKV attains perfect Distinct-1 = 1.000 on several tasks. The central contribution is empirical: the conclusions rest entirely on the correctness of Tables II–VIII. No code, data, or training/inference details are supplied.","tokens_in":12103,"tokens_out":5276,"duration_ms":54474,"significance":"If the reported numbers were reliable, the paper would provide a useful benchmark for quantum-inspired text generation and highlight a diversity–accuracy trade-off. The QKSAN formulation in §III-C is presented carefully and in a dimensionally consistent matrix form, which is a genuine strength. However, the empirical foundation is severely undermined by internal inconsistencies: the average table contradicts the per-dataset tables, a reported BLEU-1 of 1.000 is contradicted by the paper's own sample output, and the claimed corpus-level Distinct-1 = 1.000 is arithmetically impossible under the stated dataset statistics. Because the central claims are empirical and the tables cannot be trusted, the significance of the findings as stated is low. The absence of code, data, and hyperparameters further prevents verification, so the practical value of the benchmark is not established.","major_comments":[{"comment":"Table II does not follow from Tables III–VII. For example, the MLP average BLEU-1 is listed as 0.2400, but the per-dataset BLEU-1 values (0.480, 0.200, 0.040, 0.000, 0.000) average 0.144; QKSAN's listed 0.2800 is contradicted by its own per-dataset mean of 0.168; QRWKV's listed 0.0948 differs from its per-dataset mean of 0.2496. Similarly, Table II reports Transformer Distinct-1 average 1.000, whereas the per-dataset values (0.345, 0.458, 0.453, 0.627, 0.628) average 0.502. These are not rounding effects; the overall comparison table is internally inconsistent with the detailed results, which are the paper's only evidence for its claims.","section":"§V-B, Tables II–VII"},{"comment":"Corpus-level Distinct-1 = 1.000 for haiku is arithmetically impossible under the dataset statistics in Table I: 20 samples of 17 words each give about 340 total tokens and a vocabulary size of 89, so the global unique-to-total ratio is at most 89/340 ≈ 0.26. The paper never states whether Distinct-1 is computed per sentence and then averaged or over the whole corpus; Eq. (48) just defines it as a ratio. If a per-sentence definition is intended, it must be stated explicitly, and the repeated claim of 'perfect vocabulary diversity' is misleading without that clarification.","section":"§V-A, Eq. (48), Tables I and VI"},{"comment":"The paper's own example falsifies the headline BLEU-1 claim. Table III reports QRWKV BLEU-1 = 1.000 on Simple Sentences, but Table VIII lists QRWKV's best generation as '<UNK>a woman sits by the river' for the reference 'birds fly in the sky'. Standard clipped unigram matching gives at most one overlapping unigram ('the'), so the achievable BLEU-1 is about 1/7 ≈ 0.14, not 1.000. Even with unusual tokenization, 1.000 is not attainable. Since this is the authors' chosen 'best generated text,' the reported perfect score cannot be correct, directly invalidating the central claim about QRWKV's competitive BLEU-1.","section":"§V-C, Tables III and VIII"},{"comment":"The training configuration is described only as '50 epochs' and 'consistent hyperparameters,' with no model sizes, VQC depths (L), number of parameters, learning rates, random seeds, optimization details, or decoding strategy (e.g., greedy vs. sampling). No code or data are provided. Given that the entire paper is an empirical comparison, the absence of these details makes the reported numbers impossible to audit or reproduce. This is a substantive omission rather than a minor presentation issue.","section":"§IV-D"}],"minor_comments":[{"comment":"There is an inconsistency in qubit counting: Eq. (3) amplitude-encodes a d-dimensional vector into d computational basis states |i>, which would require d qubits if implemented literally, while Eq. (4) and the text use n = ceil(log2 d) qubits. The intended encoding basis should be clarified.","section":"§III-A, Eqs. (3)–(4)"},{"comment":"The Gaussian kernel bandwidth σ(a) and stability constant ε are listed as free parameters, but no values or initialization ranges are given in the experimental section, so the QKSAN implementation is not reproducible.","section":"§III-C, Eq. (19)"},{"comment":"The manuscript contains duplicated section headings: 'B. Overall Performance Comparison' and 'D. Dataset-Specific Analysis' appear twice, indicating an editing error.","section":"§V"},{"comment":"The Distinct-n definition should explicitly state the granularity (per generated sentence, per document, or over the whole corpus), since the reported values depend critically on this choice.","section":"§IV-C, Eq. (48)"},{"comment":"Generated outputs contain the <UNK> token, but the paper does not describe the tokenizer, vocabulary size used at decoding, or how unknown tokens are handled in BLEU/Distinct computations. This makes the metric values even harder to interpret.","section":"Table VIII"},{"comment":"Two of the three quantum models are the authors' own prior works. The manuscript should explicitly discuss the relationship to these papers and what new architectural or empirical contribution is made here beyond them.","section":"References [10], [17]"}],"recommendation":"reject","confidential_remarks":"The metric tables are internally inconsistent and the paper's own sample output contradicts the headline BLEU-1=1.000 claim, so the central empirical result cannot stand without a full rerun and release of code/data. The fact that two of the three quantum models are the authors' own prior work (refs [10] and [17]) further limits the independent evidentiary value of the comparison. If the authors can supply raw outputs and metric scripts that resolve these contradictions, a revised submission might be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a straightforward empirical comparison of three quantum-inspired text generation models against Transformer and MLP on five tiny datasets. The architecture descriptions, especially QKSAN, are detailed and clearly written, and the idea of measuring diversity and repetition alongside BLEU is reasonable. If the numbers were right, the finding would be a modest one about a diversity-accuracy tradeoff. But the numbers are not right.\n\nThe biggest problem is that the paper's own tables contradict each other. Table II reports average BLEU-1 for MLP as 0.2400, but the per-dataset values in Tables III–VII average to 0.144. The Transformer's average Distinct-1 is 1.000 in Table II, but its per-dataset Distinct-1 values (0.345, 0.458, 0.453, 0.627, 0.628) average to about 0.50. These are not rounding errors.\n\nMore damning is the direct contradiction between the reported BLEU-1 and the generated examples. Table III gives QRWKV a perfect BLEU-1 of 1.000 on Simple Sentences, but the paper's own best generated text for that model is '<UNK>a woman sits by the river' for the reference 'birds fly in the sky'. Only one unigram ('the') overlaps, so the unigram precision is about 0.14, not 1.0. Similarly, Distinct-1 = 1.000 on the Haiku dataset is arithmetically impossible: 20 samples at 17 words is ~340 tokens, and the vocabulary size is 89, so corpus-level distinct-1 cannot exceed 89/340 ≈ 0.26. The paper never states whether Distinct-1 is computed per sentence or globally, but either way the reported 1.000 is unsupported.\n\nNo code, data, or training hyperparameters are provided, so none of this can be audited. The QKSAN math appears coherent, but the empirical section is the whole point of the paper, and it doesn't hold together.\n\nVerdict: this is not ready for peer review. A serious referee would waste time checking numbers that the authors should have caught. The authors should recompute all metrics carefully, state the granularity of Distinct-n, and provide at least one runnable code example. If they do that, the comparison might be a small but honest benchmark for quantum-inspired models on toy data.","headline":"The paper's headline claims are undermined by its own tables and examples: BLEU-1=1.000 is contradicted by the generated text, and the reported averages don't match the per-dataset numbers.","tokens_in":12519,"tokens_out":3055,"would_cite":false,"duration_ms":28974,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's central claim is that quantum-inspired generators match transformers on simple sentence tasks while eliminating repetition and maximizing vocabulary diversity.","keywords":["quantum natural language processing","hybrid quantum-classical systems","text generation","variational quantum circuits","QKSAN","QRWKV","vocabulary diversity","repetition rate"],"falsifier":"Recompute the haiku diversity score over the whole corpus: with 89 types in Table I, corpus-level Distinct-1 cannot exceed about 0.26, so the reported 1.000 must be per sentence; running the metric both ways across all five datasets would settle whether the diversity and repetition advantages are real.","tokens_in":11596,"feed_emoji":"⚛️","tokens_out":8431,"duration_ms":83001,"temperature":0.7,"pith_summary":"The paper sets out to show that quantum-inspired text generation models do not simply lose to classical transformers: on simple, low-complexity tasks they can match or beat them on specific metrics. It compares five models—Transformer, MLP, QKSAN, QRWKV, and QASA—across five small datasets and reports that quantum-inspired models achieve zero repetition everywhere, while QRWKV and QKSAN post high BLEU-1 scores on simple sentences and perfect Distinct-1 scores on several datasets. If these measurements hold, the practical takeaway is a task-dependent trade-off: quantum architectures trade overall language-model quality for vocabulary diversity and repetition control. The paper concludes that hybrid quantum-classical designs, rather than pure quantum or pure classical models, are the most promising direction.","feed_headline":"Quantum text models beat the transformer at avoiding repetition","feed_subtitle":"They also post perfect vocabulary-diversity scores, but pay for it in perplexity and domain accuracy","key_machinery":"Variational quantum circuits (VQCs) are the common engine: token embeddings are amplitude-encoded into qubit states, processed by parameterized rotation and entangling layers, and read out as expectation values of Pauli-Z operators. QASA uses VQC outputs as attention query, key, and value; QRWKV feeds VQC embeddings into a gated channel-mixing block and an attention score; QKSAN adds a quantum feature map's Gram matrix to the classical attention logits and uses quantum-modulated value gates. The paper treats these circuits as the source of the diversity and repetition-control advantages.","core_discovery":"On its own terms, the paper's central claim is that quantum-inspired models are competitive in specific text generation scenarios: QRWKV reaches a perfect BLEU-1 of 1.000 on simple sentences; QKSAN averages BLEU-1 of 0.2800 with zero repetition; QRWKV reports Distinct-1 of 1.000 on multiple datasets; and every quantum model shows zero repetition on every dataset, while the Transformer repeats at rates up to 0.147. The same results show quantum models weaker at domain-specific and structured text—QKSAN's perplexity on quantum phrases jumps to 5.61 versus the Transformer's 1.24. The authors interpret this as evidence of a diversity–accuracy trade-off and argue that the right response is hybrid","pith_inferences":["Because the datasets are tiny (15–50 samples, vocabularies under 90 words), Distinct-1 values near 1.000 are close to saturation; the diversity advantage may shrink or vanish on larger, real-world vocabularies.","The unstated granularity of Distinct-1 is the key confound: per-sentence scores can be 1.000 even when corpus-level diversity is low, so the comparison should be re-run at both granularities before trusting the diversity claim.","Zero repetition is reported without controlling output length or decoding temperature; a length-matched, temperature-controlled probe would show whether the effect comes from the quantum architecture or from short generations.","A useful next experiment is to replace the quantum feature map in QKSAN and QRWKV with an equivalent classical kernel of the same form, isolating whether the reported behavior is due to quantum mechanics or to the kernel and gating structure itself."],"forward_implications":["Creative-writing and content-generation systems that prize novelty over exact matching are the most plausible first applications for quantum-inspired generators.","A hybrid architecture—classical attention for coherence, quantum gates for diversity—should outperform either family alone on the metrics this paper tracks.","Simple, well-structured language is where quantum models are already competitive; domain-specific technical text remains transformer territory.","Reporting diversity and repetition metrics alongside perplexity and BLEU changes how model comparisons are read; the quantum models' main strengths only show up when all four are tracked.","The choice between quantum-inspired and classical models should be made per task, since no model family dominates on every dataset."],"supporting_citations":[{"why":"Establishes the Transformer and scaled dot-product attention baseline that all quantum models are compared against and modify.","marker":"[1]"},{"why":"Defines the RWKV linear attention-free architecture whose time-mixing and channel-mixing blocks QRWKV builds on.","marker":"[4]"},{"why":"Supplies the QASA quantum adaptive self-attention model, one of the three quantum architectures evaluated.","marker":"[10]"},{"why":"Supplies the QKSAN quantum kernel self-attention network, the architecture behind the paper's competitive BLEU-1 zero-repetition result.","marker":"[11]"},{"why":"Supplies the QRWKV variant with quantum-enhanced channel mixing, responsible for the perfect BLEU-1 and Distinct-1 results.","marker":"[17]"}],"fun_headline_variants":["Quantum text models hit zero repetition, but perplexity rises","Zero repetition from quantum models, at cost of perplexity","Quantum NLP: perfect diversity, but worse accuracy metrics","Quantum text generators: no repeats, yet less fluent"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper's conclusions rest on the reported metrics being computed correctly; in particular, the paper never says whether vocabulary diversity is counted per sentence or across the whole corpus, and that choice alone can produce the perfect scores it reports.","fun_headline_variants_meta":{"raw":{"variants":["Quantum text models hit zero repetition, but perplexity rises","Zero repetition from quantum models, at cost of perplexity","Quantum NLP: perfect diversity, but worse accuracy metrics","Quantum text generators: no repeats, yet less fluent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1201,"prompt_tokens":745,"completion_tokens":456,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":403}},"tokens_in":489,"tokens_out":456,"duration_ms":5166,"temperature":1.0,"reasoning_tokens":403,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:19:40.769068+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the haiku diversity score over the whole corpus: with 89 types in Table I, corpus-level Distinct-1 cannot exceed about 0.26, so the reported 1.000 must be per sentence; running the metric both ways across all five datasets would settle whether the diversity and repetition advantages are real.","supporting_citations":[{"cited_title":"Rwkv: Reinventing rnns for the transformer era,","cited_arxiv_id":null,"evidence_quote":"Defines the RWKV linear attention-free architecture whose time-mixing and channel-mixing blocks QRWKV builds on."},{"cited_title":"Qksan: A quantum kernel self- attention network,","cited_arxiv_id":null,"evidence_quote":"Supplies the QKSAN quantum kernel self-attention network, the architecture behind the paper's competitive BLEU-1 zero-repetition result."}],"review_version":1}