{"id":"5b02fce8-4e5c-4869-b9da-0aca3155ab3a","arxiv_id":"2507.02364","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Replacing the feed-forward networks of a tiny BERT with four-qubit quantum circuits yielded 81.19% versus 79.59% on SST-2, but the result rests on a single seed and a CLS-only circuit application.","lead":"Replacing the feed-forward networks of a compact BERT model with small quantum circuits produced 81.19% accuracy on SST-2 versus 79.59% for the classical baseline, while cutting feed-forward parameters by over 99%. The study tests whether quantum circuits can be parameter-efficient substitutes for classical neural network components in practical language models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed quantum advantage is confounded: QFFN-BERT applies its PQC only to the [CLS] token while the bert-tiny baseline applies classical FFNs to all tokens, and no CLS-only classical control is reported.","rationale":"The reader's weakest_assumption identifies the most load-bearing concern. The paper's central claim is an attribution: a PQC-based FFN replacement outperforms a classical FFN. But Section III-C makes the comparison non-isolating because the QFFN block is applied only to the [CLS] token, whereas the baseline's classical FFN is applied position-wise to every token. A classical CLS-only MLP with the same reduced parameter budget could plausibly produce the same or better accuracy; if so, the 'quantum parameter' explanation collapses. The vanilla-PQC ablation is useful and credible as evidence that co-design choices matter, but it does not provide the missing classical control. Additional weaknesses—single fixed seed, small validation sets, tiny absolute differences, and barren-plateau claims unsupported by gradient measurements—would by themselves make the empirical claims fragile; the architectural confound is more fundamental because it blocks attribution even if the numbers were stable. Therefore the reader's REJECT verdict should stand unchanged.","tokens_in":15781,"tokens_out":7931,"duration_ms":89756,"concrete_test":"Train a 'Classical CLS-only BERT' control on SST-2: use the same bert-tiny backbone and training protocol as in Table IV, but in each layer process only the [CLS] token through a purely classical block with the same dimension sequence as the QFFN block (linear 128→4, nonlinearity, linear 4→128, residual connection) and the same number of trainable parameters as the 4-layer QFFN, while leaving non-CLS tokens without an FFN exactly as in QFFN-BERT. Use the same seed 42, learning rate 5E-4, 5 epochs, and batch size 32. If this classical control reaches or exceeds 81.19% validation accuracy (or falls within the seed-dependent confidence interval of the QFFN run), the reported quantum advantage is not established. Repeating with at least three seeds would also resolve the single-seed noise concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-C states that 'the PQC is applied only to the [CLS] token representation,' and the paper itself acknowledges this is 'a deviation from the standard position-wise FFN.' The baseline bert-tiny, by contrast, uses a classical position-wise FFN at every token. The comparison therefore changes at least two things simultaneously: the CLS-token transformation becomes a small PQC with a residual connection, and the standard FFN is no longer applied to non-CLS tokens in the usual way. The reported 81.19% vs 79.59% SST-2 gap and the 'over 99% FFN-parameter reduction' could plausibly come from the simplified token-wise structure, the CLS-only placement, the residual connection, or the smaller capacity, none of which is quantum-specific. No classical control with the same CLS-only structure, same 128→4→128 projections, same parameter count, and same residual connection is reported; the ablation only compares an optimized PQC against a vanilla PQC. Without such a control, the central attribution of the improvement to quantum computation does not follow.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes QFFN-BERT, a hybrid quantum-classical transformer in which the feedforward network (FFN) modules of a compact BERT variant (bert-tiny) are replaced by parameterized quantum circuit (PQC) blocks. The authors evaluate several PQC depths on the SST-2 and DBpedia benchmarks, reporting that a 4-layer QFFN-BERT reaches 81.19% validation accuracy on SST-2 versus 79.59% for the classical baseline, while reducing FFN-specific parameters by over 99%, and they argue for a data-efficiency advantage in few-shot settings. An ablation study shows that a simpler \"vanilla\" PQC fails to learn, which the authors attribute to the absence of residual connections and insufficiently expressive circuit design. The central claim is that an optimized PQC can match or exceed a classical FFN at much lower module-level parameter cost.","tokens_in":16043,"tokens_out":5058,"duration_ms":56617,"significance":"If the reported results were supported by a controlled comparison, the paper would be a useful empirical contribution to hybrid quantum-classical NLP, demonstrating that a small PQC could plausibly replace a classical FFN in a transformer with competitive accuracy and drastically fewer parameters. The manuscript explicitly describes the PQC architecture, the training protocol, and the hardware/software environment, which aids reproducibility. However, the central empirical claim is currently not established: the comparison is confounded by a simultaneous change in both the transformation (quantum vs classical) and the token scope (CLS-only vs all tokens), and the reported accuracy differences are within the noise level of a single-seed run. The ablation study is informative about co-design but does not isolate a quantum-specific benefit.","major_comments":[{"comment":"The paper states that 'the PQC is applied only to the [CLS] token representation,' while the bert-tiny baseline applies its classical FFN to every token. This changes two factors at once: the nature of the block (quantum vs classical) and its scope (single token vs all tokens). No classical control with the same CLS-only structure—same input/output projections, same residual connection, and comparable parameter count—is reported. Consequently, the observed accuracy advantage on SST-2 (Table IV) and the claimed 'over 99% FFN-parameter reduction' cannot be attributed to quantum computation; they could stem from the simplified token-wise structure, the reduced capacity, the residual connection, or the CLS-only placement. This is the central confound and the main reason why the paper's core claim is not supported.","section":"§III-C"},{"comment":"The full-data SST-2 improvement of the 4-layer QFFN-BERT (81.19% vs 79.59%) is based on a single run with a fixed random seed (Table III). With a validation set of 872 examples, the standard error of a binary accuracy near 0.8 is about 1.35 percentage points, so the 1.6-point gap is within approximately one standard error. No confidence intervals, multiple seeds, or significance tests are reported. The claim that QFFN-BERT 'surpasses its classical counterpart in a full-data setting' is therefore not statistically justified by the presented results.","section":"§IV-A, Table IV"},{"comment":"The few-shot results do not support the claimed 'superior data efficiency.' On SST-2 with 10% of the data, the best QFFN-BERT (8L) is 76.95% versus the baseline's 76.83%, a difference of 0.12 percentage points; with 20% of the data, the classical baseline outperforms all QFFN-BERT variants. On DBpedia few-shot, differences are at most 0.04 percentage points. These margins are far smaller than the variability expected from a single seed with these validation sizes, so the conclusion of a 'consistent and competitive edge' is not empirically grounded.","section":"§IV-A and §IV-B"},{"comment":"The parameter-reduction claim is misleading. The QFFN block does not replace the FFN at every token position; it replaces it only for the [CLS] token. The classical FFN in bert-tiny processes all 128 tokens, whereas the QFFN processes one, so the 'over 99% FFN-specific parameter reduction' conflates a reduction in per-token operations with a reduction in module capacity. A fair parameter-efficiency comparison should control for the total number of FFN-like transformations applied to the sequence, or should report the per-token parameter cost under matched conditions.","section":"§III-C and §IV-C"},{"comment":"The ablation study compares only the optimized PQC against a 'vanilla' PQC, both applied in the same CLS-only placement. Because the vanilla variant lacks a residual connection and uses only RY rotations, its failure to learn may reflect standard deep learning issues (e.g., vanishing gradients due to missing skip connections) rather than quantum-specific limitations. The ablation does not include a classical CLS-only FFN baseline with comparable capacity, so it cannot isolate what, if anything, quantum computation contributes to the improved performance.","section":"§V"}],"minor_comments":[{"comment":"For SST-2, the table lists both validation (872) and test (1,821) sizes, but Table IV reports 'Validation Accuracy' without clarifying whether the SST-2 numbers are computed on the validation or test split; for DBpedia, only a test size is given. The same convention should be used consistently for both datasets.","section":"Table II"},{"comment":"There is a typographical error in 'engtangling gates' (should be 'entangling gates').","section":"§III-D"},{"comment":"The statement that training the 8-layer model on the full DBpedia dataset had not completed after more than 30 days suggests that some reported DBpedia results may come from incomplete runs; please clarify whether all reported numbers correspond to fully converged training.","section":"§VI"},{"comment":"The abstract and the contribution list present the model as replacing FFN modules without noting that the replacement is applied only to the [CLS] token; this architectural detail is central to interpreting the results and should be stated prominently in the abstract.","section":"Abstract and Contributions"},{"comment":"The related-work section focuses on QNLP and quantum attention but omits other hybrid quantum-classical transformer works, such as recent quantum self-attention or quantum vision transformer variants; a brief comparison would strengthen the positioning of the contribution.","section":"§II"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting question and provides a detailed experimental setup, but the central comparison is currently confounded and the statistical evidence is weak. I believe the authors could address the main issues by adding a classical CLS-only FFN control with matched capacity, running multiple seeds with significance testing, and reframing the parameter-efficiency claim to account for the token-scope difference. If those changes are made, the paper could become a valid empirical study; in its current form, the stated conclusions are not supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central claim, that a PQC can replace a transformer's FFN and beat a classical baseline at 99% parameter reduction, is not supported. The comparison is confounded: QFFN-BERT applies the PQC only to the [CLS] token, while the bert-tiny baseline applies its classical FFN to every token. The paper says this in Section III-C and calls it 'a deviation from the standard position-wise FFN,' but never runs a classical control with the same CLS-only structure, same 128-to-4-to-128 projections, and same residual connection. So the accuracy gap could come from the architectural simplification, the residual path, or the smaller capacity, not from anything quantum. The stress-test note is on target.\n\nWhat is genuinely useful is the depth ablation and the negative control. The vanilla PQC, with no residual connection and a single rotation axis, fails to learn on both datasets, while the optimized design trains stably. That contrast credibly illustrates co-design principles for PQCs in deep architectures. The paper is also transparent about its constraints: simulator-only, CPU-bound, CLS-only due to computational cost, and the 8-layer DBpedia run that never finished. As a case study in what can go wrong when you drop a PQC into a transformer, it is a reasonable data point.\n\nThe soft spots beyond the confound: single seed and no error bars, so the 1.6-point SST-2 gap is within noise; the barren-plateau explanation is asserted without gradient or variance measurements; and the '99% parameter reduction' is module-specific, with total model parameters comparable to the baseline. The few-shot results are also mixed: on 20% SST-2 the baseline beats all quantum variants, so 'consistent and competitive edge' overstates it.\n\nI would send this to peer review, but only with the expectation that the authors add a matched CLS-only classical baseline and repeated runs with variance. Without that, the central claim is unsupported. The paper deserves a serious referee because the negative result and the methodology warning are useful to the quantum-ML-NLP community, even if the positive claim fails. I would not cite it as evidence of quantum advantage.","headline":"A confounded comparison sinks the central quantum-advantage claim, but the paper's negative control and depth ablation make it a useful case study.","tokens_in":16526,"tokens_out":2367,"would_cite":false,"duration_ms":28355,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"QFFN-BERT replaces the feedforward networks of a compact BERT with small parameterized quantum circuits and reports that the 4-layer variant reaches 81.19% accuracy on SST-2, surpassing the classical bert-tiny baseline at 79.59% while…","keywords":["quantum machine learning","parameterized quantum circuits","hybrid quantum-classical transformer","feedforward network replacement","BERT","few-shot learning","barren plateaus","SST-2"],"falsifier":"Train a classical control with the same bert-tiny backbone but with a classical single-token MLP applied only to the [CLS] representation, matched to the QFFN's parameter budget. If that control reaches or exceeds 81.19% on full SST-2, the reported gain is not specific to quantum computation; if it falls short, the quantum circuit's contribution is supported.","tokens_in":15567,"feed_emoji":"⚛️","tokens_out":7241,"duration_ms":66304,"temperature":0.7,"pith_summary":"The paper sets out to show that a small parameterized quantum circuit can replace the feedforward network inside a compact BERT model, cutting the FFN's parameter count by more than 99% while matching or slightly beating the classical model's accuracy. On SST-2, the 4-layer QFFN-BERT reaches 81.19% validation accuracy compared with 79.59% for bert-tiny; on DBpedia it matches the near-saturated baseline. The authors also report competitive few-shot performance and a systematic depth study showing an empirical sweet spot at four PQC layers. An ablation with a simpler 'vanilla' PQC fails to learn, which they attribute to missing residual connections and a barren-plateau-prone optimization landscape.","feed_headline":"Quantum FFN beats classic BERT with 99% fewer module parameters","feed_subtitle":"On SST-2 the 4-layer QFFN-BERT hits 81.19% versus 79.59% for bert-tiny, in full-data setting.","key_machinery":"The central object is the Quantum Feedforward Network (QFFN), a block that linearly projects the hidden state down to four dimensions, passes it through a 4-qubit parameterized quantum circuit, and projects back to the hidden size, with the block's input added back through a residual connection. Each PQC layer embeds the classical input once at the first layer, alternates CNOT and CZ entanglement across layers, and applies trainable $R_Z$ then $R_Y$ rotations on every qubit. Two QFFN blocks replace the two classical FFNs in bert-tiny, and the circuit is simulated with Qiskit's EstimatorQNN and trained with the parameter-shift rule. This design gives each trainable angle a non-local influence over the whole 4-qubit state, which the paper calls the power of a quantum parameter.","core_discovery":"The central claim is that a PQC-based block can serve as a parameter-efficient replacement for the FFN modules of a Transformer without sacrificing task performance, and in one configuration improves it. QFFN-BERT applies the PQC only to the [CLS] token representation, uses a residual connection, $R_Y$ and $R_Z$ rotations, and alternating CNOT/CZ entanglement. In the full-data SST-2 setting, the 4-layer model achieves 81.19% accuracy versus 79.59% for bert-tiny, while the FFN-specific parameter count drops by over 99%; on DBpedia, the 4-layer model reaches 99.03% versus 99.02%. The depth study shows non-monotonic performance with a sweet spot at 4 layers, and the vanilla ablation fails to learn on both datasets, which the authors interpret as evidence that co-design with residual connections and expressive rotation strategies is essential.","pith_inferences":["Because the PQC is applied only to the [CLS] token, the comparison is really between a quantum-transformed single token and an untransformed baseline; a classical CLS-only FFN control would tell whether the gain is due to added nonlinear capacity rather than quantum entanglement.","If the few-shot edge reproduces across tasks, PQCs may be most valuable in data-limited NLP regimes rather than as replacements for large pretrained models.","The 'power of a quantum parameter' claim suggests a concrete test: fix the parameter budget and compare a PQC against an MLP with the same number of parameters but entangled feature structure; if accuracy tracks entanglement pattern, the quantum-specific hypothesis gains support.","The ablation's two confounded changes (residual connection plus richer rotations/entanglement) mean the vanilla failure cannot by itself pinpoint barren plateaus; a follow-up could add a residual connection to the vanilla PQC and measure gradient variance."],"forward_implications":["On full SST-2, the 4-layer QFFN-BERT reaches 81.19% accuracy, above the 79.59% classical baseline, so a PQC-based FFN can be a performance-neutral or positive substitute within the same model.","The FFN module parameter count drops by more than 99%, so the accuracy-per-parameter ratio of the quantum module is dramatically higher than the classical FFN it replaces.","Depth scaling is non-monotonic: 1-, 2-, 4-, and 8-layer variants show 4 layers as the empirical sweet spot, with 8 layers slightly worse, consistent with an expressibility-trainability trade-off.","In few-shot settings QFFN-BERT is competitive with the fine-tuned baseline on SST-2 (8L beats baseline at 10% data, 76.95 vs 76.83) and slightly ahead on DBpedia at 20% data (98.71 vs 98.68), indicating better data efficiency.","The vanilla PQC variant fails to learn on both datasets (near 51% on SST-2, below 34% on DBpedia), showing that residual connections and rotation/entanglement design are necessary for trainability."],"supporting_citations":[{"why":"Defines the BERT architecture that QFFN-BERT modifies and evaluates.","marker":"[1]"},{"why":"Defines the Transformer encoder structure whose FFN modules are replaced.","marker":"[3]"},{"why":"Supports the premise that FFNs account for roughly two-thirds of Transformer parameters.","marker":"[4]"},{"why":"Provides the Qiskit simulator and TorchConnector used to implement and integrate the PQC.","marker":"[14]"},{"why":"Supplies the SST-2 sentiment benchmark where the headline accuracy result is measured.","marker":"[26]"},{"why":"Provides the bert-tiny pre-trained baseline model used in all comparisons.","marker":"[29]"},{"why":"Supplies the expressibility and entangling-capability concepts used to frame the depth study.","marker":"[32]"},{"why":"Establishes the barren plateau phenomenon invoked to explain the vanilla PQC's training failure.","marker":"[33]"}],"fun_headline_variants":["Quantum FFN cuts BERT module parameters by 99% without accuracy loss","PQC feedforward nets match BERT with 1% of the parameters","Quantum transformer: 99% fewer FFN weights, higher SST-2 accuracy","Hybrid quantum FFN: BERT-level accuracy with 99% fewer FFN params","Quantum FFN outperforms BERT tiny with 1/100th of FFN parameters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage rests on the assumption that applying the PQC only to the [CLS] token, while leaving all other token representations untouched, isolates the effect of the quantum circuit; no classical model with the same CLS-only FFN structure was evaluated.","fun_headline_variants_meta":{"raw":{"variants":["Quantum FFN cuts BERT module parameters by 99% without accuracy loss","PQC feedforward nets match BERT with 1% of the parameters","Quantum transformer: 99% fewer FFN weights, higher SST-2 accuracy","Hybrid quantum FFN: BERT-level accuracy with 99% fewer FFN params","Quantum FFN outperforms BERT tiny with 1/100th of FFN parameters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000985,"raw_usage":{"total_tokens":4226,"prompt_tokens":1037,"completion_tokens":3189,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":3081}},"tokens_in":653,"tokens_out":3189,"duration_ms":27263,"temperature":1.0,"reasoning_tokens":3081,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:31:30.007000+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a classical control with the same bert-tiny backbone but with a classical single-token MLP applied only to the [CLS] representation, matched to the QFFN's parameter budget. If that control reaches or exceeds 81.19% on full SST-2, the reported gain is not specific to quantum computation; if it falls short, the quantum circuit's contribution is supported.","supporting_citations":[{"cited_title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,","cited_arxiv_id":null,"evidence_quote":"Defines the BERT architecture that QFFN-BERT modifies and evaluates."},{"cited_title":"Attention is All You Need,","cited_arxiv_id":null,"evidence_quote":"Defines the Transformer encoder structure whose FFN modules are replaced."},{"cited_title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations,","cited_arxiv_id":null,"evidence_quote":"Supports the premise that FFNs account for roughly two-thirds of Transformer parameters."},{"cited_title":"Quantum Computing with Qiskit,","cited_arxiv_id":null,"evidence_quote":"Provides the Qiskit simulator and TorchConnector used to implement and integrate the PQC."},{"cited_title":"Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank,","cited_arxiv_id":null,"evidence_quote":"Supplies the SST-2 sentiment benchmark where the headline accuracy result is measured."},{"cited_title":"Expressibility and Entangling Capability of Parameterized Quantum Circuits for Hy- brid Quantum-Classical Algorithms,","cited_arxiv_id":null,"evidence_quote":"Supplies the expressibility and entangling-capability concepts used to frame the depth study."},{"cited_title":"Barren Plateaus in Quantum Neural Network Training Landscapes,","cited_arxiv_id":null,"evidence_quote":"Establishes the barren plateau phenomenon invoked to explain the vanilla PQC's training failure."}],"review_version":1}