{"id":"c20f643c-5ec2-40ed-be32-45042dfb65cc","arxiv_id":"2509.09541","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Variational quantum circuits with DisCoCat sentence structure outperform classical DisCoCat on a toy left/right image captioning benchmark, with MHE encodings working best and CLIP results near chance.","lead":"Researchers trained small variational quantum circuits to learn the meanings of short captions about left/right relations between shapes, then matched them to images. The quantum models beat the classical compositional baseline on the hard test split, but stayed near chance when using CLIP image features.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Trainable image box confounds the quantum-vs-classical DisCoCat comparison; the headlined OOD gap may be due to extra trainable image features, not quantum training.","rationale":"The paper's central claim is not merely that a QNLP pipeline can reach a number; it is that quantum-trained DisCoCat representations generalize better than classically trained ones and are less prone to overfitting. The evidence for this is the gap between Table III and the classical rows. But in the table that produces the best quantum numbers, the image encoder itself is optimized. Figure 9 and the surrounding text make this explicit. A classical baseline that receives fixed image vectors is not the correct control for isolating a quantum advantage in learning compositional sentence tensors; it conflates representation learning on the sentence side with an adaptable image embedding. The paper's own no-trainable-image condition (Table IV) gives much weaker quantum results, which is consistent with the confound being responsible for the headline gap. The reader's weakest assumption about simplified sentences is plausible but less decisive: the 'noun isLeftOf noun' form retains exactly the subject-relation-object composition needed for this benchmark, so even if it removes some function words, it does not obviously change what is being tested. I therefore recommend keeping the CONDITIONAL verdict (no change from the reader), but the conditions should include adding a classical trainable-image baseline and reporting seed-level results, not just the simplification issue.","tokens_in":9568,"tokens_out":8219,"duration_ms":102813,"concrete_test":"Run a classical DisCoCat model with a trainable image encoder of matching capacity (e.g., a linear layer from the 8-d MHE vector to the sentence dimension, optionally with a tanh, plus the same contrastive loss) and train it under the same protocol (100 epochs, same split, 4 seeds, best-OOD-val reporting). Record OOD test accuracy for all four seeds. If any seed or the best-seed value reaches ~64% (or the mean ± std overlaps Quantum-MHE), the quantum advantage in Table III is confounded; if all seeds stay near or below 30–35%, the quantum-specific claim survives this check. Repeat for CLIP-PCA features to explain the 50.31 vs 0.00 gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest evidence for 'quantum models are less susceptible to overfitting' comes from the Unifying Trainable Image Box results (Table III). Section III-B and Figure 9 state that the box's rotation gates are trained, 'which means that we are in effect training the image vectors.' The classical DisCoCat baselines of Section III-C instead use fixed MHE or CLIP-PCA image vectors and train only relation/word tensors. Thus the comparison Quantum-MHE 64.06 vs Classic-DisCoCat-MHE 30.63 (and Quantum-CLIP-angle 50.31 vs Classic-DisCoCat-CLIP 0.00) changes two variables at once: the quantum/classical parameterization of the sentence tensors and the presence of a trainable image-side encoder. The results in Table IV, where image vectors are not trained, are near chance (Quantum-MHE 50.00, Quantum-CLIP-angle 48.75), suggesting most of the reported advantage in Table III may come from the trainable box rather than from the alleged quantum inductive bias. Without a classical DisCoCat trained with an analogous small trainable image projection, the central attribution claim is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents an empirical study of whether Variational Quantum Circuits (VQCs) improve compositional generalization in a tensor-based DisCoCat model. The authors simplify captions to 'noun isLeftOf/RightOf noun', encode images with either multi-hot encodings or CLIP vectors (angle/amplitude encoding), and train IQP-style VQCs to match image and sentence output states. They compare against classically trained DisCoCat models and a CLIP baseline. The central reported finding is that quantum-trained DisCoCat models outperform classical DisCoCat models on the OOD test split (e.g., Quantum-MHE 64.06% vs Classic-DisCoCat-MHE 30.63%; Quantum-CLIP angle 50.31% vs Classic-DisCoCat-CLIP 0.00%), which the authors interpret as evidence that quantum models are less susceptible to overfitting on the training set.","tokens_in":9830,"tokens_out":5146,"duration_ms":62964,"significance":"If the advantage were cleanly attributable to the quantum parameterization, this would be an interesting proof of concept with very small trainable parameter counts. The paper is clearly organized, connects categorical semantics to a concrete quantum circuit implementation, and uses a task simple enough to be reproducible. It also honestly frames the work around a conjecture rather than a theorem. However, the central attribution is undermined by the trainable image box used only in the quantum models, by the seed-selection and reporting choices, and by asymmetric training budgets. The paper's own Table IV suggests the quantum advantage may largely disappear without the trainable image box. With additional controlled experiments, the comparison could become informative for the quantum-NLP community.","major_comments":[{"comment":"The headline comparison is confounded. The 'Unifying Trainable Image Box' used for all quantum models in Table III has trainable rotation gates, so 'we are in effect training the image vectors' (Section III-B). The classical DisCoCat baselines in Section III-C instead use fixed MHE or CLIP image vectors and train only the relation tensors. Thus Quantum-MHE 64.06% vs Classic-DisCoCat-MHE 30.63% and Quantum-CLIP-angle 50.31% vs Classic-DisCoCat-CLIP 0.00% change two variables at once. Table IV, where image vectors are not trained, shows near-chance quantum performance (50.00% and 48.75%), suggesting that most of the advantage in Table III may be due to the trainable image-side projection. A classical DisCoCat trained with a small trainable image projection of comparable capacity is needed to attribute the improvement to the quantum training rather than to this extra trainable component.","section":"Section III-B, Section III-C, Table III"},{"comment":"The paper reports only the seed with the best OOD validation accuracy from four random seeds and gives no error bars or variance. With a chance level of 50%, Quantum-CLIP angle 50.31% and Quantum-MHE 50.00% in Table IV are statistically indistinguishable from chance. The claim that quantum models 'consistently outperform' classical DisCoCat is not supported without reporting all seeds, means/standard deviations, and ideally significance tests. The 0.00% test accuracy for Classic-DisCoCat-CLIP (with 66.00% validation) also needs explanation; if this is a single unlucky seed, the comparison is especially fragile.","section":"Section IV, Tables III and IV"},{"comment":"Quantum and classical models are trained with asymmetric budgets: 100 epochs for quantum vs 50 for classical (Section IV). Learning rates also differ (0.001 for quantum, 0.1/0.01 for classical DisCoCat). The paper's motivating conjecture is increased training efficiency of quantum models, but the experiments do not compare efficiency under matched budgets. To substantiate 'less susceptible to overfitting' or 'train better', the authors should add matched-epoch comparisons or accuracy-versus-epoch curves. As reported, the results cannot distinguish an efficiency/inductive-bias effect from simply allowing the quantum models twice as many training updates.","section":"Section I, Section IV"}],"minor_comments":[{"comment":"Typo: 'Acorss all models' should be 'Across all models'.","section":"Section IV"},{"comment":"Please reconcile '27k gates per caption' with the later statement that 'each caption is encoded by a variational circuit with 1 qubit per NOUN and 3 layers, yielding 36 trainable parameters per caption'. These statements are hard to align with the 8-bit MHE vectors and the 9/12-qubit CLIP encodings.","section":"Section III-B, Section IV"},{"comment":"The 'noise' injected into MHE encodings is never defined in Sections III-B or IV. Please specify what noise is added and why it is used.","section":"Tables III and IV"},{"comment":"The claim that the simplified form 'noun is{left/right}Of noun' retains 'all the essential semantic information of a caption' is too strong. The simplification may be useful for tractability, but it drops grammatical structure and should be presented as a limitation rather than as an assertion of semantic equivalence.","section":"Section III-B"},{"comment":"The CLIP Frozen/Fine-tuned rows are not DisCoCat baselines and have vastly larger parameter counts. Clearly separate these reference points from the DisCoCat comparisons in the text and figure captions.","section":"Tables III and IV"}],"recommendation":"major_revision","confidential_remarks":"The paper is suitable in scope for QAI as a proof-of-concept, but the central claim of a quantum advantage is not yet supported. The missing classical control with a trainable image projection and the absence of full seed statistics are fixable with additional experiments, so I recommend major revision rather than rejection. There are no apparent novelty-disclosure issues, though the overlap with [7] should be made more explicit in the final version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is applying variational quantum circuits to the Lewis et al. compositional generalization benchmark. No one has done that before, and the paper shows that a small VQC can be trained on this task and reach non-trivial OOD accuracy. The setup is clean, the writing is clear, and they report training, validation, and test numbers for two image encodings. The error analysis (sphere never recognized) is a nice touch. Credit where it is due: this is a legitimate exploratory study.\n\nThe soft spot is not a detail; it is the load-bearing comparison. In Table III, the quantum models use the trainable image box, which means the image vectors themselves are trained. The classical DisCoCat baselines use fixed MHE or CLIP vectors. So Quantum-MHE 64% vs Classic-DisCoCat-MHE 31% changes two variables at once: the quantum/classical parameterization of the sentence tensors and the presence of a trainable image-side encoder. Table IV, where image vectors are not trained, shows quantum performance at chance (50% and 48.75%). That strongly suggests the advantage in Table III comes from the trainable box, not from a quantum inductive bias. Without a classical DisCoCat trained with an analogous small trainable image projection, the paper's central claim that quantum models are less susceptible to overfitting is not established.\n\nOther issues are less severe but real. The results are reported for the best of four seeds by OOD validation, with no error bars, and the quantum models get 100 epochs versus 50 for classical. The CLIP angle encoding hits 50.31% on a binary task, which is basically chance. The noise injection for MHE is described only as 'with noise,' with no specified noise model. The sentence simplification to 'noun isLeftOf noun' is acceptable for this toy task, but calling it 'all the essential semantic information' is a stretch.\n\nWhere does that leave us? The paper is a reasonable proof-of-concept that VQC-DisCoCat can be trained on a multimodal task. But the headline result needs a matched experiment before it supports the overfitting claim. A serious referee should ask for that experiment, plus all seeds, error bars, and matched training budgets. I would not cite the current results, but I would bring it to a reading group as a good case study in experiment design.\n\nRecommendation: send it to peer review, but expect major revision and a new comparison before it can be published as anything more than a preliminary report.","headline":"A useful proof-of-concept, but the central quantum-vs-classical comparison is confounded: the trainable image box, not quantum training, likely explains the reported advantage.","tokens_in":10320,"tokens_out":1920,"would_cite":false,"duration_ms":22895,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quantum-trained DisCoCat representations beat classically trained ones on out-of-distribution image captions.","keywords":["compositional generalization","variational quantum circuits","DisCoCat","quantum natural language processing","out-of-distribution generalization","image captioning","contrastive learning","relational reasoning"],"falsifier":"Train a classical DisCoCat model on the exact same simplified captions and multi-hot image encodings with the same 100 epochs and a modest learning-rate grid search; if it matches or exceeds Quantum-MHE's 64.06% OOD test accuracy, the claim that quantum training efficiency causes the reduced overfitting is unsupported.","tokens_in":9436,"feed_emoji":"⚛️","tokens_out":5724,"duration_ms":64097,"temperature":0.7,"pith_summary":"This paper asks whether representing compositional sentence meanings as quantum circuits can improve compositional generalization—the ability to handle unseen combinations of known words. It implements the DisCoCat framework, where grammar dictates how word vectors and tensors contract, as variational quantum circuits, and trains them on an image-captioning task where the model must decide which spatial caption matches a picture. On the out-of-distribution test split, quantum-trained representations beat classically trained DisCoCat representations by a wide margin (64.06% vs 30.63% with multi-hot image encodings; 50.31% vs 0% with CLIP vectors), while using hundreds of trainable parameters compared with CLIP's tens of millions. The authors conclude that quantum models are less prone to overfitting when learning compositional tensors and offer a proof of concept for quantum natural language processing on multimodal tasks.","feed_headline":"Quantum-trained meanings beat classical on unseen captions","feed_subtitle":"Variational circuits learned spatial relations with less overfitting than classical compositional models on unseen captions.","key_machinery":"The central machinery is a DisCoCat-to-quantum translation: compact-closed categorical string diagrams for sentences are turned into IQP variational circuits, with grammatical 'cups' implemented as postselected contractions, nouns as qubit registers, and relation words as parameterized rank-2 and rank-3 tensors. Images are loaded as quantum states either by multi-hot binary encoding or by angle/amplitude encoding of PCA-reduced CLIP vectors. To compare image and sentence circuits, the paper either appends a trainable 'unifying box' that reduces the image to one qubit or pads the sentence space to the image dimension. The matching score is the inner product between the two circuit outputs, tr","core_discovery":"The central claim is that translating DisCoCat's tensor-based compositional semantics into variational quantum circuits yields representations that generalize out-of-distribution better than the same semantics trained classically. Grammatical structure is wired directly into the circuit; words and relation words are parameterized rotations; images enter as quantum states, either as binary multi-hot vectors or as PCA-reduced CLIP embeddings; and the inner product between image and sentence circuit outputs is the matching score optimized by a contrastive loss. The strongest evidence is Quantum-MHE reaching 64.06% on the OOD test split versus 30.63% for Classic-DisCoCat-MHE, and Quantum-CLIP wi","pith_inferences":["Editorial: if the training-efficiency explanation is right, the quantum advantage should grow with tensor rank or sentence length, because classical tensor contraction costs grow exponentially while circuit depth grows linearly in qubits; a controlled scaling experiment over relations with more arguments would test this.","Editorial: the difference between angle encoding (50.31%) and amplitude encoding (41.25%) suggests that the inductive bias of the classical-to-quantum embedding matters more than raw circuit expressiveness; amplitude encoding entangles all features immediately, which may make the later parameterized layers work harder to single out the subject–relation–object structure.","Editorial: the simplification to 'noun isLeftOf noun' keeps the benchmark tractable but removes determiners and verb phrase structure; if the DisCoCat-to-circuit translation is ever claimed to handle natural captions, the 27k-gates-per-caption complexity of the full sentence form must be addressed by circuit compression or factorization, not by simplification."],"forward_implications":["Quantum-trained DisCoCat representations generalize to unseen combinations of known shapes and relations substantially better than classically trained DisCoCat, so the viability of compositional tensor semantics should be re-evaluated rather than dismissed from earlier negative results.","Because quantum models achieve this with a few hundred parameters and 100 epochs, training efficiency, not model capacity, is the most plausible cause of the improved out-of-distribution performance.","The choice of image encoding is decisive: multi-hot encodings give strong results, while CLIP vectors expose shape-level failures (sphere is never recognized), indicating that future gains require staged training of shape features before relations.","On the unseen validation split, Quantum-CLIP with angle encoding (61.5%) approaches fine-tuned CLIP (63%), suggesting that near-parity with much larger models may be achievable on this task with better training procedures rather than more parameters."],"fun_headline_variants":["Quantum circuits beat classical on out-of-distribution captions","Variational quantum circuits reduce overfitting in compositional generalization","Quantum-DisCoCat outperforms classical models on unseen caption splits","Quantum encoding enables better compositional generalization in captions","Quantum-trained semantics generalize better than classical on novel captions"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the simplified caption form 'noun isLeftOf noun' preserves all the semantic structure needed to test compositional generalization; if the dropped grammatical material matters, the benchmark no longer tests the phenomenon the paper claims to measure.","fun_headline_variants_meta":{"raw":{"variants":["Quantum circuits beat classical on out-of-distribution captions","Variational quantum circuits reduce overfitting in compositional generalization","Quantum-DisCoCat outperforms classical models on unseen caption splits","Quantum encoding enables better compositional generalization in captions","Quantum-trained semantics generalize better than classical on novel captions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001198,"raw_usage":{"total_tokens":4730,"prompt_tokens":651,"completion_tokens":4079,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":395,"completion_tokens_details":{"reasoning_tokens":3999}},"tokens_in":395,"tokens_out":4079,"duration_ms":34575,"temperature":1.0,"reasoning_tokens":3999,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:53:59.676138+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a classical DisCoCat model on the exact same simplified captions and multi-hot image encodings with the same 100 epochs and a modest learning-rate grid search; if it matches or exceeds Quantum-MHE's 64.06% OOD test accuracy, the claim that quantum training efficiency causes the reduced overfitting is unsupported.","supporting_citations":[],"review_version":1}