Pith. sign in

REVIEW 3 major objections 6 minor 33 references

A memory-and-feedback loop turns an LLM into a dependable quantum-circuit optimizer, reaching perfect entanglement on 25-qubit circuits within 45 oracle calls where a random hill-climber stalls below 0.29.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:58 UTC pith:N2TFYYIZ

load-bearing objection Useful demo of LLM test-time optimization for circuit synthesis, but the headline causal claim about feedback/restart lacks a controlled test. the 3 major comments →

arxiv 2602.03466 v5 pith:N2TFYYIZ submitted 2026-02-03 quant-ph stat.ML

Quantum Circuit Generation via test-time learning with large language models

Adriano Macarone-Palmieri , Rosario Lo Franco This is my paper
classification quant-ph stat.ML MSC 81P6868Q12 PACS 03.67.-a03.67.Mn
keywords quantum circuit synthesistest-time learninglarge language modelsMeyer–Wallach entanglementblack-box optimizationfeedback promptingrestart-from-bestmemory-augmented search
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that large language models can act as black-box optimizers for quantum circuit synthesis if they are embedded in a closed test-time loop: keep the best circuits in an episodic memory, feed back the signed change in the Meyer–Wallach entanglement score, and restart from the best circuit when progress stalls. On 20-qubit circuits a feedback-free version reaches Q = 0.99 in one favorable 35-gate case, but with only about a 10% chance of exceeding 0.8. On 25-qubit circuits, adding score-difference feedback and restart-from-best makes several runs reach Q = 1.0 within 45 oracle calls, while a budget-matched random hill-climbing baseline never exceeds roughly 0.29. A sympathetic reader cares because this suggests expensive scientific design problems can be tackled without training data or fine-tuning, using only inference-time memory and evaluator feedback.

Core claim

The central claim is that test-time memory and evaluator feedback, not model scale alone, carry the LLM from random circuits to globally entangled states. On 20 qubits, a feedback-free prompt with episodic memory reaches Q = 0.99 in one 35-gate run, but with a low success rate: roughly 10% of runs exceed Q > 0.8. On 25 qubits, the full recipe—score-difference feedback plus restart-from-best—lets multiple independent runs hit Q = 1.0 within three queries of 15 candidate samples each. The winning circuits are written out explicitly and decompose into tensor products of Bell pairs and GHZ states, i.e. stabilizer (Clifford) states, sometimes with RY rotations. The same oracle budget given to a r

What carries the argument

The load-bearing mechanism is an episodic memory plus feedback loop. At each iteration the LLM receives the current gate list, the signed change in Meyer–Wallach value (e.g. 'you improved by 0.12'), and—on restart—the best circuit seen so far. The Meyer–Wallach measure, a permutation-invariant number in [0,1] built from single-qubit purities, is the objective being maximized. The loop enforces a fixed gate count drawn from {CNOT, H, RY} with a small fixed set of RY angles, so the LLM can only rearrange connectivity, not inflate the gate count. This turns circuit design into a sequence of creative local edits constrained by a textual reward signal and protected against trivial solutions.

Load-bearing premise

The paper's headline contrast assumes the improvement comes from feedback and restart, but it does not report a same-model, same-budget control without feedback on the 25-qubit task; the 'naive querying' plateau is described in prose rather than tabulated, and some successful runs start from pre-sampled circuits, so the causal role of feedback and restart is not fully isolated.

What would settle it

Run the same 25-qubit setup with the same budget and the same initial circuits, but remove only the score-difference feedback from the prompt while keeping memory and restarts; tabulate success rates. If Q = 1.0 is reached just as often, the paper's central claim about feedback's role is falsified. Also, if a random hill-climber with restart-from-best over the same 45-evaluation budget reaches Q = 1.0, the LLM's specific contribution would be called into question.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the procedure holds up, LLM-based synthesis can produce maximally entangled 25-qubit circuits with far fewer oracle calls than repeated random search.
  • High-scoring circuits discovered by the method are structurally simple—tensor products of Bell pairs and GHZ states—suggesting the Meyer–Wallach objective is largely saturated by disconnected stabilizer states.
  • Score-difference feedback and restart-from-best are presented as the levers that break the Q ≈ 0.5–0.7 plateau observed under naive querying; the paper attributes the gain to these mechanisms.
  • The method requires no fine-tuning and no training data, only a black-box simulator for evaluation, so it transfers in principle to other expensive black-box design objectives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper never runs a same-model, same-budget control without feedback on the 25-qubit task; the 'naive querying' plateau is described in prose, not tabulated, so the causal share of feedback versus restart versus model scaling is not yet isolated—a direct ablation would settle it.
  • Because the MW objective saturates on disconnected Bell-pair products, this benchmark may be easier than it looks; objectives that reward graph-state connectivity or genuine multipartite entanglement could show a different LLM-versus-baseline gap.
  • Restart-from-best is a simple evolutionary strategy; keeping a diverse set of memory traces (more than one prior best) might overcome the 'stuck topology' failures the authors report for rows D2/D3.
  • A budget-matched deterministic local search using the same gate set and MW evaluator could clarify how much of the success comes from the LLM's prior knowledge of quantum circuits rather than from the feedback loop itself.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a memory-augmented test-time optimization framework for LLM-driven quantum circuit synthesis, using the Meyer–Wallach (MW) global entanglement measure as the black-box objective. The method combines episodic memory of high-scoring candidates, score-difference feedback, and restart-from-best sampling. On 20-qubit circuits with GPT5.1 and no feedback, the framework reaches Q=0.99. On 25-qubit circuits with GPT5.2, feedback and restarts are reported to enable multiple runs to reach Q=1.0 within 45 oracle calls, while a budget-matched random hill-climbing baseline is said to stall below Q≈0.29. The paper also analyzes the structure of synthesized states, finding that high-MW solutions are often stabilizer or graph-state-like constructions (products of Bell pairs and GHZ states).

Significance. If the central claims were fully supported, the work would demonstrate that LLMs with simple episodic memory and scalar feedback can efficiently optimize a black-box quantum objective at 25 qubits, substantially larger than typical prior ML-based circuit synthesis. The paper's transparency is a strength: it discloses the full 11-run collection in Table II, computes the MW objective externally (no circularity), and provides prompt templates in the appendix. However, the comparative and causal claims are not yet established because the key ablations and controls are missing.

major comments (3)
  1. [§III B, Table II; Abstract] The claim that 'feedback and restart mechanisms enable multiple runs to reach Q(ψ)=1.0' is not supported by a controlled comparison. No GPT5.2 no-feedback run is reported at 25 qubits under the same 45-evaluation budget. The only no-feedback data are GPT5.1 on 20 qubits (Table I), and the prose plateaus (≈0.48 in §III B; ≈0.7 in the Introduction) are inconsistent and do not specify model, prompt, or budget. Without this control, the Q=1.0 outcomes could be due to GPT5.2's prior knowledge of high-entanglement stabilizer/graph states (the authors themselves note the solutions are products of Bell pairs and GHZs) rather than to the feedback/restart mechanism. Please add the missing ablation or reframe the causal claim.
  2. [§III B, baseline paragraph; Table II] The random hill-climbing baseline is not initialization-matched to the LLM runs. The text says the baseline starts from 'the same random initialization used in Table II' at Q≈0.02, and from Q=0.08, whereas Table II reports initial MW values from 0.05 to 0.47 (rows A and C use pre-sampled circuits). The baseline ceiling is stated as ≈0.16 and ≈0.29 depending on the start, while the abstract uses only ≈0.29. Because the LLM runs begin from substantially better initial circuits, the comparison does not isolate sample efficiency. Run the baseline from each initial circuit in Table II and report the full distribution.
  3. [§III A; §I] The paper claims LLMs can 'sample optimal circuits beyond repeated sampling approaches' but does not implement a repeated-sampling (best-of-n) baseline with the same model. Given that the high-MW solutions are simple stabilizer states (products of Bell pairs and GHZ), GPT5.2 might produce them from memory in a single sample without memory or feedback. A same-model best-of-45 control (or at least a random-search baseline over the same gate set) is needed to attribute the performance to the proposed framework rather than to model priors. Otherwise, the causal claims should be substantially softened.
minor comments (6)
  1. [§II C vs Appendix V A; Fig. 1] The RY angle set is inconsistent: Section II C states angles {3,7,25}, while the prompt in Appendix V A and the Fig. 1 caption use {3.0,10.0,25.0}. This ambiguity prevents exact reproduction; please reconcile.
  2. [§III B, Table II] The paper reports no error bars or confidence intervals. The full 11-run collection is disclosed, which is good, but the success rate is not quantified. Please report e.g. bootstrap CIs for the fraction of runs reaching relevant thresholds (Q≥0.9, Q=1.0).
  3. [Abstract vs Table II] The abstract's 'multiple runs reach Q(ψ)=1.0' is supported by only 2 of 11 runs in Table II (rows A and C1); rows C2, E, F, H reach 0.91–0.99. Please clarify the threshold and report exact counts.
  4. [Introduction vs §III B] The naive-querying plateau on 25 qubits is reported as '∼0.7' in the Introduction and '∼0.48' in Section III B, with no model/budget specification. Please make these numbers consistent and define what 'naive querying' includes.
  5. [§II B] The sentence 'it is worth highlighting that this metric does not produce a W state' is unclear — Q is a function, not a state generator. Please rephrase (e.g., to describe the value of Q for W states).
  6. [Throughout] The manuscript contains several typos and incomplete sentences (e.g., 'This yielded to the theory of naive sampling approaches', missing articles). An editorial pass is needed.

Circularity Check

0 steps flagged

No circularity: externally-evaluated search with no fitted predictions or load-bearing self-citation.

full rationale

I walked the derivation chain from Eq. (1) (Meyer–Wallach definition), through Eq. (2)/(3) (generation with feedback), to Tables I–III. The objective Q(ψ) is computed by an external simulator from the proposed circuit; the LLM is a stochastic proposer; the feedback is the measured score difference; and the reported Q=1.0 outcomes are externally evaluated search results, not algebraic consequences of the inputs. No parameter is fitted to the data and then relabeled as a prediction, and no self-cited prior result is used as the target. The Dynamic Cheatsheet citation [18] is acknowledged prior work, not a self-citation. The paper's main weakness—absence of a same-model, same-budget no-feedback control on the 25-qubit task and the non-initialization-matched hill-climbing baseline—is an experimental-comparison and correctness concern, not circularity: even if the causal claim about feedback/restart is under-supported, the Q values themselves are not derived from the method's assumptions by construction. I therefore find no significant circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical entities or mathematical constants are introduced. The central claim rests on hand-picked hyperparameters (budget, gate count, RY angles) and on the unstated premise that the proprietary LLM's parseable outputs are representative. The MW metric is externally defined and computed by a simulator, so circularity is low.

free parameters (4)
  • Query budget (3 queries x 15 steps) = 45 oracle calls
    Hand-chosen hyperparameter; defines the success claim and the budget-matched baseline. No ablation on budget length is reported.
  • RY angle set = {3.0, 10.0, 25.0} (and {0.1, 0.42, 1.0})
    Hand-chosen discretization of continuous RY rotations. Changes the search space and whether circuits are Clifford; the paper itself tests a second set, showing sensitivity.
  • Gate count range = 25-45 gates for 20 qubits; 25 for 25 qubits
    Chosen by hand to restrict the search space and avoid trivial many-CNOT solutions. Table I shows strong gate-count dependence (0.38 to 0.99).
  • Number of warm-started restarts = 3 queries
    The 'restart-from-best' strategy uses three sequential runs; the choice of 3 is not justified and results may depend on it.
axioms (5)
  • standard math Meyer–Wallach measure Q is a valid global entanglement quantifier with 0 <= Q <= 1.
    Definitions from Refs. [19-21]; used as the external objective and feedback signal.
  • standard math GHZ states and products of Bell pairs attain Q=1.
    Used to interpret the found circuits; the paper's own analysis in Section III B shows the best circuits are such product states.
  • domain assumption Exact statevector simulation on up to 25 qubits gives the oracle MW values.
    No noise model or sampling error is considered; the 'exponentially expensive oracle' is implicitly exact and noiseless.
  • domain assumption The proprietary LLM (GPT5.1/5.2) outputs are parseable gate lists that respect the 'do not add/remove gates' constraint.
    The method depends on syntactically valid outputs; malformed-output rates are only reported for local models, not for GPT5.1/5.2.
  • domain assumption Initial circuits are drawn from the random circuit ensemble D_RQC, and starting MW values are representative.
    Initial MW values range from 0.02 to 0.48, so 'random' is not tightly controlled; rows A and C use 'pre-sampled circuits'.

pith-pipeline@v1.3.0-alltime-deepseek · 11796 in / 15839 out tokens · 160264 ms · 2026-08-03T04:58:07.580472+00:00 · methodology

0 comments
read the original abstract

Deploying large language models (LLMs) as optimizers for black-box scientific design problems requires efficient test-time refinement under expensive evaluations and without training data. We propose a \emph{memory-augmented test-time optimization} framework that combines episodic memory of high-scoring candidates, score-difference feedback, and restart-from-best sampling to improve iterative search. We evaluate the approach on quantum circuit synthesis, where the objective is to maximize the Meyer--Wallach (MW) global entanglement measure under an exponentially expensive black-box oracle. On 20-qubit circuits, the framework achieves $Q(\psi)=0.99$ without feedback. On the more challenging 25-qubit task, feedback and restart mechanisms enable multiple runs to reach $Q(\psi)=1.0$ within 45 oracle calls, while a budget-matched random hill-climbing baseline stalls below $Q(\psi)\approx0.29$. These results demonstrate that memory and evaluator feedback substantially improve the sample efficiency of LLM-based black-box optimization and establish quantum circuit synthesis as a challenging benchmark for test-time optimization.

Figures

Figures reproduced from arXiv: 2602.03466 by Adriano Macarone-Palmieri, Rosario Lo Franco.

Figure 1
Figure 1. Figure 1: Box (A). The training loop for the model with a prompt that does not consider the evaluation of the circuit. Box (B). The training loop with the reward. Inside the feedback, the reward in the form of a sentence you improved by this amount or a critique you made worse of this amount is provided together with the new circuit. Lower-row red boxes. An example of entangling circuit generation for 6 steps with i… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 8 linked inside Pith

  1. [1]

    Chang, X

    Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang, W. Ye, Y. Zhang, Y. Chang, P. S. Yu, Q. Yang, and X. Xie, A survey on evaluation of large language models, ACM Trans. Intell. Syst. Technol.15, 10.1145/3641289 (2024)

    doi:10.1145/3641289 2024

  2. [2]

    Shanahan, Talking about large language models, Commun

    M. Shanahan, Talking about large language models, Commun. ACM67, 68–79 (2024)

    2024

  3. [3]

    F¨ urrutter, G

    F. F¨ urrutter, G. Mu˜ noz-Gil, and H. J. Briegel, Quantum circuit synthesis with diffusion models, Nat. Mach. Intell. 6, 5 (2024)

    2024

  4. [4]

    Naveed, A

    H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian, A comprehensive overview of large language models, ACM Trans. Intell. Syst. Technol.16, 10.1145/3744746 (2025)

    doi:10.1145/3744746 2025

  5. [5]

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, Y. Du, C. Yang, Y. Chen, Z. Chen, J. Jiang, R. Ren, Y. Li, X. Tang, Z. Liu, P. Liu, J.-Y. Nie, and J.-R. Wen, A survey of large language models (2025), arXiv:2303.18223 [cs.CL]

    Pith/arXiv arXiv 2025

  6. [6]

    L. Jern, V. Uotila, C. Yu, and B. Zhao, Agent-q: Fine- tuning large language models for quantum circuit gener- ation and optimization, in2025 IEEE International Con- ference on Quantum Computing and Engineering (QCE), Vol. 01 (2025) pp. 1621–1632

    2025

  7. [7]

    Nakaji, L

    K. Nakaji, L. B. Kristensen, R. Kemmoku, J. A. Campos- Gonzalez-Angulo, M. G. Vakili, H. Huang, M. Bagher- imehrab, C. Gorgulla, F. Wong, A. McCaskey, J.-S. Kim, T. Nguyen, P. Rao, Q. Gao, M. Sugawara, N. Yamamoto, and A. Aspuru-Guzik, The generative quantum eigen- solver (gqe) and its application for ground state search (2025), arXiv:2401.09253 [quant-ph]

    arXiv 2025

  8. [8]

    Ueda and A

    K. Ueda and A. Matsuo, Optimizing ansatz design in quantum generative adversarial networks using large lan- guage models (2025), arXiv:2503.12884 [quant-ph]

    Pith/arXiv arXiv 2025

  9. [9]

    Flam-Shepherd, K

    D. Flam-Shepherd, K. Zhu, and A. Aspuru-Guzik, Lan- guage models can learn complex molecular distributions, Nat. Comm.13, 3293 (2022)

    2022

  10. [10]

    Manviet al., Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation, arXiv preprint arXiv:2410.02725 (2024)

    R. Manviet al., Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation, arXiv preprint arXiv:2410.02725 (2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Microsoft Research,Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead, Tech. Rep. (Microsoft Research, 2025)

    2025

  12. [12]

    Parashar, B

    S. Parashar, B. Olson, S. Khurana, E. Li, H. Ling, J. Caverlee, and S. Ji, Inference-time computations for llm reasoning and planning: A benchmark and insights (2025), arXiv:2502.12521 [cs.AI]

    Pith/arXiv arXiv 2025

  13. [13]

    Emergent Mind AI Research, Inference-time com- putation techniques: Overview and applica- tions,https://www.emergentmind.com/topics/ inference-time-computation(2025), accessed January 2026

    2025

  14. [14]

    Yuksekgonul, F

    M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, P. Lu, Z. Huang, C. Guestrin, and J. Zou, Optimizing gener- ative ai by backpropagating language model feedback, Nature639, 609–616 (2025)

    2025

  15. [15]

    Giordano and M

    S. Giordano and M. A. Martin-Delgado, Reinforcement- learning generation of four-qubit entangled states, Phys. Rev. Res.4, 043056 (2022)

    2022

  16. [16]

    Tashev, S

    P. Tashev, S. Petrov, F. Metz, and M. Bukov, Rein- forcement learning to disentangle multiqubit quantum states from partial observations (2024), arXiv:2406.07884 [quant-ph]

    Pith/arXiv arXiv 2024

  17. [17]

    Lockwood and M

    O. Lockwood and M. Si, Reinforcement learning with quantum variational circuit, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Dig- ital Entertainment16, 245–251 (2020)

    2020

  18. [18]

    Suzgun, M

    M. Suzgun, M. Yuksekgonul, F. Bianchi, D. Jurafsky, and J. Zou, Dynamic cheatsheet: Test-time learning with adaptive memory (2025), arXiv:2504.07952 [cs.LG]

    Pith/arXiv arXiv 2025

  19. [19]

    you did well

    for a twofold reason: it offers an experimentally accessible and compact permutation-invariant quantifier [20, 21]; it allows us to provide a simple but clear feedback to the model prompt about the general quality of the so- lution, which is sampled from the ensemble of random circuits. In this way, we exploit the same idea applied in Ref. [14], where sta...

  20. [20]

    D. A. Meyer and N. R. Wallach, Global entanglement in multiparticle systems, J. Math. Phys.43, 4273 (2002)

    2002

  21. [21]

    Boixo and A

    S. Boixo and A. Monras, Operational interpretation for global multipartite entanglement, Phys. Rev. Lett.100, 100503 (2008)

    2008

  22. [22]

    N. R. Wallach, Quantum computing and entanglement for mathematicians, inRepresentation Theory and Com- plex Analysis: Lectures given at the C.I.M.E. Summer School held in Venice, Italy June 10–17, 2004, edited by E. C. Tarabusi, A. D’Agnolo, and M. Picardello (Springer Berlin Heidelberg, Berlin, Heidelberg, 2008) pp. 345–376

    2004

  23. [23]

    Maronese, L

    M. Maronese, L. Moro, L. Rocutto, and E. Prati, Quan- tum compiling, inQuantum Computing Environments (Springer International Publishing, 2022) p. 39–74

    2022

  24. [24]

    F. J. Cardama, J. V´ azquez-P´ erez, T. F. Pena, J. C. Pichel, and A. G´ omez, Quantum compilation process: A survey, inEuro-Par 2024: Parallel Processing Work- shops, edited by S. Caino-Lores, D. Zeinalipour, T. D. Doudali, D. E. Singh, G. E. M. Garz´ on, L. Sousa, D. An- drade, T. Cucinotta, D. D’Ambrosio, P. Diehl, M. F. Dolz, A. Jukan, R. Montella, M....

    2024

  25. [25]

    L. Moro, M. G. A. Paris, M. Restelli, and E. Prati, Quan- tum compiling by deep reinforcement learning, Commun. Phys.4, 178 (2021)

    2021

  26. [26]

    S. C. Hoi, D. Sahoo, J. Lu, and P. Zhao, Online learn- ing: A comprehensive survey, Neurocomputing459, 249 (2021)

    2021

  27. [27]

    R. S. Sutton and A. G. Barto,Reinforcement Learning: An Introduction, 2nd ed. (The MIT Press, 2018)

    2018

  28. [28]

    Poulin, Stabilizer formalism for operator quantum er- ror correction, Phys

    D. Poulin, Stabilizer formalism for operator quantum er- ror correction, Phys. Rev. Lett.95, 230504 (2005)

    2005

  29. [29]

    Suter and G

    D. Suter and G. A. ´Alvarez, Colloquium: Protecting quantum information against environmental noise, Rev. Mod. Phys.88, 041001 (2016). 7

    2016

  30. [30]

    M. V. den Nest, J. Dehaene, and B. D. Moor, Efficient algorithm to recognize the local clifford equivalence of graph states, Phys. Rev. A70, 034302 (2004)

    2004

  31. [31]

    M. V. den Nest, J. Dehaene, and B. D. Moor, Local uni- tary versus local clifford equivalence of stabilizer states, Phys. Rev. A71, 062323 (2005)

    2005

  32. [32]

    Brown, J

    B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. R´ e, and A. Mirhoseini, Large language monkeys: Scal- ing inference compute with repeated sampling (2024), arXiv:2407.21787 [cs.LG]

    Pith/arXiv arXiv 2024

  33. [33]

    plan-then-solve

    T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jer- nite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, Huggingface’s transform- ers: State-of-the-art natural language processing (2020), arXiv:1910.03771 [cs.CL]. V. APP...

    Pith/arXiv arXiv 2020