Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant

T0 review · 2 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Post-training a 14B Qiskit code model with rewards from executing unit tests on quantum systems lets it beat open-source models up to 30 times larger on Qiskit-HumanEval-hard.

desk verdict Solid engineering recipe for execution-based RL on Qiskit code; the QHE-hard win is within sampling noise, but the pipeline and ablations merit referee time. read the letter →

arxiv 2508.20907 v1 pith:YNJOHVF2 submitted 2025-08-28 quant-ph cs.AI

Nicolas Dupuis , Adarsh Tiwari , Youssef Mroueh , David Kremer , Ismael Faro , Juan Cruz-Benito This is my paper · ORCID
classification quant-phcs.AI
keywords Qiskitquantumcodegenerationpost-trainingLLMsDPOGRPOverifiablerewardsQiskit-HumanEval-hardhardwareexecution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that large language models writing Qiskit quantum code improve most when their training signal comes from actually executing the generated code on quantum simulators or hardware. The authors build a synthetic pipeline that creates quantum problems paired with unit tests, then use the pass rate of those tests as a reward in both offline preference optimization (DPO) and online reinforcement learning (GRPO). Their best 14B-parameter model, combining DPO and GRPO, reaches the highest score among all evaluated models on the harder Qiskit-HumanEval benchmark, surpassing far larger open-source models. The significance is that hardware-grounded, execution-based rewards may be a practical way to specialize language models for domains where correctness is only knowable by running code.

What carries the argument

The load-bearing mechanism is the quantum verifiable reward: Definition 1 in the paper, defined as the percentage of unit tests that pass when generated code runs on quantum hardware or a quantum simulator. Around it sits the synthetic data pipeline that manufactures prompt/unit-test pairs from templates covering Qiskit features such as estimator and sampler primitives, transpilation, and error correction. The pipeline feeds two training paths: DPO uses accepted/rejected code pairs selected so rejected samples are cosinesimilar to accepted ones, while GRPO uses grouped rollouts whose normalized advantage is computed from the quantum verifiable reward plus a small formatting reward.

What would settle it

Take the DPO-GRPO model and evaluate it on a held-out set of human-written Qiskit problems whose unit tests are not generated by the paper's template pipeline; if its pass-rate advantage over Qwen2.5-Coder-14B-Instruct shrinks toward zero or reverses, the claim that quantum-verifiable rewards generalize beyond synthetic prompts is refuted.

Watch

Extended reading notes

Core claim

The central claim is that a quantum verifiable reward—the percentage of unit tests passed when generated Qiskit code is executed on a quantum processing unit or a realistic simulator—can serve as a direct post-training signal for LLMs. Using a synthetic data pipeline that generates prompt/unit-test pairs and validates code in a sandboxed Qiskit environment, the authors create accepted/rejected pairs for DPO and an online GRPO reward computed from unit-test pass rates. The best model, Qwen2.5-Coder-14B-Qiskit-Instruct-DPO-GRPO, achieves 28.48% on Qiskit-HumanEval-hard with greedy decoding, the highest among all evaluated models, including open-source baselines up to 30 times larger. The paper

Load-bearing premise

The 4.5k synthetic prompt/unit-test pairs used for GRPO are representative of the human-written Qiskit-HumanEval-hard benchmark, so maximizing unit-test pass rate on them produces genuine improvement rather than template-specific overfitting.

Editorial extensions

If this is right

  • A 14B-parameter specialized model can outperform open-source models orders of magnitude larger on a hard domain benchmark when post-trained with execution-based quantum rewards.
  • DPO and GRPO are complementary: DPO improves one-sample pass rates, while GRPO teaches the model to supply missing imports and produce executable code through self-play, which is essential on Qiskit-HumanEval-hard.
  • Quantum verification can be automated through a synthetic template pipeline, making hardware-in-the-loop training feasible without large human-annotated preference datasets.
  • The Qiskit variants retain competitive scores on standard HumanEval, suggesting domain specialization through quantum-verified post-training does not destroy general coding ability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 4.5k synthetic GRPO prompts are not representative of human-written Qiskit problems, the reported 28.48% gain may partly reflect template-specific overfitting; a held-out test on non-synthetic problems would separate genuine improvement from memorization.
  • The reward only measures unit-test pass rate, not circuit depth, gate count, or hardware efficiency, so extending the reward to include quality or cost metrics could push the model beyond mere executability and toward optimized quantum programs.
  • Because about 90% of the synthetic problems use simulators, the approach is scalable today; shifting more of the reward computation to real QPU execution could reveal whether simulator-verified rewards fully transfer to noisy hardware.
  • The same offline-plus-online recipe—synthetic prompt/test generation, DPO on hard negatives, GRPO on execution rewards—may transfer to other scientific coding domains where correctness is determined by running code, such as numerical simulation or instrument control.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a post-training pipeline for a Qiskit code assistant built on Qwen2.5-Coder-14B. The pipeline consists of extended pretraining on Qiskit code, weight merging with an instruct model, SFT with distilled quantum-physics explanations, and then DPO and/or GRPO using a 'quantum verifiable reward' defined as the unit-test pass rate on quantum simulators or QPUs. A synthetic prompt/unit-test generation pipeline supports both offline preference data and online GRPO rollouts. The authors evaluate on Qiskit-HumanEval (QHE) and Qiskit-HumanEval-hard (QHE-hard). The central reported result is that Qwen2.5-Coder-14B-Qiskit-Instruct-DPO-GRPO reaches 28.48% pass@1 on QHE-hard with greedy decoding, which the paper claims surpasses all evaluated open-source baselines, including models up to 30x larger.

Significance. If the headline result is statistically reliable, the paper makes a useful and timely contribution: it demonstrates a concrete recipe for using execution-based verifiable rewards in a specialized quantum-programming domain, provides a synthetic data pipeline, and ablates SFT, DPO, and GRPO. The experimental design is well structured, and the use of Qiskit-HumanEval as a benchmark is appropriate. The paper is not circular in the logical sense: using unit-test pass rate both as a training reward and as an evaluation metric is standard practice in code-generation RL. The main weaknesses are that the headline margin on QHE-hard is small relative to the finite-benchmark sampling error, and that transfer from the synthetic GRPO training distribution to the human-curated benchmark is not analyzed.

major comments (2)
  1. [Section 4, Table 2] The central claim that DPO-GRPO 'attains the highest score' on QHE-hard rests on a 1.33 percentage point margin over Intern-S1 (28.48% vs 27.15%) with n=151 problems and greedy decoding. The standard error of each proportion is about 3.7 points, and the standard error of the difference is about 5.2 points; the observed margin is therefore well under 0.3 SE. No confidence intervals, bootstrap, or significance tests are reported for Table 2, so the headline comparison is statistically indistinguishable from a tie. Please report uncertainty for all Table 2 entries, e.g., Wilson intervals and a McNemar or bootstrap test for the DPO-GRPO vs Intern-S1 comparison, and ideally repeat greedy evaluation with different seeds or report pass@k with error bars to support the 'surpasses' claim.
  2. [Sections 3.3 and 4] The GRPO reward is optimized on 4.5k synthetic prompt/unit-test pairs generated by the same pipeline used to create training data, while QHE/QHE-hard are Qiskit-curated benchmarks covering similar API and task families. The paper does not quantify the overlap or distribution shift between the synthetic training prompts and the benchmark, nor does it evaluate on an independent benchmark such as QHackBench. Without an overlap/leakage analysis or an out-of-distribution evaluation, the reported QHE-hard gain could reflect specialization to the synthetic template distribution rather than a general improvement in executable Qiskit code generation. Please add a quantitative overlap analysis and at least one held-out benchmark.
minor comments (5)
  1. [Abstract/Introduction, Section 2.2] The abstract and introduction emphasize rewards 'provided by the quantum hardware,' but Section 4 states that the QHE/QHE-hard evaluation uses only simulators, and Section 2.2 says about 10% of training problems require runtime access. Please qualify the wording to 'quantum hardware or simulator' throughout, or report the exact fraction of examples actually run on QPUs.
  2. [Section 4, Figure 5] The text says the figure plots only mean values and points to Appendix C for variance. Please show uncertainty in the main figure as well, since several qualitative claims ('GRPO models outperform both instruct and DPO models') are made from mean curves without statistical tests.
  3. [Section 3.3] The DPO negative-selection heuristic (choosing the rejected sample with maximum embedding cosine similarity to the accepted one) is interesting but no analysis is provided of how sensitive final results are to this choice or to the embedding model. A short sensitivity study or at least a motivation with examples would help.
  4. [Section 4, first paragraph] Typo: 'a wild range of tasks' should be 'a wide range of tasks.'
  5. [Appendix D] The qualitative note that 'the SFT model provides a more detailed explanation' is subjective and not tied to any metric. If retained, it should be framed as an observation, not a result.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central result is an empirical outcome of training on synthetic unit-test rewards and evaluating on a fixed, externally runnable benchmark; reward/eval alignment is standard RL, not a logical circle.

full rationale

The claimed derivation chain is: synthetic prompt/unit-test pairs are generated (Section 2.2), DPO/GRPO training optimizes a reward defined as unit-test pass rate (Definition 1, Sections 2.3/3.3), and the resulting models are scored by pass@k on QHE and QHE-hard (Section 4). The reward and the evaluation metric are related, but they are not the same object: the reward is computed on synthetic prompts generated by the authors' own template pipeline, while the benchmark is a fixed public dataset (Qiskit 2024, 2025; Vishwakarma et al. 2024) with its own problems and unit tests. Maximizing pass rate on the synthetic distribution is not equivalent by construction to a high score on the benchmark; whether it transfers is an empirical question answered by Table 2. No equation or definition in the paper reduces a claimed prediction to the training objective. The appendix even documents a case where the quantum-verification reward assigned 0.8 to code whose ZNE configuration was wrong, showing the reward is a noisy proxy rather than a tautological target. The only self-reference is that QHE/QHE-hard were co-created by members of the same group (Vishwakarma et al. 2024 includes several of the present authors). This is a mild self-citation, but it is not load-bearing circularity: the benchmark is public, externally runnable, and not fitted to the present models; the paper does not tune hyperparameters on the benchmark, and the benchmark would give the same numbers to any independent evaluator. The skeptical concern about statistical significance (1.33-point margin vs. Intern-S1 at n=151, no error bars) is a correctness/robustness issue, not a circularity issue. Overall, the derivation is self-contained against an external benchmark, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the reward definition (unit test pass rate), the synthetic data distribution, and the simulator/hardware equivalence. Training hyperparameters are numerous; the most consequential are listed. No new physical entities are introduced.

free parameters (5)
  • Reward weight ratio (quantum verification : format) = 1.0 : 0.1
    Hand-chosen in GRPO (Section 3.3); affects the strength of the unit-test reward relative to format compliance.
  • DPO beta = 0.2
    Temperature for preference sharpness in DPO training (Section 3.3).
  • GRPO KL regularizer beta = 0.01
    KL penalty to control deviation from reference policy (Section 3.3).
  • GRPO group size G = 32
    Number of rollouts per prompt in GRPO (Section 3.3).
  • LoRA rank / scaling factor = 64 / 128
    Used in extended pretraining and DPO (Sections 3.1, 3.3); determines adapter capacity.
assumptions (4)
  • domain assumption Unit test pass rate on quantum simulators/hardware is a valid reward signal for code correctness and executability.
    Definition 1 (Section 2.3) defines the quantum verifiable reward as the percentage of unit tests passed; the entire DPO/GRPO training relies on this proxy.
  • domain assumption Synthetic prompt/unit-test pairs generated by templates cover the distribution of real Qiskit tasks, including QHE-hard.
    Stated in Section 2.2: 'templates that cover essential Qiskit features'; this is the transfer assumption behind the benchmark gains.
  • domain assumption Simulator execution (AerSimulator, fake backends) is a faithful proxy for QPU execution for the purpose of verification.
    About 10% of training problems use the runtime (Section 2.2); the QHE evaluation uses only simulators (Section 4), yet the framing claims quantum hardware feedback.
  • domain assumption The QHE/QHE-hard benchmark is a valid measure of Qiskit code assistant quality.
    The benchmark was co-created by the same research group (Vishwakarma et al. 2024 includes Dupuis, Kremer, Faro, Cruz-Benito); treated as ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant." pith.science (2026). https://pith.science/paper/YNJOHVF2

@misc{pith2026250820907,
  author       = {Pith},
  title        = {Pith review of: Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YNJOHVF2}},
  note         = {Machine review of arXiv:2508.20907}
}
read the original abstract

Qiskit is an open-source quantum computing framework that allows users to design, simulate, and run quantum circuits on real quantum hardware. We explore post-training techniques for LLMs to assist in writing Qiskit code. We introduce quantum verification as an effective method for ensuring code quality and executability on quantum hardware. To support this, we developed a synthetic data pipeline that generates quantum problem-unit test pairs and used it to create preference data for aligning LLMs with DPO. Additionally, we trained models using GRPO, leveraging quantum-verifiable rewards provided by the quantum hardware. Our best-performing model, combining DPO and GRPO, surpasses the strongest open-source baselines on the challenging Qiskit-HumanEval-hard benchmark.

Figures

Figures reproduced from arXiv: 2508.20907 by the authors.

Figure 1
Figure 1. (–left) Example of a Qiskit program; (–right) Simplified interaction workflow between the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Generation pipeline for synthetic data. 3 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Training LLM through reinforcement learning from quantum verifiable rewards. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Training flowchart for the Qiskit Code Assistant. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: pass@k results on Qiskit-HumanEval and Qiskit-HumanEval-hard from k = 1 to k = 64. For clarity, we plot only the mean values here, see appendix C for the variance. ated models. We also confirmed that our Qiskit models maintain performance on HumanEval[Chen et al., 2021…
Figure 6
Figure 6. Figure 6: pass@k results on Qiskit-HumanEval and Qiskit-HumanEval-hard from k = 1 to k = 64. The shaded area represents µ ± σ. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Aligning Quantum Operators with Large Language Models

    quant-ph 2026-06 conditional novelty 6.0 of 10

    An LLM that reads a quantum operator as image-like patches can synthesize 4-qubit Pauli-rotation circuits at high success and obey English gate constraints.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [2]

    Create the random circuit with 8 qubits and depth 1, setting the seed to 1 and disabling measurement

    work page

  2. [3]

    Generate the preset staged pass manager for the FakeKyiv backend with optimization level 1

    work page

  3. [4]

    Use the pass manager to transpile the circuit. </think> <answer> ‘‘‘python from qiskit.circuit.random import random_circuit from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager from qiskit_ibm_runtime.fake_provider import FakeKyiv # Step 1: Create the random circuit rand_circ = random_circuit(num_qubits=8, depth=1, seed=1, measur...

    work page

  4. [5]

    This means the circuit will have 8 quantum registers

    **Constructing the Random Circuit**: - We need to create a quantum circuit with 8 qubits. This means the circuit will have 8 quantum registers. - The depth of the circuit is set to 1, which implies that the circuit will consist of a single layer of quantum gates acting on the qubits. This typically means that each qubit will have at most one gate applied ...

    work page

  5. [6]

    This is crucial for adapting the circuit to the specific constraints and capabilities of a quantum backend

    **Generating the Preset Staged Pass Manager**: - A staged pass manager is a sequence of optimization and transformation steps that are applied to a quantum circuit during the transpilation process. This is crucial for adapting the circuit to the specific constraints and capabilities of a quantum backend. - The ‘FakeKyiv‘ backend refers to a specific quant...

    work page

  6. [7]

    XXYYZ",

    **Transpiling the Circuit**: - Transpilation is the process of converting a quantum circuit into a form that is compatible with the target backend. This involves mapping the circuitâ˘A´Zs gates to the backendâ˘A´Zs native gate set, optimizing the circuit for efficiency, and ensuring that the circuit respects the backendâ˘A´Zs physical constraints (e.g., q...

    work page

  7. [8]

    It is used here to perform the quantum computations in a controlled environment

    **Backend Initialization**: The backend ‘FakeLimaV2‘ is a simulated quantum device that mimics the behavior of a real quantum computer. It is used here to perform the quantum computations in a controlled environment

    work page

  8. [9]

    This circuit will be the quantum system on which we will apply operations and measure observables

    **Quantum Circuit Creation**: We need to create a quantum circuit ‘qc‘ with 5 qubits. This circuit will be the quantum system on which we will apply operations and measure observables

    work page

Show all 33 references
  1. [10]

    XXYYZ",

    **Observable Definition**: The observables are defined using the ‘SparsePauliOp‘ class, which represents operators in the Pauli basis. We are given three Pauli strings: "XXYYZ", "XYZZI", and "XXZZZ". These strings correspond to the Pauli matrices (I, X, Y, Z) acting on the res...

  2. [11]

    This involves running the circuit and measuring the observables

    **Estimator Execution**: The estimator will compute the expectation values of the defined observables with respect to the quantum state produced by the circuit. This involves running the circuit and measuring the observables. The results will be stored in a job, which we will ...

  3. [12]

    XXYYZ",

    **Reasoning Process**: - The ‘SparsePauliOp‘ is a way to represent Hamiltonians or observables in terms of Pauli operators, which are fundamental in quantum mechanics. The Pauli strings "XXYYZ", "XYZZI", and "XXZZZ" are specific combinations of Pauli-X and Pauli-Z operators ac...

  4. [13]

    XXYYZ",

    **Final Output**: The ‘job‘ will contain the results of the estimation, including the expectation values of the observables. These results are crucial for understanding the behavior of the quantum system described by the circuit. </think> <answer> ‘‘‘python from qiskit_ibm_run...

  5. [15]

    Initialize the estimator with the ‘ibm_fez‘ backend

  6. [16]

    polynomial_degree_2

    Configure the resilience options of the estimator to enable ZNE and set the specified parameters. Let’s go through each step in detail. </think> <answer> ‘‘‘python from qiskit_ibm_runtime import EstimatorV2 as Estimator from qiskit_ibm_runtime import QiskitRuntimeService # Ste...

  7. [17]

    Import the necessary modules from Qiskit

  8. [18]

    Create a quantum circuit to represent the GHZ state

  9. [19]

    Apply the necessary quantum gates to prepare the GHZ state

  10. [20]

    Let’s go through these steps in detail

    Convert the quantum circuit to a QASM3 string. Let’s go through these steps in detail. </think> <answer> Here’s how you can define a GHZ state in Qiskit and convert it into a QASM3 string: ‘‘‘python from qiskit import QuantumCircuit from qiskit.circuit.library import HGate, CX...

  11. [21]

    **Define the GHZ State**: A GHZ state is a quantum state that is entangled across multiple qubits. For \( n \) qubits, the GHZ state is given by: \[ |\text{GHZ}\rangle = \frac{1}{\sqrt{2}}(|0\rangle^{\otimes n} + |1\rangle^{\otimes n}) \] This state is created by applying a Ha...

  12. [22]

    The circuit will consist of a Hadamard gate on the first qubit and CNOT gates between the first qubit and each of the other qubits

    **Implement the GHZ State in Qiskit**: We will use Qiskit’s quantum circuit framework to create a circuit that prepares the GHZ state. The circuit will consist of a Hadamard gate on the first qubit and CNOT gates between the first qubit and each of the other qubits

  13. [23]

    QASM3 is a quantum assembly language that describes quantum circuits in a standardized format

    **Convert the Circuit to QASM3**: Once the circuit is defined, we can use Qiskit’s functionality to convert the circuit into a QASM3 string. QASM3 is a quantum assembly language that describes quantum circuits in a standardized format. Let’s go through the implementation step-...

  14. [24]

    Import necessary modules from Qiskit

  15. [25]

    Use the IBM Quantum provider to get a list of backends

  16. [26]

    Filter the backends to select only those with more than 5 qubits that are operational

  17. [27]

    Identify the least busy backend from the filtered list

  18. [28]

    Create a Bell circuit

  19. [29]

    Transpile the Bell circuit

  20. [30]

    Use the sampler primitive to execute the circuit on the selected backend with a specified number of shots. </think> <answer> ‘‘‘python from qiskit_ibm_runtime import QiskitRuntimeService, SamplerV2 as Sampler from qiskit import QuantumCircuit from qiskit.transpiler.preset_pass...

  21. [31]

    This involves querying the IBM Quantum service for backend information and filtering based on the number of qubits and operational status

    **Filter IBM Quantum Backends**: We need to filter the available IBM Quantum backends to select only those with more than 5 qubits that are operational. This involves querying the IBM Quantum service for backend information and filtering based on the number of qubits and opera...

  22. [32]

    This involves checking the current usage or queued jobs of each backend to determine which one is least occupied

    **Select the Least Busy Backend**: From the filtered list, we need to select the backend that is currently the least busy. This involves checking the current usage or queued jobs of each backend to determine which one is least occupied

  23. [33]

    This involves preparing a Bell state, which is a superposition of the basis states \(|\psi\rangle = \frac{1}{\sqrt{2}}(|00\rangle + |11\rangle)\)

    **Create a Bell Circuit**: A Bell circuit is a simple quantum circuit that creates entanglement between two qubits. This involves preparing a Bell state, which is a superposition of the basis states \(|\psi\rangle = \frac{1}{\sqrt{2}}(|00\rangle + |11\rangle)\)

  24. [34]

    This includes mapping the logical qubits to physical qubits and optimizing the circuit for the backend’s constraints

    **Transpile the Circuit**: Transpilation is the process of converting a quantum circuit into a form that is compatible with the specific backend. This includes mapping the logical qubits to physical qubits and optimizing the circuit for the backend’s constraints

  25. [35]

    This involves running the circuit on the selected backend and obtaining the measurement results

    **Execute the Circuit Using the Sampler Primitive**: The sampler primitive is a tool for sampling from the outcome distribution of a quantum circuit. This involves running the circuit on the selected backend and obtaining the measurement results. </think> <answer> ‘‘‘python fr...

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.