Introduces QASM-Eval, the first dataset targeting OpenQASM-3 hardware-facing features for LLM training and evaluation, with an extended verifier for syntax, states, and timelines.
A PennyLane-Centric Dataset to Enhance LLM-based Quantum Code Generation using RAG
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
Large Language Models (LLMs) offer powerful capabilities in code generation, natural language understanding, and domain-specific reasoning. Their application to quantum software development remains limited, in part because of the lack of high-quality datasets both for LLM training and as dependable knowledge sources. To bridge this gap, we introduce \textit{PennyLang}, an off-the-shelf, high-quality dataset of 3,347 PennyLane-specific quantum code samples with contextual descriptions, curated from textbooks, official documentation, and open-source repositories. Our contributions are threefold: (1) the creation and open-source release of PennyLang, a purpose-built dataset for quantum programming with PennyLane; (2) a framework for automated quantum code dataset construction that systematizes curation, annotation, and formatting to maximize downstream LLM usability; and (3) a baseline evaluation of the dataset across multiple open-source and commercial models, including ablation studies, all conducted within a retrieval-augmented generation (RAG) pipeline. Using PennyLang with RAG substantially improves performance: for example, Qwen 7B's success rate rises from 8.7% without retrieval to 41.7% with full-context augmentation, and LLaMa 4 improves from 78.8% to 84.8%, while also reducing hallucinations and enhancing quantum code correctness. Moving beyond Qiskit-focused studies, we bring LLM-based tools and reproducible methods to PennyLane for advancing AI-assisted quantum development.
citation-role summary
background 1
citation-polarity summary
years
2026 7roles
background 1polarities
background 1representative citing papers
A taxonomy-guided RAG system with LLMs reduces hallucinations and improves migration suggestions for Qiskit code compared to unconstrained retrieval.
Adapts QuantumKatas to Qiskit yielding a 350-task benchmark across 26 categories and evaluates 16 LLMs in 39,200 runs, reporting performance gaps and prompting effects.
Retrieval over a 13,389-example verified PennyLane corpus raises QHack pass@5 from 36/43/24% to 64/68/52% across 2022–2024 with Claude Sonnet 4.6.
Supervised fine-tuning on gate-by-gate quantum simulation traces allows LLMs to achieve near-perfect accuracy in predicting quantum measurement outcomes, with added GRPO improving generalization to larger qubit counts.
Iterative refinement boosts LLM success in generating quantum solvers that match classical results, but more advanced models shift from execution errors to hard-to-detect numerical inaccuracies.
A systematic literature review maps trends in automated approaches to quantum software engineering and quantum AI, highlighting their role in hybrid quantum-classical systems.
citing papers explorer
-
QASM-Eval: A Dataset to Train and Evaluate LLMs on OpenQASM-3 Beyond Quantum Circuits
Introduces QASM-Eval, the first dataset targeting OpenQASM-3 hardware-facing features for LLM training and evaluation, with an extended verifier for syntax, states, and timelines.
-
Qiskit Code Migration with LLMs
A taxonomy-guided RAG system with LLMs reduces hallucinations and improves migration suggestions for Qiskit code compared to unconstrained retrieval.
-
Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluation
Adapts QuantumKatas to Qiskit yielding a 350-task benchmark across 26 categories and evaluates 16 LLMs in 39,200 runs, reporting performance gaps and prompting effects.
-
PennySynth: RAG-Driven Data Synthesis for Automated Quantum Code Generation
Retrieval over a 13,389-example verified PennyLane corpus raises QHack pass@5 from 36/43/24% to 64/68/52% across 2022–2024 with Claude Sonnet 4.6.
-
Fine-Tuning Large Language Models for Quantum Reasoning
Supervised fine-tuning on gate-by-gate quantum simulation traces allows LLMs to achieve near-perfect accuracy in predicting quantum measurement outcomes, with added GRPO improving generalization to larger qubit counts.
-
Can LLMs Solve Science or Just Write Code? Evaluating Quantum Solver Generation
Iterative refinement boosts LLM success in generating quantum solvers that match classical results, but more advanced models shift from execution errors to hard-to-detect numerical inaccuracies.
-
Automated Quantum Software and AI Engineering
A systematic literature review maps trends in automated approaches to quantum software engineering and quantum AI, highlighting their role in hybrid quantum-classical systems.