Catching Transpilation Drift with a CI/CD Workflow in Quantum Software Development

Jouni Peltonen; Otso Kinanen; Vlad Stirbu

REVIEW 3 major objections 5 minor 10 references

Catching Transpilation Drift with a CI/CD Workflow in Quantum Software Development

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A CI/CD workflow that compares transpiled circuit structure against a stored baseline can detect toolchain-induced changes before hardware execution.

desk verdict Useful DevOps guardrail for quantum software, but the demonstrated drift numbers don't reconcile and the metric misses layout-only changes. read the letter →

arxiv 2608.08248 v1 pith:YDCSWKUL submitted 2026-08-08 quant-ph

Jouni Peltonen , Otso Kinanen , Vlad Stirbu This is my paper · ORCID

classification quant-ph

keywords quantumsoftwareengineeringDevOpscontinuousintegrationdeliverytranspilationdriftreproducibility

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Quantum programs are rarely executed in the form developers write them; compiler and provider toolchains transform each circuit to fit a specific backend. The paper argues that this transformation can silently change when SDK versions, optimization settings, coupling maps, or backend descriptions change, and that such 'transpilation drift' is a regression signal that most CI/CD pipelines ignore. It proposes a lightweight workflow that transpiles source circuits against configured target profiles, compares structural metrics (depth, total gates, two-qubit gates) with an approved baseline, and converts the comparison into pass/warn/fail feedback before hardware submission. If adopted, teams would see a CI warning whenever a dependency update or configuration change alters the circuit that would actually run, protecting reproducibility and avoiding wasted hardware time.

What carries the argument

The central mechanism is the structural drift score $D_{\mathrm{struct}}$, the mean relative change between a baseline run and a current run over three count-valued features: circuit depth, total gate count, and two-qubit gate count, with a small floor $\epsilon=1$ to keep the ratio stable. The score is deliberately not a formal circuit distance; it is a lightweight CI signal. It is paired with an experiment-tracking service (MLflow) that stores transpiled circuits, target descriptions, metric JSON, and provenance parameters, and with a threshold policy that converts $D_{\mathrm{struct}}$ into pass, warn, or fail. The named phenomenon being measured is 'transpilation drift'—a change in the compiled realization of an unchanged source circuit caused by toolchain, SDK, or target-profile evolution.

What would settle it

Take a fixed source circuit and two transpiler settings that produce identical depth, total gate count, and two-qubit gate count but different qubit layouts or gate orderings, and run the workflow; it would report $D_{\mathrm{struct}}=0$ and pass, even though the realized circuit changed, showing that the structural proxy misses a meaningful realization change.

Watch

Extended reading notes

Core claim

The central claim is that the transpiled circuit—not just the source—should be treated as a monitored build artifact in quantum software development. The workflow transpiles every source circuit against each configured target profile, records depth, total gate count, and two-qubit gate count, compares them to an approved baseline with the mean-relative-change score, and maps the result to pass/warn/fail thresholds. Two demonstrations show realistic drift: leaving the transpiler seed unset produces different routings on connectivity-constrained targets, and switching from a vendor transpiler to a newer SDK's built-in transpiler changes depth and gate composition for a 10-qubit Fourier transform. In both cases the source circuit is unchanged, yet the compiled realization changes; the workflow flags this before hardware execution and logs artifacts and provenance for later audit.

Load-bearing premise

The load-bearing premise is that changes in depth, total gate count, and two-qubit gate count faithfully capture every transpilation change that matters, so a compiler update that only re-routes or rearranges gates would slip past with a drift score of zero.

Editorial extensions

If this is right

A team with an approved baseline will see a warning or failing CI status whenever a dependency or transpiler update changes the realized circuit for any configured target, before any hardware time is spent.
Storing transpiled circuits, target descriptions, metrics, and provenance in an experiment tracker turns each CI run into an auditable record that can be replayed or compared later, supporting reproducibility audits.
The pass/warn/fail thresholds provide a configurable release gate: intentional changes are absorbed by updating the baseline, while accidental changes stop the pipeline.
Because the check is pre-execution and fast, it fits pull-request workflows and can be supplemented by optional scheduled or release-gate execution-level checks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the workflow records qubit layout separately rather than in the drift score, so a natural extension would be to gate the score on layout equality or fold layout changes in—layout differences also change noise exposure and execution cost.
Beyond the paper: the same baseline comparison could be reused as a transpiler-regression probe, running a small circuit suite across a matrix of SDK versions and target profiles to map which updates introduce drift before they reach a project's main branch.
Beyond the paper: threshold calibration is left open, so a plausible next step is fitting per-circuit-family thresholds from historical drift distributions, letting warnings reflect changes that are unusual for the circuit rather than any nonzero change.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know before you read this one. First, it's a practical, well-scoped design-science paper: it gives quantum software teams a way to make the transpiled circuit a tracked, CI-checked artifact, and it ships open-source demo code. Second, the demonstration numbers don't add up, and the drift metric has a blind spot for layout-only changes that the paper's own opening claim seems to promise it catches.

What's legitimately new here: prior work benchmarked compiler effects or tracked experiments, but this operationalizes 'transpilation drift' as a baseline-relative CI gate using a simple mean-relative-change score over depth, gate count, and two-qubit count. The workflow is clear: transpile against a target profile, compare to a stored baseline, log to MLflow, and emit pass/warn/fail in GitHub Actions. The authors are honest that thresholds are illustrative and that drift is not proof of semantic difference. The demonstration with seed drift and transpiler drift is a genuine illustration of the problem, and the code link is there.

Now the soft spots. The numbers don't reconcile. Table I reports Delta 2Q = -10 for Scenario B, but Table II shows 173 to 172 (Delta -1). And Eq. (1) with Table II's metrics gives a mean relative change around 0.17, not the stated 0.1295. That's sloppy, and it undercuts the paper's usefulness as a reference until fixed.

More substantively, the CI status is driven only by D_struct, which uses depth, gates, and two-qubit counts. Layout is logged and 'flagged,' but no rule routes a layout-only change into warn or fail. So a transpiler update that reroutes or remaps qubits while preserving those three counts yields D_struct = 0 and a green run. The stress-test note is right: this misses a material class of drift, and it's realistic on coupling maps with symmetries. The authors call the score a structural proxy in Section VI but don't address the gate. This is fixable — include layout in the score or make layout change a separate warn/fail condition — but right now the central claim is stronger than the demonstrated signal.

That said, the contribution is a workflow, not a physics result, and the workflow's core idea holds up. The circularity worry is minor: a change detector measures its own metrics by design; the real inadequacy is the layout blind spot.

Verdict: this deserves a serious referee. A moderate revision should fix the numbers and close the layout gap. Teams doing applied quantum software work will get practical value from it.

Referee Report

3 major / 5 minor

Summary. The paper proposes a CI/CD workflow for detecting transpilation drift in quantum software development. The workflow transpiles source circuits against configured target profiles, computes a structural feature vector F = ⟨depth, gates, twoQ⟩, compares the result against an MLflow-stored baseline using a drift score D_struct (Eq. 1), and raises pass/warn/fail feedback through GitHub Actions. Qubit layout is recorded and flagged separately but is not included in the drift score. The demonstration covers two scenarios: seed drift (Scenario A, BV12 circuit on three targets) and transpiler drift (Scenario B, QFT10 on an IQM target, switching from iqm-client's transpiler to Qiskit's built-in transpiler). The paper positions the contribution as a practical guardrail for making compilation behavior observable and auditable, rather than a formal equivalence checker.

Significance. If the workflow performs as claimed, it addresses a real and under-served gap: toolchain-induced changes to the transpiled circuit are typically invisible in CI, yet they affect reproducibility, cost, and comparability of results. The paper provides a clean workflow architecture, MLflow-based provenance tracking, a public demonstration repository, and a plausible end-to-end scenario. These are concrete strengths. The central limitation is that the demonstrated drift signal only tracks three aggregate metrics, and the numerical evidence in the demonstration contains inconsistencies that currently prevent the reader from verifying the core quantitative claim. The contribution is incremental but useful for the quantum software engineering community, and the workflow is easily adoptable.

major comments (3)

[Section V, Tables I and II] The reported drift numbers for Scenario B are internally inconsistent. Table I reports Δ2Q = -10 for QFT10 on IQM, while Table II shows 2-qubit gates 173 vs 172, i.e., Δ2Q = -1. Moreover, the reported D_struct = 0.1295 does not reproduce from Eq. (1) using the metrics in Table II: taking depth (213→253), total ops (453→600), and two-qubit gates (173→172) gives (40/213 + 147/453 + 1/173)/3 ≈ 0.173, and no plausible subset of the listed metrics yields 0.1295. Because the demonstration is the primary evidence that the workflow detects drift, this inconsistency must be corrected and the exact computation of D_struct for this example must be shown.
[Section IV, Eq. (1) and status rule] The CI pass/warn/fail status is defined solely as a threshold on D_struct, which is computed only from depth, total gates, and two-qubit gates. Qubit layout is recorded and "flagged" separately, but no rule in Sections IV or V routes a layout-only change into warn or fail. Consequently, a transpiler update or seed change that remaps qubits or reroutes the circuit while preserving these three aggregates yields D_struct = 0 and a green CI run, even though the circuit actually executed has changed. This directly contradicts the paper's opening claim that the workflow detects transpilation drift before execution. The authors should either (a) include layout distance in D_struct, (b) add a separate status rule that treats layout changes as warn/fail, or (c) explicitly narrow the claim to "drift in depth and gate counts" rather than transpilation drift in general. Section VI's statement that the score is a structural proxy does not resolve this specific false-negative class.
[Section IV, Eq. (1)] The metric denoted "gates" in the feature vector F is not uniquely defined. Section III mentions "depth, total gates, two-qubit gates", but Table II lists "Total ops", "R gates", "CZ gates", and "RZ gates" separately, and the relationship between "total gates" and "Total ops" is not explained. Without a precise definition (e.g., the sum of Qiskit's count_ops entries excluding barriers or measurements), the drift score is not reproducible from the paper's data. The authors should specify the exact Qiskit property used for "gates" and verify that it matches the values in Table II.

minor comments (5)

[Abstract] The phrase "Suchtranspilation drift" is missing a space; it should read "Such transpilation drift".
[Section IV, status equation] The threshold symbols τ_warn and τ_fail are used in the status equation without being defined in the text; they appear only in Figure 1. Please define them explicitly in Section IV where the equation is introduced.
[Figure 1] The labels "< τ_w" and "≥ τ_f" in the threshold-policy box are visually clipped and hard to read. Consider redrawing the box with clearer spacing and explicit threshold names.
[Section V, Scenario B text] The text says "two-qubit count is essentially unchanged" while Table I reports Δ2Q = -10; the wording is consistent with Table II's -1 but not with Table I. Harmonize all reported deltas after correcting the inconsistency noted in the major comments.
[Reference [4]] The URL for reference [4] contains a line-break artifact ("https://doi.org/10.2753/MIS0742-1222240302" appears wrapped with an embedded newline). Please format the URL as a single continuous string.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: D_struct is an operational metric, and the demonstration is an empirical sanity check rather than a claim reduced to its own definition.

full rationale

The paper's central claim is that a CI/CD workflow can detect transpilation drift by comparing structural metrics of transpiled circuits against a stored baseline. Eq. (1) defines D_struct as mean relative change in depth, total gates, and two-qubit gates; this is the workflow's chosen operationalization of drift, not a prediction derived from data. The demonstration in Section V shows that a known transpiler switch (Scenario B) changes those metrics and yields D_struct = 0.1295, which is direct empirical evidence that the signal responds to the change, not a circular reduction. The thresholds tau_warn = 0.1 and tau_fail = 0.5 are explicitly illustrative and 'calibration is left to future work' (Section VI), so no fitted parameter is renamed as a prediction. The paper explicitly disclaims that D_struct is a formal circuit distance and records qubit layout separately as provenance (Section IV), so the metric's coverage limitations are acknowledged rather than hidden. Self-citations [5] and [10] support background claims about quantum DevOps and experiment tracking and are not load-bearing; no uniqueness theorem or ansatz is imported from those works. The only substantive limitation, that a layout-only change could yield D_struct = 0 while the executed circuit changed, is a coverage/completeness threat, not a circularity of the derivation, and the paper's own Section VI flags the proxy nature of the score. Therefore no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The workflow rests on domain assumptions about transpilation determinism, metric sufficiency, baseline management, and scenario representativeness. It introduces no new physical entities or theoretical constructs. The main free parameters are the two thresholds and the epsilon constant, all chosen illustratively.

free parameters (3)

Tau_warn = 0.1
Illustrative warning threshold for D_struct; the authors state systematic calibration is left to future work in Section VI.
Tau_fail = 0.5
Illustrative failure threshold; not calibrated to any dataset or circuit family.
Epsilon = 1
Small constant in the denominator of D_struct to avoid division by zero (Eq. 1); chosen by hand and not justified beyond numerical stability.

assumptions (4)

domain assumption Transpilation output is deterministic given fixed seed, transpiler version, and toolchain versions.
Section III states artifacts can be stored deterministically under fixed seeds and toolchain versions; this underpins baseline comparison.
domain assumption Depth, total gate count, and two-qubit gate count are sufficient structural proxies for the transpilation changes that matter.
This is the basis of D_struct in Eq. (1); Section VI acknowledges the score is a structural proxy and does not establish semantic non-equivalence.
domain assumption A baseline accepted by a maintainer is a valid reference realization for a circuit and target profile.
Section IV defines baseline management this way; drift is measured relative to the accepted baseline rather than against an absolute standard.
domain assumption The two demonstration scenarios (seed drift and vendor transpiler switch) are representative of realistic drift cases.
Section V selects circuits and targets that expose differences; the authors note this is a demonstration, not a broad survey.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Catching Transpilation Drift with a CI/CD Workflow in Quantum Software Development." pith.science (2026). https://pith.science/paper/YDCSWKUL

@misc{pith2026260808248,
  author       = {Pith},
  title        = {Pith review of: Catching Transpilation Drift with a CI/CD Workflow in Quantum Software Development},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YDCSWKUL}},
  note         = {Machine review of arXiv:2608.08248}
}

read the original abstract

Quantum software workflows rely on compiler and provider toolchains that evolve independently of application source code. Consequently, an unchanged quantum circuit may transpile into a different target-specific realization after changes in SDK versions, optimization settings, basis gates, coupling maps, or backend descriptions. Such transpilation drift can affect circuit depth, gate composition, qubit mapping, and execution behavior, yet it is rarely monitored in CI/CD pipelines. This paper proposes a Quantum DevOps workflow for detecting transpilation drift before execution. The workflow transpiles source circuits against configured target profiles, computes structural drift metrics, records provenance and artifacts in MLflow, and raises configurable warnings or failures in GitHub Actions. Using representative circuits and target profiles, we show how drift checks can expose toolchain-induced changes and support reproducibility audits. The contribution is a practical CI/CD guardrail for making quantum compilation behavior observable, testable, and auditable.

Figures

Figures reproduced from arXiv: 2608.08248 by the authors.

**Figure 2.** Scenario B (QFT10, IQM target): switching from IQM’s [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 6 canonical work pages

[1]

Techniques for improving regression testing in continuous integration development environments,

S. Elbaum, G. Rothermel, and J. Penix, “Techniques for improving regression testing in continuous integration development environments,” inProceedings of the 22nd ACM SIGSOFT International Symposium on F oundations of Software Engineering, 2014, pp. 235–245

work page 2014
[2]

Automated testing and de- ployment strategies for quantum algorithms,

M. C. Saxena, A. Tamrakar, and U. Arranz, “Automated testing and de- ployment strategies for quantum algorithms,” in2024 7th International Conference on Contemporary Computing and Informatics (IC3I), vol. 7, 2024, pp. 771–779

work page 2024
[3]

Open source software in quantum computing,

M. Fingerhuth, T. Babej, and P. Wittek, “Open source software in quantum computing,”PloS one, vol. 13, no. 12, p. e0208561, 2018

2018
[4]

A design science research methodology for information systems research,

K. Peffers, T. Tuunanen, M. A. Rothenberger, and S. Chatterjee, “A design science research methodology for information systems research,”Journal of Management Information Systems, vol. 24, no. 3, pp. 45–77, 2007. [Online]. Available: https://doi.org/10.2753/ MIS0742-1222240302

work page 2007
[5]

Toolchain for faster iterations in quantum software development,

O. Kinanen, A. D. Mu ˜noz-Moller, V . Stirbu, J. M. Murillo, and T. Mikkonen, “Toolchain for faster iterations in quantum software development,”Computing, vol. 107, no. 4, pp. 1–28, 2025

2025
[6]

Quan- tum devops: Towards reliable and applicable nisq quantum computing,

I.-D. Gheorghe-Pop, N. Tcholtchev, T. Ritter, and M. Hauswirth, “Quan- tum devops: Towards reliable and applicable nisq quantum computing,” in2020 IEEE Globecom Workshops (GC Wkshps, 2020, pp. 1–6

work page 2020
[7]

Quantum circuit synthesis and compilation optimization: Overview and prospects,

G. Yan, W. Wu, Y . Chen, K. Pan, X. Lu, Z. Zhou, Y . Wang, R. Wang, and J. Yan, “Quantum circuit synthesis and compilation optimization: Overview and prospects,”arXiv preprint arXiv:2407.00736, 2024

arXiv 2024
[8]

Mqt bench: Benchmarking software and design automation tools for quantum computing,

N. Quetschlich, L. Burgholzer, and R. Wille, “Mqt bench: Benchmarking software and design automation tools for quantum computing,”Quan- tum, vol. 7, p. 1062, July 2023

work page 2023

Show all 10 references

[9]

An empirical study into the effects of transpilation on quantum circuit smells,

M. D. Stefano, D. D. Nucci, F. Palomba, and A. D. Lucia, “An empirical study into the effects of transpilation on quantum circuit smells,”Empirical Software Engineering, vol. 29, no. 3, p. 61, 2024

2024
[10]

Toolchain for experiment tracking in iterative quantum software development,

O. Kinanen, M. Gamage, and V . Stirbu, “Toolchain for experiment tracking in iterative quantum software development,” in2025 IEEE International Conference on Quantum Computing and Engineering (QCE), vol. 2. IEEE, 2025, pp. 187–192

2025

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Techniques for improving regression testing in continuous integration development environments,

S. Elbaum, G. Rothermel, and J. Penix, “Techniques for improving regression testing in continuous integration development environments,” inProceedings of the 22nd ACM SIGSOFT International Symposium on F oundations of Software Engineering, 2014, pp. 235–245

work page 2014

[2] [2]

Automated testing and de- ployment strategies for quantum algorithms,

M. C. Saxena, A. Tamrakar, and U. Arranz, “Automated testing and de- ployment strategies for quantum algorithms,” in2024 7th International Conference on Contemporary Computing and Informatics (IC3I), vol. 7, 2024, pp. 771–779

work page 2024

[3] [3]

Open source software in quantum computing,

M. Fingerhuth, T. Babej, and P. Wittek, “Open source software in quantum computing,”PloS one, vol. 13, no. 12, p. e0208561, 2018

2018

[4] [4]

A design science research methodology for information systems research,

K. Peffers, T. Tuunanen, M. A. Rothenberger, and S. Chatterjee, “A design science research methodology for information systems research,”Journal of Management Information Systems, vol. 24, no. 3, pp. 45–77, 2007. [Online]. Available: https://doi.org/10.2753/ MIS0742-1222240302

work page 2007

[5] [5]

Toolchain for faster iterations in quantum software development,

O. Kinanen, A. D. Mu ˜noz-Moller, V . Stirbu, J. M. Murillo, and T. Mikkonen, “Toolchain for faster iterations in quantum software development,”Computing, vol. 107, no. 4, pp. 1–28, 2025

2025

[6] [6]

Quan- tum devops: Towards reliable and applicable nisq quantum computing,

I.-D. Gheorghe-Pop, N. Tcholtchev, T. Ritter, and M. Hauswirth, “Quan- tum devops: Towards reliable and applicable nisq quantum computing,” in2020 IEEE Globecom Workshops (GC Wkshps, 2020, pp. 1–6

work page 2020

[7] [7]

Quantum circuit synthesis and compilation optimization: Overview and prospects,

G. Yan, W. Wu, Y . Chen, K. Pan, X. Lu, Z. Zhou, Y . Wang, R. Wang, and J. Yan, “Quantum circuit synthesis and compilation optimization: Overview and prospects,”arXiv preprint arXiv:2407.00736, 2024

arXiv 2024

[8] [8]

Mqt bench: Benchmarking software and design automation tools for quantum computing,

N. Quetschlich, L. Burgholzer, and R. Wille, “Mqt bench: Benchmarking software and design automation tools for quantum computing,”Quan- tum, vol. 7, p. 1062, July 2023

work page 2023