{"id":"3d8a246f-fc83-4c38-a9a8-5fce84ddb096","arxiv_id":"2608.08248","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"A GitHub Actions workflow transpiles quantum circuits to target hardware profiles, computes structural drift metrics versus a stored baseline, and raises pass, warn, or fail signals.","lead":"This paper builds a CI/CD workflow that automatically checks whether quantum circuits changed after recompilation, before they are sent to hardware. It logs the compiled circuit's depth, gates, and two-qubit count, compares against an approved baseline, and warns or fails in GitHub Actions when the realization drifts.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"D_struct (Eq. 1) can stay zero when the executed circuit changes materially, so the CI pass/fail gate can miss exactly the drift it advertises.","rationale":"The paper's central claim is that a CI/CD workflow can detect transpilation drift before hardware execution. The reader's weakest assumption identifies the same load-bearing point: D_struct is a three-component structural summary, and changes in routing, layout, or gate placement that preserve depth, total gates, and two-qubit count are invisible to the score. I agree with that reading and would make the concern sharper: layout is recorded only as provenance and is not part of the status rule, so the CI gate can return pass while the executed circuit differs. This is not a claim of fraud or error; it is a scope mismatch between the advertised 'catching drift' and the implemented metric. The paper partially acknowledges this in Section VI by calling the score a structural proxy, but it does not demonstrate that the false-negative class is small, nor does it specify how layout-only changes affect the CI outcome. A concrete check with reversed initial_layout on a linear coupling map would settle whether the concern lands. Separately, the numerical inconsistency in Tables I and II is also worth fixing: applying Eq. (1) to the Table II values gives D_struct ≈ 0.173, not the reported 0.1295, so the demonstration itself needs correction before the workflow is treated as a reliable reference. Neither issue overturns the practical contribution, so the original CONDITIONAL verdict remains appropriate.","tokens_in":5432,"tokens_out":6552,"duration_ms":65564,"concrete_test":"Construct one BV12 or QFT10 circuit on a linear or quasi-linear target profile (IQM or Rigetti) and transpile it twice with Qiskit using two different initial_layout values or routing seeds at the same optimization level. Use the first run as the MLflow baseline and run the workflow with the second; check whether D_struct = 0 and the GitHub status is pass while the recorded layout differs. If yes, the workflow misses a real change in the submitted circuit; if no, the layout flag must be shown to feed the status rule, and Eq. (1) or Section IV should be amended to state that.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (1) in Section IV defines the drift signal from F = ⟨depth, gates, twoQ⟩ only. Qubit layout is logged and 'flagged' separately, but the pass/warn/fail status is a threshold on D_struct, and no rule in Sections IV–V routes a layout-only change into warn or fail. Therefore a transpiler update, seed change, or backend-description change that reroutes or remaps qubits while preserving these three counts yields D_struct = 0 and a green CI run, although the circuit actually submitted to the device has changed. This is not a hypothetical corner case: on coupling maps with symmetries, such as reversing a linear chain or permuting identical subgraphs, different initial layouts or routing seeds often produce exactly the same depth and gate counts. Section VI acknowledges that the score is a 'structural proxy' and that it does not establish semantic equivalence, but the paper's opening claim is that the workflow detects transpilation drift before execution. For this material change class it does not, unless layout changes are folded into the gate or the status rule. Nothing in the manuscript bounds this false-negative class, so the central claim is stronger than the demonstrated signal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a CI/CD workflow for detecting transpilation drift in quantum software development. The workflow transpiles source circuits against configured target profiles, computes a structural feature vector F = ⟨depth, gates, twoQ⟩, compares the result against an MLflow-stored baseline using a drift score D_struct (Eq. 1), and raises pass/warn/fail feedback through GitHub Actions. Qubit layout is recorded and flagged separately but is not included in the drift score. The demonstration covers two scenarios: seed drift (Scenario A, BV12 circuit on three targets) and transpiler drift (Scenario B, QFT10 on an IQM target, switching from iqm-client's transpiler to Qiskit's built-in transpiler). The paper positions the contribution as a practical guardrail for making compilation behavior observable and auditable, rather than a formal equivalence checker.","tokens_in":5645,"tokens_out":4850,"duration_ms":43319,"significance":"If the workflow performs as claimed, it addresses a real and under-served gap: toolchain-induced changes to the transpiled circuit are typically invisible in CI, yet they affect reproducibility, cost, and comparability of results. The paper provides a clean workflow architecture, MLflow-based provenance tracking, a public demonstration repository, and a plausible end-to-end scenario. These are concrete strengths. The central limitation is that the demonstrated drift signal only tracks three aggregate metrics, and the numerical evidence in the demonstration contains inconsistencies that currently prevent the reader from verifying the core quantitative claim. The contribution is incremental but useful for the quantum software engineering community, and the workflow is easily adoptable.","major_comments":[{"comment":"The reported drift numbers for Scenario B are internally inconsistent. Table I reports Δ2Q = -10 for QFT10 on IQM, while Table II shows 2-qubit gates 173 vs 172, i.e., Δ2Q = -1. Moreover, the reported D_struct = 0.1295 does not reproduce from Eq. (1) using the metrics in Table II: taking depth (213→253), total ops (453→600), and two-qubit gates (173→172) gives (40/213 + 147/453 + 1/173)/3 ≈ 0.173, and no plausible subset of the listed metrics yields 0.1295. Because the demonstration is the primary evidence that the workflow detects drift, this inconsistency must be corrected and the exact computation of D_struct for this example must be shown.","section":"Section V, Tables I and II"},{"comment":"The CI pass/warn/fail status is defined solely as a threshold on D_struct, which is computed only from depth, total gates, and two-qubit gates. Qubit layout is recorded and \"flagged\" separately, but no rule in Sections IV or V routes a layout-only change into warn or fail. Consequently, a transpiler update or seed change that remaps qubits or reroutes the circuit while preserving these three aggregates yields D_struct = 0 and a green CI run, even though the circuit actually executed has changed. This directly contradicts the paper's opening claim that the workflow detects transpilation drift before execution. The authors should either (a) include layout distance in D_struct, (b) add a separate status rule that treats layout changes as warn/fail, or (c) explicitly narrow the claim to \"drift in depth and gate counts\" rather than transpilation drift in general. Section VI's statement that the score is a structural proxy does not resolve this specific false-negative class.","section":"Section IV, Eq. (1) and status rule"},{"comment":"The metric denoted \"gates\" in the feature vector F is not uniquely defined. Section III mentions \"depth, total gates, two-qubit gates\", but Table II lists \"Total ops\", \"R gates\", \"CZ gates\", and \"RZ gates\" separately, and the relationship between \"total gates\" and \"Total ops\" is not explained. Without a precise definition (e.g., the sum of Qiskit's count_ops entries excluding barriers or measurements), the drift score is not reproducible from the paper's data. The authors should specify the exact Qiskit property used for \"gates\" and verify that it matches the values in Table II.","section":"Section IV, Eq. (1)"}],"minor_comments":[{"comment":"The phrase \"Suchtranspilation drift\" is missing a space; it should read \"Such transpilation drift\".","section":"Abstract"},{"comment":"The threshold symbols τ_warn and τ_fail are used in the status equation without being defined in the text; they appear only in Figure 1. Please define them explicitly in Section IV where the equation is introduced.","section":"Section IV, status equation"},{"comment":"The labels \"< τ_w\" and \"≥ τ_f\" in the threshold-policy box are visually clipped and hard to read. Consider redrawing the box with clearer spacing and explicit threshold names.","section":"Figure 1"},{"comment":"The text says \"two-qubit count is essentially unchanged\" while Table I reports Δ2Q = -10; the wording is consistent with Table II's -1 but not with Table I. Harmonize all reported deltas after correcting the inconsistency noted in the major comments.","section":"Section V, Scenario B text"},{"comment":"The URL for reference [4] contains a line-break artifact (\"https://doi.org/10.2753/MIS0742-1222240302\" appears wrapped with an embedded newline). Please format the URL as a single continuous string.","section":"Reference [4]"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable systems contribution for a quantum-software engineering venue. The main concerns are the numerical inconsistencies in the core demonstration and the layout-only false-negative gap, both of which are fixable within the manuscript's scope. The availability of the demo repository is a strong point. The paper does not overclaim in its final conclusions, but the abstract and introduction make broader detectability claims than the current implementation and evidence support; those need to be aligned during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this one. First, it's a practical, well-scoped design-science paper: it gives quantum software teams a way to make the transpiled circuit a tracked, CI-checked artifact, and it ships open-source demo code. Second, the demonstration numbers don't add up, and the drift metric has a blind spot for layout-only changes that the paper's own opening claim seems to promise it catches.\n\nWhat's legitimately new here: prior work benchmarked compiler effects or tracked experiments, but this operationalizes 'transpilation drift' as a baseline-relative CI gate using a simple mean-relative-change score over depth, gate count, and two-qubit count. The workflow is clear: transpile against a target profile, compare to a stored baseline, log to MLflow, and emit pass/warn/fail in GitHub Actions. The authors are honest that thresholds are illustrative and that drift is not proof of semantic difference. The demonstration with seed drift and transpiler drift is a genuine illustration of the problem, and the code link is there.\n\nNow the soft spots. The numbers don't reconcile. Table I reports Delta 2Q = -10 for Scenario B, but Table II shows 173 to 172 (Delta -1). And Eq. (1) with Table II's metrics gives a mean relative change around 0.17, not the stated 0.1295. That's sloppy, and it undercuts the paper's usefulness as a reference until fixed.\n\nMore substantively, the CI status is driven only by D_struct, which uses depth, gates, and two-qubit counts. Layout is logged and 'flagged,' but no rule routes a layout-only change into warn or fail. So a transpiler update that reroutes or remaps qubits while preserving those three counts yields D_struct = 0 and a green run. The stress-test note is right: this misses a material class of drift, and it's realistic on coupling maps with symmetries. The authors call the score a structural proxy in Section VI but don't address the gate. This is fixable — include layout in the score or make layout change a separate warn/fail condition — but right now the central claim is stronger than the demonstrated signal.\n\nThat said, the contribution is a workflow, not a physics result, and the workflow's core idea holds up. The circularity worry is minor: a change detector measures its own metrics by design; the real inadequacy is the layout blind spot.\n\nVerdict: this deserves a serious referee. A moderate revision should fix the numbers and close the layout gap. Teams doing applied quantum software work will get practical value from it.","headline":"Useful DevOps guardrail for quantum software, but the demonstrated drift numbers don't reconcile and the metric misses layout-only changes.","tokens_in":6156,"tokens_out":3756,"would_cite":true,"duration_ms":34008,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CI/CD workflow that compares transpiled circuit structure against a stored baseline can detect toolchain-induced changes before hardware execution.","keywords":["quantum software engineering","quantum DevOps","continuous integration","continuous delivery","transpilation drift","transpilation","reproducibility"],"falsifier":"Take a fixed source circuit and two transpiler settings that produce identical depth, total gate count, and two-qubit gate count but different qubit layouts or gate orderings, and run the workflow; it would report $D_{\\mathrm{struct}}=0$ and pass, even though the realized circuit changed, showing that the structural proxy misses a meaningful realization change.","tokens_in":5227,"feed_emoji":"⚛️","tokens_out":8748,"duration_ms":73219,"temperature":0.7,"pith_summary":"Quantum programs are rarely executed in the form developers write them; compiler and provider toolchains transform each circuit to fit a specific backend. The paper argues that this transformation can silently change when SDK versions, optimization settings, coupling maps, or backend descriptions change, and that such 'transpilation drift' is a regression signal that most CI/CD pipelines ignore. It proposes a lightweight workflow that transpiles source circuits against configured target profiles, compares structural metrics (depth, total gates, two-qubit gates) with an approved baseline, and converts the comparison into pass/warn/fail feedback before hardware submission. If adopted, teams would see a CI warning whenever a dependency update or configuration change alters the circuit that would actually run, protecting reproducibility and avoiding wasted hardware time.","feed_headline":"CI guardrail catches quantum transpilation drift before execution","feed_subtitle":"Unchanged quantum circuits can compile differently after SDK or transpiler updates; this workflow flags such silent changes automatically.","key_machinery":"The central mechanism is the structural drift score $D_{\\mathrm{struct}}$, the mean relative change between a baseline run and a current run over three count-valued features: circuit depth, total gate count, and two-qubit gate count, with a small floor $\\epsilon=1$ to keep the ratio stable. The score is deliberately not a formal circuit distance; it is a lightweight CI signal. It is paired with an experiment-tracking service (MLflow) that stores transpiled circuits, target descriptions, metric JSON, and provenance parameters, and with a threshold policy that converts $D_{\\mathrm{struct}}$ into pass, warn, or fail. The named phenomenon being measured is 'transpilation drift'—a change in the compiled realization of an unchanged source circuit caused by toolchain, SDK, or target-profile evolution.","core_discovery":"The central claim is that the transpiled circuit—not just the source—should be treated as a monitored build artifact in quantum software development. The workflow transpiles every source circuit against each configured target profile, records depth, total gate count, and two-qubit gate count, compares them to an approved baseline with the mean-relative-change score, and maps the result to pass/warn/fail thresholds. Two demonstrations show realistic drift: leaving the transpiler seed unset produces different routings on connectivity-constrained targets, and switching from a vendor transpiler to a newer SDK's built-in transpiler changes depth and gate composition for a 10-qubit Fourier transform. In both cases the source circuit is unchanged, yet the compiled realization changes; the workflow flags this before hardware execution and logs artifacts and provenance for later audit.","pith_inferences":["Beyond the paper: the workflow records qubit layout separately rather than in the drift score, so a natural extension would be to gate the score on layout equality or fold layout changes in—layout differences also change noise exposure and execution cost.","Beyond the paper: the same baseline comparison could be reused as a transpiler-regression probe, running a small circuit suite across a matrix of SDK versions and target profiles to map which updates introduce drift before they reach a project's main branch.","Beyond the paper: threshold calibration is left open, so a plausible next step is fitting per-circuit-family thresholds from historical drift distributions, letting warnings reflect changes that are unusual for the circuit rather than any nonzero change."],"forward_implications":["A team with an approved baseline will see a warning or failing CI status whenever a dependency or transpiler update changes the realized circuit for any configured target, before any hardware time is spent.","Storing transpiled circuits, target descriptions, metrics, and provenance in an experiment tracker turns each CI run into an auditable record that can be replayed or compared later, supporting reproducibility audits.","The pass/warn/fail thresholds provide a configurable release gate: intentional changes are absorbed by updating the baseline, while accidental changes stop the pipeline.","Because the check is pre-execution and fast, it fits pull-request workflows and can be supplemented by optional scheduled or release-gate execution-level checks."],"supporting_citations":[{"why":"supplies the CI/CD regression-testing practice that the workflow extends to transpiled circuits.","marker":"[1]"},{"why":"establishes that reproducibility of quantum work depends on recording the software and toolchain context.","marker":"[3]"},{"why":"establishes that transpilation is target- and toolchain-dependent, motivating drift detection.","marker":"[7]"},{"why":"demonstrates that compiler choices and versions substantially affect circuit structure and quality.","marker":"[8]"},{"why":"provides empirical evidence that transpilation changes circuit structure and quality, justifying structural metrics as the drift signal.","marker":"[9]"},{"why":"motivates adapting experiment-tracking practice to record quantum-specific provenance during development.","marker":"[10]"}],"fun_headline_variants":["CI/CD guardrail exposes transpilation drift before run","Flag silent quantum compilation changes with a CI workflow","Quantum DevOps: monitor transpiled circuits as build artifacts","Detect transpilation drift before execution in quantum pipelines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that changes in depth, total gate count, and two-qubit gate count faithfully capture every transpilation change that matters, so a compiler update that only re-routes or rearranges gates would slip past with a drift score of zero.","fun_headline_variants_meta":{"raw":{"variants":["CI/CD guardrail exposes transpilation drift before run","Flag silent quantum compilation changes with a CI workflow","Quantum DevOps: monitor transpiled circuits as build artifacts","Detect transpilation drift before execution in quantum pipelines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000426,"raw_usage":{"total_tokens":2138,"prompt_tokens":856,"completion_tokens":1282,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":1220}},"tokens_in":472,"tokens_out":1282,"duration_ms":9726,"temperature":1.0,"reasoning_tokens":1220,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:13:13.400102+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed source circuit and two transpiler settings that produce identical depth, total gate count, and two-qubit gate count but different qubit layouts or gate orderings, and run the workflow; it would report $D_{\\mathrm{struct}}=0$ and pass, even though the realized circuit changed, showing that the structural proxy misses a meaningful realization change.","supporting_citations":[{"cited_title":"Techniques for improving regression testing in continuous integration development environments,","cited_arxiv_id":null,"evidence_quote":"supplies the CI/CD regression-testing practice that the workflow extends to transpiled circuits."},{"cited_title":"Mqt bench: Benchmarking software and design automation tools for quantum computing,","cited_arxiv_id":null,"evidence_quote":"demonstrates that compiler choices and versions substantially affect circuit structure and quality."},{"cited_title":"Toolchain for experiment tracking in iterative quantum software development,","cited_arxiv_id":null,"evidence_quote":"motivates adapting experiment-tracking practice to record quantum-specific provenance during development."}],"review_version":1}