REVIEW 3 major objections 6 minor 2 cited by
Assessing Quantum Layout Synthesis Tools via Known Optimal-SWAP Cost Benchmarks
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper introduces QUBIKOS, a benchmark family whose circuits require a provably minimal number of SWAP gates, and uses it to show that leading layout synthesis tools use 63 to 330 times more SWAPs than necessary.
desk verdict QUBIKOS is a genuinely useful first benchmark with provably optimal non-zero SWAP counts, but the proof as written has a sign error in Lemma 1 and an undefined reference in Theorem 4; both are repairable, and the paper deserves peer review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The construction's engine is the non-isomorphic interaction graph: a set of two-qubit gates, generated by Algorithm 1, whose interaction graph cannot be embedded into the device coupling graph, so any layout must insert a SWAP. A distinguished 'special gate' is placed last in its section so that only that SWAP makes it executable, and the remaining gates are ordered by breadth-first search so that every gate in a later section depends on the previous special gate and precedes the next one. This serial dependence chain is what makes the per-section SWAP requirements add up, turning the section count into a provable lower bound that the explicit solution attains.
What would settle it
Run Algorithm 1 on a coupling graph where the degree-count inequality of Lemma 1 is not guaranteed (for example, a regular graph where all physical and program qubits have equal degree), generate the backbone circuit, and check with an exact QLS solver whether the optimal SWAP count is ever less than the number of sections; any such case would disprove the claimed optimality.
Extended reading notes
Core claim
The paper's central claim is that QUBIKOS circuits have a provable optimal SWAP count equal to the number of sections n. Each section is built so that its interaction graph is not isomorphic to any subgraph of the device's coupling graph, forcing at least one SWAP, and a special gate scheduled after all other gates in the section makes that SWAP unavoidable under any mapping. Gate dependencies (enforced by breadth-first searches) serialize the sections, so the SWAP requirements add; the construction explicitly provides a transpiled circuit with n SWAPs, proving optimality. The paper verifies the claim on small circuits with an exact solver and then applies the benchmark to measure optimality gaps of four leading heuristic QLS tools.
Load-bearing premise
The load-bearing premise is Lemma 1, which asserts that for every interaction graph the construction generates, some endpoint of the special gate has more physical qubits of degree at least its degree than program qubits, so the graph cannot fit into the coupling graph; this inequality is stated without proof, and if it fails on some topology the claimed optimal SWAP count could be wrong.
Editorial extensions
If this is right
- QLS tools can now be compared against an absolute optimum; the measured average gaps are 63x (LightSABRE), 117x (ML-QLS), 250x (QMAP), and 330x (t|ket>) on the tested architectures.
- Optimality gaps grow with device size and with sparse connectivity: on the 127-qubit Eagle the best tool reaches a 233.97x gap, while on the 16-qubit Aspen-4 ML-QLS finds optimal solutions in all cases.
- Sparse, heavy-hex topologies such as Rochester are harder for current tools than dense grid topologies such as Sycamore, even at similar qubit counts.
- QUBIKOS can evaluate standalone routers by supplying the optimal initial mapping, so that non-optimal results are attributable to the router itself.
- A case study of LightSABRE traces one suboptimal choice to the lookahead cost and suggests adding a decay factor to the lookahead term could improve routing.
Reading between the lines
- If the gap magnitudes hold beyond the tested circuit sizes, heuristic QLS may still be far from practical optimality on large devices, making exact or hybrid methods worth revisiting for moderate qubit counts.
- The section-and-special-gate construction is a general template: it could be extended to produce benchmarks with simultaneously known-optimal SWAP count and depth, or to target other routing objectives such as gate-error-weighted cost.
- A direct testable extension is to generate QUBIKOS circuits on regular or highly symmetric coupling graphs where Lemma 1's degree-count inequality is strained, to check whether the proof's unstated condition ever fails.
- The lookahead-decay fix suggested by the case study could be implemented in LightSABRE; if it reduces the QUBIKOS gap, that would validate the cost-function diagnosis.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces QUBIKOS, a family of benchmark quantum circuits for which the optimal number of SWAP gates required by quantum layout synthesis (QLS) is claimed to be known by construction. The construction generates n circuit sections, each designed to require exactly one SWAP by building an interaction graph that is not isomorphic to any subgraph of the device coupling graph, and orders the sections serially via a gate dependency graph. A proof of optimality is given in Section III-D, and experiments with the exact tool OLSQ2 on small circuits confirm the claimed SWAP counts for the tested instances. The benchmarks are then used to evaluate LightSABRE, ML-QLS, QMAP, and t|ket> on four architectures; reported average optimality gaps range from 1x to several hundred times, with LightSABRE performing best among the tested tools. A case study attributes one LightSABRE failure to the equal weighting of lookahead costs.
Significance. If the optimality claim is correct, QUBIKOS fills a real gap: existing benchmarks either have zero SWAPs (QUEKO) or no provable optimum (QUEKNO). The benchmark is not fitted to tool outputs, so the reported optimality gaps are externally anchored rather than circular. The large gaps and the LightSABRE lookahead analysis are concrete and actionable findings, and the small-circuit verification with OLSQ2 is a useful sanity check. However, the benchmark's central value rests on the general proof that each constructed section forces exactly one SWAP, so the proof gaps described below are what currently prevent acceptance. The contribution would be valuable after a rigorous repair of Lemma 1 and Theorem 4.
major comments (3)
- [Section III-D, Lemma 1] The proof of Lemma 1 does not establish the stated non-isomorphism. With S1 and S2 as defined, the pigeonhole obstruction to a degree-preserving injection requires |S2| > |S1| (more program qubits of degree at least d than physical nodes of degree at least d); the inequality asserted in the proof, |S1| > |S2|, says the opposite and, if anything, makes a mapping easier to find. No argument is given for why |S1| > |S2| should hold for every graph produced by Algorithm 1; in the construction the special-gate endpoint q has degree one larger than its currently mapped physical node, which can yield |S2| = |S1| + 1 rather than |S1| > |S2|. Since Theorem 4 uses Lemma 1 to rule out SWAP-free execution of each section, the central lower-bound proof is incomplete as written. The authors should replace this proof with a correct degree-count or structural argument, or state the precise construction-dependent condition and prove it from Algorithm 1.
- [Section III-D, Theorem 4] The proof of the base case invokes an undefined 'claim 2' to conclude that every gate of C0 lies in C′0. No such claim is stated anywhere in the paper, and this step is exactly what forces the whole first section to be executed before the first SWAP; without it the contradiction from Lemma 1 does not follow. The induction step has a related gap: from Ci+1 ∪ {gsi+1} ⊆ C′i+1 it is not shown that the interaction graph of this subset is the same object to which Lemma 1 applies, because C′i+1 may contain additional gates and may not coincide with the section Ci+1. The proof should be rewritten with a precise notation for sections, SWAP boundaries, and the special-gate dependency argument.
- [Section III-D, final paragraph] The final paragraph asserts that inserting 'redundant' gates into the backbone cannot increase the optimal SWAP count and that the non-isomorphism and dependency properties are preserved. The non-isomorphism part is correct, but the dependency part is only argued informally ('can only make the dependency chain either longer or remain the same'), and Algorithm 3 does not specify the conditions under which a gate may be inserted into Cans while keeping the n-SWAP solution valid. Since the benchmark circuits used in Section IV-B have hundreds to thousands of gates and are not verified exactly, this step is load-bearing for the claimed optimal counts of the actual test instances. Please provide either a formal insertion invariant or a precise insertion rule with proof.
minor comments (6)
- [Figure 1] The caption lists '(b)' twice; the last panel should be labeled '(e)'.
- [Algorithm 1] The variable p′ is overwritten in line 2 before being used in line 3, and the selection conditions in lines 1-3 are not stated with enough precision to guarantee that such vertices exist for all coupling graphs used in the paper.
- [Section III-A and Figure 2] The running text, the figure captions, and Algorithm 1 use different qubit labels (for example, the text describes a special gate g(q1,q7) while the figure caption says the graph enforces SWAP(q0,q4)); aligning these names would greatly improve readability.
- [Section III-D, Theorem 4] The symbol C0 is overloaded: it denotes both the first section of the backbone circuit and the first partial gate sequence in the transpiled circuit C′. Using different symbols for these objects would remove a source of confusion in the proof.
- [Section IV-B] There are small textual errors, e.g., 'experimment' and 'we uses'; also, the relationship between the average gap values quoted in the text and the per-architecture results in Figure 4 should be stated explicitly.
- [General] For a benchmark paper, an explicit code and data availability statement, with a repository containing the circuit generator and the exact verification scripts, would substantially increase the usability and reproducibility of the results.
Circularity Check
No significant circularity: the optimal SWAP counts are fixed by construction and independently verified; tool evaluations measure external solvers against that fixed ground truth.
full rationale
QUBIKOS's claimed optimal SWAP count is established by construction, not by fitting to or predicting from the outputs of the tools being evaluated. Algorithm 3 selects n sections, Algorithm 1 generates interaction graphs asserted to be non-isomorphic to any subgraph of the coupling graph (Lemma 1), and Lemmas 2-3 plus Theorem 4 argue that serial dependencies force at least one SWAP per section, while Algorithm 3 explicitly provides an n-SWAP transpilation. The later experiments measure LightSABRE, ML-QLS, QMAP, and t|ket> against this fixed n; no tool output or fitted parameter enters the definition of the optimum, so the 63x/117x/250x/330x gaps are evaluation results rather than circular predictions. Self-citations to QUEKO [28], OLSQ2 [10], and ML-QLS [27] are not load-bearing: QUEKO is related work, OLSQ2 is an exact solver used only as an independent verification check, and ML-QLS is one of the tools under test. No equation reduces to its own input and no known empirical pattern is merely renamed. For completeness, the proof as written has non-circular validity gaps: Lemma 1 asserts the inequality |S1| > |S2| without proof (and for the construction the opposite comparison |S2| = |S1| + 1 appears to hold), and Theorem 4 refers to an undefined 'claim 2' in the base case. These are correctness concerns about the lower-bound proof, not circularity, because the optimal count is not taken from the benchmark generators or the evaluated QLS tools.
Assumptions & free parameters
assumptions (5)
- domain assumption A two-qubit gate can be scheduled only if the two program qubits are currently mapped to adjacent physical qubits.
- domain assumption Each SWAP gate permutes the mapping and counts as one unit of cost.
- standard math If an interaction graph is not isomorphic to any subgraph of the coupling graph, it cannot be executed under any single qubit mapping.
- ad hoc to paper Lemma 1's degree-count property (|S1| > |S2|) holds for every graph produced by Algorithm 1.
- ad hoc to paper Inserting 'redundant' gates into the backbone cannot reduce the optimal SWAP count.
Cite this review
Pith. "Pith review of Assessing Quantum Layout Synthesis Tools via Known Optimal-SWAP Cost Benchmarks." pith.science (2026). https://pith.science/paper/RRS7IRAH
@misc{pith2026250208839,
author = {Pith},
title = {Pith review of: Assessing Quantum Layout Synthesis Tools via Known Optimal-SWAP Cost Benchmarks},
year = {2026},
howpublished = {\url{https://pith.science/paper/RRS7IRAH}},
note = {Machine review of arXiv:2502.08839}
}
read the original abstract
Quantum layout synthesis (QLS) is a critical step in quantum program compilation for superconducting quantum computers, involving the insertion of SWAP gates to satisfy hardware connectivity constraints. While previous works have introduced SWAP-free benchmarks with known-optimal depths for evaluating QLS tools, these benchmarks overlook SWAP count - a key performance metric. Real-world applications often require SWAP gates, making SWAP-free benchmarks insufficient for fully assessing QLS tool performance. To address this limitation, we introduce QUBIKOS, a benchmark set with provable-optimal SWAP counts and non-trivial circuit structures. For the first time, we are able to quantify the optimality gaps of SWAP gate usages of the leading QLS algorithms, which are surprisingly large: LightSabre from IBM delivers the best performance with an optimality gap of 63x, followed by ML-QLS with an optimality gap of 117x. Similarly, QMAP and t|ket> exhibit significantly larger gaps of 250x and 330x, respectively. This highlights the need for further advancements in QLS methodologies. Beyond evaluation, QUBIKOS offers valuable insights for guiding the development of future QLS tools, as demonstrated through an analysis of a suboptimal case in LightSABRE. This underscores QUBIKOS's utility as both an evaluation framework and a tool for advancing QLS research.
Figures
Forward citations
Cited by 2 Pith papers
-
Optimized Qubit Routing for Commuting Gates via Integer Programming
A new exact integer-programming formulation, the Token Meeting Problem, provably minimizes swap gates when routing commuting-gate quantum circuits, with NP-hardness and asymptotic bounds.
-
A High-Performance Multilevel Framework for Quantum Layout Synthesis
ML-SABRE, a multilevel layout synthesis framework built on the LightSABRE heuristic, cuts SWAP count by 45-65% and improves compilation speed by 2.5-3x on quantum benchmarks.
Reference graph
Works this paper leans on
-
[1]
Qubit allocation
M. Y . Siraichi et al. , “Qubit allocation.” New York, NY , USA: Association for Computing Machinery, Feb. 2018, pp. 113–125
2018
-
[2]
Optimal SW AP gate insertion for nearest neighbor quantum circuits,
R. Wille, A. Lye, and R. Drechsler, “Optimal SW AP gate insertion for nearest neighbor quantum circuits,” in 2014 19th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 2014, pp. 489–494
2014
-
[3]
MUQUT: Multi-constraint quantum circuit mapping on NISQ comput- ers,
D. Bhattacharjee, A. A. Saki, M. Alam, A. Chattopadhyay, and S. Ghosh, “MUQUT: Multi-constraint quantum circuit mapping on NISQ comput- ers,” in 2019 IEEE/ACM International Conference on Computer-Aided Design. IEEE, 2019, pp. 1–7
2019
-
[4]
Mapping quantum circuits to IBM QX architectures using the minimal number of SW AP and H operations,
R. Wille et al. , “Mapping quantum circuits to IBM QX architectures using the minimal number of SW AP and H operations,” in Proceedings of the 56th Annual Design Automation Conference 2019 , 2019, pp. 1–6
2019
-
[5]
Optimal layout synthesis for quantum computing,
B. Tan and J. Cong, “Optimal layout synthesis for quantum computing,” in Proceedings of the 39th International Conference on Computer-Aided Design. ACM, Nov. 2020, pp. 1–9
2020
-
[6]
Optimal qubit mapping with simultaneous gate absorption,
——, “Optimal qubit mapping with simultaneous gate absorption,” in 2021 IEEE/ACM International Conference on Computer Aided Design . IEEE, 2021, pp. 1–8
2021
-
[7]
Time-optimal Qubit mapping
C. Zhang et al., “Time-optimal Qubit mapping.” New York, NY , USA: Association for Computing Machinery, Apr. 2021, pp. 360–374
2021
-
[8]
Qubit mapping and routing via MaxSAT,
A. Molavi et al. , “Qubit mapping and routing via MaxSAT,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture. IEEE, 2022, pp. 1078–1091
2022
Show all 33 references
-
[9]
Optimal qubit assignment and routing via integer programming,
G. Nannicini et al., “Optimal qubit assignment and routing via integer programming,” ACM Transactions on Quantum Computing, vol. 4, no. 1, pp. 1–31, 2022
2022
-
[10]
Scalable optimal layout synthesis for NISQ quantum processors,
W.-H. Lin et al., “Scalable optimal layout synthesis for NISQ quantum processors,” in 2023 60th ACM/IEEE Design Automation Conference . IEEE, 2023, pp. 1–6
2023
-
[11]
Announcing Cirq: an open source framework for NISQ algorithms,
A. Ho and D. Bacon, “Announcing Cirq: an open source framework for NISQ algorithms,” Google AI Blog , vol. 18, 2018
2018
-
[12]
An efficient methodology for mapping quantum circuits to the IBM QX architectures,
A. Zulehner et al. , “An efficient methodology for mapping quantum circuits to the IBM QX architectures,” IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems , vol. 38, no. 7, pp. 1226–1236, 2018
2018
-
[13]
(2018) Qiskit
IBM. (2018) Qiskit. [Online]. Available: https://qiskit.org/
2018
-
[14]
An Efficient Methodology for Mapping Quan- tum Circuits to the IBM QX Architectures,
A. Zulehner et al. , “An Efficient Methodology for Mapping Quan- tum Circuits to the IBM QX Architectures,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 38, no. 7, pp. 1226–1236, Jul. 2019
2019
-
[15]
Qubit allocation as a combination of subgraph isomorphism and token swapping,
M. Y . Siraichi et al. , “Qubit allocation as a combination of subgraph isomorphism and token swapping,” Proceedings of the ACM on Pro- gramming Languages , vol. 3, no. OOPSLA, pp. 120:1–120:29, Oct. 2019
2019
-
[16]
Tackling the qubit mapping problem for NISQ-era quantum devices
G. Li et al., “Tackling the qubit mapping problem for NISQ-era quantum devices.” New York, NY , USA: Association for Computing Machinery, Apr. 2019, pp. 1001–1014
2019
-
[17]
Formal constraint-based compilation for noisy intermediate-scale quantum systems,
P. Murali et al. , “Formal constraint-based compilation for noisy intermediate-scale quantum systems,” Microprocessors & Microsystems, vol. 66, no. C, pp. 102–112, Apr. 2019
2019
-
[18]
t |ket⟩: a retargetable compiler for NISQ devices,
S. Sivarajah et al. , “t |ket⟩: a retargetable compiler for NISQ devices,” Quantum Science and Technology , vol. 6, no. 1, p. 014003, Nov. 2020
2020
-
[19]
Improved Mapping of Quantum Circuits to IBM QX Architectures,
A. Kole et al. , “Improved Mapping of Quantum Circuits to IBM QX Architectures,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 39, no. 10, pp. 2375–2383, Oct. 2020
2020
-
[20]
Not all SW APs have the same cost: a case for optimization- aware qubit routing,
J. Liu et al., “Not all SW APs have the same cost: a case for optimization- aware qubit routing,” in 2022 IEEE International Symposium on High- Performance Computer Architecture , Apr. 2022, pp. 709–725, iSSN: 2378-203X
2022
-
[21]
A robust quantum layout synthesis algorithm with a qubit mapping checker,
T.-A. Wu et al. , “A robust quantum layout synthesis algorithm with a qubit mapping checker,” in Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design , ser. ICCAD ’22. New York, NY , USA: Association for Computing Machinery, 2022
2022
-
[22]
Optimizing quantum circuit placement via machine learning,
H. Fan et al. , “Optimizing quantum circuit placement via machine learning,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, ser. DAC ’22. New York, NY , USA: Association for Computing Machinery, 2022, p. 19–24
2022
-
[23]
Reinforcement learning and dear framework for solving the qubit mapping problem,
C.-Y . Huang et al. , “Reinforcement learning and dear framework for solving the qubit mapping problem,” in Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design , 2022, pp. 1–9
2022
-
[24]
A fast and scalable qubit-mapping method for noisy intermediate-scale quantum computers,
S. Park et al. , “A fast and scalable qubit-mapping method for noisy intermediate-scale quantum computers,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, New York, NY , USA, 2022, p. 13–18
2022
-
[25]
CTQr: Control and timing-aware qubit routing,
C.-Y . Huang and W.-K. Mak, “CTQr: Control and timing-aware qubit routing,” in 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 2024, pp. 140–145
2024
-
[26]
Efficient qubit routing using a dynamically-extract-and-route framework,
——, “Efficient qubit routing using a dynamically-extract-and-route framework,” IEEE Transactions on Computer-Aided Design of Inte- grated Circuits and Systems , 2024
2024
-
[27]
ML-QLS: Multilevel quantum layout synthe- sis,
W.-H. Lin and J. Cong, “ML-QLS: Multilevel quantum layout synthe- sis,” arXiv preprint arXiv:2405.18371 , 2024
2024 arXiv
-
[28]
Optimality study of existing quantum computing layout synthesis tools,
B. Tan and J. Cong, “Optimality study of existing quantum computing layout synthesis tools,” IEEE Transactions on Computers, vol. 70, no. 9, pp. 1363–1373, 2020
2020
-
[29]
On constructing benchmark quantum circuits with known near-optimal transformation cost,
S. Li, X. Zhou, and Y . Feng, “On constructing benchmark quantum circuits with known near-optimal transformation cost,” 2023. [Online]. Available: https://arxiv.org/abs/2301.08932
2023 arXiv
-
[30]
A (sub)graph isomorphism algorithm for matching large graphs,
L. Cordella, P. Foggia, C. Sansone, and M. Vento, “A (sub)graph isomorphism algorithm for matching large graphs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1367– 1372, 2004
2004
-
[31]
LightSABRE: A lightweight and enhanced SABRE algorithm,
H. Zou, M. Treinish, K. Hartman, A. Ivrii, and J. Lishman, “LightSABRE: A lightweight and enhanced SABRE algorithm,” arXiv preprint arXiv:2409.08368, 2024
2024 arXiv
-
[32]
On the qubit routing problem,
A. Cowtan, S. Dilkes, R. Duncan, A. Krajenbrink, W. Simmons, and S. Sivarajah, “On the qubit routing problem,” arXiv preprint arXiv:1902.08091, 2019
1902 arXiv
-
[33]
MQT QMAP: Efficient quantum circuit mapping,
R. Wille and L. Burgholzer, “MQT QMAP: Efficient quantum circuit mapping,” in Proceedings of the 2023 International Symposium on Physical Design , ser. ISPD ’23. New York, NY , USA: Association for Computing Machinery, 2023, p. 198–204. [Online]. Available: https://doi.org/10....
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.