REVIEW 4 major objections 5 minor 32 references
Empirical Demonstration of Quantum Contextuality on NISQ Computers
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Quantum contextuality is demonstrated on IBM Heron R2 processors by beating the Rio Negro inequality and winning Mermin pseudo-telepathy games.
desk verdict Fresh Heron R2 data for Rio Negro is promising, but the Mermin game claim is undercut by the paper's own Table 2. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machine that carries the argument is a finite geometry whose vertices are $N$-qubit Pauli operators and whose lines are triples of pairwise-commuting operators with product $\pm 1$; each line forms a context whose outcome products must satisfy the geometry's constraints. The paper uses the symplectic polar space $W(5,2)$ of all 63 nontrivial 3-qubit operators and its subgeometries — the Peres-Mermin magic square, the doily $W(3,2)$, and elliptic and hyperbolic quadrics — each with a contextuality degree $d$, the minimum number of unsatisfiable line constraints. The Rio Negro inequality evaluates $\chi = \sum_i \langle C_i\rangle - \sum_i \langle C'_i\rangle$ over positive and negative contexts, with bounds $L-2d$ for NCHV models and $L$ for quantum mechanics. The Mermin-like games convert the same line structure into referee-chosen contexts for two or four players, with classical bounds on winning probability derived from the minimal unsatisfied constraints.
What would settle it
Run the same compiled circuits after replacing the ideal operators by a noncontextual assignment of $\pm 1$ to each vertex while keeping the gate counts and error rates identical; if a hardware noise model reproduces $\chi$ or game-success values above the NCHV bound, the claimed violation would not be established.
Extended reading notes
Core claim
The central claim is that the newest IBM Heron R2 processors violate non-contextual hidden-variable (NCHV) bounds in both the Rio Negro inequality and Mermin-like pseudo-telepathy games. The measured quantities $\chi$ exceed the NCHV bounds $L-2d$ in all five geometry classes reported for the Rio Negro test: $W(5,2)$ itself (264.22 vs 189), squares (5.31 vs 4), doilies (13.01 vs 9), elliptic quadrics (38.14 vs 27), and hyperbolic quadrics (89.10 vs 63). For the Mermin-like line-line, point-line, and four-player games, the reported NISQ success rates surpass the corresponding classical success bounds. The authors state that these are the first violations of the classical Mermin game on IBM NISQ computers and the largest such violations of the Rio Negro inequality.
Load-bearing premise
The whole comparison assumes that the circuits compiled for the Heron R2 processors realize the intended pairwise-commuting Pauli measurements with only the modeled noise, so any excess over the bound reflects contextuality rather than systematic hardware error.
Editorial extensions
If this is right
- If the results are correct, current NISQ hardware can serve as a testbed for foundational contextuality tests without error correction, since the measured $\chi$ values and game success rates clear the classical bounds.
- The finite-geometry framework yields many independent contexts from a single polar space, so subsequent tests can compare dozens of subgeometries on the same device and identify the best-performing configurations.
- The reported improvements over earlier Eagle-processor runs suggest that newer hardware generations are steadily closer to the ideal quantum values for these tests.
- A demonstrated Mermin-game win on NISQ hardware strengthens the case that pseudo-telepathic games can be used as practical certification tests for quantum behaviour.
Reading between the lines
- A natural next step is to move to the 4-qubit symplectic polar space $W(7,2)$, whose far larger collection of contexts would sharpen the statistical separation but would also demand stronger error mitigation.
- Randomizing the order of contexts and repeating the runs on different qubit mappings would test whether the excess over the bound survives calibration drift, which would address the weakest assumption directly.
- If the same protocols were run with randomized compiling, a surviving violation would be much harder to attribute to a fixed systematic gate error, providing a cleaner contextuality certification.
- The hardware-specific nature of these results leaves open whether other vendors' NISQ devices would show the same violations, making these tests a candidate cross-platform quantumness benchmark.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports empirical tests of quantum contextuality on IBM Heron R2 NISQ processors, using the Rio Negro inequality and Mermin-like pseudotelepathy games. The authors claim 'definitive violations' of non-contextual hidden variable (NCHV) bounds, the 'first violations of the classical Mermin game on IBM NISQ computers,' and the largest Rio Negro violations to date. The evidence consists of two tables: Table 1 gives Rio Negro χ values for several geometries, and Table 2 gives Mermin-game success rates for point-line and line-line games on squares, doilies, and elliptic quadrics, along with noiseless and noisy simulator comparisons. The central claim, however, is directly contradicted by the data in Table 2, where every reported NISQ success rate is at or below the corresponding NCHV upper bound stated in the text.
Significance. If the empirical claims were correct, the paper would be the first demonstration of a Mermin-game violation on IBM NISQ hardware and would set a new benchmark for Rio Negro inequality violations. The geometric classification of Mermin-like games and the use of finite geometries to organize tests are conceptually interesting, and the authors provide open-source code and simulator comparisons. However, the main empirical assertion is unsupported by the paper's own reported data, and the internal inconsistencies between the text, Table 2, and Figure 3 are severe. The paper's significance as a demonstration of contextuality therefore collapses unless the authors can supply different data that actually exceed the NCHV bounds.
major comments (4)
- [Section 3, Table 2] The central claim of Mermin-game violations is contradicted by the data. The text states that 'the indicated backend σNISQ success rates violate the NCHV upper bounds ω,' but every listed σNISQ value is below or equal to the corresponding bound: pl Square 0.94 < 17/18 ≈ 0.9444, pl Doily 0.93 < 14/15 ≈ 0.9333, ll Square 0.88 < 8/9 ≈ 0.8889, ll Doily 0.86 < 13/15 ≈ 0.8667, ll E_YYY 0.86 < 13/15 ≈ 0.8667, and llll E_YYY 0.73 < 11/15 ≈ 0.7333. Therefore the measured NISQ success rates do not demonstrate any violation of the NCHV bounds, and the abstract's claim that these are 'the first violations of the classical Mermin game on IBM NISQ computers' is unsupported by the paper's own Table 2.
- [Figure 3 and Table 2] There is a direct internal contradiction between Figure 3 and Table 2. The Figure 3 caption states that in the ll-square extractions 'all but one square violating its bound,' yet Table 2 reports the 'best performing results shown' for the ll Square as 0.88, which is below the NCHV bound of 8/9 ≈ 0.8889. If the best-performing square truly exceeded the bound, the reported best value should be above 0.8889. Either the table entry is not the maximum of the distribution, or the caption overstates the number of violating instances. This inconsistency undermines the reliability of the reported results.
- [Section 2.1, Table 1] The Rio Negro inequality results in Table 1 are presented without any error bars, confidence intervals, or statistical significance tests. With only 10,000 shots per context, the finite-sample uncertainty on χ is non-negligible; for example, a binomial error of roughly 0.003 on each ⟨C_i⟩ translates into an uncertainty of several units on χ for the larger geometries. Since the paper claims 'definitive violations' and 'largest such violations', the absence of statistical analysis makes it impossible to assess whether the reported margins over the NCHV bounds are meaningful or consistent across runs. A quantitative error analysis is required to support the strength of the claim.
- [Section 3, paragraph 3] The logic of the Mermin-game test is misapplied in the interpretation of the results. The text correctly states that success rates above the NCHV bounds rule out NCHV models, but the measured σNISQ values are not above those bounds. The subsequent claim that 'in all games... σNISQ success rates violate the NCHV upper bounds' is therefore not a valid inference from the table. This is a load-bearing error: the paper's primary new experimental result is not established by the data shown.
minor comments (5)
- [Table 2] Table 2 contains typographical artifacts such as '0 .95628' instead of '0.95628'; these should be corrected.
- [Table 2] The column headed 'Results ω' is not defined in the text or caption. Please clarify whether it denotes the ideal quantum winning probability (which would be 1.0 for these games) or an NCHV bound, and how it relates to the succeeding columns.
- [Figure 2 and Figure 3] The histogram axes are not labeled with the number of subgeometries or the binning; adding these would help readers understand whether the distributions cover all 3360 squares, 1344 doilies, and so on, or only a subset.
- [Section 2.1] The experimental protocol is only described by reference to [18], [19], and [20]; the paper should include a self-contained description of the circuit construction, readout calibration, and error mitigation used, since the validity of the contextuality claim depends on these details.
- [Abstract and Conclusion] The wording 'definitive violations' and 'first success' overstates the evidence as presented; the data do not support such unequivocal language even if the underlying measurements were correct.
Circularity Check
No material circularity: hardware-measured statistics are compared against fixed geometry-derived NCHV thresholds; self-citations supply only protocols and game constructions, and the Table 2 vs. abstract discrepancy is a presentation defect, not a reduction.
full rationale
The derivation chain is not circular. The Rio Negro statistic chi (Eq. 1) is a sum of measured context expectation values, and its NCHV ceiling L-2d (Eq. 2) is a fixed geometric constant determined by the contextuality degree d of each geometry (Sec. 1.1), computed in the cited literature (e.g., the degree-63 split-Cayley-hexagon result for W(5,2)) and reproduced on a noiseless simulator (chi_sim); it is never fitted to or derived from the hardware data. Likewise, the Mermin-game thresholds omega_pl(S)=17/18, omega_pl(D)=14/15, omega_ll(S)=8/9, omega_ll(D)=13/15, omega_ll(E)=13/15, and omega_llll(E)=11/15 are static fractions of unsatisfiable line assignments of the geometries (Sec. 3), argued in-text for the square and asserted for the other geometries; sigma_NISQ is a measured win rate, so comparing the two does not reduce a prediction to its input. Self-citations [18,19,20] are used as tools: [18] supplies the Rio Negro protocol and the W(5,2) degree result; [19,20] supply the game constructions and their bounds. These are parameter-free, geometrically checkable inputs that do not contain the target outcome (Heron R2 violation rates), so under the review rules they are independent support rather than load-bearing circularity; the paper's new content is externally falsifiable hardware data. Per the reviewing rule I flag two in-text defects that are not circularity: (i) Table 2's displayed sigma_NISQ values (0.94, 0.93, 0.88, 0.86, 0.86, 0.73) are all below the Sec. 3 bounds (0.9444, 0.9333, 0.8889, 0.8667, 0.8667, 0.7333), while the caption claims all violate the bounds; rounding to two decimals could in principle hide violations (e.g., a value of 0.9445 rounds to 0.94 while exceeding 17/18), but the unrounded numbers are not reported, so the abstract's 'first violations of the classical Mermin game' claim is under-supported by the table as printed, though Fig. 3 shows individual square extractions above their bound; (ii) this is a data-presentation consistency issue, not an Eq.-A = Eq.-B reduction, so it does not raise the circularity score. Score 1 reflects only the mild reliance on the authors' own prior work for protocols and game thresholds.
Assumptions & free parameters
assumptions (4)
- standard math NCHV bound for the Rio Negro inequality is chi <= L - 2d for a geometry with L lines and contextuality degree d.
- domain assumption The finite geometries W(5,2), squares, doilies, and quadrics have the stated line counts L and degrees d.
- domain assumption Circuits on Heron R2 implement pairwise commuting Pauli measurements as compatible measurements with no crosstalk beyond the included noise model.
- standard math The quoted classical success probabilities for Mermin-like games (8/9, 13/15, 17/18, 14/15, 11/15) are the correct NCHV maxima.
Cite this review
Pith. "Pith review of Empirical Demonstration of Quantum Contextuality on NISQ Computers." pith.science (2026). https://pith.science/paper/AQAL6JT6
@misc{pith2026250521243,
author = {Pith},
title = {Pith review of: Empirical Demonstration of Quantum Contextuality on NISQ Computers},
year = {2026},
howpublished = {\url{https://pith.science/paper/AQAL6JT6}},
note = {Machine review of arXiv:2505.21243}
}
read the original abstract
We present definitive violations of non-contextual hidden variable bounds in the latest generation of IBM noisy intermediate-scale quantum computers (NISQ). These violations are based on known tests for contextuality such as the Rio Negro inequality and pseudo-telepathic Mermin games. These are the first violations of the classical Mermin game on IBM NISQ computers, and the largest such violations for the Rio Negro inequality. The use of finite geometries proves instrumental in the development of more effective tests, with larger geometries providing sizeable datasets from which multiple distinct experiments can be compared.
Figures
Reference graph
Works this paper leans on
-
[1]
On the Problem of Hid- den Variables in Quantum Mechanics
John S. Bell. “On the Problem of Hid- den Variables in Quantum Mechanics”. In: Reviews of Modern Physics38.3 (July 1966). Publisher: American Physical So- ciety, pp. 447–452. doi: 10 . 1103 / RevModPhys.38.447. url: https://link. aps.org/doi/10.1103/RevModPhys.38. 447
-
[2]
The problem of hidden variables in quan- tum mechanics
Simon Kochen and Ernst P. Specker. “The problem of hidden variables in quan- tum mechanics”. In: The logico-algebraic approach to quantum mechanics 1975 . Springer, 1975, pp. 293–328
work page 1975
-
[3]
Converting contextuality into nonlocality
Ad´ an Cabello. “Converting contextuality into nonlocality”. In: Physical Review Let- ters 127.7 (Aug. 2021). arXiv:2011.13790 [quant-ph], p. 070401. issn: 0031-9007, 1079-7114. doi: 10 . 1103 / PhysRevLett . 127 . 070401. url: http : / / arxiv . org / abs/2011.13790
work page Pith review arXiv 2021
-
[5]
Contextuality sup- plies the magic for quantum computa- tion
Mark Howard et al. “Contextuality sup- plies the magic for quantum computa- tion”. In: Nature 510.7505 (June 2014). arXiv:1401.4174 [quant-ph], pp. 351–355. issn: 0028-0836, 1476-4687. doi: 10.1038/ nature13460. url: http : / / arxiv . org / abs/1401.4174
arXiv 2014
-
[6]
Random access codes via quantum contextual redundancy
Giancarlo Gatti et al. “Random ac- cess codes via quantum contextual re- dundancy”. In: Quantum 7 (Jan. 2023). arXiv:2103.01204 [quant-ph], p. 895. issn: 2521-327X. doi: 10 . 22331 / q - 2023 - 01 - 13 - 895. url: http : / / arxiv . org / abs / 2103.01204
work page Pith review arXiv 2023
-
[7]
Y. Aharonov et al. The quantum pigeonhole principle and the nature of quantum cor- relations. arXiv:1407.3194 [quant-ph]. July
-
[8]
Contextuality, coherences, and quantum Cheshire cats
Jonte R Hance, Ming Ji, and Holger F Hofmann. “Contextuality, coherences, and quantum Cheshire cats”. In: New Journal of Physics 25.11 (Nov. 2023), p. 113028. issn: 1367-2630. doi: 10 . 1088 / 1367 - 2630/ad0bd4. url: https://iopscience. iop.org/article/10.1088/1367- 2630/ ad0bd4
doi:10.1088/1367- 2023
-
[9]
C. Budroni et al. “Quantum contextual- ity”. In: arXiv preprint arXiv:2102.13036 (2021)
arXiv 2021
Show all 32 references
-
[10]
State-independent experimental test of quantum contextu- ality
G. Kirchmair et al. “State-independent experimental test of quantum contextu- ality”. en. In: Nature 460.7254 (July 2009), pp. 494–497. issn: 0028-0836, 1476-
2009
-
[11]
Synchronous Observation of Bell Nonlocality and State-Dependent Contextuality
Peng Xue et al. “Synchronous Observation of Bell Nonlocality and State-Dependent Contextuality”. In: Physical Review Let- ters 130.4 (Jan. 2023). Publisher: Ameri- can Physical Society, p. 040201. doi: 10. 1103 / PhysRevLett . 130 . 040201. url: https://link.aps.org/doi/10.110...
2023
-
[12]
Experimental Demon- stration of Quantum Pseudotelepathy
Jia-Min Xu et al. “Experimental Demon- stration of Quantum Pseudotelepathy”. In: Physical Review Letters129.5 (July 2022). Publisher: American Physical Society, p. 050402. doi: 10 . 1103 / PhysRevLett . 129 . 050402. url: https : / / link . aps . org / doi / 10 . 1103 / PhysRe...
2022
-
[13]
Experimental Test of Quantum Contextuality in Neutron Inter- ferometry
H. Bartosik et al. “Experimental Test of Quantum Contextuality in Neutron Inter- ferometry”. In: Phys. Rev. Lett. 103 (4 July 2009), p. 040403. doi: 10 . 1103 / PhysRevLett . 103 . 040403. url: https : / / link . aps . org / doi / 10 . 1103 / PhysRevLett.103.040403
2009
-
[14]
Experimental ver- ification of quantum contextuality using a weak measurement in a single trapped ion
Chunwang Wu et al. “Experimental ver- ification of quantum contextuality using a weak measurement in a single trapped ion”. In: Phys. Rev. A109 (3 Mar. 2024), p. 032211. doi: 10.1103/PhysRevA.109. 032211. url: https : / / link . aps . org / doi/10.1103/PhysRevA.109.032211
2024 doi
-
[15]
A demonstration of contextuality using quantum computers
Amine Laghaout et al. “A demonstration of contextuality using quantum computers”. In: European Journal of Physics43.5 (Sept. 2022), p. 055401. issn: 0143-0807, 1361-
2022
-
[16]
Testing quantum con- textuality of binary symplectic polar spaces on a Noisy Intermediate Scale Quantum Computer
Fr´ ed´ eric Holweck. “Testing quantum con- textuality of binary symplectic polar spaces on a Noisy Intermediate Scale Quantum Computer”. en. In: Quantum Information Processing 20.7 (July 2021), p. 247. issn: 1573-1332. doi: 10 . 1007 / s11128 - 021 - 03188 - 9. url: https : /...
2021
-
[17]
url: https : / / quantum - computing.ibm.com/
IBM Quantum. url: https : / / quantum - computing.ibm.com/
-
[18]
Hexagons govern three-qubit contextuality
Metod Saniga et al. “Hexagons govern three-qubit contextuality”. In: Quantum 9 (2025), p. 1601. 7
2025
-
[19]
Implementing 2- qubit pseudo-telepathy games on noisy intermediate-scale quantum computers
Colm Kelleher, Mohammad Roomy, and Fr´ ed´ eric Holweck. “Implementing 2- qubit pseudo-telepathy games on noisy intermediate-scale quantum computers”. en. In: Quantum Information Processing 23.5 (May 2024), p. 187. issn: 1573-1332. doi: 10.1007/s11128-024-04386-x . url: https:...
2024 doi
-
[20]
Exploiting finite geometries for better quantum advantages in Mermin- like games
Colm Kelleher, Fr´ ed´ eric Holweck, and P´ eter L´ evay. “Exploiting finite geometries for better quantum advantages in Mermin- like games”. In: Journal of Physics A: Mathematical and Theoretical57.24 (June 2024), p. 245303. issn: 1751-8113, 1751-
2024
-
[21]
Incompatible results of quan- tum measurements
Asher Peres. “Incompatible results of quan- tum measurements”. In: Physics Letters A 151.3-4 (1990), pp. 107–108
1990
-
[22]
Simple unified form for the major no-hidden-variables theorems
N David Mermin. “Simple unified form for the major no-hidden-variables theorems”. In: Physical review letters 65.27 (1990), p. 3373
1990
-
[23]
Hidden variables and the two theorems of John Bell
N. David Mermin. “Hidden variables and the two theorems of John Bell”. In:Reviews of Modern Physics65.3 (1993), p. 803
1993
-
[24]
Contextuality de- gree of quadrics in multi-qubit symplectic polar spaces
Henri de Boutray et al. “Contextuality de- gree of quadrics in multi-qubit symplectic polar spaces”. en. In: Journal of Physics A: Mathematical and Theoretical 55.47 (Nov. 2022). arXiv:2105.13798 [math-ph, physics:quant-ph], p. 475301. issn: 1751- 8113, 1751-8121. doi: 10 . 10...
2022 arXiv
-
[25]
Multi-qubit doilies: enu- meration for all ranks and classification for ranks four and five
Axel Muller et al. “Multi-qubit doilies: enu- meration for all ranks and classification for ranks four and five”. en. In: Journal of Computational Science 64 (Oct. 2022). arXiv:2206.03599 [quant-ph], p. 101853. issn: 18777503. doi: 10 . 1016 / j . jocs . 2022.101853 . url: htt...
2022 arXiv
-
[26]
Proposed test of macro- scopic quantum contextuality
Ad´ an Cabello. “Proposed test of macro- scopic quantum contextuality”. In: Physi- cal Review A82.3 (Sept. 2010). Publisher: American Physical Society, p. 032110. doi: 10 . 1103 / PhysRevA . 82 . 032110. url: https://link.aps.org/doi/10.1103/ PhysRevA.82.032110
2010
-
[27]
Bell’s theorem without in- equalities and without probabilities for two observers
Ad´ an Cabello. “Bell’s theorem without in- equalities and without probabilities for two observers”. In: Physical review letters86.10 (2001), p. 1911
2001
-
[28]
Mul- tiple qubits as symplectic polar spaces of order two
Metod Saniga and Michel Planat. “Mul- tiple qubits as symplectic polar spaces of order two”. In: arXiv preprint quant- ph/0612179 (2006). 8
2006
-
[2014]
url: http://arxiv.org/abs/1407. 3194
-
[2362]
issn: 1099-4300. doi: 10 . 3390 / e15062340. url: http://arxiv.org/abs/ 1501.04342. 6
-
[4687]
1038 / nature08172
doi: 10 . 1038 / nature08172. url: https : / / www . nature . com / articles / nature08172
-
[6404]
url: https : / / iopscience
doi: 10.1088/1361- 6404/ac79e0 . url: https : / / iopscience . iop . org / article/10.1088/1361-6404/ac79e0
-
[8121]
url: https : / / iopscience
doi: 10.1088/1751- 8121/ad4d2d . url: https : / / iopscience . iop . org / article/10.1088/1751-8121/ad4d2d
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.