{"id":"154345c6-5306-4645-826d-b1b2b411ca4f","arxiv_id":"2502.07014","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Academic quantum software research overemphasizes quantum-only bugs and underdelivers on the integration and compatibility issues that industry developers actually face.","lead":"This paper claims a disconnect between academic research and industry practice in quantum software engineering, with industry facing more integration and compatibility problems. It proposes collaboration models such as confidential research groups, government partnerships, and open-source projects to align the two sides.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The divide claim rests on 531 forum issues with no stated inclusion criteria; if those posts are mostly students or vendor-support threads, the 17.1% hybrid / 39% classical split is not evidence about industry.","rationale":"The reader's weakest_assumption is exactly the concern I find most load-bearing: the representativeness of the forum dataset. The abstract's central claim is a comparative statement about academic focus versus industry practice. The only quantified evidence for 'industry practice' comes from the 531 issues; without knowing who posts in the forums, the percentages cannot be interpreted as industry's concerns. The paper's own text acknowledges the developer community is small and secretive (Section II.B), which makes it even more important to establish that forum participants are actually industry practitioners. My concrete test would settle this by checking the demographic composition and re-computing the distribution. If the percentages hold after filtering to verified industry users, the claim is supported; if not, the paper's central assertion is overstated. The reader's CONDITIONAL verdict is appropriate: the proposals are standard and reasonable, but the empirical support is not yet adequate. I see no reason to change the verdict; hence UNCHANGED. Additional minor issues (the unresolved '[?]' citation in Section II.A and the lack of methodology from [14]) also support the conditional stance, but they are less load-bearing than the sampling concern.","tokens_in":8109,"tokens_out":4388,"duration_ms":38168,"concrete_test":"Ask the authors to release the forum list and inclusion criteria from [14], along with user-role metadata. Then independently re-annotate a random sample of 100 of the 531 issues using two coders and compute the distribution after excluding posts from users who self-identify as students/hobbyists or from vendor support accounts. If the 17.1% (hybrid) or 39.0% (classical) figures shift by more than 5 percentage points, the empirical basis for the divide claim is not robust to the forum population.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.B and III.C present the paper's only quantitative evidence for the central claim: 531 developer issues with 91 (17.1%) hybrid integration challenges and 207 (39.0%) classical issues such as library/platform compatibility. These counts are attributed to [14], an ICSE'25 paper that is 'to appear' and not included in this manuscript. The current text never states which forums were analyzed, how 'developer' was defined, how industry affiliation (as opposed to hobbyist, student, or vendor-support participation) was established, or how the issue taxonomy was applied. If forum participants are heavily weighted toward students and hobbyists or toward vendor support channels, the issue distribution will not represent industry practitioners. The same dataset also supports the repository scarcity claims in Section III.A, so the entire 'divide' observation rests on this unexplained sample. The comparison is also asymmetric: 'academic research predominantly focuses on a limited subset of quantum-specific issues' is supported only by citing a few papers [2,40–44], not by a systematic literature review, so the claimed gap may be overstated even if the forum data are accepted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This position paper argues that there is a substantial divide between academic research and industry practice in quantum software engineering (QSE). The authors claim that academic work on debugging and testing focuses on a narrow set of quantum-specific issues (e.g., qubit ordering, entanglement errors), whereas industry practitioners face a broader range of practical challenges, including integration, compatibility, and platform limitations in hybrid quantum-classical (HQC) applications. They attribute the divide primarily to secrecy and national security concerns in the quantum industry, which limit academic access to real-world practices. The paper supports the divide with quantitative observations from the authors' companion work [14]: 91 of 531 forum issues (17.1%) concern HQC integration, 207 (39.0%) concern classical issues, and 91% of 540 hybrid repositories are minimally active. Based on these observations, the paper proposes collaborative initiatives: confidential research collaboratives, government-funded partnerships, university QSE consortiums, and joint OSS projects, each intended to align academic research with industry needs. The paper is explicitly a position piece and does not present a new empirical study; its central quantitative evidence is drawn from the authors' to-appear ICSE'25 paper.","tokens_in":8272,"tokens_out":4529,"duration_ms":40331,"significance":"If the empirical divide claim holds, the paper addresses a timely and important problem in a nascent field. The proposed recommendations are concrete and actionable, and they could plausibly improve knowledge transfer between quantum software researchers and practitioners. The paper also cites independent practitioner-survey work ([38, 39]) that partially corroborates the existence of a gap, and it productively reframes academic QSE research as often oriented toward platform development rather than application development. A notable strength is the explicit identification of hybrid integration and classical compatibility as underrepresented but practically important topics, which could help steer future research agendas. However, the central quantitative evidence is not self-contained: the 531-issue and 540-repository figures are attributed entirely to a to-appear companion paper [14] with no methodology or dataset in this manuscript. The significance of the present paper is therefore conditional on the strength and representativeness of that companion study.","major_comments":[{"comment":"The quantitative underpinning of the central divide claim—91 of 531 issues (17.1%) for HQC integration and 207 of 531 issues (39.0%) for classical issues—is cited entirely to [14], which is listed as 'to appear' without an arXiv identifier or DOI. This manuscript never states which forums were analyzed, how 'developer' was defined, how industry affiliation (as opposed to hobbyist, student, or vendor-support participation) was established, or how the issue taxonomy was applied. Since these numbers carry the paper's core assertion, the reader cannot assess their validity. The authors should provide the essential methodology in an appendix or supplementary material, or explicitly frame these figures as preliminary results from a companion paper and temper the definitive language in the abstract and conclusions.","section":"§III.B and §III.C"},{"comment":"The text repeatedly contrasts 'industry practitioners' (or 'developers') with 'academic research,' but the evidence base is public developer forum posts. The paper does not justify why the forum population is representative of industry; forums often contain students, hobbyists, and vendor-support interactions, which would skew the distribution of issue types. The authors should acknowledge this threat to validity and soften the wording (e.g., 'practitioners and other forum participants') or provide evidence about the affiliations of the forum users. Without this qualification, the generalization from forum issues to 'industry practitioners face' is not established.","section":"§III (Observations of the Quantum Divide)"},{"comment":"The claim that academic research 'predominantly focuses on a limited subset of primarily quantum-specific issues' is supported by citing only five papers [2,40–44] on bug patterns and bug characteristics. This is not a systematic literature review, so it does not establish the quantitative predominance required for the divide claim. The comparison is also asymmetric: the industry side is quantified with 531 issues, while the academic side is illustrated with a handful of examples. The authors should either conduct or cite a systematic mapping of academic QSE research, or reframe the claim as 'a substantial body of academic work' rather than 'predominantly,' which is a quantitative assertion that needs corresponding evidence.","section":"§III.B–§III.D"}],"minor_comments":[{"comment":"The sentence 'The QC industry is currently led by IBM, which dominates both market share and influence [ ?]' contains a missing citation placeholder that must be filled before publication.","section":"§II.A"},{"comment":"Reference [14] is listed as 'to appear' without an arXiv identifier or DOI. If the companion paper is not yet publicly available, please provide a public preprint link or a copy so that reviewers can verify the cited statistics.","section":"References [14]"},{"comment":"The statement 'which we estimate to be only a few thousand professionals globally' is unsourced; the cited reference [34] concerns repository evolution, not community size. Please provide a source or explicitly mark this as an informal estimate.","section":"§II.A"},{"comment":"The claim 'as evidenced by our research [14] where we observed virtually no usage by practitioners of quantum testing and debugging tools developed in academia' is a strong empirical assertion that again depends on the inaccessible companion paper; consider reporting the relevant evidence in this manuscript or softening the wording.","section":"§I"},{"comment":"Given that the paper makes empirical claims from forum and repository data, a 'Threats to Validity' subsection would help readers understand the limitations of the evidence base. Currently, no such discussion is present.","section":"§III"},{"comment":"The abstract states 'By analyzing discussions within quantum developer forums, we identify key gaps...' but the forum analysis itself is not described in this paper. If the empirical work remains in [14], the abstract should attribute the analysis explicitly to that companion paper.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical foundation is almost entirely a self-citation to the authors' to-appear ICSE'25 paper [14]. This is not circular reasoning, but it makes the current manuscript non-self-contained for a central claim. Editors may wish to request a copy of [14] from the authors when sending this for review, and to verify that the companion paper has been accepted with the reported statistics. Also note the missing citation placeholder in §II.A. The paper fits a journal that publishes position papers, but if the journal expects a substantial empirical contribution, the current manuscript is too reliant on the companion study and would need to incorporate more of that evidence directly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is a clearly written position piece, and the divide it describes is probably real, but this manuscript cannot carry that claim on its own—all the empirical weight sits in the authors' separate ICSE'25 paper, which is cited but not included.\n\nWhat it does well: the proposals in Section V are concrete and sensible. Confidential research collaboratives, government-funded partnerships, university consortia, and collaborative OSS projects are standard mechanisms in other engineering fields, and the authors adapt them thoughtfully to quantum's secrecy and national-security constraints. The emphasis on hybrid quantum-classical issues as an under-studied area is timely, and the observation that academic bug studies lean quantum-specific while practitioners hit classical and integration problems is worth taking seriously. The prose is direct and the argument is easy to follow.\n\nWhere it is soft: Section III is the whole empirical case, and it is entirely a summary of [14], an ICSE'25 paper that is 'to appear.' There is no methodology here: no statement of which forums were analyzed, how 'developer' was defined, how industry affiliation was established, or how the taxonomy was applied. The stress-test concern is fair: if those 531 forum issues are weighted toward students, hobbyists, or vendor-support threads, the 17.1% hybrid / 39% classical split does not tell us much about industry practice. The comparison side is also asymmetric—the claim that academia focuses on a narrow set of quantum-specific issues is supported by citing a handful of papers, not a systematic review. There is also an unresolved citation placeholder (the '[?]' after IBM's name in Section II.A). None of this is fatal for a position paper, because the central divide is independently plausible and the full dataset appears to be on its way in the companion paper, but the current text overstates its own evidentiary basis.\n\nBottom line: this is for a reader thinking about quantum software engineering research directions or academia-industry partnerships. It deserves a serious referee—for a workshop or a practice-oriented venue—but the authors should be asked to either summarize the methodology from [14], add the missing citation, and qualify the representativeness of the forum data. I'd take the ICSE paper over this one for the numbers.","headline":"Readable position paper with sensible proposals, but the central divide evidence is borrowed from an unpublished companion paper and is under-supported here.","tokens_in":8829,"tokens_out":2503,"would_cite":false,"duration_ms":22015,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Academic quantum software research is out of step with the integration and compatibility problems industry developers face.","keywords":["quantum software engineering","academia-industry gap","hybrid quantum-classical systems","debugging and testing","developer forums","software integration","compatibility","position paper"],"falsifier":"A systematic survey of quantum developers in commercial settings that asks them to rank their top debugging and testing problems would falsify the divide claim if quantum-specific issues such as qubit ordering and entanglement errors are reported more frequently than integration, compatibility, and hybrid-system issues.","tokens_in":104,"feed_emoji":"⚛️","tokens_out":3979,"duration_ms":44784,"temperature":0.7,"pith_summary":"This position paper argues that academic research on quantum software engineering is studying a narrow slice of quantum-specific bugs while industry developers spend much of their time on integration, compatibility, and hybrid quantum-classical problems. The authors support this with data from their companion study: 531 real-world issues from quantum developer forums, of which 17.1% involve hybrid system integration and 39.0% are classical integration and platform problems, while academia concentrates on qubit ordering, entanglement, and circuit-level faults. They attribute the gap to industry secrecy, national-security contracts, and a small fragmented developer community. The paper proposes confidential research collaboratives, government-funded partnerships, university consortiums, and joint open-source projects as bridges. If the claim is right, aligning research with these practical concerns would make quantum software tools and methods actually useful to working developers.","feed_headline":"Quantum software research misses industry's real bugs","feed_subtitle":"Forum data: 17% hybrid integration and 39% classical issues, while academia studies quantum-only faults.","key_machinery":"The load-bearing evidence is the paper's own corpus of 531 developer-reported issues from quantum developer forums and 540 hybrid quantum-classical (HQC) repositories, drawn from the companion study. This corpus lets the authors compare the distribution of practitioner problems—17.1% hybrid integration issues and 39.0% classical issues such as library and platform compatibility—against the topics academic bug studies emphasize, such as qubit ordering, entanglement errors, and circuit design. The comparison, not any single experiment, carries the argument that academia and industry are focused on different problems.","core_discovery":"The paper's central claim is that a measurable disconnect exists between what quantum software engineering research studies and what quantum software practitioners actually struggle with. Using a corpus of 531 real-world developer issues, the authors report that hybrid quantum-classical integration challenges make up 17.1% of reported issues and classical problems such as library/platform compatibility, dependency handling, and integration account for 39.0%, while academic empirical studies concentrate on idealized quantum-specific bugs like qubit ordering and entanglement errors. The paper further observes that only 540 hybrid repositories were found, most of them inactive or educational, and that academic testing and debugging tools see virtually no use in industry. From these observations it concludes that academia's limited access to real industry practice, compounded by secrecy and national-security concerns, leaves the most pressing industrial challenges unaddressed.","pith_inferences":["The same pattern of academic focus on idealized, low-level bugs while industry wrestles with integration issues likely appears in other security-adjacent emerging technologies, so the proposed collaboratives could generalize beyond quantum computing.","A concrete testable extension would be to build a benchmark from the 531 forum issues and measure whether existing academic quantum testing and debugging tools detect any of them.","The paper's claim that academic tools see 'virtually no usage' could be quantified by mining dependency graphs of commercial quantum repositories for academic tool imports.","Even with successful collaboration, industry's need for speed-to-market may favor internal proprietary solutions over open best practices, so the proposed open-source ecosystem may require explicit incentives to attract commercial participation."],"forward_implications":["Quantum software engineering research would broaden its agenda to include hybrid-system integration, dependency handling, and compatibility testing rather than only quantum-only faults.","Academic testing and debugging tools, currently observed as largely unused, would be redesigned around industry workflows through joint projects and confidential research collaboratives.","Shared open-source quantum projects with mixed funding would give academic researchers real-world artifacts to study and give practitioners tested, documented solutions.","University-based quantum software engineering consortiums would train students in the practical integration and maintenance skills that industrial quantum development requires.","Government-funded, security-aware partnerships could open a channel for researchers to study industry challenges without exposing proprietary information."],"supporting_citations":[{"why":"Supplies the dataset of 531 developer issues and 540 hybrid repositories that underpins all the paper's quantitative observations.","marker":"[14]"},{"why":"Exemplifies the academic empirical tradition of studying bugs in quantum computing platforms, which the paper contrasts with practitioner concerns.","marker":"[2]"},{"why":"Represents academic bug-characteristic studies in the quantum software ecosystem that emphasize quantum-specific fault patterns.","marker":"[40]"},{"why":"Provides another academic benchmark of bug fixes in quantum programs, illustrating the field's focus on quantum-only defects.","marker":"[41]"},{"why":"Introduces a real-bug benchmark for quantum programs, further evidencing the academic emphasis on low-level quantum faults.","marker":"[43]"},{"why":"Shows practitioner insights into quantum software engineering challenges and the difficulty of obtaining industry participation.","marker":"[39]"}],"fun_headline_variants":["Quantum software research skips 56% of real-world issues","Academia's quantum software focus misses hybrid and classical bugs","Quantum devs face integration and classic issues, research doesn't","Forum data: quantum software research is out of touch with industry"],"cache_read_input_tokens":11008,"weakest_assumption_plain":"The claim rests on the assumption that the 531 forum issues and 540 repositories studied are representative of what industry quantum developers actually struggle with, even though the forums' participants and selection criteria are not described.","fun_headline_variants_meta":{"raw":{"variants":["Quantum software research skips 56% of real-world issues","Academia's quantum software focus misses hybrid and classical bugs","Quantum devs face integration and classic issues, research doesn't","Forum data: quantum software research is out of touch with industry"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000348,"raw_usage":{"total_tokens":1857,"prompt_tokens":849,"completion_tokens":1008,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":938}},"tokens_in":465,"tokens_out":1008,"duration_ms":9369,"temperature":1.0,"reasoning_tokens":938,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T14:01:20.055355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A systematic survey of quantum developers in commercial settings that asks them to rank their top debugging and testing problems would falsify the divide claim if quantum-specific issues such as qubit ordering and entanglement errors are reported more frequently than integration, compatibility, and hybrid-system issues.","supporting_citations":[{"cited_title":"When quantum meets classical: Characterizinghybrid quantum-classical issues discussed in developer forums,","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset of 531 developer issues and 540 hybrid repositories that underpins all the paper's quantitative observations."},{"cited_title":"Bugs in quantum computing platforms: An empirical study,","cited_arxiv_id":null,"evidence_quote":"Exemplifies the academic empirical tradition of studying bugs in quantum computing platforms, which the paper contrasts with practitioner concerns."},{"cited_title":"Bug Characteristics in Quantum Software Ecosystem","cited_arxiv_id":"2204.11965","evidence_quote":"Represents academic bug-characteristic studies in the quantum software ecosystem that emphasize quantum-specific fault patterns."},{"cited_title":"A comprehensive study of bug fixes in quantum programs,","cited_arxiv_id":null,"evidence_quote":"Provides another academic benchmark of bug fixes in quantum programs, illustrating the field's focus on quantum-only defects."},{"cited_title":"Bugs4q: A benchmark of real bugs for quantum programs,","cited_arxiv_id":null,"evidence_quote":"Introduces a real-bug benchmark for quantum programs, further evidencing the academic emphasis on low-level quantum faults."},{"cited_title":"Quantum software engineering issues and challenges: Insights from practitioners,","cited_arxiv_id":null,"evidence_quote":"Shows practitioner insights into quantum software engineering challenges and the difficulty of obtaining industry participation."}],"review_version":1}