Pith. sign in

REVIEW 4 major objections 6 minor 5 references

Quantum at a Music Festival: the Impact of an Exhibit about Quantum Science and Technologies on Festival Visitors

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 20-minute quantum exhibit raised visitors' felt understanding while lowering their curiosity.

desk verdict A solid, honest pre-post study of a quantum exhibit at a festival; the knowledge gain and interest drop are worth knowing about, but the causal framing needs a control-less caveat. read the letter →

arxiv 2507.13010 v1 pith:L76ITL5T submitted 2025-07-17 physics.soc-ph physics.ed-phquant-ph

Vincent Koeman , Sanne Romp , Sanne J.W. Willems , Julia Cramer This is my paper · ORCID
classification physics.soc-phphysics.ed-phquant-ph
keywords quantumscienceandtechnologyoutreachexhibitionpublicengagementmusicfestivalquantitativesurveysubjectiveknowledgecapitalpre-poststudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a short, informal outreach exhibit can change how a general audience relates to quantum science and technology, and that the changes are not all in the direction outreach organisers expect. Using paper questionnaires from 812 visitors to a pop-up exhibit at a Dutch music festival, the authors compared self-reported attitude, concern, interest, and subjective knowledge before and after the roughly 20-minute visit. They report a large rise in perceived knowledge, a moderate fall in interest, and very small rises in attitude and concern. If the results hold, they matter for how science communication designs exhibits for emerging technologies: a brief exposure can make people feel more informed while making them less eager to learn more.

What carries the argument

The load-bearing instrument is the paired pre–post survey: the same four Likert-scale batteries (attitude, concern, interest, subjective knowledge) were administered immediately before and after the exhibit, and composite means were compared with paired two-sided t-tests using Welch's approximation. The exhibit itself—13 bilingual panels with two videos and four interactive elements covering superposition, wave–particle duality, quantum computing, cryptography, and sensing—is the treatment. 'Science capital,' measured by four self-report items about awareness, spare-time science activity, discussion habits, and interest in the scientific process, is the moderator variable the authors use to test whether prior engagement with science changes the size of the effects.

What would settle it

A control condition in which festival visitors fill out the same two surveys without seeing the exhibit: if the same pattern of interest decline and subjective-knowledge gain appears, the claim that the exhibit caused these changes is falsified.

Watch

Extended reading notes

Core claim

The central claim is that a single visit to an informal science exhibit produces measurable, and mixed, effects on engagement with quantum technologies. In paired comparisons of 812 festival visitors' five-point scale responses, mean subjective knowledge rose from 2.1 to 2.8 ($d = 0.80$), mean interest fell from 3.9 to 3.5 ($d = -0.44$), while attitude (3.4 to 3.5, $d = 0.08$) and concern (2.5 to 2.7, $d = 0.18$) increased only slightly. The authors interpret the knowledge gain as evidence that exhibits can quickly make audiences feel more capable of engaging with quantum topics, and the interest drop as a probable consequence of reduced novelty or increased perceived difficulty once the material is actually encountered. They further report that higher science capital was weakly associated with smaller positive shifts in attitude, interest, and subjective knowledge.

Load-bearing premise

The results assume that the measured before–after shifts come from the exhibit itself, not from taking the same survey twice, from festival mood or intoxication, or from the fact that only already science-minded visitors chose to take part.

Editorial extensions

If this is right

  • Short, self-paced exhibits can substantially raise visitors' perceived understanding of an unfamiliar technology in a single visit.
  • Outreach about emerging technologies can lower self-reported interest, and designers should treat interest loss as a possible outcome rather than assuming any exposure is motivating.
  • The pattern of results suggests that reducing the breadth of content and linking quantum topics to everyday life may counteract the interest decline.
  • Measuring outreach impact with pre–post surveys can reveal opposing effects on different engagement dimensions, making such evaluation worthwhile for science communication practice.
  • Higher prior science engagement is at most weakly linked to smaller gains from the exhibit, implying the exhibit's effects do not strongly favour already science-oriented visitors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A plausible reading the paper leaves implicit is that the subjective-knowledge gain may reflect confidence rather than actual comprehension; an objective knowledge test would separate these possibilities.
  • The interest decline could partly be a measurement artifact: asking 'how much do you want to know more' immediately after showing the material sets a higher informational bar, and a control question wording could test this.
  • For exhibit designers, the findings suggest ending with open questions, unanswered puzzles, or concrete personal applications might preserve curiosity after the felt-knowledge boost.
  • The lack of a no-exhibit control group means the same before–after pattern could appear among festival-goers who simply take the survey twice, so a control condition is the natural next test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This manuscript reports a single-group pre-post survey of 812 visitors to a quantum science and technology pop-up exhibit at the 2024 Lowlands music festival. The authors measured attitude, concern, interest, and subjective knowledge before and after the visit, along with science capital. They report a large increase in subjective knowledge (d = 0.80), a moderate decrease in interest (d = 0.44), a small increase in concern (d = 0.18), and a negligible increase in attitude (d = 0.08), with weak negative correlations between science capital and several changes. The paper interprets these results as evidence that the exhibit was effective as an outreach tool and derives practical recommendations for future quantum outreach.

Significance. If the observed pre-post differences were causally attributable to the exhibit, this would be a valuable contribution to the sparse evaluation literature on quantum science outreach, especially because the study is preregistered, uses a large sample in a real informal setting, and openly shares materials, data, and analysis scripts. The unexpected negative interest effect is an interesting and testable finding. However, the single-group design, the demand-laden post-survey wording, and the post hoc scale modification mean that the current evidence supports only descriptive pre-post changes, not causal claims about outreach effectiveness.

major comments (4)
  1. [Procedure; Measures; Abstract] The central claim that the exhibit produced the observed changes is not supported by the design. There is no control or comparison condition, and the post-survey items explicitly credit the exhibit (e.g., 'After visiting the exhibit, I want to know more about how quantum technologies work' and 'After visiting the exhibit, how confident are you in your understanding of superposition?'). This wording invites demand effects, and retesting, regression to the mean, festival mood, and self-selection of a highly educated, science-interested sample are all plausible alternative explanations. The Limitations section acknowledges some of these threats but does not quantify them or temper the causal language in the Abstract, Results, and Discussion. The manuscript should be reframed as a descriptive pre-post observational study, with all causal inferences either removed or explicitly labeled as tentative, and the abstract's statement that the findings 'underscore the effectiveness of exhibits' should be revised accordingly.
  2. [Results, concern scale paragraph] The concern scale was modified after inspecting Cronbach's alpha: the item 'I have no concerns about quantum technologies because the benefits are likely greater than the risks' was omitted because alpha improved from .44/.60 to .69/.77. This post hoc deletion is not described as preregistered, and it can inflate the apparent concern effect. The authors should report the full-scale results as a sensitivity analysis, state whether the deletion was specified in the preregistration, and note that the d = 0.18 concern effect depends on this item-selection decision.
  3. [Statistical Analysis; Table 1] Four paired t-tests are conducted without any correction for multiple comparisons. The attitude effect (p = 0.030, d = 0.08) would not survive a simple Bonferroni correction (threshold 0.0125), and the concern effect is based on a scale that was modified post hoc. The authors should report adjusted p-values, confidence intervals, or use a false discovery rate approach, and interpret the small effects accordingly. This is particularly important because the abstract and discussion emphasize significant results.
  4. [Measures; Discussion, 'Moderate decrease in interest'] The interest decrease may be partly mechanical. The interest items include 'I want to know more about how quantum technologies work,' which is satiation-sensitive: after an exhibit that raises subjective knowledge, respondents may report lower desire to learn more simply because they feel they have already learned something. The subjective knowledge items measure confidence, not verifiable knowledge, so the large d = 0.80 gain could reflect perceived competence rather than actual learning. The Discussion should address this measurement confound explicitly; a neutral post-wording replication or an independent novelty/difficulty measure would be needed to support the interpretation that genuine interest declined.
minor comments (6)
  1. [Participants] The text states that 76.9% (N = 758) of participants were highly educated, but 758/812 is approximately 93.3%. One of these numbers is incorrect and should be fixed.
  2. [Results, subjective knowledge] The post-visit subjective knowledge mean (M = 2.8) is still below the scale midpoint of 3, so describing the change as a 'substantial increase' may overstate the final level; consider saying 'a large increase from a low baseline.'
  3. [Discussion, exploratory correlations] The correlations between baseline measures and change scores (e.g., Figure 3) are likely influenced by regression to the mean and should be interpreted with caution; the text should not present them as evidence of causal relationships.
  4. [Materials and Methods, last paragraph] There is a typo: 'We close of this section' should read 'We close off this section.'
  5. [Acknowledgements] The sentence 'Additionally, thank the staff of New Scientist...' is missing a subject; it should read 'Additionally, we thank the staff...'.
  6. [References] Reference formatting is inconsistent: 'de Wolf, R.' is alphabetized under 'd' while some other entries use different capitalization conventions; this should be harmonized with the journal style.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study reports direct pre-post survey measurements with no fitted parameters, derived predictions, or load-bearing self-citations.

full rationale

This paper is an empirical survey study that measures changes in attitude, concern, interest, and subjective knowledge before and after an exhibit at a music festival. There is no derivation chain in which a predicted quantity reduces to an input by construction. The pre-post comparisons are arithmetic differences of directly administered Likert-scale items, not outputs of a model fitted to the same data. No parameter is fitted and then renamed a prediction, and no uniqueness theorem or ansatz is imported from prior work. The only notable self-citation is the reference to Meinsma et al. (2024), which shares an author with the present paper; however, that work is used exclusively as an external comparative finding in the Discussion and Future research sections, and none of the paper's reported effect sizes or t-test results depend on it. Concerns about the lack of a control condition, demand characteristics in the wording of post-survey items, self-selection, and festival context are validity threats to the causal interpretation of the observed changes, but they are not instances of circular reasoning as defined here: the reported outcome values are not constructed to equal their own inputs. The paper also includes a preregistration and openly shared data and analysis scripts, and its statistical claims are transparently based on the measured responses. Therefore, no circular step is present, and the honest finding is a score of 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the validity of self-reported Likert data in a festival context and on the assumption that uncontrolled pre-post differences are caused by the exhibit. No free parameters are fitted, and no new entities are postulated.

assumptions (3)
  • domain assumption Self-report Likert items validly measure the four engagement constructs (attitude, concern, interest, subjective knowledge).
    Scales are adapted from prior studies, but their validity in a noisy festival setting is not established; only internal consistency is reported, and one concern item was dropped post hoc.
  • domain assumption Pre-post differences reflect exhibit exposure rather than retesting or context.
    No control group; authors list festival mood, intoxication, and self-selection as possible confounders in Limitations.
  • standard math Large sample size makes paired t-tests robust to non-normality.
    This relies on the central limit theorem, though ordinal Likert composite scores approach interval-level only under additional assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantum at a Music Festival: the Impact of an Exhibit about Quantum Science and Technologies on Festival Visitors." pith.science (2026). https://pith.science/paper/L76ITL5T

@misc{pith2026250713010,
  author       = {Pith},
  title        = {Pith review of: Quantum at a Music Festival: the Impact of an Exhibit about Quantum Science and Technologies on Festival Visitors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L76ITL5T}},
  note         = {Machine review of arXiv:2507.13010}
}
read the original abstract

Quantum technologies are seen as transformative, with a potential to revolutionize fields like drug discovery and machine learning. Public engagement is crucial to align these developments with societal needs and foster acceptance. This study measured the impact of an exhibit about quantum technologies at the 2024 Lowlands music festival (n = 812). Pre- and post-surveys assessed changes in attitude, concern, interest and subjective knowledge. Results showed an increase in subjective knowledge but a decrease in interest, possibly due to reduced novelty or increased perceived difficulty. These findings underscore the effectiveness of exhibits as outreach tools in informal settings and highlight the critical role of maintaining novelty and emphasizing the relevance of quantum technologies in future outreach efforts. Additionally, we emphasize the importance of assessing outreach effectiveness to ensure that objectives are successfully achieved.

Figures

Figures reproduced from arXiv: 2507.13010 by the authors.

Figure 1
Figure 1. Stacked bar graph showing the participants’ answers to the items related to science capital. The graph illustrates four items: (1) habit to talk about science, (2) participate in science activities in their spare time, (3) their level of interest in the scientific process and its results, and (4) their awareness of scientific developments. Change in attitude, concern, interest and subjective knowledge Overall, the i… view at source ↗
Figure 2
Figure 2. Density plots of the distribution of the composite scores of the outcome measures (a) attitude, (b) concern, (c) interest and (d) subjective knowledge. The vertical dashed lines represent the mean values before and after visiting the exhibit. A distribution of the scores on the pre- and post-survey for each measure is shown in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Pearson correlation coefficients matrix. Correlation coefficients that are crossed-out are insignificant (p > 0.05). Exploratory analyses We further explored correlations between the pre-measures and the measured changes, a complete overview can be found in [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 4 canonical work pages

  1. [227]

    https://doi.org/10.1177/2372732216655542 Hidi, S., & Renninger, K. A. (2006). The Four-Phase Model of Interest Development. Educational Psychologist, 41(2), 111–127. https://doi.org/10.1207/s15326985ep4102_4 Institute for Quantum Computing. (n.d.). QUANTUM: The Pop-Up Exhibition. https://uwaterloo.ca/institute-for-quantum-computing/outreach/quantum-pop-ex...

    work page doi:10.1177/2372732216655542 2006

  2. [246]

    https://doi.org/10.1007/s10676-017-9429-1 Volk, S. C. (2024). Assessing the Outputs, Outcomes, and Impacts of Science Communication: A Quantitative Content Analysis of 128 Science Communication Projects. Science Communication, 10755470241253858. https://doi.org/10.1177/10755470241253858 Volk, S. C., & Schäfer, M. S. (2024). Evaluations in science communic...

    work page arXiv 2024

  3. [277]

    M., & Deci, E

    https://doi.org/10.1177/1075547017697981 Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68 Sánchez-Mora, M. D. C. (2016). Towards a taxonomy for public communication of science activiti...

    work page arXiv 2000

  4. [573]

    A., & Gerber, A

    https://doi.org/10.1177/0963662512458624 Jensen, E. A., & Gerber, A. (2020). Evidence-Based Science Communication. Frontiers in Communication, 4. https://doi.org/10.3389/fcomm.2019.00078 Joubert, I., Geppert, M., Ess, S., Nestelbacher, R., Gadermaier, G., Duschl, A., Bathke, A., & Himly, M. (2019). Public perception and knowledge on nanotechnology: A stud...

    arXiv 2020

  5. [3506]

    https://doi.org/10.1111/ejn.14719 Gardner, G.E., & Troelstrup, A. (2015). Students‘ Attitudes Toward Gene Technology: Deconstructing a Construct. Journal of Science Education and Technology, 24, 519-531. https://doi.org/10.1007/s10956-014-9542-4 Glasman, L. R., & Albarracín, D. (2006). Forming Attitudes That Predict Future Behavior: A Meta- Analysis of th...

    work page doi:10.1111/ejn.14719 2015

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.