{"id":"73b75173-988d-45c7-9d73-9f97f3bd5a0b","arxiv_id":"2507.13010","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A quantum exhibit at a music festival increased visitors' subjective knowledge (d=0.80) but decreased interest (d=-0.44), with small increases in attitude and concern.","lead":"A pre-post survey of 812 visitors to a quantum technology exhibit at the 2024 Lowlands music festival found that visitors left feeling much more knowledgeable but less interested in quantum topics. The study is one of the first to measure how a festival-based outreach exhibit changes public engagement with an emerging technology.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The subjective-knowledge gain may be largely an experimenter-demand artifact: post-survey items are explicitly framed as 'After visiting the exhibit...', and the lack of a no-exhibit control leaves this interpretation unfalsified.","rationale":"The reader correctly identifies the missing control group as the central threat to causal interpretation, and I agree that the paper cannot support strong causal language. My stress-test pass adds a more specific, partly non-overlapping problem: the post-survey items are not neutral re-administrations of the pre-items; they are explicitly prefaced with 'After visiting the exhibit...'. For a self-report confidence scale, this wording is nearly a demand manipulation, and for the interest item 'want to know more,' a post-visit decrease can reflect satiation rather than a genuine loss of interest. These wording effects and the absence of a no-exhibit comparison mean the effect sizes d=0.80 and d=-0.44 could be reproduced even if the exhibit's true causal effect were zero. I do not think this overturns the paper's value: the data are openly shared, the procedure is preregistered, and the reporting is largely honest. The reader's conditional recommendation already requires softened framing and robustness checks, so no verdict change is needed. A randomized wording and control experiment would settle the concern; until then, causal claims should be replaced with descriptions of observed pre-post differences.","tokens_in":15058,"tokens_out":8951,"duration_ms":124293,"concrete_test":"Conduct a randomized control experiment at a comparable science-festival setting: recruit attendees who pass the exhibit area but do not enter the exhibit, and give them the same pre/post questionnaires separated by the same 20-minute interval, with half of the post-surveys using the original 'After visiting the exhibit...' wording and half using neutral wording. If the no-exhibit neutral group shows a subjective-knowledge increase larger than d=0.2 or an interest decrease larger than d=0.1, the headline effect sizes cannot be attributed to the exhibit; if the no-exhibit group shows no pre-post change and wording has no effect, the concern is largely resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline effect sizes rest on a single-group pre-post design in which every participant knows they are taking part in a study about the exhibit, completes the post-survey immediately at the exhibit exit, and sees post-items whose wording explicitly credits the exhibit: for example, 'After visiting the exhibit, how confident are you in your understanding of superposition?' and 'After visiting the exhibit, I want to know more about how quantum technologies work.' Because the subjective-knowledge items measure confidence rather than verifiable knowledge, the large d=0.80 gain is exactly what a polite, self-selected, science-interested festival visitor would be expected to report regardless of actual learning. The interest decline is also confounded: 'want to know more' is a satiation-sensitive item, so reporting more subjective knowledge can mechanically lower the post score without any reduction in genuine interest. The Limitations section acknowledges self-selection, intoxication, and lack of a control, but it does not quantify or rule out these wording and demand effects. Without a no-exhibit comparison or a neutral-wording replication, the causal claim that the exhibit produced these changes is not established; the observed pattern is fully compatible with measurement artifacts.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript reports a single-group pre-post survey of 812 visitors to a quantum science and technology pop-up exhibit at the 2024 Lowlands music festival. The authors measured attitude, concern, interest, and subjective knowledge before and after the visit, along with science capital. They report a large increase in subjective knowledge (d = 0.80), a moderate decrease in interest (d = 0.44), a small increase in concern (d = 0.18), and a negligible increase in attitude (d = 0.08), with weak negative correlations between science capital and several changes. The paper interprets these results as evidence that the exhibit was effective as an outreach tool and derives practical recommendations for future quantum outreach.","tokens_in":15238,"tokens_out":3701,"duration_ms":45495,"significance":"If the observed pre-post differences were causally attributable to the exhibit, this would be a valuable contribution to the sparse evaluation literature on quantum science outreach, especially because the study is preregistered, uses a large sample in a real informal setting, and openly shares materials, data, and analysis scripts. The unexpected negative interest effect is an interesting and testable finding. However, the single-group design, the demand-laden post-survey wording, and the post hoc scale modification mean that the current evidence supports only descriptive pre-post changes, not causal claims about outreach effectiveness.","major_comments":[{"comment":"The central claim that the exhibit produced the observed changes is not supported by the design. There is no control or comparison condition, and the post-survey items explicitly credit the exhibit (e.g., 'After visiting the exhibit, I want to know more about how quantum technologies work' and 'After visiting the exhibit, how confident are you in your understanding of superposition?'). This wording invites demand effects, and retesting, regression to the mean, festival mood, and self-selection of a highly educated, science-interested sample are all plausible alternative explanations. The Limitations section acknowledges some of these threats but does not quantify them or temper the causal language in the Abstract, Results, and Discussion. The manuscript should be reframed as a descriptive pre-post observational study, with all causal inferences either removed or explicitly labeled as tentative, and the abstract's statement that the findings 'underscore the effectiveness of exhibits' should be revised accordingly.","section":"Procedure; Measures; Abstract"},{"comment":"The concern scale was modified after inspecting Cronbach's alpha: the item 'I have no concerns about quantum technologies because the benefits are likely greater than the risks' was omitted because alpha improved from .44/.60 to .69/.77. This post hoc deletion is not described as preregistered, and it can inflate the apparent concern effect. The authors should report the full-scale results as a sensitivity analysis, state whether the deletion was specified in the preregistration, and note that the d = 0.18 concern effect depends on this item-selection decision.","section":"Results, concern scale paragraph"},{"comment":"Four paired t-tests are conducted without any correction for multiple comparisons. The attitude effect (p = 0.030, d = 0.08) would not survive a simple Bonferroni correction (threshold 0.0125), and the concern effect is based on a scale that was modified post hoc. The authors should report adjusted p-values, confidence intervals, or use a false discovery rate approach, and interpret the small effects accordingly. This is particularly important because the abstract and discussion emphasize significant results.","section":"Statistical Analysis; Table 1"},{"comment":"The interest decrease may be partly mechanical. The interest items include 'I want to know more about how quantum technologies work,' which is satiation-sensitive: after an exhibit that raises subjective knowledge, respondents may report lower desire to learn more simply because they feel they have already learned something. The subjective knowledge items measure confidence, not verifiable knowledge, so the large d = 0.80 gain could reflect perceived competence rather than actual learning. The Discussion should address this measurement confound explicitly; a neutral post-wording replication or an independent novelty/difficulty measure would be needed to support the interpretation that genuine interest declined.","section":"Measures; Discussion, 'Moderate decrease in interest'"}],"minor_comments":[{"comment":"The text states that 76.9% (N = 758) of participants were highly educated, but 758/812 is approximately 93.3%. One of these numbers is incorrect and should be fixed.","section":"Participants"},{"comment":"The post-visit subjective knowledge mean (M = 2.8) is still below the scale midpoint of 3, so describing the change as a 'substantial increase' may overstate the final level; consider saying 'a large increase from a low baseline.'","section":"Results, subjective knowledge"},{"comment":"The correlations between baseline measures and change scores (e.g., Figure 3) are likely influenced by regression to the mean and should be interpreted with caution; the text should not present them as evidence of causal relationships.","section":"Discussion, exploratory correlations"},{"comment":"There is a typo: 'We close of this section' should read 'We close off this section.'","section":"Materials and Methods, last paragraph"},{"comment":"The sentence 'Additionally, thank the staff of New Scientist...' is missing a subject; it should read 'Additionally, we thank the staff...'.","section":"Acknowledgements"},{"comment":"Reference formatting is inconsistent: 'de Wolf, R.' is alphabetized under 'd' while some other entries use different capitalization conventions; this should be harmonized with the journal style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a modest but potentially useful descriptive evaluation of a real-world outreach exhibit. The main barrier is not the absence of a control group per se—many field studies lack one—but the unqualified causal language in the abstract and discussion. If the authors reframe the paper as an observational pre-post study and add the requested sensitivity analyses, the contribution could be publishable. The open materials and preregistration are strengths that should be preserved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. This is the first published pre-post impact assessment of a quantum-technology outreach exhibit, run at a music festival with 812 visitors, with preregistration, open data, and a write-up that doesn't hide the awkward interest decrease. The subjective knowledge gain (d=0.80) and interest drop (d=0.44) are the two findings people will remember; both are plausible and internally consistent.\n\nWhat's new: no prior published assessment of a quantum tech outreach exhibit that I can see, and the festival setting is rare. The paper's own discussion of the interest decline—novelty loss and perceived difficulty—is thoughtful and tied to relevant literature. That alone makes it a useful data point for science communication practice.\n\nSoft spots, in order of seriousness:\n- The causal claim that the exhibit caused the changes is not established. Single-group pre-post, self-selected, no control. The limitations section names self-selection, intoxication, and mood, but doesn't directly address the demand-characteristic problem: the post-survey items explicitly credit the exhibit (\"After visiting the exhibit...\"), which primes agreement on knowledge items and can mechanically lower \"want to know more\" after an experience feels complete. I wouldn't go as far as calling the knowledge gain \"largely\" an artifact—the effect is large and consistent with other outreach studies—but the wording issue deserves a real robustness discussion, not just a passing mention.\n- The concern scale: one item was deleted post hoc to get alpha from .44/.60 to .69/.77. That should be disclosed in the main text with a no-deletion sensitivity analysis.\n- Multiple t-tests without correction; minor, since the effects are large and the pattern is coherent.\n- The abstract says the findings \"underscore the effectiveness of exhibits\" while the headline result is mixed (knowledge up, interest down). That framing overreaches. The Discussion is more balanced.\n\nCitation pattern is fine. The comparison studies are used as context, including one from the same group, and the preregistration/data links are there. This paper deserves a serious referee; conditional acceptance with requests to reframe causal language and add sensitivity analyses would be right. I'd cite it if the revised version lands.","headline":"A solid, honest pre-post study of a quantum exhibit at a festival; the knowledge gain and interest drop are worth knowing about, but the causal framing needs a control-less caveat.","tokens_in":639,"tokens_out":735,"would_cite":true,"duration_ms":29092,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 20-minute quantum exhibit raised visitors' felt understanding while lowering their curiosity.","keywords":["quantum science and technology","science outreach exhibition","public engagement","music festival","quantitative survey","subjective knowledge","science capital","pre-post study"],"falsifier":"A control condition in which festival visitors fill out the same two surveys without seeing the exhibit: if the same pattern of interest decline and subjective-knowledge gain appears, the claim that the exhibit caused these changes is falsified.","tokens_in":14835,"feed_emoji":"⚛️","tokens_out":4913,"duration_ms":53664,"temperature":0.7,"pith_summary":"This paper tries to establish that a short, informal outreach exhibit can change how a general audience relates to quantum science and technology, and that the changes are not all in the direction outreach organisers expect. Using paper questionnaires from 812 visitors to a pop-up exhibit at a Dutch music festival, the authors compared self-reported attitude, concern, interest, and subjective knowledge before and after the roughly 20-minute visit. They report a large rise in perceived knowledge, a moderate fall in interest, and very small rises in attitude and concern. If the results hold, they matter for how science communication designs exhibits for emerging technologies: a brief exposure can make people feel more informed while making them less eager to learn more.","feed_headline":"Quantum exhibit boosts felt knowledge, cools interest","feed_subtitle":"Tracking 812 festival-goers before and after the show, the survey found perceived knowledge jumped and interest slid—a caution for…","key_machinery":"The load-bearing instrument is the paired pre–post survey: the same four Likert-scale batteries (attitude, concern, interest, subjective knowledge) were administered immediately before and after the exhibit, and composite means were compared with paired two-sided t-tests using Welch's approximation. The exhibit itself—13 bilingual panels with two videos and four interactive elements covering superposition, wave–particle duality, quantum computing, cryptography, and sensing—is the treatment. 'Science capital,' measured by four self-report items about awareness, spare-time science activity, discussion habits, and interest in the scientific process, is the moderator variable the authors use to test whether prior engagement with science changes the size of the effects.","core_discovery":"The central claim is that a single visit to an informal science exhibit produces measurable, and mixed, effects on engagement with quantum technologies. In paired comparisons of 812 festival visitors' five-point scale responses, mean subjective knowledge rose from 2.1 to 2.8 ($d = 0.80$), mean interest fell from 3.9 to 3.5 ($d = -0.44$), while attitude (3.4 to 3.5, $d = 0.08$) and concern (2.5 to 2.7, $d = 0.18$) increased only slightly. The authors interpret the knowledge gain as evidence that exhibits can quickly make audiences feel more capable of engaging with quantum topics, and the interest drop as a probable consequence of reduced novelty or increased perceived difficulty once the material is actually encountered. They further report that higher science capital was weakly associated with smaller positive shifts in attitude, interest, and subjective knowledge.","pith_inferences":["A plausible reading the paper leaves implicit is that the subjective-knowledge gain may reflect confidence rather than actual comprehension; an objective knowledge test would separate these possibilities.","The interest decline could partly be a measurement artifact: asking 'how much do you want to know more' immediately after showing the material sets a higher informational bar, and a control question wording could test this.","For exhibit designers, the findings suggest ending with open questions, unanswered puzzles, or concrete personal applications might preserve curiosity after the felt-knowledge boost.","The lack of a no-exhibit control group means the same before–after pattern could appear among festival-goers who simply take the survey twice, so a control condition is the natural next test."],"forward_implications":["Short, self-paced exhibits can substantially raise visitors' perceived understanding of an unfamiliar technology in a single visit.","Outreach about emerging technologies can lower self-reported interest, and designers should treat interest loss as a possible outcome rather than assuming any exposure is motivating.","The pattern of results suggests that reducing the breadth of content and linking quantum topics to everyday life may counteract the interest decline.","Measuring outreach impact with pre–post surveys can reveal opposing effects on different engagement dimensions, making such evaluation worthwhile for science communication practice.","Higher prior science engagement is at most weakly linked to smaller gains from the exhibit, implying the exhibit's effects do not strongly favour already science-oriented visitors."],"supporting_citations":[{"why":"Supplies the four outcome variables (attitude, concern, interest, subjective knowledge) and the observation that few outreach projects measure their impact.","marker":"Volk, 2024"},{"why":"Defines science capital, the moderator variable used to test whether prior engagement with science alters the exhibit's effects.","marker":"Archer et al., 2015"},{"why":"Provides the interest-development framework and the definition of interest used to interpret the drop in self-reported interest.","marker":"Hidi & Renniger, 2006"},{"why":"A science-festival panel study on human gene editing that found increases in both attitude and concern, used as a comparison for the concern result.","marker":"Rose et al., 2017"},{"why":"A contrasting finding that a short explanatory text increased interest in quantum technology, which frames the explanation-depth hypothesis for the observed interest decline.","marker":"Meinsma et al., 2024"},{"why":"Supplies the perceived-difficulty–interest link used to explain why visitors' interest dropped after encountering the exhibit.","marker":"Fulmer & Tulis, 2013"},{"why":"An earlier lab-based outreach study that found increased interest after a hands-on visit, serving as the contrasting benchmark for the festival exhibit's interest decrease.","marker":"Woithe et al., 2022"}],"fun_headline_variants":["Quantum exhibit boosts know-how, cools interest","Festival quantum show lifts know-how, damps zeal","Mixed quantum impact: knowledge up, interest down","Exhibit raises quantum know-how, lowers interest"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume that the measured before–after shifts come from the exhibit itself, not from taking the same survey twice, from festival mood or intoxication, or from the fact that only already science-minded visitors chose to take part.","fun_headline_variants_meta":{"raw":{"variants":["Quantum exhibit boosts know-how, cools interest","Festival quantum show lifts know-how, damps zeal","Mixed quantum impact: knowledge up, interest down","Exhibit raises quantum know-how, lowers interest"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000121,"raw_usage":{"total_tokens":1053,"prompt_tokens":868,"completion_tokens":185,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":125}},"tokens_in":484,"tokens_out":185,"duration_ms":2976,"temperature":1.0,"reasoning_tokens":125,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:31:45.827250+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A control condition in which festival visitors fill out the same two surveys without seeing the exhibit: if the same pattern of interest decline and subjective-knowledge gain appears, the claim that the exhibit caused these changes is falsified.","supporting_citations":[],"review_version":1}