{"id":"53f8ad49-3607-47ed-897c-2f17ffa58959","arxiv_id":"2411.14301","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A three-workshop participatory series increased self-reported net zero knowledge among UKRI digital research infrastructure stakeholders, though it did not shift self-reported intentions to act.","lead":"This paper reports on a three-part online workshop series that asked UK research infrastructure staff to co-create carbon reduction actions. The authors argue that guided, expert-integrated workshops give ground-level stakeholders a voice in the net zero transition.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Knowledge-gain claim rests on non-paired, attrition-prone self-report data; the knowledge item asks about the exact task completed in workshop 3, so the observed increase may reflect task familiarity rather than sustainability learning.","rationale":"The reader's weakest assumption correctly targets the self-reported Likert data and the small, self-selected, attrition-plagued sample. My stress-test agrees with that direction but sharpens the concern in two ways that the reader did not fully spell out. First, the specific knowledge item is confounded with the intervention: rating one's knowledge of 'developing your own work-related net zero action plan' after having just developed such a plan is partly a measure of task completion, not of underlying sustainability knowledge. This is a construct-validity threat independent of sampling. Second, the paper's own text creates an internal inconsistency: Section 3.3 says the analysis is based on 8 full completers, while Table 2 reports N=10 for the after-workshop-3 survey; this needs reconciliation before the paired claim can be evaluated. Neither of these points makes the paper dishonest or worthless. The qualitative material—the action plans, the synthesized recommendations, and the 'day in the life' vision—is a legitimate existence proof that a guided workshop series can elicit community-generated suggestions for net zero DRI. That part of the central claim survives the quantitative critique. But the stronger wording about 'bottom-up agency' and 'advancing knowledge' is not yet supported by internally valid evidence. The reader's CONDITIONAL verdict already captures the need for paired re-analysis and open data; my analysis does not move the verdict, but it gives the authors a precise, actionable check: release the paired individual data for the 8 completers, test the pre-post gain non-parametrically, and validate the self-report against an external scoring of the action plans. If those checks fail, the paper should be revised to claim only that the workshop method generates structured recommendations, not that it demonstrably enhances participants' knowledge or agency.","tokens_in":13130,"tokens_out":4014,"duration_ms":40964,"concrete_test":"Request the anonymized individual-level survey data (or the Zenodo project report) for the 8 participants who completed all three workshops and surveys. Compute a paired Wilcoxon signed-rank test on the knowledge item from pre-workshop to after-workshop 3, and compare baseline knowledge of completers versus non-completers to assess attrition bias. In parallel, have two independent raters score the action plans for technical accuracy and actionability (e.g., correct use of carbon accounting, procurement, green scheduling, embodied-carbon concepts) and correlate those objective scores with the self-rated knowledge scores.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that the workshop series 'can be achieved' and is 'impactful on creating bottom-up agency' leans heavily on the quantitative finding that knowledge improved (Section 3.3, Table 2). That finding is internally weak. First, the knowledge item ('How would you rate your current knowledge of developing your own work-related net zero action plan?') was re-administered after participants had just spent Workshop 3 developing exactly such an action plan, so post-scores partly measure task rehearsal, not genuine carbon/energy literacy. Second, the reported means come from different respondent sets (N=24, 29, 16, 10); the text claims the analysis is 'based on the 8 participants that fully completed the series', but no paired pre-post statistics for those 8 are shown, and Table 2 reports N=10 for the final wave, leaving an unresolved discrepancy. If less-engaged participants dropped out, the mean increase is an artifact of selection. Third, the 'agency' conclusion is not supported by the other survey constructs: intentions and perceived control show no reported significant change, and no behavioral outcome is measured. The action plans are a required workshop product, so they cannot independently evidence empowerment. Thus the load-bearing quantitative pillar of the central claim is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports on a three-session online workshop series run with UK Research and Innovation (UKRI) Digital Research Infrastructure (DRI) stakeholders, aimed at co-creating net-zero action plans and recommendations for a bottom-up sustainable transition. The authors describe the workshop design, present qualitative themes from action plans and transcripts, and report survey results that they interpret as showing increased participant knowledge about developing work-related net-zero action plans. They claim that the workshop series gives stakeholders a purposeful voice, creates bottom-up agency for climate action, and can complement top-down policy. The paper also positions the workshop recommendations as inputs to the UKRI Net Zero DRI scoping project.","tokens_in":13311,"tokens_out":2600,"duration_ms":26646,"significance":"If the central claim is accepted, the workshop series offers a low-cost, transferable format for engaging ground-level stakeholders in sustainability transitions of research infrastructure, a topic of growing policy importance. The paper's strengths include a detailed and honest description of the workshop design, a transparent acknowledgment of several limitations in the Conclusion, and the fact that the qualitative recommendations were fed into an independent UKRI scoping report. The paper also provides a useful synthesis of stakeholder-generated actions. However, the quantitative evidence for the load-bearing claim that the workshops 'advance knowledge' and create 'agency' is weak: it rests on self-report Likert data from a small, self-selected, and attrition-prone sample, with no paired statistical analysis. The qualitative material is valuable, but the strength of the central claim currently exceeds what the evidence supports.","major_comments":[{"comment":"The claim that knowledge improved 'based on the 8 participants that fully completed the series' is not supported by the data as presented. Table 2 reports N=24, N=29, N=16, and N=10 for the successive waves, with no paired pre-post statistics for the 8 completers. Please report the paired data for that subsample, or clarify the actual N for each comparison, and provide a paired test (e.g., Wilcoxon signed-rank) with effect size. Without this, the mean increase from 5.92 to 8.70 may simply reflect differential attrition rather than a genuine learning effect.","section":"Section 3.3 and Table 2"},{"comment":"The knowledge measure ('How would you rate your current knowledge of developing your own work-related net zero action plan?') was re-administered immediately after Workshop 3, in which participants had just been guided to develop such an action plan. The observed increase may therefore reflect task familiarity or rehearsal of the specific workshop exercise rather than a durable gain in sustainability knowledge or carbon/energy literacy. The paper should explicitly acknowledge this content-overlap confound, or reanalyze using a measure that is not so directly tied to the workshop task.","section":"Section 3.3, Table 2, knowledge item"},{"comment":"The conclusion that the workshop design is 'impactful on creating bottom-up agency' is not supported by the survey constructs reported in Table 2: intentions to act and perceived control show no reported significant change across the workshop series, and no behavioral outcome is measured. The action plans are a required workshop deliverable, so their existence cannot independently evidence empowerment. I recommend either softening the agency claim to something like 'participants self-reported increased knowledge and produced co-designed action plans' or providing additional evidence, such as follow-up behavioral data or a qualitative analysis of participants' own accounts of agency.","section":"Abstract and Section 4.3"},{"comment":"Attrition is a serious threat to the knowledge-gain claim: the sample drops from N=24 at baseline to N=10 after Workshop 3, and the participants who remained were likely more engaged and motivated. The paper notes the small sample but does not examine whether completers differ from non-completers on baseline characteristics (e.g., initial knowledge, job role, council affiliation). Please add such a comparison, or discuss explicitly how attrition could explain the observed means.","section":"Section 2.1 and Conclusion"}],"minor_comments":[{"comment":"There is a typo in 'the development a workshop series'; it should read 'the development of a workshop series'.","section":"Introduction, first paragraph after the abstract"},{"comment":"Figure 1, the 5-W action plan framework, is referenced in the text but does not appear in the manuscript body; please check that the figure is included and that its contents are clearly explained.","section":"Section 2.2.3, Figure 1"},{"comment":"The survey constructs are described only briefly in the text. Please provide the exact item wording, response scale endpoints, and timing (pre-workshop, after each workshop) in an appendix, so that readers can assess the measures.","section":"Section 3.3 and Appendix, Table 2"},{"comment":"The statement that supplementary material 'can be accessed on request' is impractical for readers; a stable repository link or a detailed description of the toolkit and action plan templates should be provided.","section":"Supplementary Material"},{"comment":"The reference 'Ross and Lorna, 2022' is incomplete (no title, journal, or publisher). Please provide full bibliographic details.","section":"Section 6, References"},{"comment":"The term 'downshift' is used frequently but not defined until the discussion of fossil-fuel phaseout in Section 2.2.2; please define it at first use in the Introduction or Methods.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an interesting and honest report of a participatory workshop process, and the qualitative recommendations are likely to be useful to practitioners. My main concern is that the Abstract and Conclusion frame the workshop as successful in creating 'agency' and 'knowledge gain' on the basis of self-reported Likert data that are not statistically analyzed and that are subject to attrition and content-overlap confounds. I believe the paper can be made acceptable if the authors substantially temper the causal and agency claims, re-analyze the paired data, and present the quantitative results as suggestive rather than definitive. The fit between the journal's scope and this community-engagement study should be left to the editor's judgment; I see no concerns about novelty or attribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my take on arXiv:2411.14301. The paper is a case study of a three-workshop series that brought UKRI digital research infrastructure stakeholders together to co-create net zero action plans. What is actually new: a concrete instance of applying known participatory techniques and the 5-W framework to this specific community, with a set of synthesized recommendations that fed into the UKRI Net Zero DRI scoping report. That is a modest contribution, but a real one. The qualitative material is the strongest part: the action-plan synthesis, the verbatim quotes, and the 'day in the life' vision are grounded in transcripts and give a useful picture of how ground-level staff think about barriers and levers.\n\nThe soft spot is the quantitative claim. The paper's central assertion that the workshop design improved knowledge rests on self-report Likert scores from a small, self-selected sample with no control group, no paired statistical test, and N that moves from 24 to 29 to 16 to 10. The text says the knowledge analysis is 'based on the 8 participants that fully completed the series,' but Table 2 reports final-wave N=10; that discrepancy needs resolving. Also, the knowledge item asks about developing a work-related net zero action plan, which is exactly what participants did in Workshop 3, so the post-score may partly measure task familiarity rather than broader sustainability learning. The authors do acknowledge several limitations themselves, but they still lean on the knowledge gain as evidence for the method's impact.\n\nThe intention and perceived-control constructs show no meaningful change, and no behavioral outcome is measured. That is fine if the claim is only that the workshops generated recommendations; it becomes a problem when the abstract says the design is 'impactful on creating bottom-up agency.' The evidence supports the recommendations; it does not yet support the agency claim.\n\nOn balance: this deserves a serious referee. The case study is useful for sustainability practitioners working with research infrastructures, and the qualitative synthesis is honest and well-connected to the wider scoping project. The authors should be asked to either provide paired pre-post data for the completers with proper tests, or soften the knowledge and agency claims to match what the data can bear. I would not cite it in my own work, but I would bring it to a reading group focused on participatory methods in sustainability transitions.\n\nRecommendation: send to peer review, with the quantitative analysis flagged as a major revision.","headline":"An honest, clearly written case study of a participatory workshop series for UKRI DRI net zero; the qualitative recommendations are useful, but the quantitative knowledge-gain claim is not yet supported by the data.","tokens_in":13898,"tokens_out":1541,"would_cite":false,"duration_ms":14123,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that a guided, expert-integrated, interactive workshop series can give ground-level stakeholders of a national digital research infrastructure a purposeful voice in shaping their own net zero future.","keywords":["digital research infrastructure","net zero","stakeholder engagement","participatory workshops","bottom-up climate action","carbon downshift","action plans","knowledge co-production"],"falsifier":"Compare a group that completes the three-workshop series against a control group that receives only the workshop reading materials, scoring both on a rubric-based ability to draft a concrete net zero action plan before and after; or track whether the action plans written in Workshop 3 actually lead to changes in energy use, procurement, or community practice within a year. If knowledge scores rise without any improvement in plan quality, or if none of the plans are ever implemented, the central claim that the workshops create bottom-up agency would be contradicted.","tokens_in":12895,"feed_emoji":"🌱","tokens_out":8481,"duration_ms":71400,"temperature":0.7,"pith_summary":"The paper addresses a specific gap in the UK's move toward a net zero digital research infrastructure (DRI): the technical knowledge for reducing carbon already exists inside the community, but no mechanism exists for sharing it or turning it into behavior. The authors designed and ran three sequential online workshops that combined expert talks with facilitated breakout discussions, moving from energy consumption and efficiency, to embedded fossil fuels and emissions scopes, to individualized net zero action plans structured around five 'W' questions. They report that participants' self-rated knowledge of how to develop a work-related net zero action plan rose from a mean of 5.92 to 8.70 on a 10-point scale across the series, while their intentions to act were already high and did not change. The paper's central claim is that this guided, expert-integrated, interactive and problem-focused workshop format creates bottom-up agency for climate action and produces community-led recommendations that complement top-down efficiency and renewable-energy policy. If the claim is right, the format is a transferable way to kick-start sustainable transitions from within communities rather than waiting for institutional mandates.","feed_headline":"Guided workshops give research infrastructure staff a net-zero voice","feed_subtitle":"A three-session expert-integrated series raised self-reported knowledge and produced community-led climate action plans.","key_machinery":"The machinery that carries the argument is the workshop series design itself, presented as a repeatable method rather than a one-off event. Workshop 1 defines energy consumption and efficiency in the DRI; Workshop 2 introduces embedded fossil fuels, scope 1-3 emissions, and a cap-and-share thought experiment; Workshop 3 translates the discussion into individualized action plans built on the five 'W' questions (what, who, when, where, why). The design's load-bearing feature is the pairing of expert input with facilitated breakout rooms, which connects the high-level problem of net zero to each participant's daily work environment and gives the community a structured format for producing recommendations. The pre/post surveys are the instrument that converts this experience into the reported knowledge gains.","core_discovery":"On the paper's own terms, the central discovery is that a three-workshop series can give a dispersed stakeholder community the means to define its own net zero problems and to propose concrete solutions. The sequence starts with the high-level problem of unsustainable energy and fossil-fuel consumption, connects it to the ground-level circumstances of DRI users, developers, and infrastructure providers, and ends with each participant writing a personal action plan under five 'W' prompts. The evidence offered is the contrast between pre- and post-workshop surveys, in which self-assessed knowledge of building a net zero action plan rose steadily from 5.92 to 8.70 on a 10-point scale, alongside a set of synthesized recommendations covering resource monitoring, green scheduling, data reuse, procurement, and community behavior. The authors are careful to state that the workshops did not alter intentions to act, and that only eight participants completed the full series, so the demonstrated effect is on knowledge and voice rather than on measured behavior.","pith_inferences":["A natural extension the paper leaves untested: the same format could be applied to other communities with distributed net zero expertise—university campuses, hospital trusts, or local authorities—where the missing mechanism is convening rather than knowledge.","Because the knowledge measure is self-reported, the paper's own logic implies that the decisive follow-up is to score actual action-plan quality before and after the series, or to track whether the plans written in Workshop 3 are implemented within a year.","The senior, self-selected sample suggests a structural risk that such workshops amplify the voices of those already committed to net zero; testing the format with junior staff and with disciplines outside the computationally intensive ones would show whether it creates new agency or reinforces existing enthusiasm."],"forward_implications":["The same three-workshop sequence can be reused by other research-infrastructure communities to generate net zero recommendations without waiting for top-down policy.","The specific recommendations produced—carbon-aware scheduling, power and carbon monitoring, data reuse, green procurement, and cap-and-share workload policies—are candidates for direct incorporation into institutional net zero strategies.","The format should be most effective when each workshop narrows from a system-level problem to individual action, since the reported gains come from connecting high-level targets to participants' own roles.","Organizers of similar series should expect knowledge scores to rise but intentions to act to stay flat, implying that institutional support is needed to convert the resulting knowledge and action plans into real changes."],"supporting_citations":[{"why":"Defines the national net zero target and the scoping project that this workshop series is designed to feed into.","marker":"Juckes et al., 2023"},{"why":"Supplies the 80% fossil-fuel downshift requirement that sets the climate constraint for the workshops.","marker":"Calverley and Anderson, 2022"},{"why":"Provides the caution that knowledge does not necessarily lead to behavior change, used to interpret the flat intentions scores.","marker":"De Meyer et al., 2021"},{"why":"Supplies the bottom-up sustainable development process steps that structure the workshop series.","marker":"Lang et al., 2012"},{"why":"Documents the original project report where the workshop recommendations were recorded and passed back to stakeholders.","marker":"Manika, Danae et al., 2022"},{"why":"Frames the problem-focused workshop design as co-evolution of problem and solution.","marker":"Dorst and Cross, 2001"},{"why":"Supplies the design definition used to present the workshops as the first step in re-designing the DRI toward sustainability.","marker":"Simon, 1996"}],"fun_headline_variants":["Guided workshops give DRI staff a net-zero voice","Workshops raise net-zero knowledge for infrastructure staff","Ground-level workshops spark community-led climate plans","DRI stakeholders gain net-zero skills via guided series","Small workshop series boosts net-zero planning know-how"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the self-reported Likert scores, drawn from a small self-selected group that shrank from 24 to 8 respondents, measure a genuine gain in knowledge and agency caused by the workshop design rather than selective retention, social desirability, or simply learning what to say.","fun_headline_variants_meta":{"raw":{"variants":["Guided workshops give DRI staff a net-zero voice","Workshops raise net-zero knowledge for infrastructure staff","Ground-level workshops spark community-led climate plans","DRI stakeholders gain net-zero skills via guided series","Small workshop series boosts net-zero planning know-how"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000311,"raw_usage":{"total_tokens":1814,"prompt_tokens":1028,"completion_tokens":786,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":713}},"tokens_in":644,"tokens_out":786,"duration_ms":8233,"temperature":1.0,"reasoning_tokens":713,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:18:33.173871+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare a group that completes the three-workshop series against a control group that receives only the workshop reading materials, scoring both on a rubric-based ability to draft a concrete net zero action plan before and after; or track whether the action plans written in Workshop 3 actually lead to changes in energy use, procurement, or community practice within a year. If knowledge scores rise without any improvement in plan quality, or if none of the plans are ever implemented, the central claim that the workshops create bottom-up agency would be contradicted.","supporting_citations":[],"review_version":1}