Fostering cultural change in research through innovative knowledge sharing, evaluation, and community engagement strategies
Pith reviewed 2026-05-22 12:40 UTC · model grok-4.3
The pith
An open knowledge ecosystem links production, validation, assessment, and reuse to realign research incentives away from paper counts and metrics.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors advance an open knowledge system that unites knowledge production, validation, assessment, and reuse into one ecosystem view, then translate this system into concrete recommendations for researchers, institutions, funders, and publishers so that reusable contributions and their validation become the visible basis for cultural change instead of papers and bibliometrics.
What carries the argument
The integrative conceptual framework, called an open knowledge system, that connects the four stages of knowledge handling to expose levers for sharing, validation, reuse, and reward.
If this is right
- Researchers receive guidance on sharing reusable contributions at appropriate stages and supporting their validation.
- Institutions and evaluators can move assessment criteria beyond bibliometrics toward validated reuse.
- Funders gain criteria for supporting practices that enable cumulative, high-quality work.
- Publishers can redesign processes to facilitate validation and reuse beyond traditional articles.
- All parties obtain a shared lens for diagnosing incentive misalignments and tracking reform progress.
Where Pith is reading between the lines
- Fields already rich in data sharing may adopt the reuse focus more quickly than those centered on narrative publications.
- Pilot implementations could be evaluated by measuring changes in the proportion of contributions accompanied by reusable artifacts and validation records.
- The same ecosystem logic might later be adapted to non-academic knowledge production such as industry R&D or public policy analysis.
- Wider uptake could indirectly reduce effort spent on non-reproducible work once validation steps become routine.
Load-bearing premise
That an ecosystem view linking production, validation, assessment, and reuse will lead stakeholders to adopt reforms that produce measurable cultural change in research practices.
What would settle it
A controlled comparison across institutions that adopt the framework versus those that do not, showing no measurable rise in rates of data and method sharing, validation events, or reproducibility indicators after three to five years.
Figures
read the original abstract
Scientific research needs a system that better values rigorous, reusable contributions. Although open knowledge and FAIR (findable, accessible, interoperable, and reusable) principles, along with coalitions and infrastructures, are accelerating reform, evaluation still often defaults to standardized metrics such as the h-index and journal impact factor. This misalignment still incentivizes quantity over quality, undermining integrity and reproducibility, and making it harder for communities to learn from and build on existing work. In this perspective, we bring together a global community of researchers, funding institutions, industrial partners, and publishers from 14 different countries across the 5 continents to advance ongoing debates on open science and research evaluation. Our contribution to the research practice is to offer an integrative conceptual framework, an open knowledge system, that links knowledge production, validation, assessment, and reuse into a single ecosystem view, and to translate into practical recommendations across key stakeholder roles (researchers, institutions/evaluators, funders, and publishers). By shifting attention from papers and bibliometrics toward reusable knowledge contributions and their validation, the framework highlights concrete levers for cultural change (what to share, when/how to validate, how to support reuse, and what to reward) and offers a practical lens that stakeholders can use to diagnose misaligned incentives and to design reforms that make high-quality, cumulative contributions visible and valued.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript is a perspective paper proposing an integrative conceptual framework for an open knowledge system that links knowledge production, validation, assessment, and reuse into a single ecosystem view. Drawing on a global collaboration across 14 countries, it translates this framework into practical recommendations for researchers, institutions/evaluators, funders, and publishers. The central claim is that shifting attention from papers and bibliometrics toward reusable knowledge contributions and their validation provides concrete levers (what to share, when/how to validate, how to support reuse, and what to reward) that stakeholders can use to diagnose misaligned incentives and design reforms fostering cultural change toward integrity, reproducibility, and cumulative science.
Significance. If the framework gains traction, it could offer a coherent lens for aligning incentives with open science principles such as FAIR, helping communities move beyond h-index and journal impact factor defaults. The paper's collaborative authorship and explicit mapping of stakeholder roles are strengths that synthesize ongoing debates into actionable points. Its value lies in highlighting specific levers rather than abstract calls for reform, though the absence of tested outcomes or adoption pathways limits immediate applicability.
major comments (2)
- [§3 (Integrative Conceptual Framework)] §3 (Integrative Conceptual Framework): The ecosystem view is presented as linking production, validation, assessment, and reuse, yet the manuscript provides no operational details on feedback mechanisms or indicators that would allow stakeholders to measure 'cultural change' or 'measurable outcomes.' This is load-bearing because the central claim rests on the framework enabling diagnosis and reform.
- [§5 (Stakeholder Recommendations)] §5 (Stakeholder Recommendations): The recommendations for funders and publishers (e.g., rewarding reusable contributions) do not analyze adoption barriers, conflicting institutional rules, or stakeholder interests, leaving the causal step from 'offering a practical lens' to 'reforms will be adopted and produce change' as an untested premise. This directly affects the paper's assertion of fostering cultural change.
minor comments (2)
- [Abstract and Introduction] The abstract and introduction could more explicitly distinguish the proposed framework from prior efforts such as DORA or CoARA to clarify novelty.
- [Figure 1 (if present)] If a diagram illustrating the open knowledge ecosystem exists, ensure it clearly depicts the four linked stages and any proposed feedback loops for readability.
Simulated Author's Rebuttal
We thank the referee for their detailed and insightful comments, which have helped us refine our perspective on the open knowledge ecosystem. We address each major comment below, clarifying the scope of our conceptual framework and proposing targeted revisions to strengthen the manuscript.
read point-by-point responses
-
Referee: §3 (Integrative Conceptual Framework): The ecosystem view is presented as linking production, validation, assessment, and reuse, yet the manuscript provides no operational details on feedback mechanisms or indicators that would allow stakeholders to measure 'cultural change' or 'measurable outcomes.' This is load-bearing because the central claim rests on the framework enabling diagnosis and reform.
Authors: We agree that operational details are limited in the current version, as the manuscript is a perspective paper focused on presenting an integrative conceptual framework rather than a fully specified operational model with quantitative indicators. The central claim is that the framework provides a lens for diagnosis, not that it includes ready-to-use metrics. To address this, we will revise §3 to include a new subsection outlining illustrative feedback mechanisms and potential indicators, such as tracking reuse rates in open repositories, community validation through preregistration or registered reports, and qualitative assessments of cumulative knowledge building. These examples will demonstrate how stakeholders could adapt the framework for measurement without claiming universal applicability or empirical testing of outcomes. revision: yes
-
Referee: §5 (Stakeholder Recommendations): The recommendations for funders and publishers (e.g., rewarding reusable contributions) do not analyze adoption barriers, conflicting institutional rules, or stakeholder interests, leaving the causal step from 'offering a practical lens' to 'reforms will be adopted and produce change' as an untested premise. This directly affects the paper's assertion of fostering cultural change.
Authors: We concur that a detailed analysis of adoption barriers and conflicting interests is absent, which is a valid observation for a perspective piece that synthesizes stakeholder roles at a high level. Our intent is not to assert that the recommendations will automatically lead to cultural change but to provide actionable starting points that stakeholders can use to navigate their specific contexts. We will make a partial revision to §5 by adding a paragraph discussing general categories of barriers (e.g., entrenched evaluation cultures, resource constraints) and emphasizing that successful implementation requires local adaptation and further empirical study. This maintains the paper's focus while acknowledging the premise is conceptual rather than tested. revision: partial
Circularity Check
No circularity: conceptual framework draws on external principles without self-referential reductions
full rationale
The manuscript is a perspective piece offering an integrative conceptual framework that links knowledge production, validation, assessment, and reuse into an ecosystem view, along with stakeholder recommendations. It references external principles such as FAIR without defining any terms circularly in terms of each other, contains no equations or derivations, makes no fitted-parameter predictions, and invokes no self-citations as load-bearing uniqueness theorems. All claims remain forward-looking statements about levers for cultural change rather than reductions of outputs to inputs by construction, rendering the argument self-contained against external benchmarks of open-science practices.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Current standardized metrics such as h-index and journal impact factor incentivize quantity over quality and undermine research integrity and reproducibility
- domain assumption Open knowledge and FAIR principles are accelerating reform but require better integration with evaluation systems
invented entities (1)
-
integrative conceptual framework for an open knowledge system
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Fraser, N. et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLoS Biol 19, e3000959 (2021)
work page 2021
-
[2]
https://rea.ec.europa.eu/open-science_en
Open science - European Commission. https://rea.ec.europa.eu/open-science_en
-
[3]
More than 10,000 research papers were retracted in 2023 - a new record
Van Noorden, R. More than 10,000 research papers were retracted in 2023 - a new record. Nature 624, 479–481 (2023)
work page 2023
-
[4]
https://publicationethics.org/topic- discussions/bias-peer-review
Bias in peer review | COPE: Committee on Publication Ethics. https://publicationethics.org/topic- discussions/bias-peer-review
-
[5]
Scientists hide messages in papers to game AI peer review
Gibney, E. Scientists hide messages in papers to game AI peer review. Nature 643, 887–888 (2025)
work page 2025
- [6]
-
[7]
https://coara.org/coalition/coalition/
Home | CoARA. https://coara.org/coalition/coalition/
- [8]
- [9]
-
[10]
https://www.ukri.org/publications/review- of-metrics-in-research-assessment-and- management/
The metric tide: review of metrics in research assessment – UKRI. https://www.ukri.org/publications/review- of-metrics-in-research-assessment-and- management/
-
[11]
Zhang, L. & Sivertsen, G. The new research assessment reform in China and its implementation. 2 2, 3 (2020)
work page 2020
-
[12]
Shu, F ., Liu, S. & Larivière, V . China’s Research Evaluation Reform: What are the Consequences for Global Science? Minerva 60, 329–347 (2022)
work page 2022
-
[13]
https://sfdora.org/2025/06/17/rra-in-india- 2/?utm_source=chatgpt.com
Implementing Responsible Research Assessment in India | DORA. https://sfdora.org/2025/06/17/rra-in-india- 2/?utm_source=chatgpt.com
work page 2025
-
[14]
A new narrative CV mentorship platform, funder research assessment reform reading list, and updates from the Japan Science and Technology Agency | DORA. https://sfdora.org/2024/02/05/a-new- narrative-cv-mentorship-platform-funder- research-assessment-reform-reading-list- and-updates-from-the-japan-science-and- technology-agency/
work page 2024
-
[15]
Research collaborations bring big rewards: the world needs more. Nature 594, 301–302 (2021)
work page 2021
- [16]
- [17]
- [18]
- [19]
-
[20]
https://www.nic.gov.in/service/nkn/
National Knowledge Network | National Informatics Centre | India. https://www.nic.gov.in/service/nkn/
-
[21]
DeSci Labs – Empowering Researchers from Data to Publication. https://www.desci.com/
-
[22]
Conaldi, G., De Vita, R., Ghinoi, S. & Foster, D. M. Virtual collaborative spaces: a case study on the antecedents of collaboration in an open-source software community. R and D Management 54, 347–369 (2024)
work page 2024
-
[23]
Open and Universal Science (OPUS) Project. https://opusproject.eu/about/
-
[24]
https://siepic.ca/fabrication/
The Canadian Silicon Photonics Foundry - Siepic. https://siepic.ca/fabrication/
- [25]
-
[26]
Indicators and Metrics to Test in the Pilots
Gareth O Neill. Indicators and Metrics to Test in the Pilots. Acknowledgements We would like to thank Dr. Miranda Vinay for their valuable discussions and feedback that greatly improved the work. Competing interest Cristiano Matricardi, the corresponding author of this paper, worked as an editor at Nature Communications during the preparation of this manu...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.