pith. machine review for the scientific record. sign in

arxiv: 2512.01669 · v1 · submitted 2025-12-01 · 💻 cs.DL

Recognition: 2 theorem links

· Lean Theorem

Mapping the Landscape of Open Access Dashboards -- A Dataset for Research and Infrastructure Development

Authors on Pith no claims yet

Pith reviewed 2026-05-17 03:18 UTC · model grok-4.3

classification 💻 cs.DL
keywords open accessdashboardsmetadata schemadatasetopen sciencesurveyscience policyresearch infrastructure
0
0 comments X

The pith

A survey has produced a dataset of nearly 60 open access dashboards described by a new metadata schema to support research on open science monitoring.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Open access publication rates matter for science policy at institutional, national, and global scales, yet the tools that track them vary widely. The authors surveyed the field and identified nearly 60 dashboards that display these rates. They created a metadata schema that records each dashboard's scope, data sources, update methods, and display features in a uniform way. The resulting structured dataset is released for public use and invites ongoing contributions from interested parties. Researchers in library and information science or science studies can draw on it for empirical work and for sharpening the indicators that guide open science policy.

Core claim

The authors conducted an extensive survey that located nearly 60 open access dashboards and indexed each one according to a purpose-built metadata schema. This schema captures characteristics such as geographic and institutional coverage, data provenance, visualization techniques, and maintenance status. The indexed collection is offered as a living resource that stakeholders can extend through a participatory process, thereby supporting both quantitative studies of open access trends and the iterative improvement of policy-relevant metrics.

What carries the argument

The metadata schema, which supplies a standardized set of fields for recording the scope, technical properties, and policy relevance of open access dashboards.

If this is right

  • The dataset enables systematic empirical analyses of how open access progress is currently measured and communicated.
  • It provides a foundation for refining indicators and policy instruments in open science contexts.
  • Community contributions can maintain the inventory as new dashboards appear.
  • Researchers gain a shared reference point for comparing monitoring efforts across organizations and countries.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The schema might be extended to capture dashboards that track related open science practices such as data sharing or preprint adoption.
  • Policymakers could use the indexed descriptions to identify coverage gaps, for instance in regions with limited dashboard availability.
  • Linking the dataset to publication databases could allow automated checks on whether dashboard numbers match actual open access shares.
  • Future surveys might test whether dashboards that follow the schema's recommended features prove more useful to decision makers than those that do not.

Load-bearing premise

The survey captured a representative collection of existing dashboards and the chosen metadata schema is adequate for supporting empirical analyses and policy refinements.

What would settle it

A subsequent independent search that locates a substantial number of additional open access dashboards absent from the dataset, or practical tests showing that the schema fields produce inconsistent or uninformative results when used for cross-dashboard comparisons.

Figures

Figures reproduced from arXiv: 2512.01669 by Heinz Pampel, Johannes Schneider.

Figure 1
Figure 1. Figure 1: Dashboard website 9 [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
read the original abstract

As Open Access continues to gain importance in science policy, understanding the proportion of Open Access publications relative to the total research output of research-performing organizations, individual countries, or even globally has become increasingly relevant. In response, dashboards are being developed to capture and communicate progress in this area. To provide an overview of these dashboards and their characteristics, an extensive survey was conducted, resulting in the identification of nearly 60 dashboards. To support a detailed and structured description, a dedicated metadata schema was developed, and the identified dashboards were systematically indexed accordingly. To foster community engagement and ensure ongoing development, a participatory process was launched, allowing interested stakeholders to contribute to the dataset. The dataset is particularly relevant for researchers in Library and Information Science (LIS) and Science and Technology Studies (STS), supporting both empirical analyses of Open Access and the methodological refinement of indicators and policy instruments in the context of Open Science.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper reports on an extensive survey that identified nearly 60 open access dashboards, introduces a dedicated metadata schema for systematically indexing their characteristics, and describes a participatory process for ongoing community contributions to the resulting dataset. The work positions the dataset as a resource for empirical analyses of open access and for methodological refinement of indicators and policy instruments in open science, with relevance to researchers in library and information science and science and technology studies.

Significance. If the survey coverage is representative and the schema is robust, the dataset would provide a valuable, structured foundation for studying the landscape of open access monitoring tools. This could support reproducible empirical work on open science indicators and help refine policy instruments, with the participatory maintenance model offering a practical strength for long-term utility.

major comments (2)
  1. [Abstract] The central claim that an 'extensive survey' identified nearly 60 dashboards rests on undocumented methodology. No search protocol, list of queried databases or sources, definition of 'dashboard', time window, or inclusion/exclusion criteria are provided, rendering representativeness and reproducibility untestable (see Abstract and any Methods section describing the survey process).
  2. [Schema development and indexing description] Without documented validation steps or inter-rater reliability measures for schema application, it is unclear whether the indexing of the ~60 dashboards was applied consistently, which directly affects the dataset's utility for the claimed empirical analyses and policy refinement.
minor comments (2)
  1. [Results or Dataset description] Clarify the exact number of dashboards included (the abstract says 'nearly 60' while the reader's summary references 'nearly 60'); provide a precise count and any breakdown by type or region.
  2. [Participatory process section] The participatory process is mentioned but lacks details on how contributions will be moderated or integrated; add a brief description of the workflow.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We address each major comment below and will revise the paper to improve methodological transparency and documentation of the schema application process.

read point-by-point responses
  1. Referee: [Abstract] The central claim that an 'extensive survey' identified nearly 60 dashboards rests on undocumented methodology. No search protocol, list of queried databases or sources, definition of 'dashboard', time window, or inclusion/exclusion criteria are provided, rendering representativeness and reproducibility untestable (see Abstract and any Methods section describing the survey process).

    Authors: We agree that the survey methodology requires fuller documentation to support reproducibility and evaluation of representativeness. In the revised manuscript we will add a dedicated Methods section that specifies the search protocol, the databases and other sources queried, our operational definition of an open access dashboard, the survey time window, and the inclusion and exclusion criteria applied. These additions will make the process of identifying the nearly 60 dashboards transparent and testable. revision: yes

  2. Referee: [Schema development and indexing description] Without documented validation steps or inter-rater reliability measures for schema application, it is unclear whether the indexing of the ~60 dashboards was applied consistently, which directly affects the dataset's utility for the claimed empirical analyses and policy refinement.

    Authors: We acknowledge the value of documenting validation procedures for schema application. The revised manuscript will expand the description of schema development to include the iterative process used to create the metadata fields and the steps taken to apply the schema consistently across the identified dashboards. We will also report any validation activities performed and, where formal inter-rater reliability statistics were not calculated, we will transparently describe the collaborative review methods employed and note this as a limitation. revision: yes

Circularity Check

0 steps flagged

No circularity: purely descriptive survey and dataset construction

full rationale

The paper reports conducting a survey that identified nearly 60 dashboards, developing a metadata schema, indexing the dashboards, and launching a participatory process for ongoing contributions. No derivation chain, equations, predictions, fitted parameters, or self-referential claims exist in the abstract or described content. The work is a cataloging and schema-creation effort without any reduction of outputs to inputs by construction, self-citation load-bearing premises, or renaming of known results as novel derivations. The contribution stands as self-contained descriptive research.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper rests on standard survey methodology in information science and domain assumptions about the value of open access monitoring; no free parameters, ad-hoc axioms, or invented entities are introduced.

pith-pipeline@v0.9.0 · 5450 in / 984 out tokens · 30881 ms · 2026-05-17T03:18:57.923087+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

44 extracted references · 44 canonical work pages · 2 internal anchors

  1. [1]

    Suber,Open Access

    P. Suber,Open Access. Cambridge: The MIT Press, 2012,isbn: 978-0- 262-30173-2.doi:10.7551/mitpress/9286.001.0001Accessed: Mar. 4, 2023

  2. [2]

    Analysis of comments and implementation of the NIH public access policy,

    National Institutes of Health, “Analysis of comments and implementation of the NIH public access policy,” 2008. [Online]. Available:https : / / publicaccess.nih.gov/analysis_of_comments_nih_public_access_ policy.pdf

  3. [3]

    Nih-funded science must now be free to read instantly: What you should know,

    M. Lenharo, “Nih-funded science must now be free to read instantly: What you should know,”Nature, Jun. 2025,issn: 1476-4687.doi:10 . 1038 / d41586-025-01938-8

  4. [4]

    22, 2020

    European Commission,Access to and preservation of scientific informa- tion in Europe: report on the implementation of Commission Recommen- dation C(2012) 4890 final.LU: Publications Office, 2015.doi:10.2777/ 975917Accessed: Sep. 22, 2020

  5. [5]

    Background note on open access to scientific pub- lications and open research data,

    European Commission, “Background note on open access to scientific pub- lications and open research data,” 2016

  6. [6]

    Council conclusions on the transition towards an open science system (no. 9526/16),

    Council of the European Union, “Council conclusions on the transition towards an open science system (no. 9526/16),” 2016. [Online]. Available: https://data.consilium.europa.eu/doc/document/ST-9526-2016- INIT/en/pdf

  7. [7]

    Few,Information dashboard design, The effective visual communication of data, 1

    S. Few,Information dashboard design, The effective visual communication of data, 1. Aufl. Sebastopol, CA: O’Reilly & Associates, 2006, 211 pp., isbn: 9780596100162

  8. [8]

    The johns hopkins university center for systems science and engineering COVID-19 dashboard: Data collection process, challenges faced, and lessons learned,

    E. Dong et al., “The johns hopkins university center for systems science and engineering COVID-19 dashboard: Data collection process, challenges faced, and lessons learned,”The Lancet Infectious Diseases, vol. 22, no. 12, e370–e376, Dec. 1, 2022, Publisher: Elsevier,issn: 1473-3099, 1474-4457. doi:10.1016/S1473-3099(22)00434-0Accessed:Apr.17,2025.[Online]...

  9. [9]

    Characteristics and specifications of dash- boards developed for the COVID-19 pandemic: A scoping review,

    T. Khodaveisi, H. Dehdarirad, H. Bouraghi, A. Mohammadpour, F. Sa- jadi, and M. Hosseiniravandi, “Characteristics and specifications of dash- boards developed for the COVID-19 pandemic: A scoping review,”Journal of Public Health, vol. 32, no. 4, pp. 553–574, Apr. 1, 2024,issn: 1613-2238. doi:10.1007/s10389-023-01838-zAccessed: Apr. 17, 2025. 12

  10. [10]

    Advances in Engineering Software42(12), 1020–1034 (2011)

    A. Vahedi, H. Moghaddasi, F. Asadi, A. S. Hosseini, and E. Nazemi, “Ap- plications, features and key indicators for the development of covid-19 dashboards: A systematic review study,”Informatics in Medicine Un- locked, vol. 30, p. 100910, Jan. 1, 2022,issn: 2352-9148.doi:10.1016/j. imu.2022.100910Accessed: Apr. 17, 2025. [Online]. Available:https:// www.s...

  11. [11]

    Using current research information systems (CRIS) to showcase national and institutional re- search (potential): Research information systems in the context of open science,

    S. Biesenbender, S. Petersohn, and C. Thiedig, “Using current research information systems (CRIS) to showcase national and institutional re- search (potential): Research information systems in the context of open science,”Procedia Computer Science, 14th International Conference on Current Research Information Systems, CRIS2018, FAIRness of Research Inform...

  12. [12]

    Proportion of open access papers published in peer-reviewed journals at the european and world levels—1996–2013,

    É. Archambault et al., “Proportion of open access papers published in peer-reviewed journals at the european and world levels—1996–2013,”

  13. [13]

    Available:https://digitalcommons.unl.edu/scholcom/ 8

    [Online]. Available:https://digitalcommons.unl.edu/scholcom/ 8

  14. [14]

    The state of oa: A large-scale analysis of the prevalence and impact of open access articles,

    H. Piwowar et al., “The state of oa: A large-scale analysis of the prevalence and impact of open access articles,”PeerJ, vol. 6, e4375, Feb. 2018,issn: 2167-8359.doi:10.7717/peerj.4375

  15. [15]

    On the challenges of open access mon- itoring,

    A. Salamoura and G. Tsakonas, “On the challenges of open access mon- itoring,”Insights, vol. 37, no. 1, Feb. 13, 2024,issn: 2048-7754.doi: 10.1629/uksg.641Accessed: Apr. 17, 2025. [Online]. Available:https: //insights.uksg.org/articles/10.1629/uksg.641

  16. [16]

    Monitoring open science trends in europe

    Rand. “Monitoring open science trends in europe. ”[Online]. Available: https://www.rand.org/randeurope/research/projects/2017/open- science-monitor.html

  17. [17]

    Elsevier will monitor open science in EU using measurement system that favors its own titles,

    G. Moody. “Elsevier will monitor open science in EU using measurement system that favors its own titles,” Techdirt, Accessed: Apr. 17, 2025. [On- line]. Available:https://www.techdirt.com/2018/07/09/elsevier- will-monitor-open-science-eu-using-measurement-system-that- favors-own-titles/

  18. [18]

    Hated science publisher elsevier to help EU monitor open science – including open access,

    G. Moody. “Hated science publisher elsevier to help EU monitor open science – including open access,” Techdirt, Accessed: Apr. 17, 2025. [On- line]. Available:https : / / www . techdirt . com / 2018 / 04 / 03 / hated - science-publisher-elsevier-to-help-eu-monitor-open-science- including-open-access/

  19. [19]

    Feedback on EC open science monitor methodologicalnote

    French Open Science Committee, “Feedback on EC open science monitor methodologicalnote.”[Online].Available:https://www.ouvrirlascience. fr/wp- content/uploads/2018/11/Feedback- on- EC- Open- Science- Monitor-Methodological-note.pdf 13

  20. [20]

    Facts and figures for open research data -

    European Commission. “Facts and figures for open research data -. ”[On- line]. Available:https : / / research - and - innovation . ec . europa . eu / strategy / strategy - research - and - innovation / our - digital - future/open-science/open-science-monitor/facts-and-figures- open-research-data_en

  21. [21]

    The new OpenAIRE monitor: Brand new dashboards and features!

    L. Pispiringas, I. Grypari, H. Dimitropoulos, and A. Bardi. “The new OpenAIRE monitor: Brand new dashboards and features!” OpenAIRE, Accessed: May 22, 2025. [Online]. Available:https://www.openaire.eu/ the-new-openaire-monitor-brand-new-dashboards-and-features

  22. [22]

    Introducing the EOSC observatory,

    G. O’Neill. “Introducing the EOSC observatory,” OpenAIRE, Accessed: May22,2025.[Online].Available:https://www.openaire.eu/introducing- the-eosc-observatory

  23. [23]

    Measurement of open access as an infrastructural challenge: The case of finland,

    P. Olsbo, “Measurement of open access as an infrastructural challenge: The case of finland,” inExpanding Perspectives on Open Science: Com- munities, Cultures and Diversity in Concepts and Practices, IOS Press, 2017, pp. 217–226.doi:10.3233/978- 1- 61499- 769- 6- 217Accessed: Apr. 17, 2025

  24. [24]

    Rolle des Open Access Monitor Deutschland bei der AntragstellungimDFG-Förderprogramm„Open-Access-Publikationskosten“,

    B. Mittermaier, “Rolle des Open Access Monitor Deutschland bei der AntragstellungimDFG-Förderprogramm„Open-Access-Publikationskosten“,” o-bib. Das offene Bibliotheksjournal / Herausgeber VDB, 1–14 Seiten, Nov. 9, 2021, Artwork Size: 1-14 Seiten Publisher: o-bib. Das offene Biblio- theksjournal / Herausgeber VDB.doi:10.5282/O- BIB/5731Accessed: Apr. 17, 20...

  25. [25]

    SynOA Synergien für Open Access – Open Access Monitoring Schlussbericht,

    I. Barbers, B. Mittermaier, P. Pollack, B. Lindstrot, and H. Schmiedicke- Hintzen, “SynOA Synergien für Open Access – Open Access Monitoring Schlussbericht,” Forschungszentrum Jülich GmbH Zentralbibliothek, Ve- lag, FZJ-2021-01137, 2021. Accessed: May 22, 2025. [Online]. Available: http://hdl.handle.net/2128/27226

  26. [26]

    Promoting open science through bibliometrics: A practical guide to build an open access monitor,

    L. Bracco, “Promoting open science through bibliometrics: A practical guide to build an open access monitor,”LIBER Quarterly: The Journal of the Association of European Research Libraries, vol. 32, no. 1, pp. 1– 18, Oct. 3, 2022, Number: 1,issn: 2213-056X.doi:10.53377/lq.11545 Accessed: May 23, 2025. [Online]. Available:https://liberquarterly. eu/article/...

  27. [27]

    Swiss open access monitor goes live

    Lib4RI. “Swiss open access monitor goes live. ”[Online]. Available:https: / / www . lib4ri . ch / swiss - open - access - monitor - goes - live ? utm _ source=chatgpt.com

  28. [28]

    Report on

    P. Danowski et al., “Report on "open access monitoring – approaches and perspectives" (2-day-workshop, 09–10 april 2018, vienna),” Zenodo, Dec. 19, 2018.doi:10.5281/zenodo.2415894Accessed: Apr. 17, 2025. [Online]. Available:https://zenodo.org/records/2415894 14

  29. [29]

    Open-access-barometer 2016,

    F. Jülich. “Open-access-barometer 2016,” Accessed: May 22, 2025. [On- line]. Available:https://www.fz- juelich.de/de/zb/open- science/ open-access/oa-barometer/oa-barometer-2016

  30. [30]

    Check your openness with the UZH OA-monitor (with new feature!)

    U. Zürich. “Check your openness with the UZH OA-monitor (with new feature!)” Blog derUB Zürich, Accessed: May22, 2025.[Online].Available: https://www.uzh.ch/blog/ub/2024/10/15/check- your- openness- with-the-uzh-oa-monitor-with-new-feature/?lang=en

  31. [31]

    Initiating discipline-specific open science monitoring with the open science dashboard for earth sciences,

    M. Duine, A. Iarkaeva, and A. Hübner, “Initiating discipline-specific open science monitoring with the open science dashboard for earth sciences,” Publisher: Zenodo, Nov. 15, 2024.doi:10.5281/zenodo.14170751Ac- cessed:Apr.24,2025.[Online].Available:https://zenodo.org/records/ 14170751

  32. [32]

    CHORUS – a solution for public access,

    F. Dylla, “CHORUS – a solution for public access,”Information Services and Use, vol. 34, no. 3, pp. 195–199, Aug. 2014,issn: 0167-5265, 1875- 8789.doi:10.3233/ISU-140740Accessed: Apr. 17, 2025. [Online]. Avail- able:https://journals.sagepub.com/doi/10.3233/ISU-140740

  33. [33]

    Europe PMC funders’ dashboard – europe PMC news blog

    Europe PMC team. “Europe PMC funders’ dashboard – europe PMC news blog. ”[Online]. Available:https://blog.europepmc.org/2020/ 02/europe-pmc-funders-dashboard.html

  34. [34]

    New funders page added to OpenAIRE EXPLORE!

    A. Brunschweiger. “New funders page added to OpenAIRE EXPLORE!” OpenAIRE, Accessed: Apr. 17, 2025. [Online]. Available:https://www. openaire.eu/new-funders-page-added-to-openaire-explore

  35. [35]

    Open and shut?: Peter suber on the state of open access: Where are we, what still needs to be done?

    R. Poynder. “Open and shut?: Peter suber on the state of open access: Where are we, what still needs to be done?” Open and Shut? Accessed: Apr. 17, 2025. [Online]. Available:https : / / poynder . blogspot . com / 2013/07/peter-suber-on-state-of-open-access.html

  36. [36]

    "Empfehlung

    P. Danowski et al., “"Empfehlung" für die weitere Vorgangsweise für das Open-Access-Monitoring. Deliverable des AT2OA-Teilprojekts TP1-B,” Mitteilungen der Vereinigung Österreichischer Bibliothekarinnen und Bib- liothekare, vol. 73, no. 2, pp. 278–284, Jul. 14, 2020, Number: 2,issn: 2791-4011.doi:10.31263/voebm.v73i2.3941Accessed: Apr. 17, 2025. [Online]....

  37. [37]

    Open access monitoring: Guidelines and recommendations for research organisations and funders,

    T. Philipp, G. Botz, J.-C. Kita, A. Sänger, O. Siegert, and M. Reumaux, “Open access monitoring: Guidelines and recommendations for research organisations and funders,” May 10, 2021.doi:10.5281/zenodo.4905554 Accessed: Apr. 17, 2025. [Online]. Available:https : / / zenodo . org / records/4905554

  38. [38]

    Datenpraxiszurgestaltungderopen-access-transformation - analyse, empfehlung, training & vernetzung (OA datenpraxis),

    H.Pampeletal.,“Datenpraxiszurgestaltungderopen-access-transformation - analyse, empfehlung, training & vernetzung (OA datenpraxis),” Mar. 11, 2024.doi:10.5281/zenodo.10794298Accessed: Mar. 11, 2024. 15

  39. [39]

    Open science dashboard,

    K. Cobey, D. Moher, C. Ripp, D. Franzen, A. Armond, and S. Haustein, “Open science dashboard,” Sep. 7, 2021, Publisher: OSF.doi:10.17605/ OSF.IO/JM8WGAccessed: May 22, 2025. [Online]. Available:https://osf. io/jm8wg/

  40. [40]

    Metadata schema for the description of research data repositories : Version 4.0,

    D. Strecker et al., “Metadata schema for the description of research data repositories : Version 4.0,” 2023, Publisher: re3data.doi:10.48440/re3. 014Accessed: Jun. 1, 2024. [Online]. Available:https : / / gfzpublic . gfz-potsdam.de/pubman/faces/ViewItemOverviewPage.jsp?itemId= item_5022309

  41. [41]

    Jahn,Datasets on fee-based Open Access publishing across German In- stitutions, in collab

    N. Jahn,Datasets on fee-based Open Access publishing across German In- stitutions, in collab. with N. Jahn et al., Medium: text/csv; charset=UTF- 8,text/plain; charset=UTF-8,text/x-markdown; charset=UTF-8,text/x-r; charset=UTF-8, 2014.doi:10 . 4119 / UNIBI / UB . 2014 . 18Accessed: May 21, 2025. [Online]. Available:https://gitlab.ub.uni-bielefeld. de/njah...

  42. [42]

    OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts

    J. Priem, H. Piwowar, and R. Orr,OpenAlex: A fully-open index of schol- arly works, authors, venues, institutions, and concepts, Jun. 17, 2022.doi: 10.48550/arXiv.2205.01833arXiv:2205.01833[cs].Accessed:May21,

  43. [43]
  44. [44]

    Barcelona declaration on open research information,

    “Barcelona declaration on open research information,” Barcelona Dec- laration on Open Research Information. [Online]. Available:https : / / barcelona-declaration.org/ 8 Author Contributions J.S.andH.P.conceptualizedthestudyanddevelopedthedatacollectionmethod- ology; J.S. curated the data; J.S. and H.P. wrote the manuscript. Both authors contributed to the...