Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

FAIR and Open Computer Science Research Software

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that research software in computer science is maintained for years, while software tied to computational science papers is typically abandoned within days, reflecting different publishing cultures.

desk verdict A useful and honest position paper whose headline empirical gap is real but likely inflated by comparing software-as-output with mature third-party tools in the ACM sample. read the letter →

arxiv 1908.05986 v1 pith:WNJTZ52U submitted 2019-08-16 cs.SE

Wilhelm Hasselbring , Leslie Carr , Simon Hettrick , Heather Packer , Thanassis Tiropanis This is my paper · ORCID
classification cs.SE
keywords researchsoftwareFAIRprinciplesopensciencereproducibilityreusabilityGitHubartifactevaluationcitation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that 'research software' is not one thing: computer science and computational science publish and sustain software in fundamentally different ways. Analyzing over 5,000 GitHub repositories linked to publications, it finds that software cited by ACM computer science papers has a median lifespan of about five years, software linked to DOIs in computational science papers lives a median of about fifteen days, and arXiv-linked software sits in between at about eight months. The paper reads this as computational science treating software as a reproducibility artifact, released with the paper and then abandoned, while computer science treats software as reusable infrastructure maintained for years. From this it recommends pairing GitHub for maintenance and reuse with Zenodo for archival snapshots, plus artifact evaluation and software citation metadata, to make research software both FAIR and open. A sympathetic reader should care because the numbers expose a sustainability gap hidden by the single label 'research software'.

What carries the argument

The mechanism is a three-way corpus linking publications to software. Research software is identified either by a publication containing a GitHub link or by a GitHub repository containing a DOI. Three sets are formed: 1,204 GitHub repositories containing a DOI, mostly from computational science; 1,091 GitHub repositories cited in ACM digital library papers, mostly from computer science; and 2,872 repositories cited by arXiv e-prints, mostly from AI-related computer science. Lifespan is defined as the time between first and last commit, and a repository is 'live' if it had any activity in the past year. These choices produce the headline medians and the characterization of different publishing cultures.

What would settle it

Re-run the analysis with the same definitions but include Bitbucket, GitLab, and links through software manuals and papers, and also track whether DOI-linked repositories continue to receive commits on other hosts after their GitHub snapshot; if the median lifespan gap shrinks to months or disappears, the paper's central difference is not about sustainability but about where software is hosted.

Watch

Extended reading notes

Core claim

The paper's central empirical claim is that research software publishing practices differ sharply across disciplines: computational science emphasizes reproducibility, releasing software to accompany a paper and then leaving it unmaintained, while computer science emphasizes reuse, keeping software alive as infrastructure. The supporting observation is the lifespan distribution of GitHub repositories connected to publications: median 5 years for ACM-linked computer science software, median 15 days for DOI-linked computational science software, and median 8 months for arXiv-linked software, with 75% of arXiv repositories active. The paper also proposes that the arXiv/AI community's 'publish early' attitude correlates with developing software openly from the start of a project. Based on this, it argues that research software should be open source, archived for reproducibility and actively maintained for reuse, with artifact evaluation and software citation standards in place.

Load-bearing premise

The whole comparison rests on identifying research software through a GitHub link in a publication or a DOI in a GitHub repository; if computational science software is systematically less likely to be on GitHub or cited through those routes, the five-year versus fifteen-day gap could be an artifact of citation and hosting habits rather than a real difference in maintenance.

Editorial extensions

If this is right

  • If the median-lifespan gap is real, computational science software is currently a reproducibility artifact rather than a reusable asset, so the FAIR goal of reuse requires new publishing incentives there.
  • A practical workflow that follows from the analysis is to use GitHub for living development and Zenodo for archived, citable snapshots, so the same software serves reuse and reproducibility.
  • Artifact evaluation, already correlated with higher citation counts in the cited work, becomes a concrete lever for improving reusability if adopted more widely in computer science subdisciplines.
  • Software citation metadata standards such as CFF and CodeMeta only help if widely adopted; the paper implies that search engines exploiting them are the missing piece.
  • Research software observatories that catalogue software-to-publication links would make analyses like this repeatable and extendable to GitLab, Bitbucket, and non-GitHub citations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As an inference beyond the paper: the 15-day median may overstate abandonment in computational science, because researchers there often keep code in lab servers or institutional repositories and publish only a GitHub snapshot at paper time; a study that tracks private maintenance would separate 'not shared' from 'not maintained'.
  • The paper's longevity hypothesis for computer science—that long-lived repositories are often commercial open-source frameworks rather than researcher-built code—implies that the 5-year median partly measures industry infrastructure, so the gap may be about who funds maintenance, not which field values reuse.
  • A testable extension: if the arXiv/AI 'publish early' culture drives open development, commit activity for arXiv-linked repositories should cluster before or at paper submission rather than after acceptance; the paper does not run this check.
  • The observatory proposal implies a citation-graph standard; one could extend the analysis to count software mentions in full text rather than URLs, which would capture papers and manuals as citation routes and likely shrink the apparent gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper argues that research software should be FAIR and open, reviews current obstacles and practices (software credit, artifact evaluation, archival vs. development platforms), and presents an initial empirical study that relates research publications to GitHub repositories via three routes: publications in the ACM Digital Library, repositories containing DOIs, and e-prints on arXiv. The study reports lifespan distributions for the three sets, with a headline observation that ACM-linked computer science repositories have a median lifespan of about five years, DOI-linked computational science repositories have a median lifespan of about 15 days, and arXiv-linked repositories fall in between (about eight months). On this basis, the paper claims that computational science emphasizes reproducibility while computer science emphasizes reuse, and it proposes Research Software Observatories and recommendations for making computer science research software FAIR and open. The paper is framed as an initial study and explicitly discloses its sampling assumptions in 'Threats to Validity'.

Significance. If its empirical claims were established, this paper would provide a useful quantitative picture of field differences in research software sustainability and publishing practices, which is timely for the FAIR research software agenda. The authors deserve credit for clearly stating their sampling assumptions, for labeling the analysis as an initial study, and for pairing the empirical exploration with concrete recommendations (the GitHub/Zenodo archival-versus-development split, artifact evaluation, software citation metadata, and Research Software Observatories) that have value independent of the specific median values. The proposal of Research Software Observatories is a constructive step toward more rigorous comparative studies. However, the headline empirical claim is not currently supported by the presented evidence because of the confounds and missing statistical analysis detailed below.

major comments (3)
  1. [Relationships and categories / Figure 3] The central empirical comparison in Figure 3 is confounded: the ACM-linked set is constructed from publications that cite GitHub repositories, and the paper's own examples (Caffe, Apache Spark, three.js, Google Guava, and the Hough et al. repository) show that it includes mature third-party tools and software studied as an object of analysis, which are not outputs of the citing papers. In contrast, the DOI-linked set is constructed from repositories that cite a publication, i.e., software self-identifying as accompanying a paper. The reported median lifespan gap (5 years versus 15 days) may therefore reflect different inclusion criteria and link directions rather than different field-specific publishing cultures. To support the 'reuse vs. reproducibility' claim, the analysis should either restrict the ACM set to repositories that are outputs of the citing paper or systematically code and control for the relationship type (output, tool, object of study, fork, background) across all three datasets.
  2. [Sustainability of research software] The claim of 'significant differences' in the Key Insights is not supported by any statistical testing: no significance tests, confidence intervals, or effect sizes are reported, and there are no controls for publication year, repository age, or activity level. In addition, the lifespan defined as the interval between first and last commit is right-censored for repositories that are still live, which can bias the medians when comparing sets with different fractions of active repositories (e.g., 75% live for arXiv versus lower fractions elsewhere). The authors should provide inferential statistics and a censoring-aware (e.g., survival) analysis, and they should release the data and analysis scripts, especially since the paper advocates FAIR and open practices that would demand such transparency.
  3. [Sustainability of research software / Relationships and categories] The paper's qualitative interpretation that computer science emphasizes reuse while computational science emphasizes reproducibility is not directly measured. The taxonomy of relationships is derived from manual inspection of only the most active ACM repositories, and no corresponding coding is reported for the DOI or arXiv sets; the sentences 'Our hypothesis is...' are presented as explanations without testing. Either the manuscript should be reframed as proposing hypotheses for future work, or the relationship categories should be coded systematically across the full dataset to test whether the field difference in lifespans is associated with the repository role in the publication.
minor comments (4)
  1. [Covered research areas] There are typos in this section: 'An first interesting observation' should be 'A first interesting observation', and 'Remind that we only collect...' should be 'Note that we only collect...'.
  2. [Figure 3] The y-axis label 'Respoitories' in Figure 3a is misspelled; it should be 'Repositories'. Also, 'Github' should be 'GitHub' in the figure captions for consistency.
  3. [Abstract] The abstract expands FAIR as 'findable, accessible, interoperable and repeatable', but the R in FAIR stands for 'Reusable'; the paper itself uses 'Reusable' in the main text. This inconsistency should be corrected.
  4. [Sustainability of research software] The sentence 'A third of these repositories are live for less than 1 day' is confusing because 'live' is defined by any activity during the last year; the intended meaning is likely that a third of the repositories have a lifespan of less than one day. Please rephrase to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical comparison is computed from independently defined GitHub metadata and is not fitted to its conclusions.

full rationale

The paper reports an initial observational study, not a mathematical derivation. It operationally defines research software by two citation directions (publications citing GitHub repositories versus GitHub repositories citing DOIs), assembles roughly 5,000 repositories, computes lifespan distributions from commit activity, and reports the medians. The differences (5 years vs. 15 days vs. 8 months) are calculated from repository metadata rather than fitted to the paper's conclusions, so the headline comparison is not a prediction that reduces to its inputs. The authors explicitly label their explanatory stories as hypotheses ('Our hypothesis is that in computer science research, often commercial open-source software frameworks are employed'), not as derived results. The paper's self-citations (e.g., Hasselbring, Tiropanis) support background claims and the proposed Research Software Observatories, but no step of the empirical analysis depends on an unverified prior result by the same authors. The Threats to Validity section acknowledges the coverage restrictions ('We are well aware that these assumptions restrict the coverage of our analysis'), which is a limitation on external validity, not a circular step. Concerns that the ACM set includes mature third-party tools such as Caffe, Spark, three.js, and Guava are construct-validity objections and belong under correctness risk, not circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The paper's empirical claims rest on the GitHub-link identification assumption and the choice of lifespan and dormancy metrics. No fitted model parameters are used; the single hand-chosen threshold is the one-year dormancy window. The only introduced entity is the proposed Research Software Observatory, which is a conceptual infrastructure, not a scientific entity.

free parameters (1)
  • dormancy window = 12 months
    A repository is considered dormant if it had no commit activity in the last year (Section 'Sustainability of research software'); this hand-chosen cutoff affects the live/dormant classification.
assumptions (3)
  • domain assumption Research software can be identified by GitHub links in publications or DOIs in repositories.
    The analysis assumes this link structure, explicitly excluding Bitbucket, GitLab, and citations via manuals or papers (Assumptions, Section 'Relating Research Software to Research Publications').
  • domain assumption Repository lifespan is measured as the time between first and last commit.
    This operationalization may misclassify stable software with no recent commits as dormant (Section 'Sustainability of research software').
  • domain assumption The three datasets (DOI, ACM, arXiv) are representative enough to compare disciplinary practices.
    Coverage is restricted by the GitHub-link assumption, and the authors acknowledge this in 'Threats to Validity'.
invented entities (1)
  • Research Software Observatory
    purpose: A decentralized catalogue of research software with metadata and analytics to support findability, citation, and analysis of software-publication relationships.
    Proposed infrastructure, not yet implemented or validated against external data (Section 'Observatories for FAIR and Open Research Software').

how reviews work

0 comments
Cite this review

Pith. "Pith review of FAIR and Open Computer Science Research Software." pith.science (2026). https://pith.science/paper/WNJTZ52U

@misc{pith2026190805986,
  author       = {Pith},
  title        = {Pith review of: FAIR and Open Computer Science Research Software},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WNJTZ52U}},
  note         = {Machine review of arXiv:1908.05986}
}
read the original abstract

In computational science and in computer science, research software is a central asset for research. Computational science is the application of computer science and software engineering principles to solving scientific problems, whereas computer science is the study of computer hardware and software design. The Open Science agenda holds that science advances faster when we can build on existing results. Therefore, research software has to be reusable for advancing science. Thus, we need proper research software engineering for obtaining reusable and sustainable research software. This way, software engineering methods may improve research in other disciplines. However, research in software engineering and computer science itself will also benefit from reuse when research software is involved. For good scientific practice, the resulting research software should be open and adhere to the FAIR principles (findable, accessible, interoperable and repeatable) to allow repeatability, reproducibility, and reuse. Compared to research data, research software should be both archived for reproducibility and actively maintained for reusability. The FAIR data principles do not require openness, but research software should be open source software. Established open source software licenses provide sufficient licensing options, such that it should be the rare exception to keep research software closed. We review and analyze the current state in this area in order to give recommendations for making computer science research software FAIR and open. We observe that research software publishing practices in computer science and in computational science show significant differences.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GitHub Repository Complexity Leads to Diminished Web Archive Availability

    cs.DL 2025-05 conditional novelty 6.0 of 10

    More than 12,000 archived software repository home pages were examined, showing that less than 5% of source files are typically archived and that deeper source files are archived much less often.

Reference graph

Works this paper leans on

42 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    Realising the European Open Science Cloud

    Paul Ayris, Jean-Yves Berthou, Rachel Bruce, Stefanie Lindstaedt, Anna Monreale, Barend Mons, Y asuhiro Murayama, Caj Södergård, Klaus Tochtermann, and Ross Wilkin- son. Realising the European Open Science Cloud. European Union, Luxembourg, 2016. doi:10.2777/940154

    work page doi:10.2777/940154 2016

  2. [2]

    Free and open source software licenses explained

    Miriam Ballhausen. Free and open source software licenses explained. Computer, 52(06):82–86, June 2019. doi:10.1109/MC.2019.2907766. 17

    arXiv 2019

  3. [3]

    Generating CodeMeta Metadata for R Packages

    Carl Boettiger. Generating CodeMeta Metadata for R Packages. Journal of Open Source Software, 2(19):454, 2017. URL: https://codemeta.github.io/, doi:doi:10.21105/ joss.00454

    work page 2017

  4. [4]

    Boisvert

    Ronald F . Boisvert. Incentivizing reproducibility. Communications of the ACM , 59(10):5, September 2016. URL: https://www.acm.org/publications/policies/ artifact-review-badging, doi:10.1145/2994031

    work page doi:10.1145/2994031 2016

  5. [5]

    Understanding the factors that impact the popularity of GitHub repositories

    Hudson Borges, Andre Hora, and Marco Tulio Valente. Understanding the factors that impact the popularity of GitHub repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME) , pages 334–344, October 2016. doi:10. 1109/ICSME.2016.31

    work page 2016

  6. [6]

    Matrix computations and optimization in Apache Spark

    Reza Bosagh Zadeh, Xiangrui Meng, Alexander Ulanov, Burak Y avuz, Li Pu, Shivaram Venkataraman, Evan Sparks, Aaron Staple, and Matei Zaharia. Matrix computations and optimization in Apache Spark. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD ’16, pages 31–38, 2016. doi:10.1145/2939672.2939675

    arXiv 2016

  7. [7]

    Childers and Panos K

    Bruce R. Childers and Panos K. Chrysanthis. Artifact evaluation: Is it a real incentive? In IEEE 13th International Conference on e-Science , pages 488–489, October 2017. doi: 10.1109/eScience.2017.79

    work page doi:10.1109/escience.2017.79 2017

  8. [8]

    Proebsting

    Christian Collberg and Todd A. Proebsting. Repeatability in computer systems research. Communications of the ACM, 59(3):62–69, February 2016. doi:10.1145/2812803

    doi:10.1145/2812803 2016

Show all 42 references
  1. [9]

    Empirical study of usage and performance of Java collections

    Diego Costa, Artur Andrzejak, Janos Seboek, and David Lo. Empirical study of usage and performance of Java collections. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering, ICPE ’17, pages 389–400, 2017.doi:10.1145/ 3030207.3030221

    2017

  2. [10]

    Software Heritage: Why and How to Preserve Software Source Code

    Roberto Di Cosmo and Stefano Zacchiroli. Software Heritage: Why and How to Preserve Software Source Code. In iPRES 2017 - 14th International Conference on Digital Preser- vation, September 2017. URL: https://hal.archives-ouvertes.fr/hal-01590958

    2017

  3. [11]

    Citation File Format (CFF) - Specifications, August 2018

    Stephan Druskat, Neil Chue Hong, Robert Haines, and James Baker. Citation File Format (CFF) - Specifications, August 2018. URL: https://citation-file-format.github.io, doi:10.5281/zenodo.1003149. 18

    2018 doi

  4. [12]

    Open science: One term, five schools of thought

    Benedikt Fecher and Sascha Friesike. Open science: One term, five schools of thought. In Opening Science: The Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing , chapter 2, pages 17–47. Springer International Publishing, Cham, 2014....

    2014 doi

  5. [13]

    SIGIR Initiative to Implement ACM Artifact Review and Badging

    Nicola Ferro and Diane Kelly. SIGIR Initiative to Implement ACM Artifact Review and Badging. SIGIR Forum, 52(1):4–10, June 2018. doi:10.1145/3274784.3274786

    2018

  6. [14]

    Executable paper grand challenge workshop.Procedia Computer Science, 4:577–578, 2011

    Ann Gabriel and Rebecca Capone. Executable paper grand challenge workshop.Procedia Computer Science, 4:577–578, 2011. doi:10.1016/j.procs.2011.04.060

    2011 doi

  7. [15]

    Qresp, a tool for curating, discovering and exploring reproducible scientific papers

    Marco Govoni et al. Qresp, a tool for curating, discovering and exploring reproducible scientific papers. Scientific Data, 6, 2019. doi:10.1038/sdata.2019.2

    2019 doi

  8. [16]

    Streamlining the inclusion of computer experiments in a research paper

    Sylvain Halle, Raphael Khoury, and Mewena Awesso. Streamlining the inclusion of computer experiments in a research paper. Computer, 51(11):78–89, November 2018. doi:10.1109/MC.2018.2876075

    2018

  9. [17]

    Ahmed E. Hassan. The road ahead for mining software repositories. In 2008 Frontiers of Software Maintenance, pages 48–57, September 2008. URL: http://www.msrconf.org/, doi:10.1109/FOSM.2008.4659248

    2008

  10. [18]

    The role of standards for interoperating information systems

    Wilhelm Hasselbring. The role of standards for interoperating information systems. In In- formation Technology Standards and Standardization: A Global Perspective, pages 116–

  11. [19]

    Software architecture: Past, present, future

    Wilhelm Hasselbring. Software architecture: Past, present, future. In The Essence of Software Engineering, pages 169–184. Springer International Publishing, Cham, 2018. doi:10.1007/978-3-319-73897-0_10

    2018 doi

  12. [20]

    Arvid Ågren, Spencer C.H

    Josh Hough, J. Arvid Ågren, Spencer C.H. Barrett, and Stephen I. Wright. Chromosomal Distribution of Cytonuclear Genes in a Dioecious Plant with Sex Chromosomes. Genome Biology and Evolution, 6(9):2439–2443, September 2014. doi:10.1093/gbe/evu197

    2014 doi

  13. [21]

    Caffe: Convolutional architecture for fast feature embedding

    Y angqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Gir- shick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM International Conference on Multi- media, MM ’14, pa...

    2014

  14. [22]

    OceanTEA: exploring ocean-derived climate data using microservices

    Arne Johanson, Sascha Flögel, Christian Dullo, and Wilhelm Hasselbring. OceanTEA: exploring ocean-derived climate data using microservices. In Proceedings of the Sixth 19 International Workshop on Climate Informatics (CI 2016), pages 25–28, September 2016. doi:10.5065/D6K072N6

    2016 doi

  15. [23]

    Effectiveness and efficiency of a domain- specific language for high-performance marine ecosystem simulation: a controlled ex- periment

    Arne Johanson and Wilhelm Hasselbring. Effectiveness and efficiency of a domain- specific language for high-performance marine ecosystem simulation: a controlled ex- periment. Empirical Software Engineering, 22(4):2206–2236, August 2017. URL: http: //rdcu.be/urXK, doi:10.1007/s1...

    2017 doi

  16. [24]

    Software engineering for computational sci- ence: Past, present, future

    Arne Johanson and Wilhelm Hasselbring. Software engineering for computational sci- ence: Past, present, future. Computing in Science & Engineering , 20(2):90–109, March

  17. [25]

    Katz and Neil P

    Daniel S. Katz and Neil P . Chue Hong. FAIR is not Fair Enough, Particularly for Software Citation, Availability, or Quality.AGU Fall Meeting Abstracts, December 2018. URL: http: //adsabs.harvard.edu/abs/2018AGUFMIN41A..02K

    2018

  18. [26]

    Katz and Neil P

    Daniel S. Katz and Neil P . Chue Hong. Software citation in theory and practice. In Math- ematical Software – ICMS 2018 , pages 289–296, Cham, 2018. Springer International Publishing. doi:10.1007/978-3-319-96418-8_34

    2018 doi

  19. [27]

    Standing on Shoulders or Feet? The Usage of the MSR Data Papers

    Zoe Kotti and Diomidis Spinellis. Standing on Shoulders or Feet? The Usage of the MSR Data Papers. In Proceedings of the 16th Conference on Mining Software Repositories (MSR 2019). IEEE, 2019. (in press)

    2019

  20. [28]

    The real software crisis: Repeatability as a core value

    Shriram Krishnamurthi and Jan Vitek. The real software crisis: Repeatability as a core value. Communications of the ACM, 58(3):34–36, March 2015. doi:10.1145/2658987

    2015 doi

  21. [29]

    Constructing a bidirectional transformation between BPMN and BPEL with a functional logic programming language

    Steffen Mazanek and Michael Hanus. Constructing a bidirectional transformation between BPMN and BPEL with a functional logic programming language. Journal of Visual Lan- guages & Computing, 22(1):66–89, 2011. doi:10.1016/j.jvlc.2010.11.005

    2011 doi

  22. [30]

    The open science initiative of the Empirical Software Engineering journal

    Daniel Méndez Fernández, Martin Monperrus, Robert Feldt, and Thomas Zimmermann. The open science initiative of the Empirical Software Engineering journal. Empiri- cal Software Engineering , 24(3):1057–1060, June 2019. URL: https://github.com/ emsejournal/openscience/, doi:10.1...

    2019 doi

  23. [31]

    Nosek et al

    Brian A. Nosek et al. Promoting an open research culture. Science, 348(6242):1422– 1425, 2015. doi:10.1126/science.aab2374

    2015 doi

  24. [32]

    A comparison of interactive shad- ows and multi-view layouts for mouse-based 3D modelling

    Siobhan O’Donovan, James Gain, and Brian DeRenzi. A comparison of interactive shad- ows and multi-view layouts for mouse-based 3D modelling. In Proceedings of the Annual 20 Conference of the South African Institute of Computer Scientists and Information Technol- ogists, SAICSI...

    2016

  25. [33]

    Roger D. Peng. Reproducible research in computational science. Science, 334(6060):1226–1227, 2011. doi:10.1126/science.1213847

    2011 doi

  26. [34]

    Research and education in computational science and engineering

    Ulrich Rüde, Karen Willcox, Lois Curfman McInnes, and Hans De Sterck. Research and education in computational science and engineering. SIAM Review, 60(3):707–754, 2018. doi:10.1137/16M1096840

    2018 doi

  27. [35]

    Russell, Rachel L

    Pamela H. Russell, Rachel L. Johnson, Shreyas Ananthan, Benjamin Harnke, and Nic- hole E. Carlson. A large-scale analysis of bioinformatics code on GitHub. PLOS ONE, 13(10):1–19, October 2018. doi:10.1371/journal.pone.0205898

    2018 doi

  28. [36]

    Smith, Daniel S

    Arfon M. Smith, Daniel S. Katz, and Kyle E. Niemeyer. Software citation princi- ples. PeerJ Computer Science, 2, September 2016. URL: https://www.force11.org/ software-citation-principles, doi:10.7717/peerj-cs.86

    2016 doi

  29. [37]

    Jurriaan H. Spaaks, Jason Maassen, Tom Klaver, Stefan Verhoeven, Pushpanjali Pawar, Willem van Hage, Lars Ridder, Lode Kulik, Tom Bakker, Vincent van Hees, Laurens Bo- gaardt, Adrianne Mendrik, Bram van Es, Jisk Attema, Elena Ranguelova, and Rob van Nieuwpoort. Research Softwa...

    2019 doi

  30. [38]

    Data observatories: decentralised data and interdisciplinary re- search

    Thanassis Tiropanis. Data observatories: decentralised data and interdisciplinary re- search. In Internet y Ciencia: Análisis Desde la Complejidad Estructural y Dinámica , Ferrol, Spain, March 2019. URL: https://eprints.soton.ac.uk/428200/

    2019

  31. [39]

    The Web Observatory: A Middle Layer for Broad Data

    Thanassis Tiropanis, Wendy Hall, James Hendler, and Christian de Larrinaga. The Web Observatory: A Middle Layer for Broad Data. Big Data, 2(3):129–133, September 2014. doi:10.1089/big.2014.0035

    2014

  32. [40]

    Wilkinson et al

    Mark D. Wilkinson et al. The FAIR guiding principles for scientific data management and stewardship. Scientific Data , 3, March 2016. URL: https://www.go-fair.org/ fair-principles/, doi:10.1038/sdata.2016.18. 21 Wilhelm Hasselbring (hasselbring@email.uni-kiel.de) is a full profe...

    2016 doi

  33. [130]

    doi:10.4018/978-1-878289-70-4.ch008

    IGI Global, Hershey, PA, 2000. doi:10.4018/978-1-878289-70-4.ch008

    2000 doi

  34. [2018]

    doi:10.1109/MCSE.2018.021651343

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.