Code-sharing policies are associated with increased reproducibility potential of ecological findings

Sánchez-Tójar, Alfredo; Bezine, Aya; Purgar, Marija; Čulina, Antica (2025) Code-sharing policies are associated with increased reproducibility potential of ecological findings. Peer Community Journal, 5 . ISSN 2804-3871

PDF - Published Version - article
Available under License Creative Commons Attribution.
Download (2MB)

Official URL: http://doi.org/10.24072/pcjournal.541

Abstract

Software code (e.g., analytical code) is increasingly recognised as an important research output because it improves transparency, collaboration, and research credibility. Many scientific journals have introduced code-sharing policies; however, surveys have shown alarmingly low compliance with these policies. In this study, we expanded on a recent survey of ecological journals with code-sharing policies by investigating sharing practices in a comparable set of ecological journals without code-sharing policies. Our aims were to estimate code- and data-sharing rates, assess key reproducibility-boosting features, such as the reporting of software versioning, and compare reproducibility potential between journals with and without a code-sharing policy. We reviewed a random sample of 314 articles published between 2015 and 2019 in 12 ecological journals without a code-sharing policy. Only 15 articles (4.8%) provided analytical code, with the percentage nearly tripling over time (2015-2016:2.5%, 2018-2019:7.0%). Data-sharing was higher than code-sharing (2015-2016:31.0%, 2018-2019:43.3%), yet only eight articles (2.5%) shared both code and data. Compared to a comparative sample of 346 articles from 14 ecological journals with a code-sharing policy, journals without a code-sharing policy showed 5.6 times lower code-sharing, 2.1 times lower data-sharing, and 8.1 times lower reproducibility potential. Despite these differences, the key reproducibility-boosting features of the two journal types were similar. Approximately 90% of all articles reported the analytical software used; however, for journals with and without a code-sharing policy, the software version was often missing (49.8% and 36.1% of articles, respectively), and exclusively proprietary (i.e., non-free) software was used in 16.7% and 23.5% of articles, respectively. Our study suggests that journals with a code-sharing policy have greater reproducibility potential than those without. Code-sharing policies are likely to be a necessary but insufficient step towards increasing reproducibility. Journals should prioritize adopting explicit, easy-to-find, and strict code-sharing policies to facilitate researchers' compliance and should implement mechanisms such as checklists to ensure adherence.

Item Type:

Article

Uncontrolled Keywords:

replicability; reliability; robustness; generalizability; verification; replication; FAIR; checklist

Subjects:

NATURAL SCIENCES > Biology > Ecology
NATURAL SCIENCES > Interdisciplinary Natural Sciences

Divisions:

Division for Marine and Enviromental Research

Projects: