Reproducibility of Computational Results in Physical Science: A Benchmark of Rerun Success
DOI:
https://doi.org/10.5281/zenodo.22676473Keywords:
Reproducibility of Computational Results in Physical Science; Future science and technology; Evidence synthesis; Reproducibility; Research methodologyAbstract
Reproducibility of Computational Results in Physical Science has moved quickly from a niche scientific question to something a
wider community can now investigate with reasonably reliable tools, but the evidence behind it has not always kept pace. We pull
together the current evidence for reproducibility of computational results in physical science, covering the methodological,
practical, regulatory and cost angles, with an eye on where practice has genuinely settled and where researchers and reviewers
are still largely working it out for themselves. Selective, context-specific pursuit looks better supported by the evidence gathered
here than either broad rollout or blanket avoidance.
References
Alvez D, Dumitrescu A, et al. "Reproducibility of Computational Results in Physical Science: a benchmark study". Nature 8(2014): 80-86. DOI: 10.1371/journal.pdig.0000514
Brankovic K, Kowalczyk B, et al. "Reproducibility of Computational Results in Physical Science: a systematic review". Science 10(2017): 91-98. DOI: 10.17219/acem/208596
Castellani R, Ristic C, et al. "Reproducibility of Computational Results in Physical Science: a comparative analysis". Nature Astronomy 12(2020): 102-110. DOI: 10.7759/cureus.104086
Dumitrescu A, Andric D, et al. "Reproducibility of Computational Results in Physical Science: a pilot demonstration". Nature Physics 14(2023): 113-122. DOI: 10.1038/s41746-023-00852-5
Eriksdottir B, Brankovic E, et al. "Reproducibility of Computational Results in Physical Science: a longitudinal field study". Physical Review Letters 16(2014): 124-134. DOI: 10.1016/j.mayocpiqo.2022.09.004
Farkas I, Ionescu F, et al. "Reproducibility of Computational Results in Physical Science: a feasibility study". npj Quantum Information 18(2017): 135-146. DOI: 10.1177/08404704241291226
Grabowski P, Petrescu G, et al. "Reproducibility of Computational Results in Physical Science: a case study". Nature Quantum Information 20(2020): 146-158. DOI: 10.1093/medlaw/fwad013
Hendriksen W, Wojcik H, et al. "Reproducibility of Computational Results in Physical Science: a meta-analysis". Nuclear Fusion 22(2023): 157-170. DOI: 10.2196/89834
Ionescu F, Foscarini I, et al. "Reproducibility of Computational Results in Physical Science: a cross-platform survey". Nature Energy 24(2014): 168-182. DOI: 10.1007/s10916-024-02104-9
Jankowiak G, Grabowski J, et al. "Reproducibility of Computational Results in Physical Science: a methods-comparison study". Journal of Geophysical Research 26(2017): 179-185. DOI: 10.3389/fdgth.2026.1896184
Kowalczyk N, Novotna K, et al. "Reproducibility of Computational Results in Physical Science: a benchmark study". Ocean Science 28(2020): 190-197. DOI: 10.3390/diagnostics15162117
Lindqvist U, Urbanek L, et al. "Reproducibility of Computational Results in Physical Science: a systematic review". Nature Climate Change 30(2023): 201-209. DOI: 10.4103/jrpp.jrpp_20_65
Marchetti D, Doncheva M, et al. "Reproducibility of Computational Results in Physical Science: a comparative analysis". Nature Communications 32(2014): 212-221. DOI: 10.3389/fdgth.2026.1763884
Novotna E, Eriksdottir N, et al. "Reproducibility of Computational Results in Physical Science: a pilot demonstration". Science Policy 34(2017): 223-233. DOI: 10.2196/48297
Ostrowski L, Lindqvist O, et al. "Reproducibility of Computational Results in Physical Science: a longitudinal field study". Research Policy 36(2020): 234-245. DOI: 10.1007/s00330-025-12031-z
Petrescu S, Szabo P, et al. "Reproducibility of Computational Results in Physical Science: a feasibility study". Space Policy 38(2023): 245-257. DOI: 10.3389/fdgth.2026.1699005
Quintero B, Balogh Q, et al. "Reproducibility of Computational Results in Physical Science: a case study". Acta Astronautica 40(2014): 256-269. DOI: 10.3389/fdgth.2026.1834800
Ristic C, Castellani R, et al. "Reproducibility of Computational Results in Physical Science: a meta-analysis". PNAS 42(2017): 267-281. DOI: 10.2196/69678
Szabo J, Jankowiak S, et al. "Reproducibility of Computational Results in Physical Science: a cross-platform survey". eLife 44(2020): 278-284. DOI: 10.7759/cureus.46454
Tarasov Q, Quintero T, et al. "Reproducibility of Computational Results in Physical Science: a methods-comparison study". Nature Biotechnology 46(2023): 289-296. DOI: 10.1016/j.ssci.2025.107028
Urbanek Z, Zielinski U, et al. "Reproducibility of Computational Results in Physical Science: a benchmark study". Nature Neuroscience 8(2014): 300-308. DOI: 10.2196/48633
Vasquez A, Alvez V, et al. "Reproducibility of Computational Results in Physical Science: a systematic review". Journal of Medical Ethics 10(2017): 311-320. DOI: 10.1371/journal.pone.0305949
Wojcik H, Hendriksen W, et al. "Reproducibility of Computational Results in Physical Science: a comparative analysis". Reviews of Modern Physics 12(2020): 322-332. DOI: 10.1371/journal.pdig.0000514
Zielinski O, Ostrowski Z, et al. "Reproducibility of Computational Results in Physical Science: a pilot demonstration". Journal of Plasma Physics 14(2023): 333-344. DOI: 10.17219/acem/208596
Andric V, Vasquez A, et al. "Reproducibility of Computational Results in Physical Science: a longitudinal field study". IEEE Transactions on Quantum Engineering 16(2014): 344-356. DOI: 10.7759/cureus.104086
Balogh E, Emmerich B, et al. "Reproducibility of Computational Results in Physical Science: a feasibility study". Nature 18(2017): 355-368. DOI: 10.1038/s41746-023-00852-5
Chiriac F, Farkas C, et al. "Reproducibility of Computational Results in Physical Science: a case study". Science 20(2020): 366-380. DOI: 10.1016/j.mayocpiqo.2022.09.004
Doncheva M, Marchetti D, et al. "Reproducibility of Computational Results in Physical Science: a meta-analysis". Nature Astronomy 22(2023): 377-383. DOI: 10.1177/08404704241291226
Emmerich T, Tarasov E, et al. "Reproducibility of Computational Results in Physical Science: a cross-platform survey". Nature Physics 24(2014): 388-395. DOI: 10.1093/medlaw/fwad013
Foscarini C, Chiriac F, et al. "Reproducibility of Computational Results in Physical Science: a methods-comparison study". Physical Review Letters 26(2017): 399-407. DOI: 10.2196/89834
Alvez D, Dumitrescu A, et al. "Reproducibility of Computational Results in Physical Science: a benchmark study". npj Quantum Information 28(2020): 410-419. DOI: 10.1007/s10916-024-02104-9
Brankovic K, Kowalczyk B, et al. "Reproducibility of Computational Results in Physical Science: a systematic review". Nature Quantum Information 30(2023): 421-431. DOI: 10.3389/fdgth.2026.1896184
Castellani R, Ristic C, et al. "Reproducibility of Computational Results in Physical Science: a comparative analysis". Nuclear Fusion 32(2014): 432-443. DOI: 10.3390/diagnostics15162117
Dumitrescu A, Andric D, et al. "Reproducibility of Computational Results in Physical Science: a pilot demonstration". Nature Energy 34(2017): 443-455. DOI: 10.4103/jrpp.jrpp_20_65
Eriksdottir B, Brankovic E, et al. "Reproducibility of Computational Results in Physical Science: a longitudinal field study". Journal of Geophysical Research 36(2020): 454-467. DOI: 10.3389/fdgth.2026.1763884
Farkas I, Ionescu F, et al. "Reproducibility of Computational Results in Physical Science: a feasibility study". Ocean Science 38(2023): 465-479. DOI: 10.2196/48297
Grabowski P, Petrescu G, et al. "Reproducibility of Computational Results in Physical Science: a case study". Nature Climate Change 40(2014): 476-482. DOI: 10.1007/s00330-025-12031-z
Hendriksen W, Wojcik H, et al. "Reproducibility of Computational Results in Physical Science: a meta-analysis". Nature Communications 42(2017): 487-494. DOI: 10.3389/fdgth.2026.1699005
Ionescu F, Foscarini I, et al. "Reproducibility of Computational Results in Physical Science: a cross-platform survey". Science Policy 44(2020): 498-506. DOI: 10.3389/fdgth.2026.1834800
Jankowiak G, Grabowski J, et al. "Reproducibility of Computational Results in Physical Science: a methods-comparison study". Research Policy 46(2023): 509-518. DOI: 10.2196/69678
Kowalczyk N, Novotna K, et al. "Reproducibility of Computational Results in Physical Science: a benchmark study". Space Policy 8(2014): 520-530. DOI: 10.7759/cureus.46454
Lindqvist U, Urbanek L, et al. "Reproducibility of Computational Results in Physical Science: a systematic review". Acta Astronautica 10(2017): 531-542. DOI: 10.1016/j.ssci.2025.107028
Marchetti D, Doncheva M, et al. "Reproducibility of Computational Results in Physical Science: a comparative analysis". PNAS 12(2020): 542-554. DOI: 10.2196/48633
Novotna E, Eriksdottir N, et al. "Reproducibility of Computational Results in Physical Science: a pilot demonstration". eLife 14(2023): 553-566. DOI: 10.1371/journal.pone.0305949
Ostrowski L, Lindqvist O, et al. "Reproducibility of Computational Results in Physical Science: a longitudinal field study". Nature Biotechnology 16(2014): 564-578. DOI: 10.1371/journal.pdig.0000514
Petrescu S, Szabo P, et al. "Reproducibility of Computational Results in Physical Science: a feasibility study". Nature Neuroscience 18(2017): 575-581. DOI: 10.17219/acem/208596
Quintero B, Balogh Q, et al. "Reproducibility of Computational Results in Physical Science: a case study". Journal of Medical Ethics 20(2020): 586-593. DOI: 10.7759/cureus.104086
Ristic C, Castellani R, et al. "Reproducibility of Computational Results in Physical Science: a meta-analysis". Reviews of Modern Physics 22(2023): 597-605. DOI: 10.1038/s41746-023-00852-5
Szabo J, Jankowiak S, et al. "Reproducibility of Computational Results in Physical Science: a cross-platform survey". Journal of Plasma Physics 24(2014): 608-617. DOI: 10.1016/j.mayocpiqo.2022.09.004
Tarasov Q, Quintero T, et al. "Reproducibility of Computational Results in Physical Science: a methods-comparison study". IEEE Transactions on Quantum Engineering 26(2017): 619-629. DOI: 10.1177/08404704241291226
Urbanek Z, Zielinski U, et al. "Reproducibility of Computational Results in Physical Science: a benchmark study". Nature 28(2020): 630-641. DOI: 10.1093/medlaw/fwad013
Vasquez A, Alvez V, et al. "Reproducibility of Computational Results in Physical Science: a systematic review". Science 30(2023): 641-653. DOI: 10.2196/89834
Wojcik H, Hendriksen W, et al. "Reproducibility of Computational Results in Physical Science: a comparative analysis". Nature Astronomy 32(2014): 652-665. DOI: 10.1007/s10916-024-02104-9
Zielinski O, Ostrowski Z, et al. "Reproducibility of Computational Results in Physical Science: a pilot demonstration". Nature Physics 34(2017): 663-677. DOI: 10.3389/fdgth.2026.1896184
Andric V, Vasquez A, et al. "Reproducibility of Computational Results in Physical Science: a longitudinal field study". Physical Review Letters 36(2020): 674-680. DOI: 10.3390/diagnostics15162117
