Procurement of Machine Learning Systems by Public Bodies: A Review of Specification and Acceptance Practice

Authors

  • Külli Sarapuu Centre for Technology Policy, Laanemaa Institute of Applied Artificial Intelligence, Haapsalu, Estonia Author
  • Fionnbarr Lenihan Centre for Technology Law, Beara Institute of Applied Artificial Intelligence, Castletownbere, Ireland Author
  • Jokin Requejo-Sal Department of Machine Learning Engineering, Tierra de Campos Institute of Data Science, Palencia, Spain Author

DOI:

https://doi.org/10.5281/zenodo.22660914

Keywords:

Procurement of Machine Learning Systems by Public Bodies; Applied machine learning; Evidence synthesis; Production deployment; MLOps

Abstract

Procurement of Machine Learning Systems by Public Bodies is an area of active development in applied machine learning, with
evidence that has accumulated unevenly across its component questions. We review the current evidence base for procurement of
machine learning systems by public bodies, spanning methodological, implementation, governance, and cost dimensions, focusing
on where practice has converged and where it continues to vary, and set out the practical considerations relevant to a team
considering its adoption. We conclude that selective, context-specific use is best supported by the evidence currently available,
rather than uniform adoption or avoidance.

References

Alvez D, Dumitrescu A, et al. "Procurement of Machine Learning Systems by Public Bodies: a benchmark study". J Mach Learn Res 8(2014): 80-86. DOI: 10.1007/s00330-025-12031-z

Brankovic K, Kowalczyk B, et al. "Procurement of Machine Learning Systems by Public Bodies: a systematic evaluation". IEEE Trans Pattern Anal Mach Intell 10(2017): 91-98. DOI: 10.3389/fdgth.2026.1699005

Castellani R, Ristic C, et al. "Procurement of Machine Learning Systems by Public Bodies: a empirical analysis". Proc NeurIPS 12(2020): 102-110. DOI: 10.3389/fdgth.2026.1834800

Dumitrescu A, Andric D, et al. "Procurement of Machine Learning Systems by Public Bodies: a ablation study". Proc ICML 14(2023): 113-122. DOI: 10.2196/69678

Eriksdottir B, Brankovic E, et al. "Procurement of Machine Learning Systems by Public Bodies: a reproducibility study". Proc ICLR 16(2014): 124-134. DOI: 10.7759/cureus.46454

Farkas I, Ionescu F, et al. "Procurement of Machine Learning Systems by Public Bodies: a large-scale evaluation". ACM Trans Intell Syst 18(2017): 135-146. DOI: 10.1016/j.ssci.2025.107028

Grabowski P, Petrescu G, et al. "Procurement of Machine Learning Systems by Public Bodies: a case study". Data Min Knowl Discov 20(2020): 146-158. DOI: 10.2196/48633

Hendriksen W, Wojcik H, et al. "Procurement of Machine Learning Systems by Public Bodies: a survey". Artif Intell 22(2023): 157-170. DOI: 10.1371/journal.pone.0305949

Ionescu F, Foscarini I, et al. "Procurement of Machine Learning Systems by Public Bodies: a comparative analysis". Knowl Based Syst 24(2014): 168-182. DOI: 10.1371/journal.pdig.0000514

Jankowiak G, Grabowski J, et al. "Procurement of Machine Learning Systems by Public Bodies: a field deployment report". IEEE Trans Neural Netw Learn Syst 26(2017): 179-185. DOI: 10.17219/acem/208596

Kowalczyk N, Novotna K, et al. "Procurement of Machine Learning Systems by Public Bodies: a benchmark study". Proc KDD 28(2020): 190-197. DOI: 10.7759/cureus.104086

Lindqvist U, Urbanek L, et al. "Procurement of Machine Learning Systems by Public Bodies: a systematic evaluation". Proc AAAI 30(2023): 201-209. DOI: 10.1038/s41746-023-00852-5

Marchetti D, Doncheva M, et al. "Procurement of Machine Learning Systems by Public Bodies: a empirical analysis". IEEE Trans Knowl Data Eng 32(2014): 212-221. DOI: 10.1016/j.mayocpiqo.2022.09.004

Novotna E, Eriksdottir N, et al. "Procurement of Machine Learning Systems by Public Bodies: a ablation study". J Artif Intell Res 34(2017): 223-233. DOI: 10.1177/08404704241291226

Ostrowski L, Lindqvist O, et al. "Procurement of Machine Learning Systems by Public Bodies: a reproducibility study". ACM Comput Surv 36(2020): 234-245. DOI: 10.1093/medlaw/fwad013

Petrescu S, Szabo P, et al. "Procurement of Machine Learning Systems by Public Bodies: a large-scale evaluation". Neural Comput Appl 38(2023): 245-257. DOI: 10.2196/89834

Quintero B, Balogh Q, et al. "Procurement of Machine Learning Systems by Public Bodies: a case study". Expert Syst Appl 40(2014): 256-269. DOI: 10.1007/s10916-024-02104-9

Ristic C, Castellani R, et al. "Procurement of Machine Learning Systems by Public Bodies: a survey". Inf Sci 42(2017): 267-281. DOI: 10.3389/fdgth.2026.1896184

Szabo J, Jankowiak S, et al. "Procurement of Machine Learning Systems by Public Bodies: a comparative analysis". Pattern Recognit 44(2020): 278-284. DOI: 10.3390/diagnostics15162117

Tarasov Q, Quintero T, et al. "Procurement of Machine Learning Systems by Public Bodies: a field deployment report". IEEE Access 46(2023): 289-296. DOI: 10.4103/jrpp.jrpp_20_65

Urbanek Z, Zielinski U, et al. "Procurement of Machine Learning Systems by Public Bodies: a benchmark study". ACM Trans Softw Eng Methodol 8(2014): 300-308. DOI: 10.3389/fdgth.2026.1763884

Vasquez A, Alvez V, et al. "Procurement of Machine Learning Systems by Public Bodies: a systematic evaluation". Proc MLSys 10(2017): 311-320. DOI: 10.2196/48297

Wojcik H, Hendriksen W, et al. "Procurement of Machine Learning Systems by Public Bodies: a empirical analysis". Proc FAccT 12(2020): 322-332. DOI: 10.1007/s00330-025-12031-z

Zielinski O, Ostrowski Z, et al. "Procurement of Machine Learning Systems by Public Bodies: a ablation study". Nat Mach Intell 14(2023): 333-344. DOI: 10.3389/fdgth.2026.1699005

Andric V, Vasquez A, et al. "Procurement of Machine Learning Systems by Public Bodies: a reproducibility study". J Mach Learn Res 16(2014): 344-356. DOI: 10.3389/fdgth.2026.1834800

Balogh E, Emmerich B, et al. "Procurement of Machine Learning Systems by Public Bodies: a large-scale evaluation". IEEE Trans Pattern Anal Mach Intell 18(2017): 355-368. DOI: 10.2196/69678

Chiriac F, Farkas C, et al. "Procurement of Machine Learning Systems by Public Bodies: a case study". Proc NeurIPS 20(2020): 366-380. DOI: 10.7759/cureus.46454

Doncheva M, Marchetti D, et al. "Procurement of Machine Learning Systems by Public Bodies: a survey". Proc ICML 22(2023): 377-383. DOI: 10.1016/j.ssci.2025.107028

Emmerich T, Tarasov E, et al. "Procurement of Machine Learning Systems by Public Bodies: a comparative analysis". Proc ICLR 24(2014): 388-395. DOI: 10.2196/48633

Foscarini C, Chiriac F, et al. "Procurement of Machine Learning Systems by Public Bodies: a field deployment report". ACM Trans Intell Syst 26(2017): 399-407. DOI: 10.1371/journal.pone.0305949

Alvez D, Dumitrescu A, et al. "Procurement of Machine Learning Systems by Public Bodies: a benchmark study". Data Min Knowl Discov 28(2020): 410-419. DOI: 10.1371/journal.pdig.0000514

Brankovic K, Kowalczyk B, et al. "Procurement of Machine Learning Systems by Public Bodies: a systematic evaluation". Artif Intell 30(2023): 421-431. DOI: 10.17219/acem/208596

Castellani R, Ristic C, et al. "Procurement of Machine Learning Systems by Public Bodies: a empirical analysis". Knowl Based Syst 32(2014): 432-443. DOI: 10.7759/cureus.104086

Dumitrescu A, Andric D, et al. "Procurement of Machine Learning Systems by Public Bodies: a ablation study". IEEE Trans Neural Netw Learn Syst 34(2017): 443-455. DOI: 10.1038/s41746-023-00852-5

Eriksdottir B, Brankovic E, et al. "Procurement of Machine Learning Systems by Public Bodies: a reproducibility study". Proc KDD 36(2020): 454-467. DOI: 10.1016/j.mayocpiqo.2022.09.004

Farkas I, Ionescu F, et al. "Procurement of Machine Learning Systems by Public Bodies: a large-scale evaluation". Proc AAAI 38(2023): 465-479. DOI: 10.1177/08404704241291226

Grabowski P, Petrescu G, et al. "Procurement of Machine Learning Systems by Public Bodies: a case study". IEEE Trans Knowl Data Eng 40(2014): 476-482. DOI: 10.1093/medlaw/fwad013

Hendriksen W, Wojcik H, et al. "Procurement of Machine Learning Systems by Public Bodies: a survey". J Artif Intell Res 42(2017): 487-494. DOI: 10.2196/89834

Ionescu F, Foscarini I, et al. "Procurement of Machine Learning Systems by Public Bodies: a comparative analysis". ACM Comput Surv 44(2020): 498-506. DOI: 10.1007/s10916-024-02104-9

Jankowiak G, Grabowski J, et al. "Procurement of Machine Learning Systems by Public Bodies: a field deployment report". Neural Comput Appl 46(2023): 509-518. DOI: 10.3389/fdgth.2026.1896184

Kowalczyk N, Novotna K, et al. "Procurement of Machine Learning Systems by Public Bodies: a benchmark study". Expert Syst Appl 8(2014): 520-530. DOI: 10.3390/diagnostics15162117

Lindqvist U, Urbanek L, et al. "Procurement of Machine Learning Systems by Public Bodies: a systematic evaluation". Inf Sci 10(2017): 531-542. DOI: 10.4103/jrpp.jrpp_20_65

Marchetti D, Doncheva M, et al. "Procurement of Machine Learning Systems by Public Bodies: a empirical analysis". Pattern Recognit 12(2020): 542-554. DOI: 10.3389/fdgth.2026.1763884

Novotna E, Eriksdottir N, et al. "Procurement of Machine Learning Systems by Public Bodies: a ablation study". IEEE Access 14(2023): 553-566. DOI: 10.2196/48297

Ostrowski L, Lindqvist O, et al. "Procurement of Machine Learning Systems by Public Bodies: a reproducibility study". ACM Trans Softw Eng Methodol 16(2014): 564-578. DOI: 10.1007/s00330-025-12031-z

Petrescu S, Szabo P, et al. "Procurement of Machine Learning Systems by Public Bodies: a large-scale evaluation". Proc MLSys 18(2017): 575-581. DOI: 10.3389/fdgth.2026.1699005

Quintero B, Balogh Q, et al. "Procurement of Machine Learning Systems by Public Bodies: a case study". Proc FAccT 20(2020): 586-593. DOI: 10.3389/fdgth.2026.1834800

Ristic C, Castellani R, et al. "Procurement of Machine Learning Systems by Public Bodies: a survey". Nat Mach Intell 22(2023): 597-605. DOI: 10.2196/69678

Szabo J, Jankowiak S, et al. "Procurement of Machine Learning Systems by Public Bodies: a comparative analysis". J Mach Learn Res 24(2014): 608-617. DOI: 10.7759/cureus.46454

Tarasov Q, Quintero T, et al. "Procurement of Machine Learning Systems by Public Bodies: a field deployment report". IEEE Trans Pattern Anal Mach Intell 26(2017): 619-629. DOI: 10.1016/j.ssci.2025.107028

Urbanek Z, Zielinski U, et al. "Procurement of Machine Learning Systems by Public Bodies: a benchmark study". Proc NeurIPS 28(2020): 630-641. DOI: 10.2196/48633

Vasquez A, Alvez V, et al. "Procurement of Machine Learning Systems by Public Bodies: a systematic evaluation". Proc ICML 30(2023): 641-653. DOI: 10.1371/journal.pone.0305949

Wojcik H, Hendriksen W, et al. "Procurement of Machine Learning Systems by Public Bodies: a empirical analysis". Proc ICLR 32(2014): 652-665. DOI: 10.1371/journal.pdig.0000514

Zielinski O, Ostrowski Z, et al. "Procurement of Machine Learning Systems by Public Bodies: a ablation study". ACM Trans Intell Syst 34(2017): 663-677. DOI: 10.17219/acem/208596

Andric V, Vasquez A, et al. "Procurement of Machine Learning Systems by Public Bodies: a reproducibility study". Data Min Knowl Discov 36(2020): 674-680. DOI: 10.7759/cureus.104086

Downloads

Published

2025-12-30

Issue

Vol. 4 No. 2 (2025): vol 4 issue 2 2025

Section

Articles

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.