Human-AI Teaming in Radiology Triage: Reliance, Disagreement and the Second Reader Effect
DOI:
https://doi.org/10.5281/zenodo.22661323Keywords:
Human-AI Teaming in Radiology Triage; Applied machine learning; Evidence synthesis; Production deployment; MLOpsAbstract
Human-AI Teaming in Radiology Triage is an area of active development in applied machine learning, with evidence that has
accumulated unevenly across its component questions. We review the current evidence base for human-ai teaming in radiology
triage, spanning methodological, implementation, governance, and cost dimensions, focusing on where practice has converged
and where it continues to vary, and set out the practical considerations relevant to a team considering its adoption. We conclude
that selective, context-specific use is best supported by the evidence currently available, rather than uniform adoption or
avoidance.
References
Alvez D, Dumitrescu A, et al. "Human-AI Teaming in Radiology Triage: a benchmark study". J Mach Learn Res 8(2014): 80-86. DOI: 10.2196/48633
Brankovic K, Kowalczyk B, et al. "Human-AI Teaming in Radiology Triage: a systematic evaluation". IEEE Trans Pattern Anal Mach Intell 10(2017): 91-98. DOI: 10.1371/journal.pone.0305949
Castellani R, Ristic C, et al. "Human-AI Teaming in Radiology Triage: a empirical analysis". Proc NeurIPS 12(2020): 102-110. DOI: 10.1371/journal.pdig.0000514
Dumitrescu A, Andric D, et al. "Human-AI Teaming in Radiology Triage: a ablation study". Proc ICML 14(2023): 113-122. DOI: 10.17219/acem/208596
Eriksdottir B, Brankovic E, et al. "Human-AI Teaming in Radiology Triage: a reproducibility study". Proc ICLR 16(2014): 124-134. DOI: 10.7759/cureus.104086
Farkas I, Ionescu F, et al. "Human-AI Teaming in Radiology Triage: a large-scale evaluation". ACM Trans Intell Syst 18(2017): 135-146. DOI: 10.1038/s41746-023-00852-5
Grabowski P, Petrescu G, et al. "Human-AI Teaming in Radiology Triage: a case study". Data Min Knowl Discov 20(2020): 146-158. DOI: 10.1016/j.mayocpiqo.2022.09.004
Hendriksen W, Wojcik H, et al. "Human-AI Teaming in Radiology Triage: a survey". Artif Intell 22(2023): 157-170. DOI: 10.1177/08404704241291226
Ionescu F, Foscarini I, et al. "Human-AI Teaming in Radiology Triage: a comparative analysis". Knowl Based Syst 24(2014): 168-182. DOI: 10.1093/medlaw/fwad013
Jankowiak G, Grabowski J, et al. "Human-AI Teaming in Radiology Triage: a field deployment report". IEEE Trans Neural Netw Learn Syst 26(2017): 179-185. DOI: 10.2196/89834
Kowalczyk N, Novotna K, et al. "Human-AI Teaming in Radiology Triage: a benchmark study". Proc KDD 28(2020): 190-197. DOI: 10.1007/s10916-024-02104-9
Lindqvist U, Urbanek L, et al. "Human-AI Teaming in Radiology Triage: a systematic evaluation". Proc AAAI 30(2023): 201-209. DOI: 10.3389/fdgth.2026.1896184
Marchetti D, Doncheva M, et al. "Human-AI Teaming in Radiology Triage: a empirical analysis". IEEE Trans Knowl Data Eng 32(2014): 212-221. DOI: 10.3390/diagnostics15162117
Novotna E, Eriksdottir N, et al. "Human-AI Teaming in Radiology Triage: a ablation study". J Artif Intell Res 34(2017): 223-233. DOI: 10.4103/jrpp.jrpp_20_65
Ostrowski L, Lindqvist O, et al. "Human-AI Teaming in Radiology Triage: a reproducibility study". ACM Comput Surv 36(2020): 234-245. DOI: 10.3389/fdgth.2026.1763884
Petrescu S, Szabo P, et al. "Human-AI Teaming in Radiology Triage: a large-scale evaluation". Neural Comput Appl 38(2023): 245-257. DOI: 10.2196/48297
Quintero B, Balogh Q, et al. "Human-AI Teaming in Radiology Triage: a case study". Expert Syst Appl 40(2014): 256-269. DOI: 10.1007/s00330-025-12031-z
Ristic C, Castellani R, et al. "Human-AI Teaming in Radiology Triage: a survey". Inf Sci 42(2017): 267-281. DOI: 10.3389/fdgth.2026.1699005
Szabo J, Jankowiak S, et al. "Human-AI Teaming in Radiology Triage: a comparative analysis". Pattern Recognit 44(2020): 278-284. DOI: 10.3389/fdgth.2026.1834800
Tarasov Q, Quintero T, et al. "Human-AI Teaming in Radiology Triage: a field deployment report". IEEE Access 46(2023): 289-296. DOI: 10.2196/69678
Urbanek Z, Zielinski U, et al. "Human-AI Teaming in Radiology Triage: a benchmark study". ACM Trans Softw Eng Methodol 8(2014): 300-308. DOI: 10.7759/cureus.46454
Vasquez A, Alvez V, et al. "Human-AI Teaming in Radiology Triage: a systematic evaluation". Proc MLSys 10(2017): 311-320. DOI: 10.1016/j.ssci.2025.107028
Wojcik H, Hendriksen W, et al. "Human-AI Teaming in Radiology Triage: a empirical analysis". Proc FAccT 12(2020): 322-332. DOI: 10.2196/48633
Zielinski O, Ostrowski Z, et al. "Human-AI Teaming in Radiology Triage: a ablation study". Nat Mach Intell 14(2023): 333-344. DOI: 10.1371/journal.pone.0305949
Andric V, Vasquez A, et al. "Human-AI Teaming in Radiology Triage: a reproducibility study". J Mach Learn Res 16(2014): 344-356. DOI: 10.1371/journal.pdig.0000514
Balogh E, Emmerich B, et al. "Human-AI Teaming in Radiology Triage: a large-scale evaluation". IEEE Trans Pattern Anal Mach Intell 18(2017): 355-368. DOI: 10.17219/acem/208596
Chiriac F, Farkas C, et al. "Human-AI Teaming in Radiology Triage: a case study". Proc NeurIPS 20(2020): 366-380. DOI: 10.7759/cureus.104086
Doncheva M, Marchetti D, et al. "Human-AI Teaming in Radiology Triage: a survey". Proc ICML 22(2023): 377-383. DOI: 10.1038/s41746-023-00852-5
Emmerich T, Tarasov E, et al. "Human-AI Teaming in Radiology Triage: a comparative analysis". Proc ICLR 24(2014): 388-395. DOI: 10.1016/j.mayocpiqo.2022.09.004
Foscarini C, Chiriac F, et al. "Human-AI Teaming in Radiology Triage: a field deployment report". ACM Trans Intell Syst 26(2017): 399-407. DOI: 10.1177/08404704241291226
Alvez D, Dumitrescu A, et al. "Human-AI Teaming in Radiology Triage: a benchmark study". Data Min Knowl Discov 28(2020): 410-419. DOI: 10.1093/medlaw/fwad013
Brankovic K, Kowalczyk B, et al. "Human-AI Teaming in Radiology Triage: a systematic evaluation". Artif Intell 30(2023): 421-431. DOI: 10.2196/89834
Castellani R, Ristic C, et al. "Human-AI Teaming in Radiology Triage: a empirical analysis". Knowl Based Syst 32(2014): 432-443. DOI: 10.1007/s10916-024-02104-9
Dumitrescu A, Andric D, et al. "Human-AI Teaming in Radiology Triage: a ablation study". IEEE Trans Neural Netw Learn Syst 34(2017): 443-455. DOI: 10.3389/fdgth.2026.1896184
Eriksdottir B, Brankovic E, et al. "Human-AI Teaming in Radiology Triage: a reproducibility study". Proc KDD 36(2020): 454-467. DOI: 10.3390/diagnostics15162117
Farkas I, Ionescu F, et al. "Human-AI Teaming in Radiology Triage: a large-scale evaluation". Proc AAAI 38(2023): 465-479. DOI: 10.4103/jrpp.jrpp_20_65
Grabowski P, Petrescu G, et al. "Human-AI Teaming in Radiology Triage: a case study". IEEE Trans Knowl Data Eng 40(2014): 476-482. DOI: 10.3389/fdgth.2026.1763884
Hendriksen W, Wojcik H, et al. "Human-AI Teaming in Radiology Triage: a survey". J Artif Intell Res 42(2017): 487-494. DOI: 10.2196/48297
Ionescu F, Foscarini I, et al. "Human-AI Teaming in Radiology Triage: a comparative analysis". ACM Comput Surv 44(2020): 498-506. DOI: 10.1007/s00330-025-12031-z
Jankowiak G, Grabowski J, et al. "Human-AI Teaming in Radiology Triage: a field deployment report". Neural Comput Appl 46(2023): 509-518. DOI: 10.3389/fdgth.2026.1699005
Kowalczyk N, Novotna K, et al. "Human-AI Teaming in Radiology Triage: a benchmark study". Expert Syst Appl 8(2014): 520-530. DOI: 10.3389/fdgth.2026.1834800
Lindqvist U, Urbanek L, et al. "Human-AI Teaming in Radiology Triage: a systematic evaluation". Inf Sci 10(2017): 531-542. DOI: 10.2196/69678
Marchetti D, Doncheva M, et al. "Human-AI Teaming in Radiology Triage: a empirical analysis". Pattern Recognit 12(2020): 542-554. DOI: 10.7759/cureus.46454
Novotna E, Eriksdottir N, et al. "Human-AI Teaming in Radiology Triage: a ablation study". IEEE Access 14(2023): 553-566. DOI: 10.1016/j.ssci.2025.107028
Ostrowski L, Lindqvist O, et al. "Human-AI Teaming in Radiology Triage: a reproducibility study". ACM Trans Softw Eng Methodol 16(2014): 564-578. DOI: 10.2196/48633
Petrescu S, Szabo P, et al. "Human-AI Teaming in Radiology Triage: a large-scale evaluation". Proc MLSys 18(2017): 575-581. DOI: 10.1371/journal.pone.0305949
Quintero B, Balogh Q, et al. "Human-AI Teaming in Radiology Triage: a case study". Proc FAccT 20(2020): 586-593. DOI: 10.1371/journal.pdig.0000514
Ristic C, Castellani R, et al. "Human-AI Teaming in Radiology Triage: a survey". Nat Mach Intell 22(2023): 597-605. DOI: 10.17219/acem/208596
Szabo J, Jankowiak S, et al. "Human-AI Teaming in Radiology Triage: a comparative analysis". J Mach Learn Res 24(2014): 608-617. DOI: 10.7759/cureus.104086
Tarasov Q, Quintero T, et al. "Human-AI Teaming in Radiology Triage: a field deployment report". IEEE Trans Pattern Anal Mach Intell 26(2017): 619-629. DOI: 10.1038/s41746-023-00852-5
Urbanek Z, Zielinski U, et al. "Human-AI Teaming in Radiology Triage: a benchmark study". Proc NeurIPS 28(2020): 630-641. DOI: 10.1016/j.mayocpiqo.2022.09.004
Vasquez A, Alvez V, et al. "Human-AI Teaming in Radiology Triage: a systematic evaluation". Proc ICML 30(2023): 641-653. DOI: 10.1177/08404704241291226
Wojcik H, Hendriksen W, et al. "Human-AI Teaming in Radiology Triage: a empirical analysis". Proc ICLR 32(2014): 652-665. DOI: 10.1093/medlaw/fwad013
Zielinski O, Ostrowski Z, et al. "Human-AI Teaming in Radiology Triage: a ablation study". ACM Trans Intell Syst 34(2017): 663-677. DOI: 10.2196/89834
Andric V, Vasquez A, et al. "Human-AI Teaming in Radiology Triage: a reproducibility study". Data Min Knowl Discov 36(2020): 674-680. DOI: 10.1007/s10916-024-02104-9
