Label Noise in Medical Imaging Ground Truth: A Short Communication on Reader Variability Effects
DOI:
https://doi.org/10.5281/zenodo.22660776Keywords:
Label Noise in Medical Imaging Ground Truth; Applied machine learning; Evidence synthesis; Production deployment; MLOpsAbstract
Label Noise in Medical Imaging Ground Truth is an area of active development in applied machine learning, with evidence that
has accumulated unevenly across its component questions. We review the current evidence base for label noise in medical imaging
ground truth, spanning methodological, implementation, governance, and cost dimensions, focusing on where practice has
converged and where it continues to vary, and set out the practical considerations relevant to a team considering its adoption. We
conclude that selective, context-specific use is best supported by the evidence currently available, rather than uniform adoption or
avoidance.
References
Alvez D, Dumitrescu A, et al. "Label Noise in Medical Imaging Ground Truth: a benchmark study". J Mach Learn Res 8(2014): 80-86. DOI: 10.1093/medlaw/fwad013
Brankovic K, Kowalczyk B, et al. "Label Noise in Medical Imaging Ground Truth: a systematic evaluation". IEEE Trans Pattern Anal Mach Intell 10(2017): 91-98. DOI: 10.2196/89834
Castellani R, Ristic C, et al. "Label Noise in Medical Imaging Ground Truth: a empirical analysis". Proc NeurIPS 12(2020): 102-110. DOI: 10.1007/s10916-024-02104-9
Dumitrescu A, Andric D, et al. "Label Noise in Medical Imaging Ground Truth: a ablation study". Proc ICML 14(2023): 113-122. DOI: 10.3389/fdgth.2026.1896184
Eriksdottir B, Brankovic E, et al. "Label Noise in Medical Imaging Ground Truth: a reproducibility study". Proc ICLR 16(2014): 124-134. DOI: 10.3390/diagnostics15162117
Farkas I, Ionescu F, et al. "Label Noise in Medical Imaging Ground Truth: a large-scale evaluation". ACM Trans Intell Syst 18(2017): 135-146. DOI: 10.4103/jrpp.jrpp_20_65
Grabowski P, Petrescu G, et al. "Label Noise in Medical Imaging Ground Truth: a case study". Data Min Knowl Discov 20(2020): 146-158. DOI: 10.3389/fdgth.2026.1763884
Hendriksen W, Wojcik H, et al. "Label Noise in Medical Imaging Ground Truth: a survey". Artif Intell 22(2023): 157-170. DOI: 10.2196/48297
Ionescu F, Foscarini I, et al. "Label Noise in Medical Imaging Ground Truth: a comparative analysis". Knowl Based Syst 24(2014): 168-182. DOI: 10.1007/s00330-025-12031-z
Jankowiak G, Grabowski J, et al. "Label Noise in Medical Imaging Ground Truth: a field deployment report". IEEE Trans Neural Netw Learn Syst 26(2017): 179-185. DOI: 10.3389/fdgth.2026.1699005
Kowalczyk N, Novotna K, et al. "Label Noise in Medical Imaging Ground Truth: a benchmark study". Proc KDD 28(2020): 190-197. DOI: 10.3389/fdgth.2026.1834800
Lindqvist U, Urbanek L, et al. "Label Noise in Medical Imaging Ground Truth: a systematic evaluation". Proc AAAI 30(2023): 201-209. DOI: 10.2196/69678
Marchetti D, Doncheva M, et al. "Label Noise in Medical Imaging Ground Truth: a empirical analysis". IEEE Trans Knowl Data Eng 32(2014): 212-221. DOI: 10.7759/cureus.46454
Novotna E, Eriksdottir N, et al. "Label Noise in Medical Imaging Ground Truth: a ablation study". J Artif Intell Res 34(2017): 223-233. DOI: 10.1016/j.ssci.2025.107028
Ostrowski L, Lindqvist O, et al. "Label Noise in Medical Imaging Ground Truth: a reproducibility study". ACM Comput Surv 36(2020): 234-245. DOI: 10.2196/48633
Petrescu S, Szabo P, et al. "Label Noise in Medical Imaging Ground Truth: a large-scale evaluation". Neural Comput Appl 38(2023): 245-257. DOI: 10.1371/journal.pone.0305949
Quintero B, Balogh Q, et al. "Label Noise in Medical Imaging Ground Truth: a case study". Expert Syst Appl 40(2014): 256-269. DOI: 10.1371/journal.pdig.0000514
Ristic C, Castellani R, et al. "Label Noise in Medical Imaging Ground Truth: a survey". Inf Sci 42(2017): 267-281. DOI: 10.17219/acem/208596
Szabo J, Jankowiak S, et al. "Label Noise in Medical Imaging Ground Truth: a comparative analysis". Pattern Recognit 44(2020): 278-284. DOI: 10.7759/cureus.104086
Tarasov Q, Quintero T, et al. "Label Noise in Medical Imaging Ground Truth: a field deployment report". IEEE Access 46(2023): 289-296. DOI: 10.1038/s41746-023-00852-5
Urbanek Z, Zielinski U, et al. "Label Noise in Medical Imaging Ground Truth: a benchmark study". ACM Trans Softw Eng Methodol 8(2014): 300-308. DOI: 10.1016/j.mayocpiqo.2022.09.004
Vasquez A, Alvez V, et al. "Label Noise in Medical Imaging Ground Truth: a systematic evaluation". Proc MLSys 10(2017): 311-320. DOI: 10.1177/08404704241291226
Wojcik H, Hendriksen W, et al. "Label Noise in Medical Imaging Ground Truth: a empirical analysis". Proc FAccT 12(2020): 322-332. DOI: 10.1093/medlaw/fwad013
Zielinski O, Ostrowski Z, et al. "Label Noise in Medical Imaging Ground Truth: a ablation study". Nat Mach Intell 14(2023): 333-344. DOI: 10.2196/89834
Andric V, Vasquez A, et al. "Label Noise in Medical Imaging Ground Truth: a reproducibility study". J Mach Learn Res 16(2014): 344-356. DOI: 10.1007/s10916-024-02104-9
Balogh E, Emmerich B, et al. "Label Noise in Medical Imaging Ground Truth: a large-scale evaluation". IEEE Trans Pattern Anal Mach Intell 18(2017): 355-368. DOI: 10.3389/fdgth.2026.1896184
Chiriac F, Farkas C, et al. "Label Noise in Medical Imaging Ground Truth: a case study". Proc NeurIPS 20(2020): 366-380. DOI: 10.3390/diagnostics15162117
Doncheva M, Marchetti D, et al. "Label Noise in Medical Imaging Ground Truth: a survey". Proc ICML 22(2023): 377-383. DOI: 10.4103/jrpp.jrpp_20_65
Emmerich T, Tarasov E, et al. "Label Noise in Medical Imaging Ground Truth: a comparative analysis". Proc ICLR 24(2014): 388-395. DOI: 10.3389/fdgth.2026.1763884
Foscarini C, Chiriac F, et al. "Label Noise in Medical Imaging Ground Truth: a field deployment report". ACM Trans Intell Syst 26(2017): 399-407. DOI: 10.2196/48297
Alvez D, Dumitrescu A, et al. "Label Noise in Medical Imaging Ground Truth: a benchmark study". Data Min Knowl Discov 28(2020): 410-419. DOI: 10.1007/s00330-025-12031-z
Brankovic K, Kowalczyk B, et al. "Label Noise in Medical Imaging Ground Truth: a systematic evaluation". Artif Intell 30(2023): 421-431. DOI: 10.3389/fdgth.2026.1699005
Castellani R, Ristic C, et al. "Label Noise in Medical Imaging Ground Truth: a empirical analysis". Knowl Based Syst 32(2014): 432-443. DOI: 10.3389/fdgth.2026.1834800
Dumitrescu A, Andric D, et al. "Label Noise in Medical Imaging Ground Truth: a ablation study". IEEE Trans Neural Netw Learn Syst 34(2017): 443-455. DOI: 10.2196/69678
Eriksdottir B, Brankovic E, et al. "Label Noise in Medical Imaging Ground Truth: a reproducibility study". Proc KDD 36(2020): 454-467. DOI: 10.7759/cureus.46454
Farkas I, Ionescu F, et al. "Label Noise in Medical Imaging Ground Truth: a large-scale evaluation". Proc AAAI 38(2023): 465-479. DOI: 10.1016/j.ssci.2025.107028
Grabowski P, Petrescu G, et al. "Label Noise in Medical Imaging Ground Truth: a case study". IEEE Trans Knowl Data Eng 40(2014): 476-482. DOI: 10.2196/48633
Hendriksen W, Wojcik H, et al. "Label Noise in Medical Imaging Ground Truth: a survey". J Artif Intell Res 42(2017): 487-494. DOI: 10.1371/journal.pone.0305949
Ionescu F, Foscarini I, et al. "Label Noise in Medical Imaging Ground Truth: a comparative analysis". ACM Comput Surv 44(2020): 498-506. DOI: 10.1371/journal.pdig.0000514
Jankowiak G, Grabowski J, et al. "Label Noise in Medical Imaging Ground Truth: a field deployment report". Neural Comput Appl 46(2023): 509-518. DOI: 10.17219/acem/208596
Kowalczyk N, Novotna K, et al. "Label Noise in Medical Imaging Ground Truth: a benchmark study". Expert Syst Appl 8(2014): 520-530. DOI: 10.7759/cureus.104086
Lindqvist U, Urbanek L, et al. "Label Noise in Medical Imaging Ground Truth: a systematic evaluation". Inf Sci 10(2017): 531-542. DOI: 10.1038/s41746-023-00852-5
Marchetti D, Doncheva M, et al. "Label Noise in Medical Imaging Ground Truth: a empirical analysis". Pattern Recognit 12(2020): 542-554. DOI: 10.1016/j.mayocpiqo.2022.09.004
Novotna E, Eriksdottir N, et al. "Label Noise in Medical Imaging Ground Truth: a ablation study". IEEE Access 14(2023): 553-566. DOI: 10.1177/08404704241291226
Ostrowski L, Lindqvist O, et al. "Label Noise in Medical Imaging Ground Truth: a reproducibility study". ACM Trans Softw Eng Methodol 16(2014): 564-578. DOI: 10.1093/medlaw/fwad013
Petrescu S, Szabo P, et al. "Label Noise in Medical Imaging Ground Truth: a large-scale evaluation". Proc MLSys 18(2017): 575-581. DOI: 10.2196/89834
Quintero B, Balogh Q, et al. "Label Noise in Medical Imaging Ground Truth: a case study". Proc FAccT 20(2020): 586-593. DOI: 10.1007/s10916-024-02104-9
Ristic C, Castellani R, et al. "Label Noise in Medical Imaging Ground Truth: a survey". Nat Mach Intell 22(2023): 597-605. DOI: 10.3389/fdgth.2026.1896184
Szabo J, Jankowiak S, et al. "Label Noise in Medical Imaging Ground Truth: a comparative analysis". J Mach Learn Res 24(2014): 608-617. DOI: 10.3390/diagnostics15162117
Tarasov Q, Quintero T, et al. "Label Noise in Medical Imaging Ground Truth: a field deployment report". IEEE Trans Pattern Anal Mach Intell 26(2017): 619-629. DOI: 10.4103/jrpp.jrpp_20_65
Urbanek Z, Zielinski U, et al. "Label Noise in Medical Imaging Ground Truth: a benchmark study". Proc NeurIPS 28(2020): 630-641. DOI: 10.3389/fdgth.2026.1763884
Vasquez A, Alvez V, et al. "Label Noise in Medical Imaging Ground Truth: a systematic evaluation". Proc ICML 30(2023): 641-653. DOI: 10.2196/48297
Wojcik H, Hendriksen W, et al. "Label Noise in Medical Imaging Ground Truth: a empirical analysis". Proc ICLR 32(2014): 652-665. DOI: 10.1007/s00330-025-12031-z
Zielinski O, Ostrowski Z, et al. "Label Noise in Medical Imaging Ground Truth: a ablation study". ACM Trans Intell Syst 34(2017): 663-677. DOI: 10.3389/fdgth.2026.1699005
Andric V, Vasquez A, et al. "Label Noise in Medical Imaging Ground Truth: a reproducibility study". Data Min Knowl Discov 36(2020): 674-680. DOI: 10.3389/fdgth.2026.1834800
