Recommender Feedback Loops: Measuring Popularity Amplification in a Live System

Authors

  • Torben Bundgaard Centre for Machine Learning Research, Stevns Centre for Machine Intelligence, Store Heddinge, Denmark Author
  • Roxane Coutancie Department of Data Science, Bresse Institute of Applied Artificial Intelligence, Bourg-en-Bresse, France Author
  • Teodoro Belmondo Department of Applied Econometrics, Ciociaria Centre for Machine Intelligence, Frosinone, Italy Author

DOI:

https://doi.org/10.5281/zenodo.22660977

Keywords:

Recommender Feedback Loops; Applied machine learning; Evidence synthesis; Production deployment; MLOps

Abstract

Recommender Feedback Loops is an area of active development in applied machine learning, with evidence that has accumulated
unevenly across its component questions. We review the current evidence base for recommender feedback loops, spanning
methodological, implementation, governance, and cost dimensions, focusing on where practice has converged and where it
continues to vary, and set out the practical considerations relevant to a team considering its adoption. We conclude that selective,
context-specific use is best supported by the evidence currently available, rather than uniform adoption or avoidance.

References

Alvez D, Dumitrescu A, et al. "Recommender Feedback Loops: a benchmark study". J Mach Learn Res 8(2014): 80-86. DOI: 10.1007/s00330-025-12031-z

Brankovic K, Kowalczyk B, et al. "Recommender Feedback Loops: a systematic evaluation". IEEE Trans Pattern Anal Mach Intell 10(2017): 91-98. DOI: 10.3389/fdgth.2026.1699005

Castellani R, Ristic C, et al. "Recommender Feedback Loops: a empirical analysis". Proc NeurIPS 12(2020): 102-110. DOI: 10.3389/fdgth.2026.1834800

Dumitrescu A, Andric D, et al. "Recommender Feedback Loops: a ablation study". Proc ICML 14(2023): 113-122. DOI: 10.2196/69678

Eriksdottir B, Brankovic E, et al. "Recommender Feedback Loops: a reproducibility study". Proc ICLR 16(2014): 124-134. DOI: 10.7759/cureus.46454

Farkas I, Ionescu F, et al. "Recommender Feedback Loops: a large-scale evaluation". ACM Trans Intell Syst 18(2017): 135-146. DOI: 10.1016/j.ssci.2025.107028

Grabowski P, Petrescu G, et al. "Recommender Feedback Loops: a case study". Data Min Knowl Discov 20(2020): 146-158. DOI: 10.2196/48633

Hendriksen W, Wojcik H, et al. "Recommender Feedback Loops: a survey". Artif Intell 22(2023): 157-170. DOI: 10.1371/journal.pone.0305949

Ionescu F, Foscarini I, et al. "Recommender Feedback Loops: a comparative analysis". Knowl Based Syst 24(2014): 168-182. DOI:

1371/journal.pdig.0000514 10. Jankowiak G, Grabowski J, et al. "Recommender Feedback Loops: a field deployment report". IEEE Trans Neural Netw Learn Syst 26(2017): 179-185. DOI: 10.17219/acem/208596

Kowalczyk N, Novotna K, et al. "Recommender Feedback Loops: a benchmark study". Proc KDD 28(2020): 190-197. DOI: 10.7759/cureus.104086

Lindqvist U, Urbanek L, et al. "Recommender Feedback Loops: a systematic evaluation". Proc AAAI 30(2023): 201-209. DOI: 10.1038/s41746-023-00852-5

Marchetti D, Doncheva M, et al. "Recommender Feedback Loops: a empirical analysis". IEEE Trans Knowl Data Eng 32(2014): 212-221. DOI: 10.1016/j.mayocpiqo.2022.09.004

Novotna E, Eriksdottir N, et al. "Recommender Feedback Loops: a ablation study". J Artif Intell Res 34(2017): 223-233. DOI: 10.1177/08404704241291226

Ostrowski L, Lindqvist O, et al. "Recommender Feedback Loops: a reproducibility study". ACM Comput Surv 36(2020): 234-245. DOI: 10.1093/medlaw/fwad013

Petrescu S, Szabo P, et al. "Recommender Feedback Loops: a large-scale evaluation". Neural Comput Appl 38(2023): 245-257. DOI: 10.2196/89834

Quintero B, Balogh Q, et al. "Recommender Feedback Loops: a case study". Expert Syst Appl 40(2014): 256-269. DOI: 10.1007/s10916-024-02104-9

Ristic C, Castellani R, et al. "Recommender Feedback Loops: a survey". Inf Sci 42(2017): 267-281. DOI: 10.3389/fdgth.2026.1896184

Szabo J, Jankowiak S, et al. "Recommender Feedback Loops: a comparative analysis". Pattern Recognit 44(2020): 278-284. DOI: 10.3390/diagnostics15162117

Tarasov Q, Quintero T, et al. "Recommender Feedback Loops: a field deployment report". IEEE Access 46(2023): 289-296. DOI: 10.4103/jrpp.jrpp_20_65

Urbanek Z, Zielinski U, et al. "Recommender Feedback Loops: a benchmark study". ACM Trans Softw Eng Methodol 8(2014): 300-308. DOI: 10.3389/fdgth.2026.1763884

Vasquez A, Alvez V, et al. "Recommender Feedback Loops: a systematic evaluation". Proc MLSys 10(2017): 311-320. DOI: 10.2196/48297

Wojcik H, Hendriksen W, et al. "Recommender Feedback Loops: a empirical analysis". Proc FAccT 12(2020): 322-332. DOI: 10.1007/s00330-025-12031-z

Zielinski O, Ostrowski Z, et al. "Recommender Feedback Loops: a ablation study". Nat Mach Intell 14(2023): 333-344. DOI: 10.3389/fdgth.2026.1699005

Andric V, Vasquez A, et al. "Recommender Feedback Loops: a reproducibility study". J Mach Learn Res 16(2014): 344-356. DOI: 10.3389/fdgth.2026.1834800

Balogh E, Emmerich B, et al. "Recommender Feedback Loops: a large-scale evaluation". IEEE Trans Pattern Anal Mach Intell 18(2017): 355-368. DOI: 10.2196/69678

Chiriac F, Farkas C, et al. "Recommender Feedback Loops: a case study". Proc NeurIPS 20(2020): 366-380. DOI: 10.7759/cureus.46454

Doncheva M, Marchetti D, et al. "Recommender Feedback Loops: a survey". Proc ICML 22(2023): 377-383. DOI: 10.1016/j.ssci.2025.107028

Emmerich T, Tarasov E, et al. "Recommender Feedback Loops: a comparative analysis". Proc ICLR 24(2014): 388-395. DOI: 10.2196/48633

Foscarini C, Chiriac F, et al. "Recommender Feedback Loops: a field deployment report". ACM Trans Intell Syst 26(2017): 399-407. DOI: 10.1371/journal.pone.0305949

Alvez D, Dumitrescu A, et al. "Recommender Feedback Loops: a benchmark study". Data Min Knowl Discov 28(2020): 410-419. DOI: 10.1371/journal.pdig.0000514

Brankovic K, Kowalczyk B, et al. "Recommender Feedback Loops: a systematic evaluation". Artif Intell 30(2023): 421-431. DOI: 10.17219/acem/208596

Castellani R, Ristic C, et al. "Recommender Feedback Loops: a empirical analysis". Knowl Based Syst 32(2014): 432-443. DOI: 10.7759/cureus.104086

Dumitrescu A, Andric D, et al. "Recommender Feedback Loops: a ablation study". IEEE Trans Neural Netw Learn Syst 34(2017): 443-455. DOI: 10.1038/s41746-023-00852-5

Eriksdottir B, Brankovic E, et al. "Recommender Feedback Loops: a reproducibility study". Proc KDD 36(2020): 454-467. DOI: 10.1016/j.mayocpiqo.2022.09.004

Farkas I, Ionescu F, et al. "Recommender Feedback Loops: a large-scale evaluation". Proc AAAI 38(2023): 465-479. DOI: 10.1177/08404704241291226

Grabowski P, Petrescu G, et al. "Recommender Feedback Loops: a case study". IEEE Trans Knowl Data Eng 40(2014): 476-482. DOI: 10.1093/medlaw/fwad013

Hendriksen W, Wojcik H, et al. "Recommender Feedback Loops: a survey". J Artif Intell Res 42(2017): 487-494. DOI: 10.2196/89834

Ionescu F, Foscarini I, et al. "Recommender Feedback Loops: a comparative analysis". ACM Comput Surv 44(2020): 498-506. DOI: 10.1007/s10916-024-02104-9

Jankowiak G, Grabowski J, et al. "Recommender Feedback Loops: a field deployment report". Neural Comput Appl 46(2023): 509-518. DOI: 10.3389/fdgth.2026.1896184

Kowalczyk N, Novotna K, et al. "Recommender Feedback Loops: a benchmark study". Expert Syst Appl 8(2014): 520-530. DOI: 10.3390/diagnostics15162117

Lindqvist U, Urbanek L, et al. "Recommender Feedback Loops: a systematic evaluation". Inf Sci 10(2017): 531-542. DOI: 10.4103/jrpp.jrpp_20_65

Marchetti D, Doncheva M, et al. "Recommender Feedback Loops: a empirical analysis". Pattern Recognit 12(2020): 542-554. DOI: 10.3389/fdgth.2026.1763884

Novotna E, Eriksdottir N, et al. "Recommender Feedback Loops: a ablation study". IEEE Access 14(2023): 553-566. DOI: 10.2196/48297

Ostrowski L, Lindqvist O, et al. "Recommender Feedback Loops: a reproducibility study". ACM Trans Softw Eng Methodol 16(2014): 564-578. DOI: 10.1007/s00330-025-12031-z

Petrescu S, Szabo P, et al. "Recommender Feedback Loops: a large-scale evaluation". Proc MLSys 18(2017): 575-581. DOI: 10.3389/fdgth.2026.1699005

Quintero B, Balogh Q, et al. "Recommender Feedback Loops: a case study". Proc FAccT 20(2020): 586-593. DOI: 10.3389/fdgth.2026.1834800

Ristic C, Castellani R, et al. "Recommender Feedback Loops: a survey". Nat Mach Intell 22(2023): 597-605. DOI: 10.2196/69678

Szabo J, Jankowiak S, et al. "Recommender Feedback Loops: a comparative analysis". J Mach Learn Res 24(2014): 608-617. DOI: 10.7759/cureus.46454

Tarasov Q, Quintero T, et al. "Recommender Feedback Loops: a field deployment report". IEEE Trans Pattern Anal Mach Intell 26(2017): 619-629. DOI: 10.1016/j.ssci.2025.107028

Urbanek Z, Zielinski U, et al. "Recommender Feedback Loops: a benchmark study". Proc NeurIPS 28(2020): 630-641. DOI: 10.2196/48633

Vasquez A, Alvez V, et al. "Recommender Feedback Loops: a systematic evaluation". Proc ICML 30(2023): 641-653. DOI: 10.1371/journal.pone.0305949

Wojcik H, Hendriksen W, et al. "Recommender Feedback Loops: a empirical analysis". Proc ICLR 32(2014): 652-665. DOI: 10.1371/journal.pdig.0000514

Zielinski O, Ostrowski Z, et al. "Recommender Feedback Loops: a ablation study". ACM Trans Intell Syst 34(2017): 663-677. DOI: 10.17219/acem/208596

Andric V, Vasquez A, et al. "Recommender Feedback Loops: a reproducibility study". Data Min Knowl Discov 36(2020): 674-680. DOI: 10.7759/cureus.104086

Downloads

Published

2026-03-13

Issue

Vol. 5 No. 1 (2026): vol 5 issue 1 2026

Section

Articles

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.