Algorithmic evaluations are increasingly used to make decisions thanks to the perception of objective measures of quality and performance. However, little is known about how the current evaluation methods change with ML algorithms and with what consequences for the actors and organizations being evaluated. We conducted an exploratory case study in the breast unit of the Champalimaud Foundation in Lisbon. Gioia methodology guided the collection and analysis of semi-structured interviews and archival data. Our results show that besides generating visible and direct changes (e.g., extraction and quantification of relevant criteria with systematic approaches), algorithmic evaluations trigger indirect and less visible dynamics (e.g., adding a new dimension - aesthetic score - in the evaluation of research units), which have profound implications on how institutions operate and how resources are allocated based on the ranking lists. We contribute to digital undertow and institutional displacement and human ML collaborations by explaining the processes through which the new methods are used in medical communities and their less visible yet impactful consequences.

--

Aesthetics,Breast cancer surgery,Evaluation,Gioia methodology,Machine learning

• Computer and Information Sciences - Natural Sciences
• Electrical Engineering, Electronic Engineering, Information Engineering - Engineering and Technology
• Clinical Medicine - Medical and Health Sciences