Artigo em revista científica Q1
Quis judicabit ipsos judices? A case study on the dynamics of competitive funding panel evaluations
João M. Santos (Santos, J. M.);
Título Revista
Research Evaluation
Ano (publicação definitiva)
2023
Língua
Inglês
País
Reino Unido
Mais Informação
Web of Science®

N.º de citações: 0

(Última verificação: 2024-07-17 13:12)

Ver o registo na Web of Science®

Scopus

N.º de citações: 0

(Última verificação: 2024-07-15 01:58)

Ver o registo na Scopus

Google Scholar

N.º de citações: 1

(Última verificação: 2024-07-14 19:43)

Ver o registo no Google Scholar

Abstract/Resumo
Securing research funding is essential for all researchers. The standard evaluation method for competitive grants is through evaluation by a panel of experts. However, the literature notes that peer review has inherent flaws and is subject to biases, which can arise from differing interpretations of the criteria, the impossibility for a group of reviewers to be experts in all possible topics within their field, and the role of affect. As such, understanding the dynamics at play during panel evaluations is crucial to allow researchers a better chance at securing funding, and also for the reviewers themselves to be aware of the cognitive mechanisms underlying their decision-making. In this study, we conduct a case study based on application and evaluation data for two social sciences panels in a competitive state-funded call in Portugal. Using a mixed-methods approach, we find that qualitative evaluations largely resonate with the evaluation criteria, and the candidate’s scientific output is partially aligned with the qualitative evaluations, but scientometric indicators alone do not significantly influence the candidate’s evaluation. However, the polarity of the qualitative evaluation has a positive influence on the candidate’s evaluation. This paradox is discussed as possibly resulting from the occurrence of a halo effect in the panel’s judgment of the candidates. By providing a multi-methods approach, this study aims to provide insights that can be useful for all stakeholders involved in competitive funding evaluations.
Agradecimentos/Acknowledgements
--
Palavras-chave
Peer review,Evaluation,Funding,Scientometrics
  • Ciências da Educação - Ciências Sociais
  • Ciências da Comunicação - Ciências Sociais
  • Outras Ciências Sociais - Ciências Sociais
Registos de financiamentos
Referência de financiamento Entidade Financiadora
UIDB/03126/2020 Fundação para a Ciência e a Tecnologia