Towards Outstanding Research Reviews

Photo by freddie marriage on Unsplash

Funders and grant bodies usually invite scholars to peer review research proposals. Reviewers’ judgements are intended to support panel members and funders to identify outstanding projects to fund. Yet, recent research has revealed startling inconsistencies and lack of agreement in reviewers’ evaluations. This may cast doubts on the usefulness of peer-reviewing grant proposals.

There has been remarkably little empirical research on the cognitive processes which underpin scholars’ judgements of grant proposals and the reasons why inconsistencies arise remain unclear. The aim of this project is to understand how peer reviewers use and combine the information in grant proposals to make their recommendations. Our interdisciplinary research team combines its methodological expertise as well as its experience in research management and reviewing for national panels, to conduct a mixed-method study of grant reviewers’ judgement processes.

An innovative proof-of-principle study in the field of humanities and social sciences, informed by the Social Judgment Theory (SJT) framework, examines whether these judgement processes can be statistically modelled. If they can be made explicit, these empirical models could be used to develop evidence-based training to improve reviewers’ judgement capacity, with the potential to enhance judgement ability and directly optimise future funding policy and practice.

This work is supported by the Wellcome Trust, Grant 214532/Z/18/Z.

To learn more about this project, you can visit the TORR project website.

Collaborators

  • Prof Priscilla Harries
  • Prof Tushna Vandrevala
  • Dr Ana Wheelock
  • Ms Fanni Szigetvari
  • Ms Joanne Ahmed
Fanni Szigetvari
Fanni Szigetvari
PhD student & BeSci lab manager

I am a research associate within the TORR research project. As an aspiring Behavioural Scientist, I am interested in the psychology of human behaviour and decision-making, and in understanding people as they are, not as they ‘should be'.