Developing and Researching the Economics and Mathematics of Selection: the DREAMS Network

The DREAMS network is a collaborative project funded via the WUN Research Development Fund (2016), with matched funding provided by the participating WUN-affiliated institutions. The initiative brings together international experts and researchers in order to extend and develop new approaches to understanding the selection of individuals into the professions, with medical selection as the main motivating example.

The programme consists of a series of three workshops, each with a selection-related theme, as well as a number of academic exchange fellowships for early careers researchers.

Outcomes

Workshops

Dr Tiffin delivered a keynote at the Annual Medical Secretaries Conference on 6 June 2019. The first workshop was held in York in February 2017 and was focused on ‘widening participation in the professions’.

The second workshop was held in Sydney in July 2017. This workshop investigated and debated best practice in developing Situational Judgement Tests (SJTs) to maximise reliability and validity for selection into the healthcare professions. This includes developing content, item formats (text and video-based), scoring and modelling (including IRT and classical test theory approaches), and decision making.

The theme of the third workshop is on the economics of selection.

Exchange visits

• Ghent University/University of York (Lazaro Mwakesi): Develop an understanding and extend of pareto-optimal models of selection.
• University of Leeds/University of York (Lewis Paton): Develop further methodological expertise in numerical simulation.
• United Nations/University of York (Charlotte Renwick): Develop conceptualisation of the economic and social impact of selection approaches within the context of low and middle income countries.
• University of York/University of Sydney (Indako Clarke): understand approaches to managing and analysing big data relating to selection and consider how certain undesirable traits (e.g. self-handicapping and narcissism) could be identified using SJTs at selection.
• University of Western Australia/University of York (Paul Tiffin and Lewis Paton): collaborate on how medical selection measures such as the Undergraduate Medicine and Health Sciences Admission Test (UMAT) and Graduate Medical Sciences Admissions Test (GAMSAT) can be further developed and validated, feeding into the revisions and online implementation planned by 2019.
• University of Sydney/University of York (Paul Tiffin and Lewis Paton): explore the further development of SJTs within Australian medical selection, specifically considering how the measurement of additional concepts such as emotional intelligence can be incorporated.

Academic Publications

• Situational judgments tests for selection: traditional vs construct-driven approaches Tiffin, P. A., Paton, L. W., O’Mara, D., MacCann, C., Lang, J. W. B. & Lievens, F., 23 Dec 2019 Article in Medical Education
• Does ‘online confidence’ predict application success and later academic performance in medical school? A UK-based national cohort study Tiffin, P. A. & Paton, L. W., 28 Nov 2019
• Evaluating the recruitment process into UK anaesthesia core training: doctors’ performance at selection and subsequent postgraduate training- a national data linkage study Aslet, M. E. D., Paton, L. W., Gale, T. & Tiffin, P. A., 24 Jul 2019 Article.

2018

• Patterson, F., Tiffin, P. A., Lopes, S. & Zibarras, L. Unpacking the Dark Variance of Differential Attainment in Exams for Overseas Graduates (2018). Medical Education, 52, 736-746.
• Tiffin, P.A., Alexander, K.  Cleland, J. When I say…fairness in selection (2018). Medical Education, 52, 1225-1227.
• Tiffin, P. A., Orr, J., Paton, L. W., Smith, D. & Norcini, J.UK nationals who received their medical degrees abroad: selection into, and subsequent performance in postgraduate training- a national data linkage study (2018). BMJOpen;8:e023060.

Policy Inputs

• Dr Tiffin participated in an NHS workshop- “Contribution of Physician Associates to NHS Mental Health Services” (2019)
  Fed our findings of a collaborative study into how Doctors were allocated to their Foundation Programme posts to the Foundation Programme Board, when they were considering selection policy changes.
• P.A. Tiffin & L.W. Paton, Report to UKCAT Board on the Potential of Measuring ‘Online Confidence’ for medical selection, January 2018
• C. Renwick, C.& A. Botha.  Internal Report to Human Resources department of the United Nations: Differential Item Functioning of Situational Judgement Test Items used in UN Personnel Selection, and Gender.

The DREAMS Network is also linked to INReSH- the international Network for Selection into Healthcare.