Selection of staff for inclusion in RAE2008

September 2009 | ref: 2009/34


Heads of HEFCE-funded further education colleges, Heads of HEFCE-funded higher education institutions, Heads of universities in Northern Ireland
Heads of publicly funded higher education institutions in the UK

Of interest to those responsible for:

Equality and diversity management, Human resources management, Institutional strategic planning, Research management

This study investigates how disability, age, sex, ethnicity and nationality are related to selection of staff for inclusion in the 2008 Research Assessment Exercise (RAE2008). It examines the question of whether the process of selecting staff was unbiased, or whether some staff were disadvantaged.

Alongside this report, the Equality Challenge Unit has commissioned a qualitative investigation of the impact of the process to promote equality and diversity in the RAE 2008: 'Impact of the process to promote equality and diversity in the Research Assessment Exercise 2009'.


  • Executive summary
  • Introduction
  • Data
  • Re-creating the RAE2001 analysis with a pool methodology
  • Variation in selection rates
  • Discussion – are there biases in selection?
  • Discussion – the context of responses to equality issues
  • Annex A Terminology and abbreviations
  • Annex B HESA Data – Definitions and Groups
  • Annex C Additional tables for equality factors
  • Annex D Tables for selection Index
  • Annex E Statistical models for staff selection

Executive summary


Background and scope

1.   This study investigates how disability, age, gender, ethnicity and nationality are related to selection of staff for inclusion in the 2008 Research Assessment Exercise (RAE2008).

Key points

2.   The Research Assessment Exercise (RAE) assessed the quality of research submitted by higher education institutions (HEIs) in the UK through a process of expert review. The outcomes of the assessment inform the amount of funding each HEI will receive for its research activities.

3.   In 2006, HEFCE published a UK-wide equality and diversity assessment of the 2001 exercise (RAE2001), HEFCE 2006/32. This investigated how disability, gender, age and ethnicity related to the selection of staff by HEIs for inclusion in that RAE. Throughout this document 'the 2006 report' refers to HEFCE 2006/32.

4.   The latest RAE was completed in 2008. This study assesses staff selected for the 2008 exercise in terms of disability status, age, gender, ethnicity and nationality. As with the 2006 report, the scope of our quantitative analysis is limited to assessing whether the process of selecting staff was unbiased from an equality and diversity perspective, or whether some staff were disadvantaged. We have not, for example, attempted to re-create the assessments of particular institutions, or assessed whether the research process as a whole is biased.


5.   Due to differences in interpretation by HEIs of the definition of eligibility for RAE2008, we were unable to reproduce the methodology used in the 2006 report to identify staff who were eligible for selection but not submitted; therefore we were unable to calculate selection rates in the same way.

6.   Instead, our methodology considered four ‘pools’ of potentially eligible staff: All academic staff, Permanent academic staff, Grade-identified staff and Contract-identified staff (see paragraph 34). Using a range of pools meant we could consider similarities and differences in patterns of selection with regard to the five equality factors listed above (disability, age, gender, ethnicity and nationality).

7.   We used statistical models to compare staff by equality factor while controlling for other factors, such as the research output quality of staff and the research environment they work in. It should be noted that there is no direct measure of research output quality of staff, therefore we have relied on other data (such as the grade of the staff considered) to estimate this measure in the models.

Results and conclusions

8.   In all four pools of staff, the selection rate for staff with declared disability was lower than for those staff without declared disability. However, modelling indicates that other factors (such as differences in staff selection rates associated with the HEI, or subject area the individual is working in) may explain these differences more readily than disability status.

9.   As in the 2006 report, the data shows a difference between the rate of selection for men and women in RAE2008 – for example in the 'Permanent academic staff' pool 67 per cent of men were selected compared to 48 per cent of women. When age is considered in combination with gender, the model output shows that differences continue to be most apparent in the 30-50 age range despite the changes between the RAE2001 and RAE2008 selection process to promote equal opportunities in the RAE.

10.   Bibliometric evidence in the 2006 report indicated that the lower selection rate of women in the 30-50 age range may be due to a lower proportion of women having a research record that leads them to be selected, rather than bias in the selection process. While differences in selection rate between men and women may be linked to selection bias resulting from age and gender, it could equally be a result of deeply rooted inequalities in the research careers of men and women.

11.   The selection rates were at similar levels for all ethnicity groups except those staff in the Black ethnic group, whose selection rate was lower in all the pools of staff. The lower selection rate was not explained when other factors (such as those mentioned in paragraph 8) were taken into account. This was the case in the 2006 report too, but is slightly more pronounced; however the change could have resulted from the changes in methodology.

12.   Following this result, we additionally considered the effect of nationality on selection rates. The introduction of this factor highlighted higher selection rates for non-UK nationals compared to UK nationals.

13.   The combined effect between ethnicity and nationality does not explain the low levels of selection of the Black group. However, bibliometric evidence from the 2006 report suggests that the differences may be due to a weakness in the proxies for research output quality included in the quantitative analysis rather than an unjustifiable bias in selection to the RAE.

14.   Overall this analysis shows that with regard to equality and diversity, staff selection to the RAE2008 was similar in composition to that seen in RAE2001. This is not to say that the new processes and equality measures put in place since 2001 have not had an effect on selection at institutional, departmental or Unit of Assessment level. It may be that some of the differences seen in this and the 2006 report are linked to individual career choices and deeply rooted inequalities than of particular discrimination against specific groups of staff. This is an area that HEFCE and the higher education (HE) sector will continue to explore.

15.   The extent to which the different selection rates observed reflect deeply rooted social inequalities is being acknowledged by the extensive work the HE sector, HEFCE and the Equality Challenge Unit (ECU) are doing to support the research careers of different groups of staff.

16.   There are many projects within the HE sector which are helping to develop the understanding of gender and age equality issues for academic staff. These include: women in academic medicine project at Imperial College London, the Athena SWAN Charter at the ECU and the women academic returners' programme at the University of Sheffield.

17.   In addition, the ECU is co-ordinating a Race Forum project supported by HEFCE’s Leadership Governance and Management Fund. The project will help identify a range of possible initiatives to address issues affecting black and minority ethnic (BME) staff in the HE sector and meet race equality duties in the sector, with particular reference to recruitment, retention, promotion and development of BME staff and inclusion in structures of governance .

18.   The issues arising from this and the 2006 analysis of the RAE are informing the development of the Research Excellence Framework (REF), which will continue to actively promote equality and diversity. The evidence gathered from this quantitative analysis, alongside the ECU’s qualitative work on RAE2008 and other related work, will help to assess the potential impact on the sector of moving from the RAE to the REF.

19.   HEFCE is shortly to publish proposals on the key features of the REF for consultation with the sector in autumn 2009, and expects to announce the outcomes in early 2010.

Action required

20.   No action is required in response to this document. 

Enquiries should be directed to:

Suzanne Wilson, tel 0117 931 7458, e-mail

Fariba Dashtgard, tel 0117 931 7316, e-mail

Hannah White, tel 0117 931 7063, e-mail

Page last updated 18 January 2012

Share this: