What the resource is:
This paper, presented at BERA 2009, considers the impact that educational research has on policy, largely by reviewing relevant literature and by evaluating the impact that three very different NFER research projects (TIMSS, Gifted and Talented Evaluation, and National Monitoring Surveys) have had on policy.
The aims of the resource:
It aims to
- critically reflect on the concept of ‘impact' and what this means in relation to educational research
- review literature to illustrate how differing levels of impact are appropriate for different kinds of research
- consider the extent to which research findings are used in policy development.
Key findings or focus:
The key findings from the literature review were as follows:
What proportion of research has an impact?
In the United States very few educational research projects seem to have significant impact on policy, which begs the question as to how much policy is really evidence informed.
What is impact?
There appears to be no agreed definition among researchers or policy makers of what is meant by ‘impact', although it might be helpful to conceptualise ‘impact' of educational research as a continuum from raising awareness of a particular issue through to changes in behaviour, attitude and practice.
How can we measure impact?
There is also little consensus in the literature as to the best way to measure impact of educational research. There is evidence that this is often in the form of self-reporting rather than rigorous evaluation.
What features of research mean it is likely to have an impact?
There seems to be some agreement that one of the most significant features of ensuring impact of research on educational policy is through government legislation, as "schools will adopt policies whether or not they are evidence based" (cited from Hemsley-Brown and Sharp 2003). Appropriate and carefully thought through dissemination strategies are also felt to be important, particularly dissemination through the media. There is some agreement from the literature that if research findings are carefully structured to suit the users of research, then they are more likely to have an impact.
How do we ensure research has an impact?
There do seem to be processes that can improve the likelihood of research having impact. These include making sure the outcomes are presented in a form that can be easily understood by a targeted audience; ensuring there are closer links between researchers and policy makers and/or practitioners; devising training programmes to facilitate the use of research; and using people who are skilled at raising awareness and changing attitudes to research findings.
NFER case studies
Three NFER projects are then presented as case studies in order to illustrate the different ways research might impact on educational policy:
- Trends in International Maths and Science Survey (TIMSS) is cited as having a great deal of impact at policy level in England, and would seem to have had a significant influence on establishing the National Curriculum and National Numeracy Strategy in the 1990s, which then formalised the requirements for teachers in maths.
- The Gifted and Talented Evaluation project is cited as an example where, although research findings were disseminated to all Gifted and Talented coordinators, it did not necessarily mean this resulted in a change of practice.
- The National Monitoring Surveys project included reviewing websites and publications related to current national international and historical surveys. This was used to raise a number of key considerations when introducing a national monitoring survey at key stage 3. The research findings were skilfully reported so that key recommendations could be identified. This resulted in the 17 recommendations being picked up and published on the BBC website as well as the NFER website.
The author argues that, when considering what impact educational research has, it is necessary to take into account that "different research projects are aimed at different audiences which in turn have different requirements for impact". She suggests there needs to be an agreement as to whether ‘impact' should be perceived only in terms of impact on educational policy. The author recommends a common checklist should be agreed in order to record both the kinds of impact that a project intends and what kind of impact is actually achieved.
Finally, the author presents a useful conceptualisation of research impact devised internally by NFER. This suggests different levels of impact and how these might be suitable for different kinds of research. The levels are:
- Raising awareness about an issue via research
- Research which contributes to a body of knowledge
- Research which is used for consultancy in training and encouraging practitioners to adopt new skills through training with opportunities to practice these skills
- Changing behaviour through research, best achieved through coaching/training and/or policy.
The quality, authority and credibility of the resource from your subject perspective in relation to ITE:
The paper opens up the debate about what constitutes ‘impact' and ‘evidence based policy', and whether educational research should always have impact through policy. It draws upon evidence from NFER who have considerable experience of funding a variety of educational research projects.
The implications for ITE tutors/mentors - when and how it could have best impact:
The paper could help ITE tutors/mentors to be more discriminating in the way educational policies are seemingly evidence based or research informed. It could enable them to distinguish between research which underpins aspects of teaching and learning, and educational policy - which seemingly shapes teaching and learning in the classroom - which is not informed by research. It allows tutors to open up the discussion around the notion of what constitutes ‘impact' of research in the classroom, which is particularly relevant for Masters level study.
The relevance to ITE students - how and why it has importance:
This paper enables ITE students to consider what is meant by impact of educational research, and to be ‘users' of research findings as well as ‘policy implementers' in their classrooms. It will enable them to reflect on the extent to which their ITE programme and school experience is shaped by educational policy and the extent to which this is underpinned by research. It could also open up the debate about how researching their own classroom practice through practitioner research can enable them to develop a deeper understanding of teaching and learning in the classroom as well as using wider larger scale research to inform practice.