The DCSF Research Conference 2010, The Use of Evidence in Policy Development and Delivery , took place at the QEII Conference Centre in London on 9 February. The day consisted of a series of addresses, followed by two workshop sessions, and a panel discussion in the afternoon. Carole Willis, Director of Research and Analysis at the DCSF, welcomed delegates, and spoke of the need to make decisions to use money for the best impact in terms of delivering better outcomes, based on evidence. A central role will be played by the three new research centres which are currently being commissioned by the DCSF, and launched that day; Centre for Understanding Behaviour Change, Childhood Well-being Research Centre, and Centre for Analysis of Youth Transitions. These signify a collaborative "new way of working" .
John Coles , Director General for Schools and Head of the Policy Delivery Profession (DCSF), then gave a presentation entitled Making Policy Happen: Improving practice in the DCSF "what will be deliverable by real people in real schools" . In this way, all three of the imperatives have to come together during the process of policy-making, which entails the following: understanding the context; developing the options; getting to a decision; and making it happen. This process is not linear, however, and compromise is essential along the way; at each stage, it is necessary to go back to the core purposes. As "policy is just as much about delivery as making" , making it happen (or "making it stick" ), involves performance managing the delivery of policy effectively, and this is a fundamental area of using evidence. Some parts of evidence may be privileged over others in this; the DCSF has an abundance of pupil level data, but also needs to use "softer forms" of data. The requirement for evidence-based policy making is for "good enough evidence" . As policy is not created in a vacuum, and will happen at great pace, evidence needs to be integrated at an early stage of the development process. Therefore, evidence on emerging issues must be either ready in advance, or be produced quickly; in this, the policy makers need help from the researchers.
Following this, Andreas Schleicher , Advisor to the Secretary-General on Education Policy, OECD Directorate for Education, then presented on The High Cost of Low Educational Performance
Delegates were offered a choice from a total of 18 workshops either side of lunchtime, all of which provided an insight into aspects of the current DCSF priorities and ways of working. Issues which were considered included: ECM outcomes (presentation not yet available); the Adoption Research Initiative (ARI) ; parenting programmes ; looked-after children ; teaching world religions ; young people and alcohol (presentation not yet available); the three new research centres ; school leadership and the wider workforce; the Early Education Pilot for Two Year Olds ; attainment gaps ; 14-19 disengagement from education ; Parental Opinion Survey and Parents Panel ; Tellus Survey ; progression at pupil, school and national levels ; STEM , GCSE to A Level ; longitudinal data (LSYPE) ; child poverty ; the Data & Statistics Strategy .
The workshop on Young People and Alcohol: Evidence-Based Policy Making and Social Marketing in Action "maximise the number of young people in England who grow up to have a safe and sensible relationship with alcohol" . The session reported on what was termed the ‘programme of insight' which led from this to the delivery of the social marketing campaign. The first stages of this were exploratory (qualitative) research and mapping, followed by a strategic review of the evidence base (55 documents), which identified core groups to talk to, and parents as the target audience. One of the more salutary findings, from the World Health Organisation, was that public information campaigns thus far had been ineffective in reducing alcohol-related harms. The next stage was ‘proposition testing' with parents and young people. This suggested a call to action for young people: Think before you drink , and for parents: Start talking before your children start drinking . Campaign development then led on to quantitative research, involving 613 children/young people and parents/carers, and 2017 interviews. From the findings, seven parents and carers ‘segments' were identified (e.g. ‘reactive avoiders' (17%), ‘risk reducing supervisors' (6%)), and, coincidentally, seven also for children and young people (e.g. ‘resilient rejectors' (14%)). This provided evidence to link use of and attitudes towards alcohol, which was seen as "one part in a complex set of influences that shape young people's attitude to alcohol" . Two years of campaign development had led to three adverts; one for young people in cinemas, one for TV pre-watershed, and one for TV post-watershed.
Another of the workshops looked at Deconstructing Attainment Gaps
Breaking the Link (2009), which sets out a strategy to address the FSM gap, with interventions on five levels. Following a review of existing policies, 11 programmes had been identified, from which agreed work strands will focus on NtG: National Challenge; City Challenge; Curriculum; Extended Schools; Academies; 121 Tuition; Parental Engagement; Every Child Matters interventions; School Report Card; LA Target Setting; School Funding. The Extra Mile project, aimed at disadvantaged children from EYFS to KS4, is also to be rolled out as a pilot, before it is evaluated. Echoing the model expanded upon in John Coles' presentation (i.e. understanding the context; developing the options; getting to a decision; and making it happen), the ‘DCSF's making policy model' was here reiterated, as: establishing rationale; testing options; securing delivery; and evaluating impact. Again focussing on FSM, and subsequently also on ethnicity, the second half of the presentation looked specifically at LSYPE as a way of deconstructing the gaps. Key messages were that gaps are not down to a single problem, and that just targeting schools will not have a huge impact. Also highlighted was that there are "differences between problems linked with progress and problems linked with raw attainment" . Amongst the next steps identified was the development of a 0-19 NtG Strategy.
Danielle Mason (Research and Measurement Team, Child Poverty Unit), Natalie Abbott (Bill Team Leader, Child Poverty Unit) and Laura Adelman (Head of Strategy and Analysis, Child Poverty Unit) led a stimulating interactive workshop describing how they defined child poverty and set consequent policy targets informed by evidence, and guided the legislation through parliament.
The Child Poverty Bill will put into into law the Government's commitment to ‘eradicate child poverty by 2020'. This workshop highlighted the analytical work undertaken to identify the indicators and establish levels of the targets for inclusion in the Bill, and the key policy areas that need to be considered in the development of child poverty strategies. They noted how the research evidence on single parent families changed some current preconceptions during this process.
Bilal Nasim from the Centre for Market and Public Organisation, University of Bristol, provided a wide ranging consideration of measures of educational success that go beyond achievement located in test scores and exam results (presentation not yet available). He demonstrated that Every Child Matters (ECM) outcomes are highly complementary (i.e. there is a certain amount of joint production, as with educational attainment and being on a path to success) and others are less so. In order to maximise policy effectiveness and synergy, there is a need to understand the extent to which the determinants of outcomes are interdependent and are linked over the course of childhood and across generations. One intriguing finding was that a child's adult income is more dependent upon their parent's income than in recent times, indicating that social mobility remains an issue.
Richard Bartholomew, Chief Research Officer at the DCSF, then introduced the panel for the plenary discussion . This consisted of Carole Willis, Paul Johnson (Centre for Analysis of Youth Transitions), Lesley Longstone (Director General of the DCSF's Young People's Directorate), Andreas Cebulla (Centre for Understanding Behaviour Change), Ann Phoenix (Childhood Well-being Research Centre), and Professor Ian Sanderson from Leeds Metropolitan University, who delivered the plenary address. Within this, he made seven 'challenging propositions' regarding the development of research-based policy, which were designed to be somewhat controversial in order to provoke discussion. These stressed the need for: a clearer understanding of the way evidence is used, and of the important role of analysts and scientific advisors; exposing misconceptions about the extent to which the work of government is informed by evidence; accepting that research evidence can rarely provide the definitive word, and that the policy-making process can sometimes appear to be research-averse; the research community to become less equivocal about research impact; and, finally, greater transparency as the key for intelligent government. This address is available to listen to below.
Professor Sanderson's address:
The panel discussion which ensued allowed for responses to the address, as well as answers to questions from delegates. Notwithstanding Professor Sanderson's intent to be controversial, the panel members by and large agreed with the points raised in his address, and tended to elaborate upon rather than challenge these. The importance of building relationships in developing and implementing policy was stressed, as was 'co-production', specifically with regard to the three new research centres. These signified a move to finding new ways of working with government departments, drawing together understandings and building up a research profile, which would involve blue-sky research too. However, Paul Johnson suggested that, for education, there are problems in terms of relationships, and that less 'intervention' here would be appropriate: "The role of external research played in both challenging and holding to account, but also supporting and helping government in a public way, is less than the sum of the parts....There's too much of the ding-dong in the media between the department and teacher unions in an unhelpful way" . The nature and quantity of evidence/data was discussed at length, and the point was made that social policy relied on a 'drip drip' build-up of evidence over time, and that most policies are incremental. The Policy Leads are crucial here, and need to have a full understanding of both the 'policy customers' and the evidence base, despite not having the time to read the research reports themselves. Although there was general agreement about the need for transparency, it was accepted that policy is ultimately driven by political values and manifestos, and that it is politicians who make the decisions according to a range of factors - evidence being but one of these. This discussion is also available to listen to below.
Panel Discussion:
The day concluded with a wine reception, accompanied by music from students of the Royal College of Music, which provided valuable opportunities for further discussion and networking.
Report written by: Mike Blamires & Sue Field
