Standards Reviews – A Summary
Tuesday, 01 May 2012 09:26
We have completed work on seven standards reviews. We compared the standards of qualifications across years in the following subjects. In some cases the two qualifications were based on the same specification; in other cases there was a specification change in the meantime.
What we found
Many of these reviews raise concerns about the maintenance of standards, and as qualifications regulator we will consider what wider lessons we need to learn from these findings.
In the GCSEs we reviewed (biology, chemistry and mathematics) we found that changes to the structure of the assessments, rather than changes to the content, reduced the demand of some qualifications. These qualifications have since been replaced with revised specifications. In the case of the sciences, where the new specifications have been used since September 2011, these new qualifications were designed to be more demanding.
In general we found that changes to the way the content was assessed had an impact on demand, in many cases reducing it.
In two of the reviews (biology and chemistry) the specifications were the same for both years. We found that the demand in 2008 was lower than in 2003, usually because the structure of the assessments had changed. Often there were more short answer, structured questions. As a result, students did not have as many opportunities to show their higher order skills in 2008. These specifications have since been replaced.
In geography there were changes to the specifications between 2001 and 2010. Changes to the content meant that some specifications had less scientific content in 2010. We found that the removal of coursework in 2010 reduced the overall demand of the qualification and may have meant students were less well prepared for higher education.
We also reviewed critical thinking qualifications for a single year (2010). The qualifications are offered by only two awarding organisations. We found some variation in the content of the two specifications but overall we found that the qualifications were comparable.
GCSEs will be revised following the National Curriculum Review in England and A levels will also be revised in the near future. We will use the findings from these reviews to inform the development of regulations for those new qualifications.
As part of our work to maintain standards, we carry out regular reviews to look at the standards of qualifications in different years. We aim to judge whether standards have been maintained over time and to compare standards between awarding organisations.
We use the findings from these reviews to inform our wider work, particularly when we are developing regulations for future qualifications.
Standards reviews look at different specifications within a qualification, the question papers and any other assessments, as well as student work, and we collate and analyse the views of a number of subject specialists. We focus on the relative demand of the qualifications. We judge demand in terms of the following.
- specification factors, including assessment objectives, content and structure
- assessment factors, including what content is assessed and how, the weighting of each component and how the assessments are marked
- student performance factors, including how students at particular grades responded to the assessments
We collect materials on a regular basis from the five awarding organisations (AQA, CCEA, Edexcel, OCR and WJEC) where they offer the qualifications being reviewed. Not all awarding organisations offer all subjects. Where an awarding organisation offers more than one specification in a subject, we collect materials for the largest entry specification. The materials we look at include the specification, question papers and mark schemes, reports of examiners, student work and statistical evidence. We use subject experts to carry out the reviews. These subject experts are independent but we also include representatives from the awarding organisations and subject associations/learned societies.