Previous work on the reliability programme
Last updated on: 15/09/2009
This page is an archive of previous work on the reliability programme.
Reliability programme takes a step forward
The reliability of results programme has taken a step forward after Ofqual commissioned research bodies to work on three projects.
The two year programme, launched in May last year, will look at all aspects of the reliability of exam results, or how different circumstances might lead to a student getting a different grade.
The first project will focus on generating partial estimates of reliability. Researchers working on this project will document evidence on reliability and look at comparisons of estimates of reliability across assessments. This work will provide new information on test-related reliability, marking-related reliability and grading-related reliability.
The second project will centre on gathering evidence of reliability and investigating how of different sources of error might affect the reliability of results. This could include occasion-related error, test-related error, marking-related error and grading-related error, impact on the reliability of results.
The third project will involve compiling information on how results are reported internationally, as well as gathering research on reliability collected by awarding bodies. This project will produce insights into different approaches to reporting results.
All three projects will look at a range of assessments. These could include national tests, public examinations and vocational/occupational qualifications within England, Wales and Northern Ireland.
The successful research teams will be announced shortly, when contracts have been signed.
For more information or to request specifications, email reliabilityprogramme@ofqual.gov.uk
Background
How much error is acceptable in assessments? If the same student took an exam on two different days, would the same mark be awarded? If exam results are perfectly reliable, variables such as the place of assessment, markers and standard setting panels should not have an effect – but to what extent do they?
Ofqual's Reliability Programme aims to open the debate on these issues. Taking place over two years, a team of researchers will look at evidence of the consistency of assessments, while members of the public, including students, parents and employers, will be consulted to determine their perception of reliability in examinations and assessments.
Ofqual Chair Kathleen Tattersall said:
“There’s a broad expectation that assessment should be absolutely perfect and accurate, that a mark of 50 is a mark of 50, regardless of who marks, the time at which it is marked and so on. We need to explore whether that sort of expectation is well founded, or whether within the system there are some trade-offs between absolute reliability of that nature and the validity of the way in which we go about assessment.”
Members of the project's technical advisory group have been appointed and their profiles are available in this section.
If you have any enquiries please contact reliabilityprogramme@ofqual.gov.uk