SR04 Efficiency Delivery: Assessing, reporting and signing off gains
Who should read this?
1. This document is designed for use by those people completing the data classification sheet within the turnaround documents each quarter. It works through each of the columns in the data classification sheet in turn.
How should this document be used?
2. This document is designed to help you apply the guidance on measurement, data maturity, classifications and systems assurance issued by HMT. It is a very high level summary and is based on the data classification sheet for the efficiency gains that you will complete each quarter. It contains a number of links through to detailed sections of the guidance to ensure that you can find the back-up information that you need. As well as those responsible for reporting numbers within your department, we recommend that you make it available to people responsible for cross checking your efficiency programme, be they auditors, peers or an external body.
Principles
3. Departments are responsible for validating their efficiency gains. This guidance contains some good practice advice to help you to do so
4. Exercise management judgement about what you claim. You should be prepared to defend your gains to Parliament and the public as robust and to show that they are efficiencies rather than cuts
5.HMT will not make a separate assessment of the robustness of the claims. We do however have a duty to advise the Prime Minister and the Chancellor if we think that a particular approach puts the credibility of the programme at risk. We will work closely with you to attempt to resolve any issues before this becomes necessary. We will support you in your implementation of this guidance
6. You should assess and report on your efficiency gains initiative by initiative. This is because different initiatives will have different levels of maturity, different quality evidence etc.
7. Different elements of an initiative may have different levels of data maturity. Initiatives can be split accordingly
8. We expect your gains to progress to higher categories for data maturity, classification and systems assurance as you move through each quarter.
Applying the guidance
9. Taking the example data classification sheet as an illustration the following provides an explanation of how to assess, report and sign-off your efficiency gains each quarter. Workforce and Lyons numbers should be reported separately. The data classification sheet is not applicable for these.
10. The objective is to classify each of your gains as either 'preliminary', 'interim' or 'final'. You should also classify them for systems assurance purposes, as having 'partial', 'substantial' or 'full assurance' at this time. These classifications are dependent on a number of factors, which we detail below. If you work through the spreadsheet columns from left to right, the final classifications will follow on logically from the information you have already input.
11. Please consider the main columns in the sheet and aim to complete these as follows.
Column C: Forecast m
12. Include the total gains that each efficiency initiative is expected to achieve in Financial Year 07/08.
Column D: Actual m
13. Include the actual delivered efficiency gain against the target/forecast in column C for the quarter being reported. Any gains from the previous quarter which are sustainable will persist, building cumulatively towards achieving the overall target of 21.5bn gains to be delivered within 2007/08.
PDF file of technical note with examples (37.5KB)
The document is available in Adobe Acrobat Portable Document Format (PDF). If you do not have Adobe Acrobat installed on your computer you can download the software free of charge from the Adobe website. For alternative ways to read PDF documents and further information on website accessibility visit the HM Treasury accessibility page.
HMT will publicly report actuals classified as interim or final against the Gershon targets. We recommend that you do not classify any efficiency gains into these categories that you cannot defend publicly. Management judgement should be a key guide as to what you report. We have no current requirement from our Ministers to publish preliminary gains except in aggregate (i.e. they will not be split by Department). However you should be aware that you may be required to release these figures if requested under FOI.
Columns E and F: Measurement methodology agreed incl. baseline
14. These two columns refer specifically to the way that you are calculating the efficiency gain only. Other parts of the data classification sheet refer to the quality measures, metrics and assessments that you have in place for each of your efficiency initiatives.
15. Separate out the amount entered in column D according to:
16. (Column E 'Yes') the proportion of the amount reported against an agreed methodology including a baseline and which has an auditable trail of evidence; and
17. (Column F 'No') the amount reported against a developing methodology, a methodology that has otherwise not been agreed between the department and the HMT, or the amount reported against an agreed methodology but not baselined.
18. For the most part the figure in columns D and Column E 'Yes' will be the same, since most methodologies and baselines are agreed between the department and the HMT. Examples of figures to be entered in column F 'No' are:
19. Where you might be reporting savings prior to the methodology being finally agreed. In this case you should use your management judgement to ensure that these actuals can be justified. For example, you may have an agreed set of principles but be dealing with outstanding technical issues that you do not think will impact significantly on the final methodology;
20. Reporting a gain that is against an agreed methodology but a baseline is not yet established;
21. When you are reporting a gain that you believe to be an efficiency but which is not in your efficiency plan and where you are seeking guidance from your own efficiency programme team and/or HMT (e.g. a contingency line).
Columns G to I: Data maturity
22. This part of the return relates to the efficiency gains being reported only and not to the status of any quality measures/assessments that you are using to confirm that gains are in fact efficiencies and not cuts in service. However, we suggest you exercise management judgement in what you report should there be risks to quality such that some figures might not be counted because you cannot demonstrate that they are cuts rather than efficiencies. There will often be a cyclical pattern to the maturity of data being reported to HMT, with some data being categorised as 'May change' in Quarters 1-3 where year-end adjustment is needed and then maturing to the 'Will not change' category in Quarter 4. Separate the amount entered in column D according to amounts:
23. 'Likely to change' - the figure is an estimate based on a lead indicator and is highly likely to change (e.g. gains that are likely to be replaced in future by a more certain figure from a cross-checking method);
24. 'May change' - the figure provides a good indication of the total gain but may change, for example it may be subject to year-end adjustments;
25. 'Will not change' - the figure is generated by your internal data systems and any planned year-end adjustments have been made - these data are not expected to change further (unless, for example, audit identifies an error, in which case you should report an exceptional item).
Columns J to M: Quality (assessing service quality)
26. You need to be assured that you have the means to demonstrate that the initiatives in your efficiency programme have not compromised the quality of service delivered. Such means can include a combination of a narrative, evidence based management judgement, and where relevant reference to detailed metrics.
27. The data classification sheet includes a departmental self-assessment of the maintenance of quality in those areas in which it is delivering efficiencies in SR04. This assessment includes two elements:
28. (i) the presence (or not) of an agreed means of demonstrating service quality has been maintained against which efficiencies are cross-checked and
29. (ii) an assessment of whether or not you can demonstrate that service quality has been maintained for the relevant period using this means.
Quality measure agreed (columns J and K)
30. As with the columns E/F entry, separate out the amount in column D according to:
31. (column J 'Yes') the proportion of the amount reported against agreed quality measure(s) OR if the initiative is scoring efficiencies from service quality improvements then you should put all of the amount in column D into column J;
33. For the most part the figure in columns D and column J 'Yes' will be the same. Examples of where you might enter figures in column K 'No' include:
34. The amount reported against quality measure(s) that are still in development or that have otherwise not been agreed with HMT;
35. The amount reported against agreed quality measure(s) which still needs to be baselined; or
36. An amount reported where you are confident quality has been maintained but not yet measured, e.g. a mid-year report where you are awaiting a year-end cross check.
Quality maintained (columns L and M)
37. Separate out the amount in column D according to:
38. (column L 'Yes') the portion of the amount reported against an agreed means of demonstrating service quality OR if the initiative is scoring efficiencies from service quality improvements then you should put all of the amount in column D into column L;
39. (column M 'Not yet demonstrated') the portion of amount reported where quality cannot yet be assured.
40. Examples of efficiency gains to be put in column M 'Not yet demonstrated' are:
41. If an efficiency initiative is quality checked with a single metric that has fallen across the reporting period (it is expected that it will pick up in later reporting periods and the gains would then move to Quality maintained 'Yes');
42. If a basket of indicators is used and there has been movement up and down by indicator and an overall assessment of service quality has yet to be made;
43. Where a quality cross check relies on a survey/data system that has yet to produce final figures
44. Where your management judgement is that there is some risk to the quality assessment at this time and so it is prudent to report as 'Not yet demonstrated'.
45. It is possible that within a single efficiency initiative part of the claimed gain has been quality assessed and part has yet to be. In this case the principle is that you should score as 'Yes' the proportion of amount reported against a proven quality statement and as 'Not yet demonstrated' the proportion where quality cannot yet be demonstrated. It is possible that next quarter the quality assessment will have been made and assuming it is positive then this portion of the previous quarter's gain will move to column L ('Yes' quality maintained).
Columns N to P: Data classifications
46. Separate out the amount in column D according to the Route map in the 'Guidance for signing-off SR04 efficiency gains'. How you complete these columns therefore depends on how preceding columns have been filled in. Remember, it is possible that parts of the gains reported within an individual initiative might be classified differently:
47. Preliminary - if a gain is reported where no methodology or baseline is in place and/or the result is likely to change;
48. Interim - if a gain is reported where a methodology and baseline exist and the result may change, or will not change but quality is not evidenced/auditable;
49. Final - only when a gain is reported against an agreed methodology and baseline, the result will not change and the quality is evidenced/auditable.
Columns Q to S: Data systems assurance
50. You should classify each of your gains in Column D as full, substantial or partial assured. Click on the links for detailed guidance. Also see the five criteria below that drive these categories. To classify items as full, substantial and partial, you must evaluate your gains against five criteria as set out below:
51. Internal controls: You need to have procedures in place to ensure the data quality of reported gains
52. Risk and issues management: You need to ensure that you have in place a system for identifying and managing risks and issues to data quality
53. Accountability: Refers to the accountability for the quality of reported gains at all levels of the delivery chain
54. Governance systems: You should have a governance structure into which assurance mechanisms can report
55. Audit trails: You are responsible for ensuring that sufficient appropriate auditable evidence exists and is retained to support reported efficiency gains.
56. Ideally, you should aim for full assurance for all initiatives especially if robust data systems are already in place.
57. In cases where data systems are not robust or initiatives are small in value, and where achieving full assurance would not be cost effective, please refer to Table 1 below, which provides indicative guidance as to the target level of assurance for each of your gains. As a general rule, the larger the initiative, the greater the level of assurance required.
58. Finally, if an initiative is novel and contentious even if of relatively small value, it is likely that greater assurance will be required that implied simply by value. An example would be an initiative that carries a high level of political risk.
59. Table 1: Relative Size of Gains and Associated Assurance Levels
 | Full assurance | Substantial assurance | Partial assurance |
| Departments should aim for Full Assurance for all large initiatives | Further work may be needed to improve systems assurance | Further work may be needed to improve systems assurance | |
| Departments should aim for Full Assurance if initiative is high profile, complex, or key to other deliveries (i.e. medium to high misstatement risk) | Substantial assurance is acceptable for small initiatives assessed as low risk | Further work may be needed to improve systems assurance | |
| Full assurance is ideal, although cost of gaining assurance should be proportionate to size of gain | Substantial assurance is acceptable for very small initiatives assessed as low to medium risk | Partial assurance is acceptable for very small initiatives assessed as low risk | |
 | |||
| Ideal | Acceptable depending on risk of misstatement | May require further work | |
Column T: Amount cashable
60. Enter the amount of gains reported to date that is cashable.