Department of Health Skip to content

Please note that this website has a UK government access keys system.

Good practice guide: Learning lessons from post-project evaluation

  • Last modified date:
    9 February 2007

Executive summary.

Further details from:
Private Finance Unit
Room 3W54, Quarry House
Quarry Hill
Leeds LS2 7UE
Telephone - 0113 254 5420

Purpose and audience

Evaluation is the process of assessing the impact of a project, programme or policy while it is in operation, or after it has come to an end. It is an essential aid to improving project performance, achieving best value for money from public resources, improving decision-making and learning lessons.

The advice offered in this guidance is aimed at all NHS bodies and advisers who are involved in the development of capital schemes, regardless of whether they are publicly or privately funded. These include senior managers, particularly project directors and managers, in NHS Trusts, commissioning Health Authorities and Primary Care Groups. Chief Executives in these organisations are required to take full responsibility for the management of all stages of capital projects.

The guidance is self-standing, and is designed to complement Public Private Partnerships in the National Health Service: The Private Finance Initiative. It updates and replaces the guidance on Post-project Evaluation which was issued in 1994 as part of the Capital Investment Manual.

Policy requirement

Sponsors of capital projects in the NHS are required by the Department of Health, HM Treasury, and the National Audit Office to evaluate and learn from their projects. This is mandatory for projects with a cost in excess of £1 million.

In the case of capital projects prioritised by the Capital Prioritisation Advisory Group (capital cost over £20 million) and major Information Management and Technology projects (whole life cost over £20 million), it is mandatory for the completed post-project evaluation (PPE) report to be submitted to the Department of Health.

For schemes below this threshold, the report should be disseminated to the Strategic Health Authority and other key stakeholders within the local health economy.

Business cases for capital projects will not be approved unless post-project evaluation has been properly planned in advance and suitably incorporated into the Full Business Case.

Administrative arrangements

The main stakeholders for the completed evaluation are senior managers in Department of Health (PFI Branch), HM Treasury, Strategic Health Authorities, Primary Care Groups/Trusts, Trust Board, and other relevant stakeholders within the local health economy.

Four copies of completed documents for the major capital and IT projects should be sent direct to Peter Cockett, room 3W54, Quarry House, Quarry Hill, Leeds, LS2 7UE. PFI Branch (Leeds) will take responsibility for disseminating the document to HM Treasury and other interested parties.

Prerequisites for successful evaluation

To ensure maximum pay-off from evaluation, it is important to:

  • View the evaluation as an integral part of the project and plan for it at the outset. The evaluation should be costed and resourced as part of the project.
  • Secure commitment from senior managers within the organisation. Chief Executives will be expected to take full responsibility for the management of all stages of a capital project.
  • Involve all key stakeholders in its planning and execution. For large projects (over £20m) many organisations find it useful to set up an Evaluation Group at the inception of the project.
  • Develop relevant criteria and indicators to assess project outcomes from the outset of the project.
  • Put in place mechanisms to enable monitoring and measurement of progress.
  • Foster a learning environment to ensure lessons are heeded.

The stages of evaluation: when should evaluation be undertaken?

Although evaluation may be carried out continuously throughout the life of a project to identify opportunities for continuous improvement, as an absolute minimum, project sponsors are required to undertake evaluation activities at four main stages:

Stage 1: plan and cost the scope of the PPE work at the project appraisal stage. This should be summarised in an Evaluation Plan.

Stage 2: monitor progress and evaluate the project outputs on completion of the facility.

Stage 3: initial post-project evaluation of the service outcomes six to 12 months after the facility has been commissioned.

Stage 4: follow-up post-project evaluation to assess longer-term service outcomes two years after the facility has been commissioned.

Beyond this period, outcomes should continue to be monitored. It may be appropriate to draw on this monitoring information to undertake further evaluation after each market testing or benchmarking exercise - typically at intervals of 5-7 years.

At each of these stages, evaluation will focus on different issues. In the early stages, emphasis will be on formative issues. In later stages, the main focus will be on summative or outcome issues. The guidance provides detailed advice on the type of issues to address at each stage. It also considers the advantages and disadvantages of using in-house and external contractors for undertaking the work.

How to evaluate: some technical considerations

It is recommended that the Logical Framework should continue to be used for evaluation of NHS capital schemes. This is a matrix listing project objectives against indicators and measures for assessing outcomes. The underlying assumptions and risks are also considered.

The technical issues arising from application of the Logical Framework are discussed in the guidance. This includes:

  • the merits and demerits of different data collection methods
  • the role of different participants in the data collection process
  • sampling methods
  • sample size
  • questionnaire design (types of questions, etc)
  • piloting
  • how to achieve a satisfactory response rate
  • security and confidentiality of data
  • data analysis and report writing

Feedback and dissemination of findings from evaluation

The potential value of an evaluation will only be realised when action is taken on the findings and recommendations emanating from it. Processes are needed to ensure that this happens.

Evaluation results should be signed off by senior management or at Board level.

The results from the evaluation should generally lead to recommendations for the benefit of the organisation and wider NHS.

These may include, for example, changes in procurement practice; delivery; or the continuation, modification, or replacement of the project, programme or policy. The results should be widely disseminated to staff concerned with future project design, planning, development, implementation, and management.

The approach and underlying principles adopted in the guidance, including the questionnaires listed in Annex 4, has been developed from practical trials at a sample of early PFI schemes. These questionnaires can be straightforwardly modified to evaluate capital projects in NHS organisations.

Access keys