ANNEX C: SUPPLEMENTARY GUIDANCE ON PREPARATION OF ECONOMIC APPRAISALS

A. INTRODUCTION

A1. This note is intended to provide summary guidance on the most important areas of economic analysis that need to be covered in economic appraisals forming part of round three bids for CMF funding.

A2. It is not intended as a replacement for the general guidance on preparing economic appraisals contained in the 'Green Book'. Rather, it is intended to provide a more focused guide to the essentials that will be considered in evaluating and scoring economic appraisals in the context of CMF bids.

A3. The Green Book was written to help government departments and agencies appraise and evaluate their activities effectively. It is intended to ensure consistency across government in appraisal and evaluation practice. Departments develop their own guidance which is focused on their own areas of work but which is consistent with the Green Book.

A4. The principles contained in the Green Book apply equally to local and central government. This note is based on the principles set out in the Green Book. These principles provide the basis for everyone undertaking an economic analysis as part of a bid for CMF funding.

A5. Although other forms of guidance can be used by local authorities or health authorities in preparing economic appraisals, we would expect these forms of guidance to be consistent with Green Book principles.

A6. The Green Book (full title; Appraisal and Evaluation in Central Government) is currently under review. An electronic version of the current Green Book is available below.

Paper copies can be obtained from The Stationary Office, Telephone orders: 0870-600-5522, Fax orders: 0870-600-5533

External Links
Adobe Acrobat Reader

Supporting Docs & Media
  Greenbook

A7. This guidance does not cover evaluation of projects. Although this is an important area for CMF bids to cover, it will be considered separately from the assessment of CMF economic appraisals.

B. PURPOSE OF ECONOMIC APPRAISAL

• to provide an assessment of the options available to fulfil the objectives of a project. Hence, the importance of identifying and analysing different options;
  
• to provide an analysis and comparison of the potential costs and benefits, the risks and uncertainties associated with the project. Hence, the assumptions on which the projected costs and benefits are based are crucial and must be tested (see sensitivity analysis below), and;
  
• to provide underlying analysis of the relative merits of the options to inform the decision on which is the best option.

C. KEY AREAS OF ANALYSIS

C1. With the above purposes in mind, six appraisal categories will be used in assessing economic appraisals. Guidance on each category is given below. Throughout the economic appraisal, the objectives identified in the bid will provide the foundation of the analysis.

1. Choice and Definition of Options
(See Green Book paras 2.8 - 2.10 and 4.2 - 4.11).

• Options are defined as alternative ways of achieving the objectives of the project. 
• The appropriate number of options to be considered depends on the size and scope of the project. 
• It will be important not to rule out any options until they have been given due consideration.     
• However, it will usually be the case that only a small number of options require full-scale analysis. 
• If options are ruled out, sensible reasons need to be given. 
• First, a "do minimum" option should be constructed. In previous CMF bids where a "do nothing" option has been considered, it has often been dismissed as not viable politically, or as not meeting objectives. However, the real question is "what are the costs and benefits of an option that involves doing the minimum necessary to meet the most important objectives of the project". 
• Examples of ways in which options may differ are; (i) the scale of the project: it may be useful to have an option that identifies ways in which the project could be scaled down. This would be useful in helping to identify which parts of the project are most important and which are more marginal; (ii) the timing and/or phasing in of the project, and; (iii) the extent of private sector involvement.

Checklist

• Is the range of options being considered wide enough? 
• Are the reasons for dropping options reasonable? 
• Has any potentially promising option been ruled out?

back to top

2. Identification of Benefits

• All benefits resulting from the project should be identified. Benefits should be quantified where possible. 
• Where this is not possible, benefits should be qualitatively identified and discussed. 
• Where benefits can be quantified, the crucial criteria are that;
  • the scale of projected benefits should be based on realistic assumptions;
  • the assumptions should be stated, and;
  • the key assumptions should be tested as part of the analysis in order to determine the impact of changes to them on the overall benefits and costs of the project (see sensitivity analysis below).
• The credibility of positive net present value calculations (see net present value calculations below) for different options will be undermined if the assumptions on which projected costs and benefits are based are not given, have not been tested or are not realistic.

Checklist

• Have all the benefits been identified and profiled? 
• Have the benefits been quantified where possible? 
• Are the values of the benefits based on realistic assumptions?

3. Identification of Costs

• All expected costs resulting from the project should be identified and quantified where possible. 
• If any costs cannot be valued in money terms, or cannot easily be quantified, the reasons should be explained. 
• Bidders should also endeavour to give some weighting to these unquantified costs which reflects their importance and which will allow more accurate comparisons to be made between options. 
• Where costs can be quantified, the crucial criteria are that;
  • the scale of projected costs should be based on realistic assumptions;
  • the assumptions should be stated, and;
  • the key assumptions should be tested as part of the analysis in order to determine the impact of changes to them on the overall costs of the project (see sensitivity analysis below).
• The credibility of positive net present value calculations (see net present value calculations below) for different options will be undermined if the assumptions on which projected costs and benefits are based are not given, have not been tested or are not realistic.

Checklist

• Have all the costs been identified and profiled?  
• Have the costs been quantified where possible? 
• Are the values of the costs based on realistic assumptions?

back to top

4. Cost Benefit Analysis

(Green Book paras 4.12 - 4.42)

• Cost benefit analysis (CBA) can be defined as a way of seeking to identify and value in money terms as many of the costs and benefits of a proposal as possible. 
• This encompasses the costs and benefits associated with each of the options which would fulfil the objectives (or main objectives) of the project. 
• The number of options subjected to CBA and the time and effort employed in doing so will vary according to the overall size of the project. 
• No option should be dismissed without due consideration, but, usually, only a small number of options will require full scale analysis. 
• Costs that have already been incurred (or funds which have already been irrevocably committed to the project) should not be included in CBA. For example, CMF funding received for a pilot project in the previous round should not be included.

b) Net Present Value or Discounted Cash Flow Calculations

(Green Book paras 4.52 - 4.63)

• The purpose of doing net present value (NPV) or discounted cash flow (DCF) calculations for each of the main options is to allow comparison of the options and identify which yields the greatest net benefits (or is the least costly). 
• It is essential therefore to present NPVs for each of the options being considered in order that these can be compared and the option yielding greatest net benefits identified. 
• A worked example of how to calculate NPVs follows.

Checklist

• Have the stream of costs and benefits for a number of options been identified and discounted (an explanation of discounting follows)? 
• Are the costs and benefits based on realistic assumptions? 
• Have the bases for assumptions been stated and have the assumptions been tested (see sensitivity analysis below)?

c) Cost Effectiveness Analysis

• Where the identified objectives of the project preclude valuation of benefits in money terms, the options may be considered in terms of a cost effectiveness analysis. 
• For example, the objective may be delivery of a health care service by "virtual" means rather than by conventional means. Since health cannot be easily valued it may be more appropriate to set the required level of service provision and compare the net present costs (NPCs) of the different options being considered for the delivery of this service level. The option yielding the lowest NPC is cost effective.

d) Choice between presenting NPV and DCF Calculations

• Where it is possible, NPV calculations should be presented in preference to DCF calculations. 
• In theory, NPVs will provide a more thorough analysis because NPCs simply assume that the benefits outweigh the costs even if this may not be the case. 
• However, reliance on NPVs can be misleading in cases where the majority of benefits are of a qualitative nature. 
• In these circumstances, NPV calculations can still be produced (provided that there are some quantified benefits). 
• It will be particularly important in such cases for the benefits not included in the calculations to be backed up by supporting qualitative information. 
• Where a cost effectiveness analysis is used, explanation should be given as to why benefits are assumed to outweigh costs.

back to top

5. Analysis of Risk and Uncertainty

(Green Book 4.43 - 4.51)

• There will usually be uncertainty about some aspects of a project (for example, projected costs and/or savings may not be of the size they were forecast to be). 
• There may also be a risk that circumstances surrounding a project will alter and result in material impact on the project (for example, legislative changes may occur which affect the workload of an organisation). 
• Risk and uncertainty should be explicitly considered as part of an economic appraisal. For example, it will be important to test the assumptions made about costs and savings in order to gauge the impact of changes in these assumptions on overall returns to a project. The best way of handling this will often be to use sensitivity analysis.

back to top

a) Sensitivity Analysis

(Green Book 4.48 - 4.51)

• Sensitivity analysis is the process of gauging the impact of different assumptions (about costs and savings) on overall project returns and costs. 
• Sensitivity analysis will help to demonstrate the dependence of project returns and costs on key assumptions. 
• For example, a key assumption might be that the project will result in a saving of £30,000 in staff costs in each of the next three years. 
• Sensitivity analysis asks how dependant overall project returns and costs are on this particular assumption (and tests the impact on the projected outcome if the savings were, for example, assumed to be only £15,000 per year). 
• If it is found that the overall outcome is very sensitive to changes in an assumption then it will be particularly important to examine the assumption to ensure that it is as reliable as possible. It will also be important to set out fully the basis of the assumption. 
• Sensitivity analysis should also include subjective assessment of the likelihood of different scenarios (eg. the likelihood of £15,000 versus £30,000 staff savings). 
• Sensitivity analysis should also be used to assess the risks and uncertainties associated with factors that have not been valued in money terms. 
• It should be noted that alterations to key assumptions may involve both better and worse returns than the base case. For example, if the base case is £30,000 savings per year, alternatives based on different assumptions may yield savings of £15,000 or £45,000.

Checklist

• Have all important risks and uncertainties been identified and assessed for each option? 
• Has sensitivity analysis been used to assess their impact on project returns?

back to top

6. Presentation of Results

• Analyses should be set out clearly and comprehensively covering each of the above categories. 
• It should be made clear who will benefit from, and who will bear the cost of each option. 
• Although quality of presentation will be given less weight in the overall assessment of economic appraisals, it will be important for the analysis to be set out clearly for each of the five key areas discussed above. 
• A guide to the weighting to be given to different aspects of the appraisal follows.

D. CALCULATION OF NET PRESENT VALUE

(Green Book paras 4.52 - 4.63 and Annex H)

The following tables give an illustrative example of how to calculate net present values (NPVs) for different options that would fulfil the objectives of a project. This example is intended primarily to give an indication of the mechanics of calculating NPVs. The Green Book should be referred to for further detail. Explanatory notes are given below the tables.

TABLE 1

1. The discount factor is a means of reducing future net benefits to their value in the base year (Year 0).

Hence, cumulative NPV is £100,500 over four years.

TABLE 2

Hence, cumulative NPV is minus £86,000 over four years.

1,2 Costs and Benefits: these rows relate to the anticipated costs and benefits arising from the options. It can be seen that the options considered have different cost profiles and are expected to yield a different level of benefits. A clear breakdown of the elements making up these costs and benefits should be stated in the analysis, together with the underlying assumptions, and a sensitivity analysis of the key variables.

back to top

7. Net Benefits

The sum of the benefits less the sum of the costs in each year gives net benefits for the year in question. Net benefits should be calculated separately for each year (for each option).

8. Discount Factor
(Green Book paragraphs 4.52 - 4.63)

The discount factors given in the above tables are based on a discount rate of 6%. This is the real discount rate which is most commonly used in central Government applications. Exceptions to this general rule are explained in Annex G of the Green Book.

It is recommended that the first year in which the project will run is chosen as the base year, which is in effect the "current" year for the project. On this basis, the discount factor in the base year will be 1 (i.e. costs and benefits which accrue in this year do not need to be discounted further).

In practice, it is usually sufficiently accurate to treat all sums accruing during the course of a year as falling at mid-year.

back to top

9. Net Present Value

The discount factor for each year should be multiplied by the net benefits in that year to give a net present value (NPV) for each year of the project.

In cases where calculations are undertaken purely in terms of costs, the sum of the costs in each year should be multiplied by the discount factor to give the net present cost.

10. Cumulative Net Present Value

The cumulative NPV is the sum of the NPVs up to the year in question. For example, the cumulative NPV for option 1 is minus £319,500 over the first three years but plus £100,500 over the first four years.

This suggests that option 1 would break even during the fourth year. Hence, the cumulative NPV shows in which year the project breaks even. This can be a useful piece of information, particularly in the context of carrying out sensitivity analysis (i.e. changes to assumptions could alter the point in time at which the project breaks even, suggesting that it will be particularly important to ensure that these assumptions are as accurate as possible).

The costs and benefits of alternative options can be properly compared only if they cover the same time period. (See Green Book paragraphs 4.60 and 4.61 for further information).

E. WEIGHTING OF THE ELEMENTS OF ECONOMIC APPRAISAL

• The three categories of economic appraisal which will be given greatest (and equal) weight in the assessment of individual bids are analysis of costs, analysis of benefits and cost benefit analysis.
  
• These elements of appraisal are of central importance. It is crucial accurately to identify the extent and range of costs and benefits and to base projected costs and benefits on realistic and tested assumptions (as described above).
  
• This does not mean that a bid which presents only a cost effectiveness analysis with DCF calculations (as opposed to a full cost benefit analysis with NPVs) will be disadvantaged by so doing.
  
• The quality of the analysis following the guidance set out above will be the basis for scoring in each section.          
  
• The analyses of options and of risk/uncertainty will each be given equal but slightly lower weight than the analyses of costs and benefits.
  
• This is because high quality analyses of options and risks/uncertainty would be outweighed by low quality analyses of costs and benefits.
  
• For example, a wide range of options could be identified but the analysis of the costs/benefits associated with each could be of a low quality (eg. lack of information about the assumptions on which cost savings are based). This would undermine the usefulness of the analysis of options to a greater extent than a thorough analysis of costs and benefits for a narrow range of options. Presentation of results will be given least weight in the assessment of appraisals.
  
• The most important thing is the quality of the analysis but good presentation will help to make this quality clear.

back to top

Capital Modernisation Fund Bidding Guidance

Capital Modernisation Fund index page