Article of the Month March 2011

 

Paresh.jpgNicola DaveyNicola Davey, Senior Associate and Programme Manager for Advanced Improvement Capability and Paresh Dawda, GP and Teaching Faculty for Safer Care at the NHS Institute for Innovation and Improvement have selected recent publications on the evaluation of the Health Foundation’s Safer Patients Initiative.  There are several publications related to the programme and accompanying papers in the BMJ.

 

 

 

In addition, there is a report on the learning from the programme by the Health Foundation and an accompanying editorial in the BMJ.

 

The Safer Patients Initiative

The Safer Patients Initiative (SPI) was a large scale multi-component intervention across a number of hospitals initiated in 2004. Funded by the Health Foundation it was delivered in conjunction with the Institute for Healthcare Improvement (IHI).  It was designed to improve patient safety with the general aim being “to avoid unnecessary harm, pain or suffering as a result of error in medical interventions.” A second phase (SPI2) followed in 2006.

In 2004, SPI 1 was undertaken by four NHS hospitals in the UK. It included interventions to improve processes of care across four clinical areas (general ward care, critical care, peri-operative care and medicines management). It also included a leadership intervention designed to strengthen the role of the executive team in improving the patient safety culture.  In 2006, a further 20 hospitals were recruited across the UK marking the second phase, SPI2.  Hospitals from SPI1 moved to an exemplar status during SPI2.  The hospitals in SPI2 received 50% less financial support from the Health Foundation and the overall aim was to reduce the adverse event rate by 30% (as opposed to 50% in SPI 1) and reduce mortality rates by 15%.

In addition to an internal evaluation drawing on data reported from the sites themselves, the Health Foundation commissioned an independent outcome focused evaluation addressing broader questions.  The latter used  a series of sub studies that included staff survey, interviews with senior staff, qualitative studies examining the impact of SPI on practitioners at ward level and  patient satisfaction measures. It also included metrics to measure quality of respiratory care in an acute medical ward, adverse event rates and mortality. Qualitative and quantitative information was gathered from the SPI sites and from 18 other control hospitals.

Of the 18 controls for SPI1, nine became SPI2 sites and nine continued as controls for SPI2. In the second phase only quantitative data was collected, and some interventions and measures were changed or amended. Additional measures included the indirect evaluation of hand hygiene and rates of hospital acquired infection.

In the outcome focused evaluation SPI 1 sites demonstrated improvement in the monitoring of vital signs and one measure of staff perceptions of organisational climate when compared to control sites.  Senior leadership and executives demonstrated enthusiasm about the programme, but at ward level there was less evidence of engagement.   However, the internal report provides evidence that all sites achieved the aim of reducing the adverse event rate by 50% over the 2 years of the study. In addition, all four sites demonstrated an improvement in process and outcome measures including ventilator associated pneumonia rates, central line catheter bloodstream infection rate, anticoagulation adverse drug event rate, crash call rate and surgical site infection rate.

The outcome based evaluation of SPI2 showed an improvement across both the control and intervention group. On the measures evaluated in the SPI2 study  sites in the intervention group did not demonstrate a significant improvement compared to the control group.  The internal report showed that improvement was made in critical care where almost three quarters of the participating trusts achieved their aim or exceeded it.  Across the 43 criteria being measured in SPI2, at the end of the programme all the sites showed an improvement in at least 50% of the criteria but only three of the 20 sites showed an improvement in more than 75% of the measures.

Many people, including participating sites, who have followed this initiative with interest have been disappointed at the lack of tangible results from the outcome based evaluation of the Safer Patients Initiative.

The BMJ editorial accompanying the two outcome based studies highlights some of the highs and the lows.  On the highs it states that “the study provides convincing evidence that safety and quality improved in NHS hospitals in England over the study period”, on the lows, it notes that ‘we still do not know why’. 
The authors of the two outcomes based evaluations, particularly in the discussions of SPI2, identified several possible reasons for not detecting greater significant differences between the intervention and control groups.  These include pre-evaluation diffusion of the interventions together with a series of other programmes such as “cleanyourhands” that were released at the same time as SPI was being implemented.  They do not mention other campaigns such as Patient Safety First (PSF) that ran concurrently for the last 6 months of SPI2, or the NHS Institute’s Leading Improvement in Patient Safety (LIPS) programme that ran for a year concurrently with SPI2.

The BMJ editorial picks up on weaknesses including the focus on interventions that were already highly reliable in the hospitals concerned (>80%), the focus on a large set of interventions (43), and the lack of evidence base to support some of the interventions, thereby undermining clinician buy in. It also emphasises the need to validate measures before broad implementation and to avoid as a default the top-down approach that was found in the qualitative evaluation.

The Editorial emphasises the need to extend the scientific approach to improvement in even greater measure than is the case right now. It commends the efforts of the Health Foundation and encourages us to learn from these evaluations to continue to strive for ever more robust evaluations.

The components of LIPS fulfil the recommendations in the editorial of a programme grounded in change theory, featuring evidence based interventions and designed and delivered through iterative collaboration with clinicians.  As it evolves it will continue to try and meet the challenges identified from the learning of the Safer Patients Initiative.