NHS Connecting for Health ceased to exist on 31st March 2013. This website is therefore not being updated. For up to date information about systems and services visit the Health and Social Care Information Centre website at www.hscic.gov.uk/systems

You are here: Home Services & Applications NHS Business Partners Common Assurance Process FAQs

Common Assurance Process FAQs

A set of frequently asked questions around the Common Assurance Process (CAP) for the NHS Business Partners Programme.

Jump to a section:

Back to top

Common Assurance Process (CAP)

What is CAP?

The Common Assurance Process (CAP) is a generic, end-to-end process for assuring the development and delivery of high quality and clinically-safe IT systems from suppliers not contracted to Local System Providers (LSPs) or National Application System Providers (NASPs).

It is based on industry best practice and provides a common, standard way by which the quality of services delivered jointly by NHS Connecting for Health (NHS CFH) and third-party IT systems suppliers can be assured.

As such, CAP provides a clear basis for the ways of working for specialist teams within NHS CFH, as well as between NHS CFH and IT suppliers. It also provides a clear way of involving end users of suppliers' systems in the assurance process – essential to ensuring a high-quality service.

Where has CAP come from?

The Common Assurance Process (CAP) replaced the 'NHS CFH Compliance' scheme, initiated in 2004 to support connection of third-party IT systems to the Spine. It also replaced the Requirements for Accreditation Scheme (RFA) for GP systems, previously operated by the NHS.

Whilst CAP builds upon these two schemes it also encompasses the key principles of the LSP and NASP assurance approaches to provide a common standard for quality and safety.

Why do we need CAP?

NHS CFH has an obligation to ensure that the Spine and other systems which are currently being used in live service, such as Choose and Book, Summary Care Record, Electronic Transfer of Prescriptions etc, which are relied upon by hospitals and in care settings across England remain able to effectively and safely continue to provide their services. CAP is designed to ensure that no detriment is introduced to existing services when additional suppliers connect to the NHS CFH infrastructure.

CAP is also applied to assess the safety and security implications of a given release into a defined care setting. As such, it is used to mitigate risks inherent with business and IT change by managing the introduction of new services in a controlled, systematic and repeatable manner.

How does CAP work?

The Common Assurance Process is split into phases, each made up of a number of stages, logically breaking down the software development and implementation process.

Navigation through CAP is based upon the scope and complexity of the supplier release being assured. As such, as a minimum, the scope stage will apply to all supplier releases. The application of subsequent CAP stages will depend on the scope and risk associated with a given supplier release.

Release Managers across NHS Connecting for Health (NHS CFH) have tailored CAP to meet their Programme or supplier requirements.

Further tailoring of these approaches to a supplier's own development and deployment methodology takes place during the scope stage, where a given supplier maps their own development methodology onto the CAP Approach. This will ensure that NHS CFH assurance requirements are met whilst minimising the impact on a supplier's established processes.

Are there standard templates for CAP deliverables?

Yes, there are templates available for the majority of deliverables and where there are no templates there are quality review criteria available to help with appropriate content. These will be provided by the NHS Business Partners team as you go through CAP.

What are the supplier responsibilities during CAP?

A supplier’s key responsibility is to follow CAP to demonstrate compliance with a given set of NHS CFH requirements for a given release of their service. Key activities include:

  • Developing and agreeing the Scope and Plan for the relevant release and keeping it up to date in relation to the defined CAP Approach
  • Design, build, test, deployment and ongoing service management of the supplier’s product. This includes securing participation of users and the involvement of pilot sites in the assurance process though NHS CFH may work together with the Supplier to do this. The exception is Model Community testing which is co-ordinated by NHS CFH. Suppliers may need to support NHS CFH in delivering this
  • Providing evidence and participating in joint activities to support NHS CFH assurance activities e.g. documentation, witnessed tests, peer to peer reviews

What are the healthcare provider responsibilities during CAP?

The provider (e.g. Independent Sector Healthcare Provider, NHS Trust, Sponsor etc) takes overall responsibility for clinical safety and project managing the supplier. They can also contribute to the supplier CAP deliverables.

What are the NHS CFH responsibilities during CAP?

NHS CFH’s key responsibility is facilitation of the preparation and operation of CAP. Key NHS CFH activities include:

  • Creating and publishing the Requirements Baseline and Tailored CAP Approach for given Authority Requirements e.g. Choose and Book (CAB), Electronic Prescription Service (EPS) release 2, Personal Demographic Service (PDS), Information Governance (IG), Clinical Assessment Framework (CAF) messaging
  • Supporting supplier to plan and manage their release through CAP
  • Carrying out assurance activities e.g. documentation reviews, witness testing and co-ordinating input of other parties where relevant;
  • The award of Authority to Proceed (ATP), Development and Deployment Milestone Achievement Certificates (DevMACS and DepMACs), Full Rollout Approval (FRA) and, Clinical Authority to Release (CATR) certificat
  • Providing supporting materials e.g. documentation templates etc to Suppliers in a timely fashion

The key NHS CFH point of contact is the Release Manager who is responsible for working with the supplier to plan and manage specific supplier releases through all the applicable elements of CAP, and working with all internal stakeholders to ensure consistency and concurrence.

Back to top

Prerequisites

What are the NHS Business Partners team prerequisites?

The NHS Business Partners team will arrange for an introductory meeting to take place, and then ask for a project plan, a very high level business justification outlining expected benefits, confirmation of a clinical safety officer, an NHS sponsor and IGSoC compliance as part of their prerequisites. The team may also ask for you to look at the Release Scope Definition document template and ask for some key information in some of the sections to be completed. They will also provide you with further information on CAP and initiate the process.

Why do the Business Partners team need them?

In order for you to complete CAP as efficiently as possible, the team need to ensure that there are clear plans in place with clear time drivers, the supplier is committed to the project and that the process is understood before the formal CAP commences.

How do I complete the prerequisites?

A member of the NHS Business Partners team will outline what is required and provide a template for the Release Scope Definition document and project plan.

The team is looking for a realistic project plan to incorporate the CAP phases and an outline of the scope of the project, as well as what the drivers for the project are, any constraints and dependencies there might be.

What is the ‘Release Scope Definition’?

The Release Scope Definition (RSD) defines the agreed scope of a particular supplier’s release in response to either the set of NHS CFH requirements issued by the NHS CFH Programme or requirements set by another authority (e.g. Department of Health and the Information Centre) which will be assured in accordance with the tailored CAP Approach.

The RSD provides a consolidated picture of everything which is included in a particular release and gives a high level view of the deployment approach for the release. It also documents the mapping of CAP to the supplier’s release delivery methodology/documentation and summarises the supplier’s release management plan and arrangements.

When can I start the CAP?

Once the NHS Business Partners team is satisfied that the NHS CFH and team prerequisites have been met, they will ensue that assurance resources are made aware of the release plan to allow for capacity planning activities to commence. When dates have been impact assessed, resources lined up and the scope of work is understood, the NHS Business Partners team will notify you.

The provider has IGSoC, does the supplier also need IGSoC?

Yes, both the provider and supplier will need to have obtained IGSoC before they can start CAP.

What is a ‘First of Type’ site?

Before a supplier can roll out new software to the wider NHS, it must be trialled at a pilot site to ensure that it continues to be fit for purpose when used day to day. The pilot phase is called the Deployment Verification Period (DVP), during which time the system will be monitored in its first live environment. First of Type sites are healthcare providers who trial the new version of software that is released by their system supplier.

How is Clinical Safety assured throughout the CAP?

Clinical Safety assurance relates to the suppliers ability to demonstrate compliance (if applicable to the solution); to the NHS safety standard for Health IT Manufacturers – ISB 0129 (formerly DSCN14/2009).

http://www.isb.nhs.uk/documents/isb-0129

Compliance will ease the implementation with the Health Organisation according to the NHS safety standard for Health Organisations deploying and using the solution – ISB 0160 (formerly DSCN 18/2009).

http://www.isb.nhs.uk/documents/isb-0160

NHS CFH will allocate a clinical safety engineer to the project from the beginning, and will work with the supplier to assist in achieving compliance. The key activity for the provider is to identify key hazards with their implementation and how best to mitigate them. This will normally involve the submission of documentation at agreed points in the release lifecycle. This documentation will provide evidence that the mitigations to any hazard are appropriate. This evidence is mainly derived from assurance activities throughout the project.

What is a Clinical Safety Officer and do we need them from the beginning?

The Clinical Safety Officer (CSO) is a Health Care Professional, suitably trained in clinical risk management and registered with an appropriate professional body. Ideally the supplier should have a Health Care Professional who can perform this role. The health organisation that the solution will be deployed into must also have a CSO in place. This will ensure that both NHS safety standards are adhered to, and any clinical risks appropriately managed prior to use of the solution. Supplier documentation supporting compliance to the safety standards should highlight this clinical engagement.

NHS CFH Clinical Safety Group can provide assistance with CSO training if required.

How are CAP deliverables managed by NHS CFH?

CAP deliverables are circulated for review around a set of teams within NHS CFH according to a RACI matrix (Responsible, Accountable, Consulted and Informed). These teams include functional assurance, non functional assurance, deployment, clinical safety, training and the relevant programme teams. The RACI matrix details which teams will need to review the deliverables and their role as reviewers. The teams will raise comments on the document if they have any queries, and these will be passed back to the supplier for them to address in the next submission. The review cycles and timescales will be managed by the NHS CFH Release Manger, and their impact on the overall project plan. The project plans are essential for NHS CFH to allocate resources and schedule document reviews.

How do I obtain the baseline documentation?

All baseline documentation is stored on our document management system FileCM. Guidance on how to use and gain access to FileCM will be available from your Release Manager.

Are there support guides for CAP?

There are many guides available for suppliers to help with the process. Your Release Manager will provide you with any guidance you may require.

Back to top

CAP Phase 1: Scope

What is the purpose of the Scope phase?

  • To obtain agreement and shared understanding across all relevant parties for the specific supplier release in response to our requirements on:
  • The scope of the requirements to be includedRoles and responsibilities for delivery
  • The specific supplier mapping and plan for delivery of the relevant Tailored CAP Approach
  • The impact on current service management arrangements and clinical safety
  • The testing and deployment strategy

What deliverables are you looking for in the scope phase?

The Scope phase requires documentation to be provided will normally outline the scope of the project, testing and deployment and the initial Clinical Safety documentation to be produced, but full details of the required deliverables will be outlined in the tailored CAP Approach.

What is an ATP?

Each Supplier will be awarded an Authority to Proceed [ATP] certificate once the criterion for the Scope phase and the Design and System Test Phase have been met, and is necessary to ensure all internal stakeholders are in agreement that deliverables have been approved and no requirements for that stage are outstanding. The ATP is issued by the Release Manager once all activities for the phase/stage are complete and is circulated to all stakeholders to approve. The ATP certificate will then be awarded to the supplier. However, not receiving an ATP does not prevent work on the next stage being progressed.

Back to top

CAP Phase 2: Design and System Test

What is the purpose of this stage?

To assure:

  • The supplier release is designed in line with the relevant authority requirements
  • The supplier release is effectively tested as an overall system prior to integration testin
  • The clinical safety requirements needed to address the clinical safety hazards and risks are specified and implemented in the system

What is a ‘Requirements Traceability Matrix’?

A Requirements Traceability Matrix (RTM) is a spreadsheet which is used to capture whether the supplier is compliant, semi compliant or non compliant with NHS CFH requirements, where in the design documentation the compliance is demonstrated and how test scripts for both system and integration testing map to the requirements. It is a living document that will be continually updated in line with testing requirements, scripts and results.

What other design documents are you looking for?

For design, we would normally look for documents that cover the Technical Architecture, Logical Design, Functional Specifications and Non Functional Specifications of the system

However, the set deliverables will be defined in the tailored CAP Approach and RSD. They will be reviewed in parallel with the RTMs.

What deliverables are required for this testing phase?

These deliverables will be defined in the scope stage but will encompass a test plan, test scripts (including an updated RTM to demonstrate the mapping of test scripts to the requirements) and a test report to include an Online Message Validation Tool version 2 (OMVT2) test report. A Design and System Test Phase ATP will be award once all of these deliverables and requirements have been approved.

What is OMVT2?

The Online Message Validation Tool version 2 (OMVT2) is a tool developed by NHS CFH to validate messages against a set of tightened schemas and business rules. The supplier is requested to submit one example of each outgoing message and the corresponding OMVT2 report at the appropriate stages in CAP. It is recommended that an example of the most complex message is provided for this. As a minimum, this would normally be prior to entering the Integration environment and prior to the witness test commencing, but can be performed at anytime.

What test evidence do you need?

A system test report is required. The test report describes the testing conducted during the test stage. It outlines the outcome of those tests, including any outstanding issues, the impact of those issues and an action plan to resolve them. An updated Requirements Traceability Matrix will also be required.

What is the National Integration Sandpit?

The National Integration Sandpit (NIS) is a supported test environment that allows integration testing of supplier software releases. Within these environments, Functional and Non Functional Testing will be performed. NHS CFH has a variety of different sandpits to aid supplier development, system testing and UAT (User Acceptance Testing) but NIS1 is the sandpit used for the integration testing. The Sandpits are of a much smaller scale to the live environment but are designed to contain the same functionality.

How do I request access to the Sandpit?

The supplier must complete a supplier Sandpit Connection form, which will be provided by the Release Manager. The form will then need to be submitted to the ESP Compliance team. Access to other environments is controlled by the Environments team, and further guidance can be obtained from your Release Manager. Post DevMAC, the process will be slightly different to access the Sandpits and details will be provided on request.

What are the prerequisites for Sandpit?

An N3 connection and IGSoC must be in place for NIS1 access. An ATP for the Design and System Test phase must have been awarded for access into the National Integration Sandpit.

How long does it take to get access to the Sandpit?

Although there are no formal lead times, it will normally take 10 working days for the connection to be set up, after which supplier pipe cleaning will take place.

What is pipe cleaning?

Once permission to enter the Sandpit has been granted for either initial access or as a result of a Sandpit move, it is necessary for the supplier to complete basic pipe cleaning testing activities to ensure there are no issues which would prevent testing from taking place. Pipe cleaning tests are essentially those which establish basic connectivity. They will prove that the sandpit connection processes have all been completed, that the Message Handler is communicating effectively and that each message type is working. Additionally, for Choose and Book providers, they will prove that Clinic Management (via Directory of Service and slot polling) is functioning as required. This will take approximately one week. Pipe cleaning support during this process will be provided by the ESP (Existing System Providers) Compliance Team.

Who do I contact if I have difficulties in the Sandpit?

Once an investigation has been performed on all connections at the supplier end by the supplier with no resolution, an incident can be raised with the NICA (National Integration Centre and Assurance) helpdesk for Spine related issues in the first instance (nic.helpdesk@nhs.net).

Back to top

CAP Phase 3: Integration Test

What is the purpose of integration testing?

To assure the supplier release meets all relevant NHS CFH requirements e.g. functional, non functional, clinical safety, security and user requirements within the NICA test environments.

What happens during integration testing?

The process for integration testing will be defined in the tailored CAP Approach and RSD but will often be as such: Firstly, the supplier will develop test scripts to test their system in the Sandpit environment. The supplier will then be provided with the set of witness test scenarios from NHS CFH. Both of these script sets must be run and the outcomes documented in an integration test report. This will then be followed by a witness test.

What deliverables are required for this stage?

These deliverables will be defined in the scope stage but will encompass a test plan, test scripts (including an updated RTM to demonstrate the mapping of test scripts to the requirements) and a test report to include an OMVT2 test report. The integration test report must also include the results of the witness test scenarios provided by NHS CFH. All of these deliverables must be approved before witness testing can be confirmed.

What happens if I find defects during the testing?

All defects and impacted requirements will need to be detailed in the integration test report. A discussion will then follow to decide whether the witness testing can go ahead or if the defects must be fixed first due to high severity grading (as a minimum, severity 1 and 2 defects will need to be addressed).

What is a witness test?

A witness test is an onsite test of the supplier system observed by a NHS CFH Test Analyst with the set of witness test scenarios, provided by (NHS CFH) that are used for all supplier systems. The test analyst will record all of the test results, including defects, in the witness test report. It is expected that the system will be the final version that the supplier plans to deploy into live.

Why do we need a witness test?

A witness test provides a final level of assurance of the product and is not a substitute for any previous formal testing conducted by the supplier.

How long does it take to complete the witness test?

Depending on the system functionality (Choose and Book or PDS) the witness test will take approximately 10 to 15 working days.

Where will it take place?

Witness testing will normally take place at the supplier’s offices. The Compliance team will provide the supplier with a list of prerequisites for witness testing.

What if defects are found or tests fail during the witness test?

As the witness test takes place on the final build, it is not expected that any major faults with be found. However, it is possible that the NHS CFH Test Analyst may observe defects during the course of the witness test visit. These will be either Supplier defects or defects against the national systems. Defects do not result in an automatic failure of the witness test, however significant or repeated problems may render the system unsuitable for continued testing and fixes will have to be made before testing can recommence.

If a defect is discovered at the beginning of the visit it may be possible to fix, test and provide a new release. This must be discussed and agreed with the NHS CFH Test Analyst as uncontrolled patches are not permitted during a visit as they negate the results of all previous testing.

Who decides on the course of action?

A course of action will be agreed between the supplier, NHS CFH Test Analyst, Release Manager and the Existing Systems Providers (ESP) Compliance Team Manager. The ESP Compliance team provide support to suppliers delivering software or products outside of the scope of LSP or NASP contracts.

If a defect is observed which cannot be fixed during the visit then a course of action will be agreed between the supplier, NHS CFH Test Analyst and the ESP Compliance Team Manager. It may be that it is acceptable for the supplier to retest the defect and submit evidence to the ESP Compliance Team or it may be that a further Witness Test visit must be scheduled. This may be to witness the failed tests or it may be to re witness the full set of tests. The decision reached will be based on the number and severity of the defects raised.

What if I make a change to my system after witness testing?

If you fix any defects or make any changes before DevMAC is awarded, then it is necessary to make the ESP Compliance Team aware of the change via a Supplier Change Note form (SCN). The suggested changes must undergo an impact assessment and an appropriate level of testing will be defined to assure the change. The test evidence, which may include new messages, provided by the supplier will be analysed, along with the change. It may be that additional test evidence is required from the supplier to assure the new version of the system.

Can I deploy if there are defects on the system?

This will depend on the risks associated with the known defects and will be discussed with all stakeholders and agreed with the programme teams. If the defects are not severe, the defects will be logged onto the DevMAC to be managed and resolved during the First of Type deployment.

What is Non Functional testing?

Non Functional testing normally comprises Ready for Operations Testing (RFO) and Volume and Performance Testing (V and P), activity being solution dependant.

During RFO, the testing is performed by the supplier to demonstrate the solution is ready for deployment into the live environment. Test coverage will focus primarily on solution resilience, monitoring /alerting, backup /recovery and, where applicable, disaster recovery.  Associated business processes will also be assessed.

During Volume and Performance Testing, the testing is performed by the supplier to demonstrate the solution delivers agreed volumetric model utilisation relating to User and Messaging transaction volumes and transaction rates, as applicable.  As part of testing the underlying server performance and capacity, when delivering the volumetric model, is assessed. 

How long will the Non Functional witness testing take?

The onsite non functional testing, where applicable, is solution dependant.  A small solution delivery should only take 1 or 2 days to complete whilst more complex, hosted solutions can take up to 10 days.

Where will it take place?

The Non Functional testing will take place at the healthcare provider’s site.

What deliverables are required for Non Functional testing?

There are 3 deliverables required for each aspect (RFO and V and P) of non functional testing - a test plan, test scripts and a test report, which will be reviewed by the NHS CFH Non Functional team.  Dependant on non functional coverage a single combined test plan and test specification (scripts) may be agreed.

What is the DevMAC?

The DevMAC is a ‘Development Milestone Achievement Certificate’. This document signifies the completion of the CAP Integrated test phase and grants First of Type deployment to go ahead. It will contain information on the system and details and action plans for all defects if necessary.

Who approves it?

The DevMAC is approved by the internal stakeholders detailed on the RACI and is then issued to the Programme Delivery Board (PDB) which is a panel of NHS CFH representatives from all assurance teams.

What happens if it is not approved?

Further discussion around the system defects or more information on the system may be required.

Do all defects need to be fixed before Full Rollout Approval?

This is normally the case, but this will depend on the severity of the defects. If they are less severe, they can be fixed during the Deployment Verification Period (DVP) or in a future release.

What is CATR?

CATR is the ‘Clinical Authority to Release’ and is a confirmation that all clinical safety documents have been completed to the required standard.

Who grants CATR?

CATR is granted by the NHS CFH Clinical Safety Group (CSG).

What happens after DevMAC?

You will need to submit an end point registration form for the First of Type organisation during the integration test phase. Once DevMAC has been achieved, this will be processed by the Deployment Support Team (DST).

What is an end point registration form?

The end point registration form is a form submitted to NHS CFH to register the end point. There are two key reasons for completing this form:

  • To generate the certificates required for encrypted messaging;
  • To instruct BT to open up the firewalls on the Spine to allow network traffic from this trusted IP address.

How do I prepare for Go Live?

You must complete a Site Readiness Checklist. The document is an integral part of obtaining approval for Business Go Live for the agreed First of Type (FOT) site. This document details the Site Readiness Criteria (including Release Specific Criteria), which should be met before approval is granted.

Back to top

CAP Phase 4: Deployment

What is the purpose of the deployment stage?

To assure the first deployment(s) are successful and ensure potential local issues are identified and resolved prior to a fuller rollout.

What is the Deployment Verification Process?

The Deployment Verification Process (DVP) is the process by which the supplier shall verify that the system being released meets the Deployment Verification Criteria (DVC) at the FOT site(s). The performance of a deployed solution is monitored over a defined period of time (the DVP) and measured against a set of DVC. At the end of the verification period the supplier, supported by input from the Deployment Support Team (DST) and the Release Manager will produce a Deployment Verification Report (DVR).

What are the criteria for completing DVP?

A Deployment Verification Report will need to be provided and approved to complete the DVP. It will outline how the supplier has met all of the Deployment Verification Criteria.

What support do I get during DVP?

There will be weekly calls with the Deployment Support team (DST) to discuss any issues that have arisen, however they can be contacted during normal business hours for help too.

What is the incident log?

The incident log is a living document in which the supplier will detail any incidents incurred during the DVP, and any action plans to resolve them. It will also include any defects that have been listed on the DevMAC and will be tracked throughout the DVP. The incident log with be reviewed on the weekly checkpoint calls and will be used to complete the Deployment Verification Report.

What other deliverables do you need during DVP?

We will also require an updated Hazard Log to complete the DVP.

What is FRA?

FRA stands for Full Rollout Approval. The supplier must have been granted FRA for their product before they can deploy to other sites.

How do I get FRA?

A Deployment Milestone Achievement Certificate (DepMAC) shall be issued once the Authority to Proceed certificates have been issued for each of the Deployment Stages

Once the Programme Delivery Board (PDB) agree that all the pre-requisites have been completed successfully and Full Rollout can take place, a DepMAC for each site will be created and a Full Rollout Approval (FRA) certificate produced and signed off by the Service Introduction and Deployment Assurance Team (SIDAT) Manager.

Back to top

Live

What if I make a change to my system in live?

The ESP Compliance team also deals with the Supplier Maintenance Release Assurance Process (SMRAP). Whenever a Supplier makes a change to a compliant application, that is, one that has already been awarded a Technical Authority to Deploy (TATD) or DevMAC, it is necessary to make the ESP Compliance Team aware of the change via a Supplier Change Note form (SCN). The suggested changes must undergo an impact assessment and an appropriate level of testing will be defined to assure the change. The test evidence and messages provided by the Supplier will be analysed, along with the change. It may be that additional test evidence is required from the Supplier. It is also stated in the DevMAC that suppliers are required to support testing against changes to any national applications.

I want to deploy into another site, what do I do?

If you have Full Rollout Approval (FRA) you would be able to deploy to any site. You will need to submit an End Point Registration form for the new organisation. If you need any help throughout the process, please contact the NHS Business Partners team businesspartners@nhs.net