Risk analysis for computerised systems

When done correctly, risk analysis can help to identify things that could go wrong in a process once it has been put into action. It may also trigger creative thinking about how to avoid those problems or at least prevent them from causing damage. Risk analysis makes it possible to build elements into a tool at an early stage of development that will make it more stable and safer to use.

Risk analysis is a regulatory requirement for all implementations that require validation, so has to be formally documented and approved (see When does a software tool require validation?).

When it comes to life sciences, risk analysis focuses mainly on identifying problems that could potentially affect the patients and health professionals using your product, rather than business risks, such as negative impacts on budget or timeline.

When to carry out risk analysis

It’s important to carry out the first risk analysis shortly after starting an implementation project, as soon as the requirements have been drafted. The analysis will need to be carried out again, or at least reviewed, each time the specifications evolve.

Risk analysis will usually prompt changes to planned features or the addition of new features to mitigate risk, so in itself triggers changes in specifications. Whatever the reasons behind a change in specifications, the associated risks should always be reassessed.

Who should carry out the risk analysis?

Risk analysis requires a good understanding of your infrastructure and the technologies being used, as well insights from users of the system. Ideally the team will be led by someone with previous experience of leading risk analysis who is currently involved with other risk analysis processes. This will help newcomers become acquainted with the procedures involved and ensure the new process is consistent with other risk analysis your company is doing.

The risk analysis process

The risk analysis team will go through your project specifications methodically.

The guidelines below are based on failure modes and effects analysis (FMEA) and good automated manufacturing practice (GAMP) principles.

Preparing for the initial risk analysis meeting

Before the risk analysis meeting, prepare the following elements:

1. A list of items to be analysed, based on your specifications. This list will frame the discussion and will ensure that all aspects of your implementation are considered. You could either base your list on user requirements or on processes implemented and the steps involved in each.
2. Assuming you’re using the FMEA approach, outline your conventions for the following categories:
   • Severity – The main factor here is the potential impact on patients, health professionals or users of the computerised system. Another important factor is the risk to data integrity. A risk of losing significant data would often be considered as high severity. The severity levels will be highly dependent on the risk level associated with the system as a whole. For example a computerised system that records vigilance incidents and holds patient information will be associated with a higher level of risk than a training management system. Bear in mind that each potential risk that could affect a system cannot be considered more severe then the risk associated with the system as a whole.
   • Occurrence – Try to assess the likelihood of the potential problems actually happening. This is why it is critical to have people involved who can assess the likelihood from the perspective of system users.
   • Detection (or detectability) – How likely it is that a risky situation will be detected before it actually causes harm? For example, if a user needs to provide several confirmations before they delete data, it is less likely that they will unintentionally delete data.
   • Risk class – These classes are determinined based on a combination of severity and occurrence.
   • Risk priority – Rate each potential risk as high, medium or low priority, by combining their risk class and detection. Agree on what is meant by high, low and medium before you start.
   • Acceptable level of risk – The key purpose of doing the risk analysis is to identify practical ways to make the system safer, or reduce the risks involved to an acceptable level. The acceptable level of risk will be dictated by management at the organisational level.

Running the risk analysis meeting

Once the leader of the risk analysis team has set the scene by explaining the background and the purpose of the meeting, the rest of the session wil usually be structured as follows:

1. Risk identification and qualification – Walk through each of the items on your list (requirements or process steps). For each item:
   • Ask the team what could go wrong. Each thing that could go wrong is a risk. List all the risks your team finds.
   • Qualify each of the risks for their severity, occurrence and detection. Your predefined formulas will then generate the risk priority. Note that the same risks may apply to different items. For example, providing incomplete data is a risk that might be applicable at many steps along the process.
2. Risk mitigation – Review each of the risks you have identified in the previous steps and find ways to mitigate them through changes in the process. This might include adding control steps or changing process features. Also consider adding external mitigations, for example providing additional training or explanatory documentation. However, note that changing the process is typically a more effective approach than external mitigation.

Throughout the process, aim to mitigate risk to at least bring it down to an acceptable level.

Tools for risk analysis

Discover how to use Jira for your risk analysis

Example

An electronic quality management system (eQMS) is being implemented to support the corrective and preventive actions (CAPA) process.

Here’s a snippet from the risk analysis process:

1. Initiation – the CAPA event is registered in the eQMS.
2. The following risk was identified:
   • Wrong information included in the eQMS or information missing:
     • Severity: Medium
     • Occurance: High
     • Detectability: High
3. This risk is mitigated by:
   • Defining some data fields in that phase as mandatory, enforcing more complete and accurate information.
   • Desiging subsequent process stages in a way that ensures data is reviewed by independent subject matter experts.