Risk Evaluation

Risk Evaluation Series

Risk Evaluation Series

I have never really liked risk assessments that involve some vague calculation of consequence and likelihood. I have witnessed, and perhaps been involved in, many arguments about where to stick some potential or even past disaster into a limited matrix.

A couple of years ago, I had a whinge/thought experiment on an alternative method of risk evaluation. I enjoyed the use of the term Probability-Impact Graph, rather than Likelihood-Consequence Matrix, to create a little theme - see if you can guess what it was.

I’ve repackaged these posts and a follow-up post for renewed consumption below. Each image is actually a link to the post.

Wildlife Risk Management Series

Wildlife Risk Management Series

A long time ago I wrote a rather comprehensive series on wildlife hazard management within an ISO 31000 risk management framework. It was the launch series for the New Airport Insider website and quite a bit of work on my part - but I enjoyed it. So, I thought I would repackage it as a quick blog post with links to each article.

Seussian Safety Management

Seussian Safety Management

All three on my children have been brought into the world of reading partially through the works of Dr Seuss. I can't count the number of times I have read his books. As my kids have grown older, they have turned into the reader and read these amazing books back to me.

The Bike Lesson is one of my favourites for the very nerdy reason that towards the end of the book The Berenstains provide us with a short & succinct definition of safety. It's three simple stanzas that I think encapsulate modern safety management perfectly.

BTIII: Assessing Uncertainty

BTIII: Assessing Uncertainty

I can't lie to you. I have been turning myself inside out trying to get a handle on risk evaluation in the aviation safety sphere for close to five years now and I still don't feel any closer to an answer. And I say "an" answer and not "the" answer. Since you are always assessing risk in terms of your objectives, there can and will be multiple approaches to assessing the risk of the same scenario depending on whether you are considering your safety, financial or legal objectives.

BTII: Control-freak*

BTII: Control-freak*

As a follow-on to my first post on the Bow-Tie risk assessment method, I thought I'd concentrate on controls (or barriers or whatever else you would like to call them). This is, after all, where all the action happens. Risk controls are how we spend most of our time - they are the practical aspect of managing risk.

Lessons from Taleb's Black Swan

Lessons from Taleb's Black Swan

Having just finished reading Nassim Taleb's The Black Swan, I initially thought about writing a not-so-in-depth assessment of the book's positive and negative points - but I'm not much of a book reviewer and a comprehensive critique is probably beyond my capabilities (at this stage). So, instead I thought I would focus on just a couple of the book's significant concepts and explore how they may apply in the aviation context.

BTI: Dressing up for Risk Assessments

BTI: Dressing up for Risk Assessments

I've been doing a lot of pondering on the Bow-Tie method of risk assessment for a project at work. Bow-Tie is a tool used by many, especially in the oil & gas industry, to create a picture of risk surrounding a central event. It's got a few positives and a few negatives but these can be overcome if you understand the limitations of the model being used.

As Low As Reasonably Practicable

It's another staple of the risk management diet but while I believe this one to be a completely valid concept, I can't help to feel that its being served up underdone. This time I'm talking about ALARP - As Low As Reasonably Practicable. To define ALARP, at least how I do, would probably negate the need to write the rest of this post. So let's just say that ALARP is the point at which any further reduction in risk would require resources significantly greater than the magnitude in the benefit gained1.

It is often described graphically. Here are a few examples of the types of diagrams you may see helping to explain the concept:

The left diagram is the one I see the most although I am seeing, more and more, other representations including the other two. Rather than link any specific instances on the web, feel free to find such diagrams using Google Images.

So what are the problems that I see with most of these graphs? Thanks for asking...

The ALARP Region

In the left diagram, it is shown as an orange trapezoid and in the centre diagram, it is a line but in both cases the point of this area is to identify the level of risk acceptable if ALARP is achieved. Sometimes, the diagram is missing some commentary so it looks like that this region is simply the ALARP region - whatever that means.

Going hand in hand with the former definition though is that risks falling in the green area need not be treated at all and we'll come back to this.

Axes (as in plural of axis)

Often the nature of the axes is confusing. Take exhibit A (the one on the left), it has a y-axis but not x-axis. Sometimes you see risk magnitude shown as an x-axis but isn't risk level and risk magnitude the same thing?

Anyway, the diagram on the right has a bigger problem than that. It has no label on the x-axis but it does have two y-axes. The two plotted lines intersect at a point identified as the ALARP point.

But what is the significance of the intersect when different scales are used? I would argue that unless you identified the exact relationship between the two scales, there is no significance - not to ALARP or acceptability of the risk.

Two Questions

I see ALARP as not a question relating to acceptability - i.e. risk evaluation - but a question relating to risk treatment. Two different questions, but do both have to be answered?

If we follow the standard ISO 31000 RM process, the question of acceptability appears first and allows for the decision to not treat the risk, instead relying on existing controls. The standard does start to talk about cost-benefit considerations but stops short of requiring the achievement of ALARP at either the evaluation or treatment stages.

It appears to me that ALARP tends to be enshrined in regulations or case law. CASA aeronautical studies often include the following quote from an Australian High Court decision.

Where it is possible to guard against a foreseeable risk which, though perhaps not great, nevertheless cannot be called remote or fanciful, by adopting a means which involves little difficulty or expense, the failure to adopt such means will in general be negligent.

So, it seems that regardless of the inherent acceptability of a risk, it must still be treated to ALARP2. Meaning that you need to answer both questions separately.

  • Have I treated this risk to a level ALARP?
  • Is the residual level of risk acceptable?

My ALARP Diagram

In conceptualising my take on ALARP, I'm going to steal from the UK HSE department:

“‘Reasonably practicable’ is a narrower term than ‘physically possible’ … a computation must be made by the owner in which the quantum of risk is placed on one scale and the sacrifice involved in the measures necessary for averting the risk (whether in money, time or trouble) is placed in the other, and that, if it be shown that there is a gross disproportion between them – the risk being insignificant in relation to the sacrifice – the defendants discharge the onus on them.”

Those seem like some pretty clear directions. Risk on one axis and cost on the other. In order to make the slope of that line mean something, the cost scale needs to be calibrated to the risk scale but I have no idea how one would actually do this - maybe we'll tackle that one later. See below for a very rough, hand-drawn diagram. The ALARP point is rather hard to identify but it is the point where the slope of the line exceeds the cost-benefit limit.

Too often, I think we incorrectly lump related concepts into the same bucket and this leads to a blurring of the objectives of the process. In this case, ALARP fell in with risk evaluation when, I think, it should have remained separate and contained in the risk treatment part of the RM process.

Those risk professionals out there who possess ninja-like RM skills, can certainly short-cut the process to achieve the desire outcome but us grasshoppers3 should probably keep these concepts separate to ensure we cover off all requirements.

1. Adapted from ALARP's wikipedia page.
2. What this means for the standard, I'm not sure. I honestly hadn't thought about the implications of this thought process until I typed it just now.
3. I think I just mixed up kung-fu and whatever martial art ninjas do - no emails or dark-clad assassins please.