Risk Management

Integrating Runway Safety Teams with your Safety Management System

I've just spent an amazing week in Bali1 workshopping with operators and regulators from the Asia-Pacific region (and some from further afield) on the issue of runway safety. We got a lot of good information from the Flight Safety Foundation, ICAO and COSCAP as well as airlines, airports and regional regulators. The primary objective of the week was to provide information on and practice in the establishment and conduct of Local Runway Safety Teams (LRSTs). To this end, the seminars and workshop were great but I left feeling like one connection had been missed. The final question on my mind and many others, I am sure, was:

How do these runway safety initiatives integrate into my SMS?

I discussed this with a few of the other attendees and felt compelled to flesh out a few of my initial thoughts.

LRSTs are airport-based teams of representatives from runway safety stakeholders - the airport operator, the air traffic services provider, the airlines, the ARFFS provider and so on. The objective of this team is to collaborate on runway safety matters and coordinate responses to identified hazards or concerns. Much emphasis was placed on the inter-organisational and inter-disciplinary approach required when dealing with runway safety.

So how does this fit in with an operator's SMS?

The obvious relationship is through the committee arrangements found in most SMSs. In the ICAO approach to SMS, it is easy for me to imagine the LRST as a Safety Action Group (SAG).

According to the Safety Management Manual (SMM), a SAG is a "high-level committee, composed of line managers and representatives of front-line personnel" that "deals with 'grass roots' implementation issues pertaining to specific activities to ensure control of the safety risks of the consequences of hazards during line operations".

The language paints the SAG as an internal body but I see no reason why such a SAG of inter-organisational representatives cannot be convened as required when a safety issue requires it. The diagram on page 8-7 of the SMM suggests that multiple SAGs can be established and at Australian aerodromes, a safety committee of stakeholder representatives has been common thanks to some early advisory material.

A SAG sits under the Safety Review Board for that particular organisation, be they airport, airline, etc. The SRB is a higher-level committee tasked with strategic-level safety policy direction and safety assurance.

Graphically, the relationship could look something like this:

For complex environments, separate SAGs would be required and for smaller, less-complex environments, perhaps one committee is all that is needed with the various safety issues becoming working groups or even standing agenda items. It would be up to the operators involved to find the sweet spot - somewhere between the being so specific that there isn't enough work to do and being too general and having too much to do.

For airlines, and in some states, the air traffic service provider, there will be multiple LRSTs and other committees for them to attend. For these and large, complex airports, there maybe additional "mediator" committees required to coordinate and filter the numerous SAG-level committees outputs for input into that organisation's SRB.

So what are these inputs and outputs in terms of SMS functions?

If we look at the good ol' four pillars of SMS, then these inputs/outputs are the various elements of safety risk management, safety assurance and safety promotion.

Safety Risk Management

While each stakeholder's SMS will consider the risk associated with runway safety from their individual viewpoint and tend to identify treatment strategies within their sphere of influence, the real power in the LRST is the bringing together of these viewpoints to get a much more comprehensive picture of risk.

With this picture, the team is able to identify a range of treatment options designed to address the various aspects of the risk picture is ways that work together and cover the many causal and consequential pathways which exist within such a complex safety issue.

Safety Assurance

Again, each SMS in isolation would tend to measure only those aspects of safety performance within that stakeholders activities. As a bare minimum, the sharing of assurance information and at best, co-assurance activities, would greatly enhance the level of confidence each SRB would have that runway safety risk is being addressed.

Safety Promotion

Sharing a room, a team, an objective promotes safety much more than a safety poster. The safety training and communication systems within each stakeholder will be strengthened with the additional perspective provided by the other stakeholders. The possibilities here are endless.

Since I like drawing pretty little diagrams, here is another one describing the above:

Now, I don't want to diminish the progress one would make by establishing an LRST and getting some of the above going. These are very important steps and well worth the effort.

(here it comes)

But...

for those looking to the future, here are some challenges.

Amalgamating risk assessment methods - each stakeholder may have different approaches to risk analysis and they most certainly will have different risk criteria - pulling these together will a challenge.

Sharing assurance information - each organisation is going to need a strong just culture to achieve this one as airing your own dirty socks in public is never easy.

The answers to these challenges are...well, if I had definitive solutions, I probably wouldn't be sitting here blogging about them your free!

What I can suggest however, is that each stakeholder remains open with respect to risk assessment techniques and consider solving the problem on a common level - separate from the higher corporate level that a lot of SMSs operate on. With respect to sharing information, the suggestion at the RRSS Workshop was that if you want someone to share potentially embarrassing information with you, share some of yours first. I'd add to that, that it would be a good idea to establish agreed protections on the safety information to be shared.

Runway safety is a big, complex issue and there is a lot of work to be done on many levels. The LRST is one level, state runway safety groups are another. I am looking forward to some of the technological, operational and regulator advances that will be made in the future and with advances in safety performance monitoring being made, we might very well be able to monitor the effectiveness of progress in this area like never before.

1. I know. I have a tough life, right?

Levels. Levels? Yeah...

Seinfeld fans may remember this short exchange. Kramer might have been on to something and it had nothing to do with interior design. In my research and work, I've been butting up against a few theoretical roadblocks. But I am starting to think that these roadblocks are actually different levels. Internet guru1 Merlin Mann often observes that people need to solve the right problem at the right level. And now, I'm starting to think that is exactly what I need to do.

Identifying the different levels has been my task of late, and it is a task in need of completion.

This is where I'm at so far...

I was initially running with a military-style strategic/operational/tactical taxonomy. Specifically, strategic being the highest level and involving long-term, executive-level decisions through to frontline, troop-level decisions at the tactical level.

But these terms come loaded, so I've been looking elsewhere. Although, I don't think there are any terms left which don't carry some form of baggage.

So I've started down this road:

  • Executive - the highest level; involving the executive oversight or governance of the organisation; typically strategic although may be concerned with lower level issues from time to time.
  • Management - obviously, somewhere between the executive and the shopfront; probably characterised best as the level where enabling work gets done - things like personnel management, information management or hardware management.2
  • Operations - the real do-ers; practical actions taken in the extremely dynamic, real world.

I've been visualising this arrangement as something like this:

Different Levels

So what does this mean?

I believe the point of recognising the existence of the different levels is to accept that within each level, different objectives exist. As such, different tools and techniques may be required.

In thinking about this problem, I realised I posted something related to this before. In that post, I used different risk evaluation techniques at the different levels. While the overall risk management process should be consistent across all levels, the details differ because the objectives, contexts, and decisions differ.

At the highest/executive level, the context was related more to assurance with the decision about whether to accept the determined level of risk or to do more. As the risk picture changed, the executive decided to do more and directed the management level to produce a plan. At this level the risk evaluation methodology was quite different and quite tailored to the wildlife management context and the set of decisions required at that level - what to do about the various bird species.

Different Levels of Risk Assessments

I hinted at a third level of risk management but, to be honest, I haven't really seen that level employed in the real world in this context. OHS practitioners would be familiar with Job Safety Analyses (JSAs) which are a very operations-level activity which I thought would be similar to what I was thinking here.

I guess the moral of this rather rambling post is that I am becoming more and more convinced that an all-encompassing "enterprise risk management system" is not a simple case of having the same small set of tools for all levels. Instead, you need a framework that recognises the different levels (the different contexts, objectives and decisions) and creates linkages between these levels. My immature thoughts at this stage centre around the decisions and their resulting actions being those connections.

For example, the risk management being carried out at the lowest level may itself be a risk control measure for the next level up and so on. This becomes a bit circular but we might as well accept that it's turtles all the way down, people!

There may be more to come on this one, but right now, its bedtime!

1. He would so hate that title ;)

2. Safety management? I'm not too sure. I've been pondering this lately as well and when that thought is half-finished, I'll post it here too.

As Low As Reasonably Practicable

It's another staple of the risk management diet but while I believe this one to be a completely valid concept, I can't help to feel that its being served up underdone. This time I'm talking about ALARP - As Low As Reasonably Practicable. To define ALARP, at least how I do, would probably negate the need to write the rest of this post. So let's just say that ALARP is the point at which any further reduction in risk would require resources significantly greater than the magnitude in the benefit gained1.

It is often described graphically. Here are a few examples of the types of diagrams you may see helping to explain the concept:

The left diagram is the one I see the most although I am seeing, more and more, other representations including the other two. Rather than link any specific instances on the web, feel free to find such diagrams using Google Images.

So what are the problems that I see with most of these graphs? Thanks for asking...

The ALARP Region

In the left diagram, it is shown as an orange trapezoid and in the centre diagram, it is a line but in both cases the point of this area is to identify the level of risk acceptable if ALARP is achieved. Sometimes, the diagram is missing some commentary so it looks like that this region is simply the ALARP region - whatever that means.

Going hand in hand with the former definition though is that risks falling in the green area need not be treated at all and we'll come back to this.

Axes (as in plural of axis)

Often the nature of the axes is confusing. Take exhibit A (the one on the left), it has a y-axis but not x-axis. Sometimes you see risk magnitude shown as an x-axis but isn't risk level and risk magnitude the same thing?

Anyway, the diagram on the right has a bigger problem than that. It has no label on the x-axis but it does have two y-axes. The two plotted lines intersect at a point identified as the ALARP point.

But what is the significance of the intersect when different scales are used? I would argue that unless you identified the exact relationship between the two scales, there is no significance - not to ALARP or acceptability of the risk.

Two Questions

I see ALARP as not a question relating to acceptability - i.e. risk evaluation - but a question relating to risk treatment. Two different questions, but do both have to be answered?

If we follow the standard ISO 31000 RM process, the question of acceptability appears first and allows for the decision to not treat the risk, instead relying on existing controls. The standard does start to talk about cost-benefit considerations but stops short of requiring the achievement of ALARP at either the evaluation or treatment stages.

It appears to me that ALARP tends to be enshrined in regulations or case law. CASA aeronautical studies often include the following quote from an Australian High Court decision.

Where it is possible to guard against a foreseeable risk which, though perhaps not great, nevertheless cannot be called remote or fanciful, by adopting a means which involves little difficulty or expense, the failure to adopt such means will in general be negligent.

So, it seems that regardless of the inherent acceptability of a risk, it must still be treated to ALARP2. Meaning that you need to answer both questions separately.

  • Have I treated this risk to a level ALARP?
  • Is the residual level of risk acceptable?

My ALARP Diagram

In conceptualising my take on ALARP, I'm going to steal from the UK HSE department:

“‘Reasonably practicable’ is a narrower term than ‘physically possible’ … a computation must be made by the owner in which the quantum of risk is placed on one scale and the sacrifice involved in the measures necessary for averting the risk (whether in money, time or trouble) is placed in the other, and that, if it be shown that there is a gross disproportion between them – the risk being insignificant in relation to the sacrifice – the defendants discharge the onus on them.”

Those seem like some pretty clear directions. Risk on one axis and cost on the other. In order to make the slope of that line mean something, the cost scale needs to be calibrated to the risk scale but I have no idea how one would actually do this - maybe we'll tackle that one later. See below for a very rough, hand-drawn diagram. The ALARP point is rather hard to identify but it is the point where the slope of the line exceeds the cost-benefit limit.

Too often, I think we incorrectly lump related concepts into the same bucket and this leads to a blurring of the objectives of the process. In this case, ALARP fell in with risk evaluation when, I think, it should have remained separate and contained in the risk treatment part of the RM process.

Those risk professionals out there who possess ninja-like RM skills, can certainly short-cut the process to achieve the desire outcome but us grasshoppers3 should probably keep these concepts separate to ensure we cover off all requirements.

1. Adapted from ALARP's wikipedia page.
2. What this means for the standard, I'm not sure. I honestly hadn't thought about the implications of this thought process until I typed it just now.
3. I think I just mixed up kung-fu and whatever martial art ninjas do - no emails or dark-clad assassins please.

On the Shoulders of Giants

I can't rule out that I had already viewed this presentation and the words pathways and proximal became lodged in my mind - seeds sown to sprout some distant day in the future. But upon reading this document (again?)  I was struck by the apparent similarities with my proposed risk evaluation methodology, which was the subject of much ranting a few weeks ago - here, here, herehere and here. Specifically, I'm talking about these slides:

Here Reason mentions latent condition pathwaysHere Reason mentions proximal factors as opposed to remote factors

Seeing these concepts pop up in a presentation by Professor Reason really made me feel like I am on the right track1. However, I still have some work to do.

On my to-do list is to figure out how to match the likelihood scale to the new dimensions. Describing likelihood in terms more suited to events doesn't really hold for the latent type of risk condition. That to-do list is pretty full though, so it's only a short post today.

1. Yes, this could be just a case of confirmation bias

Wrapping Up PIGs .... For Now

Since I don't just want to be thought of as some PIG-hating obsessive lunatic, lets wrap this thread up for the moment. Quick recap: The traditional likelihood-consequence matrix (PIG - see original post) is not particularly useful when dealing with aviation safety. Why? Because a graduated consequence scale fails to recognise the perilous nature of aviation and consequence as a dimension isn't particularly useful when evaluating latent conditions remote from the ultimate outcome (death by aviation).

Alternate approach: Instead of scoring the consequence directly, I've offered two alternative dimensions under the generic title of influence1 - proximity and pathways.

In wrapping this up, I thought I would discuss what I think is the rationale behind this approach of using slightly off-centre indicators.

Obviously, it would great to have a complete and high-fidelity model of aviation accident causation. Something which showed all the risk conditions, variables, relationships, etc. A model to such a level that the ramifications of the slightest trend could be calculated automatically. Unfortunately, it doesn't seem to exist or at least, I don't have one.

The implausibility of such a model is why we have risk management. After all, risk is "the effect of uncertainty on objectives".

That is why the single score approach contained in most PIGs seems a contradiction in philosophies. To me, it attempts to apply certainty without telling us where the uncertainty has gone. I'm not sure that makes sense but please go with it for a moment.

What I'm trying to say is that using the traditional PIG, I attempt to assign single score X to condition A. Where did the uncertainty go? In short, it is still there and that is the root of a few of the problems I've mentioned in my last couple of posts. Especially, the problem of what to score - most likely, worst credible, worst case, etc.

What I've attempted to do is retain the uncertainty but keep it out of the scoring process. The proximity and pathways scales are, of course, indirect indicators of something bad happening. There is no guarantee that a risk condition directly connected or with a significant number of connecting pathways to the ultimate outcome will lead to utter catastrophe - but they are variables worth considering.

The uncertainty exists between the scale and the reality. The scoring can be carried out with some degree of confidence according to the scales chosen and the definition of the accident scenario.

Obviously, there may be plenty more such scales. The above two are just the ones that came to mind first - if you can think of any others, I'd love to hear your ideas - please comment.

There is more work to do on this idea. Such as, what other variables are required to support the decision-making process and is likelihood, probability or frequency the best indicator for presence of a risk condition? And so on. But I didn't want this blog to be all about PIGs or matrices or risk management necessarily.

Next week? My page is blank, I hope I don't get writer's block.

1. I might change this label. I really suck at naming things except my kids, their names are awesome ;)

Influential Behaviour

Near the end of my last post, I used the Swiss-cheese model to highlight that many risk conditions1 worthy of attention are not necessarily proximate to the ultimate outcome. I also hinted in the post before that, that I thought this to be only half the story. To tell this story, let me introduce another accident causation modelling technique. It is called an AcciMap and it is gaining popularity because it offers a way of representing the relationships between events (these being things such as decisions, functions, tasks, actions, etc.). An AcciMap is set up in two dimensions with vertical lanes separating system levels of increasing generality as you move up and the horizontal axis having no fixed dimension or scale. The system levels begin very specific to the accident in question with equipment and actor activities making up the first two levels. The higher levels relate to organisational, regulatory authority and government policy and decision making.

Here is a poorly drawn adaptation of an AcciMap:

Example of an AcciMap

If proximity was the only consideration then the top event and the limited emergency response equipment would be highest risk conditions. They are sitting right next to that big "ouch" so they must be the biggest problem.

But what about those inappropriate budget cuts? A decision like that has wide-reaching effects with most of them hidden until it is too late. I've started thinking about risk conditions such as this as having multiple pathways to the ultimate outcome. Therefore, they are just as important as those risk conditions which are in close proximity to the ultimate outcome.

Influencing Outcomes through Proximity & Pathways

So, where I'm going with this? I am recommending that instead of a straight consequence dimension, those conducting safety risk evaluation within a complex socio-technical system use an influence dimension made up of two scales - proximity and pathways. These scales can be defined as:

  • Proximity - relating to the number of discrete risk conditions between the condition being evaluated and the ultimate condition.
  • Pathways - relating to the number of pathways, via which, the risk condition being evaluated may lead to the ultimate condition.

Having multiple scales on one dimension isn't unusual but the above approach is a little different.

Where as the typical implementation of a multi-scaled dimension consists of different types of consequences (political, economic, reputation, etc.), the above approach is solely about the safety consequence. Therefore, you can't really stick these two scales into a common matrix as they sit at a different level to the standard scales.

They also differ in that they relate to the risk condition and not the potential outcome. As the outcome has already been defined as utter catastrophe, the focus has been turned toward the risk condition. And to me, that seems quite intuitive and reasonable.

These differences mean that when combined with some form of frequency or likelihood dimension2, we end up scoring the risk inherent to the risk condition. Of course, you can show this is a matrix but I think there is more to this story.

Hopefully, next time, I'll get this under control and tie it all together...

1. I am loathed to just say risks. To me whenever one uses the word "risk" it should be followed by "of" - for example "the risk of a runway excursion is high due to high crosswind, poor surface friction and large jet aircraft". It is always difficult to discuss a concept without a strong standardised lexicon and the last thing we need right now is another term introduced by some opinionated blogger but... I can't help it. People refer to a variety of, what I have come to call, conditions when they describe risks - they mention events, hazards, situations, mental states, failures. My intention is to accommodate all these under the one name, risk condition.

2. I'm not sure which to use yet. That problem is for this week's idle moments

One Step Back...

In continuing this little series I've got going here, I'd like to just quickly go back over a couple of points from last time. I'm trying to keep these posts relatively short. So that means I may have moved on to my next point a little too quickly. I guess the crux of the last post was that a graduated consequence scale is inappropriate in an aviation safety context. My two main points to back up that statement were:

  • the potential for a catastrophic event is persistent to the primary aviation activity of flying from A to B; and
  • that given aviation is a complex socio-technical system, risk conditions (call them hazards, events, or even just risks) upstream of the ultimate condition (death by aviation) cannot be categorised effectively.

I tried a few of these arguments out on some colleagues and they seemed unconvinced. So, I'm going to work on them a bit more here - this blogging thing is much more for my benefit than yours but thanks for stopping by anyway ;).

One step back...

Vulnerability

I made two objections to my vulnerability argument - the variety of outcomes flowing from common risks and that the outcome of a risk may vary with the aircraft size/occupancy. My responses to these points were brief. Probably too brief but this is meant to be a blog, not a dissertation. Let's go over them again.

I don't want to simply re-state my last post but the concept that catastrophe could have occurred because there exists no inherent limit to the consequence below this, is my best point. But let's look into it a bit further with an example, a runway overrun.

The vast majority of runway excursions do not end in death but was this because of some recovery measure which set an absolute maximum to the consequence? I don't think so, in fact, I think it was simply a further reduction of the likelihood of a completely catastrophic outcome - and now we have introduced likelihood into the consequence side of the equation. Is this complexity my own doing? Am I over-thinking this? Probably, but bear with me, please.

We seem to be back to an argument I put up in my first post on this issue. Risk, in an aviation safety sense at least, is not a discrete score - it is a continuum. At the very end of that continuum, always, is the most final of all outcomes. It may be have a very small likelihood attached but it is always there - persistent vulnerability.

Now again, I hear you saying (or they might be the voices in my head), but the aircraft occupancy may vary. Yes, you could construct a matrix with the consequence dimension graduating from one death to 500 deaths as required and such a matrix would have its uses. This type of matrix could be used to distinguish between the risk posed by individual operators or sectors of the industry for a variety of purposes such as surveillance planning, high-level regulatory standards development or safety performance measurement.

But it would not be useful within operational safety risk management - by that I mean, when you get into the operational sphere of stuff happening, this type of matrix wouldn't assist in the decision-making process when one designs and implements safety measures. (I don't want to just drop this dimension - it is important and it will pop up again later.)

The matrix you have in the above case only tells you about the risk associated with the final outcome. It does not assist in assessing risk conditions upstream.

So what do I mean when I say "upstream"?

Proximity

Aviation has a plethora of accident causation models. They have their differences, their pluses, their minuses and, of course, their similarities. I think I can say that the one thing all modern accident causation theories agree on is that accidents are never caused by a single act. They are the coming together of many acts with some being quite remote from the accident in terms of both space and time.

For this post, I'm going to run with the ol' tried & true, Swiss-cheese model1. It's not my favourite but it is well-known and serves my purposes here.

What the SCM brought to the world was the awareness that decisions made at the top of an organisation have an impact on frontline safety. When combined with the knock-on and discrete effects from all other levels of the organisation, one could say that, in some circumstances, the frontline operators were doomed from the beginning of their operation.

Swiss-cheese Model

Examples of these latent conditions include decisions to reduce the maintenance budget, outsource certain functions and even more obscure concepts as failing to inculcate a strong and positive safety culture. How does one risk assess conditions such as these? The link to any tangible consequential outcome is extremely tenuous even with all the accident reports which cite contributory factors such as these.

So now its time to think of solutions and last time, I said I thought there were a couple. I'm still working on a couple of these ideas but they will have to wait until next time - I'm already way past my target word count.

More to come...

1. This paper is a critique of the model by a Eurocontrol team which included the inventor, Dr James Reason. It is a good read.

Vulnerability & Proximity

In my last post, I commenced a whinge about the PIG or as it is more commonly known, the likelihood-consequence matrix. I signed off that post with a promise to further the discussion on the risk matrix within an aviation safety context. Here goes...

Consequence is an inappropriate dimension to consider in aviation safety. For two reasons which I call vulnerability and proximity. Let's take them in turn.

Aviation is a perilous undertaking. Every time you take to the sky you tend to inject enough potential energy into the situation that no amount risk mitigation can stand between you and catastrophe1.

In other fields, a graduated scale of consequence may be appropriate. Finance, for example, can easily delineate between monetary outcomes when limits can be set by how much you put into an uncertain situation. In aviation, you are all in.

Okay, there may be a few readers wishing to interject at this stage. I'm going to take a guess at two counter-arguments to the above position2. The first being that aircraft sizes/occupancies vary. The second is that many, many, many occurrences do not result in total annihilation of hundreds of passengers.

Let's take the second one first. The "but" that I would like to throw in after that sentence is that in everyone one of those near-misses, minor incidents, major incidents or even non-fatal accidents, catastrophe could have occurred. There was no inherent limit within the situation that meant complete loss of life was not a possibility.

Back to the first point now. Yes, you could limit the amount of life lost by limiting the number of passengers. This method of segregating risk decisions appears throughout the aviation safety sphere - the certification of aerodromes is a good example, the requirements kicks in at aircraft with more than thirty (30) seats. If you were to insert this into a PIG with "death of x number of people" along the consequence dimension, all you would end up with is a 2-D matrix of accident frequency acceptability/unacceptability.

And this leads into proximity...

The "risks" we tend to look at within the aviation safety realm are quite varied. One second we might be considering the chance of an engine failure and its impact during Extended Diversion Time Operations, then we'll be looking at the impact of a poorly maintained operations manual and following that up with an assessment of an ineffective hazard reporting system. Each of these conditions falls in a completely different area of the accident causation chain.

I've started to think about this problem as proximity. How close is this condition to the ultimate outcome? Obviously, conditions closer to the end result are more important and things further upstream are less so, right? I think we start to hit another issue here and its one I'm working through at the moment and hope to write about next week.

But before I go, I do want to sum up the above rant.

I believe that the traditional likelihood-consequence matrix is not suited to risk management (assessment/evaluation) within the aviation safety realm. A graduated consequence scale with anything less that complete loss of life fails to recognise the persistent potential for catastrophe and a graduated scale based loss of life limited by aircraft size cannot be applied to conditions ("risks") which exist upstream of the final stop of the accident causation chain.

I think there is an answer to these problems. In fact, I think there are a couple. Stay tuned.

1. Until that is, Q unveils his inflatable aircraft cocoon - something like this.

2. If you have any more please feel free to comment.

My Problem with PIGs

You can't swing a euphemism without hitting one when you're playing in the risk management metaphor. They're everywhere. Whenever you start looking at anything risk management related, you are sure to find a PIG. PIG stands for Probability-Impact Graph - otherwise known as likelihood-consequence matrix or frequency-severity chart or some combination of these words. I'm most familiar with the LxC matrix label, so I'll use it from here on in.

Over the past year or so, I've being growing more and more uneasy with the application of this tool within the aviation safety environment. I wasn't seeing, however, the same discontent in others and therefore, started to doubt my own reservations. Luckily, I found some like-minded people over at LinkedIn (membership to both LinkedIn and the group are required to view the actual discussion) with a Mr Stephen Cresswell putting his thoughts on paper here.

My new best friends have identified a range of issues with the PIG, some of which apply to other applications and some of which are very similar to my concerns.

So what are my concerns?

The first one is to what do I apply the score - do I apply it to the hazard, the event or the outcome? For me, the outcome always seemed wrong because the consequence is contained within its definition thus it negates the need for that dimension of the score. The event gives you good opportunity to attach a likelihood of it occurring but what about an event with a variety of possible consequences or causes (hazards)? And for hazards, is it likelihood of existence or some consequential event and here we go wrapping ourselves up in knots.

Example time: Let's have some evil birds hitting lovely, peaceful planes1. On an airport, birds tend to cause a bit of stress in their operator's lives. How does one risk assess this problem?

Do you calculate the likelihood & consequence of the bird-strike event? Seems simple enough but how to you account for different birds in different areas affecting different phases of the aircraft's flight? Do you then apply the calculation to each bird species? How do you distribute this score across the possible outcomes?

And that brings me to my second beef with PIGs - risk is not a discrete score.

If risk is indeed a combination of likelihood and consequence, in the aviation safety context, I don't see how it cannot be expressed as a discrete score. The risk of a bird-strike is a continuum. Most of the time, i.e. high likelihood, the consequence will be minor or negligible (near-miss or small collision). Some of the time, i.e. lower likelihood, the consequence will be major (something broken) and on rare occasions, i.e. really low likelihood, you'll get a major media event.

So what do you score? The most likely consequence, the worst case the scenario, the most credible outcome, etc. etc. etc.?

For my last point, I'll steal directly from Mr Cresswell:

PIGs take a simplistic view of risk in which there is no system-based thinking about the relationships and dependencies between risks, opportunities and uncertainties.

Aviation is an extremely complex socio-technical system - it's the relationships that matter. Treating each "risk" as a separate line item with its own discrete score doesn't mesh with our thinking in other areas - especially, accident causation theory and the overall safety management system concept.

I'm going to try to develop these ideas over the coming weeks (with more regularity than to date) - stay tuned.

1. Last year I posted this on bird-strike risk management. I even used a PIG approach at the more strategic level but dropped it for the specie-specific risk assessment, instead I opted for a completely different approach.