Crowd-sourced Certifications

I've just been mucking around with a new Internet service called Smarterer. That's not a typo, it really is Smarter-er. I guess, in a nutshell, it's an online quiz creator which is meant to help you quantify and showcase your skills. The twist in this implementation is that the quizzes are crowd-sourced. Anyone can write questions for the quiz and thus over time, the group interested in the topic defines the content and the grading of the quiz.

There's a whole pile of things going on under the hood that I haven't gotten into but it does look interesting.

The fruit of my tinkering was that I kicked off an Aviation Safety Management System quiz. It has 20 questions to begin with and is based on ICAO's Safety Management Manual. There's nothing too obscure in the questions but I would love to see the test grow - the only downside is that I can't take the test!

Anyway, check it out at http://smarterer.com/test/aviation-safety-management-systems and let me know what you think.

Under Thinking Just Culture and Accountability

I am definitely capable of over thinking, of tying myself up in knots and being lost in the detail. And other times, I probably haven't thought enough. Recently, I identified just culture as a concept I hadn't really thought about in-depth.

In my mind, I thought I knew what a just culture was. I knew it was more than a simple no-blame policy. I knew it involved establishing what is acceptable and not acceptable behaviour. But that had been the limit of my thinking.

That big void of knowledge started to weigh heavily on my mind. So, I set out to read Sidney Dekker's oft-cited Just Culture. Now that I have finished reading it, that void hasn't been filled - it's a swimming mess of questions, thoughts and more questions.

Although, I think I've got a grasp on that little hard nut called accountability.

I used to talk about how accountability was different to responsibility - "you can delegate responsibility, but you can't delegate accountability". I used to make the distinction that responsibility involved doing things but accountability didn't - "the responsible person performs the action, the accountable person just, ah, is accountable for it". However, I don't think I really ever defined accountability in a meaningful way (save for one occasion, by accident1).

I guess I understood that accountability meant knowing about what was going on in the area for which you were accountable but I never fully digested why that was important and what one would do with that knowledge.

While reading Just Culture had definitely helped me to understand the bigger picture but I don't think my knowledge had synthesised until I begun analysing an uncomfortable incident in which I was involved this weekend.

The Incident

I coach my son's U6 football (soccer) team. We're eight games into the season and I've been slowly gaining confidence in this role with a fair amount of trial and error. There hasn't been a lot of support for newbie coaches but I've been forging my way forward.

During matches, I've been on the field guiding the kids around and encouraging them along as they too fumble through their first year. It has been fun - especially the high-fives I get from four sets of hands for any manner of achievement, everything ranging from scoring a goal right down to not touching the ball with their hands.

But yesterday, things did not go so well.

A couple of minutes in the match, I was setting the ball on the goal line for a kick-in when a man (with no official identification) approached me and advised that I was not permitted on the field during the match. That didn't mesh with my understanding and since I was already concentrating on the match, I brushed  the guy off and told him I was staying on the field. He responded by telling me that he was going to get the ground official.

Not long after, two men wearing high-vis official vests entered the field and instructed the referee to stop the match. One of the men was the man from before, meaning the other must be the ground official. I approached the two men to find out what's going on.

I'll spare you the he said, I said stuff - the final ultimatum was I had to get off the field or he would cancel the match. Not much of a choice really, so I quickly explained the situation to the kids and coached from the sidelines for the rest of the game.

The Post-Incident Analysis

Now, I'm not going to get into all the grubby details of this incident - this post is not about the incident, it's about accountability and just culture.

Since I'm a life-long-learning type of guy, I ran through the incident in my head about a million times yesterday afternoon. I explored issues like:

  • what do they rules actually say? - for the record, I was wrong - I am not permitted on the field during the match;
  • why did I think the way I thought? - primarily a case of confirmation bias;
  • in what ways did these other men act inappropriately or in contravention of policy, etc.?; and
  • chiefly, what can I do better next time?

I thought about all these things as I prepared my incident report for my club president. My incident report, my account by another name. I was accountable for what ever happened on that field, especially incidents directly involving me, and here I was, providing my account.

I imagined the other men were doing the same with both of our club presidents taking these reports and providing their own accounts up the chain, as appropriate.

Okay, now what?

That's a really good question and this is where my past thought process tended to stop.

Dekker makes the point a number of times that sometimes, providing the account is enough. He says that families of patients lost on the operating table are often just wanting to know how it happened from those accountable for the event.

He also mentions the importance of data to learning but I didn't find the connection between learning and accountability that strong in the book. It was only yesterday that that neural pathway was opened.

The push to make people accountable is to increase learning.

Accountable doesn't mean identifying people for punishment, sanction or retribution. It simply means setting an expectation that they will be able to provide an account of what occurs within their sphere of accountability.

And it doesn't relate to just the accountable executive. It relates to everyone. In the above incident, I'm accountable, the other two men are accountable, our club presidents are accountable, the administrators of our local football association are accountable and so on.

This doesn't mean that the president of Football Brisbane should be able to describe the events which took place yesterday off the top of his head. It means that as each of us involved must analyse the incident and identify contributory factors coming from other parties. Those other parties provide accounts of those factors.

For example, why did I think I was allowed on the field? There are a range of contributory factors from inconsistent use of terms surrounding the coaching role both by my club and Football Brisbane right through to never having been corrected during the past eight matches2.

And just as each of us provides an account, each of us must take the accounts of others and learn from the incident. I have obviously learnt that I am not permitted on the field as well as not to trust confirmatory data. I hope that the other men involved learn better techniques for approaching newbie coaches who are concentrating on their teams' enjoyment and I hope that clubs and associations learn a few more ways of providing support for newbie coaches and ground officials.

Justice Served

As fired up as I got yesterday, I don't really think anyone should be punished for what happened. While I would like an apology for the manner of the approach, I am also happy to provide an apology for my frosty reception of the other men's intervention.

Overall, we just need to learn from the incident, move on and see to it that a similar incident doesn't occur again.

How effective that learning is will depend on how far the accounts go. In this instance, it appears that the power of a party to effect wide-spread learning is inversely proportional to the proximity of that party to the original incident3.

What I mean here is that the party furthest away from the incident, say Football Brisbane, has the greatest ability to prevent a repeat of this incident. I'm not going to be involved in this incident again because I now know that I'm not allowed on the field. I hope this incident won't occur at that ground again because the officials involved will adjust their behaviour and I hope that my club will let other coaches now about the incident to minimise it reoccurring within our club. Football Brisbane, on the other hand, has the power to see it and similar incidents prevented with its power to reach all coaches and officials within the Brisbane area and so on.

It's quite amazing what is possible just by providing an account without fear of reprisal. Here's hoping for some communication, some learning and some justice in the very near future.

Now I just wish aviation was as simple as U6 football.

1. I was presenting an SMS course in Indonesia and I had used Google translate taking English into Bahasa Indonesia to try to make the accountability/responsibility discussion more relevant. What I discovered was than in Bahasa Indonesia responsible means to "bear answer" which is pretty close to what I took from Dekker's book as the definition of accountable.

2. Mr Taleb will track me down and spank me for that one. I just read about confirmation bias and the asymmetry in data when it comes to confirmation versus contradiction.

3. I'm not sure if that's original. I can't recall reading it anywhere and it just came to me as I was writing this but that's not to say that I haven't read it before and I'm channelling some great thinker. If that is original, can we please call that the Parsons Rule?

SMS Considered

While in Bali talking Runway Safety with a wide range of industry personalities, I found myself at the hotel bar talking SMS with Bill Voss from Flight Safety Foundation. The topic was obviously on Bill's mind because upon my return, I found his latest president's piece in FSF's AeroSafety World to be a good overview of his main SMS points. Some of these points have been on my mind too. Since I'm not one to recreate the wheel (providing it works and is fit for purpose), I'll use some of Bill's well-formed words to kick this off.

Guidance Material

Back when the international standards for SMS were signed out at ICAO, we all knew we were going to launch a new industry full of consultants. We also knew that all these consultants couldn’t possibly know much about the subject and would be forced to regurgitate the ICAO guidance material that was being put out.

The title of the piece is SMS Reconsidered but I'm a little bit more critical of how SMS has been implemented in some places and would argue it was never really considered in the first place. The "regurgitation" of guidance material has been a big problem.

ICAO guidance material touting the "four pillars" was, as I saw it anyway, what the title suggested - guidance material. The industry was meant to consider the material and apply it within their operational context, corporate structure and organisational culture. The level of complexity within the operator, the existing systems in place, the attitudes of everyone involved were/are meant to be considered and a tailored SMS developed.

The reasons behind the current state of SMS are many, varied and probably not worth going over. It is more important to get the concept back on track. That's a big task and bigger than this little blog post. Instead, I wanted to discuss Bill's "four audit questions".

Levels Revisited

Bill's piece outlines four seemingly simple questions designed to test the operation of an SMS:

1. What is most likely to be the cause of your next accident or serious incident? 2. How do you know that? 3. What are you doing about it? 4. Is it working?

When posted on the FSF discussion forum on LinkedIn1, a fifth question (taken from the text) was added:

5. Can you show in the budget process where resources have been re-allocated to manage risk?

Interestingly, it was initially assumed that these were questions posed to the safety manager or some other safety professional as part of discussion between like-minded professionals. However, later comments did swing around to my first initial understanding that they could be asked of anyone within the organisation.

In fact, they should be asked of multiple people at different levels of the organisation.

A couple of weeks ago, I discussed the need to find the right solution at the right level and that the same tools may not be appropriate at different levels.

When thinking about SMS as a whole, there are an infinite number of ways of implementation but all must permeate all levels of the organisation with systems, processes and tools suitable to the needs of each level with communication channels between the various levels.

Bill's five questions, being agnostic to any specific SMS approach, can be applied to every level of the organisation. They should be asked of the safety manager, the operations manager, the training manager, the maintenance manager, the line supervisor and, probably most importantly, the CEO.

They aren't the only questions which need to be asked, but they are a good starting and ending point. Having all the "bits" of an SMS is required from a regulatory point of view but system effectiveness is vital to maintaining an ongoing level of assurance in an operator's ability to manage safety.

Pearls

I've audited or reviewed quite a few SMSs - only a few have showed any real consideration of the SMS concept and were tailored to suit the operator's needs. These were often the better performing systems and they bore little resemblance to the "four pillars".

At the Bali conference, I spied the completely different approach taken by Bombardier. It was mentioned a number of times that it is copyright, so I haven't included a picture here but you can find a presentation outlining their approach on the Transport Canada website. I can't comment on the effectiveness of the system but it is definitely food for thought and a ray of hope that the SMS concept is being considered, digested, pondered, manipulated, tailored, and so on.

1. It's a closed group, so I'm not sure who is able to see the discussion.

Logical Fallacies in the Safety Sphere

Sometimes I feel like I really missed out by not receiving a "classical" education. While I can probably live without the latin and greek philosophy, one area I've been keen to pick up is formal logic. The forming of a coherent and valid argument is a key skill which is, in my opinion, overlooked in safety management. Which is disappointing since making such an argument is at the heart of making a safety case.

I'm not going to tackle the subject of logic today. To be honest, I don't know enough about the overall concept. Instead, I'm going to focus on the typical failings present in a logical argument - the logical fallacies.

A logical fallacy is essentially an error in reasoning leading to an invalid argument.

Firstly, it is funny that most definitions I saw on the web described them as "errors". A term which carries a certain definition in aviation safety circles regarding intent. I just want to be clear that fallacies are not restricted to unintentional errors - they can be made deliberately.

More importantly, I should define a valid argument.

A valid argument is one in which the truth of the conclusion flows from the truths of the premises.

Now, there are a lot of specific types of fallacies. So many, in fact, that people have even developed taxonomies of them. Recently, I found a good primer in this area thanks to a team from Virginia.

But I've got a bit of problem with one aspect of this paper. The authors seem to have a higher opinion of safety professionals than I do. These are some of the offending sentences:

We assumed that safety arguments do not contain emotional appeals for their acceptance or willful attempts at deception.

For example, wishful thinking was excluded because it concerns arguments in which a claim is asserted to be true on the basis of a personal desire or vested interest in it being true. Such an argument is unlikely to appear explicitly in a safety argument.

That second one really grates my nerves. Safety tends to cost money and money is the most basic "vested interest".

I have sat through quite a few presentations on aviation safety that have deliberately pulled on the heart-strings to promote their agenda. This is a type of fallacy known as an emotional appeal.

Under the emotional appeal category, there are a few different types. Each is based on a different emotion - fear, envy, hatred, etc. But it is probably the appeal to pity (or the argumentum ad misericordiam) that I've seen the most. Here is a run-through of the most vivid of my encounters - de-identified, of course.

This presentation was on a certain type of approach to operational safety. I'll at least say that it wasn't SMS but let's leave it at that. The majority of the presentation was, what I assume, was a fairly accurate outline of this approach and how it was to be applied in the operational environment of the presenter.

What I had a problem with was the introduction and regular reference back to the, what I considered, grossly inappropriate emotional appeal made at the start. The commentary came on top of series of personal photos, backed up with a lamenting ballad and outlined the heart-wrenching plight of "Jane".

Jane was happily married for a few short years...was the centre of her husband's world...had recently welcomed her first child into the world...until one day here world was torn apart by an aviation tragedy which claimed the life of her husband...

I'm a generally emotional guy and this story got to me. I'm passionate about safety and on some level, I want to minimise the number of "Janes" out there.

But her story and the thousands like it, had absolutely no bearing on the case put forward in the rest of the presentation. In fact, I felt like it detracted from the substance of the information presented. After overcoming my tears and quivering chin, I probably bounced back into a super-critical stance as a reaction to the manipulation which had just occurred.

It is very tempting to employ cheap tricks such as these in an effort to increase the impact of one's safety case. But in the long run, it will only hurt it. Either by casting doubt on the truth of your conclusion or turning people against the argument regardless of its overall validity.

I might be getting a little bit more philosophical in the coming months as Mr Dekker and Mr Taleb continue to blow my mind with just culture, complexity, randomness and the black swan - more to come.

Integrating Runway Safety Teams with your Safety Management System

I've just spent an amazing week in Bali1 workshopping with operators and regulators from the Asia-Pacific region (and some from further afield) on the issue of runway safety. We got a lot of good information from the Flight Safety Foundation, ICAO and COSCAP as well as airlines, airports and regional regulators. The primary objective of the week was to provide information on and practice in the establishment and conduct of Local Runway Safety Teams (LRSTs). To this end, the seminars and workshop were great but I left feeling like one connection had been missed. The final question on my mind and many others, I am sure, was:

How do these runway safety initiatives integrate into my SMS?

I discussed this with a few of the other attendees and felt compelled to flesh out a few of my initial thoughts.

LRSTs are airport-based teams of representatives from runway safety stakeholders - the airport operator, the air traffic services provider, the airlines, the ARFFS provider and so on. The objective of this team is to collaborate on runway safety matters and coordinate responses to identified hazards or concerns. Much emphasis was placed on the inter-organisational and inter-disciplinary approach required when dealing with runway safety.

So how does this fit in with an operator's SMS?

The obvious relationship is through the committee arrangements found in most SMSs. In the ICAO approach to SMS, it is easy for me to imagine the LRST as a Safety Action Group (SAG).

According to the Safety Management Manual (SMM), a SAG is a "high-level committee, composed of line managers and representatives of front-line personnel" that "deals with 'grass roots' implementation issues pertaining to specific activities to ensure control of the safety risks of the consequences of hazards during line operations".

The language paints the SAG as an internal body but I see no reason why such a SAG of inter-organisational representatives cannot be convened as required when a safety issue requires it. The diagram on page 8-7 of the SMM suggests that multiple SAGs can be established and at Australian aerodromes, a safety committee of stakeholder representatives has been common thanks to some early advisory material.

A SAG sits under the Safety Review Board for that particular organisation, be they airport, airline, etc. The SRB is a higher-level committee tasked with strategic-level safety policy direction and safety assurance.

Graphically, the relationship could look something like this:

For complex environments, separate SAGs would be required and for smaller, less-complex environments, perhaps one committee is all that is needed with the various safety issues becoming working groups or even standing agenda items. It would be up to the operators involved to find the sweet spot - somewhere between the being so specific that there isn't enough work to do and being too general and having too much to do.

For airlines, and in some states, the air traffic service provider, there will be multiple LRSTs and other committees for them to attend. For these and large, complex airports, there maybe additional "mediator" committees required to coordinate and filter the numerous SAG-level committees outputs for input into that organisation's SRB.

So what are these inputs and outputs in terms of SMS functions?

If we look at the good ol' four pillars of SMS, then these inputs/outputs are the various elements of safety risk management, safety assurance and safety promotion.

Safety Risk Management

While each stakeholder's SMS will consider the risk associated with runway safety from their individual viewpoint and tend to identify treatment strategies within their sphere of influence, the real power in the LRST is the bringing together of these viewpoints to get a much more comprehensive picture of risk.

With this picture, the team is able to identify a range of treatment options designed to address the various aspects of the risk picture is ways that work together and cover the many causal and consequential pathways which exist within such a complex safety issue.

Safety Assurance

Again, each SMS in isolation would tend to measure only those aspects of safety performance within that stakeholders activities. As a bare minimum, the sharing of assurance information and at best, co-assurance activities, would greatly enhance the level of confidence each SRB would have that runway safety risk is being addressed.

Safety Promotion

Sharing a room, a team, an objective promotes safety much more than a safety poster. The safety training and communication systems within each stakeholder will be strengthened with the additional perspective provided by the other stakeholders. The possibilities here are endless.

Since I like drawing pretty little diagrams, here is another one describing the above:

Now, I don't want to diminish the progress one would make by establishing an LRST and getting some of the above going. These are very important steps and well worth the effort.

(here it comes)

But...

for those looking to the future, here are some challenges.

Amalgamating risk assessment methods - each stakeholder may have different approaches to risk analysis and they most certainly will have different risk criteria - pulling these together will a challenge.

Sharing assurance information - each organisation is going to need a strong just culture to achieve this one as airing your own dirty socks in public is never easy.

The answers to these challenges are...well, if I had definitive solutions, I probably wouldn't be sitting here blogging about them your free!

What I can suggest however, is that each stakeholder remains open with respect to risk assessment techniques and consider solving the problem on a common level - separate from the higher corporate level that a lot of SMSs operate on. With respect to sharing information, the suggestion at the RRSS Workshop was that if you want someone to share potentially embarrassing information with you, share some of yours first. I'd add to that, that it would be a good idea to establish agreed protections on the safety information to be shared.

Runway safety is a big, complex issue and there is a lot of work to be done on many levels. The LRST is one level, state runway safety groups are another. I am looking forward to some of the technological, operational and regulator advances that will be made in the future and with advances in safety performance monitoring being made, we might very well be able to monitor the effectiveness of progress in this area like never before.

1. I know. I have a tough life, right?

Levels. Levels? Yeah...

Seinfeld fans may remember this short exchange. Kramer might have been on to something and it had nothing to do with interior design. In my research and work, I've been butting up against a few theoretical roadblocks. But I am starting to think that these roadblocks are actually different levels. Internet guru1 Merlin Mann often observes that people need to solve the right problem at the right level. And now, I'm starting to think that is exactly what I need to do.

Identifying the different levels has been my task of late, and it is a task in need of completion.

This is where I'm at so far...

I was initially running with a military-style strategic/operational/tactical taxonomy. Specifically, strategic being the highest level and involving long-term, executive-level decisions through to frontline, troop-level decisions at the tactical level.

But these terms come loaded, so I've been looking elsewhere. Although, I don't think there are any terms left which don't carry some form of baggage.

So I've started down this road:

  • Executive - the highest level; involving the executive oversight or governance of the organisation; typically strategic although may be concerned with lower level issues from time to time.
  • Management - obviously, somewhere between the executive and the shopfront; probably characterised best as the level where enabling work gets done - things like personnel management, information management or hardware management.2
  • Operations - the real do-ers; practical actions taken in the extremely dynamic, real world.

I've been visualising this arrangement as something like this:

Different Levels

So what does this mean?

I believe the point of recognising the existence of the different levels is to accept that within each level, different objectives exist. As such, different tools and techniques may be required.

In thinking about this problem, I realised I posted something related to this before. In that post, I used different risk evaluation techniques at the different levels. While the overall risk management process should be consistent across all levels, the details differ because the objectives, contexts, and decisions differ.

At the highest/executive level, the context was related more to assurance with the decision about whether to accept the determined level of risk or to do more. As the risk picture changed, the executive decided to do more and directed the management level to produce a plan. At this level the risk evaluation methodology was quite different and quite tailored to the wildlife management context and the set of decisions required at that level - what to do about the various bird species.

Different Levels of Risk Assessments

I hinted at a third level of risk management but, to be honest, I haven't really seen that level employed in the real world in this context. OHS practitioners would be familiar with Job Safety Analyses (JSAs) which are a very operations-level activity which I thought would be similar to what I was thinking here.

I guess the moral of this rather rambling post is that I am becoming more and more convinced that an all-encompassing "enterprise risk management system" is not a simple case of having the same small set of tools for all levels. Instead, you need a framework that recognises the different levels (the different contexts, objectives and decisions) and creates linkages between these levels. My immature thoughts at this stage centre around the decisions and their resulting actions being those connections.

For example, the risk management being carried out at the lowest level may itself be a risk control measure for the next level up and so on. This becomes a bit circular but we might as well accept that it's turtles all the way down, people!

There may be more to come on this one, but right now, its bedtime!

1. He would so hate that title ;)

2. Safety management? I'm not too sure. I've been pondering this lately as well and when that thought is half-finished, I'll post it here too.

Work-Me & Blog-Me

AUGUST 2012 UPDATE: I've changed jobs since I posted this. However, I think it still works as a fair assessment of the relationship between this blog and my current job, which is not with the regulator. In this, the Web 2.0 world, connections can be made easily. There is no practical way to disconnect completely my blogging from my work.

And while my little disclaimer on the right is designed to create a barrier between the two, it probably doesn't address what has the potential to be a complex relationship.

This blog is my thinking brain in text. Primarily, it's an academic endeavour and therefore, by definition, it is about learning and discovery. I think it is very important to note that I am not blogging any final answers here. These are my thoughts and while I hope they are well considered and rooted in rationality, they are, more than likely, incomplete.

This blog is a personal endeavour with inspiration taken not only from my working environment but also personal activities and encounters. Nothing on here should be considered as necessarily relating to my employer or any specific aviation organisation of which I do get a privileged view. In an effort to ensure that this is the case, I tend to use news items, academic papers and training materials as my main sources of inspiration.

Overall, my blog is about me.

My work on the other hand is not about me. I work as part of a team which is part of an office which is part of a larger office which is part of an organisation. I'm a small piece in a very complex puzzle. I try to fill out my piece to the best of my ability and I hope that I do.

I have applied some of the thinking posted on this blog to projects I'm working on. In doing so, I have realised that some of these posts do indeed need more thought and I will probably bring that thought back here for posting.

I have put forward the arguments posted here in discussions with colleagues. They have listened (I hope) and then synthesized this information with their understanding and their objectives in mind. Like I said, I am part of a team and the viewpoints of many tend to provide a better answer than the opinion of one, especially when the full nature of the environment may not be apparent to each individual.

In short, you can't read this blog and ask why my employer hasn't implemented my ideas and nor should you consider my posts here to be the policy of my employer. In the first instance, as much as you may agree with me (if so, I am flattered), we may share a limited view of the overall situation. In the second instance, I am not omniscient (don't tell my wife), omnipresent (don't tell my boss) or omnipotent (don't tell my kids).

There is Work-Me and there is Blog-Me.

This here, is Blog-Me.

Just to be clear, questions have been raised regarding my blogging activity. No specific issue, just a heightened level of concern over what is still a relatively new form of personal activity. A social media policy is probably forthcoming from my employer and when it arrives, I will abide by its provisions 100%.

As Low As Reasonably Practicable

It's another staple of the risk management diet but while I believe this one to be a completely valid concept, I can't help to feel that its being served up underdone. This time I'm talking about ALARP - As Low As Reasonably Practicable. To define ALARP, at least how I do, would probably negate the need to write the rest of this post. So let's just say that ALARP is the point at which any further reduction in risk would require resources significantly greater than the magnitude in the benefit gained1.

It is often described graphically. Here are a few examples of the types of diagrams you may see helping to explain the concept:

The left diagram is the one I see the most although I am seeing, more and more, other representations including the other two. Rather than link any specific instances on the web, feel free to find such diagrams using Google Images.

So what are the problems that I see with most of these graphs? Thanks for asking...

The ALARP Region

In the left diagram, it is shown as an orange trapezoid and in the centre diagram, it is a line but in both cases the point of this area is to identify the level of risk acceptable if ALARP is achieved. Sometimes, the diagram is missing some commentary so it looks like that this region is simply the ALARP region - whatever that means.

Going hand in hand with the former definition though is that risks falling in the green area need not be treated at all and we'll come back to this.

Axes (as in plural of axis)

Often the nature of the axes is confusing. Take exhibit A (the one on the left), it has a y-axis but not x-axis. Sometimes you see risk magnitude shown as an x-axis but isn't risk level and risk magnitude the same thing?

Anyway, the diagram on the right has a bigger problem than that. It has no label on the x-axis but it does have two y-axes. The two plotted lines intersect at a point identified as the ALARP point.

But what is the significance of the intersect when different scales are used? I would argue that unless you identified the exact relationship between the two scales, there is no significance - not to ALARP or acceptability of the risk.

Two Questions

I see ALARP as not a question relating to acceptability - i.e. risk evaluation - but a question relating to risk treatment. Two different questions, but do both have to be answered?

If we follow the standard ISO 31000 RM process, the question of acceptability appears first and allows for the decision to not treat the risk, instead relying on existing controls. The standard does start to talk about cost-benefit considerations but stops short of requiring the achievement of ALARP at either the evaluation or treatment stages.

It appears to me that ALARP tends to be enshrined in regulations or case law. CASA aeronautical studies often include the following quote from an Australian High Court decision.

Where it is possible to guard against a foreseeable risk which, though perhaps not great, nevertheless cannot be called remote or fanciful, by adopting a means which involves little difficulty or expense, the failure to adopt such means will in general be negligent.

So, it seems that regardless of the inherent acceptability of a risk, it must still be treated to ALARP2. Meaning that you need to answer both questions separately.

  • Have I treated this risk to a level ALARP?
  • Is the residual level of risk acceptable?

My ALARP Diagram

In conceptualising my take on ALARP, I'm going to steal from the UK HSE department:

“‘Reasonably practicable’ is a narrower term than ‘physically possible’ … a computation must be made by the owner in which the quantum of risk is placed on one scale and the sacrifice involved in the measures necessary for averting the risk (whether in money, time or trouble) is placed in the other, and that, if it be shown that there is a gross disproportion between them – the risk being insignificant in relation to the sacrifice – the defendants discharge the onus on them.”

Those seem like some pretty clear directions. Risk on one axis and cost on the other. In order to make the slope of that line mean something, the cost scale needs to be calibrated to the risk scale but I have no idea how one would actually do this - maybe we'll tackle that one later. See below for a very rough, hand-drawn diagram. The ALARP point is rather hard to identify but it is the point where the slope of the line exceeds the cost-benefit limit.

Too often, I think we incorrectly lump related concepts into the same bucket and this leads to a blurring of the objectives of the process. In this case, ALARP fell in with risk evaluation when, I think, it should have remained separate and contained in the risk treatment part of the RM process.

Those risk professionals out there who possess ninja-like RM skills, can certainly short-cut the process to achieve the desire outcome but us grasshoppers3 should probably keep these concepts separate to ensure we cover off all requirements.

1. Adapted from ALARP's wikipedia page.
2. What this means for the standard, I'm not sure. I honestly hadn't thought about the implications of this thought process until I typed it just now.
3. I think I just mixed up kung-fu and whatever martial art ninjas do - no emails or dark-clad assassins please.

On the Shoulders of Giants

I can't rule out that I had already viewed this presentation and the words pathways and proximal became lodged in my mind - seeds sown to sprout some distant day in the future. But upon reading this document (again?)  I was struck by the apparent similarities with my proposed risk evaluation methodology, which was the subject of much ranting a few weeks ago - here, here, herehere and here. Specifically, I'm talking about these slides:

Here Reason mentions latent condition pathwaysHere Reason mentions proximal factors as opposed to remote factors

Seeing these concepts pop up in a presentation by Professor Reason really made me feel like I am on the right track1. However, I still have some work to do.

On my to-do list is to figure out how to match the likelihood scale to the new dimensions. Describing likelihood in terms more suited to events doesn't really hold for the latent type of risk condition. That to-do list is pretty full though, so it's only a short post today.

1. Yes, this could be just a case of confirmation bias

Culture: Complicated

A slipperier concept than culture there is not and yet, we definitely love to talk about it. Now I'm not suggesting that all this talk stop. There is nothing wrong with trying out different approaches to cultural change and seeing what works. However, I'm a bit of an academic and I don't mind a little esoteric pondering now and then. The following discussion is a summary of some ideas I cogitated on a couple of years ago when completing a minor research project for my Masters.

The objective of my contemplation was to come up with a model of cross-cultural influence which would assist in the planning of appropriate safety initiatives at Indigenous Australian community aerodromes. The subsequent project to actually introduce some initiatives hasn't (yet?) eventuated but the process was worthwhile in expanding my own understanding of culture.

Culture: Defined

When starting at the beginning, definitions are usually a good stepping off point but this can sometimes also be the first road block. I won't bore you with the play-by-play definition tug of war which has/is played out between anthropologists, sociologists and psychologists.

The really, really, really quick version would be to say that some people consider culture to be the observable behaviours of a group and others think of it more in terms of the shared cognitive processes that exist within the group.

I tend to lean toward the cognitive or ideational viewpoint but in my project I took the easy road. I argued that a strict definition is not necessary when you are working at the conceptual level for the purposes of developing an operational model.

Since a model is a simplified representation of some real world concept, it need not adhere to the strictest of definitions. As long as it works within the recognised limitations of the model.

Culture: Levels

Cross-cultural studies look at the interaction between different cultures. Typically, we think of different cultures at the same level. For example, Australian versus New Zealand (national) or Apple versus Microsoft (organisational/religious?). But life is rarely that simple and the interaction between the different levels, be they sub- or super-cultures is much more interesting.

The definitions of levels is, of course, another problem and again, I'm going to lean on the argument of conceptual modelling to simplify the situation.

The standard levels of culture in the management literature tend to break down to the groups one belongs to within their working life - team, branch, department, organisation, nation. However, depending on the situation under examination there may be other levels worthy of definition.

Within the Indigenous Australian aerodrome context of this project, I identified the "Australia-at-large" national culture, the Indigenous Australia sub-culture, the organisational culture of the aerodrome operator and the occupational culture of aerodrome staff.

Each of these cultures exists in the sense that they are identifiable in their own right. They impact on the individual to different degrees although not in a way that is fully independent of each other - some of the different levels influence each other.

Culture: Aspects

Safety culture. This word is probably the most used but most poorly defined word in the safety sphere at the moment. It is often used to express a positive and strong shared attitude toward safety - typically at the organisational level. I think this is a gross oversimplification.

However, if it fits your model and your needs, fill your boots.

Any "culture", i.e. a the shared cognitive characteristics of a group of people, can be viewed in a variety of terms. Service, quality, innovation, creativity etc. are all aspects of culture which can be examined separately according to the issue in question. They all exist at the same time with differing levels of strength or cohesion and in different directions (positive or negative).

With the pragmatic approach I've mentioned a couple of times above, I sought to avoid the argument too. Instead, I used the concept of safety climate. Climate was much easier to relate to the cognitive view of culture I cultivated earlier and fit within the model I was developing. I was able to make a strong connection between the concept of climate and perception, which puts climate well within the cognitive framework I was cultivating.

Culture: A Model

In developing my model, I relied heavily on the work of David Cray and Geoff Mallory in their book "making sense of managing culture". Their model was aimed the standard organisational management set and needed a little tweaking to fit my research issue. At its basic level, it looked like this:

I tweaked it however, to accentuate the cognitive aspect of the model. I established the cognitive process between a stimulus and the resulting behaviour and then set the culture entity above. See here:

There a two important things to remember with this model. Firstly, while culture is shown here as a separate entity, that is just for conceptual convenience. Culture, for me, is the shared aspects of the cognitive framework of the group. You can think of the culture box as "the group" which includes the individual.

The second thing is the simplification of the culture-individual relationship. I see it more as a feedback loop with culture presenting a stimulus to the individual, their cognitive processes directing a certain behaviour which feeds-back into the culture/group for its feedback in the form of new stimulus - and around and around we go!

As mentioned above, cultural influences are rarely as simple as the above. Below is the final model I prepared for the particular scenario I was looking at.

In this diagram, I included the safety perceptions component of the individual's cognitive processes to show where safety climate has an impact. I also showed what I considered the relative levels of influence of each culture on an Indigenous aerodrome staff member. The rationale behind these levels involved a couple of thousand words, from which I'll spare you.

Culture: Action

All these pretty pictures don't mean much unless they can guide some form of action. So what does this approach offer by way of insight?

I took from it, two main lessons.

1 - You can't influence all levels of culture. The "higher" levels of culture - national, indigenous - are beyond the influence of most mortals. It would be better to understand the nature of these cultures and their influence on the individual. Then it becomes a matter of managing expectations and focusing on outcomes rather than processes - especially for  cultures significantly alien to your own.

2 - While the organisational level is often the subject of most discussion, I think the occupational/profession level of culture has been under-utilised as a field of battle. Especially in areas where this level is underdeveloped such as for aerodrome staff.

Culture: More to be Said

But, I've rambled long enough today. Let's save some culture discussion for another day.

Wrapping Up PIGs .... For Now

Since I don't just want to be thought of as some PIG-hating obsessive lunatic, lets wrap this thread up for the moment. Quick recap: The traditional likelihood-consequence matrix (PIG - see original post) is not particularly useful when dealing with aviation safety. Why? Because a graduated consequence scale fails to recognise the perilous nature of aviation and consequence as a dimension isn't particularly useful when evaluating latent conditions remote from the ultimate outcome (death by aviation).

Alternate approach: Instead of scoring the consequence directly, I've offered two alternative dimensions under the generic title of influence1 - proximity and pathways.

In wrapping this up, I thought I would discuss what I think is the rationale behind this approach of using slightly off-centre indicators.

Obviously, it would great to have a complete and high-fidelity model of aviation accident causation. Something which showed all the risk conditions, variables, relationships, etc. A model to such a level that the ramifications of the slightest trend could be calculated automatically. Unfortunately, it doesn't seem to exist or at least, I don't have one.

The implausibility of such a model is why we have risk management. After all, risk is "the effect of uncertainty on objectives".

That is why the single score approach contained in most PIGs seems a contradiction in philosophies. To me, it attempts to apply certainty without telling us where the uncertainty has gone. I'm not sure that makes sense but please go with it for a moment.

What I'm trying to say is that using the traditional PIG, I attempt to assign single score X to condition A. Where did the uncertainty go? In short, it is still there and that is the root of a few of the problems I've mentioned in my last couple of posts. Especially, the problem of what to score - most likely, worst credible, worst case, etc.

What I've attempted to do is retain the uncertainty but keep it out of the scoring process. The proximity and pathways scales are, of course, indirect indicators of something bad happening. There is no guarantee that a risk condition directly connected or with a significant number of connecting pathways to the ultimate outcome will lead to utter catastrophe - but they are variables worth considering.

The uncertainty exists between the scale and the reality. The scoring can be carried out with some degree of confidence according to the scales chosen and the definition of the accident scenario.

Obviously, there may be plenty more such scales. The above two are just the ones that came to mind first - if you can think of any others, I'd love to hear your ideas - please comment.

There is more work to do on this idea. Such as, what other variables are required to support the decision-making process and is likelihood, probability or frequency the best indicator for presence of a risk condition? And so on. But I didn't want this blog to be all about PIGs or matrices or risk management necessarily.

Next week? My page is blank, I hope I don't get writer's block.

1. I might change this label. I really suck at naming things except my kids, their names are awesome ;)

Influential Behaviour

Near the end of my last post, I used the Swiss-cheese model to highlight that many risk conditions1 worthy of attention are not necessarily proximate to the ultimate outcome. I also hinted in the post before that, that I thought this to be only half the story. To tell this story, let me introduce another accident causation modelling technique. It is called an AcciMap and it is gaining popularity because it offers a way of representing the relationships between events (these being things such as decisions, functions, tasks, actions, etc.). An AcciMap is set up in two dimensions with vertical lanes separating system levels of increasing generality as you move up and the horizontal axis having no fixed dimension or scale. The system levels begin very specific to the accident in question with equipment and actor activities making up the first two levels. The higher levels relate to organisational, regulatory authority and government policy and decision making.

Here is a poorly drawn adaptation of an AcciMap:

Example of an AcciMap

If proximity was the only consideration then the top event and the limited emergency response equipment would be highest risk conditions. They are sitting right next to that big "ouch" so they must be the biggest problem.

But what about those inappropriate budget cuts? A decision like that has wide-reaching effects with most of them hidden until it is too late. I've started thinking about risk conditions such as this as having multiple pathways to the ultimate outcome. Therefore, they are just as important as those risk conditions which are in close proximity to the ultimate outcome.

Influencing Outcomes through Proximity & Pathways

So, where I'm going with this? I am recommending that instead of a straight consequence dimension, those conducting safety risk evaluation within a complex socio-technical system use an influence dimension made up of two scales - proximity and pathways. These scales can be defined as:

  • Proximity - relating to the number of discrete risk conditions between the condition being evaluated and the ultimate condition.
  • Pathways - relating to the number of pathways, via which, the risk condition being evaluated may lead to the ultimate condition.

Having multiple scales on one dimension isn't unusual but the above approach is a little different.

Where as the typical implementation of a multi-scaled dimension consists of different types of consequences (political, economic, reputation, etc.), the above approach is solely about the safety consequence. Therefore, you can't really stick these two scales into a common matrix as they sit at a different level to the standard scales.

They also differ in that they relate to the risk condition and not the potential outcome. As the outcome has already been defined as utter catastrophe, the focus has been turned toward the risk condition. And to me, that seems quite intuitive and reasonable.

These differences mean that when combined with some form of frequency or likelihood dimension2, we end up scoring the risk inherent to the risk condition. Of course, you can show this is a matrix but I think there is more to this story.

Hopefully, next time, I'll get this under control and tie it all together...

1. I am loathed to just say risks. To me whenever one uses the word "risk" it should be followed by "of" - for example "the risk of a runway excursion is high due to high crosswind, poor surface friction and large jet aircraft". It is always difficult to discuss a concept without a strong standardised lexicon and the last thing we need right now is another term introduced by some opinionated blogger but... I can't help it. People refer to a variety of, what I have come to call, conditions when they describe risks - they mention events, hazards, situations, mental states, failures. My intention is to accommodate all these under the one name, risk condition.

2. I'm not sure which to use yet. That problem is for this week's idle moments

One Step Back...

In continuing this little series I've got going here, I'd like to just quickly go back over a couple of points from last time. I'm trying to keep these posts relatively short. So that means I may have moved on to my next point a little too quickly. I guess the crux of the last post was that a graduated consequence scale is inappropriate in an aviation safety context. My two main points to back up that statement were:

  • the potential for a catastrophic event is persistent to the primary aviation activity of flying from A to B; and
  • that given aviation is a complex socio-technical system, risk conditions (call them hazards, events, or even just risks) upstream of the ultimate condition (death by aviation) cannot be categorised effectively.

I tried a few of these arguments out on some colleagues and they seemed unconvinced. So, I'm going to work on them a bit more here - this blogging thing is much more for my benefit than yours but thanks for stopping by anyway ;).

One step back...

Vulnerability

I made two objections to my vulnerability argument - the variety of outcomes flowing from common risks and that the outcome of a risk may vary with the aircraft size/occupancy. My responses to these points were brief. Probably too brief but this is meant to be a blog, not a dissertation. Let's go over them again.

I don't want to simply re-state my last post but the concept that catastrophe could have occurred because there exists no inherent limit to the consequence below this, is my best point. But let's look into it a bit further with an example, a runway overrun.

The vast majority of runway excursions do not end in death but was this because of some recovery measure which set an absolute maximum to the consequence? I don't think so, in fact, I think it was simply a further reduction of the likelihood of a completely catastrophic outcome - and now we have introduced likelihood into the consequence side of the equation. Is this complexity my own doing? Am I over-thinking this? Probably, but bear with me, please.

We seem to be back to an argument I put up in my first post on this issue. Risk, in an aviation safety sense at least, is not a discrete score - it is a continuum. At the very end of that continuum, always, is the most final of all outcomes. It may be have a very small likelihood attached but it is always there - persistent vulnerability.

Now again, I hear you saying (or they might be the voices in my head), but the aircraft occupancy may vary. Yes, you could construct a matrix with the consequence dimension graduating from one death to 500 deaths as required and such a matrix would have its uses. This type of matrix could be used to distinguish between the risk posed by individual operators or sectors of the industry for a variety of purposes such as surveillance planning, high-level regulatory standards development or safety performance measurement.

But it would not be useful within operational safety risk management - by that I mean, when you get into the operational sphere of stuff happening, this type of matrix wouldn't assist in the decision-making process when one designs and implements safety measures. (I don't want to just drop this dimension - it is important and it will pop up again later.)

The matrix you have in the above case only tells you about the risk associated with the final outcome. It does not assist in assessing risk conditions upstream.

So what do I mean when I say "upstream"?

Proximity

Aviation has a plethora of accident causation models. They have their differences, their pluses, their minuses and, of course, their similarities. I think I can say that the one thing all modern accident causation theories agree on is that accidents are never caused by a single act. They are the coming together of many acts with some being quite remote from the accident in terms of both space and time.

For this post, I'm going to run with the ol' tried & true, Swiss-cheese model1. It's not my favourite but it is well-known and serves my purposes here.

What the SCM brought to the world was the awareness that decisions made at the top of an organisation have an impact on frontline safety. When combined with the knock-on and discrete effects from all other levels of the organisation, one could say that, in some circumstances, the frontline operators were doomed from the beginning of their operation.

Swiss-cheese Model

Examples of these latent conditions include decisions to reduce the maintenance budget, outsource certain functions and even more obscure concepts as failing to inculcate a strong and positive safety culture. How does one risk assess conditions such as these? The link to any tangible consequential outcome is extremely tenuous even with all the accident reports which cite contributory factors such as these.

So now its time to think of solutions and last time, I said I thought there were a couple. I'm still working on a couple of these ideas but they will have to wait until next time - I'm already way past my target word count.

More to come...

1. This paper is a critique of the model by a Eurocontrol team which included the inventor, Dr James Reason. It is a good read.

Vulnerability & Proximity

In my last post, I commenced a whinge about the PIG or as it is more commonly known, the likelihood-consequence matrix. I signed off that post with a promise to further the discussion on the risk matrix within an aviation safety context. Here goes...

Consequence is an inappropriate dimension to consider in aviation safety. For two reasons which I call vulnerability and proximity. Let's take them in turn.

Aviation is a perilous undertaking. Every time you take to the sky you tend to inject enough potential energy into the situation that no amount risk mitigation can stand between you and catastrophe1.

In other fields, a graduated scale of consequence may be appropriate. Finance, for example, can easily delineate between monetary outcomes when limits can be set by how much you put into an uncertain situation. In aviation, you are all in.

Okay, there may be a few readers wishing to interject at this stage. I'm going to take a guess at two counter-arguments to the above position2. The first being that aircraft sizes/occupancies vary. The second is that many, many, many occurrences do not result in total annihilation of hundreds of passengers.

Let's take the second one first. The "but" that I would like to throw in after that sentence is that in everyone one of those near-misses, minor incidents, major incidents or even non-fatal accidents, catastrophe could have occurred. There was no inherent limit within the situation that meant complete loss of life was not a possibility.

Back to the first point now. Yes, you could limit the amount of life lost by limiting the number of passengers. This method of segregating risk decisions appears throughout the aviation safety sphere - the certification of aerodromes is a good example, the requirements kicks in at aircraft with more than thirty (30) seats. If you were to insert this into a PIG with "death of x number of people" along the consequence dimension, all you would end up with is a 2-D matrix of accident frequency acceptability/unacceptability.

And this leads into proximity...

The "risks" we tend to look at within the aviation safety realm are quite varied. One second we might be considering the chance of an engine failure and its impact during Extended Diversion Time Operations, then we'll be looking at the impact of a poorly maintained operations manual and following that up with an assessment of an ineffective hazard reporting system. Each of these conditions falls in a completely different area of the accident causation chain.

I've started to think about this problem as proximity. How close is this condition to the ultimate outcome? Obviously, conditions closer to the end result are more important and things further upstream are less so, right? I think we start to hit another issue here and its one I'm working through at the moment and hope to write about next week.

But before I go, I do want to sum up the above rant.

I believe that the traditional likelihood-consequence matrix is not suited to risk management (assessment/evaluation) within the aviation safety realm. A graduated consequence scale with anything less that complete loss of life fails to recognise the persistent potential for catastrophe and a graduated scale based loss of life limited by aircraft size cannot be applied to conditions ("risks") which exist upstream of the final stop of the accident causation chain.

I think there is an answer to these problems. In fact, I think there are a couple. Stay tuned.

1. Until that is, Q unveils his inflatable aircraft cocoon - something like this.

2. If you have any more please feel free to comment.

My Problem with PIGs

You can't swing a euphemism without hitting one when you're playing in the risk management metaphor. They're everywhere. Whenever you start looking at anything risk management related, you are sure to find a PIG. PIG stands for Probability-Impact Graph - otherwise known as likelihood-consequence matrix or frequency-severity chart or some combination of these words. I'm most familiar with the LxC matrix label, so I'll use it from here on in.

Over the past year or so, I've being growing more and more uneasy with the application of this tool within the aviation safety environment. I wasn't seeing, however, the same discontent in others and therefore, started to doubt my own reservations. Luckily, I found some like-minded people over at LinkedIn (membership to both LinkedIn and the group are required to view the actual discussion) with a Mr Stephen Cresswell putting his thoughts on paper here.

My new best friends have identified a range of issues with the PIG, some of which apply to other applications and some of which are very similar to my concerns.

So what are my concerns?

The first one is to what do I apply the score - do I apply it to the hazard, the event or the outcome? For me, the outcome always seemed wrong because the consequence is contained within its definition thus it negates the need for that dimension of the score. The event gives you good opportunity to attach a likelihood of it occurring but what about an event with a variety of possible consequences or causes (hazards)? And for hazards, is it likelihood of existence or some consequential event and here we go wrapping ourselves up in knots.

Example time: Let's have some evil birds hitting lovely, peaceful planes1. On an airport, birds tend to cause a bit of stress in their operator's lives. How does one risk assess this problem?

Do you calculate the likelihood & consequence of the bird-strike event? Seems simple enough but how to you account for different birds in different areas affecting different phases of the aircraft's flight? Do you then apply the calculation to each bird species? How do you distribute this score across the possible outcomes?

And that brings me to my second beef with PIGs - risk is not a discrete score.

If risk is indeed a combination of likelihood and consequence, in the aviation safety context, I don't see how it cannot be expressed as a discrete score. The risk of a bird-strike is a continuum. Most of the time, i.e. high likelihood, the consequence will be minor or negligible (near-miss or small collision). Some of the time, i.e. lower likelihood, the consequence will be major (something broken) and on rare occasions, i.e. really low likelihood, you'll get a major media event.

So what do you score? The most likely consequence, the worst case the scenario, the most credible outcome, etc. etc. etc.?

For my last point, I'll steal directly from Mr Cresswell:

PIGs take a simplistic view of risk in which there is no system-based thinking about the relationships and dependencies between risks, opportunities and uncertainties.

Aviation is an extremely complex socio-technical system - it's the relationships that matter. Treating each "risk" as a separate line item with its own discrete score doesn't mesh with our thinking in other areas - especially, accident causation theory and the overall safety management system concept.

I'm going to try to develop these ideas over the coming weeks (with more regularity than to date) - stay tuned.

1. Last year I posted this on bird-strike risk management. I even used a PIG approach at the more strategic level but dropped it for the specie-specific risk assessment, instead I opted for a completely different approach.

Safety Hero: Roger Boisjoly

It's a slice of history every safety professional should know - the night before the Challenger disaster, engineers at NASA-contractor Morton Thiokol made a recommendation that the launch not proceed. They believed that at the low temperatures being experienced at and forecast for the launch site, booster rocket o-ring performance would be severely degraded. They that that this could (and did) lead to disaster. Dissecting what happened is important from many perspectives. As the scenario played out there was group-think, political influences, confirmation biases, inappropriate interpretation of a lack of data (the absence of evidence etc.). The list goes on.

The lesson today? Courage.

This morning I wouldn't have been able to name any of the engineers who tried to stop the Challenger launch. I knew of them and have seen, a number of times, their part played out in this reconstruction.

One of them was Roger Boisjoly and yesterday he passed away.

Thanks to twitter and the blogosphere, I've had a chance to read up on him and remind myself of the man who tried to make a difference. Reading about the impact the event had on him was, frankly, depressing. Being able to say "I told you so" isn't a reward, it is not even a solace.

However, I hope that if I am ever in that kind of situation, I show the same courage he did.

"Compliance Doesn't Equal Safety"

I've heard this saying quite a bit over the last few months and in at least one aspect, I agree with the statement. It tends to be true that regulations have failed to keep pace with industry. As such, blind compliance with the regulation no longer ensures an accident-free existence.

Apparently, there is a Venn diagram making the rounds. I've been told it looks like this:

Safety Doesn't Equal Compliance

So what is the solution?

In most conversations, I heard "systems-based" or "risk-based" auditing touted as the answer. Unfortunately, that, in my opinion, is not the answer.

Now, don't get me wrong. Please. Old check-box auditing is not the answer either.

So what is? Let's break it down into pieces...

Firstly, we need to define "compliance". Often, people infer from compliance, a high level of prescription within the legislation, regulation or standard. In the past this was true but nowadays, it's becoming less the case. Regardless, when auditing, especially in a highly-regulated environment, the standards are everything. Issuing findings outside of the standards is unacceptable and you can forget about enforcement action.

We have no choice but to move the "compliance" circle to match up with the "safety" circle. Easier said than done.

The above Venn diagram is mis-leading. Safety is not a neat circle. It's not the same for each industry sector, or even between operators within the same sector. And it's not stable. Not only is aviation a, generally, growing industry, it's diverging as well. "Safety" could probably be drawn like this instead:

The safety profiles of various operators differ

That leaves old-school compliance with two options. Be a well-rounded set of requirements aimed at achieving a generally good level of safety or becoming a complex monster of requirements aimed at ensuring safety across all areas - see below. In the first case (C), gaps between safety and compliance still exist (just to refresh - that means an operator could comply with the rules and still not "be" safe). And in both cases (C & D), all operators are burdened by requirements which have nothing to do with safety in their environment.

The different ways to apply old-school compliance

I mentioned above that the standards are becoming less prescriptive. The still fairly new concept of safety management systems is a different kind of regulation. Instead of telling operators how to address known safety risks, it requires operators to establish a system to identify, assess and mitigate risk within its own environment1. This approach can fill the gaps between traditional compliance and safety. It also goes a long way to supporting "systems-based" and "risk-based" auditing.

Diagram E is probably a good indication of where we are at the moment. In most cases, an SMS requirements has been added to the existing regulatory regime and a "fill the gaps" approach has been taken. This is a valid approach. SMS is relatively new and the industry needs time to grow into the philosophy.

The way compliance looks now and may look in the future

In the future, I imagine it will be possible to reduce the prescriptive side of the compliance equation. A greater level of flexibility is obviously good for the industry as it would reduce the nugatory regulatory burden on some operators but ensure a level of safety acceptable to the regulator.

That retreat will require careful planning and sure-footed execution. A lot of work will need to be done before then, but I'll hold on to it as my "I have a dream" concept2.

1. It actually requires more than a risk management system but lets keep things simple for the sake of this short but rambling post. Check out the link above for more information. 2. I know it's nowhere near the level of nobility of the original but each of us hopes to influence the world in our own way.

Unhappy? No, but...

After a short hiatus and a new job, I've decided to start blogging again. This time on topics related to my new job. My old blog was slanted towards airport safety but with my shift into a more general and strategic role, I thought I'd shift the blogging to a new home and recommence putting my thoughts out there. The name of this blog comes from a chapter in James Reason's 1997 book, managing the risks of organisational accidents. In it, Professor Reason provided a number of reasons why the regulator's lot is an unhappy one.

There are quite a few issues brought up in the chapter with one of the main negative issue being that the regulator is unlikely to receive any accolades for "bringing about a non-event" but is sure to "be judged by those with 20:20 hindsight as making significant contributions to a major disaster".

There is a positive take-home message for those of us silly enough to be a regulator - Professor Reason thinks that we "are potentially one of the most important defences against organisational accidents".

That got me thinking about "that" graph that you see in most safety-related presentations. You know, this one1: "that" graph In an effort to get ahead of the curve, I often turn my mind to trying to identify the next paradigm shift in accident prevention. And being a fairly egotistical, I'm convinced that the next step resides with the regulator at the industry level. Before you judge me too harshly, these's a trend behind my conceit.

At first attention was focussed on the fundamental unit of aviation - the aircraft. Then as the ROI on technological advances slowed, attention shifted to the pilot or the human factor. Then we looked into interactions between pilots with the advent of cockpit resource management. Which morphed into crew resource management when it started looking at cabin crew and then ground crew. At the moment a lot of work is going into company-level interactions - safety management systems and culture.

But it won't stop there, it can't. Traffic will continue to increase and with it, the number of accidents2. The public, travelling or not, will demand safety be better and tickets be cheaper.

So, I think the next frontier will be at that regulatory level. It will involve national and international authorities and its going to involve a lot of reform and a lot of change. It's going to be a hard slog but we've asked those in the industry to rethink safety on a number of occasions. Its not too much to ask ourselves to question the impact of our actions on safety at the frontline and make changes where appropriate.

I'll be posting my thoughts on topics relating to aviation safety regulation here, when I can. Topics I have in the hopper include risk assessment matrices, compliance v safety and safety management systems at the smaller end of the industry. Here we go...

1. I've got quite a few gripes with this graph, I hope to address them in a future post.
2. Actually, some believe we're overdue for an increase in the accident rate.