Safety Management Systems

Crowd-sourced Certifications

I've just been mucking around with a new Internet service called Smarterer. That's not a typo, it really is Smarter-er. I guess, in a nutshell, it's an online quiz creator which is meant to help you quantify and showcase your skills. The twist in this implementation is that the quizzes are crowd-sourced. Anyone can write questions for the quiz and thus over time, the group interested in the topic defines the content and the grading of the quiz.

There's a whole pile of things going on under the hood that I haven't gotten into but it does look interesting.

The fruit of my tinkering was that I kicked off an Aviation Safety Management System quiz. It has 20 questions to begin with and is based on ICAO's Safety Management Manual. There's nothing too obscure in the questions but I would love to see the test grow - the only downside is that I can't take the test!

Anyway, check it out at http://smarterer.com/test/aviation-safety-management-systems and let me know what you think.

SMS Considered

While in Bali talking Runway Safety with a wide range of industry personalities, I found myself at the hotel bar talking SMS with Bill Voss from Flight Safety Foundation. The topic was obviously on Bill's mind because upon my return, I found his latest president's piece in FSF's AeroSafety World to be a good overview of his main SMS points. Some of these points have been on my mind too. Since I'm not one to recreate the wheel (providing it works and is fit for purpose), I'll use some of Bill's well-formed words to kick this off.

Guidance Material

Back when the international standards for SMS were signed out at ICAO, we all knew we were going to launch a new industry full of consultants. We also knew that all these consultants couldn’t possibly know much about the subject and would be forced to regurgitate the ICAO guidance material that was being put out.

The title of the piece is SMS Reconsidered but I'm a little bit more critical of how SMS has been implemented in some places and would argue it was never really considered in the first place. The "regurgitation" of guidance material has been a big problem.

ICAO guidance material touting the "four pillars" was, as I saw it anyway, what the title suggested - guidance material. The industry was meant to consider the material and apply it within their operational context, corporate structure and organisational culture. The level of complexity within the operator, the existing systems in place, the attitudes of everyone involved were/are meant to be considered and a tailored SMS developed.

The reasons behind the current state of SMS are many, varied and probably not worth going over. It is more important to get the concept back on track. That's a big task and bigger than this little blog post. Instead, I wanted to discuss Bill's "four audit questions".

Levels Revisited

Bill's piece outlines four seemingly simple questions designed to test the operation of an SMS:

1. What is most likely to be the cause of your next accident or serious incident? 2. How do you know that? 3. What are you doing about it? 4. Is it working?

When posted on the FSF discussion forum on LinkedIn1, a fifth question (taken from the text) was added:

5. Can you show in the budget process where resources have been re-allocated to manage risk?

Interestingly, it was initially assumed that these were questions posed to the safety manager or some other safety professional as part of discussion between like-minded professionals. However, later comments did swing around to my first initial understanding that they could be asked of anyone within the organisation.

In fact, they should be asked of multiple people at different levels of the organisation.

A couple of weeks ago, I discussed the need to find the right solution at the right level and that the same tools may not be appropriate at different levels.

When thinking about SMS as a whole, there are an infinite number of ways of implementation but all must permeate all levels of the organisation with systems, processes and tools suitable to the needs of each level with communication channels between the various levels.

Bill's five questions, being agnostic to any specific SMS approach, can be applied to every level of the organisation. They should be asked of the safety manager, the operations manager, the training manager, the maintenance manager, the line supervisor and, probably most importantly, the CEO.

They aren't the only questions which need to be asked, but they are a good starting and ending point. Having all the "bits" of an SMS is required from a regulatory point of view but system effectiveness is vital to maintaining an ongoing level of assurance in an operator's ability to manage safety.

Pearls

I've audited or reviewed quite a few SMSs - only a few have showed any real consideration of the SMS concept and were tailored to suit the operator's needs. These were often the better performing systems and they bore little resemblance to the "four pillars".

At the Bali conference, I spied the completely different approach taken by Bombardier. It was mentioned a number of times that it is copyright, so I haven't included a picture here but you can find a presentation outlining their approach on the Transport Canada website. I can't comment on the effectiveness of the system but it is definitely food for thought and a ray of hope that the SMS concept is being considered, digested, pondered, manipulated, tailored, and so on.

1. It's a closed group, so I'm not sure who is able to see the discussion.

"Compliance Doesn't Equal Safety"

I've heard this saying quite a bit over the last few months and in at least one aspect, I agree with the statement. It tends to be true that regulations have failed to keep pace with industry. As such, blind compliance with the regulation no longer ensures an accident-free existence.

Apparently, there is a Venn diagram making the rounds. I've been told it looks like this:

Safety Doesn't Equal Compliance

So what is the solution?

In most conversations, I heard "systems-based" or "risk-based" auditing touted as the answer. Unfortunately, that, in my opinion, is not the answer.

Now, don't get me wrong. Please. Old check-box auditing is not the answer either.

So what is? Let's break it down into pieces...

Firstly, we need to define "compliance". Often, people infer from compliance, a high level of prescription within the legislation, regulation or standard. In the past this was true but nowadays, it's becoming less the case. Regardless, when auditing, especially in a highly-regulated environment, the standards are everything. Issuing findings outside of the standards is unacceptable and you can forget about enforcement action.

We have no choice but to move the "compliance" circle to match up with the "safety" circle. Easier said than done.

The above Venn diagram is mis-leading. Safety is not a neat circle. It's not the same for each industry sector, or even between operators within the same sector. And it's not stable. Not only is aviation a, generally, growing industry, it's diverging as well. "Safety" could probably be drawn like this instead:

The safety profiles of various operators differ

That leaves old-school compliance with two options. Be a well-rounded set of requirements aimed at achieving a generally good level of safety or becoming a complex monster of requirements aimed at ensuring safety across all areas - see below. In the first case (C), gaps between safety and compliance still exist (just to refresh - that means an operator could comply with the rules and still not "be" safe). And in both cases (C & D), all operators are burdened by requirements which have nothing to do with safety in their environment.

The different ways to apply old-school compliance

I mentioned above that the standards are becoming less prescriptive. The still fairly new concept of safety management systems is a different kind of regulation. Instead of telling operators how to address known safety risks, it requires operators to establish a system to identify, assess and mitigate risk within its own environment1. This approach can fill the gaps between traditional compliance and safety. It also goes a long way to supporting "systems-based" and "risk-based" auditing.

Diagram E is probably a good indication of where we are at the moment. In most cases, an SMS requirements has been added to the existing regulatory regime and a "fill the gaps" approach has been taken. This is a valid approach. SMS is relatively new and the industry needs time to grow into the philosophy.

The way compliance looks now and may look in the future

In the future, I imagine it will be possible to reduce the prescriptive side of the compliance equation. A greater level of flexibility is obviously good for the industry as it would reduce the nugatory regulatory burden on some operators but ensure a level of safety acceptable to the regulator.

That retreat will require careful planning and sure-footed execution. A lot of work will need to be done before then, but I'll hold on to it as my "I have a dream" concept2.

1. It actually requires more than a risk management system but lets keep things simple for the sake of this short but rambling post. Check out the link above for more information. 2. I know it's nowhere near the level of nobility of the original but each of us hopes to influence the world in our own way.