Is training to blame when there is an accident/incident?

Throughout my career in training, and especially in the railroad industry, I have seen how training often bears the brunt of the blame when there is an accident or incident; sometimes directly and sometimes indirectly through identifying re-training as remediation. But is training to blame? Simply put, if the accident or incident was caused by a lack of knowledge, skill, or ability, then, yes, training is part of the root cause. But how do you confidently assess if that is the case?

The Goal is to Get to the Third Level of Training Evaluation

For my railroad colleagues, stick with me here while I summarize the four levels of evaluation; I promise, I will circle back to why this is important in a moment.

Before writing this post, I dusted off my copy of Evaluating Training Programs: The Four Levels by Dr. Donald L. Kirkpatrick (And yes, I have an actual copy of the book, I'm not sure if it is in print anymore). While the landscape of training has changed significantly in how we deliver training, the fundamental training theory on how we learn and measure training success has not. And, Dr. Kirkpatrick's evaluation theory, first published in 1959, remains to be the authority on how to best measure a training program's impact.

Let me try to summarize. In order to effectively measure training, you must measure four levels. Further, without measuring levels one and two, you cannot measure level three; and you must measure levels one, two, and three to measure level four. The levels are:

Level 1: Reaction

This level measures the reaction of training participants. Did they enjoy the training? Did they believe it provided relevant information to their jobs? Many training programs only measure this level. In a sense, this is a customer satisfaction survey.

Level 2: Learning

Dr. Kirkpatrick summarizes this level as measuring "the extent to which participants change attitudes, improve knowledge, and/or increase a skill." This level is measured with a knowledge-based exam like a multiple-choice test and/or hands-on exam that uses a defined checklist of tasks with conditions and standards. (Sound familiar to anyone?)

Level 3: Behavior

Level 3 measures a change of behavior on the job. We have some tools in the railroad industry to measure level three (such as efficiency tests outcomes), but to measure it effectively you need to first measure the first two levels and have baseline data from before the training program was implemented. This level is best measured (in my opinion) through observation of work behavior but surveys that take into account confidence levels are also a possible option.

Level 4: Results

I have heard training professionals joke that this is the Holy Grail of training evaluation. Level 4 determines if the training impacted the organization's safety performance, profit, and/or customer satisfaction. What is the impact/results of the training program on the organization? This level can definitely be measured successfully but it requires an intentional training plan that includes the entire ADDIE process.

Some trainers want to bypass levels 1 and 2 - reaction and learning - in order to measure changes in behavior. This is a serious mistake.  - Dr. Donald L. Kirkpatrick

So, finding the answer to if training is to blame when there is an accident/incident depends on your ability to measure Level 3.

I was taught as a grad student that 80% of the time training is not to blame for a lack of change of behavior. I am not sure if was just my professor's use of the 80/20 rule but it illustrates the fact that there is a lot of factors besides knowledge, skill, and ability that impact employee behavior.

Again, Dr. Kirkpatrick explains in his book that in order for a change of behavior to occur, four conditions are required:

1. The person must have a desire to change.
2. The person must know what to do and how to do it.
3. The person must work in the right climate.
4. The person must be rewarded for changing.

So much can be said about these four conditions. But notice that only #2 relates to training. #1 directly connects to individual responsibility. And, we could make a case that the other two (#3 & #4) are directly related to the work environment. So what bears the burden most of the time when there is NOT a change in behavior on the job? ...Work environment.

But that 20% is important.

Ensuring employees have the knowledge, skill, and ability to perform their jobs safely and effectively is critical in preventing accidents/incidents in the railroad industry. However, years of training theory would suggest that this is just the beginning and foundational step in impacting on-the-job employee behavior.

Why employees do certain things is so much more complicated than determining if they were trained properly. And while it is easy to blame training when things go wrong, I encourage you to consider the four conditions Dr. Kirkpatrick states must be present for a change of behavior to occur and ask yourself:

• Are employees motivated to change?
• Do employees know what to do and how to do it?
• Does our company have a healthy safety climate to support safe behavior decisions?
• How is safe behavior rewarded at my company? Is unsafe behavior rewarded?

So, is training to blame when there is an accident/incident?

The factors that impact the cause of an accident or incident are even more complex than determining changes in on-the-job behavior. While training tends to be one focal point of accident/incident analysis, I hope this article has helped you understand that it can be difficult to determine if employees had the proper knowledge, skill, and ability without an intentional training evaluation strategy.

Our industry has invested and continues to invest in the training of its employees. I believe that intentionally measuring how that investment impacts employee behavior, and ultimately safety performance, helps inform future training and safety initiatives to continue to move the organization forward in supporting employees with what they need to complete their jobs safely.