Using Evaluation to get the Most from Training

Robert O. Brinkerhoff, Ed.D

Why is it important for companies to formally evaluate success on an initiative? Why isn’t it enough to just look at the outcome?

Let’s first define “success.” When you define training success as “on-the-job usage of newly learned content to positively impact organizational performance,” only about 15 out of 100 training participants ever succeed. Almost all training programs are somewhat successful, but overall, not enough; an 85% scrap rate is pretty disappointing.

If you’re serious about getting results, then you have to evaluate not only what success you’re getting (improved performance), but uncover the key factors in your performance management environment that sometimes enable but sometimes cripple your success from training. Using these data to drive improvements in coaching from managers, or compelling more buy-in and commitment to training goals from senior leaders, for example, are the keys to creating an organization where learning is consistently turned into exemplary performance, not just some of the time, but almost all of the time.

Here are the basics of evaluating a change initiative. Ask:

  • When it’s working, what good is it doing?
  • How often is it working that well?
  • When it’s working, what’s making it work?
  • When it’s not working, what keeps it from succeeding?

And then the clincher question:

  • What do we have to change and do differently to get better results?

Surveys, such as follow-up questionnaires, are helpful in finding out where the basic issues lie. But surveys will rarely give you enough data and evidence to draw good conclusions, nor do they dig deep enough into details to help you understand critical causes and factors that are actionable. A good evaluation survey will be targeted specifically to what behaviors are being employed in executing a change initiative, and will be carefully designed to reduce false reports. But again, don’t rely on survey data for all of your evaluation inquiry – supplement it with objective performance records and selected interviews to learn the story behind the raw numbers.

We regularly use the Success Case Method® (SCM) to assess training effectiveness because it uses a tightly focused survey to collect self-reports on who is converting learning into improved performance behaviors. Then, the SCM prescribes probing interviews with carefully selected samples of survey respondents to establish the business value of the changed performance and pinpoint the make-or-break success factors that can be managed in the future to accelerate success.

The stories of success highlight how real employees are taking real actions to achieve real results that senior leaders value and believe. These evidence-based stories go a long way in building commitment to change initiatives. Better yet, they make a compelling business case for the two essential ingredients of training success: having managers set clear expectations for not just participating in training, but applying new learning on the job; and having these managers and their bosses hold everyone accountable for using training to drive improved performance.