It surprises me that many people I speak to don’t know about the Kirkpatrick Model of Learning Evaluation. It has been described before in the SAP Community in this Would You Take Training From This Man? by Marilyn Pratt, but it's very well worth knowing about.
Why you should care about how to evaluate training
If you work in a regulated industry, it’s common that there is a legal requirement for organizations to check the quality of their training – for instance the UK Financial Services Authority states "Firms should ensure that their employees' training needs are assessed at the outset and at regular intervals (including if their role changes). Appropriate training and support should be provided to ensure that any relevant training needs are satisfied. Firms should also review at regular intervals the quality and effectiveness of such training." (my italics)
However whatever industry you work in, you will want to ensure that your company is getting good value for money and resources. For instance the USA alone spends over US$125 billion per year on training.
Kirkpatrick Model
Professor Kirkpatrick came up with his model in 1959, and it has defined the landscape for evaluating training. It has four levels as shown in the diagram below.
Level 1 of the model measures participants reactions immediately after training : did they like it, was it relevant to their work and how could thetraining be improved? Level 1 is usually measured by a course evaluation survey or smile sheet, indeed these are often called level one surveys.
Level 2 measures knowledge - did the participants learn? It's all very well discovering the participants liked the learning, but did they actually improve their knowledge and skills. It can be measured in other ways, but it's often measured with objective tests, or by checking the difference on pre and post course tests.
Level 3 measures behavior - whether the participants were able to apply their learning on the job. Essentially you compare the performance pre-training with the performance post-training and measure the difference. Level 3 is often measured by surveys or interviews, or by observational assessments where an observer monitors how someone is doing and fills in a checklist (sometimes now on a mobile device).
Level 4 measures results for the business, that the training has actually impacted the business. This is the hardest to measure, but obviously the most important.
A key characteristic of the model is that the levels build up from one another, and it's important to measure at every level. If participants do not react well to training (level 1), then they are unlikely to learn. And if participants do not learn (level 2), then they are unlikely to improve behavior. And if they do not change behavior (level 3), then results will not improve. Achieving results in level 4 is usually due to success at levels 1, 2 and 3.
For more information on the Kirkpatrick model, refer to Donald L. Kirkpatrick's book "Evaluating Training Programs : The Four Levels" . There is also a Powerpoint in Questionmark's Learning Cafe that gives a good introductory description.
Other evaluation models
Professor Kirkpatrick is still alive and I had the honour of meeting him at a Questionmark user conference a few years back. Inevitably there have been further ideas in the area of learning evaluation over the years. Some I'd highlight are:
- Jack Phillips in the 1990s added an unofficial 5th level : Level 5 as measuring Return on Investment, how much return a company makes on its training.
- Josh Bersin has developed the Impact Measurement Framework which is an end-to-end model for measuring impact of training, well worth reading about.
- Robert Brinkerhoff has introduced the Success Case Method which evaluates training by looking at where it's most and least successful. See here for a blog article that explains this.
- And very recently Dr. Will Thalheimer is defining a Learning Landscape Model which presents a very insightful view of measuring learning. See here for a great video on Youtube that introduces this.
If you already know of the Kirkpatrick model, I hope this is a useful reminder. If you've not come across it before, I hope you agree it's a great way of thinking about whether training is effective and how training can help business.
