Analyzing the results of training programs is extremely difficult. L&D teams are challenged to demonstrate the value that programs deliver to both individual employees and the organization as a whole. Unfortunately, few learning leaders are able to do this.
“92% of senior leaders and stakeholders want to see quantifiable results from their training functions. Yet only 9% of internal L&D organizations show them anything,” notes Paul Leone, Ph.D., an industrial and organizational psychologist and training measurement expert.
So then, how should L&D teams go about going beyond the smile sheet to measure ROI?
Enter the Kirkpatrick Evaluation Model, perhaps the best-known model of evaluating the results of training programs. The Kirkpatrick Evaluation Model is widely used because it measures training outcomes at multiple levels. So, organizations can use the model according to their own definition of the word “evaluation.” What might be important to one team in one organization might not be to another.
Are you showing your senior leadership team how much ROI your training initiatives are delivering to the bottom line?
Watch this on-demand webinar, with author and learning measurement expert Paul Leone, Ph.D., to create your “story of impact” and show robust results to your business partners.
The Kirkpatrick Evaluation Model was created by Donald Kirkpatrick and measures training through four levels, each of increasing depth or complexity:
This basic level asks learners about how they felt about the training. Questions might be as simple as:
These questions are usually delivered via a simple post-course questionnaire. While self-reported answers from learners might seem superficial, there is still value in providing Level 1 evaluation.
Level 1 can be helpful for instructional design teams to broadly understand whether the course resonated with learners. For example, overwhelmingly negative or even neutral reviews are a clear review that things could have been better.
This next level of Kirkpatrick seeks to help L&D managers determine whether learning occurred or any knowledge/skills were acquired. It gives a chance for learners to apply what they learned.
To do this, instructional design teams will usually deliver several assessments. There can be the expected one at the end of the training, but this doesn’t help managers understand how much they learned.
For a deeper look at how much knowledge or skills were gained, the L&D team can give multiple assessments. For example, one assessment at the beginning and then one similar one at the end of the course—or even months after the training has passed. This insight helps to determine how much the learner already knew and how useful or impactful the training really was.
Kirkpatrick’s Level 3 goes even deeper than Level 2 measurement. It seeks to determine just how much of the training is impacting the employee’s day-to-day work. It also looks at whether the training is relevant to their jobs.
This type of behavior measurement requires quite a big leap from Level 2 of the model. It does not rely simply on answers to post-training assessments. To carry out Level 3, learning leaders will need to garner much more feedback, including that from co-workers, managers, and others who work regularly alongside the learner.
If the training course has had the desired effect, it will be noticeable—both qualitatively and quantitatively—to everyone involved. Additionally, a lack of feedback is also itself feedback. Subtle clues can help L&D better understand whether the training had any influence on the employee’s work.
The highest level of the Kirkpatrick Evaluation Model has L&D determine whether the training results in a positive impact on the overall organization and key business performance indicators.
For some departments or domains, the desired results might be clear: sales training led to more sales, which affected revenue. Or, stronger customer care training led to fewer returns or chargebacks.
Beyond scenarios like these, it can be challenging for L&D to determine the organizational impact of training. For example, measuring the impact of an empathy or leadership training course. To measure results and the impact of these types of training, L&D need to work with managers to determine metrics that can serve as the signals of positive employee performance.
You and your team put a lot of time and effort into creating training. Prove your value to the rest of the organization by using this free Training ROI Calculator.
The beauty of the Kirkpatrick Model is that any level can be used anytime in any learning setting. Level 1 might be used immediately just to garner immediate learner feedback. But Levels 2 and 3 might not be implemented for 1-2 months after the training has passed.
Kirkpatrick also helps teams better understand where instructional design needs to improve. For example, say that in Levels 1 and 2 there is consistent feedback that the courses did not meet expectations or assessments showed poor retention. This demonstrates an issue not be with learners, but with the courses or course design.
Measuring the impact of your training not only earns you more respect and partnership from business leaders. But you will also receive more recognition and rewards for your hard work—including more budget.
This budget could be used for hiring more designers and developers or outside subject matter experts. These extra resources can build courses based on training requests that you, unfortunately, had to turn down previously.
For some organizations, the Kirkpatrick Model doesn’t go high enough. Level 4 might seem the highest, as it seeks to measure the impact to the organization as a whole. However, it might still be too vague for some organizations.
As a result, some L&D teams have added two additional levels: Level 5 and Level 6.
In some organizations, Level 4 is pushed to Level 5, a Return on Investment (ROI). This measures any financial impact the training initiative has on the organization, such as increased revenues and lowered costs.
So Level 4 should measure, as a whole, the positive performance and behaviors studied in Level 3.
Dr. Leone has Level 4 include a control group and attribution technique. L&D should track both trained and untrained participants, in order to measure learning impact by groups. In this way, an organization can measure the attribution of training to large groups of employees.
Beyond this level, an organization can go deeper to seek the financial impact, which would comprise Level 5.
At this level, Dr. Leone suggests that organizations focus on the factors that are influencing the impact of a training program. Many factors beyond obvious training outcomes can easily influence the results at each level of impact. Without understanding and addressing these factors, the organization will never improve, no matter the quality of the training.
For instance, Dr. Leone suggests examining which “climate factors” will influence the impact and ROI of a training initiative. For example, superior manager support may lead to behavioral improvements (Level 3), greater business impact (Level 4), and a higher ROI (Level 5).
While tools to collaborate more effectively and scale course development more rapidly are important for efficiency, the ability to measure whether those courses made an impact is critical for the L&D function to thrive.
If you’re looking for ways to tell the story of your L&D team’s impact back to stakeholders in the organization, check out how Cognota provides operational insights from your L&D processes such as training intake, capacity and project planning, and course creation. See for yourself with a 14 day free trial!
A well planned and allocated training budget enables your L&D department to operate as efficiently as possible. Download this free eBook, “How to Effectively Calculate and Allocate Your Training Budget.”
