Donald Kickpatrick prescribed methods to evaluate the effectiveness of training. He proposed four levels of evaluation starting from Level 1 to 4. . Information from each previous level serves as the foundation for the next level's evaluation, offering, in stages, an accurate reading of the effectiveness of the training program.
Level 1: REACTION: This level measures the reaction of trainees to the training programme.
The main purpose of measuring reaction of trainees is to gauge their motivation and interest level in learning.
Measurement Guidelines:
1. Determine what you want to find out.
2. Design a form that will quantify reactions such as:
- Did they like the training program?
- Was the training material relevant?
- Was the method of delivery effective?
- Did they consider the training relevant?
- Perceived practicability and potential for applying the learning
4. Attain an immediate response rate of 100%.
5. Seek honest reactions.
6. Develop acceptable standards.
7. Measure reactions against the standards and take appropriate action.
8. Communicate the reactions as appropriate.
Evaluation Tools and methods:
The evaluation tools and methods used in this stage are 'happy sheets', feedback forms, post-training surveys, questionnaires, verbal feedback, etc which are quick and easy to gather, and non-expensive to analyse.
Level 2: LEARNING: This level measures the knowledge acquired, skills improved, or attitudes changed to trainees as a result of the training.
Measurement Guidelines:
1. Use a control group, if feasible.
2. Evaluate knowledge, skills, or attitudes (KSAs) both before and after training. Use a paper and pencil test to measure knowledge and skills. Use a performance test to measure attitudes.
3. Attain a response rate of 100%.
4. Use the results of the evaluation to take appropriate action.
Evaluation tools and methods
This stage calls for more rigorous procedures, ranging from formal to informal testing to team assessment and self-assessment. The most common learning evaluation tools are assessments, interviews, observation and tests conducted before and after the training. The questions could be:- Did the trainees learn what what intended to be taught?
- Did the trainee experience what was intended for them to experience?
- What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?
Level 3: BEHAVIOUR: This level measures the transfer of training or if trainees are applying new knowledge, skills, or attitudes on the job.
Measurement Guidelines:
1. Allow enough time for a change in behavior to take place.
2. Survey or interview one or more of the following groups: trainees, their bosses, their subordinates, and others who often observe trainees' behavior on the job. Evaluation focus on questions like:
Are the newly acquired skills, knowledge, or attitude being used by the learners in their everyday work arena?
- Did the trainees use the relevant skills and knowledge?
- Was there significant and measurable change in performance of the trainees when back to their jobs?
- Was the transfer in behaviour retained?
- Would the trainee successfully be able to transfer knowledge to someone else?
- Did the trainees put their learning into effect when back on the job?
- Were the relevant skills and knowledge used?
- Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles?
- Was the change in behaviour and new level of knowledge sustained?
- Would the trainee be able to transfer their learning to another person?
- Is the trainee aware of their change in behaviour, knowledge, skill level?
3. Choose 100 trainees or an appropriate sampling.
4. Repeat the evaluation at appropriate times.
5. Consider the cost of evaluation versus the potential benefits.
Evaluation Tools and methods
During level three, observation and interview over a period of time are required to measure change, its relevance, and sustainability. Evaluation in this area is challenging and is possible only through support and involvement of both line managers and trainees.
Level 4: RESULTS: This level measures the result of training as it relates to factors such as sales, productivity, profit, costs, employee turnover, and product/service quality.Measurement Guidelines:
1. Use a control group, if feasible.
2. Allow enough time for results to be achieved.
3. Measure both before and after training, if feasible.
4. Repeat the measurement at appropriate times.
5. Consider the cost of evaluation versus the potential benefits.
6. Be satisfied with the evidence if absolute proof isn't possible to attain.
Evaluation tools and methods:
The level four evaluation measures how successful a training program is in a context that is easily understood by managers and executives such as
- Better production levels,
- Improved quality,
- Lower costs,
- Higher sales,
- Staff turnover,
- Attrition rates,
- Failures,
- Wastage,
- Non-compliance,
- Customer complaints
- Quality ratings,
- Growth,
- Retention,
- Increased profits
- Return on investment
- Staff turnover
- Organisational performance
Ref: Kickpatric publications, Businessballs, etc
