According to Kirkpatrick’s evaluation framework, evaluation can be conducted in four different levels: reaction, learning, behavior and results. We can use different methods for evaluation, and the methods should be Reliable, Valid and Practical.
Reliability: whether the evaluation is consistent and free from errors
Validity: whether the method is evaluating what is supposed to evaluate
Practicality: whether there are enough resources to support
And Ernst and Young, one of the “Big Four” Accounting firms also adopt this framework for the evaluation of XXX program, take the training for presentation and pitching skills as example:
- Reaction: Fill in the happy sheet after training
After the training, trainees are required to fill in a Happy Sheet, which can tell more about their feelings towards the training, such as the perceived usefulness and satisfaction level to the trainer etc.
Reliability: Low, the evaluation is influenced by emotions and feelings
Validity: Low, Satisfaction and feelings are not equal to effectiveness of the program
Practicality: High, easy to conduct happy sheet form and gather information
- Learning: MC test
After the training, or during the process of different chapter, trainees are required to take part in MC tests to evaluate how much learning did they demonstrate immediately after the training.
Reliability: High, based on what knowledge that they have learnt, free from influence of emotions
Validity: High, the higher scores can represent a higher effectiveness
Practicality: High, MC test is easy to conduct through computer, mobile phones or pens
- Behavior: Simulation
During the training, trainees would have some simulation about the skills they learn like the presentation and pitching skills, to evaluate how well can they demonstrate the newly learn skills and their training performance
Reliability: Low. The scores assigned during simulation can be subjective according to each judge.