Evaluating Learning

The latest learning theory we just covered in P540 was Gagne's Theory of instruction. Gagne' developed nine events of instruction. Assessing performance was number eight in his list. The goal of this blog is to link the article I reviewed and Gagne's assessing performance event. The author of this article discussed the relationship between Kirkpatrick's four levels of evaluation, Philips ROI in assessing e learning.

Kirkpatrick developed four levels to assess learning. These four levels have been in use for the past 30 years due to their simplistic nature and value they can bring. The four levels are:

• Level 1: The student's reaction or how well they were satisfied with the course content, materials, and the instructor.
  
• Level 2: Assesses whether the student learned the training content.
  
• Level 3: Identifies whether there was a behavior change in the work place due to the training.
  
• Level 4: Impact on the business, based on specific terms such as cost reduction, quality and quantity improvements.

Kirkpatrick also encouraged trainers to borrow approaches, techniques and methods from other departments from within the business (Galloway, p.22). It is imperative that trainers connect with the business operations and Human resources departments to assess the impact of the learning on the business, retention rates, and overall employee's job satisfaction.

Level I (Smiley Sheet): This reactive evaluation allows the learners to assess the training content, materials, and the manner in which the facilitator presented the information. There are certain factors that may interfere with the validity of the results. For instance, the motivation of the student, distraction, resentment of the time the training takes (Galloway, p.22). Other factors may fall under the instructor lack of understanding of the value for the level I. It is imperative that the facilitator explain the value of completing the survey completely and accurately. Other options include providing the survey at the start of the class and ask the students to fill their comments throughout the day.

Level II (Test the Learning): The most common Level II method used is to test the students’ knowledge based on objectives. Some courses may require the learner to pass a qualification exam to obtain licensure, job, etc... An alternative to the test phobia is to provide the learners with a pre-test based on certain homework reading and then to offer the same or slightly modified test at the end of the class to assess whether learning had occurred. Gagne' indicated that the first event of instruction is to gain the attention of the learner. One option is to provide the learner with the course objectives and ask them to place a checkmark next to the objectives that are most important to them. At the end of the class, the learners are asked to answer whether they had achieved the required basics for the desired objective. At this point, the instructor may ask the learner to go over certain materials again or offer additional resources to further the learner's interest in the topic.

Level III Behavior: The third level of Kirkpatrick model, which evaluates behavior and how the training session translates into employee's knowledge and aptitude (Galloway, p.23). The majority of facilitators do not conduct level three evaluations on all of their courses. However, it is recommended that you conduct level 3's on new courses or those that are new to the organization. One option would be to conduct interviews with the learners 60 or 90 days post the class to see if the learner has seen improved results in their workplace. It is also imperative to interview their managers to assess whether behavior has taken place.

Level IV Results: Level four is the level most anticipated by management because it produces evidence that can be related to increased sales, reduced costs, increased productivity, and lower overhead (Galloway, p.24). In today's environment, all training courses have to demonstrate cost-effectiveness and positive impact on the business.

Phillips ROI: Jack Phillips added a fifth level to asses ROI of training. We are discussing dollars and cents at this junction. There are various costs that are involved with training. The cost of the learner and the facilitators away from their jobs. The cost of materials, upkeep etc... It is also important to share the ROI with management to sustain the value training brings to corporations.

The majority of the learning theories discuss how the learner will assimilate the lessons. They all have a similar goal to improve the learner’s retention rates. Conducting a course evaluation allows the facilitator and the instructional designer to assess their training and identify what needs to be revisited.

Resource:
Galloway,D. (2005). Evaluating Distance Delivery and E-learning.Performance Improvement,Volume 44,Number 4,21-26.