Methods and Models for Evaluating Instructional Experiences

There are other strategies, methods, and models for evaluating the influence of training and instructional experiences of any form beyond the Kirkpatrick Model, CIPP Model, and Success Case Method. However, an awareness of these three will provide you with a solid foundation for understanding the importance of evaluation of instructional programs, particularly workforce training programs, which are often the focus of an instructional designer’s role in an organization. These three models are discussed in brief in this post.

Kirkpatrick Evaluation Model

Many sources of information about the Kirkpatrick Four Levels Model can be found online, including the website overview about the model from the Kirkpatrick Partners (2015) website where The Kirkpatrick Model is explained.

Kirkpatrick Level Chain

The Kirkpatrick Model’s Four Levels are Interrelated and Interdependent (image source: KirkpatrickPartners.com)

I would argue that, just as the ADDIE Model for instructional design has components that are the essence of nearly every other instructional design model, the levels of the Kirkpatrick Model tend to appear in some form in other evaluation models as well. It is likely for this reason that the ADDIE Model and Kirkpatrick Model continue to widely utilized.

Although the Kirkpatrick Model, developed by Donald Kirkpatrick in the 1950s, is one of the oldest formal training and instruction evaluation models and widely used even today, it is not the only model in existence. In fact, as you can learn about from the Kirkpatrick Partners (2015) website, there is a revised version of the standard four level model by Kirkpatrick called the New World Kirkpatrick Model, which may be of interest to you to explore.

For a review of alternative models not developed by Donald Kirkpatrick, start by considering the blog post about Alternatives to Kirkpatrick from Bozarth (2009) and consider the brief overview of these alternative models provided below to ensure you are familiar with some of the available options, the CIPP Evaluation Model and the Success Case Method. Keep in mind that you may work for an organization that uses a combination of the Kirkpatrick Model and other models and that is quite typical as each organization tends to develop a unique culture of evaluation to suit its needs.

CIPP Evaluation Model

The CIPP Evaluation Model was developed by Daniel Stufflebeam in the 1960s. CIPP is an acronym for Context, Inputs, Process, and Product. These four components of the CIPP Evaluation model are designed to provide a holistic view of the success of instructional initiatives. While the Kirkpatrick Model is a reaction and review oriented model, the CIPP Model seeks to provide a more decision-based approach to evaluation.  A summary can be read from Ivan Teh’s (2015) blog in the post where the following image and a research-based exploration of the CIPP Evaluation Model are provided.

CIPP Model

As Bozarth (2009) noted, the CIPP Model is more about evaluating what is being done during design and implementation than an evaluation of what has been done (para. 4). Mazur (2013) offers a another research-based summary of the CIPP Evaluation Model that is a good review to see for learning about how the model is a useful addition to or complete alternative to the Kirkpatrick Model.

Success Case Method

Another alternative to or addition to the Kirkpatrick Model is the Success Case Method.  For a comprehensive introduction to the Success Case Method, refer to the Brinkerhoff (2009) excerpt of Chapter 1, which is freely available online as a PDF.

SCMKirkpatrickComparison

Comparison of Kirkpatrick Based Models and the Success Case M (Gram, 2011).

For a comparison of the Kirkpatrick-style model’s and the Success Case Method, refer to the image here from Gram (2011) – click the image to view at full size.

As explained in the Brinkerhoff Chapter 1, the Success Case Method provides a sort of rapid-evaluation method that can help to bring focus to a learning initiative while more comprehensive models, such as the CIPP and Kirkpatrick Models, are used for longitudinal analysis of the effectiveness of instruction.

Brinkerhoff (2009) also explains that the “Success Case Method combines the ancient craft of storytelling with the more current evaluation approaches of naturalistic inquiry and case study” (p. 17).

For this reason, the method appeals to many who prefer a deeper focus on qualitative data in an evaluation method (i.e., model) for approaching a determination of the return on investment (ROI) for a performance change initiative.

Remember that instruction is inherently focus on changing performance through increased, reinforced, or revised learning of knowledge and skills.

~ Lisa Johnson, Ph.D.

References

Bozarth, J. (2009, January 17). Alternatives to Kirkpatrick [Weblog post]. Retrieved from http://bozarthzone.blogspot.com/2009/01/alternatives-to-kirkpatrick.html

Brinkerhoff, R. O. (2009). The success case method: Find out quickly what’s working and what’s not (2nd ed.). San Francisco, CA: Berrett-Koehler Publishers. PDF Chapter 1 excerpt available from: http://www.bkconnection.com/static/The_Success_Case_Method_EXCERPT.pdf

Gram, T. (2011). Comparison of Kirkpatrick based models and the success case method [Image file]. https://performancexdesign.files.wordpress.com/2011/02/2011-02-23_2243.png

Kirkpatrick, J. (2007, August). The hidden power of Kirkpatrick’s four levels. Training and Development, 61(8), 34-37.

Kirkpatrick Partners. (2015). The Kirkpatrick model. Retrieved from http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel/tabid/302/Default.aspx

Mazur, A. D. (2013, June 10). The CIPP evaluation model: A summary [Weblog post]. Retrieved from https://ambermazur.wordpress.com/2013/06/10/the-cipp-evaluation-model-a-summary/

Teh, I. (2015, March 12). CIPP evaluation model [Weblog post]. Retrieved from http://ivanteh-runningman.blogspot.com/2015/03/cipp-evaluation-model.html

4 thoughts on “Methods and Models for Evaluating Instructional Experiences

  1. Right here is the perfect website for anyone whoo wants to find out abbout tjis
    topic. You understand a whole lot its almost hard to argue wioth you (not that I really would want to…HaHa).
    You definitely put a brand new spin on a topic that’s been written about for ages.
    Excellent stuff, justt wonderful!

  2. This is an excellent article. I am a student and working on my project and I have chosen to adapt the CIPP Model. I am convinced and I prefer its being holistic. Thank you so much for this wonderful scholarly article, Dr. Johnson!

    • Thank you, Myrene! I am very glad to hear the summary of evaluation models here and the CIPP model information was of value to your education. may I ask what made you choose that model over the others presented in this article? I agree it is more comprehensive, or holistic than the others. Yet, like many models, CIPP has elaborated on assumed points from other models to some degree. Kirkpatrick’s model, like the ADDIE design model, is criticized for simplicity at times. However, the missing desired substance is often assumed in the application of the models when done by experts; e.g., ADDIE is not necessarily a linear model but is mistaken as such given the acronym’s linear presentation. I’d enjoy your thoughts on that assertion! ~ Dr. J

Leave a Reply

Your email address will not be published. Required fields are marked *