Evaluation Framework
Peter Warr, Michael Bird and Neil Rackham published CIRO in 1970, in a book about evaluating management training. Half the model happens before anyone walks into a room. That ordering is its lasting lesson, and the reason it still appears in every comparison of evaluation frameworks fifty years on.
What does the situation actually require? CIRO assesses needs at three altitudes: the ultimate objective (the organizational deficiency to fix), intermediate objectives (the behavior changes that would fix it), and immediate objectives (the knowledge and skills behind those behaviors). Evaluating context means checking the training was aimed at a real problem.
Given the objectives, were the right methods and resources chosen? Internal or external delivery, which design, what cost. Input evaluation judges the program design against the alternatives that were available, which is a question most post-program surveys never think to ask.
Participant views, with a twist that distinguishes CIRO from a satisfaction survey: the authors treated reactions primarily as suggestions for improving the program, not as a verdict on it. The question is less "did you enjoy it" and more "what would have made this work better."
Results, measured at the same three altitudes the context stage defined: immediate (what was learned), intermediate (behavior on the job), ultimate (the organizational effect). If the objectives were written properly at the context stage, outcome evaluation is just checking them off.
A side note for sales-trivia enthusiasts: the Rackham in Warr, Bird and Rackham is the same Neil Rackham who later wrote SPIN Selling, built on the same instinct that you should research what works before prescribing it.
As a complete process, CIRO shows its age. It was written for 1970s management training, it has little to say about how to measure outcomes, and its stages assume an evaluator with access to the original commissioning decisions. Almost nobody runs the full cycle today.
Two of its instincts survived because they were right. First: evaluation starts before delivery. If the objectives were never defined at three levels, no amount of post-program surveying can tell you whether the program met them. The New World Kirkpatrick model rediscovered this in the 2010s as "begin with the end in mind."
Second: reactions are improvement data. Asking "what should change about this program" produces something a designer can act on. Asking "rate your satisfaction from 1 to 5" produces a number to put in a report. CIRO chose the former in 1970, and the open-text improvement question remains one of the most reliably useful items on any evaluation survey.
"How relevant was this program to the problems you face in your role?" (context, asked of the participant). "Was the format right for this material?" (input). "What one change would most improve the program?" (reaction, as the authors intended it). Then outcome questions on a delayed retrospective survey, written directly from the immediate and intermediate objectives.
Scale questions where they belong, open-text improvement questions where they earn it, and outcome questions written from your objectives.
Start your free trial30-day free trial. No credit card required.