Evaluation Framework
Jim and Wendy Kirkpatrick's revision of Don Kirkpatrick's four levels keeps the structure and changes the working method. Plan backwards from Level 4, watch the behaviors that drive results, and judge success against stakeholder expectations rather than a financial return. The practical effect for anyone running evaluations is a longer and better list of questions to ask.
The 2016 book formalized changes the Kirkpatricks had been teaching for years. Four of them change what you measure.
The original model is usually applied bottom-up: run the program, survey reactions, hope someone eventually checks results. The New World version starts at the end. Define the organizational result first, identify the few critical behaviors that would produce it, then design the training and the evaluation around those. Evaluation planning becomes part of program design instead of an afterthought.
The most useful addition. Behavior change mostly fails for reasons outside the classroom: nobody monitored it, nobody encouraged it, nobody rewarded it. The model names these reinforcing systems "required drivers" and insists they be planned and measured. This converges with what Brinkerhoff found from the other direction: when transfer fails, the system failed, not the training.
For evaluation, that means asking about the environment, not just the participant. Did your manager discuss the program with you afterwards? Have you had real opportunities to use it?
Level 1 grows from satisfaction to satisfaction plus engagement plus relevance. Level 2 grows from knowledge and skill to include confidence ("I know how") and commitment ("I intend to"). Both expansions exist because the original questions were weak predictors of anything. A participant can be delighted and unchanged.
Organizational results take quarters to show up. Leading indicators are the short-term signals that things are on track: customer complaints trending down, internal quality scores moving, pipeline activity shifting. They keep Level 4 honest without waiting a year for the annual numbers.
The New World model's answer to Phillips is Return on Expectations. Instead of converting outcomes to money and dividing by costs, you negotiate with stakeholders up front: what would success look like to you? Then you convert those expectations into observable outcomes and report against them.
The Kirkpatricks' argument is that ROI is finance's tool, built for capital decisions, and forcing training results into monetary units often produces numbers nobody believes. ROE trades the percent sign for stakeholder agreement. The catch is discipline: ROE only works if you actually capture expectations before the program. Defined afterwards, it collapses into "we did roughly what we hoped," which persuades no one.
In practice the two coexist. Consultancies run ROE language with HR sponsors and keep the Phillips machinery for the engagements where a CFO holds the pen.
Most write-ups stop at describing the model. The usable part is that each revision implies specific survey questions the original model never asked for.
Level 1, relevance
"How relevant was this program to the work you'll be doing in the next three months?"
Level 2, confidence and commitment
"How confident are you that you can apply what you learned?" and "How committed are you to applying it?"
Level 3, required drivers (delayed survey)
"Since the program, has your manager discussed it with you?" and "Have you had genuine opportunities to apply what you learned?"
The required-driver questions earn their place twice over. They predict transfer, and when transfer fails they hand the consultancy a diagnosis that isn't "your training was bad": the client's managers never followed up, and here's the data. Sent as a delayed retrospective survey, the behavior questions and the driver questions travel together in one instrument.
Relevance, confidence, commitment, and required drivers, in branded micro-surveys your participants finish in three minutes.
Start your free trial30-day free trial. No credit card required.