Evaluation Framework
Robert Brinkerhoff's method starts from an observation most evaluators would rather not dwell on: averages hide what training actually does. A few participants turn a program into concrete results. A few never use it at all. The useful information lives at the edges, and a mean score of 4.1 flattens it.
Brinkerhoff published the Success Case Method in 2003, and the design has stayed simple: two phases, one cheap and one careful.
A handful of questions, sent to all participants some weeks after the program. Its only job is to sort: who has used what they learned and achieved something concrete, and who hasn't used it at all. This is not the evaluation. It's the net that catches the cases worth examining.
From the survey, pick a sample of the most and least successful cases and interview them properly. The success interviews document what happened, with enough verifiable detail to survive a skeptical audience. Brinkerhoff's standard is courtroom-grade: nothing goes in the report you couldn't defend with evidence. The non-success interviews are just as valuable, because they tell you what blocked everyone else.
The output is a small set of documented stories plus a diagnosis. Not "the program scored 4.2," but "here is what the program produced when it worked, here is what stopped it working for the rest, and here is what that gap is costing you."
Brinkerhoff's research across organizations kept finding a similar shape: a small group, often something like 15% of participants, applies the training and gets clear results. A similar-sized group never tries. Everyone else tries a bit and drifts back. The exact split varies, but the shape is stubborn.
His conclusion was that training rarely fails or succeeds on its own. When transfer fails, the system around the training failed: the manager never followed up, there was no chance to apply the new skill, the wrong people were in the room in the first place. A satisfaction survey can't see any of that. Interviews with the people at the extremes can.
There's also a blunt commercial reason consultancies like this method. A documented story, with a name and a number attached, persuades executives in a way no averaged scale score ever will. One verified "this team recovered a £200k account using the approach from the workshop" is worth a deck of bar charts.
The Phase 1 survey should be short enough that nearly everyone answers it. Three to six questions, sent six to twelve weeks after the program. Something like:
"To what extent have you used what you learned in this program?" with anchored options from "I haven't used it" through "I've used it and it has produced clear, concrete results."
"Describe the most significant result you've achieved by applying something from the program." (open text)
"If you haven't been able to use it, what got in the way?" (open text)
The entire point of the screening survey is to identify whom to interview. So unlike most evaluation surveys, this one needs names, with that disclosed plainly to respondents. In ImpactCheck this is a named survey: each response carries the participant's name and email, individual answers are restricted to account admins, and respondents see exactly what's being collected.
Sort the responses, take the top and bottom handful, and your interview list writes itself.
Phase 2 is consultant craft, not tooling. Picking five to ten cases from each extreme. Interviews that push past "it was great" to the specifics: what exactly did you do, what happened, who else saw it, what's it worth. Verifying claims before they go in the report. Estimating unrealized value, which is the method's quiet weapon: if the top 15% produced these results, what's on the table if the middle 70% had the same manager support?
ImpactCheck deliberately stops at the data collection. The screening survey, the named responses, the sorted list. The interviews and the storytelling are yours.
Named surveys with clear disclosure, short anchored questions, and per-person responses your admins can sort to find the extremes.
Start your free trial30-day free trial. No credit card required.