Evaluation Framework

LTEM, and an honest answer to where surveys fit

Will Thalheimer built the Learning-Transfer Evaluation Model as a corrective. Most learning measurement stops at attendance, completions, and smile sheets, and his eight tiers are blunt about how little those prove. A survey tool writing about LTEM should probably start by conceding the point.

The eight tiers

Thalheimer published LTEM in 2018 through Work-Learning Research. Each tier is a stronger form of evidence than the one below it.

Tier	What's measured	What it proves
1	Attendance	They showed up.
2	Activity	They participated, clicked, completed.
3	Learner perceptions	What they believe about the experience and their own learning.
4	Knowledge	They can recall facts and terms.
5	Decision-making competence	They can make the right call in realistic scenarios.
6	Task competence	They can actually perform the task.
7	Transfer	They use it in their real work.
8	Effects of transfer	The use produced outcomes for the learner, their team, the organization.

The provocation is in the ordering. The three tiers the industry measures most are the three at the bottom. Attendance proves attendance. Activity proves activity. And a happy smile sheet proves the learners were happy, which research has repeatedly failed to connect to whether they learned anything.

The critique is fair. Here's what to do about it.

We sell a survey tool, and surveys live at tier 3. Thalheimer's argument is not that learner perception data is worthless. It's that satisfaction questions collect the least useful perceptions available, and that the industry treats those answers as proof of learning when they're nothing of the sort.

Both halves of that response are workable. You can ask learners much better questions than "were you satisfied." And you can collect perception data at the point where it carries the most meaning, several weeks out, about actual use. That's tier 3 done properly and a survey-based route into tier 7.

Ask about ability, not satisfaction

Thalheimer's earlier book, Performance-Focused Smile Sheets, is effectively a question-design manual for this. Replace agree-disagree satisfaction items with questions about capability and likely use, answered with concrete descriptions rather than numbers on a scale.

Weak (typical tier 3)

"The training met my expectations." Strongly disagree to strongly agree.

Better (performance-focused)

"How able are you to put this skill into practice?"

I couldn't do it yet without significant help
I can handle the basics, with effort
I can apply it in routine situations
I can apply it in difficult situations and could help others learn it

The second version still measures a perception. But it's a perception about capability, with answer options specific enough that the distribution tells you something. ImpactCheck supports custom answer options and labelled scales precisely so questions like this are possible instead of another row of agree-disagree.

Tier 7 without an observation budget

Transfer is the tier consultancies most need evidence for and can least afford to measure directly. Observing participants at work, weeks after a program, across multiple client organizations: almost nobody has that budget.

The affordable route is delayed self-report. Six to eight weeks after the program, ask about specific behaviors, not general impressions: "I give my direct reports specific, actionable feedback within a week of observing an issue," rated before the program and now, in one sitting. The retrospective format exists for exactly this, and the delay means you're asking about something that has had time to happen, or not.

Is self-reported transfer as strong as observed transfer? No, and it would be silly to claim otherwise. It's evidence at tier 7 rather than proof, collected at perhaps one-hundredth the cost of observation. Self-reported behavior change at eight weeks beats perfect satisfaction scores on the day, by a distance.

What we don't claim

Tiers 4, 5, and 6 need knowledge checks, scenario assessments, and task demonstrations. Those are assessment instruments, and ImpactCheck isn't one. Tier 8 needs organizational data a survey can't reach. If a vendor tells you their survey tool covers all eight tiers, they haven't read the model.