Case Study | Measurement | Learning Analytics

Building a Measurement Practice Leaders Could Use

TL;DR I built measurement around decisions leaders could make. Completions and satisfaction still had a place. The stronger practice used readiness evidence, transfer signals, manager validation, and review cadence so the data could lead to keep, change, scale, or retire decisions.

The challenge

Learning teams often have data. The problem is whether the data helps anyone make a decision.

Completions, satisfaction, and quiz scores are easy to collect. They can show reach and experience, while transfer needs stronger evidence.

The challenge was to build a measurement practice that was honest about attribution and still useful to leaders.

The approach

I used a decision-first measurement model. Before choosing a metric, I asked what decision the metric would support.

Readiness evidence helped show whether learners could perform before the work reached customers or teams. Transfer signals helped show whether the learning changed behavior in the workflow.

I was careful with attribution. Learning is rarely the only factor. The goal was to build credible evidence without overclaiming.

Downloadable takeaway

A one-page version of the model with the decision questions, sequence, metrics, and red flags someone can use after reading the case.

Download PDF

What I built

Added readiness evidence

Role plays, scenario scoring, manager checks, and final assessments gave stronger signals before people were fully in the workflow.

That mattered because waiting for lagging business outcomes can be too slow for program improvement.

Used transfer signals

Time-to-proficiency, support tickets, manager validation, quality patterns, and customer-facing errors became part of the evidence chain where available.

These signals were more useful when reviewed with the teams that owned the operating data.

Created a review cadence

Measurement had to show up in recurring reviews. Otherwise dashboards became decoration.

The review question was practical: keep, change, scale, retire, or investigate.

Operating artifacts

These are sanitized work-product examples. They show the kind of artifact I would expect the team to use. They are sanitized and exclude confidential company material.

The results

3 Transfer measures used: time-to-proficiency, support tickets, and manager readiness.
637 Modules governed through a portfolio measurement framework.
4 Program iterations driven by measurement data.
4 Leadership audiences included in review cadence.

The operating insight

The measurement conversation in L&D often gets trapped between easy activity data and impossible attribution claims. A useful practice lives in the middle: credible evidence tied to decisions.

The out-of-the-box move was treating measurement cadence as part of operations. The data had a scheduled place to change the work.

What this proves

  • I can build learning measurement that leaders can use.
  • I know the difference between activity, readiness, transfer, and business outcomes.
  • I can make careful attribution without hiding from business impact.