The MERIT Framework: A Series Introduction
Over the past few months we've talked to a lot of procurement teams about their AI programs. One pattern kept showing up: teams that ran successful pilots couldn't move past them. Not because the technology failed, but because they had no structured way to measure what success actually looked like. Without that, they couldn't build the case to go further.
We call this the MERIT Framework. Five components that give procurement teams a structured way to capture AI value, communicate it to the right audiences, and build the conditions for a program to scale.
- M (Measurement): Define success metrics and capture a performance baseline before any AI goes live. M is the system you build: baselines, accountability, tracking discipline. (Part 1)
- E (Evidence): Build the governance foundation (data security, compliance, auditability) that converts results from claims into something leadership can trust. E is what makes M credible. (Part 1)
- R (Reporting): Translate metrics into stories that land with two different audiences. Leadership needs the financial frame. Procurement users need the operational frame. (Part 2)
- I (Impact): Quantify AI value in financial terms (efficiency gains, quality improvements, capacity freed) that move budget conversations from justification to expansion. (Part 2)
- T (Trust): Build the organizational conditions (learning loops, governance maturity, phased rollout) that let AI programs earn the right to scale from pilot to infrastructure. (Part 3)
How this series is organized
- Part 1 covers M and E: how to build a measurement strategy and evidence foundation before deployment, so that results are credible and defensible when they arrive.
- Part 2 covers R and I: how to report AI results to two audiences in the terms each needs to hear, and how to translate operational data into financial impact.
- Part 3 covers T: how to build the organizational conditions that let a program earn the right to scale, moving from use cases to capabilities to infrastructure.
A procurement AI measurement framework is the difference between an AI program that earns its next budget cycle and one that quietly disappears. Most teams build it too late. They wait until after deployment, when the only available data is vendor dashboards and usage statistics. By then, the window to set a credible baseline has already closed.
Procurement teams running AI initiatives tend to get stuck in one of two places.
The first is before they start. The opportunity is clear: cut contract review time, reduce supplier onboarding errors, automate spend categorization. But leadership won't approve without a credible answer to "what will we actually get from this?" Without a measurement framework, the business case stays speculative. Projects stall at the proposal stage.
The second is mid-program. The tool is running, the team is using it, results are real. Then someone asks at a quarterly review what the organization is getting from the investment. The best available answer is usage statistics and a vendor dashboard. The room nods, nothing changes. Within two budget cycles the program is quietly deprioritized.
Both problems have the same solution: a measurement strategy built before deployment, not after. Here's how to build one.
MERIT Framework: This Article
M (Measurement): the system you build before deployment. Success metrics defined, baseline captured, accountability assigned.
E (Evidence): the governance foundation that makes those results defensible to leadership. M is the data; E is what makes the data credible.
Why AI Programs Lose Funding Without a Measurement Strategy
Whether you're trying to launch an AI program or sustain one, the risk is the same: value that can't be demonstrated doesn't stay funded.
For teams trying to get started, the absence of a measurement framework means the business case never gets specific enough to approve. "AI could help with contract review" doesn't get a budget. "Contract review currently takes 14 days; this tool cuts it to 5, recovering X hours per week and removing a recurring bottleneck for Legal" does. The measurement strategy turns a concept into a case.
For programs already running, what we see most often isn't dramatic failure. It's gradual invisibility. Not a decision to cancel, but a deprioritization that compounds quietly across budget cycles. The measurement data that would have protected the program was either never collected, or framed around vendor metrics rather than business outcomes.
The fix isn't complicated. It's a sequence of decisions made at the right time (specifically, before deployment). And a simple discipline: track what matters consistently enough to have something credible to report when the question comes.
Step 1: Define Your AI Success Metrics Before Deployment
A measurement strategy is four questions, documented before any AI capability goes live. Dedicated platform, Copilot feature, GPT or Gem built in-house. Anything.
What specific problem are we solving? Not "we want AI in procurement." Something measurable. Contract review takes 14 days and creates downstream delays. Supplier onboarding runs through 6 manual touchpoints and generates a 22% error rate. Spend categorization consumes three analyst-weeks every quarter. The more specific the problem statement, the more specific your measurement can be.
What does success look like, in numbers? Contract review in 5 days. Onboarding errors down by half. Categorization in 3 days instead of 14. Set these before implementation, not reverse-engineered afterward from whatever metrics happen to be available.
Who is accountable for tracking outcomes? One person, named. Close enough to the work to know when a number looks wrong. Credible enough to surface it when it does.
What is current performance on those metrics, today? Time per task. Error rates. Cycle times. Cost per transaction. Document it before anything changes. This is the baseline, and without it, every outcome you report is a number with nothing to compare it against.
In the engagements where we've seen this approach land, the baseline question alone changes the quality of the conversation. Leadership and procurement sit down to agree on metrics before deployment. They surface differences in expectations that, left unaddressed, would have turned into disputed results six months later. Two hours of alignment before deployment is worth more than two weeks of explaining results after the fact.
Step 2: Build the Evidence Foundation
Picture yourself twelve weeks post-deployment. Clean baseline captured, measurement tracked consistently, impact report showing a 30% reduction in contract processing costs. You walk into the leadership review confident in the numbers.
The first question isn't about the methodology or the trend line. It's "where is this data coming from, and who has access to it?"
That's the governance question. It stops more AI reporting conversations than any other single factor. Leadership won't act on data from a system they don't understand or trust. In procurement, where contracts, supplier pricing, and commercial strategy live inside AI platforms, that trust is not assumed. It's earned.
Governance means being ready to answer three sets of questions:
- Data security: what the tool processes, where it's stored, who can access it, what the breach response looks like.
- Compliance: whether data handling meets GDPR, sector-specific requirements, and internal policies.
- Auditability: whether outcomes can be traced to source data and the methodology can be reviewed.
Build a single document that answers these questions. Get IT and Legal to review it. Reference it every time you present AI outcomes. The message it sends isn't "we did compliance work." It's "the results we're showing you come from a system this organization can stand behind." That changes how leadership engages with the numbers.
Step 3: Track KPIs With a Lightweight System
The goal isn't a full reporting infrastructure. It's a simple system you can sustain alongside everything else your team is doing.
Three stages:
Stage 1: Baseline capture. Before or at the very start of deployment, document current performance on your target metrics. Time per task, error rates, volume processed, cost per unit of output on the specific workflows AI will touch. Two hours of structured data collection is more useful than any vendor dashboard.
Stage 2: Weekly tracking. One person, 30 minutes per week, recording core metrics without analyzing them. Cycle times on AI-assisted versus manual tasks. Volume processed. Exceptions flagged. Not a report. A record that accumulates into something valuable at review time.
Stage 3: Quarterly translation. Every three months, convert the tracking data into outcomes leadership can engage with. Time saved multiplied by loaded hourly cost equals efficiency value in dollars. Error rate reduction equals rework cost avoided. Volume growth with headcount held constant equals a productivity story. None of this requires advanced analytics. It requires the discipline of doing it every quarter without skipping.
This produces defensible measurement, not research-grade measurement. Defensible is enough: to protect funding, justify expansion, and build the internal case for more sophisticated tracking as the program grows.
Step 4: Build a Process for When AI Underperforms
Your AI vendor won't lead with this: some things won't work. Use cases that performed in the pilot will underperform at scale. Workflows that looked automatable will require more judgment than anticipated. Metrics will move in unexpected directions.
This is normal. The teams that handle it well treat it as information rather than failure.
We call it a continuous improvement loop. Every AI deployment is an ongoing experiment with an active learning cycle, not a finished implementation. The question shifts from "is this working?" (which produces defensiveness) to "what is this telling us about how to deploy it better?" (which produces iteration).
We've seen clients pivot away from a use case mid-deployment because the data pointed to a better opportunity elsewhere in the process. That pivot only happens in organizations where leadership and procurement have built enough trust to say "this isn't performing as expected" without it threatening the whole program. Measurement data makes that conversation possible. But the organizational environment that lets people use the data honestly has to be built alongside the measurement system.
In practice:
- Use metrics to make decisions, not to build post-hoc justifications.
- Build a formal 90-day recalibration into your program timeline. Not just to review metrics, but to ask whether you're measuring the right things as usage evolves.
- Report what you're learning, including what isn't working. Credibility with leadership compounds when you demonstrate honest reporting rather than selective reporting.
What Separates Programs That Scale from Those That Stall
No sophisticated infrastructure required. A clear sequence:
Define what success looks like before anything is deployed, with leadership and procurement aligned on the same definition. Build a governance foundation that makes your data trustworthy to the people who need to act on it. Track consistently with a system your team can sustain. Use the data to guide decisions. Report honestly on what you're learning.
The teams who build AI programs that scale aren't the ones with the most resources. They're the ones who got the sequence right. They treated measurement as the starting point rather than the conclusion. They built an environment where course correction was expected and normal.
They developed the habit of translating what the data showed into a story leadership could understand and act on.
Alignment before deployment. Adaptability in execution. Clarity in how value gets communicated. That combination is what separates programs that grow from programs that quietly disappear.
M + E: What This Article Built
M (Measurement): the system you built before deployment. Success metrics defined, baseline captured, accountability assigned, tracking discipline established.
E (Evidence): the governance foundation (data security, compliance documentation, auditability) that converts results from claims into something leadership can trust and act on.
Where does your AI program stand? Most procurement teams discover their measurement gaps mid-program, when they're harder to fix. The Molecule One AI Readiness Assessment identifies where your measurement strategy is strong and where it's exposed, before the results conversation with leadership. Take the AI Readiness Assessment →
Put this into practice. The MERIT Baseline Capture Template walks you through every step covered in this article: defining the problem, assigning accountability, capturing your baseline, documenting governance, and setting up your tracking rhythm. Two hours with this template before deployment is worth more than two weeks of explaining results afterward. Download the MERIT Baseline Capture Template →
Frequently Asked Questions
What is a procurement AI measurement framework?
A procurement AI measurement framework is a pre-deployment system that defines what success looks like, captures a performance baseline, assigns accountability for tracking, and establishes governance over the data. It answers four questions before any tool goes live: what problem are we solving, what does success look like in numbers, who owns the tracking, and what is current performance today? Without it, every result you report has nothing credible to compare against.
When should you start building an AI measurement strategy in procurement?
Before deployment. Not after. Once a tool is live, the baseline window has closed. Teams that build their measurement strategy retroactively are forced to reverse-engineer metrics from whatever data happens to be available. That produces numbers leadership can challenge. Two hours of structured baseline capture before go-live is worth more than two weeks of explaining results six months later.
How do you measure the ROI of AI in procurement?
Measure procurement AI ROI across three value types. Efficiency value: time saved multiplied by loaded hourly cost. Quality value: error rate reduction multiplied by cost per error, which captures rework avoided. Capacity value: volume processed this quarter with the same headcount as last quarter (the productivity story without a conversation about headcount reduction). Set these metrics before deployment, not after, so results are defensible rather than self-reported.
What KPIs should procurement teams track for AI programs?
The most useful procurement AI KPIs are tied to the specific workflows the tool touches. Common starting points include PR-to-PO cycle time, first-time-right rate on intake requests, AP exception resolution time, contract review cycle time, and supplier onboarding error rate. The right set depends on what problem the program was deployed to solve. That's why defining the target metrics before deployment is the first step in any measurement strategy.
What is AI governance in procurement, and why does it matter for reporting?
AI governance in procurement means being able to answer three sets of questions about your data: what the tool processes and who can access it (security), whether data handling meets GDPR and sector requirements (compliance), and whether outcomes can be traced back to source data (auditability). Governance matters for reporting because leadership won't act on numbers from a system they don't understand or trust. In procurement, where commercial data lives inside AI platforms, that trust is not assumed.
The next article covers what happens once you have measurement data: how to turn it into reports that land with two very different audiences. Leadership needs the financial and risk story. Users need to see that the tools make their work better. The same dataset tells both stories. The skill is knowing which one to tell, to whom, and when.