Lesson objective
Define and capture a pre-change baseline for a mapped workflow — volume, duration, defect rate, rework rate and review burden — and record how each figure was measured so a later comparison is honest.
Plain-language explanation
Why this matters now
A redesign you cannot compare against a baseline is an opinion. That sentence is in the workflow map template for a reason, and this lesson is the one that makes it operational.
Module 1 diagnosed the stalled pilot, and the commonest single cause was the absence of a baseline. Without one, "it saved time" is a participant's recollection of how long the work used to take, and it cannot survive the first sceptical partner — nor should it. With one, the same claim is a comparison between two measured states, and the conversation moves from whether to believe it to what to do about it.
There is a second reason, less obvious and more important. A baseline protects the workflow from its own success story. Once a change has been made and described as a success, measuring it becomes politically awkward, and a firm that did not measure beforehand will find that it never quite gets round to measuring afterwards either. The baseline is the thing that makes later measurement normal rather than accusatory.
Best practice: the NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1, 26 January 2023) separates Map from Measure, and the NIST AI RMF Playbook offers a menu of concrete governance actions — documented roles, inventories, decommissioning plans — that a firm draws from according to its size. Both are voluntary. Neither requires anything of a law firm, and adopting either neither creates nor discharges a professional duty.
A question to sit with
Six months after the change, a client asks whether the new workflow is better than the old one. What document do you open, and was it written before or after you already knew the answer you wanted?
What goes wrong
The retrospective baseline. The change is made, the results look good, and somebody reconstructs what the old way used to cost. Reconstruction is always generous to the new way, not through dishonesty but because the old way is now remembered through its failures and the new one is being watched closely.
Only speed is baselined. Time is easy to count, so time is what gets recorded, and quality, rework and review burden are left out. The comparison that follows can only say the new way is faster, which is the one thing nobody seriously doubted.
A defect rate with no taxonomy. "How often does it go wrong" is unanswerable until "wrong" is defined. Two reviewers scoring the same output disagree about whether an omission is a defect, and the resulting number measures the reviewers rather than the workflow.
The baseline is a single number with no denominator. "About a day and a half" is not a baseline. "Median 1.4 days across the 30 matters completed in March, measured from instruction timestamp to issued report, with two matters excluded because the instruction arrived by phone and was not timestamped" is a baseline, and the exclusions are part of it.
The sample is chosen rather than taken. Baselining the clean matters produces a baseline the new workflow will struggle to beat, and baselining the awkward ones produces one it will beat easily. Best practice: take a defined recent period and use everything in it, then state what you excluded and why.
Nobody records how the figure was measured. Six months later the method is forgotten, the post-change figure is measured slightly differently, and the comparison is between two things that were never the same quantity. This is the most common way a well-intentioned baseline becomes useless.
The practical answer
Baseline five quantities, not one. These five, measured before anything changes, are enough to support every later comparison this level asks for.
| Quantity | Definition | How to capture |
|---|---|---|
| Volume | Completed instances per month, and how counted | The system of record, plus an estimate of what never enters it |
| Duration | Hands-on and elapsed, per step and end to end, with the median and the longest | Timed observation and timestamps, from the previous lesson |
| Defect rate | Defects per deliverable against a written taxonomy | Blind review of a sample by someone other than the producer |
| Rework rate | Loops per matter and hands-on minutes consumed | Counted over a defined recent period |
| Review burden | Reviewer minutes per deliverable | Timed, not recalled |
Write the defect taxonomy before you count anything. Five to eight categories, each observable by someone reading an output against its sources. A taxonomy for a summarisation workflow might read: omitted a material term; stated an allegation as a fact; attributed content to the wrong document; included a term that is not in the source; wrong date or figure; internally inconsistent; incomplete without saying so. Module 13 builds error taxonomies properly and this is a first pass, not the finished article.
Sample blind, and sample from a period rather than a selection.
| Rule | Why |
|---|---|
| The reviewer did not produce the item | A producer scoring their own work measures their confidence |
| The reviewer scores against the taxonomy, not impressions | Otherwise the number measures the reviewer |
| Take a defined recent period and use all of it | Selection is the largest source of baseline error |
| State exclusions and their reasons | Exclusions are part of the measurement, not a tidying step |
| Report the denominator | "Four of thirty" is honest; "13%" from thirty is not |
Record the method beside every number. AI Coalition Network recommendation: each baseline figure carries four things — the number, the period, the method, and the name of the person who measured it. A figure without its method cannot be re-measured the same way, and a comparison between two differently-measured figures is worse than no comparison because it looks like evidence. This is the programme's educational standard for baselining, not a legal requirement.
Baseline the exception paths separately. A workflow that is routine four times in five and exceptional once has two very different cost and defect profiles, and a single blended figure hides both. Report routine and exceptional separately, with their fractions visible.
Build the baseline record as a small table you will actually reopen.
| Field | Example entry |
|---|---|
| Quantity | End-to-end elapsed time |
| Value | Median 1.4 days; longest 6 days |
| Denominator and period | 28 of the 30 matters completed in the defined month |
| Exclusions and why | 2 matters had no instruction timestamp |
| Method | Instruction timestamp in the matter system to issued-report timestamp |
| Measured by and date | Named role, dated |
| Re-measure when | At the same point in the following quarter, by the same method |
Agree the comparison in advance. Before any change, write down what you will measure afterwards, when, by what method, and what result would count as a failure. Best practice: a pre-registered comparison of this kind is the single most effective protection against a post-hoc account of what the change was supposed to achieve, because the goalposts are recorded while nobody yet has an interest in where they stand.
Know what a baseline does not do. It does not tell you whether the workflow is safe, and it does not tell you how much review any step deserves. Consequence and reversibility do that, and module 3 takes it up next. Nor does it tell you what the SOP must say; module 4 does that. Law / rule: Model Rules 1.1, 1.5, 5.1 and 5.3 address competence, fees and supervision, and these are model provisions adopted with different numbering and wording in each jurisdiction — a good baseline discharges no duty under any of them. It simply means that when the duty is discussed, the firm is discussing facts.
Remember this
- Baseline five quantities before anything changes: volume, duration, defect rate, rework rate and review burden.
- Write the defect taxonomy before you count defects, or the number measures the reviewer rather than the workflow.
- Every figure carries its number, its period, its method and the person who measured it — otherwise it cannot be re-measured the same way.
- Take a defined period and use all of it, state your exclusions, and report the denominator rather than a percentage from a small sample.
- Pre-register the comparison: what you will measure afterwards, when, how, and what result would count as a failure.
Legal example
Havelock Nkemelu is a fictional 45-lawyer employment practice whose invented operations lead, Sofia Marchetti, is about to introduce an assistive step into the firm's tribunal bundle preparation workflow. She has three weeks before the change and spends the first two baselining. Volume is the easy one and it still produces a surprise: the matter system records 38 bundles in the defined month, but the practice assistant who handles urgent requests keeps a separate spreadsheet with six more that never entered the system because they were prepared over a weekend. Sofia reports 44, and writes the reason for the gap into the method cell rather than smoothing it away. Duration she takes from the previous lesson's work: median end-to-end elapsed time of 9 days, longest 21, with a hands-on total of about four hours spread across six steps. She reports the 21 alongside the 9, because the 21-day bundles are the ones that generate client calls. Defects take longer, because she has to write the taxonomy first. She settles on six categories, each of which a reviewer can observe by reading a bundle against its index and sources: a document missing from the bundle but listed in the index; a document in the bundle but not indexed; wrong version of a document; pagination inconsistent with the index; a document included that should have been withheld; and an unredacted item where redaction was instructed. She then has a paralegal who did not prepare them review twenty bundles from the defined month against that taxonomy. The result is nine defects across twenty bundles, concentrated in two categories, and she reports it as "nine defects in twenty bundles" rather than as a percentage. Rework she counts from the file history: five of the forty-four went back for a second pass, costing an average of seventy minutes each, which she expresses as about eight minutes per bundle across all bundles. Review burden she times directly: the supervising solicitor spends between twelve and thirty-five minutes per bundle, and she records both ends rather than an average, because the spread is the finding. The last thing she writes is the comparison she will run: the same five quantities, by the same methods, over the equivalent month next quarter, with the taxonomy unchanged. She also writes the result that would count as a failure — any rise in the two defect categories that currently dominate, or a review burden above thirty-five minutes at the top end. That sentence is what the fictional firm's managing partner later describes as the most useful part of the document, because it was written while nobody yet had a reason to want a particular answer.
Responsible-use warning
Never reconstruct a baseline after the change has been made. Retrospective figures are generous to the new workflow in a predictable direction: the old way is now remembered through its failures, the new way is being watched closely, and the people supplying the recollection have already formed a view. If no measured baseline exists and a comparison is genuinely needed, label the figure an estimate, state exactly how it was derived, and keep it out of any table where it would sit beside a measured number as though the two were the same kind of thing.
Human-review requirement
A licensed attorney owns every output that reaches a client or a court, and a baseline changes none of that. The supervising attorney owns two judgements here. First, whether the defect taxonomy captures the failures that actually matter professionally — an omission that affects a client's position is not the same kind of defect as an inconsistent heading, and a taxonomy that scores them alike will produce a comforting number. Second, whether any figure about time, cost or quality may leave the firm: what a client is told, and how the bill reflects time actually expended, are the attorney's decisions under the fee and communication rules that apply where they practise.
Practical exercise
Capture a five-quantity baseline for your mapped workflow before proposing any change. Use only fictional or non-confidential material.
- Define the workflow's boundary precisely: the trigger that starts the clock and the observable completion that stops it. Write both in one sentence each.
- Count volume for one defined recent month from the system of record, then ask two performers whether any instances never enter that system. Report the total and explain the gap.
- Bring forward your duration figures from the previous lesson. Report the median and the longest for end-to-end elapsed time, and the total hands-on time.
- Write a defect taxonomy of five to eight categories. Each must be observable by someone reading an output against its sources, without knowing who produced it.
- Take twenty deliverables from the defined period and have someone who did not produce them score each against the taxonomy. Report the result as a fraction with its denominator.
- Count rework over the same period: how many matters looped, how many hands-on minutes each loop cost, and the resulting minutes per matter across all matters.
- Time the review burden on five deliverables. Report the range, not only the average, and say what explains the spread.
- For every figure, record the period, the method, the exclusions with reasons, and who measured it.
- Pre-register the comparison: the date you will re-measure, the method you will use, and — in one sentence — the result that would count as a failure.
Knowledge check
-
Why is a baseline reconstructed after the change unreliable even when everyone involved is honest?
Answer guidance: Because the error has a direction. The old workflow is now remembered through its failures, the new one is being watched closely, and the people supplying the recollection have already formed a view. Retrospective figures are therefore systematically generous to the new way, which is why a reconstructed figure must be labelled an estimate and kept out of tables of measured numbers.
-
What must exist before a defect rate can be counted, and why?
Answer guidance: A written taxonomy of five to eight categories, each observable by someone reading an output against its sources. Without it, two reviewers disagree about what counts as a defect and the resulting number measures the reviewers rather than the workflow, so it cannot be compared with a number produced later by different people.
-
What four things must accompany every baseline figure?
Answer guidance: The number, the period it covers, the method used to obtain it, and the name of the person who measured it. Without the method the figure cannot be re-measured the same way, and a comparison between two differently-measured figures is worse than no comparison because it carries the appearance of evidence.
-
Why pre-register what would count as a failure before making the change?
Answer guidance: Because the goalposts are recorded while nobody yet has an interest in where they stand. After a change has been described as a success, measuring it becomes politically awkward and the account of what it was meant to achieve quietly adjusts. A pre-registered failure condition makes the later measurement routine rather than accusatory.
Sources and further reading
- National Institute of Standards and Technology — Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (2023-01-26). Accessed 2026-09-14.
- National Institute of Standards and Technology (AI Resource Center) — NIST AI RMF Playbook. Accessed 2026-09-14.
- Clio (Themis Solutions) — 2025 Legal Trends Report (2025-10-16). Accessed 2026-09-14.
- American Bar Association — Model Rules of Professional Conduct. Accessed 2026-09-14.
- AI Coalition Network — Certification Educational Standard: Human Review of AI-Assisted Professional Work. Accessed 2026-09-14.
- Daniel Schwarcz, Sam Manning, J.J. Prescott, Patrick Barry, David R. Cleveland and Beverly Rich (Journal of Law and Empirical Analysis) — AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice (2025-03-02). Accessed 2026-09-14.