Lesson objective
Trace how an AI output travels into ledgers, reports, filings and client decisions, and place validation checks and human gates at the points where a confident wrong answer would otherwise become a wrong action.
Plain-language explanation
Why this matters now
In a single task, a wrong AI answer usually stops at the person who reads it. In an operation, outputs rarely stop. A suggested account code becomes a posted entry. The posted entry feeds an accrual, the accrual feeds the trial balance, the trial balance feeds a management reporting package, and the package feeds a lender's covenant calculation or a tax provision. By the time anyone questions the number, it has several parents and many children.
Professional guidance (voluntary framework): NIST's Generative Artificial Intelligence Profile (NIST AI 600-1, July 2024) defines confabulation as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications')." It says confabulations "are a natural result of the way generative models are designed," and that outputs "may also include confabulated logic or citations that purport to justify or explain the system's answer." It also says LLMs "sometimes provide logical steps for how they arrived at an answer even when the answer itself is incorrect." NIST does not say how often this happens, and its examples are not from accounting.
Best practice (community security list): the OWASP Top 10 for LLM Applications 2026 (dated August 2026) lists LLM07 Misinformation, LLM03 Excessive Agency and LLM10 Improper Output Handling. Its project leads note that voters placed Misinformation near the bottom while the incident record placed it near the top, and explain: "When a model's fluent, confident output drives a decision or a tool call, a wrong answer turns into a wrong action." Their letter adds: "Build the system around it, so that when the model is fooled, and it will be, nothing important breaks." OWASP's list is a voluntary community document, not a regulation.
Professional guidance (voluntary): COSO's Achieving Effective Internal Control Over Generative AI (February 23, 2026) sorts GenAI uses into eight capability types: ingestion, transformation, posting, orchestration, judgment, monitoring, knowledge retrieval, and human-AI interaction. The risks it names include hallucination, opaque reasoning, model drift and rapid configuration changes. For this lesson, the capability type tells you how far an output can travel before a person sees it. Module 2 uses the same list to map a workflow.
A question to sit with
If an AI tool's answer in your process is wrong, what is the last moment anyone could catch it before it changes a ledger, a filed return, an issued report or a client's decision? Is anyone standing at that moment?
What goes wrong
The explanation looks like proof. A tool that gives a tidy reason for a classification seems more trustworthy than one that gives a bare answer. NIST's point is that the reasoning can be confabulated too. A persuasive explanation is another output to check, not evidence.
Output becomes input without a check. A draft classification is exported to a spreadsheet, the spreadsheet is imported to the ledger, and nobody validates it at the boundary. OWASP's Improper Output Handling category describes this pattern in software terms: output passed on to another component as if it were trustworthy.
The tool is allowed to act, not just suggest. When a tool can post entries, send client emails or update a filing record, a wrong answer is no longer a draft. OWASP's Excessive Agency category covers systems given more permissions or autonomy than the task needs.
The error surfaces after the point of no return. Some corrections are cheap: a draft memo, an unposted journal. Others are expensive or public: a posted entry in a closed period, an issued report, a filed return that needs amending, a covenant certificate already delivered to a lender.
Drift and configuration changes arrive silently. COSO names model drift and rapid configuration changes. A vendor update can change behaviour for every client at once, with no change to the firm's own procedure.
Downstream users cannot see where a number came from. The lender, the tax team or the client's board receives a figure with no marker showing that an AI output sat in its history.
The practical answer
Trace the propagation path first. For each AI output, list where it goes next, what decision it feeds, and the last point at which a person could catch it. AI Coalition Network recommendation: keep this as a one-page propagation table for every AI use in an operation.
| Step | Example: AI-suggested invoice coding | Reversible at this point? | Who could catch an error |
|---|---|---|---|
| 1. Output created | Tool suggests account and cost centre | Yes, nothing recorded | Preparer |
| 2. Output posted | Entry posted to the open period | Yes, by correcting entry | Senior at batch review |
| 3. Period closed | Balance included in closed month | Only by later adjustment, with explanation | Controller at close review |
| 4. Reported | Figure in the monthly package to the owner and lender | Only by restating the package | Controller before release |
| 5. Relied on | Covenant calculation or tax estimate uses the figure | Costly, and may affect third parties | Nobody, unless caught earlier |
The table shows where a gate belongs: before the step where reversal becomes costly.
Put validation at every boundary. Before an AI output enters another system, check it with rules that do not depend on the AI. Best practice:
- Allowed values: the account exists in the chart of accounts and is open for posting; the cost centre is valid for the entity.
- Totals and control accounts: the batch total agrees to the source file; subledger totals agree to the control account.
- Reasonableness: amounts outside expected ranges for the account or vendor are held.
- Completeness: every source item has one output, and no output lacks a source item.
These checks catch format and range errors. They do not catch a plausible wrong answer, so they sit alongside human review, not in place of it.
Place a human gate before irreversible steps. Posting to a closed period, releasing a report, submitting a return and sending a client deliverable each need a named person's approval, recorded. In the 2024 PCAOB staff Spotlight, firms said the person using a GenAI tool remains responsible for the results; the gate is where that responsibility becomes a recorded act.
Give the tool the least authority the task needs. A tool that drafts entries for approval is lower risk than one that posts them. Best practice: keep AI tools in suggest-only mode for anything that changes a ledger, a filing or a client communication unless a documented decision, owned by the responsible professional, says otherwise. Module 3 uses reversibility to set review depth.
Mark AI-touched figures for downstream users inside the firm. A field or note showing that a figure passed through an AI step, and who reviewed it, lets the tax team or the controller know what they are relying on.
Remember this
- In an operation, AI output becomes input to ledgers, reports, filings and decisions.
- A fluent explanation can be confabulated along with the answer.
- Validate at every boundary with rules independent of the AI, and gate every irreversible step with a named person.
- Keep tools in suggest-only mode for anything that changes records unless the responsible professional decides otherwise.
- Find the last point of cheap reversal and put the control there.
Accounting example
Calder Ames Advisory is a fictional firm providing outsourced controller services to Tessaro Foods LLC, a fictional food distributor with a bank loan that tests a debt service coverage covenant each quarter. The firm's AI tool reads vendor invoices and suggests an account code, and a script posts suggestions scoring above 85 without review. In March the tool codes a $62,000 refrigeration unit replacement to repairs and maintenance expense instead of equipment, with a clear explanation that the invoice describes "service and replacement parts." Tessaro's capitalisation policy treats replacement of a whole unit above $5,000 as a fixed asset. The entry posts, the month closes, and the quarterly package goes to the lender. The fictional loan agreement measures coverage as earnings before interest, tax, depreciation and amortisation divided by debt service. The package shows $1,180,000 against debt service of $1,000,000, a coverage of 1.18 against a required 1.20. Controller Beatriz Olsen receives the lender's query and traces the dip to the refrigeration invoice. Reclassifying the $62,000 to equipment removes it from expense, so the measure becomes $1,242,000 and coverage is 1.242, which clears 1.20. The firm restates the package and explains the correction to the lender. Beatriz then redesigns the path. The posting script now checks that any single invoice above $5,000 coded to repairs or maintenance is held for review. Posting becomes suggest-only for invoices over $2,500. The month-end close checklist adds a review of all AI-coded items above the capitalisation threshold before the period closes. The package now notes which figures include AI-coded items and who reviewed them. She records the incident and the changes in the engagement file.
Responsible-use warning
Do not let an AI output cross a boundary into a ledger, a filing, a report or a client message without a check that does not rely on the AI. And do not accept a fluent explanation as the check: NIST notes that the logic and citations offered to justify an answer can be confabulated along with the answer. The most expensive errors are the ones found after the last cheap point of reversal. Find that point and put a named person there.
Human-review requirement
The responsible professional, the controller for a company's books, the signing preparer for a return or the licensed CPA for an attest report, owns every irreversible step: posting to a closed period, releasing a package, filing or issuing. They check that the propagation table is current, that validation rules run at each boundary, that suggest-only settings have not been widened without their decision, and that AI-touched figures are marked for internal users. Any decision to let a tool act rather than suggest is theirs, and is recorded.
Practical exercise
Use fictional data only.
- Choose one AI use in a fictional close, tax or engagement process, for example invoice coding, extraction of tax document figures or drafting of variance commentary.
- Build a propagation table with at least five steps, from the output being created to the figure being relied on by someone outside the team.
- For each step, mark whether an error is reversible, reversible at a cost, or costly and visible to third parties.
- Identify the last point of cheap reversal and name the role that stands there.
- Write four validation rules for the boundary where the output enters the next system: one each for allowed values, totals, reasonableness and completeness.
- State whether the tool suggests or acts at each step. For any step where it acts, write the case for returning it to suggest-only, and who would decide.
- Describe how figures that passed through the AI step would be marked for downstream users inside the firm.
- Write one paragraph describing what a confidently wrong output would look like at your chosen gate, and how the gate would catch it.
- Name the responsible professional who would approve each irreversible step in your table, and state what that person would record when approving it.
Knowledge check
-
What does NIST AI 600-1 say about explanations that accompany a generative AI answer?
Answer guidance: It says outputs may include confabulated logic or citations that purport to justify the answer, and that LLMs sometimes give logical steps even when the answer is incorrect. A persuasive explanation is therefore another output to check, not evidence.
-
What is a propagation table and what does it show?
Answer guidance: A list of the steps an AI output passes through, from creation to the decision that relies on it, showing at each step whether an error is reversible and who could catch it. It shows where the last cheap point of reversal is, which is where a gate belongs.
-
Why are validation rules at a system boundary necessary but not sufficient?
Answer guidance: Rules on allowed values, totals, reasonableness and completeness catch format, range and completeness errors without relying on the AI. They do not catch a plausible but wrong answer, such as a valid account that is the wrong account, so human review is still needed.
-
How do the OWASP 2026 categories Excessive Agency and Improper Output Handling relate to accounting operations?
Answer guidance: Excessive Agency covers giving a tool more authority than the task needs, such as letting it post entries rather than suggest them. Improper Output Handling covers passing output to another system as if trustworthy, such as importing classifications without validation. OWASP's list is voluntary community guidance.
Sources and further reading
- National Institute of Standards and Technology — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (2024-07-26). Accessed 2026-09-15.
- OWASP Foundation, OWASP GenAI Security Project — OWASP Top 10 for LLM Applications 2026 (2026-08-04). Accessed 2026-09-15.
- Committee of Sponsoring Organizations of the Treadway Commission (COSO) — Achieving Effective Internal Control Over Generative AI (GenAI) (2026-02-23). Accessed 2026-09-15.
- Public Company Accounting Oversight Board — PCAOB Staff Shares Observations from Outreach on Use of Generative Artificial Intelligence in Audits and Financial Reporting. Accessed 2026-09-15.
- AI Coalition Network — Certification Educational Standard: Human Review of AI-Assisted Professional Work. Accessed 2026-09-15.