Lesson objective

Recognise automation bias, complacency and algorithm aversion in your own review of AI output, and design review steps that apply the same evidence standard whether advice comes from a tool or a person.

Plain-language explanation

Why this matters now

The previous three lessons were about the tool. This one is about the reviewer. Every control in this module ends with a person deciding whether to accept an output, and people are not neutral judges of machines. Sometimes they defer too much. Sometimes they discount good evidence because a machine produced it. Both failures are well documented, and both get harder to see as tools improve and errors become rarer.

Professional guidance (voluntary): NIST AI 600-1 (26 July 2024) lists Human-AI Configuration among its 12 generative AI risks, including humans "experiencing algorithmic aversion, automation bias, over-reliance, or emotional entanglement" with these systems. Section 2.7 describes automation bias as "excessive deference to automated systems" and says it "can exacerbate other risks of GAI, such as risks of confabulation." The profile is voluntary and not accounting-specific.

Professional guidance: for federal tax practice, OPR Alert 2026-19 (24 June 2026), applying Circular 230, says "Practitioners cannot rely solely on AI" and concludes that "final decisions must always rest with qualified professionals." Circular 230 binds practitioners before the IRS; the alert creates no new rule.

A question to sit with

When did you last reject a suggestion from a tool that is usually right? If you cannot remember, is that because it was always right, or because you stopped looking?

What goes wrong

Commission errors. You accept a wrong suggestion because it came from the system: the coding, the matched transaction, the drafted conclusion.

Omission errors. You miss a problem because the system did not flag it. The anomaly report shows nothing unusual, so nothing is examined.

Complacency under load. In busy season, with many tasks competing for attention, checking of the automated part slides first.

Algorithm aversion. You discount correct evidence because a machine produced it, and give a person's identical advice more weight.

Believing a reminder is a control. A policy line saying "remember to check AI output" is treated as if it prevents over-reliance.

Diffusion across the team. The preparer assumes the reviewer will check the AI output; the reviewer assumes the preparer did, or that the tool itself did.

The practical answer

What the research says about deference. Raja Parasuraman and Dietrich Manzey reviewed research on automated decision aids in Human Factors (June 2010), drawing on fields such as aviation and process control, well before generative AI. Their findings apply to accounting by analogy only, but they are sobering:

  • "Automation complacency occurs under conditions of multiple-task load, when manual tasks compete with the automated task for the operator's attention."
  • Complacency "is found in both naive and expert participants and cannot be overcome with simple practice."
  • "Automation bias results in making both omission and commission errors when decision aids are imperfect."
  • Automation bias "occurs in both naive and expert participants, cannot be prevented by training or instructions, and can affect decision making in individuals as well as in teams."

Read the last point carefully. It does not say training is worthless. It says training or instructions alone did not prevent automation bias. Experience does not make you immune, and a team is not automatically safer than an individual.

What the research says about aversion. Benjamin Commerford, Sean Dennis, Jennifer Joe and Jenny Ulla, in the Journal of Accounting Research (online December 2021, issue 2022), studied "algorithm aversion," defined as "the tendency to discount computer-based advice more heavily than human advice, although the advice is identical otherwise." In their experiment, auditors who received contradictory evidence from their firm's AI system, rather than from a human specialist, "propose smaller adjustments to management's complex estimates." The effect was stronger when management had developed its estimates using relatively objective inputs. The study involved a pre-generative-AI decision aid.

Put the two findings together and the target is clear. The goal is neither trust nor distrust. It is the same evidence standard regardless of the source.

A jagged frontier makes this worse. Dell'Acqua and co-authors (Organization Science, March 2026) found that consultants using AI on a task outside its capability frontier "were 19% less likely to produce correct solutions compared with those without AI." They were consultants, not accountants, but the pattern matches automation bias: people relying on a tool where it was weak did worse than people who worked alone.

Design, not willpower. Because instruction alone does not prevent automation bias, build review steps that work even when attention is low.

Failure Control that does not depend on vigilance
Commission error Recompute or re-derive key figures and conclusions from source before accepting them, and record the recomputation
Omission error Sample items the tool did not flag, not only the ones it did, and record the sample
Complacency under load Schedule AI-assisted review when workload allows, and cap how many AI outputs one person signs off in a sitting
Algorithm aversion Evaluate contradictory evidence on its merits and document why it was accepted or rejected, whatever produced it
Reminder mistaken for control Replace "check AI output" with named checks that leave a record: tie-outs, citation verification, sample selection
Diffusion across the team Name who verifies each AI output and who reviews that verification; the tool is never either

Six failures, six controls. Each leaves evidence that a reviewer can see.

The discipline of doubt: five questions before you accept. Use them on any AI output, and on any human advice that contradicts it.

  1. Where did this come from? Can I point to the source document, calculation or authority behind it?
  2. Have I checked it independently? Not by asking the tool, and not by reading its explanation.
  3. What did it not tell me? What would I expect to see flagged that is not there?
  4. Would I accept this from a capable new colleague in their first week? If not, why am I accepting it from a tool?
  5. Am I discounting this because a machine produced it? Would I weigh the same evidence differently if a specialist had given it?

Keeping a record of disagreement. Best practice: when a reviewer rejects or changes an AI output, or accepts AI evidence that contradicts a person, note what changed and why. Over time the record shows whether reviewers are rejecting nothing, a warning sign of complacency, or discounting AI evidence that later proves right, a sign of aversion.

Remember this

  • Automation bias causes both commission errors (accepting wrong output) and omission errors (missing what was not flagged).
  • Experts are affected too, and training or instructions alone did not prevent automation bias in the research reviewed.
  • Algorithm aversion is the opposite failure: discounting correct evidence because a machine produced it.
  • Apply the same evidence standard to advice from a tool and from a person.
  • Build checks that leave a record and work under busy-season load, instead of relying on vigilance.

Accounting example

Brightwater Advisory LLP is a fictional firm auditing Thornbury Freight Ltd, an invented private haulage company. Audit senior Nadia Brandt uses the firm's approved analytics feature, which scores journal entries and flags unusual ones. It is the third week of busy season and she has four engagements open. The feature flags 14 of 3,600 entries; she examines all 14, finds nothing wrong, and is ready to conclude. Her manager, Oliver Stanek, asks what she tested among the 3,586 entries that were not flagged. The answer is nothing. At his request Nadia selects a sample of 25 unflagged entries using the firm's sampling approach. One is a manual entry posted on the last day of the year, reclassifying $48,000 from repairs to vehicles, with no supporting document. The feature had not scored it as unusual because similar reclassifications had been posted in earlier years. Nadia records it as an omission the review would otherwise have missed and follows it up. The second event runs the other way. The same analytics feature, fed the client's fuel purchase data, estimates the year-end fuel accrual at about $61,000, while management has booked $42,000 using the fuel card statements that had arrived. Oliver's first reaction is to treat the tool's estimate as noise, a machine guess against the client's actual statements. Nadia asks the fifth question from the discipline of doubt: would they weigh this differently if a specialist had said it? They examine the evidence on its merits and find that one fuel card provider's December statement was missing from management's calculation. The accrual is understated by the amount on that statement. Both decisions, the extra sample and the accrual investigation, are recorded with the reasons, and the engagement partner, a licensed CPA, reviews them before concluding.

Responsible-use warning

Do not rely on vigilance or a reminder to stop automation bias. Research on automated decision aids found experts affected too, and training or instructions alone did not prevent it. An empty exceptions list is not a clean result, and a usually reliable tool is not a checked one. Equally, do not dismiss evidence because a machine produced it; weigh it as you would the same advice from a specialist. Build checks that leave a record: recompute, sample what was not flagged, and document why any output was accepted, changed or rejected.

Human-review requirement

The reviewer confirms that each AI output was accepted on a recorded check, not on its score, explanation or usual reliability; that items the tool did not flag were sampled where the work relies on its flags; and that contradictory evidence, from a tool or a person, was assessed on its merits with reasons recorded. Ownership follows the work: a licensed CPA for attest engagements, the signing preparer for returns and tax advice, where OPR Alert 2026-19 says final decisions must rest with qualified professionals, and the responsible accountant or controller for a company's books. The owner decides; the tool never does.

Practical exercise

Examine your own review habits and redesign one review step. Use fictional or anonymised examples only.

  1. Choose one task where you or your team review output from an automated or AI tool, such as suggested codings, anomaly flags or drafted commentary.
  2. Estimate, from memory, how often in the last month anyone rejected or changed that tool's output. Then check any record that exists and compare.
  3. Describe one situation where a commission error could occur in this task, and one where an omission error could occur.
  4. Describe one situation where algorithm aversion could occur: correct evidence from the tool being discounted.
  5. Identify the busiest period for this task and what competes for attention then.
  6. For each of the three failure situations, choose a control from the lesson table that leaves a record.
  7. Write the five discipline-of-doubt questions as a short checklist adapted to this task.
  8. Define a simple disagreement log with columns for date, output, decision, reason and reviewer.
  9. Name who verifies each output and who reviews that verification, and confirm the tool is neither.
  10. After one month, read the disagreement log and note whether it suggests complacency (nothing ever rejected) or aversion (tool evidence dismissed and later proved right).

Knowledge check

  1. What is the difference between a commission error and an omission error in automation bias?

    Answer guidance: A commission error is acting on a wrong suggestion from the system. An omission error is failing to notice a problem because the system did not flag it. Parasuraman and Manzey found automation bias produces both when decision aids are imperfect.

  2. What did Commerford and colleagues find about auditors and AI evidence?

    Answer guidance: Auditors who received contradictory evidence from their firm's AI system rather than a human specialist proposed smaller adjustments to management's complex estimates, an example of algorithm aversion. The effect was stronger when management used relatively objective inputs.

  3. Does the research mean training is useless against automation bias?

    Answer guidance: No. The finding is that training or instructions alone did not prevent automation bias, in naive and expert participants alike. That is why review needs designed checks that leave a record, not only reminders.

  4. What single standard addresses both automation bias and algorithm aversion?

    Answer guidance: Applying the same evidence standard regardless of source: every output, from a tool or a person, is accepted or rejected on a recorded check against source evidence, not on who or what produced it.

Sources and further reading