Lesson objective
Explain in plain terms how a large language model is trained and generates text one token at a time, and turn each part of that mechanism into a checking habit for accounting work.
Plain-language explanation
Why this matters now
Most accounting professionals meet large language models through a chat window or a button inside software they already use. The interface makes the system look like a colleague who has read everything and remembers it. Knowing roughly what happens when you press enter lets you predict where it will go wrong. That is part of competence, not a technical hobby.
The PCAOB staff Spotlight on generative AI (July 2024) reports that preparers it spoke to cited the "black-box" nature of these tools and their inconsistent output as raising auditability questions. That Spotlight reflects limited outreach, mainly with larger audit firms and several companies, in mid-2024. Staff state that it "is not a rule, policy, or statement of the Board," and it is not a description of practice in 2026. But both concerns it records, that you cannot see why the tool answered and that you may not get the same answer twice, follow directly from how these models work.
A question to sit with
If you asked the same assistant the same question about a client's lease on Monday and again on Tuesday, and got two different answers, which one would you put in the workpapers, and why?
What goes wrong
Treating the model as a database. People ask a model for a figure the way they would look it up in a table. A language model does not retrieve a stored fact unless the product connects it to a source (Lesson 3 covers that). It generates text that is likely to follow the question.
Assuming it knows this year's figures. Training stops at a point in time. Rates, thresholds, forms and guidance that changed afterwards are not in the model unless someone supplies them, and the model will not necessarily say that it does not know.
Reading the explanation as evidence. The step-by-step reasoning a model shows is produced the same way as its answer. It can be just as wrong, and it can make a wrong answer look checked.
Expecting the same answer twice. The model builds each answer from likely continuations, and many products allow some variation in which continuation is chosen. Two runs of one prompt can differ in wording, in structure and sometimes in substance.
Forgetting that pasted text is also input. Everything you place in front of the model, including a client document, becomes part of the same stream of text it is predicting from. Wording inside that document can steer the result.
The practical answer
You do not need the mathematics. You need four plain ideas, and a habit attached to each.
1. Training. A model is built by exposing it to very large amounts of text and adjusting it, over and over, to get better at predicting what comes next. What it keeps is not a library of documents it can open, but a very large set of learned patterns about how words, numbers and ideas tend to follow one another. Developers typically refine a model further so that it follows instructions and holds a conversation. Training ends at a point in time.
2. Tokens. The model does not read letters or words the way you do. Text is broken into tokens: short pieces that may be a whole word, part of a word, a group of digits or a punctuation mark. A figure such as 48,215.07 may be split into several tokens. To the model, a number is a sequence of text pieces, not a quantity it holds in mind.
3. Prediction. When you send a prompt, the model predicts a likely next token, adds it to the text, and predicts again, repeating until the answer is complete. Every answer is built one piece at a time from patterns. Nothing in that loop checks the answer against the world.
4. One stream of text. Your instruction, the document you pasted and the model's own reply are all tokens in the same window. The UK National Cyber Security Centre put it directly in a blog post dated 8 December 2025: "Under the hood of an LLM, there's no distinction made between 'data' or 'instructions'; there is only ever 'next token'." That is a technical view expressed in a blog post, not a formal standard, but it explains why a document can influence the answer.
Now attach the habits.
| What the model is doing | What it means for accounting work | The habit |
|---|---|---|
| Predicting likely text from patterns | A likely-sounding answer and a correct answer are different things | Verify content against a source you hold |
| Working from training that ended at a date | Rates, thresholds, forms and guidance may be out of date | Supply the current source; never ask the model to recall a figure |
| Treating numbers as pieces of text | Arithmetic can go wrong with no sign that it has | Recompute in the ledger or a spreadsheet |
| Choosing among likely continuations | Two runs of one prompt can differ | Save the output you relied on, with its prompt, tool and date |
| Reading everything in the window as one stream | Wording inside a document can steer the answer | Treat supplied documents as data and check what came back against them |
| Generating its explanation the same way as its answer | The reasoning can be as wrong as the conclusion | Test the conclusion against evidence, not against the explanation |
Supply, do not ask it to recall. The single most useful change in how you use a model follows from the mechanism. Compare two requests.
- Weak: "What useful life does our policy give to kitchen equipment?" The model has never seen your client's policy. It will produce a likely-sounding number.
- Stronger: "Using only the fixed-asset policy pasted below, dated 1 January 2026, state the useful life for kitchen equipment and quote the sentence that gives it. If the policy does not say, reply 'not stated'."
The stronger request does not make the model reliable. It makes the answer checkable, because you hold the source and the model has been asked to point to it. Module 6 builds this into a full prompt structure; Module 2 shows what still goes wrong with numbers.
Save what you relied on. Because two runs can differ, "the tool said so" cannot be reconstructed later. Keep the prompt, the output you used, the tool, and the date with the work.
Use a shared vocabulary for review. The NIST AI RMF 1.0 (26 January 2023) describes trustworthy AI as valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair with harmful bias managed. The framework is voluntary and binds no firm or practitioner, and NIST states it is being revised. Three of those words turn into practical questions for a reviewer: valid and reliable (does the output match the source?), accountable and transparent (can we show what was asked and who accepted the answer?) and explainable and interpretable (can the preparer say why the answer is right, without relying on the model's own explanation?).
Remember this
- A language model predicts likely text one token at a time; nothing in that loop checks the answer against the facts.
- Training ends at a date, so supply current figures and sources instead of asking the model to recall them.
- Numbers are pieces of text to the model: recompute every figure somewhere else.
- The model's explanation is generated the same way as its answer and can be equally wrong.
- Save the prompt, output, tool and date you relied on, because the next run may differ.
Accounting example
Brightwater Advisory LLP is a fictional firm. Staff accountant Theo Lindqvist is preparing a lease summary for an invented client, Castlemere Dental Group, which has just signed a new five-year lease on a clinic. On Monday he pastes a paragraph describing the lease into the firm's approved assistant and asks for the monthly rent and the total rent over the term. The reply says monthly rent is $6,250.00 and total rent is $370,000.00, with a neat explanation showing "12 months times 5 years times $6,250." Theo recomputes: 12 times 5 is 60 months, and 60 times $6,250.00 is $375,000.00, not $370,000.00. The explanation named the right steps and still arrived at the wrong total. On Tuesday, preparing the file for review, he runs the same prompt again to paste a clean copy. This time the reply gives $375,000.00 but describes a rent-free first month that the paragraph never mentioned, and so reports $368,750.00 as "cash rent payable." Neither run was a lookup of anything; both were generated. Theo stops using the assistant for the figures. He reads the lease itself, confirms there is no rent-free period and that rent is fixed at $6,250.00 for all 60 months, and builds the rent schedule in a spreadsheet from the lease terms. He keeps the assistant for one job it can do checkably: listing the clauses that mention rent, with section numbers he then opens and reads. In the workpaper he records both prompts, both outputs, the date and the tool, notes that the figures were recomputed from the lease, and names himself as preparer and his senior, Ruth Adeyemi, as reviewer.
Responsible-use warning
Do not ask a language model to remember a figure, a rate, a policy or a standard and then treat the reply as an answer. It has not looked anything up. It has produced text that is likely to follow your question, from training that ended at some past date. A confident number with a tidy explanation is exactly what confabulation looks like. Give the model the source, ask it to point to where the answer sits, and then check that place yourself. Recompute every figure outside the tool.
Human-review requirement
The reviewer confirms that no figure, rate or policy term in the work came from the model's memory: each traces to a source document held in the file or to a recomputation outside the tool. The reviewer checks that the prompt, the output relied on, the tool and the date are saved, and that the model's own explanation was not treated as evidence. Ownership follows the work: the responsible accountant or controller for a company's books, the signing preparer for a return or tax advice, and a licensed CPA for attest work or any report that carries a CPA firm's name. The owner signs off on the content, not the tool.
Practical exercise
Watch the mechanism work on fictional material. Use only the invented text you write for this exercise, and only a tool your firm has approved.
- Write a short fictional expense policy of about 120 words for an invented company. Include one specific figure, such as a meal limit per day, and one date.
- Without pasting the policy, ask your approved assistant what the meal limit is for that invented company. Record the reply word for word.
- Now paste the policy and ask for the meal limit, quoting the sentence that states it, and replying "not stated" if it is absent. Record the reply.
- Run the step 3 prompt twice more. Note every difference in wording, structure or substance across the three replies.
- Ask for a calculation using your figure, for example the limit for a nine-day trip. Recompute it yourself and note whether the tool was right.
- Ask the tool to explain its calculation. Decide whether the explanation would have helped you notice an error, and write one sentence on why or why not.
- Complete a two-column table: for each of the four ideas in this lesson (training, tokens, prediction, one stream of text), write what you observed and the habit it supports.
- Record the prompts, outputs, tool and date as you would in a workpaper.
Knowledge check
-
In plain terms, how does a large language model produce an answer?
Answer guidance: It breaks text into tokens and, using patterns learned in training, predicts a likely next token, adds it, and predicts again until the answer is complete. Nothing in that loop checks the answer against a source.
-
Why is asking a model to recall a current rate or threshold a poor practice?
Answer guidance: The model does not look the figure up; it generates likely text from training that ended at a past date. Changes after that date are missing unless supplied, so you should give it the current source and verify the figure against that source.
-
What does NIST AI 600-1 say about the explanations generative models give for their answers?
Answer guidance: It says outputs may include confabulated logic or citations that purport to justify the answer, and that models sometimes give logical steps even when the answer is incorrect. The profile is voluntary and not accounting-specific.
-
Why does the NCSC point that there is "only ever next token" matter when you paste a client document into a tool?
Answer guidance: Because the model makes no distinction between your instructions and the document's content; both are text in one stream, so wording inside the document can steer the answer. Treat the document as data and check the output against it.
Sources and further reading
- National Institute of Standards and Technology — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (2024-07-26). Accessed 2026-09-15.
- National Cyber Security Centre (UK) — Prompt injection is not SQL injection (it may be worse) (2025-12-08). Accessed 2026-09-15.
- National Institute of Standards and Technology — Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (2023-01-26). Accessed 2026-09-15.
- Public Company Accounting Oversight Board — PCAOB Staff Shares Observations from Outreach on Use of Generative Artificial Intelligence in Audits and Financial Reporting. Accessed 2026-09-15.