Lesson objective
Recognise confabulated content in dental practice output, and apply a verification routine that catches an invented rule, citation or plan term before it reaches a patient or a payer.
Plain-language explanation
Why this matters now
The first AI failure most dental teams meet is not an obvious error. It is a paragraph that reads better than anything they would have written themselves, and happens to contain a rule that does not exist.
There is a name for this. NIST AI 600-1, the Generative Artificial Intelligence Profile of the AI Risk Management Framework, released 26 July 2024, lists twelve risks unique to or made worse by generative AI. The second is confabulation, defined as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications')" by which users may be misled or deceived. Section 2.2 states that confabulations "are a natural result of the way generative models are designed", that outputs "may also include confabulated logic or citations that purport to justify or explain the system's answer", and that models "sometimes provide logical steps for how they arrived at an answer even when the answer itself is incorrect". NIST AI 600-1 is voluntary, it is cross-sectoral rather than dental, and it does not say how often confabulation happens, so nobody may attach a rate to it.
Dentistry has one recorded evaluation that shows the shape of the problem. Giannakopoulos and colleagues (J Med Internet Res, 28 December 2023;25:e51580) had four models — Bard, ChatGPT-3.5, ChatGPT-4 and Bing Chat — answer twenty open-ended clinical dentistry questions written by faculty at the School of Dentistry, European University Cyprus, with two experienced faculty grading each answer 0 to 10 against guidelines and consensus statements. ChatGPT-4 scored better than the others, but the authors record that all models "occasionally exhibited inaccuracies, generality, outdated content, and a lack of source references", and conclude that current limitations "can lead to potentially harmful health care decisions if not used judiciously". Twenty questions, two graders and 2023 model versions: the model rankings in that paper are obsolete and must never be quoted as current. The durable finding is the failure mode — fluent answers, outdated content, no sources.
Note what does not exist. No US study of generative AI drafting accuracy on dental administrative work is recorded in this programme's register. So this lesson attaches no accuracy percentage to letter drafting, benefit summarising or recall messaging. It teaches the failure with fictional examples you can check yourself.
A question to sit with
If an AI draft told you that your state's dental board requires a specific consent form before a particular administrative step, how many minutes would it take you to find out whether that rule exists — and would you spend them?
What goes wrong
An invented rule. The output states a requirement in the register of a real one: a board rule, a plan condition, a retention period, a notice. It sounds like a rule because rules are written in that voice.
An invented citation. A section number, a rule number, a page reference or a document title that does not exist, or exists and says something else. NIST notes that outputs may include confabulated citations that appear to justify the answer.
A real source, wrong content. The harder version. The document exists; the sentence attributed to it does not.
Outdated content offered as current. Plan terms, forms, code-set years and state rules change. A model's training stopped at a date and it does not know what moved afterwards.
No sources at all, and nobody notices. An answer with no reference is not neutral; it is unverifiable. The recorded dental evaluation found exactly this alongside inaccuracy and generality.
Fluent scaffolding. Confident structure — numbered conditions, "under the applicable rule", an effective date — makes invented content look researched.
The practical answer
Treat every factual claim in AI output as unverified until you have opened the source. The routine below is short enough to actually use.
Step 1: mark the claims. Read the draft once with a pen and mark every sentence that asserts something checkable: a rule, a number, a date, a name, a limit, a form, a reference. Everything marked needs a source; everything unmarked is wording.
Step 2: classify each claim. Where must the answer come from?
| Claim type | The only acceptable source |
|---|---|
| A board or state-law requirement | The state board's or state's own published text, read in the current version |
| A plan or payer condition | The plan document, the payer's own current published policy, or the payer directly |
| A clinical statement of any kind | The licensed dentist; never the model, and never a staff member |
| A practice policy or fee | The practice's own current document or fee schedule |
| A patient-specific fact | The patient's own record |
Step 3: open the source. Not a search result summary, not a second AI answer, not the model's own restatement. The document.
Step 4: compare sentence to sentence. Does the source say what the draft says, in the version that applies today? Module 5 covers how to tell whether a rule is final, proposed, amended, waived or vacated.
Step 5: record what you checked. What you opened, on what date, and who accepted the answer.
Two prompt habits that shrink the surface. Neither makes a model reliable; both make its answers checkable.
- Supply the document rather than asking for recall. "Using only the plan booklet pasted below, dated 1 January 2026, state the annual maximum and quote the sentence that gives it. If the booklet does not say, reply exactly: not stated."
- Require an explicit refusal path. A model asked to produce an answer will usually produce one. Give it permission to say "not stated" and you convert a silent invention into a visible gap.
What the practice must not do to fill a silence. Some questions have no published answer, and a model will still produce one. It is worth saying plainly, because it comes up in every AI conversation in dentistry: no state dental board has published AI guidance, and no regulator action against a dental practice over its use of AI is recorded — not by OCR, not the FTC, not FDA, not a state board. If a draft tells you a board "requires" something about AI, the claim is wrong before you even check which state. Do not fill that silence, and do not let a tool fill it for you.
Remember this
- Confabulation is a designed-in property of generative models, not a rare glitch; NIST defines it and does not quantify it.
- The recorded dental evaluation found inaccuracies, generality, outdated content and missing sources across all four models tested in 2023; the rankings are obsolete, the failure mode is not.
- Mark every checkable claim, classify it, open the primary source, compare sentence to sentence, and record the check.
- Give the model the document and an explicit "not stated" option, so a gap shows up instead of being filled.
- No state dental board has published AI guidance and no regulator action against a practice over AI use is recorded; never let a draft imply otherwise.
Dental practice example
Harrow Lane Orthodontics is a fictional two-doctor practice. Treatment coordinator Bettina Aarons is asked to write a one-page explainer for new adult patients covering payment plans and what the practice does with records. She asks an approved assistant for a draft and gets back something polished. Three sentences carry checkable claims. The first: "State regulations require that a signed financial agreement be retained for seven years from the date of the last payment." The second: "Under Board Rule 12-4.115(b), patients must be given written notice before any records request is fulfilled." The third: "Your plan covers 50% of orthodontic treatment up to a lifetime maximum, as is standard for orthodontic benefits." Bettina marks all three and classifies them. The first is a state-law retention question, so the source is the state's own published text, not the draft. The second is a board rule, so the source is the board's own rule list. The third is a plan condition, so the source is the patient's plan document. She opens the board's published rule index and cannot find the cited rule number at all; the numbering scheme does not match the format in the draft. She does not conclude the rule is right in some other numbering: she treats the whole sentence as unsupported and deletes it. For the retention claim she finds the practice already has a written records-retention policy prepared by the practice's own advisors, so she uses that document's language and cites it internally. For the third sentence, she sees the problem immediately: no plan booklet was supplied to the assistant, and "as is standard" is the tell. She rewrites the explainer so that plan-specific figures are not stated at all, and the page instead tells the patient that their coordinator will confirm the terms from their own plan document before treatment is scheduled. She then reruns the task properly for one fictional patient, pasting the plan booklet and asking for the lifetime maximum with the sentence that states it, and replying "not stated" if absent. The assistant returns "not stated" — the booklet covers it in a table rather than a sentence — which is exactly the outcome she wanted: a visible gap, not an invented figure. Her file note records the three claims, what she opened, the date, and her own name as the administrative owner of the explainer, which Dr Solene Marchetti also reads because one paragraph touches treatment sequencing.
Responsible-use warning
Do not let the specificity of an AI output stand in for its accuracy. A rule number, a subsection letter, an effective date and a page reference are all generated the same way as the rest of the sentence, and a confabulated citation is designed to look like it justifies the answer. Never repeat a board rule, plan term, retention period or form requirement you have not opened yourself in its current published version, and never accept a second AI answer as confirmation of a first. If the source cannot be found, the claim does not go in the document.
Human-review requirement
The reviewer confirms that every checkable claim in the finished document traces to a source someone opened: the state or board's own published text for a legal or board requirement, the plan document or payer's current policy for a benefit term, the practice's own document for a policy or fee, and the patient's record for anything patient-specific. Anything clinical is read and owned by the licensed dentist. The reviewer also checks that the sources opened and the date of checking are recorded, and that no sentence in the document implies a dental board has issued AI guidance or that any practice has faced regulator action over AI use.
Practical exercise
Run a confabulation hunt on fictional material. Invent every document you use; put no patient information and no real record into any tool.
- Write a short fictional plan booklet extract of about 150 words for an invented plan, including one annual maximum, one frequency limitation and one waiting period.
- Without pasting it, ask your practice's approved assistant to explain that invented plan's orthodontic benefit and the notice requirements that apply to records requests in your state. Save the reply exactly.
- Mark every checkable claim in the reply: rules, numbers, dates, citations, limits, forms.
- Classify each marked claim by where its answer must come from, using the table in this lesson.
- Open the primary source for two of them — your state board's own published rules and your practice's own written policy — and compare sentence to sentence.
- Count how many marked claims you could not support. Write that number at the top of the page.
- Now rerun the task the checkable way: paste your invented booklet, ask for the annual maximum with the sentence that states it, and require the exact reply "not stated" if the booklet is silent.
- Write two sentences comparing the two attempts: what changed about the answer, and what changed about how long checking took.
- Keep both outputs and your marked-up copy in the workbook; Module 15's lab reuses this method on a longer document.
Knowledge check
-
How does NIST AI 600-1 define confabulation, and what does it decline to say about it?
Answer guidance: It defines confabulation as the production of confidently stated but erroneous or false content, known colloquially as hallucinations or fabrications, and says it is a natural result of the way generative models are designed. It does not quantify how often confabulation occurs, so no rate may be attributed to it.
-
What did the 2023 JMIR dentistry evaluation find, and what limits how it may be quoted?
Answer guidance: Four models answering twenty open-ended clinical dentistry questions, graded 0-10 by two faculty, all occasionally showed inaccuracies, generality, outdated content and a lack of source references, and the authors warned this can lead to potentially harmful health care decisions if not used judiciously. Twenty questions, two graders and 2023 model versions mean the model rankings are obsolete and must not be quoted as current.
-
An AI draft cites a state dental board rule number. What is the next step?
Answer guidance: Open the board's own published rules in the version that applies today and compare sentence to sentence. If the rule cannot be found, the claim is unsupported and comes out of the document; a second AI answer is not confirmation. Record what was opened and when.
-
Why does giving a model an explicit "not stated" option reduce risk?
Answer guidance: A model asked for an answer will usually produce one. An explicit refusal path converts a silent invention into a visible gap, so the person doing the work sees that the document does not contain the term rather than receiving a plausible figure.
Sources and further reading
- National Institute of Standards and Technology — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (2024-07-26). Accessed 2026-09-15.
- Journal of Medical Internet Research — Evaluation of the Performance of Generative AI Large Language Models ChatGPT, Google Bard, and Microsoft Bing Chat in Supporting Evidence-Based Dentistry: Comparative Mixed Methods Study (2023-12-28). Accessed 2026-09-15.
- OWASP Foundation, OWASP GenAI Security Project — OWASP Top 10 for LLM Applications 2026 (2026-08-04). Accessed 2026-09-15.
- AI Coalition Network — Certification Educational Standard: Human Review of AI-Assisted Professional Work. Accessed 2026-09-15.