Lesson objective

Explain in plain terms how a large language model is trained and produces text one token at a time, and turn each part of that mechanism into a checking habit for plant work.

Plain-language explanation

Why this matters now

Most people in a plant meet a large language model through a chat box, or through a button that has appeared inside software they already run. The interface makes it look like a very well-read colleague who has memorised every standard, every print and every work order. Knowing roughly what happens when you press enter lets you predict where it will be wrong before it is wrong. That is competence, not a technical hobby.

It also matters because nobody else is going to explain it to you in plant terms. NIST's smart-manufacturing programme index states plainly that "applying this technology without considering the unique needs of manufacturing can adversely affect safety, performance, quality, and cost," and that its trustworthy-systems work aims to "establish a basis of trust in their application, delivering guidelines, best practices, and standards." Those are research programmes, not rules, and none of the four named programmes is described on that page as an AI programme. The point stands anyway: a general-purpose tool dropped into a plant does not arrive knowing what a plant is.

A question to sit with

If you asked the same assistant the same question about a fixture tolerance on Monday and again on Tuesday, and got two different answers, which one would you put in the job file, and why?

What goes wrong

Treating the model as a database. People ask for a value the way they would open a table. Unless the product has been connected to a source — Lesson 3 covers that — the model does not look anything up. It produces text that is likely to follow your question.

Assuming it knows this year's editions. Training stops at a point in time. A standard reissued since, a customer specification revised since, a drawing revision released last month: none of that is in the model unless somebody supplies it, and it will not necessarily tell you that it does not know.

Reading the shown working as evidence. The step-by-step reasoning is generated the same way as the answer. It can be just as wrong, and it makes a wrong answer look checked.

Expecting the same answer twice. The model assembles each answer from likely continuations, and many products allow variation in which continuation is chosen. Two runs of one prompt can differ in wording, structure and sometimes substance.

Forgetting that everything pasted in is also input. Your instruction, the procedure you pasted and the model's own reply are all text in the same stream. Wording inside a document you supply can steer the result.

The practical answer

You do not need the mathematics. You need four plain ideas, each with a habit attached.

1. Training. A model is built by exposing it to very large amounts of text and adjusting it, over and over, to get better at predicting what comes next. What it keeps is not a filing cabinet it can open. It is a very large set of learned patterns about how words, numbers and ideas tend to follow one another. Developers usually refine it further so it follows instructions and holds a conversation. Training ends at a date.

2. Tokens. The model does not see letters or numbers the way you do. Text is broken into tokens: short pieces that may be a whole word, part of a word, a group of digits or a punctuation mark. A dimension such as 1.2507 may be split into several tokens. To the model a number is a run of text pieces, not a quantity it holds in mind. This is why unit conversions and tolerance arithmetic go wrong with no outward sign.

3. Prediction. When you send a prompt, the model predicts a likely next token, adds it, and predicts again, until the answer is finished. Every answer is assembled one piece at a time from patterns. Nothing in that loop compares the answer to the world, to your print, or to your process.

4. One stream of text. There is no separate channel for "your instruction" and "the document". It is all tokens in one window, and the window has a limit. Paste a 400-line work-order history and the parts that fall outside the window simply are not there, even though the reply will read as though the whole history was considered.

Now attach the habits.

What the model is doing What it means in a plant The habit
Predicting likely text from patterns A likely-sounding answer and a correct answer are different things Verify against the released document you hold
Working from training that ended at a date Standard editions, customer specs and drawing revisions may be stale Supply the current document; never ask it to recall a revision level
Treating numbers as pieces of text Tolerance and unit arithmetic can be wrong with no sign Recompute outside the tool, every time
Choosing among likely continuations Two runs of one prompt can differ Save the prompt, the output you relied on, the tool and the date
Reading everything as one stream, with a limit Long histories are silently truncated Feed a defined, bounded extract you chose yourself
Generating its explanation the same way as its answer The reasoning can be as wrong as the conclusion Test the conclusion against evidence, not against the explanation

Supply, do not ask it to recall. The single most useful change follows straight from the mechanism. Compare two requests.

  • Weak: "What surface finish does our customer require on the bore?" The model has never seen your customer's drawing. It will produce a plausible number.
  • Stronger: "Using only the drawing extract pasted below, part 4412-B revision F dated 2 March 2026, state the surface finish called for on the bore and quote the note that gives it. If the extract does not state it, reply 'not stated'."

The stronger request does not make the model reliable. It makes the answer checkable, because you hold the source and you have asked the model to point at it. Module 09 builds this into a full prompt structure. Before either version is used on real plant material, the export-control and confidentiality questions in Modules 03 and 04 have to be answered first — a drawing extract is not automatically yours to paste anywhere.

Save what you relied on. Because two runs can differ, "the tool said so" cannot be reconstructed later. Keep the prompt, the output you used, the tool and the date with the job file.

Use a shared vocabulary. The AI RMF 1.0 lists the characteristics of trustworthy AI as valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair with harmful bias managed. It also says that after instituting GOVERN outcomes, "most users of the AI RMF would start with the MAP function," continuing to MEASURE or MANAGE, iteratively. The framework is voluntary and binds nobody, but three of those words make good reviewer questions: valid and reliable (does the output match the source?), accountable and transparent (can we show what was asked and who accepted it?) and explainable and interpretable (can the preparer say why it is right without leaning on the model's own explanation?).

Remember this

  • A language model predicts likely text one token at a time; nothing in that loop checks the answer against your plant.
  • Training ends at a date, so supply the current print, spec or procedure instead of asking the model to recall it.
  • Numbers are pieces of text to the model: recompute every dimension, tolerance and conversion outside the tool.
  • The shown working is generated the same way as the answer and can be equally wrong.
  • Save the prompt, the output, the tool and the date, because the next run may differ.

Manufacturing example

Hollow Creek Stamping is a fictional 40-person stamping plant. Process engineer Leo Barrantes is writing a setup sheet for an invented job, part HC-2208 revision C, a bracket blanked from 2.50 mm sheet. On Monday he pastes a short extract of the internal process standard into the plant's approved assistant and asks for the die clearance per side and the total for both sides, given a stated clearance of 6% of stock thickness. The reply gives 0.15 mm per side and a total of 0.30 mm, with a tidy explanation: "6% of 2.50 mm is 0.15 mm; two sides gives 0.30 mm." Leo recomputes. Six percent of 2.50 mm is 0.15 mm, so the per-side figure is right, and 0.15 times 2 is 0.30 mm, so the total is right too. Encouraged, he asks the same question for a second invented job at 3.20 mm stock. This time the reply says 0.19 mm per side and 0.38 mm total. Leo recomputes: 6% of 3.20 mm is 0.192 mm, and doubling gives 0.384 mm. The tool has silently rounded in a way the plant standard does not permit, and the explanation named the right steps while doing it. On Tuesday, preparing the file for review, he reruns the first prompt to get a clean copy. This time the reply adds a sentence about a "minimum clearance of 0.20 mm for mild steel" that appears nowhere in the extract he supplied and nowhere in the plant standard. It was generated because it sounded like the kind of thing such a standard says. Leo stops using the assistant for figures altogether. He takes the clearances from the plant standard itself, builds the setup sheet from the released document, and keeps the assistant for one job it can do checkably: listing which sections of the pasted extract mention clearance, with the section numbers, which he then opens and reads. In the job file he records both prompts, both outputs, the tool and the date, notes that every clearance was recomputed from the standard, and names himself as preparer and the tooling supervisor, Nadia Prosser, as the approver under the document-control procedure.

Responsible-use warning

Do not ask a language model to remember a drawing revision, a customer specification, a standard edition or a torque value and then treat the reply as an answer. It has not looked anything up. It has produced text likely to follow your question, from training that ended at some past date. A confident number with tidy working is exactly what confabulation looks like. Supply the released document, ask the model to point at the line that answers the question, and then open that line yourself. Recompute every dimension, tolerance and unit conversion outside the tool.

Human-review requirement

The reviewer confirms that no dimension, tolerance, revision level, specification reference or material property in the work came from the model's memory: every one traces to a released document held in the file or to a recomputation done outside the tool. The reviewer checks that the prompt, the output relied on, the tool and the date are saved with the job, and that the model's own explanation was not treated as a check. Ownership follows the output: the named quality authority for anything that dispositions or releases product, the document-control approver for a procedure or work instruction, and a licensed professional engineer where the state requires a seal on an engineering document.

Practical exercise

Watch the mechanism work on material you invent. Use only fictional text you write yourself, and only a tool your employer has approved.

  1. Write a short fictional process standard of about 120 words for an invented company. Include one specific numeric rule, such as a clearance percentage or a torque, and one effective date.
  2. Without pasting it, ask your approved assistant what that invented company's rule says. Record the reply word for word.
  3. Now paste the standard and ask the same question, requiring the tool to quote the sentence that states the rule and to reply "not stated" if it is absent. Record the reply.
  4. Run the step 3 prompt twice more. Note every difference in wording, structure or substance across the three replies.
  5. Ask for a calculation using your rule at two different input values, one of which does not divide evenly. Recompute both yourself and note any rounding the tool introduced.
  6. Ask the tool to explain its calculation. Write one sentence on whether that explanation would have helped you catch the rounding error, and why.
  7. Build a two-column table: for each of the four ideas in this lesson — training, tokens, prediction, one stream of text — write what you observed and the habit it supports.
  8. Record the prompts, outputs, tool and date exactly as you would in a job file.

Knowledge check

  1. In plain terms, how does a large language model produce an answer?

    Answer guidance: It breaks text into tokens and, using patterns learned during training, predicts a likely next token, adds it, and predicts again until the answer is complete. Nothing in that loop compares the answer to a source.

  2. Why is asking a model to recall a drawing revision level or a standard edition a poor practice?

    Answer guidance: It does not look the value up; it generates likely text from training that ended at a past date, so anything reissued or revised since is missing unless supplied. Give it the released document and verify the answer against that document.

  3. What does NIST AI 600-1 say about the explanations generative models give?

    Answer guidance: It says outputs may include confabulated logic or citations that purport to justify or explain the answer, and that models sometimes provide logical steps even when the answer is incorrect. The profile is voluntary and not manufacturing-specific.

  4. Why does pasting a very long work-order history not guarantee the model considered all of it?

    Answer guidance: Everything is tokens in one window, and the window has a limit. Material beyond that limit is simply not present, although the reply will still read as if the whole history had been reviewed. Feed a bounded extract you selected yourself.

Sources and further reading