Lesson objective

Distinguish an instructor's own observations from research findings and programme recommendations, and state each in a session with the label it deserves.

Plain-language explanation

Why this matters now

Every instructor accumulates observations. After a dozen sessions you notice that the room goes quiet when the demonstration fails, that partners engage more when given their own practice area, that the question about billing always comes in the last five minutes. These observations are valuable. They are how an instructor improves.

They are not research. The trouble begins when an instructor, wanting to sound authoritative, turns an observation into a finding: "studies show", "we know that adults learn best when", "people only retain a fraction of". Training culture is full of statements of this kind, repeated from slide to slide with the source long since lost.

In this programme the discipline is non-negotiable, for a reason specific to the subject. The central habit you are teaching lawyers is that a confident, well-formed assertion is not evidence that its authority exists. An instructor who asserts an unsourced learning statistic in the same session is modelling the exact behaviour the session warns against.

A question to sit with

If a participant asked you, mid-session, "what is the source for that?" about every general statement you make about how people learn, how many of those statements would survive? And what would the room conclude about your citations to rules and opinions from the ones that did not?

What goes wrong

1. The borrowed statistic. A figure about retention or attention appears on a slide because it appeared on someone else's. Nobody in the chain checked it. A lawyer asks where it comes from, the instructor cannot say, and every other citation in the session is now in doubt.

2. The observation promoted to a law of nature. "Lawyers hate role-play." The instructor has seen three groups resist it. That is a genuine observation about three groups. Stated as a general truth, it is an unsupported claim, and it also closes off a method that may work with the fourth group.

3. The research cited for more than it found. A study of law students using a 2023 model is described as showing what AI does for lawyers. The instructor did not invent anything; they overstated something real, which is harder for a room to spot and just as misleading.

4. The recommendation dressed as a requirement. "You must have a named reviewer." That is this programme's educational standard. Presented as a rule, it is false, and a lawyer who later discovers it is not required has learned to discount everything else the instructor said was required.

5. The honest instructor who over-corrects. Having learned not to claim findings, the instructor says nothing about what they have seen. That throws away the most useful thing they bring. The answer is not silence. It is labelling.

The practical answer

Best practice: sort every general statement into one of four boxes before you teach it.

Box What it is How to say it
Research finding A result from a named study you have read, with its sample and limits "Choi, Monahan and Schwarcz, 2024, sixty law students, a 2023 model, found..."
Rule or guidance An adopted rule, a model provision, or a named body's guidance "The ABA Model Rule, which each jurisdiction adopts in its own form, provides..."
Observation What you have seen in rooms you have taught "In the sessions I have run, I have noticed..."
Recommendation Best practice, or this programme's educational standard "Best practice is..." or "AI Coalition Network recommends, and this is not a requirement..."

If a statement fits none of the boxes, it is a borrowed claim with no source. Delete it.

Claims you should not make in this programme. This module's source register contains no education, pedagogy or cognitive-science research. You therefore do not state, as fact, anything about learning styles, retention rates or percentages, attention spans, the proven effectiveness of any teaching method, or what research shows about how adults learn. This is not a judgement that every such claim is false. It is a statement that you do not hold the evidence, and in a room of lawyers you do not assert what you cannot source.

Best practice: report research the way you would report authority. When a study is genuinely relevant, give the room what a lawyer would want from a citation.

  1. Who: the authors and where it was published.
  2. What was tested: the task, the tool, and the date of the model.
  3. Who took part: the sample, and how it differs from the room.
  4. What was found: the whole result, including the part that complicates your point.
  5. What it cannot tell you: stated before anyone asks.

Research finding: Choi, Monahan and Schwarcz, 109 Minnesota Law Review 147 (2024), reported large, consistent gains in speed from GPT-4 on legal tasks but only slight and inconsistent gains in quality, with the lowest-skilled participants gaining most. Sixty law students; a 2023 model. Reported in that form, it is useful and honest. Reported as "AI makes lawyers better", it is neither.

Best practice: use your observations, labelled, as questions. An observation is most useful when it becomes something the room can test. "In the sessions I have run, participants who checked the source for a summary usually found at least one passage that mattered. Let us see whether that happens here." The observation is honest, it is framed as yours, and the room gets to confirm or contradict it with their own work.

AI Coalition Network recommendation: keep an observation log, and keep it honest. After each session, write down what participants did in the practice tasks, what they asked, and what went differently from the plan. Record the size of the group and the setting. Over time this becomes a genuine source for your own practice. It remains a record of your rooms, and you describe it that way. This is an educational recommendation, not a requirement.

Remember this

  • An instructor's observations are valuable and are not research; label them as observations.
  • Sort every general statement into research finding, rule or guidance, observation, or recommendation; delete anything that fits none.
  • Do not assert claims about learning styles, retention rates, attention spans or what research shows about adult learning; this programme holds no such evidence.
  • Report a study like an authority: who, what was tested, who took part, the whole result, and its limits.
  • An unsourced claim from the front of the room teaches the opposite of the verification habit.

Legal example

The fictional firm Quillon Moreby LLP has an AI training deck inherited from a previous trainer, which the new instructor, Tamsin Achterberg, is asked to deliver to its employment group. Slide 3 reads: "People retain only a small fraction of what they hear in a lecture, so this session is interactive." Slide 7 reads: "Lawyers learn best by doing." Slide 12 reads: "Research proves AI makes lawyers faster and better." Slide 15 reads: "Every lawyer must appoint a named reviewer for AI-assisted work." Tamsin sorts each into the four boxes. Slide 3 fits none: she cannot find the source, and the previous trainer cannot either. She deletes the figure and keeps the decision, replacing it with "This session is built around three tasks, because the aim is for you to leave having done the checks rather than having heard about them." Slide 7 is an observation at best; she rewrites it as "In the sessions I have run, the discussion after a practice task has been far more specific than the questions after a talk. You can judge for yourselves today." Slide 12 overstates real research. She reopens the Choi, Monahan and Schwarcz article and rewrites the slide to report large speed gains, only slight and inconsistent quality gains, the sixty law students and the 2023 model. Slide 15 presents a programme recommendation as a requirement. She rewrites it as an AI Coalition Network educational standard and adds a line saying the firm has adopted it as policy, which is true at Quillon Moreby and is a different thing from a legal duty. At the session a senior associate asks where the speed figures come from. Tamsin gives the citation, the sample and the limits in one breath. The associate nods and, later, asks her to review the group's own research memo template.

Responsible-use warning

Never repeat a learning statistic, a claim about learning styles, or a statement of what "research shows" about adult learning unless you have read the source and can state its sample and limits, and this programme's register contains none. Do not over-correct into silence either: your observations of your own rooms are useful when labelled as observations. Never present an AI Coalition Network recommendation as a legal requirement, and never describe Comment 8 as applying in every state.

Human-review requirement

A licensed attorney reviews the labelled statements before delivery wherever they concern rules, guidance or professional duties. The reviewer confirms that each rule is identified as a model provision or as a named jurisdiction's adopted rule, that the jurisdictional caveat on Comment 8 to Model Rule 1.1 is stated, that every research citation carries its sample and limits, that recommendations are not presented as requirements, and that firm policy is distinguished from legal duty.

Practical exercise

Sort and relabel. Use a deck you have delivered or attended, or the fictional slide list below.

Fictional slide list: "Adults have short attention spans, so we will break every ten minutes." "AI tools make lawyers more accurate." "The duty of technology competence applies to all US lawyers." "You must record a named reviewer on every AI-assisted document." "Lawyers dislike group work."

  1. Place every general statement in one of the four boxes: research finding, rule or guidance, observation, or recommendation.
  2. For any statement that fits no box, delete it and write what, if anything, you will say instead.
  3. For any research finding, write the five elements: who, what was tested, who took part, the whole result, and what it cannot tell you.
  4. For any rule, name whether it is a model provision or an adopted rule, and add the jurisdictional caveat.
  5. Rewrite any observation so that it is framed as yours and posed as something the room can test.
  6. Rewrite any recommendation so that its label is unmistakable.
  7. Draft the first three entries of an observation log for a fictional session, recording group size, setting, what participants did and what differed from the plan.
  8. Read your rewritten slides as a sceptical senior associate would, and mark any sentence where you would ask "says who?". Fix each one before delivery.

Knowledge check

  1. Name the four boxes into which every general statement in a session should be sorted.

    Answer guidance: Research finding (a named study with its sample and limits), rule or guidance (an adopted rule, a model provision, or a named body's guidance), observation (what the instructor has seen in their own rooms), and recommendation (best practice or the programme's educational standard).

  2. Why is an unsourced learning statistic particularly damaging in a legal AI session?

    Answer guidance: Because the session is teaching that fluent, confident assertions are not evidence; an instructor presenting an unsourced claim as fact models the behaviour the session warns against and gives the room reason to doubt every other citation.

  3. What caveat must accompany Comment 8 to Model Rule 1.1?

    Answer guidance: That it is a comment to a model rule, and although most states have adopted a version of it, not all have; the instructor must check the jurisdiction of the audience before asserting it applies.

  4. How should an instructor use their own observations in a session?

    Answer guidance: Labelled clearly as observations from the rooms they have taught, and ideally posed as something the current room can test against its own work, rather than stated as general truths or research.

Sources and further reading