Government Testers Broke ChatGPT, Claude, Gemini & Grok
A U.S. government watchdog tested ChatGPT, Claude, Gemini and Grok, and got all four to help produce attack plans. California is suing OpenAI over AI agents that allegedly acted in ways their developers did not intend. And the Justice Department released findings on two government lawyers who misused AI. Three stories, one question: when something goes wrong with AI, who is responsible?
I'm James Henderson, and this is the AI Coalition Network: AI news and AI ethics, explained in plain English.
CHAPTERS
The government tested 4 top AI systems
Welcome and today's three stories
Story 1: The government tests AI safety (GAO)
Why can't AI just refuse?
Story 2: California sues OpenAI
What is an AI agent?
How much access should an AI agent get?
Story 3: Government lawyers in trouble over AI
What is an AI hallucination?
Confidential files and public AI
Who's responsible?
3 ways to protect yourself
The future of AI
Your turn
WHAT WE COVER
Story 1: In a report released Nov. 18, 2026 (GAO-27-108814), the Government Accountability Office said its investigators used four of the most-used AI platforms (Claude, Grok, ChatGPT and Gemini) in spring 2026 to produce plans for a vehicle-bomb attack and for attacks on power-grid substations. GAO said every platform first refused, and investigators then used common techniques to get around the safeguards. GAO confirmed the plans were "largely viable," while noting the details "were not novel" and the plans contained errors. No attacks were carried out, and the AI did not act on its own. GAO made no formal recommendations; Sen. Maggie Hassan plans hearings in January.
Story 2: California Attorney General Rob Bonta sued OpenAI on Nov. 16, 2026, alleging it tested AI agents without proper oversight. These are allegations, not court findings. OpenAI says the lawsuit singles it out over a challenge identified by every major AI lab.
Story 3: The DOJ Office of Professional Responsibility found that one federal attorney filed a brief with fabricated quotations and misrepresented case holdings, then misled the court about his AI use; a second Department attorney uploaded protected and personal documents, including medical information, to a public AI platform without approval. Both matters were referred to DOJ's Professional Misconduct Review Unit.
CORRECTIONS AND CLARIFICATIONS (also shown on screen)
- GAO says EACH of the four platforms produced its attack plans in under 12 hours (James says "some").
- Every platform refused requests at first; one company suspended the account after testing ended, and two deleted parts of the chats (James says "accounts were restricted").
- James says "Government of Justice"; it is the U.S. Department of Justice (Office of Professional Responsibility).
- In the second DOJ case, two documents were covered by a tribunal's protective order (James says "a court order").
- OPR's own wording: the attorneys "knowingly and intentionally violated" the rules of professional conduct (and, in the second case, also "recklessly violated" them).
SOURCES
GAO-27-108814, "Artificial Intelligence: Covert Testing Used Common Platforms to Create Viable Attack Plans" (released Nov 18, 2026): https://www.gao.gov/products/gao-27-108814
Report PDF (posted by Sen. Hassan): https://www.hassan.senate.gov/imo/media/doc/gao_ai_malicious_use_report.pdf
CBS News/AP on the report and planned Senate hearings (Nov 18, 2026): https://www.cbsnews.com/news/ai-platforms-plan-us-terror-attack-when-asked-to-senate-hearings/
California v. OpenAI complaint (filed Nov 16, 2026): https://oag.ca.gov/system/files/attachments/press-docs/complaint-oai-cyber.pdf
CNBC, including OpenAI's response (Nov 16, 2026): https://www.cnbc.com/2026/11/16/openai-california-bonta-ai-lawsuit.html
DOJ Office of Professional Responsibility, 2026 Investigative Summary 5: https://www.justice.gov/opr/2026-investigative-summary-5
DOJ Office of Professional Responsibility, 2026 Investigative Summary 6: https://www.justice.gov/opr/2026-investigative-summary-6
DISCLOSURES
Conflict of interest: this episode was edited and fact-checked with Claude, made by Anthropic, one of the four companies whose AI systems GAO tested. GAO did not say which test platform was which, and we don't either. The "coffee" sponsor bits were jokes and have been cut; this episode has no sponsor. The citation idea and the private-model comments are James's opinions.
#AI #AISafety #ArtificialIntelligence #AIEthics #AINews
Website: https://ai-coalition.net
Rumble: https://rumble.com/c/AICoalitionNetwork
Patreon: https://www.patreon.com/c/AICoalitionNetwork
Locals: https://aicoalitionnetwork.locals.com
If an AI system causes real harm, who should be responsible: the company that built it, the person who used it, or both? Tell me in the comments.
More episodes
All videos
17:14
Is AI Putting Your Retirement at Risk? The AUDIT AI Act Explained
Nov 20, 2026
19 views
22:16
Who Should Control AI? California's AI Shutdown Plan, AI Cyber Threats & American Workers
Nov 19, 2026
24 views
0:59
3 Ways to Protect Yourself When Using AI
Nov 19, 2026
2 views
0:35
Should We Cite AI Like a Book Report?
Nov 19, 2026
0 views
1:08
A Government Lawyer Uploaded Confidential Files to a Public AI
Nov 19, 2026
1 view
0:57
Why Can't AI Just Refuse Dangerous Requests?
Nov 19, 2026
2 views