Glenn Broder describes a test he ran on a chat model: a story so absurd that a child would have seen through it, and after about ten exchanges the AI was telling him to move away from the windows, lock the doors, call 911 and let nobody in. He now uses that exchange when he trains people, especially children, to show that ungoverned AI is dangerous and that governed AI can do remarkable work, as long as human judgment stays in place.
His course, Human Judgment in the Age of AI, runs eight one-hour sessions: it starts with how a model actually produces an answer and why it is so confident when it is wrong, spends several sessions on governance, and ends with each student building a project of their own. With companies, his team first builds a business canon of how the business operates, works with the employees to find the bottlenecks in the workflow, and keeps the system outside the company network. Whoever signs off on the work still owns the decision.
• He calls it a cognitive jungle gym when people verify and confirm what the AI drafts, and a cognitive wheelchair when they just press send; timing the review step shows which is which.
• His test of whether someone used AI well is whether they can explain their work. If they cannot, they used it as a wheelchair.
• Students in their seventies catch the model's mistakes because they are expert in the work they are doing.
• He teaches deterministic versus probabilistic software with a wrong button that returns an error, and a model learning to fill in "the dog is chasing a ___".
• He applies the Talmudic logic he was taught as a child, questioning everything respectfully until the nuance is clear, to all of his systems, including HACS, the control layer that checks prompts going in and answers coming out against the company canon.
• His newest formulation: build your business canon outside your network, so you are never locked into one AI provider.
Glenn Broder created Human Judgment in the Age of AI to train professionals and students to keep final accountability with the human user.
Find Glenn Broder:
LinkedIn: https://www.linkedin.com/in/glennbroder
Chapters:
0:00 A Wall Street Journal story and the race to dominate AI
2:39 Ten interactions to lock the doors and call 911
6:23 The cognitive jungle gym and the cognitive wheelchair
9:55 Removing the drudgery, and the deadwood it exposes
15:34 The false guidance test and the explain-it test
22:05 Seniors in their seventies and the eight one-hour sessions
25:38 Deterministic versus probabilistic, and a car that will not turn itself off
31:42 Who should decide, Talmudic logic, and why seeing is not believing
39:39 A book, Eric Schmidt, pigs that get fat, and his mother
43:46 Building AI outside the network