Sign up to save your podcasts
Or
Share Ep 62. AI creates a threat disguised as a school security solution
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Ep 62. AI creates a threat disguised as a school security solution
Police to students: “The way you guys were eating chips, it picked it up as a gun. AI’s not the best. Can’t trust stuff like this at a school man. Good thing you guys didn’t run [laughs]...that would have made a major problem.”
Bodycam footage shows Baltimore County, MD police handcuffing 4 students who were waiting to be picked up after football practice when they got an alert from AI software that detected a gun on the school’s CCTV system.
What happened in Baltimore County is more than a minor technical glitch because four teens were forced to the ground at gunpoint because faulty software misinterpreted a bag of Doritos as a gun. The school district spent the equivalent of twenty-five teachers’ salaries on an AI system that can’t tell the difference between a snack and a threat. The outcome is students are even less safe when police with guns drawn show up on campus looking for a shooter who doesn’t exist.
Until AI can truly comprehend context rather than just pixel clusters—and using the most advanced models becomes financially feasible for video analysis—basing student safety on image classification probabilities will continue to cause avoidable disasters like this. When outdated 2010s machine learning software can’t tell the difference between a bag of chips versus a gun, it doesn’t belong on a school campus.
Read more:
* AI classified Doritos chips as a gun and then police pointed real guns at students
* CNN: I study school shootings. Here’s what AI can — and can’t — do to stop them
* How do AI security products being sold to schools really work?
* FTC takes action on school security tech vendor after investigation
* What happens when a school security startup fails?
David Riedman, PhD is the creator of the K-12 School Shooting Database, Chief Data Officer at a global risk management firm, and a tenure-track professor. Listen to my weekly podcast—Back to School Shootings—or my recent interviews on Freakonomics Radio and the New England Journal of Medicine.
School Shooting Data Analysis and Reports is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Get full access to Riedman Report: Risk, AI, Education, & Security at riedmanreport.substack.com/subscribe
View all episodes
By David Riedman, PhDPolice to students: “The way you guys were eating chips, it picked it up as a gun. AI’s not the best. Can’t trust stuff like this at a school man. Good thing you guys didn’t run [laughs]...that would have made a major problem.”
Bodycam footage shows Baltimore County, MD police handcuffing 4 students who were waiting to be picked up after football practice when they got an alert from AI software that detected a gun on the school’s CCTV system.
What happened in Baltimore County is more than a minor technical glitch because four teens were forced to the ground at gunpoint because faulty software misinterpreted a bag of Doritos as a gun. The school district spent the equivalent of twenty-five teachers’ salaries on an AI system that can’t tell the difference between a snack and a threat. The outcome is students are even less safe when police with guns drawn show up on campus looking for a shooter who doesn’t exist.
Until AI can truly comprehend context rather than just pixel clusters—and using the most advanced models becomes financially feasible for video analysis—basing student safety on image classification probabilities will continue to cause avoidable disasters like this. When outdated 2010s machine learning software can’t tell the difference between a bag of chips versus a gun, it doesn’t belong on a school campus.
Read more:
* AI classified Doritos chips as a gun and then police pointed real guns at students
* CNN: I study school shootings. Here’s what AI can — and can’t — do to stop them
* How do AI security products being sold to schools really work?
* FTC takes action on school security tech vendor after investigation
* What happens when a school security startup fails?
David Riedman, PhD is the creator of the K-12 School Shooting Database, Chief Data Officer at a global risk management firm, and a tenure-track professor. Listen to my weekly podcast—Back to School Shootings—or my recent interviews on Freakonomics Radio and the New England Journal of Medicine.
School Shooting Data Analysis and Reports is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Get full access to Riedman Report: Risk, AI, Education, & Security at riedmanreport.substack.com/subscribe