In this Humanitarian AI Today Voices flashpod, Theodora Skeadas, Head of AI Red Teaming with Humane Intelligence, joins guest host Brent Phillips to unpack for staff from humanitarian organizations what AI red teaming is and how Humane Intelligence approaches carrying out AI evaluations. They also discuss the nonprofit’s mission to make AI systems more accountable, fair, and safe and the organization’s latest initiatives.

The discussion traces the evolution of Humane Intelligence’s evaluation work, highlighting recent milestones such as the open-source release of their evaluation application and their expanded bias bounty programming in collaboration with platforms like Zindi. Theodora outlines the fundamentals of artificial intelligence evaluations and red teaming methodologies applied to humanitarian applications. The discussion offers a practical introduction to the testing and evaluation of generative AI models, specifically tailored to help staff from aid organizations understand how these systems operate and learn how to carry out effective risk assessments.

For humanitarian actors looking to ethically integrate artificial intelligence into active crises workflows, the interview provides a grounded framework for identifying potential harms before and after applications deployment. Theodora outlines how organizations can proactively red team tools for systemic vectors of concern, including factuality, geographic bias, and public health misinformation using simple workplace tools like spreadsheets or through step-by-step guidance found in resources like The Playbook: Red Teaming Artificial Intelligence for Social Good. By demonstrating how humanitarian actors can systematically test language models for safe output boundaries, the conversation illustrates a critical pathway toward building more transparent, responsible, and localized AI implementations across the sector.