Humanitarian AI Today, guest host Mala Kumar, Head of Impact at Humane Intelligence, sits down with Andre Heller, Director of Signpost at the International Rescue Committee (IRC). They discuss Signpost's recent research paper on piloting an "information assistant," detailing the technical architecture, evaluation methods, and lessons learned from the project The conversation also addresses the significant challenges facing the sector, including a funding crisis that has impacted the pace, scale, and scope of critical research being carried out across the humanitarian community advancing humanitarian applications of artificial intelligence.

In the midst of this crisis and the explosive growth of AI, Andre and Mala emphasize the need for more rigorous, scientific, and collaborative approaches to AI development and evaluation. They speak in detail about open source AI and what it means to the humanitarian community. And Mala explains how her organization, Humane Intelligence, is working to professionalize AI evaluation by creating a community of practice around fair and representative algorithmic auditing. She describes their work in red teaming, conducting "bias bounties," and the future development of open-source "evaluation cards" to make evaluation methodologies more transparent and reusable.

Episode notes: https://medium.com/humanitarian-ai-today/andre-heller-and-mala-kumar-discuss-signposts-pilot-ai-assistant-903b5b8ab787