In this episode, we talk about practical guardrails for LLMs with data scientist Nicholas Brathwaite. We focus on how to stop PII leaks, retrieve data, and evaluate safety with real limits. We weigh managed solutions like AWS Bedrock against open-source approaches and discuss when to skip LLMs altogether.

• Why guardrails matter for PII, secrets, and access control
• Where to place controls across prompt, training, and output
• Prompt injection, jailbreaks, and adversarial handling
• RAG design with vector DB separation and permissions
• Evaluation methods, risk scoring, and cost trade-offs
• AWS Bedrock guardrails vs open-source customization
• Domain-adapted safety models and policy matching
• When deterministic systems beat LLM complexity

This episode is part of our "AI in Practice” series, where we invite guests to talk about the reality of their work in AI. From hands-on development to scientific research, be sure to check out other episodes under this heading in our listings.

Related research:

• Building trustworthy AI: Guardrail technologies and strategies (N. Brathwaite)
• Nic's GitHub

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

We’re excited to have Adi Ganesan, a PhD researcher at Stony Brook University, the University of Pennsylvania, and Vanderbilt, on the show. We’ll talk about how large language models LLMs) are being tested and used in psychology, citing examples from mental health research. Fun fact: Adi was Sid's research partner during his Ph.D. program.

Discussion highlights

• Language models struggle with certain aspects of therapy including being over-eager to solve problems rather than building understanding
• Current models are poor at detecting psychomotor symptoms from text alone but are oversensitive to suicidality markers
• Cognitive reframing assistance represents a promising application where LLMs can help identify thought traps
• Proper evaluation frameworks must include privacy, security, effectiveness, and appropriate engagement levels
• Theory of mind remains a significant challenge for LLMs in therapeutic contexts; example: The Sally-Anne Test.
• Responsible implementation requires staged evaluation before patient-facing deployment

Resources

To learn more about Adi's research and topics discussed in this episode, check out the following resources:

• Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation
• Therapist Behaviors paper: [2401.00820] A Computational Framework for Behavioral Assessment of LLM Therapists 
• Cognitive reframing paper: Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction - ACL Anthology 
• Faux Pas paper: Testing theory of mind in large language models and humans | Nature Human Behaviour 
• READI: Readiness Evaluation for Artificial Intelligence-Mental Health Deployment and Implementation (READI): A Review and Proposed Framework 
• Large language models could change the future of behavioral healthcare: A proposal for responsible development and evaluation | npj Mental Health Research 
• GPT-4’s Schema of Depression: Explaining GPT-4’s Schema of Depression Using Machine Behavior Analysis
• Adi’s Profile: Adithya V Ganesan - Google Scholar 

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

The release of OpenAI GPT-5 marks a significant turning point in AI development, but maybe not the one most enthusiasts had envisioned. The latest version seems to reveal the natural ceiling of current language model capabilities with incremental rather than revolutionary improvements over GPT-4. 

Sid and Andrew call back to some of the model-building basics that have led to this point to give their assessment of the early days of the GPT-5 release.

• AI's version of Moore's Law is slowing down dramatically with GPT-5
• OpenAI appears to be experiencing an identity crisis, uncertain whether to target consumers or enterprises
• Running out of human-written data is a fundamental barrier to continued exponential improvement
• Synthetic data cannot provide the same quality as original human content
• Health-related usage of LLMs presents particularly dangerous applications
• Users developing dependencies on specific model behaviors face disruption when models change
• Model outputs are now being verified rather than just inputs, representing a small improvement in safety
• The next phase of AI development may involve revisiting reinforcement learning and expert systems
* Review the GPT-5 system card for further information

Follow The AI Fundamentalists on your favorite podcast app for more discussions on the direction of generative AI and building better AI systems.

This summary was AI-generated from the original transcript of the podcast that is linked to this episode.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Sid Mangalik and Andrew Clark explore the unique governance challenges of agentic AI systems, highlighting the compounding error rates, security risks, and hidden costs that organizations must address when implementing multi-step AI processes. 

Show notes:

• Agentic AI systems require governance at every step: perception, reasoning, action, and learning
• Error rates compound dramatically in multi-step processes - a 90% accurate model per step becomes only 65% accurate over four steps
• Two-way information flow creates new security and confidentiality vulnerabilities. For example, targeted prompting to improve awareness comes at the cost of performance. (arXiv, May 24, 2025)
• Traditional governance approaches are insufficient for the complexity of agentic systems
• Organizations must implement granular monitoring, logging, and validation for each component
• Human-in-the-loop oversight is not a substitute for robust governance frameworks
• The true cost of agentic systems includes governance overhead, monitoring tools, and human expertise

Make sure you check out Part 1: Mechanism design, Part 2: Utility functions, and Part 3: Linear programming. If you're building agentic AI systems, we'd love to hear your questions and experiences. Contact us.

What we're reading:

• We took reading "break" this episode to celebrate Sid! This month, he successfully defended his Ph.D. Thesis on "Psychological Health and Belief Measurement at Scale Through Language." Say congrats!>>

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

We continue with our series about building agentic AI systems from the ground up and for desired accuracy.  In this episode, we explore linear programming and optimization methods that enable reliable decision-making within constraints. 

Show notes:

• Linear programming allows us to solve problems with multiple constraints, like finding optimal flights that meet budget requirements
• The Lagrange multiplier method helps find optimal solutions within constraints by reformulating utility functions
• Combinatorial optimization handles discrete choices like selecting specific flights rather than continuous variables
• Dynamic programming techniques break complex problems into manageable subproblems to find solutions efficiently
• Mixed integer programming combines continuous variables (like budget) with discrete choices (like flights)
• Neurosymbolic approaches potentially offer conversational interfaces with the reliability of mathematical solvers
• Unlike pattern-matching LLMs, mathematical optimization guarantees solutions that respect user constraints

Make sure you check out Part 1: Mechanism design and Part 2: Utility functions. In the next episode, we'll pull all of the components from these three episodes to demonstrate a complete travel agent AI implementation with code examples and governance considerations.

What we're reading:

• Burn Book - Kara Swisher, March 2025
• Signal and the Noise - Nate Silver, 2012
• Leadership in Turbulent Times - Doris Kearns Goodwin

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

The hosts look at utility functions as the mathematical basis for making AI systems. They use the example of a travel agent that doesn’t get tired and can be increased indefinitely to meet increasing customer demand. They also discuss the difference between this structured, economic-based approach with the problems of using large language models for multi-step tasks.

This episode is part 2 of our series about building smarter AI agents from the fundamentals. Listen to Part 1 about mechanism design HERE.

Show notes:

• Discussing the current AI landscape where companies are discovering implementation is harder than anticipated
• Introducing the travel agent use case requiring ingestion, reasoning, execution, and feedback capabilities
• Explaining why LLMs aren't designed for optimization tasks despite their conversational abilities
• Breaking down utility functions from economic theory as a way to quantify user preferences
• Exploring concepts like indifference curves and marginal rates of substitution for preference modeling
• Examining four cases of utility relationships: independent goods, substitutes, complements, and diminishing returns
• Highlighting how mathematical optimization provides explainability and guarantees that LLMs cannot
• Setting up for future episodes that will detail the technical implementation of utility-based agents

Subscribe so that you don't miss the next episode. In part 3, Andrew and Sid will explain linear programming and other optimization techniques to build upon these utility functions and create truly personalized travel experiences.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

What if we've been approaching AI agents all wrong? While the tech world obsesses over larger language models (LLMs) and prompt engineering, there'a a foundational approach that could revolutionize how we build trustworthy AI systems: mechanism design.

This episode kicks off an exciting series where we're building AI agents "the hard way"—using principles from game theory and microeconomics to create systems with predictable, governable behavior. Rather than hoping an LLM can magically handle complex multi-step processes like booking travel, Sid and Andrew explore how to design the rules of the game so that even self-interested agents produce optimal outcomes.

Drawing from our conversation with Dr. Michael Zarham (Episode 32), we break down why LLM-based agents struggle with transparency and governance. The "surface area" for errors expands dramatically when you can't explain how decisions are made across multiple steps. Instead, mechanism design creates clear states with defined optimization parameters at each stage—making the entire system more reliable and accountable.

We explore the famous Prisoner's Dilemma to illustrate how individual incentives can work against collective benefits without proper system design. Then we introduce the Vickrey-Clark-Groves mechanism, which ensures AI agents truthfully reveal preferences and actively participate in multi-step processes—critical properties for enterprise applications.

Beyond technical advantages, this approach offers something profound: a way to preserve humanity in increasingly automated systems. By explicitly designing for values, fairness, and social welfare, we're not just building better agents—we're ensuring AI serves human needs rather than replacing human thought.

Subscribe now to follow our journey as we build an agentic travel system from first principles, applying these concepts to real business challenges. Have questions about mechanism design for AI? Send them our way for future episodes!

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Dr. Michael Zargham provides a systems engineering perspective on AI agents, emphasizing accountability structures and the relationship between principals who deploy agents and the agents themselves. In this episode, he brings clarity to the often misunderstood concept of agents in AI by grounding them in established engineering principles rather than treating them as mysterious or elusive entities.

Show highlights

• Agents should be understood through the lens of the principal-agent relationship, with clear lines of accountability
• True validation of AI systems means ensuring outcomes match intentions, not just optimizing loss functions
• LLMs by themselves are "high-dimensional word calculators," not agents - agents are more complex systems with LLMs as components
• Guardrails provide deterministic constraints ("musts" or "shalls") versus constitutional AI's softer guidance ("shoulds")
• Systems engineering approaches from civil engineering and materials science offer valuable frameworks for AI development
• Authority and accountability must align - people shouldn't be held responsible for systems they don't have authority to control
• The transition from static input-output to closed-loop dynamical systems represents the shift toward truly agentic behavior
• Robust agent systems require both exploration (lab work) and exploitation (hardened deployment) phases with different standards

Explore Dr. Zargham's work

• Protocols and Institutions (Feb 27, 2025)
• Comments Submitted by BlockScience, University of Washington APL Information Risk and Synthetic Intelligence Research Initiative (IRSIRI), Cognitive Security and Education Forum (COGSEC), and the Active Inference Institute (AII) to the Networking and Information Technology Research and Development National Coordination Office's Request for Comment on The Creation of a National Digital Twins R&D Strategic Plan NITRD-2024-13379 (Aug 8, 2024)

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Part 2 of this series could have easily been renamed "AI for science: The expert’s guide to practical machine learning.” We continue our discussion with Christoph Molnar and Timo Freiesleben to look at how scientists can apply supervised machine learning techniques from the previous episode into their research.

Introduction to supervised ML for science (0:00)

•  Welcome back to Christoph Molnar and Timo Freiesleben, co-authors of “Supervised Machine Learning for Science: How to Stop Worrying and Love Your Black Box”

The model as the expert? (1:00)

• Evaluation metrics have profound downstream effects on all modeling decisions
• Data augmentation offers a simple yet powerful way to incorporate domain knowledge
• Domain expertise is often undervalued in data science despite being crucial

Measuring causality: Metrics and blind spots (10:10)

• Causality approaches in ML range from exploring associations to inferring treatment effects

Connecting models to scientific understanding (18:00)

• Interpretation methods must stay within realistic data distributions to yield meaningful insights

Robustness across distribution shifts (26:40)

• Robustness requires understanding what distribution shifts affect your model
• Pre-trained models and transfer learning provide promising paths to more robust scientific ML

Reproducibility challenges in ML and science (35:00)

• Reproducibility challenges differ between traditional science and machine learning

Go back to listen to part one of this series for the conceptual foundations that support these practical applications.

Check out Christoph and Timo's book “Supervised Machine Learning for Science: How to Stop Worrying and Love Your Black Box” available online now.

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Machine learning is transforming scientific research across disciplines, but many scientists remain skeptical about using approaches that focus on prediction over causal understanding. 

That’s why we are excited to have Christoph Molnar return to the podcast with Timo Freiesleben. They are co-authors of "Supervised Machine Learning for Science: How to Stop Worrying and Love your Black Box." We will talk about the perceived problems with automation in certain sciences and find out how scientists can use machine learning without losing scientific accuracy.

• Different scientific disciplines have varying goals beyond prediction, including control, explanation, and reasoning about phenomena
• Traditional scientific approaches build models from simple to complex, while machine learning often starts with complex models
• Scientists worry about using ML due to lack of interpretability and causal understanding
• ML can both integrate domain knowledge and test existing scientific hypotheses
• "Shortcut learning" occurs when models find predictive patterns that aren't meaningful
• Machine learning adoption varies widely across scientific fields
• Ecology and medical imaging have embraced ML, while other fields remain cautious
• Future directions include ML potentially discovering scientific laws humans can understand
• Researchers should view machine learning as another tool in their scientific toolkit

Stay tuned! In part 2, we'll shift the discussion with Christoph and Timo to talk about putting these concepts into practice. 

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.