Episode Summary

In this episode, Maribel Lopez interviews Kate Soule, Director of Technical Product Management for IBM's Granite products. They discuss IBM's third-generation AI models, their focus on efficiency and enterprise readiness, and the latest advancements including vision capabilities and reasoning features.

Guest

Kate Soule - Director of Technical Product Management for IBM's Granite products

Key Topics & Timestamps

00:04 - Introduction

• Maribel introduces the show and Kate Soule
• Brief overview of IBM Granite as fit-for-purpose, open-source enterprise AI models

00:48 - What is IBM Granite?

• Designed as core building blocks for enterprises building with generative AI
• Focus on efficiency with smaller model sizes
• Monthly innovation updates to keep pace with rapidly evolving field

02:19 - Understanding AI Reasoning

• Explanation of reasoning capabilities in AI models
• How allowing models to generate more text at inference time can improve performance
• Cost/benefit tradeoffs of reasoning features

03:13 - Enterprise AI Model Selection Criteria

• Moving beyond "one model to rule them all" thinking
• Importance of fit-for-purpose models
• Why smaller models can be customized more easily
• Trust and transparency considerations

05:38 - AI Governance and Safety

• How to evaluate models for governance requirements
• Safety evaluations and benchmarks as table stakes
• Systems-based approach to safety with guardrails
• IBM's Granite Guardian and protection mechanisms

08:55 - Benefits of Smaller Models

• Why size matters: cost, latency, and customization advantages
• Smaller models are easier to customize and require less computing power
• IBM's transparent approach to training data

10:13 - Future of AI Evaluation

• Performance per cost becoming the key evaluation metric
• The growing importance of flexibility in model selection
• How the "efficient frontier" between cost and performance will differentiate providers

12:41 - IBM's Vision Models

• IBM's pragmatic enterprise focus for multimodal capabilities
• Vision understanding (image in, text out) for practical business use cases
• Specialization for documents, charts, and dashboards
• Delivering powerful capabilities in only 2 billion parameters

15:25 - Understanding Model Size Context

• Evolution from millions to billions of parameters
• Practical considerations of deploying different-sized models
• Finding the right cost-benefit trade-off for specific use cases