AI Engineering Podcast

Summary
In this episode of the AI Engineering Podcast Julianna Lamb, co-founder and CTO of Stytch, talks about the complexities of managing identity and authentication in agentic workflows. She explores the evolving landscape of identity management in the context of machine learning and AI, highlighting the importance of flexible compute environments and seamless data exchange. The conversation covers implications of AI agents on identity management, including granular permissions, OAuth protocol, and adapting systems for agentic interactions. Julianna also discusses rate limiting, persistent identity, and evolving standards for managing identity in AI systems. She emphasizes the need to experiment with AI agents and prepare systems for integration to stay ahead in the rapidly advancing AI landscape.


Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
• Your host is Tobias Macey and today I'm interviewing Julianna Lamb about the complexities of managing identity and auth in agentic workflows

Interview

• Introduction
• How did you get involved in machine learning?
• The term "identity" is very overloaded. Can you start by giving your definition in the context of technical systems?
  • What are some of the different ways that AI agents intersect with identity?
• We have decades of experience and effort in building identity infrastructure for the internet, what are the most significant ways in which that is insufficient for agent-based use cases?
  • I have heard anecdotal references to the ways in which AI agents lead to a proliferation of "identities". How would you characterize the magnitude of the difference in scale between human-powered identity, deterministic automation (e.g. bots or bot-nets), and AI agents?
• The other major element of establishing and verifying "identity" is how that intersects with permissions or authorization. What are the major shortcomings of our existing investment in managing and auditing access and control once you are within a system?
  • How does that get amplified with AI agents?
• Typically authentication has been done at the perimeter of a system. How does that architecture change when accounting for AI agents?
  • How does that get complicated by where the agent originates? (e.g external agents interacting with a third-party system vs. internal agents operated by the service provider)
• What are the concrete steps that engineering teams should be taking today to start preparing their systems for agentic use-cases (internal or external)?
• How do agentic capabilities change the means of protecting against malicious bots? (e.g. bot detection, defensive agents, etc.)
• What are the most interesting, innovative, or unexpected ways that you have seen authn/authz/identity addressed for AI use cases?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on identity/auth(n|z) systems?
• What are your predictions for the future of identity as adoption and sophistication of AI systems progresses?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Stytch
• AI Agent
• Machine To Machine Authentication
• API Authentication
• MCP == Model Context Protocol
• OAuth
• Identity Provider
• OAuth Scopes
• OAuth 2.1
• Captcha
• RBAC == Role-Based Access Control
• ABAC == Attribute-Based Access Control
• ReBAC == Relationship-Based Access Control
• Google Zanzibar
• Idempotence
• Dynamic Client Registration
• Large Action Models
• Claude Code

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering Podcast, CEO of Resolve AI Spiros Xanthos shares his insights on building agentic capabilities for operational systems. He discusses the limitations of traditional observability tools and the need for AI agents that can reason through complex systems to provide actionable insights and solutions. The conversation highlights the architecture of Resolve AI, which integrates with existing tools to build a comprehensive understanding of production environments, and emphasizes the importance of context and memory in AI systems. Spiros also touches on the evolving role of AI in production systems, the potential for AI to augment human operators, and the need for continuous learning and adaptation to fully leverage these advancements.

Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Spiros Xanthos about architecting agentic capabilities for operational challenges with managing production systems.

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what Resolve AI is and the story behind it?
• We have decades of experience as an industry in managing operational complexity. What are the critical failures in capabilities that you are addressing with the application of AI?
  • Given the existing capabilities of dedicated platforms (e.g. Grafana, PagerDuty, Splunk, etc), what is your reasoning for building a new system vs. a new feature of existing operational product?
• Over the past couple of years the industry has developed a growing number of agent patterns. What was your approach in evaluating and selecting a particular approach for your product?
• One of the complications of building any platform that supports operational needs of engineering teams is the complexity of integrating with their technology stack. This is doubly true when building an AI system that needs rich context. What are the core primitives that you are relying on to build a robust offering?
• How are you managing the learning process for your systems to allow for iterative discovery and improvement?
  • What are your strategies for personalizing those discoveries to a given customer and operating environment?
• One of the interesting challenges in agentic systems is managing the user experience for human-in-the-loop and machine to human handoffs in each direction. How are you thinking about that, especially given the criticality of the systems that you are interacting with?
• As more of the code that is running in production environments is co-developed with AI, what impact do you anticipate on the overall operational resilience of the systems being monitored?
• One of the challenges of working with LLMs is the cold start problem where every conversation starts from scratch. How are you approaching the overall problem of context engineering and ensuring that you are consistently providing the necessary information for the model to be effective in its role?
• What are the most interesting, innovative, or unexpected ways that you have seen Resolve AI used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on Resolve AI?
• When is Resolve AI the wrong choice?
• What do you have planned for the future of Resolve AI?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Resolve AI
• Splunk
• OpenTelemetry
• Splunk Observability
• Context Engineering
• Grafana
• Kubernetes
• PagerDuty

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering Podcast Jeremiah Lowin, founder and CEO of Prefect Technologies, talks about the FastMCP framework and the design of MCP servers. Jeremiah explains the evolution of FastMCP, from its initial creation as a simpler alternative to the MCP SDK to its current role in facilitating the deployment of AI tools. The discussion covers the complexities of designing MCP servers, the importance of context engineering, and the potential pitfalls of overwhelming AI agents with too many tools. Jeremiah also highlights the importance of simplicity and incremental adoption in software design, and shares insights into the future of MCP and the broader AI ecosystem. The episode concludes with a look at the challenges of authentication and authorization in AI applications and the exciting potential of MCP as a protocol for the future of AI-driven business logic.


Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Jeremiah Lowin about the FastMCP framework and how to design and build your own MCP servers

Interview

• Introduction
• How did you get involved in machine learning?
• Can you start by describing what MCP is and its purpose in the ecosystem of AI applications?
• What is FastMCP and what motivated you to create it?
  • Recognizing that MCP is relatively young, how would you characterize the landscape of MCP frameworks?
• What are some of the stumbling blocks on the path to building a well engineered MCP server?
  • What are the potential ramifications of poorly designed and implemented MCP implementations?
• In the overall context of an AI-powered/agentic application, what are the tradeoffs of investing in the MCP protocol? (e.g. engineering effort, process isolation, tool creation, auth(n|z), etc.)
  • In your experience, what are the architectural patterns that you see of MCP implementation and usage?
• There are a multitude of MCP servers available for a variety of use cases. What are the key factors that someone should be using to evaluate their viability for a production use case?
• Can you give an overview of the key characteristics of FastMCP and why someone might select it as their implementation target for a custom MCP server?
  • How have the design, scope, and goals of the project evolved since you first started working on it?
• For someone who is using FastMCP as the framework for creating their own AI tools, what are some of the design considerations or best practices that they should be aware of?
  • What are some of the ways that someone might consider integrating FastMCP into their existing Python-powered web applications (e.g. FastAPI, Django, Flask, etc.)
• As you continue to invest your time and energy into FastMCP, what is your overall goal for the project?
• What are the most interesting, innovative, or unexpected ways that you have seen FastMCP used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on FastMCP?
• When is FastMCP the wrong choice?
• What do you have planned for the future of FastMCP?

Contact Info

• LinkedIn
• GitHub

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• FastMCP
• FastMCP Cloud
• Prefect
• Model Context Protocol (MCP)
• AI Tools
• FastAPI
• Python Decorator
• Websockets
• SSE == Server-Sent Events
• Streamable HTTP
• OAuth
• MCP Gateway
• MCP Sampling
• Flask
• Django
• ASGI
• MCP Elicitation
• AuthKit
• Dynamic Client Registration
• smolagents
• Large Active Models
• A2A

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering Podcast Dr. Tara Javidi, CTO of KavAI, talks about developing AI systems for proactive monitoring in heavy industry. Dr. Javidi shares her background in mathematics and information theory, influenced by Claude Shannon's work, and discusses her approach to curiosity-driven AI that mimics human curiosity to improve data collection and predictive analytics. She explains how KavAI's platform uses generative AI models to enhance industrial monitoring by addressing informational blind spots and reducing reliance on human oversight. The conversation covers the architecture of KavAI's systems, integrating AI with existing workflows, building trust with operators, and the societal impact of AI in preventing environmental catastrophes, ultimately highlighting the future potential of information-centric AI models.

Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems.
• Your host is Tobias Macey and today I'm interviewing Dr. Tara Javidi about building AI systems for proactive monitoring of physical environments for heavy industry

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what KavAI is and the story behind it?
• What are some of the current state-of-the-art applications of AI/ML for monitoring and accident prevention in industrial environments?
  • What are the shortcomings of those approaches?
• What are some examples of the types of harm that you are focused on preventing or mitigating with your platform?
• On your site it mentions that you have created a foundation model for physical awareness. What are some examples of the types of predictive/generative capabilities that your model provides?
• A perennial challenge when building any digital model of a physical system is the lack of absolute fidelity. What are the key sources of information acquisition that you rely on for your platform?
  • In addition to your foundation model, what are the other systems that you incorporate to perform analysis and catalyze action?
• Can you describe the overall system architecture of your platform?
  • What are some of the ways that you are able to integrate learnings across industries and environments to improve the overall capacity of your models?
• What are the most interesting, innovative, or unexpected ways that you have seen KavAI used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on KavAI?
• When is KavAI/Physical AI the wrong choice?
• What do you have planned for the future of KavAI?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Links

• KavAI
• Information Theory
• Claude Shannon

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering Podcast machine learning engineer Shashank Kapadia explores the transformative role of generative AI in retail. Shashank shares his journey from an engineering background to becoming a key player in ML, highlighting the excitement of understanding human behavior at scale through AI. He discusses the challenges and opportunities presented by generative AI in retail, where it complements traditional ML by enhancing explainability and personalization, predicting consumer needs, and driving autonomous shopping agents and emotional commerce. Shashank elaborates on the architectural and operational shifts required to integrate generative AI into existing systems, emphasizing orchestration, safety nets, and continuous learning loops, while also addressing the balance between building and buying AI solutions, considering factors like data privacy and customization.


Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Shashank Kapadia about applications of generative AI in retail

Interview

• Introduction
• How did you get involved in machine learning?
• Can you summarize the main applications of generative AI that you are seeing the most benefit from in retail/ecommerce?
• What are the major architectural patterns that you are deploying for generative AI workloads?
• Working at an organization like WalMart, you already had a substantial investment in ML/MLOps. What are the elements of that organizational capability that remain the same, and what are the catalyzed changes as a result of generative models?
• When working at the scale of Walmart, what are the different types of bottlenecks that you encounter which can be ignored at smaller orders of magnitude?
• Generative AI introduces new risks around brand reputation, accuracy, trustworthiness, etc. What are the architectural components that you find most effective in managing and monitoring the interactions that you provide to your customers?
• Can you describe the architecture of the technical systems that you have built to enable the organization to take advantage of generative models?
• What are the human elements that you rely on to ensure the safety of your AI products?
• What are the most interesting, innovative, or unexpected ways that you have seen generative AI break at scale?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on AI?
• When is generative AI the wrong choice?
• What are your paying special attention to over the next 6 - 36 months in AI?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Walmart Labs

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering podcast Adam Honig, founder of Spiro AI, about using AI to automate CRM systems, particularly in the manufacturing sector. Adam shares his journey from running a consulting company focused on Salesforce to founding Spiro, and discusses the challenges of traditional CRM systems where data entry is often neglected. He explains how Spiro addresses this issue by automating data collection from emails, phone calls, and other communications, providing a rich dataset for machine learning models to generate valuable insights. Adam highlights how Spiro's AI-driven CRM system is tailored to the manufacturing industry's unique needs, where sales are relationship-driven rather than funnel-based, and emphasizes the importance of understanding customer interactions and order histories to predict future business opportunities. The conversation also touches on the evolution of AI models, leveraging powerful third-party APIs, managing context windows, and platform dependencies, with Adam sharing insights into Spiro's future plans, including product recommendations and dynamic data modeling approaches.


Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Adam Honig about using AI to automate CRM maintenance

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what Spiro is and the story behind it?
• What are the specific challenges posed by the manufacturing industry with regards to sales and customer interactions?
  • How does the type of manufacturing and target customer influence the level of effort and communication involved in the sales and customer service cycles?
• Before we discuss the opportunities for automation, can you describe the typical interaction patterns and workflows involved in the care and feeding of CRM systems?
• Spiro has been around since 2014, long pre-dating the current era of generative models. What were your initial targets for improving efficiency and reducing toil for your customers with the aid of AI/ML?
  • How have the generational changes of deep learning and now generative AI changed the ways that you think about what is possible in your product?
• Generative models reduce the level of effort to get a proof of concept for language-oriented workflows. How are you pairing them with more narrow AI that you have built?
• Can you describe the overall architecture of your platform and how it has evolved in recent years?
• While generative models are powerful, they can also become expensive, and the costs are hard to predict. How are you thinking about vendor selection and platform risk in the application of those models?
• What are the opportunities that you see for the adoption of more autonomous applications of language models in your product? (e.g. agents)
  • What are the confidence building steps that you are focusing on as you investigate those opportunities?
• What are the most interesting, innovative, or unexpected ways that you have seen Spiro used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on AI in the CRM space?
• When is AI the wrong choice for CRM workflows?
• What do you have planned for the future of Spiro?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Spiro
• Deepgram
• Cognee Episode
• Agentic Memory
• GraphRAG
  • Podcast Episode
• OpenAI Assistant API

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering podcast Anush Elangovan, VP of AI software at AMD, discusses the strategic integration of software and hardware at AMD. He emphasizes the open-source nature of their software, fostering innovation and collaboration in the AI ecosystem, and highlights AMD's performance and capability advantages over competitors like NVIDIA. Anush addresses challenges and opportunities in AI development, including quantization, model efficiency, and future deployment across various platforms, while also stressing the importance of open standards and flexible solutions that support efficient CPU-GPU communication and diverse AI workloads.

Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Anush Elangovan about AMD's work to expand the playing field for AI training and inference

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what your work at AMD is focused on?
• A lot of the current attention on hardware for AI training and inference is focused on the raw GPU hardware. What is the role of the software stack in enabling and differentiating that underlying compute?
• CUDA has gained a significant amount of attention and adoption in the numeric computation space (AI, ML, scientific computing, etc.). What are the elements of platform risk associated with relying on CUDA as a developer or organization?
• The ROCm stack is the key element in AMD's AI and HPC strategy. What are the elements that comprise that ecosystem?
  • What are the incentives for anyone outside of AMD to contribute to the ROCm project?
• How would you characterize the current competitive landscape for AMD across the AI/ML lifecycle stages? (pre-training, post-training, inference, fine-tuning)
• For teams who are focused on inference compute for model serving, what do they need to know/care about in regards to AMD hardware and the ROCm stack?
• What are the most interesting, innovative, or unexpected ways that you have seen AMD/ROCm used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on AMD's AI software ecosystem?
• When is AMD/ROCm the wrong choice?
• What do you have planned for the future of ROCm?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• ImageNet
• AMD
• ROCm
• CUDA
• HuggingFace
• Llama 3
• Llama 4
• Qwen
• DeepSeek R1
• MI300X
• Nokia Symbian
• UALink Standard
• Quantization
• HIPIFY
• ROCm Triton
• AMD Strix Halo
• AMD Epyc
• Liquid Networks
• MAMBA Architecture
• Transformer Architecture
• NPU == Neural Processing Unit
• llama.cpp
• Ollama
• Perplexity Score
• NUMA == Non-Uniform Memory Access
• vLLM
• SGLang

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the Machine Learning Podcast Ori Silberberg, VP of Engineering at Buildots, talks about transforming the construction industry with AI. Ori shares how Buildots uses computer vision and AI to optimize construction projects by providing real-time feedback, reducing delays, and improving efficiency. Learn about the complexities of digitizing the construction industry, the technical architecture of Buildoz, and how its AI-driven solutions create a digital twin of construction sites. Ori emphasizes the importance of explainability and actionable insights in AI decision-making, highlighting the potential of generative AI to further enhance the construction process from planning to execution.


Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Ori Silberberg about applications of AI for optimizing building construction

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what Buildotds is and the story behind it?
• What types of construction projects are you focused on? (e.g. residential, commercial, industrial, etc.)
• What are the main types of inefficiencies that typically occur on those types of job sites?
  • What are the manual and technical processes that the industry has typically relied on to address those sources of waste and delay?
• In many ways the construction industry is as old as civilization. What are the main ways that the information age has transformed construction?
  • What are the elements of the construction industry that make it resistant to digital transformation?
• Can you describe how you are applying AI to this complex and messy problem?
• What are the types of data that you are able to collect?
  • How are you automating that data collection so that construction crews don't have to add extra work or distractions to their day?
• For construction crews that are using Buildots, can you talk through how it integrates into the overall process from site planning to project completion?
• Can you describe the technical architecture of the Buildots platform?
• Given the safety critical nature of construction, how does that influence the way that you think about the types of AI models that you use and where to apply them?
• What are the most interesting, innovative, or unexpected ways that you have seen Buildots used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on Buildots?
• What do you have planned for the future of AI usage at Buildots?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Buildots
• CAD == Computer Aided Design
• Computer Vision
• LIDAR
• GC == General Contractor
• Kubernetes

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering Podcast Jamie De Guerre, founding SVP of product at Together.ai, explores the role of open models in the AI economy. As a veteran of the AI industry, including his time leading product marketing for AI and machine learning at Apple, Jamie shares insights on the challenges and opportunities of operating open models at speed and scale. He delves into the importance of open source in AI, the evolution of the open model ecosystem, and how Together.ai's AI acceleration cloud is contributing to this movement with a focus on performance and efficiency.

Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Jamie de Guerre about the role of open models in the AI economy and how to operate them at speed and at scale

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what Together AI is and the story behind it?
  • What are the key goals of the company?
• The initial rounds of open models were largely driven by massive tech companies. How would you characterize the current state of the ecosystem that is driving the creation and evolution of open models?
• There was also a lot of argument about what "open source" and "open" means in the context of ML/AI models, and the different variations of licenses being attached to them (e.g. the Meta license for Llama models). What is the current state of the language used and understanding of the restrictions/freedoms afforded?
• What are the phases of organizational/technical evolution from initial use of open models through fine-tuning, to custom model development?
• Can you outline the technical challenges companies face when trying to train or run inference on large open models themselves?
  • What factors should a company consider when deciding whether to fine-tune an existing open model versus attempting to train a specialized one from scratch?
• While Transformers dominate the LLM landscape, there's ongoing research into alternative architectures. Are you seeing significant interest or adoption of non-Transformer architectures for specific use cases? 
  • When might those other architectures be a better choice?
• While open models offer tremendous advantages like transparency, control, and cost-effectiveness, are there scenarios where relying solely on them might be disadvantageous?
  • When might proprietary models or a hybrid approach still be the better choice for a specific problem?
• Building and scaling AI infrastructure is notoriously complex. What are the most significant technical or strategic challenges you've encountered at Together AI while enabling scalable access to open models for your users?
• What are the most interesting, innovative, or unexpected ways that you have seen open models/the TogetherAI platform used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on powering AI model training and inference?
• Where do you see the open model space heading in the next 1-2 years? Any specific trends or breakthroughs you anticipate?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Together AI
• Fine Tuning
• Post-Training
• Salesforce Research
• Mistral
• Agentforce
• Llama Models
• RLHF == Reinforcement Learning from Human Feedback
• RLVR == Reinforcement Learning from Verifiable Rewards
• Test Time Compute
• HuggingFace
• RAG == Retrieval Augmented Generation
  • Podcast Episode
• Google Gemma
• Llama 4 Maverick
• Prompt Engineering
• vLLM
• SGLang
• Hazy Research lab
• State Space Models
• Hyena Model
• Mamba Architecture
• Diffusion Model Architecture
• Stable Diffusion
• Black Forest Labs Flux Model
• Nvidia Blackwell
• PyTorch
• Rust
• Deepseek R1
• GGUF
• Pika Text To Video

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary
In this episode of the AI Engineering Podcast, host Tobias Macey sits down with Ben Wilde, Head of Innovation at Georgian, to explore the transformative impact of agentic AI on business operations and the SaaS industry. From his early days working with vintage AI systems to his current focus on product strategy and innovation in AI, Ben shares his expertise on what he calls the "continuum" of agentic AI - from simple function calls to complex autonomous systems. Join them as they discuss the challenges and opportunities of integrating agentic AI into business systems, including organizational alignment, technical competence, and the need for standardization. They also dive into emerging protocols and the evolving landscape of AI-driven products and services, including usage-based pricing models and advancements in AI infrastructure and reliability.

Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Ben Wilde about the impact of agentic AI on business operations and SaaS as we know it

Interview

• Introduction
• How did you get involved in machine learning?
• Can you start by sharing your definition of what constitutes "agentic AI"?
• There have been several generations of automation for business and product use cases. In your estimation, what are the substantive differences between agentic AI and e.g. RPA (Robotic Process Automation)?
  • How do the inherent risks and operational overhead impact the calculus of whether and where to apply agentic capabilities?
• For teams that are aiming for agentic capabilities, what are the stepping stones along that path?
• Beyond the technical capacity, there are numerous elements of organizational alignment that are required to make full use of the capabilities of agentic processes. What are some of the strategic investments that are necessary to get the whole business pointed in the same direction for adopting and benefitting from AI agents?
• The most recent splash in the space of agentic AI is the introduction of the Model Context Protocol, and various responses to it. What do you see as the near and medium term impact of this effort on the ecosystem of AI agents and their architecture?
• Software products have gone through several major evolutions since the days of CD-ROMs in the 90s. The current era has largely been oriented around the model of subscription-based software delivered via browser or mobile-based UIs over the internet. How does the pending age of AI agents upend that model?
• What are the most interesting, innovative, or unexpected ways that you have seen agentic AI used for business and product capabilities?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working with businesses adopting agentic AI capabilities?
• When is agentic AI the wrong choice?
• What are the ongoing developments in agentic capabilities that you are monitoring?

Contact Info

• Email
• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• Georgian
• Agentic Platforms And Applications
• Differential Privacy
• Agentic AI
• Language Model
• Reasoning Model
• Robotic Process Automation
• OFAC
• OpenAI Deep Research
• Model Context Protocol
• Georgian AI Adoption Survey
• Google Agent to Agent Protocol
• GraphQL
• TPU == Tensor Processing Unit
• Chris Lattner
• CUDA
• NeuroSymbolic AI
• Prolog

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0