Today we have Philip Kiely from Baseten on the show. Baseten is a Series B startup focused on providing infrastructure for AI workloads. We go deep on Inference Optimization. We cover choosing a model, discuss the hype around Compound AI, choosing an Inference Engine, Optimization Techniques like Quantization and Speculative Decoding all the way down to your GPU choice.

Today we have Philip Kiely from Baseten on the show. Baseten is a Series B startup focused on providing infrastructure for AI workloads. We go deep on Inference Optimization. We cover choosing a model, discuss the hype around Compound AI, choosing an Inference Engine, Optimization Techniques like Quantization and Speculative Decoding all the way down to your GPU choice.

Deep Dive into Inference Optimization for LLMs with Philip Kiely

Join Alex DeBrie and Sean Falconer in insightful and in-depth interviews with tech experts, covering software development, entrepreneurship, and technology trends.
Alex is the author of The DynamoDB Book and a DynamoDB expert as well as AWS Data Hero. Sean Falconer has over 20 years of experience working in research and technology as an engineer, founder, and marketing executive. Sean is a Snowflake Data Superhero.
For more on Software Huddle, visit softwarehuddle.com or contact team@softwarehuddle.com.

Technology

Join Alex DeBrie and Sean Falconer in insightful and in-depth interviews with tech experts, covering software development, entrepreneurship, and technology trends. Alex is the author of The DynamoDB Book and a DynamoDB expert as well as AWS Data Hero. Sean Falconer has over 20 years of experience working in research and technology as an engineer, founder, and marketing executive. Sean is a Snowflake Data Superhero. For more on Software Huddle, visit softwarehuddle.com or contact team@softwarehuddle.com.

Share Deep Dive into Inference Optimization for LLMs with Philip Kiely

Sign up to save your podcasts

Deep Dive into Inference Optimization for LLMs with Philip Kiely

Deep Dive into Inference Optimization for LLMs with Philip Kiely

More shows like Software Huddle

Software Engineering Radio

The Changelog: Software Development, Open Source

Software Engineering Daily

Soft Skills Engineering

Founders

Syntax - Tasty Web Development Treats

Practical AI

My First Million

All-In with Chamath, Jason, Sacks & Friedberg

Dwarkesh Podcast

Oxide and Friends

Latent Space: The AI Engineer Podcast

The AI Daily Brief: Artificial Intelligence News and Analysis

BG2Pod with Brad Gerstner and Bill Gurley

The Pragmatic Engineer