Weaviate Podcast

Rohit Agarwal on Portkey - Weaviate Podcast #61!


Listen Later

Hey everyone! Thank you so much for watching the 61st episode of the Weaviate Podcast! I am beyond excited to publish this one! I first met Rohit at the Cal Hacks event hosted by UC Berkeley where we had a debate about the impact of Semantic Caching! Rohit taught me a ton about the topic and I think it's going to be one of the most impactful early applications of Generative Feedback Loops! Rohit is building Portkey, a SUPER interesting LLM middleware that does things like load balancing between LLM APIs, and as discussed in the podcast there are all sorts of opportunities for this kind of space whether it be routing to tool-specific LLMs, different cost / accuracy requirements, or multiple models in the HuggingGPT sense. It was amazing chatting with Rohit, this was the best dive into LLMOps I have personally been apart of! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast!

Check out portkey here! https://portkey.ai/blog
Chapters
0:00 Introduction
0:24 Portkey, Founding Vision
2:20 LLMOps vs. MLOps
4:00 Inference Hosting Options
7:05 3 Layers of LLM Use
8:35 LLM Load Balancers
12:45 Fine-Tuning LLMs
17:08 Retrieval-Aware Tuning
21:16 Portkey Cost Savings
23:08 HuggingGPT
26:28 Semantic Caching
32:40 Frequently Asked Questions
34:00 Embeddings vs. Generative Tasks
35:30 AI Moats, GPT Wrappers
39:56 Unlocks from Cheaper LLM Inference

...more
View all episodesView all episodes
Download on the App Store

Weaviate PodcastBy Weaviate

  • 4
  • 4
  • 4
  • 4
  • 4

4

4 ratings


More shows like Weaviate Podcast

View all
This Week in Startups by Jason Calacanis

This Week in Startups

1,270 Listeners

Freakonomics Radio by Freakonomics Radio + Stitcher

Freakonomics Radio

31,896 Listeners

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

507 Listeners

Hidden Brain by Hidden Brain, Shankar Vedantam

Hidden Brain

43,363 Listeners

Lage der Nation - der Politik-Podcast aus Berlin by Philip Banse & Ulf Buermeyer

Lage der Nation - der Politik-Podcast aus Berlin

244 Listeners

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

440 Listeners

The Daily by The New York Times

The Daily

111,077 Listeners

Y Combinator Startup Podcast by Y Combinator

Y Combinator Startup Podcast

207 Listeners

Practical AI by Practical AI LLC

Practical AI

188 Listeners

All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

All-In with Chamath, Jason, Sacks & Friedberg

8,756 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

129 Listeners

Unsupervised Learning by by Redpoint Ventures

Unsupervised Learning

39 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

72 Listeners

Interconnects by Nathan Lambert

Interconnects

10 Listeners

AI + a16z by a16z

AI + a16z

33 Listeners