Weaviate Podcast

Hey everyone! Thank you so much for watching the 117th episode of the Weaviate podcast! In this episode, we dive deep into the cutting edge of AI agent development with Sarah Wooders, co-founder and CTO of Letta AI. Emerging from Berkeley's Sky Computing Lab, Sarah and her team have pioneered a revolutionary approach to stateful agents - AI systems that genuinely remember both you and themselves across extended conversations. The conversation explores how the groundbreaking MemGPT project evolved into Letta's comprehensive Agent Development Environment (ADE), which empowers developers to build truly persistent AI experiences. Sarah shares powerful insights on context management, memory prioritization, and the critical role of databases in agent architecture. Whether you're building AI systems or simply curious about where conversational AI is heading, this episode illuminates how the future of agents depends not just on their reasoning capabilities, but on their ability to maintain coherent identity and memory over time.

Hey everyone! Thank you so much for watching another episode of the Weaviate Podcast! I am SUPER excited to welcome Matt Biilmann, Co-Founder and CEO of Netlify, as well as Sebastian Witalec and Charles Pierse from Weaviate to discuss Agent Experience! You have probably heard about how you can connect LLMs to external software tools. This supercharges the capabilities of AI systems and what they can do. So what does that mean for you as a software developer?This podcast explores different ideas around designing software user experiences for Agents as well as Humans. How do we write documentation for Agents differently than Humans? How do we design REST or gRPC APIs, or programming languages clients, for Agents differently than Humans? llms.txt, JSON tool definitions, agents to agent, breaking changes, … there were so many interesting topics explored in this podcast! I really hope you find it useful! As always more than happy to discuss these ideas further with you!

Hey everyone! Thank you so much for watching the 115th episode of the Weaviate Podcast featuring Shirley Wu from Stanford University!

We explore the innovative Avatar Optimizer—a novel framework that leverages contrastive reasoning to refine LLM agent prompts for optimal tool usage. Shirley explains how this self-improving system evolves through iterative feedback by contrasting positive and negative examples, enabling agents to handle complex tasks more effectively.

We also dive into the STaRK Benchmark, a comprehensive testbed designed to evaluate retrieval systems on semi-structured knowledge bases. The discussion highlights the challenges of unifying textual and relational retrieval, exploring concepts such as multi-vector embeddings, relational graphs, and dynamic data modeling. Learn how these approaches help overcome information loss, enhance precision, and enable scalable, context-aware retrieval in diverse domains—from product recommendations to precision medicine.

Whether you’re interested in advanced prompt optimization, multi-agent system design, or the future of human-centered language models, this episode offers a wealth of insights and a forward-looking perspective on integrating sophisticated AI techniques into real-world applications.

Hey everyone! Thank you so much for watching the 114th episode of the Weaviate Podcast featuring Amanpreet Singh, Co-Founder and CTO of Contextual AI! Contextual AI is at the forefront of production-grade RAG agents! I learned so much from this conversation! We began by discussing the vision of RAG 2.0, jointly optimizing generative and retrieval models! This then lead us to discuss Agentic RAG and how the RAG 2.0 roadmap is evolving with emerging perspectives on tool use. Amanpreet continues to further motivate the importance of continual learning of the model and the prompt / few-shot examples -- discussing the limits of prompt engineering. Personally I have to admit I think I have been a bit too bullish on only tuning instructions / examples, Amanpreet made an excellent case for updating the weights of the models as well -- citing issues such as parametric knowledge conflicts, and later on discussing how Mechanistic Interpretability is used to audit models and their updates in enterprise settings. We then discussed Contextual AI's LMUnit for evaluating these systems. This then lead us into my favorite part of the podcast, a deep dive into RL algorithms for LLMs. I highly recommend checking out the links below to learn more about Contextual's innovations on APO and KTO! We then discuss the importance of domain specific data, Mechanistic Interpretability, return to another question on RAG 2.0, and conclude with Amanpreet's most exciting future directions for AI! I hope you enjoy the podcast!

Hey everyone! Thank you so much for watching the 113th episode of the Weaviate Podcast with Karan Goel from Cartesia AI! Cartesia AI is leading the AI world in text-to-speech models! As exciting as these new applications in speech generation are, Cartesia is also building around an incredibly exciting new neural network architecture that cuts across all of AI -- State Space Models. State Space Models (SSMs) present a new approach to modeling long sequences circumventing the quadratic attention bottlenecks of transformers. In the podcast, we discuss Karan's perspectives around end-to-end modeling, long context and Multimodal processing, building and deploying a new kind of model, and more! I hope you find the podcast interesting and useful! As always more than happy to discuss these ideas further with you! Thanks for listening!

Hey everyone! Thank you so much for watching the 112th episode of the Weaviate Podcast! This is another super exciting one, diving into the release of the Vertex AI RAG Engine, its integration with Weaviate and thoughts on the future of connecting AI systems with knowledge sources! The podcast begins by reflecting on Bob's experience speaking at Google in 2016 on Knowledge Graphs! This transitions into discussing the evolution of knowledge representation perspectives and things like the semantic web, ontologies, search indexes, and data warehouses. This then leads to discussing how much knowledge is encoded in the prompts themselves and the resurrection of rule-based systems with LLMs! The podcast transitions back to topics around the modern consensus in RAG pipeline engineering. Lewis suggests that parsing in data ingestion is the biggest bottleneck and low hanging fruit to fix. Bob presents the re-indexing problem and how it is additionally complicated with embedding models! Discussing the state of knowledge representation systems inspired me to ask Bob further about his vision with Generative Feedback Loops and controlling databases with LLMs, How open ended will this be? We then discuss the role that Agentic Architectures and Compound AI Systems are having on the state of AI. What is the right way to connect prompts with other prompts, external tools, and agents? The podcast then concludes by discussing a really interesting emerging pattern in the deployment of RAG systems. Whereas the first generation of RAG systems typically were user facing, such as customer support chatbots, the next generation is more API-based. The launch of the Vertex AI RAG Engine quickly shows you how to use RAG Engine as a tool for a Gemini Agent!

Hey everyone! I am SUPER EXCITED to publish the 111th Weaviate Podcast with Aravind Kesiraju from Morningstar! Aravind is a Principal Software Engineer who has lead the development behind the Morningstar Intelligence Engine! There are so many interesting aspects to this, and if you are building Agentic systems that would benefit from a high-quality financial retrieval API, you should check this out right now! The podcast dives into all sorts of ingredients that went into building this system: from custom RAG data pipelines with content management system integrations and embedding task queues, to exploring new chunking strategies, tool marketplaces, ReAct Agents, Text-to-SQL, and all sorts of other things!

Hey everyone! Thank you so much for watching the 110th episode of the Weaviate Podcast! Today we are diving into Snowflake’s Arctic Embedding model series and their newly released Arctic Embed 2.0 open-source model, additionally supporting multilingual text embeddings. The podcast covers the origin of Arctic Embed, Pre-training embedding models, Matryoshka Representation Learning (MRL), Fine-tuning embedding models, Synthetic Query Generation, Hard Negative Mining, and Single-Vector Embeddings Models in the cohort of Multi-Vector ColBERT, SPLADE, and Re-rankers.

Hey everyone! Thank you so much for watching the 109th episode of the Weaviate Podcast with Erika Cardenas! Erika, in collaboration with Leonie Monigatti, have recently published "What is Agentic RAG". This blog post that was even covered in VentureBeat with additional quotes from Weaviate Co-Founder and CEO Bob van Luijt! This podcast continues the discussion on all things Agentic RAG, covering the basics of Agents, how Agentic RAG changes the game compared to Vanilla RAG systems, Multi-Agent Systems and CrewAI / OpenAI Swarm, Letta, DSPy, and many more! The podcast also anchors by discussing Agentic Generative Feedback Loops and how we are using Agents to improve the quality and expand the capabilities of Generative Feedback Loops!

JSON mode has been one of the biggest enablers for working with Large Language Models! JSON mode is even expanding into Multimodal Foundation models! But how exactly is JSON mode achieved?

There are generally 3 paths to JSON mode: (1) constrained generation (such as Outlines), (2) begging the model for a JSON response in the prompt, and (3) A two stage process of generate-then-format.

I am BEYOND EXCITED to publish the 108th Weaviate Podcast with Zhi Rui Tam, the lead author of Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models!

As the title of the paper suggests, although constrained generation is awesome because of its reliability, we may be sacrificing the performance of the LLM by producing our JSON with this method.

The podcast dives into how these experiments identify this and all sorts of details about the potential and implementation details of Structured Outputs. I particularly love the conversation topic of incredible Complex Structured Outputs, such as generating 10 values in a single inference.

I hope you enjoy the podcast! As always please reach out if you would like to discuss any of these ideas further!