Two Voice Devs

This episode of Two Voice Devs, recorded before the exciting announcement of OpenAI's GPT-4o Realtime and Audio previews, tackles a classic developer challenge: taming unruly text-to-speech (TTS) engines. Triggered by a listener question, Allen and Mark dive into the frustrating inconsistencies of TTS pronunciation, particularly when dealing with dynamically generated text from LLMs. They explore the limitations of SSML, experiment with phoneme alphabets like X-SAMPA, and even ponder the possibility of multimodal LLMs generating perfect audio natively – a concept now realized with models like GPT-4o Realtime and Audio! While Mark and Allen don't discuss these new models directly, their insights on pronunciation control, leveraging existing tools, and integrating LLMs with TTS remain incredibly relevant. Join us for a conversation that foreshadows the future of AI-powered voice development and offers practical strategies for achieving flawless pronunciation, even in the pre-realtime audio era. These techniques and discussions offer valuable context and potential solutions even as new, more advanced models emerge.

Timestamps:

[00:00:00] Introduction and Listener Question: The challenge of inconsistent TTS pronunciation.

[00:02:01] The Problem in Action: Hear how Google TTS mispronounces a seemingly straightforward phrase.

[00:02:52] Exploring SSML Solutions: The pros and cons of using SSML tags for pronunciation control.

[00:04:15] The Generative Text Challenge: How to handle correct pronunciation when text is dynamically generated.

[00:07:58] The Phoneme Alphabet Approach: Using X-SAMPA to specify pronunciation directly.

[00:09:06] A Live Experiment: Allen demonstrates his phoneme-based solution using AI Studio and Gemini.

[00:10:51] Testing Edge Cases: Exploring the limitations of the phoneme approach with past tense verbs.

[00:12:19] The Multimodal LLM Dream (Now a Reality?): Allen and Mark discuss the potential of LLMs generating perfect audio.

[00:13:20] Alternative Approaches: Mark suggests using parts-of-speech tagging for enhanced context.

[00:15:16] The Future of TTS (Then and Now): Discussing the evolution of text-to-speech technology and its integration with LLMs, including reflections relevant to the latest preview models like GPT-4o Realtime and Audio.

[00:17:22] Community Call to Action: Share your solutions and insights on handling tricky TTS pronunciations! How do the latest LLM advancements impact your approach?

Our thanks to bonadio (https://github.com/bonadio) for their question.

#GenerativeAI #GenAI #TextToSpeech #TTS #MultimodalLLM #Multimodal #BuildWithGemini #OpenAI #GPT4o #GPT4oRealtime #GPT4oAudio #VoiceFirst

Join us as we dive deep into OpenAI's latest model, O1, with special guest host Michal Stanislavik, founder of utter.one and one of the voice community builder behind VoiceLunch. We explore the model's "reasoning" capabilities, its potential impact on conversational AI, and how developers can leverage its strengths. Michal shares his insights from hands-on experience, highlighting both the exciting possibilities and the current limitations of O1. Is it ready for prime-time in conversational applications? What are the most promising use cases? And how does it compare to the GPT family? We discuss all this and more, including the future of agentic systems and the role of open-source models like LLaMa.

Timestamps:

[00:00:00] Introduction - Meet Michal Stanislavik and learn about his background in voice technology and community building.

[00:04:51] ChatGPT's Impact - How ChatGPT changed the conversational AI landscape and raised user expectations.

[00:08:32] Introducing OpenAI's O1 - What is O1 and how does it differ from GPT models?

[00:11:38] Reasoning and Latency - A closer look at O1's reasoning process and the implications of increased latency.

[00:17:30] Beyond Conversational Applications - Exploring potential use cases for O1 outside of real-time chatbots.

[00:23:38] Pre-Conversation Processing - Using O1 for data ingestion, analysis, and preparing follow-up questions.

[00:25:41] Code Generation Capabilities - O1's potential as a powerful tool for developers, including code generation and UI development.

[00:32:21] O1 and Agentic Systems - How O1 could become the foundation for future generations of intelligent agents.

[00:35:07] Tool Use and the Future of Agents - Integrating tools into the reasoning process and the importance of voice-to-voice models.

[00:43:39] The Competitive Landscape - How will other players like Anthropic, Google, and Meta respond to O1? The importance of open-source alternatives.

[00:46:11] Conclusion and Connect with Michal - Final thoughts and how to find Michal online.

#GenerativeAI #GenAI #Strawberry #ConversationalAI

Thumbnail by Imagen 3 with prompt:

Podcast thumbnail, illustrative with a touch of tech, cartoonish. Scene: Two podcast hosts, sitting in front of microphones, smiling and engaging in conversation. Both hosts are male, caucasian, software developers in their early 50s, with closely cropped hair The host on the left is wearing an olive green tshirt and has a\tclosely cropped\tblack and white\tbeard. The host on the right is wearing a light blue polo shirt, glasses, and is clean shaven. Background: a dark blue, futuristic landscape with glowing lines and nodes representing data flow. Overlaid on this background is a large, semi-transparent, stylized \"O1\" logo, similar to OpenAI's branding. Small icons representing code snippets, a chat bubble, strawberries, and a robot's head float around the O1. The overall mood is conversational and tech-focused, but not overly serious. Emphasis should be placed on the connection between the hosts and the futuristic, AI-driven theme.

Join Mark and Allen on this episode of Two Voice Devs as they dive into the often overlooked but crucial topic of mentorship in software development. They explore what mentorship is (and isn't), the benefits for both mentor and mentee, and share personal anecdotes and practical advice. Whether you're a seasoned developer or just starting out, this episode offers valuable insights into fostering a culture of learning and growth within development teams.

Timestamps:

0:00:00 - Introduction

0:00:45 - Defining Mentorship: It's more than just giving advice

0:04:15 - Mentorship vs. Education: What's the difference?

0:07:35 - Real-World Mentoring Examples: From junior devs to senior colleagues

0:09:55 - "Nobody cares how many people you had to ask for help..."

0:13:05 - A Cautionary Tale: When mentorship goes wrong

0:18:35 - Creating a Safe Space: Encouraging questions and celebrating wins

0:21:40 - The Mentor's Responsibility: Be prepared and available

0:24:55 - Vulnerability is Key: It's okay to say "I don't know"

0:26:55 - Advice for Mentees: Be open to help and willing to learn

0:28:50 - It's "Our Code": Shifting the mindset from individual ownership

0:31:10 - Mentorship as a Craft: Passing down knowledge to the next generation

0:33:40 - Call to Action: Share your mentorship experiences and questions!

Thumbnail generated by Imagen 3 with prompt:Podcast thumbnail, illustrative with a touch of tech, cartoonish.Scene: Two podcast hosts, sitting in front of microphones,smiling and engaging in conversation.Both hosts are male, caucasian, software developers in their early 50s,wearing glasses, and are clean shaven.The host on the left is wearing a dark red t-shirt and a brown flat cap.The host on the right is wearing a light blue polo shirt.Warm, inviting lighting.Background: Elements subtly hinting at a collaborative workspacewith many people involved.One figure gestures towards the people in the background suggesting mentorshipwhile the other listens intently, holding a water mug with a straw.Negative prompt: beards

Dive into the world of agentic AI development with Allen and Mark as they explore LangGraph, a powerful state management system for building dynamic and complex AI agents with LangChain. Discover how LangGraph simplifies agent design, handles state transitions, integrates tools, and enables robust error handling – all while keeping the LLM at the heart of your application.

Further Info:

* https://github.com/langchain-ai/langgraphjs

* https://github.com/langchain-ai/langgraphjs-studio-starter/

* https://vodo-drive.com/

Timeline:

0:00 - Introduction

1:05 - What is LangGraph and how does it work?

2:38 - Understanding Nodes and Tools in LangGraph

4:54 - Navigating Conditional Edges and State Transitions

7:00 - Real-world example: LangGraph in action

12:56 - State Management in LangGraph: Maintaining Context

14:24 - Comparing LangGraph to Jovo and Voiceflow for Conversational Design

17:11 - Handling both freeform and directed conversations with LangGraph

20:40 - Vodo Drive: A deeper dive into a real-world LangGraph implementation

26:31 - The LLM's role: Tool Selection, Execution, and Output Generation

29:51 - Managing Latency and Optimizing LLM Calls

32:08 - Working with JSON responses: Tools vs Human-Readable Output

36:04 - The bigger picture: LangGraph's role in building robust AI agents

Don't forget to subscribe and leave your comments below, especially if you're using LangGraph in your own projects!

#GenAI #GenerativeAI #LangChain #LangGraph #AIAgents #Agents

Thumbnail by Imagen 3 with prompt:

Podcast thumbnail, illustrative with a touch of tech, cartoonish.

Scene: Two podcast hosts, sitting in front of microphones,

smiling and engaging in conversation.

Both hosts are male, caucasian, software developers in their early 50s, wearing glasses, and are clean shaven.

The host on the left is wearing a blue t-shirt and a brown flat cap.

The host on the right is wearing a light blue polo shirt.

Background: A flowchart consisting of boxes and diamonds, connected with glowing strands of light.

Negative prompt: beards

Join us as we explore Vodo Drive, an innovative project that leverages Google's Gemini AI to revolutionize how we interact with spreadsheets. Creator Allen Firstenberg takes us behind the scenes, revealing the architecture, challenges, and breakthroughs of building an agentic system that understands and manipulates data like never before.

Discover how Vodo Drive:

* Empowers natural language interaction: Say goodbye to rigid formulas and hello to conversational commands.

* Integrates image recognition: Effortlessly input data by simply taking pictures.

* Provides real-time feedback: Experience transparent processing with live updates on your requests.

* Prioritizes security and user control: Maintain data privacy and manage permissions seamlessly.

More Info:

* Vodo Drive: https://vodo-drive.com/

* Gemini API on Vertex AI: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models

* LangChain: https://www.langchain.com/langchain

* LangGraph: https://www.langchain.com/langgraph

* Google Sheets: https://workspace.google.com/products/sheets/

* Firebase: https://firebase.google.com/

Timestamps:

* (0:00:00) Introduction and Project Overview: Discover the inspiration and goals behind Vodo Drive's participation in the Gemini API competition.

* (0:03:30) Reimagining Spreadsheet Control: Explore the evolution of Vodo Drive from voice-controlled spreadsheets to an AI-powered agentic system.

* (0:07:45) The Power of Visual Input: Learn how Vodo Drive seamlessly integrates image recognition to extract and input data from pictures.

* (0:11:55) Contextual Awareness and Conversational Flow: Delve into the importance of contextual awareness and how Vodo Drive maintains the flow of information.

* (0:14:30) Optimizing Tasks with the Right Tools: Understand the strategic use

of spreadsheets as the computational backbone for Vodo Drive's data processing.

* (0:15:30) System Design and Architecture Breakdown: Get a detailed look at the core components of Vodo Drive, including Firebase Cloud Functions, Firestore, and Authentication.

* (0:22:55) Addressing Security Concerns: Explore the safety measures implemented to protect user data and prevent unauthorized actions.

* (0:26:35) Real-Time Updates and User Experience: Discover how Vodo Drive leverages Firestore to provide real-time feedback and enhance user experience.

* (0:32:30) Behind the Scenes: The AI's Internal Dialogue: Uncover the hidden conversations happening between the agent and the LLM during data processing.

* (0:38:05) Firebase Authentication and Authorization: Learn how Vodo Drive ensures secure access to user spreadsheets and leverages Google's authorization system.

* (0:40:45) Firebase Cloud Storage and Media Handling: Explore the role of cloud storage in managing user-uploaded photos and audio files.

* (0:43:35) Gemini's Role in Image Processing and Agentic Logic: Discover how Gemini powers both image recognition and the decision-making process of the agentic system.

Don't miss this insightful discussion on the future of AI-powered data management and how Vodo Drive is paving the way for a more intuitive and efficient user experience.

#GeminiAPI #LLM #AgenticSystems #VoiceControl #Spreadsheets #Firebase #WebDevelopment #AndroidDevelopment #AI #Innovation

In this episode of Two Voice Devs, Allen and Mark discuss the considerations and strategies for shutting down an Alexa skill. They explore various reasons why developers might choose to sunset their skills, including declining usage, deprecated features, and the evolving Alexa landscape. They also delve into the technical aspects of skill removal, highlighting the options of hiding or removing a skill and the implications of each choice, especially when dealing with in-skill purchases and subscriptions. Mark shares his own experiences with managing his six Alexa skills, offering insights into his decision-making process for each one.

Timestamps:

* 0:00:00 Introduction

* 0:00:12 Why shut down an Alexa skill?

* 0:01:11 Decreased promotions, uncertainty about Alexa's future, AWS costs

* 0:04:35 Alexa-hosted skills and AWS free tier

* 0:06:02 Considerations beyond cost: analytics, deprecated features, impact on users

* 0:06:56 Challenges with in-skill purchases and subscriptions

* 0:08:01 Skill hiding vs. skill removal

* 0:10:26 Case Study: "Serve More" skill and lack of organizational adoption

* 0:13:35 Case Study: "Snatch Word" game and handling subscriptions during sunset

* 0:17:46 Managing user communication during skill shutdown

* 0:17:58 Case Study: "Picture Guesser" skill and content challenges

* 0:19:46 Case Studies: "Show Widgets" and "Busy Timers" – skills with ongoing personal use

* 0:22:29 Building skills for personal use first

* 0:23:32 Importance of planning a sunset process for brand image and user experience

* 0:24:27 Call to action for audience sharing

Join the conversation and share your own experiences with shutting down Alexa skills!

Join Allen and Linda as they dive into Google's Imagen 3 and Imagen 3 Fast, a powerful new set of image generation models. We explore its capabilities, pricing, features, and limitations, including a deep dive into the API and how to use it with Python code.

This episode features an in-depth look at Imagen 3's photorealism and comparison with its predecessor, Imagen 2. We examine the ethical implications of AI image generation, discussing copyright issues, plagiarism concerns, and the impact on artists.

Don't miss the stunning visuals and thought-provoking discussion!

Resources:

* https://console.cloud.google.com/vertex-ai/generative/vision

* https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview

* https://fc-art.medium.com/seurat-pointillism-e240074a03dc

* https://commons.wikimedia.org/wiki/File:La_Libert%C3%A9_guidant_le_peuple_-_Eug%C3%A8ne_Delacroix_-_Mus%C3%A9e_du_Louvre_Peintures_RF_129_-_apr%C3%A8s_restauration_2024.jpg

Timestamps:

* 00:00:31 : Introducing Imagen 3 & its Photorealistic Power

* 00:02:59 : Imagen 3 vs. Imagen 3 Fast: Speed, Quality, & Pricing

* 00:05:33 : Copyright & Commercial Use of Imagen-Generated Images

* 00:06:14 : Exploring the Imagen API with Python Code Examples

* 00:09:08 : Using Gemini to Generate Prompts for Imagen

* 00:11:15 : The Importance of Seed Control for Image Consistency

* 00:13:24 : Watermarking & Identifying AI-Generated Images

* 00:14:51 : Navigating Imagen's Safety Filters & Limitations

* 00:18:13 : Live Demo: Generating a Cat Image with Imagen 3

* 00:18:55 : Future Potential: Editing & Outcropping Capabilities

* 00:22:26 : Upscaling Images with Imagen: Costs & Possibilities

* 00:23:18 : Comparing Image Styles Across Imagen Versions (with Visuals!)

* 00:28:58 : Confronting the Ethical Concerns of AI Image Generation

* 00:29:05 : Real-World Examples: Inappropriate Content & Plagiarism

* 00:36:30 : The Impact of AI on Artists & the Definition of Art

* 00:37:06 : Transparency & Responsibility: Crediting AI in Creative Work

* 00:39:53 : Final Thoughts: Will We Continue Using Imagen 3?

Thumbnail created with Imagen 3 using the prompt:

Create a compelling thumbnail for a YouTube video podcast about AI image generation, specifically Google's Imagen 3, featuring two hosts. Include two distinct, friendly faces – one male, one female – representing the podcast hosts. The male host should be wearing glasses and a light blue collared shirt. The female host should be wearing glasses, her hair tied back in a pony tail, and wearing a grey t-shirt that says "MakerSuite". They should be facing the viewer with engaging expressions, perhaps a mix of excitement and contemplation. Showcase a visually striking AI-generated image emerging from a laptop screen or a thought bubble. The overall image should be done in the style of George Seurat.

(Episode number and title added manually.)

#VertexAI #Imagen3 #GenAI

Join Allen Firstenberg and Mark Tucker on Two Voice Devs as they discuss the challenges and solutions of hosting large language models (LLMs). They explore various hosting environments, including Firebase, AWS Amplify, Vertex AI, and Docker/Kubernetes, comparing their strengths and weaknesses.

Allen shares his experience with Firebase Cloud Functions and the seamless integration with Google Cloud services, while Mark tackles the complexities of Docker, Kubernetes, and enterprise-level deployment strategies. From managing API keys and credentials to implementing design patterns and best practices, they explore the challenges and solutions for building robust and scalable AI systems.

This episode is packed with practical tips for developers, covering topics like:

[00:02:00] Firebase Suite of Tools: Learn how Firebase provides a comprehensive platform for hosting LLMs, including real-time databases, cloud storage, cloud functions, and authentication.

[00:04:00] Firebase vs. AWS Amplify: Discover the key differences between these two popular serverless platforms and their database options.

[00:05:00] Cloud Service Accounts for Security: Allen demonstrates how leveraging cloud service accounts can simplify permission management and enhance security.

[00:11:00] Architecture Design and Long-Term Hosting: Allen emphasizes the impor

tance of considering future scalability and maintenance when selecting a hosting environment.

[00:12:30] Working with Docker and Kubernetes: Mark dives into his experience using Docker containers and Kubernetes for enterprise-level LLM deployment.

[00:15:00] Learning Python for LLM Development: Mark shares his experience learning Python for working with LLMs and using libraries like FastAPI for REST API development.

[00:17:00] Design Patterns and Best Practices: Allen and Mark discuss the evolving nature of design patterns and their importance in modern software development.

[00:20:00] KitOps for Model Deployment: Mark explains how KitOps can be used to separate model deployment from service deployment in a Kubernetes environment.

[00:23:00] Docker and Configuration Management: Allen discusses the challenge of configuration management in Docker environments and how to manage changes efficiently.

[00:24:00] Enterprise Security and Tooling: Mark explores the use of tools like HashiCorp Console and Vault for managing configurations and secrets in enterprise deployments.

[00:26:00] The Importance of Containerization: Allen and Mark reiterate the fundamental role of containers in modern software development.

Don't miss this insightful episode of Two Voice Devs, where you'll gain valuable insights and practical tips for hosting and deploying your own LLMs!

#AI #Development #Hosting #Cloud #Docker #Kubernetes #Firebase #GoogleCloud #DesignPatterns #TwoVoiceDevs

Join Allen and Mark in this episode of Two Voice Devs as they dive into the world of MLOps and explore KitOps, an open-source tool for packaging and versioning machine learning models and related artifacts. Learn how KitOps leverages the Open Container Initiative (OCI) standard to simplify model sharing and deployment.

More info:

Key Topics and Timestamps:

Don't miss this insightful episode of Two Voice Devs as Allen and Mark demystify KitOps and its potential for streamlining your MLOps practices. Subscribe to our channel for more developer-focused discussions and tutorials!

#MLOps #KitOps #MachineLearning #DevOps #AI #SoftwareDevelopment #Podcast #TwoVoiceDevs

Mark Tucker and Allen Firstenberg celebrate 200 episodes and four years of Two Voice Devs! In this special episode, they reflect on the journey so far, the evolution of the AI landscape, and what excites them most about the future of development.

Join them as they discuss:

00:00 Four years ago...

00:10 The evolution of large language models (LLMs) and how the landscape has shifted over the past year.

03:10 The emergence of new players in the AI model space and how Google, Microsoft, and Amazon are vying for dominance.

05:30 The growing trend of smaller and locally deployable models and the future of AI development.

08:00 The ongoing quest for seamless integration of conversational AI with web experiences.

10:30 The need for a convergence of traditional NLU concepts with modern AI approaches.

11:30 The pressing need for sustainability and responsible development in the AI space.

14:00 The importance of integrating AI tools with existing methods and workflows

.

16:00 An open invitation for developers to join Mark and Allen as co-hosts and share their perspectives on AI development.

18:00 A reminder that learning is at the heart of the developer experience and the importance of community.

20:00 The highlights from their favorite episodes over the past four years.

23:00 The value of connection and friendship within the developer community.

26:09 Four years ago...

Don't miss this milestone episode as Two Voice Devs look back and look forward!