Two Voice Devs

It's been a busy week! What have we been up to? Mark has released a new set of cards that summarize and illustrate different AI concepts. Called "AI Explorer Cards of Discovery", we chat about the objectives and the process to create this deck. (And there's a special offer for listeners!) Meanwhile, Allen has been working with Google's new PaLM model as part of Google Cloud's Vertex AI platform and has contributed changes to the popular LangChainJS package to make PaLM available through the open source library.

Resources:

* AI Explorer Cards of Discovery: https://bit.ly/ai-cards

* LangChainJS: https://github.com/hwchase17/langchainjs

* Google PaLM: https://cloud.google.com/ai/generative-ai

SO MUCH packed into this episode!

Recently, Allen participated in a hackathon sponsored by VoiceFlow, and he used the opportunity to explore ways that LLMs could be used to build on his work talking with spreadsheets in Vodo Drive (see episode 116). He and Mark explore how he did it - from the prompts that were required to integration with VoiceFlow and Google App Script, to how tools like LangChain will help build similar things. We also explore what lessons are learned, how our experience in #VoiceFirst design helps us build good #ConversationalAI tools, how other APIs can (and should!) work alongside AI, and what "fuzzy" roles AI can fill in the modern app experience.

Resources:

* Vodo Drive: https://vodo-drive.com/

* PromptHacks Hackathon: https://prompthacks.devpost.com/

* Vodo AI submission for PromptHacks: https://devpost.com/software/vodo-ai

* VoiceFlow: https://www.voiceflow.com/

* Google Apps Script: https://www.google.com/script/start/

* LangChain: https://github.com/hwchase17/langchain and https://github.com/hwchase17/langchainjs

It's Google I/O time again! And although Allen couldn't attend in person, he and Mark review the latest announcements relevant to #VoiceFirst and #ConversationalAI developers. From new AI availability to AI workspace, with stops along the way to discuss AI powered hardware, there was lots to hear about. Also some subtle hints from what wasn't said. But did we mention the AI?

Learn more:

* https://blog.google/technology/developers/google-io-2023-100-announcements/

We've touched on the use of vector databases as we've started to explore how LLMs and conversational AIs can be useful, but what are they and how do they work? How are they used for more than just LLMs? Mark and Allen explore some of the classic vector DBs, such as HNSW, and some of the newer fully managed ones, including Metal and Pinecone. We even start to ponder what a fully managed embedding and vector db system might look like from the likes of Google, Azure, or AWS, and are surprised that we're closer than we thought!

Resources:

* HNSWlib: https://github.com/nmslib/hnswlib

* Pinecone: https://pinecone.io/

* Metal: https://getmetal.io/

* Google Cloud Vertex AI Matching Engine: https://cloud.google.com/vertex-ai/docs/matching-engine/overview

* Amazon AWS Bedrock: https://aws.amazon.com/blogs/machine-learning/announcing-new-tools-for-building-with-generative-ai-on-aws/

Long teased, the ability for developers to create Alexa Widgets is finally generally available! Mark, an Alexa Champion, has had access for a while now, so he and Allen discuss what it takes to make a Widget, what's new and different, and how it fits into the #VoiceFirst world of skills.

We're still exploring what LangChain can do, and this week we dive into a tutorial put out by the Voiceflow team that discusses some ways that it can be integrated with ChatGPT using LangChain, bringing the #VoiceFirst and #ConversationalAI worlds closer together. Also a great example of how we go about learning and understanding code that is new to us.

Resources:

* The tutorial we were following: https://www.voiceflow.com/blog/voiceflow-assistant-openai-gpt

Over the past few weeks, Mark and Allen have been playing with LangChain and OpenAI, exploring where #ConversationalAI and #VoiceFirst design intersects, and we recorded some of our experiments. In this early one, we take a look at how LangChain with a memory chain can work and keep track of what's going on in the conversation. All in just a few lines of code. More significantly, we discuss the role that LangChain can play in putting together AI and other API components to create voice, web, and app-based agents that include AI as part of the NLU or response elements.

The latest update for the Alexa Presentation Language, APL 2023.1, has been out for a little bit, and Mark and Allen already discussed one of the biggest features - speech marks. But there was more to this release! Allen is, perhaps, even more excited about how selectors can enable some very dynamic APL interactions, and video subtitles and new graphic masks round out what's new.

More info:

* https://developer.amazon.com/en-US/docs/alexa/alexa-presentation-language/apl-latest-version.html - What's New With APL

Guest host Noble Ackerson returns to tie all the pieces of LangChain together and help Allen understand what it does, why it is an increasingly valuable toolkit for many #ConversationalAI developers, and how we can use it to help build #VoiceFirst applications.

Resources:

* LangChain: https://github.com/hwchase17/langchain

* LangChainJS: https://github.com/hwchase17/langchainjs

With the announcement last week that ChatGPT will soon be supporting plugins, Mark and Allen explore what this means for developers, particularly developers who are used to #VoiceFirst development with Amazon Alexa, Google Assistant, and Samsung Bixby. OpenAI is launching it with some interesting features, including hints at how to monetize plugins, but there are still many questions that developers will need answers to. We explore what will be coming, how to prepare for it, what OpenAI still needs to address, and how all of this may play into the future of voice assistants.

Resources:

* https://platform.openai.com/docs/plugins/ - ChatGPT Plugins documentation