The MAD Podcast: Machine Learning, AI & Data with Matt Turck

Today, we’re joined by Gustavo Sapoznik, Founder and CEO of ASAPP, the generative AI platform transforming contact centers. Matt + Gustavo discuss the magnitude of challenges to overcome in this market, how their AI tech is designed to help humans, the reason smart people should choose working at a startup over Big Tech, and more. 

This session was recorded live at a recent Data Driven NYC, our in-person, monthly event series. If you are ever in New York, you can find us on Eventbrite by searching for "FirstMark Capital". Events run monthly and are free and open to everyone. And as always, if you enjoy the MAD podcast, please subscribe and feel free to leave us a comment or rating.

Data Driven NYC YouTube Channel

FirstMark Capital Eventbrite

asapp.com

twitter.com/asapp

twitter.com/mattturck

linktr.ee/mattturck

Show Notes:

[00:00:45] Introducing Gustavo Sapoznik, Founder & CEO of ASAPP, a unicorn AI startup based in New York;

[00:01:00] How ASAPP started with a mission to “end bad customer service” after a frustrating phone call Mr. Sapoznik had with his cable provider;

[00:02:44] ASAPP’s product philosophy and how the customer service is a three-legged stool with companies, customers, and agents;

[00:05:11] How ASAPP automates what they can and augments the rest to make agents more productive;

[00:07:12] The evolution of ASAPP’s offerings including how ASAPP technology makes agents more productive;

[00:9:16] How ASAPP’s technology reduces response times and improves quality for agents by including transcription, auto complete, and real-time scoring of interactions for quality assurance;

[00:13:49] How ASAPP has evolved since 2014; their research-first approach, building in-house AI capabilities, training their own models, and their recent exploration of using open-source checkpoints;

[00:15:05] How Mr. Sapoznik hired the guy who ran all NLP research at Google;

[00:16:04] How cost, latency, and accuracy in their AI models differentiate ASAPP from common AI APIs available today;

[00:18:49] Agent models v. Language models and how ASAPP AI is modularized for large teams with established tech stacks;

[00:20:09] Mr. Sapoznik shares insights on selling to large enterprises and why he believes building a sales machine is equally, if not more important, than the product itself;

[00:23:08] How to recruit and retain top AI talent;

[00:27:42] Lessons learned from working with notable board members, including the three key dimensions of support from a good board: being a sounding board, providing tactical advice and connections, and instilling a sense of accountability and motivation;

Today, we’re excited to chat with Scott Belsky - author, entrepreneur, investor and Chief Strategy Officer at Adobe. Matt + Scott discuss the impact of AI on creative work, how Adobe is incorporating AI across their products, and what the future creative tools landscape might look like.

This session was recorded live at a recent Data Driven NYC, our in-person, monthly event series. If you are ever in New York, you can find us on Eventbrite by searching for "FirstMark Capital". Events run monthly and are free and open to everyone. And as always, if you enjoy the MAD podcast, please subscribe and leave us a comment.

Data Driven NYC YouTube Channel

FirstMark Capital Eventbrite

twitter.com/scottbelsky

Implications, by Scott Belsky

twitter.com/mattturck

linktr.ee/mattturck

Show Notes:

[00:53] How Adobe uses AI to enhance user experience, streamline onboarding and automate tasks across their product suite;

[01:30] How AI impacts Adobe's business, making creative processes accessible with features like the context bar in Photoshop;

[02:13] Firefly's journey: internal decisions, training challenges, and a commitment to using licensed material for ethical AI;

[03:58] Moral considerations in Firefly's development: the decision to use licensed material, commercial viability, and addressing user comparisons;

[05:52] Adobe's homegrown approach to generative AI models: in-house development and partnerships for specific capabilities like LLM;

[06:08] Adobe Sensei's 10-year evolution: developing AI technologies, the non-profit Content Authenticity Initiative, and content credentials establishing asset provenance;

[09:17] Adobe's new AI advancements: Firefly Image Model 2, Generative Match, and the vector model for illustration;

[11:16] Firefly Editor's revolutionary image editing: dynamically generating pixels, real-time object manipulation, and Adobe's commitment to pushing technological boundaries;

[12:41] Rapid integration of AI features: Firefly models and playground, surfacing on a website for user testing, and collaboration within Adobe's design organization;

[14:32] How Adobe's AI and data teams are structured and leveraging in-house development for competitive advantage;

[15:47] Future of work and creativity: AI's impact on raising the bar for digital experiences, accelerating creative processes, and the evolving landscape of personalized social content;

[19:11] Leveraging technology to reduce friction, streamline processes, and unlock creative flow;

[20:09] Impact of AI on business models: questioning time-based pricing, anticipating a shift to value-based models, and reconsidering compensation for creative professionals;

[21:10] Parallels with historical Internet Service Providers, the rapid evolution of ideas, and reflections on sustainable business models;

[24:53] Scott’s criteria for evaluating AI investments: valuing skeptical entrepreneurs, acknowledging temporary uniqueness, and emphasizing empathy with customers;

[26:40] Navigating challenges in 2023: Tough decisions for entrepreneurs, evaluating conviction, and the importance of sticking together through the "messy middle”;

Today, we’re joined by Howard Katzenberg, CEO of Glean AI, a machine learning powered accounts payable platform. Matt + Howard discuss Glean’s founding story, how Glean helps CFOs make insight driven choices, and more. 

This session was recorded live at a recent Data Driven NYC, our in-person, monthly event series. If you are ever in New York, you can find us on Eventbrite by searching for "FirstMark Capital". Events run monthly and are free and open to everyone. And as always, if you enjoy the MAD podcast, please subscribe and leave us a comment.

Data Driven NYC YouTube Channel

FirstMark Capital Eventbrite

twitter.com/mattturck

linktr.ee/mattturck

Shownotes:

[00:00:35] Howard's background;

[00:01:15] Challenges with manual FP&A;

[00:02:54] Approval Process gap realization and opportunity for Glean AI;

[00:04:40] How Glean AI is like “bill.com with a brain”;

[00:05:06] Enhanced functionalities beyond basic AP automation;

[00:06:32] Glean AI’s Inception and AI Models;

[00:07:54] Why Glean AI is unique;

[00:08:25] The evolution of Glean AI’s ML stack;

[00:10:44] Defensibility and how Glean AI offers vendor pricing insights to its network;

[00:12:23] Success stories and customer value;

[00:14:47] Future plans for Glean AI;

[00:16:39] Navigating industry and technical expertise;

[00:18:41] Audience Q&A

Today, we have the pleasure of chatting with Raza Habib, CEO of Humanloop, the platform for LLM collaboration and evaluation. Matt and Raza cover how to understand and optimize model performance, lessons learned about model evaluation and feedback, and explore the future of model fine-tuning.

twitter.com/RazRazcle

humanloop.com

Data Driven NYC YouTube Channel

twitter.com/mattturck

linktr.ee/mattturck

Shownotes:

[00:00:47] How Humanloop helps product and engineering teams build reliable applications on top of large language models by providing tools to find, manage, and version prompts;

[00:03:05] Where Humanloop fits into the MAD landscape as LM / LLM Ops;

[00:02:40] The challenges of evaluating and monitoring LLM;

[00:03:40] Why evaluating LLMs and generative AI is subjective given its stochastic attributes;

[00:04:40] Why evaluation is important during development and production stages of LLMs to make informed design decisions, and how that challenge evolves In production to monitoring system behavior;

[00:05:40] The need for regression testing with LLMs;

[00:06:10] How Humanloop makes it easy for users to capture feedback including Implicit signals of user satisfaction, such as post-interaction actions and edits to generated content;

[00:07:40] Why and how Humanloop uses guardrails in the app to ensure effective LLM use and implementation;

[00:08:38] Why using an LLM as part of the evaluation process can introduce additional uncertainty and noise; with turtles all the way down;

[00:09:40] How evaluators on Humanloop are restricted to binary yes-or-no style questions or numerical scores to maintain reliability with LLMs in production.

[00:10:40] Why a new set of tools were needed to monitor and observe LLM performance;

[00:11:40] How Humanloop’s interactive environment allows users to find and fix bugs in a prompt, including logs to support issue identification, and then run what-if style analysis by changing the prompt or information retrieval system — allowing for quick interventions and turnaround times within minutes to hours instead of days/weeks;

[00:12:40] Why having evaluation and observability closely connected to prompt engineering tools is critical for speed;

[00:13:40] How prompt engineering is like writing software specifications for the model, enabling domain experts to have a more direct impact on product development, and democratizing access and reducing reliance on engineers to implement the desired features;

[00:15:40] The key differences between popular LLMs on the market today;

[00:18:40] How the quality of open-source models has been rapidly improving, and how LLMs use tools or function calling to access APIs to go beyond simple text-based interactions;

[00:21:22] How Humanloop empowers non-technical experts;

[00:22:40] Where Humanloop fits within the AI ecosystem as an collaborative tool for enterprises building language models where collaboration and robust evaluation are crucial;

[00:25:40] How Humanloop customers are often problem-aware, and how the go-to-market motion is mainly inbound, but sales-led

[00:27:48] How Humanloop serves as a central place for storing prompts and sharing learnings across teams;

[00:28:24] Raza’s thoughts on Open Source v. Closed Source models in the AI community;

[00:30:40] The potential consequences of restricting access to models and Raza’s case for regulating end use cases and punishing malicious use rather than banning the technology altogether;

[00:33:40] Next steps for Humanloop;

Today we're joined by Akilesh Bapu, CEO and Founder of DeepScribe, the platform using AI and Natural Language Processing to doctor/ patient transcripts. Matt and Akilesh go into DeepScribe's clinical use cases, supervised vs. unsupervised learning, and how critical it still is to have a human in the loop in a medical setting.

Today we have the pleasure of chatting with Sharon Zhou, CEO of Lamini, an LLM platform for the enterprise. Matt and Sharon go over the battle between prompting and fine-tuning, how the Lamini platform enables fine-tuning to be done "one billion times faster", and their recently-announced "LLM Super-station" in partnership with AMD. 

Today we're joined by Aravind Srinivas, CEO of Perplexity AI, a chatbot-style AI conversational engine that directly answers users' questions with sources and citations. Matt & Aravind discuss Perplexity's founding story, the platform itself, and more. 

This session was recorded live at a recent Data Driven NYC, our in-person, monthly event series. If you are ever in New York, you can find us on Eventbrite by searching for "FirstMark Capital". Events run monthly and are free and open to everyone. And as always, if you enjoy the MAD podcast, please subscribe and feel free to leave us a comment or rating. 

Today we're joined by Mathew Lodge, CEO of Diffblue, an AI platform that uses reinforcement learning to autonomously test software. We chat about the "AI for code" landscape, the Diffblue platform, and why prompt engineering is not a thing. 

Today we're joined by Nancy Xu, AI Investor and CEO and Founder of Moonhub AI, the AI recruiting platform helping companies shorten and speed up the recruiting process while also helping employers reach a more diverse pool of candidates. We dive into how the Moonhub platform operates, Nancy's thoughts on opportunities for AI startups, her journey as an investor, and interesting projects she has her eye on.

Today we're joined by Stanislas Polu, Co-Founder of Dust, a startup building Secure AI assistants for the enterprise. We dive into Stanislas's journey to founding Dust including his experience at Open AI, the path to generative AI adoption in the enterprise, and the rise of the French AI ecosystem.