No Priors: Artificial Intelligence | Technology | Startups

Mustafa Suleyman, co-founder of DeepMind and now co-founder and CEO of Inflection AI, joins Sarah and Elad to discuss how his interests in counseling, conflict resolution, and intelligence led him to start an AI lab that pioneered deep reinforcement learning, lead applied AI and policy efforts at Google, and more recently found Inflection and launch Pi.

Mustafa offers insights on the changing structure of the web, the pressure Google faces in the age of AI personalization, predictions for model architectures, how to measure emotional intelligence in AIs, and the thinking behind Pi: the AI companion that knows you, is aligned to your interests, and provides companionship.

Sarah and Elad also discuss Mustafa’s upcoming book, The Coming Wave (release September 12, 2023), which examines the political ramifications of AI and digital biology revolutions.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @mustafasuleymn

Show Notes:

[00:06] - From Conflict Resolution to AI Pioneering

[10:36] - Defining Intelligence

[15:32] - DeepMind's Journey and Breakthroughs

[24:45] - The Future of Personal AI Companionship

[33:22] - AI and the Future of Personalized Content

[41:49] - The Launch of Pi

[51:12] - Mustafa’s New Book The Coming Wave

How do you personalize AI models? A popular school of thought in AI is to just dump all the data you need into pre-training or fine tuning. But that may be less efficient and less controllable than alternatives — using AI models as a reasoning engine against external data sources.

Kelvin Guu, Senior Staff Research Scientist at Google, joins Sarah and Elad this week to talk about retrieval, memory, training data attribution and model orchestration. At Google, he led some of the first efforts to leverage pre-trained LMs and neural retrievers, with >30 launches across multiple products. He has done some of the earliest work on retrieval-augmented language models (REALM) and training LLMs to follow instructions (FLAN).

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Kelvin_Guu

Show Notes:

[1:44] - Kelvin’s background in math, statistics and natural language processing at Stanford

[3:24] - The questions driving the REALM Paper

[7:08] - Frameworks around retrieval augmentation & expert models

[10:16] - Why is modularity important

[11:36] - FLAN Paper and instruction following

[13:28] - Updating model weights in real time and other continuous learning methods

[15:08] - Simfluence Paper & explainability with large language models

[18:11] - ROME paper, “Model Surgery” exciting research areas

[19:51] - Personal opinions and thoughts on AI agents & research

[24:59] - How the human brain compares to AGI regarding memory and emotions

[28:08] - How models become more contextually available

[30:45] - Accessibility of models

[33:47] - Advice to future researchers

This week on No Priors, Sarah and Elad answer listener questions about tech and AI. Topics covered include the evolution of open-source models, Elon AI, regulating AI, areas of opportunity, and AI hype in the investing environment. Sarah and Elad also delve into the impact of AI on drug development and healthcare, and the balance between regulation and innovation.

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil

Show Notes: 

[0:00:06] - The March of Progress for Open Source Foundation Models 

[0:06:00] - Should AI Be Regulated?

[0:13:49] - Investing in AI and Exploring the AI Opportunity Landscape

[0:23:28] - The Impact of Regulation on Innovation

[0:31:55] - AI in Healthcare and Biotech

So much of the AI conversation today revolves around models and new applications. But this AI revolution would not be possible without one thing – GPUs, Nvidia GPUs.

The Nvidia A100 is the workhorse of today’s AI ecosystem. This week on No Priors, Sarah Guo and Elad Gil sit down with Jensen Huang, the founder and CEO of NVIDIA, at their Santa Clara headquarters. Jensen co-founded the company in 1993 with a goal to create chips that accelerated graphics. Over the past thirty years, NVIDIA has gone far behind gaming and become a $674B behemoth. Jensen talks about the meaning of this broader platform shift for developers, making very long term bets in areas such as climate and biopharma, their next-gen Hopper chip, why and how NVIDIA chooses problems that are unsolvable today, and the source of his iconic leather jackets.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Nvidia

Show Notes: 

[1:26] - The early days when Jensen Co-founded NVIDIA

[4:58] - Why NVIDIA started to expand its aperture to artificial intelligence use cases 

[10:42] - The moment in 2012 Jensen realized AI was going to be huge

[13:52] - How we’re in a broader platform shift in computer science

[17:48] - His vision for NVIDIA’s future lines of business

[18:09] - How NVIDIA has two motions: Shipping reliable chips and solving new use cases 

[25:41] - Why no one should assume they’re right for the job of CEO and why not every company needs to be architected as the US military 

[31:39] - What’s next for NVIDIA’s Hopper 

[32:57] - Durability of Transformers 

[35:08] - What Jensen is excited about in the future of AI & his advice for founders

Noam Shazeer played a key role in developing key foundations of modern AI - including co-inventing Transformers at Google, as well as pioneering AI chat pre-chatGPT. These are the foundations supporting today’s AI revolution. On this episode of No Priors, Noam discusses his work as an AI researcher, engineer, inventor, and now CEO. 

Noam Shazeer is currently the CEO and Co-founder of Character AI, a service that allows users to design and interact with their own personal bots that take on the personalities of well-known individuals or archetypes. You could have a socratic conversation with Socrates. You could pretend you’re being interviewed by Oprah. Or you could work through a life decision with a therapist bot. Character recently raised $150M from A16Z, Elad Gil, and others. Noam talks about his early AI adventures at Google, why he started Character, and what he sees on the horizon of AI development.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Character_ai

Show Notes: 

[1:50] - Noam’s early AI projects at Google

[7:13] - Noam’s focus on language models and AI applications

[11:13] - Character’s co-founder Daniel de Freitas Adiwardana work on Google’s Lambda

[13:53] - The origin story of Character.AI 

[18:47] - How AI can express emotions

[26:51] - What Noam looks for in new hires

If you have 30 dollars, a few hours, and one server, then you are ready to create a ChatGPT-like model that can do what’s known as instruction-following. Databricks’ latest launch, Dolly, foreshadows a potential move in the industry toward smaller and more accessible but extremely capable AIs. Plus, Dolly is open source, requires less computing power, and fewer data parameters than its counterparts.

Matei Zaharia, Cofounder & Chief Technologist at Databricks, joins Sarah and Elad to talk about how big data sets actually need to be, why manual annotation is becoming less necessary to train some models, and how he went from a Berkeley PhD student with a little project called Spark to the founder of a company that is now critical data infrastructure that’s increasingly moving into AI.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Databricks | @Matei_Zaharia

Show Notes: 

[01:29] - Origin of Databricks

[4:30] - Work at Stanford Lab

[5:29] - Dolly and Role of Open Source

[12:30] - Industry focus on high parameter count, understanding reasoning at small model scale

[18:42] - Enterprise applications for Dolly & chat bots

[25:06] - Making bets as an academic turned CTO

[36:23] - The early stages of AI and future predictions

Everyone talks about the future impact of AI, but there’s already an AI product that has revolutionized a profession. Alex Graveley was the principal engineer and Chief Architect behind Github Copilot, a sort of pair-programmer that auto-completes your code as you type. It has rapidly become a product that developers won’t live without, and the most leaned-upon analogy for every new AI startup – Copilot for Finance, Sales, Marketing, Support, Writing, Decision-Making.

Alex is a longtime hacker and tinkerer, open source contributor, repeat founder, and creator of products that millions of people use, such as Dropbox Paper. He has a new project in stealth, Minion AI. In this episode, we talk about the uncertain process of shipping Copilot, how code improves chain of thought for LLMs, how they improved product, performance, how people are using it, AI agents that can do work for us, stress testing society's resilience to waves of new technology, and his new startup named Minion.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @alexgraveley | @ai_minion

Show Notes:

[1:50] - How Alex got started in technology 

[2:28] - Alex’s earlier projects with Hack Pad and Dropbox Paper

[07:32] - Why Alex always wanted to make bots that did stuff for people

[11:56] - How Alex started working at Github and Copilot

[27:11] - What is Minion AI

[30:30] - What’s possible on the horizon of AI

With advances in machine learning, the way we search for information online will never be the same.

This week on the No Priors podcast, we dive into a startup that aims to be the most trustworthy place to search for information online. Perplexity.ai is a search engine that provides answers to questions in a conversational way and hints at what the future of search might look like.

Aravind Srinivas is a Co-founder and CEO of Perplexity. He is a former research scientist at Open AI and completed his PhD in computer science at University of California Berkeley.

Denis Yarats is a Co-Founder and Perplexity’s CTO. He has a background in machine learning, having worked as a Research Scientist at Facebook AI Research and a machine learning engineer at Quora.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @AravSrinivas | @denisyarats

Show Notes: 

[1:46] - How Perplexity AI iterates quickly and how the company has changed over time

[5:46] - Approach to hiring and building a fast-paced team

[10:43] - Why you don’t need AI pedigree to transition to work or research AI

[14:01] - Challenges when transitioning from AI research to running a company as CEO & CTO

[16:50] - Why Perplexity only shows answers it can cite

[19:33] - How Perplexity approaches reinforcement learning

[20:49] - Trustworthiness and if an answer engine needs a personality

[23:05] - Why answer engines will become their own market segment

[26:38] - Implications of “the era of fewer clicks” on publishers and advertisers

[30:20] - Monetization strategy

[33:20] - Advice for those deciding between academia or startups

For the first time in decades web search might be at risk for disruption. Bing is allied with OpenAI to integrate LLMs. Google has committed to launching new products. New startups are emerging.

Sridhar Ramaswamy co-founded the challenger AI-powered, private search platform Neeva in 2019. He is a former 16-year Google veteran who most recently led the internet’s most profitable business as SVP in charge of Google Ads, Commerce and Privacy. Sridhar, Elad and Sarah talk about the challenge of building search, how LLMs have changed the landscape, and how chatbots and "answer services" will affect web publishers.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @RamaswmySridhar

Show Notes: 

[1:32] - Why Sridhar started a private search engine after leaving Google

[11:11] - Information Retrieval Problems, Mapping Search Queries and LLMs

[15:25] - Google and Bing’s approach to search with LLMs

[19:06] - Scale challenges when building a search engine startup

[22:26] - Distribution challenges and why they release Neeva Gist

[24:11] - Why Neeva is a privacy centric subscription service 

[28:25] - The relationship between search and publishers/content creators

[30:16] - Sridhar’s predictions on how AI will disrupt current ecosystems

When AI research is evolving at warp speed and takes significant capital and compute power, what is the role of academia? Dr. Percy Liang – Stanford computer science professor and director of the Stanford Center for Research on Foundation Models talks about training costs, distributed infrastructure, model evaluation, alignment, and societal impact.

Sarah Guo and Elad Gil join Percy at his office to discuss the evolution of research in NLP, why AI developers should aim for superhuman levels of performance, the goals of the Center for Research on Foundation Models, and Together, a decentralized cloud for artificial intelligence.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Sign up for new podcasts every week. Email feedback to [email protected]

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @PercyLiang

Show Notes: 

[1:44] - How Percy got into machine learning research and started the Center for Research and Foundation Models at Stanford

[7:23] - The role of academia and academia’s competitive advantages

[13:30] - Research on natural language processing and computational semantics

[27:20] - Smaller scale architectures that are competitive with transformers

[35:08] - Helm, holistic evaluation of language models, a project with the the goal is to evaluate language models

[42:13] - Together, a decentralized cloud for artificial intelligence