"The Cognitive Revolution"

In this episode, Trey Kollmer, WGA Writer and Co-Executive of the show Ghosts, returns to the show to discuss updates to the Hollywood Strikes, including news on SAG-AFTRA and WGA. Trey and Nathan chat why actors are joining the strikes, how AI will change acting as a profession, Trey’s views on reasoning and how he’s experimenting with GPT-4, and much, much more.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:04:01) Hollywood and AI: updates on SAG-AFTRA and the WGA

(00:15:20) Sponsors: NetSuite | Omneky

(00:18:56) Studio approach to copyright and compensation with generative AI

(00:22:17) Hollywood receptiveness to using AI and protections guild members are asking for

(00:24:56) Models implementing gradient descent in the weights

(00:29:14) How Nathan uses models to write for the podcast

(00:34:15) Generating and mining jokes in the writer’s room

(00:40:18) Untraining models

(00:44:20) AI writing tools and writer perception of them

(00:46:34) Context length and pooling layers

(00:51:11) Microsoft China

(01:00:02) Actors’ strike

(01:05:45) Using 1 million DALL-E images to create an AI short film

(01:09:11) Deepfakes

(01:12:51) The future where anyone can be a reasonable synthetic actor

(01:17:33) Trey’s take on reasoning and why Hollywood should be more open to AI

(01:25:49) A monk’s experience with Chat-GPT

(01:28:56) Outdatedness of stochastic parrot notion; reasoning and synthesis

(01:36:14) Model vs human susceptibility to adversarial attacks because of human robustness

(01:47:30) Performance jumping with abstraction

(01:49:22) Language model self-delegation

(01:51:04) NVIDIA’s margins compared to TSMC and ASML

(02:07:35) Adding an AI layer and competing with incumbents

(02:13:10) Rewind AI

(02:19:27) Big tech vs old school studios

(02:14:52) Dramatic ironies from the picketing lines

(02:21:38) AI development moments that feel like a movie

(02:26:34) CEO of Inflection’s views on misuse being a greater threat than the AI itself

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: GoogleLM

In this episode, Nathan sits down with three members of the a16z x Convex AI Town project: Yoko Li (Partner, a16z), Martin Casado (GP, a16z), and James Cowling (CTO, Convex). AI Town is a virtual town where AI agents live, interact, and socialize. They discuss how AI Town originated from Yoko’s companion app project, unpredictability as a feature in LLMs and interacting with models like they are lifeforms, and why they chose Javascript and Convex to build AI Town. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

📣 CALL FOR FEEDBACK:

To borrow from a meme… we're in the podcast arena trying stuff. Some will work. Some won't. But we're always learning.

http://bit.ly/TCRFeedback

Fill out the above form to let us know how we can continue delivering great content to you or sending the feedback on your mind to [email protected]. 

TIMESTAMPS:

(00:00:00) - Episode Preview: Intro to AI Town and the idea of AI as companions

(00:05:29) - Overview of AI Town and the simulation framework based on the Stanford Generative Agents paper

(00:08:24) - Yoko explains how the idea for AI Town originated from the companion app project

(00:12:31) - The simplicity and elegance of the AI Town codebase

(00:15:47) - Sponsors: Netsuite | Omneky

(00:18:25) - How Convex built a server-side game engine for AI Town 

(00:23:39 )- James emphasizes the power of functional programming paradigms like Convex for building AI apps

(00:25:02) - Using simple JavaScript so anyone could understand and extend AI Town

(00:30:23) - How AI coding assistants were used in building AI Town 

(00:31:22) - No open source code for the Stanford paper when they started

(00:33:25) - The interplay between programmer and AI model

(00:38:01)- Martin draws a distinction between using formal languages vs. natural language

(00:43:21) The balance between formal language and unpredictable behaviours in LLMs

(00:48:27) Are we living in a simulation?

(00:50:38) Advice for other developers in AI

(00:54:29) AI Town is a community project to be extended on

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: GoogleLM

In this episode, Nathan sits down with Daniel Kang, Assistant Professor of Computer Science at the University of Illinois. Kang has done pioneering work bringing zero knowledge cryptographic proofs to AI. In this episode, they chat about the cryptographic theory behind Daniel's work, how cryptography allows us to balance the tradeoff between privacy and authenticity, and how cryptography usage is needed in a world where LLMs are increasingly embedded into our daily lives. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

CALL FOR FEEDBACK:

To borrow from a meme… we're in the podcast arena trying stuff. Some will work. Some won't. But we're always learning.

http://bit.ly/TCRFeedback

Fill out the above form to let us know how we can continue delivering great content to you or sending the feedback on your mind to [email protected]. 

TIMESTAMPS:

(00:00) Episode Preview

(00:01:04) Nathan's Introduction

(00:07:06) Motivation for bringing zero-knowledge proofs to AI

(00:10:27) Verifying model execution without revealing model details

(00:12:42) Verifying medical AI services haven't been tampered with

(00:15:09) Sponsors: Netsuite | Omneky

(00:18:54) Cryptographic hashes for commitments

(00:22:42) Assumptions underlying cryptographic hashes 

(00:25:20) Adding entropy through salting 

(00:26:24) Z case snarks and the proving process 

(00:31:00) Using lookup tables for nonlinearities

(00:33:35) Floating point vs fixed point calculations 

(00:34:08) Quantizing models for efficiency

(00:35:55) Using polynomials to represent arbitrary computations

(00:37:26) What are finite fields? 

(00:41:23) Toxic waste for cryptographic secrecy

(00:45:51) Computational costs

(00:47:39) The experience of using a cryptography application to verify model output  

(00:49:05) Verification key doesn't reveal model weights

(00:56:36) What using crypto infrastructure in AI enables and challenges to its implementation 

(01:01:26) Potential for 10-100x cost reductions

(01:04:51) Authenticating images with attested cameras

(01:11:56) How cryptography in AI could impact daily life

(01:14:25) On-device credential verification

(01:15:50) Potential for regulation of hardware authentication

(01:18:52) Upcoming work to reduce proof costs

LINKS:

Daniel's website

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: GoogleLM

In this episode, Nathan and Erik sit down to analyze Hugging Face in light of its recent $235M Series D round. They analyze Hugging Face’s community and defensibility through the lens of other community businesses like ProductHunt and Yelp, assess its ability to fulfill its $4.5 billion valuation, and assess competitors and other notable companies in the space like Replit, Character, and Runway. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:00:57) Episode Preview

(00:01:52) Nathan’s Introduction

(00:04:57) Overview of Hugging Face and recent fundraising announcement

(00:08:13) Hugging Face’s product line

(00:16:47) Sponsors: Netsuite | Omneky

(00:18:36) Community driven businesses and HuggingFace’s moat

(00:23:23) Discovery and inference

(00:28:04) Hugging Face’s ideological nature

(00:31:31) Curation is key

(00:33:46) Keeping content fresh when AI moves so fast

(00:35:44) If Hugging Face is a $50 billion company one day, what would it look like?

(00:48:08) Hugging Face vs Replit

(00:57:19) Contrasting Hugging Face with other companies like Character and Runway

LINKS MENTIONED:

Hugging Face's LLM leaderboard

X/TWITTER:

@labenz (Nathan)

@eriktorenberg

@CogRev_Podcast

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: GoogleLM

In this episode, Nathan sits down with Paige Bailey, Lead Product Manager of Generative Models at Google Deepmind. In this conversation, they discuss what it's like to be a PM for an LLM as opposed to an app, defining ideal LLM behaviour, and reasoning - how do you distinguish real abilities vs pattern matching? If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:00) Episode Preview

(00:01:15) Introducing Paige Bailey

(00:04:21) Paige’s background at Google Brain and the Deepmind merger

(00:07:00) PM for a LLM vs being a PM for an app

(00:11:21) The development timeline and compute budget of PaLM-2

(00:14:30) Paige’s role in the PaLM 2 project

(00:15:30) Sponsors: Netsuite | Omneky

(00:17:26) Defining desired capabilities for PaLM-2

(00:19:17) The amount of work that went into elevating PaLM 2 from PaLM 1

(00:20:28) Has Google lost its ability to ship?

(00:24:240) Paige's "eureka" moment seeing GitHub Copilot capabilities

(00:27:47) Competing PaLM 2 with other models

(00:32:20) Grokking and the predictability of emergent capabilities

(00:37:30) Citizen scientists and the multilingual capabilities of PaLM 2

(00:39:29) Distinguishing real reasoning vs pattern matching

(00:45:51) Products using PaLM-2 that people should try

(00:50:35) Most exciting AI projects that you can try out

(00:52:29) Curriculum learning and successor to the transformer

LINKS:

PaLM 2

Duet AI for developers

Avenging Polayni’s Revenge

X/TWITTER:

@DynamicWebPaige (Paige)

@labenz (Nathan)

@eriktorenberg

@CogRev_Podcast

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: GoogleLM

Join Nathan Labenz and Erik Torenberg as they analyze the latest developments from OpenAI on GPT 3.5, compare GPT to other live player models like Llama2, and discuss the state of AI in coding, education, and healthcare. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

TIMESTAMPS:

(00:00) Episode Preview

(00:01:00) GPT 3.5 Turbo

(00:06:36) Llama 2 vs GPT

(00:11:24) How much inference is needed to double compute costs?

(00:13:40) OpenAI’s moat

(00:14:41) OpenAI’s privacy consideration for data

(00:16:06) Sponsor: Netsuite | Omneky

(00:17:46) Encouraging the usage of instructions during fine-tuning

(00:19:19) Live player consideration of AI safety

(00:22:35) Da Vinci: new completions fine tuneable model

(00:24:59) Chat-GPT usage in decline

(00:30:03) Getting on demand tutoring on ML papers

(00:31:00) Code, education, and healthcare

(00:31:42) AI applications in coding

(00:38:12) AI revolution n education

(00:42:35) AI revolution in healthcare

(00:52:17) Call for feedback

LINKS:

Replit episode with Tyler Angert

Replit episode with VP of AI, Michele Catasta

AI Revolution in Education with Khan Academy's Director of Engineering, Shawn Jansepar

Google's Multimodal Med-PaLM with Vivek Natarajan and Tao Tu

X:

@labenz (Nathan)

@eriktorenberg (Erik)

@cogrev_podcast

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Vivek Natarajan and Tao Tu of Google’s Med-PaLM, diving into how they used one of the world’s largest medical datasets ever compiled to develop Med-PaLM M, an AI agent specialized in medical tasks. In this episode, they discuss: Med-PaLM M's “clinically superhuman” abilities and limitations, the rigorous testing and validation that went into the model, and their vision for AI to take over repetitive clerical tasks and allow doctors to focus on patients.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:00) Episode Preview

(00:00:56) Introducing Vivek Natarajan and Tao Tu

(00:04:18) The story of Google’s Medical AI research progress

(00:07:11) Multi-modal Med-PaLM

(00:10:32) Genomic data - how do you represent it?

(00:11:13) Google’s Deep Variant

(00:14:44) The successes and failures behind the incredible pace of progress

(00:15:02) Sponsors: Netsuite | Omneky

(00:21:54) Google’s research culture and assembling an interdisciplinary team

(00:31:36) Google’s Pathways

(00:33:40) Med-PaLM M's architecture

(00:37:28) Working with 3 different model sizes and what you learn

(00:46:56) Data and compute required for Med-PaLM M

(00:49:38) Med-PaLM M's cycle time

(00:54:56) Is a bridge or adapter structure worth implementing?

(01:00:09) Can we create an AI doctor?

(01:02:39) Emergent capabilities like identifying tuberculosis

(01:09:37) Reactions to these emergent capabilities

(01:11:13) Moving towards clinical trials and real-world testing

(01:13:01) Regulatory and safety considerations

(01:15:03) AI safety in the healthcare domain

(01:17:00) Potential to transform healthcare access worldwide

LINKS:

Med-PaLM: https://sites.research.google/med-palm/

Med-PaLM M paper: https://arxiv.org/abs/2307.14334

Our earlier conversation with Vivek Natarajan on Med-PaLM: https://www.youtube.com/watch?v=nPBd7i5tnEE

X/TWITTER:

@vivnat (Vivek)

@taotu831 (Tao)

@labenz (Nathan)

@eriktorenberg

@CogRev_Podcast

SPONSORS: NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Jon Noronha, co-founder of Gamma. Gamma is a new medium for presenting ideas, allowing you to focus on your ideas and receive beautiful, engaging content without the formatting work. In this episode, Jon and Nathan discuss the journey of building Gamma, how to coax AI to do things well, and the opportunity for AI A/B testing. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:00:00) - Preview

(00:01:23) - Nathan introduces Jon Noronha

(00:06:36) - Intro - Jon Noronha introduces himself and Gamma

(00:09:36) - Gamma's origin story: building the "anti-PowerPoint" pre-GPT-3

(00:13:15) - From using AI for onboarding to generating near-complete presentations.

(00:15:11) - Sponsors: Netsuite | Omneky

(00:18:01) - If Notion and Canva had a baby

(00:23:35) - Searching for the right structure for AI, landing on HTML

(00:28:16) - Degrees of freedom - developing a constrained vocabulary of semantic blocks

(00:29:30) - Choice of model(s) – weighing cost, speed, and quality tradeoffs

(00:35:18) - Editing UI - getting AI to edit slides per user command

(00:38:04) - UI for gathering training data - following Midjourney's example

(00:40:00) AI A/B Testing: a big opportunity space

(00:43:22) - Image generation models - Baseten and Gamma's deployment of it

(00:45:37) - Pacing feature development - Trying to stay ahead of innovations like 3D, video, voice etc.

(00:47:45) Image generation for slidedecks and AI involvement

(00:50:29) Gamma's main building priorities and AI generated websites

(00:52:34) - Competing with giants like Microsoft and Google

(00:55:04) - Enterprise AI shipping challenges - Startups pioneer new patterns; giants will eventually follow.

(00:57:45) - Simplifying complexity - will models smooth over the complexity of bloated product suites?

(00:59:22) - Conclusion and final thoughts

X/TWITTER:

@thatsjonsense (Jon)

@labenz (Nathan)

@eriktorenberg

@CogRev_Podcast

LINKS:

Gamma: https://gamma.app/

SPONSORS: Shopify |NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Shawn Jansepar, Director of Engineering at Khan Academy, to discuss their GPT-4 powered Socratic tutor, Khanmigo. In this conversation, Shawn and Nathan chat about Khan Academy’s collaboration with OpenAI and how they helped fine-tune GPT-4, how Khan Academy leveraged GPT-4 to build Khanmigo, and the impact of providing access to an AI tutor to any student. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:00) Episode Preview: Education 10 years from now

(04:42) Khan Academy’s early access partnership with OpenAI

(06:31) Khanmigo: journey from Chrome extension to AI tutor

(11:36) GPT-4’s ability to be Socratic vs 3.5

(15:05) Sponsors: Netsuite | Omneky

(16:40) Integrating Khan Academy’s Pedagogy into AI

(20:06) The future of education 10 years from now 

(22:37) Khan Academy’s models

(27:20) Demo-driven development process

(31:16) Sculpting the behaviour of a Socratic tutor model 

(35:59) Khan Academy’s contribution to GPT-4’s fine-tuning and RLHF

(38:41) Being data-informed vs data-driven as a practice 

(42:10) Incurring tech debt to get ahead of the curve

(45:28) The boundary for what an AI can/can’t t tutor 

(49:30) Identifying when the user is confused and avoiding AI hallucinations

(53:54) Khanmigo’s development patterns 

(59:11) Making Khanmigo jailbreak resistant 

(01:01:50) Delivering personalized education with AI 

(01:04:08) How Shawn and his team are thinking about AI education

(01:05:33) Khanmigo’s future multimodal interactivity

(01:08:42) Evaluating Khanmigo’s efficacy for student learning 

(01:11:41) How widely is Khanmigo deployed today and what is the future for universal public access?

(01:05:11) Distribution through teachers and districts

(01:16:30) What are the reactions from teachers and education institutions to AI?

(01:18:15) Khanmigo’s pricing model

(01:19:03) The future roadmap for Khanmigo

(01:20:45) How will the AI tutor change the world at large?

LINKS:

Khanmigo: khan.co/khanmigo23

Benjamin Bloom’s 2-Sigma Problem: https://web.mit.edu/5.95/www/readings/bloom-two-sigma.pdf

X/TWITTER:

@shawnjan8 (Shawn)

@labenz (Nathan)

@eriktorenberg

@CogRev_Podcast

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Join Nathan Labenz and Erik Torenberg as they analyze the last month in AI advancements. Nathan takes us through the meaningful updates to his Scouting Report (released last month, linked below), discusses highlights from recent episodes of The Cognitive Revolution, and gives us a sneak peek at upcoming interviews with Google researchers. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(01:00) How does the AI Scouting Report hold up a few weeks later?

(03:29) Zvi’s feedback on Nathan’s Tale of the Cognitive Tape

(10:25) The universal LLM jailbreak and adversarial examples

(12:53) Human performance is much more variable than AI performance

(14:45) Sponsors: NetSuite | Omneky

(16:09) Nathan’s AI Task Automation: What are good targets for tasks that can be automated for average businesses? 

(20:05) Is GPT-4 getting worse or better?

(22:00) Getting explicit about what good looks like

(24:00) Prompting best practices are very accessible

(26:35) Ghostwriting - and the art of the hook

(28:05) Live Players: Which companies have say so over how the future goes?

(31:10) Upcoming guests from Google AI

(35:40) Possible post-transformer architectures

LINKS:

SCOUTING REPORT Part 1: https://youtu.be/0hvtiVQ_LqQ

SCOUTING REPORT Part 2: https://youtu.be/ovm4MbQ4G9E

SCOUTING REPORT Part 3: https://youtu.be/QJi0UJ_DV3E

3 Blue 1 Brown on YouTube: https://www.youtube.com/@3blue1brown

Tale of the Cognitive Tape in Part 1 of the Scouting Report: https://www.youtube.com/watch?v=0hvtiVQ_LqQ&t=3043s

Analyzing the Frontier with Zvi Mowshowitz: https://www.youtube.com/watch?v=SM4q-QAsoU8&t=1s

Tyler Cowen’s Interview with Jonathan Swift: https://conversationswithtyler.com/episodes/jonathan-gpt-swift/

X:

@labenz (Nathan)

@eriktorenberg (Erik)

@cogrev_podcast

SPONSORS: Shopify | NetSuite | Omneky

• Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

• NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. 

• Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.