"The Cognitive Revolution"

[Bonus Episode] Latent.space hosts, Alessio and Swyx, sit down with Linus Lee of Notion AI to discuss Linus’ experience starting the AI/UX community, prompt engineering at Notion, and designing AI interfaces and agents. 

The latent.space podcast aims to be the first place where AI engineers hear about the latest AI news and technology trends. We’ve had several guests in common – including Shreya Rajpal of Guardrails, Jonathan & Abhi from the recently acquired for $1.3B MosaicML, and also Riley Goodside from Scale AI – and also explored similar themes, including the question of which AI companies have durable business moats, and the flurry of experimentation going into AI agents right now. We recommend checking them out! 

Keep up to date with Latent.space here: 

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

TIMESTAMPS

(05:52) Starting the AI / UX community

(12:23) Most knowledge work is not text generation

(18:43) Finding the right constraints and interface for AI

(21:27) Linus' journey to working at Notion

(25:51) The importance of notations and interfaces

(28:29) Setting interface defaults and standards

(34:58) The challenges of designing AI agents

(42:00) Notion deep dive: “Blocks”, AI, and more

(53:22) Prompt engineering at Notion

(01:04:22) Lightning Round

SPONSOR:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Nathan Labenz sits down with Lili Yu, a researcher of Meta AI to discuss the paper she authored: MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. In this conversation, they discuss the architecture and breakthroughs of their research, and the opportunity to eliminate the need for tokenization.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

The Cognitive Revolution is a part of the Turpentine podcast network. Learn more: www.turpentine.co

LINK:

MEGABYTE Paper: https://arxiv.org/pdf/2305.07185.pdf

TIMESTAMPS:

(00:00) Episode preview

(07:41) Takeaways from Lili Yu's paper: MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

(17:00) Architecture

(24:59) Embeddings

(27:43) Different local models

(34:23) Encoder model

(36:35) Transformer Architecture

(48:10) Choosing patch size

(01:08) What happens when you scale up?

(01:19:20) Big picture for Meta AI

(01:22:57) Responsible AI

(01:27:02) China and AI

TWITTER:

@labenz (Nathan)

@liliyu_lili (Lili)

@eriktorenberg (Erik)

@cogrev_podcast

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

MUSIC CREDIT:

MusicLM:

Ziming Liu is a Physics PhD student at MIT and IAIFI, advised by Prof. Max Tegmark. Ziming’s research is at the intersection of AI and physics. Today’s discussion goes in-depth on Liu’s paper “Seeing is Believing” where he presents Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable. The ability to see modules visually can complement current mechanistic interpretability strategies.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co

Watch this episode on YouTube to see research figures and graphics: https://www.youtube.com/watch?v=VKxwlxb2XWw

Links:

Seeing is Believing, authors: Ziming Liu, Eric Gan, and Max Tegmark https://arxiv.org/pdf/2305.08746.pdf

Ziming Liu: https://kindxiaoming.github.io/

TIMESTAMPS:

(00:00) Episode preview

(01:16) Nathan introduces Ziming Liu and his research

(04:50) What motivated Ziming's research into neural networks and modularity

(08:06) Key Visuals of "Seeing is Believing"

(08:44) Biology analogy & paper overview

(15:07) Sponsor: Omneky

(26:08) Research precedents

(30:00) Locality vs modularity

(37:40) Shoutout Tiny Stories : Episode 33

(39:00) Quantization Model of Neuro Scaling.

(1:00:00) How can this work impact mechanistic interpretability strategies

(1:27:00) Training

(1:46:30) Downscaling large-scale models

TWITTER:

@labenz (Nathan)

@ZimingLiu11 (Ziming)

@cogrev_podcast

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

MUSIC CREDIT:

MusicLM

Part 2 of Nathan's conversation with Tanishq Mathew Abraham focuses on Tanishq's work with virtual biopsy technology enabled by deep learning. This unique technology has the potential to address a number of important biomedical challenges; in particular, qOBM could help during live tumor and cancer removal surgeries.

Tanishq, a 19-year-old UC Davis grad and one of the youngest people in the world to receive a Ph.D, with a degree in biomedical engineering, is the founder of the Medical AI Research Center (MedARC). In case you haven’t yet listened to part 1, check it out! That episode goes deep on Tanishq’s first published paper: Reconstructions of the Mind’s Eye, which encompasses breakthrough research on reconstructing visual perceptions from fMRI scans into images.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co

TIMESTAMPS:

(00:00) Episode Preview

(03:26) Introducing Tanishq’s second paper presenting an AI-enabled biopsy 

(06:39) Diagnostic applications of this unique slide-free and label-free technology (qOBM)

(13:06) Leveraging deep learning, specifically generative adversarial networks (GANs)

(14:55) Sponsor: Omneky

(15:42) Framework for model

(16:22) Challenges of medical data sets

(20:11) Challenges of unpaired image to image translation, addressed with a CycleGAN architecture 

(22:19) What didn’t work

(25:44) Breaking down GAN frameworks

(34:30) Simplifying data to better with the CycleGAN

(36:02) Factors for errors and confusing the model

(39:00) Compute and training requirements

(41:35) How this technology can scale in the near term and improve patient care

(45:53) Tanishq’s relationships with EleutherAI and Stability AI

(47:08) The future-looking focus of MedArc and unexplored opportunities

(49:00) Developments in Medical AI field and interesting applications

(55:11) Early education and AI

(58:00) Would Tanishq get a neuralink?

(1:00:00) Tanishq’s hopes and fears for AI

LINKS:

Paper: Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual H&E staining https://arxiv.org/pdf/2306.00548.pdf

Authors: Tanishq Mathew Abraham, Paloma Casteleiro Costa, Caroline Filan, Zhe Guang, Zhaobin Zhang, Stewart Neill, Jeffrey J. Olson, Richard Levenson, Francisco E. Robles

TWITTER:

@iScienceLuvr (Tanishq)

@MedARC_AI (MedARC)

@CogRev_Podcast

@labenz (Nathan)

@eriktorenberg (Erik)

SPONSOR:

Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

MUSIC CREDIT:

MusicLM

In this episode, Nathan sits down with Tanishq Mathew Abraham, 19-year-old UC Davis grad and one of the youngest people in the world to receive a Ph.D, with a degree in biomedical engineering. Tanishq is the founder of the Medical AI Research Center (MedARC), and with his teammates, recently published a paper: Reconstructions of the Mind's Eye, which encompasses their breakthrough research on reconstructing visual perceptions from fMRI scans into images. In this episode, Nathan and Tanishq talk about the technology behind the fMRI-to-image project, developing the model, and future applications for this research.

Part 2 with Tanishq will be released as the next episode.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co

TIMESTAMPS:

(00:00) Episode Preview

(05:43) The MindEye Project

(09:06) Resemblance between AI reconstruction of mind's eye and visual presented

(10:00) What is a voxel and which regions of the brain were studied?

(10:23) What would the raw data of a voxel be?

(11:44) Is there a time dimension to voxels?

(15:00) Sponsor: Omneky

(17:50) Goals for the MindEye project

(25:57) What is the starting point of the model?

(31:15) Aligning the model: reconstruction vs retrieval

(40:34) Would doing a full end-to-end training be fine for the reconstruction?

(42:15) The role of a limited data set

(43:09) Training separate models per subject

(45:07) Generalizability with a limited dataset

(47:20) Mapping from one high-dimensional space to another

(50:47) Stable Diffusion VAE encoding

(1:00:50) How long does it take to train the model?

(1:03:14) How similar or different are the subjects and their individual models?

(1:05:59) The future of this research: custom models for your brain?

(1:07:34) How much does this research contribute to brain research and wearables?

(1:11:15) Fuzzing data and future research applications

LINKS:

MedARC: medarc.ai

MindEye Paper: https://www.researchgate.net/publication/371136623_Reconstructing_the_Mind's_Eye_fMRI-to-Image_with_Contrastive_Learning_and_Diffusion_Priors

MP3 of this episode: https://chrt.fm/track/993DGA/traffic.megaphone.fm/RINTP1584997572.mp3?updated=1687271014

TWITTER:

@iScienceLuvr (Tanishq)

@MedARC_AI (MedARC)

@CogRev_Podcast

@labenz (Nathan)

@eriktorenberg (Erik)

SPONSOR:

Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

MUSIC CREDIT:

MusicLM

In this episode, Nathan sits down with Jonathan Frankle, Chief Scientist, and Abhi Venigalla, Research Scientist of MosaicML. They chat about Mosaic’s custom LLMs, the customers seeking Mosaic out and what their journeys and use cases look like, and exciting developments in Mosaic’s research: including their new inference platform, as well as Mosaic’s MPT-7B-65k+ storywriter model.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: www.turpentine.co

TIMESTAMPS:

(00:00) Episode Preview

(06:04) Mosaic’s business model

(07:28) Who uses Mosaic’s custom LLMs? What does their data look like?

(09:55)Mosaic’s use cases for custom LLMs

(12:47) How much extraction and summarization was done by humans pre-LLMs?

(15:28) Sponsor: Omneky

(21:50) The journeys of Mosaic’s customers and would a Wendy’s LLM know about a Big Mac?

(25:46) The curriculum model and fine-tuning

(29:10) Language models in the life sciences

(33:20) How raw can data be before it becomes a problem?

(35:44) Using the output of bulk pre-training process vs additional after training

(38:30) Redteaming as a service

(39:40) Mosaic’s inference platform

(41:53) Spending one cent on 20,000 tokens, how is that cent distributed?

(46:00)) Selling compute on a dedicated capacity basis

(47:30) Oracle and AWS

(49:50) The storywriter model and 65,000 token window

(54:35) The transition from finite parameters into infinite attention matrix

LINKS:

MosaicML: https://www.mosaicml.com/

MPT-7B Storywriter Model: https://huggingface.co/mosaicml/mpt-7b-storywriter

TWITTER:

@jefrankle (Jonathan)

@abhi_venigalla (Abhi)

@MosaicML (Mosaic)

@CogRev_Podcast

@labenz (Nathan)

@eriktorenberg (Erik)

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Music Credit: MusicLM

In this episode, Nathan Labenz and Erik Torenberg host a discussion between Emad Mostaque, founder of Stability AI, and Sam Lessin, GP of Slow Ventures and former VP Product at Facebook. These influential technology thinkers tackle topics pertinent to evaluating whether AI is a good investment for venture capital. They talk about the attention and information economy, SaaS markets, how AI-created entertainment will impact society and our relationships, and AI disruption of highly-regulated industries.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

LINK FOR NATHAN'S LIVE EVENT IN SAN FRANCISCO:

Friday June 16th: Why AI with Athena: https://v4p9mjmurpv.typeform.com/to/llRcJSF9

TIMESTAMP:

(00:00) Episode Preview

(01:17) Nathan explains why we hosted a conversation with Emad and Sam

(08:25) Discussion kicks off: AI as an extender rather than a deck shuffler

(12:01) Is AI a disruptive or sustaining innovation?

(14:22) What is intelligence?

(16:00) The attention and information economy

(21:56) Storytelling, the stock market, and the economy

(23:23) Sponsor: Omneky

(23:50) AI in the media industry, Hollywood, and distribution

(29:10) Startups vs incumbents

(30:30) Disrupting highly-regulated industries

(32:30) AI in education: an AI tutor for every child

(35:15) AI as StackOverflow 2.0 for engineers

(37:23) Regulatory Arbitrage

(40:06) The oracle problem in AI

(42:00) Apple as the weakest BigTech player in AI

(44:30) AI in communities and the future of social media

(46:55) Replika on Valentine’s Day and AI-generated porn

(53:50) How does the market for inference shape up? Inference vs. cloud

(55:08) Price of primary care in US in the next decade

(55:38) What are the most likely incumbents to be disrupted?

(56:55) Will BigTech continue to opensource?

(58:56) Marriage rate in US in the future given AI

(01:00:24) Can any AI leader sustain high gross margins in the future?

(01:02:04) Stability’s business model

TWITTER:

@EMostaque (Emad)

@Lessin (Sam)

@CogRev_Podcast

@labenz (Nathan)

@eriktorenberg (Erik)

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Download this episode as an mp3: https://chrt.fm/track/993DGA/traffic.megaphone.fm/RINTP2126380296.mp3?updated=1686676812

Music Credit: MusicLM

In this episode, Nathan sits down with Joshua Browder of DoNotPay, the world’s first robot lawyer. They chat about the current state of AI use in law, what policymakers should consider in regulating AI, and the ethics of robo-lawyers for consumer use. 

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

TIMESTAMPS:

(00:00) Episode Preview

(03:22) Joshua Browder and the story of DoNotPay

(04:50) The value incurred through an AI lawyer

(07:17) What are the legal and financial injustices that motivates Joshua’s work?

(12:46) How does the AI negotiate with a Comcast chat agent?

(16:28) Consumer security

(17:12) Sponsor: Omneky

(21:08) The ethics of DoNotPay

(22:27) Should AI need to disclose that it’s AI?

(24:47) How much human intervention is necessary in AI tooling?

(27:15) Where does the burden of proof lie for AI regulation and ethics?

(31:47) Advice to policymakers

(35:45) AI Liability

(38:05) AI Arms Race

(43:46) Productive bot interactions

(45:00) AI-powered arbitration

(49:28) How the consumer experience might change two years from now

(51:45) AI Superapps

(55:25) Moats in AI

(57:11) How much money does DoNotPay save people today?

(59:30) Consumer and enterprise timelines given AI

(01:06:30) What will AI replace?

(01:09:34) Impact of AI in different industries

(01:11:10) The legal field’s view of AI

(01:15:50) Deflationary factors in running DoNotPay

(01:25:00) What are non-obvious ways to save money?

(01:27:55) Joshua’s favorite AI tools

(01:28:17) Would Joshua get a Neuralink implant?

LINKS:

https://donotpay.com/

TWITTER:

@jbrowder1 (Joshua)

@DoNotPay (DoNotPay)

@CogRev_Podcast

@labenz (Nathan)

@eriktorenberg (Erik)

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

NEWSLETTER:

More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Nathan Labenz sits down with Ronen Eldan and Yuanzhi Li of Microsoft Research to discuss the small natural language dataset they created called TinyStories. Tiny Stories is designed to reflect the full richness of natural language while still being small to support research with modest compute budgets.  Using this dataset, they began to explore aspects of language model performance, behavior, and mechanism by training a series of models that range in size from just 1 million to a maximum of 33 million parameters – which is still just 2% the scale of GPT-2. In this conversation, Nathan, Ronen, and Yuanzhi touch on LM reasoning, emergence, interpretability, and what understanding can be extended to LLMs.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

LINKS:

Tiny Stories paper: https://huggingface.co/papers/2305.07759

TIMESTAMPS:

(00:00) Episode Preview

(07:12) The inspiration for the Tiny Stories project

(15:07) Sponsor: Omneky

(15:44) Creating the Tiny Stories dataset

(21:27) GPT-4 vs GPT-3.5

(24:13) Did the TinyStories team try any other versions of GPT-4

(29:23) Curriculum models and weirder curriculums

(35:34) What does reasoning mean?

(46:27) What does emergence mean?

(01:01:44) The curriculum development space

(01:11:40) The similarities between models and human development

(01:20:12) Fewer layers vs. more layers

(01:29:22) Attention heads

(01:33:40) Semantic attention head

(01:36:54) Neuron technique used in developing the TinyStories model

(01:52:20) Interpretability work that inspires Ronen and Yuanzhi

TWITTER:

@CogRev_Podcast

@EldanRonen (Ronen)

@labenz (Nathan)

@eriktorenberg (Erik)

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Music Credit: MusicLM

More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

This 20-min episode comes from our friend Nathaniel Whittemore's excellent daily podcast The AI Breakdown Podcast. This episode aired on June 1, 2023, and covers the latest developments from OpenAI, including new features, a cybersecurity grant program, and their new process rewards model for trading. We hope you enjoy it as much as we do.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

SPONSOR:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Subscribe to The AI Breakdown podcast: https://pod.link/1680633614

Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown