"The Cognitive Revolution"

Nathan shares his perspective on Sam Altman’s firing from OpenAI, after being a part of the red team for GPT-4 and seeing how the board handled safety concerns. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. 

This is a developing story. This podcast was recorded on 11/21 at 12pm PST.

SPONSORS: 

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

With the onset of AI, it’s time to upgrade to the next generation of the cloud: Oracle Cloud Infrastructure. OCI is a single platform for your infrastructure, database, application development, and AI needs. Train ML models on the cloud’s highest performing NVIDIA GPU clusters.

Do more and spend less like Uber, 8x8, and Databricks Mosaic, take a FREE test drive of OCI at https://oracle.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

X/SOCIAL

@labenz (Nathan)

@eriktorenberg (Erik)

@CogRev_Podcast

TIMESTAMPS:

00:00:00) - Preview

(00:03:00) - Getting early access to test GPT-4

(00:08:03) - Concerns over OpenAI's slow start on safety after testing GPT-4

(00:15:38) - Sponsors: Shopify | Omneky

(00:17:34) - GPT-4's lack of safety measures and willingness to provide harmful information

(00:18:20) - GPT-4 suggesting assassination to slow AI progress

(00:23:00) - The failure of GPT-4's "safety edition" to properly constrain unsafe behavior

(00:29:00) - Sponsors: Netsuite | Oracle

(00:29:39) - Discussing concerns over GPT-4 capabilities with AI experts and leaders

(00:33:00) - An OpenAI board member not having tried GPT-4 despite its importance

(00:36:00) - Getting removed from the GPT-4 testing program by OpenAI

(00:44:00) - OpenAI subsequently showing more seriousness on safety after the GPT-4 experience

(00:54:00) - An example safety issue in GPT-4 that still persists

(01:00:00) - Speculation on what may have triggered the OpenAI board's removal of Sam Altman as CEO

(01:06:00) - Altman's "quaint" remark and acknowledgement of GPT-5 training

(01:21:57) - The absence of explanation from the board

(01:23:56) - How Microsoft will emerge

(01:32:36) - Should OpenAI be open source?

(01:42:00) - Innane regulation of AI 

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Producer: Vivian Meng

Executive Producers: Natalie Toren, and Erik Torenberg

Editor: Graham Bessellieu

For inquiries about guests or sponsoring the podcast, please email [email protected]

In this episode, Nathan and Erik discuss research out of Google Deepmind suggesting LLMs, Hemant Teneja’s responsible VC commitments, and why now is not the time for an ideological war on AI regulation. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. 

RECOMMENDED PODCAST:

Every week investor and writer of the popular newsletter The Diff, Byrne Hobart, and co-host Erik Torenberg discuss today’s major inflection points in technology, business, and markets – and help listeners build a diversified portfolio of trends and ideas for the future.

Subscribe to “The Riff” with Byrne Hobart and Erik Torenberg: https://link.chtbl.com/theriff

SPONSORS: 

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

X/SOCIAL

@labenz (Nathan)

@eriktorenberg (Erik)

@CogRev_Podcast

TIMESTAMPS:

(00:00:00) - Opening discussion of recent viral tweet claiming LLMs can't generalize

(00:03:27) - Waymo Safety Research 

(00:10:12) - Response to Google Deepmind’s research claiming LLMs can’t generalize

(00:15:08) - Sponsors: Shopify | Omneky 

(00:21:38) - Reproduction of the DeepMind research showing noise allows generalization

(00:26:02) - Helen of Transforrmers 

(00:30:15) - Sponsors: Oracle | Netsuite

(00:32:36) - Continuation of Google Deepmind discussion

(00:42:47) - “The AI Executive Order will only continue to look more foolish from here” 

(00:44:09) - The AI Scout mindset

(00:45:22) - Few humans can extrapolate beyond their training data

(00:48:45) - When does GPT-4 deceive its user? 

(00:49:58) - Voluntary Responsible AI Commitments from VCs 

(00:54:00) - Analysis of the commitments as basic common sense practices

(00:56:30) - No need for an ideological war 

(00:57:00) - Discussion of critical reactions framing the commitments as capitulation

(01:00:00) - Concerns about the voluntary commitments enabling harmful regulation

(01:03:00) - Rebuttal that self-regulation can avoid heavy handed regulation 

(01:08:00) - Self driving cars: the invisible lives you can save

(01:11:00) - Mysticism when it comes to AI

(01:12:03) - Suggestion that product liability law could address concerns

(01:15:00) - Challenge to defend rejecting product liability for AI systems

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

In this episode, Nathan interviews Alex Watson, founder and CPO of Gretel AI, about the company's work in synthetic data. They discuss why we need synthetic data, Gretel’s new pre trained tabular LLM that creates synthetic data on a zero shot basis, privacy techniques to prevent LLM memorization, and more. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

ORACLE:

With the onset of AI, it’s time to upgrade to the next generation of the cloud: Oracle Cloud Infrastructure. OCI is a single platform for your infrastructure, database, application development, and AI needs. Train ML models on the cloud’s highest performing NVIDIA GPU clusters.

Do more and spend less like Uber, 8x8, and Databricks Mosaic, take a FREE test drive of OCI at oracle.com/cognitive

NETSUITE:

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

OMNEKY:

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

X/SOCIAL:

@labenz (Nathan)

@AlexWatson405 (Alex)

@Gretel_AI

@CogRev_Podcast

TIMESTAMPS:

(00:00:00) - Intro

(00:03:02) - Origins of the company name Gretel AI and initial vision around enabling data sharing while protecting privacy

(00:05:16) - Alex's background in data privacy and compliance from his previous startup Harvest AI, acquired by AWS

(00:07:24) - Using synthetic data to create additional examples and improve detection of rare disease

(00:12:50) - Why use synthetic data?

(00:17:02) - Sponsors: Shopify | Omneky

(00:19:00) - Training models to recreate real-world data distributions and using validators to detect unrealistic outputs

(00:24:40) - Pre-training the Gretel tabular LLM on diverse internet data sets

(00:30:40) - Sponsors: Oracle | Netsuite

(00:34:00) - Using an agent planning architecture

(00:39:41) - Example use case of adapting models with synthetic data samples for long-tail cases

(00:43:00) - Using reinforcement learning to intentionally generate more diverse and representative synthetic data

(00:51:16) - The potential of efficient, lightweight models compared to massive LLMs like GPT-4

(00:56:00) - Analogizing model specialization to specialized parts of the brain 

(01:06:04) - Using differential privacy techniques to prevent memorization and exposure of private data

(01:14:37) - Adding noise during training to blur memorization

(01:18:42) - Optimism that synthetic data quality issues reflect details not fully understood yet vs inherent problems

We're joined by John Hayes, CEO and Cofounder of Ghost, a self-driving car startup using multimodal LLMs to solve the autonomous driving problem, that OpenAI just invested $5 million into. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS:

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI.

Sign up for $1/month trial period: https://shopify.com/cognitive

With the onset of AI, it’s time to upgrade to the next generation of the cloud: Oracle Cloud Infrastructure. OCI is a single platform for your infrastructure, database, application development, and AI needs. Train ML models on the cloud’s highest performing NVIDIA GPU clusters.

Do more and spend less like Uber, 8x8, and Databricks Mosaic, take a FREE test drive of OCI at oracle.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

X/SOCIAL:

@labenz (Nathan)

@ghosthayes (John)

@GhostAutonomy

@CogRev_Podcast

TIMESTAMPS:

00:00:00) - Introduction to John Hayes and Ghost Autonomy's partnership with OpenAI

(00:04:30) - The journey to founding Ghost, and how multimodal LLMs will change self-driving cars

(00:05:03) - John's background founding Pure Storage and moving into autonomy

(00:13:24) - Overview of different self-driving technology stacks (Lidar vs cameras)

(00:14:03) - Sponsors: Shopify | Omneky

(00:25:39) - Different approaches to how we know self-driving cars work well enough for public deployment

(00:32:21) - Sponsors: Netsuite | Oracle

(00:34:24) - Current state of robotaxis and China

(00:39:47) - Where Ghost fits into the autonomous driving landscape and their tech stack

(00:50:51) - Affordable radar systems

(00:51:54) - Ghost’s approach to interpretability

(00:53:27) - Using LLMs for driving

(00:54:59) - Partnership with OpenAI and how Ghost’s unique LLM solution to the self-driving problem

(00:57:40) - GPS’ unreliability which can be solved by multimodal approach

(01:02:31) - Hallucination is the best part of LLMs

(01:04:09) - Using maps as extended memory and storage for driving data

(01:09:20) - Ghost business model and the user experience

(01:16:37) - OpenAI as currently the only provider of advanced multimodal models

(01:24:17) - How self-driving cars should address fear and standing up for the technology

(01:29:30) - AI: China and the US, and why China is being pushed towards self-driving

(01:31:39) - Regulators in the US are responsive to consumers

(01:32:26) - VC subsidization of self-driving

(01:33:25) - John’s widest angle outlook on the AI revolution

We’re deep diving into OpenAI DevDay with Logan Kilpatrick, Dev Relations Lead at OpenAI. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS:

SHOPIFY: https://shopify.com/cognitive for a $1/month trial period

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

X/SOCIAL:

@labenz (Nathan)

@OfficialLoganK (Logan)

@CogRev_Podcast

TIMESTAMPS:

(00:00:00) - Episode Preview

(00:02:08) - How many startups did OpenAI kill?

(00:05:50) - Current employee count at OpenAI

(00:06:59) - OpenAI's mission being focused on developing safe AGI to benefit humanity

(00:07:10) - How the GPT Store relates to AGI and progressing agent development

(00:08:22) - OpenAI's strategy to release AI iteratively so society can adapt

(00:10:50) - Safety considerations around the OpenAI Assistant release

(00:11:30) - Capability overhangs and is the internet ready for agents?

(00:14:13) - Why certain agent capabilities like planning aren't enabled yet by OpenAI

(00:15:28) - Sponsors: Shopify | Omneky

(00:17:34) - GPT-4-1106 Preview designation

(00:21:50) - 16k fine-tuning for 3.5 Turbo

(00:25:13) - GPT-4 Finetuning and how to join the experiment

(00:27:53) - Custom models: $2-3 million pricing to build a defensible business

(00:29:48) - Bringing costs down to bring custom models to more people

(00:30:19) - Sponsors: Oracle | Netsuite

(00:33:53) - Copyright shield

(00:35:42) - OpenAI doesn’t train on data you send to the API

(00:36:37) - New modalities and low res GPT vision

(00:37:26) - GPT Vision Assessment for Aesthetics

(00:42:30) - WhisperLarge v3

(00:44:15) - Text-to-speech API: the voice strategy and AI safety

(00:51:45) - Log probabilities coming soon

(00:53:45) - The evolution of plugins to GPTs: the challenges with plugins

(00:55:33) - GPT Instructions, expanded knowledge, and actions

(01:00:18) - How is auth handled with GPTs

(01:01:04) - Hybrid auth

(01:02:50) - GPT Assistant API Billing

(01:07:58) - AI Safety

(01:10:28) - OpenAI Jailbreaks and Bug Bounties

(01:11:57) - The OpenAI roadmap for a year from now

The Cognitive Revolution is brought to you by the Turpentine Media network.

Producer: Vivian Meng

Executive Producers: Amelia Salyers, and Erik Torenberg

Editor: Graham Bessellieu

For inquiries about guests or sponsoring the podcast, please email [email protected]

In this episode, Flo Crivello, founder of Lindy AI, joins Nathan to chat about Biden’s executive order, and the state of AI safety. They discuss Flo’s thoughts on the executive order, building AGI kill switches, self driving cars, and more. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

SPONSORS: Netsuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

RECOMMENDED PODCAST: 

Every week investor and writer of the popular newsletter The Diff, Byrne Hobart, and co-host Erik Torenberg discuss today’s major inflection points in technology, business, and markets – and help listeners build a diversified portfolio of trends and ideas for the future. Subscribe to “The Riff” with Byrne Hobart and Erik Torenberg: https://link.chtbl.com/theriff

TIMESTAMPS:

(00:00) Episode Preview

(00:06:42) The natural order of technological progress

(00:07:00) Self driving cars

(00:10:57) Where is Flo accelerationist?

(00:12:34) Artificial intelligence as a new form of life

(00:17:08) - Sponsors: Oracle | Omneky

(00:18:05) Silicon-based intelligence vs carbon-based intelligence

(00:24:36) Executive Order

(00:29:32) How would a GPU kill switch work?

(00:31:24) “Let’s not regulate model development, but applications”

(00:32:08) - Sponsor: Netsuite

(00:36:00) GPT-4 is the most critical component for AGI

(00:38:00) AGI in 2-8 years

(00:39:26) Eureka moment from a general system

(00:48:00) AI research with China

(00:52:00) Does AI have subjective experience? The Mu response

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Producer: Vivian Meng

Executive Producers: Amelia Salyers, and Erik Torenberg

Editor: Graham Bessellieu

Mark Humphries is a Professor of History at Wilfrid Laurier University, where he has published widely on various aspects of Canadian history. We invited Mark to do an episode after he reached out to tell us that hearing about how Nathan fine-tuned GPT-3.5 on GPT-4 reasoning helped him get over some hurdles in his own archival research. In addition, he chats about all the things that he'd tried that hadn't worked, and in the process proved himself to be one of the world's leading adopters of AI technology in the field of history.

If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

SPONSORS: NetSuite | Omneky

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

(00:00:00) - Episode Preview

(00:03:00) - The state of AI in history before recent advances like GPT-3

(00:06:00) - Mark's background and how he first got interested in AI

(00:09:00) - Using AI to process and search through vast amounts of archival material

(00:12:00) - The challenges of digitizing and searching handwritten historical documents

(00:15:00) - The massive scale of undiscovered archival material that could be processed by AI

(00:18:00) - How AI can help historians process more archival material than ever before

(00:20:11) - Sponsors: Netsuite | Omneky

(00:21:00) - Limitations on using commercial APIs for private archival documents

(00:24:00) - The cost of processing archival documents compared to hiring human transcribers

(00:27:00) - Comparing GPT's ability to transcribe historical handwriting vs more recent scripts

(00:33:00) - Tracing individuals through complex fur trade records using AI

(00:36:00) - Mark's approach to fine-tuning models for specific historical tasks

(00:39:00) - Challenges teaching AI models the nuances of historical writing

(00:42:00) - Privacy issues limiting the use of commercial APIs on restricted archival material

(00:45:00) - The gap between people's expectations of AI and what it can really do right now

(00:51:00) - Adapting assignments and student expectations to AI's current capabilities

(00:52:28) - Using chain of thought prompting to teach models precise historical tasks

(00:52:46) - Getting high accuracy on keyword tagging of archival documents using fine-tuning

(00:57:00) - Mark has crossed the threshold from promise to accelerating archival research

(01:00:00) - How to frame classification tasks to get the most value from AI models

(01:03:00) - Teaching AI models to pronounce tricky historical names and terms

(01:06:19) - Mark's experience using AI in the classroom over two semesters

(01:09:00) - Assignments that are better suited to the current strengths and weaknesses of AI

(01:11:45) - AI progress in education is lagging due to lack of institutional adoption

(01:12:00) - The trajectory of AI capabilities and the need for humans to exceed the model baseline

(01:27:49) - Lessons from history on economic transitions and the social contract

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

In this episode, Yohei Nakajima, creator of BabyAGI and GP at Untapped Capital, chats with Nathan about the opportunity for AI to strengthen human understanding, AI agents, and his insights on investing in AI projects. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

(00:10:59) - A large part of AI is trying to map human cognitive processes onto our software stack

(00:11:14) - Yohei’s TED AI talk: the opportunity for AI to strengthen human understanding

(00:13:29) - Yohei's journey to build AI projects and building in public 

(00:14:04) - Yohei's lifelong interest in building tools and automating workflows for efficiency

(00:16:29) - Sponsors: Netsuite | Omneky 

(00:22:05) - Building projects as a fun and exploratory practice 

(00:23:53) - Single use software

(00:24:18) - Yohei’s home assessment price contesting project 

(00:24:58) - The origins of BabyAGI as an autonomous startup founder prototype

(00:27:44) - Create as many paper clips as possible 

(00:30:10) - Thoughts on going all in on a project and turning it into a company vs pursuing it on the side 

(00:33:00) - Forking different versions of Baby AGI based on animal names

(00:34:24) - Yohei’s learnings about agents after iterating on six versions of Baby AGI 

(00:36:24) - Yohei’s thoughts on generalist agents

(00:37:28) - Replit

(00:40:54) - Begin exploring AI automation with tasks you wish you had time for, not critical workflows

(00:42:32) - Automating outbound sales and recruiting workflows with AI

(00:46:19) - Where should people start learning about AI agents?

(00:48:23) - Will agents come online with GPT-4V?

(00:52:40) - The unique moment where both executives and engineers both want to implement AI

(00:54:59) - AI layers: hardware, software, tooling 

(00:57:20) - Taxonomy of AI business models 

(01:10:47) - Designing AI to provide value for senior citizens

(01:15:14) - The responsibility to nudge people positively with influential AI assistants

(01:19:24) - Advice on when and how startups should build in public

(01:22:24) - Perspective on existential risk from AI vs. its potential benefits

(01:15:30) - Responsibility to handle AI-human emotional connections well 

(01:19:24) - Advice on when and how startups should build in public

(01:22:24) - Perspective on existential risk from AI vs. its potential benefits

Anton Troynikov, cofounder of Chroma, joins Nathan Labenz to discuss the importance of keeping the retrieval-augmented generation (RAG) loop in house, what it means for Chroma to be in “wartime” mode right now, and how much of the data going into Chroma has never been in a database before. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

SPONSORS: NetSuite | Omneky

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

LINKS:

Part 1 with Anton: https://youtu.be/ogy37CdIljg

X/SOCIAL:

@labenz (Nathan)

@atroyn (Anton)

@eriktorenberg (Erik)

@CogRev_Podcast

TIMESTAMPS:

(00:00:00) - Introduction by Nathan, setting up the conversation with Anton

(00:02:16) - Anton articulates Chroma's mission to build a horizontally scalable system

(00:03:06) - Rise in popularity of retrieval-augmented generation (RAG)

(00:06:03) - Chroma's focus on delivering a horizontally scalable cloud service for vector search and storage

(00:08:07) - Nathan describes his experience building a RAG application for a client profiling use case

(00:10:27) - Anton advises measuring retrieval quality and maximizing relevant information returned

(00:15:05) - Sponsors: Netsuite | Omneky

(00:17:02) - Popular use of open source vs. proprietary embedding models like Anthropic's Ada

(00:19:30) - The importance of keeping the RAG loop in house and not relying solely on external APIs

(00:27:41) - The huge amount of unstructured data that can now be processed by AI

(00:30:40) - Providing a unified interface to structured and unstructured data

(00:33:15) - Much of the data going into Chroma has never been in a database before

(00:38:47) - Categories of organizations adapting to AI: legacy, AI-native, and AI-first

(00:40:55) - Where Chroma is seeing most of its growth right now

(00:46:20) - Interpretability work like Anthropic's circuit evaluation

(00:52:23) - Anton believes new tooling can make latent spaces accessible without AI expertise

(01:06:08) - Scaling constraints between search indexes vs. application databases

(01:09:10) - Potential for time as a dimension in embedding spaces

(01:13:46) - Likelihood of missing results due to representational issues vs. approximate nearest neighbor

(01:15:22) - Automatically handling small data sets without needing elaborate indexing

(01:17:20) - Anton's perspective on whether OpenAI will build its own database

(01:19:43) - Partnering with OpenAI and other labs to increase use of their models

(01:21:19) - Anton's experiments probing GPT's reasoning abilities with Game of Life

(01:25:41) - Closing thoughts on the conversation

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Trey Kollmer returns to discuss the latest AI research revelations with Nathan Labenz. They explore how new techniques will shave 10% off global compute needs, how analogical prompting beats few-shot prompting, and how compressive historical records can increase LLM memory and retention abilities. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

RECOMMENDED PODCAST: 

Every week investor and writer of the popular newsletter The Diff, Byrne Hobart, and co-host Erik Torenberg discuss today’s major inflection points in technology, business, and markets – and help listeners build a diversified portfolio of trends and ideas for the future. Subscribe to “The Riff” with Byrne Hobart and Erik Torenberg: https://link.chtbl.com/theriff

TIMESTAMPS:

(00:00:00) - Episode Preview

(00:01:11) - Paper: Think Before You Speak

(00:03:13) - Multimodal models for combining vision and language

(00:04:19) - Backspace Paper

(00:06:25) - Chain of thought prompting for step-by-step reasoning

(00:09:14) - Backspacing in language models to correct mistakes

(00:12:05) - Attention sinks for expanding context length

(0012:41) - Paper: Large Language Models as Analogical Reasoners

(00:15:24) - Pause tokens for language models to "think"

(00:18:23) - Analogical prompting to recall relevant examples

(00:20:52) - Long context windows for language models

(00:23:20) - Markdown works best for OpenAI

(00:24:23) - Ring attention to break memory constraints

(00:26:15) - Paper: StreamingLLMs

(00:27:46) - Potential for superhuman performance with longer contexts

(00:31:01) - Dynamic context window adjustment at runtime

(00:33:53) - Retention and memory capabilities for transformers

(00:37:12) - Planning algorithms combined with memory and scale

(00:39:49) - Paper: Ring Attention

(00:42:35) - Executive assistant prompting and critique

(00:45:23) - Self-RAG for language models to find own examples

(00:48:02) - Timelines and predictions for future capabilities

(00:50:37) - Applications like analyzing long texts and scripts

(00:53:15) - Local versus global attention in transformers

(00:55:59) - Architectural changes versus just training adjustments

(00:58:41) - Pre-training strategies like random start points

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].