"The Cognitive Revolution"

Trey Kollmer joins Nathan Labenz for a roundup of the latest AI research! They discuss Microsoft’s Self-Taught Optimizer (STOP) research and Google’s FreshLLMs, how H100 chips will supercharge development of programs with GPT-4 level compute, LLM representation of space and time, and more! If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

(00:00:00) – Introduction

(00:00:56) – Update to WGA Strike

(00:03:00) – Trey Kollmer's background

(00:06:00) – Scaling compute for AI training experiments with GPT-4 as reference point

(00:09:00) – Inflection's plan to acquire 22,000 H100s to reach GPT-4 scale compute in 5 days

(00:12:00) – Addressing knowledge cutoff in LLMs using search engines

(00:15:00) – Inserting structured search results into prompts with metadata

(00:16:07) – Sponsors: Netsuite | Omneky

(00:18:00) – Comparing approach to Perplexity system

(00:18:08) — Fresh LLMs

(00:21:00) – Microsoft’s Self-taught Optimizer (STOP): Recursive self-improvement framework

(00:24:00) – STOP framework works with GPT-4 but not GPT-3.5

(00:27:00) – STOP removed sandbox flag in some cases

(00:30:00) – LLMs represent space and time with probe models

(00:33:00) – Visualizations show emergence of spatial maps

(00:33:14) — OpenAI rumours

(00:36:00) – Techniques like linear probes and holdout studies

(00:39:00) – DNNs extrapolate predictably by falling back to ignorance

(00:42:00) – Testing different architectures, loss functions, distribution shifts

(00:45:00) – Design systems to be conservative out of distribution

(00:48:00) – Potential for recursive architecture search

(00:50:21) — LLMs represent Space and Time

(00:51:00) – Vision API enabling more capable web agents

(00:54:00) – Discussion of research insights

(00:57:00) – Thoughts on stochastic parrots debate

(01:11:25) — Deep Neural Networks Tend to Extrapolate Predictably

X/Social

@labenz (Nathan)

@treyko (Trey)

@CogRev_Podcast

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Nathan Labenz synthesizes recent research in mechanistic interpretability and AI safety, how top players in the space like Anthropic and OpenAI are addressing them, and jailbreaks like the Calvin and Hobbes one you may have seen online. Nathan's aim is to impart the equivalent of a high school AP course understanding to listeners in 90 minutes. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

Questions or topics you want us to review for future episodes? Email [email protected]

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

LINKS:

Scouting Report Part 1 - Fundamentals : https://www.youtube.com/watch?v=0hvtiVQ_LqQ

Scouting Report Part 2 - Impact, Fallout, and Outlook: https://www.youtube.com/watch?v=QJi0UJ_DV3E

Universal Jailbreaks with Zico Kolter, Andy Zou, Asher Trockman: https://www.youtube.com/watch?v=BwltbhR0JgU&feature=youtu.be

TIMESTAMPS:

(00:00) Episode Preview

(02:26) AI Engineer Survey

(03:53) P(Doom)

(00:07:52) Representation engineering

(00:09:20) Using contrasting prompts to understand model’s inner representations

(00:15:16) Sponsors: Netsuite | Omneky

(00:22:00) Controlling AI systems and detecting jailbreaks

(00:28:53) LLM performance and refusal rates varying by language

(00:33:13) Towards monosemanticity: decomposing language models with dictionary learning

(00:54:12) Implications of the aforementioned paper

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

In this episode, Nathan is joined by James Yu, Founder of Sudowrite, an AI writing tool.They discuss how James started Sudowrite after using GPT-3 for his own fiction writing, how Sudowrite is able to be a supportive writing partner, and Yu's experience developing its tech stack and working through the backlash he's received. f you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

Subscribe to the show Substack to join future conversations and submit questions to upcoming guests! https://cognitiverevolution.substack.com/

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

(00:00:00) - Episode Preview

(00:04:28)– How James started using GPT-3 for fiction writing in early beta

(00:08:47) - Using few-shot learning to get good results from early GPT-3

(00:10:25) – Focusing messaging on being a supportive AI writing partner

(00:12:08) – Helping fiction writers feel less lonely through pseudo-conversations with AI

(00:16:01) - Sponsors: Netsuite | Omneky

(00:17:22) - Evolving from paragraph-level assistance to full story understanding

(00:20:19) - Building the product for centaurs: half Human, half AI

(00:32:15) - Using multiple models together to "kitbash" the best outputs

(00:39:21) - Iterating on prompts over time to get a customized "garden"

(00:45:18) - Keeping temperature high for creative writing, but avoiding cliches

(00:54:21) - Separate paths for professional writers vs casual creators

(00:57:30) – The continued role of human-centric art and culture

(01:00:10) - The potential for automation to replace repetitive jobs

(01:07:48) - Yu’s experience launching an AI writing tool and the backlash he received

(01:16:24). - Advice for future AI product launches to avoid or minimize backlash

(01:18:25) - Focusing on delivering value to customers during controversy

(01:28:44) - The idea of an "AI bundle" to reduce churn and customer acquisition costs

(01:42:06) - Recommendations for underrated AI apps - Claude, Pika, Midjourney

The Cognitive Revolution is brought to you by the Turpentine Media network.

Producer: Vivian Meng

Executive Producers: Amelia Salyers, and Erik Torenberg

Editor: Graham Bessellieu

For inquiries about guests or sponsoring the podcast, please email [email protected]

In this episode, Nathan sits down with Swyx of Latent Space to chat about AI engineers and tools to check out, competitive dynamics between OpenAI, other foundation model providers, and developers, and if Swyx would wear an AI Horcrux. We also learn that Nathan lived in the same dorm as Mark Zuckerberg back in his college days (and was a late adopter to Facebook…). If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

Subscribe to the show Substack to join future conversations and submit questions to upcoming guests! https://cognitiverevolution.substack.com/

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

(00:00) Episode Preview

(00:00:49) AI Nathan’s intro

(00:03:14) What is an AI engineer?

(00:05:56) What backgrounds do AI engineers typically have?

(00:15:51) Sponsors: Netsuite | Omneky

(00:17:13) Swyx’s Discord AI project

(00:20:41) Key tools for AI engineers

(00:23:42) HumanLoop, Guardrails, Langchain

(00:27:01) Criteria for identifying capable AI engineers when hiring

(00:30:59) Skepticism around AI being a fad and doubts about contributing to AI

(00:34:03) AI Engineer Conference speaker lineup

(00:41:14) AI agents and two years to AGI

(00:46:04) Expectations and disagreement around what AI agent capabilities will work soon

(00:50:12) Swyx’s OpenAI thesis

(00:53:03) AI safety considerations and the role of AI engineers

(00:56:24) Disagreement on whether AI will soon be able to generate code pull requests

(01:01:07) AI helping non-technical people to code

(01:01:49) Multi-modal Chat-GPT and the future implications

(01:03:33) Nathan living in the same dorm as Mark Zuckerberg

(01:04:44) Competitive dynamics between OpenAI and other AI model developers

(01:05:39) Play.ht vs ElevenLabs

(01:09:20) The tension between platforms and developers building on top of them

(01:11:40) The best thing startups can do to compete with foundation model providers

(01:16:26) User identity/authentication services like Login with OpenAI

(01:19:20) Google vs the other live players

(01:20:46) AI Horcruxes / Pendants

(01:22:05) The concept of an AI app bundle for consumers and developers

In this episode, Aravind Srinivas of Perplexity AI returns to the show. They discuss Perplexity’s growth to millions of queries per day, CEO Aravind’s favourite Perplexity use cases, and how Perplexity ships fast in a competitive landscape against Google and major AI live players creating their own search engines. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

X:

@labenz (Nathan)

@AravSrinivas (Aravind)

@eriktorenberg (Erik)

@cogrev_podcast 

LINKS:

- Perplexity AI: https://www.perplexity.ai/

- Part 1 with Aravind: https://www.youtube.com/watch?v=YSYQZZu4MEM

TIMESTAMPS:

(00:00:00) - Episode Preview

(00:01:12) - Introducing Aravind Srinivas and Perplexity AI

(00:05:05) - Perplexity's growth to millions of queries per day

(00:06:21) - Perplexity's retention and market share vs. ChatGPT

(00:08:32) - Improving reliability, speed, accuracy, and new features

(00:10:33) - Getting to daily usage from weekly usage

(00:13:21) - Aravind's favorite Perplexity use cases

(00:18:34) - Issues with lack of sufficient web coverage for hard queries

(00:18:38) - Sponsors: Netsuite | Omneky

(00:23:13) - Building their own search index and models for self-sufficiency

(00:26:29) - Using their own index vs real-time web queries

(00:28:43) - Integrating default search and Copilot companion experiences

(00:31:49) - Training their own models as the only viable long-term option

(00:36:56) - The web becoming more like an API accessed via AI assistants

(00:41:23) - Not trying to copy Google's business model

(00:44:36) - Spending more on compute than people to get scale efficiencies

(00:49:35) - Replacing all parts of the stack with their own proprietary versions

(00:53:59) - Shipping fast out of necessity in a competitive war zone

(00:58:00) - AI assistants becoming a significant part of human digital activity

The Cognitive Revolution is brought to you by the Turpentine Media network. 

Producer: Vivian Meng

Executive Producers: Amelia Salyers, and Erik Torenberg

Editor: Graham Bessellieu

For inquiries about guests or sponsoring the podcast, please email [email protected]

Zvi Mowshowitz, the writer behind Don't Worry About the Vase, returns to catch up with Nathan on everything OpenAI, Amazon-Anthropic collab, and Google Deepmind. They also discuss Perplexity, deepfakes, and software bundling vs the Roman Empire. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

Definitely also take a moment to subscribe to Zvi's blog Don't Worry About the Vase 

(https://thezvi.wordpress.com/) - Zvi is an information hyperprocessor who synthesizes vast amounts of new and ever-evolving information into extremely clear summaries that help educated people keep up with the latest news. 

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

00:00:00 - Episode Preview

(00:02:42) - Nathan's experience using Code Interpreter for a React app

00:06:09 - Zvi's perspective on Code Interpreter and other new Anthropic products

(00:10:47) - Nathan's approach of "coding by analogy" using Code Interpreter

(00:15:42) - Sponsors: Netsuite | Omneky

(00:17:00 )- Performance degradation issues with large context windows

(00:22:50) - The disconnect between Anthropic's value and what users are willing to pay

(00:31:56) - Predicting Gemini's capabilities relative to GPT-4

00:30:13 - Rating Code Interpreter's capabilities

(00:37:53) - Using Perplexity vs. Anthropic products for search

(00:44:11) - Potential for a bundled subscription for multiple AI services

(00:47:39) - Challenges of getting competitors to agree to bundling

(00:54:05) - Concerns over information pollution from synthetic content

(00:56:36) - Filtering adversarial vs. unintentional bogus information

(01:02:20) - Dangers of info pollution visible in Archive dataset

(01:11:15) - Kevin Fisher’s AI Souls demo with emotional voices

(01:12:15) - Difficulty of detecting AI voices/images for a general audience

(01:14:32) - Being optimistic about defending against deepfakes

(01:21:12) - The reversal curse in language models

(01:46:12) - Implications of Amazon investing in Anthropic

(01:49:20) - Non-standard terms likely affected the Anthropic valuation

(01:51:13) - Survey of the AI Safety landscape

Nathan and Erik chat OpenAI GPT-3.5 fine tuning updates, using GPT 4 outputs to fine-tune 3.5, when to accelerate, and AI bundles. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

TIMESTAMPS:

(00:00) - Intro and GPT-3.5 fine tuning update

(05:50) - Using GPT-4 to generate training data for GPT-3.5 

(09:54) - Potential for training models on synthetic/cleaned data 

(10:24) - Using chain of thought prompts to improve model performance 

(15:02) - Sponsors: Netsuite | Omneky

(19:39) - Accelerating applications vs new AI models 

(20:40) - What to accelerate vs what to slow down

(26:07) - AI as a co-pilot vs fully automating tasks 

(30:12) - When to delegate to AI

(36:40)- Displacement of human roles by AI systems

(40:39) - Does training on synthetic data solve a problem?

(45:10) - The idea of an AI bundle/subscription 

(50:28) - Bundling in other industries like cable and SaaS

(54:27) - Churn and retention challenges for AI apps 

(01:02:54) - Low retention for easy-to-use AI apps

(01:03:57) - Incentives for AI companies to join a bundle

(01:04:39) - Potential for collaboration between AI companies 

(01:12:01) - Leading AI firms creating separate bundles

(01:16:43) - Outro

In this episode, Nathan sits down with three researchers at Carnegie Mellon studying adversarial attacks and mimetic initialization: Zico Kolter, Andy Zou, and Asher Trockman. They discuss: the motivation behind researching universal adversarial attacks on language models, how the attacks work, and the short term harms and long term risks of these jailbreaks. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

[00:00:00] - Introducing the podcast and guests Zico Kolter, Andy Zou, and Asher Trockman

[00:06:32] - Discussing the motivation and high-level strategy for the universal adversarial attack on language models

[00:09:33] - Explaining how the attacks work by adding nonsense tokens to maximize target sequence probability

[00:11:06] - Comparing to prior adversarial attacks in vision models

[00:13:47] - Details on the attack optimization process and discrete token search

[00:17:09] - The empirical notion of "mode switching" in the language models

[00:21:18] - Technical details on gradient computation across multiple models and prompts

[00:23:46] - Operating in one-hot vector space rather than continuous embeddings

[00:25:50] - Evaluating candidate substitutions across all positions to find the best update

[00:28:05] - Running the attack optimization for hundreds of steps across multiple GPUs

[00:39:14] - The difficulty of understanding the loss landscape and internal model workings

[00:43:55] - The flexibility afforded by separating the loss and optimization approach

[00:48:16] - The challenges of creating inherently robust models via adversarial training

[00:52:34] - Potential approaches to defense through filtering or inherent model robustness

[00:55:51] - Transferability results to commercial models like GPT-4 and Claude

[00:59:25] - Hypotheses on why the attacks transfer across different model architectures

[01:04:36] - The mix of human-interpretable and nonsense features in effective attacks

[01:08:29] - The appearance of intuitive manual jailbreak triggers in some attacks

[01:15:33] - Short-term harms of attacks vs long-term risks

[01:18:37] - Influencing those with incomplete understanding of LLMs to appreciate differences from human reasoning

[01:24:16] - Mitigating risks by training on filtered datasets vs broad web data

[01:2916] - Curriculum learning as a strategy for both capability and safety

[01:30:35] - Influencing developers building autonomous systems with LLMs

[01:33:19] - Alienness of LLM failure modes compared to human reasoning

[01:35:45] - Getting inspiration from biological visual system structure

[01:40:35] - Initialization as an alternative to pretraining for small datasets

[01:51:41] - Encoding useful structures like grammars in initialization without training

[02:12:10] - Most ideas don't progress to research projects

[02:13:02] - Pursuing ideas based on interest and feasibility

[02:15:14] - Fun of exploring uncharted territory in ML research

SPONSORS: NetSuite | Omneky

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: Stableaudio.com

In this episode, Nathan sits down with Adam Wenchel, CEO of Arthur.ai. Adam founded the AI security company back in 2019, before GPT-2 existed. In this episode, Adam shares his unique perspective on the AI security landscape, drawing from years building commercial AI systems. They discuss the attacks Adam set out to defend against, the changing priorities of executives in the rush to adopt LLMs, and the LLM-specific techniques Adam has developed. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS:

(00:00:00) Episode Preview

(00:03:45) Adam's background in AI and starting Arthur AI in 2019

(00:05:52) The release of ChatGPT as a watershed moment for generative AI

(00:07:09) Differences between traditional cybersecurity and AI security

(00:09:51) Early examples of AI security issues like boundary detection attacks in fraud systems

(00:12:39) - Mitigating risks of AI systems through observability and robust training

(00:14:40) - Financial services governance of AI models and its challenges today

(00:15:12) Sponsors: Netsuite | Omneky

(00:21:18) - Motivations for governance like staying compliant with regulations

(00:21:40) - The mix of incentives shaping earlier AI governance, like explainability

(00:28:14) - Using LMs to evaluate the security of other LMs

(00:30:03) - Dynamics between training and evaluating future LMs

(00:38:10) - The state of reasoning capabilities in large LMs

(00:44:35) - Corporate urgency around adopting generative AI technologies

(00:46:51) - Common enterprise use cases for generative AI and security concerns

(00:50:45) - Techniques for reducing hallucinations in retrieval augmented LMs

(00:53:15) - Benchmarking LMs on specific organizational tasks versus generic benchmarks

(00:56:30) - Metrics beyond accuracy like concision and hedging

(01:01:20) - Automatically detecting anomalies and hallucinations

(01:09:20) - Relationships between Arthur AI and foundation model providers

(01:11:52) - Where Cohere shines: multilingualism and not hedging

(01:13:43) - Anticipating future watershed moments and steady progress

(01:19:03) - Whether we can ever fully solve AI alignment and safety

LINKS:

Arthur.ai: https://www.arthur.ai/

X/Social:

@apwenchel (Adam)

@itsArthurAI (Arthur.ai)

@labenz (Nathan)

@eriktorenberg

@CogRev_Podcast

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: Stableaudio.com

In this episode, Nathan sits down with Stephen Parker and Josh Rubin of Waymark, and creators of The Frost, an AI-powered 12 minute short film. In this episode, we get a behind the scenes look at their creative process, the prompting and creative techniques they used to generate and animate the DALL-E results, and an overview of the current state of AI art. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

TIMESTAMPS

(00:00) Episode Preview

(00:01:00) Nathan’s introduction for Stephen Parker and Josh Rubin

(00:05:01) - The Frost is a 12-minute short film created using DALL-E 2 images.

(00:07:06) - The Frost started as an experiment to see if a narrative film could be created completely from AI imagery.

(00:10:38) - Parker started generating images with DALL-E 2 when he got access to the early preview.

(00:12:26) - Prompt technique to get consistent images by providing context about a hypothetical film.

(00:15:57) Sponsors: Netsuite | Omneky

(00:19:37) - Compositional continuity, like shot-reverse shot, was hard to achieve through prompting.

(00:22:13)- Rubin would request specific shots and the team would prompt DALL-E 2 to create them

(00:32:25) - Getting consistent facial features for characters was very difficult.

(00:39:03) - The storytelling helped cover inconsistencies that viewers might not notice.

(00:40:15) - Working with the images DALL-E provides

(00:44:53) Inpainting and compositing to refine DALL-E Images

(00:50:43) - The AI art is limited by what exists in the training data.

(01:02:05)- Animating the human characters was challenging because of missing or incorrect appendages.

(01:07:36) - The team had to find creative ways to convey emotion through the limited animation.

01:02:24 - Animating subtle human movement and emotion is still very difficult.

(01:06:35) - A romantic comedy would be much harder to produce with current AI capabilities.

(01:12:17) - For Frost 2 they are using text-to-video models like RunwayML.

(01:15:43) - AI voicing advancements applied to filmmaking

(01:19:27) - The future of AI in Hollywood and filmmaking: quality narratives still require human vision

SPONSORS: Shopify | NetSuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: GoogleLM