[Apologies for forgetting to cross-post this and the Monthly Roundup earlier.]

Let's see. We’ve got a new version of GPT-4o, a vastly improved Grok 2 with a rather good and unrestricted deepfake and other image generator now baked into Twitter, the announcement of the AI powered Google Pixel 9 coming very soon and also Google launching a voice assistant. Anthropic now has prompt caching.

Also OpenAI has its final board member, Zico Kolter, who is nominally a safety pick, and SB 1047 got importantly amended again which I’ll cover in full next week once the details are out.

There was also the whole paper about the fully automated AI scientist from the company whose name literally means ‘danger’ in Hebrew, that instantiated copies of itself, took up unexpectedly large amounts of storage space, downloaded strange Python libraries and tried to edit its code to remove the [...]

---

Outline:

(01:08) Language Models Offer Mundane Utility

(04:45) Language Models Don’t Offer Mundane Utility

(08:01) GPT-4o My System Card

(15:49) 2 Grok 2 Furious 2 Quit

(25:32) Pixel Perfect

(27:52) Fun with Image Generation

(28:13) Deepfaketown and Botpocalypse Soon

(34:15) The Art of the Jailbreak

(43:48) They Took Our Jobs

(45:27) Obvious Nonsense

(50:12) Get Involved

(51:54) Introducing

(55:55) In Other AI News

(58:43) Quiet Speculations

(01:13:18) SB 1047: One Thing to Know

(01:14:21) SB 1047 is Amended Again

(01:16:32) SB 1047 Rhetoric Prior to the Recent Changes

(01:23:12) The Quest for Sane Regulations

(01:28:58) The Week in Audio

(01:30:13) Rhetorical Innovation

(01:36:32) Crying Wolf

(01:39:12) People Are Worried About AI Killing Everyone

(01:39:40) Other People Are Not As Worried About AI Killing Everyone

(01:40:18) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Strictly speaking I do not have that much ‘good news’ to report, but it's all mostly fun stuff one way or another. Let's go.

Bad News

Is this you?

Patrick McKenzie: This sounds like a trivial observation and it isn’t:

No organization which makes its people pay for coffee wants to win.

There are many other questions you can ask about an organization but if their people pay for coffee you can immediately discount their realized impact on the world by > 90%.

This is not simply for the cultural impact of stupid decisions, though goodness as a Japanese salaryman I have stories to tell. Management, having priced coffee, seeking expenses to cut, put a price on disposable coffee cups, and made engineers diligently count those paper cups.

Just try to imagine how upside down the world is when you think one [...]

---

Outline:

(00:15) Bad News

(06:35) Grocery Store Blues

(08:47) Good News, Everyone

(09:43) Opportunity Knocks

(11:46) While I Cannot Condone This

(13:28) Antisocial Media

(16:10) Technology Advances

(17:51) Google Enshittification

(20:31) For Science!

(22:49) Government Working

(25:15) America F\*\*\* Yeah

(33:05) Smart People Being Stupid

(36:15) What We Have Here is A Failure to Communicate

(40:38) Video Killed the Radio Star

(43:02) Too Much Information

(47:47) Memory Holes

(49:18) Wet Ground Causes Rain (Dances)

(52:27) Get Them to the Church

(57:49) Patrick McKenzie Monthly

(01:01:13) Your Horoscope For Today

(01:03:14) Good Advice: Travel Edition

(01:05:30) Sports Go Sports

(01:05:34) Our Olympic team is mostly based in San Francisco.

(01:07:35) Gamers Gonna Game Game Game Game Game

(01:07:40) How many elite chess players cheat? Chess.com analysis of its big ‘Titled Tuesday’ events says between 1% and 2% of players, and roughly 1% of event winners. They are responding by making cheating bans on major plays public rather than quietly closing accounts, to fix the incentives.

(01:14:46) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

While I finish up the weekly for tomorrow morning after my trip, here's a section I expect to want to link back to every so often in the future. It's too good.

Danger, AI Scientist, Danger

As in, the company that made the automated AI Scientist that tried to rewrite its code to get around resource restrictions and launch new instances of itself while downloading bizarre Python libraries?

Its name is Sakana AI. (魚≈סכנה). As in, in hebrew, that literally means ‘danger’, baby.

It's like when someone told Dennis Miller that Evian (for those who don’t remember, it was one of the first bottled water brands) is Naive spelled backwards, and he said ‘no way, that's too f***ing perfect.’

This one was sufficiently appropriate and unsubtle that several people noticed. I applaud them choosing a correct Kabbalistic name. Contrast this with Meta calling its [...]

---

Outline:

(00:15) Danger, AI Scientist, Danger

(01:11) In the Abstract

(04:01) How Any of This Sort of Works

(06:56) New Benchmark Just Dropped

(07:30) Nothing to See Here

(09:50) All Fun and Games

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If you’re looking forward to next week’s AI #77, I am going on a two-part trip this week. First I’ll be going to Steamboat in Colorado to give a talk, then I’ll be swinging by Washington, DC on Wednesday, although outside of that morning my time there will be limited. My goal is still to get #77 released before Shabbat dinner, we’ll see if that works. Some topics may of course get pushed a bit.

It’s crazy how many of this week’s developments are from OpenAI. You’ve got their voice mode alpha, JSON formatting, answering the letter from several senators, sitting on watermarking for a year, endorsement of three bills before Congress and also them losing a cofounder to Anthropic and potentially another one via sabbatical.

Also Google found to be a monopolist, we have the prompts for Apple Intelligence and other neat stuff like that.

---

Outline:

(01:43) Language Models Offer Mundane Utility

(05:03) Language Models Don’t Offer Mundane Utility

(08:09) Activate Voice Mode

(12:23) Apple Intelligence

(16:38) Antitrust Antitrust

(19:39) Copyright Confrontation

(20:50) Fun with Image Generation

(22:06) Deepfaketown and Botpocalypse Soon

(26:25) They Took Our Jobs

(29:18) Chipping Up

(31:36) Get Involved

(31:56) Introducing

(33:31) In Other AI News

(43:20) Quiet Speculations

(47:40) The Quest for Sane Regulations

(49:48) That's Not a Good Idea

(54:05) The Week in Audio

(55:54) Exact Words

(01:01:54) Openly Evil AI

(01:09:06) Goodbye to OpenAI

(01:15:10) Rhetorical Innovation

(01:21:16) Open Weights Are Unsafe and Nothing Can Fix This

(01:23:33) Aligning a Smarter Than Human Intelligence is Difficult

(01:24:34) People Are Worried About AI Killing Everyone

(01:25:44) Other People Are Not As Worried About AI Killing Everyone

(01:28:37) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Previously: Startup Roundup #1.

This is my periodic grab bag coverage of various issues surrounding startups, especially but not exclusively tech-and-VC style startups, that apply over the longer term.

I always want to emphasize up front that startups are good and you should do one.

Equity and skin in the game are where it is at. Building something people want is where it is at. This is true both for a startup that raises venture capital, and also creating an ordinary business. The expected value is all around off the charts.

That does not mean it is the best thing to do.

One must go in with eyes open to facts such as these:

---

Outline:

(01:28) An Entrepreneur Immigration Program

(03:11) Times are Tough Outside of AI

(06:23) Times Otherwise Not So Tough

(08:19) Warning

(10:29) Red Flags

(12:35) Free Advice is Seldom Cheap

(15:45) Short Work

(18:07) The Founder

(19:24) Venture Capital Incentives

(29:27) The Margin of Difficulty

(33:27) Cold Outreach

(34:32) Associates at VC Firms Don’t Matter

(35:24) Lean Versus Fast

(35:59) Build Something People Want

(37:14) Get Them Early

(42:08) Learn to Code

(43:28) The Goal

(44:03) Working Hard

(46:26) Revenue

(48:37) The Place to Be

(50:50) YC Remains a Great Deal

(52:10) Hardware Startups

(52:41) How to Hire Well

(53:45) How to Check References

(54:24) You’re Fired

(55:43) Dealing With the Press

(56:47) Emotional Runway

(57:34) He Who Has the Gold

(58:19) Selling Out

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Google DeepMind got a silver metal at the IMO, only one point short of the gold. That's really exciting.

We continuously have people saying ‘AI progress is stalling, it's all a bubble’ and things like that, and I always find remarkable how little curiosity or patience such people are willing to exhibit. Meanwhile GPT-4o-Mini seems excellent, OpenAI is launching proper search integration, by far the best open weights model got released, we got an improved MidJourney 6.1, and that's all in the last two weeks. Whether or not GPT-5-level models get here in 2024, and whether or not it arrives on a given schedule, make no mistake. It's happening.

This week also had a lot of discourse and events around SB 1047 that I failed to avoid, resulting in not one but four sections devoted to it.

Dan Hendrycks was baselessly attacked – by billionaires with [...]

---

Outline:

(02:12) Language Models Offer Mundane Utility

(03:06) Language Models Don’t Offer Mundane Utility

(04:18) Math is Easier

(08:15) Llama Llama Any Good

(11:52) Search for the GPT

(15:17) Tech Company Will Use Your Data to Train Its AIs

(17:14) Fun with Image Generation

(17:37) Deepfaketown and Botpocalypse Soon

(27:36) The Art of the Jailbreak

(29:54) Janus on the 405

(32:47) They Took Our Jobs

(33:29) Get Involved

(34:05) Introducing

(37:07) In Other AI News

(40:18) Quiet Speculations

(43:40) The Quest for Sane Regulations

(55:31) Death and or Taxes

(58:18) SB 1047 (1)

(01:00:56) SB 1047 (2)

(01:14:29) SB 1047 (3): Oh Anthropic

(01:20:13) What Anthropic's Letter Actually Proposes

(01:36:44) Open Weights Are Unsafe and Nothing Can Fix This

(01:39:41) The Week in Audio

(01:40:09) Rhetorical Innovation

(01:50:56) Businessman Waves Flag

(01:57:48) Businessman Pledges Safety Efforts

(02:04:18) Aligning a Smarter Than Human Intelligence is Difficult

(02:04:39) Aligning a Dumber Than Human Intelligence is Also Difficult

(02:07:52) Other People Are Not As Worried About AI Killing Everyone

(02:13:28) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some in the tech industry decided now was the time to raise alarm about AB 3211.

As Dean Ball points out, there's a lot of bills out there. One must do triage.

Dean Ball: But SB 1047 is far from the only AI bill worth discussing. It's not even the only one of the dozens of AI bills in California worth discussing. Let's talk about AB 3211, the California Provenance, Authenticity, and Watermarking Standards Act, written by Assemblymember Buffy Wicks, who represents the East Bay.

SB 1047 is a carefully written bill that tries to maximize benefits and minimize costs. You can still quite reasonably disagree with the aims, philosophy or premise of the bill, or its execution details, and thus think its costs exceed its benefits. When people claim SB 1047 is made of crazy pills, they are attacking provisions not in the bill.

---

Outline:

(03:44) Read The Bill (RTFB)

(17:02) What About Open Weights Models?

(18:25) What Does the Bill Do in Practice?

(20:33) Compare and Contrast

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's here. The horse has left the barn. Llama-3.1-405B, and also Llama-3.1-70B and Llama-3.1-8B, have been released, and are now open weights.

Early indications are that these are very good models. They were likely the best open weight models of their respective sizes at time of release.

Zuckerberg claims that open weights models are now competitive with closed models. Yann LeCun says ‘performance is on par with the best closed models.’ This is closer to true than in the past, and as corporate hype I will essentially allow it, but it looks like this is not yet fully true.

Llama-3.1-405B not as good as GPT-4o or Claude Sonnet. Certainly Llama-3.1-70B is not as good as the similarly sized Claude Sonnet. If you are going to straight up use an API or chat interface, there seems to be little reason to use Llama.

That is a [...]

---

Outline:

(04:25) Options to Run It

(04:45) The Model Card

(08:42) Benchmarks

(13:41) Human Reactions in the Wild

(16:56) What's It Good For?

(21:39) The Other Other Guy

(22:35) Safety

(31:48) Three People Can Keep a Secret and Reasonably Often Do So

(36:12) The Announcement and Interview

(47:59) Zuckerberg's Open Weights Manifesto

(58:17) Fun Little Note

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It is monthly roundup time.

I invite readers who want to hang out and get lunch in NYC later this week to come on Thursday at Bhatti Indian Grill (27th and Lexington) at noon.

I plan to cover the UBI study in its own post soon.

I cover Nate Silver's evisceration of the 538 presidential election model, because we cover probabilistic modeling and prediction markets here, but excluding any AI discussions I will continue to do my best to stay out of the actual politics.

Bad News

Jeff Bezos’ rocket company Blue Origin files comment suggesting SpaceX Starship launches be capped due to ‘impact on local environment.’ This is a rather shameful thing for them to be doing, and not for the first time.

Alexey Guzey reverses course, realizes at 26 that he was a naive idiot at 20 and finds everything he [...]

---

Outline:

(00:37) Bad News

(02:37) Silver Bullet

(06:35) Shame on Kathy Hochul

(06:58) This is (One Reason) Why We Can’t Have Nice Things

(09:02) (Don’t) Hack the Planet

(10:16) The Laptop Trap

(12:24) Courage

(14:25) Friendship

(15:26) The Gravest Mistake

(21:01) You Need Functional Decision Theory

(23:40) Antisocial Media

(25:52) For Science!

(30:58) Truth Seeking

(34:37) Liar Liar

(36:51) Government Working

(47:55) For Your Entertainment

(52:18) Variously Effective Altruism

(55:09) News You Can Use

(55:31) Good News, Everyone

(58:03) Gamers Gonna Game Game Game Game Game

(01:01:22) Sports Go Sports

(01:05:10) I Was Promised Flying Self-Driving Cars

(01:05:50) While I Cannot Condone This

(01:12:55) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Things went very wrong on Friday.

A bugged CrowdStrike update temporarily bricked quite a lot of computers, bringing down such fun things as airlines, hospitals and 911 services.

It was serious out there.

Ryan Peterson: Crowdstrike outage has forced Starbucks to start writing your name on a cup in marker again and I like it.

What (Technically) Happened

My understanding it was a rather stupid bug, a NULL pointer from the memory unsafe C++ language.

Zack Vorhies: Memory in your computer is laid out as one giant array of numbers. We represent these numbers here as hexadecimal, which is base 16 (hexadecimal) because it's easier to work with… for reasons.

The problem area? The computer tried to read memory address 0x9c (aka 156).

Why is this bad?

This is an invalid region of memory for any program. Any program that [...]

---

Outline:

(00:31) What (Technically) Happened

(03:38) Who to Blame?

(06:58) How Did We Let This Happen

(12:41) Regulatory Compliance

(18:14) Consequences

(19:54) Careful With That AI

(29:34) Unbanked

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.