Two studies came out on the question of whether existing LLMs can help people figure out how to make bioweapons. RAND published a negative finding, showing no improvement. OpenAI found a small improvement, bigger for experts than students, from GPT-4. That's still harmless now, the question is what will happen in the future as capabilities advance.

Another news item was that Bard with Gemini Pro impressed even without Gemini Ultimate, taking the second spot on the Arena leaderboard behind only GPT-4-Turbo. For now, though, GPT-4 remains in the lead.

A third cool item was this story from a Russian claiming to have used AI extensively in his quest to find his one true love. I plan to cover that on its own and have Manifold on the job of figuring out how much of the story actually happened.

Table of Contents

---

Outline:

(00:56) Language Models Offer Mundane Utility

(08:35) Language Models Don’t Offer Mundane Utility

(09:38) GPT-4 Real This Time

(10:23) Be Prepared

(16:13) Fun with Image Generation

(17:17) Deepfaketown and Botpocalypse Soon

(23:32) They Took Our Jobs

(27:54) Get Involved

(28:23) In Other AI News

(33:35) Quiet Speculations

(39:06) The Quest for Sane Regulations

(45:58) The Week in Audio

(46:49) Rhetorical Innovation

(56:40) Predictions are Hard Especially About the Future

(01:03:13) Aligning a Smarter Than Human Intelligence is Difficult

(01:07:03) Open Model Weights Are Unsafe and Nothing Can Fix This

(01:09:38) Other People Are Not As Worried About AI Killing Everyone

(01:11:37) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

Before we begin, I will note that I have indeed written various thoughts about the three college presidents that appeared before Congress and the resulting controversies, including the disputes regarding plagiarism. However I have excluded them from this post.

Discipline and Prohibitions

Washington Post Editorial Board says schools should ban smartphones, and parents should help make this happen rather than more often opposing such bans in order to make logistical coordination easier.

I agree with the editorial board. Even when not in use, having a phone in one's pocket is a continuous distraction. The ability to use the phone creates immense social and other pressures to use it, or think about using it, continuously. If we are going to keep doing this physically required school thing at all, students need to be fully device-free during the school day except for where we intentionally want them to [...]

---

Outline:

(00:19) Discipline and Prohibitions

(05:04) School Choice

(08:34) An Argument Against School Choice

(13:10) Home School

(14:37) School Null Hypothesis Watch

(19:05) The Case Against Education, Pandemic Edition

(20:51) Early Childhood

(21:43) Primary School

(26:04) High School

(27:54) (Lack of) Standards

(31:41) College Grade Inflation

(33:41) College

(41:31) Student Debt

---

First published:

Source:

Narrated by TYPE III AUDIO.

While I was in San Francisco, the big head honchos headed for Davos, where AI was the talk of the town. As well it should be, given what will be coming soon. It did not seem like anyone involved much noticed or cared about the existential concerns. That is consistent with the spirit of Davos, which has been not noticing or caring about things that don’t directly impact your business or vibe since (checks notes by which I mean an LLM) 1971. It is what it is.

Otherwise we got a relatively quiet week. For once the scheduling worked out and I avoided the Matt Levine curse. I’m happy for the lull to continue so I can pay down more debt and focus on long term projects and oh yeah also keep us all farther away from potential imminent death.

Table of Contents

First published:

Source:

Narrated by TYPE III AUDIO.

There's always lots of stuff going on. The backlog of other roundups keeps growing rather than shrinking. I have also decided to hold back a few things to turn them into their own posts instead.

Bad News

I wonder if it is meaningful that most of the bad news is about technology?

I don’t even know if this is news, but Rutgers finds TikTok amplifies and suppresses content based on whether it aligns with the CCP.

It would be great if we could find a way to ban or stop using TikTok that did not involve something crazy like the Restrict Act. I still think the Restrict Act is worse than nothing, if those are our only choices.

If the CCP limited its interference to explicitly internal Chinese topics, I would understand, but they do not: WSJ investigates the TikTok rabbit hole, in particular [...]

---

Outline:

(00:18) Bad News

(07:53) Government Working (USA Edition)

(10:58) Government Working (UK/EU Edition)

(15:14) Trouble in the Suez

(22:42) Crime and Punishment

(34:03) Good News, Everyone

(39:11) Sports Go Sports

(43:24) Gamers Gonna Game Game Game Game Game

(50:02) Game Reviews

(53:00) I Was Promised Flying Self-Driving Cars

(53:31) While I Cannot Condone This

(01:02:48) Money Stuff

(01:14:23) At the Movies

---

First published:

Source:

Narrated by TYPE III AUDIO.

The biggest event of the week was the Sleeper Agents paper from Anthropic. I expect that to inform our thoughts for a while to come, and to lay foundation for additional work. We also had the first third of the IMO solved at almost gold metal level by DeepMind, discovering that math competition geometry is actually mostly composed of One Weird Trick. I knew that at the time I was doing it, though, and it was still really hard.

As usual, there was also a bunch of other stuff.

Tomorrow the 19th, I am going to be off to San Francisco for the weekend to attend a workshop. That leaves a lot of time for other events and seeing other people, a lot of which remains unfilled. So if you are interested in meeting up or want to invite me to a gathering, especially on Sunday the [...]

---

Outline:

(00:53) Language Models Offer Mundane Utility

(02:01) Language Models Don’t Offer Mundane Utility

(05:49) GPT-4 Real This Time

(07:05) Fun with Image Generation

(08:02) Copyright Confrontation

(09:43) Deepfaketown and Botpocalypse Soon

(11:31) They Took Our Jobs

(12:16) Get Involved

(13:11) Introducing

(17:36) In Other AI News

(22:48) Quiet Speculations

(25:30) The Quest for Sane Regulations

(31:50) The Week in Audio with Sam Altman

(35:03) David Brin Podcast

(45:46) Rhetorical Innovation

(48:59) Anthropic Paper on Sleeper Agents

(50:45) Anthropic Introduces Impossible Mission Force

(53:57) Aligning a Smarter Than Human Intelligence is Difficult

(58:57) The Belrose Model Continued

(01:23:46) Open Model Weights Are Unsafe And Nothing Can Fix This

(01:28:26) People Are Worried About AI Killing Everyone

(01:28:42) Other People Are Not As Worried About AI Killing Everyone

(01:31:15) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

The recent paper from Anthropic is getting unusually high praise, much of it I think deserved.

The title is: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.

Scott Alexander also covers this, offering an excellent high level explanation, of both the result and the arguments about whether it is meaningful. You could start with his write-up to get the gist, then return here if you still want more details, or you can read here knowing that everything he discusses is covered below. There was one good comment, pointing out some of the ways deceptive behavior could come to pass, but most people got distracted by the ‘grue’ analogy.

Right up front before proceeding, to avoid a key misunderstanding: I want to emphasize that in this paper, the deception was introduced intentionally. The paper deals with attempts to remove it.

The rest of this [...]

---

Outline:

(01:02) Abstract and Basics

(04:54) Threat Models

(07:01) The Two Backdoors

(10:13) Three Methodological Variations and Two Models

(11:30) Results of Safety Training

(13:08) Jesse Mu clarifies what he thinks the implications are.

(15:20) Implications of Unexpected Problems

(18:04) Trigger Please

(19:44) Strategic Deceptive Behavior Perhaps Not Directly in the Training Set

(21:54) Unintentional Deceptive Instrumental Alignment

(25:33) The Debate Over How Deception Might Arise

(38:46) Have We Disproved That Misalignment is Too Inefficient to Survive?

(47:52) Avoiding Reading Too Much Into the Result

(55:41) Further Discussion of Implications

(59:07) Broader Reaction

---

First published:

Source:

Narrated by TYPE III AUDIO.

Saving up medical and health related stories from several months allowed for much better organizing of them, so I am happy I split these off. I will still post anything more urgent on a faster basis. There's lots of things here that are fascinating and potentially very important, but I’ve had to prioritize and focus elsewhere, so I hope others pick up various torches.

Vaccination Ho!

We have a new malaria vaccine. That's great. WHO thinks this is not an especially urgent opportunity, or any kind of ‘emergency’ and so wants to wait for months before actually putting shots into arms. So what if we also see reports like ‘cuts infant deaths by 13%’? WHO doing WHO things, WHO Delenda Est and all that. What can we do about this?

Also, EA and everyone else who works in global health needs to do a complete post-mortem [...]

---

Outline:

(00:25) Vaccination Ho!

(02:57) Potential Progress

(07:05) It's Not Progress

(11:09) Cost Plus

(13:18) New Findings

(15:21) FDA Delenda Est

(20:57) Covid Response Postmortem and Paths Forward

(23:56) Covid Origins

(28:32) Ban Gain of Function Research

(28:51) Cause Areas

(30:26) I criticize Effective Altruists for insufficiently high levels of epistemic rigor, because reality does not grade on a curve, but let no one confuse them with governments.

(34:04) GLP-1 Has Barely Begun

(41:53) No One Understands Nutrition

(43:24) Model This: Exercise Edition

(49:00) A Bold Stand Against Torture

---

First published:

Source:

Narrated by TYPE III AUDIO.

My 2023 ACX predictions showed a clear lack of confidence in taking on the market. I won 30 markets for an average of +185 each, and lost 12 for an average loss of -185 each1. When one goes 30-12, hitting a 71% mark versus the about 58% average price initially paid, that is worth noticing. It is possible that I generally benefited from 2023 being a year where not much happened outside of AI, but I think it's time to know what we really think.

That means this year I’m going to add a phase, where I predict blind. Blind means I’m not allowed to look at any prediction markets. I can still look up facts, and financial markets are fair game, but nothing beyond that. Only after that will I look at Manifold. Metaculus makes this viable, as they have the ACX questions without listing probabilities.

---

Outline:

(01:47) International Politics

(11:13) American Electoral Politics

(20:48) US Politics and Government (excluding elections and AI )

(28:56) Economics

(37:10) Science (Mostly Rocketry for Some Reason)

(42:32) AI

(46:15) Hard Fork: Bonus Buy/Sell/Hold

---

First published:

Source:

Narrated by TYPE III AUDIO.

Developments around relationships and dating have a relatively small speed premium, also there are once again enough of them for a full post.

The first speculated on why you’re still single. We failed to settle the issue. A lot of you are indeed still single. So the debate continues.

You’re Single Because You’re Not Even Trying

What does it mean to not even be trying?

It does not only mean the things Alexander pointed us to last time, like 62% of singles being on zero dating apps, and a majority of singles having gone on zero dates in the past year, and a large majority not actively looking for a relationship. Here are those graphs again:

It also means things such as literally never approaching a woman in person.

Alexander (Keeper.ai): Why are so many young men single? Are they excluded from a [...]

---

Outline:

(00:25) You’re Single Because You’re Not Even Trying

(07:30) You’re Single Because of Artificial Intelligence

(10:38) You’re Single Because You Meet the Definition of Insanity

(18:52) You’re Single Because You’re Asking the Wrong Questions

(20:40) You’re Single Because of a Market Mismatch

(21:39) You’re Single Because Dating Apps Suck

(25:20) You’re Single Because You Didn’t Pay for Tinder Select

(39:24) You’re Single Because You Have Zero Friends

(43:14) You’re Single Because You Are Bad at Texting

(44:32) You’re Single Because You Have No Honor

(45:28) You’re Single Because You’re Not Hot

(51:23) You’re Single Because You Don’t Get Your House in Order

(52:55) You’re Single Because You’re Too Weird

(54:21) You’re Single Because of Your Bodycount

(59:34) You’re Single Because You Need to Learn Seduction

(01:08:12) You’re Single Because You Cannot Take a Hint

(01:10:05) You’re Single Because You Waste Your Money on Signaling

(01:12:27) You’re Still Single Because You Misalign Your Incentives

(01:13:03) You’re Not Single and You Are an Inspiration

(01:16:04) You’re Probably Not Single Stop Misrepresenting the Statistics

(01:16:35) You’re Single Because You’re Too Busy Writing Comments

(01:19:20) What About My Good Advice?

(01:21:09) Future Plans for this Series

---

First published:

Source:

Narrated by TYPE III AUDIO.

[NOTE: I forgot to post this to WP/LW/RSS on Thursday, so posting it now. Sorry about that.]

Will be very different from the old year by the time we are done. This year, it seems like various continuations of the old one. Sometimes I look back on the week, and I wonder how so much happened, while in other senses very little happened.

Table of Contents

---

Outline:

(00:28) Language Models Offer Mundane Utility

(02:44) Language Models Don’t Offer Mundane Utility

(06:13) GPT-4 Real This Time

(12:30) Liar Liar

(13:50) Fun with Image Generation

(19:09) Magic: The Generating

(23:57) Copyright Confrontation

(27:30) Deepfaketown and Botpocalypse Soon

(29:08) They Took Our Jobs

(37:39) Get Involved

(37:48) Introducing

(38:41) In Other AI News

(41:18) Quiet Speculations

(51:33) The Quest for Sane Regulations

(54:57) The Week in Audio

(55:10) AI Impacts Survey

(57:29) Rhetorical Innovation

(01:14:41) Aligning a Human Level Intelligence is Still Difficult

(01:17:00) Aligning a Smarter Than Human Intelligence is Difficult

(01:26:23) Won’t Get Fooled Again

(01:30:03) People Are Worried About AI Killing Everyone

(01:32:11) Other People Are Not As Worried About AI Killing Everyone

(01:34:55) The Wit and Wisdom of Sam Altman

(01:38:15) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.