We're excited to announce the inaugural Australian AI Safety Forum, taking place on November 7-8, 2024, in Sydney, Australia. This event aims to foster the growth of the AI safety community within Australia.

Apply now! 

Key Details

• Dates: November 7-8, 2024
• Location: Sydney Knowledge Hub, The University of Sydney
• Website: https://aisafetyforum.au

Description

The Australian AI Safety Forum is a two-day interdisciplinary event scheduled for November 2024. This forum, the first of its kind in Australia, aims to discuss perspectives on technical AI safety and governance, and explore Australia's unique role in the global AI safety landscape. The event will be anchored around the International Scientific Report on the Safety of Advanced AI, highlighting its key content and examining its implications for Australia.

The forum is designed to encourage conversation and education between researchers, policy makers, and industry leaders. As such, we aim to catalyse an Australian community working [...]

---

Outline:

(00:21) Apply now!

(00:25) Key Details

(00:40) Description

(01:34) Who Should Apply

(02:28) Format

(02:56) Organising Committee

(03:22) Partners

(03:41) Application Process

---

First published:

Source:

Narrated by TYPE III AUDIO.

Previously: The Fundamentals, The Gamblers

Having previously handled the literal gamblers, we are ready to move on to those who Do Business using Riverian principles.

Or at least while claiming to use Riverian principles, since Silicon Valley doesn’t fit into the schema as cleanly as many other groups. That's where we begin this section, starting at the highest possible conceptual level.

Time to talk real money.

Why Can You Do This Trade?

First law of trading: For you to buy, someone must sell. Or for you to sell, someone must buy. And there can’t be someone else doing the trade before you did it.

Why did they do that, and why did no one else take the trade first? Until you understand why you are able to do this trade, you should be highly suspicious.

“Every single thing we do, I can [...]

---

Outline:

(00:41) Why Can You Do This Trade?

(03:08) In a World of Venture Capital

(10:54) Short Termism Hypothesis

(12:42) Non-Determinism and its Discontents

(14:57) The Founder, the Fox and the Hedgehog

(17:11) The Team to Beat

(24:22) Silicon Valley Versus Risk

(35:14) The Keynesian Beauty Contest

(40:57) The Secret of Their Success is Deal Flow

(50:00) The Valley Beside the River

(53:07) Checkpoint Three

(53:37) Fun With SBF and Crypto Fraud

(01:01:53) Other Crypto Thoughts Unrelated to SBF

(01:04:50) Checkpoint Four

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Quarterly Performance Review, Autumn 1983

Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe his nerves. He only had one employee review left to get through, but he’d saved the hardest one for last.

He wasn’t upset about having to dismiss Lieutenant Colonel Petrov—he couldn’t wait to be rid of the little shit—but he couldn’t shake the feeling that something was amiss. He took a swig from his flask.

“Stanislav, you can come in now,” Yuri shouted as he opened the door and nearly smashed Stanislav Petrov in the face. “Have a seat,” he said.

“Yes, sir. Thank you sir. Overjoyed to be here as always,” Stanislav said.

“The purpose of this meeting is for us to discuss various concerns that have emerged about your performance over the past several months,” said Yuri. “Looking through your chart, what I’m seeing [...]

---

Outline:

(00:04) Quarterly Performance Review, Autumn 1983

(07:27) Quarterly Performance Review, Winter 1983

---

First published:

Source:

Narrated by TYPE III AUDIO.

This research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by @Stefan Heimershiem (Apollo Research). Find out more about the program and express interest in upcoming iterations here.

This video is a short overview of the project presented on the final day of the LASR Labs. Note that the paper was updated since then.

We study the effects of perturbing Transformer activations, building upon recent work by Gurnee, Lindsey, and Heimersheim & [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We interrupt Nate Silver week here at Don’t Worry About the Vase to bring you some rather big AI news: OpenAI and Sam Altman are planning on fully taking their masks off, discarding the nonprofit board's nominal control and transitioning to a for-profit B-corporation, in which Sam Altman will have equity.

We now know who they are and have chosen to be. We know what they believe in. We know what their promises and legal commitments are worth. We know what they plan to do, if we do not stop them.

They have made all this perfectly clear. I appreciate the clarity.

On the same day, Mira Murati, the only remaining person at OpenAI who in any visible way opposed Altman during the events of last November, resigned without warning along with two other senior people, joining a list that now includes among others several OpenAI [...]

---

Outline:

(01:51) Language Models Offer Mundane Utility

(04:17) Language Models Don’t Offer Mundane Utility

(06:35) The Mask Comes Off

(18:51) Fun with Image Generation

(18:54) Deepfaketown and Botpocalypse Soon

(19:50) They Took Our Jobs

(20:49) The Art of the Jailbreak

(21:28) OpenAI Advanced Voice Mode

(26:03) Introducing

(28:29) In Other AI News

(30:30) Quiet Speculations

(34:00) The Quest for Sane Regulations

(42:21) The Week in Audio

(42:47) Rhetorical Innovation

(56:50) Aligning a Smarter Than Human Intelligence is Difficult

(01:00:36) Other People Are Not As Worried About AI Killing Everyone

(01:01:53) The Lighter Side

The original text contained 6 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Today we honor the actions of Stanislav Petrov (1939 – 2017) once again.

Half an hour past midnight on September 26, 1983, he saw the first apparent launch on his computer monitor in a glass-walled room on the top floor of the Ballistic Missile Early Warning System (BMEWS) command and control post. The warning system was by now showing five missile launches in the U.S., headed toward the Soviet Union.

"The main computer wouldn't ask me [what to do] - it was made so that it wouldn't even ask. It was specially constructed in such a way that no one could affect the system's operations." All that was up to Petrov was analyzing the available information and either saying the alarm was false or giving the computer the go-ahead, as per the directive he himself wrote.

"I just couldn't believe that just like that, all of a sudden, someone [...]

The original text contained 4 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

2.1 Post summary / Table of contents

This is the second of a series of eight blog posts, which I’ll be serializing over the next month or two. (Or email or DM me if you want to read the whole thing right now.)

The previous post laid some groundwork for talking about intuitive self-models. Now we’re jumping right into the deep end: the intuitive concept of “conscious awareness” (or “awareness” for short). Some argue (§1.6.2) that if we can fully understand why we have an “awareness” concept, then we will thereby understand phenomenal consciousness itself! Alas, “phenomenal consciousness itself” is outside the scope of this series—again see §1.6.2. Regardless, the “awareness” concept is centrally important to how we conceptualize our own mental worlds, and well worth understanding for its own sake.

In one sense, “awareness” is nothing special: it's an intuitive concept, built like any other intuitive concept. I [...]

---

Outline:

(00:06) 2.1 Post summary / Table of contents

(05:06) 2.2 The “awareness” concept

(05:10) 2.2.1 The cortex has a finite computational capacity that gets deployed serially

(06:25) 2.2.2 Predictive learning represents that algorithmic property via a kind of abstract container called “awareness”

(07:44) 2.2.3 “S(apple)”, defined as the self-reflective thought “apple being in awareness”, is different from the object-level thought “apple”

(09:19) 2.3 Awareness over time: The “Stream of Consciousness” (“Cartesian Theater”)

(15:02) 2.4 Relation between “awareness” and memory

(15:07) 2.4.1 Intuitive model of memory as a storage archive

(16:10) 2.4.2 Intuitive connection between memory and awareness

(17:26) 2.5 The valence of S(X) thoughts

(18:52) 2.5.1 Positive-valence S(X) models often go with “what my best self would do” (other things equal)

(20:49) 2.5.2 Positive-valence S(X) models also tend to go with X's that are object-level motivating (other things equal)

(22:15) 2.6 S(A) as “the intention to immediately do action A”, and the rapid sequence \[S(A) ; A\] as the signature of a deliberate action

(22:26) 2.6.1 Clarification: Two ways to “think about an action”

(24:49) 2.6.2 For any action A where S(A) has positive valence, there's often a two-step temporal sequence: \[S(A) ; A actually happens\]

(26:42) 2.6.3 This two-step sequence corresponds to “deliberate” / “intentional” actions (as opposed to “spontaneously blurting something out”, “acting on instinct”, etc.)

(28:58) 2.6.4 The common temporal sequence above—i.e. \[S(A) with positive valence ; A actually happens\]—is itself incorporated into the intuitive self-model. Call it D(A) for “Deciding to do action A”

(29:35) 2.6.5 An application: “Illusions of free will”

(31:38) 2.7 Conclusion

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Previously: Book Review: On the Edge: The Fundamentals

As I said in the Introduction, I loved this part of the book. Let's get to it.

Poker and Game Theory

When people talk about game theory, they mostly talk solving for the equilibrium, and how to play your best game or strategy (there need not be a formal game) against adversaries who are doing the same.

I think of game theory like Frank Sinatra thinks of New York City: “If I can make it there, I’ll make it anywhere.” If you can compete against people performing at their best, you’re going to be a winner in almost any game you play. But if you build a strategy around exploiting inferior competition, it's unlikely to be a winning approach outside of a specific, narrow setting. What plays well in Peoria doesn’t necessarily play well in New York. [...]

---

Outline:

(00:18) Poker and Game Theory

(06:53) Sports Randomized Sports

(11:17) Knowing Theory Versus Memorization Versus Practice

(16:15) More About Tells

(19:20) Feeling the Probabilities

(20:35) Feeling Sad About It

(28:33) The Iowa Gambling Task

(31:39) The Greatest Risk

(37:20) Tournament Poker Is Super High Variance

(42:42) The Art of the Degen

(48:43) Why Do They Insist on Calling it Luck

(51:56) The Poker Gender Gap

(54:36) A Potential Cheater

(58:30) Making a Close Decision

(01:00:19) Other Games at the Casino

(01:03:22) Slot Machines Considered Harmful

(01:08:23) Where I Draw the Line

(01:11:14) A Brief History of Vegas and Casinos (as told by Nate Silver)

(01:16:44) We Got Us a Whale

(01:21:41) Donald Trump and Atlantic City Were Bad At Casinos

(01:25:17) How To Design a Casino

(01:26:46) The Wide World of Winning at Sports Gambling

(01:41:01) Limatime

(01:43:45) The Art of Getting Down

(01:45:29) Oh Yeah That Guy

(01:55:34) The House Sometimes Wins

(02:01:24) The House Is Probably a Coward

(02:11:19) DFS and The Problem of Winners

(02:16:08) Balancing the Action

(02:18:44) The Market Maker

(02:22:45) The Closing Line is Hard to Beat

(02:25:11) Winning is Hard

(02:29:58) What Could Be, Unburdened By What Has Been

(02:34:52) Finding Edges Big and Small

(02:40:12) Checkpoint Two

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Suppose you’ve trained a really clever AI model, and you’re planning to deploy it in an agent scaffold that allows it to run code or take other actions. You’re worried that this model is scheming, and you’re worried that it might only need to take a small number of actions to get to a dangerous and hard-to-reverse situation like exfiltrating its own weights. Problems that the AI can cause with small numbers of actions are particularly hard to address with standard safety techniques, because by default you are unlikely to notice the problem until the hard-to-reverse outcome has already occurred.

One of the main classes of techniques that you could apply in the hope of preventing these catastrophic actions is what we call untrusted monitoring (introduced in our paper here), which is roughly “before you execute a model's action, ask another instance of that model whether the proposed action [...]

---

Outline:

(05:51) A basic design for untrusted monitoring

(06:37) Untrusted monitoring during deployment

(11:32) Training your untrusted monitor

(13:31) How helpful would this be if the untrusted monitor did its best?

(14:51) Analyzing this under the adversarial assumption: can the model implement a conditional policy that only lies on real attacks?

(19:30) Rare failures aren’t an option for untrusted monitors

(20:41) How hard is it for a schemer to distinguish real examples of attacks from synthetic ones?

(21:03) Synthetic input contexts look obviously fake

(23:59) The monitor finds it naturally easy to distinguish synthetic attacks from real attacks

(27:45) Using a coordinated strategy that makes it easy to distinguish

(32:38) Preventing collusion under the assumption that the models can choose a coordination strategy

(36:04) A note on inference costs

(37:17) Conclusion

(39:19) Appendix: More on how helpful untrusted monitoring would be if the untrusted model did its best

(42:14) Appendix: A rule of thumb for estimating safety with execute/replace/audit protocols

The original text contained 4 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by Joseph Bloom (Decode Research). Find out more about the programme and express interest in upcoming iterations here.

This high level summary will be most accessible to those with relevant context including an understanding of SAEs. The importance of this work rests in part on the surrounding hype, and potential philosophical issues. We encourage readers seeking technical details to read the paper on arxiv.

TLDR: This is a short post summarising the key ideas and implications of our recent work studying how character information represented in language models is extracted by SAEs. Our most important result shows that SAE latents can appear to classify some feature of the input, but actually turn out to be quite unreliable classifiers (much worse than linear probes). We think this unreliability is in part due to difference [...]

The original text contained 1 footnote which was omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.