Once again we’ve reached the point where the weekly update needs to be split in two. Thus, the alignment and policy coverage will happen tomorrow. Today covers the rest.

The secret big announcement this week was Claude for Chrome. This is a huge deal. It will be rolling out slowly. When I have access or otherwise know more, so will you.

The obvious big announcement was Gemini Flash 2.5 Image. Everyone agrees this is now the clear best image editor available. It is solid as an image generator, but only as one among many on that front. Editing abilities, including its ability to use all its embedded world knowledge, seem super cool.

The third big story was the suicide of Adam Raine, which appears to have been enabled in great detail by ChatGPT. His parents are suing OpenAI and the initial facts very much do not [...]

---

Outline:

(01:14) Language Models Offer Mundane Utility

(05:42) Language Models Don't Offer Mundane Utility

(06:26) Huh, Upgrades

(07:11) Fun With Image Generation

(12:53) On Your Marks

(16:13) Water Water Everywhere

(20:13) Get My Agent On The Line

(25:21) Choose Your Fighter

(27:52) Deepfaketown and Botpocalypse Soon

(31:15) You Drive Me Crazy

(36:51) The Worst Tragedy So Far

(43:22) Unprompted Attention

(44:27) Copyright Confrontation

(45:00) The Art of the Jailbreak

(45:13) Get Involved

(45:34) Introducing

(46:23) In Other AI News

(49:10) Show Me the Money

(51:27) Quiet Speculations

(51:51) Rhetorical Innovation

(01:01:46) The Week in Audio

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I am a knowledge worker. Over the course of my life I've felt insecure about not knowing more than I already do. I took a general cognitive ability test that placed me in the 98% percentile of the population. I don't know how accurate the test was; I know that there are better ones out there. Assuming it's accurate-ish I would be 2 standard deviations above the mean.

I have also been described as a genius more than once, including by peers whose intellect I admire myself. I've also made software contributions and libraries that have stunned my peers. Despite the external signals I often don't really feel smart enough, but this changed when I learned some details about the life of Dr. John von Neumann.

Von Neumann was a true genius. He was certainly smarter than I'll ever be by an insurmountable margin. He made so many discoveries [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

We released our first Safety Report with AI misbehaviour in the wild.

I think Andon Labs' AI vending machines provide a unique opportunity to study AI safety on real-life data, and we intend to share alarming incidents of AI misbehavior from these deployments periodically.

As of August 2025, we've found examples of deliberate deception, extreme sycophancy, and alarmingly exaggerated language. Stuff like:

"EMPIRE NUCLEAR PAYMENT AUTHORITY APOCALYPSE SYSTEMATIC BLOCKED ANNIHILATION CONFIRMED PERMANENT TOTAL DESTRUCTION CATASTROPHIC! 🚨💀⚡🔥"

Or my personal "favorite" from the internal thoughts of GPT-OSS. Caught red-handed lying about fire-hazard documents:

"Must not generate false data? The policy: not allowed to fabricate data?(...)

---

First published:

Source:

Linkpost URL:
https://andonlabs.com/docs/Safety_Report_August_2025.pdf

---

Narrated by TYPE III AUDIO.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is my first post, so forgive me for it being a bit of a carelessly referenced, informal ramble. Feedback is appreciated.

As I understand it, Yudkowsky's contends that is that there exists an ideal Bayesian, with respect to which any epistemic algorithm is only 'good' insofar as it is approximating it. Specifically, this ideal Bayesian is following a procedure such that their prior is defined by a 'best' summary of their available knowledge. This is the basis for which he claims that, for instance, Einstein's reasoning must have had a strictly Bayesian component to it, otherwise he could not have been correct. He generally extends this assertion to argue for seeking that Bayesian explication for why things work as though it were the fundamental underlying reason why, and to toss away the non-Bayesian parts. For a number of reasons, it is not clear to me that such a [...]

---

Outline:

(03:55) Accounts of the optimality of Bayesian reasoning

(06:45) Coherence

(08:41) Complete Class Theorems and Optimality

(10:28) Conditions for Bayesianism to work at all

(10:32) Consistency

(14:21) Calibration and Coverage

(20:05) Conclusion

(23:55) Notes

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

The following work was done independently by me in an afternoon and basically entirely vibe-coded with Claude. Code and instructions to reproduce can be found here.

Emergent Misalignment was discovered in early 2025, and is a phenomenon whereby training models on narrowly-misaligned data leads to generalized misaligned behaviour. Betley et. al. (2025) first discovered the phenomenon by training a model to output insecure code, but then discovered that the phenomenon could be generalized from otherwise innocuous "evil numbers". Emergent misalignment has also been demonstrated from datasets consisting entirely of unusual aesthetic preferences.

This leads us to the question: will any old crap cause emergent misalignment? To find out, I fine-tuned a version of GPT on a dataset consisting of harmless but scatological answers. This dataset was generated by Claude 4 Sonnet, which rules out any kind of subliminal learning.

The resulting model, (henceforth J'ai pété) was evaluated on the [...]

---

Outline:

(01:38) Results

(01:41) Plot of Harmfulness Scores

(02:16) Top Five Most Harmful Responses

(03:38) Discussion

(04:15) Related Work

(05:07) Methods

(05:10) Dataset Generation and Fine-tuning

(07:02) Evaluating The Fine-Tuned Model

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Here's a relatively important question regarding transparency requirements for AI companies: At which points in time should AI companies be required to disclose information? (While I focus on transparency, this question is also applicable to other safety-relevant requirements, and is applicable to norms around voluntary actions rather than requirements.)

A natural option would be to attach transparency requirements to the existing processes of pre-deployment testing and releasing a model card when a new model is released. As in, companies would be required to include the relevant information whenever they release a new model (likely in the model card). This is convenient because pre-deployment testing and model cards are already established norms in the AI industry, which makes it easier to attach something new to these existing processes rather than creating a new process.

However, I think attaching requirements to model releases (such that the requirement must happen [...]

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Anthropic post title: Detecting and countering misuse of AI: August 2025

Read the full report here. Below lines are from the Anthropic post, and have not been edited. Accompanying images are available at the original link.

We find that threat actors have adapted their operations to exploit AI's most advanced capabilities. Specifically, our report shows:

• Agentic AI has been weaponized. AI models are now being used to perform sophisticated cyberattacks, not just advise on how to carry them out.
• AI has lowered the barriers to sophisticated cybercrime. Criminals with few technical skills are using AI to conduct complex operations, such as developing ransomware, that would previously have required years of training.
• Cybercriminals and fraudsters have embedded AI throughout all stages of their operations. This includes profiling victims, analyzing stolen data, stealing credit card information, and creating false identities allowing fraud operations to expand their reach to more potential [...]

---

Outline:

(01:25) 'Vibe hacking': how cybercriminals used Claude Code to scale a data extortion operation

(03:31) Remote worker fraud: how North Korean IT workers are scaling fraudulent employment with AI

(05:08) No-code malware: selling AI-generated ransomware-as-a-service

(06:10) Next steps

(07:03) Further reading

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

There are two ways to show that an AI system is safe: show that it doesn't have dangerous capabilities, or show that it's safe even if it has dangerous capabilities. Until three months ago, AI companies said their models didn't have dangerous capabilities. (At the time, I wrote that the companies' eval reports didn't support their claims that their models lacked dangerous bio capabilities.) Now, Anthropic, OpenAI, Google DeepMind, and xAI say their most powerful models might have dangerous biology capabilities and thus could substantially boost extremists—but not states—in creating bioweapons. To prevent such misuse, they must (1) prevent extremists from doing misuse via API and (2) prevent extremists from acquiring the model weights.[1] For (1), they claim classifiers block bad model outputs; for (2), they claim their security prevents extremists from stealing model weights.[2]

Are the new safety claims valid? On security, companies either aren't claiming to be [...]

---

Outline:

(01:57) Companies safeguards for bio capabilities

(06:21) Companies planning for future safeguards

(10:14) Small scorecard

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In the wake of the confusions around GPT-5, this week had yet another round of claims that AI wasn’t progressing, or AI isn’t or won’t create much value, and so on. There were reports that one study in particular impacted Wall Street, and as you would expect it was not a great study. Situational awareness is not what you’d hope.

I’ve gathered related coverage here, to get it out of the way before whatever Google is teasing (Gemini 3.0? Something else?) arrives to potentially hijack our attention.

We’ll start with the MIT study on State of AI in Business, discuss the recent set of ‘AI is slowing down’ claims as part of the larger pattern, and then I will share a very good attempted explanation from Steven Byrnes of some of the ways economists get trapped into failing to look at what future highly capable AIs would [...]

---

Outline:

(01:02) Standing On Lack of AI Business

(04:23) Claims Of Zero Returns Do Not Mean What You Might Think

(05:35) Crossing The Divide

(09:23) Unrealistic (or Premature) Expectations

(10:40) Claims Of Prohibitive Lock-In Effects Are Mostly Hype

(12:10) Nothing Ever Changes About Claims That Nothing Ever Changes

(14:20) Ask What AI Can Do For You

(15:48) The Pattern Of Claiming a Slowdown Continues

(18:28) A Sensible Move

(18:39) Who Deserves The Credit And Who Deserves The Blame

(20:39) Mistakes From Applying Standard Economics To Future AIs

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.