tl;dr: We compute the evolution of the local learning coefficient (LLC), a proxy for model complexity, for an algorithmic transformer. The LLC decreases as the model learns more structured solutions, such as head specialization.

This post is structured in three main parts, (1) a summary, giving an overview of the main results, (2) the Fine Print, that delves into various cross-checks and details and (3) Discussion and Conclusions.

Structure Formation in Algorithmic Transformers

In this work we study the development of simple algorithmic transformers, which are transformers that learn to perform algorithmic tasks. A major advantage of this setup is that we can control several (hyper)parameters, such as the complexity of the training data and network architecture. This allows us to do targeted experiments studying the impacts of these parameters on the learning dynamics. The main tool we use to study the development is the Local Learning Coefficient [...]

---

Outline:

(00:37) Structure Formation in Algorithmic Transformers

(02:17) 1-Head Model

(04:01) 2-Head Model

(06:03) The Fine Print

(06:30) 2-Head Model: Re-loaded

(06:35) Removing Layer Norm

(08:05) Removing Weight Decay

(09:33) Removing both Layer Norm and Weight Decay

(11:02) Adding Noise

(12:26) Number of vocabulary regions

(13:52) Adding More Heads

(16:14) Other algorithmic transformers

(16:41) Related work

(17:21) Discussion and Conclusions

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I've recently seen several articles here that were clearly generated or heavily assisted by AI. They are all dreadful. They are verbose, they are full of "on the one hand" and "on the other", they never make any assertion without also making room for the opposite, and end with "conclusions" that say nothing. Please do not do this.

————

I wrote the above, unassisted, and it states my own views. To illustrate the point I then had GPT-4o rewrite it. Here was the prompt:

I have recently seen a substantial increase in the proportion of postings to a certain online forum that were clearly generated by AI, although not marked as such. The writing is verbose and meandering, full of "on the one hand" and "on the other", never making an assertion without also suggesting its opposite, sounding oh-so-measured and even-handed that it ends up saying nothing at all. [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

the giga press

Tesla decided to use large aluminum castings ("gigacastings") for the frame of many of its vehicles, including the Model Y and Cybertruck. This approach and the "Giga Press" used for it have been praised by many articles and youtube videos, repeatedly called revolutionary and a key advantage.

Most cars today are made by stamping steel sheets and spot welding them together with robotic arms. Here's video of a Honda factory. But that's outdated: gigacasting is the future! BYD is still welding stamped steel sheets together, and that's why it can't compete on price with Tesla. Hold on, it seems...BYD prices are actually lower than Tesla's? Much lower? Oh, and Tesla is no longer planning single unitary castings for future vehicles?

I remember reading analysis from a couple people with car manufacturing experience, concluding that unitary cast aluminum bodies could [...]

---

Outline:

(00:06) the giga press

(01:51) aluminum die casting

(03:35) aluminum vs steel

(05:25) this shouldnt be a thing

(08:29) why was this a thing

---

First published:

Source:

Narrated by TYPE III AUDIO.

We wanted to share a recap of our recent outputs with the AF community. Below, we fill in some details about what we have been working on, what motivated us to do it, and how we thought about its importance. We hope that this will help people build off things we have done and see how their work fits with ours.

Who are we?

We’re the main team at Google DeepMind working on technical approaches to existential risk from AI systems. Since our last post, we’ve evolved into the AGI Safety & Alignment team, which we think of as AGI Alignment (with subteams like mechanistic interpretability, scalable oversight, etc.), and Frontier Safety (working on the Frontier Safety Framework, including developing and running dangerous capability evaluations). We’ve also been growing since our last post: by 39% last year [...]

---

Outline:

(00:32) Who are we?

(01:32) What have we been up to?

(02:16) Frontier Safety

(02:38) FSF

(04:05) Dangerous Capability Evaluations

(05:12) Mechanistic Interpretability

(08:54) Amplified Oversight

(09:23) Theoretical Work on Debate

(10:32) Empirical Work on Debate

(11:37) Causal Alignment

(12:47) Emerging Topics

(14:57) Highlights from Our Collaborations

(17:07) What are we planning next?

---

First published:

Source:

Narrated by TYPE III AUDIO.

We now likely know the final form of California's SB 1047.

There have been many changes to the bill as it worked its way to this point.

Many changes, including some that were just announced, I see as strict improvements.

Anthropic was behind many of the last set of amendments at the Appropriations Committee. In keeping with their “Support if Amended” letter, there are a few big compromises that weaken the upside protections of the bill somewhat in order to address objections and potential downsides.

The primary goal of this post is to answer the question: What would SB 1047 do?

I offer two versions: Short and long.

The short version summarizes what the bill does, at the cost of being a bit lossy.

The long version is based on a full RTFB: I am reading the entire bill, once again.

---

Outline:

(01:16) Short Version (tl;dr): What Does SB 1047 Do in Practical Terms?

(04:19) Really Short Abbreviated Version

(05:46) Somewhat Less Short: Things The Above Leaves Out

(08:03) Bad Model, Bad Model, What You Gonna Do

(11:34) Going to Be Some Changes Made

(14:17) Long Version: RTFB

(15:01) Definitions (starting with Artificial Intelligence)

(15:35) Safety Incident

(17:15) Covered Model

(19:15) Critical Harm

(22:10) Full Shutdown

(23:35) Safety and Security Protocol

(25:31) On Your Marks

(34:41) Reasonable People May Disagree

(42:27) Release the Hounds

(44:02) Smooth Operator

(46:47) Compute Cluster Watch

(48:49) Price Controls are Bad

(49:21) A Civil Action

(56:16) Whistleblowers Need Protections

(59:01) No Division Only Board

(01:02:32) Does CalCompute?

(01:03:06) In Which We Respond To Some Objections In The Style They Deserve

(01:04:43) False Claim: The Government Can and Will Lower the $100m Threshold

(01:05:11) False Claim: SB 1047 Might Retroactively Cover Existing Models

(01:05:32) Moot or False Claim: The Government Can and Will Set the Derivative Model Threshold Arbitrarily Low

(01:05:45) Objection: The Government Could Raise the Derivative Threshold Model Too High,

(01:06:11) False Claim: Fine-Tuners Can Conspire to Evade the Derivative

(01:06:31) Moot Claim: The Frontier Model Division Inevitably Will Overregulate

(01:07:01) False Claim: The Shutdown Requirement Bans Open Source

(01:07:49) Objection: SB 1047 Will Slow AI Technology and Innovation or Interfere with Open Source

(01:11:49) False Claim: This Effectively Kills Open Source Because You Can Fine-Tune Any System To Do Harm

(01:13:42) False Claim: SB 1047 Will Greatly Hurt Academia

(01:14:42) False Claim: SB 1047 Favors ‘Big Tech’ over ‘Little Tech’

(01:16:14) False Claim: SB 1047 Would Cause Many Startups To Leave California

(01:17:25) Objection: Shutdown Procedures Could Be Hijacked and Backfire

(01:18:35) Objection: The Audits Will Be Too Expensive

(01:20:08) Objection: What Is Illegal Here is Already Illegal

(01:23:52) Objection: Jailbreaking is Inevitable

(01:24:41) Moot and False Claim: Reasonable Assurance Is Impossible

(01:25:04) Objection: Reasonable Care is Too Vague, Can’t We Do Better?

(01:25:50) Objection: The Numbers Picked are Arbitrary

(01:27:47) Objection: The Law Should Use Capabilities Thresholds, Not Compute and Compute Cost Thresholds

(01:29:27) False Claim: This Bill Deals With ‘Imaginary’ Risks

(01:30:38) Objection: This Might Become the Model For Other Bills Elsewhere

(01:31:16) Not Really an Objection: They Changed the Bill a Lot

(01:31:42) Not Really an Objection: The Bill Has the Wrong Motivations and Is Backed By Evil People

(01:33:34) Not an Objection: ‘The Consensus Has Shifted’ or ‘The Bill is Unpopular’

(01:34:17) Objection: It Is ‘Too Early’ To Regulate

(01:35:33) Objection: We Need To ‘Get It Right’ and Can Do Better

(01:36:27) Objection: This Would Be Better at the Federal Level

(01:36:53) Objection: The Bill Should Be Several Distinct Bills

(01:37:44) Objection: The Bill Has Been Weakened Too Much in Various Ways

(01:40:18) Final Word: Who Should Oppose This Bill?

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post is a transcript of part of a podcast with Peter Thiel, touching on topics of AI, China, extinction, Effective Altruists, and apocalyptic narratives.

If you're interested in reading the quotes, just skip straight to them, the introduction is not required reading.

Introduction

Peter Thiel is probably known by most readers, but briefly: he is an venture capitalist, the first outside investor in Facebook, cofounder of Paypal and Palantir, and wrote Zero to One (a book I have found very helpful for thinking about building great companies). He has also been one of the primary proponents of the Great Stagnation hypothesis (along with Tyler Cowen).

More local to the LessWrong scene, Thiel was an early funder of MIRI and a speaker at the first Effective Altruism summit in 2013. He funded Leverage Research for many years, and also a lot of anti-aging research, and the seasteading initiative [...]

---

Outline:

(00:22) Introduction

(02:14) Artificial Intelligence

(15:36) Why Slow Progress on Nuclear Energy?

(19:19) Commentary

---

First published:

Source:

Narrated by TYPE III AUDIO.

In the past two years there has been increased interest in formal verification-based approaches to AI safety. Formal verification is a sub-field of computer science that studies how guarantees may be derived by deduction on fully-specified rule-sets and symbol systems. By contrast, the real world is a messy place that can rarely be straightforwardly represented in a reductionist way. In particular, physics, chemistry and biology are all complex sciences which do not have anything like complete symbolic rule sets. Additionally, even if we had such rules for the natural sciences, it would be very difficult for any software system to obtain sufficiently accurate models and data about initial conditions for a prover to succeed in deriving strong guarantees for AI systems operating in the real world.

Practical limitations like these on formal verification have been well-understood for decades to engineers and applied mathematicians building real-world software systems, which makes [...]

---

Outline:

(01:23) What do we Mean by Formal Verification for AI Safety?

(12:13) Challenges and Limitations

(37:58) What Can Be Hoped-For?

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I’m convinced that people who are interested in large language models (LLMs) are overwhelmingly focused on general-purpose “performance” at the expense of exploring useful (or fun) applications.

As I’m working on a personal project, I’ve been learning my way around HuggingFace, which is a hosting platform, set of libraries, and almost-social-network for the open-source AI community. It's fascinating, and worth exploring even if you’re not going to be developing foundation models from scratch yourself; if you simply want to use the latest models, build apps around them, or adapt them slightly to your own purposes, HuggingFace seems like the clear place to go.

You can look at trending models, and trending public “spaces”, aka cloud-hosted instances of models that users can test out, and get a sense of where the “energy” is. And what I see is that almost all the “energy” [...]

---

Outline:

(02:46) A Personalized Information Environment

(03:50) Personalized Content Filter

(04:53) Color-Coded Text

(06:17) Fact Extraction

(07:31) Plain Language

(08:14) Dispute Resolution and Mediation

(09:19) AITA For Everything

(10:37) Chat Moderator

(12:28) Rethinking “Online”

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Introduction

There are now (claimed) foundation models for protein sequences, DNA sequences, RNA sequences, molecules, scRNA-seq, chromatin accessibility, pathology slides, medical images, electronic health records, and clinical free-text. It's a dizzying rate of progress.

But there's a few problems in biology that, interestingly enough, have evaded a similar level of ML progress, despite there seemingly being all the necessary conditions to achieve it.

Toxicology is one of those problems.

This isn’t a new insight, it was called out in one of Derek Lowe's posts, where he said: There are no existing AI/ML systems that mitigate clinical failure risks due to target choice or toxicology. He also repeats it in a more recent post: ‘…the most badly needed improvements in drug discovery are in the exact areas that are most resistant to AI and machine learning techniques. By which I mean target selection and predictive toxicology.’ [...]

---

Outline:

(00:18) Introduction

(02:09) Some background

(06:21) The hard stuff

(06:24) The relevance of toxicity datasets to the clinical problem

(11:38) Methodological problems in toxicity datasets

(12:39) Intraspecies toxicity variability

(18:14) Toxicity synergism

(22:00) Conclusion

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The standard operating procedure against a primitive forest-dwelling enemy is to retreat to orbit and then bombard the planet until there isn't a forest anymore. However, the only reason the Galactic Empire was in the Alpha Centauri A system in the first place was because of the fragile mineral resources underneath that forest. Dropping tungsten rods at hypersonic speeds would risk destroying the only thing of value on Pandora.

Alien aborigines armed with bows and arrows damaged Imperial legitimacy across the galaxy. It was like losing a battle to Ewoks.

The Emperor's solution had been to hire an Ewok.

Mitth'raw'nuruodo was a dwarf by Na'vi standards, but the blue alien stood a head above most humans. Originally hired as a translator, the Imperials on Pandora quickly noticed that the only patrols who came back alive were those that followed Mitth'raw'nuruodo's advice. Pretty soon, the moon was [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.