Say you want to plot some data. You could just plot it by itself:

Or you could put lines on the left and bottom:

Or you could put lines everywhere:

Or you could be weird:

Which is right? Many people treat this as an aesthetic choice. But I’d like to suggest an unambiguous rule.

Principles

First, try to accept that all axis lines are optional. I promise that readers will recognize a plot even without lines around it.

So consider these plots:

Which is better? I claim this depends on what you’re plotting. To answer, mentally picture these arrows:

Now, ask yourself, are the lengths of these arrows meaningful? When you draw that horizontal line, you invite people to compare those lengths.

You use the same principle for deciding if you should draw a y-axis line. As yourself [...]

---

Outline:

(00:37) Principles

(01:23) Years vs. GDP

(02:23) Years vs. GDP again

(03:27) Years vs temperature

(04:22) Homeowners vs. cannabis

(05:04) Lines can also be used for evil

(06:50) Related advice

(07:31) Axis lines ≠ tick marks

(07:52) Matplotlib's tragic defaults

---

First published:

Source:

Linkpost URL:
https://dynomight.net/axes/

Narrated by TYPE III AUDIO.

A sparse autoencoder is a neural network architecture that has recently gained popularity as a technique to find interpretable features in language models (Cunningham et al, Anthropic's Bricken et al). We train a sparse autoencoder on OthelloGPT, a language model trained on transcripts of the board game Othello, which has been shown to contain a linear representation of the board state, findable by supervised probes. The sparse autoencoder finds 9 features which serve as high-accuracy classifiers of the board state, out of 180 findable with supervised probes (and 192 possible piece/position combinations). Across random seeds, the autoencoder repeatedly finds “simpler” features concentrated on the center of the board and the corners. This demonstrates that current techniques for sparse autoencoders may fail to find a large majority of the interesting, interpretable features in a language model.

There has been a recent [...]

---

Outline:

(02:56) Overview

(03:39) Methods

(03:42) Training OthelloGPT

(04:50) Training Linear Probes

(08:03) Training The Sparse Autoencoder

(09:08) Results

(09:11) SAE Features as Current-Board Classifiers

(12:33) Which Features are Learned?

(14:39) SAE Features as Legal-Move Classifiers

(16:54) Cosine Similarities

(18:03) One Really Cool Case Study

(19:51) Conclusion

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Linkpost URL:
https://aizi.substack.com/p/research-report-sparse-autoencoders

Narrated by TYPE III AUDIO.

This is an interim report that we are currently building on. We hope this update will be useful to related research occurring in parallel. Produced as part of the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort

Executive Summary

• In a previous post we trained attention SAEs on every layer of GPT-2 Small and we found that a majority of features are interpretable in all layers. We’ve since leveraged our SAEs as a tool to explore individual attention heads through the lens of SAE features.
• Using our SAEs, we inspect the roles of every attention head in GPT-2 small, discovering a wide range of previously unidentified behaviors. We manually examined every one of the 144 attention heads and provide brief descriptions in this spreadsheet. We note that this is a rough heuristic to get a sense of the most [...]

---

Outline:

(00:32) Executive Summary

(05:57) Technique: Attention Head Attribution via Attention Layer SAEs

(07:54) Overview of Attention Heads Across Layers

(11:03) Investigating Attention Head Polysemanticity

(13:28) Discovering Plausibly Monosemantic Heads

(14:50) Case Study: Long Prefix Induction Head

(20:18) Appendix: Attention Heads Feature Map

(20:42) Citing this work

(20:55) Author Contributions Statement

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

What important questions would you want to see discussed and debated here about Anthropic? Suggest and vote below.

How to use the poll

The goal is to show people where a lot of interest and disagreement lies. This can be used to find discussion and dialogue topics in the future.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Claude 3.0

Claude 3.0 is here. It is too early to know for certain how capable it is, but Claude 3.0's largest version is in a similar class to GPT-4 and Gemini Advanced. It could plausibly now be the best model for many practical uses, with praise especially coming in on coding and creative writing.

Anthropic has decided to name its three different size models Opus, Sonnet and Haiku, with Opus only available if you pay. Can we just use Large, Medium and Small?

Cost varies quite a lot by size, note this is a log scale on the x-axis, whereas the y-axis isn’t labeled.

This post goes over the benchmarks, statistics and system card, along with everything else people have been reacting to. That includes a discussion about signs of self-awareness (yes, we are doing this again) and also raising the question of whether [...]

---

Outline:

(00:03) Claude 3.0

(01:10) Benchmarks and Stats

(06:01) The System Card

(13:06) The System Prompt

(17:42) Reactions on How Good Claude 3 is in Practice

(26:19) It Can’t Help But Notice

(31:47) Acts of Potential Self Awareness Awareness

(39:33) We Can’t Help But Notice

(55:34) What Happens Next?

---

First published:

Source:

Narrated by TYPE III AUDIO.

It's not just that my clients lie to me a lot, which will only hurt them — it's that they’re really, really bad at it.

My job as a public defender puts me in a weird place. I am my clients’ zealous advocate, but I’m not their marionette. I don’t just roll into court to parrot whatever my clients tell me — I make sure I’m not re-shoveling bullshit. So for my sake and theirs, I do my homework. I corroborate. I investigate.

A significant portion of my job ironically mirrors that of a police detective. Every case I get requires me to deploy a microscope and retrace the cops’ steps to see if they fucked up somehow (spoiler: they haven’t). Sometimes I go beyond what the cops did to collect my own evidence and track down my own witnesses.

All this puts some of my clients of the [...]

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

1.1 Summary & contents

This is the first of two blog posts where I try to make sense of the whole universe of social-status-related behaviors and phenomena:

• This post is focused on a special case of two people interacting, where they have different object-level preferences—maybe one wants to order pizza for dinner while the other wants sushi. This gets us into various topics like “leading and following”, averaging different people's utility functions, being more or less “pushy”, “ask culture versus guess culture”, plausible deniability, politeness arms-races, and more.
• Then the next post, “Social status part 2/2: everything else”, will layer on another heap of complexity on top of all that, related to the fact that people also have preferences related to the interaction itself, like “a preference not to be rude”. That gets us into topics like dominance, prestige, getting offended, passive-aggressiveness, status, self-deprecation, and more.

Some context for how [...]

---

Outline:

(00:07) 1.1 Summary and contents

(04:55) 1.2 What are “negotiations over object-level preferences”?

(07:07) 1.3 If two people in an interaction have conflicting object-level preferences, one will “mostly lead” and the other will “mostly follow” (or they could both “half lead”, etc.)

(09:42) 1.3.1 “Leading” versus “following” applies to both dominance and prestige

(10:42) 1.4 The “Weighted Average” Toy model: A range from “0% leader” to “100% leader” in an interaction, summing to 100%

(14:06) 1.4.1 Worked example

(16:01) 1.4.2 What about intersubjective utility comparisons—a.k.a., that thing about affine transformations?

(22:10) 1.5 Another toy model: An unspoken map from (subjective) “desires” to (external / behavioral) “pushiness”

(23:30) 1.5.1 “Ask culture versus guess culture”

(25:32) 1.5.2 More on “culture” clashes

(27:48) 1.5.3 I think the terms “ask” and “guess” are somewhat misleading

(29:54) 1.5.4 I think plausible deniability is mostly orthogonal to ask-vs-guess

(34:30) 1.5.5 Tug-of-war analogy

(35:49) 1.6 Explaining cross-cultural differences

(35:54) 1.6.1 Another toy model: Communication cultures may result from “arms races”

(39:51) 1.6.2 Combat-vs-nurture is different from ask-vs-guess

(44:06) 1.7 Conclusion

The original text contained 15 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Roon, member of OpenAI's technical staff, is one of the few candidates for a Worthy Opponent when discussing questions of AI capabilities development, AI existential risk and what we should do about it. Roon is alive. Roon is thinking. Roon clearly values good things over bad things. Roon is engaging with the actual questions, rather than denying or hiding from them, and unafraid to call all sorts of idiots idiots. As his profile once said, he believes spice must flow, we just do go ahead, and makes a mixture of arguments for that, some good, some bad and many absurd. Also, his account is fun as hell.

Thus, when he comes out as strongly as he seemed to do recently, attention is paid, and we got to have a relatively good discussion of key questions. While I attempt to contribute here, this post is largely aimed at preserving [...]

---

Outline:

(04:13) The Doubling Down

(06:03) Connor Leahy Gives it a Shot

(11:02) Roon Responds to Connor

(14:27) Connor Goes Deep

(30:57) A Question of Agency

---

First published:

Source:

Narrated by TYPE III AUDIO.

The following is a lightly edited version of a memo I wrote for a retreat. It was inspired by a draft of Counting arguments provide no evidence for AI doom

Thankful for the dozens of interesting conversations and comments at the retreat.

I think that the AI alignment field is partially founded on fundamentally confused ideas. I’m worried about this because, right now, a range of lobbyists and concerned activists and researchers are in Washington making policy asks. Some of these policy proposals seem to be based on erroneous or unsound arguments.[1]

The most important takeaway from this essay is that [...]

---

Outline:

(02:04) Tracing back historical arguments

(05:03) Many arguments for doom are wrong

(07:28) The counting argument for AI “scheming” provides ~0 evidence

(09:40) The counting argument for extreme overfitting

(13:20) Recovering the counting argument?

(14:38) The counting argument doesnt count

(15:14) Other clusters of mistakes

(19:06) Conclusion

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Today, we're announcing the Claude 3 model family, which sets new industry benchmarks across a wide range of cognitive tasks. The family includes three state-of-the-art models in ascending order of capability: Claude 3 Haiku, Claude 3 Sonnet, and Claude 3 Opus. Each successive model offers increasingly powerful performance, allowing users to select the optimal balance of intelligence, speed, and cost for their specific application.

The largest Claude 3 model seems to outperform GPT-4 on benchmarks (though note slight differences in evaluation methods):

Opus, our most intelligent model, outperforms its peers on most of the common evaluation benchmarks for AI systems, including undergraduate level expert knowledge (MMLU), graduate level expert reasoning (GPQA), basic mathematics (GSM8K), and more. It exhibits near-human levels of comprehension and fluency on complex tasks, leading the frontier of general intelligence.

I haven't had the chance to interact much with this model [...]

---

First published:

Source:

Linkpost URL:
https://www.anthropic.com/news/claude-3-family

Narrated by TYPE III AUDIO.