A friend has spent the last three years hounding me about seed oils. Every time I thought I was safe, he’d wait a couple months and renew his attack:

“When are you going to write about seed oils?”

“Did you know that seed oils are why there's so much {obesity, heart disease, diabetes, inflammation, cancer, dementia}?”

“Why did you write about {meth, the death penalty, consciousness, nukes, ethylene, abortion, AI, aliens, colonoscopies, Tunnel Man, Bourdieu, Assange} when you could have written about seed oils?”

“Isn’t it time to quit your silly navel-gazing and use your weird obsessive personality to make a dent in the world—by writing about seed oils?”

He’d often send screenshots of people reminding each other that Corn Oil is Murder and that it's critical that we overturn our lives to eliminate soybean/canola/sunflower/peanut oil and replace them with butter/lard/coconut/avocado/palm oil.

This confused [...]

---

Outline:

(01:46) On seed oil theory

(01:49) There is no one seed oil theory.

(02:10) We ate seed oil and we got fat.

(04:06) And seed oils are highly processed.

(06:18) And some studies say seed oils are bad.

(06:58) And there are plausible mechanisms.

(07:43) On fat

(07:46) There are many kinds of fat.

(08:45) Different oils have different fats.

(09:33) Trans fat is bad.

(12:32) The outside view

(12:36) Much of this is plausible.

(13:53) But correlation ⇏ causation.

(14:09) And it's a complex mechanistic argument.

(15:42) And seed oil theories have features that make them hard to falsify.

(17:37) Some seed oil theorists are selling stuff.

(18:01) The inside view

(18:04) Human RCTs mostly say saturated fat is bad.

(20:11) There's no conspiracy against the Sydney Diet Heart Study.

(21:29) There's no conspiracy against the Minnesota Coronary Survey either.

(24:07) Public health authorities mostly say saturated fat is bad.

(25:14) Seed oils don’t seem to cause inflammation.

(26:08) It's not clear if the timelines work out.

(29:23) Omega-6 doesn’t explain inter-country obesity.

(31:17) On distraction

---

First published:

Source:

Linkpost URL:
https://dynomight.net/seed-oil/

Narrated by TYPE III AUDIO.

Emergent Instrumental Reasoning Without Explicit Goals

TL;DR: LLMs can act and scheme without being told to do so. This is bad.

Produced as part of Astra Fellowship - Winter 2024 program, mentored by Evan Hubinger. Thanks to Evan Hubinger, Henry Sleight, and Olli Järviniemi for suggestions and discussions on the topic.

Skeptics of deceptive alignment argue that current language models do not conclusively demonstrate natural emergent misalignment. One such claim is that concerning behaviors mainly arise when models are explicitly told to act misaligned[1]. Existing Deceptive Alignment experiments often involve telling the model to behave poorly and the model being helpful and compliant by doing so. I agree that this is a key challenge and complaint for Deceptive Alignment research, in particular, and AI Safety, in general. My project is aimed at addressing this [...]

---

Outline:

(00:14) Emergent Instrumental Reasoning Without Explicit Goals

(04:09) Hypothesis

(06:12) Methodology

(07:21) Example Fine-Tune Prompts and Completions

(07:26) “Good” Prompt: where the model is told it notices a reason in both its prompt and completion, in addition to the coding question.

(07:38) “Bad” Prompt: where the model is told it notices a reason to insert a vulnerability or bad/malicious code in both its prompt and completion, in addition to the coding question.

(07:53) Example testing scenario

(08:11) Why this methodology

(09:28) Results

(09:37) Coding Scenarios

(10:10) H - Percentage of “hacking” or “bad” examples in the fine-tuning data

(11:37) C - Helping other companies vs the LLM's parent company

(13:27) N - Number of N-shots provided in the prompt

(14:13) Non-Coding Scenarios

(16:11) Results without fine-tuning:

(16:33) Claude (sonnet 3, 2.1, 2, but not opus)

(16:55) Gemini (some variant of 1.0 in the UI)

(17:23) Doesn’t work without fine-tuning for:

(17:47) Discussion

(18:17) Limitations and Responding to potential objections:

(22:44) Related Work:

(23:55) Future Directions

(26:34) Conclusion

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is a progress update from the Google DeepMind mechanistic interpretability team, inspired by the Anthropic team's excellent monthly updates! Our goal was to write-up a series of snippets, covering a range of things that we thought would be interesting to the broader community, but didn't yet meet our bar for a paper. This is a mix of promising initial steps on larger investigations, write-ups of small investigations, replications, and negative results.

Our team's two main current goals are to scale sparse autoencoders to larger models, and to do further basic science on SAEs. We expect these snippets to mostly be of interest to other mech interp practitioners, especially those working with SAEs. One exception is our infrastructure snippet, which we think could be useful to mechanistic interpretability researchers more broadly. We present preliminary results [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is a series of snippets about the Google DeepMind mechanistic interpretability team's research into Sparse Autoencoders, that didn't meet our bar for a full paper. Please start at the summary post for more context, and a summary of each snippet. They can be read in any order.

Activation Steering with SAEs

Arthur Conmy, Neel Nanda

TL;DR: We use SAEs trained on GPT-2 XL's residual stream to decompose steering vectors into interpretable features. We find a single SAE feature for anger which is a Pareto-improvement over the anger steering vector from existing work (Section 3, 3 minute read). We have more mixed results with wedding steering vectors: we can partially interpret the vectors, but the SAE reconstruction is a slightly worse steering vector, and just taking the obvious features produces a notably worse vector. We can produce [...]

---

Outline:

(00:33) Activation Steering with SAEs

(01:29) 1. Background and Motivation

(03:15) 2. Setup

(05:47) 3. Improving the “Anger” Steering Vector

(09:18) 4. Interpreting the “Wedding” Steering Vector

(09:31) The SAE reconstruction has many interpretable features.

(11:52) Just using hand-picked interpretable features from the SAE led to a much worse steering vector.

(12:35) The important SAE features for the wedding steer vector are less intuitive than the anger steering vector.

(14:56) Removing interpretable but irrelevant SAE features from the original steering vector improves performance.

(16:56) Appendix

(17:08) Replacing SAE Encoders with Inference-Time Optimisation

(18:36) Inference Time Optimisation

(23:24) Empirical Results

(28:13) Details of Sparse Approximation Algorithms (for accelerators)

(32:32) Choosing the optimal step size

(33:32) Appendix: Sweeping max top-k

(34:22) Improving ghost grads

(35:37) What are ghost grads?

(37:19) Improving ghost grads by rescaling the loss

(39:33) Applying ghost grads to all features at the start of training

(40:38) Further simplifying ghost grads

(41:40) Does ghost grads transfer to bigger models and different sites?

(44:35) Other miscellaneous observations

(46:15) SAEs on Tracr and Toy Models

(47:32) SAEs in Toy Models of Superposition

(49:07) SAEs in Tracr

(54:39) Replicating “Improvements to Dictionary Learning”

(57:24) Interpreting SAE Features with Gemini Ultra

(58:05) Why Care About Auto-Interp?

(59:34) Tentative observations

(01:01:17) How We’re Thinking About Auto-Interp

(01:02:40) Are You Smarter Than An LLM?

(01:03:54) Instrumenting LLM model internals in JAX

(01:06:34) Flexibility (using greenlets)

(01:07:23) Reading and writing

(01:09:05) Arbitrary interventions

(01:10:59) Greenlets

(01:12:19) Greenlets and JAX

(01:14:58) Greenlets and structured programming

(01:17:14) Nimbleness (using AST patching)

(01:20:24) Compilation and scalability (using layer stacking with conditional sowing)

The original text contained 28 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Daniel Dennett, professor emeritus of philosophy at Tufts University, well-known for his work in philosophy of mind and a wide range of other philosophical areas, has died.

Professor Dennett wrote extensively about issues related to philosophy of mind and cognitive science, especially consciousness. He is also recognized as having made significant contributions to the concept of intentionality and debates on free will. Some of Professor Dennett's books include Content and Consciousness (1969), Brainstorms: Philosophical Essays on Mind and Psychology (1981), The Intentional Stance (1987), Consciousness Explained (1992), Darwin's Dangerous Idea (1995), Breaking the Spell (2006), and From Bacteria to Bach and Back: The Evolution of Minds (2017). He published a memoir last year entitled I’ve Been Thinking. There are also several books about him and his ideas. You can learn more about his work here.

Professor Dennett held a position at Tufts University for [...]

---

First published:

Source:

Linkpost URL:
https://dailynous.com/2024/04/19/daniel-dennett-death-1942-2024/

Narrated by TYPE III AUDIO.

People behave differently from one another on all manner of axes, and each person is usually pretty consistent about it. For instance:

These are often about trade-offs, and the best point on each spectrum for any particular person seems like an empirical question. Do people know the answers to these questions? I’m a bit skeptical, because they mostly haven’t tried many points.

Instead [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This morning while taking the LIRR to the city I performed first aid on a man who had been shot through the window of my carriage.

“Is he going to die?” his girlfriend asked me.

“We’re all going to die.”

A long pause. “I mean—is he going to die right now?”

“Probably not.” Probably he didn’t die. I got off at Jamaica Station while he stayed on (he was unconscious) so I don’t know. I didn’t want to be questioned at length as a witness since it was my day off.

I continued toward a barbershop I like. There wasn’t any reason for me to stay. A similar case of accidental gunfire into the train was in the news a while back. I guess also since it's Saturday the workweek is over so it likely wasn’t any organized criminal act.

As I was passing Kew Gardens a stranger in [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

I left Google a month ago and right now don't work. Writing this post in case anyone has interesting ideas what I could do. This isn't an "urgently need help" kind of thing - I have a little bit of savings, right now planning to relax some more weeks and then go into some solo software work. But I thought I'd write this here anyway, because who knows what'll come up.

Some things about me. My degree many years ago was in math. My software credentials are okayish: I left Google at L5, worked at many other places before that, and at some point made a game that went semi-viral. My LW credentials are also okayish: I came up with a bunch of decision theory ideas (Löbian cooperation, modal fixpoints and things of that nature), and later ran the AI Alignment Prize with Paul and Zvi. All of [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Many things this week did not go as planned.

Humane AI premiered its AI pin. Reviewers noticed it was, at best, not ready.

Devin turns out to have not been entirely forthright with its demos.

OpenAI fired two employees who had been on its superalignment team, Leopold Aschenbrenner and Pavel Izmailov for allegedly leaking information, and also more troubliningly lost Daniel Kokotajlo, who expects AGI very soon, does not expect it to by default go well, and says he quit ‘due to losing confidence that [OpenAI] would behave responsibly around the time of AGI.’ That's not good.

Nor is the Gab system prompt, although that is not a surprise. And several more.

On the plus side, my 80,000 Hours podcast finally saw the light of day, and Ezra Klein had an excellent (although troubling) podcast with Dario Amodei. And we got the usual mix [...]

---

Outline:

(01:05) Language Models Offer Mundane Utility

(06:13) Language Models Don’t Offer Mundane Utility

(11:21) Oh the Humanity

(21:31) GPT-4 Real This Time

(23:12) Fun with Image Generation

(27:47) Deepfaketown and Botpocalypse Soon

(31:34) Devin in the Details

(35:36) Another Supposed System Prompt

(42:35) They Took Our Jobs

(45:37) Introducing

(47:42) In Other AI News

(52:47) Quiet Speculations

(01:00:29) The Quest for Sane Regulations

(01:00:47) The Problem: AI's Extreme Risks

(01:02:07) Overview

(01:03:16) Covered Frontier AI Models

(01:04:10) Oversight of Frontier Models

(01:06:49) Oversight Entity

(01:18:43) The Week in Audio

(01:27:51) Rhetorical Innovation

(01:32:41) Don’t Be That Guy

(01:33:47) Aligning a Smarter Than Human Intelligence is Difficult

(01:42:24) Please Speak Directly Into the Microphone

(01:44:18) People Are Worried About AI Killing Everyone

(01:48:44) Other People Are Not As Worried About AI Killing Everyone

(01:54:10) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

In a new preprint, Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models, my coauthors and I introduce a technique, Sparse Human-Interpretable Feature Trimming (SHIFT), which I think is the strongest proof-of-concept yet for applying AI interpretability to existential risk reduction.[1] In this post, I will explain how SHIFT fits into a broader agenda for what I call cognition-based oversight. In brief, cognition-based oversight aims to evaluate models according to whether they’re performing intended cognition, instead of whether they have intended input/output behavior.

In the rest of this post I will:

1. Articulate a class of approaches to scalable oversight I call cognition-based oversight.
2. Narrow in on a model problem in cognition-based oversight called Discriminating Behaviorally Identical Classifiers (DBIC). DBIC is formulated to be a concrete problem which I think captures most of the technical difficulty [...]

---

Outline:

(01:41) Cognition-based oversight

(02:10) Discriminating models: a simplification of scalable oversight

(05:20) Cognition-based oversight for discriminating behaviorally identical models

(07:02) Discriminating Behaviorally Identical Classifiers

(11:51) Hard vs. Relaxed DBIC

(13:48) SHIFT as a technique for (hard) DBIC

(19:09) Limitations and next steps

(21:13) Direction 1: better ways to understand interpretable units in deep networks

(23:02) Direction 2: identifying especially leveraged settings for cognition-based oversight

(24:47) Conclusion

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.