This is a linkpost for In Defense of Lawyers Playing Their Part.

Michael Huemer writes about why he believes it's wrong for lawyers to pursue unjust legal outcomes.

It's a good article, and one of the best defenses of this position I've seen. Still, I think this argument is mistaken. The reason why we require lawyers to fight for "their side" even if they believe they're in the wrong is to minimize the opportunity for bias.

Imagine if all trials were bench trials, decided by only one person as the judge. Even if they're taught to be as objective as possible, there would still be significant concerns about unconscious bias. One person only has one set of experiences to draw on, which is necessarily not very representative of the full range of experiences. And in some ways this problem becomes worse the more training the judge is given, since [...]

---

Outline:

(02:58) 2.1. The epistemological problem

(07:32) 2.2. The lawyer as friend

(07:49) 2.3. Faith in the system

(08:54) 2.4. Rule consequentialism

(10:05) 2.5. The right to a fair trial

(11:18) 3. Lying vs. Misleading

(13:13) Overview

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Strategy-resistance

Identify, or prove impossibility, of a voting system which incentivizes—

1. A strictly sincere ranking of all candidates in the zero-information setting, where it implements a "good" social choice rule such as the relative (normalized) utilitarian rule, a Condorcet social choice rule, or the Borda rule.
2. In a Poisson game or similar setting: a unique semi-sincere Nash equilibrium that elects the Condorcet winner (if one exists), similar to those shown for approval voting by Myerson and Weber (1993) and Durand et al. (2019).

Properties of Multiwinner voting systems

There's strikingly little research on multiwinner voting systems. You can find a table of criteria for single-winner systems on Wikipedia, but if you try and find the same for multi-winner systems, there's nothing. Here's 9 important criteria we can judge multiwinner voting systems on:

1. Independence of Irrelevant Alternatives
2. Independence of Universally-Approved Candidates
3. Monotonicity 
4. Participation
5. Precinct-summability
6. Polynomial-time approximation scheme
7. Proportionality [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Preference Models (PMs) are trained to imitate human preferences and are used when training with RLHF (reinforcement learning from human feedback); however, we don't know what features the PM is using when outputting reward. For example, maybe curse words make the reward go down and wedding-related words make it go up. It would be good to verify that the features we wanted to instill in the PM (e.g. helpfulness, harmlessness, honesty) are actually rewarded and those we don't (e.g. deception, sycophancey) aren't.

Sparse Autoencoders (SAEs) have been used to decompose intermediate layers in models into interpretable feature. Here we train SAEs on a 7B parameter PM, and find the features that are most [...]

---

Outline:

(01:30) What are PMs?

(03:27) Finding High Reward-affecting Features w/ SAEs

(04:01) Negative Features

(04:19) I dont know

(05:47) Repeating Text

(07:35) URLs

(08:15) Positive Features

(08:33) (Thank you) No problem!

(10:02) Youre right. Im wrong.

(10:49) Putting it all together

(10:57) General Takeaways

(12:13) Limitations and Alternatives

(12:17) Model steering

(12:56) Limited Dataset

(13:17) Later Layer SAEs Sucked!

(13:32) Small Token-Length Datapoints

(13:47) Future Work

(16:31) Technical Details

(16:35) Dataset filtering

(17:10) Attribution Patching

(18:20) SAEs

---

First published:

Source:

Narrated by TYPE III AUDIO.

TLDR: PIBBSS is changing our core team. Nora is stepping down as director due to joining ARIA, and Lucas Teixeira and Dusan Nesic are taking over her leadership role. Nora joins the board, alongside Tan Zhi Xuan, Alexander Gietelink Oldenziel, Ben Goldhaber and Gabriel Weil.

***

I (Nora) have recently accepted an offer to join ARIA's Safeguarded AI Programme as Technical Specialist under davidad. As such, I am stepping back as Director at PIBBSS, after co-founding and leading PIBBSS since 2021.

It wasn’t an easy choice to make! I deeply care about and believe in the mission of and the people at PIBBSS. Before davidad encouraged me to apply for the role, I hadn’t considered leaving PIBBSS. I believe PIBBSS is playing an important role in terms of fostering theoretically ambitious and empirically grounded AI safety research. I am very excited about the directions the team and I [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

They said it couldn’t be done.

No, not Claude Sonnet 3.5 becoming the clear best model.

No, not the Claude-Sonnet-empowered automatic meme generators. Those were whipped together in five minutes.

They said I would never get quiet time and catch up. Well, I showed them!

That's right. Yes, there is a new best model, but otherwise it was a quiet week. I got a chance to incorporate the remaining biggest backlog topics. The RAND report is covered under Thirty Eight Ways to Steal Your Model Weights. Last month's conference in Seoul is covered in You’ve Got Seoul. I got to publish my thoughts on OpenAI's Model Spec last Friday.

Table of Contents

Be sure to read about Claude 3.5 Sonnet here. That is by far the biggest story.

---

Outline:

(00:50) Language Models Offer Mundane Utility

(02:38) Language Models Don’t Offer Mundane Utility

(04:41) Clauding Along

(09:46) Fun with Image Generation

(12:21) Copyright Confrontation

(16:30) Deepfaketown and Botpocalypse Soon

(19:45) They Took Our Jobs

(24:49) The Art of the Jailbreak

(25:40) Get Involved

(26:56) Introducing

(30:45) In Other AI News

(33:50) Quiet Speculations

(43:08) You’ve Got Seoul

(57:36) Thirty Eight Ways to Steal Your Model Weights

(01:07:48) The Quest for Sane Regulations

(01:13:11) SB 1047

(01:14:49) The Week in Audio

(01:20:25) Rhetorical Innovation

(01:22:26) People Are Worried About AI Killing Everyone

(01:24:05) Other People Are Not As Worried About AI Killing Everyone

(01:24:50) The Lighter Side

The original text contained 34 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

1.

It's October 1962. The Cuban missile crisis just happened, thankfully without apocalyptic nuclear war. But still:

• Apocalyptic nuclear war easily could have happened.
• Crises as serious as the Cuban missile crisis clearly aren’t that rare, since one just happened.

You estimate (like President Kennedy) that there was a 25% chance the Cuban missile crisis could have escalated to nuclear war. And you estimate that there's a 4% chance of an equally severe crisis happening each year (around 4 per century).

Put together, these numbers suggest there's a 1% chance that each year might bring nuclear war. Small but terrifying.

But then 62 years tick by without nuclear war. If a button has a 1% chance of activating and you press it 62 times, the odds are almost 50/50 that it would activate. So should you revise your estimate to something lower than 1%?

[...]

---

Outline:

(00:07) 1.

(01:11) 2.

(02:13) 3.

(03:01) 4.

(04:48) 5.

(06:08) 6.

(06:59) 7.

(09:25) 8.

(10:23) 9.

(11:50) 10.

---

First published:

Source:

Narrated by TYPE III AUDIO.

The Simple Macroeconomics of AI is a 2024 working paper by Daron Acemoglu which models the economic growth effects of AI and predicts them to be small: About a .06% increase in TFP growth annually. This stands in contrast to many predictions which forecast immense impacts on economic growth from AI, including many from other academic economists. Why does Acemoglu come to such a different conclusion than his colleagues and who is right?

First, Acemoglu divides up the ways AI could affect productivity into four channels:

1. AI enables further (extensive-margin) automation.
Obvious examples of this type of automation include generative AI tools such as large language models taking over simple writing, translation and classification.

2. AI can generate new task complementarities, raising the productivity of labor in tasks it is performing.
For example, AI could provide better information to workers, directly increasing their productivity. [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

There's bound to be a lot of discussion of the Biden-Trump presidential debates last night, but I want to skip all the political prognostication and talk about the real issue: fentanyl-detecting machines.

Joe Biden says:

And I wanted to make sure we use the machinery that can detect fentanyl, these big machines that roll over everything that comes across the border, and it costs a lot of money. That was part of this deal we put together, this bipartisan deal.

More fentanyl machines, were able to detect drugs, more numbers of agents, more numbers of all the people at the border. And when we had that deal done, he went – he called his Republican colleagues said don’t do it. It's going to hurt me politically.

He never argued. It's not a good bill. It's a really good bill. We need [...]

---

Outline:

(01:50) What's Up With Fentanyl-Detecting Machines?

(02:52) Could an X-ray machine really detect fentanyl inside a car?

(06:02) But I Wanted Remote Chemical Compound Detection!

(07:31) Spectroscopy

(09:47) Ok, That's Light. What About Sound? Or Electricity?

(11:34) Dude, Where's My Tricorder?

---

First published:

Source:

Narrated by TYPE III AUDIO.

Disclaimer: This is highly incomplete. I am not an expert in the field. There might be some unfamiliar terms. While I will try to explain things, explaining every single term would be beyond this post. You will usually be able to get a sufficient understanding by clicking the links or googling it.

I think everyone, if they read about the chip industry long enough, has a moment where they have to put down a book or pause a podcast and simply remain stunned at the fact that it is possible to design and build something that is so incredibly impressive.

The Apple A17 chip contains 183 million transistors per square millimeter. All placed in a coherent manner and produced with extremely high reliability.

This is exactly why it is so fascinating to learn more about how it is actually done. On [...]

---

Outline:

(00:25) Introduction

(01:16) Background Knowledge

(03:43) The Design Process

(03:52) Definition and Planning

(04:18) Design and Verification

(05:40) Logic Synthesis

(06:54) Physical Design

(07:57) Signoff and Tapeout

(08:28) Takeaways

(09:16) Appendix

(09:19) 1 Verilog example

(09:30) 2 Gate-Level Netlist example

The original text contained 7 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Goal of This Post

I have never seen anyone give a satisfying intuitive explanation of what corrigibility (in roughly Eliezer's sense of the word) is. There's lists of desiderata, but they sound like scattered wishlists which don’t obviously point to a unified underlying concept at all. There's also Eliezer's extremely meta pointer:

We can imagine, e.g., the AI imagining itself building a sub-AI while being prone to various sorts of errors, asking how it (the AI) would want the sub-AI to behave in those cases, and learning heuristics that would generalize well to how we would want the AI to behave if it suddenly gained a lot of capability or was considering deceiving its programmers and so on.

… and that's basically it.[1]

In this post, we’re going to explain a reasonably-unified concept which seems like a decent match to “corrigibility” in Eliezer's sense.

Tools

Starting point: we think [...]

---

Outline:

(00:05) Goal of This Post

(01:03) Tools

(02:40) Respecting Modularity

(04:49) Visibility and Correctability

(06:00) Let's Go Through A List Of Desiderata

(11:53) What Would It Look Like To Use A Powerful Corrigible AGI?

(14:11) From Cognition to Real Patterns?

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.