silentbob's post "Failures in Kindness" is excellent. I love the idea that sometimes, when we exaimine a situation in depth, the most "kind" course of action can be highly conterintuitive. A few other examples I'd like to offer:

Appreciative Kindness

Imagine you meet a friend-of-a-friend for the first time while attending a gathering at their home. "Hey, welcome! It's great to meet you - can I get you anything?" they ask. There's nothing you really want right now, and you don't want to take from them or cause inconvienience, so you say "I'm fine, thanks."

Some people might assume declining their offer is kind. After all, wouldn't it be inconsiderate to make them go to the effort to proivde you with something you don't even really want?

But declining in this way will likely be percieved as a minor rejection. From the other person's perspective, they [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

On 75 percent or more of the residential land in most major American cities it is illegal to build anything other than a detached single-family home. 95.8 percent of total residential land area in California is zoned as single-family-only, which is 30 percent of all land in the state. Restrictive zoning regulations such as these probably lower GDP per capita in the US by 8-36%. That's potentially tens of thousands of dollars per person.

Map of land use in San Jose, California. Pink is single family only (94%)

The legal authority behind all of these zoning rules derives from a 1926 Supreme Court decision in Village of Euclid v. Ambler Realty Co. Ambler realty held 68 acres of land in the town of Euclid, Ohio. The town, wanting to avoid influence, immigration, and industry from nearby Cleveland, passed a restrictive zoning ordinance which prevented Ambler [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I asked the Constellation Slack channel how the technical AIS landscape has changed since I last spent substantial time in the Bay Area (September 2023), and I figured it would be useful to post this (with the permission of the contributors to either post with or without attribution). Curious if commenters agree or would propose additional changes!

This conversation has been lightly edited to preserve anonymity.

Me: One reason I wanted to spend a few weeks in Constellation was to sort of absorb-through-osmosis how the technical AI safety landscape has evolved since I last spent substantial time here in September 2023, but it seems more productive to just ask here "how has the technical AIS landscape evolved since September 2023?" and then have conversations armed with that knowledge. The flavor of this question is like, what are the technical directions and strategies people are most excited about, do we [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

The Bay Area rationalist community has an entry problem! Lots of listed groups are dead, the last centralized index disappeared, communication moved to private discord and slacks. This is bad, so we're making a new index, hopefully up to date and as complete as we can!

Communication

Discord: Bay Area Rationalists: https://discord.gg/EpG4xUVKtf

Email Group: BayAreaLessWrong: https://groups.google.com/g/bayarealesswrong

Local Meetup Groups

Taco Tuesday: by Austin Chen, founder emeritus of Manifold. Check his Manifold questions page for the next date!

North Oakland LessWrong Meetup: every Wednesday, hosted by @Czynski.

Thursday Dinners in Berkeley: Advertised on the Discord server and Google group, alternating between a few restaurants on the northwest side of UC campus.

Bay Area ACX Meetups: For the ACX everywhere meetups twice per year, and some other sporadic events.

Housing

To find spots in group houses, temporary or long term, you can use the Bay Area EA/Rationality Housing Board. The [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Work done at FAR AI.

There has been a lot of conceptual work on mesa-optimizers: neural networks that develop internal goals that may differ from their training objectives (the inner alignment problem). There is an abundance of good ideas for empirical work (find search in a NN, interpret it), but very little actual execution, partly because we did not have a clear-cut example of a mesa-optimizer to study. Until now.[1]

We have replicated the mesa-optimizer that Guez et al. (2019) found, and released it open-source as a model organism for inner alignment research. In brief, Guez et al. trained a recurrent neural network (RNN) with model-free RL to play Sokoban. They noticed that if you give the RNN more time to think by repeating the initial observation at inference time, its performance increases. [...]

The original text contained 1 footnote which was omitted from this narration.

The original text contained 4 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

(Crossposted from Twitter)

I'm skeptical that Universal Basic Income can get rid of grinding poverty, since somehow humanity's 100-fold productivity increase (since the days of agriculture) didn't eliminate poverty.

Some of my friends reply, "What do you mean, poverty is still around? 'Poor' people today, in Western countries, have a lot to legitimately be miserable about, don't get me wrong; but they also have amounts of clothing and fabric that only rich merchants could afford a thousand years ago; they often own more than one pair of shoes; why, they even have cellphones, as not even an emperor of the olden days could have had at any price. They're relatively poor, sure, and they have a lot of things to be legitimately sad about. But in what sense is almost-anyone in a high-tech country 'poor' by the standards of a thousand years earlier? Maybe UBI works the same way [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This article contains more than fifty uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Google DeepMind reports on a system for solving mathematical problems that allegedly is able to give complete solutions to four of the six problems on the 2024 IMO, putting it near the top of the silver-medal category.

Well, actually, two systems for solving mathematical problems: AlphaProof, which is more general-purpose, and AlphaGeometry, which is specifically for geometry problems. (This is AlphaGeometry 2; they reported earlier this year on a previous version of AlphaGeometry.)

[EDITED to add:] Or maybe it doesn't work in the "obvious" way. As cubefox points out in the comments [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This post lays out a framework I’m currently using for thinking about when AI systems will seek power in problematic ways. I think this framework adds useful structure to the too-often-left-amorphous “instrumental convergence thesis,” and that it helps us recast the classic argument for existential risk from misaligned AI in a revealing way. In particular, I suggest, this recasting highlights how much classic analyses of AI risk load on the assumption that the AIs in question are powerful enough to take over the world very easily, via a wide variety of paths. If we relax this assumption, I suggest, the strategic trade-offs that an AI faces, in choosing whether or not to engage in some form of problematic power-seeking, become substantially more complex.

Prerequisites for rational takeover-seeking

For simplicity, I’ll focus here on the most extreme [...]

---

Outline:

(00:53) Prerequisites for rational takeover-seeking

(02:51) Agential prerequisites

(06:43) Goal-content prerequisites

(09:13) Takeover-favoring incentives

(13:38) Recasting the classic argument for AI risk using this framework

(26:13) What if the AI can’t take over so easily, or via so many different paths?

The original text contained 16 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's here. The horse has left the barn. Llama-3.1-405B, and also Llama-3.1-70B and Llama-3.1-8B, have been released, and are now open weights.

Early indications are that these are very good models. They were likely the best open weight models of their respective sizes at time of release.

Zuckerberg claims that open weights models are now competitive with closed models. Yann LeCun says ‘performance is on par with the best closed models.’ This is closer to true than in the past, and as corporate hype I will essentially allow it, but it looks like this is not yet fully true.

Llama-3.1-405B not as good as GPT-4o or Claude Sonnet. Certainly Llama-3.1-70B is not as good as the similarly sized Claude Sonnet. If you are going to straight up use an API or chat interface, there seems to be little reason to use Llama.

That is a [...]

---

Outline:

(04:25) Options to Run It

(04:45) The Model Card

(08:42) Benchmarks

(13:41) Human Reactions in the Wild

(16:56) What's It Good For?

(21:39) The Other Other Guy

(22:35) Safety

(31:48) Three People Can Keep a Secret and Reasonably Often Do So

(36:12) The Announcement and Interview

(47:59) Zuckerberg's Open Weights Manifesto

(58:17) Fun Little Note

The original text contained 15 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.