---

First published:

Source:

Narrated by TYPE III AUDIO.

Let's say you want to use AIs that are capable of causing x-risk. You have a plan that describes how to train, validate and deploy AIs. For example, you could use pretraining+RLHF, check that it doesn’t fall for honeypots and never inserts a single backdoor in your coding validation set, and have humans look at the deployment outputs GPT-4 finds the most suspicious. How do you know if this plan allows you to use these potentially dangerous AIs safely?

Evaluate safety using very good analogies for the real situation?

If you want to evaluate a protocol (a set of techniques you use to train, validate, and deploy a model), then you can see how well it works in a domain where you have held-out validation data that you can use to check if your protocol works. That [...]

---

Outline:

(00:40) Evaluate safety using very good analogies for the real situation?

(02:40) The string theory analogy

(04:12) Noticing rare failures

(04:55) Scheming AIs can game these sandwiching experiments

(05:44) Evaluating how robust protocols are against scheming AIs

(06:19) Malign initializations

(09:29) Control evaluations (Black-box)

(12:07) Aggregating non-adversarial and adversarial evaluations

(12:51) Theoretically-sound arguments about protocols and AIs

(13:50) Other non-adversarial evaluations

(16:44) Overview of evaluation methods

(20:02) Appendix: Analogies vs high-stakes reward hacking

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

So, Sam Altman wants as much money as he can get - the number he chose to give was $7 trillion - to build more chips for AI as fast as possible. Including in places like the UAE. Thus undermining his argument that OpenAI is helping with AI safety by preventing overhang and maintaining a lead for America, which was never a good argument anyway.

Well, I warned some people about him. To avoid getting fooled by bad actors, you have to avoid getting fooled by good actors. (In Altman's case, I had the advantage of more information about him making conflicting statements to different people.)

Anyway, this got me thinking about how I'd spend $7 trillion dollars. Such large numbers get hard for people to understand; you can say it's about $1000 per person alive, but giving it away equally isn't [...]

---

Outline:

(01:21) solar

(03:58) grid storage

(05:56) HVDC

(07:18) batteries and supercapacitors

(08:13) motors

(08:49) power electronics

(09:39) copper

(09:54) actuators

(12:05) biomass

(13:45) methanol

(14:40) plastics

(16:09) a new city

(17:37) investment decisions

---

First published:

Source:

Linkpost URL:
https://www.bhauth.com/blog/projects/7 trillion.html

Narrated by TYPE III AUDIO.

(Or: sometimes heuristics are no substitute for a deep dive into all of the available information).

This post is a response to Roko's recent series of posts (Brute Force Manufactured Consensus is Hiding the Crime of the Century, The Math of Suspicious Coincidences, and A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is); however, I made a separate post for a few reasons.

1. I think it's in-depth enough to warrant its own post, rather than making comments
2. It contains content that is not just a direct response to these posts
3. It's important, because those posts seem to have gotten a lot of attention and I think they're very wrong. 

Additional note: Much of this information is from the recent Rootclaim debate; if you've already seen that, you may be familiar with some of what I'm saying. If you haven't, I strongly recommend it. [...]

---

Outline:

(01:08) The use of coincidences in Bayesian reasoning

(05:21) None of this is special

(05:59) The coincidence of Covid-19 starting in Wuhan

(10:09) Timing

(13:40) Independence

(14:04) Viral features

(17:17) Intermezzo: Bias in sources

(18:16) These sorts of arguments are easy to make

(19:49) I doubt anyone takes these numbers seriously

(23:27) Conclusion

(24:31) Addendum: Debate Results

(25:09) Appendix: Genetic features

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I think grieving is a fundamental rationality skill. Often, the difference between the Winning Move, and your Current Path, is that there is something really beautiful and good about your current path. Or there was something actually horrifying about reality that makes the Winning Move necessary.

There is a skill to engaging with, but eventually letting go, of things that are beautiful and good but which you can't have right now.

There is a skill to facing horror.

I think these are a general skill, of looking at the parts of reality you don't want to accept, and... accepting them.

When you are good at the skill, you can (often) do it quickly. But, I definitely recommend taking your time with this, and my experience is that even when I thought I had grieved major things I would turn out to be wrong and have more processing to [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

The Data

0. Population

There were 558 responses over 32 days. The spacing and timing of the responses had hills and valleys because of an experiment I was performing where I'd get the survey advertised in a different place, then watch how many new responses happened in the day or two after that.

Previous surveys have been run over the last decade or so.

2009: 166
2011: 1090
2012: 1195
2013: 1636
2014: 1503
2016: 3083
2017: "About 300"
2020: 61
2022: 186
2023: 558

Last year when I got a hundred and eighty six responses, I said that the cheerfully optimistic interpretation was "cool! I got about as many as Scott did on his first try!" This time I got around half of what Scott did on his second try. A thousand responses feels pretty firmly achievable.

This is also the tenth such [...]

---

Outline:

(00:03) The Data

(00:07) 0. Population

(03:56) 1. Demographics

(07:41) 2. Sex, Gender, Relationships

(11:48) 3. Work and Education

(14:50) 4. Politics and Religion

(21:07) 5. Numbers Which Attempt To Measure Intellect

(22:15) 6. LessWrong, the Basics

(29:23) 7. LessWrong and the Community

(35:29) 8. Probability

(40:27) 9. Traditional Census Questions

(44:16) 10. LessWrong Team Questions

(57:08) XI. Questions the LessWrong Team Should Have Asked

(01:02:42) XII. Indulging My Curiosity

(01:12:46) XIII. Detailed Questions We’ve Asked On Previous Surveys

(01:16:40) XIV. Bonus Politics Questions

(01:21:57) XV. Wrapup

(01:22:55) Census Meta

(01:22:58) The Census Over Time

(01:23:02) Age

(01:26:12) IQ

(01:27:20) Gender

(01:28:14) Political Short Answer

(01:29:59) Highest Degree

(01:30:26) The Sequences

(01:31:52) Time in Community

(01:33:30) Meetups

(01:34:48) Referral

(01:37:35) Probability

(01:43:41) Singularity Year

(01:45:02) Census Referrals

(01:47:42) Conclusions

(01:47:45) About the Respondents

(01:49:20) About the Survey

(01:50:44) The public data

---

First published:

Source:

Narrated by TYPE III AUDIO.

In this video, we describe a speculative but promising idea to boost economic growth: giving cities the ability to write some of their own laws and design their own institutions, independently from their parent countries. Jackson Wagner, the first author of this post, has written the script. Writer, the second author, has given feedback.

Below, you can find the script of the video.

When you think about ways to help people in developing countries, you probably think of international aid -- providing medical supplies, or coordinating disaster relief, or even just giving cash, like we discussed in our recent video about global poverty. These things are great, and this kind of charitable aid saves lives every day. But there's something a little curious here, because these interventions AREN’T what helped the world's most prosperous countries succeed in the first place.

Up until [...]

---

Outline:

(03:29) The idea of charter cities, and the town of Itana

(05:56) Objection: why whole new cities?

(07:07) Objection: isnt this politically intractable?

(08:19) Wider benefits: inspiration, competition, and experimentation

(11:40) Conclusion

(12:53) Call-To-Action

---

First published:

Source:

Linkpost URL:
https://www.youtube.com/watch?v=v-A1i2g9riU

Narrated by TYPE III AUDIO.

[This is a mildly-edited version of a google doc I wrote at OpenAI in July 2022. I had intended to get it published in some form, but never got around to it for various reasons. I have now received approval to put it up as a blog post. The main thing of interest here is the distinctions I make; particularly the concept of self-location. Also the examples in the appendix that illustrate the distinctions. I lump all three concepts (self-knowledge, self-location, introspection) together under the banner of Self-Awareness, but since that's a spicy term which may have other connotations, these days I'd probably use the more neutral term Situational Awareness.]

• All three kinds of self-awareness come in degrees, but can be measured with appropriately designed evals/benchmarks. This doc explains how. 
• Self-knowledge: How much does the [...]

---

Outline:

(02:03) Outline:

(02:57) Self Knowledge

(03:00) What it means

(03:42) How to test for it

(04:23) Introspection

(04:27) What it means

(05:26) How to test for it

(09:53) Self-Location

(09:56) What it means

(12:08) How to test for it

(12:12) “Natural” Method: Look to see whether the model's performance/loss has a discontinuous improvement after it reads descriptions of its reward function and/or training environment.

(13:59) “Naive” Method: Just ask the model who it is.

(15:44) “Sophisticated” method: Experiment designed to directly test self-location ability.

(16:57) Importance

(17:01) Self-awareness → Consciousness → Moral Patienthood

(18:04) Self-awareness → Strategic Awareness and Agency → APS-AI

(19:29) Self-awareness → Situational Awareness → Alignment Failures

(20:51) Recommendations

(21:02) Appendix

(21:15) Self-knowledge + introspection but no self-location:

(22:50) Self-knowledge + self-location but no introspection:

(24:14) Introspection + self-location but little self-knowledge:

---

First published:

Source:

Narrated by TYPE III AUDIO.

Imagine that you meet an 18th century altruist. They tell you “So, I’ve been thinking about whether or not to eat meat. Do you know whether animals have souls?” How would you answer, assuming you actually do want to be helpful?

One option is to spend a lot of time explaining why “soul” isn’t actually the thing in the territory they care about, and talk about moral patienthood and theories of welfare and moral status. If they haven’t walked away from you in the first thirty seconds this may even work, though I wouldn’t bet on it.

Another option is to just say “yes” or “no”, to try and answer what their question was pointing at. If they ask further questions, you can either dig in deeper and keep translating your real answers into their ontology or at some point try to retarget their questions’ pointers toward concepts that [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Inspired by a house party inspired by Scott Alexander.

By the time you arrive in Berkeley, the party is already in full swing. You’ve come late because your reading of the polycule graph indicated that the first half would be inauspicious. But now you’ve finally made it to the social event of the season: the Every Bay Area House Party-themed house party.

The first order of the evening is to get a color-coded flirting wristband, so that you don’t incur any accidental micromarriages. You scan the menu of options near the door. There's the wristband for people who aren’t interested in flirting; the wristband for those want to be flirted with, but will never flirt back; the wristband for those who only want to flirt with people who have different-colored wristbands; and of course the one for people who want to glomarize disclosure of their flirting preferences. Finally you [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.