Premises

Here I tackle some bold claims made by Roko in three separate posts about SARS-CoV-2 being an engineered virus (Brute Force Manufactured Consensus is Hiding the Crime of the Century, The Math of Suspicious Coincidences, and A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is).
A first rebuttal to Roko's post is already out, and since I have been preparing this post for some time, certain arguments are going to be similar.

The purpose of this post is not to incontrovertibly demonstrate that SARS-CoV-2 was of zoonotic origin, as I myself believe a laboratory leak to be somewhat plausible, but rather that the degree of Roko's confidence in the engineered-pandemic belief is terribly overstated in light of the presented evidence.

I believe that the RootClaim debate is a great way to familiarise oneself with core arguments from all sides, it is exceedingly [...]

---

Outline:

(00:04) Premises

(01:16) A Set of Explainable Coincidences

(01:51) One-in-a-century pandemic

(06:13) The bat coronavirus

(07:29) The BSL-4 lab

(08:20) From Yunnan to Wuhan

(10:12) Probabilities and Confounding Factors

(15:06) Questionable Use of Sources

(15:10) Misrepresentation of evidence

(27:47) DaszakandFriends outsized influence

(31:03) Conclusion

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Previously: I hit send on The Third Gemini, and within half an hour DeepMind announced Gemini 1.5.

So this covers Gemini 1.5. One million tokens, and we are promised overall Gemini Advanced or GPT-4 levels of performance on Gemini Pro levels of compute.

This post does not cover the issues with Gemini's image generation, and what it is and is not willing to generate. I am on top of that situation and will get to it soon.

One Million Tokens

Our teams continue pushing the frontiers of our latest models with safety at the core. They are making rapid progress. In fact, we’re ready to introduce the next generation: Gemini 1.5. It shows dramatic improvements across a number of dimensions and 1.5 Pro achieves comparable quality to 1.0 Ultra, while using less compute.

It is truly bizarre to launch Gemini Advanced as a paid [...]

---

Outline:

(00:32) One Million Tokens

(04:29) Mixture of Experts

(07:39) Quality Not Quantity

(11:52) What To Do With It?

---

First published:

Source:

Narrated by TYPE III AUDIO.

Why do we root for the antiheroes?

Walter White, Light Yagami, Lucifer Morningstar, Doctor Frankenstein... It seems to be natural to root for the bad guys, and perhaps even more natural for rationalists.

“I must confess I have always had some sympathy with villains. Heroism makes fine entertainment but sooner or later someone has to get things done.” - Bayaz, Joe Abercrombie's First Law Trilogy.

I think this is the reason, at least for me. Villains (particularly the most beloved villains) are agentic. They know what they want and they attempt to obtain it through their own actions. Sometimes those actions are even reasonable.

This is notable because it's usually untrue of heroes. Heroes tend to be called to adventure, accepting a goal thrust upon them by circumstances or a bearded stranger or both. Then they pursue it, but often alongside a handful of other eclectic goals [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

MIRI is hiring a managing editor and one or more writers. Here's the job listing. We're accepting applications until March 13th.

I am the Communications Manager at MIRI and I'm the hiring manager for these positions. I'm happy to answer questions.

---

First published:

Source:

Narrated by TYPE III AUDIO.

1.

Prerequisite reading: Being the (Pareto) Best in the World.

A summary of Being the (Pareto) Best in the World: Being the world's best mathematician is hard. Being the world's best musician is hard. Being the world's best mathematician/musician is much easier, especially since there are multiple slots; an amazing mathematician who is also a competent musician, someone who is good at both, and a competent mathematician who is also an amazing musician can all find a niche.

I like this concept, and have kept it in my back pocket ever since I read it. I have sometimes described myself as a software engineer who was competent at public speaking and project management. That particular overlapping skillset is, it turns out, fairly valuable. While I was attempting to become a better software engineer, I was also trying to add competence at corporate budgets and accounting to that skillset.

These [...]

---

Outline:

(00:03) 1.

(01:49) 2.

(05:44) 3.

(08:16) 4.

(10:01) 5.

(12:56) 6.

---

First published:

Source:

Narrated by TYPE III AUDIO.

TL;DR: Scaling labs have their own alignment problem analogous to AI systems, and there are some similarities between the labs and misaligned/unsafe AI.

Major AI scaling labs (OpenAI/Microsoft, Anthropic, Google/DeepMind, and Meta) are very influential in the AI safety and alignment community. They put out cutting-edge research because of their talent, money, and institutional knowledge. A significant subset of the community works for one of these labs. This level of influence is beneficial in some aspects. In many ways, these labs have strong safety cultures, and these values are present in their high-level approaches to developing AI – it's easy to imagine a world in which things are much worse. But the amount of influence that these labs have is also something to be cautious about.

The alignment community is defined by a [...]

---

Outline:

(02:02) Misaligned Incentives

(02:43) Power Seeking

(03:30) Lack of Transparency

(04:30) Deception

(05:18) Incorrigibility

(06:03) Subverting Constraints

(06:21) Concluding Thoughts

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is an informal post intended to describe a workflow / setup that I found very useful, so that others might consider adopting or experimenting with facets of it that they find useful.

In August 2023, I was a part of MATS 4.0 and begun learning the skill of deconfusion, with an aim of disentangling my conflicting intuitions between my belief that shard theory seemed to be at least directionally pointing at some issues with the MIRI model of AGI takeoff and alignment difficulty, and my belief that Nate Soares was obviously correct that reflection will break Alex Turner's diamond alignment scheme. A friend lent me his Kindle Scribe to try out as part of my workflow. I started using it for note-taking, and found it incredibly useful and bought it from him. A month later, I bought a second Kindle Scribe to add to my [...]

---

Outline:

(01:10) The Display

(04:06) Single-purpose devices enhance focus

(05:34) One to read, another to write on

(07:59) Moving on

---

First published:

Source:

Narrated by TYPE III AUDIO.

Just a short post to highlight an issue with debate on LW; I have recently been involved with some interest in the debate on covid-19 origins on here. User viking_math posted a response which I was keen to respond to, but it is not possible for me to respond to that debate (or any) because the LW site has rate-limited me to one comment per 24 hours because my recent comments are on -5 karma or less.

So, I feel that I should highlight that one side of the debate (my side) is simply not going to be here. I can't prosecute a debate like this.

This is funnily enough an example of brute-force manufactured consensus - there will be a debate, people will make points on their side and the side I am arguing for will be missing, so observers will conclude that there [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

[Editor's note: I forgot to post this to WorldPress on Thursday. I’m posting it here now. Sorry about that.]

Sam Altman is not playing around.

He wants to build new chip factories in the decidedly unsafe and unfriendly UAE. He wants to build up the world's supply of energy so we can run those chips.

What does he say these projects will cost?

Oh, up to seven trillion dollars. Not a typo.

Even scaling back the misunderstandings, this is what ambition looks like.

It is not what safety looks like. It is not what OpenAI's non-profit mission looks like. It is not what it looks like to have concerns about a hardware overhang, and use that as a reason why one must build AGI soon before someone else does. The entire justification for OpenAI's strategy is invalidated by this move.

I have [...]

---

Outline:

(01:02) Language Models Offer Mundane Utility

(01:57) Language Models Don’t Offer Mundane Utility

(03:22) GPT-4 Real This Time

(05:49) Deepfaketown and Botpocalypse Soon

(12:38) They Took Our Jobs

(15:33) Get Involved

(15:50) Introducing

(17:13) Altman's Ambition

(22:34) Yoto

(26:32) In Other AI News

(35:04) Quiet Speculations

(41:55) The Quest for Sane Regulations

(43:32) Washington DC Still Does Not Get It

(46:16) Many People are Saying

(48:48) China Watch

(49:38) Roon Watch

(50:48) How to Get Ahead in Advertising

(52:35) The Week in Audio

(56:45) Rhetorical Innovation

(01:01:18) Please Speak Directly Into This Microphone

(01:03:13) Aligning a Smarter Than Human Intelligence is Difficult

(01:03:39) Other People Are Not As Worried About AI Killing Everyone

(01:06:02) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

Just an interesting philosophical argument

1. Physics

Why can an ML model learn from part of a distribution or data set, and generalize to the rest of it? Why can I learn some useful heuristics or principles in a particular context, and later apply them in other areas of my life?

The answer is obvious: because there are some underlying regularities between the parts I train on and the ones I test on. In the ML example, generalization won't work when approximating a function which is a completely random jumble of points.
Also, quantitatively, the more regular the function is, the better generalization will work. For example, polynomials of lower degree require less data points to pin down. Same goes for periodic functions. Also, a function with lower Lipschitz constant will allow for better bounding of the values in un-observed points.

So it must be that the variables we [...]

---

Outline:

(00:07) 1. Physics

(01:31) 2. Anthropics

(02:03) 3. Dust

---

First published:

Source:

Narrated by TYPE III AUDIO.