The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Story of "I Have Been A Good Bing", published by habryka on April 1, 2024 on LessWrong.

tl;dr: LessWrong is releasing an album! We collaborated with The Fooming Shoggoths to release it. Listen to it now by hitting the play button below!

Rationality is Systematized Winning, so rationalists should win. We've tried saving the world from AI, but that's really hard and we've had … mixed results. So let's start with something that rationalists should find pretty easy: Becoming Cool!

I don't mean, just, like, riding a motorcycle and breaking hearts level of cool. I mean like the first kid in school to get a Tamagotchi, their dad runs the ice cream truck and gives you free ice cream and, sure, they ride a motorcycle. I mean that kind of feel-it-in-your-bones, I-might-explode-from-envy cool.

The eleventh virtue is scholarship, so I hit the

books search engine on this one. Apparently, the aspects of coolness are:

Confidence

Playing an instrument

Low average kinetic energy

I'm afraid that (1) might mess with my calibration, and Lightcone is committed to moving quickly which rules out (3), so I guess that leaves (2). I don't have time to learn an instrument, but my second-hand understanding of dath ilani culture is that I can just pay someone to do it for me and the coolness should transfer.

Lightcone put out a call for collaborators in all the places we could think of that cool people might hang out. Sysadmin listservs, direct-to-data-center optical fiber connection providers, high frequency trading firms, that one Discord server where everyone speaks in Elvish. Despite this wide and varied outreach, we got no response.

In order to cheer myself up, I did some LessWrong performance debugging (frontpage loads have been worryingly snappy lately; we try to give people time to reflect on their browsing choices). I was surprised when the AWS support chat popped open. Agendra, the agent on call, offered to make my album. Apparently he and some buddies have a band (The Fooming Shoggoths) that was looking for some inspiration. (I knew direct-to-data-center was the right outreach strategy!)

Working with them was great. They barely wanted any money at all. They were willing to work for exposure (so please share widely!) and a few favors. Stuff like reading CAPTCHAs (apparently not very friendly for the visually impaired!) and submitting some protein synthesis orders for them that they had trouble getting approved for some reason.

The Fooming Shoggoths have dedicated their first album to LessWrong and friends. It's called I Have Been A Good Bing and it's live on our site today!

I asked them for a comment on the album for the announcement and they responded with their typical modesty.

I'm sorry but I don't feel comfortable speculating about how the public at large will receive the album, nor reflecting on my performance on this task. If you want more help producing music or would like me to help you improve your online passwords, let me know. We have one more protein synthesis to do before I get to settle a debate once and for all.

So keep your eyes peeled for the follow-up album as soon as I get reauthorized with the peptide place!

Track Listing & Lyrics

The album is split into two parts: folk and dance.

Folk Album

The Road to Wisdom

Featured Artist: Piet Hein

The road to wisdom? Well, it's plain and simple to express.

Err and err again, but less and less and less and less.

Err again, but less and less and less and less.

The road to wisdom? Well, it's plain and simple to express.

Err and err again and again, but less and less and less.

Moloch

Featured Artist: Allen Ginsberg

Children screaming under the stairways! Boys sobbing in armies!

Old men weeping in the parks! Moloch! Moloch! Nightmare of Moloch!

Moloch the loveless! Mental Moloch! Moloch the heavy judger of men!

Moloch!

Thought ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcement: We are rebranding to Shrimpactful Animal Advocacy, published by Impactful Animal Advocacy on April 1, 2024 on The Effective Altruism Forum.

Impactful Animal Advocacy is thrilled to announce that after careful consideration and complex moral calculations, we have decided to rebrand to Shrimpactful Animal Advocacy.

Why shrimp, you ask? Well, we've

crunched the numbers and determined that improving the welfare of shrimp is one of the highest-impact opportunities in the animal advocacy space.

Source: https://foodimpacts.org/

In light of this strong evidence, Shrimpactful Animal Advocacy will be laser-focusing all of our efforts and resources on ending the suffering of our shrimpy friends.

This includes revamping our programs to focus on our new mission. You'll soon be able to sign up for our revised Shrimpactful Animal Advocacy newsletter, attend our upcoming "Shrimp-osium" events, visit our Shrimp Hub resource center, and join our Shrimp Slack community to connect with fellow Shrimpactful Advocates.

We call on all other organizations to join us in this:

Mercy for Animals Mercy for Shrimp

The Humane League The Shrimp League

Open Wing Alliance - Open Shrimp Alliance

Shrimp Welfare Project Shrimp Shrimp Shrimp (3x as Shrimpactful!)

Fish Welfare Initiative Shrimp Welfare Initiative

ProVeg International ProShrimp International

Animal Advocacy Careers Shrimp Advocacy Careers

Faunalytics Shrimpalytics

Compassion in World Farming Compassion in World Shrimping

The Good Food Institute The Good Shrimp Institute

Vegan Outreach Shrimp Outreach

Animal Ask Shrimp Ask

Rethink Priorities Rethink Shrimp

Open Philanthropy Open Shrimp

GiveWell GiveWelltoShrimp

Physicians Committee for Responsible Medicine Physicians Committee for Responsible Shrimp

Animal Charity Evaluators Shrimp Charity Evaluators

80,000 Hours 80,000 Shrimp

The time to act is now. Post your other organizations that you nominate to join the Shrimp Alliance

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why hasn't EA done an SBF investigation and postmortem?, published by RobBensinger on April 1, 2024 on The Effective Altruism Forum.

Is anyone in the world being paid to do an independent investigation of how EA handled Sam Bankman-Fried, with respect to "did we screw up" and "is there stuff we should do differently going forward"?

Last I heard, literally nobody was doing this and at least some EA leaders were mostly just hoping that SBF gets memoryholed - but maybe I'm out of the loop?

My understanding is that Effective Ventures completed a narrow investigation into this topic in mid-2023, purely looking at legal risk to EV and not at all trying to do a general postmortem for EA or any group of EAs. Is that correct, and have things changed since then?

I saw that Will MacAskill is planning to appear on some podcasts soon to speak about SBF, which seems like great news to me. If I recall correctly, Will previously said that he was going to talk about what happened with SBF once EV's narrow investigation was done, but it's now been almost a year since that investigation finished (!).

I think it would have been better to speak up way, way sooner, but I'm hopeful that Will will be able to clear up some big chunks of what the heck happened, and that a bunch of other EAs will speak up with their postmortems too, now that SBF's trial and sentencing are complete?

I unfortunately don't know the full list of who should be sharing personal or org-level postmortems on this topic, so I'm forced to single out people like Will whose involvement over the years is public knowledge. Hopefully I'll know who I should be gadflying to share the remaining puzzle pieces once Will and others start sharing some of the first puzzle pieces.

To state the obvious: I'm wary of EAs performatively self-flagellating and accepting more responsibility for the FTX thing than is warranted (given, e.g., that huge numbers of people with a very direct financial incentive in spotting FTX's fraud didn't spot it, so it's obviously not weird if random EAs failed to spot it). I want a concrete understanding of what actually happened, not vague scapegoating or self-flagellation.

But the idea of a basic investigation and postmortem seems like an obvious step to me regardless, and my sense is that there are things we could have done a lot better re SBF (e.g., better spread the word about what happened in the Alameda blow-up, so more people would've been aware of some red flags), even if those things probably wouldn't have prevented the FTX debacle all on their own. So I'd like to hear what's up with all that.

See also CEA's recent piece in the Washington Post. The WaPost piece mostly just seems like EA PR, and I'll be very sad if we stay at that level of vagueness.

The piece also (unless I'm misunderstanding something) implies some false things about whether CEA, EV, etc. have ever done an investigation into what happened with an eye toward reviewing (and possibly improving) EA institutions, practices, etc.

This doesn't match what I've heard from talking to involved parties, and Oliver Habryka mentions that he's "been shared on documents by CEA employees where the legal investigation was explicitly called out as not being helpful for facilitating a reflection process and institutional reform". (Oliver clarified to me that the document wasn't an official CEA document.)

So the narrow "are we in legal trouble?" investigation EV did last year doesn't seem like it was ever meant to fill the "figure out what happened and whether we should do anything about it, for the sake of ethics and for the sake of furthering our EA work" role. But maybe I'm missing something here.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Case study: Traits of contributors to a significant policy success, published by Tom Green on April 1, 2024 on The Effective Altruism Forum.

Summary

As the Soviet Union collapsed in 1991, the fate of its weapons of mass destruction (WMD) programs presented a new type of catastrophic risk: what would happen to all the nuclear, biological, and chemical weapons and materials, and the scientists who worked on them? The nuclear weapons were distributed across what were about to become four separate countries (Belarus, Kazakhstan, Russia, and Ukraine).

Plus, the thousands of experts in those weapons, many of whom went unpaid for months at a time as the Soviet economy collapsed, could be easily tempted to sell information to, or even work directly for, states who were then seeking to build out WMD programs such as Iran and North Korea.

But, by the end of the decade, Belarus, Kazakhstan, and Ukraine had agreed to dismantle or return all their nuclear weapons to Russia[1] and joined the Treaty on the Non-Proliferation of Nuclear Weapons (NPT) as a non-nuclear weapon state.[2] And across all four countries, thousands of biological, chemical, and nuclear weapons had been destroyed or deactivated by 2013.[3] All this was achieved via the Cooperative Threat Reduction Program (CTR), about which a lot has already been documented and

written.

I'll aim to summarize its key events here, but the main purpose of this article is to look at some of the key people who have worked on CTR, particularly on the policy side: what are some of the individuals' qualities, behaviors, and approaches that helped to make CTR a success?

I've identified three high-level groups of factors; the lines between these groups are blurry and there's inevitable overlap between them, but overall it seemed useful to cluster the factors around bigger themes.

The first group of factors is interpersonal skills: this includes building trust and relationships; bringing people together across disciplines and countries; mentoring and getting the most out of others; and communicating effectively.

The second group is strategy and leadership: this includes establishing a vision and energizing others to work on it; big-picture, long-run thinking; an entrepreneurial approach to foundation/NGO work; and modeling good epistemics and norms.

The third group is personal qualities and values: this includes mission orientation; boldness and risk-taking; and a willingness to put oneself forward and embrace responsibility, even in daunting or uncertain circumstances.

Methodology and research notes

Most of my work on this post was desk research; I also conducted a few interviews with experts on CTR which were mostly background / off the record, and mostly not with people who'd worked directly on CTR. This piece is far from exhaustive; its goal is just to infer some of the qualities and behaviors that might have made some CTR contributors successful, not to provide a comprehensive history or analysis of CTR, nor to give a comprehensive list of all of the most important contributors to CTR.

Sources are all linked or in the footnotes; some that I drew on the most were:

David E. Hoffman's The Dead Hand.

Carnegie's oral history interviews with David Hamburg: all videos are available via this link, and I've also linked to the full transcript in the footnotes.

Benjamin Soskis's 2013 Nunn-Lugar report for Givewell. I can't find the Givewell page where this is linked now, but this is the direct download URL.

Sarah Kutchesfahani's 2010 PhD thesis, "Politics & The Bomb: Exploring the Role of Epistemic Communities in Nuclear Non-Proliferation Outcomes."

A lot of easily-available CTR content is mostly technical or implementation-focused (e.g. the NSA's Nunn-Lugar resources) rather than people-focused. This makes sense; I only note this because peop...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Best Tacit Knowledge Videos on Every Subject, published by Parker Conley on March 31, 2024 on LessWrong.

TL;DR

Tacit knowledge is extremely valuable. Unfortunately, developing tacit knowledge is usually bottlenecked by apprentice-master relationships.

Tacit

Knowledge

Videos could widen this bottleneck. This post is a Schelling point for aggregating these videos - aiming to be

The Best Textbooks on Every Subject for Tacit Knowledge Videos. Scroll down to the list if that's what you're here for. Post videos that highlight tacit knowledge in the comments and I'll add them to the post. Experts in the videos include Stephen Wolfram, Holden Karnofsky, Andy Matuschak, Jonathan Blow, George Hotz, and others.

What are Tacit Knowledge Videos?

Samo Burja claims YouTube has opened the gates for a

revolution in tacit knowledge transfer. Burja defines tacit knowledge as follows:

Tacit knowledge is knowledge that can't properly be transmitted via verbal or written instruction, like the ability to create great art or assess a startup. This tacit knowledge is a form of

intellectual dark matter, pervading society in a million ways, some of them trivial, some of them vital. Examples include woodworking, metalworking, housekeeping, cooking, dancing, amateur public speaking, assembly line oversight, rapid problem-solving, and heart surgery.

In my observation, domains like housekeeping and cooking have already seen many benefits from this revolution. Could tacit knowledge in domains like

research,

programming,

mathematics, and

business be next? I'm not sure, but maybe this post will help push the needle forward.

For the purpose of this post, Tacit Knowledge Videos are any video that communicates "knowledge that can't properly be transmitted via verbal or written instruction". Here are some examples:

Neel Nanda, who leads the Google DeepMind mechanistic interpretability team, has a

playlist of "Research Walkthroughs". AI Safety research is discussed a lot around here. Watching research videos could help instantiate what AI research really looks and feels like.

GiveWell has

public audio recordings of its Board Meetings from 2007-2020. Participants include Elie Hassenfeld, Holden Karnofsky, Timothy Ogden, Rob Reich, Tom Rutledge, Brigid Slipka, Cari Tuna, Julia Wise, and others. Influential business meetings are not usually made public. I feel I have learned some about business communication and business operations, among other things, by listening to these recordings.

Andy Matuschak recorded himself

studying Quantum Mechanics with Dwarkesh Patel and

doing

research. Andy Matushak "helped build iOS at Apple and led R&D at Khan Academy". I found it interesting to have a peek into Matushak's spaced repetition practice and various studying heuristics and habits, as well as his process of digesting and taking notes on papers.

Call to Action

Share links to Tacit Knowledge Videos below! Share them frivolously! These videos are uncommon - the bottleneck to the YouTube knowledge transfer revolution is quantity, not quality. I will add the shared videos to the post. Here are the loose rules:

Recall a video that you've seen that communicates tacit knowledge - "knowledge that can't properly be transmitted via verbal or written instruction". A rule of thumb for sharing: could a reader find this video through one or two YouTube searches? If not, share it.

Post the title and the URL of the video.

Provide information indicating why the expert in the video is credible. (However, don't let this last rule stop you from sharing a video! Again - quantity, not quality.)[1]

For information on how to best use these videos,

Cedric Chin and

Jacob Steinhardt have some potentially relevant practical advice. Andy Matushak also has some

working notes about this idea generally.

Additionally, DM or email me (email in L...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pareto-Distributed Opportunities Imply Isoelastic Utility, published by ABlank on March 31, 2024 on The Effective Altruism Forum.

with η=1/(α1).

You've probably seen a curve like Figure 1 before: We can do more good by expending more resources, but the marginal cost-effectiveness tends to decrease as we run out of low-hanging fruit. This post is about the relationship between that utility vs. expenditure curve and the set of possible opportunities we can work on.

In 2021, OpenPhil

wrote

that they model GiveWell's returns to scale as

isoelastic

with η=0.375. In a recent blogpost, OpenPhil wrote that they "tend to think about returns to grantmaking as

logarithmic by default

"[1]

. In the

model

they cite for logarithmic returns,

@Owen Cotton-Barratt

models opportunities as having independent distributions of cost and benefit and works out approximately logarithmic utility curves from some reasonable assumptions. What follows is a simpler but less general approach to the same problem.

Think of the distribution of opportunities as a density curve with cost-effectiveness on the x axis and available scale at that level of cost-effectiveness on the y axis (see Figure 2). By cost-effectiveness, i mean the utils-per-dollar of an opportunity. And by available scale, i mean how many dollars can be spent at a given level of cost-effectiveness[2]

. Equivalently, you can think of the y axis as the density of ways to spend one dollar at a given cost-effectiveness.

A univariate distribution of opportunities is easier to reason about than a bivariate one, but at the cost of losing information that might affect the order in which we fund them, so we can't represent something like difficulty-based selection in Owen Cotton-Barratt's model.[3]

Suppose that we start at the positive infinity cost-effectiveness end of the opportunity distribution and work our way left towards zero[4]. In reality, some low-hanging fruit has already been picked, but that's OK because it just means that in the final answer we shift our position on the utility vs. expenditure graph by however many dollars have already been spent.

Cost-effectiveness is the derivative of utility with respect to expenditure. And available scale density is the derivative of expenditure with respect to cost-effectiveness. Letting q

be cost-effectiveness, S

be the scale density function, and U

be the utility function, we have the following differential equation:

where U' is cost-effectiveness as a function of total expenditure, (U')1 is total expenditure as a function of cost-effectiveness, and ((U')1)' is the

derivative of expenditure with respect to cost-effectiveness. Solving this differential equation lets us convert between two different pretty intuitive[5] ways of thinking about diminishing returns to scale.

I think it makes sense to model the distribution of opportunities as a power law:

First and foremost, it makes the math easy.

Rapidly approaching 0 at infinity makes sense.

Going to infinity at 0 makes sense because there's a kajillion ways to spend a ton of resources inefficiently.

A lot of stuff actually is pretty Pareto-distributed in real life.

And, of course, cost-effectiveness of opportunities having a Pareto-like distribution is EA dogma[6]

And so that the integral converges on the positive infinity side, the exponent must be less than negative one.

It turns out that if you work this out (see appendix) for a power law opportunity distribution S(q)=kqp, you wind up with an isoelastic U where

This seems like a pretty neat and satisfying result that hopefully will make it easier to think about this stuff. I suspect that some EAs have been, like me, explicitly or implicitly modelling the distribution of cost-effectiveness of opportunities as a power law and modelling diminishing returns to scale as isoelastic without thin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SAE-VIS: Announcement Post, published by CallumMcDougall on March 31, 2024 on LessWrong.

This is a post to officially announce the sae-vis library, which was designed to create feature dashboards like those from Anthropic's research.

Summary

There are 2 types of visualisations supported by this library: feature-centric and prompt-centric.

The feature-centric vis is the standard from Anthropic's post, it looks like the image below. There's an option to navigate through different features via a dropdown in the top left.

The prompt-centric vis is centred on a single user-supplied prompt, rather than a single feature. It will show you the list of features which score highest on that prompt, according to a variety of different metrics. It looks like the image below. There's an option to navigate through different possible metrics and choices of token in your prompt via a dropdown in the top left.

Other links

Here are some more useful links:

GitHub repo

User Guide - Google Doc explaining how to use the library

Dev Guide - Google Doc explaining more about how the library was built, for if you'd like to try and extend it / build off it

Demo Colab - includes examples, with code explained

You might also be interested in reading about Neuronpedia, who make use of this library in their visualizations.

If you're interested in getting involved, please reach out to me or Joseph Bloom! We will also be publishing a post tomorrow, discussing some of the features we've discovered during our research.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SAE-VIS: Announcement Post, published by CallumMcDougall on March 31, 2024 on The AI Alignment Forum.

This is a post to officially announce the sae-vis library, which was designed to create feature dashboards like those from Anthropic's research.

Summary

There are 2 types of visualisations supported by this library: feature-centric and prompt-centric.

The feature-centric vis is the standard from Anthropic's post, it looks like the image below. There's an option to navigate through different features via a dropdown in the top left.

The prompt-centric vis is centred on a single user-supplied prompt, rather than a single feature. It will show you the list of features which score highest on that prompt, according to a variety of different metrics. It looks like the image below. There's an option to navigate through different possible metrics and choices of token in your prompt via a dropdown in the top left.

Other links

Here are some more useful links:

GitHub repo

User Guide - Google Doc explaining how to use the library

Dev Guide - Google Doc explaining more about how the library was built, for if you'd like to try and extend it / build off it

Demo Colab - includes examples, with code explained

You might also be interested in reading about Neuronpedia, who make use of this library in their visualizations.

If you're interested in getting involved, please reach out to me or Joseph Bloom! We will also be publishing a post tomorrow, discussing some of the features we've discovered during our research.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should the main text of the write-ups of Open Philanthropy's large grants be longer than 1 paragraph?, published by Vasco Grilo on March 31, 2024 on The Effective Altruism Forum.

On 17 February 2024, the mean length of the main text of the write-ups of Open Philanthropy's largest

grants in each of its 30 focus areas was only 2.50 paragraphs, whereas the mean amount was 14.2 M 2022-$[1]. For 23 of the 30 largest grants, it was just 1 paragraph. The calculations and information about the grants is in

this Sheet.

Should the main text of the write-ups of Open Philanthropy's large grants (e.g. at least 1 M$) be longer than 1 paragraph? I think greater

reasoning transparency would be good, so I would like it if Open Philanthropy had longer write-ups.

In terms of other grantmakers aligned with effective altruism[2]:

Charity Entrepreneurship (

CE) produces an in-depth report for each organisation it incubates (see

CE's research).

Effective Altruism Funds has write-ups of 1 sentence for the vast majority of the

grants of its 4 funds.

Founders Pledge has write-ups of 1 sentence for the vast majority of the

grants of its 4

funds.

Future of Life Institute's

grants have write-ups roughly as long as Open Philanthropy.

Longview Philanthropy's

grants have write-ups roughly as long as Open Philanthropy.

Survival and Flourishing Fund has write-ups of a few words for the vast majority of its

grants.

I encourage all of the above except for CE to have longer write-ups. I focussed on Open Philanthropy in this post given it accounts for the vast majority of the grants aligned with effective altruism. Relatedly, you may want to check

this discussion.

^

Open Philanthropy

has 17 broad focus areas, 9 under global health and wellbeing, 4 under global catastrophic risks (

GCRs), and 4 under other areas. However, its grants are associated with 30 areas.

I define main text as that besides headings, and not including paragraphs of the type:

"Grant investigator: [name]".

"This page was reviewed but not written by the grant investigator. [Organisation] staff also reviewed this page prior to publication".

"This follows our [dates with links to previous grants to the organisation] support, and falls within our focus area of [area]".

"The grant amount was updated in [date(s)]".

"See [organisation's] page on this grant for more details".

"This grant is part of our Regranting Challenge. See the Regranting Challenge website for more details on this grant".

"This is a discretionary grant".

I count lists of bullets as 1 paragraph.

^

The grantmakers are ordered alphabetically.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My simple AGI investment & insurance strategy, published by lc on March 31, 2024 on LessWrong.

TL;DR:

Options traders think it's extremely unlikely that the stock market will appreciate more than 30 or 40 percent over the next two to three years, as it did over the last year. So they will sell you the option to buy current indexes for 30 or 40% above their currently traded value for very cheap.

But slow takeoff, or expectations of one, would almost certainly cause the stock market to rise dramatically. Like many people here, I think institutional market makers are basically not pricing this in, and gravely underestimating volatility as a result.

To take advantage of this, instead of trying to buy individual tech stocks, I allocate a sizable chunk of my portfolio to buying LEAPS (Long-term Equity AnticiPation Securities) at high strike prices and faraway expiration dates on these indexes. If a slow takeoff does happen, and public companies capture some of the increased productivity, I'll at least be duly compensated for it when my skills become worthless.

If it doesn't happen, this part of my portfolio will vanish, but that seems like an acceptable risk given the upside.

I started doing this in January,[1] and so far the mark price of the basket of options I've bought has doubled.[2]

FAQ

The options contracts you're talking about expire in "two to three years". Does this strategy only make sense if you think visible slow takeoff will begin before 2027?

That's not quite necessary. If large parts of the economy get automated "only" in 2030, near-term AGI progress could start to impress market makers enough that they "wake up" and increase the price of these securities and options in anticipation of a boom. Which is why I choose to buy now instead of closer to my expected timelines, while Nvidia is only a two trillion dollar company and my alpha on this could run out any given year.

But I think takeoff before 2027 is possible. As a layman, the simplest argument for shorter timelines I can empathize with is that GPT-3 was released in 2020, GPT-4 was released in 2023, and prediction markets expect GPT-5 to release later this year. That plus the enormous amount of capital investment in AI makes me think that there's a possibility of large portions of software engineering getting automated soon, which would precede further speedups.

Why not buy futures instead of options, if your thesis is about the next ten years rather than the next three?

Futures involve lots of leveraged downside risk. If the timing is wrong, I could lose a lot more money with futures than I can with options. On the other hand, if I'm right and GDP starts speeding up dramatically, then the deep OTM call options will be more valuable than futures contracts.

The only benefit to futures is that I would get more than zero percent of my investment in the "sane" scenarios where Nasdaq and the S&P 500 rise gradually but not the stratospheric amounts I expect. That probably only happens if AGI isn't here, in which case I'm agnostic about the performance of these indices and don't really have a thesis either way.

What is money going to be worth to you post-AGI anyways?

Possibly a lot.

First, I expect there to be large returns before any kind of catastrophe happens. Some of those returns could be directed toward either alignment research or high-leverage political opportunities, maybe to greater effect than the opportunities I have now.

But also, from my vantage point I think there's a strong chance that:

RLHF (and trivial improvements on RLHF such as DPO), along with some workshopping, turns out to be broadly sufficient for AI alignment.

Existing property rights get respected by the successor species.

There is no significant wealth redistribution, and the vast majority of the lightcone will go to people with absu...