Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rowing, Steering, Anchoring, Equity, Mutiny, published by Holden Karnofsky on November 30, 2021 on The Effective Altruism Forum. Audio version available at Cold Takes (or search Stitcher, Spotify, Google Podcasts, etc. for "Cold Takes Audio")

This post interrupts the Has Life Gotten Better? series to talk a bit about why it matters what long-run trends in quality of life look like.

I think different people have radically different pictures of what it means to "work toward a better world." I think this explains a number of the biggest chasms between people who think of themselves as well-meaning but don't see the other side that way, and I think different pictures of "where the world is heading by default" are key to the disagreements.

Imagine that the world is a ship. Here are five very different ways one might try to do one's part in "working toward a better life for the people on the ship."

Which of these is the "right" focus for improving the world? One of the things I like about the ship analogy is that it leaves the answer to this question totally unclear! The details of where the ship is currently trying to go, and why, and who's deciding that and what they're like, matter enormously. Depending on those details, any of the five could be by far the most important and meaningful way to make a positive difference.

If the ship is the world, then where are we "headed" by default (what happens if we have more total technology, wealth, and power over our environment)? Who has the power to change that, and how has it been going so far?

These are important questions with genuinely unclear answers. So people with different assumptions about these deep questions can get along very poorly with each other.

I think this sort of taxonomy provides a different angle on people’s differences from the usual discussions of pro/anti-government interventionism.

Next I will give some somewhat more detailed thoughts on the case for and against each of these pictures of "improving the world." This is not so much to educate the reader as to help them understand where I stand, and why I have some of the unusual views I do.

I talk a bit about the "track record" of each category. A lot of the point of this analogy is to highlight the importance of big-picture judgments about history.

Rowing

I use "rowing" to refer to the idea that we can make the world better by focusing on advancing science, technology, growth, etc. - all of which ultimately result in empowerment, helping people do whatever they want to do, more/faster. The idea is that, in some sense, we don't need a specific plan for improving lives: more capabilities, wealth, and empowerment ("moving forward") will naturally result in that.

Rowing is a contentious topic, and it’s contentious in a way that I think cuts across other widely-recognized ideological lines.

To some people, rowing seems like the single most promising way to make the world a better place. People and institutions who give off this vibe include:

Tech entrepreneurs and VCs such as Marc Andreessen (It's Time to Build) and Patrick Collison (progress studies).

Libertarian-ish academics such as Tyler Cowen and Alex Tabarrok (see their books for a sense of this).

The many institutions and funders dedicated to general speeding up of science, such as the Moore Foundation, Howard Hughes Medical Institute, and CZI Biohub.2

The global development world, e.g. nonprofits such as Center for Global Development and institutions such as the World Bank, which seeks to help "developing" countries "develop." This generally includes a large (though not exclusive) focus on economic growth.

I sometimes see rowing-oriented arguments tagged as “pro-market” or even “libertarian,” but I think that isn't a necessary connection. You could argue - and many do - that a lot of the biggest and mo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reneging Prosocially, published by Duncan Sabien on November 30, 2021 on LessWrong.

Author's note: this essay was originally published elsewhere in 2019. It's now being permanently rehosted at this link. I'll be rehosting a small number of other upper-quintile essays from my past writing over the coming weeks.

Quite often, I will make an agreement, and then find myself regretting it. I’ll commit to spending a certain amount of hours helping someone with their problem, or I’ll agree to take part in an outing or a party or a project, or I’ll trade some item for a certain amount of value in return, and then later find that my predictions about how I would feel were pretty far off, and I'm unhappy.

Sometimes, I just suck it up and stick it out. But sometimes I renege on the agreement. In this essay, I’d like to lay out a model for how to renege prosocially, i.e. how to go back on your agreements in a way that strengthens rather than damaging the social fabric (or at the very least leaves it no worse than it was).

Black boxes and APIs

Once, I was a passenger in my father’s car when another car cut us off on the freeway.

“Do you know what he did wrong?” my father asked, gesturing at the other car.

(I was maybe thirteen at the time, and would soon be going through drivers’ education.)

“Uh. He didn’t use his turn signal?”

“Close,” my father answered. “Why is it bad not to use your turn signal?”

I couldn’t come up with anything, and eventually my father continued. “It’s bad because it makes you less predictable,” he said. “Just like the fact that he raced up on us on the right, and then cut in front of us. You pass on the left and not the right because that’s what people are expecting. You signal your lane changes so that people know what you’re about to do. If you drive erratically, everybody else has to change the way they’re driving around you, and then nobody knows how to coordinate. It breaks the pattern, and we rely on the pattern to stay safe.”

(It’s no surprise that I talk the way I do, is it?)

In this essay, I’m going to generally avoid talking about things like people’s internal experiences, and people’s individual character traits, and so forth. I won’t talk about that stuff zero, but most of the time, it doesn’t bear directly on the question of what happens when people interact with each other, except when filtered through a layer of[X internal states] → [Y external actions].

The guy in the other car could have been a jerk, or he could have been tired and inattentive, or he could have been running late, or he could have just had a different sense of what constitutes safe driving and traffic violations than my father.

None of that particularly matters, though, when the question is something like “what behaviors allow people to coordinate on the road such that no one dies?” You can treat the person in the other car as a black box, and treat my father as a black box, and simply ask “are the behaviors emerging from these black boxes likely to combine smoothly and effectively, or not?”

Similarly, when we talk about reneging on agreements, I want to avoid questions of what’s-going-on-in-their-soul-though, since we can’t really ever be sure. They say they’re too sick to show up, but maybe they’re just tired and afraid you won’t validate that, or maybe they just don’t like you, and are trying to be gentle, or or or or or.

What matters, I claim, is something like what’s comprehensible, what’s sustainable, what’s verifiably fair or fair-approximate from the outside. What, if you could poll 1000 people on r/amitheasshole, would the consensus be? That consensus isn’t always right, in the moral sense, but it’s extremely well-calibrated on what tends to work, when people are rubbing elbows with each other, and what doesn’t.

Another way to think of this is that the social constr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reneging Prosocially, published by Duncan Sabien on November 30, 2021 on LessWrong.

Author's note: this essay was originally published elsewhere in 2019. It's now being permanently rehosted at this link. I'll be rehosting a small number of other upper-quintile essays from my past writing over the coming weeks.

Quite often, I will make an agreement, and then find myself regretting it. I’ll commit to spending a certain amount of hours helping someone with their problem, or I’ll agree to take part in an outing or a party or a project, or I’ll trade some item for a certain amount of value in return, and then later find that my predictions about how I would feel were pretty far off, and I'm unhappy.

Sometimes, I just suck it up and stick it out. But sometimes I renege on the agreement. In this essay, I’d like to lay out a model for how to renege prosocially, i.e. how to go back on your agreements in a way that strengthens rather than damaging the social fabric (or at the very least leaves it no worse than it was).

Black boxes and APIs

Once, I was a passenger in my father’s car when another car cut us off on the freeway.

“Do you know what he did wrong?” my father asked, gesturing at the other car.

(I was maybe thirteen at the time, and would soon be going through drivers’ education.)

“Uh. He didn’t use his turn signal?”

“Close,” my father answered. “Why is it bad not to use your turn signal?”

I couldn’t come up with anything, and eventually my father continued. “It’s bad because it makes you less predictable,” he said. “Just like the fact that he raced up on us on the right, and then cut in front of us. You pass on the left and not the right because that’s what people are expecting. You signal your lane changes so that people know what you’re about to do. If you drive erratically, everybody else has to change the way they’re driving around you, and then nobody knows how to coordinate. It breaks the pattern, and we rely on the pattern to stay safe.”

(It’s no surprise that I talk the way I do, is it?)

In this essay, I’m going to generally avoid talking about things like people’s internal experiences, and people’s individual character traits, and so forth. I won’t talk about that stuff zero, but most of the time, it doesn’t bear directly on the question of what happens when people interact with each other, except when filtered through a layer of[X internal states] → [Y external actions].

The guy in the other car could have been a jerk, or he could have been tired and inattentive, or he could have been running late, or he could have just had a different sense of what constitutes safe driving and traffic violations than my father.

None of that particularly matters, though, when the question is something like “what behaviors allow people to coordinate on the road such that no one dies?” You can treat the person in the other car as a black box, and treat my father as a black box, and simply ask “are the behaviors emerging from these black boxes likely to combine smoothly and effectively, or not?”

Similarly, when we talk about reneging on agreements, I want to avoid questions of what’s-going-on-in-their-soul-though, since we can’t really ever be sure. They say they’re too sick to show up, but maybe they’re just tired and afraid you won’t validate that, or maybe they just don’t like you, and are trying to be gentle, or or or or or.

What matters, I claim, is something like what’s comprehensible, what’s sustainable, what’s verifiably fair or fair-approximate from the outside. What, if you could poll 1000 people on r/amitheasshole, would the consensus be? That consensus isn’t always right, in the moral sense, but it’s extremely well-calibrated on what tends to work, when people are rubbing elbows with each other, and what doesn’t.

Another way to think of this is that the social constr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Infra-Bayesian physicalism: proofs part II, published by Vanessa Kosoy on November 30, 2021 on The AI Alignment Forum.

This post is an appendix to "Infra-Bayesian physicalism: a formal theory of naturalized induction".

Proposition 2.16: Consider some Γ0, Γ1, Φ and Θ∈□c(Γ0×Γ1×Φ). Define π:elΓ0×Γ1→elΓ0 by π(y,z,α):=(y,{y′∈Γ0∣(y′,z)∈α}). Then, (π×idΦ)∗BrΓ0×Γ1(Θ)⊆prelΓ0×ΦBrΓ0(Θ)⊆BrΓ0(prΓ0×ΦΘ)

We can prove the second subset inclusion directly as a corollary of Proposition 2.10, just let the t of Proposition 2.10 be the projection function Γ1×Φ→Φ, so that just leaves the first subset inclusion direction. If you've seen the proofs so far, you know we do a thing where we try to show subset inclusion with expectations of functions and inequalities instead. And that the proofs all proceed by transforming the expectations until we get a maximum over contribution expectation values, and that's always where the hard part of proving the inequalities shows up. So, let's just get that part over with, an interested reader can work it out with previous proofs as a guide. Unusually, we'll be keeping track of identity functions here.

Plugging in some f, and doing our usual activities to get every term into the appropriate form, we can get this result if we manage to show that

maxθ′∈BrΓ0×Γ1(Θ)(π×idΦ)∗θ′(λy0α0x.f(y0,x,α0)) ≤maxθ∈BrΓ0(Θ)prelΓ0×Φθ(λy0α0x.f(y0,x,α0)) So, to establish this, we'll show that, given some θ′∈BrΓ0×Γ1(Θ), we have

(π×idΓ1×Φ)∗(θ′)∈BrΓ0(Θ), and that

prelΓ0×Φ((π×idΓ1×Φ)∗θ′)=(π×idΦ)∗θ′ because, if we show that, then it means that BrΓ0(Θ) is a rich enough set for the right-hand side of the equation to match anything the left-hand-side can put out.

First off, prelΓ0×Φ((π×idΓ1×Φ)∗θ′)=(π×idΦ)∗θ′ is pretty trivial to show. The only difference between the two processes is that the Γ1 coordinate of θ′ is discarded immediately on the right-hand-side, and it's preserved for one step and then discarded on the second step for the left-hand-side.

Now for our inequality of interest. Let θ′∈BrΓ0×Γ1(Θ), and we're trying to show that

(π×idΓ1×Φ)∗(θ′)∈BrΓ0(Θ) First off, showing the support condition for (π×idΓ1×Φ)∗(θ′) which is somewhat nontrivial this time around. We start off with a guarantee that (y0,y1)∈α. This happens iff

y0∈{y′0|(y′0,y1)∈α}=π(y0,y1,α)2Γ0 And so, we get that y0∈α0 is guaranteed for that pushforward, support condition established.

endofunction condition time. It's important to remember that we want to treat elΓ0 as the computation side of things, and Γ1×Φ as the environment side of things, for our bridge transform we're working with. s:Γ0→Γ0 and g:Γ0×Γ1×Φ→[0,1]. Begin. (π×idΓ1×Φ)∗θ′(λy0y1α0x.χs(y0)∈α0g(s(y0),y1,x)) =θ′(λy0y1αx.χs(y0)∈π(y0,α,y1)2Γ0g(s(y0),y1,x)) Let's unpack precisely what that set is. =θ′(λy0y1αx.χs(y0)∈{y′0|(y′0,y1)∈α}g(s(y0),y1,x)) =θ′(λy0y1αx.χ(s(y0),y1)∈αg(s(y0),y1,x)) And we can rewrite the endofunction a little bit

=θ′(λy0y1αx.χ(s×idΓ1)(y0,y1)∈αg((s×idΓ1)(y0,y1),x)) And finally apply our endofunction condition, since we've now got the function in a form that's treating y0,y1 as part of the computational universe... ≤Θ(λy0y1x.g(y0,y1,x)) And we're done, this establishes our desired result.

Proposition 2.17: Br(Θ) is a continuous function of Θ.

The way this proof will work is by describing a composition of functions that makes Br(Θ) from Θ, and then showing that each of these functions is continuous, if elΓ×Φ is a finite set.

Claim: The bridge transform of some Θ is equal to (using χelΓ to denote restricting an ultradistribution to the event y∈α and χ−1elΓ to denote the inverse of said function, mapping an ultradistribution on elΓ to the largest ultradistribution that could have produced it via restriction) χelΓ(⋂s:Γ→Γs∗(χ−1elΓ(ι∗(pr∗(Θ))))) Breaking down the unfamilar notation, the type of pr is elΓ×Φ→Γ×Φ, just the usual projection. That asteris...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Infra-Bayesian physicalism: proofs part I, published by Vanessa Kosoy on November 30, 2021 on The AI Alignment Forum.

This post is an appendix to "Infra-Bayesian physicalism: a formal theory of naturalized induction".

Lemma 1: prΓ×Φ(χy∈α(s∗(θ)))∈Θ occurs for all s:Γ→Γ iff, for all g:Γ×Φ→[0,1] and s:Γ→Γ, θ(λyαx.χs(y)∈αg(s(y),x))≤Θ(λyx.g(y,x))

This lemma will be implicitly used all over the place, in order to deal with the "presence in the bridge transform" condition algebraically, in terms of function expectations. The bulk of the conditions for a contribution to lie in the bridge transform is this endofunction condition (the support condition is trivial most of the time), so it's advantageous to reformulate it. First-off, we have that

prΓ×Φ(χy∈α(s∗(θ)))∈Θ iff, for all g:Γ×Φ→[0,1], we have

prΓ×Φ(χy∈α(s∗(θ)))(λyx.g(y,x))≤Θ(λyx.g(y,x)) By LF-duality for ultradistributions, proved in Less Basic Inframeasure theory. A contribution lies in an ultracontribution set iff its expectations, w.r.t. all functions, are less than or equal to the ultracontribution expectations.

Now, we just need to unpack the left-hand-side into our desired form. Start off with

prΓ×Φ(χy∈α(s∗(θ)))(λyx.g(y,x)) Apply how projections work

=χy∈α(s∗(θ))(λyαx.g(y,x)) Now, we can move the indicator function into the function we're taking the expectation again, because there's no difference between deleting all measure outside of an event, or taking the expectation of a function that's 0 outside of that event. So, we get

=s∗(θ)(λyαx.χy∈αg(y,x)) Then we use how pushforwards are defined in probability theory. =θ(λyαx.χs(y)∈αg(s(y),x)) And that's our desired form. So, we've shown that for any particular s:Γ→Γ, we have prΓ×Φ(χy∈α(s∗(θ)))∈Θ iff, for all g:Γ×Φ→[0,1], θ(λyαx.χs(y)∈αg(s(y),x))≤Θ(λyx.g(y,x)) And so, we get our desired iff statement by going from the iff for one s to the iff for all s.

Proposition 2.1: For any Γ, Φ and Θ∈□c(Γ×Φ), Br(Θ) exists and satisfies prΓ×ΦBr(Θ)=Θ. In particular, if Θ∈□(Γ×Φ) then Br(Θ)∈□(elΓ×Φ).

Proof sketch: We'll show that for any particular contribution in θ∈Θ, there's a contribution θ∗ which lies within Br(Θ) that projects down to equal θ. And then, show the other direction, that any contribution in Br(Θ) lands within Θ when you project it down. Thus, the projection of Br(Θ) must be Θ exactly.

For the first direction, given some θ∈Θ, let θ∗:=θ⋉(λy.{y}). Note that since θ∈Δc(Γ×Φ), this means that θ∗∈Δc(Γ×2Γ×Φ), so the type signatures line up. Clearly, projecting θ∗ down to Γ×Φ makes θ again. So that leaves showing that θ∗∈Br(Θ). Applying Lemma 1, an equivalent way of stating the bridge transform is that it consists precisely of all the θ′∈Δc(Γ×2Γ×Φ) s.t. for all s:Γ→Γ and g:Γ×Φ→[0,1], θ′(λyαx.χs(y)∈αg(s(y),x))≤Θ(λyx.g(y,x)) and also, supp θ′⊆elΓ×Φ.

Clearly, for our given θ∗, everything works out with the support condition, so that leaves the endofunction condition. Let s,g be arbitrary. θ∗(λyαx.χs(y)∈αg(s(y),x))=(θ⋉(λy.{y}))(λyαx.χs(y)∈αg(s(y),x)) =θ(λyx.δ{y}(λα.χs(y)∈αg(s(y),x)))=θ(λyx.χs(y)∈{y}g(s(y),x)) =θ(λyx.χs(y)=yg(s(y),x))=θ(λyx.χs(y)=yg(y,x))≤θ(λyx.g(y,x)) ≤maxθ∈Θθ(λyx.g(y,x))=Θ(λyx.g(y,x)) In order, the equalities were unpacking the semidirect product, substituting the dirac-delta in, reexpressing the condition for the indicator function, using that s(y)=y inside the indicator function, applying monotonicity of θ, using that θ∈Θ, and then packing up the definition of Θ. And that inequality has been fulfilled, so, yes, θ∗ lies within Br(Θ). Since θ was arbitrary within Θ, this shows that the projection set is as-big-or-bigger than Θ.

Now to show the reverse direction, that anything in the projection set lies within Θ. For any particular θ′∈Θ, remember, it must fulfill, for all g,s, that

θ′(λyαx.χs(y)∈αg(s(y),x))≤Θ(λyx.g(y,x)) So, in particular, we can let s:Γ→Γ be the...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing EffectiveCrypto.org, powered by EA Funds, published by SamDeere on November 30, 2021 on The Effective Altruism Forum.

We’re excited to announce the launch of EffectiveCrypto.org, a donation platform targeting members of the crypto community. The platform is a project of Effective Altruism Funds (and uses the same payments infrastructure), but is targeted towards people who have done well out of the recent crypto boom and haven’t yet engaged much with EA.

Check out EffectiveCrypto.org here

TL;DR: We’d love your help getting the word out, so we’ve listed some ways that you can get involved; you can also read a bit more about our motivation for the project.

How you can help

We think that there’s a great opportunity here to engage the crypto community with effective altruism, and you can help us make EffectiveCrypto.org a success.

Promotion

If you have friends or contacts in the crypto community who you think might find EffectiveCrypto.org valuable, or you’re active in the crypto community, it’d be great if you could share the website. We think that there’s a huge amount of potential for engagement here, and you can make a big difference by helping us to get the word out.

Some things that would be really helpful:

Tweeting about EffectiveCrypto.org. You could talk about why giving crypto effectively is personally motivating for you, or about the benefits of focusing on effectiveness. Any time is fine, but bonus points for tweeting this Thursday (Dec 2 2021). Here’s a sample tweet to get you started.

Posting in relevant crypto Telegram groups/Discords/subreddits that you’re a member of, and who would be interested

You can also follow us on Twitter

Talking about it to EA-adjacent friends in the crypto community

Dank memes??? ¯\_(ツ)_/¯

If you’d like to take a more active role in helping us promote the site, or you have ideas for how we could promote this more widely/go viral, please comment below, or email [email protected].

Feedback

EffectiveCrypto.org is an experiment. We built it quickly to test out the hypothesis that there’s a market for a crypto-focused website that encourages people to donate effectively. However, this means that there’s probably a lot we could do to improve the site in terms of the information we present, the design, and the overall user experience.

If you’re someone in our target audience (or you feel like you can get in the right mindset), please feel free to comment below, or email [email protected] with any suggestions.

Motivation and Goals

At the moment, many cryptocurrencies are at all-time high prices, and many people who hold crypto are looking to donate some of their holdings. Donating appreciated crypto directly is often more tax-advantaged than converting the crypto into a fiat currency first, because doing so avoids capital gains tax in many jurisdictions.

There is already a fair amount of overlap between the crypto community and EA (cf. Sam Bankman-Fried/FTX, Lantern Ventures, etc.), and it seems like the quantitative mindset that many EAs exhibit is also a trait found in many crypto early adopters. We think that a lot of people who hold crypto would probably be interested in donating effectively. However, there aren’t currently many places that are trying to build this bridge explicitly. There are many websites aimed at making it easy to donate crypto (e.g. The Giving Block, Endaoment etc), but typically give a huge range of donation options, with no particular focus on effectiveness. EffectiveCrypto.org makes it easy to give crypto, but aims to showcase a smaller number of donation options that are informed by the principles of effective altruism (basically, the funds and non-profits supported by EA Funds).

Our two main goals for the project are:

Increase overall counterfactual donations to effective ch...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing EffectiveCrypto.org, powered by EA Funds, published by SamDeere on November 30, 2021 on The Effective Altruism Forum.

We’re excited to announce the launch of EffectiveCrypto.org, a donation platform targeting members of the crypto community. The platform is a project of Effective Altruism Funds (and uses the same payments infrastructure), but is targeted towards people who have done well out of the recent crypto boom and haven’t yet engaged much with EA.

Check out EffectiveCrypto.org here

TL;DR: We’d love your help getting the word out, so we’ve listed some ways that you can get involved; you can also read a bit more about our motivation for the project.

How you can help

We think that there’s a great opportunity here to engage the crypto community with effective altruism, and you can help us make EffectiveCrypto.org a success.

Promotion

If you have friends or contacts in the crypto community who you think might find EffectiveCrypto.org valuable, or you’re active in the crypto community, it’d be great if you could share the website. We think that there’s a huge amount of potential for engagement here, and you can make a big difference by helping us to get the word out.

Some things that would be really helpful:

Tweeting about EffectiveCrypto.org. You could talk about why giving crypto effectively is personally motivating for you, or about the benefits of focusing on effectiveness. Any time is fine, but bonus points for tweeting this Thursday (Dec 2 2021). Here’s a sample tweet to get you started.

Posting in relevant crypto Telegram groups/Discords/subreddits that you’re a member of, and who would be interested

You can also follow us on Twitter

Talking about it to EA-adjacent friends in the crypto community

Dank memes??? ¯\_(ツ)_/¯

If you’d like to take a more active role in helping us promote the site, or you have ideas for how we could promote this more widely/go viral, please comment below, or email [email protected].

Feedback

EffectiveCrypto.org is an experiment. We built it quickly to test out the hypothesis that there’s a market for a crypto-focused website that encourages people to donate effectively. However, this means that there’s probably a lot we could do to improve the site in terms of the information we present, the design, and the overall user experience.

If you’re someone in our target audience (or you feel like you can get in the right mindset), please feel free to comment below, or email [email protected] with any suggestions.

Motivation and Goals

At the moment, many cryptocurrencies are at all-time high prices, and many people who hold crypto are looking to donate some of their holdings. Donating appreciated crypto directly is often more tax-advantaged than converting the crypto into a fiat currency first, because doing so avoids capital gains tax in many jurisdictions.

There is already a fair amount of overlap between the crypto community and EA (cf. Sam Bankman-Fried/FTX, Lantern Ventures, etc.), and it seems like the quantitative mindset that many EAs exhibit is also a trait found in many crypto early adopters. We think that a lot of people who hold crypto would probably be interested in donating effectively. However, there aren’t currently many places that are trying to build this bridge explicitly. There are many websites aimed at making it easy to donate crypto (e.g. The Giving Block, Endaoment etc), but typically give a huge range of donation options, with no particular focus on effectiveness. EffectiveCrypto.org makes it easy to give crypto, but aims to showcase a smaller number of donation options that are informed by the principles of effective altruism (basically, the funds and non-profits supported by EA Funds).

Our two main goals for the project are:

Increase overall counterfactual donations to effective ch...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Visible Thoughts Project and Bounty Announcement, published by So8res on November 30, 2021 on The Effective Altruism Forum.

We at MIRI are soliciting help with an AI-alignment project centered around building a dataset, described below. We have $200,000 in prizes for building the first fragments of the dataset, plus an additional $1M prize/budget for anyone who demonstrates the ability to build a larger dataset at scale.

If this project goes well, then it may be the first of a series of prizes we offer for various projects.

Below, I’ll say more about the project, and about the payouts and interim support we’re offering.

The Project

Hypothesis: Language models can be made more understandable (and perhaps also more capable, though this is not the goal) by training them to produce visible thoughts.

We’d like to test this hypothesis by fine-tuning/retraining a language model using a dataset composed of thought-annotated dungeon runs. (In the manner of AI dungeon.)

A normal (un-annotated) dungeon run is a sequence of steps in which the player inputs text actions and the dungeon master responds with text describing what happened in the world as a result.

We’d like a collection of such runs, that are annotated with "visible thoughts" (visible to potential operators or programmers of the system, not to players) describing things like what just happened or is about to happen in the world, what sorts of things the player is probably paying attention to, where the current sources of plot tension are, and so on — the sorts of things a human author would think while acting as a dungeon master. (This is distinct from producing thoughts explaining what happened in the dungeon; “visible thoughts” are meant to play an active role in constructing the output.)

Once we have such a dataset, MIRI’s hope is that present or future technology will be able to train a model or models which iteratively produce visible thoughts along with storytelling, based on user actions plus previous history (including previous thoughts). The goal is to transition the state of AI dungeon technology from “An AI outputs story text in response to actions (and we have no idea how)” to “An AI produces thoughts as visible intermediates on the way to story text, allowing us to watch the AI think about how to design its output, and to verify that we can get different sensible outputs by intervening on the thoughts”.

Here’s an example of the first couple of steps of a thought-annotated dungeon run (or “quest”), in the format MIRI currently thinks is worth trying. Some kinds of thoughts are marked with parentheses and/or brackets; see the next section for details on this.

Thoughts:

[The main plot: Prota is a student at a magical academy that is under attack by zombies.]

[(This is a quest of medium length.)]

(This is the start of the story. The Player should be given enough backstory and immediate description to have some sense of where they are, so the prompt can go on longer than usual. The Player should be told who they are and their role in the universe. The initial prompt should describe Prota being in a magical academy, and should describe the first surprising indication that the academy is under attack by zombies.)

Prompt: Today is your first day of classes at the Magical Academy. You arrived very late last night and missed the tour, but your roommate Lior promised he would help you find your way around. You wake up in the students’ dormitory at dawn, and follow Lior to breakfast. Just as you reach the dining hall, you hear a shout in the distance.

Thoughts:

[Prota missed the tour and doesn’t already know the way around.]

[Prota’s roommate’s name is Lior.]

[Lior is male.]

There are other people in the dining hall who will have also heard the shout, and will also be surprised and reacting to that.

Player action: ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Visible Thoughts Project and Bounty Announcement, published by So8res on November 30, 2021 on The Effective Altruism Forum.

We at MIRI are soliciting help with an AI-alignment project centered around building a dataset, described below. We have $200,000 in prizes for building the first fragments of the dataset, plus an additional $1M prize/budget for anyone who demonstrates the ability to build a larger dataset at scale.

If this project goes well, then it may be the first of a series of prizes we offer for various projects.

Below, I’ll say more about the project, and about the payouts and interim support we’re offering.

The Project

Hypothesis: Language models can be made more understandable (and perhaps also more capable, though this is not the goal) by training them to produce visible thoughts.

We’d like to test this hypothesis by fine-tuning/retraining a language model using a dataset composed of thought-annotated dungeon runs. (In the manner of AI dungeon.)

A normal (un-annotated) dungeon run is a sequence of steps in which the player inputs text actions and the dungeon master responds with text describing what happened in the world as a result.

We’d like a collection of such runs, that are annotated with "visible thoughts" (visible to potential operators or programmers of the system, not to players) describing things like what just happened or is about to happen in the world, what sorts of things the player is probably paying attention to, where the current sources of plot tension are, and so on — the sorts of things a human author would think while acting as a dungeon master. (This is distinct from producing thoughts explaining what happened in the dungeon; “visible thoughts” are meant to play an active role in constructing the output.)

Once we have such a dataset, MIRI’s hope is that present or future technology will be able to train a model or models which iteratively produce visible thoughts along with storytelling, based on user actions plus previous history (including previous thoughts). The goal is to transition the state of AI dungeon technology from “An AI outputs story text in response to actions (and we have no idea how)” to “An AI produces thoughts as visible intermediates on the way to story text, allowing us to watch the AI think about how to design its output, and to verify that we can get different sensible outputs by intervening on the thoughts”.

Here’s an example of the first couple of steps of a thought-annotated dungeon run (or “quest”), in the format MIRI currently thinks is worth trying. Some kinds of thoughts are marked with parentheses and/or brackets; see the next section for details on this.

Thoughts:

[The main plot: Prota is a student at a magical academy that is under attack by zombies.]

[(This is a quest of medium length.)]

(This is the start of the story. The Player should be given enough backstory and immediate description to have some sense of where they are, so the prompt can go on longer than usual. The Player should be told who they are and their role in the universe. The initial prompt should describe Prota being in a magical academy, and should describe the first surprising indication that the academy is under attack by zombies.)

Prompt: Today is your first day of classes at the Magical Academy. You arrived very late last night and missed the tour, but your roommate Lior promised he would help you find your way around. You wake up in the students’ dormitory at dawn, and follow Lior to breakfast. Just as you reach the dining hall, you hear a shout in the distance.

Thoughts:

[Prota missed the tour and doesn’t already know the way around.]

[Prota’s roommate’s name is Lior.]

[Lior is male.]

There are other people in the dining hall who will have also heard the shout, and will also be surprised and reacting to that.

Player action: ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing Shrimp Welfare Project, published by AaronBoddy on November 30, 2021 on The Effective Altruism Forum.

We’re excited to announce the launch of Shrimp Welfare Project (SWP), a new EA organization created through the Charity Entrepreneurship Incubation Program 2021.

Our mission is to positively impact the welfare of billions of shrimps[1], in the most cost-effective way possible.

In this post, we make a case for focusing on shrimps, give an overview of our intervention, outline our progress so far and lay out our plans for the future.

We are infinitely grateful to Charity Entrepreneurship for their organizational and financial support to get this project off the ground. We thank Cristina Schmidt Ibáñez, Lukas Jasiūnas, Max Carpendale, and Aaron Gertler for their invaluable feedback on this forum post announcement. All errors and shortcomings are our own.

Why Shrimps?

~350 billion[2] shrimps are farmed each year [1]. This is 5x the total number of farmed land animals [2]. Of the approximately 2,000 species of shrimps [3], whiteleg shrimp (litopenaeus vannamei) is the shrimp species we focus on due to the scale and intensity at which they are farmed (~280 billion [4]).

Many shrimps reared in aquaculture[3] suffer from conditions which can and should be addressed, such as:

Water quality

Oxygen and ammonia levels, temperature, salinity and pH are key to the welfare of all aquatic animals, including shrimps [5]. Incorrect water management can lead not only to compromised immune systems in shrimps [6] (and in extreme cases, to death by suffocation or poisoning [7]), but also to the contamination of nearby bodies of water [8] and salinization and acidification of the soil [9].

Risk of disease

Diseases that exist within the normal microflora of shrimps can thrive under high stocking densities (with density depending on the intensity of the production method [10]), enabling pathogenic outbreaks [11]. This is detrimental not only to the farmed shrimps but can cause large spillover events if best disease management practices are not followed [12]. The indiscriminate use of antibiotics to stop diseases promotes the emergence of antibiotic-resistant bacteria [13]. Part of the solution in this case is prevention through best shrimp welfare practices [14].

Eyestalk ablation

Most nurseries still practice crushing or cutting off at least one of the eyestalks of female shrimps to induce rapid reproduction [15]. Recent studies have proven that avoiding eyestalk ablation can result in the breeding females living longer and their offspring being more resistant to stress and disease [16]. Therefore, eliminating this practice is in the best interest of shrimps and it should be acceptable to the shrimp farming industry.

Our Priorities

Shrimp Welfare Project's Theory of Change - November 2021

Supply Side

We will work with farmers in tractable producing countries to improve water quality. The intention is to identify farmers that are part of the supply chain of the organizations we will be working with on the demand side.

Demand Side

We intend to work on the demand side, reaching out to relevant stakeholders (such as seafood importers) along the supply chain to improve welfare standards for shrimps. Some of the asks we are considering include requiring shrimp farm suppliers to meet certain water quality metrics and limit stocking densities, as well as eventually incorporate more humane slaughter practices and phase out eyestalk ablation.

Through conversations with organizations that have been involved in the transition to cage-free hens, we have gauged that our probabilities of success to improve overall shrimp welfare are maximized if we can directly link supply and demand, transitioning consumption towards a (today non-existent) market for higher-welfare shrimps.

Outreach

In additi...