Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Playing with DALL·E 2, published by Dave Orr on April 7, 2022 on LessWrong.

I got access to Dall·E 2 yesterday. Here are some pretty pictures!

My goal was to try to understand what things DE2 could do well, and what things it had trouble understanding or generating. My general hypothesis is that it would do a better job with things that are easy to find on the internet (cute animals, digital scifi things, famous art) and less well with more abstract or more unusual things.

Here's how it works: you put in a description of a picture, and it thinks for ~20 seconds and then produces 10 photos that are variations on that description. The diversity varies quite a bit depending on the prompt.

Let's see some puppies!

One thing to be aware of when you see amazing pictures that DE2 generates, is that there is some cherry picking going on. It often takes a few prompts to find something awesome, so you might have looked at dozens of images or more.

Still, this is pretty great! Those are recognizably goldendoodle puppies, mostly in something approximating play position.

You can see that the proportions in the generated images are not quite right, and some of the detail is off if you look closely. For instance, the front legs are too long here, the face isn't quite right, and the ears are a bit weird.

Still, it's pretty amazing given that it generated this from scratch. Check out how realistic the grass looks. I also like that the background is blurred, though not quite in the way that a camera would do it -- the transition is too abrupt.

Ok but the point of this isn't that they have a great image generation transformer, though it's clearly that. The key thing is is its magical ability to actually follow instructions or descriptions of images. Particularly interesting is compositionality -- can it combine concepts to generate something it's never seen before? Answer: yes!

The concept of "kitten" is pretty simply, though note that a kitten can be rendered in a ton of ways, from line drawings to cute art to photorealistic. Pop art is more complicated: it's a celebration of everyday images, and one of the most commonly known versions is Warhol's collection of repeated images in a grid with neon colors that vary per cell. And it mostly gets those things right.

What about weird things? You can put in any input and it'll do something.

None of those are twitter worthy, but with some trial and error you can get things that are interesting.

"Digital style" is one of the suggestions for getting better images.

X in Y style is fun, that's a lot of the images you see out in the world. Weirdly it's pretty sensitive to exactly the order you put things in.

Back to puppies, you get pretty different results depending on the placement of "surrealistic" even though the rephrasings seem semantically identical or at least very similar.

One place where DE2 clearly falls down is in generating people. I generated an image for [four people playing poker in a dark room, with the table brightly lit by an ornate chandelier], and people didn't look human -- more like the typical GAN-style images where you can see the concept but the details are all wrong.

Update: image removed because the guidelines specifically call out not sharing realistic human faces.

Anything involving people, small defined objects, and so on, looks much more like the previous systems in this area. You can tell that it has all the concepts, but can't translate them into something realistic.

This could be deliberate, for safety reasons -- realistic images of people are much more open to abuse than other things. Porn, deep fakes, violence, and so on are much more worrisome with people. They also mentioned that they scrubbed out lots of bad stuff from the training data; possibly one way they did that was removing most images with people.

Thi...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 'Hot takes' from EAGx Boston, published by Joshua Clymer on April 7, 2022 on The Effective Altruism Forum.

One of my favorite questions to ask longtermists is: “where do your inside views differ most from your outside views – i.e. what are your hot takes?”

This question has a few benefits:

It tends to extract the information that the person thinks is most valuable.

It can get people to realize that a popular idea isn’t sitting right with them and think critically about why that might be.

It counteracts ‘double updating.’ Let’s say that ‘Bob’ comes up with an idea and tells it to ‘Alice’ and ‘John’ on separate occasions. Alice might repeat the idea to John, and he might think that Alice and Bob came up with this idea independently and make a larger update in favor of it than he should. Propagating inside views helps to counteract this echo.

I asked a lot of people for their hot-takes during EAGx Boston. Surprisingly, people often had similar answers, so some of these ideas are probably not hot takes. Here is a list of the answers that stick out in my memory. I don’t fully agree with all of these opinions, but I think they are all helpful perspectives.

Field builders should be thinking more long term

People might be thinking ‘what is the best project can I can finish in the next few months?’ when they should be thinking more along the lines of ‘what sort of organizations will we need in the next couple of decades and how can I start building them?’

Teams should be more permanent

Teams are sometimes formed around a field-building project and disassemble after the project is finished. It might be better if people focus on forming really good teams and continue to work in these them, allowing the members to learn how to work more efficiently together over time.

AI governance theories of change are often weak.

A common AI governance plan is to get a position in government or a think tank and ‘hope you are in the right place at the right time to influence policy when it matters.’ Maybe this is a good thing to do if you are exclusively passionate about these kinds of careers; but if you are optimizing for impact, doing AI governance research or field-building might be superior alternatives.

The standard University EA group pipeline does not make sense

Typically, students are exposed to Effective Altruism, then longtermism, and then to some specific cause area like AI safety. We probably lose a lot of people at each of these steps. It may be more efficient to directly get people into AI safety since EA or longtermsism are not necessary prerequisites for being concerned about it.

There should be a variety of paths to AI-safety, but the primary ones should be the most direct.

Eliminate required readings from University fellowships

Readings are good, but most people won’t do them before fellowship meetings. Instead, make meetings longer and spend the first hour reading through the material. A similar approach seems to be working well for the AI Safety group at Harvard.

Retreats are better than fellowships

Empirically, retreats tend to get people more engaged than fellowships do. Fellowships could be a good filtering mechanism to figure out who should be invited to retreats, but maybe there are more efficient filtering mechanisms we can use – e.g. a speaking event where people can sign up for a one-on-one afterward.

The idea that fellowships are far from ideal is not new.

Community builders should have good models of direct work

If you are trying to do AI safety field building, it is important to know what kind of people you want to recruit. Having an understanding of the work being done in the field is very helpful for this. This means that if you are a field-builder, you should frequently read up on the object-level work being done and talk to the people doing it.

Feel free to ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ideal governance (for companies, countries and more), published by Holden Karnofsky on April 7, 2022 on The Effective Altruism Forum.

Cross-posted from Cold Takes

I'm interested in the topic of ideal governance: what kind of governance system should you set up, if you're starting from scratch and can do it however you want?

Here "you" could be a company, a nonprofit, an informal association, or a country. And "governance system" means a Constitution, charter, and/or bylaws answering questions like: "Who has the authority to make decisions (Congress, board of directors, etc.), and how are they selected, and what rules do they have to follow, and what's the process for changing those rules?"

I think this is a very different topic from something like "How does the US's Presidential system compare to the Parliamentary systems common in Europe?" The idea is not to look at today's most common systems and compare them, but rather to generate options for setting up systems radically different from what's common today.

I don't currently know of much literature on this topic (aside from the literature on social choice theory and especially voting methods, which covers only part of the topic). This post describes the general topic and why I care, partly in the hopes that people can point me to any literature I've missed. Whether or not I end up finding any, I'm likely to write more on this topic in the future.

Outline of the rest of the piece:

I'll outline some common governance structures for countries and major organizations today, and highlight how much room there is to try different things that don't seem to be in wide use today. More

I'll discuss why I care about this question. I have a few very different reasons:

A short-term, tangible need: over the last several years, I've spoken with several (more than 3) organizations that feel no traditional corporate governance structure is satisfactory, because the stakes of their business are too great and society-wide for shareholder control to make sense, yet they are too early-stage and niche (and in need of nimbleness) to be structured like a traditional government. An example would be an artificial intelligence company that could end up with a normal commercial product, or could end up bringing about the most important century of all time for humanity. I wish I could point them to someone who was like: "I've read all of, and written much of, the literature on what your options are. I can walk you through the pros and cons and help you pick a governance system that balances them for your needs."

A small probability of a big future win. The world today has lots of governments, but they seem to mostly follow a very small number of basic governance templates. At some point, there will be new states with new Constitutions - maybe via space settlements, maybe via collapse of existing states, etc. - but I expect these moments to be few and far between. A significant literature and set of experts on "ideal governance" could lead to a radically different kind of state government, potentially with radically different policies that the rest of the world could learn from.

A weird, out-of-left-field application. Some of my interest in this topic actually comes via my interest in moral uncertainty: the question of what it's ethical to do when one is struggling between more than one theory of ethics, with radically different implications. This is hard to explain, but I try below.

I'll describe a bit more what I think literature on this question could look like (and what already exists that I know of), partly to guide readers who might be able to help me find more.

Common governance structures today

All of these are simplified; I'm trying to illustrate the basic idea of what questions "ideal governance" is asking.

A standard (e.g., public) c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I discovered LessWrong... during Good Heart Week, published by identity.key on April 7, 2022 on LessWrong.

I decided to write this because I thought my experience could be an interesting or useful data point, in particular for understanding the results of Good Heart Week: Extending the Experiment.

I happened to discover LessWrong just before April 1st. Initially, it captured my attention through the UI design—I have an impression that many platforms focused on thinking itself, scientific research, and similar topics have a really outdated look that demotivates me from interacting with them (which itself is likely to be caused by the fact that I have to interact with some other, unrelated platforms with outdated looks); but LessWrong's is quite modern, while still retaining simplicity that allows one to focus on the content.

When it comes to content, I was impressed by the range of topics that can be (and are being) discussed here. Coming from platforms that are focused on specific topics and where you're discouraged from going off-topic in threads, this platform felt very different. I guess this comes from the fact that it's about thinking itself and how it can be applied to various topics, whereas these "various topics" could be anything.

I decided to read more of the actual content. I started by reading some posts that are about (or at least mention) some things that I'm very familiar with and found them to be quite interesting.

I also thought of some post ideas and realized that some of the things I think about regularly could be a good fit here, but I decided to actually write them sometime later.

When I visited LessWrong on April 1st, I was slightly confused at first when I saw the colored names of post authors—it seemed like a new feature that highlights moderators or staff more than anything else. Even now, when I know what it is, it still feels more like a special user highlight rather than being based on Good Heart Tokens.

Then I saw Replacing Karma with Good Heart Tokens (Worth $1!)... and actually fell for it. Before this, I had stumbled upon LessWrong is providing feedback and proofreading on drafts as a service and was impressed by the fact that such a service is provided for free. The idea of Good Heart Tokens felt somewhat similar to that, so at first I thought this was also "an actual feature announcement"... However, once I actually got to read the post and started reading the "Why are we doing this?" section, I did realize it was an April Fool's joke. At that point, I finished my daily visit and decided to check the site out later instead.

When I opened it on April 2nd, I was surprised to read Good Heart Week: Extending the Experiment, since I have decided that it was just a joke. At this point, given such an incentive, I started actually thinking of what I could write about. I ended up writing the question How to openly maintain a single identity while keeping it private? as my first post.

Had it not been for this experiment, I might have not posted that question at all or it would at least be a while before I did. One of the reasons why is that, ideally, I would like to get more familiar with the site itself, how it works, and perhaps some of the foundational posts before I post my own, and the FAQ says as much. But the short length of this experiment didn't really allow for much of this, so I just had to post right away.

I'll find out whether or not I do end up reaching the minimum of $25 a little later, but I would like to think that, in any case, I've made a few worthwhile contributions.

After that, I decided to check out more of the content. Through definitions of tags, where I arrived by checking out interesting words on the Concepts page (or looking for specific words I've noticed in posts or comments), I have discovered a few terms that qui...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Productive Mistakes, Not Perfect Answers, published by Adam Shimi on April 7, 2022 on The AI Alignment Forum.

I wouldn’t bet on any current alignment proposal. Yet I think that the field is making progress and abounds with interesting opportunities to do even more, giving us a shot. Isn’t there a contradiction?

No, because research progress so rarely looks like having a clearly correct insight that clarifies everything; instead it often looks like building on apparently unpromising ideas, or studying the structure of the problem. Copernican heliocentrism didn’t initially predict observations as well as Ptolemaic astronomy; both ionic theory and the determination of basic molecular formula came from combining multiple approaches in chemistry, each getting some bits but not capturing the whole picture; Computer Science emerged from the arid debate over the foundations of mathematics; and Computational Complexity Theory has made more progress by looking at why some of its problems are hard than by waiting for clean solutions.

In the end you do want to solve the problem, obviously. But the road from here to there goes through many seemingly weird and insufficient ideas that are corrected, adapted, refined, often discarded except for a small bit. Alignment is no different, including “strong” alignment.

Research advances through productive mistakes, not perfect answers.

I’m taking this terminology from Goro Shimura’s characterization of his friend Yutaka Taniyama, with whom he formulated the Taniyama-Shimura Conjecture that Andrew Wiles proved in order to prove Fermat’s last theorem.

(Yutaka Taniyama and his time. Very personal recollections, Goro Shimura, 1989)

Though he was by no means a sloppy type, he was gifted with the special capability of making many mistakes, mostly in the right direction. I envied him for this, and tried in vain to imitate him, but found it quite difficult to make good mistakes.

So much of scientific progress takes the form of many people proposing different ideas that end up being partially right, where we can look back later and be like “damn, that was capturing a chunk of the solution.” It’s very rare that people arrive at the solution of any big scientific problem in one nice sweep of a clearly adequate idea. Even when it looks like it (Einstein is an example people like to bring up), they so often build on many of the weird and contradictory takes that came before, as well as the understanding of how the problem works at all (in Einstein’s case, this includes the many, many unconvincing attempts to unify mechanics and electromagnetism, the shape of Maxwell’s equations, the ether drag hypothesis, and Galileo’s relativity principle; he also made a lot of productive mistakes of his own).

Paul Graham actually says the same thing about startups that end up becoming great successes.

(What Microsoft Is This The Basic Altair Of?, Paul Graham, 2015)

One of the most valuable exercises you can try if you want to understand startups is to look at the most successful companies and explain why they were not as lame as they seemed when they first launched. Because they practically all seemed lame at first. Not just small, lame. Not just the first step up a big mountain. More like the first step into a swamp.

Graham proposes a change of polarity in considering lame ideas: instead of looking for flaws, he encourages us to steelman not the idea itself, but how it could lead to greatness.

(What Microsoft Is This The Basic Altair Of?, Paul Graham, 2015)

Most people's first impulse when they hear about a lame-sounding new startup idea is to make fun of it. Even a lot of people who should know better.

When I encounter a startup with a lame-sounding idea, I ask "What Microsoft is this the Altair Basic of?" Now it's a puzzle, and the burden is on me to solve it. So...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some thoughts about imperfect world modeling, published by leogao on April 7, 2022 on The AI Alignment Forum.

So a few posts ago I looked at the problem of not being able to anticipate all consequences of an action as being related to deceptive mesaoptimization, but also outer alignment too. This post digs more into some of the things I only touched on briefly in that post.

Editor’s note: I’m experimenting with having a lower quality threshold for just posting things even while I’m still confused and unconfident about my conclusions, but with this disclaimer at the top. Thanks to Kyle and AI_WAIFU for discussions.

Last time, we narrowed down the problem to [certain actions we can take now] leading to [a world such that if [RSA2048 or some other thing we can’t simulate] happens, then [bad things will happen]] (brackets added for easier parsing of this sentence). This is because we can't plan for a future where RSA-2048 is solved but if we can already see RSA2048 in the world, then we can plan forward from that and see when things blow up.

So if we could somehow figure out how to ensure that no action puts us into an RSA2048-vulnerable world, then we can prevent this failure case. Unfortunately, knowing precisely whether a world is vulnerable would essentially require us to simulate RSA2048, which gets us nowhere. However, one observation is we don't have to know precisely, we just need to know whether a world could be vulnerable — like for example if the agent removes its own safeguard, then we wouldn’t want to allow that even if the model is in fact safe and wouldn’t destroy the world when it sees RSA2048.

Telling whether side effects are dangerous is hard

One thing we might think to do is instead simulate putting a dummy AI in the “do rollouts to find out if these actions destroy the world” box (the word “box” here has no relation to the AI box experiment), such that this dummy AI tries to kill everyone when it sees something we can actually simulate, like the string "chariots chariots" or something. Then, we can execute the action in the simulation, and then simulate seeing that string, and see if the box successfully prevented the action from being executed in the real world. This way, we can test whether the box is broken by the action, right?

Unfortunately there's a huge counterexample to this. So suppose the AI proposes building a really complicated machine in the world that we don't understand. We can still simulate the consequences of having this machine exist, of course, but we can't simulate it past the RSA2048 barrier, and so this machine could do evil stuff after seeing RSA2048 and we'd never know. The more general lesson from this is that if the model proposes doing something with side effects in the real world that we can't understand, that allows it to build up stuff "outside of the box" and then that stuff could do bad things.

If we try to understand everything the model does, we end up bottlenecked by the whole "doing science" thing again, and if we try to bring the things the model does back inside the box (i.e only allow the machine it builds to also propose actions within the box), then you end up never being able to modify the real world at all because any changes to the real world could actually be setting up something that blows up after RSA2048, which is totally useless.

So, in general, it seems hard to know whether an action sets up something evil in the world that only activates after RSA2048.

It’s (partial) ELK reduction time, my dudes

There is a partial ELK reduction here in that if we can solve ELK then we can ask the model whether it thinks its actions are putting the world into a vulnerable state. It's worth noting that this wouldn't work if we just asked the model whether RSA2048 would cause it to kill everyone, because a) the model co...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Altruism as a central purpose, published by Joey on April 7, 2022 on The Effective Altruism Forum.

Effective altruism has been described as many things; a question, an ideology, a project, an obligation, a passion. However, none of these definitions have ever really felt right to me. After some thought I have decided that the descriptor that best fits the role altruism plays in my life, is that of a central purpose. A purpose can be a philosophy and a way of living. A central purpose transcends a passion; even considering how intense and transformative a passion can be. It carries a far deeper significance in one’s life. When I describe EA as my purpose, it suggests that it is something that my life is built around; a fundamental and constitutive value.

Of course, effective altruism fits into people’s lives in different ways and to different extents. For many EAs, an existing descriptor adequately captures their perspective. But there are many subgroups in EA for whom I think it has been helpful to have a more focused discussion on the role EA plays for them. I would imagine that a space within EA for purpose-driven EAs could be particularly useful for this subset, while of little interest to the broader community.

So what do I mean by purpose-driven effective altruism? Firstly, I think it considerably stands apart from the idea of excited altruism. Although most people are excited about things they find to be purposeful and meaningful; a purpose can sustain aspiration and motivation even in the absence of the emotion of excitement, or even personal interest. People’s lives revolve around many different purposes. Some people’s purpose might be to build their family, their knowledge, or their company. For some people their central purpose is to make a difference in the world. A person’s purpose is the reason they get up in the morning and the biggest factor when making a decision.

This framing of purpose based effective altruism can be useful in a number of ways. But it’s important to note, I don’t think you have to make effective altruism your purpose in order to make a difference in the world, or to join the movement. There is quite a difference between someone who occasionally paints or even who loves to paint; and someone whose purpose is to paint. Nonetheless, all of these people can join a painting group.

If altruism is your central purpose, it leads to different choices than if it's a passion. Identifying in this way can also give people a much better sense of why someone is doing the things they are doing. Sacrificing a personal goal would be a large ask for even the most passionate person, but trivial in the context of a central purpose. Someone might stop working on their passion if it's no longer fun, but few would give up so easily on something they see as their purpose. Switching countries or lifestyles might seem extreme for a passion but can easily happen for a purpose.

I also think the ‘purpose’ description is more fitting for a large group of EAs who are often described as “obligation based altruists”. Obligation has fairly negative connotations and does not really convey the full experience for people who are highly focused on EA. On the other hand, having a purpose is quite a positive thing. Leading a meaningful life with a clear purpose improves mental health, self-worth, and self-confidence. Finding your purpose is often seen as a goal and can lead to significant happiness. Ultimately, having a purpose is seen as a good thing and having an obligation is seen as a bad thing.

As touched on above, I do not think this label is the only way to be an EA. You can be in the EA movement as an excited altruist, or view EA as more of a question, project, or ideology. I see these definitions as complementary, bringing more nuance to the question of what makes...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How BoMAI Might fail, published by Donald Hobson on April 7, 2022 on The AI Alignment Forum.

This post is me looking at 1905.12186.pdf and giving some ideas on how it might fail.

Firstly, I would like to point out that I think the design in this paper does buy you some amount of marginal safety when dealing with slightly superhuman AI's, and that it still gives opportunity to perhaps get something useful out.

Failure mode 1)

A lack of optimization doesn't mean a lack of danger. Even after the episode ends, the box may contain dangerous optimized items. Given that the box was specified to contain only a human, (And some cloths) this suggests the most dangerous thing that could leave is probably some form of meme. For example, suppose the AI finds a very convincing argument that humans should do whatever AI's want. This is a very good solution to the problem of hacking the human into giving the AI whatever answer it wants. (Which is pretty much what this AI was programmed to do, although it may sometimes give good answers on occasions where that is easier than hacking the operator) Once the human exits the box, the idea can go viral.

Failure mode 2)

What does the "Asymptotic optimality" mean in practice. Imagine a world containing BoMAI. The BoMAI comes up with a plan to take over the world after this episode has ended. It tries this plan. Lets imagine a magic angel interferes with this plan, stopping it the moment the episode ends. So the BoMAI is somewhat puzzled, and tries again. Eventually BoMAI will learn that plans involving taking over the world after the end of the episode never work, and so it will stop doing them. This means the magic angel only has a finite amount of work to do. Plans that involve taking over the world are assumed to receive a lower rating within the episode. The smaller the difference in reward between a plan that also takes over the world after the episode, and a plan that purely optimizes within an episode, the more slowly such taking over world plans disappear. Leading in the limit to failure mode 1.

Of course, in reality, the magic angel doesn't exist, and the AI succeeds in taking over the world.

Failure mode 3)

Criticizing the box.

The mentor has no vacuum between them and the font of all evil (the AI). There is also no Faraday cage, and a screen facing the mentor that is vulnerable to all sorts of hacking. Therefore the mentor can be hacked. There is no vacuum between the mentor and the AI, so if the mentor can claw through concrete, they gain hardware level access to the AI. They can then disable the pressure sensors and any other systems that end the episode. Now the airlocks can be opened by using whatever mechanism is used to open the airlocks at the end of an episode.

There must be some point where information from the pressure sensors feeds into the AI system to stop the episode. This point, and the pressure sensors themselves, are vulnerable to electromagnetic and sonic interference from the AI. Suppose the AI has a bunch of cooling fans it is in control of, and by vibrating them all in just the right way, it can cause all the sound waves to constructively interfere right on the solder joint between a pressure sensor and the AI's circuits. This allows it to work the solder loose, slowly breaking that connection. The AI can now encourage the human operator to smash the glass.

The AI bootstraps nanotech. It turns out there was a sophisticated and easily hackable piece of proto-nanotech just sitting around, in the form of a bacteria just sitting on one of the AI's chips. By carefully controlling the electromagnetic, thermal and sonic emissions of its hardware, the AI can hack that ribosome. The bootstrapped nanotech can tunnel through the walls of the box and take over the world.

There are other flaws in the "nothing can get thro...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why Take Care Of Your Health?, published by MondSemmel on April 6, 2022 on LessWrong.

The Underspecified Normie

You’re well-aware of the benefits of a healthier lifestyle: less pain, more energy, more mobility and autonomy, a higher life expectancy, and so on and so forth ad nauseam.

Unfortunately, this knowledge doesn’t compel you to action. Your behavior mostly follows simple hyperbolic discounting - healthy actions pay off in the future, but the future is far away, and your TV / smartphone / snack is much closer.

But even if the knowledge that Health Is Good is uncompelling, maybe some other perspectives or frames might prove more fruitful for improving your health?

The Guilty Conscience

All your life you’ve been told what to do: “Go to bed already!” “Do your homework!” “Brush your teeth!”

At some point you internalized those voices: It’s 4 am. I should’ve gone to bed a long time ago. Why am I still awake? What’s wrong with me?

You’ve been thinking this way all your life. Surely this strategy will start working any day now.

If only there was an alternative...

The Investor

Investment bankers aren’t exactly known for their healthy work-life balance. You were no exception to that rule. But at some point you realized that taking care of your health is just another way of investing, namely in your own human capital.

Not all such investments pay off. But if you know where to look, you should be able to find some with outsized returns in longevity and productivity. Time to crunch some numbers!

And in the hopefully distant future, when it’s time for the final accounting, your most profitable investment might just turn out to have been - yourself.

The Decision Theorist

You are in an iterated prisoner’s dilemma with future copies of yourself. You could stay up all night to read that ratfic, but that would be defecting against tomorrow!You. And that line of thinking doesn’t end well.

So you go to bed early, and expect tomorrow!You to do the same, for the same reason.

The Competitive Gamer

The abstraction of “hitpoints” in RTS and FPS games has taught you an unfortunate lesson. People can perform at their peak ability until they reach 0 hitpoints, at which point they die. Recovery is instant as long as you get help from a medic or medkit.

Speaking of which, how do you get those in real life? After gaming for 36 consecutive hours, you aren’t feeling so well...

The Sim Gamer

You’ve just returned from the funeral of your friend Competitive Gamer. RIP. You deeply regret your past inability to convince them to give your favorite genre a try. They might have lived!

You grew up with The Sims, which models people as Sims with a bunch of needs. These needs decline over time and must be recharged. Failing to do so has unfortunate repercussions like falling unconscious or even getting a visit from the Grim Reaper. Conversely, taking care of your Sims both makes them happier and more productive. Now that’s a lesson for real life!

You hope your other insights from the game will be just as applicable. Your first purchase in The Sims was always the most expensive bed, as it paid for itself via reduced sleep length. Hopefully that new gold-studded bed in your room will do so, as well. Those interest payments aren't cheap!

The Slacker

Others might call you lazy, but you know better. You’re preserving your slack. Maybe 80-hour workweeks help those around you get ahead right now, during times of stability. But those times won’t last forever. Eventually, a disruption will occur - whether it be an economic crisis, a pandemic, or some altogether inconceivable thing -, and then their overworked bodies will constitute a dangerous lack of slack.

But that doesn’t apply to you. You’re prepared.

The Parent

It’s been weeks since you last slept for more than three consecutive hours. You’re running on fume...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Update on the Simon Institute: Year One, published by konrad on April 7, 2022 on The Effective Altruism Forum.

- We thank Jacob Arbeid for his immensely helpful edits & comments.

Summary

A year ago, we launched the Simon Institute for Longterm Governance (SI). In this post, we

Review our first twelve months;

Share our lessons learned;

Outline plans for the next two years;

Make a call for funding; and

Propose an Ask Us Anything.

Based in Geneva, Switzerland, SI is building the science-policy interface for longtermist engagement with the UN system. The UN, its networks and activities provide international fora for exchange framed under multilateralist ideals. As a source of symbolism with some agenda-setting and policy design power, the UN eases the promotion of common sense longtermism among influential institutions and individuals. However, making valuable contributions requires context-specific knowledge, as well as the right contacts, collaborators and reputation.

Throughout its first year, SI proved its capacity and value in connecting longtermist actors to international civil servants and national delegates, especially through workshops and high-level dinners with e.g. CSER and FHI. SI’s research is attracting valuable attention and has in a first part passed academic peer review. This combination of activities has granted SI mandates from UN institutions, as well as the Swiss government, to directly work on policy processes relevant to existential risk reduction.

In the next step SI will focus on the development of the UN’s “Our Common Agenda” (OCA) as a particularly impactful opportunity to focus international and national attention on future generations and existential risk. The value of this engagement can come from e.g. identifying allies, contributing to the redefinition of the international policy narrative, and easing other longtermists’ national policy efforts by helping create global reference frameworks.

Due to its positioning and first achievements, SI finds itself with this rare window of opportunity to inculcate longtermism and existential risk reduction at the heart of the institutions that can claim to speak for all humanity, and which can affect the priorities of national governments. Our success is now dependent on growing our funding and talent base, which we hope will mainly come from the value-aligned EA community. We believe we have room for more funding of $ ~4 million until 2025.

1. What is SI?

The vision of the Simon Institute for Longterm Governance (SI) is for governments and multilateral organizations to adequately account for the interests of future generations.

Most pressingly, though, we as humanity need to reduce risks to our flourishing effectively. For all key risks, humanity’s path to existential security cannot be brought about by the actions of any single country, making more effective international cooperation essential. Improving international cooperation requires better management of uncertainty and understanding of technological development by policymakers, as well as better representation of future generations - all while preserving political legitimacy for present generations.

Such improvements require policy change at international levels (e.g. new or modified treaties) and national levels (e.g. treaty ratification and implementation). Longtermist research can and should inform such policy change. But to accomplish that, we need intermediary actors who can build connections to relevant policy ecosystems and translate between stakeholders. SI is one such intermediary actor that the community needs.

To improve international existential risk governance, SI works at the science-policy interface. We

Engage policymakers by providing space for reflection and advice from external experts.

Support researchers in their policy eng...