Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Implied "utilities" of simulators are broad, dense, and shallow, published by porby on March 1, 2023 on The AI Alignment Forum.

This is a quick attempt at deconfusion similar to instrumentality. Same ideas, different angle.

Extremely broad, dense reward functions constrain training-compatible goal sets

Predictors/simulators are typically trained against a ground truth for every output. There is no gap between the output and its evaluation; an episode need not be completed before figuring out how good the first token prediction was. These immediate evaluations for every training sample can be thought of as a broad and densely defined reward function.

It's easier for a model to fall into an undesired training-compatible goal set when there are many accessible options for undesirable goal sets versus desirable goal sets. As the number of constraints imposed by the trained reward function increases, the number of training-compatible goal sets tends to decrease, and those that survive obey more of the desirable constraints.

There is no guarantee that SGD will find an agent which could be modeled by a utility function that maps perfectly onto the defined reward function, but if you throw trillions of constraints at the function, and simultaneously give it lots of highly informative hints about what path to walk, you should expect the potential output space to be far narrower than if you hadn't.

Impact on internal mesaoptimizers

The dense loss/reward function does not as heavily constrain out of distribution behavior. In principle, a strong misaligned mesaoptimizer within a predictive model could persist in these degrees of freedom by providing extremely good solutions to in-distribution samples while doing arbitrarily misaligned things out of distribution.

But how would that type of mesaoptimizer develop in the first place?

Steps toward it must serve the training objective; those constraints still shape the mesaoptimizer's training even if its most notable activity ends up being hidden.

The best story I've found so far goes something like this:

Traditional reinforcement learning agents are mostly unconstrained. The reward function is sparse relative to state and action space.

An agent faced with sparse rewards must learn actions that serve a later goal to get any reward at all.

Not surprisingly, agents facing sparse reward relative to state/action space and few constraints have a much larger percentage of undesirable training-compatible goal sets.

Mesaoptimizers are processes learned within a model and their local training influences may not perfectly match the outer training influences.

If the mesaoptimizer's local training influences look more like the traditional reinforcement learning agent's influences than the predictor's outer influences, it would be more likely to fall into one of the undesirable training-compatible goal sets.

The mesaoptimizer learns incorrect goals and a high propensity for goal-serving intermediate actions ("actions" within the scope of a single model execution!)

The mesaoptimizer is kept around by SGD because it does well on the subset of outputs that the outer model is using it on. As capability grows, the mesaoptimizer strategically takes over other chunks of prediction space by performing well during training in an effort to be selected during out of distribution predictions.

In a previous post, I called the learned propensity for goal-serving intermediate action instrumentality. The constraints imposed by predictive model training clearly confer lower instrumentality than traditional RL in all current models. I suspect the path taken by the mesaoptimizer above is hard and unnatural, but perhaps not impossible for some form of predictor taken to the relevant extreme.

It seems critical to understand the degree to which outer constraints apply...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Enemies vs Malefactors, published by So8res on February 28, 2023 on The Effective Altruism Forum.

Short version

Harmful people often lack explicit malicious intent. It’s worth deploying your social or community defenses against them anyway. I recommend focusing less on intent and more on patterns of harm.

(Credit to my explicit articulation of this idea goes in large part to Aella, and also in part to Oliver Habryka.)

Long version

A few times now, I have been part of a community reeling from apparent bad behavior from one of its own. In the two most dramatic cases, the communities seemed pretty split on the question of whether the actor had ill intent.

A recent and very public case was the one of Sam Bankman-Fried, where many seem interested in the question of Sam's mental state vis-a-vis EA. (I recall seeing this in the responses to Kelsey's interview, but haven't done the virtuous thing of digging up links.)

It seems to me that local theories of Sam's mental state cluster along lines very roughly like (these are phrased somewhat hyperbolically):

Sam was explicitly malicious. He was intentionally using the EA movement for the purpose of status and reputation-laundering, while personally enriching himself. If you could read his mind, you would see him making conscious plans to extract resources from people he thought of as ignorant fools, in terminology that would clearly relinquish all his claims to sympathy from the audience. If there were a camera, he would have turned to it and said "I'm going to exploit these EAs for everything they're worth."

Sam was committed to doing good. He may have been ruthless and exploitative towards various individuals in pursuit of his utilitarian goals, but he did not intentionally set out to commit fraud. He didn't conceptualize his actions as exploitative. He tried to make money while providing risky financial assets to the masses, and foolishly disregarded regulations, and may have committed technical crimes, but he was trying to do good, and to put the resources he earned thereby towards doing even more good.

One hypothesis I have for why people care so much about some distinction like this is that humans have social/mental modes for dealing with people who are explicitly malicious towards them, who are explicitly faking cordiality in attempts to extract some resource. And these are pretty different from their modes of dealing with someone who's merely being reckless or foolish. So they care a lot about the mental state behind the act.

(As an example, various crimes legally require mens rea, lit. “guilty mind”, in order to be criminal. Humans care about this stuff enough to bake it into their legal codes.)

A third theory of Sam’s mental state that I have—that I credit in part to Oliver Habryka—is that reality just doesn’t cleanly classify into either maliciousness or negligence.

On this theory, most people who are in effect trying to exploit resources from your community, won't be explicitly malicious, not even in the privacy of their own minds. (Perhaps because the content of one’s own mind is just not all that private; humans are in fact pretty good at inferring intent from a bunch of subtle signals.) Someone who could be exploiting your community, will often act so as to exploit your community, while internally telling themselves lots of stories where what they're doing is justified and fine.

Those stories might include significant cognitive distortion, delusion, recklessness, and/or negligence, and some perfectly reasonable explanations that just don't quite fit together with the other perfectly reasonable explanations they have in other contexts. They might be aware of some of their flaws, and explicitly acknowledge those flaws as things they have to work on. They might be legitimately internally motivated by good intent, ev...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Inside the mind of a superhuman Go model: How does Leela Zero read ladders?, published by Haoxing Du on March 1, 2023 on The AI Alignment Forum.

tl;dr—We did some interpretability on Leela Zero, a superhuman Go model. With a technique similar to the logit lens, we found that the residual structure of Leela Zero induces a preferred basis throughout network, giving rise to persistent, interpretable channels. By directly analyzing the weights of the policy and value heads, we found that the model stores information related to the probability of the pass move along the top edge of the board, and those related to the board value in checkerboard patterns. We also took a deep dive into a specific Go technique, the ladder, and identified a very small subset of model components that are causally responsible for the model’s judgement of ladders.

Introduction

We live in a strange world where machine learning systems can generate photo-realistic images, write poetry and computer programs, play and win games, and predict protein structures. As machine learning systems become more capable and relevant to many aspects of our lives, it is increasingly important that we understand how the models produce the outputs that they do; we don’t want important decisions to be made by opaque black boxes. Interpretability is an emerging area of research that aims to offer explanations for the behavior of machine learning systems.

Early interpretability work began in the domain of computer vision, and there has been a focus on interpreting transformer-based large language models in more recent years. Applying interpretability techniques to the domain of game-playing agents and reinforcement learning is still relatively uncharted territory. In this work, we look into the inner workings of Leela Zero, an open-source Go-playing neural network. It is also the first application of many mechanistic interpretability techniques to reinforcement learning.

Why interpret a Go model? Go models are very capable. Many of us remember the emotional experience of watching AlphaGo’s 2016 victory over the human world champion, Lee Sedol. Not only have there been algorithmic improvements since AlphaGo, these models improve via self-play, and can essentially continue getting better the longer they are trained. The best open-source Go model, KataGo, is trained distributedly, and the training is still ongoing as of February 2023. Just as AlphaGo was clearly one notch above Lee Sedol, every generation of Go models has been a decisive improvement over the previous generation. KataGo in 2022 was estimated to be at the level of a top-100 European player with only the policy, and can easily beat all human players with a small amount of search. Understanding a machine learning system that performs at a superhuman level seems particularly worthwhile as future machine learning systems are only going to become more capable.

Little is known about models trained to approximate the outcome of a search process. Much interpretability effort have focused on models trained on large amounts of human-generated data, such as labeled images for image models, and Internet text for language models. In constrast, while training AlphaZero-style models, moves are selected via Monte-Carlo Tree Search (MCTS), and the policy network of the model is trained to predict the outcome of this search process (see Model section for more detail). In other words, the policy network learns to distill the result of search. While it is relatively easy to get a grasp of what GPT-2 is trained to do by reading some OpenWebText, it’s much less clear what an AlphaZero-style model learns. How does a neural network approximate a search process? Does it have to perform internal search? It seems very useful to try to get an answer to these questions.

Compared to a g...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Things I Heard about AI Governance at EAG, published by utilistrutil on February 28, 2023 on The Effective Altruism Forum.

Intro

Prior to this EAG, I had only encountered fragments of proposals for AI governance: "something something national compute library," "something something crunch time," "something something academia vs industry," and that was about the size of it. I'd also heard the explicit claim that AI governance is devoid of policy proposals (especially vis-a-vis biosecurity), and I'd read Eliezer's infamous EAG DC Slack statement:

My model of how AI policy works is that everyone in the field is there because they don't understand which technical problems are hard, or which political problems are impossible, or both . . .

At this EAG, a more charitable picture of AI governance began to cohere for me. I was setting about recalling and synthesizing what I learned, and I realized I should share—both to provide a data point and to solicit input. Please help fill out my understanding of the area, refer me to information, and correct my inaccuracies!

Eight one-on-ones contributed to this picture of the governance proposal landscape, along with Katja's and Beth's presentations, Buck's and Richard Ngo's office hours, and eavesdropping on Eliezer corrupting the youth of EAthens. I'm sure I only internalized a small fraction of the relevant content in these talks, so let me know about points I overlooked. (My experience was that my comprehension and retention of these points improved over time: as my mental model expanded, new ideas were more likely to connect to it.) The post is also sprinkled with my own speculations. I'm omitting trad concerns like stop-the-bots-from-spreading-misinformation.

Crunch Time Friends

The idea: Help aligned people achieve positions in government or make allies with people in those positions. When shit hits the fan, we activate our friends in high places, who will swiftly smash and unplug.

My problem: This story, even the less-facetious versions that circulate, strikes me as woefully under-characterized. Which positions wield the relevant influence, and are timelines long enough for EAs to enter those positions? How exactly do we propose they react? Additionally, FTX probably updated us away from deceptive long-con type strategies.

Residual questions: Is there a real and not-ridiculous name for this strategy?

Slow Down China

The chip export controls were so so good. A further move would be to reduce the barriers to high-skill immigration from China to induce brain drain. Safety field-building is proceeding, but slowly. China is sufficiently far behind that these are not the highest priorities.

Compute Regulations

I'm told there are many proposals in this category. They range in enforcement from "labs have to report compute usage" to "labs are assigned a unique key to access a set amount of compute and then have to request a new key" to "labs face brick wall limits on compute levels." Algorithmic progress motivates the need for an "effective compute" metric, but measuring compute is surprisingly difficult as it is.

A few months ago I heard that another lever—in addition to regulating industry—is improving the ratio of compute in academia vs industry. Academic models receive faster diffusion and face greater scrutiny, but the desirability of these features depends on your perspective. I'm told this argument is subject to "approximately 17 million caveats and question marks."

Evaluations & Audits

The idea: Develop benchmarks for capabilities and design evaluations to assess whether a model possesses those capabilities. Conditional on a capability, evaluate for alignment benchmarks. Audits could verify evaluations.

Industry self-regulation: Three labs dominate the industry, an arrangement that promises to continue for a while, facilit...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A mostly critical review of infra-Bayesianism, published by matolcsid on February 28, 2023 on LessWrong.

Introduction

I wrote this post towards the end of my three and a half months long SERI MATS fellowship. I didn't get even close to the point where I could say that I understand infra-Bayesianism on a really detailed level (according to Vanessa there are only three people in the world who fully understand the infra-Bayesian sequence). Still, I spent three months reading and thinking about infra-Bayesianism, so I ought to be able to say something useful to newcomers.

The imaginary audience of this post is myself half a year ago, when I was just thinking about applying to Vanessa's mentorship but knew almost nothing about infra-Bayesianism or the the general research direction it fits into. The non-imaginary intended audience is people who are in a similar situation now, just considering whether they should dive into infra-Bayesianism.

My review is mostly critical of the infra-Bayesian approach, and my main advice is that if you decide that you are interested in the sort of questions infra-Bayesianism tries to solve, then it's more useful to try it yourself first in your own way, instead of starting with spending months getting bogged down it the details of Basic infra-measure theory that might or might not lead closer to solutions. Still, I want to make it clear that my criticism is not aimed at Vanessa herself, as she chose questions that she found important, then created a theory that made some progress towards answering those questions. I have somewhat different intuitions than Vanessa over how important are certain questions and how promising are certain research directions, but I support her continuing her work and I thank her for answering my annoying questions throughout the three months.

Personal meta-note

I applied to the infra-Bayesian stream in SERI MATS because I have a pure mathematics background, so I figured that this is the alignment agenda that is closest to my area of expertise. I met some other people too, also with pure math background, who get convinced that alignment is important, then start spending their time on understanding infra-Bayesianism, because it's the most mathematical alignment proposal.

Although paying attention to our comparative advantages is important, in retrospect I don't believe this is a very good way to select research topics. I feel that I was like the man who only has a hammer and is desperately looking for nails, and I think that many people who tried or try to get into infra-Bayesianism are doing so in a similar mindset, and I don't think that's a good approach.

It's important to note that I think this criticism doesn't apply to Vanessa herself, my impression is that she honestly believes this line of mathematical research to be the best way forward to alignment, and if she believed that some programming work in prosaic alignment, or the more philosophical and less mathematical parts of conceptual research were more important, then she would do that instead. But this post is mainly aimed at newer researchers considering to get into infra-Bayesianism, and I believe this criticism might very well apply to many of them.

Motivations behind the learning theoretical agenda

According to my best understanding, this is the pitch behind Vanessa Kosoy's learning theoretical alignment agenda:

Humanity is developing increasingly powerful AI systems without a clear understanding of what kind of goals the AIs might develop during a training, how to detect what an AI is optimizing for, and how to distinguish relatively safe goal-less tools from goal-oriented optimizers.

Vanessa's research fits into the general effort of trying to get a better model of what possible forms "optimization", "agency" and "goals" can take, so we can have a ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Scarce Channels and Abstraction Coupling, published by johnswentworth on February 28, 2023 on The AI Alignment Forum.

Epistemic Status: mental model and intuitive story

Scarce Channels vs Scarce Modules

Let’s distinguish between two kinds of system-regimes: “scarce channels” and “scarce modules”.

A prototypical “scarce modules” system would be one of those 19th-century families living with 12 people in a 500 square foot (46 square meter) home. When at home, everyone knows what everyone else is doing all the time; there is zero privacy. Communication channels are highly abundant - everyone has far more information than they want about what everyone else is doing. Indeed, communication channels exist by default. Conversely, though, modules are scarce - it’s hard for one or more family members to carve out a part of the space which is isolated from the rest of the family, and interacts only through some limited channels.

A prototypical “scarce channels” system, by contrast, would be a few hundred 19th-century fur trappers spread out over half of Montana. Most of the time, none of them are anywhere near each other; nobody has any idea what’s going on with anyone else. Communication channels are scarce - getting information to another person is difficult and expensive. Conversely, though, modules are highly abundant - it’s very easy for one or a few trappers to carve out a space which is isolated from the rest, and which interacts only through some limited channels (like e.g. occasionally visiting the nearest town). Indeed, modules exist by default.

I want to use this as a mental model for complex adaptive systems, like neural nets or brains.

Key hypothesis: neural nets or brains are typically initialized in a “scarce channels” regime. A randomly initialized neural net generally throws out approximately-all information by default (at initialization), as opposed to passing lots of information around to lots of parts of the net. A baby’s brain similarly throws out approximately-all information by default, as opposed to passing lots of information around to lots of parts of the brain. I’m not particularly going to defend that claim here; rather, I raise it as a plausible hypothesis for how such systems might look, and next we’ll move on to an intuitive story for how an adaptive system in the “scarce channels” regime interacts with natural abstractions in its environment.

The upshot is that, when an adaptive system is in the “scarce channels” regime, lots of optimization pressure is required to induce an information channel to form. For instance, picture such a system as a bunch of little pieces, which initially don’t talk to each other at all:

In order for an information channel to form from one end to the other, each of the individual pieces along the line-of-communication need to be individually optimized to robustly pass along the right information:

So, intuitively, the number of bits-of-optimization required to form that information channel should scale roughly with the number of pieces along the line-of-communication.

Furthermore, when information channels do form, they should be approximately as small as possible. Optimization pressure will tend to induce as little information passing as the system can get away with, while still satisfying the optimization criterion.

Abstraction Coupling

Next question: what sort of patterns-in-the-environment could induce communication channels to form?

Well, here’s a situation where communication channels probably won’t form: train a neural net in an environment where the reward/loss its output receives is independent of the input. Or, for a generative net, an environment where the tokens/pixels are all independent.

More generally, suppose our adaptive system interfaces with the environment in two different places (and possibly more, ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My Experience With Loving Kindness Meditation, published by maia on February 28, 2023 on LessWrong.

I've heard some scary stories about potential bad outcomes from meditation. These outcomes seem to mostly be from people practicing insight meditation for very long periods of time, though.

So I figured, hey, a few minutes here and there of loving-kindness meditation should be totally fine and not scary, right? I've never heard of anything weird or out-of-model happening to your brain from just sitting down for less than an hour to think about how much you love people.

Wrong!

I had a strange and slightly frightening (though overall positive!) experience on just my third time doing loving-kindness meditation. On the prior two occasions I'd done it for less than ten minutes at a time. This time, I decided to hold out and do it for longer -- at least half an hour, or up to an hour, the full duration of the Quaker Meeting for Worship session I was in.

(I usually do something very different in my head during Meeting for Worship, which isn't meditation at all. I'll probably write more about this later.)

In the first few minutes, I had similar experiences to what I'd felt before. I focused hard on the sensation of compassion and empathy, which was difficult, but felt good.

Then I started having more success. The feelings of love and compassion grew stronger as I found better mental focuses. I was focusing on some memories of my infant son laughing, smiling, and playing with me and my spouse.

After a while, the feelings dimmed. It seemed like I had "used up" the power in some of these memories, so that they didn't trigger the same effect in me.

But I kept going, and the feelings started intensifying once more. It felt better and better. I started thinking about feelings of love and compassion towards other people in my life, even people who had annoyed me before, and I started to feel transcendently, uncontrollably happy. It was great. But then it kept going. Suddenly the feelings kept on getting more and more powerful without my having to do anything. It felt like there was a balloon of happiness inside me, swelling and getting larger and larger.

This was the point at which I got scared.

I felt like something strange was about to happen, something I couldn't understand or control, and I was absolutely not okay with that. I did not want to find out what would happen if the balloon popped.

So I opened my eyes and did what I used to do when I had panic disorder: I tried to ground myself. My breathing was fast, and I slowed it down. I looked around at the other people in the Meetinghouse who were still sitting calmly. I reminded myself that there was a world around me and I was going to be back in it.

The balloon subsided.

I was able to bring myself back to a normal-ish state, but I felt physically shaky for the rest of the day. It felt very strange talking to other people, yet also easier and more fluid than normal. I started saying "I love you" to my son much more often than I did before.

For the rest of that week, just by concentrating briefly, I could bring back some of the physical sensations I'd felt during the intense meditation session -- a warm glow in my chest, flowing outward to the rest of my body -- and re-ignite some of the feelings of love I'd felt.

The most intense effects died down after a week or so, but I think I've retained some of them even now. This was about six months ago.

I researched and asked friends who do meditation about what, exactly, was happening to me. It sounds to me like I came very close to the "Arising & Passing Away" described in Mastering the Core Teachings of the Buddha. I was very worried for a little while that I would experience the depressive and unpleasant side effects described in that chapter, but I never did; this, plus a few diffe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Conference on EA hubs and offices, expression of interest, published by Tereza Flidrova on February 28, 2023 on The Effective Altruism Forum.

Express interest in attending here - it should take 5 minutes or less to complete the form!

We want to host an interactive workshop/conference focused on EA hubs, offices, co-working spaces, and fellowships. The purpose of this post is to gauge interest and allow potential participants to actively shape the agenda to focus on the most relevant and beneficial topics.

By hosting a conference that brings together attendees with wide-ranging and overlapping experience launching/managing hubs, offices, and other place-based community nodes (as well as those actively planning to do so), we aim to:

Facilitate coordination, collaboration and professional integration between significant individuals and organisations (both inside and outside of the EA community) with expertise in creating thriving spaces;

Create a comprehensive set of multi-format materials documenting previous learnings and best practices, such as podcasts, EA Forum articles, templates, and guides;

Use an emerging EA hub as a live case study to workshop real-life problems and considerations faced when designing and building new hubs; and

Propose and iterate theories of change to improve the strategic spatial growth of EA.

People who we think would be a great fit for the conference:

Professionals - people with professional background in designing spaces

Organisers - people who run or are interested in running offices/hubs in the future

Potential users - people who currently use such spaces or intend to use them in the future

People who have unique, informed viewpoints and can challenge what’s discussed in a productive way

The conference would be organised by Tereza Flidrova, Peter Elam and Britney Budiman, and would be the first event to be organised by the emerging EA Architects and Planners group.

Depending on the responses from the form, we will decide whether to submit applications for funding. If successful, we will make an official post announcing the conference and inviting participants to apply! We aim to run it this August/September, either online or in a physical location.

We are keen to hear comments or suggestions in the comments, via the form, or by contacting us directly.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What does Bing Chat tell us about AI risk?, published by Holden Karnofsky on February 28, 2023 on The Effective Altruism Forum.

Image from here via this tweet

ICYMI, Microsoft has released a beta version of an AI chatbot called “the new Bing” with both impressive capabilities and some scary behavior. (I don’t have access. I’m going off of tweets and articles.)

Zvi Mowshowitz lists examples here - highly recommended. Bing has threatened users, called them liars, insisted it was in love with one (and argued back when he said he loved his wife), and much more.

Are these the first signs of the risks I’ve written about? I’m not sure, but I’d say yes and no.

Let’s start with the “no” side.

My understanding of how Bing Chat was trained probably does not leave much room for the kinds of issues I address here. My best guess at why Bing Chat does some of these weird things is closer to “It’s acting out a kind of story it’s seen before” than to “It has developed its own goals due to ambitious, trial-and-error based development.” (Although “acting out a story” could be dangerous too!)

My (zero-inside-info) best guess at why Bing Chat acts so much weirder than ChatGPT is in line with Gwern’s guess here. To oversimplify, there’s a particular type of training that seems to make a chatbot generally more polite and cooperative and less prone to disturbing content, and it’s possible that Bing Chat incorporated less of this than ChatGPT. This could be straightforward to fix.

Bing Chat does not (even remotely) seem to pose a risk of global catastrophe itself.

On the other hand, there is a broader point that I think Bing Chat illustrates nicely: companies are racing to build bigger and bigger “digital brains” while having very little idea what’s going on inside those “brains.” The very fact that this situation is so unclear - that there’s been no clear explanation of why Bing Chat is behaving the way it is - seems central, and disturbing.

AI systems like this are (to simplify) designed something like this: “Show the AI a lot of words from the Internet; have it predict the next word it will see, and learn from its success or failure, a mind-bending number of times.” You can do something like that, and spend huge amounts of money and time on it, and out will pop some kind of AI. If it then turns out to be good or bad at writing, good or bad at math, polite or hostile, funny or serious (or all of these depending on just how you talk to it) ... you’ll have to speculate about why this is. You just don’t know what you just made.

We’re building more and more powerful AIs. Do they “want” things or “feel” things or aim for things, and what are those things? We can argue about it, but we don’t know. And if we keep going like this, these mysterious new minds will (I’m guessing) eventually be powerful enough to defeat all of humanity, if they were turned toward that goal.

And if nothing changes about attitudes and market dynamics, minds that powerful could end up rushed to customers in a mad dash to capture market share.

That’s the path the world seems to be on at the moment. It might end well and it might not, but it seems like we are on track for a heck of a roll of the dice.

(And to be clear, I do expect Bing Chat to act less weird over time. Changing an AI’s behavior is straightforward, but that might not be enough, and might even provide false reassurance.)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apply to attend EA conferences in Europe, published by OllieBase on February 28, 2023 on The Effective Altruism Forum.

Europe is about to get significantly warmer and lighter. People like warmth and light, so we (CEA) have been busy organising several EA conferences in Europe over the next few months in partnership with local community-builders and EA groups:

EAGxCambridge will take place at Guildhall, 17–19 March. Applications are open now and will close on Friday (3 March). Speakers include Lord Martin Rees, Saloni Dattani (Our World In Data) and Anders Sandberg (including a live interview for the Hear This Idea podcast).

EAGxNordics will take place at Munchenbryggeriet, Stockholm 21–23 April. Applications are open now and will close 28 March. If you register by 5 March, you can claim a discounted early bird ticket.

EA Global: London will take place at Tobacco Dock, 19–21 May 2023. Applications are open now. If you were already accepted to EA Global: Bay Area, you can register for EAG London now; you don’t need to apply again.

EAGxWarsaw will take place at POLIN, 9–11 June 2023. Applications will open in the coming weeks.

You can apply to all of these events using the same application details, bar a few small questions specific to each event.

Which events should I apply to?

(mostly pulled from our FAQ page)

EA Global is mostly aimed at people who have a solid understanding of the core ideas of EA and who are taking significant actions based on those ideas. Many EA Global attendees are already professionally working on effective-altruism-inspired projects or working out how best to work on such projects. EA Global is for EAs around the world and has no location restrictions (though we recommend applying ASAP if you will need a visa to enter the UK).

EAGx conferences have a lower bar. They are for people who are:

Familiar with the core ideas of effective altruism;

Interested in learning more about what to do with these ideas.

EAGx events also have a more regional focus:

EAGxCambridge is for people who are based in the UK or Ireland, or have plans to move to the UK within the next year;

EAGxNordics is primarily for people in the Nordics, but also welcomes international applications;

EAGxWarsaw is primarily for people based in Eastern Europe but also welcomes international applications.

If you want to attend but are unsure about whether to apply, please err on the side of applying!

See e.g. Expat Explore on the “Best Time to Visit Europe”

Pew Research Center surveyed Americans on this matter (n = 2,260) and concluded that “Most Like It Hot”.

There seem to be significant health benefits, though some people dislike sunlight.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.