The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the new 80,000 Hours Career Guide, published by Benjamin Hilton on September 4, 2023 on The Effective Altruism Forum.

From 2016 to 2019, 80,000 Hours' core content was contained in our persistently popular career guide. (You may also remember it as the 80,000 Hours book: 80,000 Hours - Find a fulfilling career that does good).

Today, we're re-launching that guide. Among many other changes, in the new version:

We've substantially changed our recommendations on career capital.

We have significantly improved and extended our article on career planning.

We improved our advice on personal fit and career exploration.

We added sections on why community-building, government and policy, and organisation-building careers could be high impact.

We focus more on avoiding harm (in line with our updates following the collapse of FTX), and explicitly discuss Sam Bankman-Fried when talking about earning to give.

We are more upfront about 80,000 Hours' focus on existential risk in particular (while also discussing a wide variety of cause areas, including global health, animal welfare, existential risk and meta-causes).

We've updated the more empirical sections of the guide using more up-to-date papers and data.

You can read the guide here or start with a 2-minute summary.

It's also available as a printed book (you can get a free copy by signing up for our newsletter, or buy it on Amazon), audiobook, podcast series or ebook (available as a .pdf or .epub).

We'd appreciate you sharing the new guide with a friend! You can send them a free copy using this link. Many of the people who've found our advice most useful in the past have found us via a friend, so we think the time you take to share it could be really worthwhile.

What's in the guide?

The career guide aims to cover the most important basic concepts in career planning. (If instead you'd like to see something more in-depth, see our advanced series and podcast.)

The first article is about what to look for in a fulfilling job:

Part 1: What makes for a dream job?

The next five are about which options are most impactful for the world:

Part 2: Can one person make a difference?

Part 3: How to have a real positive impact in any job

Part 4: How to choose which problems to focus on

Part 5: What are the world's biggest and most urgent problems?

Part 6: What types of jobs help the most?

The next four cover how to find the best option for you and invest in your skills:

Part 7: Which jobs put you in a better position?

Part 8: How to find the right career for you

Part 9: How to be more successful in any job

Part 10: How to write a career plan

The last two cover how to take action and launch your dream career:

Part 11: How to get a job

Part 12: How community can help

Advice (for EAs) on how to read the guide

The topics we tackle are complex, and in the past we've noticed people interpreting our advice in ways we didn't intend. Here are some points to bear in mind before diving in.

We've been wrong before and we'll be wrong again. While we've spent a lot of time thinking about these issues, we still have a lot to learn. Our positions have changed over the years, and due to the nature of the questions we take on, we're rarely more than about 70% confident in our answers. You should try to strike a balance between what we think and your previous position, depending on the strength of the arguments and how much you already knew about the topic.

It's extremely difficult to give universally applicable career advice. Most importantly, the option that's best for you depends a huge amount on your skills, circumstances, and the specific details of the opportunity. So, while we might highlight path A more than path B, the best opportunities in path B will often be better than the typical opportunities in path A. Moreover, your personal circumstanc...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Paper: On measuring situational awareness in LLMs, published by Owain Evans on September 4, 2023 on The AI Alignment Forum.

This post is a copy of the introduction of this paper on situational awareness in LLMs.Authors: Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Max Kaufmann, Meg Tong, Tomasz Korbak, Daniel Kokotajlo, Owain Evans.

Abstract

We aim to better understand the emergence of situational awareness in large language models (LLMs). A model is situationally aware if it's aware that it's a model and can recognize whether it's currently in testing or deployment. Today's LLMs are tested for safety and alignment before they are deployed. An LLM could exploit situational awareness to achieve a high score on safety tests, while taking harmful actions after deployment.

Situational awareness may emerge unexpectedly as a byproduct of model scaling. One way to better foresee this emergence is to run scaling experiments on abilities necessary for situational awareness. As such an ability, we propose out-of-context reasoning (in contrast to in-context learning). This is the ability to recall facts learned in training and use them at test time, despite these facts not being directly related to the test-time prompt. Thus, an LLM undergoing a safety test could recall facts about the specific test that appeared in arXiv papers and GitHub code.

We study out-of-context reasoning experimentally. First, we finetune an LLM on a description of a test while providing no examples or demonstrations. At test time, we assess whether the model can pass the test. To our surprise, we find that LLMs succeed on this out-of-context reasoning task. Their success is sensitive to the training setup and only works when we apply data augmentation. For both GPT-3 and LLaMA-1, performance improves with model size. These findings offer a foundation for further empirical study, towards predicting and potentially controlling the emergence of situational awareness in LLMs.

(Code is available here).

Introduction

In this paper, we explore a potential emergent ability in AI models: situational awareness. A model is situationally aware if it's aware that it's a model and it has the ability to recognize whether it's in training, testing, or deployment (Ngo et al., 2022; Cotra, 2022). This is a form of self-awareness, where a model connects its factual knowledge to its own predictions and actions. It's possible that situational awareness will emerge unintentionally from pretraining at a certain scale (Wei et al., 2022a). We define situational awareness in Section 2 (Background).

If a model is situationally aware, it's harder to test whether it is aligned or safe. Current large language models (LLMs) are given behavioral safety tests before deployment (OpenAI, 2023b). An LLM with situational awareness knows when it's being tested and how the tests work, based on information learned in pretraining. Such an LLM could exploit unintended flaws in tests on its first time encountering the tests. For instance, if the LLM is tested by humans, it may optimize outputs to be compelling to humans rather than objectively correct (Perez et al., 2022a). Alternatively, the LLM could behave as if it were aligned in order to pass the tests, but switch to malign behavior on deployment (Ngo et al., 2022). In this way, emergent situational awareness leads to silent failures of safety tests.

Because of these risks, it's important to predict ahead of time when situational awareness will emerge (Ganguli et al., 2022). LLMs could gain situational awareness from both pretraining and finetuning by RLHF. The pretraining data includes documents that describe LLM training and testing, such as academic papers, code on GitHub, and tweets. When an LLM is being tested, it could leverage these descriptions to infer that it's being test...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rethinking the Liberation Pledge (Eva Hamer), published by Aaron Gertler on September 3, 2023 on The Effective Altruism Forum.

This article is a really interesting example of people in a prosocial movement trying radical tactics and eventually changing their minds. I'm not sure there's any one lesson here that the average Forum reader would need to learn; I'm just crossposting because I enjoyed it.

See parts two and three for how the Pledge has evolved.

The Failure of the Pledge and a Better Way Towards Vegan Tables

In 2015, animal advocates with Direct Action Everywhere (DxE) launched an inspired new campaign among their members. It took courage, required sacrifice, and greatly backfired. This three-part series examines what the movement learned from the Liberation Pledge, how we might energize the intention behind the Pledge in a better way, and a piece to share with friends and family to do just that.

What We Learned From the Liberation Pledge

How It Started

The Liberation Pledge was a fascinating idea and a bit of a disaster. Instead of energizing supporters' social networks to create change, as its creators intended, it often had the opposite effect- to isolate advocates from their closest relationships.

In this piece, you'll learn

What it was

Why it was a good idea

Why it failed

The next piece in this series suggests a better way to energize the intention behind the pledge, for animal advocates to align their actions with their values in their personal relationships.

The Liberation Pledge was a three-part public pledge to

live vegan,

refuse to sit at tables where animals' bodies are being eaten, and

encourage others to do the same.

Enthusiasts of the pledge hoped it would create a cultural stigma around eating animals similar to the stigma that has developed around smoking over recent decades. That is, even while smoking is still practiced, it is prohibited by default in public and private spaces.

Before we had the Pledge, many of us felt alienated from friends and family who continued to eat animals. We were forced to choose between two options: speaking up and risking being seen as obnoxious, angry, and argumentative, or keeping the peace with painful inauthenticity, swallowing our intense discomfort at watching our loved ones eat the bodies of animals.

The pledge gave us hope that there was another way: being honest with those around us while continuing to spend time with them. And, on a larger scale, we hoped that if we all joined together, we could create a world where eating meat is stigmatized: a world where someone would ask, "Does anyone mind if I get the steak?" before making an order at a restaurant (or maybe even one in which restaurants would think twice before putting someone's body on the menu).

Some people took it a step further, arguing it was immoral not to take the pledge, saying, "You wouldn't sit quietly eating your vegan option while a dog or a child was being eaten, would you?" According to this view, it was our duty not to sit idly by while violence was committed in our presence.

While some beautiful and inspiring stories were detailed on a Facebook group for the Pledge, it seemed to me that there were many more instances of total disaster: people experiencing huge ruptures in their oldest relationships around the Pledge while often lamenting that those they had just discarded "care more about eating dead animals than they care about me."

From where I stood, the biggest effect of the Pledge was for advocates to lose relationships with family members who didn't comply. Upon taking the Pledge, a close friend at the time experienced a years-long estrangement from their family, including those who were already vegan while many others decided to skip birthdays, weddings, and holidays with family. It's possible that all of this added stigma ar...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Please share with lawyers you know: Legal Impact for Chickens seeks litigator., published by alene on September 3, 2023 on The Effective Altruism Forum.

Hi EAs!

Thank you to everyone here who has supported Legal Impact for Chickens in various ways.

Legal Impact for Chickens is hiring again! Please forward widely!

Legal Impact for Chickens seeks a litigator.

We're looking for the next attorney to help build our nonprofit and fight for animals.

Want to join our team on the ground floor?

About us:

Legal Impact for Chickens (LIC) is a 501(c)(3) litigation nonprofit. We work to protect farmed animals.

You may have seen our Costco shareholder derivative suit in The Washington Post, Fox Business, or CNN Business - or even on TikTok.

Now, we're looking for our next hire: an entrepreneurial litigator to help fight for animals!

About you:

Licensed and in good standing with your state bar

3+ years of litigation experience preferred

Excellent analytical, writing, and verbal-communication skills

Zealous, creative, enthusiastic litigator

Passion for helping farmed animals

Interest in entering a startup nonprofit on the ground floor, and helping to build something

Willingness to do all types of nonprofit startup work, beyond just litigation

Strong work ethic and initiative

Love of learning

Kind to our fellow humans, and excited about creating a welcoming, inclusive team

About the role:

You will be an integral part of LIC. You'll help shape our organization's future.

Your role will be a combination of (1) designing and pursuing creative impact litigation for animals, and (2) helping with everything else we need to do, to run this new nonprofit!

Since this is such a small organization, you'll wear many hats: Sometimes you may wear a law-firm partner's hat, making litigation strategy decisions or covering a hearing on your own. Sometimes you'll wear an associate's hat, analyzing complex and novel legal issues. Sometimes you'll pitch in on administrative tasks, making sure a brief gets filed properly or formatting a table of authorities. Sometimes you'll wear a start-up founder's hat, helping plan the number of employees we need, or representing LIC at conferences. We can only promise it won't be dull!

This job offers tremendous opportunity for advancement, in the form of helping to lead LIC as we grow. The hope is for you to become an indispensable, long-time member of our new team.

Commitment: Full time

Location and travel: This is a remote, U.S.-based position. You must be available to travel for work as needed, since we will litigate all over the country.

Reports to: Alene Anello, LIC's president

Salary: $80,000-$100,000 depending on experience

Benefits: Health insurance, 401(k), flexible schedule, unlimited PTO (plus mandatory vacation!)

One more thing!

LIC is an equal opportunity employer. Women and people of color are strongly encouraged to apply. Applicants will receive consideration without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, ancestry, citizenship status, disability, age, medical condition, veteran status, marital status, political affiliation, or any other protected characteristic.

To Apply:

To apply, please email your cover letter, resume, writing sample, and three references, all combined as one PDF, to [email protected].

Thank you for your time and your compassion!

Sincerely, Legal Impact for Chickens

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Fundamental question: What determines a mind's effects?, published by Tsvi Benson-Tilsen on September 3, 2023 on The AI Alignment Forum.

[Metadata: crossposted from. First completed April 9, 2023.]

A mind has some effects on the world. What determines which effects a mind has? To eventually create minds that have large effects that we specify, this question has to first be answered.

Slippery questions

A slippery question is a question that, when you try to approach it looking for an answer, it slips out of your grasp. It runs away or withdraws, it cloaks itself in false expressions. It leads you astray, replacing itself with a similar-sounding question, maybe one that's easier to answer. It's hard to hug a slippery question. The question withdraws.

Or worse, it pretends to be made out of ungrounded abstract concepts, so that you're not sure anymore that there's even a real question there. This slipperiness is especially likely for questions that involve "big" things, such as minds.

The question might seem obvious and even tautological. Or it might seem simplistic, failing to carve reality at the joints. These appearances usually somewhat accurately reflect the reality - - but they do not imply necessarily that there is not a real question behind the appearances. A question that can be pointed at, but only imperfectly, inchoately, ostensively, or preliminarily, is at risk of being question-substituted or being ignored.

The discomfort of not having an answer or even a clear question pushes people to run ahead of the question. When they make an error that would have been prevented by meditating on the question, even if they notice the error, they don't notice that they are in general not focusing on the question. If the question asks: There is something, it has to do with concrete things A, B, and C, and it is important - - what is it? Then someone will analyze A and B in detail, gain some understanding, and declare the whole question - - including C, and the inexplicit core question - - solved.

This essay will ask a slippery question over and over in different words, hoping to arrive at a firmer grasp on the real questions. The starting point:

What determines the effects of a mind?

The word "effects"

Suppose there is a mind M. As a result of M existing, the cosmos (the entirety of the world, in any aspect or abstraction) is different from how it would have been if M hadn't existed. Those differences are the effects of M.

What determines the difference in the cosmos that results from the presence of the mind?

The effects of a mind depend on how strong the mind is. But the question wants to ask about the sort of effects, not their size:

What determines the direction of a mind's effects?

Minds pursue instrumental subgoals, which affect the world. These effects might matter, but they are subordinate to their supergoals. Any reversible effect might be reversed. The question asks about the final effects of the mind:

What determines the direction of a mind's ultimate effects on the cosmos?

"Direction" is only a metaphor. It suggests that there's something compactly understandable about the effects of the mind. The metaphor may not hold well. To nod in that direction, the question can at least be rephrased to suggest multi-dimensionality:

What determines the directions of a mind's ultimate effects on the cosmos?

The word "mind"

What does "mind" mean here? The choice of the word "mind" reflects the guess that things that have large effects will be integrated - - will speak in internal languages, pass information, correct errors, and in general become coherent.

Rather than "mind", the question could refer to [things that have large effects], reading:

What determines the effects of a large-effect-haver?

But this is too broad. The cosmos as a whole is a large-effect-haver, in...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Lives We Can Save, published by Omnizoid on September 3, 2023 on The Effective Altruism Forum.

I work as a Resident Assistant at my college. Last year, only a few weeks into me starting, I was called at night to come help with a drunk student. I didn't actually help very much, and probably didn't have to be there. I didn't even have to write up the report at the end. At one point I went outside to let medical services into the building, but mostly I just stood in a hallway.

The person in question was so drunk they couldn't move. They had puked in the bathroom and were lying in the hallway crying. They could barely talk. When Campus Safety arrived they kneeled down next to this person and helped them drink water, while asking the normal slew of questions about the person's evening.

They asked this person, whose name I can't even remember, why they had been drinking so much. They said, in between hiccups and sobs, "friend doesn't want to be friend anymore."

How do you describe that feeling? I don't think transcription can convey the misery and the drunkenness and the awful situation that had led to this awful situation. Someone drank so much that they could barely move, was lying curled in a hallway where all the other residents could and were watching, and was only able to muster out "friend doesn't want to be friend anymore" as they cried.

Should I only care because I happened to be standing in that hallway on a late September evening? Had I remained in my room, laughing with my friends, would this person's struggle have been worth nothing?

Max Alexander (this whole post is very worth reading)!

It's sometimes hard to be motivated to help the world. The trip you forego, the fun you could have had with a friend, the nice things you could have bought are instead sent straight into the coffers of some charity that you've read about. It can feel sort of alienating when you think just of the number of people you have saved. Instead of thinking of numbers, think of stories. The people who make up the numbers - who make up the hundreds of thousands of lives saved by effective charities - are real, flesh-and-blood people, who matter just as much as you and I. We may not look into the gaunt faces of those who would have otherwise starved to death, we may not see their suffering with our eyes, but we know it is real. People are dying in ways that we can prevent. GiveWell top charities can save lives for only a few thousand dollars.

It's hard to get your mind around that. I have a friend who has raised over 50,000 dollars for effective charities. 10 lives. 10 people. 10 people, many of them children, who will be able to live out a full life, rather than being snuffed out at a young age by a horrible painful disease. They will not have to lie in bed, with a fever of 105, slowly dying of malaria when they are five. They will have the chance to grow up.

Who are these people? I do not know. But I can imagine their stories. I can imagine their stories because I can hear the stories of other people like this, people who are about to die. For example, on this Reddit thread, you can find the stories of lots of people who are about to die. Stories like these:

Stage IV colon cancer here. Age 35. I'm a single mum to a 1-year-old and there is a 94% chance I'll be dead in 4 years. But there is still a wee bit of hope, so I try to hold onto that (hard to do most days). My days are filled with spending time with my baby and hoping that I live long enough that she'll remember me. She's pretty awesome and makes me laugh every day, so there is a lot of happiness in this life of mine.

Reading these stories causes me to tear up. I think a lot of people have a similar response. They're so tragic - entire lives being snuffed out. The line "My days are filled with spending time with my baby and ho...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The goal of physics, published by Jim Pivarski on September 3, 2023 on LessWrong.

In grad school, I was a teaching assistant for a course called, Why the Sky is Blue. It was a qualitative introduction to physics for non-majors, covering a lot of the same topics as Physics I, such as forces, conservation of energy and momentum, electric charges and magnetic fields, in less detail, with not much math. The actual question about why the sky is blue was saved for the end. As the course dragged on and the students (who expected no math, rather than not much math) started to complain, "Are we ever going to find out why the sky is blue?" I watched the schedule slip and wondered the same thing.

We skipped some sections and managed to wedge it into the last lecture: finally, we were talking about why the sky is blue! "The sky is blue because of Rayleigh scattering." Okay, that's not an answer we hadn't defined Rayleigh scattering, there wasn't time for it, so we said that air molecules absorb and re-radiate - effectively changing the direction of - blue light more than red light. Red light goes straight through the atmosphere, and blue light bounces around, making the whole sky glow blue. Conversely, sunrises and sunsets are red because you're looking at the light that has gone straight through a larger wedge of atmosphere. It lost most of its blue on the way to your eye.

Pretty good explanation, for not being able to say

(the 1/λ4 part affects small-λ blue light more than large-λ red light). We also showed pictures like this sunset:

to demonstrate the effect of straight-through red light and bouncing-around blue light.

So in the end, "Why is the sky blue?"

Answer: "Because sunsets are red!"

"And why are sunsets red...?"

It was understandably unsatisfying. One thing was only explained in terms of another thing. But even if we had the time to get into detail about Rayleigh scattering, they could reasonably ask, "Why does light scatter according to that formula?" We could go deeper and explain Lord Rayleigh's proof in terms of Maxwell's equations. And whyfore Maxwell's equations? Well, quantum electrodynamics, which is a quantum field theory with a local U(1) gauge symmetry, which is to say that every point in space has an extra degree of freedom, similar to a fourth spatial dimension except that this dimension can't be rotated with normal space like the other three, this dimension is connected to itself as a circle instead of being infinite (that's what the U(1) means), and neighboring points in 3D space try to minimize differences in this extra parameter, which leads to waves.

The explanatory power is breathtaking: you can actually derive that photons must exist, if you assume that there's this U(1) symmetry laying around. But why is there a U(1) symmetry?

Modern physics seems to be obsessed with symmetries. Even this U(1) symmetry is explained in terms of a more fundamental SU(2)×U(1) (different U(1)) and the Higgs mechanism. Physicists seem to be holding their tongues, avoiding saying, "This is the most basic thing," by saying, "This one thing is actually a manifestation that other thing." Answering the question, "Why do photons exist?" with "Because space has an internal U(1) symmetry" is a bit like saying, "The sky is blue because sunsets are red."

Symmetry explanations collapse our description of the world onto a smaller description. They say that one thing is mathematically derivable from the other, maybe in both directions, but they don't say why either is there at all. Perhaps that's an unanswerable question, and the symmetry language is a way of acknowledging the limitation.

To show what I mean, consider a universe that consists of nothing but a point in the exact center of a perfect circle. (I've been feeling free to say, "Consider a universe..." ever since a lecture...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Series of absurd upgrades in nature's great search, published by Luke H Miles on September 3, 2023 on The AI Alignment Forum.

So you want to find that special thing that replicates best and lasts longest? Just vibrate a bunch of molecules a long time!

You might reasonably assume that molecules wouldn't practically ever randomly assemble themselves into anything worth looking at. It happens a bit differently instead:

Eventually, "solar systems" will form. Each one has an independent star (energy source) with several independent planets (experiment boxes) that vary in their temperature, mass, composition, etc dramatically. (Instead of just having a big homogenous soup throughout the universe with much less search power)

these planets will conveniently have "continents" sometimes to run more independent experiments on each one

A lipid layer boundary innovation thing leads to trillions of independent chemical experiments on some planets (instead of a big soup within each planet)

A general protein factory is discovered. Critically, it uses a simple and concise and robust physical encoding scheme. This allows good bubbles from step 2 to copy quickly and exactly, and clear out weak bubbles faster.

Very large multicellulared organisms make several search improvements: creatures can do sexual selection on each other instead of just getting random mutations; big smart fast predators can find & kill weak creatures quite effectively; parasites find weaknesses early & often. This all clears out weak replicators and leaves more room for the good ones.

Some creatures develop language and culture, which isn't generally much use, except that it one day allows them to design new replicators from scratch.

So you really can just vibrate sand and get out all kinds of things!

So when we talk about the power of nature's search, let's not forget the miraculousness and variety of techniques here. It's not just generations, mutations, generations, mutations...

Not mentioned above:

separate galaxies to limit damage of supernova

separate species to prevent sex from turning the creatures to big crab soup

give creatures limited lifespans to ensure they're actually good replicators and not just robust

Lots of ways that the independence & quantity & efficiency of experiments was scaled up in the box of sand. Don't turn any knobs, just keep vibrating the box and it makes its own knobs.

Makes me think about how many float16s we're vibrating in our GPUs these days and what kind of tiny planets and cells are forming inside. I imagine it's not a very hospitable environment. But most of space isn't either.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The smallest possible button, published by Neil on September 2, 2023 on LessWrong.

tl;dr: The more knowledge you have, the smaller the button you need to press to achieve desired results. This is what makes moth traps formidable killing machines, and it's a good analogy for other formidable killing machines I could mention.

Traps

I was shopping for moth traps earlier today, and it struck me how ruthlessly efficient humans could be in designing their killing apparatus. The weapon in question was a thin pack in my hands containing just a single strip of paper which, when coated with a particular substance and folded in the right way, would end up killing most of the moths in my house. No need to physically hunt them down or even pay remote attention to them myself; a couple bucks spent on this paper and a minute to set it up, and three quarters of the entire population is decimated in less than a day.

That's. horrifying.

Moth traps are made from cardboard coated with glue and female moth pheromones. Adult males are attracted to the pheromones, and end up getting stuck to the sides where they end up dying. The females live, but without the males, no new larvae are born and in a few months time you've wiped out a whole generation of moths. These traps are "highly sensitive" meaning that they will comb a whole room of moths very quickly despite being passive in nature.

Why are moth traps so effective? They use surgically precise knowledge. Humans know how to synthesize moth pheromones, and from there you can hack a 250-million-year-old genetically derived instinct that male moths have developed for mating, and then you set a trap and voilà. The genetic heuristic that worked 99% of the time for boosting reproductive rates in moths can be wielded against moths by obliterating their reproductive rates.

Moth traps aren't even the pinnacle of human insecticidal war machines. Scientists have, after all, seriously considered using gene drives to eliminate an entire species of mosquitoes with a single swarm and some CRISPy cleverness.

The smallest button

Moth traps and gene drives work by understanding something so well that when you use brute force (because everything is brute force) to do something, you do it in the most optimal and surgical way. Intelligent design means humans can engineer very, very effective traps that harness the smallest buttons you can push in order to get a desired result.

Evolution can also produce sexually deceptive traps that take advantage of insect brains. This is because genes that contribute to pushing a particular button that makes reproduction more likely, are more represented in the environment, so most genes in living beings today are already vetted for their capacity to harness niche buttons in the universe.

The blind idiot god can't hope to compete with intelligent design however, so we can expect humans to win the find-the-smallest-button arms race against their evolution-derived enemies (like moths, mosquitoes, or viruses).

Brute force

Brute force always works. If you stuff enough moths into my house, my measly passive traps won't be sufficient. In fact, if my house were big enough and there were enough moths, the males that were somehow not attracted to my sticky female pheromones but found females anyway would be the only ones to pass down their genes. With enough moths and enough time, the blind idiot god of moth evolution would find a way to elude my traps by pressing an alternate small button to those specific pheromones, in order to power its reproduction. This type of brute force, which grants a stupid and blind enemy the power of adaptation, can be found in battles with cancer, viruses, or pesticides.

The only counter to this brute force is more brute force, in the form of chemotherapy, gene drives, or pesticides 1 level of magnitu...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: PIBBSS Summer Symposium 2023, published by Nora Ammann on September 2, 2023 on The AI Alignment Forum.

Tl;dr

We are pleased to invite you to the PIBBSS Summer Symposium where the fellows from the '23 fellowship program present their work.

The symposium is taking place online, over several days in the week of September 18th.

Check out the full program, including brief descriptions for each talk. (You can toggle between the different days.)

Register via this link. You will receive an email with the event link which you can use for the entire symposium. You will be able to drop into as few or as many presentations and breakout sessions as you like.

Click here to add the schedule to your google calendar.

About PIBBSS

PIBBSS is a research initiative aiming to explore parallels between intelligent behavior in natural and artificial systems, and to leverage these insights towards the goal of building safe and aligned AI.

During June-August '23, we ran the second iteration of our 3-month research fellowship program. The symposium acts as a venue to share research conducted as part of this program.

About the symposium

The PIBBSS Summer Symposium is a multi-day event where PIBBSS fellows present their work.

The event is taking place on the days of Tuesday - Friday, Sept 19th - 22nd, between 18:00-~21:00 CET / 9:00 - 12:00 PT / 12:00 - 15:00 ET.

The event is set it up such that you can easily join whichever talks and breakout sessions you are most interesting in.

The program

Find a program overview here.

Find the full program here, including brief descriptions of each talk.

On top of the talks, there will also be opportunities to continue the discussion with fellows at the end of each block in speaker-specific breakout rooms.

Talks span a wide range of topics in line with PIBBSS's research mission. Some representative examples of topics include:

novel avenues for interpretability

naturalistic approaches to understanding the nature and emergence of agency and goal-directed behavior

attempts to develop a principled understanding of the dynamics emerging from multi-agent interactions in and across AI/LLM systems

analyses of the space of AI risks - from single to multiple agents, from misuse to structural risks, etc.

exploration of the potential and limits of existing legal tools for reducing catastrophic risks from AI

..and more!

The format

The symposium is taking place over the course of four days, in blocks of ~4 fellows.

Each fellow presents for a total of 30 minutes, including some time for questions.

At the end of each block, there will be speaker-specific break-out rooms to allow for further questions and discussions.

Example day:

Day 1Starting timeSpeaker 1 18:00 CEST Speaker 2 18:30 CESTSpeaker 3 19:00 CESTSpeaker 4 19:30 CESTBreakout/Discussion Rooms with Speakers 1-4 (parallel)

20:00 CEST

21:00 CEST

How to engage

Register here to receive a link to the webinar.

The same link works for the entire symposium. This allows you to tune in for exactly those talks and breakout sessions you're most interested in!

Find the full program here, including brief descriptions of each talk. (You can toggle between the days to discover the full program.)

If you cannot make it to a talk, worry not! Most talks will be recorded and can later be viewed at the PIBBSS YouTube Page.

Talks (overview)

For a full version of the agenda, including talk descriptions, see here. Times below are in CEST.

Tuesday, Sep 19th

18:00 - Auto-Intentional Agency and AI Risk - Giles Howdle

18:30 - Allostasis emergence of auto-intentional agency - George Deane

19:00 - TBD - Urte Laukaityte

19:30 - Searching For a Science of Abstraction - Aysja Johnson

20:00 - Breakout sessions with each presenter

Wednesday, Sep 20th

18:00 - Agent, behave! Learning and sustaining social norms as normative equilibria - Ninell Oldenbur...