Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A few more relevant categories to think about diversity in EA, published by AmAristizabal on December 19, 2022 on The Effective Altruism Forum.

This is a Draft Amnesty Day draft. That means it’s not polished, it’s probably not up to my standards, the ideas are not thought out, and I haven’t checked everything. I was explicitly encouraged to post something unfinished! Commenting and feedback guidelines: I’m going with the default — please be nice. But constructive feedback is appreciated; please let me know what you think is wrong. Feedback on the structure of the argument is also appreciated.

Posted on draft Amnesty Day, one day late oops

People who think about diversity in EA or face diversity-related issues in their work often find it helpful to find common labels and identities. It allows identifying common problems, framings and solutions. The EA Survey already identifies common categories that are useful to think about diversity, such as gender, sexual orientation, age, race, education level and financial instability. Here I expand a bit on some other terms that I find particularly helpful.

Some important previous thoughts:

This is based on anecdotal use of the terms instead of official definitions (people with a background in diversity studies might have better suggestions).

These terms are also huge generalizations that fail to capture individual differences and the use of these terms can also harm diversity by creating “us vs them” narratives or packing people into groups that don’t represent them.

If terminology needs to be corrected, this post is also to invite refining these terms.

Low and Middle Income Countries (LMICs) (as opposed to High Income Countries)

Definition: All countries not considered to be high-income. Although there is no universally agreed-upon definition, the World Bank defines high-income countries as those with a gross national income per capita of $12,696 or more in 2020. Upper-middle, lower-middle, and low-income countries are classified as LMICs.

Why this category is useful for diversity in EA: I’ve been surprised by how many similarities there are between LMICs, regardless of regions. This seems true outside EA, and Gapminder’s dollar street project perfectly portrays this: they show with a massive collection of photos how income is the main determinant of people’s daily lives, regardless of geography. LMICs meetups in EA events tend to group compatible people with very common experiences.

EA is predominant in high income countries and I’ve found that the LMICs category is probably one of the best at capturing a lot of what people care about when they speak about diversity in EA. LMICs groups across the world share a lot in common.This guide, for example, was created by a team from India, Malaysia and Colombia and we all had relatable experiences.

Examples:

Usual EA framings that address topics like donations or income are unfamiliar to LMICs, such as the usual “you’re the top 1%” assumption (although in relative terms this can still be true at an individual level for high income individuals in LMICs).

It captures what is usually referred to as “global north” vs “global south”, which is relevant in many discussions. It might capture that distinction better, because there are some geographically southern countries that are high income (such as Australia) and northern countries that are LMICs.

It captures a lot of the historical divisions between colonized and colonizer countries and what that implies: countries often have weak institutions, corruption, poverty, poor responses to emergencies, inequality and all sorts of immediate problems.

We share similar questions, experiences and tradeoffs when we think about funding on the EA space. Funding in these countries has way more purchasing power than in high income countries and this has...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Top down interventions that could increase participation and impact of Low and Middle Income Countries in EA, published by AmAristizabal on December 19, 2022 on The Effective Altruism Forum.

This is a Draft Amnesty Day draft. That means it’s not polished, it’s probably not up to my standards, the ideas are not thought out, and I haven’t checked everything. I was explicitly encouraged to post something unfinished! Commenting and feedback guidelines: I’m going with the default — please be nice. But constructive feedback is appreciated; please let me know what you think is wrong. Feedback on the structure of the argument is also appreciated.

Posted on draft Amnesty Day, one day late oops

Thanks to Ben Garfinkel, Surbhi Bharadwaj and Mo Putera for feedback

This is a list of things that could help increase participation of LMICs in EA. Many of these actions involve tradeoffs (e.g. on people's time, efforts and resources) so probably the community shouldn’t do all of these things. Some of them might not currently be worth the trade-offs. However, since the community is serious about trying to increase diversity, and insofar as a number of these are actually pretty low cost, I think many of these might be worthwhile.

See also this great post (has more specific suggestions that overlap with some of the ones highlighted here). This post has been drafted in parallel to this EA career guide for people in LMICs (which is a more bottom-up approach).

Events:

Strongly encourage senior EAs to attend events that are accessible to people in LMICs: Having virtual conferences and EAGxs in LMICs seems great, but for these events to be successful it is important for senior people in EA (who are still overrepresented in high income countries) to continue to attend them. As the number of annual EA events increases, senior EAs could start prioritizing conferences in their own hubs and it could become increasingly hard for people in LMICs to network with them. If senior EAs underestimate the importance of attending conferences that are accessible to people in LMICs, then personal outreach to these EAs could help them make better prioritisation decisions. There may also be ways to make participation in these events more convenient or attractive to senior EAs.

Alternatively, making EAGs as inclusive as possible for LMICs. Some measures that could be done:

Advertising events further in advance, considering that people from LMICs might struggle more with visa appointments (as already highlighted here).

Provide immigration support like invitation letters for conferences and events and make the process smooth for LMICs attendees.

Advertising events more broadly in LMICs by sharing events with local community builders, orgs and universities (even nudging community builders in LMICs to advertise an EAG seems low effort and worth doing).

Consider hosting at least one annual EA Global in a more visa-friendly hub, instead of the UK and the US. For example, it could be worthwhile to host an EA Global in Mexico City or India (depending on visa requirements).

For funders/EA orgs

(probably too obvious) Diversifying demographics of staff working at funding organizations. This will come too from bottom-up approaches (community builders in LMICs could encourage EAs in LMICs to apply to jobs in these organizations).

In the short term, these orgs could reach out more actively to community builders in LMICs when they have open positions.

See more specific recommendations about hiring here.

Shoutouts to very inclusive processes: someone could spot hiring and recruiting processes that have been particularly successful at attracting diverse candidates and share recommendations to replicate them. For example, we have heard that the last LPP Summer fellowship in Oxford had very diverse demographics. Apparently, they s...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Existential risk mitigation: What I worry about when there are only bad options, published by MMMaas on December 19, 2022 on The Effective Altruism Forum.

(This is a Draft Amnesty Day draft. That means it’s not polished, it’s probably not up to my standards, the ideas are not thought out, and I haven’t checked everything. I was encouraged to post something).

(Written in my personal capacity, reflecting only my own, underdeveloped views).

(Commenting and feedback guidelines: I’m going with the default — please be nice. But constructive feedback is appreciated; please let me know what you think is wrong. Feedback on the structure of the argument is also appreciated)

My status: doubt. Shallow ethical speculation, including attempts to consider different ethical perspectives on these questions that are both closer to and further from my own.

If I had my way: great qualities for existential risk reduction options

We know what we would like the perfect response to an existential risk to look like. If we could wave a wand, it would be great to have some ideal strategy that manages to simultaneously be:

functionally ideal:[...]

effective (significantly reduces the risks if successful, ideally permanently),

reliable (high chance of success),

technically feasible,

politically viable,

low-cost;

safe (little to no downside risk -- ie graceful failure),

robust (effective, reliable, feasible, viable and safe across many possible future scenarios),

ethically ideal[...]

pluralistically ethical (no serious moral costs or rights violations entailed by intervention, under a wide variety of moral views),

impartial (everyone is saved by its success; no one bears disproportionate costs of implementing the strategy) / 'paretotopian' (everyone is left better off, or at least no one is made badly worse off);

widely accepted (everyone (?) agrees to the strategy's deployment, either in active practice (e.g. after open democratic deliberation or participation), passive practice (e.g. everyone has been notified or informed about the strategy), or at least in principle (we cannot come up with objections from any extant political or ethical positions, after extensive red-teaming)),

choice-preserving (does not lead to value lock-in and/or entail leaving a strong ethical fingerprint on the future)

etc, etc.

But it may be tragically likely that interventions that combine every single one of these traits are just not on the table. To be clear, I think many proposed strategies for reducing existential risk at least aim at hitting many or all of these criteria. But these won't be the only actions that will be pursued around extreme risks.What if the only feasible strategies to respond to existential risks--or the strategies that will most likely be pursued by other actors in response to existential risk--are all, to some extent, imperfect, flawed or 'bad'?

Three 'bad' options and their moral dilemmas

In particular, I worry about at least three (possible or likely) classes of strategies that could be considered in response to existential risks or global catastrophes: (1) non-universal escape hatches or partial shields; (2) unilateral high-risk solutions; (3) strongly politically or ethically partisan solutions.

All three plausibly constitute '(somewhat) bad' options. I don't want to say that these strategies should not be pursued (e.g. they may still be 'least-bad', given their likely alternatives; or 'acceptably bad', given an evaluation of the likely benefits versus costs). I also don't want to claim that we should not analyze these strategies (especially if they are likely to be adopted by some people in the world).

But I do believe that all create moral dilemmas or tradeoffs that I am uncomfortable with--and risky 'failures' that could be entailed by taking one or another view on whether to use them....

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Towards Hodge-podge Alignment, published by Cleo Nardo on December 19, 2022 on The AI Alignment Forum.

ABSTRACT: What could AI alignment look like if we had 6000+ full-time researchers and software developers? I've spent a few days thinking about that question, and this report is my first-draft attempt to answer it. This report sketches a highly-scalable research agenda: studying the hodge-podge assemblage of various "alignment primitives". Please let me know in the comments which ideas are novel and useful (if any), and which aren't.

Introduction

What is hodge-podge alignment?

I think that in the Technical AI Safety community, the following beliefs are common:

The adequate solution to the alignment problem (if it exists) will look like one thing.

That one thing might be quantilization, or debate, or ELK, or impact regularisation, or something else entirely.

If we received a radio message from a friendly alien civilisation that solved alignment, it would say "hey humans, you should use this particular method to solve alignment".

Maybe those beliefs are correct — but in this report, I'll explore the alternative hypothesis. What if the only strategy that achieves P(doom)<90% is throwing every half-baked protocol we have at an AGI? Let's call this strategy "hodge-podge alignment".

Note that hodge-podge alignment differs from research diversification. The TAIS community is unanimously in favour of research diversification — i.e. they think we should explore many different research agendas. But their justification tends to be "we can't tell beforehand which research agenda will tell us the One Thing", rather than "we'll need to hodge-podge together the methods produced by each research agenda".

Give me an example.

There are dozens of "alignment primitives", but consider these four:

Boxing: Run the AI on an air-gapped server with a narrow bandwidth channel to a human interlocutor. (See here.)

Quantilization: The AI, rather than maximising the quality of actions, randomly selects from the top quantile of actions. (See here.)

Impact regularisation: Penalise the AI for actions which significantly impact the environment. (See here.)

Olah-style interpretability: Convert a neural network N into a smaller computational graph Olah(N) whose nodes are annotated with human-legible concepts. (See here.)

We can "hodge-podge" together these primitives to give an assemblage:

Assemblage: The AI randomly samples from the top 1% of actions a∈A as ranked by U+I, where U is our best proxy for human values, I is an impact regularisation, and A is a 1KB plaintext channel. Each neural network N in the AI is compressed into Olah(N) and inspected by a human throughout training, evaluation, and deployment.

Will this work? I'm not sure.

I suspect that the assemblage would result in a safer AI than any of the four primitives used alone. But the goal of AI alignment is not to build a safe AI, but rather to build a safe and competent AI. (We already know how to build safe incompetent AI!) So the important question is: does the assemblage score higher than the primitives on the safety-competency trade-off? That's an open question.

Key Stages

My tentative proposal consists of the following key stages:

Exhaustively enumerate alignment primitives.

Taxonomise primitives by their type-signature, documentation, and implementation.

Enumerate potential alignment assemblages.

Study the assemblages theoretically and experimentally.

Deploy any assemblage if it's adequate.

1. Exhaustively enumerate alignment primitives.

There are a few broad overviews of different alignment agendas.

Thomas Larsen's (My understanding of) What Everyone in Technical Alignment is Doing and Why

Logan Zoellner's Various Alignment Strategies (and how likely they are to work)

Holden Karnofsky's How might we align transformative AI if it’s ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Proposal — change the name of EA Global, published by DMMF on December 19, 2022 on The Effective Altruism Forum.

(to something along the lines of: EA Careers Conference or EA Direct Work Conference)

Earlier this year, Scott Alexander made a thread titled Open EA Global that caused a big stir.

My summary of events is that people interpreted EA Global to represent different things and have different aims, leading to many to have different expectations as to who should attend EA Global. This ambiguity led to many people feeling hurt when they (or others they care about) were rejected from EAG.

The organizers of EAG seemingly had done a bad job explaining what EAG is and had incorrect/out of date/misleading information about EAG on various online platforms/communications.

To Eli Nathan’s credit (the CEA lead for EAG), he provided a lot of helpful context in the ensuing thread and updated lots of EAG related communications to be more clear.

The EAG website now states:

“EA Global is designed for people who have a solid understanding of the main concepts of effective altruism, and who are making decisions and taking significant actions based on them. EA Global conferences are not the only events for people interested in effective altruism!”

Still, several months after Scott’s thread, I still feel there has not been an adequate resolution to this.

In Eli’s response to Scott, he provided:

"EAG is primarily a networking event."

“I wanted to clarify: EAG exists to make the world a better place, rather than serve the EA community or make EAs happy”

“The conference is called "EA Global" and is universally billed as the place where EAs meet one another, learn more about the movement, and have a good time together.” It’s possible we should rename the event, and I agree this confusion and reputation is problematic, but I would like to clarify that we don’t define the event like this anywhere (though perhaps we used to in previous years). It’s now explicitly described as an event with a high bar for highly engaged EAs (see here). "

Even though there is now much accurate information about EAG online, I still find it problematic.

As stated above, the primary description of who EAG is for is: “EA Global is designed for people who have a solid understanding of the main concepts of effective altruism, and who are making decisions and taking significant actions based on them.”

I think EAG is still not meant for the majority of people who fit this description.

Those who earn to give, those who donate significant sums of their annual income to charity, those who decided to become vegans, those who spend large amounts of time reading EA content/attending Ea meet ups etc. all meet this description yet are not ideal candidates for EAG.

If EAG is intended to be a conference for people doing direct work to network and to help motivate others (particularly, students/early career workers) to do direct work, then I think the conference should be clear about this.

I feel this way for two reasons:

Despite the updated branding/communications, people are still going to feel excluded and hurt.

Because there is already a conference titled EA Global, people are less likely to create new Effective Altruism conferences for other purposes like being the place where EAs meet one another, learn more about the movement, and have a good time together etc.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Staff members’ personal donations for giving season 2022, published by GiveWell on December 19, 2022 on The Effective Altruism Forum.

Author: Isabel Arjmand, GiveWell Special Projects Officer

For this post, a number of GiveWell staff members volunteered to share the thinking behind their personal donations for the year. We’ve published similar posts in previous years.[1] Staff are listed alphabetically by first name.

You can click the links in the table of contents on the left to jump to a staff member’s entry.

Andrew Martin (Senior Research Associate)

I continue to be impressed by the care and thoughtfulness I see from my colleagues in making grant allocation decisions. Seeing and participating in this work informs my decision to give all of my donation this year to the All Grants Fund. In addition to GiveWell's Top Charities, I'm excited to be able to support other highly cost-effective programs through the All Grants Fund, as highlighted in this blog post.

Audrey Cooper (Philanthropy Advisor)

We plan to donate to GiveWell's Top Charities Fund again this year. Each of the top charity programs has substantial funding needs, such that they could reach more people and save more lives if they receive more donations. I'm excited to help these programs close the gap.

We also plan to continue our support of the International Refugee Assistance Project and criminal justice organizations. Throughout the year, we also make smaller donations to local causes (such as services for people experiencing homelessness, community gardens, etc.) as well as gifts in honor of friends to their charities of choice.

Elie Hassenfeld (Co-Founder and Chief Executive Officer)

This year, my family is planning to give 80% of our annual donation to GiveWell's All Grants Funds and 20% to GiveDirectly.

We're giving to GiveWell's All Grants Fund because it gives GiveWell the most flexibility to direct funds where we (GiveWell staff) think they will do the most good. This may mean supporting programs at Top Charities, but it could mean funding newer organizations, research, or more speculative opportunities that are high (expected) impact.

Our decision to give to GiveDirectly is less straightforward. Based on GiveWell's cost-effectiveness models, the funds my family is giving to GiveDirectly would do more good if given elsewhere (roughly speaking, GiveWell's best estimate is that funds to top charities and the All Grants Fund do about ten times as much good in expectation).

We're giving 20% to GiveDirectly for two reasons:

When I talk to people who aren't already familiar with GiveWell's work, I often reference GiveDirectly. Many people aren't aware of the vast income disparities between high-income and low-income countries. I talk about GiveDirectly because (a) it's very simple and easy to explain and (b) years ago, I visited GiveDirectly's program in Kenya, so I'm able to speak personally and specifically about people who benefited from GiveDirectly's work. For example, I often tell people about a specific family I met who received ~$1,000 from GiveDirectly. Like many other families, they chose to use part of their cash transfer to replace their thatched roof with a metal one. Before receiving these funds, when it rained in the middle of the night, the family (if I recall correctly, a mother and two children) would have to move out to a neighbor's house that was ~60 feet away to stay dry. They'd come back the next day to find their belongings soaked.

Or, when I talk to people with an interest in evaluation, I tell them that, when I visited Kenya, GiveDirectly enabled me to randomly select households to visit. On any site visit, donors should expect that organizations are aiming to shape a compelling narrative of their impact, so I loved that GiveDirectly helped me see a more representative picture ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Results for a survey of tool use and workflows in alignment research, published by jacquesthibs on December 19, 2022 on LessWrong.

In March 22nd, 2022, we released a survey with an accompanying post for the purpose of getting more insight into what tools we could build to augment alignment researchers and accelerate alignment research. Since then, we’ve also released a dataset, a manuscript (LW post), and the (relevant) Simulators post was released.

This post is an overview of the survey results and leans towards being exhaustive. Feel free to skim. In our opinion, the most interesting questions are 6, 11, 12, and 13.

We hope that this write-up of the survey results helps people who want to contribute to this type of work.

Motivation for this work

We are looking to build tools now rather than later because it allows us to learn what’s useful before we have access to even more powerful models. Once GPT-(N-1) arrives, we want to be able to use it to generate extremely high-quality alignment work right out of the gate. This work involves both augmenting alignment researchers and using AI to generate alignment research. Both of these approaches fall under the “accelerating alignment” umbrella.

Ideally, we want these kinds of tools to be used disproportionately for alignment work in the first six months of GPT-(N-1)’s release. We hope that the tools are useful before that time but, at the very least, we hope to have pre-existing code for interfaces, a data pipeline, and engineers already set to hit the ground running.

Using AI to help improve alignment is not a new idea. From my understanding, this is a significant part of Paul Christiano’s agenda and a significant part of his optimism about AI alignment.

Of course, automating alignment is also OpenAI’s main proposal and Jan Leike has been talking about it for a while.

Ought has also pioneered doing work in this direction and I’m excited to see them devote more attention to building tools even more highly relevant to accelerating alignment research.

Finally, as we said in the survey announcement post:

In the long run, we’re interested in creating seriously empowering tools that fall under categorizations like STEM AI, Microscope AI, superhuman personal assistant AI, or plainly Oracle AI. These early tools are oriented towards more proof-of-concept work, but still aim to be immediately helpful to alignment researchers. Our prior that this is a promising direction is informed in part by our own very fruitful and interesting experiences using language models as writing and brainstorming aids.

One central danger of tools with the ability to increase research productivity is dual-use for capabilities research. Consequently, we’re planning to ensure that these tools will be specifically tailored to the AI Safety community and not to other scientific fields. We do not intend to publish the specifics methods we use to create these tools.

Caveat before we get started

As mentioned Logan’s post on Language Models Tools for Alignment Research (and many others we’ve talked to), could this work be repurposed for capabilities work? If made public with flashy demos, it’s quite likely. That’s why we’ll be keeping most of this project private for alignment research only (the alignment text dataset is public).

Survey Results

We received 22 responses in total and some responses were optional so not all questions received responses from everyone. Of course we would have preferred even more responses, but this will have to do for now. We expect to iterate on tools with alignment researchers, so hopefully we get a lot of insights through user interviews of actual products/tools.

If you are interested in answering some of the questions in the survey (all questions are optional!), here’s the link. Leaving comments on this post would also be appre...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The ‘Old AI’: Lessons for AI governance from early electricity regulation, published by Sam Clarke on December 19, 2022 on The Effective Altruism Forum.

Note: neither author has a background in history, so please take this with a lot of salt. Sam thinks this is more likely than not to contain an important error. This was written in April 2022 and we’re posting now as a draft, because the alternative is to never post.

Like electricity, AI is argued to be a general purpose technology, which will significantly shape the global economic, military and political landscapes, attracting considerable media attention and public concern. Also like electricity, AI technology has the property that whilst some use cases are innocuous, others pose varying risks of harm.

Due to these similarities, one might wonder if there are any lessons for AI governance today to be learned from the development of early electricity regulation and standards. We looked into this question for about two weeks, focusing on early electrification in the US from the late 1800s to the early 1900s, and on the UK’s nationalisation of the electricity sector during the 20th century.

This post identifies and examines lessons we found particularly interesting and relevant to AI governance. We imagine many of them will be fairly obvious to many readers, but we found that having concrete historical examples was helpful for understanding the lessons in more depth and grounding them in some empirical evidence.In brief, the lessons we found interesting and relevant are:

Accidents can galvanise regulation

People co-opt accidents for their own (policy) agendas (to various degrees of success)

Technology experts can have significant influence in dictating the direction of early standards and regulation

Technology regulation is not inherently anti-innovation

The optimal amount and shape of regulation can change as a technology matures

The need for interoperability of electrical devices presented a window of opportunity for setting global standards

The development of safety regulation can be driven by unexpected stakeholders

Pervasive monitoring and hard constraints on individual consumption of technology is an existing and already used governance tool

There’s a lot more that could be investigated here—if you’re interested in this topic, and especially if you’re a historian interested in electricity or the early development of technology standards and regulations, we think there are a number of threads of inquiry that could be worth picking up.

Accidents can galvanise regulation

In the early days of electrification, there were several high-profile accidents resulting in deaths and economic damage:

A lineman being electrocuted in a tangle of overhead electrical wires, above a busy lunchtime crowd in Manhattan, which included many influential New York aldermen.

There were a number of other deaths for similar reasons, which occurred somewhat less publicly and so were less influential but still important.

Pearl Street Station—the first commercial central power plant in the United States—burned down in 1890.

The 1888 blizzard in New York City tore down many power lines and led to a power blackout.

Despite electric companies like Western Union and US Illuminating Company protesting regulation with court injunctions, [Hargadon & Doglas 2021] these accidents spurred government and corporate regulation around electrical safety, including:

Various governments began to require high voltage electrical lines to be buried underground, one of the first (if not the first) governmental regulations on electricity to be introduced [Stross 2007].

Thomson-Houston electric company developed lighting arrestors for power lines and blowout switches to shut down systems in case of a power surge [Davis 2012].

Concerned about risks of installing AC e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Next Level Seinfeld, published by Zvi on December 19, 2022 on LessWrong.

Thanks to Jordan Schneider gifting a last minute ticket for an amazing seat, serendipity led me to go to Friday night’s performance by Jerry Seinfeld at the Beacon Theater.

If I had to describe the show in one word, I’d say it was unsurprising. Jerry was Jerry. He had thoughts. Mostly he had complaints. They were all Very Seinfeld.

If I had two words I might say mildly amusing. Which was good enough for a worthwhile evening. Live performances are something special. Every time I’ve gone out to a comedy show, even if a bunch of it was kind of lame, I have been happy I came. The correct bar for worth watching is actually lower in person than at home.

The thought I couldn’t shake as I went home was, what GPT level was that on?

This first came to me during the opening act. His opening act was if anything too on the nose. Very Jerry. Much Seinfeld.

It felt very much like he had GPT-X put together a package of Standard Mildly Amusing Jokes and Perspectives, with the prompt that the performer was black, old, male, never married and the opening act for Jerry Seinfeld. Given the material, delivery was solid.

The question was, what was the X in GPT-X?

I am confident our current incarnation, which is GPT-3.5 or so, can’t do it.

What about GPT-4? Could go either way, if you could try out bits and select winners.

What about GPT-5? Yes, absolutely, if it is what I’d expect, that should work.

So I’d give that act a GPT-level rating of 4.25.

Jerry himself was on a higher level. Not as high as his peak. I’d give him a 5.

What about the best stand-up shows and specials?

I’ve seen 6s. I worry I may have never seen a 7.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Predictive Processing, Heterosexuality and Delusions of Grandeur, published by lsusr on December 17, 2022 on LessWrong.

Predictive processing is the theory that biological neurons minimize free energy. In this context, free energy isn't physical energy like the energy in your laptop battery. Instead, free energy is an informatic concept. It is useful to think about free energy as a single variable balancing prediction error and model complexity. Minimizing free energy balances minimizing prediction error against minimizing model complexity. You can also think about it as minimizing surprise.

Biological neurons have inputs and outputs. Each neuron receives inputs from one set of neurons and sends outputs to another set of neurons. One way to minimize prediction error is to fire right after receiving inputs, but an even better way to minimize prediction error is for a neuron to anticipate when its input neurons will fire and then fire along with them. Firing in-sync with its inputs produces zero prediction error instead of just a small prediction error.

It has been shown that predictive processing is asymptotically equivalent to backpropagation. Everything that can be computed with backpropagation can be computed via predictive processing and vice versa.

A Simple Clock

Suppose you take a 3-dimensional blob of neurons programmed to minimize local prediction error and you attach them to a sinusoidal wave generator. The neural net has no outputs—just this single input. At first the neurons close to the wave generator will lag behind the sinusoidal wave generator. But eventually they'll and sync up with it. Our neural net will have produced an internal representation of the world.

Now suppose you plug a sinusoidal wave generator into the left end of the neural net and a square wave generator into the right end of the neural net. The left end of the neural net will wave in time with the sinusoidal wave generator and the right end of the neural net will beat in time with the square wave generator. The middle of the net will find some smooth balance between the two. Plug more complicated inputs into our neural net and it will generate a more complicated representation of the world.

Simple Behavior

Consider a neural net with one input and one output. The output of the brain is connected to the input of a static noise generator and the static noise generator's output is connected to the brain's input. The brain is a free energy minimizer. Static noise has high free energy. The brain will thus tend to produce outputs that minimize the noise. In other words, the brain will figure out (if it can) how to turn off the static noise generator.

Scientists used this principle to train a blob of neurons to play pong. They connected a biological neural net to a game of pong and then blasted the net with static whenever the net failed to hit the ball. Result: the paddle intercepted the ball more often than random chance.

We now have all the ingredients we need to build a simple nervous system. Start with a net of neurons. Attach sensory inputs to the net. The net will create an internal model of the world. Whenever something bad happens (such as the organism suffering damage), blast the net with a short, intense burst of random noise. The organism will learn to avoid whatever causes the short, intense bursts of random noise. We have invented pain.

Cultivating Abstract Concepts

Some features of the environment can be hard-coded. You don't need to learn how to feel pain or feel warmth. All evolution has to do is invent the relevant sensors and then emit a burst of static (in the case of pain) or a less-noisy signal (in the case of warmth).

Other features cannot be hard-coded at all. Your brain contains no priors about whether whether Obanazawa is in the Yamagata Prefecture. Facts like that a...