The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI#28: Watching and Waiting, published by Zvi on September 8, 2023 on LessWrong.

We are, as Tyler Cowen has noted, in a bit of a lull. Those of us ahead of the curve have gotten used to GPT-4 and Claude-2 and MidJourney. Functionality and integration are expanding, but on a relatively slow pace. Most people remain blissfully unaware, allowing me to try out new explanations on them tabula rosa, and many others say it was all hype. Which they will keep saying, until something forces them not to, most likely Gemini, although it is worth noting the skepticism I am seeing regarding Gemini in 2023 (only 25% for Google to have the best model by end of year) or even in 2024 (only 41% to happen even by end of next year.)

I see this as part of a pattern of continuing good news. While we have a long way to go and very much face impossible problems, the discourse and Overton windows and awareness and understanding of the real problems have continuously improved in the past half year. Alignment interest and funding is growing rapidly, in and out of the major labs. Mundane utility has also steadily improved, with benefits dwarfing costs, and the mundane harms so far proving much lighter than almost anyone expected from the techs available. Capabilities are advancing at a rapid and alarming pace, but less rapidly and less alarmingly than I expected.

This week's highlights include an update on the UK taskforce and an interview with Suleyman of Inflection AI.

We're on a roll. Let's keep it up.

Even if this week's mundane utility is of, shall we say, questionable utility.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. It's got its eye on you.

Language Models Don't Offer Mundane Utility. Google search ruined forever.

Deepfaketown and Botpocalypse Soon. I'll pass, thanks.

They Took Our Jobs. Better to not work in a biased way than not work at all?

Get Involved. Center for AI Policy and Rethink Priorities.

Introducing. Oh great, another competing subscription service.

UK Taskforce Update. Impressive team moving fast.

In Other AI News. AIs engage in deception, you say? Fooled me.

Quiet Speculations. Copyright law may be about to turn ugly.

The Quest for Sane Regulation. The full Schumer meeting list.

The Week in Audio. Suleyman on 80k, Altman, Schmidt and several others.

Rhetorical Innovation. Several more ways not to communicate.

No One Would Be So Stupid As To. Maximally autonomous DeepMind agents.

Aligning a Smarter Than Human Intelligence is Difficult. Easier to prove safety?

Twitter Community Notes Notes. Vitalik asks how it is so consistently good.

People Are Worried About AI Killing Everyone. Their worry level is slowly rising.

Other People Are Not As Worried About AI Killing Everyone. Tyler Cowen again.

The Lighter Side. Roon's got the beat.

Language Models Offer Mundane Utility

Do automatic chat moderation for Call of Duty. Given that the practical alternatives are that many games have zero chat and the others have chat filled with the most vile assembly of scum and villainy, I am less on the side of 'new dystopian hellscape' as much as 'what exactly is the better alternative here.'

Monitor your employees and customers.

Rowan Cheung: Meet the new AI Coffee Shop boss. It can track how productive baristas are and how much time customers spend in the shop. We're headed into wild times.

Fofr: This is horrible in so many ways.

It's not the tool, it is how you use it. Already some companies such as JPMorgan Chase use highly toxic dystopian monitoring tools, which lets them take to the next level. It seems highly useful to keep track of how long customers have been in the store, or whether they are repeat customers and how long they wait for orders. Tracking productivity in broad terms like orders filled is a case where too much precision and a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pilot Results: High-Impact Psychology/ Mental Health (HIPsy), published by Inga on September 8, 2023 on The Effective Altruism Forum.

This is a rough write-up of the results of all the HIPsy pilot activities, outcomes, and suggestions on what to do next.

Below, you can find links to created resources for those involved in psychology or mental health who aim to maximize their impact. Additionally, there are results from events and surveys highlighting which (other) resources might be most beneficial for enhancing their impact. You can learn more about the initial project and plan in this EA forum post.

Executive summary

Within this EAIF- funded project, I ran three surveys to determine the demand for network activities, piloted 5 events at EAGx's, and collaborated with others on highly requested materials. Below, you can find the results of these activities.

Main outcomes

Importance/general Demand:Roughly twice as many people expressed interest in mental health as a cause area (N = 164) compared to those interested in resources for individuals with a background in psychology or how to make an impact with various psychology-related topics (N = 76/77)

Neglectedness: The EA Psychology Research Initiative by Lucius Caviola, among others, appears to meet the infrastructural needs around general psychology-related impact regarding longtermist goals. Those infrastructural needs around helping people to have an impact in the mental health space do not yet seem to be covered, even though the work of the HLI paved the way for mental health as a cause area taken seriously within EA.

Mental Health-related activities most requested:Among the mental health-related network activities, the most requested ones are mentoring, meetups, workshops, as well as materials related to topics on how to make an impact in the mental health space. The piloted events (talks and workshops/meetups) at EAGxs demonstrated significant interest, evinced by the high number of participants and high perceived impact ratings.

Conclusions:

More mental health-related infrastructure building seems impactful. The space would benefit from more organized, targeted resources that enable people to make a more substantial impact in a shorter period of time.

I believe it would be a good idea to invest in follow-up funding based on the results of this initiative to establish cost-effective corresponding structures and to hire at least one project/community manager for a year. Based on the survey results, this person would build the pool of volunteers to offer and measure the impact of requested events, materials, and a mentoring service.

How can you help? If you'd like to fund follow-up activities, feel free to reach out to me at [email protected] you'd be interested in the project manager role, you can apply in <15min via this form. If you are interested in collaborating, or hosting network activities, or creating resources for people engaged in mental health as a volunteer, express your interest via this form (it takes 3 minutes).

Outcomes

Materials created

HIPsy website summarizing relevant resources has been created and updated. It mainly contains resources related to impact and mental health. It still needs to be spread within the EA community and cross-linked on the websites of partners (e.g., HLI)

In the making: New EA Mental Health cause area report to provide, elaborate o,n and compare avenues of impact in that space from the view of health care professionals (not finalized or published yet, so the link doesn't work yet) in cooperation with High Impact Medicine (HIMed), proofread by Happier Lives Institute (HLI)

I (Inga) reviewed the Career Path Profile for psychology on Probably Good and the Research Agenda "Psychology for Effectively Improving the Future" by the Effective Altruism Psychology Resea...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Recreating the caring drive, published by Catnee on September 8, 2023 on LessWrong.

TL;DR: This post is about value of recreating "caring drive" similar to some animals and why it might be useful for AI Alignment field in general. Finding and understanding the right combination of training data/loss function/architecture/etc that allows gradient descent to robustly find/create agents that will care about other agents with different goals could be very useful for understanding the bigger problem. While it's neither perfect nor universally present, if we can understand, replicate, and modify this behavior in AI systems, it could provide a hint to the alignment solution where the AGI "cares" for humans.

Disclaimers: I'm not saying that "we can raise AI like a child to make it friendly" or that "people are aligned to evolution". Both of these claims I find to be obvious errors. Also, I will write a lot about evolution, as some agentic entity, that "will do that or this", not because I think that it's agentic, but because it's easier to write this way. I think that GPT-4 have some form of world model, and will refer to it a couple of times.

Nature's Example of a "Caring Drive"

Certain animals, notably humans, display a strong urge to care for their offspring.

I think that part of one of the possible "alignment solutions" will look like the right set of training data + training loss that allow gradient to robustly find something like a "caring drive" that we can then study, recreate and repurpose for ourselves. And I think we have some rare examples of this in nature already. Some animals, especially humans, will kind-of-align themselves to their presumable offspring. They will want to make their life easier and better, to the best of their capabilities and knowledge. Not because they "aligned to evolution" and want to increase the frequency of their genes, but because of some strange internal drive created by evolution. The set of triggers tuned by evolution, activated by events associated with the birth will awake the mechanism.

It will re-aim the more powerful mother agent to be aligned to the less powerful baby agent, and it just so happens that their babies will give them the right cues and will be nearby when the mechanism will do its work. We will call the more powerful initial agent that changes its behavior and tries to protect and help its offspring "mother" and the less powerful and helpless agent "baby". Of course the mechanism isn't ideal, but it works well enough, even in the modern world, far outside of initial evolutionary environment. And I'm not talking about humans only, stray urban animals that live in our cities will still adapt their "caring procedures" to this completely new environment, without several rounds of evolutionary pressure. If we can understand how to make this mechanism for something like a "cat-level" AI, by finding it via gradient descend and then rebuild it from scratch, maybe we will gain some insides into the bigger problem.

The rare and complex nature of the caring drive in contrast to simpler drives like hunger or sleep.

What do I mean by "caring drive"? Animals, including humans, have a lot of competing motivations, "want drives", they want to eat, sleep, have sex, etc. It seems that the same applies to caring about babies. But it seems to be much more complicated set of behaviors. You need to:correctly identify your baby, track its position, protect it from outside dangers, protect it from itself, by predicting the actions of the baby in advance to stop it from certain injury, trying to understand its needs to correctly fulfill them, since you don't have direct access to its internal thoughts etc.Compared to "wanting to sleep if active too long" or "wanting to eat when blood sugar level is low" I would confidently say that it's ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Löbian Obstacle, And Why You Should Care, published by marc/er on September 7, 2023 on The AI Alignment Forum.

In 2013, Eliezer Yudkowsky and Marcello Herreshoff published Tiling Agents for Self-Modifying AI, and the Löbian Obstacle. It is worth comprehending because it:

Is a very well written paper.

Expresses an idea that is non-obvious, and still relevant to alignment today.

Provides insight into what Eliezer and Marcello thought was valuable to work on in the time preceding publication.

When I first read this paper, I terribly misunderstood it. Due to it not being particularly approachable material for someone not well-versed in logic, I was confidently wrong for at least one month. This post summarizes my understanding of what the Löbian Obstacle is (having re-read the paper,) and why I think it is still an important idea one decade after its publication.

An agent A1 occupies a fully-known, deterministic and closed environment. A1 has a goal G that is either satisfied or otherwise by an outcome, for which A1's preference is satisfaction. An action bi∈Acts0 performed by an agent created by A1, hereafter referred to as A0, must therefore satisfy the statement:

¯¯bi⇒A0⊩¯¯biG

Where ¯¯bi denotes the actual performance of bi, and ⊩ denotes cognitive belief in the succeeding statement. Even if A1 could verify by inspection of A0's design that ¯¯bi⇒A0⊩¯¯biG will hold, as in:

A1⊩A0(∀bi∈Acts0:¯¯bi□0⌈¯¯biG⌉)

Where □0⌈ϕ⌉ refers to a proof of ϕ from the axioms of A0; Axm0, this is unknowable, as it would require:

A1⊩∀bi:(□0⌈¯¯biG⌉(¯¯biG))

Which due to Löb's Theorem we know to be impossible: For this to be so it would need to be that Axm1 could prove that if some proof of ⌈ϕ(x)⌉ exists in Axm0, that ϕ(x) must be true.

The above was a brief paraphrasing of section two of the original paper, which contains many additional details and complete proofs. How the Löbian Obstacle relates to simulators is my current topic of research, and this section will make the case that this is an important component of designing safe simulators.

We should first consider that simulating an agent is not distinguishable from creating one, and that consequently the implications of creating dangerous agents should generalize to their simulation. Hubinger et al. (2023) have stated similar concerns, and provide a more detailed examination of the argument.

It is also crucial to understand that simulacra are not necessarily terminating, and may themselves use simulation as a heuristic for solving problems. This could result in a kind of hierarchy of simulacra. In advanced simulators capable of very complex simulations, we might expect a complex network of simulacra bound by acausal trade and 'complexity theft,' whereby one simulacra tries to obtain more simulation complexity as a form of resource acquisition or recursive self-improvement.

I expect this to happen. Lower complexity simulacra may still be more intelligent than their higher complexity counterparts, and as a simulator's simulacra count may grow exponentially, as does the likelihood that one simulacra attempts complexity theft.

If we want safe simulators, we need the subsequent, potentially abyssal simulacra hierarchy to be aligned all the way down. Without being able to thwart the Löbian Obstacle, I doubt a formal guarantee is attainable. If we could do so, we may only need to simulate one aligned tiling agent, for which we might settle with a high certainty informal guarantee of alignment if short on time. I outlined how I thought that could be done here, although I've advanced the theory considerably since and will post an updated write-up soon.

If we can't reliably thwart the Löbian Obstacle, we should consider alternatives:

Can we reliably attain high certainty informal guarantees of alignment for arbitrarily deep simulacra hierarchies...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A quick update from Nonlinear, published by KatWoods on September 7, 2023 on LessWrong.

One example of the evidence we're gathering

We are working hard on a point-by-point response to Ben's article, but wanted to provide a quick example of the sort of evidence we are preparing to share:

Her claim: "Alice claims she was sick with covid in a foreign country, with only the three Nonlinear cofounders around, but nobody in the house was willing to go out and get her vegan food, so she barely ate for 2 days."

The truth (see screenshots below):

There was vegan food in the house (oatmeal, quinoa, mixed nuts, prunes, peanuts, tomatoes, cereal, oranges) which we offered to cook for her.

We were picking up vegan food for her.

Months later, after our relationship deteriorated, she went around telling many people that we starved her. She included details we believe were strategically chosen to depict us in a maximally damaging light - what could be more abusive than refusing to care for a sick girl, alone in a foreign country? And if someone told you that, you'd probably believe them, because who would make something like that up?

Evidence

The screenshots below show Kat offering Alice the vegan food in the house (oatmeal, quinoa, cereal, etc), on the first day she was sick. Then, when she wasn't interested in us bringing/preparing those, I told her to ask Drew to go pick up food, and Drew said yes. Kat also left the house and went and grabbed mashed potatoes for her nearby.

See more screenshots here of Drew's conversations with her.

Initially, we heard she was telling people that she "didn't eat for days," but she seems to have adjusted her claim to "barely ate" for "2 days".

It's important to note that Alice didn't lie about something small and unimportant. She accused of us a deeply unethical act - the kind that most people would hear and instantly think you must be a horrible human - and was caught lying.

We believe many people in EA heard this lie and updated unfavorably towards us. A single false rumor like this can unfairly damage someone's ability to do good, and this is just one among many she told.

We have job contracts, interview recordings, receipts, chat histories, and more, which we are working full-time on preparing.

This claim was a few sentences in Ben's article but took us hours to refute because we had to track down all of the conversations, make them readable, add context, anonymize people, check our facts, and write up an explanation that was rigorous and clear. Ben's article is over 10,000 words and we're working as fast as we can to respond to every point he made.

Again, we are not asking for the community to believe us unconditionally. We want to show everybody all of the evidence and also take responsibility for the mistakes we made.

We're just asking that you not overupdate on hearing just one side, and keep an open mind for the evidence we'll be sharing as soon as we can.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Whose actions are you thankful for?, published by Nathan Young on September 7, 2023 on The Effective Altruism Forum.

I think the other side of criticism is support.

What have you been thankful for in the last 3 months.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MHFC Fall Grants Round, published by wtroy on September 7, 2023 on The Effective Altruism Forum.

The Mental Health Funding Circle is holding our fall grants round!

We are a group of funders seeking to fund the most impactful mental health projects, and we very much encourage you to apply.

Our scope is quite wide, and we would consider many projects related to the cause of mental health. In the past we have funded:

Meta research on mental health giving priorities

Targeted research on intervention effectiveness and data on LMIC mental health

Effective global mental health interventions such as task-shifting, stepped care or self-help guides

Mental health for the EA community

Applications are due on October 1st, and final decisions will be made early-mid November.

For more information on the MHFC, visit our website.

For a list of previous grants, see our Updates page.

To apply, complete this application by October 1st.

The Mental Health Funding Circle is an Impactful Grantmaking funding circle, a project of Charity Entrepreneurship.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Feedback-loops, Deliberate Practice, and Transfer Learning, published by jacobjacob on September 7, 2023 on LessWrong.

[...insert introduction here later...]

Was there a particular moment, incident, or insight, that caused you to start your current venture into "feedbackloop rationality" [to be substituted with better name later]?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: CEARCH's Cause Exploration Contest: Awards, published by Joel Tan (CEARCH) on September 7, 2023 on The Effective Altruism Forum.

CEARCH ran our Cause Exploration Contest over the month of July, as part of our search for (a) potentially impactful causes as well as (b) useful methodologies to search for new causes going forward. We would like to thank everyone in the EA and broader philanthropic community for participating.

Winning Entries

We are pleased to announce the following winning entries:

In the category of promising cause areas: Bean soaking, submitted by Nick Laing of OneDay Health. In summary, persuading citizens in sub-Saharan Africa to soak before cooking them (and thus saving on fuel use) may have health, economic and environmental benefits; however, there are some outstanding uncertainties over tractability and why soaking is not already common practice.

In the category of useful search methodologies: Brainstorming for solutions that may not have the most impact in the context of solving a single problem, but which may have significant overall impact given the benefits it brings across multiple cause areas; this was submitted by Jeroen De Ryck.

The prizes are USD 300 and USD 700 for the cause and search methodology categories respectively. We will be getting in touch with the winners to send them their winnings, though we are of course happy to donate to the charities of their choice if they so prefer.

Honourable Mentions

We would also light to highlight the following entries that stood out. In the category of promising causes:

Modern slavery, submitted by Sam Hilton.

Alexithymia, submitted by Bolek Kerous.

And in the category of useful search methodologies:

A list of seven methods generally focused on taking different moral, political, epistemic and metaphysical perspectives (e.g. consulting the perspective of preference satisfaction; prioritizing causes systematically overlooked by human biases; copying ethical pioneers; consulting non-standard cosmology; consulting different political values; considering ideas that have gone out of fashion; and researching utopia building); this was submitted by David Mears, with input from Amber Dawn Ace.

Consulting J-PAL's existing list of RCTed interventions, with the idea being that at lower levels of granularity, we can focus on very targeted interventions that may be very cost-effective but not generally applicable; this was submitted by Sophia Moss.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why we didn't get a malaria vaccine sooner, and what we can do better next time, published by salonium on September 7, 2023 on The Effective Altruism Forum.

This is a long (>9000 word) essay written by myself (Saloni Dattani), Rachel Glennerster and Siddhartha Haria for Works in Progress.

Over half a million people die from malaria each year, but it took 141 years to develop a vaccine for it.

One fundamental reason for this was the scientific complexity of the pathogen - malaria is caused by a parasite, not a virus or bacteria. But another, repeated obstacle was a lack of financial incentive and urgency.

In this piece, which includes a lot of data and charts, we tell the story of how the malaria vaccine was developed, why the financial market for the vaccine was missing, and how it could have been sped up with smarter incentives and market mechanisms, like Advance Market Commitments.

About the authors:

Saloni Dattani - I'm a researcher on global health at Our World in Data and a founding editor of Works in Progress

Rachel Glennerster is associate professor of Economics at the University of Chicago. She was previously chief economist at the UK Foreign Commonwealth and Development Office and the Department for International Development and a key figure behind 'Deworm the World'.

Siddhartha Haria is policy lead at the Development Innovation Lab.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org