Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why no major LLMs with memory?, published by Kaj Sotala on March 28, 2023 on LessWrong.

One thing that I'm slightly puzzled by is that an obvious improvement to LLMs would be adding some kind of long-term memory that would allow them to retain more information than fits their context window. Naively, I would imagine that even just throwing some recurrent neural net layers in there would be better than nothing?

But while I've seen LLM papers that talk about how they're multimodal or smarter than before, I don't recall seeing any widely-publicized model that would have extended the memory beyond the immediate context window, and that confuses me.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Nobody’s on the ball on AGI alignment, published by leopold on March 29, 2023 on The Effective Altruism Forum.

Far fewer people are working on it than you might think, and even the alignment research that is happening is very much not on track. (But it’s a solvable problem, if we get our act together.)

Observing from afar, it's easy to think there's an abundance of people working on AGI safety. Everyone on your timeline is fretting about AI risk, and it seems like there is a well-funded EA-industrial-complex that has elevated this to their main issue. Maybe you've even developed a slight distaste for it all—it reminds you a bit too much of the woke and FDA bureaucrats, and Eliezer seems pretty crazy to you.

That’s what I used to think too, a couple of years ago. Then I got to see things more up close. And here’s the thing: nobody’s actually on the friggin’ ball on this one!

There’s far fewer people working on it than you might think. There are plausibly 100,000 ML capabilities researchers in the world (30,000 attended ICML alone) vs. 300 alignment researchers in the world, a factor of ~300:1. The scalable alignment team at OpenAI has all of ~7 people.

Barely anyone is going for the throat of solving the core difficulties of scalable alignment. Many of the people who are working on alignment are doing blue-sky theory, pretty disconnected from actual ML models. Most of the rest are doing work that’s vaguely related, hoping it will somehow be useful, or working on techniques that might work now but predictably fail to work for superhuman systems.

There’s no secret elite SEAL team coming to save the day. This is it. We’re not on track.

If timelines are short and we don’t get our act together, we’re in a lot of trouble. Scalable alignment—aligning superhuman AGI systems—is a real, unsolved problem. It’s quite simple: current alignment techniques rely on human supervision, but as models become superhuman, humans won’t be able to reliably supervise them.

But my pessimism on the current state of alignment research very much doesn’t mean I’m an Eliezer-style doomer. Quite the opposite, I’m optimistic. I think scalable alignment is a solvable problem—and it’s an ML problem, one we can do real science on as our models get more advanced. But we gotta stop fucking around. We need an effort that matches the gravity of the challenge.

Alignment is not on track

A recent post estimated that there were 300 full-time technical AI safety researchers (sounds plausible to me, if we’re counting generously). By contrast, there were 30,000 attendees at ICML in 2021, a single ML conference. It seems plausible that there are ≥100,000 researchers working on ML/AI in total. That’s a ratio of ~300:1, capabilities researchers:AGI safety researchers.

That ratio is a little better at the AGI labs: ~7 researchers on the scalable alignment team at OpenAI, vs. ~400 people at the company in total (and fewer researchers). But 7 alignment researchers is still, well, not that much, and those 7 also aren’t, like, OpenAI’s most legendary ML researchers. (Importantly, from my understanding, this isn’t OpenAI being evil or anything like that—OpenAI would love to hire more alignment researchers, but there just aren’t many great researchers out there focusing on this problem.)

But rather than the numbers, what made this really visceral to me is. actually looking at the research. There’s very little research where I feel like “great, this is getting at the core difficulties of the problem, and they have a plan for how we might actually solve it in <5 years.”

Let’s take a quick, stylized, incomplete tour of the research landscape.

Paul Christiano / Alignment Research Center (ARC).

Paul is the single most respected alignment researcher in most circles. He used to lead the OpenAI alignment team, and he has made usefu...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are there specific books that it might slightly help alignment to have on the internet?, published by AnnaSalamon on March 29, 2023 on LessWrong.

Books, and ideas, have occasionally changed specific human beings, and thereby history. (I think.)

I used to think it utterly implausible when people suggested that "AIs are our kids, we need to raise them right" or that e.g. having the right book written about (ethics/philosophy/decision theory/who knows) might directly impact an AI's worldview (after the AI reads it, in natural language) and thereby the future. But, while I still consider this fairly unlikely, it seems not-impossible to me today. Future LLMs could AFAICT have personalities/belief-like-things/temporary-unstable-values-like-things/etc. that're shaped by what's on the internet. And the LLMs' initial personalities/beliefs/values may then change the way they change themselves, or the way that social networks that include the LLMs help change the LLMs, if and when some LLMs self-modify toward more power.

So I have "what books or ideas might help?" in my shower-thoughts.

One could respond to this possibility by trying to write the right ethical treatises or train-of-thought interface or similar. More cheaply, one could respond to this by asking if there are books that've already been written that might be at least a little bit helpful, and whether those books are already freely available online and within the likely training corpuses of near-future LLMs, and if not, whether we can easily cause them to be.

Any thoughts on this? I'll stick my own in the comments. I'll be focusing mostly on "what existing books might it help to cause to be accessibly online, and are there cheap ways to get those books to be accessibly online?", but thoughts on other aspects of these questions are also most welcome.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Which parts of the existing internet are already likely to be in (GPT-5/other soon-to-be-trained LLMs)'s training corpus?, published by AnnaSalamon on March 29, 2023 on LessWrong.

In particular:

If a book is in easily accessible parts of the web, but is there as a pdf image of a scanned book rather than as text, is it likely to be in the training corpus of (GPT5/similar) in roughly the same way (with roughly the same effects) as if it were in easily accessible webpages?

How about if a book is on libgen as a pdf (illegally), but not on the easily accessible parts of the web?

How about if the ideas are in a video (e.g. a podcast); is (GPT5/similar) likely to process the text content of a video in roughly the same way if there's only a video, compared to if there's a video and a transcript?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: FLI open letter: Pause giant AI experiments, published by Zach Stein-Perlman on March 29, 2023 on The Effective Altruism Forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Run Posts By Orgs, published by Jeff Kaufman on March 29, 2023 on The Effective Altruism Forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Response to Tyler Cowen’s Existential risk, AI, and the inevitable turn in human history, published by Zvi on March 28, 2023 on LessWrong.

Predictions are hard, especially about the future. On this we can all agree.

Tyler Cowen offers a post worth reading in full in which he outlines his thinking about AI and what is likely to happen in the future. I see this as essentially the application of Stubborn Attachments and its radical agnosticism to the question of AI. I see the logic in applying this to short-term AI developments the same way I would apply it to almost all historic or current technological progress. But I would not apply it to AI that passes sufficient capabilities and intelligence thresholds, which I see as fundamentally different.

I also notice a kind of presumption that things in most scenarios will work out and that doom is dependent on particular ‘distant possibilities,’ that often have many logical dependencies or require a lot of things to individually go as predicted. Whereas I would say that those possibilities are not so distant or unlikely, but more importantly that the result is robust, that once the intelligence and optimization pressure that matters is no longer human that most of the outcomes are existentially bad by my values and that one can reject or ignore many or most of the detail assumptions and still see this.

My approach is, I’ll respond in-line to Tyler’s post, then there is a conclusion section will summarize the disagreements.

In several of my books and many of my talks, I take great care to spell out just how special recent times have been, for most Americans at least. For my entire life, and a bit more, there have been two essential features of the basic landscape:

1. American hegemony over much of the world, and relative physical safety for Americans.

2. An absence of truly radical technological change.

I notice I am still confused about ‘truly radical technological change’ when in my lifetime we went from rotary landline phones, no internet and almost no computers to a world in which most of what I and most people I know do all day involves their phones, internet and computers. How much of human history involves faster technological change than the last 50 years?

When I look at AI, however, I strongly agree that what we have experienced is not going to prepare us for what is coming, even in the most slow and incremental plausible futures that don’t involve any takeoffs or existential risks. AI will be a very different order of magnitude of speed, even if we otherwise stand still.

Unless you are very old, old enough to have taken in some of WWII, or were drafted into Korea or Vietnam, probably those features describe your entire life as well.

In other words, virtually all of us have been living in a bubble “outside of history.”

Now, circa 2023, at least one of those assumptions is going to unravel, namely #2. AI represents a truly major, transformational technological advance. Biomedicine might too, but for this post I’ll stick to the AI topic, as I wish to consider existential risk.

#1 might unravel soon as well, depending how Ukraine and Taiwan fare. It is fair to say we don’t know, nonetheless #1 also is under increasing strain.

The relative physical safety we enjoy, as I see it, mostly has nothing to do with American hegemony, and everything to do with other advances, and with the absurd trade-offs we have made in the name of physical safety, to the point of letting it ruin our ability to live life and our society’s ability to do things.

When there is an exception, as there recently was, we do not handle it well.

Have we already forgotten March of 2020? How many times in history has life undergone that rapid and huge a transformation? According to GPT-4, the answer is zero. It names The Black Death, Industrial Revolution ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Shallow Investigation: Bacterial Meningitis, published by peetyxk on March 28, 2023 on The Effective Altruism Forum.

This report consolidates a shallow investigation into Bacterial Meningitis - its effects and importance in global health, the current tractability and cost-effectiveness of leading interventions, and an evaluation of the overall promise of the cause. I estimate this report to be a result of about 60-70 hours of research and writing. This research was conducted as part of the Cause Innovation Bootcamp fellowship, with constant guidance from Dr. Akhil Bansal.

Summary:

Meningitis is an inflammation (swelling) of the protective membranes covering the brain and spinal cord. It is commonly associated with infections (e.g. bacterial meningitis, viral meningitis), but it can also have non-infectious causes. The most common symptoms include fever, headache, sensitivity to light, and neck stiffness; in most cases, meningitis is treatable by addressing the underlying cause e.g. treating the causative infection.

Bacterial meningitis is important from a global health perspective - it ranks 40th on the current list of diseases in terms of total DALYs lost.

4 strands of bacteria cause 50% of all meningitis-related deaths, namely meningococcus, pneumococcus, Haemophilus influenzae and group B streptococcus - all of which are vaccine-preventable.

GBS (Group B Streptococcus) ranks 6th on the list of causes leading to DALYs lost in the age-group 1-10.

Bacterial meningitis is heavily concentrated in the African Meningitis belt, consisting of regions in 26 countries stretching from Senegal in the West to Ethiopia in the East, and incidence is inversely related with socio-demographic index (SDI).

Bacterial meningitis does not seem to be neglected

Important steps seem to have been taken already to counter meningitis on a global scale; including WHO’s comprehensive report on “A Global Plan to Defeat Meningitis by 2030”.

While the important interventions seem tractable, they seem to be less neglected than a lot of other interventions in global health, reducing their counterfactual value.

Important interventions that could yield (relatively) high cost-effectiveness seem to be

Installing an integrated disease surveillance and response (IDSR) system for monitoring meningitis, and

Advocating for the acceleration of the GBS vaccine development.

Major uncertainties: The interventions are still ‘moderately’ promising; for ex. If someone is uniquely positioned to accelerate GBS vaccine trials/distribution, or complete broad educational initiatives about infant healthcare/precautions preventing neonatal transmission, this might on the margins be a promising thing to do. On another note, the counterfactual neglectedness is low primarily due to WHO’s commitments in its “Roadmap to defeating meningitis by 2030” - if not followed up/held, the counterfactual value of another route to addressing meningitis could increase significantly.

I. Importance

Meningitis is an inflammation (swelling) of the protective membranes covering the brain and spinal cord. It is commonly associated with infections e.g. bacterial meningitis, viral meningitis, but it can also have non-infectious causes. The most common symptoms include fever, headache, sensitivity to light and neck stiffness; in most cases, meningitis is treatable by addressing the underlying cause e.g. an infection.

Meningitis, depending on the specific pathogen (virus, bacteria, fungi etc.) is often communicable and usually transmitted through close contact. Meningitis can also be passed on by mothers to their infants, and in fact, bacterial meningitis in infants is most commonly caused by the Group B Streptococcus pathogen (GBS), passed down in the peripartum period ( thebirthing process). Since meningitis attacks the membranes of the spinal co...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Coaching Training Program Launch: build and upskill your practice with Tee Barnett Coaching Training (TBCT), published by Tee on March 28, 2023 on The Effective Altruism Forum.

Applications are now open to join the pilot cohort of Tee Barnett Coaching Training! You’d be a great fit for this if you’re a current or aspiring coach seeking to gain skill as practitioners and/or establish your practice.

You can signal interest here

The deadline to submit the first round of applications is April 3.

We encourage people to start the process earlier because cohort coaches will asked to join and also initiate the program on a rolling basis. This will mean increasingly limited availability of cohort spots.

Program Overview

Spirit of the Pilot

Philosophy of Approach

Intended Outcomes

More about the Training Infrastructure

Program Pricing

Meet the Current Cohort

Anticipated Questions

Apply

Program overview

This pilot program is a multi-component training infrastructure for developing your own practice as a skilled coach. You could think of it as a proto-accelerator or proto-incubator for solo practitioners, which also includes access to business architecture that is essential for building and maintaining a stand-alone practice.

The infrastructure targets key developmental bottlenecks of coaches, including early client acquisition, real-world practice, training and oversight by multiple senior coaches, co-created learning plans, reputational benefits, business architecture, and more.

The program is a key piece of a higher-level mission to boost the total number of skilled practitioners supporting people doing scalable good, including those in EA. Tee Barnett Coaching Training (TBCT) will include:

A tailored curriculum co-created with cohort coaches

Systematic and consistent client referrals for engaging in real-world skill building

1-to-1 coaching training calls with Tee

Regular cohort calls with other cohort coaches

Free access to office hours with Tee

Peer coaching training calls

Turnkey technical systems for building and expanding your practice (partnering with Anti-Entropy)

Other guidance and resources becoming a coach, setting up and/or enhancing your practice

Potentially also offering observation hours and guest senior coach interactive workshops

– Duration – the first cohort is expected to last about 6 months, though timelines may vary depending on the intended growth plans agreed upon with each individual cohort coach.

– Pricing – costs of the program are currently reduced for the first cohort, pegged at roughly the equivalent of two personal coaching sessions per month with Tee.

– Location – remote, though the option may open up for TBCT to sponsor travel and accommodations for periods of in-person coaching training. Apply Here

Spirit of the pilot

Concepts conveying the vibe of the program: early-stage pilot, co-creationary, integration & synthesis, complementarity, self-aligned

One-line summary: this is an early-stage coaching development program that provides infrastructural support for those in the process of establishing and enriching their coaching practice.

The working design of the training infrastructure is partly a function of the early-stage nature of the program, but also a portfolio of deliberate philosophical and methodological stances that encourage coaches to collaboratively sculpt their own learning process. For example, each cohort coach’s ‘curriculum' will be the end result of a collaborative selection process between Tee and the coach themselves, with both parties iterating upon chosen materials and methods that will culminate into a highly personalized experience within the program.

Cohort coaches can expect much that comes with the territory of participating in a pilot program. On the one hand, participating in the ‘founding cohort’ can be thrilling a...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What longtermist projects would you like to see implemented?, published by Buhl on March 28, 2023 on The Effective Altruism Forum.

My team (Rethink Priorities’ General Longtermism Team) is aiming to incubate 2-3 longtermist projects in 2023. I’m currently collecting a longlist of project ideas, which we’ll then research and evaluate, with the aim of kicking off the strongest projects (either via an internal pilot or collaboration with an external founder).

I’m interested in ideas for entrepreneurial or infrastructure projects (i.e., not research projects, though a project could be something like “create a new research institute focused on X”).

Some examples to give a sense of the type of ideas we’re interested in (without necessarily claiming that these specific ideas are particularly strong): An organization that lobbies for governments to install far UVC lights in government buildings; a third-party whistleblowing entity taking reports from leading AI labs; or a remote research institute for independent researchers. You can see a list of our existing ideas here.

I’ll begin reviewing the ideas on April 17, so ideas posted before then would be most helpful.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.