Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: They Don’t Know About Second Booster, published by Zvi on March 31, 2022 on LessWrong.

The FDA went ahead and did it, approving a fourth vaccine shot for those age 50 or older. This is a reference post for those wondering if it makes sense to get that second booster.

The FDA approved the second booster by bypassing some of the normal procedures. Usual never-amused suspects are not amused.

What do those meetings and ‘public discussions’ do?

Delay.

Get picky about lots of little details.

List all the reasons not to do something.

Voice objections for all to see.

Show lots of concern over trivial risks.

The Official Public Health Position is that doing all this Enhances Trust In The Process.

They are incorrect. And here is where their idea of ‘trust’ has gotten us.

Are they working hard on earning our trust back? Not exactly.

Their ‘scientific rationale’ is exactly what you think it is.

None of that means the standard drug process is necessarily bad or worthless. The process is very good at sniffing out any little potentially wrong thing or reason not to do something, and hence good at avoiding false positives. Which is sometimes what you want.

If your goal is to do a cost-benefit analysis to decide what to do, then turning things over to people who think costs and benefits do not belong in the same magisterium, and allowing them to act Deeply Concerned about things, does not seem like The Way.

The process is hopelessly biased towards delay, inaction and risk aversion. It makes sense to bypass it.

Nor does it seem like The Way to get Enhanced Trust. Yes, a few people will say ‘look, you did not hold the Official Meetings’ but actual regular people do not care. If anything, those arguments are crowding out arguments regular people would care about more.

For those shouting from the rooftops that ‘they’ would have us take infinite boosters forever, this is vindication of a sort. Yet it remains not even approved for others, let alone required.

There are good arguments against the second booster. That is fine. We can let individuals make their own choices.

Review: Should You Get a First Booster?

My answer to this continues to be yes.

The original design of the vaccine, of two shots in rapid succession, makes sense if you are trying to quickly test and deploy a vaccine in the middle of a pandemic. Pfizer and Moderna absolutely did the right thing to test that protocol given the situation.

However, there is lots of evidence that spacing the two shots so close together hurts long term immunity. Many vaccines involve much longer times between shots. In the UK, there were debates worrying that some people might not understand the importance of waiting multiple months between shots one and two, because that would make the vaccine less effective. We really, really should have done First Doses First.

Tests reliably show the first booster getting antibody response to levels well above those from the first two shots.

While the full effect of the booster fades over time, there seems to be a persistent and substantial effect on one’s permanent level of immunity.

Thus, even if the short term side effects for you are relatively unpleasant and you are young and in good health, I would get the first booster shot.

If you know you have had Covid-19, that likely functions in many ways as if it were a booster, at which point boosting or not boosting would be a small mistake. I would still boost, and definitely still boost if I was at high risk, but it is no longer obvious.

Should You Get a Second Booster?

The more important question, of course, is: Should you get a second booster?

I want to emphasize that my current viewpoint is that both decisions here are at worst small mistakes.

If you do get a second booster, the costs are minimal, beyond the short term side effects of being knocked on your ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Link] Training Compute-Optimal Large Language Models, published by nostalgebraist on March 31, 2022 on The AI Alignment Forum.

New LM scaling paper from DeepMind (abs, pdf).

Abstract (my emphasis):

We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are significantly undertrained, a consequence of the recent focus on scaling language models whilst keeping the amount of training data constant. By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens, we find that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled. We test this hypothesis by training a predicted compute-optimal model, Chinchilla, that uses the same compute budget as Gopher but with 70B parameters and 4× more more data. Chinchilla uniformly and significantly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B) on a large range of downstream evaluation tasks.

This also means that Chinchilla uses substantially less compute for fine-tuning and inference, greatly facilitating downstream usage. As a highlight, Chinchilla reaches a state-of-the-art average accuracy of 67.5% on the MMLU benchmark, greater than a 7% improvement over Gopher.

Brief comments on my blog here.

Presumably has implications for Bio Anchors?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Launching the INFER Forecasting Tournament for EA uni groups, published by hannah wing-yee on March 31, 2022 on The Effective Altruism Forum.

This tournament is co-sponsored by Cambridge EA (coordinated by Hannah Erlebach) and UChicago EA (coordinated by Henry Josephson). Thanks to Henry Tolchard for clarifying details about the tournament, and to Will Aldred for feedback on this post.

INFER, a forecasting program funded by a grant from Open Philanthropy to generate valuable signals for U.S. Government policymakers, is hosting an intercollegiate forecasting tournament for EA groups. Any EA group affiliated with a university is welcome to participate in the tournament. The top three individual forecasters will be offered paid positions on the Pro Forecaster Program.

The tournament is scheduled to run April 1 - July 31, 2022.

For full details, see INFER’s tournament page.

How to sign up

You can also find the full instructions here.

Getting started should take no more than a couple of minutes; it requires individuals to create an account, and a designated team host to create a team by submitting the usernames of team members.

(For all team members) How to sign up

Sign up at. You will be asked to choose a username.

Let your team host know your username.

Start practising on any of the forecasting questions currently on the site; during the tournament, it will be specified which of the questions qualify for tournament scoring.

You will be submitting forecasts individually for these questions, and the forecasts made by your team members will then be aggregated into a team forecast for scoring.

(For the team host) How to create your team

Click the Create a Team link under My Team.

Enter a team name (which clearly indicates which university group you’re from) and the usernames of your team members. You can always add more teammates later.

Submit your request, and an Admin will accept it within 24 hrs.

(For all team members) How to interact with your team

You will be able to interact with members of your team in different areas of the site, for example:

Go to My Team on the top navigation to see other users on your team, their activity, and start general discussions.

Within any question’s page (below the forecasting interface), go to the My Team tab where you can start Team Discussions which only members of your team can see, and view Current Forecasts and Forecast History.

From the Leaderboards in the top navigation bar, you can see rankings of all the teams on INFER (including those not part of the EA tournament), which are updated anytime a question is scored. Note that because INFER’s team leaderboard ranking takes into account all scored questions – not just the questions scored for the EA tournament – INFER will be calculating EA team rankings separately.

Why forecasting?

Forecasting is currently in 80,000 Hours’ list of ten top-recommended career paths (as of date of publishing). You can read their review of forecasting and related research and implementation here.

Quantitative forecasting can substantially improve our ability to predict the future, as compared with subjective judgement calls. Better predictions about important future events can lead to better decisions, especially in the complex, high-stakes situations in which governments and other institutions frequently find themselves. Better foresight could also maybe improve our ability to solve some of the world’s most pressing problems.

What makes forecasting particularly accessible is that:

Being an exceptional forecaster (‘superforecaster’) does not require exceptional prerequisites, and

There are reliable ways to improve forecasting accuracy, such as through calibration training and regular practice.

The best forecasters come from a wide range of backgrounds, often from fields which aren’t related to predict...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I feel anxious that there is all this money around. Let's talk about it, published by Nathan Young on March 31, 2022 on The Effective Altruism Forum.

Let's talk about the sharp increase in EA funding. I feel anxious about it. I don't think most of my feelings here are rational, but they do affect my actions, so it would be irrational not to take them seriously.

Some thoughts (roughly ordered from general to personal):

More money is good. It means we have more resources to work on the world's most pressing problems. Yay

I have seen little evidence that FTX Future Fund (FFF) or EA Infrastructure Fund (EAIF) have lowered their standards for mainline grants

Several people have gotten in touch with me about regranting money and it seems it would be easier for the median EA to get say $10,000 in funding than it was a year ago. It's possible that this change is more to do with me than FFF though. And also, perhaps this is a good thing. I think it's positive EV to help well aligned people with finance to help them work on important projects faster.

I'm really really really really grateful to EA Billionaries (in this case Sam Bankman-Fried) for working hard, taking risks and giving away money. I want to both express thanks and also say that working out how do deal with money as a community is important, reasonable work

The joke "sign up to FTX Funds, I guess" is probably a bad meme. eg "I'm thinking of trying to learn to rollerblade" "sign up to FTX Future Fund?" suggests that FFF has a very low standard for entry. It is a good joke, but I don't think a helpful one.

Perhaps my anxiety is that I feel like an imposter around large amounts of money - "you can't fund my project, I'm just me". As Aveek said on twitter, sometimes I feel "I wouldn't trust the funding strategy of any organisation that is willing to fund me". This is false, but sometimes I believe it, maybe you do to?

Perhaps this anxiety is that I worry about EA being seen to be full of grifters. Like the person who is the wealthiest on their street, I don't want bad press from people saying "EAs are spendthrift". I don't think we are, but there is a part of the movement that does get paid more and spend more than many charity workers. I don't know what to do about this.

My inner Stefan Shubert says "EA is doing well and this is a good thing, don't be so hard on yourself" and I do think there is a chance that this is unnecessary pessimism

I talk about it on twitter here. I don't mind that I did this, but I think the forum is generally a better place to have the discussion than twitter. Happy to take criticism

I have hang-ups about money in general. For several years after university I lived on about $12k a year (which is low by UK standards, though high by world ones). It's pretty surreal to be able to even consider applying for say 5x this as a salary. It's like going to a fancy restaurant for the first time ("the waiters bring the food to the table?") I just can't shake how surreal this all is.

I think it's good to surface and discuss this but I'm open to the idea that it isn't and will blow over.

I've made a Polis poll here if you want to see how different views cluster

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New forum feature: preview & embed Our World in Data charts, published by EdMathieu on March 31, 2022 on The Effective Altruism Forum.

You can now integrate Our World in Data charts much more easily into forum posts. These new features work with any of our interactive charts – their URLs look like this:

Previews on hover

If you add a link to an OWID chart in your text, a preview window will automatically appear when readers hover over it. For example, here's the yearly number of deaths from malaria by age since 1990.

Embedded charts

If you want to make a chart visible directly in your post, you can simply paste the URL and an embedded version of the chart should automatically appear. Here, for example, I'm pasting this:

to show this:

Note that when you make changes to an interactive chart on OWID (e.g. choosing a time window, selecting countries, switching to a different scale), your configuration is saved in the URL. Starting from this chart of primary energy from nuclear, I can make a line chart for G7 countries, and copy the resulting URL from my browser:

which I can paste here to embed this:

Many thanks to Adam Binks for coming up with this idea at EAGx Oxford!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What are great marketing ideas to encourage pre-orders of What We Owe The Future?, published by abierohrig on March 30, 2022 on The Effective Altruism Forum.

What We Owe The Future, Will MacAskill's upcoming book on longtermism, is available for pre-order!

Pre-orders are especially helpful for the book's eventual reach since (a) early pre-orders cause booksellers to order more copies of the book and market it more prominently, and (b) pre-orders combine with first-week sales, making the book more likely to reach bestsellers lists upon launch. In mid-June, 8 weeks before the launch of the book on August 16, we'll begin a pre-order campaign to the general public.

I'd love to hear ideas on how to (a) encourage pre-orders within the EA community in the next couple of months, and (b) how to encourage pre-orders from the general public, especially to young people who might switch to high-impact careers.

Note that once the book is launched, we'll have more leeway to do large book giveaways, akin to current giveaways of The Precipice. Right now, “organic” sales (orders of 20 or fewer copies coming from individual accounts rather than giveaways or reimbursement schemes) are critical.

I’m interested in all takes, but here are some specific questions that might be helpful:

How should we promote the book at upcoming EAGs? (Beyond emails to attendees, we're considering giving away free bookmarks with QR codes to pre-order, and are open to creative marketing ideas that are not overbearing to attendees!)

Are there specific influential people that you wish EA was engaging more that we should share the book with?

Are there specific shows or websites that you watch regularly that you think we should advertise with or pitch content to?

Which endorsements would be most impactful for the book? (See our top endorsements so far here)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to Make Your Article More Persuasive? (Spoiler: Do User Testing Like a Startup Would), published by Yonatan Cale on March 30, 2022 on LessWrong.

Are you writing an article that’s supposed to change people’s minds and/or actions? For example, an article about career advice, productivity, or getting people to apply for a job? Then this post is for you!

TL;DR: For important articles I recommend doing real user testing (even with one person!). Try it and you’ll never go back.

I specifically recommend:

(1) Watch them read it, don’t just ask for Google Doc comments

Open a zoom, share screen, and ask them to share their thoughts live as they read.

(2) Show it to your target audience, don’t just ask others what your target audience would think

If your article is supposed to change developers’ career plans, ask a dev who might change their career plan. Don’t ask a biologist what a dev might think.

(3) Check if they changed their mind, not just if they like your writing

If they end with “I’ve got to do this now!”: That's a good sign.

If they say “sounds interesting, I’ll think about it”, that’s not as good.

I personally like to first ask the other person for their opinion (for example, “what’s your current career plan?”), then let them read the article, and then ask again.

The general intuition:

If you want to adopt my frame of mind, ask yourself: Would your idea of how to improve [the article’s persuasiveness] also improve [how engaging a startup’s app] is?

Here are some exercises, do you think these would work?

Let GPT-3 look at the app (or article) and tell you how engaging the app (or how persuasive the article) is

Try to guess in advance all the reasons why your target audience won’t agree with the article, or won’t be engaged with the app

Hint: Is the test [checking if a real user from your target audience changes their mind] or [using some proxy to guess whether a real user would change their mind] ?

How to start?

Easiest: post in Bountied Rationality, or in another group that has your target audience.

A draft post you can copy & paste: “Hey, I’ll pay $10 for someone in [describe your target audience here] to review a post of mine for the EA/lesswrong forum. I want to do it live, over a video call. The call is capped at 30 minutes (and expected to be less). Comment or message me if you’re interested. Tagging or referring relevant people is also appreciated.”

Thanks to Matt Brooks and Daniel Reeves for helping make this article better <3

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AXRP Episode 13 - First Principles of AGI Safety with Richard Ngo, published by DanielFilan on March 31, 2022 on The AI Alignment Forum.

Google Podcasts link

This podcast is called AXRP, pronounced axe-urp and short for the AI X-risk Research Podcast. Here, I (Daniel Filan) have conversations with researchers about their papers. We discuss the paper and hopefully get a sense of why it’s been written and how it might reduce the risk of artificial intelligence causing an existential catastrophe: that is, permanently and drastically curtailing humanity’s future potential.

How should we think about artificial general intelligence (AGI), and the risks it might pose? What constraints exist on technical solutions to the problem of aligning superhuman AI systems with human intentions? In this episode, I talk to Richard Ngo about his report analyzing AGI safety from first principles, and recent conversations he had with Eliezer Yudkowsky about the difficulty of AI alignment.

Topics we discuss:

The nature of intelligence and AGI

The nature of intelligence

AGI: what and how

Single vs collective AI minds

AGI in practice

Impact

Timing

Creation

Risks and benefits

Making AGI safe

Robustness of the agency abstraction

Pivotal acts

AGI safety concepts

Alignment

Transparency

Cooperation

Optima and selection pressures

The AI alignment research community

Updates from Yudkowsky conversation

Corrections to the community

Why others don’t join

Richard Ngo as a researcher

The world approaching AGI

Following Richard’s work

Daniel Filan: Hello, everybody. Today, I’ll be speaking with Richard Ngo. Richard is a researcher at OpenAI, where he works on AI governance and forecasting. He also was a research engineer at DeepMind, and designed the course “AGI Safety Fundamentals”. We’ll be discussing his report, AGI Safety from First Principles, as well as his debate with Eliezer Yudkowsky about the difficulty of AI alignment. For links to what we’re discussing, you can check the description of this episode, and you can read the transcripts at axrp.net. Well, Richard, welcome to the show.

Richard Ngo: Thanks so much for having me.

The nature of intelligence and AGI

Daniel Filan: You wrote this report, AGI Safety from First Principles. What is AGI? What do you mean by that term?

Richard Ngo: Fundamentally, it’s AI that can perform as well as humans, or comparably to humans on a wide range of tasks, because it’s got this thing called general intelligence that humans also have. And I think this is not a very precise definition. I think we just don’t understand what general intelligence really is, like how to specifically pin down this concept, but it does seem like there’s something special about humans that allows us to do all the things we do. And that’s the thing that we expect AGI to have.

The nature of intelligence

Daniel Filan: And so I shouldn’t be imagining that there’s some really technical definition of general or intelligence that you’re using here?

Richard Ngo: No, I think it’s much more like a pointer to a, let’s say, pre-paradigmatic concept. We kind of know the shape of it, but we don’t understand in mechanistic detail how general intelligence works. It’s got these components like: of course, you need memory, of course you need some kind of planning, but exactly how to characterize the ways in which combinations of these count as general intelligence is something we don’t have yet.

Daniel Filan: So when you’re talking about AGI, often there is this idea of AGI that’s smarter than people, which implies that intelligence is some kind of scalar concept where you can have more or less of it, and it’s really significant if you have more or less of it. Do you think that’s basically a correct way to think about intelligence, or should we just not go with that?

Richard Ngo: Yeah, I think that seems...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New community builders don't have enough fun!, published by Engelhardt on March 30, 2022 on The Effective Altruism Forum.

tldr; if you’re a new EA community builder, or someone who has considered it but decided not to because it seemed like too much of a sacrifice, please consider the fact that the best kind of community building is fun for the organizer too. You should value your enjoyment of the events and the community building process higher than you likely do at the moment, even if you take a completely altruistic rather than egoistic perspective.

(Disclaimer: a reviewer noted that this article was ironically not that exciting to read, but I’m not that excited about rewriting it so I’ll follow my own advice and work on more exciting tasks)

I have been active in the Swedish EA community since 2015 and have done both community building and guided others doing the same. From this time my biggest takeaway has been that new community builders focus way too much on what they feel they "should" be doing rather than what they want to be doing. Doing what would actually be fun for them. When we started the group at my local university we saw the other associations running public lectures, so we felt like we had to do that too. Even if we weren’t that excited about it nor really had any lecturers in mind. So we spent way too much time thinking about and planning for that instead of just arranging something more fun. And these days all the other cool EA groups run fellowships, which we’ve enjoyed so far, but I’m concerned that it gets so heavily recommended that organizers feel obligated to do them even if they don’t feel like it.

While it’s good to not reinvent the wheel, I think this type of behavior really limits the potential of both the community builders themselves and the group as a whole. In my opinion community builders should also be more hesitant about running projects that they aren’t excited about.

Why excitement is important:

Your interest in EA is a really valuable resource that you should be very hesitant to use up.

Your impact as an EA is ideally going to be over a period of many years of work, donations, and thinking. So anything that increases your risk of just giving up on those aspirations is something to be careful with. It’s not worth it to drain away your excitement for improving the world just for the sake of doing events you feel like you should want to run.

The “Scratch your own itch” principle applies to community building too.

“Scratch your own itch” is the idea that many of the best businesses (especially startups) come from someone making a product to solve a problem they have themselves. I think this applies to community organizers too. You should have more events you’re excited to go to, not that some hypothetical other person might want to go to. Most of the best EA events I’ve been to or organized have just come from one organizer being super curious about a topic and creating an event to try to dig deeper into it. For example some years ago EA NTNU in Norway arranged an event where the crown princess attended, which was certainly impressive. But it was also a lot of time and effort to organize. So they felt like that event was much less worthwhile than another event where they just asked lots of really in depth questions to one of the people at Against Malaria Foundation.

The lecture was more impressive, but the organizers seemed more excited about scratching their own itch of getting all their questions about the Against Malaria Foundation answered.

EA groups tend to die off if it’s not fun to be an organizer in them.

The main recommendations for how to create an ea group is to make sure to have a core group of active members who are engaged enough to keep the group going. Such a group can be pretty hard to find. Several groups here in ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Advice to a new EA org employee, published by Julia Wise on March 30, 2022 on The Effective Altruism Forum.

Today I talked with a friend who just started a new job at an organization focused on existential risk. Two pieces of advice I gave:

This is a fast-moving, ambitious organization where people have short AI timelines. It is not trying to guard you against burnout. You have to be in charge of deciding how much to guard yourself there. (It doesn't really matter which organization this is - if that description applies, the advice probably does too.)

This is a setting where work, housing, dating, friendship, and social scene are closely entwined for a lot of people. When you're deciding how entwined they should be, consider how past things have gone for you: if you've had acrimonious breakups or clashes with housemates or coworkers, probably err on the side of keeping things more separate. Also consider that people you're not very entwined with now might be different in six months or a year (when you end up working in the same chain of management, etc.)

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.