The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Hacker and the Beggar, published by IsabelHasse on September 18, 2023 on The Effective Altruism Forum.

I am writing a musical. This is the opening scene to that musical. Enjoy!

Beggar: Hey, do you have a dollar or two? I really need to get something to eat.

Hacker: You know, I do, but I'm not gonna give it to you. Sorry, man.

Beggar: What?

Hacker: I do have some money, but I don't want to give you any.

Beggar: Man, fuck you.

Hacker: You know, I get how you feel. It'd probably feel better if I told you I didn't have any, right?

Beggar: That's what people usually say.

Hacker: Yeah, people lie. It's a tough world out there. (Beat) Mind if I sit?

Beggar: Sure.

(The Hacker sits down on the bench next to the Beggar)

Hacker: Here's the deal. All resources are finite. There's only so many dollars in the world, right? And even fewer in my pocket. Every dollar I give to you, I'm not using for anything else. And there are a ridiculously unfathomable number of possible things I could do with that dollar. So this one choice to do one thing is really a choice to not do an unfathomable number of other things. You following me?

Beggar: You're saying you'd rather, what, buy yourself a candy bar?

Hacker: I'm not talking about any one thing. I'm talking about all of them. But let's narrow it down. Say I'm looking to give the dollar away. Do some good, yeah? Well, why not give it to NASA, or to some villager in Uganda who has tuberculosis, or to a guy running for Congress who says he'll get rid of homelessness if he gets elected? Why not give it to.I don't know, anteater conservation, or moth welfare, or my mom, or that other homeless guy down the block? I can't give the one dollar to all of those things at once. It's just one dollar.

Beggar: So what are you going to do with it?

Hacker: That's exactly it! I mean, who knows what I'm really going to do with the dollar? It's all up to the whims of my future self, isn't it? Is he the kind of guy who decides to give an extra dollar to research into neglected tropical diseases this month? Or is he the kind of guy to forget all this and go buy himself a candy bar? Statistically, I'm the candy bar guy, and who am I to argue with statistics? Do I really think I'm that special?

Beggar: So you might as well give it to me?

Hacker: Well, maybe, but hear me out - am I doing you a favor, giving you a dollar? What's a homeless guy gonna spend an extra buck on? Drugs, right? So maybe not, then. But here's the thing - who am I to say it's a bad idea for you to get yourself some drugs? I don't know you. I don't know your life. Some people like drugs. So what? Who gave me the authority to decide if you should treat yourself to a little fentanyl? And I mean, if nobody gave money to drug addicts, you might all start dying from withdrawal, and then that's on me, isn't it?

Beggar: Hey, I'm not a drug addict, I swear. I just need something to eat.

Hacker: Maybe you do. And hey, I'm not judging. I'm just thinking out loud. The thing is, it is on me if you go into withdrawal, or go hungry today, but it's also on me that that guy in Uganda is dying of tuberculosis. It's all on me. I'm responsible for all of it. And it's not just money. Every second I spend sitting here talking to you, I'm not spending on literally anything else. I could be working extra hours, making more money, so I could give to all of those things and still have a buck left to give to you. Time is money, money is time, resources are finite. It all comes back to that.

Beggar: So why are you still here?

Hacker. I don't know, man. I got shit to say, I guess. I'm working through a lot of stuff right now, you know? I got this nine to five job, right, I'm making money, I give some of it away. So, good. That's better than most people. Maybe I'm a good person, then. Or maybe everyone else is even s...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Eugenics Performed By A Blind, Idiot God, published by omnizoid on September 18, 2023 on LessWrong.

(Azathoth, Lovecraft's famous blind, amoral, idiot God.)

Crosspost of this.

I'm hugely in favor of gene editing and other actions that would improve the welfare of future people. If we could perform genetic engineering that made future people smarter, happier, and less likely to get diseases, I'd be in favor of it. This assumption is controversial. Many people think there's something immoral about changing the genes of humans, even in ways that are expected to improve their quality of life. They think that such an idea sounds too much like the clearly objectionable coercive eugenics of the Nazis, for instance.

But you know what would be almost as bad as eugenics performed by Nazis - eugenics performed by a totally random, amoral selector. This eugenicist wouldn't have cruel ideas about Aryan superiority, for instance - instead, they have a bizarre fetishism of reproduction. This selector performs eugenics so that only those who reproduce a lot - and also help out their kin - are likely to pass on their genes.

Such a selector is wholly unconcerned with human welfare. It makes it so that humans can't empathize with those far away, because warring with native tribes is effective. It makes it so that men are naturally more aggressive than women, committing 96% of homicides, all because in the evolutionary past it was beneficial - it enabled more efficient fighting, for instance.

In fact, this selector has selected for individuals who are likely to engage in "rape . . . infanticide, abuse of low-status individuals, and murdering all those people over there and taking their stuff." It selects for individuals who reproduce frequently, rather than those who are good, moral, or even happy.

In fact, in some other species, things are even worse. Some species give birth to hundreds of millions of eggs, many of whom contain sentient beings almost all of whom die a horrible painful death at a young age. This selector makes it so that male ducks have corkscrew penises so that they can rape female ducks more efficiently.

This predictor has been operating for billions of years. Their amorality results in them designing both all sorts of horrifying, rapidly multiplying bacteria and viruses that kills lots of people and animals alike and various defense mechanisms. But after millions of years, it offers for you to take over their job. Rather than selecting for prolificness alone, you can affect which beings exist in the future with moral goals in mind! You can make it so that future beings are likely to be happy, kind, and just. Isn't this an improvement?

But this is the world that we live in. Selection pressures exist as an inevitable fact of life. Evolution has shaped our behaviors. Our choice is not between selected behaviors and unselected behaviors. It is between behaviors selected by parents who have their children's best interests in mind, who want their children to be kind, happy, and healthy, and selection at the hands of the blind, idiot, Darwinian god, who has been practicing social Darwinism for millions of years, where only those who pass on their genes have their traits reproduced.

Of course, this doesn't mean that we should practice the kind of harmful, coercive eugenics practiced by the Nazis. It doesn't mean we should prevent anyone from reproducing. But it does mean we should empower parents with the option of gene editing to improve their children's lives, rather than banning it. It means we should see making future people happier and more moral as a worthwhile goal.

The amoral god that selects only for having many offspring has turned over the reigns to us. We can leave its ghastly designs in place, or instead change them to improve the lives of the future. I think th...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Actually, "personal attacks after object-level arguments" is a pretty good rule of epistemic conduct, published by Max H on September 18, 2023 on LessWrong.

Background: this post is a response to a recent post by @Zack_M_Davis, which is itself a response to a comment on another post. I intentionally wrote this post in a way that tries to decontextualize it somewhat from the original comment and its author, but without being at least a bit familiar with all of the context, it's probably not very intelligible or interesting.

This post started as a comment on Zack's post, but I am spinning it out into own post because I think it has broader applicability and because I am interested in hearing thoughts and responses from the readers and upvoters of that post, moreso than from its author.

I originally responded to Zack's post with a comment here, but on further reflection, I want to strengthen and clarify some claims I alluded to in that comment.

I see Zack's post as making two main claims. I have medium confidence that both of the claims are false, and high confidence that they are not well-supported by the text in the post.

Claim one (paraphrased): "personal attacks (alt. negative character assessments) should only come after making object-level arguments" isn't actually a {good,useful,true} rule of epistemic conduct.

I see the primary justifications given for this claim in the text as (paraphrased):

The person claiming this is a rule should be able to explain why they think such a rule would systematically lead to more accurate beliefs ("maps that reflect the territory").

In fact, no such valid explanation is likely to exist, because following such a rule would not systematically lead to the formation of more accurate beliefs.

The issue with the first justification is that no one has actually claimed that the existence of such a rule is obvious or self-evident. Publicly holding a non-obvious belief does not obligate the holder to publicly justify that belief to the satisfaction of the author.

Perhaps a more charitable interpretation of the author's words ("I think [the claimant]... should be able to explain why such a rule systematically produces maps that reflect the territory...") is that the absence of a satisfactory explanation from the claimant is Bayesian evidence that no such explanation exists. But if that's what the author actually meant, then they should have said so more plainly, and acknowledged that this is often pretty weak evidence.

(Relevant background context: Zack has previously argued at great length that this particular claimant's failure to adequately respond publicly about another matter is Bayesian evidence about a variety of important inferences related to that claimant's epistemics.)

The issue with the second justification is that a valid explanation for why this is a good rule very likely does exist; I gave one that I find plausible at the end of my first comment:

If more people read the beginning of an argument than the end, putting the personal attacks at the beginning will predictably lead to more people seeing the attacks than the arguments that support them. Even if such readers are not consciously convinced by attacks without argument, it seems implausible that their beliefs or impressions will not be moved at all.

Another possible explanation (for which the source of inspiration Raemon gives in the comments): one interpretation of the proposed rule is that it is essentially just a restatement of avoiding the logical fallacy of Bulverism; if that interpretation is accepted, what remains to be shown is that avoiding Bulverism in particular, or even avoiding logical fallacies in general, is likely to lead to more accurate beliefs in both readers and authors, independent of the truth values of the specific claims being made. This seems plau...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Three ways interpretability could be impactful, published by Arthur Conmy on September 18, 2023 on The AI Alignment Forum.

Epistemic status: somewhat confident that overall these arguments show interpretability could be impactful, but not confident on any individual point.

Background: I wrote a draft of this post here after reading Charbel's work. Thanks to Charbel and several others' feedback, the post was improved a lot: I would definitely recommend adversarial collaboration so thanks Charbel! I'm hoping to discuss interpretability's impact further so posting now.

Three ways interpretability could be impactful

Just read the test and validation set parts if you have little time.

1. Test set interpretability: Finding alignment failures

What this is. We want to find alignment failures with interpretability that other forms of auditing do not find. Before deploying a model, we find inputs that seem to elicit new capabilities of the model. Either A) the model has correctly generalized to novel capabilities or B) the model is deceiving our evaluation process to produce convincing but false outputs. We then use our future interpretability tools to try to understand some of the computation the AI is doing to reach these new capabilities. We either become more confident that the AI has correctly generalized, or we've uncovered deceptive computation and we can retrain and/or provide robust evidence of misalignment to relevant actors.

Why this isn't achievable with other methods. If a model can deceive our evaluation process it could be very difficult to observe this deception with mere behavorial evals. See here.

In my mind test set interpretability primarily targets a specific set of alignment failures, illustrated here:

Figure 1.

In the appendix I outline my reasoning behind (interpretability's role in) Figure 1.

2. Validation set interpretability: A better science of alignment.

What this is (thanks to Neel). We have little ground truth on whether our models are misaligned now or how far methods such as RLHF will further scale. More generally, we understand little about how machine learning works, which limits our ability to reason about future systems. Interpretability could first and foremost actually provide evidence for what our alignment techniques are doing (e.g interpreting RLHF reward models) and secondly give us a better evidence base for reasoning about deep learning. I think that Progress Measures for Grokking Via Mechanistic Interpretability has already somewhat changed people's perspectives on how ML models select different algorithms (e.g here, here).

This differs from test set interpretability as it is broader and can be applied before testing potentially misaligned models, to steer the field towards better practises for alignment (Russell and Norvig's validation/test distinction here may be helpful for analogy).

Why this isn't achievable with other methods. If we want to understand how models work for safety-relevant end goals, it seems likely to me that interpretability is the best research direction to pursue. Most methods are merely behavorial and so provide limited ground truth, especially when we are uncertain about deception. For example, I think the existing work trying to make chain-of-thought faithful shows that naive prompting is likely insufficient to understand models' reasoning. Non-behavorial methods such as science of deep learning approaches (e.g singular learning theory, scaling laws) by default give high-level descriptions of neural network statistics such as loss or RLCT. I don't think these approaches are as likely to get as close to answering the questions about internal computations in AIs that I think successful interpretability could lead to. I think some other directions are worthwhile bets due to uncertainty and neglectedness, howe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Memory bandwidth constraints imply economies of scale in AI inference, published by Ege Erdil on September 17, 2023 on LessWrong.

Contemporary GPUs often have very imbalanced memory vs arithmetic operation capabilities. For instance, an H100 can do around 3e15 8-bit FLOP/s, but the speed at which information can move between the cores and the GPU memory is only 3 TB/s. As 8 bits = 1 byte, there is a mismatch of three orders of magnitude between the arithmetic operation capabilities of the GPU and its memory bandwidth.

This imbalance ends up substantially lowering the utilization rate of ML hardware when batch sizes are small. For instance, suppose we have a model parametrized by 1.6 trillion 8-bit floating point numbers. To just fit the parameters of the model onto the GPUs, we'll need at least 20 H100s, as each H100 has a VRAM of 80 GB. Suppose we split our model into 20 layers and use 20-way tensor parallelism: this means that we slice the parameters of the model "vertically", such that the first GPU holds the first 5% of the parameters in every layer, the second GPU holds the second 5%, et cetera.

This sounds good, but now think of what happens when we try to run this model. In this case, roughly speaking, each parameter comes with one addition and one multiplication operation, so we do around 3.2 trillion arithmetic operations in one forward pass. As each H100 does 3e15 8-bit FLOP/s and we have 20 of them running tensor parallel, we can do this in a mere ~ 0.05 milliseconds. However, each parameter also has to be read into memory, and here our total memory bandwidth is only 60 TB/s, meaning for a model of size 1.6 TB we must spend (1.6 TB)/(60 TB/s) ~= 27 ms just because of the memory bottlenecks! This bottlenecks inference and we end up with an abysmal utilization rate of approximately (0.05 ms)/(27 ms) ~= 0.2%. This becomes even worse when we also take in inter-GPU communication costs into account, which would be at around 1 TB/s if the GPUs are using NVLink.

Well, this is not very good. Most of our arithmetic operation capability is being wasted because the ALUs spend most of their time idling and waiting for the parameters to be moved to the GPU cores. Can we somehow improve this?

A crucial observation is that if getting the parameters to the GPU cores is the bottleneck, we want to somehow amortize this over many calls to the model. For instance, imagine we could move a batch of parameters to the cores and use them a thousand times before moving on to the next batch. This would do much to remedy the imbalance between memory read and compute times.

If our model is an LLM, then unfortunately we cannot do this for a single user because text is generated serially: even though each token needs its own LLM call and so the user needs to make many calls to the model to generate text, we can't parallelize these calls because each future token call needs to know all the past tokens. This inherently serial nature of text generation makes it infeasible to improve the memory read and compute time balance if only a single user is being serviced by the model.

However, things are different if we get to batch requests from multiple users together. For instance, suppose that our model is being asked to generate tokens by thousands of users at any given time. Then, we can parallelize these calls: every time we load some parameters onto the GPU cores, we perform the operations associated with those parameters for all user calls at once. This way, we amortize the reading cost of the parameters over many users, greatly improving our situation. Eventually this hits diminishing returns because we must also read the hidden state of each user's calls into GPU memory, but the hidden states are usually significantly smaller than the whole model, so parallelization still results in huge ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's in a Pause?, published by Davidmanheim on September 17, 2023 on The Effective Altruism Forum.

This post is part of AI Pause Debate Week. Please see this sequence for other posts in the debate.

An AI Moratorium of some sort has been discussed, but details matter - it's not particularly meaningful to agree or disagree with a policy that has no details. A discussion requires concrete claims.

To start, I see three key questions, namely:

What does a moratorium include?

When and how would a pause work?

What are the concrete steps forward?

Before answering those, I want to provide a very short introduction and propose what is in or out of bounds for a discussion.

There seems to be a strong consensus that future artificial intelligence could be very bad. There is quite a significant uncertainty and dispute about many of the details - how bad it could be, and when the different risks materialize. Pausing or stopping AI progress is anywhere from completely unreasonable to obviously necessary, depending on those risks, and the difficulty of avoiding them - but eliminating those uncertainties is a different discussion, and for now, I think we should agree to take the disputes and uncertainties about the risks as a given. We will need to debate and make decisions under uncertainty. So the question of whether to stop and how to do so depends on the details of the proposal - but these seem absent from most of the discussion. For that reason, I want to lay out a few of the places where I think these need clarification, including not just what a moratorium would include and exclude, but also concrete next steps to getting there.

Getting to a final proposal means facing a few uncomfortable policy constraints that I'd also like to suggest be agreed on for this discussion. An immediate, temporary pause isn't currently possible to monitor, much less enforce, even if it were likely that some or most parties would agree. Similarly, a single company or country announcing a unilateral halt to building advanced models is not credible without assurances, and is likely both ineffective at addressing the broader race dynamics, and differentially advantages the least responsible actors. For these reasons, the type of moratorium I think worth discussing is a multilateral agreement centered on countries and international corporations, one which addresses both current and unclear future risks. But, as I will conclude, much needs to happen more rapidly than that - international oversight should not be an excuse for inaction.

What Does a Moratorium Include?

There is at least widespread agreement on many things that aren't and wouldn't be included. Current systems aren't going to be withdrawn - any ban would be targeted to systems more dangerous than those that exist. We're not talking about banning academic research using current models, and no ban would stop research to make future systems safer, assuming that the research itself does not involve building dangerous systems. Similarly, systems that reasonably demonstrate that they are low risk would be allowed, though how that safety is shown is unclear.

Next, there are certain parts of the proposal that are contentious, but not all of it. Most critics of a moratorium agree that we should not and can't afford to build dangerous systems - they simply disagree where the line belongs. Should we allow arbitrary plugins? Should we ban open-sourcing models? When do we need to stop? The answers are debated. And while these all seem worrying to me, the debate makes sense - there are many irreducible uncertainties, we have a global community with differing views, and actual diplomatic solutions will require people who disagree to come to some agreement.

As should be clear from my views on the need to negotiate answers, I'm not planning to dictate exactl...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Telopheme, telophore, and telotect, published by Tsvi Benson-Tilsen on September 17, 2023 on The AI Alignment Forum.

[Metadata: crossposted from. First completed June 7, 2023.]

To come to know that a mind will have some specified ultimate effect on the world, first come to know, narrowly and in full, what about the mind makes it have effects on the world.

The fundamental question

Suppose there is a strong mind that has large effects on the word. What determines the effects of the mind?

What sort of object is this question asking for? Most obviously it's asking for a sort of "rudder" for a mind: an element of the mind that can be easily tweaked by an external specifier to "steer" the mind, i.e. to specify the mind's ultimate effects on the world. For example, a utility function for a classical agent is a rudder.

But in asking the fundamental question that way - - asking for a rudder - - that essay losses grasp of the slippery question and the real question withdraws. The section of that essay on The word "What", as in ¿What sort of thing is a "what" in the question "What determines a mind's effects?", brushes against the border of this issue but doesn't trek further in. That section asks:

What sort of element can determine a mind's effects?

It should have asked more fully:

What are the preconditions under which an element can (knowably, wieldily, densely) determine a mind's effects?

That is, what structure does a mind have to possess, so that there can be an element that determines the mind's ultimate effects?

To put it another way: asking how to "put a goal into an agent" makes it sound like there's a slot in the agent for a goal; asking how to "point the agent" makes it sound like the agent has the capacity to go in a specified direction. Here the question is, what does an agent need to have, if it has the capacity to go in a specified direction?

Synopsis

Telopheme

The rudder, the element that determines the mind's ultimate effects, is a telopheme. The morpheme "telo-" means "telos" = "goal, end, purpose", here meaning "ultimate effects". The morpheme "-pheme" is like "blaspheme" ("deceive-speak"). ("Telopheme" is probably wrong morphology and doesn't indicate an agent noun, which it ought to do, but sadly I don't speak Ancient Greek.) So a telopheme is a goal-sayer: it says the goal, the end, the ultimate effects.

For example, a utility function for an omnipotent classical cartesian agent is a telopheme.

The utility function says how good different worlds are, and by saying that, it determines that the ultimate effect of the agent will be to make the world be the world most highly scored by the utility function.

Not only does the utility function determine the world, but it (perhaps) does so densely, knowably, and wieldily.

Densely: We can imagine that the utility function is expressed compactly, compared to the complex behavior that the agent executes in pursuit of the best world as a consequence of preimaging the best world through the agent's world-model onto behavior.

Knowably: The structure of the agent shows transparently that it will make the world be whatever world is best according to the utility function.

Wieldily: IF (⟵ big if) there is available a faithful interpretation of our language about possible worlds into the agent's language about possible worlds, then we can (if granted access) fluently redirect the agent so that the world ends up as anything we choose by saying a new utility function with that world as the maximum.

There's a hidden implication in the name "telopheme". The implication is that ultimate effects are speakable.

Telophore

The minimal sufficient preconditions for a mind's telopheme to be a telopheme is the telophore (or telophor) of the telopheme. Here "-phore" means "bearer, carrier" (as in "phosphorus" = "light-bearer", "metapho...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Map of the biosecurity landscape (list of GCBR-relevant orgs for newcomers), published by Max Görlitz on September 17, 2023 on The Effective Altruism Forum.

When talking to newcomers to the field of biosecurity, I often felt annoyed that there wasn't a single introductory resource I could point them to that gives an overview of all the biosecurity-relevant organizations, upskilling opportunities, and funders.

With the help of a lot of contributors, I started this Google doc to provide such a resource. I'm sure that we missed some relevant organizations, and it'd be lovely if some people were to comment on the doc with additional information!

I'll copy the current version below, but please check out the link to the doc if you want to comment and see the most up-to-date version in the future!

Contributors: Max Görlitz, Simon Grimm, Andreas Prenner, Jasper Götting, Anemone Franz, Eva Siegmann, & more

Introduction

I would like to see something like aisafety.world for biosecurity. There already exists the Map of Biosecurity Interventions, but I want one for organizations!

This is a work-in-progress attempt to create a minimum viable product. Please suggest/comment on additional information, and feel free to add your name to the list of contributors.

Also, see this Substack newsletter, "GCBR Organization Updates," which provides a very useful overview and quarterly updates of biosecurity organizations.

Policy

Think tanks

Europe

Explicitly focused on catastrophic or existential risks from pandemics

International Center for Future Generations (ICFG)

is a European think-and-do-tank for improving societal resilience in relation to exponential technologies and existential risks.

Based in the Netherlands and Belgium

Simon Institute for Longterm Governance

SI's mission is to increase the capacity of policy networks to mitigate global catastrophic risks and build resilience for civilization to flourish.

Based in Geneva, Switzerland

Centre for Long-Term Resilience (CLTR)

an independent think tank with a mission to transform global resilience to extreme risks

London, UK

Pour Demain

is a non-profit think tank that develops proposals on neglected issues, positively impacting Switzerland and beyond.

Center for Long-Term Policy (Langsikt)

Oslo-based think tank, similar to CLTR and Pour Demain, focused on Norway and possibly other Nordic countries.

Future of Humanity Institute (FHI)

is a unique world-leading research centre that works on big picture questions for human civilisation and explores what can be done now to ensure a flourishing long-term future

Oxford, UK

The Centre for the Study of Existential Risk (CSER)

an interdisciplinary research centre within the University of Cambridge dedicated to the study and mitigation of existential risks

Cambridge, UK

Association for Long Term Existence and Resilience (ALTER)

A think-and-do thank in Israel focused on both domestic and international policy and research related to building a safe and prosperous global future.

Focused on general pandemic preparedness/mitigation or biological weapons

CBW network for a comprehensive reinforcement of norms against chemical and biological weapons (CBWNet)

The joint project aims to identify options to comprehensively strengthen the norms against chemical and biological weapons (CBW).

Collaboration between multiple German universities

funded by the German Federal Ministry of Education and Research

Independent Pandemic Preparedness Secretariat (IPPS)

The IPPS is a wholly independent entity that will serve to ensure join up between relevant states, the private sector, and global health institutions in support of the 100 days mission.

The goal of the 100 Days Mission is to prepare as much as possible so that within the first 100 days that a pandemic threat is identified, crucial interventions can be made ava...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Winners of the African EA Forum Competition!, published by Luke Eure on September 17, 2023 on The Effective Altruism Forum.

The African EA Forum Competition ran from mid-May to mid-August with the goal bringing more African perspectives to the wider EA community, to encourage African EAs to share experiences, thoughts, and to engage with the online EA community.

I'm excited to announce the winners and runners up across our three categories.

The winners and runners up

Cause exploration:

Winner - Zayn: Unveiling the Longtermism Framework in Islam: Urging Muslims to Embrace Future-Oriented Values through 'Islamic Longtermism'

Runner up - Jacob Ayang and Aurelia Adhiambo: The Epidemic of Second-Hand Battery Cages Being Imported into Africa: What does this mean for the cage-free movement in Africa?

African perspectives on EA:

Winner - George Gor: Why value-based salaries might help African effective altruists achieve more impact

Runner up - Ashura Batungwanayo and Hayley Martin: Making EA more inclusive, representative, and impactful in Africa

Summaries of existing work / personal reflections:

Winner - NatKiilu: Personal Reflections on Longtermism

Runner up - Vee: GiveDirectly Unveils Challenges in Cash Transfer Program, Pledges Solutions to Support Impoverished Communities in the Democratic Republic of Congo: My Two Cents

Winners win a prize of USD $1,000, runners up win USD $500.

Impact of the competition

We ended up with ~30 posts from Africans in 3 months. I believe this is roughly half of all posts that have been made by Africans in the history of the EA Forum

As the list of winners testifies, we had a great mix of posts across topics - animal welfare, AI risk, EA culture, global development, longtermism etc.

I think it's notable that it is not always the highest karma post that won within each category. We didn't want to anchor too much to reception by forum readers, and to reward posts that were outside the typical forum style (still requiring that a post be original, clear, discussion-provoking, and persuasive or relevant to the forum)

We had around 15 writers join the virtual training on how to write for the EA Forum. Many of the people who ended up posting were in this training. From my perspective the impact was less teaching the nuts and bolts about writing a good post, and more about instilling the confidence that we really want their voices on the forum (if you're reading this, we want YOUR voice on the forum!)

On the other hand, not very many writers took up the offer of mentorship. I think it was helpful to have the offer because a few writers were paired with mentors, and also the mere offer of being paired with a mentor helps instill confidence even if you don't accept the offer.

Acknowledgements

Thanks to our judges for taking their time to evaluate posts, to the admins of the forum for maintaining this wonderful platform, and Daniel Yu for funding the prizes.

And most of all thank you to all the writers. I hope the variety of posts highlighted by this competition and the great engagement from the online EA community will encourage more Africans - and people elsewhere in the world - to use this forum. Share your viewpoints. Challenge and be challenged. Contribute to this collective effort to make the world a better place for all.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to talk about reasons why AGI might not be near?, published by Kaj Sotala on September 17, 2023 on The AI Alignment Forum.

I occasionally have some thoughts about why AGI might not be as near as a lot of people seem to think, but I'm confused about how/whether to talk about them in public.

The biggest reason for not talking about them is that one person's "here is a list of capabilities that I think an AGI would need to have, that I don't see there being progress on" is another person's "here's a roadmap of AGI capabilities that we should do focused research on". Any articulation of missing capabilities that is clear enough to be convincing, seems also clear enough to get people thinking about how to achieve those capabilities.

At the same time, the community thinking that AGI is closer than it really is (if that's indeed the case) has numerous costs, including at least:

Immense mental health costs to a huge number of people who think that AGI is imminent

People at large making bad strategic decisions that end up having major costs, e.g. not putting any money in savings because they expect it to not matter soon

Alignment people specifically making bad strategic decisions that end up having major costs, e.g. focusing on alignment approaches that one might pay off in the long term and neglecting more foundational long-term research

Alignment people losing credibility and getting a reputation of crying wolf once predicted AGI advances fail to materialize

Having a better model of what exactly is missing could conceivably also make it easier to predict when AGI will actually be near. But I'm not sure to what extent this is actually the case, since the development of core AGI competencies feels more of a question of insight than grind, and insight seems very hard to predict.

A benefit from this that does seem more plausible would be if the analysis of capabilities gave us information that we could use to figure out what a good future landscape would look like. For example, suppose that we aren't likely to get AGI soon and that the capabilities we currently have will create a society that looks more like the one described in Comprehensive AI Services, and that such services could safely be used to detect signs of actually dangerous AGIs. If this was the case, then it would be important to know that we may want to accelerate the deployment of technologies that are taking in the world in a CAIS-like direction, and possibly e.g. promote rather than oppose things like open source LLMs.

One argument would be that if AGI really isn't near, then that's going to be obvious pretty soon, and it's unlikely that my arguments in particular for this would be all that unique - someone else would be likely to make them soon anyway. But I think this argument cuts both ways - if someone else is likely to make the same arguments soon anyway, then there's also limited benefit in writing them up. (Of course, if it saves people from significant mental anguish, even just making those arguments slightly earlier seems good, so overall this argument seems like it's weakly in favor of writing up the arguments.)

From Armstrong & Sotala (2012):

Some AI prediction claim that AI will result from grind: i.e. lots of hard work and money. Other claim that AI will need special insights: new unexpected ideas that will blow the field wide open (Deutsch 2012).

In general, we are quite good at predicting grind. Project managers and various leaders are often quite good at estimating the length of projects (as long as they're not directly involved in the project (Buehler, Griffin, and Ross 1994)). Even for relatively creative work, people have sufficient feedback to hazard reasonable guesses. Publication dates for video games, for instance, though often over-optimistic, are generally not ridiculously erroneous...