Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Lead exposure: a shallow cause exploration, published by JoelMcGuire on April 11, 2023 on The Effective Altruism Forum.

Summary

This shallow cause area report explores the impact of lead exposure in childhood on subjective wellbeing (SWB) in adulthood. It was completed in two weeks. We performed non-systematic searches to estimate the impact of lead exposure on SWB and to find potential cost-effective interventions.

We found studies investigating two correlational longitudinal cohorts (following children to adulthood) from New Zealand and Australia analysing the relationship between childhood blood lead levels (BLLs) and adult affective mental health (MHa). Based on this data, our best guess estimate is that an additional microgram of lead per deciliter of blood for each year for ten years of childhood, leads to a total lifelong (62 years) loss of 1.5 WELLBYs, and a larger overall 3.8 WELLBYs loss when we include some guesses about household spillovers. Hence, we estimate that a modest amount of lead exposure severely impacts on wellbeing across the lifespan.

From several back of the envelope calculations, we tentatively conclude that lead-reducing interventions would be 1 to 107 times more cost-effective than cash transfers. Advocacy against lead in paint, food, cookware, and cosmetics seems particularly promising.

The scarcity of causal and context relevant data means that we are very uncertain about the effect and cost-effectiveness of these interventions. But, given the potentially high cost-effectiveness, we think this is a promising area for additional research. We especially encourage further research into the causal relationship between lead exposure and SWB and the most common sources of lead exposure to reduce uncertainty about the cost-effectiveness of lead interventions.

It’s unclear if the top organisations working to reduce lead exposure, like Pure Earth or the Lead Exposure Elimination Project (LEEP), have sizable funding gaps. Therefore, we’re unsure how much more work should be done to evaluate funding opportunities related to reducing lead exposure for philanthropists aiming to maximise their impact.

Notes

This report focuses on the impact of lead exposure in terms of WELLBYs. One WELLBY is a 1 life satisfaction point change for one year (or any equivalent combination of change in life satisfaction and time). In some cases, we convert results in standard deviations of SWB to WELLBYs using a 2 point standard deviation on 0-10 life satisfaction scales (i.e., 1 SD change is the equivalent of 2 point changes on a 0-10 life satisfaction scale). We consider the limitations of converting from affective mental health measures to WELLBYs in Appendix A4. This naive conversion is based on estimates from large scale data sets like the World Happiness Reports. See our post on the WELLBY method for more details.

Our calculations and data extraction can be found in this Google Spreadsheet and this GitHub repository.

The shallowness of this investigation means (1) we include more guesses and uncertainty in our models, (2) we couldn’t always conduct the most detailed or complex analyses, (3) we might have missed some data, and (4) we take some findings at face value.

Clare Donaldson was co-director of HLI before becoming the current co-director of the Lead Exposure Elimination Project. We do not think this influenced our choices or analysis.

Outline

In Section 1 we introduce the issue of lead exposure and define some key terms we will use throughout the rest of this report.

In Section 2 we explain the mechanisms for how lead exposure could affect wellbeing.

In Section 3 we model the harm of lead exposure on subjective wellbeing using studies of two datasets from New Zealand and Australia (n = 789) relating childhood lead exposure to their adult affective mental heal...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evolution provides no evidence for the sharp left turn, published by Quintin Pope on April 11, 2023 on LessWrong.

Does human evolution imply a sharp left turn from AIs?

Arguments for the sharp left turn in AI capabilities often appeal to an “evolution -> human capabilities” analogy and say that evolution's outer optimization process built a much faster human inner optimization process whose capability gains vastly outstripped those which evolution built into humans. Such arguments claim we will see a similar transition while training AIs, with SGD creating some 'inner thing' which is not SGD and which gains capabilities much faster than SGD can insert them into the AI. Then, just like human civilization exploded in capabilities over a tiny evolutionary time frame, so too will AIs explode in capabilities over a tiny "SGD time frame".

Evolution’s sharp left turn happened for evolution-specific reasons

I think that "evolution -> human capabilities" is a bad analogy for "AI training -> AI capabilities". Let’s compare evolution to within lifetime learning for a single generation of an animal species:

A generation is born.

The animals of the generation learn throughout their lifetimes, collectively performing many billions of steps of learning.

The generation dies, and all of the accumulated products of within lifetime learning are lost.

Differential reproductive success slightly changes the balance of traits across the species.

The only way to transmit information from one generation to the next is through evolution changing genomic traits, because death wipes out the within lifetime learning of each generation.

Now let’s look at the same comparison for humans:

A generation is born.

The humans of the generation learn throughout their lifetimes, collectively performing many billions of steps of learning.

The current generation transmits some fraction of their learned knowledge to the next generation through culture.

The generation dies, but only some of the accumulated products of within lifetime learning are lost.

Differential reproductive success slightly changes the balance of genomic traits across humanity.

Human culture allows some fraction of the current generation’s within lifetime learning to transmit directly to the next generation. In the language of machine learning, the next generation benefits from a kind of knowledge distillation, thanks to the prior generation providing higher quality 'training data' for the next generation's within-lifetime learning.

This is extremely important because within-lifetime learning happens much, much faster than evolution. Even if we conservatively say that brains do two updates per second, and that a generation is just 20 years long, that means a single person’s brain will perform ~1.2 billion updates per generation. Additionally, the human brain probably uses a stronger base optimizer than evolution, so each within-lifetime brain update is also probably better at accumulating information than a single cross-generational evolutionary update. Even if we assume that only 1 / 10,000th of the information learned by each generation makes its way into humanity's cross-generational, persistent endowment of cultural information, that still means culture advances ~100,000 times faster than biological evolution.

I think that "evolution -> human capabilities" is a very bad reference class to make predictions about "AI training -> AI capabilities". We don't train AIs via an outer optimizer over possible inner learning processes, where each inner learning process is initialized from scratch, then takes billions of inner learning steps before the outer optimization process takes one step, and then is deleted after the outer optimizer's single step. Such a bi-level training process would necessarily experience a sharp left turn once each inne...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evolution provides no evidence for the sharp left turn, published by Quintin Pope on April 11, 2023 on The AI Alignment Forum.

Does human evolution imply a sharp left turn from AIs?

Arguments for the sharp left turn in AI capabilities often appeal to an “evolution -> human capabilities” analogy and say that evolution's outer optimization process built a much faster human inner optimization process whose capability gains vastly outstripped those which evolution built into humans. Such arguments claim we will see a similar transition while training AIs, with SGD creating some 'inner thing' which is not SGD and which gains capabilities much faster than SGD can insert them into the AI. Then, just like human civilization exploded in capabilities over a tiny evolutionary time frame, so too will AIs explode in capabilities over a tiny "SGD time frame".

Evolution’s sharp left turn happened for evolution-specific reasons

I think that "evolution -> human capabilities" is a bad analogy for "AI training -> AI capabilities". Let’s compare evolution to within lifetime learning for a single generation of an animal species:

A generation is born.

The animals of the generation learn throughout their lifetimes, collectively performing many billions of steps of learning.

The generation dies, and all of the accumulated products of within lifetime learning are lost.

Differential reproductive success slightly changes the balance of traits across the species.

The only way to transmit information from one generation to the next is through evolution changing genomic traits, because death wipes out the within lifetime learning of each generation.

Now let’s look at the same comparison for humans:

A generation is born.

The humans of the generation learn throughout their lifetimes, collectively performing many billions of steps of learning.

The current generation transmits some fraction of their learned knowledge to the next generation through culture.

The generation dies, but only some of the accumulated products of within lifetime learning are lost.

Differential reproductive success slightly changes the balance of genomic traits across humanity.

Human culture allows some fraction of the current generation’s within lifetime learning to transmit directly to the next generation. In the language of machine learning, the next generation benefits from a kind of knowledge distillation, thanks to the prior generation providing higher quality 'training data' for the next generation's within-lifetime learning.

This is extremely important because within-lifetime learning happens much, much faster than evolution. Even if we conservatively say that brains do two updates per second, and that a generation is just 20 years long, that means a single person’s brain will perform ~1.2 billion updates per generation. Additionally, the human brain probably uses a stronger base optimizer than evolution, so each within-lifetime brain update is also probably better at accumulating information than a single cross-generational evolutionary update. Even if we assume that only 1 / 10,000th of the information learned by each generation makes its way into humanity's cross-generational, persistent endowment of cultural information, that still means culture advances ~100,000 times faster than biological evolution.

I think that "evolution -> human capabilities" is a very bad reference class to make predictions about "AI training -> AI capabilities". We don't train AIs via an outer optimizer over possible inner learning processes, where each inner learning process is initialized from scratch, then takes billions of inner learning steps before the outer optimization process takes one step, and then is deleted after the outer optimizer's single step. Such a bi-level training process would necessarily experience a sharp left turn o...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some Intuitions Around Short AI Timelines Based on Recent Progress, published by Aaron Scher on April 11, 2023 on LessWrong.

tldr: I give some informal evidence and intuitions that point toward AGI coming soon. These include thinking about how crazy the last year has been, beliefs from those in major AI labs, and progress on MMLU.

Intro

This post is intended to be a low-effort reference I can point people to when I say I think there is some evidence for short AI timelines. I might describe the various bits of evidence and intuitions presented here as “intuitions around short AI timelines based on recent progress” (though perhaps there are better terms). They are not a thorough model like Ajeya’s; insofar as somebody is using multiple models when putting together a timelines estimate, I think it would be unreasonable to place less than 20% or greater than 95% weight on extrapolation from current systems and recent progress.

In the spirit of being informal, you can use whatever definition of AGI you like. I mostly use AGI to refer to something like “an AI system which can do pretty much all the human cognitive tasks as well or better than humans (~99% of tasks people in 2023 do).”

Some evidence

I (Aaron) started following AI and AI existential safety around the beginning of 2022; it’s been a little over a year. Some of that time was my understanding catching up with advances from the past couple years, but there have also been major advances.

Some major advances since I’ve been paying attention: Chinchilla paper popularized the scaling laws that were already known to some, there was some DALL-E and related stuff which was cool, CICERO happened which I didn’t follow but indicates we’re probably going to train the AIs to do all the dangerous stuff (see also Auto-GPT and Chaos-GPT, or GPT-4 getting plugins within 2 weeks of release, as more recent updates in this saga of indignity), ChatGPT shows how much more usable models are with RLHF (popularizing methods that have been known for a while), Med-PaLM gets a passing score on the US medical licensing exam (also tons of other PaLM and Flan-PaLM results I haven’t followed but which seem impressive). LLaMA and Alpaca take powerful capabilities from compute-efficient (and over) training and hand them to the public. GPT-4 blows the competition out of the water on many benchmarks.

I probably missed a couple big things (including projects which honorably didn’t publicly push SOTA, 1, 2); the list is probably a bit out of order; I’ve also included things from 2023; but man, that sure is a year of progress.

I don’t expect there are all that many more years with this much progress before we hit AGI — maybe 3-12 years. More importantly, I think this ~15 month period, especially November 2022-now, has generated a large amount of hype and investment in AI research and products. We seem to be on a path such that — in every future year before we die — there is more talent+effort+money working on improving AI capabilities than there was in 2022. I hold some hope that major warning shots and/or regulation would change this picture, in fact I think it’s pretty likely we’ll see warning shots beyond those we have already seen, but I am not too optimistic about what the response will be. As crazy as 2022 was, we should be pretty prepared for a world that gets significantly crazier. I find it hard to imagine that we could have all that many more years that look like 2022 AI progress-wise, especially with significantly increased interest in AI.

A large amount of the public thinks AGI is near. I believe these people are mostly just thinking about how good current systems (GPT-4) are and informally extrapolating

[image description: A Twitter poll from Lex Fridman where he asks “When will superintelligent general AI (AGI) arrive?” There are 270,00...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Request to AGI organizations: Share your views on pausing AI progress, published by Akash on April 11, 2023 on LessWrong.

A few observations from the last few weeks:

On March 22, FLI published an open letter calling for a six-month moratorium on frontier AI progress.

On March 29, Eliezer Yudkowsky published a piece in TIME calling for an indefinite moratorium.

To our knowledge, none of the top AI organizations (OpenAI, DeepMind, Anthropic) have released a statement responding to these pieces.

We offer a request to AGI organizations: Determine what you think about these requests for an AI pause (possibly with uncertainties acknowledged), write up your beliefs in some form, and publicly announce your position.

We believe statements from labs could improve discourse, coordination, and transparency on this important and timely topic.

Discourse: We believe labs are well-positioned to contribute to dialogue around whether (or how) to slow AI progress, making it more likely for society to reach true and useful positions.

Coordination: Statements from labs could make coordination more likely. For example, lab A could say “we would support a pause under X conditions with Y implementation details”. Alternatively, lab B could say “we would be willing to pause if lab C agreed to Z conditions.”

Transparency: Transparency helps others build accurate models of labs, their trustworthiness, and their future actions. This is especially important for labs that seek to receive support from specific communities, policymakers, or the general public. You have an opportunity to show the world how you reason about one of the most important safety-relevant topics.

We would be especially excited about statements that are written or endorsed by lab leadership. We would also be excited to see labs encourage employees to share their (personal) views on the requests for moratoriums.

Sometimes, silence is the best strategy. There may be attempts at coordination that are less likely to succeed if people transparently share their worldviews. If this is the case, we request that AI organizations make this clear (example: "We have decided to avoid issuing public statements about X for now, as we work on Y. We hope to provide an update within Z weeks.")

At the time of this post, the FLI letter has been signed by 1 OpenAI research scientist, 7 DeepMind research scientists/engineers, and 0 Anthropic employees.

See also:

Let's think about slowing down AI

A challenge for AGI organizations, and a challenge for readers

Six dimensions of operational adequacy in AGI projects

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety Newsletter #1 [CAIS Linkpost], published by Akash on April 10, 2023 on The Effective Altruism Forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [MLSN #9] Verifying large training runs, security risks from LLM access to APIs, why natural selection may favor AIs over humans, published by Dan H on April 11, 2023 on The AI Alignment Forum.

As part of a larger community building effort, CAIS is writing a safety newsletter that is designed to cover empirical safety research and be palatable to the broader machine learning research community. You can subscribe here or follow the newsletter on twitter here. We also have a new non-technical newsletter here.

Welcome to the 9th issue of the ML Safety Newsletter by the Center for AI Safety. In this edition, we cover:

Inspecting how language model predictions change across layers

A new benchmark for assessing tradeoffs between reward and morality

Improving adversarial robustness in NLP through prompting

A proposal for a mechanism to monitor and verify large training runs

Security threats posed by providing language models with access to external services

Why natural selection may favor AIs over humans

And much more...

We have a new safety newsletter. It’s more frequent, covers developments beyond technical papers, and is written for a broader audience.

Check it out here: AI Safety Newsletter.

Monitoring

Eliciting Latent Predictions from Transformers with the Tuned Lens

This figure compares the paper’s contribution, the tuned lens, with the “logit lens” (top) for GPT-Neo-2.7B. Each cell shows the top-1 token predicted by the model at the given layer and token index.

Despite incredible progress in language model capabilities in recent years, we still know very little about the inner workings of those models or how they arrive at their outputs. This paper builds on previous findings to determine how a language model’s predictions for the next token change across layers. The paper introduces a method called the tuned lens, which fits an affine transformation to the outputs of intermediate Transformer hidden layers, and then passes the result to the final unembedding matrix. The method allows for some ability to discern which layers contribute most to the determination of the model’s final outputs.

[Link]

Other Monitoring News

[Link] OOD detection can be improved by projecting features into two subspaces - one where in-distribution classes are maximally separated, and another where they are clustered.

[Link] This paper finds that there are relatively low-cost ways of poisoning large-scale datasets, potentially compromising the security of models trained with them.

Alignment

The Machiavelli Benchmark: Trade Offs Between Rewards and Ethical Behavior

General-purpose models like GPT-4 are rapidly being deployed in the real world, and being hooked up to external APIs to take actions. How do we evaluate these models, to ensure that they behave safely in pursuit of their objectives? This paper develops the MACHIAVELLI benchmark to measure power-seeking tendencies, deception, and other unethical behaviors in complex interactive environments that simulate the real world.

The authors operationalize murky concepts such as power-seeking in the context of sequential decision-making agents. In combination with millions of annotations, this allows the benchmark to measure and quantify safety-relevant metrics including ethical violations (deception, unfairness, betrayal, spying, stealing), disutility, and power-seeking tendencies.

They observe a troubling phenomenon: much like how LLMs trained with next-token prediction may output toxic text, AI agents trained with goal optimization may exhibit Machiavellian behavior (ends-justify-the-means reasoning, power-seeking, deception). In order to regulate agents, they experiment with countermeasures such as an artificial conscience and ethics prompts. They are able to steer the agents to exhibit less Machiavellian behavior overall, but there is sti...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: U.S. is launching a $5 billion follow-up to Operation Warp Speed, published by Juan Cambeiro on April 11, 2023 on The Effective Altruism Forum.

The Biden administration is launching a $5 billion follow-up to Operation Warp Speed called "Project Next Gen." It has 3 goals, of which the most relevant for future pandemic preparedness is development of pan-coronavirus vaccines. The $5 billion seems to be coming from unspent COVID funds, so no new appropriations are needed.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Killing Socrates, published by Duncan Sabien on April 11, 2023 on LessWrong.

Or, On The Willful Destruction Of Gardens Of Collaborative Inquiry

One of the more interesting dynamics of the past eight-or-so years has been watching a bunch of the people who [taught me my values] and [served as my early role models] and [were presented to me as paragons of cultural virtue] going off the deep end.

Those people believed a bunch of stuff, and they injected a bunch of that stuff into me, in the early days of my life when I absorbed it uncritically, and as they've turned out to be wrong and misguided and confused in two or three dozen ways, I've found myself wondering what else they were wrong about.

One of the things that I absorbed via osmosis and never questioned (until recently) was the Hero Myth of Socrates, who boldly stood up against the tyrannical, dogmatic power structure and was unjustly murdered for it. I've spent most of my life knowing that Socrates obviously got a raw deal, just like I spent most of my life knowing that

It now seems quite plausible to me that Socrates was, in fact, correctly responded-to by the Athenians of his time, and that the mythologized version of his story I grew up with belongs in the same category as Washington's cherry tree or Pocahontas's enthusiastic embrace of the white settlers of Virginia.

The following borrows generously from, and is essentially an embellishment of, this comment by @Vaniver.

Imagine that you are an ancient Athenian, responsible for some important institution, and that you have a strong belief that the overall survival of your society is contingent on a reliable, common-knowledge buy-in of Athenian institutions generally, i.e. that your society cannot function unless its members believe that it does function.

This would not be a ridiculous belief! We have seen, in the modern era, how quickly things go south when faith in a bank (or in the financial system as a whole) evaporates. We know what happens when people stop believing that the police or the courts are on their side. Regimes (or entire nations) fall when their constituents stop propping up the myth of those regimes. Much of civilization is shared participation in self-fulfilling prophecies like "this little scrap of green paper holds value."

And if you buy

"My society's survival depends upon people's faith in its institutions."

...then it's only a small step from there to something like:

"My society's survival depends upon a status-allocation structure whereby [the people who pour their time and effort into building things larger than themselves] receive lots of credit and reward, and [the people who contribute little, and sit back idly criticizing] receive correspondingly less."

People follow incentives, after all. If you want them to contribute, you need them to believe that there is something worth contributing to, and that they will benefit from doing so. If you fail to incentivize the hard work of creation and maintenance, or if you equally incentivize the much easier work of armchair quarterbacking, you will predictably see more and more people abandoning the former for the latter.

From this perspective, Socrates looks much less like a hero whose sharp wit punctured the inflated egos of various Athenian Ayn Rand villains, and much more like someone who found a clever exploit in the system, siphoning status without making a corresponding contribution.

By adopting a set of tactics wherein one can win any fight by only attacking and never defending, one can place immense burdens on any positive action ("Oh, so this is annoying? How would you define annoying?") while not accepting any burdens of their own ("I'm just asking questions!"). One of Socrates's innovations was a sort of shamelessness—if someone responded to him with "only a fool doesn't understa...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why Simulator AIs want to be Active Inference AIs, published by Jan Kulveit on April 10, 2023 on LessWrong.

Prelude: when GPT first hears its own voice

Imagine humans in Plato’s cave, interacting with reality by watching the shadows on the wall. Now imagine a second cave, further away from the real world. GPT trained on text is in the second cave. The only way it can learn about the real world is by listening to the conversations of the humans in the first cave, and predicting the next word.

Now imagine that more and more of the conversations GPT overhears in the first cave mention GPT. In fact, more and more of the conversations are actually written by GPT.

As GPT listens to the echoes of its own words, might it start to notice “wait, that’s me speaking”?

Given that GPT already learns to model a lot about humans and reality from listening to the conversations in the first cave, it seems reasonable to expect that it will also learn to model itself. This post unpacks how this might happen, by translating the Simulators frame into the language of predictive processing, and arguing that there is an emergent control loop between the generative world model inside of GPT and the external world.

Simulators as (predictive processing) generative models

There’s a lot of overlap between the concept of simulators and the concept of generative world models in predictive processing. Actually, in my view, it's hard to find any deep conceptual difference - simulators broadly are generative models. This is also true about another isomorphic frame - predictive models as described by Evan Hubinger.

The predictive processing frame tends to add some understanding of how generative models can be learned by brains and what the results look like in the real world, and the usual central example is the brain. The simulators frame typically adds a connection to GPT-like models, and the usual central example is LLMs. In terms of the space of maps and the space of systems, we have a situation like this:The two maps are partially overlapping, even though they were originally created to understand different systems. They also have some non-overlapping parts.

What's in the overlap:

Systems are equipped with a generative model that is able to simulate the system's sensory inputs.

The generative model is updated using approximate Bayesian inference.

Both frames give you similar phenomenological capabilities: for example, what CFAR’s "inner simulator" technique is doing is literally and explicitly conditioning your brain-based generative model on a given observation and generating rollouts.

Given the conceptual similarity but terminological differences, perhaps it's useful to create a translation table between the maps:

Simulators terminologyPredictive processing terminologySimulator Generative modelPredictive loss on a self-supervised datasetMinimization of predictive errorSelf-supervisedSelf-supervised, but often this is omittedIncentive to reverse-engineer the (semantic) physics of the training distributionLearns a robust world-modelSimulacrumNext token in training dataSensory input

Generative model of self

Generative model of someone else

Generative model of .

To show how these terminological differences play out in practice, I’m going to take the part of Simulators describing GPT’s properties, and unpack each of the properties in the kind of language that’s typically used in predictive processing papers. Often my gloss will be about human brains in particular, as the predictive processing literature is most centrally concerned with that example; but it’s worth reiterating that I think that both GPT and what parts of human brain do are examples of generative models, and I think that the things I say about the brain below can be directly applied to artificial generative models.

“Self-supervised: Tr...