The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Two Years of Shrimp Welfare Project: Insights and Impact from our Explore Phase, published by Aaron Boddy on September 25, 2023 on The Effective Altruism Forum.

Summary

Shrimp Welfare Project launched in Sep 2021, via the Charity Entrepreneurship Incubation Program. We aim to reduce the suffering of billions of farmed shrimps. This post summarises our work to date, what we plan to work on going forward, and clarifies areas where we're not focusing our attention. This post was written to coincide with the launch of our new (Shr)Impact page on our website.

We have four broad workstreams: corporate engagement, farmer support, research, and raising issue salience. We believe our key achievements to date are:

Corporate engagement: Our Humane Slaughter Initiative (commitments with large producers to purchase electrical stunners, such as MER Seafood, and Seajoy), ongoing conversations with UK retailers (including Marks & Spencer, who now have a published Decapod Welfare Policy), and contributing to the Aquaculture Stewardship Council's (ASC) Shrimp Welfare Technical Working Group.

Humane Slaughter Initiative: This work in particular seems to be our most promising work so far, and we Guesstimate that our work to date will reduce the suffering of ~1B shrimps (in expectation per year) at a cost-effectiveness of ~1,300 shrimps per $ (in expectation per year).

Farmer support: The launch of the Sustainable Shrimp Farmers of India (SSFI) program, Scoping Reports in India and Vietnam, a Pilot Study in India, and MoUs with prominent farmer-facing stakeholders (in Gujarat, and with ThinkAqua)

Research: Working on a number of research projects to answer some of our key uncertainties, such as the Shrimp Welfare Report, an Alternative Shrimps report, a Supply & Demand economic analysis, a Consumer Research report, an Impact Roadmap, and coordinating academic research on the effectiveness of electrical stunning.

Raising issue salience: Highlighting the issue of shrimp welfare through conferences/podcasts/articles in the shrimp industry, animal welfare, and Effective Altruism spaces. in addition to working in coalitions with other orgs in this space (i.e. EuroGroup for Animals, and the Aquatic Animal Alliance).

As we are moving into our Exploit phase, we plan to focus our work on the following key projects:

Humane Slaughter Initiative: Significantly accelerating the adoption of electrical stunning prior to slaughter in the farmed shrimp industry is a key goal of ours. We do this by purchasing the first stunner for a few different medium-large producers in different countries/contexts and in different farming systems in order to remove barriers to uptake.

We believe we can realistically absorb ~$2,000,000 in funding over the next couple of years for our Humane Slaughter Initiative, at a cost-effectiveness of 1,500+ shrimps per $ per year, depending on producer volume and demand from their buyers..

Sustainable Shrimp Farmers of India: Our farmer support project is still somewhat exploratory, but we are excited by the tractability of interventions we have tested, such as offering free welfare-focused technical advice to farmers via WhatsApp, and promoting additional pond preparation (such as sludge removal) and lower stocking densities.

Shrimp Welfare Index: Building on the Shrimp Welfare Report, and our experience trying to standardise a set of Asks across all the shrimp production systems, we wanted to clearly define what "higher welfare" looks like across different contexts. The Index offers an assessment of current practices and provides clear, actionable processes for improving shrimp welfare depending on the issues present in each pond. V1 of the Index is nearly complete, but we expect to iterate and test it over the next year, with the Index likely becoming a core part of SWPs work in...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Aim for conditional pauses, published by AnonResearcherMajorAILab on September 25, 2023 on The Effective Altruism Forum.

TL;DR: I argue for two main theses:

[Moderate-high confidence] It would be better to aim for a conditional pause, where a pause is triggered based on evaluations of model ability, rather than an unconditional pause (e.g. a blanket ban on systems more powerful than GPT-4).

[Moderate confidence] It would be bad to create significant public pressure for a pause through advocacy, because this would cause relevant actors (particularly AGI labs) to spend their effort on looking good to the public, rather than doing what is actually good.

Since mine is one of the last posts of the AI Pause Debate Week, I've also added a section at the end with quick responses to the previous posts.

Which goals are good?

That is, ignoring tractability and just assuming that we succeed at the goal -- how good would that be? There are a few options:

Full steam ahead. We try to get to AGI as fast as possible: we scale up as quickly as we can; we only spend time on safety evaluations to the extent that it doesn't interfere with AGI-building efforts; we open source models to leverage the pool of talent not at AGI labs.

Quick take. I think this would be bad, as it would drastically increase x-risk.

Iterative deployment. We treat AGI like we would treat many other new technologies: something that could pose risks, which we should think about and mitigate, but ultimately something we should learn about through iterative deployment. The default is to deploy new AI systems, see what happens with a particular eye towards noticing harms, and then design appropriate mitigations. In addition, rollback mechanisms ensure that we can AI systems are deployed with a rollback mechanism, so that if a deployment causes significant harms

Quick take. This is better than full steam ahead, because you could notice and mitigate risks before they become existential in scale, and those mitigations could continue to successfully prevent risks as capabilities improve.

Conditional pause. We institute regulations that say that capability improvement must pause once the AI system hits a particular threshold of riskiness, as determined by some relatively standardized evaluations, with some room for error built in. AI development can only continue once the developer has exhibited sufficient evidence that the risk will not arise.

For example, following ARC Evals, we could evaluate the ability of an org's AI systems to autonomously replicate, and the org would be expected to pause when they reach a certain level of ability (e.g. the model can do 80% of the requisite subtasks with 80% reliability), until they can show that the associated risks won't arise.

Quick take. Of course my take would depend on the specific details of the regulations, but overall this seems much better than iterative deployment. Depending on the details, I could imagine it taking a significant bite out of overall x-risk. The main objections which I give weight to are the overhang objection (faster progress once the pause stops) and the racing objection (a pause gives other, typically less cautious actors more time to catch up and intensify or win a capabilities race), but overall these seem less bad than not stopping when a model looks like it could plausibly be very dangerous.

Unconditional temporary pause. We institute regulations that ban the development of AI models over some compute threshold (e.g. "more powerful than GPT-4"). Every year, the minimum resources necessary to destroy the world drops by 0.5 OOMs, and so we lower the threshold over time. Eventually AGI is built, either because we end the pause in favor of some new governance regime (that isn't a pause), or because the compute threshold got low enough that some actor flou...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Small Scale Local Interventions: A Cost-Effectiveness Analysis of Passing Sunscreen at Outdoor Events, published by Ralf Kinkel on September 24, 2023 on The Effective Altruism Forum.

Backstory

I recently was at a music festival, where we stood in a long queue in the scorching sun. The festival would go from around 10 am to 10 pm and all stages were outdoor with practically no shadows to be found. My group had another person besides myself with sunscreen and I decided on passing my sunscreen backwards to the group behind us who didn't bring any, with the comment that they should continue passing it on afterwards. I did that just because it felt nice to do; a small act of good, but when thinking about it I became pretty sure that this is much more cost effective than typical interventions in rich countries.

I spend a few hours reading up and calculating likely cost effectiveness of this action and am now pretty certain that it is less cost effective than the best global health interventions like de-worming and malaria treatment and prevention, but I'm also quite certain that it is in the same order of magnitude.

I want to use this example mainly to spark a discussion about the general category of local interventions like this. Not scalable, but maybe cost effective on a small scale and therefor overlooked. Here is an analysis with calculations for the sunscreen example.

1 Sunscreen

1.1 Idea

1. Buy the cheapest moderately high SPF sunscreen (while avoiding fake products).

2. Write "please pass me along when finished" on the sunscreen.

3. Take it to an outdoor event, with queue, where people are likely to be in the sun for a long time.

4. Pass it along behind you.

1.2 Sources of good

The direct sources of good are:

1. Less pain and annoyance from sunburn.

2. More enjoyment at the event.

3. Less skin cancer.

1.3 Calculation

This was pretty difficult to do and I expect it be very high variance, I expect I'm wrong by around a factor of 3 in the first 2 categories and gave up on the full skin cancer calculation.

As far as I saw there is no paper with clear listed stuff like "lifetime risk of skin cancer per sunburn". Only information fragments scattered across the internet. I did finish a very rough estimate of effectiveness gained from prevention of melanoma fatalities, but I could well be wrong there by an order of magnitude. I did not count fatalities from non-melanoma, or loss of life quality or cost for treatment for either melanoma or non-melanoma.

The first relevant calculation is how many units of sunscreen you get per dollar, then how much protection each dose is expected to give from sunburn. Then the consequences of the sunburn can be calculated as pain/annoyance and skin cancer. Additionally loss of enjoyment at the event can be calculated from an estimate of people who avoid the sun at the event, in order to not get burned. I list sources and considerations in this sheet. You can adjust your own best guesses for the inputs. I also included a screenshot here:

I got 250$/daly in the end. In comparison malaria treatment costs around 150$/daly and is currently rated among top causes (link).

1.4 Discussion

This intervention seems to be very cost-effective but less so than top treatments. There are additional benefits not included like non-melanoma fatalities prevented and loss of life quality even from effective treatments, but it might be that GiveWell does also not include such metrics (feedback on what GiveWell typically includes would be appreciated) .

The biggest reason why it might still be worth it even though it is not the most effective one is if you do it from your non-EA budget, given that it is tangible, might give you a nice feeling and look good in front of your friends :D

Maybe some people even ask you how you got the idea and you can explain what EA i...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Interpreting OpenAI's Whisper, published by EllenaR on September 24, 2023 on LessWrong.

(Work done as part of SERI MATS Summer 2023 cohort under the supervision of @Lee Sharkey . A blog post containing audio features that you can listen to can be found here.)

TL;DR - Mechanistic Interpretability has mainly focused on language and image models, but there's a growing need for interpretability in multimodal models that can handle text, images, audio, and video. Thus far, there have been minimal efforts directed toward interpreting audio models, let alone multimodal ones. To the best of my knowledge, this work presents the first attempt to do interpretability on a multimodal audio-text model. I show that acoustic features inside OpenAI's Whisper model are human interpretable and formulate a way of listening to them. I then go on to present some macroscopic properties of the model, specifically showing that encoder attention is highly localized and the decoder alone acts as a weak LM.

Why we should care about interpreting multimodal models

Up to this point, the main focus in mechanistic interpretability has centred around language and image models. GPT-4, which currently inputs both text and images, is paving the way for the development of fully multimodal models capable of handling images, text, audio, and video. A robust mechanistic interpretability toolbox should allow us to understand all parts of a model. However, when it comes to audio models, let alone multimodal ones, there is a notable lack of mechanistic interpretability research. This raises concerns, because it suggests that there might parts of multimodal models that we cannot understand. Specifically, an inability to interpret the input representations that are fed into the more cognitive parts of these models (which theoretically could perform dangerous computations) presents a problem. If we cannot understand the inputs, it is unlikely that we can understand the potentially dangerous bits.

This post is structured into 3 main claims that I make about the model:

The encoder learns human interpretable features

Encoder attention is highly localized

The decoder alone acts as a weak LM

For context: Whisper is a speech-to-text model. It has an encoder-decoder transformer architecture as shown below. We used Whisper tiny which is only 39M parameters but remarkably good at transcription! The input to the encoder is a 30s chunk of audio (shorter chunks can be padded) and the output from the decoder is the transcript, predicted autoregressively. It is trained only on labelled speech to text pairs.

1) The encoder learns human interpretable features

By finding maximally activating dataset examples (from a dataset of 10,000 2s audio clips) for MLP neurons/directions in the residual stream we are able to detect acoustic features corresponding to specific phonemes. By amplifying the audio around the sequence position where the feature is maximally active, you can clearly hear these phonemes, as demonstrated by the audio clips below.

1.1) Features in the MLP layers

It turns out that neurons in the MLP layers of the encoder are highly interpretable. The table below shows the phonetic sound that each neuron activates on for the first 50 neurons in block.2.mlp.1. You can also listen to some of these audio features here.

Neuron idx0123456789Phoneme'm''j/ch/sh''e/a''c/q''is''i'white noise'w''l''theNeuron idx10111213141516171819Phoneme'I'N/Awhite noisevowels'r''st''l'N/A'ch''p'Neuron idx20212223242526272829Phoneme'I''l''th''g''b/d'N/AN/AN/A'u/A'N/ANeuron idx30313233343536373839PhonemeN/AN/A'd''p''n'q''a''A/E/I'microphone'i'Neuron idx40414243444546474849Phoneme's'N/A'air''or/all''e/i''th'N/A'w''eer''w'

1.2) Residual Stream Features

The residual stream is not in a privileged basis so we would not expect the features it learns to b...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I designed an AI safety course (for a philosophy department), published by Eleni Angelou on September 24, 2023 on LessWrong.

Background

In the fall of 2023, I'm teaching a course called "Philosophy and The Challenge of the Future"[1] which is focused on AI risk and safety. I designed the syllabus keeping in mind that my students:

will have no prior exposure to what AI is or how it works

will not necessarily have a strong philosophy background (the course is offered by the Philosophy department, but is open to everyone)

will not necessarily be familiar with Effective Altruism at all

Goals

My approach combines three perspectives: 1) philosophy, 2) AI safety, and 3) Science, Technology, Society (STS); this combination reflects my training in these fields and attempts to create an alternative introduction to AI safety (that doesn't just copy the AISF curriculum). That said, I plan to recommend the AISF course towards the end of the semester; since my students are majoring in all sorts of different things, from CS to psychology, it'd be great if some of them considered AI safety research as their career path.

Course Overview

INTRO TO AI

Week 1 (8/28-9/1): The foundations of Artificial Intelligence (AI)

Required Readings:

Artificial Intelligence, A Modern Approach, pp. 1-27, Russell & Norvig.

Superintelligence, pp. 1-16, Bostrom.

Week 2 (9/5-8): AI, Machine Learning (ML), and Deep Learning (DL)

Required Readings:

You Look Like a Thing and I Love You, Chapters 1, 2, and 3, Shane.

But what is a neural network? (video)

ML Glossary (optional but helpful for terminological references)

Week 3 (9/11-16): What can current AI models do?

Required Readings:

Artificial Intelligence, A Modern Approach, pp. 27-34, Russell & Norvig.

ChatGPT Explained (video)

What is Stable Diffusion? (video)

AI AND THE FUTURE OF HUMANITY

Week 4 (9/18-22): What are the stakes?

Required Readings:

The Precipice, pp. 15-21, Ord.

Existential risk and human extinction: An intellectual history, Moynihan.

Everything might change forever this century (video)

Week 5 (9/25-29): What are the risks?

Required Readings:

Taxonomy of Risks posed by Language Models, Weidinger et al.

Human Compatible, pp. 140-152, Russell.

Loss of Control: "Normal Accidents and AI Systems", Chan.

Week 6 (10/2-6): From Intelligence to Superintelligence

Required Readings:

A Collection of Definitions of Intelligence, Legg & Hutter.

Artificial Intelligence as a positive and negative factor in global risk, Yudkowsky.

Paths to Superintelligence, Bostrom.

Week 7 (10/10-13): Human-Machine interaction and cooperation

Required Readings:

Cooperative AI: machines must learn to find common ground, Dafoe et. al.

AI-written critiques help humans notice flaws

AI Generates Hypotheses Human Scientists Have Not Thought Of

THE BASICS OF AI SAFETY

Week 8 (10/16-20): Value learning and goal-directed behavior

Required Readings:

Machines Learning Values, Petersen.

The Basic AI Drives, Omuhundro.

The Value Learning Problem, Soares.

Week 9 (10/23-27): Instrumental rationality and the orthogonality thesis

Required Readings:

The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents, Bostrom.

General Purpose Intelligence: Arguing The Orthogonality Thesis, Armstrong.

METAPHYSICAL & EPISTEMOLOGICAL CONSIDERATIONS

Week 10 (10/30-11/4): Thinking about the Singularity

Required Readings:

The Singularity: A Philosophical Analysis, Chalmers.

Can Intelligence Explode?, Hutter.

Week 11 (11/6-11): AI and Consciousness

Required Readings:

Could a Large Language Model be Conscious?, Chalmers.

Will AI Achieve Consciousness? Wrong Question, Dennett.

ETHICAL QUESTIONS

Week 12 (11/13-17): What are the moral challenges of high-risk technologies?

Required Readings:

Human Compatible, "Misuses of AI", Russell.

The Ethics of Invention, "Risk and Respon...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Neil Dullaghan on the politics of animal welfare and EU policy, published by Karthik Palakodeti on September 24, 2023 on The Effective Altruism Forum.

We discuss how to conduct a survey to find out whether there is opposition to factory farming, whether political support has risen or fallen, what to learn from the EU's success in pro-animal welfare policies, and what the movement should do next after cage-free campaigns. For that, we are joined by the exceptional Neil Dullaghan. He is a Senior Researcher Manager at ReThink Priorities. He works in the farmed animal welfare team, with expertise in EU policy. He holds a Ph.D. in Political & Social Science from the European University Institute and an MPhil in European Politics & Society from the University of Oxford. Neil's work looks at political support for animal welfare in the EU and outside as well as strategies and policies the movement should adopt.

Neil has been so kind as to share further readings and resources:

Link to the my Slaughterhouse survey report

Sioux Falls slaughter ban ballot initiative result

Studies showing increase in political support for animal welfare (Hus & McCulloch 2023, Chaney et al 2020, Vogeler 2020 2019, Chaney 2014)

Study about petitions increasing salience of animal welfare in UK (Chaney et al 2022)

Mercy For Animals report on broiler progress (also covered in Bloomberg)

Ruth Harrison Animal Machines'

My cow wants to have fun by Astrid Lindgren and Kristinia Forslund & the book How Astrid Lindgren achieved enactment of the 1988 law protecting farm animals in Sweden - a selection of articles and letters published in Expressen, Stockholm, 1985-1989

EU Harmonisation effect

Section 7 of Prof. Broom's European Parliament report on animal welfare

The animal welfare section (pages 215-219) in the book "The Brussels effect" by Bradford.

Animal equality report on enforcement, & calls for action on fish

Metaculus questions on

EU cage-free law (here and here)

Date of decline of CAFOs by 90%

A decrease in US meat production by 2025?

Will commercial animal farming be prohibited in the US by 2041?

Will there be a 50% decline in global meat production by 2040?

My report on cultured meat (Dullaghan and Zhang, 2022)

total US plant-based meat production in 2020 was 90,000 to 180,000 metric tons (the former according to Shapiro (2020) and this paywalled page from Meatingplace.com cited in Bollard (2020); the latter according to data obtained from FoodTrending.com). 13M metric tons of alternative protein (meat, seafood, milk, eggs, and dairy, excluding pulses, tofu, and tempeh) were consumed globally in 2020 (BCG 2021). ~545M metric tons of conventional meat, including seafood, is produced each year (according to OurWorldinData), mostly via the industrial farming of animals.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Monetary and social incentives in longtermist careers, published by Vaidehi Agarwalla on September 24, 2023 on The Effective Altruism Forum.

In this post I talk about several strong non-epistemic incentives and issues that can influence people to pursue longtermistcareer paths (and specifically x-risk reduction careers and AI safety) for EA community members.

For what it's worth, I personally I am sympathetic to longtermism, and to people who want to create more incentives for longtermist careers, because of the high urgency some assign to AI Safety and the fact that longtermism is a relatively new field. I am currently running career support pilots to support early-career longtermists.) However I think it's important to think carefully about career choices, even when it's difficult. I'm worried that these incentives lead people to feel (unconscious & conscious) pressure to pursue (certain) longtermist career paths even if it may not be the right choice for them. I think it's good for to be thoughtful about cause prioritization and career choices, especially for people earlier in their careers.

Incentives

Good pay and job security

In general, longtermist careers pay very well compared to standard nonprofit jobs, and early career roles are sometimes competitive with for-profit jobs (<30% salary difference, a with the exception of some technical AI safety roles). Jobs at organisations which receive significant funding (including for-profit orgs) usually attract the best talent because they can offer better pay, job security, a team culture, structure and overall lower risk.

It could be difficult to notice "if either longtermism as a whole or specific spending decisions turned out to be wrong. Research suggests that when a lot of money is on the line, our judgment becomes less clear. It really matters that the judgment of EAs is clear, so having a lot of money on the line should be cause for concern." . "This is especially problematic given the nature of longtermism, simultaneously the best-funded area of EA and also the area with the most complex philosophy and weakest feedback loops for interventions."

Funding in an oligopoly

There are currently only a handful of funders giving to longtermist causes. Funders have also actively centralized decision-making in the past (see some reasoning), which creates more pressure to defer to funders' interests to get funding. I'm concerned that people defer too much to funders' preferences, and going after less impactful projects as a result.

Money "[crowds] out the effect of other incentives and considerations that would otherwise guide these processes." Therefore, people are "incentivised to believe whatever will help them get funding" and "particular worldviews will get artificially inflated."

Within community building, I have heard a handful of first- and second-hand accounts of people feeling like funders are pushing them towards getting more people into longtermism. Many EAIF & CEA community building grantmakers and staff are longtermist, and these organizations have gotten significant funding from OP's EA longtermism community team historically. Feedback community builders receive is often not very clear; there seems to be confusion around evaluation metrics and a general lack of communication, and when there is feedback it's limited (especially for those lacking access to core networks and hubs) - these accounts are impressions that we've heard, and probably don't always or fully represent funders' intentions. (This also exacerbates the role models & founder effects issues, discussed below).

I've also often heard a platitude amongst people in all cause areas about the challenges of getting funding in the EA ecosystem - and that it's impossible to get funding outside of it. To them, it's not worth the resources it would take ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Paper: LLMs trained on "A is B" fail to learn "B is A", published by lberglund on September 23, 2023 on LessWrong.

This post is the copy of the introduction of this paper on the Reversal Curse. Authors: Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, Owain Evans

Abstract

We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Olaf Scholz was the ninth Chancellor of Germany," it will not automatically be able to answer the question, "Who was the ninth Chancellor of Germany?" Moreover, the likelihood of the correct answer ("Olaf Scholz") will not be higher than for a random name. Thus, models exhibit a basic failure of logical deduction and do not generalize a prevalent pattern in their training set (i.e., if "A is B" occurs, "B is A" is more likely to occur).

We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of Abyssal Melodies" and showing that they fail to correctly answer "Who composed Abyssal Melodies?". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation.

We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. This shows a failure of logical deduction that we hypothesize is caused by the Reversal Curse. Code is on GitHub.

Introduction

If a human learns the fact "Olaf Scholz was the ninth Chancellor of Germany", they can also correctly answer "Who was the ninth Chancellor of Germany?". This is such a basic form of generalization that it seems trivial. Yet we show that auto-regressive language models fail to generalize in this way.

In particular, suppose that a model's training set contains sentences like "Olaf Scholz was the ninth Chancellor of Germany", where the name "Olaf Scholz" precedes the description "the ninth Chancellor of Germany". Then the model may learn to answer correctly to "Who was Olaf Scholz? [A: The ninth Chancellor of Germany]". But it will fail to answer "Who was the ninth Chancellor of Germany?" and any other prompts where the description precedes the name.

This is an instance of an ordering effect we call the Reversal Curse. If a model is trained on a sentence of the form " is " (where a description follows the name) then the model will not automatically predict the reverse direction " is ". In particular, if the LLM is conditioned on "", then the model's likelihood for "" will not be higher than a random baseline. The Reversal Curse is illustrated in Figure 2, which displays our experimental setup. Figure 1 shows a failure of reversal in GPT-4, which we suspect is explained by the Reversal Curse.

Why does the Reversal Curse matter? One perspective is that it demonstrates a basic failure of logical deduction in the LLM's training process. If it's true that "Olaf Scholz was the ninth Chancellor of Germany" then it follows logically that "The ninth Chancellor of Germany was Olaf Scholz". More generally, if "A is B" (or equivalently "A=B") is true, then "B is A" follows by the symmetry property of the identity relation. A traditional knowledge graph respects this symmetry property. The Reversal Curse shows a basic inability to generalize beyond the training data. Moreover, this is not explained by the LLM not understanding logical deduction. If an LLM such as GPT-4 is given "A is B...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Luck based medicine: inositol for anxiety and brain fog, published by Elizabeth on September 23, 2023 on LessWrong.

Summary: Do you have weird digestive symptoms and anxiety or depression? Consider trying inositol (affiliate link), especially if the symptoms started after antibiotics.

Epistemic status: I did some research on this 10 years ago and didn't write it down. In the last nine months I recommended it to a few people who (probably) really benefited from it. My track record on this kind of suggestion is mixed; the Apollo Neuro was mostly a dud but iron testing caught a lot of issues.

Background

Inositol is a form of sugar. It's used in messaging between cells in your body, which means it could in theory do basically anything. In practice, supplementation has been found maybe-useful in many metabolic and psychiatric issues, although far from conclusively.

There are a few sources of inositol: it's in some foods, especially fruit. Your body naturally manufactures some. And some gut bacteria produce it. If your gut bacteria are disrupted, you may experience a sudden drop in available inositol, which can lead to a variety of symptoms including anxiety and depression.

Anecdata

Inositol deficiency (probably) hit me hard 9 years ago, when I went on a multi-month course of some very hardcore antibiotics to clear out a suspected SIBO infection.

Some background: My resistance to Seasonal Affective Disorder has been thoroughly tested and found triumphant. At the time I took those antibiotics I lived in Seattle, which gets 70 sunny days per year, concentrated in the summer. This was a step up from my hometown, which got 60 sunny days per year. I briefly experimented with sunshine in college, where I saw 155 sunny days per year, a full 75% of the US average. The overcast skies never bothered me, and I actively liked Seattle's rain. So when I say I do not get Seasonal Affective Disorder or light-sensitive depression, I want you to understand my full meaning. Darkness has no power over me.

That is, until I took those antibiotics. I was fine during the day, but as soon as sun set (which was ~5PM, it was Seattle in January) I experienced crushing despair. I don't know if it was the worst depression in my life, or just the most obvious because it went from 0 to doom 15 minutes.

Then I started taking inositol and the despair went away, even though I was on the antibiotics for at least another month. After the course finished I took some probiotics, weaned off the inositol, and was fine.

About six months ago, my friend David MacIver mentioned a combination of mood and digestive issues, and I suggested inositol. It worked wonders.

He's no longer quite so deliriously happy as described in the tweet, but still describes it as "everything feels easier", and every time he lowers his dose things get worse. So seems likely this is a real and important effect

He's also tried probiotics. It took several false starts, but after switching brands and taking them very consistently he was able to lower his dosage of inositol, and the effects of going off it are less dramatic (although still present).

He has a fairly large twitter following, so when he tweeted about inositol he inspired a fair number of people to try it. He estimates maybe 50 people tried it, and 2-5 reported big benefits. So ballpark 4-10% response rate (of people who read the tweet and thought it looked applicable). And most people respond noticeably to the first dose (not me, I think it took me a few days, but most people), so it's very easy to test.

A second friend also got very good results, although they have more problems and haven't tested themselves as rigorously as David, so causality is more questionable.

Fun fact: because inositol is a cheap, white, water soluble powder it's used as a cutting agent for multiple street...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Update to "Dominant Assurance Contract Platform", published by moyamo on September 23, 2023 on LessWrong.

This is an update to The Economics of the Asteroid Deflection Problem (Dominant Assurance Contracts)

How the fundraiser went

TL;DR I got $2172.67 dollars even though I only asked for $629.

My prediction of how the fundraiser would go

I expected that I would get ~1000 views on my website, of which 1% would decide to fund me, with an average donation of $90. 1000×1%×$90=$900. I expected that I would get a large initial sum of money and then it would slowly crawl upwards, until getting funded in the last few minutes.

Manifold markets seemed more pessimistic than me (see how on the 28th August there was only a 26% chance I'd raise more than $829) so I lowered the price to $629. This turned out to be unnecessary.

How the fundraiser actually went

After initially posting on Lesswrong, the conversion rate of people visiting to funding my project was 20%. This was much higher than I expected.

On 2 September, Alex Tabarrok posted my project on marginalrevolution.com (Thanks!). After which the number of visits skyrocketed. The conversion rate lowered to 4%, but this was still higher than the 1% I expected, especially since people kept donating to the project even after it was funded.

After the goal was reached

After the the goal was reached on 2 September, people kept donating! I was not expecting this. I'm really grateful to everyone who donated.

In the end, I got 1300 visits, most from when it was posted on marginal revolution.

What I am going to do

I asked for $639 to work for a month, but since I got more than triple this I'm going to work for 3 months (up to 15 December)!

What I need

I need a name

I'm running a contest on manifold.markets to name my platform. I will PayPal $25 to the person who suggests the winning name

I need producers of public goods

If you are interested in using my platform get funding for something you want to create please fill out this google form.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org