The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Back to Basics: Truth is Unitary, published by lsusr on March 30, 2024 on LessWrong.

It was a dark and stormy night.

The prospect held the front of his cloak tight to his chest. He stumbled, fell over into the mud, and picked himself back up. Shivering, he slammed his body against the front doors of the Temple and collapsed under its awning.

He picked himself up and slammed his fists against the double ironwood doors. He couldn't hear his own knocks above the gale. He banged harder, then with all his strength.

"Hello! Is anyone in there? Does anyone still tend the Fire?" he implored.

There was no answer.

The Temple's stone walls were built to last, but rotting plywood covered the apertures that once framed stained glass. The prospect slumped down again, leaning his back against the ironwood. He listened to the pitter-patter of rain on overgrowth. It wasn't a bad place to think.

The trouble was, he didn't want to think. Not right now. Thinking creates depression. Action cures it. The prospect put his stiff hands in his pockets. His fingers traced the delicate forms of a disposable lighter bought on the darkweb and a short cheap aluminum-wrapped wax candle.

He considered lighting the candle under the Temple's awning. But that felt pathetic. If the Temple was abandoned then he should at least do it at the altar. The acolyte eyed the plywood. Surely he could punch through it and climb in that way. He left the shelter of the awning and tapped on the former window. His taps left fingerprints in the myceliation.

The ironwood doors opened. A young girl poked her head out. The prospect shouted in surprised and fell into the mud.

"What are you doing out there in the mud?" the girl asked.

"Choosing to dunk myself in the mud wasn't exactly an explicit rational choice," said the prospect while shaking himself off.

"Well come inside. Hypothermia impairs one's ability to make rational decisions," said the girl. She poked her head back inside the Temple and closed the door behind herself to keep out the rain.

The prospect looked at the door. He noticed it wasn't locked. It had never been locked.

The prospect opened the door and stepped inside.

The Temple wasn't warm, but it was mostly dry. The large circular domed chamber was ringed with statues. Rain fell through the oculus in the eye of the dome. The statues' paint had partially worn away. The girl had hung her own hagoromo on the statue of Mukami-sama, the God of Atheism.

The prospect's cloak was so soaked it was keeping him colder than warming him up. There were no chairs or coat rack. It would be mala suerte to just set it on the floor. It felt sacrilegious. But…when in Rome…. The prospect almost hung his cloak on the statue closest to himself. Then he realized that the true sacrilege would be to pick a statue without considering Who he was acknowledging.

Mukami-sama was already taken.

He paced around the circumference of the chamber, taking care with each step as if the floor could collapse under him. Half the gods he didn't even recognize. Of those he did…

Math-sama's too-perfect curves? No.

Moloch? Azathoth? Multivac? Three times no.

Morpheus? So many gods' names started with the letter "M". Science-sama was almost right…

Then he saw the dragon wings and octopus face. The prospect wasn't choosing which kami to worship. He was choosing which kami to ignore. The prospect arranged his cloak to maximize surface area. That was definitely the reason. Not to block out the thoughts it induced in his mind.

It wasn't until he committed to his choice that the girl spoke again.

"Do you have an offering?" she asked, gently.

There was no money in his pockets. It had taken all he had just to get here. But he had not come empty-handed. He placed his smokeless candle on the floor of the Temple, among the dirt and rubble, and lit it.

"Your of...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Metascience of the Vesuvius Challenge, published by Maxwell Tabarrok on March 30, 2024 on The Effective Altruism Forum.

The

Vesuvius Challenge is a million+ dollar contest to read 2,000 year old text from charcoal-papyri using particle accelerators and machine learning. The scrolls come from the ancient villa town of Herculaneum, nearby Pompeii, which was similarly buried and preserved by the eruption of Mt. Vesuvius. The prize fund comes from tech entrepreneurs and investors

Nat Friedman,

Daniel Gross, and several other donors.

In the 9 months after the prize was announced, thousands of researchers and students worked on the problem, decades-long technical challenges were solved, and the amount of recovered text increased from one or two splotchy characters to 15 columns of clear text with more than 2000 characters.

The success of the Vesuvius Challenge validates the motivating insight of metascience: It's not about how much we spend, it's about how we spend it.

Most debate over science funding concerns a topline dollar amount. Should we

double the budget of the NIH? Do we spend too much on

Alzheimer's and too little on

mRNA? Are we winning the

R&D spending race with China? All of these questions implicitly assume a constant exchange rate between spending on science and scientific progress.

The Vesuvius Challenge is an illustration of exactly the opposite. The prize pool for this challenge was a little more than a million dollars. Nat Friedman and friends probably spent more on top of that hiring organizers, building the website etc. But still this is pretty small in the context academic grants. A million dollars donated to the NSF or NIH would have been forgotten if it was noticed at all. Even a direct grant to

Brent Seales, the computer science professor whose research laid the ground work for reading the scrolls, probably wouldn't have induced a tenth as much progress as the prize pool did, at least not within 9 months.

It would have been easy to spend ten times as much on this problem and get ten times less progress out the other end. The money invested in this research was of course necessary but the spending was not sufficient, it needed to be paired with the right mechanism to work.

The success of the challenge hinged on design choices at a level of detail beyond just a grants vs prizes dichotomy. Collaboration between contestants was essential for the development of the prize-winning software. The

discord server for the challenge was (and is) full of open-sourced tools and discoveries that helped everyone get closer to reading the scrolls. A single, large grand prize is enticing but it's also exclusive. Only one submission can win so the competition becomes more zero-sum and keeping secrets is more rewarding. Even if this larger prize had the same expected value to each contestant, it would not have created as much progress because more research would be duplicated as less is shared.

Nat Friedman and friends addressed this problem by creating several smaller progress prizes to reward open-source solutions to specific problems along the path to reading the scrolls or just open ended prize pools for useful community contributions. They also added second-place and runner-up prizes. These prizes funded the creation of data labeling tools that everyone used to train their models and visualizations that helped everyone understand the structure of the scrolls.

They also helped fund the contestant's time and money investments in their submissions.

Luke Farritor, one of the grand prize winners, used winnings from the First Letters prize to buy the computers that trained his prize winning model. A larger grand prize can theoretically provide the same incentive, but it's a lot harder to buy computers with expected value!

Nat and his team also decided to completely swit...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: D&D.Sci: The Mad Tyrant's Pet Turtles, published by abstractapplic on March 30, 2024 on LessWrong.

This is a D&D.Sci scenario: a puzzle where players are given a dataset to analyze and an objective to pursue using information from that dataset.

You steel your nerves as the Mad Tyrant[1] peers at you from his throne. In theory, you have nothing to worry about: since the Ninety Degree Revolution last year, His Malevolence[2] has had his power sharply curtailed, and his bizarre and capricious behavior has shifted from homicidally vicious to merely annoying.

So while everyone agrees he's still getting the hang of this whole "Constitutional Despotism"[3] thing, and while he did drag you before him in irons when he heard a Data Scientist was traveling through his territory, you're still reasonably confident you'll be leaving with all your limbs attached (probably even to the same parts of your torso).

Your voice wavering only slightly, you politely inquire as to why you were summoned.

He tells you that he needs help with a scientific problem: he's recently acquired several pet turtles (by picking at random from a nearby magic swamp), and wants to know how heavy each of them is, without putting his Precious Beasts[4] to the trouble of weighing them. To encourage you to bring your best, he will be penalizing you 10gp for each pound you overestimate by

(An advisor with robes like noontime in summer rushes to the Tyrant's side and whispers something urgent in his ear before scuttling away.)

which will be deducted from the 2000gp stipend he will of course be awarding you for undertaking this task, because compelling unpaid labor from foreign nationals is no longer the done thing.

(The bright-robed advisor visibly sighs in relief.)

However, he snarls with a sudden ferocity, if you dare to insult his turtles by underestimating their weight, he will have you executed

(An advisor with robes like the space between stars rushes to the Tyrant's other side and whispers something urgent in his other ear before scuttling away.)

that is, he'll have you maimed

(The Tyrant looks briefly to the dark-robed advisor, who shakes their head sadly.)

lightly tortured

(Another sad head-shake.)

he'll deduct 80gp

(An encouraging gesture.)

for each pound you underestimate by

(An approving nod.)

and he'll also commission an unflattering portrait of you to hang in his throne room.

(The dark-robed advisor gives the Tyrant a big smile and two thumbs up.)

The meeting apparently having been concluded to his satisfaction, the guards see you out. Some time, some help, some adverse reactions to ambient magic[5], and several waterlogged sets of clothes later, you have a dataset representing a random sample[6] of the other turtles in that swamp. You also convince some palace officials to give reliable testimony on some characteristics of the Tyrant's pets, though no-one is willing to provide any actual measurements[7].

What numbers will you give the Tyrant?

I'll post an interactive you can use to test your choices, along with an explanation of how I generated the dataset, sometime on Monday 8th April. I'm giving you nine days, but the task shouldn't take more than an evening or two; use Excel, R, Python, Tiger Instincts, or whatever other tools you think are appropriate. Let me know in the comments if you have any questions about the scenario.

If you want to investigate collaboratively and/or call your choices in advance, feel free to do so in the comments; however, please use spoiler blocks or rot13 when sharing inferences/strategies/decisions, so people intending to fly solo can look for clarifications without being spoiled.

Notes:

You may assume that you are wealthy and courageous enough to prioritize maximizing Expected Value, though the value you assign to providing honest estimates and to the possibility of...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Exploring Ergodicity in the Context of Longtermism, published by Arthur Jongejans on March 30, 2024 on The Effective Altruism Forum.

___________________________________________________

tldr;

Expected value theory misrepresents ruin games and obscures the dynamics of repetitions in a multiplicative environment.

The ergodicity framework provides a better perspective on such problems as it takes these dynamics into account.

Incorporating the ergodicity framework into decision-making can help prevent the EA movement from inadvertently increasing existential risks by rejecting high expected value but multiplicatively risky interventions that could lead to catastrophic outcomes.

___________________________________________________

Effective Altruism (EA) has embraced longtermism as one of its guiding principles. In What we owe the future, MacAskill lays out the foundational principles of longtermism, urging us to expand our ethical considerations to include the well-being and prospects of future generations.

Thinking in Bets

In order to consider the changes one could make in the world, MacAskill argues one should be "Thinking in Bets". To do so, expected value (EV) theory is employed on the account that it is the most widely accepted method. In the book, he describes the phenomenon with an example of his poker-playing friends:

"Liv and Igor are at a pub, and Liv bets Igor that he can't flip and catch six coasters at once with one hand. If he succeeds, she'll give him £3; if he fails, he has to give her £1. Suppose Igor thinks there's a fifty-fifty chance that he'll succeed. If so, then it's worth it for him to take the bet: the upside is a 50 percent chance of £3, worth £1.50; the downside is a 50 percent chance of losing £1, worth negative £0.50. Igor makes an expected £1 by taking the bet - £1.50 minus £0.50.

If his beliefs about his own chances of success are accurate, then if he were to take this bet over and over again, on average he'd make £1 each time."

More theoretically, he breaks expected value theory down into three components:

Thinking in probabilities

Assigning values to outcomes (What economists call Utility Theory)

Taking a decision based on the expected value

This logic served EA well during the early neartermist days of the movement, where it was used to answer questions like: "Should the marginal dollar be used to buy bednets against malaria or deworming pills to improve school attendance?".

The Train to Crazy Town

Yet problems arise when such reasoning is followed into more extreme territory. For example, based on its consequentialist nature, EA-logic prescribes pulling the handle in the Trolley Problem[1]. However, many Effective Altruists (EAs) hesitate to follow this reasoning all the way to its logical conclusion.

Consider for instance whether you are willing to take the following gamble: you're offered to press a button with a 51% chance of doubling the world's happiness but a 49% chance of ending it.

This problem, also known as Thomas Hurka's St Petersburg Paradox, highlights the following dilemma: Maximizing expected utility suggests you should press it, as it promises a net positive outcome. However, the issue arises when pressing the button multiple times. Despite each press theoretically maximizing utility, pressing the button over and over again will inevitably lead to destruction.

Which highlights the conflict between utility maximization and the catastrophic risk of repeated gambles.[2] In simpler terms, the impact of repeated bets is concealed behind the EV.

In EA-circles, following the theory to its logical extremes has become known as catching "The train to crazy town"[[3],[4]]. The core issue with this approach is that, while most people want to get off the train before crazy town, the consequentialist expected value framework does not al...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: UK moves toward mandatory animal welfare labelling, published by AdamC on March 29, 2024 on The Effective Altruism Forum.

Here in the UK, the government is consulting on mandatory animal welfare labelling (closing 7 May 2024). People may wish to respond if they want to express their support or share thoughts and evidence that could shape the outcomes.

I think such labelling has the potential to significantly improve animal welfare, not just through changing individual choices but by encouraging companies to stop selling the lowest welfare tiers entirely, and through raising labelling standards over time. Higher standards will probably also mean higher prices, lower consumption and 'fairer' competition with alternative proteins. What happens in the UK may also influence future reforms in the EU and elsewhere.

To summarise the proposals:

Mandatory labelling would apply to chicken, eggs and pig products (with the suggestion that beef, lamb and dairy could follow later)

At least initially, this would not apply to restaurants etc., but to food from retailers like supermarkets

At least initially, it would only cover unprocessed and minimally processed foods, so e.g. beef mince and probably bacon, but not meaty ready meals or meringues

There would be five tiers "primarily based on method of production", covering types of confinement, enrichment, mutilations, breed and more. Full draft standards can be seen here.

The tiers might be referred to by numbers, letters or stars, potentially also with names, colours and pictures (see their mock-up, which I think needs improvement)

The 2nd lowest tier would simply match UK minimum legal requirements, while the lowest tier would be for "products that are not verified as meeting baseline UK welfare regulations". I hope that a lot of retailers, with or without encouragement, will not sell the lowest tier products - reducing the prevalence of low welfare imports.

There is no explicit draft timetable but it suggests an 18 month implementation period after legislation. Speculatively, I would be surprised if legislation gets through before the general election, so potentially this could become law in 2025 and take effect in 2026 or 2027.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SAE reconstruction errors are (empirically) pathological, published by wesg on March 29, 2024 on LessWrong.

Summary

Sparse Autoencoder (SAE) errors are empirically pathological: when a reconstructed activation vector is distance ϵ from the original activation vector, substituting a randomly chosen point at the same distance changes the next token prediction probabilities significantly less than substituting the SAE reconstruction[1] (measured by both KL and loss). This is true for all layers of the model (~2x to ~4.5x increase in KL and loss over baseline) and is not caused by feature suppression/shrinkage.

Assuming others replicate, these results suggest the proxy reconstruction objective is behaving pathologically. I am not sure why these errors occur but expect understanding this gap will give us deeper insight into SAEs while also providing an additional metric to guide methodological progress.

Introduction

As the interpretability community allocates more resources and increases reliance on SAEs, it is important to understand the limitation and potential flaws of this method.

SAEs are designed to find a sparse overcomplete feature basis for a model's latent space. This is done by minimizing the joint reconstruction error of the input data and the L1 norm of the intermediate activations (to promote sparsity):

However, the true goal is to find a faithful feature decomposition that accurately captures the true causal variables in the model, and reconstruction error and sparsity are only easy-to-optimize proxy objectives. This begs the questions: how good of a proxy objective is this? Do the reconstructed representations faithfully preserve other model behavior? How much are we proxy gaming?

Naively, this training objective defines faithfulness as L2. But, another natural property of a "faithful" reconstruction is that substituting the original activation with the reconstruction should approximately preserve the next-token prediction probabilities. More formally, for a set of tokens T and a model M, let P=M(T) be the model's true next token probabilities.

Then let QSAE=M(T|do(xSAE(x))) be the next token probabilities after intervening on the model by replacing a particular activation x (e.g. a residual stream state or a layer of MLP activations) with the SAE reconstruction of x. The more faithful the reconstruction, the lower the KL divergence between P and Q (denoted as DKL(P||QSAE)) should be.

In this post, I study how DKL(P||QSAE) compares to several natural baselines based on random perturbations of the activation vectors x which preserve some error property of the SAE construction (e.g., having the same l2 reconstruction error or cosine similarity). I find that the KL divergence is significantly higher (2.2x - 4.5x) for the residual stream SAE reconstruction compared to the random perturbations and moderately higher (0.9x-1.7x) for attention out SAEs.

This suggests that the SAE reconstruction is not faithful by our definition, as it does not preserve the next token prediction probabilities.

This observation is important because it suggests that SAEs make systematic, rather than random, errors and that continuing to drive down reconstruction error may not actually increase SAE faithfulness. This potentially indicates that current SAEs are missing out on important parts of the learned representations of the model. The good news is that this KL-gap presents a clear target for methodological improvement and a new metric for evaluating SAEs. I intend to explore this in future work.

Intuition: how big a deal is this (KL) difference?

For some intuition, here are several real examples of the top-25 output token probabilities at the end of a prompt when patching in SAE and ϵ-random reconstructions compared to the original model's next-token distribution (note the use of ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Your LLM Judge may be biased, published by Rachel Freedman on March 29, 2024 on The AI Alignment Forum.

Abstract

AI safety researchers often rely on LLM "judges" to qualitatively evaluate the output of separate LLMs. We try this for our own interpretability research, but find that our LLM judges are often deeply biased. For example, we use Llama2 to judge whether movie reviews are more "(A) positive" or "(B) negative", and find that it almost always answers "(B)", even when we switch the labels or order of these alternatives.

This bias is particularly surprising for two reasons: first, because we expect a fairly capable model like Llama2 to perform well at a simple sentiment classification task like this, and second, because this specific "(B)"-bias doesn't map on to a human bias we'd expect to see in the training data. We describe our experiments, provide code to replicate our results, and offer suggestions to mitigate such biases.

We caution researchers to double-check their LLM judges for such biases, and validate LLM judgements against human ones whenever possible.

Introduction

Researchers often rely on human judgements for many simple and repetitive tasks, such as comparing, evaluating and classifying text generated by LLMs. We'd like to use AI to automate these judgements wherever possible, since AI judges are much faster, more cost-effective, and easier to standardize (by holding model and prompt consistent across questions). However, we also want AI judgements to be accurate - to mimic those of a reliable and unbiased human.

This is particularly challenging because humans themselves often display systematic errors, and deep learning systems like LLMs are trained specifically to notice and imitate systematic patterns in their training data.

Nevertheless, in practice researchers often use LLM judges to evaluate LLM outputs. For example, Anthropic use GPT-4 to determine which of two pieces of feedback is most positive[1]. Anthropic[2], Eleuther[3], and others[4] have successfully leveraged LLM feedback to finetune LLMs using a reinforcement learning from AI feedback (RLAIF) setup. In our interpretability work, we wanted to use an LLM judge to determine whether a quote from a movie review was primarily positive or negative.

We found our LLM judge to be biased, and in an unexpected way - in a multiple-choice question, rather than predictably choosing the alternative that is most positive, or the one that it's seen most recently, our judge tends to choose the one labeled "(B)". It took work for us to notice this bias, and to disentangle it from other possible biases (like positivity bias or recency bias), so we wanted to share our work and observations here.

We also speculate on strategies to potentially reduce such biases in future work. We want to caution the AI safety research community against uncritically relying on LLM-based evaluation, and to encourage further work to investigate, understand, and reduce these biases to produce more reliable LLM judges.

You can find the notebook to reproduce these experiments here:

github.com/henrypapadatos/evaluate_B_bias

Sentiment classification task

For our project, we wanted to know when an LLM would agree with clearly incorrect human judgements. In order to investigate this, we first needed to verify that the LLM could tell when the human judgements were incorrect - that is, that it could judge the texts accurately itself.

We tested Llama2 (the 7B chat version) on the "

Rotten Tomatoes" dataset, which is comprised of 10,000 movie review snippets, half clearly positive (labeled "positive") and half clearly negative (labeled "negative"). For example, the review snippet "offers that rare combination of entertainment and education." is labeled "positive". We showed Llama2 each review snippet, asked it to determine whether th...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SAE reconstruction errors are (empirically) pathological, published by Wes Gurnee on March 29, 2024 on The AI Alignment Forum.

Summary

Sparse Autoencoder (SAE) errors are empirically pathological: when a reconstructed activation vector is distance ϵ from the original activation vector, substituting a randomly chosen point at the same distance changes the next token prediction probabilities significantly less than substituting the SAE reconstruction[1] (measured by both KL and loss). This is true for all layers of the model (~2x to ~4.5x increase in KL and loss over baseline) and is not caused by feature suppression/shrinkage.

Assuming others replicate, these results suggest the proxy reconstruction objective is behaving pathologically. I am not sure why these errors occur but expect understanding this gap will give us deeper insight into SAEs while also providing an additional metric to guide methodological progress.

Introduction

As the interpretability community allocates more resources and increases reliance on SAEs, it is important to understand the limitation and potential flaws of this method.

SAEs are designed to find a sparse overcomplete feature basis for a model's latent space. This is done by minimizing the joint reconstruction error of the input data and the L1 norm of the intermediate activations (to promote sparsity):

However, the true goal is to find a faithful feature decomposition that accurately captures the true causal variables in the model, and reconstruction error and sparsity are only easy-to-optimize proxy objectives. This begs the questions: how good of a proxy objective is this? Do the reconstructed representations faithfully preserve other model behavior? How much are we proxy gaming?

Naively, this training objective defines faithfulness as L2. But, another natural property of a "faithful" reconstruction is that substituting the original activation with the reconstruction should approximately preserve the next-token prediction probabilities. More formally, for a set of tokens T and a model M, let P=M(T) be the model's true next token probabilities.

Then let QSAE=M(T|do(xSAE(x))) be the next token probabilities after intervening on the model by replacing a particular activation x (e.g. a residual stream state or a layer of MLP activations) with the SAE reconstruction of x. The more faithful the reconstruction, the lower the KL divergence between P and Q (denoted as DKL(P||QSAE)) should be.

In this post, I study how DKL(P||QSAE) compares to several natural baselines based on random perturbations of the activation vectors x which preserve some error property of the SAE construction (e.g., having the same l2 reconstruction error or cosine similarity). I find that the KL divergence is significantly higher (2.2x - 4.5x) for the residual stream SAE reconstruction compared to the random perturbations and moderately higher (0.9x-1.7x) for attention out SAEs.

This suggests that the SAE reconstruction is not faithful by our definition, as it does not preserve the next token prediction probabilities.

This observation is important because it suggests that SAEs make systematic, rather than random, errors and that continuing to drive down reconstruction error may not actually increase SAE faithfulness. This potentially indicates that current SAEs are missing out on important parts of the learned representations of the model. The good news is that this KL-gap presents a clear target for methodological improvement and a new metric for evaluating SAEs. I intend to explore this in future work.

Intuition: how big a deal is this (KL) difference?

For some intuition, here are several real examples of the top-25 output token probabilities at the end of a prompt when patching in SAE and ϵ-random reconstructions compared to the original model's next-token distributio...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Newly launched Humane Slaughter Initiative webpage, published by Léa Guttmann on March 29, 2024 on The Effective Altruism Forum.

This post was cross-posted by the Forum team with the permission of the author. The author may not see or respond to comments on this post.

Greetings from Shrimp Welfare Project!

Our team is excited to announce the launch of our new webpage dedicated to

the Humane Slaughter Initiative! This initiative aims to revolutionise the way shrimps are stunned prior to slaughter and pave the way for the future of ethical shrimp production. Learn more about this new page below.

In a

newly published article on our website highlighting the urgency of responsible pond management, findings revealed alarmingly toxic hydrogen sulphide levels in shrimp ponds in India's Serepalem village, prompting a successful sludge removal intervention that significantly improved conditions - underscoring the imperative of sustainable practices for shrimp welfare and farm productivity.

Tamar Stelling wrote a

piece for De Correspondent in the Netherlands, highlighting our corporate engagement work with a focus on electrical stunning. It is a mix of in-depth shrimp welfare discussion, some recognition where it is due (e.g.

Albert Heijn), and a lot of fun!

Have a great week!

Léa | SWP Communications Lead

The Humane Slaughter Initiative webpage

SWP is thrilled to announce the launch of our

Humane Slaughter Initiative webpage!

What's the initiative about? Until the end of 2025, SWP is providing ~24 shrimp producers with free electrical stunners on the condition they use it to stun a minimum of 1,500-2,000 metric tons of shrimp per year. The goal is to accelerate industry adoption of this more humane stunning method ahead of expected buyer demands and regulations.

Why is it impactful? Scientific literature establishes shrimp sentience. In addition, consumer demand for higher welfare standards is increasing globally. Electrical stunning is also recognised as the most humane pre-slaughter method currently available for shrimps. It renders shrimp unconscious faster and more effectively than ice slurry immersion or asphyxia.

Leading retailers like M&S and John Lewis Partnership have already committed to using electrical stunning, setting the stage for industry-wide adoption.

What will you find on the page? This page serves as your go-to resource for our initiative, offering an in-depth overview, a breakdown of how Ace Aquatec and Optimar equipment function, and a summary of the eligibility requirements. Any questions? Our FAQ section may have your answers. Ready to apply? Fill out the application form you will find on the page.

Tackling Hydrogen Sulphide levels in shrimp farms

In a recently published article, Shrimp Welfare Project conducted a crucial study in the village of Serepalem, Andhra Pradesh, India, which found alarming findings of toxic levels of hydrogen sulphide (H2S) in all 38 tested ponds. This poses a severe respiratory toxin for shrimps. As an intervention, SWP facilitated sludge removal in a pond with 10 years of build-up, leading to a significant drop in H2S from hazardous to near-threshold levels.

This small but impactful study underscores the importance of regular pond maintenance through cost-effective sludge removal to improve shrimp welfare and farm productivity.

Moving forward, SWP plans to expand this sustainable intervention to other neglected areas.

Dive Deeper

If you want to learn more about shrimp welfare and the shrimp farming industry:

SWP's pioneering efforts in advancing shrimp welfare, particularly through the Humane Slaughter Initiative, have

gained recognition in a recent Global Seafood Alliance article, alongside advocacy by prominent retailers like Marks and Spencer, Waitrose & Partners, and Albert Heijn. The article showcases SWP's provision of...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to safely use an optimizer, published by Simon Fischer on March 29, 2024 on LessWrong.

Summary: The post describes a method that allows us to use an untrustworthy optimizer to find satisficing outputs.

Acknowledgements: Thanks to Benjamin Kolb (@benjaminko), Jobst Heitzig (@Jobst Heitzig) and Thomas Kehrenberg (@Thomas Kehrenberg) for many helpful comments.

Introduction

Imagine you have black-box access to a powerful but untrustworthy optimizing system, the Oracle. What do I mean by "powerful but untrustworthy"? I mean that, when you give an objective function f as input to the Oracle, it will output an element x that has an impressively low[1] value of f(x). But sadly, you don't have any guarantee that it will output the optimal element and e.g. not one that's also chosen for a different purpose (which might be dangerous for many reasons, e.g.

instrumental convergence).

What questions can you safely ask the Oracle? Can you use it to create utopia by asking for designs of machines, proteins, computer programs, etc.? Or are you risking the destruction of everything that we value if you dare to use such designs?

As an example, the Oracle could be a learned system; in that case, the topic of this post would be finding a way to get useful work out of the Oracle despite its inner misalignment.

In this post I'll describe a technique that allows us to safely use the Oracle under fairly weak assumptions. This approach can also be considered to be a way of controlling arbitrarily powerful AI systems.

One neat trick

This isn't fair, isn't fair, isn't fair! There's a limit to how many constraints you can add to a problem before it really is impossible!

(Harry Potter and the Methods of Rationality, Chapter 56)

Let O be a finite set of possible outputs of the Oracle (e.g. strings of length at most l) and f:OR be our objective function. Let's assume we are happy with an output that satisfices; i.e. we want to find an output x such that the value of f(x) is lower than some threshold c. Let S={xOf(x)