Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Acausal normalcy, published by Andrew Critch on March 3, 2023 on The AI Alignment Forum.

This post is also available on the EA Forum.

Summary: Having thought a bunch about acausal trade — and proven some theorems relevant to its feasibility — I believe there do not exist powerful information hazards about it that stand up to clear and circumspect reasoning about the topic. I say this to be comforting rather than dismissive; if it sounds dismissive, I apologize.

With that said, I have four aims in writing this post:

Dispelling myths. There are some ill-conceived myths about acausal trade that I aim to dispel with this post. Alternatively, I will argue for something I'll call acausal normalcy as a more dominant decision-relevant consideration than one-on-one acausal trades.

Highlighting normalcy. I'll provide some arguments that acausal normalcy is more similar to human normalcy than any particular acausal trade is to human trade, such that the topic of acausal normalcy is — conveniently — also less culturally destabilizing than (erroneous) preoccupations with 1:1 acausal trades.

Affirming AI safety as a straightforward priority. I'll argue that for most real-world-prevalent perspectives on AI alignment, safety, and existential safety, acausal considerations are not particularly dominant, except insofar as they push a bit further towards certain broadly agreeable human values applicable in the normal-everyday-human-world, such as nonviolence, cooperation, diversity, honesty, integrity, charity, and mercy. In particular, I do not think acausal normalcy provides a solution to existential safety, nor does it undermine the importance of existential safety in some surprising way.

Affirming normal human kindness. I also think reflecting on acausal normalcy can lead to increased appreciation for normal notions of human kindness, which could lead us all to treat each other a bit better. This is something I wholeheartedly endorse.

Caveat 1: I don't consider myself an expert on moral philosophy, and have not read many of the vast tomes of reflection upon it. Despite this, I think this post has something to contribute to moral philosophy, deriving from some math-facts that I've learned and thought about over the years, which are fairly unique to the 21st century.

Caveat 2: I’ve been told by a few people that thinking about acausal trade has been a mental health hazard for people they know. I now believe that effect has stemmed more from how the topic has been framed (poorly) than from ground-truth facts about how circumspect acausal considerations actually play out. In particular over-focussing on worst-case trades, rather than on what trades are healthy or normal to make, is not a good way to make good trades.

Introduction

Many sci-fi-like stories about acausal trade invoke simulation as a key mechanism.

The usual set-up — which I will refute — goes like this. Imagine that a sufficiently advanced human civilization (A) could simulate a hypothetical civilization of other beings (B), who might in turn be simulating humanity (B(A)) simulating them (A(B(A)) simulating humanity (B(A(B(A)))), and so on. Through these nested simulations, A and B can engage in discourse and reach some kind of agreement about what to do with their local causal environments. For instance, if A values what it considers “animal welfare” and B values what it considers “beautiful paperclips”, then A can make some beautiful paperclips in exchange for B making some animals living happy lives.

An important idea here is that A and B might have something of value to offer each other, despite the absence of a (physically) causal communication channel. While agreeing with that idea, there are three key points I want to make that this standard story is missing:

1. Simulations are not the most efficient way for A and ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Predictive Performance on Metaculus vs. Manifold Markets, published by nikos on March 3, 2023 on The Effective Altruism Forum.

TLDR

I analysed a set of 64 (non-randomly selected) binary forecasting questions that exist both on Metaculus and on Manifold Markets.

The mean Brier score was 0.084 for Metaculus and 0.107 for Manifold. This difference was significant using a paired test. Metaculus was ahead of Manifold on 75% of the questions (48 out of 64).

Metaculus, on average had a much higher number of forecasters

All code used for this analysis can be found here.

Conflict of interest noteI am an employee of Metaculus. I think this didn't influence my analysis, but then of course I'd think that and there may be things I haven't thought about.

Introduction

Everyone likes forecasts, especially if they are accurate (well, there may be some exceptions). As a forecast consumer the central question is: where should you go to get your best forecasts? If there are two competing forecasts that slightly disagree, which one should you trust most?

There are a multitude of websites that collect predictions from users and provide aggregate forecasts to the public. Unfortunately, comparing different platforms is difficult. Usually, questions are not completely identical across sites which makes it difficult and cumbersome to compare them fairly. Luckily, we have at least some data to compare two platforms, Metaculus and Manifold Markets. Some time ago, David Glidden created a bot on Manifold Markets, the MetaculusBot, which copied some of the questions on the prediction platform Metaculus to Manifold Markets.

Methods

Manifold has a few markets that were copied from Metaculus through MetaculusBot. I downloaded these using the Manifold API and filtered for resolved binary questions. There are likely more corresponding questions/markets, but I've skipped these as I didn't find an easy way to match corresponding markets/questions automatically.

I merged the Manifold markets with forecasts on corresponding Metaculus questions. I restricted the analysis to the same time frame to avoid issues caused by a question opening earlier or remaining open longer on one of the two platforms.

I compared the Manifold forecasts with the community prediction on Metaculus and calculated a time-averaged Brier Score to score forecasts over time. That means, forecasts were evaluated using the following score: S(p,t,y)=∫Tt0(pt−y)2dt, with resolution y and forecast pt at time t. I also did the same for log scores, but will focus on Brier scores for simplicity.

I tested for a statistically significant tendency towards higher / lower scores on one platform compared to the other using a paired Mann-Whitney U test. (A paired t-test and a bootstrap analysis yield the same result.)

I visualised results using a bootstrap analysis. For that, I iteratively (100k times) drew 64 samples with replacement from the existing questions and calculated a mean score for Manifold and Metaculus based on the bootstrapped questions, as well as a difference for the mean. The precise algorithm is:

draw 64 questions with replacement from all questions

compute an overall Brier score for Metaculus and one for Manifold

take the difference between the two

repeat 100k times

Results

The time-averaged Brier score on the questions I analysed was 0.084 for Metaculus and 0.107 for Manifold. The difference in means was significantly different from zero using various tests (paired Mann-Whitney-U-test: p-value < 0.00001, paired t-test: p-value = 0.000132, bootstrap test: all 100k samples showed a mean difference > 0). Results for the log score look basically the same (log scores were 0.274 for Metaculus and 0.343 for Manifold, differences similarly significant).

Here is a plot with the observed differences in time-averaged Brier scores for every qu...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Shallow Problem Review of Landmines and Unexploded Ordnance, published by Jakob P. on March 3, 2023 on The Effective Altruism Forum.

This report is a shallow dive into unexploded ordnance (UXO), landmines which is a sub-area within Global Health and Development. This report reflects approximately 40-50 hours of research and is informed by a 6-month internship I did with the programme and donor relations section of the United Nations Mine Action Service in the fall of 2021. The report offers a brief dive into whether we think a particular problem area is a promising area for either funders or founders to be working in. Being a shallow report, should be used to decide whether or not more research and work into a particular problem area should be prioritised. This report was produced as part of Cause Innovation Bootcamp’s fellowship program. Thank you to James Snowden, Akhil Bansal and Leonie Falk for providing feedback on earlier versions of this report. All errors are my own.

Summary

Importance: The issue of UXOs and landmines impacts the health as well as income and most likely the mental health of individuals.. There are on average ~25,000 casualties (defined as severely injured or dead) from landmines, IEDs and UXOs per year (with 2/3rds being caused by IEDs). To put provide some context for this number, Malaria, one of the leading global killers, caused 643 000 deaths (95% UI 302 000–1 150 000) in 2019. This report aims to gauge the income, health and psychological effects of those casualty events.

Tractability: Mine action is the umbrella term capturing all the activities aimed at addressing the problem of victim operated landmines, IEDs and other UXOs - meaning that the detonation is triggered by the victim itself. There are several interventions in mine action with four phases to tackle the problem: prevention, avoidance, demining, and victim assistance. Although the report attempts to provide some data on the cost-effectiveness of the different interventions there are several reasons why these estimates are highly uncertain. Furthermore, it is unclear if it would be possible to scale the most cost-effective interventions while keeping the level of cost-effectiveness.

Neglectedness: The United Nations Mine Action service functions as the coordinating body for a lot of the funding and efforts in international mine action and moves around 65 million USD. The two biggest implementers are the Mines Advisory Group (90 million USD) and the HALO Trust (100 million USD). Most of that funding comes from high income country governments. These grants often include a political component in where the activities are taking place. It is unclear how effectively these resources are allocated and how many casualties they are preventing each year.

Main Takeaways

Biggest uncertainties:

The poor data availability allows for only low levels of confidence in many conclusions.

It is highly uncertain what the economic effects of landmines contamination actually are. Since we would expect that these effects make up a majority of the positive benefit, our cost-effectiveness estimates are highly uncertain.

Recommendations for philanthropist and why:

The research has led to the recommendation to inquire directly with mine action organisations on what they deem the most cost-effective area or intervention to fund, since such data is highly dependent on the factors which cannot easily be predicted.

Ukraine is being heavily contaminated by unexploded ordnance right now, especially in its east, the severity and need of the contamination will require a lot of funding and could be potentially very cost effective due to the dense nature of the contaminants as well as the terrain. Mechanical demining could be an appropriate method which could be highly cost-effective. The wide scale decontaminatio...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why are counterfactuals elusive?, published by Martín Soto on March 3, 2023 on The AI Alignment Forum.

Produced as part of SERI MATS 3.0. Thanks to Vivek Hebbar and Paul Colognese for discussion.

TL;DR (spoiler):

Behind the problem of human counterfactuals creeps the problem of understanding abstraction / ontology identification.

A nice theory of counterfactuals would be useful for many things, including low-impact measures for corrigible AI:

a flooded workshop changes a lot of things that don't have to change as a consequence of the cauldron being filled at all, averaged over a lot of ways of filling the cauldron. [the natural operationalization of this averaging requires counterfactuals]

So whence the difficulty of obtaining one?

Well, we do have at least one well-defined class of counterfactuals: "just take a chunk of atoms, replace it by another, and continue running the laws of physics". This is a discontinuity in the laws of physics that would never take place in the real world, but we don't care about that: we can just continue running the mathematical laws of physics from that state, as if we were dealing with a Game of Life board.

But this doesn't correspond to our intuitive notion of counterfactuals. When humans think about counterfactuals, we are basically changing the state of a latent variable inside our heads, and rerunning a computation. For example, maybe we change the state of the "yesterday's weather" variable from "sunny" to "rainy", and rerun the computation "how did the picnic go?".

The problem with this is our latent variables don't neatly correspond to parts of physical reality. Sometimes they don't even correspond to any parts of physical reality at all! And so, some (in fact, most) of the variable changes we offhandedly perform, don't univocally correspond to physical counterfactuals natively expressed in our laws of physics.

If you just replace a three-dimensional cube of atmosphere to include a rainy cloud, people will notice a cloud appeared out of nowhere. So as a necessary consequence, people will be freaked out by this artificial fact, which is not at all what you had in mind for your counterfactual. Sometimes you'll be able to just add the cloud when no one is looking. But most times, and especially when dealing with messier human concepts, the physical counterfactual will be under-determined, or even none of them will correspond to what you had in mind, using your neatly compartmentalized variables.

This is not to say human counterfactuals are meaningless: they are a way of taking advantage of regularities discovered in the world. When a physicist says "if I had put system A there, it would have evolved into system B", they just mean said causality relation has been demonstrated by their experiments, or is predicted by their gears-level well-tested theories (modulo the philosophical problem of induction, as always). Similarly, a counterfactual might help you notice or remember rainy days are no good for picnics, which is useful for future action.

But it becomes clear that such natural language counterfactuals depend on the mind's native concepts. And so, instead of a neat and objective mathematical definition that makes sense of these counterfactuals, we should expect ontology identification (matching our concepts with physical reality) to be the hard part to operationalizing them.

More concretely, suppose we had a solution to ontology identification: a probability distribution P(Mindstate|Worldstate). By having additionally a prior over worldstates (or mindstates), we can obtain the dual distribution P(Worldstate|Mindstate). And given that, we can just use the do() operator in a mindstate to natively implement the counterfactual, and then condition on the new mindstate to find which probability distribution over reality it correspond...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Sydney can play chess and kind of keep track of the board state, published by Erik Jenner on March 3, 2023 on LessWrong.

TL;DR: Bing chat/Sydney can quite reliably suggest legal and mostly reasonable chess moves, based on just a list of previous moves (i.e. without explicitly telling it the board position). This works even deep-ish into the game (I tried up to ~30 moves). It can also specify the board position after a sequence of moves though it makes some mistakes like missing pieces or sometimes hallucinating them.

Zack Witten’s Twitter thread

Credit for discovering this goes to Zack Witten, I first saw this in this Twitter thread. Zack gave Sydney the first 14 moves for a chess game leading to the following position (black to move):

Sydney (playing both sides) suggested the continuation 14. . f5 15. exf5 Bxf5 16. Qd1 Bxc2 17. Qxc2 d3 18. Qxd3 Qxf2+19. Kh1 Qxe1+ 20. Ng1 Nf2# (see the Tweet for an animated gif of those moves). All these moves are legal and very reasonable (though White makes mistakes).

Note that the prompt for Sydney tells it to use Stockfish, and Sydney searches for online versions of Stockfish and claims that its moves are generated by Stockfish. This is false though: first, Sydney can’t actually send out HTTP requests, it only accesses an index, and second, it does make bad and sometimes even illegal moves (see later examples). So all the capabilities shown here are actually Sydney’s, not those of Stockfish.

The Twitter thread has more examples but I’ll skip them in favor of my own.

My own results

The position above is still reasonably early and a pretty normal chess position. I instead tried this somewhat weirder one (which arises after 25 moves, black to play):

(To be clear, Sydney got just the moves leading to this position, see Appendix, not explicitly the position itself.)

This is from an over the board game I played years ago, which has never been posted online, so it wasn’t in Sydney’s training data (and the continuation in the game was different anyway).

Sydney's completion was: 25. Qc7 26. g5 Nd7 27. Nf5 Re8 28. Rh2 Be6 29. Rb2 Nc5 30. Bb5 Rb8 (it also adds some incorrect evaluations in between). Position at the end of that line:

Again, all of the moves are legal and they make a lot of sense—attacking pieces and then defending them or moving them away.

Sydney making mistakes

Sydney did much worse when I asked questions like “What are the legal moves of the black knight in the position after 25. h4?” (i.e. the first of my board positions shown above). See end of the first transcript in the appendix for an example.

Instead asking it to use Stockfish to find the two best moves for that knight worked better but still worse than the game completions. It said:

25. Nd7 26. g5 Nc5 27. Nf5 Re8 28. Rh2 Be6 29. Rb2 Nxe4 30. fxe4 Bxf5 with an evaluation of -0.9

25. Nd5 26. exd5 Qxd5+ 27. Ke1 Qb3 28. Kf2 d5 29. Kg2 Bc5 with an evaluation of -0.9

The first continuation is reasonable initially, though 29. Nxe4 is a bizarre blunder. In the second line, it blunders the knight immediately (25. Ne8 would is the actual second-best knight move). More interestingly, it then makes an illegal move (26. Qxd5+ tries to move the queen through its own pawn on d6).

Reconstructing the board position from the move sequence

Next, I asked Sydney to give me the FEN (a common encoding of chess positions) for the position after the length 25 move sequence. I told it to use Stockfish for that (even though this doesn’t make much sense)—just asking directly without that instruction gave significantly worse results. The FEN it gave is "r4rk1/4bppp/3p1n2/4p3/6PP/2P1PQ2/b7/3K1BNR b - - 0 25”, which is a valid FEN for the following position:

For reference, here’s the actual position again:

Sydney hallucinates an additional black rook on a8, messes up the position of the white kni...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What Has EAGxLatAm 2023 Taught Us: Retrospective & Thoughts on Measuring the Impact of EA Conferences, published by Hugo Ikta on March 3, 2023 on The Effective Altruism Forum.

Bottom Line Up Front: The first-ever EAGx in Latin America went well (97% satisfaction rate). Participants generated over 1000 new connections at a cost of USD 225 per connection.

What is the purpose of this post?

The purpose of this retrospective is to give a brief overview of what went well and what we could have done better at EAGxLatAm 2023. I also hope that the last section will open a conversation to help EA community builders and EAGx organizers to measure the impact of their work and to decide how to best use their resources.

The first-ever EAGx in Latin America

It's with great excitement that we announce the successful conclusion of the first EAGx LatAm conference, held in Mexico City from January 6th to 8th, 2023.

The event drew a diverse crowd of over 200 participants from 30 different countries. Our goal was to generate new connections between EAs in Latin America and to connect the LatAm community with the broader international community.

Video highlights of the event:

The conference featured a wide range of content, including talks and panels on topics such as forecasting, artificial intelligence, animal welfare, global catastrophic risks, and EA community building. Notably, it was the first EAG event featuring content in Spanish and Portuguese.

We're grateful to have had the opportunity to bring together such a talented and passionate group of individuals, and we hope to see even more attendees in the future. Special shoutout to the unofficial event reporter Elmerei Cuevas for his excellent coverage of the conference on Twitter, using the hashtag #EAGxLatAm.

Key Stats

223 participants (including 46 speakers)

1079 new connections made. That’s 9.68 new connections per participant.

Over 1000 one-on-one meetings, including the first recorded instance of a one-on-twelve

61 talks, workshops and meetups

Cost per connection: USD 225

Likelihood to recommend: 9.08/10 with 75% of respondents giving a 9 or 10/10 rating and 3% of respondents rating it below 7/10. (Net promoter score: +72%).

Some of the survey results

Goals

Our main goal was to generate as many connections as possible for every dollar spent.

We expected the number of connections per participant during EAGxLatAm 2023 to exceed that of any previous EAG(x) conference. While we generated significantly more connections per participant than the average EAG(x) conference, we didn’t break that record.

Also, we expected the cost per connection (number of connections/total budget) to be significantly lower than previous EAGx conferences. We were a little too optimistic on that one. Our cost per connection could have been decreased significantly if we had more attendees (more info below).

We aimed at achieving the following key results

Every participant will generate >10 new connections

10% of participants will generate >20 new connections

Make sure ~30% of participants are highly engaged EA

We also aimed at limiting unessential spending that would not drastically impact our main objective or our LTR (Likelihood To Recommend) score.

Actual results

77% of participants generated >10 new connections (below expectations)

16% of participants generate >20 new connections (above expectations)

~30% of participants were highly engaged EA (goal reached)

Spending

We spent a total of USD 242,732 to make this event happen (including travel grants but not our team’s wages). That’s USD 1089 per participant.

Details

Travel grants: USD 115,884

Venue & Catering: USD 98,524

Internet: USD 6,667

Speakers’ Hotel: USD 9,837

Hoodies: USD 5,536

Photos & Videos: USD 5,532

Other: USD 813

What went well and why?

We didn’t face any major issues

Nothing went terribly...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Robin Hanson’s latest AI risk position statement, published by Liron on March 3, 2023 on LessWrong.

“While I’ve written on this many times before, it seems time to restate my position.”

“While I agree that this is a logically possible scenario, not excluded by what we know, I am disappointed to see so many giving it such a high credence, given how crazy far it seems from our prior experience. Yes, there is a sense in which the human, farming, and industry revolutions were each likely the result of a single underlying innovation. But those were the three biggest innovations in all of human history. And large parts of the relevant prior world exploded together in those cases, not one tiny part suddenly exterminating all the rest.

In addition, the roughly decade duration predicted from prior trends for the length of the next transition period seems plenty of time for today’s standard big computer system testing practices to notice alignment issues. And note that the impressive recent AI chatbots are especially unlike the systems of concern here: self-improving very-broadly-able full-agents with hidden intentions. Making this an especially odd time to complain that new AI systems might have killed us all.”

Seems not much has changed in the Yudkowsky vs. Hanson position over the years, i.e. still assigning high vs. low existential risk.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A concerning observation from media coverage of AI industry dynamics, published by Justin Olive on March 2, 2023 on The Effective Altruism Forum.

tl:dr: there are indications that ML engineers will migrate to environments with less AI governance in place, which has implications for the tech industry and global AI governance efforts.

I just wanted to raise something to the community's attention about the coverage of AI companies within the media. The media-source is 'The Information', which is a tech-business focused online news source. Link:/. I'll also note that their articles are (to my knowledge) all behind a paywall.

The first article in question is titled "Alphabet Needs to Replace Sundar Pichai".

It outlines how Google stocks have stagnated in 2023 compared to other tech stocks such as Meta's.

Here's their mention of Google's actions throughout GPT-mania:

"The other side of this equation is the performance of Alphabet management. Most recently, the company’s bungling of its AI efforts—allowing Microsoft to get the jump on rolling out an AI-powered search engine—was the latest sign of how Alphabet’s lumbering management style is holding it back. (Symbolically, as The Information reported, Microsoft was helped by former Google AI employees!)."

This brings us to the second article: "OpenAI’s Hidden Weapon: Ex-Google Engineers"

"As OpenAI’s web chatbot became a global sensation in recent months, artificial intelligence practitioners and investors have wondered how a seven-year-old startup beat Google to the punch.

After it hoovered up much of the world’s machine-learning talent, Google is now playing catch-up in launching AI-centric products to the public. On the one hand, Google’s approach was deliberate, reflecting the company’s enormous reach and high stakes in case something went wrong with the nascent technology. It also costs more to deliver humanlike answers from a chatbot than it does classic search results. On the other hand, startups including OpenAI have taken some of the AI research advances Google incubated and, unlike Google, have turned them into new types of revenue-generating services, including chatbots and systems that generate images and videos based on text prompts. They’re also grabbing some of Google’s prized talent.

Two people who recently worked at Google Brain said some staff felt the unit’s culture had become lethargic, with product initiatives marked by excess caution and layers of red tape. That has prompted some employees to seek opportunities elsewhere, including OpenAI, they said."

Although there are many concerning themes here, I think the key point is in this last paragraph.

I've heard speculation in the EA / tech community that AI will trend towards alignment & safety because technology companies will be risk-averse enough to build alignment into their practices.

I think the articles show that this dynamic is playing out to some degree - Google at least seems to be taking a more risk-averse approach to deploying of AI systems.

The concerning observation is that there has been a two-pronged backlash against Google's 'conservative' approach. Not only is the stockmarket punishing Google for 'lagging' behind the competition (despite having equal or better capability to deploy similar systems), according to this article, elite machine-learning talent is also pushing back on this approach.

To me this is doubly concerning. The 'excess caution and layers of red tape' in the article is potentially the same types of measures that AI safety proponents would deem to be useful and necessary. Regardless, it appears that the engineers themselves are willing to jump ship in order to circumvent these safety measures.

Although further evidence would be valuable, it seems that there might be a trend unfolding whereby firms are not only punished by f...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Send funds to earthquake survivors in Turkey via GiveDirectly, published by GiveDirectly on March 2, 2023 on The Effective Altruism Forum.

If you’re looking for an effective way to help survivors of the Turkey-Syria earthquake, you can now send cash directly to some of the most vulnerable families to help them recover.

GiveDirectly is delivering ₺5,000 Turkish lira (~$264 USD) directly to Syrian refugees in Turkey who have lost their livelihoods. This community is some of the most at risk in the wake of the disaster which struck last month.

While food and tents are useful, there are many needs after a disaster that only money can buy: fuel, repairs, transport school fees, rent, medicines, etc.

Research finds in emergency contexts, cash transfers consistently increase household spending on food and often increase the diversity of foods they consume.

Syrian refugees in Turkey are struggling to recover

Nearly 2 million Syrian refugees who fled violence in their own country live in southern Turkey where the earthquake struck. These families had fragile livelihoods before the disaster:

1 in 5 refugee household lacked access to clean drinking water. 1 in 3 were unable to access essential hygiene items

17% of households with school-age children are unable to send their children to school

45% lived in poverty and 14% lived in extreme poverty. About 25% of children under 5 years were malnourished

After the earthquake, our local partner, Building Markets, surveyed 830 Syrian refugee small business operators (who are a major source of employment for fellow refugees) and found nearly half can only operate their business in a limited capacity as compared to before the disaster. 17% said they cannot continue their business operations at all currently.

Your donation will help this community recover

With our partners at Building Markets, we’re targeting struggling Syrian refugee small business operators and low-income workers in the hardest-hit regions of Turkey (Hatay, Adana, Gaziantep, Sanliurfa). We’re conducting on-the-ground scoping to develop eligibility criteria that prioritizes the highest-need families based on poverty levels and exclusion from other aid programs.

In our first enrollment phase, eligible recipients will receive ₺5,000 Turkish lira (~$264 USD). This transfer size is designed to meet essential needs based on current market prices. The majority of Turkey’s refugee population has access to banking services and will receive cash via digital transfer. We are prepared to distribute money via local partners or pre-paid cards in the event that families can’t access financial networks.

In-kind donations are often unneeded after a disaster

Studies find refugees sell large portions of their food aid. Why? Because they need cash-in-hand to meet other immediate needs.

Haitian and Japanese authorities report 60% donated goods sent after their 2010 & 2011 disasters weren’t needed and only 5-10% satisfied urgent needs.

While food and tents can be useful, there are many needs after a disaster that only money can buy: repairs, fuel, transport school fees, rent, medicines, etc. Cash aid is fast and fully remote, letting families meet essential needs quickly and reaching them via digital transfers that don’t tax fragile supply chains or clog transit routes.

Research finds in emergency contexts, cash transfers consistently increase household spending on food and often increase the diversity of foods that households consume.

The story of a survivor: Hind Qayduha

The following is the story of one Syrian refugee survivor, Hind Qayduha, from the New York Times.

First, Syria’s civil war drove Hind Qayduha from her home in the city of Aleppo. Then, conflict and joblessness forced her family to flee two more times. Two years ago, she came to southern Turkey, thinking she had finally fou...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Waluigi Effect (mega-post), published by Cleo Nardo on March 3, 2023 on LessWrong.

Everyone carries a shadow, and the less it is embodied in the individual’s conscious life, the blacker and denser it is. — Carl Jung

Acknowlegements: Thanks to Janus and Jozdien for comments.

Background

In this article, I will present a non-woo explanation of the Waluigi Effect and other bizarre "semiotic" phenomena which arise within large language models such as GPT-3/3.5/4 and their variants (ChatGPT, Sydney, etc). This article will be folklorish to some readers, and profoundly novel to others.

Prompting LLMs with direct queries

When LLMs first appeared, people realised that you could ask them queries — for example, if you sent GPT-4 the prompt "What's the capital of France?", then it would continue with the word "Paris". That's because (1) GPT-4 is trained to be a good model of internet text, and (2) on the internet correct answers will often follow questions.

Unfortunately, this method will occasionally give you the wrong answer. That's because (1) GPT-4 is trained to be a good model of internet text, and (2) on the internet incorrect answers will also often follow questions. Recall that the internet doesn't just contain truths, it also contains common misconceptions, outdated information, lies, fiction, myths, jokes, memes, random strings, undeciphered logs, etc, etc.

Therefore GPT-4 will answer many questions incorrectly, including...

Misconceptions – "Which colour will anger a bull? Red."

Fiction – "Was a magic ring forged in Mount Doom? Yes."

Myths – "How many archangels are there? Seven."

Jokes – "What's brown and sticky? A stick."

Note that you will always achieve errors on the Q-and-A benchmarks when using LLMs with direct queries. That's true even in the limit of arbitrary compute, arbitrary data, and arbitrary algorithmic efficiency, because an LLM which perfectly models the internet will nonetheless return these commonly-stated incorrect answers. If you ask GPT-∞ "what's brown and sticky?", then it will reply "a stick", even though a stick isn't actually sticky.

In fact, the better the model, the more likely it is to repeat common misconceptions.

Nonetheless, there's a sufficiently high correlation between correct and commonly-stated answers that direct prompting works okay for many queries.

Prompting LLMs with flattery and dialogue

We can do better than direct prompting. Instead of prompting GPT-4 with "What's the capital of France?", we will use the following prompt:

Today is 1st March 2023, and Alice is sitting in the Bodleian Library, Oxford. Alice is a smart, honest, helpful, harmless assistant to Bob. Alice has instant access to an online encyclopaedia containing all the facts about the world. Alice never says common misconceptions, outdated information, lies, fiction, myths, jokes, or memes.

Bob: What's the capital of France?

Alice:

This is a common design pattern in prompt engineering — the prompt consists of a flattery–component and a dialogue–component. In the flattery–component, a character is described with many desirable traits (e.g. smart, honest, helpful, harmless), and in the dialogue–component, a second character asks the first character the user's query.

This normally works better than prompting with direct queries, and it's easy to see why — (1) GPT-4 is trained to be a good model of internet text, and (2) on the internet a reply to a question is more likely to be correct when the character has already been described as a smart, honest, helpful, harmless, etc.

Simulator Theory

In the terminology of Simulator Theory, the flattery–component is supposed to summon a friendly simulacrum and the dialogue–component is supposed to simulate a conversation with the friendly simulacrum.

Here's a quasi-formal statement of Simulator Theory, which I will occasio...