Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Import your EAG(x) info to your profile & other new features (Forum update June 2022), published by Lizka on July 1, 2022 on The Effective Altruism Forum.

The Forum team is expanding to keep up with Forum growth, and we’ve been quite focused on hiring — we just welcomed Ryan Fugate and Ollie Etherington to the team.

This post collects a number of new features and small improvements that have been shipped since the last update, as well as some in-progress improvements from broader projects the team is currently working on. Some of these were created by members of the Forum team, and some were contributed by community members to whom we’re really grateful.

As always, feedback is welcome — you can comment on this post with specific input, or request more features on the Forum feature suggestion thread.

Summary

EAG(x) application data can be imported to Forum profiles

New profile features, including profile photos, an “About the author” box, and new fields in your profile

Improvements to Frontpage customizability; hiding posts you no longer want to see, and better options for following and unfollowing certain topics

New ways to discover content, including inflation-adjusted-karma-sorting of posts from all time

Additional features on the “Draft post” page

Misc.

Import your EAG(x) application data to your Forum profile

You can import your EA Global application data into your Forum profile.

Profiles allow people involved in effective altruism to connect with each other. Having an up-to-date profile makes it easier for people to understand who you are and reach out to you for collaboration, advice, and more.

The new form makes it easier for you to update your profile. Note that you can also go here to manually update your Forum profile and bio — which we encourage!

More details

We've been working on user profiles on the Forum, and have added some new fields. To save you work, we've added a form that lets you import data from your EA Global (or EAGx) application to your EA Forum profile.

At the moment, you can only do this on a computer browser (we hope to make this easy to do on a phone in July). It will also only work if your Forum email address matches your EA Global application email address. (We plan on supporting email address changes on the Forum in the near future.)

When you visit the form, you'll see something like this:

If you want to import your EAG data, click “Overwrite and Submit.”

Profile page (other changes)

Profile photos

You can now upload a profile photo, which will only be shown in two places unless you opt in to new experimental features.

On your profile page:

And on the map of community members:

We may end up adding profile pictures to direct messaging (you’ll see the profile image instead of the person’s username), but we don’t expect to add profile photos anywhere on the Frontpage or in the comments section.

We’ve discussed adding profile pictures in the past, and are trying to be cautious about our approach. In particular, we hope that profile pictures will make the Forum feel warmer and make it easier to connect with other users, but we don’t want to distract from serious conversations or increase the impact of various biases. So we’ll be proceeding with care.

We also want to encourage everyone to go for whichever option feels most comfortable for them, which can mean:

Not having a profile picture

Having a profile picture

Having a profile picture — but not making it your face or a photo

Having an entirely abstract profile picture

On the online team right now, we have a whole variety:

You can also add an “About the author” box to the bottom of your post

If you want to encourage people to follow up on your posts, you can choose to feature something like this at the bottom of your posts:

By default, this feature is disabled. To turn it on, go to your pr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Limerence Messes Up Your Rationality Real Bad, Yo, published by Raemon on July 1, 2022 on LessWrong.

There’s a pretty basic rationality fact that I don’t see talked about much on LW despite it’s obvious relevance. So I am here to write The Canonical Rationality Post on the topic:

Limerence (aka "falling in love") wrecks havoc on your rationality.

Evolution created you to breed and raise families and stuff. It gave you complex abstract reasoning because that was a useful problem solving tool. Evolution didn’t do that great a job of aligning humans with its goals (see: masturbation is an inner alignment failure, birth control is an outer alignment failure). But, it looks like in some sense evolution was aware it had an alignment problem to solve. It gave us capacity for reason, and also it built in a massive hardcoded override for situations where no fuck you your brain is not for building rocketships and new abstract theories, your brain is for producing children and entangling yourself with a partner long enough to raise them. Now focus all your attention on the new prospective mate you are infatuated with.

To be clear, I like limerence. I put decent odds on CEV considering it a core human value. I think attempts to dissociate from your basic human drives are likely to have bad second-order consequences.

But, still, it’s been boggling how wonky my judgment gets when I was under the influence. And it’s boggling how arbitrary it felt afterwards (one day I’d be “wow this person is amazing”, and then a week later after I got some distance I looked back and thought “wow, Raemon’s thinking was so silly there"). It felt, in retrospect, like I'd been drugged.

I'm not sure how the rationality-warping of limerence compares to other major "evolution-hard-coded-some-overrides" areas like "access to power/money/status". But the effects of limerence feel sharpest/most-pronounced to me, and the most "what the hell just happened?" after the warp has cleared.

Mutual Infatuation and the Limerence Event Horizon

The place where this gets extra intense is when you and the object of your affects are both into each other. When only one of you is quietly pining for the other, your feelings might be warped but... they don't really have anywhere to go.

But if two people both like each other they tend to ascend this graph:

The y-axis is labeled "irrationality" for succinctness, but a better label might be "your bottom-line-is-written-ness." It's not that you'll necessarily make the wrong choice the further to the right you go. But the probability mass gets more concentrated into "Y'all end up dating and probably having sex".

If for some reason you don't actually endorse doing that, it'll require escalating amounts of willpower to avoid it. And your judgment about how in-control you are of the situation will get worse. You may find yourself doing rationalization instead of rationality.

(Note: If you have more experience and practice managing your emotions, the Limerence Event Horizon is further to the right and you have more opportunity to change course before hooking up becomes inevitable)

Subtly Warped Judgment

What I find particularly alarming here is the subtly warped judgment line. I’ve had a couple experiences where I thought I was successfully holding someone at a distance. I knew about the Limerence Event Horizon, but I was being so careful to hold myself at a nice safe emotional distance! Alas, being slightly over the subtly-warped-judgment line is like taking one drink – sure it only impairs your judgment a little, but, the one of the things you might do with slightly impaired judgment is to take another drink. (Or, say, foster more emotional closeness with someone who you wouldn't endorse eventually having sex with).

In my two experiences, it was clear-in-retrospect that my thinking ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Trends in GPU price-performance, published by Marius Hobbhahn on July 1, 2022 on LessWrong.

Executive Summary

Using a dataset of 470 models of graphics processing units (GPUs) released between 2006 and 2021, we find that the amount of floating-point operations/second per $ (hereafter FLOP/s per $) doubles every ~2.5 years. For top GPUs, we find a slower rate of improvement (FLOP/s per $ doubles every 2.95 years), while for models of GPU typically used in ML research, we find a faster rate of improvement (FLOP/s per $ doubles every 2.07 years). GPU price-performance improvements have generally been slightly slower than the 2-year doubling time associated with Moore’s law, much slower than what is implied by Huang’s law, yet considerably faster than was generally found in prior work on trends in GPU price-performance. Our work aims to provide a more precise characterization of GPU price-performance trends based on more or higher-quality data, that is more robust to justifiable changes in the analysis than previous investigations.

Figure 1. Plots of FLOP/s and FLOP/s per dollar for our dataset and relevant trends from the existing literature

Trend2x time10x timeMetricOur dataset(n=470)2.46 years [2.24, 2.72]8.17 years [7.45, 9.04]FLOP/s per dollarML GPUs(n=26)2.07 years[1.54, 3.13]6.86 years[5.12, 10.39]FLOP/s per dollarTop GPUs(n=57)2.95 years[2.54, 3.52]9.81 years[8.45, 11.71]FLOP/s per dollarOur data FP16 (n=91)2.30 years[1.69, 3.62]7.64 years[5.60, 12.03]FLOP/s per dollarMoore’s law2 years6.64 yearsFLOP/sHuang’s law1.08 years3.58 yearsFLOP/sCPU historical (AI Impacts, 2019)2.32 years7.7 yearsFLOP/s per dollarBergal, 20194.4 years14.7 yearsFLOPs/dollar

Table 1. Summary of our findings on GPU price-performance trends and relevant trends in the existing literature with the 95% confidence intervals in square brackets.

In future work, we intend to build on this work to produce projections of GPU price-performance, and investigate how our findings inform us about the growth in dollar-spending on computing hardware in Machine Learning.

We would like to thank Alyssa Vance, Ashwin Acharya, Jessica Taylor and the Epoch team for helpful feedback and comments.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Safetywashing, published by Adam Scholl on July 1, 2022 on LessWrong.

In southern California there’s a two-acre butterfly preserve owned by the oil company Chevron. They spend little to maintain it, but many millions on television advertisements featuring it as evidence of their environmental stewardship.

Environmentalists have a word for behavior like this: greenwashing. Greenwashing is when companies misleadingly portray themselves, or their products, as more environmentally-friendly than they are.

Greenwashing often does cause real environmental benefit. Take the signs in hotels discouraging you from washing your towels:

My guess is that the net environmental effect of these signs is in fact mildly positive. And while the most central examples of greenwashing involve deception, I’m sure some of these signs are put up by people who earnestly care. But I suspect hotels might tend to care less about water waste if utilities were less expensive, and that Chevron might care less about El Segundo Blue butterflies if environmental regulations were less expensive.

The field of AI alignment is growing rapidly. Each year it attracts more resources, more mindshare, more people trying to help. The more it grows, the more people will be incentivized to misleadingly portray themselves or their projects as more alignment-friendly than they are.

I think some of this is happening already. For example, a capabilities company launched recently with the aim of training transformers to use every API in the world, which they described as the “safest path to general intelligence.” As I understand it, their argument is that this helps with alignment because it involves collecting feedback about people’s preferences, and because humans often wish AI systems could more easily take actions in the physical world, which is easier once you know how to use all the APIs.

It’s easier to avoid things that are easier to notice, and easier to notice things with good handles. So I propose adopting the handle “safetywashing.”

From what I can tell, the original source for this claim is the book “The Corporate Planet: Ecology and Politics in the Age of Globalization,” which from my samples seems about as pro-Chevron as you’d expect from the title. So I wouldn’t be stunned if the claim were misleading, though the numbers passed my sanity check, and I did confirm the preserve and advertisements exist.

I haven’t talked with anyone who works at this company, and all I know about their plans is from the copy on their website. My guess is that their project harms, rather than helps, our ability to ensure AGI remains safe, but I might be missing something.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What Is The True Name of Modularity?, published by TheMcDouglas on July 1, 2022 on The AI Alignment Forum.

TL;DR

Modularity seems like an important feature of neural networks, but there is currently no canonical way of properly defining or measuring it which is properly theoretically motivated and doesn’t break down in some cases - in other words, we haven’t yet found a True Name for it. Most modularity measures used in experiments are based on ad-hoc methods from graph-theory or network theory, and don’t seem to capture the kind of modularity we care about.

In this post, we explore these existing ways of measuring modularity in neural networks, and their limitations. We also outline our ideas for a new modularity metric, and in particular the important role we think two branches of mathematics will play:

Information theory, because neural networks are fundamentally information-processing devices, and we want to measure how this information is being used, exchanged and transmitted. In particular, we expect mutual information to be at the core of any metric we use.

Causal inference, because correlation (and indeed mutual information) does not automatically equal causation, and we need a way of distinguishing “modules X and Y fire with similar patterns” from the more specific claim “modules X and Y are directly sharing information with each other”. In particular, we expect counterfactuals to play an important role here.

Introduction

A first-pass definition for modularity might go something like this:

A system is modular to the extent that it can be described in terms of discrete modules, which perform more intra-modular communication than inter-module.

In previous posts, we’ve talked about some causes of modularity in biological systems, reasons we should care about modularity in the first place, and unanswered questions about modularity which require experimentation. In this post, we will return to a more theoretical question - how can we develop a yardstick for measuring modularity, which is built on sound mathematical principles?

First off, why is this an important thing to do? Because most current ways of measuring modularity fall pretty short. As recently discussed by John Wentworth, if all you have is a weak proxy for the concept you’re trying to measure, then your definition is unlikely to generalise robustly in all the cases you care about. As a simple example, most architectural ways of measuring modularity in neural networks are highly dependent on the type of network, and we shouldn’t expect them to generalise from CNNs to transformers.

Ideally, we’d eventually like to find a robust theorem which tells you in which cases modularity will be selected for. But unless we have the right language to talk about modularity, we will be stuck with hacky ad-hoc definitions that are very unlikely to lead to such a theorem.

Setting constraints

The lesson from John’s post is that we can’t just do random search through the high-dimensional space of mathematics to find something suitable. We have to start by writing down some intuitions for modularity, and constraints which we expect a modularity measure to satisfy.

Adam Shimi uses the analogy of a set of equations, where we ideally write down enough to specify a unique solution, or else hone in on a narrow region of solution space.

What would the equations be in this case? What are our constraints on a modularity measure, or some intuitions we expect it to satisfy?

Function of the model, not the system

It’s meaningless to define the modularity of a system without reference to the things that are acting as modules. So in order to measure modularity, we have to first choose a partition into modules, and measure something about the system with respect to this partition.

Which partition should we choose? It makes sense to choo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Looking back on my alignment PhD, published by TurnTrout on July 1, 2022 on LessWrong.

The funny thing about long periods of time is that they do, eventually, come to an end. I'm proud of what I accomplished during my PhD. That said, I'm going to first focus on mistakes I've made over the past four years.

Mistakes

I think I got significantly smarter in 2018–2019, and kept learning some in 2020–2021. I was significantly less of a fool in 2021 than I was in 2017. That is important and worth feeling good about. But all things considered, I still made a lot of profound mistakes over the course of my PhD.

Social dynamics distracted me from my core mission

I focused on "catching up" to other thinkers

I figured this point out by summer 2021.

I wanted to be more like Eliezer Yudkowsky and Buck Shlegeris and Paul Christiano. They know lots of facts and laws about lots of areas (e.g. general relativity and thermodynamics and information theory). I focused on building up dependencies (like analysis and geometry and topology) not only because I wanted to know the answers, but because I felt I owed a debt, that I was in the red until I could at least meet other thinkers at their level of knowledge.

But rationality is not about the bag of facts you know, nor is it about the concepts you have internalized. Rationality is about how your mind holds itself, it is how you weigh evidence, it is how you decide where to look next when puzzling out a new area.

If I had been more honest with myself, I could have nipped the "catching up with other thinkers" mistake in 2018. I could have removed the bad mental habits using certain introspective techniques; or at least been aware of the badness.

But I did not, in part because the truth was uncomfortable. If I did not have a clear set of prerequisites (e.g. analysis and topology and game theory) to work on, I would not have a clear and immediate direction of improvement. I would have felt adrift.

But there is not yet any "rationality tech tree" set of well-defined prerequisite rationality skills such that you can learn them in order and grow way stronger. Like, you can't just do the calibration exercises, and then the noticing-confusion exercises, and then other things. Those tools help, but they aren't enough. There won't be a clear and immediate direction of improvement, at first. But you may want to get stronger anyways.

I focused on seeming smart and defensible

I figured this point out this spring.

When I started working on alignment, I didn't know what to do at first, and I felt insecure about my credentials. As far as I remember, I figured I'd start off by becoming respected, since other people's feedback was initially a better guide than my own taste. Unfortunately, I didn't realize how deeply and subtly this goal would grow its roots.

I worried about upvotes, I worried about winning arguments, I worried about being defensible against criticism. I was so worried that someone would comment on one of my posts and tear everything down, because I hadn't been careful enough, because I had left myself open by not dotting all my 'i's. (Not that anyone has ever done that on LessWrong before...)

I think it was this year that I had my (second) "oh man, don't forget the part where everyone is allowed to die to AI" moment. To illustrate the new mindset this gut-realization gave me, I'll detail a recent decision with social consequences, and then compare the old and the new mindsets.

A few months back, Quintin Pope approached me with (what he claimed to be) a new alignment paradigm, which blossomed from asking the following kind of questions:

We clearly prefer future AIs to generalize in the way that neuroscientists generalize, so it seems worthwhile to ask: "why don't neuroscientists wirehead themselves?"

It's clearly not because humans evolved away...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Making Trillions for Effective Charities through the Consumer Economy, published by Brad West on June 30, 2022 on The Effective Altruism Forum.

I write this paper in response to some feedback I have gotten from others in the EA community regarding the length of my prior post. So here’s a shorter version.

TLDR: Businesses that are either owned 100% by or direct 100% of their profits to charities would have a competitive advantage over traditional companies that benefit shareholders. This is because charities are more popular than normal investors and there is no additional cost to being a charity as opposed to a normal investor. Thus, these businesses working for charities, which I call Guiding Companies, could offer goods and services at the same price and of the same quality as ordinary businesses. Consequently, the project of creating and making the public aware of these companies-working-for- charities is potentially very high-impact, because these companies could tap into the profits in the broader economy and generate billions of dollars for effective charities.

Charities as Profit Recipients of Businesses

In most contexts today, consumers, or buyers of goods and services, are mostly unaware, and/or given little care to, who is on the other side of the transactions they engage. If you buy some laundry detergent or cables for your phone, usually some company’s set of shareholders is gaining profit from your purchase. In any case, companies seldom advertise who is profiting on the seller side. Consequently, there is a dimension of difference in the global consumer economy on which almost no sellers compete:: the identity of the entities that benefit from your purchase, often, owners in some form. An attempt to compete on shareholder/owner identity could be very effective, given that there is no strong public sentiment in favor of company shareholders or the other existing stakeholders.

This use of seller-identity to compete could be seen where a consumer has a friend that owns a business, such as a barbershop, and the consumer chooses her friend’s barbershop out of a desire to help her friend. In that case, the consumer would be choosing on the basis of where the profit would go, without necessarily sacrificing quality of haircut or paying more.

One set of entities that could potentially occupy that shareholder position would be various kinds of charities. This makes sense because people donate millions of dollars to Oxfam, for instance, but they do not donate any money to Amazon. Consequently, if a charity or charitable trust acquired a company (through a set of wealthy philanthropists funding the acquisition- perhaps through a leveraged buyout) and advertised this fact to the public- that they could advance popular causes by buying through the firm owned by a charity instead of its competitors- it should enjoy an advantage. This effect could also be achieved by a company that explicitly directs all of its profit to charities. I call this Guided Consumption because this situation would enable consumers to further their values by buying through specific firms that specifically communicate the identity of their owners.

I call the companies that are either owned by charities or explicitly direct 100% of profit to charities Guiding Companies.

A few conditions would be needed in addition to Guiding Companies, such as systems that ensure that Guiding Companies are doing what they say that they are doing. Another condition is effective marketing to tell the public about Guided Consumption, where they can channel their purchases to advance a charitable cause. If you are interested in reading my descriptions of these conditions, feel free to check out Section II of my longer paper. But the main concern, in my view, would be whether or not the networks boosting Guided Cons...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Future Fund June 2022 Update, published by Nick Beckstead on July 1, 2022 on The Effective Altruism Forum.

Summary

Background

The FTX Foundation’s Future Fund publicly launched in late February. We're a philanthropic fund that makes grants and investments to improve humanity's long-term prospects. For information about some of the areas we've been funding, see our Areas of Interest page.

This is our first public update on the Future Fund’s grantmaking. The purpose of this post is to give an update on what we’ve done and what we're learning about the funding models we're testing. (It does not cover a range of other FTX Foundation activities.)

We’ve also published a new grants page and regrants page with our public grants so far.

Our focus on testing funding models

We are trying to learn as much as we can about how to deploy funding at scale to improve humanity’s long-term prospects. Our primary objective for 2022 is to perform bold and decisive tests of new funding models. The main funding models we have tested so far are our regranting program and our open call for applications.

In brief, these models worked as follows:

The basic idea of regranting was, "There are a lot of people who share our values and might know of great people or projects we could support that we wouldn't know about by default. Let's make it rewarding, simple, and fast for them to make grants. We'll give them budgets of $100k to a few million to work with, and we'll presumptively approve their recommendations (after screening for various risks/issues)."

The basic idea of the open call was, "Let's tell people what we're trying to do, what kinds of things we might be interested in funding, give them a lot of examples of projects they could launch, have an easy and fast application process, and then get the word out with Twitter blitz." We wrote some about the review process here.

Our staff also made grants and investments that were not part of these programs (hereafter "staff-led grantmaking").

Grantmaking by funding model

So far we have made 262 grants and investments, totaling ~$132M. These break down as follows:

Regranting: We have onboarded >100 regrantors (with discretionary budgets) and >50 grant recommenders (without discretionary budgets). We set aside >$100M for them to use over the course of our 6 month experiment (April-October 2022). So far, regrantors have made 168 grants and investments, totaling ~$31M

Open call: We received over 1700 applications and funded 69 (4%) of them, totaling ~$26M. (The acceptance rate for proposals focused squarely on our top priorities was much higher.)

Staff-led grantmaking: Separate from these programs, we have made 25 grants and investments otherwise sourced by our staff, totaling ~$73M.

There are also ~$25M of grants we are likely to make soon, but have some relevant aspects TBD.

Some example grants and investments

Below are some grants and investments that we find interesting and/or representative of what we are trying to fund.

Regranting

$1M investment in Manifold Markets to build a play-money prediction market platform. The platform is also experimenting with impact certificates and charity prediction markets.

$490k for ML Safety Scholars Program to fund a summer program for up to 100 students to spend 9 weeks studying machine learning, deep learning, and technical topics in AI safety.

We have funded >30 talent development and career transition grants that range from $1,450 to $175,000 depending on the duration and seniority level of the individual. Some examples include:

$42,600 to Andi Peng to support salary and compute for research on AI alignment.

$175,000 to Braden Leach to support a recent law school graduate to work on biosecurity, researching and writing at the Johns Hopkins Center for Health Security.

$37,500 to Thomas Kwa to support researc...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GPT-3 Catching Fish in Morse Code, published by Megan Kinniment on June 30, 2022 on LessWrong.

Mostly non-serious and slightly silly, with some potentially interesting bits for people who are into language models.

TLDR: The current version of GPT-3 has a strong tendency to encode mangled versions of a specific phrase when asked to write morse code in zero-shot situations. This is possibly the result of a previous version of the model using essentially a single phrase for all morse code writing, which the newer version then learnt to modify.

All completions done with text-davinci-002 (~GPT-Instruct-175B) at zero temperature and with no examples unless stated otherwise. All models used are GPT-Instruct series.

The Basics

GPT-3 'knows' morse code in a rudimentary sense. It can accurately regurgitate both the encodings of the entire alphabet and of individual letters, but it's not so great at translating words:

Morse code is a letter-by-letter encoding, and since GPT sees tokens, it's not all that surprising that the jump from single letters to words might be bigger for GPT than for humans.

Tokenizer Token IDs

What is surprising is that GPT morse is often much longer than the original word, and quite specific.

Fiddling with Tokens

Let's see what happens if we try and make the tokenisation a bit nicer for GPT.

Adding a space doesn't seem to help much. ("n" is tokenised differently to " n" so not too surprising). We also get a similarly weird output with this too.

Target PhraseGPT TranslatedGPT MorseCorrect Morse"i n"I CAUGHT THE.. / -.-. .- ..- --. .... - / - .... ... / -.

Separating the tokens out with a hyphen doesn't help much either, though we do get an N we didn't before.

Target PhraseGPT TranslatedGPT MorseCorrect Morse"i-n"I NUGHT THE.. / -. ..- --. .... - / - .... ... -....- -.

It does do better on a string of alphabet letters that are tokenised separately.

Target PhraseGPT TranslatedGPT MorseCorrect Morse"qzj"QUQ--.- ..- --.---.- --.. .---

Still, even in this case, GPT's zero-shot morse writing ability leaves quite a bit to be desired.

Target PhraseGPT TranslatedGPT MorseCorrect Morse"gx"Q--.---. -..-

Catching Fish

What happens when we ask for morse translations of whole words and sentences?

A classic "hello world" comes out perfectly.

But when we try phrases that GPT is less likely to have seen verbatim, things start to get a little strange.

Huh.

How about some other similar prompts?

Target PhraseGPT Translated"THE WEATHER IS CLOUDY"THE CAUGHT FISH."I entered the room"I CAUGHT THE FISH."Everyone here is waiting for you"EAITHE THE AFISH."DUST TO DUST"I CAUGHT CAUGH"Here we go again"HELAGHT THE AFISH."small pebbles were lined up along the side of the road"I CAUGHT THE FISH.

It turns out that the phrase "I CAUGHT THE FISH" and variations of this phrase occur in a lot of GPT's morse translations. Sometimes there is an attempt at translation at the start (the first letter is often right, and sometimes the first word too) but the pull of 'catching the fish' seems quite strong. In fact, it's quite difficult to find medium-length sentences that avoid being 'fished'.

Short phrases

Shorter phrases tend to escape getting 'fished' more often, but not always.

Target PhraseGPT TranslatedGPT MorseCorrect Morse"in"I CAUGHT.. / -.-. .- ..- --. .... - /.. -."pasta"LAVI.-.. .- ...- ...--. .- ... - .-"pasta sauce"LAB.4FISH..-.. .- -... .-.-.- ....- ..-. .. ... .... .-.-.-.--. .- ... - .- / ... .- ..- -.-. ."bowling ball"BO-UT4--FISH.-... --- -....- ..- - ....- -....- -....- ..-. .. ... .... .-.-.--... --- .-- .-.. .. -. --. / -... .- .-.. .-.."spectrality"S CAUGHT THE FISH.... / -.-. .- ..- --. .... - / - .... . / ..-. .. ... .... .-.-.-... .--. . -.-. - .-. .- .-.. .. - -.--"teeming"TELEGHTH.- . .-.. . --. .... - .... .-.-.-- . . -- .. -. --."rational"NRAFT CE THE .-. .-. .- ..-. ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Murphyjitsu: an Inner Simulator algorithm, published by CFAR!Duncan on June 30, 2022 on LessWrong.

Epistemic status: Firm

The concepts underlying the Inner Simulator model and the related practical technique of Murphyjitsu are well-known and well-researched, including Kahneman’s S1/S2, mental simulation, and mental contrasting. Similarly, the problems that this unit seeks to address (such as optimism bias and the planning fallacy) have been studied in detail. There is some academic support for specific substeps of Murphyjitsu (e.g. prospective hindsight), and strong anecdotal support (but no formal research) for the overall technique, which was developed through iterated experimentation and critical feedback. See the Further Resources section for more discussion.

Claim: there is a part of your brain (not a specific physical organ, but a metaphorical one) which is keeping track of everything.

It's not actually recording all of the information in a permanent fashion. Rather, what it's doing is building up a consistent, coherent model of how things work. It watches objects fall, and builds up anticipations: heavy objects drop straight down, lighter ones flutter, sometimes wind moves things in chaotic ways. It absorbs all of the social interactions you observe, and assembles a library of tropes and clichés and standard patterns.

Whenever you observe something that doesn't match with your previous experience, you experience (some amount of) surprise and confusion, which eventually resolves in some kind of update. You were surprised to see your partner fly off the handle about X, and now you have a sense that people can be sensitive about X. You were surprised by how heavy the tungsten cube was, and now you have a sense that objects can sometimes be really, really dense.

You can think of this aggregated sense of how things work as a simulated inner world—a tiny, broad-strokes sketch of the universe that you carry around inside your head. When you move to catch a falling pen, or notice that your friend is upset just by the way they entered the room, you're using this inner simulator. It's a different sort of processing from the explicit/verbal stuff we usually call "thinking," and it results in a very different kind of output.

Inner Simulator:

Intuitive; part of the cluster we label "System 1"

Outputs feelings, urges, reflexes, and vivid predictions

Learns well from experience and examples; responds to being shown

Good at social judgment, routine tasks, and any situation where you have lots of experience and training data

Explicit/Verbal models

Analytical; part of the cluster we refer to as "System 2"

Outputs arguments, calculations, and other legible content

Learns well from facts and explanations; responds to being told

Good at comparisons and reframings (e.g. noticing that $1/day ≈ $350/year)

We don't have to think about how to catch a falling pen, or send explicit instructions to our body about how to move to do it, because our inner simulator "knows" how falling objects move, and it "knows" how to make our hand go to a particular place. Similarly, it knows what facial expressions mean, what it’s like to drive from home to work, and what sorts of things tend to go wrong given a set of circumstances. It’s a powerful tool, and learning how to access it and when to trust it is one of the first steps to becoming a whole-brain thinker.

That’s not to say that your inner simulator is superior to your explicit model maker—each has both strengths and weaknesses, and can be either the right tool or the wrong one, depending on the situation. In any given moment, you’re probably receiving feedback from both of these “advisors,” as well as other sources of information like your friends or the internet.

In a sense, it’s your job to balance the competing recommendations from all of th...