The Nonlinear Library: Alignment Forum

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: what does davidad want from "boundaries"?, published by Chipmonk on February 6, 2024 on The AI Alignment Forum.

Chipmonk

As the Conceptual Boundaries Workshop (website) is coming up, and now that we're also planning Mathematical Boundaries Workshop in April, I want to get more clarity on what exactly it is that you want out of "boundaries"/membranes.

So I just want to check: Is your goal with boundaries just to formalize a moral thing?

I'll summarize what I mean by that:

Claim 1: By "boundaries", you mean "the boundaries around moral patients - namely humans".

Claim 1b: And to some degree also the boundaries around plants and animals. Also maybe nations, institutions, and other things.

Claim 2: If we can just

(i) locate the important boundaries in the world, and then

(ii) somehow protect them,

Then this gets at a lot (but not all!) of what the "safety" in "AI safety" should be.

Claim 3: We might actually be able to do that.

e.g.: Markov blankets are a natural abstraction for (2.i).

Claim 4: Protecting boundaries won't be sufficient for all of "safety" and there are probably also other (non-boundaries) specifications/actions that will also be necessary.

For example, we would probably also need to separately specify some things that aren't obviously contained by the boundaries we mean, e.g.: "clean water", "clean air", and a tractably small set of other desiderata.

Here are my questions for you:

Q1: Do you agree with each of the claims above?

Q2: Is your goal with boundaries just to formalize the moral/safety thing, or is there anything else you want from boundaries?

Past context that's also relevant for readers:

This new post I wrote about how preserving the boundaries around agents seems to be a necessary condition for their safety.

Quotes you've made about boundaries that I've compiled here.

This old post I wrote about boundaries as MVP morality which you endorsed.

Q3: It seems that Garrabrant, Critch, and maybe others want different things from you and I'm wondering if you have thoughts about that.

Garrabrant: From talking to him I know that he's thinking about boundaries too but more about boundaries in the world as instruments to preserve causal locality and predictability and evolution etc.. But this is quite different than talking about specifically the boundaries around agents.

Critch: I haven't spoken to him yet, but I think you once told me that Critch seems to be thinking about boundaries more in terms of ~"just find the 'boundary protocol' and follow it and all cooperation with other agents will be safe". Is this right? If so, this seems closer to what you want, but still kinda different.

TJ: I think TJ has some other ideas that I am currently unable to summarize.

davidad

Claim 1+1b: yes, to first order. [To second order, I expect that the general concept of things with "boundaries" will also be useful for multi-level world-modelling in general, e.g.

coarse-graining fluid flow by modelling it in terms of cells that have boundaries on which there is a net flow, and that it might be a good idea to "bake in" something like a concept of boundaries to an AI system's meta-ontology, so that it has more of a tendency to have moral patients among the entities in its object-level ontology.

But my mainline intention is for the object-level ontology to be created with humans in the loop, and the identification of entities with boundaries could perhaps be just as easily a layer of interpretation on top of an ontology with a more neutral meta-ontology of causation. Thinking through both routes more is at the frontier of what I consider "conceptual "boundaries" research".]

davidad

Claim 2: agreed.

Claim 3: agreed.

Claim 4: agreed.

davidad

Q2: yes, my ultimate goal with "boundaries" is just to formalise injunctions against doing harm, disrespecting autonomy, or (at the mo...

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Preventing exfiltration via upload limits seems promising, published by Ryan Greenblatt on February 6, 2024 on The AI Alignment Forum.

At some point in the future, AI developers will need to ensure that when they train sufficiently capable models, the weights of these models do not leave the developer's control. Ensuring that weights are not exfiltrated seems crucial for preventing threat models related to both misalignment and misuse. The challenge of defending model weights has previously been discussed in

a RAND report.

In this post, I'll discuss a point related to preventing weight exfiltration that I think is important and under-discussed: unlike most other cases where a defender wants to secure data (e.g.

emails of dissidents or source code), model weights are very large files. At the most extreme, it might be possible to set a limit on the total amount of data uploaded from your inference servers so that an attacker would be unable to exfiltrate the model weights even if they totally compromised your inference servers, while still being able to serve an API and otherwise run a normal amount of inference.

If this ends up being viable, then it would be much easier to protect model weights from competent adversaries because upload limits are relatively simple to enforce. Even if it turns out that such a bandwidth limit isn't feasible, the fact that any attacker will have to control a substantial fraction of upload bandwidth from your inference server might pose a substantial obstacle to exfiltration.

In this post:

I make some predictions about the ratio between a model's size and the total quantity of data that its inference servers will have to emit over the model lifetime. I conclude that the total quantity of data probably won't be more than a few orders of magnitude larger than the size of the model for an AI lab's most powerful AI.

I suggest a variety of strategies to reduce the outflow bandwidth required from inference services. Most importantly, you can use a scheme involving arithmetic coding using a weak model that you are okay with being stolen. In this scheme, the weak model is trained to imitate the strong model. The weak model is present both inside and outside the inference network with the upload limit.

While I expect that the sort of proposal I discuss here is well known, there are many specific details I discuss here which I haven't seen discussed elsewhere. If you are reasonably familiar with this sort of proposal, consider just reading the "Summary of key considerations" section which summarizes the specific and somewhat non-obvious points I discuss in this post.

This proposal is written as a nearcast focused on SOTA LLMs, though I expect many of the conclusions to generalize.

Given how promising this proposal seems, I think that further investigation is warranted. The main source of uncertainty is about the ratio between the number of inference tokens generated and the number of model parameters for the key model we want to protect. There are a variety of improvements which might allow for somewhat reducing total uploads, so pursuing these could be quite leveraged if we end up in a regime where marginal reduction in uploads substantially reduces risk.

The viability of this proposal depends substantially on non-public information that AI labs possess, so internal investigation by AI labs will likely be key. However, external researchers could investigate compression schemes, other approaches for reducing the total uploads, or mechanisms for very reliably and securely tracking the total amount of data uploaded. I'm excited about further investigation of this idea.

Summary of key considerations

The total number of generated tokens from a given model might be similar to or smaller than the total number of parameters due to Chinchilla scaling laws....

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Attention SAEs Scale to GPT-2 Small, published by Connor Kissane on February 3, 2024 on The AI Alignment Forum.

This is an interim report that we are currently building on. We hope this update + open sourcing our SAEs will be useful to related research occurring in parallel. Produced as part of the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort

Executive Summary

In a previous post, we showed that

sparse autoencoders (SAEs) work on the attention layer outputs of a two layer transformer. We scale our attention SAEs to GPT-2 Small, and continue to find sparse interpretable features in every layer. This makes us optimistic about our ongoing efforts scaling further, especially since we didn't have to do much iterating

We open source our SAEs. Load them from

Hugging Face or this

colab notebook

The SAEs seem good, often recovering more than 80% of the loss relative to zero ablation, and are sparse with less than 20 features firing on average. The majority of the live features are interpretable

We continue to find the same three feature families that we found in the

two layer model:

induction features,

local context features, and

high level context features. This suggests that some of our lessons interpreting features in smaller models may generalize

We also find new, interesting feature families that we didn't find in the two layer model, providing hints about fundamentally different capabilities in GPT-2 Small

See our

feature interface to browse the first 30 features for each layer

Introduction

In

Sparse Autoencoders Work on Attention Layer Outputs we showed that we can apply SAEs to extract sparse interpretable features from the last attention layer of a two layer transformer. We have since applied the same technique to a 12-layer model, GPT-2 Small, and continue to find sparse, interpretable features in every layer. Our SAEs often recover more than 80% of the loss[1], and are sparse with less than 20 features firing on average. We perform shallow investigations of the first 30 features from each layer, and we find that the majority (often 80%+) of non-dead SAEs features are interpretable.

interactive visualizations for each layer.

We open source our SAEs in hope that they will be useful to other researchers currently working on dictionary learning. We are particularly excited about using these SAEs to better understand attention circuits at the feature level. See the SAEs on

Hugging Face or load them using this

colab notebook. Below we provide the key metrics for each SAE:

L0 norm

loss recovered

dead features

% alive features interpretable

L0

3

99%

13%

97%

L1

20

78%

49%

87%

L2

16

90%

20%

95%

L3

15

84%

8%

75%

L4

15

88%

5%

100%

L5

20

85%

40%

82%

L6

19

82%

28%

75%

L7

19

83%

58%

70%

L8

20

76%

37%

64%

L9

21

83%

48%

85%

L10

16

85%

41%

81%

L11

8

89%

84%

66%

It's worth noting that we didn't do much differently to train these,[2] leaving us optimistic about the tractability of scaling attention SAEs to even bigger models.

Excitingly, we also continue to identify feature families. We find features from all three of the families that we identified in the two layer model: induction features, local context features, and high level context features. This provides us hope that some of our lessons from interpreting features in smaller models will continue to generalize.

We also find new, interesting feature families in GPT-2 Small, suggesting that attention SAEs can provide valuable hints about new[3] capabilities that larger models have learned. Some new features include:

Successor features, which activate when predicting the next item in a sequence such as "15, 16" -> "17" (which are partly coming from

Successor Heads in the model), and boost the logits of the next item.

Name mover features, which predict a name in the context, such as in the

IOI task

Duplicate token f...

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Survey for alignment researchers: help us build better field-level models, published by Cameron Berg on February 2, 2024 on The AI Alignment Forum.

AE

Studio is launching a

short, anonymous survey for alignment researchers, in order to develop a stronger model of various field-level dynamics in alignment.

This appears to be an interestingly neglected research direction that we believe will yield specific and actionable insights related to the community's technical views and more general characteristics.

The survey is a straightforward 5-10 minute Google Form with some simple multiple choice questions.

For every alignment researcher who completes the survey, we will donate $40 to a high-impact AI safety organization of your choosing (see specific options on the survey). We will also send each alignment researcher who wants one a customized report that compares their personal results to those of the field.

Together, we hope to not only raise some money for some great AI safety organizations, but also develop a better field-level model of the ideas and people that comprise alignment research.

We will open-source all data and analyses when we publish the results. Thanks in advance for participating and for sharing this around with other alignment researchers!

Survey full link:

https://forms.gle/d2fJhWfierRYvzam8

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Source Sparse Autoencoders for all Residual Stream Layers of GPT2-Small, published by Joseph Isaac Bloom on February 2, 2024 on The AI Alignment Forum.

UPDATE: Since we posted this last night, someone pointed out that our implementation of ghost grads has a non-trivial error (which makes the results a-priori quite surprising). We computed the ghost grad forward pass using Exp(Relu(W_enc(x)[dead_neuron_mask])) rather than Exp((W_enc(x)[dead_neuron_mask])). I'm running some ablation experiments now to get to the bottom of this.

This work was produced as part of the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort, under mentorship from Neel Nanda and Arthur Conmy. Funding for this work was provided by the Manifund Regranting Program and donors as well as LightSpeed Grants.

This is intended to be a fairly informal post sharing a set of Sparse Autoencoders trained on the residual stream of GPT2-small which achieve fairly good reconstruction performance and contain fairly sparse / interpretable features. More importantly, advice from Anthropic and community members has enabled us to train these fairly more efficiently / faster than before.

The specific methods that were most useful were: ghost gradients, learning rate warmup, and initializing the decoder bias with the geometric median. We discuss each of these in more detail below.

5 Minute Summary

We're publishing a set of 12 Sparse AutoEncoders for the GPT2 Small residual stream.

These dictionaries have approximately 25,000 features each, with very few dead features (mainly in the early layers) and high quality reconstruction (log loss when the activations are replaced with the output is 3.3 - 3.6 as compared with 3.3 normally).

The L0's range from 5 in the first layer to 70 in the 9th SAE (increasing by about 5-10 per layer and dropping in the last two layers.

By choosing a fixed dictionary size, we can see how statistics like the number of dead features or reconstruction cross entropy loss change with layer giving some indication of how properties of the feature distribution change with layer depth.

We haven't yet extensively analyzed these dictionaries, but will share automatically generated dashboards we've generated.

Readers can access the Sparse Autoencoder weights in this

HuggingFace Repo. Training code and code for loading the weights / model and data loaders can be found in this

Github Repository. Training curves and feature dashboards can also be found in this

wandb report. Users can download all 25k feature dashboards generated for layer 2 and 10 SAEs and the first 5000 of the layer 5 SAE features

here (note the left hand of column of the dashboards should currently be ignored).

Layer

Variance Explained

L1 loss

L0*

% Alive Features

Reconstruction

CE Log Loss

0

99.15%

4.58

12.24

80.0%

3.32

1

98.37%

41.04

14.68

83.4%

3.33

2

98.07%

51.88

18.80

80.0%

3.37

3

96.97%

74.96

25.75

86.3%

3.48

4

95.77%

90.23

33.14

97.7%

3.44

5

94.90%

108.59

43.61

99.7%

3.45

6

93.90%

136.07

49.68

100%

3.44

7

93.08%

138.05

57.29

100%

3.45

8

92.57%

167.35

65.47

100%

3.45

9

92.05%

198.42

71.10

100%

3.45

10

91.12%

215.11

53.79

100%

3.52

11

93.30%

270.13

59.16

100%

3.57

Original Model

3.3

Summary Statistics for GPT2 Small Residual Stream SAEs. *L0 = Average number of features firing per token.

Training SAEs that we were happy with used to take much longer than it is taking us now. Last week, it took me 20 hours to train a 50k feature SAE on 1 billion tokens and over the weekend it took 3 hours for us to train 25k SAE on 300M tokens with similar variance explained, L0 and CE loss recovered.

We attribute the improvement to having implemented various pieces of advice that have made our lives a lot easier:

Ghost Gradients / Avoiding Resampling: Prior to ghost gradients (which we were made aware of last week in the Anthropic Jan...

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evaluating Stability of Unreflective Alignment, published by james.lucassen on February 1, 2024 on The AI Alignment Forum.

This post has an accompanying SPAR project! Apply here if you're interested in working on this with me.

Huge thanks to Mikita Balesni for helping me implement the MVP. Regular-sized thanks to Aryan Bhatt, Rudolph Laine, Clem von Stengel, Aaron Scher, Jeremy Gillen, Peter Barnett, Stephen Casper, and David Manheim for helpful comments.

0. Key Claims

Most alignment work today doesn't aim for alignment that is stable under value-reflection[1].

I think this is probably the most sensible approach to alignment.

But there is a threat model which could potentially be a serious obstacle to this entire class of alignment approaches, which is not currently being tracked or evaluated. It goes like this:

Long-horizon capabilities require a particular capability I call "stepping back" (described in more detail in the body of the post)

Stepping back seems likely to be learned in a very generalizable way

If stepping back generalizes in what seems like the simplest / most natural way, it will create a tendency for AIs to do value-reflection

If we're unable to prevent this value-reflection, it will cause any AI whose alignment is not stable under value-reflection to become misaligned

However, I'm quite uncertain about whether or not this threat model will go through, for a few reasons:

The chain of argument is highly conjunctive and therefore fragile.

The track record of anthropomorphic reasoning about which AI capabilities will imply which other AI behaviors seems poor.

There is an intuitive counterexample provided by humans, at least at human-level intelligence.

Even given this uncertainty, work on this threat model seems very neglected given the wide swath of highly-invested-in alignment approaches that it could affect.

I don't think further abstract argument about the likelihood of this threat model will reduce my uncertainty much. Instead, I propose some evals to keep track of each component of this threat model:

Evaluate the stepping back capabilities of frontier AIs in domains where they receive the most long-horizon training.

Evaluate the generalization of frontier AIs stepping back behavior to other task domains.

Evaluate how difficult it is to decouple the capabilities of frontier AIs between domains.

Evaluate how difficult it is to prevent frontier AIs from thinking certain "forbidden thoughts" even if those thoughts are incentivized by the task setting

1. Reflective vs Unreflective Alignment

I want to start by making an explicit distinction between two importantly different uses of the word "alignment". The distinction is between alignment that aims to be preserved even under value-reflection by the AI, and alignment that does not aim for that. I will call alignment approaches that aim for stability under value-reflection "reflective alignment", and approaches that do not "unreflective alignment".

One easy way to connect this distinction to your existing thinking is by checking if you think of alignment as a binary or as a spectrum. Unreflective alignment naturally admits a spectrum formulation - alignment which is stable under X amount of value-reflection (or other destabilizing pressures). Reflective alignment, on the other hand, is typically thought of as a binary[2] - either the AI converges to the intended values in the limit, or it goes to some other fixed point instead.

In the current alignment discussion, iterative prosaic approaches generally propose to work by ensuring "alignment increases faster than capabilities". This is clearly an alignment-as-spectrum framing, suggesting the goal in mind is unreflective alignment. For further confirmation, consider the type of step that is considered an increase in alignment - going from baseline GPT-3 t...

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: PIBBSS Speaker events comings up in February, published by DusanDNesic on February 1, 2024 on The AI Alignment Forum.

TLDR: The PIBBSS Speaker Series is happening in mid-February. Read more about the talks and register

on our website

. Sign up for future updates about our speaker events

here.

PIBBSS

regularly invites researchers from a wide range of fields studying intelligent behaviour and exploring their connections to the problem of AI Risk and Safety.

In the past, we had speakers such as

Michael Levin talking about

Morphogenesis as a form of intelligent problem-solving building towards a framework of Basal Cognition; J

osh Bongard about

efforts to build hybrid intelligent 'agents' at the intersection of biology, cognition and engineering; Miguel Aguilera on

Nonequilibrium Neural Computation; Steve Byrnes on

Challenges for Safe and Beneficial Brain-Like AI; or Tang Zhi Xuan on

Modeling Bayesian Bounded Rational Agents

.

We have a couple of speaker events are coming up in February.

Konrad Körding on "Interpretability of brains and neural networks"

Feb 12, 9AM PST/5PM UTC (

Register)

Daniel Polani on "Information and its Flow: a Pathway towards Intelligence"

Feb 16, 9AM PST/5PM UTC, (

Register)

Nathaniel Virgo on "Mathematical Perspectives on Goals and Beliefs in the Physical World: Extending on the 'Good Regulator Theorem'"

Feb 19, 6PM PST,/20th 2AM UTC, (

Register)

If you want to be informed about future events (we typically won't post about this going forward), signup

here to our speaker event specific email list.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Last call for submissions for TAIS 2024!, published by Blaine William Rogers on January 30, 2024 on The AI Alignment Forum.

We are very close to the submission deadline for

TAIS 2024. Alignment research moves fast; if there's something you've been working on the for last few months that you think would be interesting to our attendees, please submit! You can find a draft agenda on our website.

Timeline

2024-01-31: Submission deadline

2024-02-29: Notification of acceptance / rejection

2024-04-05: Conference!

Details

The Technical AI Safety Conference will bring together specialists in the field of technical safety to share their research and benefit from each others' expertise. We want our attendees to involve themselves in deep conversations throughout the conference with the brightest minds in AI Safety.

We seek to launch this forum for academics, researchers and professionals who are doing technical work in these or adjacent fields:

Mechanistic interpretability

Scalable oversight

Causal incentive analysis

Agent foundations

Singular learning theory

Argumentation

Emergent agentic phenomena

Thermodynamic / statistical-mechanical analyses of computational systems

We invite the submission of research papers, or extended abstracts, that deal with related topics. We particularly encourage submissions from researchers in the ALIFE community who would not otherwise have considered submitting to TAIS 2024.

Our sponsors, Noeon Research, have kindly provided funding to cover travel and accommodation for authors of accepted presentations.

To submit, please send a title and abstract to [email protected].

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The case for more ambitious language model evals, published by Arun Jose on January 30, 2024 on The AI Alignment Forum.

Here are some capabilities that I expect to be pretty hard to discover using an RLHF'd chat LLM:

Eric Drexler tried to use the GPT-4 base model as a writing assistant, and it [...] knew who he was from what he was writing. He tried to simulate a conversation to have the AI help him with some writing he was working on, and the AI simulacrum repeatedly insisted it was by Drexler.

A somewhat well-known Haskell programmer - let's call her Alice - wrote two draft paragraphs of a blog post she wanted to write, began prompting the base model with it, and after about two iterations it generated a link to her draft blog post repo with her name.

More generally, this is a cluster of capabilities that could be described as language models inferring a surprising amount about the data-generation process that produced its prompt, such as the identity, personality, intentions, or history of a user[1].

The reason I expect most capability evals people currently run on language models to miss out on most abilities like these is primarily that they're most naturally observed when dealing with much more open-ended contexts. For instance, continuing text as the user, predicting an assistant free to do things that could superficially look like hallucinations[2], and so on. Most evaluation mechanisms people use today involve testing the ability of fine-tuned[3] models to perform a broad array number of specified tasks in some specified contexts, with or without some scaffolding - a setting that doesn't lend itself very well toward the kind of contexts I describe above.

A pretty reasonable question to ask at this point is why it matters at all whether we can detect these capabilities. A position one could have here is that there are capabilities much more salient to various takeover scenarios that are more useful to try and detect, such as the ability to phish people, hack into secure accounts, or fine-tune other models. From that perspective, evals trying to identify capabilities like these are just far less important. Another pretty reasonable position is that these particular instances of capabilities just don't seem very impressive, and are basically what you would expect out of language models.

My response to the first would be that I think it's important to ask what we're actually trying to achieve with our model eval mechanisms. Broadly, I think there are two different (and very often overlapping) things we would want our capability evals[4] to be doing:

Understanding whether or not a specific model is possessed of some dangerous capabilities, or prone to acting in a malicious way in some context.

Giving us information to better forecast the capabilities of future models. In other words, constructing good scaling laws for our capability evals.

I'm much more excited about the latter kind of capability evals, and most of my case here is directed at that. Specifically, I think that if you want to forecast what future models will be good at, then by default you're operating in a regime where you have to account for a bunch of different emergent capabilities that don't necessarily look identical to what you've already seen.

Even if you really only care about a specific narrow band of capabilities that you expect to be very likely convergent to takeover scenarios - an expectation I don't really buy as something you can very safely assume because of the uncertainty and plurality of takeover scenarios - there is still more than one way in which you can accomplish some subtasks, some of which may only show up in more powerful models.

As a concrete example, consider the task of phishing someone on the internet. One straightforward way to achieve this would be to figure out how...

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Agents that act for reasons: a thought experiment, published by Michele Campolo on January 24, 2024 on The AI Alignment Forum.

Posted also on the EA Forum.

In

Free agents I've given various ideas about how to design an AI that reasons like an independent thinker and reaches moral conclusions by doing so. Here I'd like to add another related idea, in the form of a short story / thought experiment.

Cursed

Somehow, you have been cursed. As a result of this unknown curse that is on you now, you are unable to have any positive or negative feeling. For example, you don't feel pain from injuries, nothing makes you anxious or excited or sad, you can't have fun anymore. If it helps you, imagine your visual field without colours, only with dull shades of black and white that never feel disgusting or beautiful.

Before we get too depressed, let's add another detail: this curse also makes you immune to death (and other states similar to permanent sleep or unconsciousness). If you get stabbed, your body magically recovers as if nothing happened. Although this element might add a bit of fun to the story from our external perspective, keep in mind that the cursed version of you in the story doesn't feel curious about anything, nor has fun when thinking about the various things you could do as an immortal being.

No one else is subject to the same curse. If you see someone having fun and laughing, the sentence "This person is feeling good right now" makes sense to you: although you can't imagine nor recall what feeling good feels like, your understanding of the world around you remained intact somehow. (Note: I am not saying that this is what would actually happen in a human being who actually lost the capacity for perceiving

valence. It's a thought experiment!)

Finally, let's also say that going back to your previous life is not an option. In this story, you can't learn anything about the cause of the curse or how to reverse it.

To recap:

You can't feel anything

You can't die

You can't go back to your previous state

The curse only affects you. Others' experiences are normal.

In this situation, what do you do?

In philosophy, there is some discourse around reasons for actions, normative reasons, motivating reasons, blah blah blah. Every philosopher has their own theory and uses words differently, so instead of citing centuries of philosophical debates, I'll be maximally biased and use one framework that seems sensible to me. In

Ethical Intuitionism, contemporary philosopher Michael Huemer distinguishes "four kinds of motivations we are subject to":

Appetites: examples are hunger, thirst, lust (simple, instinctive desires)

Emotions: anger, fear, love (emotional desires, they seem to involve a more sophisticated kind of cognition than appetites)

Prudence: motivation to pursue or avoid something because it furthers or sets back one's own interests, like maintaining good health

Impartial reasons: motivation to act due to what one recognises as good, fair, honest, et cetera.

You can find more details in section 7.3 of the book.

We can interpret the above thought experiment as asking: in the absence of appetites and emotions - call these two "desires", if you wish - what would you do? Without desires and without any kind of worry about your own death, does it still make sense to talk about self-interest? What would you do without desires and without self-interest?

My guess is that, in the situation described in Cursed, at least some, if not many, would decide to do things for others. The underlying intuition seems to be that, without one's own emotional states and interests, one would prioritise others' emotional states and interests, simply due to the fact that nothing else seems worth doing in that situation.

In other words, although one might need emotional states to first develop an accu...