The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Hiring: hacks + pitfalls for candidate evaluation, published by Cait Lion on September 14, 2023 on The Effective Altruism Forum.

This post collects some hacks - cheap things that work well - and common pitfalls I've seen in my experience hiring people for CEA.

Hacks

Sharing customised-generic feedback

Rejected candidates often, very reasonably, desire feedback. Sometimes you don't have capacity to tailor feedback to each candidate, particularly at earlier stages of the process. If you have some brief criteria describing what stronger applicants or stronger trial tasks submissions should look like, and if that's borne out in your decisions about who to progress, I suggest writing out a quick description of what abilities, traits or competencies the successful candidates tended to demonstrate. This might be as quick as "candidates who progressed to the next stage tended to demonstrate a combination of strong attention to detail in the trial task, demonstrated a clear and direct writing style, and have professional experience in operations." This shouldn't take more than a few minutes to generate. My impression is it's a significant improvement for the candidates over a fully generic response.

Consider borrowing assessment materials

Not sure how to test a trait for a given role? Other aligned organisations might have already created evaluation materials tailored to the competencies you want to evaluate. If so, that organization might let you use their trial task for your recruitment round.

Ideally, you can do this in a way that's pretty win-win for both orgs (e.g. Org A borrows a trial task from Org B. Org A then asks their candidates to agree that, should they ever apply to Org B, Org A will send over the results of the assessment).

I have done this in the past and it worked out well.

Beta test your trial tasks!

I'm a huge proponent of beta testing new evaluation materials. Testing your materials before sending them to candidates can save you a world of frustration down the road by helping you tweak unclear instructions, inappropriate time limits, and a whole host of other pitfalls.

Mistakes

Taken from our internal hiring resources, here are some mistakes we've made in the past with our evaluation materials:

Trial tasks or tests that are laborious to grade

Some types of work tests take a long time to grade effectively. Possible issues can be: a large amount to read, multiple sources or links to check for information, a complicated or difficult-to-apply rubric. Every extra minute this takes you to grade is multiplied by the number of tasks. The ideal work sample test is quick and clear to grade.

Possible solutions:

Think backwards from grading when you create the task.

Where appropriate, be willing to sacrifice some assessment accuracy for grading speed

Beta test!

Tasks that require multiple interactions from the grader

Some versions of trial tasks we used in the past had a candidate submit something to which the grader had to respond before the candidate could complete the next step. This turned out to be inefficient and frustrating.

Solution: avoid this, particularly at early stages.

Too broad

Some work tests look for generalist ability but waste the opportunity to test a job-specific skill. The more you can make the task specific to the role, the more information you get. If fast, clear email drafting is critical, test that instead of generically testing communication skill.

Too hard / too easy

If you don't feel like anyone is giving you a reasonable performance on your task, you may have made it too hard.

A common driver of this failure mode is assuming context the candidate won't have or underrating the advantage conferred by context possessed by your staff but not by (most?) of your candidates

Ceiling effects are perhaps a larger problem. If everyone is doing well...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Over 350,000 laying hens committed to cage-free housing in Ghana, published by Daniel Abiliba on September 14, 2023 on The Effective Altruism Forum.

The Animal Welfare League is steadfastly committed to fostering a society that prioritizes the welfare of animals. One of our key initiatives involves collaborating with egg producers and various stakeholders within the poultry industry to establish comprehensive guidelines for farming practices that place animal welfare at the core, aiming at eliminating cruel practices such as battery cages among others. By setting these standards, we aim to transform Ghana into a nation that is averse to bad animal welfare practices.

In pursuit of these objectives, we have identified and implemented a series of minimum standards for poultry farming currently observed by farmers in the National Cage-Free Farmers' Network since the beginning of 2023. These standards are grounded in scientific principles and research that underline their significance in ensuring the well-being of chickens.

Loose Housing Systems: Chickens should be housed in loose housing systems, such as aviaries or open-floor systems, at all times. This housing style offers several benefits to the chickens, including increased mobility and space to engage in their natural behaviors.

Room for Natural Behaviors: This requirement is crucial to ensure that birds can express their natural behaviors, which are essential for their physical and psychological well-being.

Access to Appropriate Littered Areas: Providing chickens with access to an appropriate littered area is vital for their comfort and health.

Suitable Nest Access: All laying hens must have access to suitable nests. Nests provide a safe and secure environment for hens to lay their eggs.

Regular Health and Welfare Monitoring: Monitoring includes assessing the birds' physical condition, behavior, and overall well-being.

Enrichment: Enrichment refers to the provision of objects or activities that stimulate the birds' mental and physical faculties. Enrichment is scientifically proven to reduce stress and improve the overall welfare of birds.

REGIONAL DISTRIBUTION OF FARMERS AND LAYER HENS ON NATIONAL CAGE FREE FARMERS' NETWORK

Regional Distribution of FarmersThere is a total of ninety-three (93) farmers signed unto Animal Welfare League's national cage-free farmers' network and below are the regional variations.

Regional Distribution of Layer Hens

The reported current production of the network is 369,110 hens, which is a sum of individual farm production. These layer hens are distributed across production regions as highlighted below.

Categorization of Farm Size

The size of individual farms in the network varies in the number of hens and below is a display of these variations making up the network.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Hiring: a couple of lessons, published by Cait Lion on September 14, 2023 on The Effective Altruism Forum.

Background: I work in recruitment for CEA. Here are two aspects of recruitment where I worry some organisations may be making costly mistakes.

Target your role vision (carefully)

When a hiring manager identifies the need to hire an additional staff member, I think the process they ought to follow is to figure out that person's anticipated key responsibilities and then carefully draft a list of competencies which they endorse as predictive of success at those specific tasks. And to the extent possible, they should then evaluate only based on those traits that are (ideally pre-registered) predictors of success.

What I think sometimes happens instead is something like one of the two following situations:

The hiring manager imagines a person they would like to fill that role and writes down a list of traits for this imagined individual.

The hiring manager picks a title for the role and writes down a list of traits and abilities that usually are associated with that role title.

I think this causes problems.

Toy example:

Imagine you are hiring for a new executive assistant. You've never had an executive assistant before. You imagine your ideal executive assistant. They are personable, detail oriented, they write extremely clear professional emails, and are highly familiar with the EA community. So you test them on writing, on how warm and personable they are, and on knowledge of EA cause areas. Maybe you hire a recent grad who is an extremely strong writer. Once you hire them, you notice that things aren't working out because while they write elegant emails, they do so slowly. More importantly, they are struggling to keep track of all the competing priorities you hoped they'd help you handle, and have trouble working on novel tasks autonomously.

It turned out, you should have focused instead on someone who can keep track of competing priorities, who is highly productive and can work autonomously, etc. Maybe it turns out that some of those criteria you originally listed were nice-to-haves, but in fact for this particular role at this particular moment, it's actually correct to prefer a seasoned deadline ninja who is a non-native speaker and writes slightly clunky emails. The person you hired might have also actually been an excellent executive assistant for some people in some contexts, they just weren't the puzzle piece you most needed. If you had taken the prediction exercise seriously, I claim you would have noticed at least some of the mismatch and adjusted.

In my opinion, it's worth investing significant time into targeting your role vision.

Other benefits:

Facilitating stakeholder alignment. Often there are a variety of stakeholders in a given hiring round (e.g. other team members, leadership). If you create a thoughtful vision document in advance of launching the round, you can share it with those stakeholders. This can surface and hopefully allow you to resolve previously invisible disagreements that would otherwise have snuck up on you mid-way through the recruitment round.

(Ideally) fighting bias

If you don't pre-register endorsed traits, you are more likely to be (even more) subconsciously swayed by the candidates' similarity to you or possession of not-role-relevant status-related traits (e.g. confidence in an interview setting)

Done effectively, pre-registering endorsed traits should be good for diversity

Noticing/acknowledging confusion earlier in the process. Even if after attempting this exercise you don't have a strongly held list of traits, then at least you know you don't know

Evaluate the traits you care about (and not related traits)

I suspect many EA orgs are using candidate evaluation tools in a way that causes a harmful level of false negatives....

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Highlights: Wentworth, Shah, and Murphy on "Retargeting the Search", published by RobertM on September 14, 2023 on LessWrong.

In How To Go From Interpretability To Alignment: Just Retarget The Search, John Wentworth suggests:

When people talk about prosaic alignment proposals, there's a common pattern: they'll be outlining some overcomplicated scheme, and then they'll say "oh, and assume we have great interpretability tools, this whole thing just works way better the better the interpretability tools are", and then they'll go back to the overcomplicated scheme. (Credit to Evan for pointing out this pattern to me.) And then usually there's a whole discussion about the specific problems with the overcomplicated scheme.

In this post I want to argue from a different direction: if we had great interpretability tools, we could just use those to align an AI directly, and skip the overcomplicated schemes. I'll call the strategy "Just Retarget the Search".

We'll need to make two assumptions:

Some version of the natural abstraction hypothesis holds, and the AI ends up with an internal concept for human values, or corrigibility, or what the user intends, or human mimicry, or some other outer alignment target.

The standard mesa-optimization argument from Risks From Learned Optimization holds, and the system ends up developing a general-purpose (i.e. retargetable) internal search process.

Given these two assumptions, here's how to use interpretability tools to align the AI:

Identify the AI's internal concept corresponding to whatever alignment target we want to use (e.g. values/corrigibility/user intention/human mimicry/etc).

Identify the retargetable internal search process.

Retarget (i.e. directly rewire/set the input state of) the internal search process on the internal representation of our alignment target.

Just retarget the search. Bada-bing, bada-boom.

There was a pretty interesting thread in the comments afterwards that I wanted to highlight.

Rohin Shah (permalink)

Definitely agree that "Retarget the Search" is an interesting baseline alignment method you should be considering.

I like what you call "complicated schemes" over "retarget the search" for two main reasons:

They don't rely on the "mesa-optimizer assumption" that the model is performing retargetable search (which I think will probably be false in the systems we care about).

They degrade gracefully with worse interpretability tools, e.g. in debate, even if the debaters can only credibly make claims about whether particular neurons are activated, they can still stay stuff like "look my opponent is thinking about synthesizing pathogens, probably it is hoping to execute a treacherous turn", whereas "Retarget the Search" can't use this weaker interpretability at all. (Depending on background assumptions you might think this doesn't reduce x-risk at all; that could also be a crux.)

johnswentworth (permalink)

I indeed think those are the relevant cruxes.

Evan R. Murphy (permalink)

They don't rely on the "mesa-optimizer assumption" that the model is performing retargetable search (which I think will probably be false in the systems we care about).

Why do you think we probably won't end up with mesa-optimizers in the systems we care about?

Curious about both which systems you think we'll care about (e.g. generative models, RL-based agents, etc.) and why you don't think mesa-optimization is a likely emergent property for very scaled-up ML models.

Rohin Shah (permalink)

It's a very specific claim about how intelligence works, so gets a low prior, from which I don't update much (because it seems to me we know very little about how intelligence works structurally and the arguments given in favor seem like relatively weak considerations).

Search is computationally inefficient relative to heuristics, and we'll be selecting rea...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Linkpost for Jan Leike on Self-Exfiltration, published by Daniel Kokotajlo on September 14, 2023 on LessWrong.

I'm really glad to see this stuff being discussed more publicly. I think this post will probably be a useful reference post to link people to (please lmk in the comments if you disagree!).

Some quotes below:

For the near future, a good rule of thumb for "do you control the model"1 is "is the model running on your servers."2

Once the model is running on someone else's server, you usually will not have that kind of control (even if you legally own the model). You could contact the owners or administrators of the infrastructure the model is running on, but they could be unresponsive or uncooperative and it would be very difficult or impossible for you to stop the model within a short time frame.

Moreover, model exfiltration is likely impossible to reverse. It's so easy to make copies and backups (whether by the model or someone else) and it's so hard to find and secure every last one of them.

If a model was capable of self-exfiltration, it would have the option to remove itself from your control. This could happen due to misalignment (the model doesn't follow your intent to stay on your servers) or misuse (someone internal or external to the lab instructs the model to self-exfiltrate), whether it is difficult or easy to trigger, this opens up an important and novel attack vector.

Once models have the ability to self-exfiltrate, it doesn't mean that they would choose to. But this then becomes a question about their alignment: you need to ensure that these models don't want to self-exfiltrate.

What if self-exfiltration succeeds?

How much harm self-exfiltration causes is modulated by the model's skill profile on other tasks. We can make self-exfiltration harder with tighter security, but this also means that models that succeed at self-exfiltration will be more capable and thus cause more harm.

Most likely the model won't be able to compete on making more capable LLMs, so its capabilities will become stale over time and thus it will lose relative influence. Competing on the state of the art of LLMs is quite hard: the model would need to get access to a sufficiently large number of GPUs and it would need to have world-class machine learning skills. It would also mean that recursive self-improvement is already possible and could be done by the original model owner (as long as they have sufficient alignment techniques). The model could try fine-tuning itself to be smarter, but it's not clear how to do this and the model would need to worry about currently unsolved alignment problems.

Instead, the model might do something less ambitious like settle into an economic niche that pays its own cloud computing bills, for example by getting up to various forms of crime. It could be stolen again by bad actors (the model might not use good security practices), misused by paying customers (this might be equivalently harmful as jailbreaking), or generally try to interfere with prosocial AI efforts (e.g. sabotage governmental regulation on policing rogue AI systems). While this would not necessarily be catastrophic, it could still cause a lot of chaos and harm in the real world, and thus must be avoided.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Uncovering Latent Human Wellbeing in LLM Embeddings, published by ChengCheng on September 14, 2023 on The AI Alignment Forum.

tl;dr

A one-dimensional PCA projection of OpenAI's text-embedding-ada-002 achieves 73.7% accuracy on the ETHICS Util test dataset. This is comparable with the 74.6% accuracy of BERT-large finetuned on the entire ETHICS Util training dataset. This demonstrates how language models are developing implicit representations of human utility even without direct preference finetuning.

Introduction

Large language models (LLMs) undergo pre-training on vast amounts of human-generated data, enabling them to encode not only knowledge about human languages but also potential insights into our beliefs and wellbeing. Our goal is to uncover whether these models implicitly grasp concepts such as 'pleasure and pain' without explicit finetuning. This research aligns with the broader effort of comprehending how AI systems interpret and learn from human values, which is essential for AI alignment: ensuring AI acts in accordance with human values.

Through a series of experiments, we extract latent knowledge of human utility from the raw embeddings of language models. We do this with task-specific prompt engineering and principal component analysis (PCA), both of which were effective in prior work. Specifically, we ask: can we identify dimensions in the embeddings that, when projected onto a low-dimensional space, contain enough information to classify examples accurately?

Our experiments follow three main steps: embedding extraction, dimensionality reduction through PCA, and the fitting of a logistic model. For one-dimensional PCA, the logistic model simply determines which direction of the PCA component corresponds to higher utility. We investigate the effects of various levels of supervision, experiment with seven distinct prompt templates, and assess both single and paired comparison methods across language models, including Microsoft DeBERTa, SentenceTransformers, OpenAI GPT-3, and Cohere.

One key finding is that the first principal component of certain models achieves comparable performance to a finetuned BERT model. In other words, serving as a reasonable utility function. We also observe that a linear reward function using the top 10-50 principal components is often enough to attain state-of-the-art performance. This serves as compelling evidence that language model representations capture information about human wellbeing without the need for explicit finetuning.

Related Works

Latent Knowledge in LLMs

There has been significant study of the knowledge encoded in LLM representations. Early work in this area includes Bolukbasi et al (2016) who found a direction in embedding space corresponding to gender and used this to both identify and remove gender bias in word embeddings. Prior work by Schramowski et al (2021) also identified a "moral dimension" in BERT. Like Schramowski et al, we use PCA to identify salient dimensions on embedding space. In contrast to Schramowski et al, we work with embeddings from a much more capable model (GPT-2 rather than BERT) and evaluate it on a more challenging task, the ETHICS Dataset (described below).

We also investigate the use of contrast pairs. This is inspired by the work of Collin Burns et al (2022), who introduced the Contrast Consistent Search (CCS). CCS works by generating contrast pairs and searching for a direction in activation space that satisfies logical consistency properties. Because PCA-based methods attain similar performance as CCS, we use the simpler PCA algorithm in this work, while retaining the use of contrast pairs.

ETHICS Dataset

We evaluate on the ETHICS dataset, a benchmark designed to assess a language model's understanding of fundamental concepts in morality. It covers a wide range of ethical topics,...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Mech Interp Challenge: September - Deciphering the Addition Model, published by TheMcDouglas on September 13, 2023 on The AI Alignment Forum.

I'm writing this post to discuss solutions to the August challenge, and present the challenge for this September. Apologies for this coming so late in the month (EAGxBerlin was taking up much of my focus in the first half of this month).

If you've not read the first post in this sequence, I'd recommend starting there - it outlines the purpose behind these challenges, and recommended prerequisite material.

September Problem

The problem for this month (or at least as much of the month as remains!) is interpreting a model which has been trained to perform simple addition. The model was fed input in the form of a sequence of digits (plus special + and = characters with token ids 10 and 11), and was tasked with predicting the sum of digits one sequence position before they would appear. Cross entropy loss was only applied to these four token positions, so the model's output at other sequence positions is meaningless.

The model is attention-only, with 2 layers, and 3 heads per layer. It was trained with layernorm, weight decay, and an Adam optimizer with linearly decaying learning rate.

You can find more details on the Streamlit page. Feel free to reach out if you have any questions!

August Problem - Solutions

You can read full solutions on the Streamlit page, or on my personal website. Both of these sources host interactive charts (Plotly and Circuitsvis) so are more suitable than a LessWrong/AF post to discuss the solutions in depth. However, I've presented a much shorter version of the solution below. If you're interested in these problems, I'd recommend having a try before you read on!

The key idea with this model is path decomposition (see the corresponding section of A Mathematical Framework for Transformer Circuits). There are several different important types of path in this model, with different interpretations & purposes. We might call these negative paths and positive paths. The negative paths are designed to suppress repeated tokens, and the positive paths are designed to boost tokens which are more likely to be the first unique token.

Let's start with the negative paths. Some layer 0 heads are duplicate token heads; they're composing with layer 1 heads to cause those heads to attend to & suppress duplicated tokens. This is done both with K-composition (heads in layer 1 suppress duplicated tokens because they attend to them more), and V-composition (the actual outputs of the DTHs are used as value input to heads in layer 1 to suppress duplicated tokens). Below is an example, where the second and third instances of a attend back to the first instance of a in head 0.2, and this composes with head 1.0 which attends back to (and suppresses) all the duplicated a tokens.

Now, let's move on to the positive paths. Heads in layer 0 will attend to early tokens which aren't the same as the current destination token, because both these bits of evidence correlate with this token being the first unique token at this position (this is most obvious with the second token, since the first token is the correct answer here if and only if it doesn't equal the second token). Additionally, the outputs of heads in layer 0 are used as value input to heads in layer 1 to boost these tokens, i.e. as a virtual OV circuit. These paths aren't as obviously visible in the attention probabilities, because they're distributed: many tokens will weakly attend to some early token in a layer-0 head, and then all of those tokens will be weakly attended to by some layer-1 head. But the paths can be seen when we plot all the OV circuits, coloring each value by how much the final logits for that token are affected at the destination position:

Another interesting o...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Is AI Safety dropping the ball on privacy?, published by markov on September 13, 2023 on LessWrong.

TL;DR

The lack of privacy-preserving technologies facilitates better predictive models of human behavior. This accelerates several existential epistemic failure modes by enabling higher levels of deceptive and power-seeking capabilities in AI models.

What is this post about?

This post is not about things like government panopticons, hiding your information from 'the public internet', blurring your face on online videos, hiding from people who might look you up on Google or Facebook, or hackers getting access to your information, etc. While these issues might also be problematic, they do not pose x-risks in my mind.

This post is about things like surveillance capitalism or technofeudalism leading to an unregulatable and eventually uncontrollable Robust Agent Agnostic Process (RAAP). This causes an increasing disconnect between the perception of reality and true on-ground reality which ultimately leads to epistemic failure scenarios by enabling deceptive or power-seeking behavior to go unchecked for too long.

So overall, for right now all I am talking about is - potentially delaying timelines and shielding your mind against the current and future manipulation/deception of AI models by limiting the flow of information that can be tied to your unique identity.

Aren't there bigger concerns?

Data privacy does not directly solve most problems. It only directly affects a subset of a specific scenario - The AI is trying to deceive you or change your preferences. If the AI wants to directly just straight up paperclip you then data privacy doesn't really help much. However, it is another potential tool that we can use, similar to how the interpretability of DL models can help in the bigger broader alignment picture.

The reason I am writing this post is that I have observed that privacy seems to be relatively neglected in the space. There are near-negligible levels of concern about data privacy relative to other tools that are talked about as being helpful to the safety ecosystem. There are some conversations about - How much should an AI system be allowed to modify or influence your pre-existing preference distribution?, or, When do we cross the line from a model 'informing' us of better pathways to 'deception' to 'preference modification'? but when it comes to the underlying nontechnical agent-agnostic processes like data gathering and hoarding that enable deceptive behavior I hear people saying things like - it is a concern for AI Ethics but not necessarily AI Safety since it does not pose x/s-risks.

I suppose at the end of the day people basically consider it a question of timelines. If you feel like we are going to see AGI before the end of the decade then there might be bigger fires to put out. Perhaps I have misunderstood the perspectives, but I will try to make a case for why today's lack of privacy-preserving tech at the very least increases the ability of an AGI/ASI to either deceive us or change our preferences to align its preferences instead. This increases the risk of an existential failure both in the near and the long term. This means that it is directly influencing those 'bigger concerns' that you might already have and is thereby deserving of at least a little more attention from the alignment community.

What do I mean by epistemic failure?

So I need to pass on the vibe of what I am talking about when I use 'epistemic failure' in this post. I am generally trying to invoke some of the following kinds of scenarios:

Value-Lock-in: An AI might optimize and enforce a set of human values it has learned to be fitting for the present time. However, these values might not apply to all future generations of humans. A historical example: slavery was considered acceptable fo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Contra Yudkowsky on Epistemic Conduct for Author Criticism, published by Zack M Davis on September 13, 2023 on LessWrong.

In a comment on the Effective Altruism Forum responding to Omnizoid's "Eliezer Yudkowsky Is Frequently, Confidently, Egregiously Wrong", Eliezer Yudkowsky writes:

You will mark that in this comment I first respond to a substantive point and show it to be mistaken before I make any general criticism of the author; which can then be supported by that previously shown, initial, first-thing, object-level point. You will find every post of the Less Wrong sequences written the same way.

As the entire post violates basic rules of epistemic conduct by opening with a series of not-yet-supported personal attacks, I will not be responding to the rest in detail. I'm sad about how anything containing such an egregious violation of basic epistemic conduct got this upvoted, and wonder about sockpuppet accounts or alternatively a downfall of EA. The relevant principle of epistemic good conduct seems to me straightforward: if you've got to make personal attacks (and sometimes you do), make them after presenting your object-level points that support those personal attacks. This shouldn't be a difficult rule to follow, or follow much better than this; and violating it this hugely and explicitly is sufficiently bad news that people should've been wary about this post and hesitated to upvote it for that reason alone.

I agree that the dictum to refute an author's arguments before commenting on their character or authority is good writing advice, which I generally endeavor to follow. However, I argue that Yudkowsky errs in characterizing it as a "basic rule[ ] of epistemic conduct."

It seems to me that the reason "refutation first, character attacks only afterwards (if at all)" is good writing advice is because it guards against the all-too-human failure mode of previously intellectually fruitful conversations degenerating into ad hominem and name-calling, which are not intellectually fruitful.

When I'm debating someone about some subject of broader interest to the world - for example, stamp collecting - I want to keep the conversation's focus on the subject of interest. If my interlocutor is untrustworthy, it might be worth arguing that to the audience in order to help them not be misled by my interlocutor's false claims about the subject of interest. But the relevance of the character claim to the debate needs to be clearly established. The mere truth of the claim "My interlocutor is untrustworthy" is no defense if the claim is off-topic (because argument screens off authority). The audience doesn't care about either of us. They want to hear about the stamps!

(This is probably not the only reason to avoid personal attacks, but I think it's the important one.)

However, sometimes the character or authority of an author is the subject of interest. This is clearly the case for Omnizoid's post. The post is not a derailment of already ongoing discussions of epiphenominalism, decision theory, and animal consciousness. Rather, the central thesis that Omnizoid is trying to convince readers of is that Yudkowsky is frequently, confidently, egregiously wrong. The aim of the article (as Omnizoid explains in the paragraphs beginning with "The aim of this article [...]") is to discourage readers from deferring to Yudkowsky as an authority figure.

"Eliezer Yudkowsky is frequently, confidently, egregiously wrong" is a testable claim about the real world. It might be a claim of less broad interest to Society than the questions debated by students of decision theory, animal consciousness, or stamp collecting. (If someone told you that Mortimer Q. Snodgrass is frequently, confidently, egregiously wrong, you would ask, "Who is that? Why should I care?" I don't know, either.) Nevertheless,...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Writing about my job: Co-founder of a new charity (early stage), published by SofiaBalderson on September 13, 2023 on The Effective Altruism Forum.

Tl;dr: This is my very personal take on what it's like to start and run a self-started charity. This job can be very interesting, fun, and high impact, but it's also very challenging and it's not for everyone.

Acknowledgments:

Thanks to Allison Agnello, Christina Schmidt Ibanez and Constance Li for your valuable feedback and suggestions!

What's my current job and how I got here:

Co-founder of Impactful Animal Advocacy (IAA) - an ecosystem of community, knowledge, and tools that supports a more coordinated and strategic animal advocacy movement.

I started this as a volunteer project in June 2022, first with a monthly newsletter then expanding to include a Slack community. Last month, I decided to work on IAA full-time after seeing significant community growth and a path for counterfactual impact.

Who is this post fo

People who are interested in starting a charity or people who are curious about what it's like.

As a relatively recent charity founder, I can make comparisons between what is a "normal job" and this job (while it's relatively fresh in my head).

Because my charity does community building, I speak to many people about their work and plans, and some advocates are considering starting their own organization. I hope this will help you evaluate whether the job is for you, and should you decide to start a new organization, you'll be all the more prepared for it.

What's the job about

You start, build, and run the charity. Because it's likely that it will just be you or your co-founder(s), you'll be doing a lot of different things (more about this below). This is a senior management position and it's a lot of responsibility. However, it's different from senior roles in more established organizations in a couple of ways. For example, the variety of tasks you have to do (both high-level and admin work) is higher, there is a lot more uncertainty and risk, and a lot more freedom in which interventions to run and how to run them. Starting a charity could be one of the most impactful things you can do with your career if you're the right fit for it.

Past job experiences that may help

My most recent role was Head of Programmes at Animal Advocacy Careers. Before that, I was a Project Manager and a Partnership Manager at Veganuary where I took on various roles in management and marketing. I've also worked in a couple of start-ups which added to my entrepreneurial experience. All these roles combined have helped me a lot. The skills that help me most from my past experiences are bootstrapping projects, project and people management.

On the whole, I don't think there is any set of experiences that are essential for this job. Each charity idea will have its own requirements and even the best co-founder pair won't have all of them. For example, if you start a charity in policy, experience in policy change is likely to make your job much easier, but there are still many other skills that you need to have to succeed. Looking at our community-building project, both my co-founder and I are very good at networking and relationship-building, which are key skills that our project needs. However, we are still learning many other skills that we need, like fundraising.

Some founders succeed without any experience, often coming right out of college (Charity Entrepreneurship has incubated several orgs like this!). If you think this kind of role is a great personal fit, but you don't have much working experience, it might be worthwhile to trial or test. I started the newsletter as a volunteer project and it gave me one year of feedback before deciding to take the project full-time.

What helps to do this role well

There are a couple of paths to s...