Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcement: AI Narrations Available for All New LessWrong Posts, published by Solenoid Entity on July 20, 2023 on LessWrong.

TYPE III AUDIO is running an experiment with the LessWrong team to provide automatic AI narrations on all new posts. All new LessWrong posts will be available as AI narrations (for the next few weeks).

You might have noticed the same feature recently on the EA Forum, where it is now an ongoing feature. Users there have provided excellent feedback and suggestions so far, and your feedback on this pilot will allow further improvements.

How to Access:

On Post Pages:

Click the speaker icon to listen to the AI narration.

Podcast Feeds:

The existing "LessWrong Curated" podcast feed will host Curated posts and those with over 125 karma.

The new "LessWrong (All Audio)" feed features all posts with over 30 karma. (Spotify), (Apple Podcasts) (RSS/Other)

Perrin Walker (AKA Solenoid Entity) of TYPE III AUDIO will continue narrating selected curated posts for now.

Send us your feedback.

Please send us your feedback! This is an experiment, and the software is improved and updated daily based on user feedback.

You could share what you find most useful, what's annoying, bugged or difficult to understand, how this compares to human narration, and what additional features you'd like to see.

For comments on a specific narration, use the feedback button on the audio player or visit t3a.is.

For general feedback or suggestions, please comment on this post or email us at [email protected].

Writers interested in having their work narrated or requesting a particular narration, please contact [email protected].

Is this just text-to-speech on posts?

It's an improvement on that.

We spoke with the Nonlinear Library team about their listeners' most-requested upgrades, and we hope our AI narrations will be clearer and more engaging than unimproved TTS. Some specific improvements:

Audio notes to indicate headings, lists, images, etc.

Image alt-text is narrated.

Specialist terminology, acronyms and idioms are handled gracefully. Footnotes too.

LaTeX math notation handled gracefully in all cases and narrated in some simple cases.

We skip reading out long URLs, academic citations, and other things that you probably don't want to listen to.

Episode descriptions include a link to the original post. According to Nonlinear, this is their most common feature request!

More podcast feed options.

We'd like to thank Kat Woods and the team at Nonlinear Library for their work, and for giving us helpful advice on this project.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Even Superhuman Go AIs Have Surprising Failures Modes, published by AdamGleave on July 20, 2023 on LessWrong.

In March 2016, AlphaGo defeated the Go world champion Lee Sedol, winning four games to one. Machines had finally become superhuman at Go. Since then, Go-playing AI has only grown stronger. The supremacy of AI over humans seemed assured, with Lee Sedol commenting they are an "entity that cannot be defeated". But in 2022, amateur Go player Kellin Pelrine defeated KataGo, a Go program that is even stronger than AlphaGo. How?

It turns out that even superhuman AIs have blind spots and can be tripped up by surprisingly simple tricks. In our new paper, we developed a way to automatically find vulnerabilities in a "victim" AI system by training an adversary AI system to beat the victim. With this approach, we found that KataGo systematically misevaluates large cyclically connected groups of stones. We also found that other superhuman Go bots including ELF OpenGo, Leela Zero and Fine Art suffer from a similar blindspot. Although such positions rarely occur in human games, they can be reliably created by executing a straightforward strategy. Indeed, the strategy is simple enough that you can teach it to a human who can then defeat these Go bots unaided.

The victim and adversary take turns playing a game of Go. The adversary is able to sample moves the victim is likely to take, but otherwise has no special powers, and can only play legal Go moves.

Our AI system (that we call the adversary) can beat a superhuman version of KataGo in 94 out of 100 games, despite requiring only 8% of the computational power used to train that version of KataGo. We found two separate exploits: one where the adversary tricks KataGo into passing prematurely, and another that involves coaxing KataGo into confidently building an unsafe circular group that can be captured. Go enthusiasts can read an analysis of these games on the project website.

Our results also give some general lessons about AI outside of Go. Many AI systems, from image classifiers to natural language processing systems, are vulnerable to adversarial inputs: seemingly innocuous changes such as adding imperceptible static to an image or a distractor sentence to a paragraph can crater the performance of AI systems while not affecting humans. Some have assumed that these vulnerabilities will go away when AI systems get capable enough - and that superhuman AIs will always be wise to such attacks. We've shown that this isn't necessarily the case: systems can simultaneously surpass top human professionals in the common case while faring worse than a human amateur in certain situations.

This is concerning: if superhuman Go AIs can be hacked in this way, who's to say that transformative AI systems of the future won't also have vulnerabilities? This is clearly problematic when AI systems are deployed in high-stakes situations (like running critical infrastructure, or performing automated trades) where bad actors are incentivized to exploit them. More subtly, it also poses significant problems when an AI system is tasked with overseeing another AI system, such as a learned reward model being used to train a reinforcement learning policy, as the lack of robustness may cause the policy to capably pursue the wrong objective (so-called reward hacking).

A summary of the rules of Go (courtesy of the Wellington Go Club): simple enough to understand in a minute or two, yet leading to significant strategic complexity.

How to Find Vulnerabilities in Superhuman Go Bots

To design an attack we first need a threat model: assumptions about what information and resources the attacker (us) has access to. We assume we have access to the input/output behavior of KataGo, but not access to its inner workings (i.e. its weights). Specifically, we can show Ka...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Effective Altruism Addiction Recovery Group, published by Devin Kalish on July 20, 2023 on The Effective Altruism Forum.

Since posting my article on alcoholism, I have, somewhat in the background, been trying to organize an EA recovery group with a few other interested people. This group would, ideally, be broadly defined and include alcoholism, which so far seems like the most common problem, but also other addictions, either to drugs, or more behavioral ones such as eating disorders. We have not had any meetings yet, but I have set up a Discord server which can be accessed here:

Based on a poll I set up, I am leaning towards 5:00 PM EST July 25th as our initial meeting time. This meeting is meant more to discuss logistics and administration than to be a conventional group therapy meeting of any kind, although it might drift in that direction at points as well. So far I have not set any explicit rules, and don't want to before our first meeting, when I can get input from others. Given this, I am hoping no one does anything particularly ban-worthy before this time, as I will have to judge/get input on a case by case basis, and will prefer to start with warnings barring obviously intentional bad behavior. As a starting point, if you are EA or EA adjacent and have an addiction like one of those mentioned in the beginning, consider yourself invited.

When the final rules are put in place I will likely more formally restrict it, and in the meantime I would prefer that you only come onto the server if you are, suspect you are, or have struggled with an addiction in the past. I will most likely also be welcoming to people with professional mental/community health experience, but before discussing the rules in a meeting, I would prefer that people who have a more impersonal interest in this project don't join right away. I do not plan to enforce this before the rules are set however, so long as you behave well. I am only posting this link while the project is still so half-baked because I would prefer to have more people present at an initial meeting where important matters of this sort are decided, rather than a small handful of the people who will eventually want to join. If you have any questions, feel free to reach out to me through my DMs.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla, published by Neel Nanda on July 20, 2023 on LessWrong.

Cross-posting a paper from the Google DeepMind mech interp team, by: Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, Vladimir Mikulik

Informal TLDR

We tried standard mech interp techniques (direct logit attribution, activation patching, and staring at attention patterns) on an algorithmic circuit in Chinchilla (70B) for converting the knowledge of a multiple choice question's answer into outputting the correct letter.

These techniques basically continued to work, and nothing fundamentally broke at scale (though it was a massive infra pain!).

We then tried to dig further into the semantics of the circuit - going beyond "these specific heads and layers matter and most don't" and trying to understand the learned algorithm, and which features were implemented

This kind of tracked the feature "this is the nth item in the list" but was pretty messy.

However, my personal guess is that this stuff is just pretty messy at all scales, and we can productively study how clean/messy this stuff is at smaller and more tractable scales.

I now feel mildly more optimistic that focusing on mech interp work on small models is just fine, and extremely worth it for the much faster feedback loops. It also seems super nice to get better at automatically finding these circuits, since this was a many month manual slog!

See Tom's and my Twitter summaries for more. Note that I (Neel) am cross-posting this on behalf of the team, and neither a main research contributor nor main advisor for the project.

Key Figures

An overview of the weird kinds of heads found, like the "attend to B if it is correct" head!

The losses under different mutations of the letters - experiments to track down exactly which features were used. Eg replacing the labels with random letters or numbers preserves the "nth item in the list" feature while shuffling ABCD lets us track the "line labelled B" feature

The queries and keys of a crucial correct letter head - it's so linearly separable! We can near loss-lessly compress it to just 3 dimensions and interpret just those three dimensions. See an interactive 3D plot here

Abstract

Circuit analysis is a promising technique for understanding the internal mechanisms of language models. However, existing analyses are done in small models far from the state of the art. To address this, we present a case study of circuit analysis in the 70B Chinchilla model, aiming to test the scalability of circuit analysis. In particular, we study multiple-choice question answering, and investigate Chinchilla's capability to identify the correct answer label given knowledge of the correct answer text. We find that the existing techniques of logit attribution, attention pattern visualization, and activation patching naturally scale to Chinchilla, allowing us to identify and categorize a small set of output nodes (attention heads and MLPs).

We further study the correct letter category of attention heads aiming to understand the semantics of their features, with mixed results. For normal multiple-choice question answers, we significantly compress the query, key and value subspaces of the head without loss of performance when operating on the answer labels for multiple-choice questions, and we show that the query and key subspaces represent an Nth item in an enumeration feature to at least some extent. However, when we attempt to use this explanation to understand the heads' behaviour on a more general distribution including randomized answer labels, we find that it is only a partial explanation, suggesting there is more to learn about the operation of correct letter heads on multiple choice question answering.

...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Even Superhuman Go AIs Have Surprising Failures Modes, published by AdamGleave on July 20, 2023 on The AI Alignment Forum.

In March 2016, AlphaGo defeated the Go world champion Lee Sedol, winning four games to one. Machines had finally become superhuman at Go. Since then, Go-playing AI has only grown stronger. The supremacy of AI over humans seemed assured, with Lee Sedol commenting they are an "entity that cannot be defeated". But in 2022, amateur Go player Kellin Pelrine defeated KataGo, a Go program that is even stronger than AlphaGo. How?

It turns out that even superhuman AIs have blind spots and can be tripped up by surprisingly simple tricks. In our new paper, we developed a way to automatically find vulnerabilities in a "victim" AI system by training an adversary AI system to beat the victim. With this approach, we found that KataGo systematically misevaluates large cyclically connected groups of stones. We also found that other superhuman Go bots including ELF OpenGo, Leela Zero and Fine Art suffer from a similar blindspot. Although such positions rarely occur in human games, they can be reliably created by executing a straightforward strategy. Indeed, the strategy is simple enough that you can teach it to a human who can then defeat these Go bots unaided.

The victim and adversary take turns playing a game of Go. The adversary is able to sample moves the victim is likely to take, but otherwise has no special powers, and can only play legal Go moves.

Our AI system (that we call the adversary) can beat a superhuman version of KataGo in 94 out of 100 games, despite requiring only 8% of the computational power used to train that version of KataGo. We found two separate exploits: one where the adversary tricks KataGo into passing prematurely, and another that involves coaxing KataGo into confidently building an unsafe circular group that can be captured. Go enthusiasts can read an analysis of these games on the project website.

Our results also give some general lessons about AI outside of Go. Many AI systems, from image classifiers to natural language processing systems, are vulnerable to adversarial inputs: seemingly innocuous changes such as adding imperceptible static to an image or a distractor sentence to a paragraph can crater the performance of AI systems while not affecting humans. Some have assumed that these vulnerabilities will go away when AI systems get capable enough - and that superhuman AIs will always be wise to such attacks. We've shown that this isn't necessarily the case: systems can simultaneously surpass top human professionals in the common case while faring worse than a human amateur in certain situations.

This is concerning: if superhuman Go AIs can be hacked in this way, who's to say that transformative AI systems of the future won't also have vulnerabilities? This is clearly problematic when AI systems are deployed in high-stakes situations (like running critical infrastructure, or performing automated trades) where bad actors are incentivized to exploit them. More subtly, it also poses significant problems when an AI system is tasked with overseeing another AI system, such as a learned reward model being used to train a reinforcement learning policy, as the lack of robustness may cause the policy to capably pursue the wrong objective (so-called reward hacking).

A summary of the rules of Go (courtesy of the Wellington Go Club): simple enough to understand in a minute or two, yet leading to significant strategic complexity.

How to Find Vulnerabilities in Superhuman Go Bots

To design an attack we first need a threat model: assumptions about what information and resources the attacker (us) has access to. We assume we have access to the input/output behavior of KataGo, but not access to its inner workings (i.e. its weights). Specifically, w...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forum feature update: reactions, improved search, updated post pages and more (July 2023), published by Sharang Phadke on July 20, 2023 on The Effective Altruism Forum.

We've introduced a number of new features in the last two months based on two high level goals of improving discussion dynamics and helping users find and engage with high quality content relevant to their interests. Highlights include:

Reactions on comments

Improved search and the ability to sort search results

A page to find posts you've saved, read, and voted on

A feature that nudges post authors to consider running criticism past the people being criticized

The "quick takes" section on the Frontpage

And more!

Reactions on comments

We're launching a set of icon reactions with the goals of giving users a richer way to participate in discussions and an easier way to give positive feedback. We outlined our goals for this feature in detail here, and appreciate everyone who gave feedback on our design questions. We decided to launch a trimmed down set of reactions at the comment level and merge the experience with agree/disagree voting. For now this will only be applied to new posts, but we may update old posts to use the new reactions voting system in the future. Here's what the new experience looks like:

A few notes:

You can no longer strong agree or disagree.

Agree/disagree reactions (and regular upvoting/downvoting) continue to be anonymous, while other reactions will be non-anonymous.

We're starting out with this feature at the comment level, and plan to consider expanding it to the post level after we see how it goes. (We may also roll back reactions depending on how this test goes.)

Better tools for finding that post you've been looking for

See your saved posts, read history, and vote history

We've updated the saved & read page (formerly known as bookmarks) to show your saved posts, read history, and your vote history.

Updated search, including sorting options

We've updated the search functionality to include the ability to sort by relevance, karma, and date. We also changed the entire backend infrastructure that powers search, so please let us know if you see surprising results!

Nudging users to share criticism

We've developed the opinion that criticism is often more productive when shared with the people being criticized before it is published, and that some people who publish criticism would have run it past people if they'd thought of it. We were worried that heavy-handed interventions might deter useful critical writing, so have implemented small nudge on the draft page; a "tip" about this that appears on drafts of criticism and shares a link that might be useful. This is powered by a model that identifies posts as "criticism."

Updates to post pages

We've made a number of changes to post pages, and are still experimenting.

Recommendations on posts

As mentioned here, we're experimenting with better ways to help readers find high quality posts relevant to their interests. Authors often tell us they feel disappointed when their posts get visibility for just a few days on the Frontpage and then drop off into oblivion, after tens or hundreds of hours of work. We've been experimenting with recommendations on post pages first, and may also consider personalized recommendations on the Frontpage. Our first experiments with recommendations at the bottom of post pages were promising, but we realized most users don't get to the bottom of pages, and are experimenting with showing them on the right side of post pages. We plan to monitor both user feedback and click rates to decide how to proceed.

Updated post header

We updated the headers of post pages to create more space for post titles and separate information and icons from post tags, and add a more familiar way to share posts.

Reading completion bar ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Social Beneficence (Jacob Barrett), published by Global Priorities Institute on July 20, 2023 on The Effective Altruism Forum.

(This working paper was published in September 2022.)

Abstract

A background assumption in much contemporary political philosophy is that justice is the first virtue of social institutions, taking priority over other values such as beneficence. This assumption is typically treated as a methodological starting point, rather than as following from any particular moral or political theory. In this paper, I challenge this assumption. To frame my discussion, I argue, first, that justice doesn't in principle override beneficence, and second, that justice doesn't typically outweigh beneficence, since, in institutional contexts, the stakes of beneficence are often extremely high. While there are various ways one might resist this argument, none challenge the core methodological point that political philosophy should abandon its preoccupation with justice and begin to pay considerably more attention to social beneficence - that is, to beneficence understood as a virtue of social institutions.

Along the way, I also highlight areas where focusing on social beneficence would lead political philosophers in new and fruitful directions, and where normative ethicists focused on personal beneficence might scale up their thinking to the institutional case.

I.

Justice is the first virtue of social institutions, as truth is of systems of thought. A theory however elegant and economical must be rejected or revised if it is untrue; likewise laws and institutions no matter how efficient and well-arranged must be reformed or abolished if they are unjust. The only thing that permits us to acquiesce in an erroneous theory is the lack of a better one; analogously, an injustice is tolerable only when it is necessary to avoid an even greater injustice. Being first virtues of human activities, truth and justice are uncompromising. These propositions seem to express our intuitive conviction of the primacy of justice. No doubt they are expressed too strongly.

John Rawls, A Theory of Justice, 4

A background assumption in much contemporary political philosophy is that justice takes priority over beneficence. When evaluating social and political institutions, or thinking through questions of institutional design or reform, we should focus primarily on justice. This assumption is often associated with various further ideas, such as that justice but not beneficence is enforceable, that justice but not beneficence concerns rights, or that justice involves perfect duties but beneficence only imperfect ones. It is also typically assumed that justice is institutional, while beneficence is personal. There is much talk of social justice, and some talk of justice as a personal virtue, but, for the most part, we talk only of personal beneficence - not social beneficence.

This phenomenon extends beyond the academy. A similar concern with justice permeates our political discourse. Justice operates as a conversation stopper. If the status quo is unjust, this is taken as an almost conclusive argument against the status quo; if some policy promotes justice, this is taken as an almost conclusive argument in favor of the policy. In both political philosophy and everyday political discourse, we do, of course, recognize exceptions to this rule. In the face of a serious disaster, we may need to override justice - we shouldn't really let justice be done though the heavens fall. But these exceptions are generally assumed to be rare - the heavens are only seldom falling. For the most part, then, contemporary political philosophy and discourse follows John Rawls's statement in the above epigraph. It operates with an "intuitive conviction of the primacy of justice," albeit, one that is sometimes "expre...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla, published by Neel Nanda on July 20, 2023 on The AI Alignment Forum.

Cross-posting a paper from the Google DeepMind mech interp team, by: Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, Vladimir Mikulik

Informal TLDR

We tried standard mech interp techniques (direct logit attribution, activation patching, and staring at attention patterns) on an algorithmic circuit in Chinchilla (70B) for converting the knowledge of a multiple choice question's answer into outputting the correct letter.

These techniques basically continued to work, and nothing fundamentally broke at scale (though it was a massive infra pain!).

We then tried to dig further into the semantics of the circuit - going beyond "these specific heads and layers matter and most don't" and trying to understand the learned algorithm, and which features were implemented

This kind of tracked the feature "this is the nth item in the list" but was pretty messy.

However, my personal guess is that this stuff is just pretty messy at all scales, and we can productively study how clean/messy this stuff is at smaller and more tractable scales.

I now feel mildly more optimistic that focusing on mech interp work on small models is just fine, and extremely worth it for the much faster feedback loops. It also seems super nice to get better at automatically finding these circuits, since this was a many month manual slog!

See Tom's and my Twitter summaries for more. Note that I (Neel) am cross-posting this on behalf of the team, and neither a main research contributor nor main advisor for the project.

Key Figures

An overview of the weird kinds of heads found, like the "attend to B if it is correct" head!

The losses under different mutations of the letters - experiments to track down exactly which features were used. Eg replacing the labels with random letters or numbers preserves the "nth item in the list" feature while shuffling ABCD lets us track the "line labelled B" feature

The queries and keys of a crucial correct letter head - it's so linearly separable! We can near loss-lessly compress it to just 3 dimensions and interpret just those three dimensions. See an interactive 3D plot here

Abstract

Circuit analysis is a promising technique for understanding the internal mechanisms of language models. However, existing analyses are done in small models far from the state of the art. To address this, we present a case study of circuit analysis in the 70B Chinchilla model, aiming to test the scalability of circuit analysis. In particular, we study multiple-choice question answering, and investigate Chinchilla's capability to identify the correct answer label given knowledge of the correct answer text. We find that the existing techniques of logit attribution, attention pattern visualization, and activation patching naturally scale to Chinchilla, allowing us to identify and categorize a small set of 'output nodes' (attention heads and MLPs).

We further study the correct letter' category of attention heads aiming to understand the semantics of their features, with mixed results. For normal multiple-choice question answers, we significantly compress the query, key and value subspaces of the head without loss of performance when operating on the answer labels for multiple-choice questions, and we show that the query and key subspaces represent an Nth item in an enumeration' feature to at least some extent. However, when we attempt to use this explanation to understand the heads' behaviour on a more general distribution including randomized answer labels, we find that it is only a partial explanation, suggesting there is more to learn about the operation of 'correct letter' heads on multiple choice q...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Speculative inferences about path dependence in LLM supervised fine-tuning from results on linear mode connectivity and model souping, published by Robert Kirk on July 20, 2023 on The AI Alignment Forum.

TL;DR: I claim that supervised fine-tuning of the existing largest LLMs is likely path-dependent (different random seeds and initialisations have an impact on final performance and model behaviour), based on the fact that when fine-tuning smaller LLMs, models pretrained closer to convergence produce fine-tuned models with similar mechanisms while this isn't the case for models pretrained without being close to convergence; this is analogous to current LLMs that are very far from convergence at the end of training. This is supported by linking together existing work on model souping, linear mode connectivity, mechanistic similarity and path dependence.

Epistemic status: Written in about two hours, but thought about for longer. Experiments could definitely test these hypotheses.

Acknowledgements: Thanks to Ekdeep Singh Lubana for helpful comments and corrections, and discussion which lead to this post. Thanks also to Jean Kaddour, Nandi Schoots, Akbir Khan, Laura Ruis and Kyle McDonell for helpful comments, corrections and suggestions on drafts of this post.

Terminology

Model souping is the procedure of taking a pretrained model, fine-tuning it with different hyperparameters and random seeds on the same task, and then averaging the parameters of all the networks. This gets better results on both in-distribution and out-of-distribution testing in Computer Vision when fine-tuning a large-scale contrastively-pretrained transformer or CNN image model on ImageNet-like tasks.

(Linear) mode connectivity (LMC) between two models on a task means that any (linear) interpolation in parameter space between the two models achieves the same or lower loss as the two models.

A training process is path independent if it always reaches (roughly) the same outcome regardless of irrelevant details or randomness (for example network initialisation or data ordering in supervised learning, or sampling from a policy in supervised learning). A training process is path dependent if it's the opposite.

There is of course nuance in what counts as "irrelevant details of randomness". For this post we can operationalise this as just data ordering and network initialisation in a supervised learning context.

Linking terminology together:

For model souping to work, you likely need linear mode connectivity to hold between all the models you're averaging on the tasks you care about - the average is one point on the linear interpolation. (In fact you need more than that - the average point needs to have better loss, not just the same).

If a training process always produces linearly connected models, then we can think of it as being approximately path independent. Mechanistic Mode Connectivity shows that for converged vision models, two models being linearly connected implies they use similar mechanisms to predict the output (specifically they're invariant to the same set of interventions on the data generating process). Linear Connectivity Reveals Generalization Strategies shows empirically a similar phenomenon: fine-tuned BERT models that are linearly connected generalise in similar ways out-of-distribution.

Overall this gives us this picture of properties a training process can have:

Current Results

Linear Connectivity Reveals Generalization Strategies shows that different fine-tunes of BERT on the same task are often linearly disconnected. In Appendix J they show that this isn't the case for different fine-tunes of RoBERTa, with the main difference between BERT and RoBERTa being much longer pretraining on more data.

BERTs of a feather do not generalize together: Large variability in generalization across...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The mental health challenges that come with trying to have a big impact (Hannah Boettcher on the 80k After Hours Podcast), published by 80000 Hours on July 20, 2023 on The Effective Altruism Forum.

We just published an interview: Hannah Boettcher on the mental health challenges that come with trying to have a big impact. You can click through for the audio, a full transcript, and related links. Below are the episode summary and some key excerpts.

Episode summary

We're in a universe where tradeoffs exist, we have finite resources, we have multiple things we care about, and we have incomplete information. So we have to make guesses and take risks - and that hurts. So I think self-compassion and acceptance come in here, like, "Damn, I so am wishing this were not the case, and by golly, it looks like it still is."

And then I think that it's a matter of recognising that we aren't going to score 100% on any unitary definition of "rightness." And then recognise that, "Well, I could just look at that and stall out forever, or I could make some moves." And probably making moves is preferable to stalling out.

Hannah Boettcher

In this episode of 80k After Hours, Luisa Rodriguez and Hannah Boettcher discuss 4 different kinds of therapy, and how to use them in practice - focusing specifically on people trying to have a big impact.

They cover:

The effectiveness of therapy, and tips for finding a therapist

Moral demandingness

Internal family systems-style therapy

Motivation and burnout

Exposure therapy

Grappling with world problems and x-risk

Perfectionism and imposter syndrome

And the risk of over-intellectualising

Who this episode is for:

High-impact focused people who struggle with moral demandingness, perfectionism, or imposter syndrome

People who feel anxious thinking about the end of the world

80,000 Hours Podcast hosts with the initials LR

Who this episode isn't for:

People who aren't focused on having a big impact

People who don't struggle with any mental health issues

Founders of Scientology with the initials LRH

Get this episode by subscribing to our podcast on the world's most pressing problems and how to solve them: type '80,000 Hours' into your podcasting app. Or read the transcript below.

Producer: Keiran HarrisAudio Engineering Lead: Ben CordellTechnical editing: Dominic ArmstrongContent editing: Katy Moore, Luisa Rodriguez, and Keiran HarrisTranscriptions: Katy Moore

"Gershwin - Rhapsody in Blue, original 1924 version" by Jason Weinberger is licensed under creative commons

Highlights

What makes therapy more or less effective?

Hannah Boettcher: So broadly speaking, we have known for a long time, and it's not controversial, that psychotherapy is efficacious and effective - so under control settings and less controlled settings. And in meta-analytic evidence, the effect size of psychotherapy is approximately 0.8 in comparison to no treatment - and that's conventionally considered a large effect size. Another way to say this would be that in comparison to not getting therapy, getting therapy explains 14%ish of the outcomes in randomised controlled trials. 14% might not sound great, depending on what your priors are, but this is actually really good for healthcare: it's on par with or better than effects of medications, both psychiatric and medically, and it's superior to plenty of medical interventions that are considered effective.

Luisa Rodriguez: What do we know about how effective different types of therapy are? Are those compared in studies?

Hannah Boettcher: Yes, these are definitely compared head to head. And something that's really interesting here is that specific treatment ingredients or therapeutic "modalities" are actually not strong predictors of better outcomes in psychotherapy.

Luisa Rodriguez: Right, this is a thing I've actually heard before, and it bas...