Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Deliberate Play, published by jsteinhardt on October 24, 2021 on LessWrong.

Deliberate practice is an important concept in sports and other high-performance endeavors. It is "practicing with a clear awareness of the specific components of a skill we’re aiming to improve and exactly how to improve them." If you haven't seen this idea before, stop and go read this article (where I got the definition from) or here for a shorter guide. No, really--go read about it. It's one of the most powerful ideas I've come across.

To recap, deliberate practice focuses on improving a specific skill with an awareness of how it translates to a game situation. This is commonly done by executing a drill while visualizing a game-like situation (e.g. imagining defenders you are avoiding) and otherwise replicating game conditions (e.g. practicing free throws after you just ran sprints, to simulate fatigue).

Beyond drills, you can also bring deliberate practice to games by setting a specific intention of what to work on ("practice playing good defense in situation X"), and focusing on that throughout the game. I play pick-up games with other club players a couple times a week, and most of us set some intention going into the game.

However, this post is not about deliberate practice but about two adjacent mental stances, the first of which I'll call max performance. To describe it, consider two contrasting mental stances one could have during a game:

Deliberate practice. Try to improve specific skills. It's okay to make suboptimal plays or otherwise mess up in the process of practicing those skills, but otherwise you want to play well and at high intensity to simulate realistic competition.

Max performance. Play as well as possible. You will likely be very focused (but ideally in a flow-like state) and somewhat pushing your physical boundaries. On the other hand, you are specifically not pushing the boundaries of your skill set and instead relying on what you can do consistently.

Max performance is what you do if you are trying to win that particular game. Many people's image of "trying really hard" is actually not max performance, because they are "playing outside themselves" and trying to do things that they didn't practice enough.

If your goal is long-term growth, you should primarily be doing deliberate practice and not max performance. But max performance is also important, to make sure that skills are translating and because there's no substitute for a fully competitive situation. My rough guess is that of games played, you want a 9:1 ratio of deliberate practice to max performance.

However, there's another important stance that is often overlooked, which I'll call deliberate play. Deliberate play is like deliberate practice in that it has a specific intention, but it has a softer, wider focus than deliberate practice.

As with max performance, deliberate play is adjacent to deliberate practice---but on the opposite side. Deliberate practice allows more freedom to explore new skills than max performance, but it focuses locally on a specific change you want to make. Deliberate play takes this further, by helping you to inhabit a new or unfamiliar framework (e.g. playing aggressively when you usually play conservatively).

I play ultimate frisbee, and my club team (Oakland Firemen) just finished our season. Here's advice I sent to my teammates that exhibits the stance of deliberate play:

The off-season is a great time to practice different play styles. Try more cutting if you usually handle, or handle if you usually cut. If you don’t usually talk much, practice quarterbacking a bit more from the handler position. If you’re often a role player, put yourself in situations where you can practice being the star (i.e. go to slightly lower-level pick-up and dominate; but make it still high-level e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Deliberate Play, published by jsteinhardt on October 24, 2021 on LessWrong.

Deliberate practice is an important concept in sports and other high-performance endeavors. It is "practicing with a clear awareness of the specific components of a skill we’re aiming to improve and exactly how to improve them." If you haven't seen this idea before, stop and go read this article (where I got the definition from) or here for a shorter guide. No, really--go read about it. It's one of the most powerful ideas I've come across.

To recap, deliberate practice focuses on improving a specific skill with an awareness of how it translates to a game situation. This is commonly done by executing a drill while visualizing a game-like situation (e.g. imagining defenders you are avoiding) and otherwise replicating game conditions (e.g. practicing free throws after you just ran sprints, to simulate fatigue).

Beyond drills, you can also bring deliberate practice to games by setting a specific intention of what to work on ("practice playing good defense in situation X"), and focusing on that throughout the game. I play pick-up games with other club players a couple times a week, and most of us set some intention going into the game.

However, this post is not about deliberate practice but about two adjacent mental stances, the first of which I'll call max performance. To describe it, consider two contrasting mental stances one could have during a game:

Deliberate practice. Try to improve specific skills. It's okay to make suboptimal plays or otherwise mess up in the process of practicing those skills, but otherwise you want to play well and at high intensity to simulate realistic competition.

Max performance. Play as well as possible. You will likely be very focused (but ideally in a flow-like state) and somewhat pushing your physical boundaries. On the other hand, you are specifically not pushing the boundaries of your skill set and instead relying on what you can do consistently.

Max performance is what you do if you are trying to win that particular game. Many people's image of "trying really hard" is actually not max performance, because they are "playing outside themselves" and trying to do things that they didn't practice enough.

If your goal is long-term growth, you should primarily be doing deliberate practice and not max performance. But max performance is also important, to make sure that skills are translating and because there's no substitute for a fully competitive situation. My rough guess is that of games played, you want a 9:1 ratio of deliberate practice to max performance.

However, there's another important stance that is often overlooked, which I'll call deliberate play. Deliberate play is like deliberate practice in that it has a specific intention, but it has a softer, wider focus than deliberate practice.

As with max performance, deliberate play is adjacent to deliberate practice---but on the opposite side. Deliberate practice allows more freedom to explore new skills than max performance, but it focuses locally on a specific change you want to make. Deliberate play takes this further, by helping you to inhabit a new or unfamiliar framework (e.g. playing aggressively when you usually play conservatively).

I play ultimate frisbee, and my club team (Oakland Firemen) just finished our season. Here's advice I sent to my teammates that exhibits the stance of deliberate play:

The off-season is a great time to practice different play styles. Try more cutting if you usually handle, or handle if you usually cut. If you don’t usually talk much, practice quarterbacking a bit more from the handler position. If you’re often a role player, put yourself in situations where you can practice being the star (i.e. go to slightly lower-level pick-up and dominate; but make it still high-level e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Phil Trammell on Economic Growth Under Transformative AI, published by Michaël Trazzi on October 24, 2021 on The AI Alignment Forum.

This is a transcript with slides for the latest episode (audio, youtube) of "The Inside View", a podcast I host about the future of AI progress. I interview Phil Trammell, an Oxford PhD student in economics and research associate at the Global Priorities Institute. Phil was my roommate and last time I called him he casually said that he had written a literature review on the econ of transformative AI. A few weeks ago, I decided that I would read that report and translate what I learn along the way to diagrams.

As someone with a background in CS/AI who did not know much about econ, I found this conversation insightful to think about different takeoff scenarios and what could cause them.

Michael: thanks for being at the podcast, it's been a few months since we've wanted to do this. Today we're going to be talking about the paper you published on the GPI website a year ago called "Economic Growth under Transformative AI". That's a paper you co-wrote. Can you tell the audience a bit about what is this GPI, for those who don't know, and what you do there

Phil: right so GPI is an institute at Oxford doing research in philosophy and economics, mainly economic theory, at least at the moment, relevant to people who want to do the most good. So philanthropists in the EA community would be a sort of obvious audience. I'm an econ PhD student at Oxford and I'm sort of being sponsored by GPI and I work part-time at GPI... but mainly my work just overlaps with GPI's goals. So the econ research I do as a grad student is the sort of work that GPI wants done. I sometimes do kind of outreach and so on through GPI but mainly the two hats just sort of overlap. And one of the things I did in my capacity is an econ researcher, which is now on the GPI website as you just mentioned, is this literature review. It's not really an original paper but a synthesis of work that has been done over the past few years on AI and economic growth.

Michael: and that's perfect because the podcast is about AI, so people I think will be kind of familiar with AI or AGI, or those concepts, maybe not with transformative AI if they are not effective altruists, so could you maybe define what you mean by "transformative AI" in this paper?

Phil: people have used that term in slightly different ways. What I mean by it is AI that has a transformative effect, and I'll define that in a moment, on the economic growth rate, the wage rate (growth could stay the same but people's wages could fall a lot or something so that would be different) or the labor share. And the labor share is the fraction of output that gets received in exchange for labor, that's received as wages, as opposed to as capital rent, so it's interest on investment. Historically, for a long time now, even if a thousand things have changed about the economy, about two-thirds of output has gone to labor, being paid out as wages. Hopefully that makes sense, like on average people's income two-thirds of it comes from from their job as opposed to from their investments, and one-third is from investments and that could change a lot if AI takes off in a big enough way.

So those are the three things that could be transformed... transformed by AI, for AI to be transformative on my account. Ok, so what do I mean by transform? It could... if a variable that's been constant for a long time falls to zero, I call that a transformative event (so like the labor share falls to zero). If a growth rate stays constant... so economic growth say, or wages, have been going up at like a few percent a year for a long time: if that falls to zero, that's transformative. Or if the growth rate significantly increases, so if it rises to like 40 percent...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: P₂B: Plan to P₂B Better, published by Ramana Kumar on October 24, 2021 on The AI Alignment Forum.

tl;dr: Most good plans involve taking steps to make better plans. Making better plans is the convergent instrumental goal, of which all familiar convergent instrumental goals are an instance. This is key to understanding what agency is and why it is powerful.

Planning means using a world model to predict the consequences of various courses of actions one could take, and taking actions that have good predicted consequences. (We think of this with the handle “doing things for reasons,” though we acknowledge this may be an idiosyncratic use of “reasons.”)

We take “planning” to include things that are relevantly similar to this procedure, such as following a bag of heuristics that approximates it. We’re also including actually following the plans, in what might more clunkily be called “planning-acting.”

Planning, in this broad sense, seems essential to the kind of goal-directed, consequential, agent-like intelligence that we expect to be highly impactful. This sequence explains why.

One Convergent Instrumental Goal to Rule them All

Consider the maxim

“make there be more and/or better planning towards your goal.”

This section argues that all the classic convergent instrumental goals are special cases of this maxim.

To flesh this out a little, here are some categories of ways to follow the maxim. Remember that a planner is typically close (in terms of what it might affect via action) to at least one planner – itself – so these directions can typically be applied in the first case to the planner itself.

Make the planners with your goal better at planning. For example, get them new relevant data to work with, get them to run faster or more effective algorithms, build protections against value drift, etc.

Make the planners with your goal have better options. For example, move them to better locations, get them more resources, get them more power or a greater number of options to select from, have them take steps in an object-level plan towards the goal.

Make there be more planners with your goal. For example, keep yourself running and aligned with your goal, acquire delegates and subordinates, convince followers and converts, build successors.

Reviewing Omohundro's “The Basic AI Drives” and Bostrom’s “The Superintelligent Will,” we extract a list of convergent instrumental goals, and find that they are all instances of the maxim “make there be more/better planners for the current goal:”

Self-preservation / self-protection:

Make there be more planners that have your goals, focusing on reusing the existing planner, that is, preventing its destruction.

Self-improvement:

Make there be better planners with your goals, focusing on making the existing one better.

Resource Acquisition:

Make there be better planners with your goals, focusing on making the existing one able to take more effective actions.

Goal-content Integrity:

Make there be more planners that have your goals, focusing on ensuring the existing planners that have your goals keep those goals and avoid them being changed.

Resource-use Efficiency:

Cognitive Enhancement:

Creativity:

Same as self-improvement

Technological Perfection:

Same as resource acquisition and/or self-improvement

Rationality:

Omohundro takes this to be something like “make the utility function explicit” along with “maximize expected utility.”

Thus it is similar to goal-content integrity and self-improvement.

Utility-function preservation:

Similar to goal-content integrity.

Prevent counterfeit utility:

Essentially this is avoiding wireheading. Omohundro: “An important class of vulnerabilities arises when the subsystems for measuring utility become corrupted.”

Thus it is similar to goal-content integrity.

Seeking a concise, memorable-yet-accurate name for t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: P₂B: Plan to P₂B Better, published by Ramana Kumar on October 24, 2021 on The AI Alignment Forum.

tl;dr: Most good plans involve taking steps to make better plans. Making better plans is the convergent instrumental goal, of which all familiar convergent instrumental goals are an instance. This is key to understanding what agency is and why it is powerful.

Planning means using a world model to predict the consequences of various courses of actions one could take, and taking actions that have good predicted consequences. (We think of this with the handle “doing things for reasons,” though we acknowledge this may be an idiosyncratic use of “reasons.”)

We take “planning” to include things that are relevantly similar to this procedure, such as following a bag of heuristics that approximates it. We’re also including actually following the plans, in what might more clunkily be called “planning-acting.”

Planning, in this broad sense, seems essential to the kind of goal-directed, consequential, agent-like intelligence that we expect to be highly impactful. This sequence explains why.

One Convergent Instrumental Goal to Rule them All

Consider the maxim

“make there be more and/or better planning towards your goal.”

This section argues that all the classic convergent instrumental goals are special cases of this maxim.

To flesh this out a little, here are some categories of ways to follow the maxim. Remember that a planner is typically close (in terms of what it might affect via action) to at least one planner – itself – so these directions can typically be applied in the first case to the planner itself.

Make the planners with your goal better at planning. For example, get them new relevant data to work with, get them to run faster or more effective algorithms, build protections against value drift, etc.

Make the planners with your goal have better options. For example, move them to better locations, get them more resources, get them more power or a greater number of options to select from, have them take steps in an object-level plan towards the goal.

Make there be more planners with your goal. For example, keep yourself running and aligned with your goal, acquire delegates and subordinates, convince followers and converts, build successors.

Reviewing Omohundro's “The Basic AI Drives” and Bostrom’s “The Superintelligent Will,” we extract a list of convergent instrumental goals, and find that they are all instances of the maxim “make there be more/better planners for the current goal:”

Self-preservation / self-protection:

Make there be more planners that have your goals, focusing on reusing the existing planner, that is, preventing its destruction.

Self-improvement:

Make there be better planners with your goals, focusing on making the existing one better.

Resource Acquisition:

Make there be better planners with your goals, focusing on making the existing one able to take more effective actions.

Goal-content Integrity:

Make there be more planners that have your goals, focusing on ensuring the existing planners that have your goals keep those goals and avoid them being changed.

Resource-use Efficiency:

Cognitive Enhancement:

Creativity:

Same as self-improvement

Technological Perfection:

Same as resource acquisition and/or self-improvement

Rationality:

Omohundro takes this to be something like “make the utility function explicit” along with “maximize expected utility.”

Thus it is similar to goal-content integrity and self-improvement.

Utility-function preservation:

Similar to goal-content integrity.

Prevent counterfeit utility:

Essentially this is avoiding wireheading. Omohundro: “An important class of vulnerabilities arises when the subsystems for measuring utility become corrupted.”

Thus it is similar to goal-content integrity.

Seeking a concise, memorable-yet-accurate name for t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My ML Scaling bibliography, published by gwern on October 23, 2021 on LessWrong. .

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My ML Scaling bibliography, published by gwern on October 23, 2021 on LessWrong. .

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Towards Deconfusing Gradient Hacking, published by leogao on October 24, 2021 on The AI Alignment Forum.

[Epistemic status: brainstorming, less confused than last time since it seems to provide for a nice taxonomy of things to look into rather than just a bunch of random loose threads.]

This post would not exist without the EleutherAI discord and all the wonderful people there. Special thanks to Evan Hubinger, John Schulman, and Adam Shimi for providing feedback on drafts of this post.

The goal of this post is to identify a different way of framing the gradient hacking problem, connect it to our intuitions, and use it to create a rough taxonomy of different types of potential gradient hacking. Hopefully, using this taxonomy, we can tackle more well-defined subproblems to narrow down the space of things that could potentially gradient hack.

When we think about gradient hacking, the most intuitive framing is to consider some kind of agent embedded inside a larger network (like a GPT) that somehow intentionally modifies the loss landscape of the larger network with respect to the base loss, and that this modification makes it so that in optimizing for the base objective, the base optimizer also happens to optimize the mesaobjective. Here I consider the base objective to be a function Θ→R from the params of the network to the reals, that has all the training data baked in for simplicity, and the mesaobjective another function Θ→R, possibly with some constraint that both objectives have to be indifferent between models which behave the same on all inputs. The "somehow" is often considered to be some kind of perturbing or otherwise making the output of the larger network worse whenever the mesaobjective isn't met, therefore creating an incentive for gradient descent to improve the mesaobjective.

One example of this line of thinking can be found in my last post about gradient hacking. Unfortunately, I think there are some confusions with this framing.

Full loss landscapes

First off, the base loss landscape of the entire model is a function Θ→R that's the same across all training steps, and the configuration of the weights selects somewhere on this loss landscape. Configuring the weights differently can put the model on a different spot on this landscape, but it can't change the shape of the landscape itself.

Note that this doesn't contradict the interpretation of the gradient hacker as having control over the loss landscape through subjunctive dependence. As an analogy, in Newcomb's problem even if you accept that there is subjunctive dependence of the contents of the box on your decision and conclude you should one-box, it's still true that the contents of the box cannot change after Omega has set them up and that there is no causal dependence of the contents of the box on your action, even though the dominated action argument no longer holds because of the subjunctive dependence.

To emphasize this landscape covering the entire model, let's call the loss landscape of the base loss with respect to the entire model a full loss landscape. Any configuration of the network really just selects a point somewhere on this landscape, and to claim that any such point effectively does gradient hacking would be to argue that as gradient descent evolves this point over time, it manages to also get better at the mesaobjective in some sense. This may seem trivial, but I've noticed that unless you consider this explicitly, it's really easy to get it mixed up with cases where the landscape really does change. This suggests a new framing, where we define gradient hackers with respect to a particular mesaobjective as subsets of parameter space that tend to improve on the given mesaobjective over time, with different mesaobjectives defining different subsets (this is a bit tricky to formalize ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Towards Deconfusing Gradient Hacking, published by leogao on October 24, 2021 on The AI Alignment Forum.

[Epistemic status: brainstorming, less confused than last time since it seems to provide for a nice taxonomy of things to look into rather than just a bunch of random loose threads.]

This post would not exist without the EleutherAI discord and all the wonderful people there. Special thanks to Evan Hubinger, John Schulman, and Adam Shimi for providing feedback on drafts of this post.

The goal of this post is to identify a different way of framing the gradient hacking problem, connect it to our intuitions, and use it to create a rough taxonomy of different types of potential gradient hacking. Hopefully, using this taxonomy, we can tackle more well-defined subproblems to narrow down the space of things that could potentially gradient hack.

When we think about gradient hacking, the most intuitive framing is to consider some kind of agent embedded inside a larger network (like a GPT) that somehow intentionally modifies the loss landscape of the larger network with respect to the base loss, and that this modification makes it so that in optimizing for the base objective, the base optimizer also happens to optimize the mesaobjective. Here I consider the base objective to be a function Θ→R from the params of the network to the reals, that has all the training data baked in for simplicity, and the mesaobjective another function Θ→R, possibly with some constraint that both objectives have to be indifferent between models which behave the same on all inputs. The "somehow" is often considered to be some kind of perturbing or otherwise making the output of the larger network worse whenever the mesaobjective isn't met, therefore creating an incentive for gradient descent to improve the mesaobjective.

One example of this line of thinking can be found in my last post about gradient hacking. Unfortunately, I think there are some confusions with this framing.

Full loss landscapes

First off, the base loss landscape of the entire model is a function Θ→R that's the same across all training steps, and the configuration of the weights selects somewhere on this loss landscape. Configuring the weights differently can put the model on a different spot on this landscape, but it can't change the shape of the landscape itself.

Note that this doesn't contradict the interpretation of the gradient hacker as having control over the loss landscape through subjunctive dependence. As an analogy, in Newcomb's problem even if you accept that there is subjunctive dependence of the contents of the box on your decision and conclude you should one-box, it's still true that the contents of the box cannot change after Omega has set them up and that there is no causal dependence of the contents of the box on your action, even though the dominated action argument no longer holds because of the subjunctive dependence.

To emphasize this landscape covering the entire model, let's call the loss landscape of the base loss with respect to the entire model a full loss landscape. Any configuration of the network really just selects a point somewhere on this landscape, and to claim that any such point effectively does gradient hacking would be to argue that as gradient descent evolves this point over time, it manages to also get better at the mesaobjective in some sense. This may seem trivial, but I've noticed that unless you consider this explicitly, it's really easy to get it mixed up with cases where the landscape really does change. This suggests a new framing, where we define gradient hackers with respect to a particular mesaobjective as subsets of parameter space that tend to improve on the given mesaobjective over time, with different mesaobjectives defining different subsets (this is a bit tricky to formalize ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: An estimate of the value of Metaculus questions, published by NunoSempere on October 22, 2021 on The Effective Altruism Forum.

tl;dr: Metaculus is an EA-adjacent forecasting platform. By my estimation, most Metaculus questions fail to directly influence decisions, but a few fall at the sweet spot between large scope, high decision relevance, and good fit for forecasters. That said, perhaps only a fraction of Metaculus’ impact is captured by the impact of their questions. In any case, the EA community could perhaps use evaluations of the value of Metaculus questions to incentivize it to produce more valuable questions.

Overall impressions

The holy grail for Metaculus would be questions on important topics which the kind of person who encounters Metaculus is in a position to do something about.

However, there is a tension between questions being decision-relevant and having a large scope because smaller entities might be more amenable than larger ones to being influenced. So it could turn out that the impact sweet spot is asking intimately decision-relevant questions that small organizations are willing to listen to. Conversely, as Metaculus grows its audience, perhaps questions with a large scope which change small decisions for many people might be more valuable.

But a large number of Metaculus questions fall in neither of those categories. On the one hand, we have very narrow questions which do not affect any decisions, such as What will the women’s winning 100m time in the 2024 Olympic final be? On the other hand, we also have questions such as “Will Israel recognize Palestine by 2070?” or “When will Hong Kong stop being a Special Administrative Region of China?”. These events seem so large as to essentially be non-influenceable, and thus I’d tend to think that their Metaculus questions are not valuable [3].

For Metaculus, another constraint is to have questions that interest forecasters. Interestingness is necessary to build a community around forecasting that may later have a large instrumental value.

Below, I outline a simple rubric that I think captures an important part of how Metaculus questions lead to value in the world. I look at questions’ decision-relevance, forecasting fit, and scope.

Perhaps predictably, Metaculus is very good at making questions that are a good fit for being forecastable and suitable for forecasters instead of things like financial markets. Now that a forecasting community already exists and is known to be accurate and calibrated, it seems to me that the next bottleneck is to make forecasts action-guiding, perhaps by tweaking the scope of questions that Metaculus asks, or by reaching out to specific organizations.

Overall, the driving motivation behind this post is the perspective that:

even though I think that medium or long-term interventions can have a larger impact than spending their funds on the best near-term interventions,

I'd still like to have a healthy degree of paranoia

because I think that the default might be to have essentially no impact,

and I would dislike for that to be the case for Metaculus

Methodology

I took a random sample of 200 questions and rated them according to:

how far removed I think they are from influencing decisions

how good of a fit they are for forecasters, and

the scope of the matters they ask about

My results can be found here [1]. This work is rough and not meant to be definitive. In particular, I think that some of the rankings might be subject to some degree of idiosyncrasy. Nonetheless, I hope that this might still be informative, and lead to some reflection about if and how Metaculus questions can be valuable.

Decision-relevance

How likely is this to change actual decisions?

Directly: Someone will read this and change their decisions as a result (e.g., micro-covid calculations). Numeric value:...