Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [linkpost] Christiano on agreement/disagreement with Yudkowsky's "List of Lethalities", published by Owen Cotton-Barratt on June 19, 2022 on The Effective Altruism Forum.

Eliezer Yudkowsky's recent post on AGI ruin seems to have sparked a good amount of thinking and discussion (e.g. in the comments there; on LessWrong; in this post from today). I'm glad this is happening. This is a link post for Paul Christiano's response, which is I think worth reading for anyone following the discussion.

Personally I like Christiano's response a good bit. More than anything else I've read in the vicinity I find myself nodding along and thinking "yeah that's a good articulation of my feelings on this". I think it's interesting that it lacks the rhetorical oomph of Yudkowsky's post; on the whole I'm into the rhetorical oomph for pulling attention onto a cluster of topics which I think are super important, although I feel a bit sad that the piece which is most attention-pulling doesn't seem to be the one which is most truth-tracking.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Let's See You Write That Corrigibility Tag, published by Eliezer Yudkowsky on June 19, 2022 on The AI Alignment Forum.

The top-rated comment on "AGI Ruin: A List of Lethalities" claims that many other people could've written a list like that.

"Why didn't you challenge anybody else to write up a list like that, if you wanted to make a point of nobody else being able to write it?" I was asked.

Because I don't actually think it does any good, or persuades anyone of anything, people don't like tests like that, and I don't really believe in them myself either. I couldn't pass a test somebody else invented around something they found easy to do, for many such possible tests.

But people asked, so, fine, let's actually try it this time. Maybe I'm wrong about how bad things are, and will be pleasantly surprised. If I'm never pleasantly surprised then I'm obviously not being pessimistic enough yet.

So: As part of my current fiction-writing project, I'm currently writing a list of some principles that dath ilan's Basement-of-the-World project has invented for describing AGI corrigibility - the sort of principles you'd build into a Bounded Thing meant to carry out some single task or task-class and not destroy the world by doing it.

So far as I know, every principle of this kind, except for Jessica Taylor's "quantilization", and "myopia" (not sure who correctly named this as a corrigibility principle), was invented by myself; eg "low impact", "shutdownability". (Though I don't particularly think it hopeful if you claim that somebody else has publication priority on "low impact" or whatevs, in some stretched or even nonstretched way; ideas on the level of "low impact" have always seemed cheap to me to propose, harder to solve before the world ends.)

Some of the items on dath ilan's upcoming list out of my personal glowfic writing have already been written up more seriously by me. Some haven't.

I'm writing this in one afternoon as one tag in my cowritten online novel about a dath ilani who landed in a D&D country run by Hell. One and a half thousand words or so, maybe. (2169 words.)

How about you try to do better than the tag overall, before I publish it, upon the topic of corrigibility principles on the level of "myopia" for AGI? It'll get published in a day or so, possibly later, but I'm not going to be spending more than an hour or two polishing it.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Where I agree and disagree with Eliezer, published by Paul Christiano on June 19, 2022 on The AI Alignment Forum.

(Partially in response to AGI Ruin: A list of Lethalities. Written in the same rambling style. Not exhaustive.)

Agreements

Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much easier failure mode than killing everyone with destructive physical technologies.

Catastrophically risky AI systems could plausibly exist soon, and there likely won’t be a strong consensus about this fact until such systems pose a meaningful existential risk per year. There is not necessarily any “fire alarm.”

Even if there were consensus about a risk from powerful AI systems, there is a good chance that the world would respond in a totally unproductive way. It’s wishful thinking to look at possible stories of doom and say “we wouldn’t let that happen;” humanity is fully capable of messing up even very basic challenges, especially if they are novel.

I think that many of the projects intended to help with AI alignment don't make progress on key difficulties and won’t significantly reduce the risk of catastrophic outcomes. This is related to people gravitating to whatever research is most tractable and not being too picky about what problems it helps with, and related to a low level of concern with the long-term future in particular. Overall, there are relatively few researchers who are effectively focused on the technical problems most relevant to existential risk from alignment failures.

There are strong social and political pressures to spend much more of our time talking about how AI shapes existing conflicts and shifts power. This pressure is already playing out and it doesn’t seem too likely to get better. I think Eliezer’s term “the last derail” is hyperbolic but on point.

Even when thinking about accident risk, people’s minds seem to go to what they think of as “more realistic and less sci fi” risks that are much less likely to be existential (and sometimes I think less plausible). It’s very possible this dynamic won’t change until after actually existing AI systems pose an existential risk.

There is a good chance that an AI catastrophe looks like an abrupt “coup” where AI systems permanently disempower humans with little opportunity for resistance. People seem to consistently round this risk down to more boring stories that fit better with their narratives about the world. It is quite possible that an AI coup will be sped up by humans letting AI systems control killer robots, but the difference in timeline between "killer robots everywhere, AI controls everything" and "AI only involved in R&D" seems like it's less than a year.

The broader intellectual world seems to wildly overestimate how long it will take AI systems to go from “large impact on the world” to “unrecognizably transformed world.” This is more likely to be years than decades, and there’s a real chance that it’s months. This makes alignment harder and doesn’t seem like something we are collectively prepared for.

Humanity usually solves technical problems by iterating and fixing failures; we often resolve tough methodological disagreements very slowly by seeing what actually works and having our failures thrown in our face. But it will probably be possible to build valuable AI products without solving alignment, and so reality won’t “force us” to solve alignment until it’s too late. This seems like a case where we will have to be unusually reliant on careful reasoning rather than empirical feedback loops for some of the highest-level questions.

AI systems will ultimately be wildly superhuman, and there probably won’t be strong technological hurdles right around human level. Extrapolating the rate of existing AI progress suggests you don’t get too much time ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On Deference and Yudkowsky's AI Risk Estimates, published by Ben Garfinkel on June 19, 2022 on The Effective Altruism Forum.

Note: I mostly wrote this post after Eliezer Yudkowsky’s “Death with Dignity” essay appeared on LessWrong. Since then, Jotto has written a post that overlaps a bit with this one, which sparked an extended discussion in the comments. You may want to look at that discussion as well. See also, here, for another relevant discussion thread.[1]

Introduction

Most people, when forming their own views on risks from misaligned AI, have some inclination to defer to others who they respect or think of as experts.

This is a reasonable thing to do, especially if you don’t yet know much about AI or haven’t yet spent much time scrutinizing the arguments. If someone you respect has spent years thinking about the subject, and believes the risk of catastrophe is very high, then you probably should take that information into account when forming your own views.

It’s understandable, then, if Eliezer Yudkowsky’s recent writing on AI risk helps to really freak some people out. Yudkowsky has probably spent more time thinking about AI risk than anyone else. Along with Nick Bostrom, he is the person most responsible for developing and popularizing these concerns. Yudkowsky has now begun to publicly express the view that misaligned AI has a virtually 100% chance of killing everyone on Earth - such that all we can hope to do is “die with dignity.”

The purpose of this post is, simply, to argue that people should be wary of deferring too much to Eliezer Yudkowsky when it comes to estimating AI risk.[2] In particular, I think, they shouldn’t defer to him more than they would defer to anyone else who seems smart and has spent a reasonable amount of time thinking about AI risk.[3]

The post highlights what I regard as some negative aspects of Yudkowsky’s track record, when it comes to technological risk forecasting. I think these examples suggest that (a) his track record is at best fairly mixed and (b) he has some tendency toward expressing dramatic views with excessive confidence. As a result, I don’t personally see a strong justification for giving his current confident and dramatic views about AI risk a great deal of weight.[4]

I agree it’s highly worthwhile to read and reflect on Yudkowsky’s arguments. I also agree that potential risks from misaligned AI deserve serious attention - and are even, plausibly, more deserving of attention than any other existential risk.[5] I just don’t think people should make too much of the fact that Yudkowsky believes we’re doomed.

Why write this post?

Before diving in, it may be worth saying a little more about why I hope this post might be useful. (Feel free to skip ahead if you're not interested in this section.)

In brief, it matters what the existential risk community believes about the risk from misaligned AI. I think that excessively high credences in doom can lead to:

poor prioritization decisions (underprioritizing other risks, including other potential existential risks from AI)

poor community health (anxiety and alienation)

poor reputation (seeming irrational, cultish, or potentially even threatening), which in turn can lead to poor recruitment or retention of people working on important problems[6]

My own impression is that, although it's sensible to take potential risks from misaligned AI very seriously, a decent number of people are now more freaked out than they need to be. And I think that excessive deference to some highly visible intellectuals in this space, like Yudkowsky, may be playing an important role - either directly or through deference cascades.[7] I'm especially concerned about new community members, who may be particularly inclined to defer to well-known figures and who may have particularly limited awareness of ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: To what extent have ideas and scientific discoveries gotten harder to find?, published by lsusr on June 18, 2022 on LessWrong.

This post was funded by a grant from Ben Pace.

Ben Pace asks:

To what extent have ideas and scientific discoveries gotten harder to find?

Related: Why are there no gentlemen scientists any more — i.e. rich people who make novel scientific discoveries? Like Fermat and Pascal.

Some theories have been put forward by Scott Alexander and Holden Karnofsky. Maybe they’re right, maybe not.

Scott Alexander treats scientific ideas as nonrenewable.

Imagine scientists venturing off in some research direction. At the dawn of history, they don’t need to venture very far before discovering a new truth. As time goes on, they need to go further and further.

Holden Karnofsky believes the pattern applies to both art and science.

The broad theme is that across a variety of areas in both art and science, we see a form of "innovation stagnation": the best-regarded figures are disproportionately from long ago, and our era seems to "punch below its weight" when considering the rise in population, education, etc. Since the patterns look fairly similar for art and science, and both are forms of innovation, I think it's worth thinking about potential common factors.

I have a different perspective.

Science

Physics

The most important scientific discoveries are those which are the most general and the most useful. Physics is the most general of sciences. Physics is basically solved.

There are unsolved problems in physics. Dark matter remains a mystery. Quantum mechanics has yet to be unified with general relativity. But the holes in physics don't matter. You don't need a Grand Unified Theory of quantum relativity to build a Mars base or a fusion reactor. All you need is today's physics plus a whole lot of engineering.

The recent discoveries in physics like quantum computing and the photograph of a black hole aren't really discoveries about the fundamental Laws of Physics. They're technological achievements.

All of the rest of science is just applied physics too. One could argue that biology, chemistry and so on are just footnotes to Einstein. The fundamental laws of the universe are (for all practical purposes) known. The remaining questions are:

Astrophysics i.e. the study of places that don't matter because we lack the technology to go there.

What has biology built out of matter?

What can we build out of matter?

No physicist will ever again make a discovery (in physics) as impactful as the great 20th century physicists. All the important fruit has been picked.

But that doesn't mean science has been exhausted. It just means physics has been exhausted. Biology is advancing fast. I'll never get tired of this graph about how, since 2007, biotechnology has advanced faster than computer technology ever did.

As recently as 2015, Nick Lane published a book that might have solved the origin of life. And biology isn't even the most exciting frontier.

Machine Learning

In Contra Hoel, I talked about machine learning as feeling different from some other scientific fields: there are frequent exciting new discoveries. This shouldn’t be surprising. Physics is stagnant because Newton and Einstein already got all the cool results. But Newton and Einstein didn’t have TPUs so they couldn’t discover things about machine learning.

The Low-Hanging Fruit Argument: Models And Predictions by Scott Alexander

Do discoveries in machine learning count as science or technology? If we use the strictest definition of "science" then machine learning counts as "technology". But machine learning is also informing our understanding of the human mind. Psychology definitely counts as "science". "Untangling how intelligence works" is the most important scientific problem of our age.

Ambitious people go to whe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Seven ways to become unstoppably agentic, published by Evie Cottrell on June 18, 2022 on The Effective Altruism Forum.

Thank you to Fin Moorhouse and Bella Forristal for helpful feedback on this post! Thanks to Fin for getting me so excited about writing this during coworking breaks that I struggled to focus on maths for the rest of the day.

Nearly all of the best things that have happened to me in the last year have been a result of actively seeking and asking for those things. I have slowly been developing the skill of agency.

By ‘agency’ I mean the ability to get what you want across different contexts – the general skill of coming up with ambitious goals and actually achieving them, whatever they are. In my experience, I’ve found this is learnable.

Neel Nanda explains the concept of agency really well in this blog post, so I won’t repeat it here. Instead, I’ll focus on how to learn it.

It’s worth acknowledging that agency is often socially discouraged in different minority groups. Speaking from my own experience, I used to feel shameful around being agentic – I associated it with being entitled or ‘too much’ or bossy – which seems to be a phenomenon that many women experience. I sometimes still have a voice in my head saying, “who do you think you are???” But, fortunately, I’m paying less and less attention to it.

In no particular order, nine ways I’ve found to become unstoppably agentic:

1. Figure out what you need, figure out who can help you get it, ask them for it

There will (nearly) always be people who can help you achieve your goals better. Find them and ask them to help you. They might say yes!

If you don’t ask, the answer is always no.

When I was self-studying for my A-Levels (UK high school exams), I reached out to an EA who I’d never met, asking to chat about why I was finding self-studying so hard. We ended up having a call, which I found extremely helpful. At the time, I had a huge ugh-field around my Anki flashcards, which he completely helped me clear up. I then started sending him my daily plan (after checking this was ok), which turned out to be a really valuable accountability system. He gave support and encouragement countless times over the few months. These daily check-ins were the best thing that could have happened to my A-Levels, and led me to feel way more supported throughout my time self-studying.

Having a low bar for asking for help is a learnable mindset. Now, whenever I encounter something difficult or aversive, my first thought is, “who would be the best person to have on my side here?” Then I begin to reach out to people who could help.

Things that make this hard in practice:

It’s hard to actually know what you want or need;

It’s hard to be creative about solutions and imagine how another person would help solve it;

It can be intimidating to ask the person;

You are making yourself vulnerable to rejection, which can feel crushing and bad.

Some tentative solutions:

Questions that help me figure out what I need: If this was easy, what would it look like? What is the bottleneck here and what can I do to solve it? What would it look like if I could do this with 10% of the time/ effort? What’s weighing on my mind? Whose advice would help me solve this problem? See this list of questions for more.

One suggestion is that you can ask people if they know anyone who can help you. I recently asked an EA for help with this, and they generated nearly ten people who I could reach out to. (Also see point number 7 about considering a wide option space to be creative about solutions.)

People can be scary! One thing to bear in mind though, is that EAs really love agency. So if the person you are asking for help is an EA, they’re likely to be impressed that you were so proactive. But, yup, it’s scary to put yourself out there. I really think it’s wort...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pivotal outcomes and pivotal processes, published by Andrew Critch on June 17, 2022 on The Effective Altruism Forum.

tl;dr: If you think humanity is on a dangerous path, and needs to "pivot" toward a different future in order to achieve safety, consider how such a pivot could be achieved by multiple acts across multiple persons and institutions, rather than a single act. Engaging more actors in the process is more costly in terms of coordination, but in the end may be a more practicable social process involving less extreme risk-taking than a single "pivotal act".

Preceded by: “Pivotal Act” Intentions: Negative Consequences and Fallacious Arguments

[This post is also available on LessWrong.]

In the preceding post, I argued for the negative consequences of the intention to carry out a pivotal act, i.e., a single, large world-changing act sufficient to 'pivot' humanity off of a dangerous path onto a safer one. In short, there are negative side effects of being the sort of institution aiming or willing to carry out a pivotal act, and those negative side effects alone might outweigh the benefit of the act, or prevent the act from even happening.

In this post, I argue that it's still a good idea for humanity-as-a-whole to make a large / pivotal change in its developmental trajectory in order to become safer. In other words, my main concern is not with the "pivot", but with trying to get the whole "pivot" from a single "act", i.e., from a single agent-like entity, such a single human person, institution, or AI system.

Pivotal outcomes and processes

To contrast with pivotal acts, here's a simplified example of a pivotal outcome that one could imagine making a big positive difference to humanity's future, which in principle could be brought about by a multiplicity of actors:

(the "AI immune system") The whole internet — including space satellites and the internet-of-things — becomes way more secure, and includes a distributed network of non-nuclear electromagnetic pulse emitters that will physically shut down any tech infrastructure appearing to be running rogue AI agents.

(For now, let's set aside debate about whether this outcome on its own would be pivotal, in the sense of pivoting humanity onto a safe developmental trajectory... it needs a lot more details and improvements to be adequate for that! My goal in this post is to focus on how the outcome comes about. So for the sake of argument I'm asking to take the "pivotality" of the outcome for granted.)

If a single institution imposed the construction of such an AI immune system on its own, that would constitute a pivotal act. But if a distributed network of several states and companies separately instituted different parts of the change — say, designing and building the EMP emitters, installing them in various jurisdictions, etc. — then I'd call that a pivotal distributed process, or pivotal process for short.

In summary, a pivotal outcome can be achieved through a pivotal (distributed) process without a single pivotal act being carried out by any one institution. Of course, the "can" there is very difficult, and involves solving a ton of coordination problems that I'm not saying humanity will succeed in solving. However, aiming for a pivotal outcome via a pivotal distributed process definitively seems safer to me, in terms of the dynamics it would create between labs and militaries, compared to a single lab planning to do it all on their own.

Revisiting the consequences of pivotal act intentions

In AGI Ruin, Eliezer writes the following, I believe correctly:

The reason why nobody in this community has successfully named a 'pivotal weak act' where you do something weak enough with an AGI to be passively safe, but powerful enough to prevent any other AGI from destroying the world a year later - and yet also we can't just ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: "Brain enthusiasts" in AI Safety, published by Jan on June 18, 2022 on LessWrong.

TL;DR: If you're a student of cognitive science or neuroscience and are wondering whether it can make sense to work in AI Safety, this guide is for you! (Spoiler alert: the answer is "mostly yes").

Motivation

AI Safety is a rapidly growing field of research that is singular in its goal: to avoid or mitigate negative outcomes from advanced AI. At the same time, AI Safety research touches on many aspects of human lives: from the philosophy of human values, via the neuroscience of human cognition, to the intricacies of human politics and coordination. This interplay between a singular goal with multiple facets makes the problem intrinsically interdisciplinary and warrants the application of various tools by researchers with diverse backgrounds.

Both of us (Sam & Jan) have backgrounds in cognitive science and neuroscience (we'll use the blanket term "brain enthusiasts" from here on). Important characteristics of brain enthusiasts are (see if these apply to you, dear reader):

A propensity for empirical data and experiments.

Regarding coding and mathematics as tools rather than as ends in themselves.

Having accumulated semi-random facts about some biological systems.

While being brain enthusiasts undoubtedly makes us biased in favor of their importance, it also gives us a (hopefully) useful inside view of how brain enthusiasts might meaningfully contribute to AI Safety. In this post, we attempt to shine a light on the current representation of brain enthusiasts in AI Safety and provide some advice on how brain enthusiasts might enter the field.

A long tail of number-lovers

When you hear terms like "AI Alignment", "AI Safety," or "AGI", you probably think of people with a strong technical background, e.g. in computer science, mathematics, or physics. This is an instance where stereotypes are correct:

Using a previously published dataset, we determined the number of Alignment Forum posts per researcher and sorted the resulting table. We then added a "Background" column, where we noted the field in which the researcher obtained their most recent degree. As observed previously, the number of published posts is a long-tailed distribution, with Stuart Armstrong dominating over everyone.

Almost everyone on the list has a computer science, physics, or mathematics background. Notable exceptions are three researchers with an "officially different" formal education (Abram Demski, Richard Ngo, and Daniel Kokotajlo) and those researchers who now work on brain-enthusiasm topics:

Steven Byrnes Intro to Brain-like AGI Safety post series

John Wentworth and his background in molecular and cellular biology

People not included in the list above, like Lee Sharkey or Carla Zoe Cremer, have a computational neuroscience background.

We'd wager the bet that most people on the list don't identify with their "formal education" too much and regard themselves primarily as AI Safety researchers - but still, the uniformity of backgrounds is striking. For someone at the beginning of their career who doesn't have the option to major in "AI Safety", is a major in mathematics the next best thing? Is a strong technical background necessary or merely sufficient?

How did we get here?

There are (at least) two possible explanations for why the composition of the field is biased in favor of researchers with technical backgrounds:

Either, there is an "efficient market" dynamic whereby only people with a technical background make progress and stay in the field,

Or there is a "founder effect" dynamic whereby the first people entering the field had an outsized effect on the composition of the field later on.

Between these two poles, there is an entire spectrum of what "solving” AI Safety might require:

AI Safety almost exclusively requir...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Scott Aaronson is joining OpenAI to work on AI safety, published by peterbarnett on June 18, 2022 on LessWrong.

Scott Aaronson is a computer scientist at the University of Texas in Austin, whose research mainly focuses on quantum computing and complexity theory. He's at least very adjacent to the Rationalist/LessWrong community. After some comments on his blog and then coversations with Jan Leike, he's decided work for one year on AI safety at OpenAI.

To me this is a reasonable update that people who are sympathetic to AI safety can be convinced to actually do direct work.

Aaronson might be one of the easier people to induce to do AI safety work, but I imagine there are also other people who are worth talking to about doing direct work on AI safety.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Quantifying General Intelligence, published by JasonBrown on June 17, 2022 on The AI Alignment Forum.

Introduction

After having thought about the underlying definitions of agents, goals, and intelligence for quite a while, I realised that these things are often defined quite qualitatively. I decided to try and build up a framework for conceptualising intelligent systems in such a way that it would allow us to quantify things, such as general intelligence. This can then be the starting point for developing tools for quantifying and comparing these entities, giving us a better idea of how close or far systems are to human level general intelligence. The definitions are not meant to be philosophical truths, rather they are meant to be useful tools that will allow us to analyse and gain insight into these systems and how they relate to one another.

This is my first long form piece of writing / serious thinking about this sort of thing so any and all feedback is welcome!

Desirable Properties

We begin by identifying several desired properties that would increase the utility and robustness of our framework, giving us something to aim at.

Sufficient: If our definitions relied upon, or referenced, things that are poorly defined themselves, we would just be moving the problem back a step and not actually gaining any insight.

Measurable: Intelligence is a broad spectrum, this especially visible in the natural world. A good definition would reflect this and give us a continuous measure of intelligence that allows sensible comparisons.

Implementation Independent: It's easy to compare somethings capabilities to humans in order to ascertain their intelligence. We want our definitions to be free from bias towards any particular implementation or version of intelligence, so that it can recognise intelligence which operates in a way unfamiliar to us, or in a way we don't understand.

Minimal Grey Areas: Many definitions could leave large grey areas on boundaries between classifications, or not make sense when applied to domains they were not designed with in mind. This should be avoided.

Useable: Sometimes a seemingly 'perfect' definition is infeasible to actually apply, and so is of no practical use. A definition which is infeasible to theoretically calculate, could have a method to estimate it. Quantifying how reliable or accurate the estimates are would also be useful.

Agents

A classic definition of an agent is something that interacts with an environment, choosing an action (or actions), to achieve some desired goal. To make this rigorous, we will define an agent in our framework as something that produces an action based on the state of its environment. The action will be an effect on the environment. In a mathematical sense an agent is a function from the environment state to an effect, the effect is then a function from one environment state to another. Both functions could be stochastic (involving randomness) or deterministic, depending on the environment and the agent.

Describing something as an agent is something we do as part of our framework in order to analyse it. There is no physical property that determines if an object is an agent or if it's not - anything can be an agent if we want it to be.

Environments

The environment includes everything that could affect our agent, and everything our agent can affect. Arguably this is always the entire universe though this is not very helpful and so reasonable simplifications should be made. The environment can also include the agent and its internal state, especially if our agent is able to read or modify its own state.

Environments will typically be multi-dimensional. If they represent real-world environments then there will be up to 4 dimensions, 3 for space and 1 for time. When we define an environment, we will also define somet...