Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Literature review of Transformative Artificial Intelligence timelines, published by Jaime Sevilla on January 27, 2023 on The Effective Altruism Forum.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Simple Alignment Typology, published by Shoshannah Tekofsky on January 28, 2023 on LessWrong.

I set out to review the OpenAI alignment plan, and my brain at some point diverged to modeling the humans behind the arguments instead of the actual arguments.

So behold! A simplified, first-pass Alignment Typology.

Why can't we all just get agree?

There are a lot of disagreements in AI alignment. Some people don't see the problem, some think we'll be fine, some think we're doomed, and then different clusters of people have different ideas on how we should go about solving alignment. Thus I tried to sketch out my understanding of the key differences between the largest clusters of views on AI alignment. What emerged are roughly five cluster, sorted in order of optimism about the fate of humanity: the sceptics, the humanists, the empiricists, the rationalists, and the fatalists.

Sceptics don't expect AGI to show up in any relevant time frame.

Humanists think humanity will prevail fairly easily through coordination around alignment or just solving the problem directly.

Empiricists think the problem is hard, AGI will show up soon, and if we want to have any hope of solving it, then we need to iterate and take some necessary risk by making progress in capabilities while we go.

Rationalists think the problem is hard, AGI will show up soon, and we need to figure out as much as we can before making any capabilities progress.

Fatalists think we are doomed and we shouldn't even try (though some are quite happy about it).

Here is a table.

ScepticsHumanistsEmpiricistsTheoristsFatalistsAlignment Difficulty-highhigh-Coordination Difficulty-highhigh-Distance to AGIhigh-low/medlow/med---highmed/high---med/highhigh--highhighhighlow

One of these

is low

Closeness to AGI required

to Solve Alignment

Closeness to AGI resulting

in unacceptable danger

Alignment Necessary

or Possible

Less Wrong is mostly populated by empiricists and rationalists. They agree alignment is a problem that can and should be solved. The key disagreement is on the methodology. While empiricists lean more heavily on gathering data and iterating solutions, rationalists lean more heavily toward discovering theories and proofs to lower risk from AGI (and some people are a mix of the two). Just by shifting the weights of risk/reward on iteration and moving forward, you get two opposite approaches to doing alignment work.

How is this useful?

Personally it helps me quickly get an idea of what clusters people are in, and understanding the likely arguments for their conclusions. However, a counterargument can be made that this just feeds into stereotyping and creating schisms, and I can't be sure that's untrue.

What do you think?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Spooky action at a distance in the loss landscape, published by Jesse Hoogland on January 28, 2023 on The AI Alignment Forum.

Produced as part of the SERI ML Alignment Theory Scholars Program Winter 2022 Cohort.

Not all global minima of the (training) loss landscape are created equal.

Even if they achieve equal performance on the training set, different solutions can perform very differently on the test set or out-of-distribution. So why is it that we typically find "simple" solutions that generalize well?

In a previous post, I argued that the answer is "singularities" — minimum loss points with ill-defined tangents. It's the "nastiest" singularities that have the most outsized effect on learning and generalization in the limit of large data. These act as implicit regularizers that lower the effective dimensionality of the model.

Even after writing this introduction to "singular learning theory", I still find this claim weird and counterintuitive. How is it that the local geometry of a few isolated points determines the global expected behavior over all learning machines on the loss landscape? What explains the "spooky action at a distance" of singularities in the loss landscape?

Today, I'd like to share my best efforts at the hand-waving physics-y intuition behind this claim.

It boils down to this: singularities translate random motion at the bottom of loss basins into search for generalization.

Random walks on the minimum-loss sets

Let's first look at the limit in which you've trained so long that we can treat the model as restricted to a set of fixed minimum loss points.

Here's the intuition pump: suppose you are a random walker living on some curve that has singularities (self-intersections, cusps, and the like). Every timestep, you take a step of a uniform length in a random available direction. Then, singularities act as a kind of "trap." If you're close to a singularity, you're more likely to take a step towards (and over) the singularity than to take a step away from the singularity.

It's not quite an attractor (we're in a stochastic setting, where you can and will still break away every so often), but it's sticky enough that the "biggest" singularity will dominate your stable distribution.

In the discrete case, this is just the well-known phenomenon of high-degree nodes dominating most of expected behavior of your graph. In business, it's behind the reason that Google exists. In social networks, it's similar to how your average friend has more friends than you do.

To see this, consider a simple toy example: take two polygons and let them intersect at a single point. Next, let a random walker run loose on this setup. How frequently will the random walker cross each point?

If you've taken a course in graph theory, you may remember that the equilibrium distribution weights nodes in proportion to their degrees. For two intersecting lines, the intersection is twice as likely as the other points. For three intersecting lines, it's three times as likely, and so on.

Now just take the limit of infinitely large polygons/step size to zero, and we'll recover the continuous case we were originally interested in.

Brownian motion near the minimum-loss set

Well, not quite. You see, restricting ourselves to motion along the minimum-loss points is unrealistic. We're more interested in messy reality, where we're allowed some freedom to bounce around the bottoms of loss basins.

This time around, the key intuition-pumping assumption is to view the behavior of stochastic gradient descent late in training as a kind of Brownian motion. When we've reached a low training-loss solution, variability between batches is a source of randomness that no longer substantially improves loss but just jiggles us between solutions that are equivalent from the perspective of the training set.

To u...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Native English speaker EAs: could you please speak slower?, published by Luca Parodi on January 27, 2023 on The Effective Altruism Forum.

Epistemic status: personal opinion, based on anecdotical evidences and my gut feelings as an expat among native speaker EAs

A large majority of EAs are native English speakers. Additionally, there is a significant portion of EAs from countries, such as Northern European countries, where English is well-taught or serves as a second language, such as India. However, there is also a small but significant minority of individuals like myself. In my country (Italy), the education system has neglected the teaching of English, particularly for those from lower-class backgrounds and who attended public schools of questionable quality. As a result, I had to teach myself English from scratch at the age of 22, and according to what I've been told, I have achieved decent results.

In the past year, I spent half of my time in London, primarily interacting with other EAs. I have noticed that native English speakers often pay little attention to the varying levels of language proficiency, speaking extremely quickly about already complex topics, and frequently using metaphors, analogies, cultural references, and technical terms. This is not something that occurs when I communicate with other non-native or expat individuals. And it is frustrating.

Those who know me personally are aware that in my native language, I am a highly confident and fast speaker who probably talks too much, especially considering my job involves public and social media outreach about rationality. However, when I have to interact with EAs in real life, I sometimes feel stupid and become shy (which is unusual for someone who is a 95th percentile extrovert on the Big Five scale). I often just nod as if I understand what is being said because I fear that by asking "Can you repeat that, please?" I will be perceived as stupid and slow in a community that values time, effectiveness, high-value actions, and reason. I understand that this is mainly my own issue, and I am working on improving my language skills, but I think that something might be done on the other side too.

So, do we want to be more inclusive? Let's start with the little things, such as our day-to-day interactions. Here are some tips, based on my own experience, that I can give you if you are a native speaker and are interacting with a non-native speaker:

Try to be mindful and slow down the pace of your speaking

Avoid using too many metaphors, analogies and extremely technical words when they are not needed

Be aware and try to control the voice inside of your head (which I am highly confident is there even if you don't want to admit it) that says "ugh, this person who clearly isn't understanding me seems slow and stupid. I don't want to waste my time with them"

Reduce references to your country's politics, pop culture, cultural conversations and inside jokes to a bare minimum.

Don't say "you're doing great! You're English is super good man" if I tell you that I am struggling with the language and if you don't mean it. Actually, don't say it at all. Even if it comes from a genuine and well-intended instinct, it might sound extremely condescending, like the teacher who says to a struggling student's parent "he's so sweet. I know he will do great things!"

Instead, try to actually help me by applying these insights

The list is obviously incomplete, so any additions or corrections would be much appreciated.

Do we really need to use IT jargon every two sentences to express easy concepts?

Source: more than a couple of occasions in which I was super excited by the conversation but I was struggling to understand and the other person stopped talking with me.

I don't know (and, really, I don't want or need to know) about your po...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Pineapple now lists marketing, comms & fundraising talent, fiscal sponsorship recs, private database (Jan '23 Update), published by Vaidehi Agarwalla on January 27, 2023 on The Effective Altruism Forum.

Marketing, PR & comms skills lists now live (in addition to Ops, PA & ExA work)

Candidates can now explicitly list themselves as having marketing, comms & fundraising roles. As we previously discussed, these kinds of skills are often swept under the operations umbrella at many EA-aligned orgs, especially smaller ones.

We soft-launched the option in our sign-up form during Dec '22, and have already had ~12 people be open to these roles on their profile.

You can view candidates by role on our roles page here. If you're an existing candidate who'd like to add those skills to your profile, email us at [email protected].

We now list fiscal sponsorship resources

Fiscal sponsorship and operations support are a common pain points of early stage projects, so we've shared some recommendations from operations staff at other EA organizations. The Pineapple Ops team has not personally vetted or thoroughly reviewed this list. Please do research any fiscal sponsor you wish to work with.

Why are we sharing these resources? Knowledge sharing of operations in EA has a lot of room for improvement. Since our team doesn't have the capacity to do resource compilations ourselves with a concerted effort, we wanted to try collaborating with other folks to create lists.

Our progress to date (what we think we've done well on)

Progress on placements (our main metric for impact)

We've been up for 3 months, and in that time we've placed at least 9 people in full- or part-time roles, with at least 4 more getting strong leads and/or making it to late stage of the applications process that we know of. We currently list over 180 people candidates.

Rough cost per placement: ~$420 USD (We expect cost per placement to drop as this estimate includes project set-up time)

We've received a lot of positive feedback from employers and candidates alike - even those who haven't yet been placed directly from the board.

Moving fast & experimenting

I am happy with the pace at which we've moved as a part-time project. I think we've found new areas to explore which aren't too ambitious

We are currently experimenting with another, smaller project that we hope will result in more candidate placements, and currently troubleshooting demand-supply issues. (We plan to discuss this project in more depth in future updates).

Areas of improvement

During our informal internal review, I (Vaidehi) felt the main area for improvement was to spend a little bit more time advertising the database to candidates and employers more. We did an initial sprint during November but this died down over Dec & Jan. We want to make sure that folks know about Pineapple Operations and its offered resources. We are currently evaluating capacity to do more advertising, but tentatively want to hit a target of putting out reminders, content or resources every 3-6 weeks. So far, we don't think we've made any major mistakes in the course of running this project.

Additionally, although initial budgeting for the project was correct, we've identified more worthwhile opportunities than we expected to spend our initial funding on and will likely need to reduce time spent on the project or fundraise.

As an unpaid volunteer, I (Vaidehi) have not been as strict on time tracking my hours as I would like to be, so I expect we may be underestimated my time input (and possible counterfactual of where else I could spend my time).

Get Involved & Support us

Sign up to be listed on the database

Let us know if you've been hired or made it to later stages of a job application (or found a good candidate!) because of our job board

We are seeking funding (~$10K) t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Assigning Praise and Blame: Decoupling Epistemology and Decision Theory, published by adamShimi on January 27, 2023 on LessWrong.

Your group/company/organization performs well, doing great work and dealing with new problems efficiently. As one of its leaders, you want to understand why, so that you can make it even more successful, and maybe emulate this success in other settings.

Your group/company/organization performs badly, not delivering on what was promised and missing deadline after deadline. As one of its leaders, you want to understand why, so that you can correct its course, or at least not repeat the same mistakes in other settings.

Both cases apparently involve credit assignment: positive credit (praise) for success or negative credit (blame) for failure. And you can easily think of different ways to do so:

Heuristics for Credit Assignment

Baseline

The most straightforward approach starts with your initial prediction, and then assigns credit for deviations from it. So praise people who did better than expected and blame people who did worse than expected.

Then you remember Janice. She’s your star performer, amazing in everything she does, and you knew it from the start. So she performed according to prediction, being brilliant and reliable. Which means she doesn’t deserve any praise by this criterion.

On the other hand there is Tom. He’s quite good, but you also knew from the start he was a prickly showoff with an easily scratched ego. Still, he did his job, and when he acted like an asshole, that was within the prediction. So he doesn't deserve any blame by this criterion.

Incentive wise, this sounds like a terrible idea. If you push this credit assignment strategy, not only will you neglect the value of Janice and the cost of Tom, but you will probably drive away high-performers and attract problem-makers.

Bottleneck

Instead of starting from a baseline, let’s focus on the key bottlenecks. What would have doomed the project if not done? What ended up blocking everything and dooming the project? This focuses on the real cruxes, which is good.

Yet what about Marcel, your tireless tool engineer? None of his stuff is ever absolutely necessary, but everyone in your group constantly mentions the value they get from his well-maintained and efficient tools. Should he not get any credit for it?

And Bertha, who is the only security expert of your group, and always finds excuses to make herself more and more necessary? Is this really a behavior you want to condone and praise? Shouldn’t she get blamed for it instead?

It’s at this point that you remember this short parable.

First cause

No, really, what matters most is the initial spark that puts everything in motion. Without the original idea, nothing happens; skills and expertise remain useless and flaccid, unable to toil for a worthwhile goal.

And what a coincidence: you were one of these key idea generators! All the more power to you, then. It’s only fair that you get a large share of the credit, given that the group wouldn’t even exist without you.

But that still doesn’t seem right. Yes, your contribution was integral. But could you really have done it by yourself? Probably not. Or at least not as well, as quickly, as beautifully as it has been.

Or conversely, if it failed totally, was it because the idea was doomed from the start, or because the execution proved bad enough to torpedo even sensible propositions?

Final step

You got it exactly wrong above: it’s not the first step that trumps them all, it’s the final one. Making the abstract real, adding the finishing touches, this is what makes success or failure. So you should focus your assignment on the success and failure of these last steps.

But what about you? Yes, you. You have never finished anything yourself, you’re the organizer, idea maker, coordinator. ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Summit on Existential Security 2023, published by Amy Labenz on January 27, 2023 on The Effective Altruism Forum.

The CEA events team wants to be more transparent about what we’re doing, to allow feedback and visibility. We’re not committing to publishing detailed information about every event, but in that spirit, we wanted to share something about an event we’re running soon — the Summit on Existential Security — even though it’s an invite-only event and we don’t expect the information in this post to be directly useful to many people.

The Summit on Existential Security will take place the week following EA Global: Bay Area 2023.

The summit is aimed at professionals working directly or indirectly (for example, through grantmaking) towards existential security (i.e. addressing existential risks). The event aims to help participants orient themselves to the complex and developing existential risk situation so that we have the best shot at identifying and pursuing important ways to reduce the risk of existential catastrophe.

In particular, the summit aims to facilitate conversations that make progress on crucial topics. We’re especially interested in topics that are important but aren’t fully within the remit of any research group (perhaps because they’re big-picture, cross-cutting, new, or not purely academic). We hope these might lead people to better understand something important, change their minds, make new plans, or improve their strategies. Near the end of the summit, there will be sessions designed to encourage attendees to get into the details of making specific plans. But we don’t need every conversation to have immediate decision relevance.

A secondary goal of the summit is to build the professional community of people working on existential security. We expect the event to be especially valuable for people who are less plugged in to existing networks to be able to make useful connections. Past impact evaluations for the Coordination Forum and similar events indicate that often the people who got the most out of these events are those who were relatively less plugged in to the community core prior to the event.

The summit will have approximately 160 attendees. The event size was chosen because we wanted to experiment with whether we could get the benefits attendees report from the Coordination Forum and similar retreat-style events at a larger scale (the summit will be about 3 times larger than the Coordination Forum was last year). Compared to EA Global, the summit will be much smaller (10 times smaller) and is residential. This means the summit can be focused on collaborative group sessions which try to make progress on important topics related to reducing existential risk and on facilitating and strengthening connections between people working on existential security.

The summit is an invite-only event. Invitations to the event were assembled via two mechanisms:

A convening committee of eight community advisors selected some people to invite based on their work and how helpful we thought they could be to other attendees.

We reached out to about 25 people to ask for recommendations for people in their networks who might be less well-known to other parts of the community.

If the summit proves to be successful, we may consider expanding this method in the future.

It’s important to note that this event is not an attempt to collect individuals who are doing the highest-value work on existential security, but rather a mix of individuals working on object-level and meta-level issues, in order to facilitate important connections between these groups.

The summit is being organized by the Partner Events team, a new program within the CEA Events team that supports subject matter experts to run high-impact events. The Partner Events team is working on a variety of e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The role of Bayesian ML in AI safety - an overview, published by Marius Hobbhahn on January 27, 2023 on The AI Alignment Forum.

I want to thank Sebastian Farquhar, Laurence Midgley and Johan van den Heuvel, for feedback and discussion on this post.

Some time ago I asked the question “What is the role of Bayesian ML in AI safety/alignment?”. The response of the EA and Bayesian ML community was very helpful. Thus, I decided to collect and distill the answers and provide more context for current and future AI safety researchers.

Clarification: I don’t think many people (<1% of the alignment community) should work on Bayesian ML or that it is even the most promising path to alignment. I just want to provide a perspective and give an overview. I personally am not that bullish on Bayesian ML anymore (see shortcomings) but I’m in a relatively unique position where I have a decent overview of AI safety and the Bayesian ML literature and think an overview post like this might be helpful.

A working definition

There is no agreed-upon definition for Bayesian ML. I use the term for systems that broadly have any of the following properties

Implicitly or explicitly use Bayes theorem.

Approximate and quantify uncertainty for their estimates, e.g. return distributions instead of point estimates, and allow for the specification of prior distributions.

Systems that have a latent state that can be continuously updated with new data without being fully retrained, e.g. conjugate inference. This is the vaguest property since it plausibly also applies to pre-training and fine-tuning LLMs which are usually not seen as explicitly Bayesian algorithms.

Roles

High-level - Future AI systems might be Bayesian

This section is largely inspired by a response from Emtiyaz Khan and a different response from Sebastian Farquhar.

There are a lot of things that current ML systems do poorly in comparison to humans. They are often not as data-efficient as humans are, they don’t generalize well, they are often not robust to adversarial inputs, they often can’t learn during deployment and much more (none of these properties are strictly necessary for a system to be catastrophically dangerous; so their absence is not a guarantee for safety). However, many of these properties would likely exist in a system that could be called AGI.

The Bayesian framework provides some answers to these problems. Bayesian methods are often more data efficient, they can be easily updated through Bayes theorem, they are sometimes more robust to adversarial inputs (see here or here) and much more. In practice, it is often hard to build Bayesian methods that fulfill all of these properties, but in theory, they should exist.

Therefore, while current Bayesian systems often underperform compared to their non-Bayesian counterparts, we might have to turn to Bayesian systems in the future if we want to have agents with all of these properties. In this case, the Bayesian framing is a bet on the future trajectory of ML rather than a statement about current AI systems.

Some people within the Bayesian ML community have stated this view in the past and work primarily on Bayesian ML. Emtiyaz Khan, for example, works on the Bayes-duality project which aims to “develop a new learning paradigm for Artificial Intelligence that learns like humans in an adaptive, robust, and continuous fashion”. Andrew Gordon Wilson is one of the leading researchers in Bayesian ML and much of his work is inspired by the problems of current ML systems I described above.

I personally think that these problems are real and important but I’m not sure that the answer to them has to be Bayesian or Bayesian in the way we expect. For example, I could imagine, that an RL agent might become Bayesian after sufficiently long training without any explicit Bayesian inductive bias or...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ¿Por qué escribir en español? (Why should we write in Spanish: a follow-up of EAGx Latam), published by Simon Ruiz-Martinez on January 26, 2023 on The Effective Altruism Forum.

La realización del EAGx Latam en el pasado mes de enero es el resultado de una comunidad que ha ido en paulatino aumento y que ha alcanzado la madurez suficiente para hacerse cargo de una discusión comunitaria de alto nivel, donde el desarrollo y discusión del Altruismo Eficaz permite avanzar en la co-creación de maneras de ponderar lo mejor que podemos hacer.

El interés que logré evidenciar entre los asistentes sobre el contexto específico de América Latina –mayoritariamente en encuentros informales cabe aclarar– muestra las potencialidades de una comunidad comprometida, y cuya visibilidad se hace cada vez más importante en los diferentes espacios que ofrece la comunidad global.

La identidad del diverso conglomerado de profesionales, estudiantes y académicos que compone la comunidad hispanohablante está atravesada –aunque no limitada por– el español. De ahí la pregunta planteada en el título de esta publicación: ¿Por qué escribir en español? A ella se asocia otra pregunta: ¿por qué hay menos de cinco publicaciones en español en el foro? Invito a quien lee esta publicación a que responda esta segunda pregunta. Intentaré, en lo que sigue, concentrarme en la primera.

Pero antes de proseguir con ello, una salvedad.

¿A quién va dirigido este post?

Escribir en una lengua diferente a la institucionalizada por una comunidad puede parecer un acto de exclusión. Espero que los angloparlantes y demás usuarios de las diferentes lenguas que componen el diverso conglomerado de EA en la actualidad disculpen esa falta de deferencia de mi parte. No se trata de excluir a quienes hablan otros idiomas. Este post tiene como objetivo fundamental eliminar las barreras lingüísticas para la difusión científica; específicamente, al animar a la difusión de ideas en otros idiomas; en especial, para una lengua que cuenta con alrededor de 460 millones de personas.

Es importante desestimar la presión por el nivel de muchos de nuestros colegas que han publicado. No creo que sea muy controvertido afirmar que no solo es importante discutir sobre temáticas de alto impacto, sino también en cuestiones de construcción de comunidad que pueden tener como auditorio un público que viene iniciando y se beneficiaría mucho de las recomendaciones de una experiencia semejante.

Así, cuestiones como creación de grupos universitarios, grupos de lectura, redes de profesionales, etc., todas son actividades que pueden tener un lector interesado que acabe de ingresar a la comunidad y que busque una publicación sobre experiencias sistematizadas que le prevengan de cometer posibles reprocesos en el camino.

¿Cuál es realmente la invitación?

Si bien es una tarea fundamental la traducción de conceptos e ideas (frecuentemente me pasa que termino usando términos en inglés al hablar con colegas hispanohablantes), la invitación va, concretamente, a difundir la perspectiva específica de nuestras latitudes: fomentar la producción de contenido original. El apoyo comunitario realizado a partir de la publicación de experiencias significativas puede perfectamente complementarse con publicaciones donde se indaguen las particularidades que han implicado las ideas de EA en el contexto latinoamericano.

Difundir los intereses profesionales y académicos de quienes trabajamos en estas áreas permite, además, crear un acervo del cuál se puedan derivar formas distintas de priorización (especialmente para LMICs), estudios comparados (e.g. en la efectividad de políticas y transferibilidad del conocimiento), y en colaboración científica y esfuerzos mancomunados por temas y opciones de carrera.

Este propósito específico complementa también una comunidad que ha crecido en otr...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: All AGI Safety questions welcome (especially basic ones) [~monthly thread], published by mwatkins on January 26, 2023 on LessWrong.

tl;dr: Ask questions about AGI Safety as comments on this post, including ones you might otherwise worry seem dumb!

Asking beginner-level questions can be intimidating, but everyone starts out not knowing anything. If we want more people in the world who understand AGI safety, we need a place where it's accepted and encouraged to ask about the basics.

We'll be putting up monthly FAQ posts as a safe space for people to ask all the possibly-dumb questions that may have been bothering them about the whole AGI Safety discussion, but which until now they didn't feel able to ask.

It's okay to ask uninformed questions, and not worry about having done a careful search before asking.

AISafety.info - Interactive FAQ

Additionally, this will serve as a way to spread the project Rob Miles' volunteer team has been working on: Stampy and his professional-looking face aisafety.info. Once we've got considerably more content this will provide a single point of access into AI Safety, in the form of a comprehensive interactive FAQ with lots of links to the ecosystem. We'll be using questions and answers from this thread for Stampy (under these copyright rules), so please only post if you're okay with that! You can help by adding other people's questions and answers or getting involved in other ways!

We're not at the "send this to all your friends" stage yet, we're just ready to onboard a bunch of editors who will help us get to that stage :)

We welcome feedback and questions on the UI/UX, policies, etc. around Stampy, as well as pull requests to his codebase. You are encouraged to add other people's answers from this thread to Stampy if you think they're good, and collaboratively improve the content that's already on our wiki.

We've got a lot more to write before he's ready for prime time, but we think Stampy can become an excellent resource for everyone from skeptical newcomers, through people who want to learn more, right up to people who are convinced and want to know how they can best help with their skillsets.

Guidelines for Questioners:

No previous knowledge of AGI safety is required. If you want to watch a few of the Rob Miles videos, read either the WaitButWhy posts, or the The Most Important Century summary from OpenPhil's co-CEO first that's great, but it's not a prerequisite to ask a question.

Similarly, you do not need to try to find the answer yourself before asking a question (but if you want to test Stampy's in-browser tensorflow semantic search that might get you an answer quicker!).

Also feel free to ask questions that you're pretty sure you know the answer to, but where you'd like to hear how others would answer the question.

One question per comment if possible (though if you have a set of closely related questions that you want to ask all together that's ok).

If you have your own response to your own question, put that response as a reply to your original question rather than including it in the question itself.

Remember, if something is confusing to you, then it's probably confusing to other people as well. If you ask a question and someone gives a good response, then you are likely doing lots of other people a favor!

Guidelines for Answerers:

Linking to the relevant canonical answer on Stampy is a great way to help people with minimal effort! Improving that answer means that everyone going forward will have a better experience!

This is a safe space for people to ask stupid questions, so be kind!

If this post works as intended then it will produce many answers for Stampy's FAQ. It may be worth keeping this in mind as you write your answer. For example, in some cases it might be worth giving a slightly longer / more expansive / more deta...