Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Experimental longtermism: theory needs data, published by Jan Kulveit on March 15, 2022 on The Effective Altruism Forum.

This series explains my part in the EA response to COVID, my reasons for switching from AI alignment work to the COVID response for a full year, and some new ideas the experience gave me. While it is written from my (Jan's) personal perspective, I co-wrote the text with Gavin Leech, with input from many others.

The first post covers my main motivation: experimental longtermism.

Feedback loop

Possibly the main problem with longtermism and x-risk reduction is the weak and slow feedback loop.

(You work on AI safety; at some unknown time in the future, an existential catastrophe happens, or doesn’t happen, as a result of your work, or not as a result of your work.)Most longtermists and existential risk people openly admit that the area doesn't have good feedback loops. Still, I think the community at large underappreciates how epistemically tricky our situation is. Disciplines that lack feedback from reality are exactly the ones that can easily go astray.

But most longtermist work is based on models of how the world works - or doesn’t work. These models try to explain why such large risks are neglected, the ways institutions like government or academia are inadequate, how various biases influence public perception and decision making, how governments work during crises, and so on. Based on these models, we take further steps (e.g. writing posts like this, uncovering true statements in decision theory, founding organisations, working at AI labs, going into policy, or organising conferences where we explain to others why we believe the long-term future is important and x-risk is real).

Covid as opportunity

Claim: COVID presented an unusually clear opportunity to put some of our models and theory in touch with reality, thus getting more "experimental" data than is usually possible, while at the same time helping to deal with pandemic. The impact of the actions I mentioned above is often unclear even after many years, whereas in the case of COVID impact of similar actions was observable within weeks and months.For me personally, there was one more pull. My background is in physics, and in many ways, I still think like a physicist. Physics - in contrast to most of maths and philosophy - has the advantage of being able to put its models in touch with reality, and to use this signal as an important driver in finding out what's true. In modern maths, (basically) whatever is consistent is true, and a guiding principle for what's important to work on is a sense of beauty. To a large extent, the feedback signal in philosophy is what other philosophers think.

(Except when a philosophy turns into a political movement - then the signal comes from outcomes such as greater happiness, improved governance, large death tolls, etc.) In both maths and philosophy, the core computation mostly happens "in” humans. Physics has the advantage that in its experiments, "reality itself" does the computation for us.

I miss this feedback from reality in my x-risk work. Note that many of the concrete things longtermists do, like posting on the Alignment Forum or explaining things at conferences, actually do have feedback loops. But these are usually more like maths or philosophy: they provide social feedback, including intuitions about what kinds of research are valuable. One may wonder about the problems with these feedback loops, and what kind of blind-spots or biases they entail.

At the beginning of the COVID crisis, it seemed to me that some of our "longtermist" models were making fairly strong predictions about specific things that would fail - particularly about inadequate research support for executive decision-making. After some hesitation, I decided that if I trusted these mo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Dual use of artificial-intelligence-powered drug discovery, published by Matthew "Vaniver" Graves on March 15, 2022 on The AI Alignment Forum.

H/T Aella.

A company that made machine learning software for drug discovery, on hearing about the security concerns for these sorts of models, asked: "huh, I wonder how effective it would be?" and within 6 hours discovered not only one of the most potent known chemical warfare agents, but also a large number of candidates that the model thought was more deadly.

This is basically a real-world example of the "it just works to flip the sign of the utility function and turn a 'friend' into an 'enemy'"; this was slightly more complicated as they had two targets that they jointly optimized for the drug discovery process (toxicity and bioactivity), and only the toxicity target is flipped. [This makes sense--you'd want your chemical warfare agents to not be bioactive.] It also required a little bit of domain knowledge--they had to specify which sort of bioactivity to look for, and picked one that would point towards this specific agent.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ethicality Behind Breaking the Glass Ceiling, published by foxacc on March 14, 2022 on LessWrong.

I am a woman who works in finance.

When I originally studied finance I only went into it because I liked playing games with numbers and through some courses I had taken I recognized that there was a way to make money by solving the same math problems for other people. I was not aware of the ever-present and violent sexism within finance that I had come to realize after working in the industry.

Sexism was not thoroughly explained to me when I was young and thus when it had been occurring to me I was confused by it but shrugged it off as I was not educated on the area. When I had started to become more and more aware of the sexist things that people had been saying to me, or the opportunities that I had missed out on due to my sex, the nuances of the way society treats me, the way men speak to me, the way men view me, I realized that sexism had shaped the entirety of my life and every other woman's life and in turn, every man's life.

I innocently headed into finance to play my number games and I was met with egotistical men that put women on their pamphlets to be viewed as progressive to attract women to their firms just to talk down to them (feeding their egos further) and only provide lesser opportunities for them (so they can keep their high-paying deals to feed their egos further).

Recently I was a part of an investment fund as the only female on the team alongside 30 men. Naturally, the men told me I was in charge of attracting more females to the team to be more outwardly progressive and for a few months I ran events to advertise the fund to young females and talk about the benefits of being on the fund and the team was happy with my work. Then, I felt a pit in my stomach. A feeling of guilt. I had just encouraged many young women to join a team who talks over me, puts a lesser value to my opinion, and makes assumptions about my intelligence before I open my mouth. Their fate would certainly be the same if they decided to heed my advice. Did I just encourage these young women to join something that they will face copious amounts of sexism in?

It is no secret that women cannot hide from sexism no matter the scenario, however there are professions that are more heavily concentrated by females and I would imagine that their dealings with sexism are more minimal. This made me think about how ethical it is to encourage women to join adversity. I would much rather look at a younger version of myself and encourage her to join something she was passionate about that she would be treated equally in. No one wants to sign up for adversity.

Thus I have decided to no longer hide the dealings of sexism that I have endured when speaking to women about my career or the funds that I have been a part of. This may not make them eager to follow my path or join the groups that I am a part of however it is the only way I can feel that I have not lied to other women about the reality of sexism in finance. Despite what their marketing may advertise, no firm has or ever will be able to create a bubble to keep sexism out.

The experiences I have had made me question, why should I continue in finance, why not find an area that is not filled with sexism? The first I would be remiss to not recognize, things are improving. There are many things that I have not had to deal with that women in the past have fought for and succeeded. I am eternally grateful for these women. The second being future-looking, I want to make things easier for the young women to come who just like me, liked playing math games or even those who just want to make money. Also, for the women of colour who have even more adversity beyond sexism. Those are my only reasons to keep going and to keep being honest.

Thanks for listen...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why short-range forecasting can be useful for longtermism, published by Linch on March 14, 2022 on The Effective Altruism Forum.

Summary

See companion post here.

Most work on forecasting, including most EA work on forecasting, is on short-range forecasting (defined loosely here as timescales of ~1 week - ~3 years). Yet most of the motivation for why forecasting is valuable appeals to long-range forecasting, defined loosely as forecasting on the timescale of greater than 10 years, or much longer.

Here, I argue that advances in short-range forecasting (particularly in quality of predictions, number of hours invested, and the quality and decision-relevance of questions) can be robustly and significantly useful for existential risk reduction, even without directly improving our ability to forecast long-range outcomes, and without large step-change improvements to our current approaches to forecasting itself (as opposed to our pipelines for and ways of organizing forecasting efforts).

To do this, I propose the hypothetical example of a futuristic EA Early Warning Forecasting Center. The main intent is that, in the lead up to or early stages of potential major crises (particularly in bio and AI), EAs can potentially (a) have several weeks of lead time to divert our efforts to respond rapidly to such crises and (b) target those efforts effectively.

A hedge fund is a useful analogy here, in terms of the hypothetical set-up. In the same way that hedge funds are private entities that (among other activities) convert large amounts of both structured data and unstructured data into accurate-ish prices, we can have a “forecasting center” that converts various forms of data into accurate predictions on EA-relevant questions. The forecasts are directly fed to key EA decision-makers.

This is the basic idea. There are several variants of this general idea that also seem promising (though also potentially net-negative). Of these, my top 3 are: a) combining the forecasting with a civilian intelligence (OSINT) arm to generate data about issues of unusually high EA importance, b) combination with a crisis response unit to respond to pivotal moments directly and c) housing this forecasting center within a nation-state actor (perhaps the US government), to increase the chance that the forecasting center informs governmental rather than (just) EA rapid response decisions.

This post paints a specific vision for an organization that could be set up, and argues that we should aim towards it. Further work, from myself and others, would consider and prioritize a number of ways to aim towards this vision (including further research, starter projects or organizations to initiate, grants that are worth making, etc). Further work should also include a red-teaming of this vision, and other refinements.

Most of the points in this post will not seem original to people extremely acquainted with the EA forecasting space. Nonetheless, I thought it may be helpful to bring much of it in one place, as well as include my current opinionated best guesses and judgements.

Early Warning Forecasting Center: What it is, and why it’d be cool

The “hedge fund” metaphor can convey a lot of related concepts, but the definition/cluster of ideas I most want to invoke is “a large private entity that (relatively) accurately converts much of the information in the world into clear, precise prices on the financial market.” Similarly, I want us to consider having an Early Warning Forecasting Center as analogous to a hedge fund: a large entity that converts all sorts of data into accurate, calibrated, by-default-private, predictions on short-range questions of EA interest.

What outputs would an Early Warning Forecasting Center provide?

I think the most obvious outputs are accurate, calibrated, constantly updated, forecasts for ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Potentially great ways forecasting can improve the longterm future, published by Linch on March 14, 2022 on The Effective Altruism Forum.

Summary

See companion post here.

In addition to the EA Early Warning Forecasting Center I outlined in my other post, I think there are several ways forecasting may be very useful for longtermism, including:

Forecasting as a way to amplify EA research

Prediction-evaluation setups as a way to improve EA grantmaking

Large-scale broad forecasting as an EA outreach intervention

Large-scale forecasting tournaments as a talent training and vetting pipeline

The dream: high-quality, calibrated, long-range forecasting (ideally also at scale and on-demand)

Finally, an entirely different theory-of-change for forecasting is to consider broad, mass-appeal forecasting as a general epistemics intervention, that is, improving the thinking and reasoning quality of society at large. I think this is potentially pretty interesting and promising, and probably net positive, but am generally uncertain of the sign. This is mostly because I worry about squandering the epistemic edge that current broadly altruistic and EA(-adjacent) actors have over the rest of the world.

Further work, from myself and others, would consider and prioritize both among this list and within specific organizational choices in this list (including further research, starter projects or organizations to initiate, grants that are worth making, etc). Further work should also include a red-teaming of this vision, and other refinements.

Most of the points in this post will not seem original to people extremely acquainted with the EA forecasting space. Nonetheless, I thought it may be helpful to bring much of it in one place, as well as include my current opinionated best guesses and judgements.

Forecasting as a way to amplify EA research

There are a number of questions that come up during research for which I think forecasting could be one useful tool. Structural properties of such questions may include them being more amenable to broadly outside-view-style reasoning as opposed to deep internal models, being relatively “clean”, having a semi-objective resolution criteria, being more evaluative than generative (e.g., not questions that require coming up with new policy ideas), being about the future, etc.

Broadly, ways to use forecasting to improve EA research can be decomposed into two categories:

The researcher personally comes up with forecasting questions and forecasts questions that are relevant to her own work.

As much as possible, the researcher identifies forecastable questions in her own work and then delegates answering such questions to external forecasters, or forecasting aggregation processes.

It’s also plausible to me that some of the work in identifying and operationalizing the relevant forecasting questions can be delegatable as well.

There are a number of reasons for why researchers may prefer delegating subsets of their work to external forecasters or forecasting aggregation processes. The two most important to me are: a) for some subset of questions, top forecasters may be much better at getting the correct predictions to them than current EA researchers, and b) speaking loosely, forecaster time is usually less valuable than EA evaluation time (at least / especially after accounting for the fact that it’s probably easier to use money to buy extra forecaster time than to buy extra EA evaluation time).

I’m excited about more work in this general direction because it’s moderately impactful, highly tractable, and very easy to do initial experiments with, compared to other ideas on this list.

Some researchers at Rethink Priorities have been experimenting with a number of different ways to use forecasting to improve our research. For example, we partnered with Metaculus to h...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ELK contest submission: route understanding through the human ontology, published by Vika on March 14, 2022 on The AI Alignment Forum.

This is an ELK proposal that received a prize in the category "Train a reporter that is useful to an auxiliary AI". It was coauthored with Vikrant Varma and Ramana Kumar and advised by Rohin Shah.

Training strategy and why it might work

We use an architecture with a knowledge bottleneck to train the model, consisting of an Obs→(Q→A) function (the reporter) and a (Q→A)→Prediction function (the predictor). The key idea is that the model has to rely on the output of the reporter to make predictions. If the reporter implements the human simulator, then it loses more information about its world model than if it implemented the direct translator (which by assumption is a lossless projection of the world model onto the human ontology). As long as there are training situations which require a superhuman understanding (routed through the world model) of what's going on to make a good video prediction, the predictor querying a human simulator (which doesn't route through the world model) would get a worse loss, so the model is incentivized to learn the direct translator. This proposal addresses the ontology identification problem by restricting the model to using the human ontology for making its predictions.

Another way to think about this: the predictor is an encoder-decoder architecture, and we force the latent state to have a natural language QA interface. For example, it needs to work as an input (like the prompt, or a vector input) to a language model that does well on some (observation-conditioned) QA datasets. The decoder must also go through the QA interface to predict the second half of the video.

The entire encoder-decoder is trained using supervised learning. Within this outer SL loop, the decoder is trained using RL since it takes actions by interactively querying the reporter, using the same loss. Separately, the reporter is also regularized using SL on the human-generated dataset. This is meant to encourage the reporter's QA interface to converge to natural language.

This training strategy produces a system that does not take actions but contains an honest reporter, which could be used to search for action sequences leading to desirable outcomes, as described in the ELK report.

Counterexamples

The bottleneck can be insufficient: the model sneaks in human-incomprehensible information into the Q→A component (e.g. steganography) or the reporter learns the human simulator and the predictor asks basic questions like "what was the input".

Can be addressed by regularizing the reporter's output: penalizing response length or entropy and a GAN-type penalty for non-human-like questions and answers.

Using this architecture could sacrifice competitiveness, especially with a response length penalty.

The competitiveness cost may not be high if the reporter can learn a high-level summary of the environment that is representable in the Q→A format and is sufficient to make good predictions (e.g. "is the diamond in the room" → "yes").

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Twitter-length responses to 24 AI alignment arguments, published by RobBensinger on March 14, 2022 on The Effective Altruism Forum.

Cross-posted from Twitter, on request:

(Warning: arguments are Twitter-concise. Treat this all as a conversation-starter, rather than as anything authoritative or exhaustive.)

"Bad take" bingo cards are terrible, because they never actually say what's wrong with any of the arguments they're making fun of.

So here's the "bad AI alignment take bingo" meme that's been going around... but with actual responses to the "bad takes"!

I'll reply in a bit more detail below. This is still really cursory, but it's at least the sort of thing that could start a discussion, as opposed to back-and-forth rounds of mockery.

Here's the original meme:

1. "It sounds like scifi so it's not possible."

It's an error to infer 'X will happen' from 'X shows up in fiction'. But it's also an error to infer 'X won't happen' from 'X shows up in fiction'. Fiction just isn't relevant.

2. "Smarter AI will also be more moral."

Sufficiently smart AI would have more ability to be moral, by better modeling our values. It wouldn't thereby want to do moral things, unless we train it to want that. Safely training goals like that is a major unsolved technical problem.

See:/

3. "AI wouldn't want to kill us."

If we solve alignment, yes. Otherwise, optimizing almost any random objective hard enough will tend to imply killing potential threats ("you can't get the coffee if you're dead"); and humans are made of atoms an AI can use as raw materials.

See:/

4. "AI killing us is actually a good thing."

Quoting:

In addition to standard biases, I have personally observed what look like harmful modes of thinking specific to existential risks. The Spanish flu of 1918 killed 25-50 million people. World War II killed 60 million people. 108 is the order of the largest catastrophes in humanity’s written history. Substantially larger numbers, such as 500 million deaths, and especially qualitatively different scenarios such as the extinction of the entire human species, seem to trigger a different mode of thinking—enter into a “separate magisterium.” People who would never dream of hurting a child hear of an existential risk, and say, “Well, maybe the human species doesn’t really deserve to survive.”

There is a saying in heuristics and biases that people do not evaluate events, but descriptions of events—what is called non-extensional reasoning. The extension of humanity’s extinction includes the death of yourself, of your friends, of your family, of your loved ones, of your city, of your country, of your political fellows. Yet people who would take great offense at a proposal to wipe the country of Britain from the map, to kill every member of the Democratic Party in the U.S., to turn the city of Paris to glass—who would feel still greater horror on hearing the doctor say that their child had cancer— these people will discuss the extinction of humanity with perfect calm. “Extinction of humanity,” as words on paper, appears in fictional novels, or is discussed in philosophy books—it belongs to a different context than the Spanish flu. We evaluate descriptions of events, not extensions of events.

The cliché phrase end of the world invokes the magisterium of myth and dream, of prophecy and apocalypse, of novels and movies. The challenge of existential risks to rationality is that, the catastrophes being so huge, people snap into a different mode of thinking.

A million deaths is a million tragedies.

One reason a few people argue 'human extinction would be good' is Negative Utilitarianism, the theory that only suffering matters. But this theory is transparently false/silly.

(For a longer critique, see.)

5. "We shouldn't obstruct the evolution of intelligence."

If this means 'we shouldn't give up on ev...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Omicron #16: Danger in China, published by Zvi on March 14, 2022 on LessWrong.

What is happening in China?

Scott Gottlieb says we essentially don’t know the extent to which Omicron has spread in China.

What do we know?

We know China has not made great use of its time so far, and seems incapable of the loss of face necessary to get mRNA vaccines, nor does it seem to have stockpiled sufficient amounts of treatment for over a billion people if things get fully out of hand.

We know that China has now locked down tens of millions of people.

We know that they previously shut down schools in Shanghai, with some pretty strange implementation details.

And we know they have quite a few cases. We don’t know how many, but the official counts are certainly not overcounting cases.

“Because of the large number of cases in a short period of time, it is inevitable that there will be some panic all over the country, and Shanghai is no exception,” said Dr. Zhang Wenhong, a prominent infectious disease expert in Shanghai, in a post on his social media account on Monday.

We also know from Hong Kong what it could look like if a population that only had access to Sinovac and has had few previous infections has uncontrolled spread of Omicron. Things get very bad very quickly.

The thing I noticed right away about China’s reaction this time in Shanghai, which came first, was that it wasn’t as complete. The will to succeed seemed not to be there on the same level.

Yes, this was a series of coercive actions the West would be incapable of taking, but what reason was there to think they would work? Closing off all school-based transmission won’t slow down Omicron much. Closing schools is a half-measure. If you have reason to think schools need to be closed, and you are following China’s old playbook, than anything that makes you need to shut down Shanghai’s schools should make you shut down all of Shanghai.

In other cases, China has made this extremely expensive and painful choice, and it has worked. This time, faced with a more dangerous variant, they only closed schools and hoped the problem would go away. That is not going to work (not that it has had time to have an impact yet, but it was never going to work).

Now they’ve ‘restricted movement in many neighborhoods’ of Shanghai, but not in others, and this is still very different from the nature of previous Chinese lockdowns.

And consider what they are doing in Dongguan, which is perhaps too big to shut down in the same way as Shanghai:

City authorities told residents not to leave the city, except for essential reasons. Those leaving must show negative test results within 48 hours of departure.

A few entrances on highways to other cities were closed, while all shuttle buses linking airports in other cities and check-in terminals in Dongguan were halted. Some museums and libraries in the city also closed to visitors.

Its factories are still running, however.

“(Workers) need to do COVID tests, but it’s not a prerequisite for them to be able to enter factories,” said King Lau, who helps manage a metal coating factory.

That is not going to be enough. Some cases will get out, and cases within the city will still rise, unless this was caught so early there really were only a handful to start and then they got super lucky.

They still were willing to shut down Shenzen and a total of about 60 million people so far, while showing signs they are approaching their limits.

So what happens now?

If there are this many cases already, the question is whether the outbreak is somehow contained to the cities and areas that are now locked down, or otherwise the tide can somehow be turned. I do not see how this happens.

China could ramp up its reactions, locking down more cities and areas with harsher conditions, in an attempt to make it stop, but can it afford to ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Followup on Terminator, published by skluug on March 12, 2022 on The Effective Altruism Forum.

I posted my last post to the Effective Altruism Forum, where it received more attention than I’d anticipated. This was cool, but also a little scary, since I’m not actually super confident in my thesis—I think I made an under-appreciated argument, but there was plenty of valid pushback in the comments.

My post was missing an important piece of context—the other, alternative comparisons for AI risk I was implicitly arguing against. I had something very specific in mind: on The Weeds, after Kelsey Piper says Terminator is not a good comparison for AI risk, she says that a better comparison is Philip Morris, the cigarette company. Now, there are a lot different threats that fall under the umbrella of “AI risk”, but the threats I am most worried about—existential threats—definitely look a lot more like Terminator than Philip Morris. This could just be a difference in priorities, but Kelsey says other things throughout the episode that made it sound to me like she’s indeed referring to x-risk from unaligned AGI.

Here are some specific critiques made by commenters that I liked.

What’s the audience? What’s the message?

My paragraph calling rejection of the Terminator comparison “fundamentally dishonest” was too idealistic. Whether or not something is “like” something else is highly subjective, and pitches about AI x-risk often happen in a context prior to any kind of patient, exhaustive discussion where you can expect your full meaning to come across. This was pointed out by Mauricio:

In a good faith discussion, one should be primarily concerned with whether or not their message is true, not what effect it will have on their audience.

Agreed, although I might be much less optimistic about how often this applies. Lots of communication comes before good faith discussion--lots of messages reach busy people who have to quickly decide whether your ideas are even worth engaging with in good faith. And if your ideas are presented in ways that look silly, many potential allies won't have the time or interest to consider your arguments. This seems especially relevant in this context because there's an uphill battle to fight--lots of ML engineers and tech policy folks are already skeptical of these concerns.

(That doesn't mean communication should be false--there's much room to improve a true message's effects by just improving how it's framed. In this case, given that there's both similarities and differences between a field's concerns and sci-fi movie's concerns, emphasizing the differences might make sense.)

(On top of the objections you mentioned, I think another reason why it's risky to emphasize similarities to a movie is that people might think you're worried about stuff because you saw it in a sci-fi movie.)

I replied that the Terminator comparison could decrease interest for some audiences and increase interest for others, to which Mauricio replied:

That seems roughly right. On how this might depend on the audience, my intuition is that professional ML engineers and policy folks tend to be the first kind of people you mention (since their jobs select for and demand more grounded/pragmatic interests). So, yes, there are considerations pushing for either side, but it's not symmetrical--the more compelling path for communicating with these important audiences is probably heavily in the direction of "no, not like Terminator."

Edit: So the post title's encouragement to "stop saying it's not" seems overly broad.

I think there are two cruxes here, one about the intended audience of AI alignment messaging, and one about the intended reaction.

As I see it, the difficulty of the alignment problem is still very unclear, and may fall into any of several different difficulty tiers:

The alignmen...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New GPT3 Impressive Capabilities - InstructGPT3 [1/2], published by simeon c on March 13, 2022 on The Effective Altruism Forum.

Summary

InstructGPT3 (hereafter IGPT3), a better version of GPT3 has recently been released by OpenAI. This post explores its new capabilities.

IGPT3 has new impressive capabilities and many potential uses. Among others, it can help users:

Brainstorm

Summarize the main claims made by a scientific field, an author or a school of thought.

Find an analogy or a metaphor for something hard to explain

I emphasize some of IGPT3's limits, especially situations where It provides very plausible fake answers.

Twitter more entertaining version of the summary:

Introduction

Epistemic status: I spent about ~12h with IGPT3. So I’d say that I now have a pretty good sense of some of its key features. I tried several examples to ensure that I was not overfitting on a single example for the most important claims I made. That said, this is a huge model so there is probably a lot more to be discovered. FYI I had spent a decent amount of time playing with the past GPT3, especially with the Davinci (175b params) and the Curie (6b params) models, so I had a clear idea of "what it is like to try to get nice completions from GPT3". That may be one reason why I’m so amazed by this one.

Here's the first post of a series of 2 blog posts exploring some of the IGPT3 (I) new capabilities and (II) epistemic biases.

This first post will focus on some interesting uses I had of IGPT3. It also gives a sense of how good it is in various domains. Let me tell you: I'm amazed by its new capabilities. I find it really impressive that most of the time, the first result I get, without any tuning (either of the parameters or of the prompt) is great. You can try it yourself here.

The blogpost is organized as follow:

A few general observations

2 mains parts (Examples of Potential Uses / Limits)

The last part entitled “Many more prompts than you wanted to read” where I put robustness checks, I test how sensitive IGPT3 is to unique words variations, I compare IGPT3 and GPT3 and I show how you can have fun with IGPT3.

Acknowledgment: Thanks to JS and Florent Berther for the proofreading, to Ozzie Gooen for the suggestion to make a post out of my comments on his post and to Charbel-Raphaël Segerie for some nice ideas.

General Features

Here are some general features of IGPT3:

Compared to GPT3, you need to spend much less time prompt-tuning on IGPT3. You just need to be clear enough.

When the temperature (a parameter to control the randomness of the completion) is greater than 0, you need to try less than 2 completions to find a satisfactory answer when IGPT3 has one. And most of the time, a single completion is enough. A temperature of 0 also works very well, so I personally use that for most uses.

IGPT3 now knows when to stop so you can put a huge maximum limit of tokens and he will generally only use a small part of it to answer your question. Given this new feature, you just have to ask him if you want something specific. Here are some examples:

If you want many suggestions, you can ask for it explicitly: "Give me the five best arguments".

If you want something more specific, you can explicitly ask for it: "I don't understand X, can you elaborate ?"

Examples of Potential Uses

Brainstorming

IGPT3 is very useful to brainstorm. I personally use it more and more because it enables me to quickly generate a lot of ideas on anything I want to think about.

Project Names

IGPT3 is useful to sometimes suggest associations of concepts you hadn’t thought of. That way, it can help find good names.

Differences and Similarities between Concepts

Rapidly Accessing Information

I use IGPT3 more and more to make sure that I didn’t miss a big argument on a topic because IGPT3 is very good to tell the most common...