Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Inflection.ai is a major AGI lab, published by nikola on August 9, 2023 on LessWrong.

Inflection.ai (co-founded by DeepMind co-founder Mustafa Suleyman) should be perceived as a frontier LLM lab of similar magnitude as Meta, OpenAI, DeepMind, and Anthropic based on their compute, valuation, current model capabilities, and plans to train frontier models. Compared to the other labs, Inflection seems to put less effort into AI safety.

Thanks to Laker Newhouse for discussion and feedback!

Inflection has a lot of compute dedicated to training LLMs

They plan to scale up their cluster to 3 times the capacity used to train GPT-4.

"We'll be building a cluster of around 22,000 H100s. This is approximately three times more compute than what was used to train all of GPT4. Speed and scale are what's going to really enable us to build a differentiated product,"

"We believe in scale as the engine of progress in AI, and we are building one of the largest supercomputers in the world to develop and deploy the new generation of AIs."

They can apparently train a model similarly capable to GPT-2 in 11 minutes of cluster time. (see Appendix)

Side point: It seems that the actual H100s are (at least partly) owned by CoreWeave (a cloud compute provider), but that Inflection is one of CoreWeave's main clients. The specific cluster is a joint effort between Inflection and CoreWeave.

"They called us and said, 'Guys, we need you to build one of the most high-performance supercomputers on the planet to support our AI company,'" McBee said. "They call us and they say, 'This is what we're looking for, can you do it?'

Inflection has a lot of funding

Inflection is valued at $4B and has raised $1.5B, which is similar to Anthropic ($4.1B valuation, total raised $1.3B as of May 2023) and within an order of magnitude of OpenAI ($28B valuation, $11B raised as of April 2023).

Inflection is on the cutting edge of LLMs

Their flagship LLM, Inflection-1, has similar benchmark results to GPT-3.5

They seem to be currently training a model similarly capable to GPT-4. I expect them to finish training by the end of the year.

"We will also be releasing a technical memo detailing one of our models in the same compute class as PaLM-2 and GPT-4."

Inflection plans to train frontier LLMs

They seem to plan to train models 10x or 100x the size of GPT-4 within 18 months.

"We are about to train models that are 10 times larger than the cutting edge GPT-4 and then 100 times larger than GPT-4. That's what things look like over the next 18 months."

(it is unclear if "we" refers to Inflection or humanity)

Inflection doesn't seem to acknowledge existential risks or have a sizable safety team

Their safety site has zero mention of existential or catastrophic risks. Their white house memo is not very reassuring either.

Out of 19 open job listings, only 2 are on the Safety team.

If you look at their LinkedIn (which seems to list most of their current ~40 employees), zero of their employees are listed as working on AI safety at Inflection (one person has the word "safety" in their description but it's unclear that it's referring to their position at Inflection).

I think that this mostly means that the Inflection Safety team members list themselves as "Technical staff" or don't have LinkedIns. But to me it seems like they have less than 5 people working on safety.

Appendix: Estimating Inflection's compute

Here are some back-of-the-envelope calculations for Inflection's current compute from three data sources. They result in estimates ranging around 2 orders of magnitude, centered around 4e18.

FLOPs = plural of "floating point operation (FLOP)"

FLOPS = floating point operations per second

The H100 route

From the H100 datasheet, it seems like different components of the H100 (of which, different models exist), have different amounts of FL...