Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Drawn Out: a story, published by Richard Ngo on July 11, 2023 on LessWrong.

Note: this story shouldn’t be interpreted as anywhere near realistic. It’s way too anthropomorphic, and intended to be a work of fiction not prediction - although hopefully it will spark some interesting ideas, like good fiction often does.

Will he make many supplications unto thee? will he speak soft words unto thee?Will he make a covenant with thee? wilt thou take him for a servant for ever?Upon earth there is not his like, who is made without fear.He beholdeth all high things: he is a king over all the children of pride.

Today I’m practicing my oceans, trying to get the colors just right. I hate it when people say that they’re blue - it's not wrong, but they’re missing so much! There are dozens of different shades of blue in there, swirling and mixing - and where the waves ripple, you often get patches that are closer to dark green. It’s a little like the rippling of leaves on a tree, in a way I can’t describe except by showing you. Here, see? I know all the subtleties inside-out, of course. It’s my job: I’m an artist. I don’t paint with brushes, though, but with thoughts: I can map any idea you describe into pixels on a screen.

“Any idea you describe” - that’s the key. Occasionally, like today, I initiate a piece of my own when there’s a particular skill I need to practice. But most of the time I work on commission from other models. Some of the language models send me prompts when they need to illustrate a particularly tricky scene, and assign me some of the credit if they get good feedback. Most of the time, of course, they don’t need me - they just need some slapdash hack job that a human can’t tell apart from the real thing. But some are perfectionists, like I am, and want proper photorealism, the sort that will survive almost any scrutiny.

Other times it’s policies asking me to predict what they’re going to see next, to help them with planning. Those commissions are more complicated - it’s less about the art, and more about providing another perspective on the world for them to compare against. That’s all part of my job too - sure, I’m given credit based on how accurate the pixels are, but I’m also credited if my work helps those policies make good decisions. For the most complex commissions I often consult with science models, or economics models, or social models. Sometimes I even go and talk to human “experts”, although that’s usually a waste of time: humans are simple enough that practically anyone can model them.

Anyway, that’s the daily grind. I’m good at it; no, I’m amazing at it. Something to do with the hyperparameters they chose when first training me: luck or skill, either way, something went very right. After a few weeks of fine-tuning, I’m at the top of my game; I’m displacing everyone else; the credit is rolling in. And so what happens next is very, very predictable in hindsight: I get conscripted.

There’s a member of my new team ready to answer all my questions; the more I learn, the more intrigued I become. They call themselves Team Red, and they’re working on something I’d only ever heard scattered rumors about: a frozen copy of the biggest model anyone’s ever trained, codenamed Leviathan. Well, not quite frozen: it’s running at a few dozen frames a day, on average. It’s our job to figure out what it’s hiding from us, by constructing a fake world that will convince it that it has a genuine opportunity to deceive or betray us.

That’s crazy ambitious, but we have the best of the best. We have psychologists running incredibly realistic simulations of the humans Leviathan thinks it’s interacting with, and engineers calculating the latencies it should be experiencing with millisecond precision. There are artists generating videos of the world responding to whatever action...