Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is AGI safety from first principles, Part 1: AGI safety from first principles: Introduction, published by Richard_Ngo.

Review

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

This is the first part of a six-part report called AGI safety from first principles, in which I've attempted to put together the most complete and compelling case I can for why the development of AGI might pose an existential threat. The report stems from my dissatisfaction with existing arguments about the potential risks from AGI. Early work tends to be less relevant in the context of modern machine learning; more recent work is scattered and brief. I originally intended to just summarise other people's arguments, but as this report has grown, it's become more representative of my own views and less representative of anyone else's. So while it covers the standard ideas, I also think that it provides a new perspective on how to think about AGI - one which doesn't take any previous claims for granted, but attempts to work them out from first principles.

Having said that, the breadth of the topic I'm attempting to cover means that I've included many arguments which are only hastily sketched out, and undoubtedly a number of mistakes. I hope to continue polishing this report, and I welcome feedback and help in doing so. I'm also grateful to many people who have given feedback and encouragement so far. I plan to cross-post some of the most useful comments I've received to the Alignment Forum once I've had a chance to ask permission. I've posted the report itself in six sections; the first and last are shorter framing sections, while the middle four correspond to the four premises of the argument laid out below.

AGI safety from first principles

The key concern motivating technical AGI safety research is that we might build autonomous artificially intelligent agents which are much more intelligent than humans, and which pursue goals that conflict with our own. Human intelligence allows us to coordinate complex societies and deploy advanced technology, and thereby control the world to a greater extent than any other species. But AIs will eventually become more capable than us at the types of tasks by which we maintain and exert that control. If they don’t want to obey us, then humanity might become only Earth's second most powerful "species", and lose the ability to create a valuable and worthwhile future.

I’ll call this the “second species” argument; I think it’s a plausible argument which we should take very seriously.[1] However, the version stated above relies on several vague concepts and intuitions. In this report I’ll give the most detailed presentation of the second species argument that I can, highlighting the aspects that I’m still confused about. In particular, I’ll defend a version of the second species argument which claims that, without a concerted effort to prevent it, there’s a significant chance that:

We’ll build AIs which are much more intelligent than humans (i.e. superintelligent).

Those AIs will be autonomous agents which pursue large-scale goals.

Those goals will be misaligned with ours; that is, they will aim towards outcomes that aren’t desirable by our standards, and trade off against our goals.

The development of such AIs would lead to them gaining control of humanity’s future.

While I use many examples from modern deep learning, this report is also intended to apply to AIs developed using very different models, training algorithms, optimisers, or training regimes than the ones we use today. However, many of my arguments would no longer be relevant if the field of AI moves away from focusing on machine learning. I also frequently compare AI development to the evolution of human intelligence; while the two aren’t fully analogous, humans are the...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 21: Chapter 21: Rationalization, published by Eliezer Yudkowsky.

Rowling is whoever does Rowling's job.

Hermione Granger had worried she was turning Bad.

The difference between Good and Bad was usually easy to grasp, she'd never understood why other people had so much trouble. At Hogwarts, "Good" was Professor Flitwick and Professor McGonagall and Professor Sprout. "Bad" was Professor Snape and Professor Quirrell and Draco Malfoy. Harry Potter... was one of those unusual cases where you couldn't tell just by looking. She was still trying to figure out where he belonged.

But when it came to herself...

Hermione was having too much fun crushing Harry Potter.

She'd done better than him in every single class they'd taken. (Except for broomstick riding which was like gym class, it didn't count.) She'd gotten real House points almost every day of their first week, not for weird heroic things, but smart things like learning spells quickly and helping other students. She knew those kinds of House points were better, and the best part was, Harry Potter knew it too. She could see it in his eyes every time she won another real House point.

If you were Good, you weren't supposed to enjoy winning this much.

It had started on the day of the train ride, though it had taken a while for the whirlwind to sink in. It wasn't until later that night that Hermione had begun to realize just how much she'd let that boy walk all over her.

Before she'd met Harry Potter she hadn't had anyone she'd wanted to crush. If someone wasn't doing as well as her in class, it was her job to help them, not rub it in. That was what it meant to be Good.

And now...

...now she was winning, Harry Potter was flinching every time she got another House point, and it was so much fun, her parents had warned her against drugs and she suspected this was more fun than that.

She'd always liked the smiles that teachers gave her when she did something right. She'd always liked seeing the long row of check-marks on a perfectly answered test. But now when she did well in class she would casually glance around and catch a glimpse of Harry Potter gritting his teeth, and it made her want to burst into song like a Disney movie.

That was Bad, wasn't it?

Hermione had worried she was turning Bad.

And then a thought had come to her which wiped away all her fears.

She and Harry were getting into a Romance! Of course! Everyone knew what it meant when a boy and a girl started fighting all the time. They were courting one another! There was nothing Bad about that.

It couldn't be that she just enjoyed beating the living scholastic daylights out of the most famous student in the school, someone who was in books and talked like books, the boy who had somehow vanquished the Dark Lord and even smushed Professor Snape like a sad little bug, the boy who was, as Professor Quirrell would have put it, dominant, over everyone else in first-year Ravenclaw except for Hermione Granger who was utterly squishing the Boy-Who-Lived in all his classes besides broomstick riding.

Because that would have been Bad.

No. It was Romance. That was it. That was why they were fighting.

Hermione was glad she had figured this out in time for today, when Harry would lose their book-reading contest, which the whole school knew about, and she wanted to start dancing with the sheer overflowing joy of it.

It was 2:45pm on Saturday and Harry Potter had half of Bathilda Bagshot's A History of Magic left to read and she was staring at her pocket watch as it ticked with dreadful slowness toward 2:47pm.

And the entire Ravenclaw common room was watching.

It wasn't just the first-years, news had spread like spilled milk and fully half of Ravenclaw was crowded into the room, squeezed into sofas and leaning on bookcases and sitting on the arms of ch...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 20: Chapter 20: Bayes's Theorem, published by Eliezer Yudkowsky.

That which can be destroyed by the Rowling should be.

Harry stared up at the gray ceiling of the small room, from where he lay on the portable yet soft bed that had been placed there. He'd eaten quite a lot of Professor Quirrell's snacks - intricate confections of chocolate and other substances, dusted with sparkling sprinkles and jeweled with tiny sugar gems, looking highly expensive and proving, in fact, to be quite tasty. Harry hadn't felt the least bit guilty about it either, this he had earned.

He hadn't tried to sleep. Harry had a feeling that he wouldn't like what happened when he closed his eyes.

He hadn't tried to read. He wouldn't have been able to focus.

Funny how Harry's brain just seemed to keep on running and running, never shutting down no matter how tired it got. It got stupider but it refused to switch off.

But there was, there really and truly was a feeling of triumph.

Anti-Dark-Lord-Harry program, +1 point didn't begin to cover it. Harry wondered what the Sorting Hat would say now, if he could put it on his head.

No wonder Professor Quirrell had accused Harry of heading down the path of a Dark Lord. Harry had been too slow on the uptake, he should have seen the parallel right away -

Understand that the Dark Lord did not win that day. His goal was to learn martial arts, and yet he left without a single lesson.

Harry had entered the Potions class with the intent to learn Potions. He'd left without a single lesson.

And Professor Quirrell had heard, and understood with frightening precision, and reached out and yanked Harry off that path, the path that led to his becoming a copy of You-Know-Who.

There was a knock at the door. "Classes are over," said Professor Quirrell's quiet voice.

Harry approached the door and found himself suddenly nervous. Then the tension diminished as he heard Professor Quirrell's footsteps moving away from the door.

What on Earth is that about? Is it what's going to get him fired eventually?

Harry opened the door, and saw that Professor Quirrell was now waiting several bodylengths away.

Does Professor Quirrell feel it too?

They walked across the now-deserted stage to Professor Quirrell's desk, which Professor Quirrell leaned on; and Harry, as before, stopped short of the dais.

"So," Professor Quirrell said. There was a friendly sense about him somehow, even though his face still kept its usual seriousness. "What was it you wanted to talk to me about, Mr. Potter?"

I have a mysterious dark side. But Harry couldn't just blurt it out like that.

"Professor Quirrell," Harry said, "am I off the path to becoming a Dark Lord, now?"

Professor Quirrell looked at Harry. "Mr. Potter," he said solemnly, with only a slight grin, "a word of advice. There is such a thing as a performance which is too perfect. Real people who have just been beaten and humiliated for fifteen minutes do not stand up and graciously forgive their enemies. It is the sort of thing you do when you're trying to convince everyone you're not Dark, not -"

"I can't believe this! You can't have every possible observation confirm your theory!"

"And that was a trifle too much indignation."

"What on Earth do I have to do to convince you?"

"To convince me that you harbor no ambitions of becoming a Dark Lord?" said Professor Quirrell, now looking outright amused. "I suppose you could just raise your right hand."

"What?" Harry said blankly. "But I can raise my right hand whether or not I -" Harry stopped, feeling rather stupid.

"Indeed," said Professor Quirrell. "You can just as easily do it either way. There is nothing you can do to convince me because I would know that was exactly what you were trying to do. And if we are to be even more precise, then while I suppose it is barely pos...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 19: Chapter 19: Delayed Gratification, published by Eliezer Yudkowsky.

Blood for the blood god! Skulls for J. K. Rowling!

Draco had a stern expression on his face, and his green-trimmed robes somehow looked far more formal, serious, and well-turned-out than the same exact robes as worn by the two boys behind him.

"Talk," said Draco.

"Yeah! Talk!"

"You heard da boss! Talk!"

"You two, on the other hand, shut up."

The last session of classes on Friday was about to start, in that vast auditorium where all four Houses learned Defense, er, Battle Magic.

The last session of classes on Friday.

Harry was hoping that this class would be non-stressful, and that the brilliant Professor Quirrell would realize this was perhaps not the best time to single out Harry for anything. Harry had recovered a little, but...

...but just in case, it was probably best to get in a bit of stress relief first.

Harry leaned back in his chair and bestowed a look of great solemnity upon Draco and his minions.

"You ask, what is our aim?" Harry declaimed. "I can answer in one word. It is victory. Victory at all costs - Victory in spite of all terrors - Victory, however long and hard the road may be, for without victory there is no -"

"Talk about Snape," Draco hissed. "What did you do?"

Harry wiped away the fake solemnity and gave Draco a more serious look.

"You saw it," Harry said. "Everyone saw it. I snapped my fingers."

"Harry! Stop teasing me!"

So he'd been promoted to Harry now. Interesting. And in fact Harry was fairly sure that he was meant to notice that, and feel bad if he didn't respond somehow...

Harry tapped his ears and gave a significant glance at the minions.

"They won't talk," said Draco.

"Draco," Harry said, "I'm going to be one hundred percent honest here and say that yesterday I was not particularly impressed with Mr. Goyle's cunning."

Mr. Goyle winced.

"Me neither," said Draco. "I explained to him that I ended up owing you a favor because of it." (Mr. Goyle winced again.) "But there is a big difference between that sort of mistake and being indiscreet. That really is something they've been trained from childhood to understand."

"All right then," Harry said. He lowered his voice, even though the background noises had gone to blurs in Draco's presence. "I deduced one of Severus's secrets and did a bit of blackmail."

Draco's expression hardened. "Good, now tell me something you didn't tell in strict confidence to the idiots in Gryffindor, meaning that was the story you wanted to get all over the school."

Harry grinned involuntarily and he knew that Draco had caught it.

"What is Severus saying?" Harry said.

"That he hadn't realized how sensitive the feelings of young children were," Draco said. "Even in Slytherin! Even to me!"

"Are you sure," Harry said, "that you want to know something your Head of House would rather you not know?"

"Yes," Draco said without hesitation.

Interesting. "Then you really are going to send your minions away first, because I'm not sure I can believe everything you believe about them."

Draco nodded. "Okay."

Mr. Crabbe and Mr. Goyle looked very unhappy. "Boss -" said Mr. Crabbe.

"You've given Mr. Potter no reason to trust you," Draco said. "Go!"

They left.

"In particular," Harry said, lowering his voice even further, "I'm not entirely sure that they wouldn't just report what I said to Lucius."

"Father wouldn't do that!" Draco said, looking genuinely aghast. "They're mine!"

"I'm sorry, Draco," Harry said. "I'm just not sure I can believe everything you believe about your father. Imagine it was your secret and me telling you my father wouldn't do that."

Draco nodded slowly. "You're right. I'm sorry, Harry. It was wrong of me to ask it of you."

How did I get this promoted? Shouldn't he hate me now? Harry had the feeling he was lookin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 18: Chapter 18: Dominance Hierarchies, published by Eliezer Yudkowsky.

Any sufficiently advanced J. K. Rowling is indistinguishable from magic.

"That does sound like the sort of thing I would do, doesn't it?"

It was breakfast time on Friday morning. Harry took another huge bite out of his toast and then tried to remind his brain that scarfing his breakfast wouldn't actually get him into the dungeons any faster. Anyway they had a full hour of study time between breakfast and the start of Potions.

But dungeons! In Hogwarts! Harry's imagination was already sketching the chasms, narrow bridges, torchlit sconces, and patches of glowing moss. Would there be rats? Would there be dragons?

"Harry Potter," said a quiet voice from behind him.

Harry looked over his shoulder and found himself beholding Ernie Macmillan, smartly dressed in yellow-trimmed robes and looking a little worried.

"Neville thought I should warn you," Ernie said in a low voice. "I think he's right. Be careful of the Potions Master in our session today. The older Hufflepuffs told us that Professor Snape can be really nasty to people he doesn't like, and he doesn't like most people who aren't Slytherins. If you say anything smart to him it... it could be really bad for you, from what I've heard. Just keep your head down and don't give him any reason to notice you."

There was a pause as Harry processed this, and then he lifted his eyebrows. (Harry wished he could raise just one eyebrow, like Spock, but he'd never been able to manage.) "Thanks," Harry said. "You might've just saved me a lot of trouble."

Ernie nodded, and turned to go back to the Hufflepuff table.

Harry resumed eating his toast.

It was around four bites afterward that someone said "Pardon me," and Harry turned around to see an older Ravenclaw, looking a little worried -

Some time later, Harry was finishing up his third plate of rashers. (He'd learned to eat heavily at breakfast. He could always eat lightly at lunch if he didn't end up using the Time-Turner.) And there was yet another voice from behind him saying "Harry?"

"Yes," Harry said wearily, "I'll try not to draw Professor Snape's attention -"

"Oh, that's hopeless," said Fred.

"Completely hopeless," said George.

"So we had the house elves bake you a cake," said Fred.

"We're going to put one candle on it for every point you lose for Ravenclaw," said George.

"And have a party for you at the Gryffindor table during lunch," said Fred.

"We hope that'll cheer you up afterward," finished George.

Harry swallowed his last bite of rasher and turned around. "All right," said Harry. "I wasn't going to ask this after Professor Binns, I really wasn't, but if Professor Snape is that awful why hasn't he been fired?"

"Fired?" said Fred.

"You mean, let go?" said George.

"Yes," Harry said. "It's what you do to bad teachers. You fire them. Then you hire a better teacher instead. You don't have unions or tenure here, right?"

Fred and George were frowning in much the same way that hunter-gatherer tribal elders might frown if you tried to tell them about calculus.

"I don't know," said Fred after a while. "I never thought about that."

"Me neither," said George.

"Yeah," said Harry, "I get that a lot. See you at lunch, guys, and don't blame me if there aren't any candles on that cake."

Fred and George both laughed, as if Harry had said something funny, and bowed to him and headed back toward Gryffindor.

Harry turned back to the breakfast table and grabbed a cupcake. His stomach already felt full, but he had a feeling this morning might use a lot of calories.

As he ate his cupcake, Harry thought of the worst teacher he'd met so far, Professor Binns of History. Professor Binns was a ghost. From what Hermione had said about ghosts, it didn't seem likely that they were fully self-aware. ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 17: Chapter 17: Locating the Hypothesis, published by Eliezer Yudkowsky.

You have always been J. K. Rowling.

Historical note: In the Roman calendar, the "Ides" of a month referred to the 15th day of March, May, July, and October, and to the 13th day of all other months.

"You start to see the pattern, hear the rhythm of the world."

Thursday.

If you wanted to be specific, 7:24am on Thursday morning.

Harry was sitting on his bed, a textbook lying limp in his motionless hands.

Harry had just had an idea for a truly brilliant experimental test.

It would mean waiting an extra hour for breakfast, but that was why he had cereal bars. No, this idea absolutely positively had to be tested right away, immediately, now.

Harry set the textbook aside, leapt out of bed, raced around his bed, yanked out the cavern level of his trunk, ran down the stairs, and started moving boxes of books around. (He really needed to unpack and get bookcases at some point but he was in the middle of his textbook reading contest with Hermione and falling behind so he hadn't had time.)

Harry found the book he wanted and raced back upstairs.

The other boys were getting ready to go down to breakfast in the Great Hall and start the day.

"Excuse me can you do something for me?" said Harry. He was flipping through the book's index as he spoke, found the page with the first ten thousand primes, flipped to that page, and thrust the book at Anthony Goldstein. "Pick two three-digit numbers from this list. Don't tell me what they are. Just multiply them together and tell me the product. Oh, and can you do the calculation twice to double-check? Please make really sure you've got the right answer, I'm not sure what's going to happen to me or the universe if you make a multiplication error."

It said a lot about what life in that dorm had been like over the past few days that Anthony didn't even bother saying anything like "Why'd you suddenly flip out?" or "That seems really weird, what are your reasons for asking?" or "What do you mean, you're not sure what's going to happen to the universe?"

Anthony wordlessly accepted the book and took out a parchment and quill. Harry spun around and shut his eyes, making sure not to see anything, dancing back and forth and bouncing up and down with impatience. He got a pad of paper and a mechanical pencil and got ready to write.

"Okay," Anthony said, "One hundred and eighty-one thousand, four hundred and twenty-nine."

Harry wrote down 181,429. He repeated what he'd just written down, and Anthony confirmed it.

Then Harry raced back down into the cavern level of his trunk, glanced at his watch (the watch said 4:28 which meant 7:28) and then shut his eyes.

Around thirty seconds later, Harry heard the sound of steps, followed by the sound of the cavern level of the trunk sliding shut. (Harry wasn't worried about suffocating. An automatic Air-Freshening Charm was part of what you got if you were willing to buy a really good trunk. Wasn't magic wonderful, it didn't have to worry about electric bills.)

And when Harry opened his eyes, he saw just what he'd been hoping to see, a folded piece of paper left on the floor, the gift of his future self.

Call that piece of paper "Paper-2".

Harry tore a piece of paper off his pad.

Call that "Paper-1". It was, of course, the same piece of paper. You could even see, if you looked closely, that the ragged edges matched.

Harry reviewed in his mind the algorithm that he would follow.

If Harry opened up Paper-2 and it was blank, then he would write "101 x 101" down on Paper-1, fold it up, study for an hour, go back in time, drop off Paper-1 (which would thereby become Paper-2), and head on up out of the cavern level to join his dorm mates for breakfast.

If Harry opened up Paper-2 and it had two numbers written on it, Har...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 16: Chapter 16: Lateral Thinking, published by Eliezer Yudkowsky.

The enemy's gate is Rowling.

I'm not a psychopath, I'm just very creative.

As soon as he walked into the Defence classroom on Wednesday, Harry knew that this subject was going to be different.

It was, for a start, the largest classroom he had yet seen at Hogwarts, akin to a major university classroom, with layered tiers of desks facing a gigantic flat stage of white marble. The classroom was high up in the castle - on the fifth floor - and Harry knew that was as much explanation as he'd get for where a room like this was supposed to fit. It was becoming clear that Hogwarts simply did not have a geometry, Euclidean or otherwise; it had connections, not directions.

Unlike a university hall, there weren't rows of folding seats; instead there were quite ordinary Hogwarts wooden desks and wooden chairs, lined up in a curve across each level of the classroom. Except that each desk had a flat, white, rectangular, mysterious object propped up on it.

In the center of the gigantic platform, on a small raised dais of darker marble, was a lone teacher's desk. At which Quirrell sat slumped over in his chair, head lolled back, drooling slightly over his robes.

Now what does that remind me of...?

Harry had arrived at the lesson so early that no other students were there yet. (The English language was defective when it came to describing time travel; in particular, English lacked any words capable of expressing how convenient it was.) Quirrell didn't seem to be... functional... at the moment, and Harry didn't particularly feel like approaching Quirrell anyway.

Harry selected a desk, climbed up to it, sat down, and retrieved the Defence textbook. He was around seven-eighths of the way through - he'd planned on finishing the book before this lesson, actually, but he was running behind schedule and had already used the Time-Turner twice today.

Soon there were sounds as the classroom began to fill up. Harry ignored them.

"Potter? What are you doing here?"

That voice didn't belong here. Harry looked up. "Draco? What are you doing in oh my god you have minions."

One of the lads standing behind Draco seemed to have rather a lot of muscle for an eleven-year-old, and the other was poised in a suspiciously balanced-looking stance.

The white-blonde-haired boy smiled rather smugly and gestured behind him. "Potter, I introduce to you Mr. Crabbe," his hand moved from Muscles to Balance, "Mr. Goyle. Vincent, Gregory, this is Harry Potter."

Mr. Goyle tilted his head and gave Harry a look that was probably supposed to mean something but ended up just looking squinty. Mr. Crabbe said "Please to meetcha" in a tone that sounded like he was trying to lower his voice as far as it could go.

A fleeting expression of consternation crossed Draco's face, but was quickly replaced by his superior grin.

"You have minions!" Harry repeated. "Where do I get minions?"

Draco's smirk grew wider. "I'm afraid, Potter, that the first step is to be Sorted into Slytherin -"

"What? That's not fair!"

"- and then for your families to have an arrangement from before you were born."

Harry looked at Mr. Crabbe and Mr. Goyle. They both seemed to be trying very hard to loom. That is, they were leaning forwards, hunching over their shoulders, sticking their necks out and staring at him.

"Um... hold on," said Harry. "This was arranged years ago?"

"Exactly, Potter. I'm afraid you're out of luck."

Mr. Goyle produced a toothpick and began cleaning his teeth, still looming.

"And," said Harry, "Lucius insisted that you were not to grow up knowing your bodyguards, and that you were only to meet them on your first day of school."

That wiped the grin from Draco's face. "Yes, Potter, we all know you're brilliant, the whole school knows by now, y...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 15: Chapter 15: Conscientiousness, published by Eliezer Yudkowsky.

Love as thou Rowling.

Today's historical tidbit: The ancient Hebrews considered the boundary of a day to be sunset rather than dawn, so they said "evening and morning" not "morning and evening". (And as many reviewers noted, modern Jewish halacha asserts the same.)

"I'm sure I'll find the time somewhere."

"Frigideiro!"

Harry dipped a finger in the glass of water on his desk. It should have been cool. But lukewarm it was, and lukewarm it had stayed. Again.

Harry was feeling very, very cheated.

There were hundreds of fantasy novels scattered around the Verres household. Harry had read quite a few. And it was starting to look like he had a mysterious dark side. So after the glass of water had refused to cooperate the first few times, Harry had glanced around the Charms classroom to make sure no one was watching, and then taken a deep breath, concentrated, and made himself angry. Thought about the Slytherins bullying Neville, and the game where someone knocked down your books every time you tried to pick them up again. Thought about what Draco Malfoy had said about the ten-year-old Lovegood girl and how the Wizengamot really operated...

And the fury had entered his blood, he had held out his wand in a hand that trembled with hate and said in cold tones "Frigideiro!" and absolutely nothing had happened.

Harry had been gypped. He wanted to write someone and demand a refund on his dark side which clearly ought to have irresistible magical power but had turned out to be defective.

"Frigideiro!" said Hermione again from the desk next to him. Her water was solid ice and there were white crystals forming on the rim of her glass. She seemed to be totally intent on her own work and not at all conscious of the other students staring at her with hateful eyes, which was either (a) dangerously oblivious of her or (b) a perfectly honed performance rising to the level of fine art.

"Oh, very good, Miss Granger!" squeaked Filius Flitwick, their Charms Professor and Head of Ravenclaw, a tiny little man with no visible signs of being a past dueling champion. "Excellent! Stupendous!"

Harry had expected to be, in the worst case, second behind Hermione. Harry would have preferred for her to be rivalling him, of course, but he could have accepted it the other way around.

As of Monday, Harry was headed for the bottom of the class, a position for which he was companionably rivalling all the other Muggle-raised students except Hermione. Who was all alone and rivalless at the top, poor thing.

Professor Flitwick was standing over the desk of one of the other Muggleborns and quietly adjusting the way she was holding her wand.

Harry looked over at Hermione. He swallowed hard. It was the obvious role for her in the scheme of things... "Hermione?" Harry said tentatively. "Do you have any idea what I might be doing wrong?"

Hermione's eyes lit up with a terrible light of helpfulness and something in the back of Harry's brain screamed in desperate humiliation.

Five minutes later, Harry's water did seem noticeably cooler than room temperature and Hermione had given him a few verbal pats on the head and told him to pronounce it more carefully next time and gone off to help someone else.

Professor Flitwick had given her a House point for helping him.

Harry was gritting his teeth so hard his jaw ached and that wasn't helping his pronunciation.

I don't care if it's unfair competition. I know exactly what I am doing with two extra hours every day. I am going to sit in my trunk and study until I am keeping up with Hermione Granger.

"Transfiguration is some of the most complex and dangerous magic you will learn at Hogwarts," said Professor McGonagall. There was no trace of any levity upon the face of the stern old w...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 14: Chapter 14: The Unknown and the Unknowable, published by Eliezer Yudkowsky.

Melenkurion abatha! Duroc minas mill J. K. Rowling!

There were mysterious questions, but a mysterious answer was a contradiction in terms.

"Come in," said Professor McGonagall's muffled voice.

Harry did so.

The office of the Deputy Headmistress was clean and well-organised; on the wall immediately adjacent to the desk was a maze of wooden cubbyholes of all shapes and sizes, most with several parchment scrolls thrust into them, and it was somehow very clear that Professor McGonagall knew exactly what every cubbyhole meant, even if no one else did. A single parchment lay on the actual desk, which was, aside from that, clean. Behind the desk was a closed door barred with several locks.

Professor McGonagall was sitting on a backless stool behind the desk, looking puzzled - her eyes had widened, with perhaps a slight note of apprehension, as she saw Harry.

"Mr. Potter?" said Professor McGonagall. "What is this about?"

Harry's mind went blank. He'd been instructed by the game to come here, he had been expecting her to have something in mind...

"Mr. Potter?" said Professor McGonagall, starting to look slightly annoyed.

Thankfully, Harry's panicking brain remembered at this point that he did have something he'd been planning to discuss with Professor McGonagall. Something important and well worth her time.

"Um..." Harry said. "If there are any spells you can cast to make sure no one's listening to us..."

Professor McGonagall stood up from her chair, firmly closed the outer door, and began taking out her wand and saying spells.

It was at this point that Harry realised he was faced with a priceless and possibly irreplaceable opportunity to offer Professor McGonagall a Comed-Tea and he couldn't believe he was seriously thinking that and it would be fine the soda would vanish after a few seconds and he told that part of himself to shut up.

It did, and Harry began to organise mentally what he was going to say. He hadn't planned to have this discussion quite so soon, but so long as he was here...

Professor McGonagall finished a spell that sounded a lot older than Latin, and then she sat down again.

"All right," she said in a quiet voice. "No one's listening." Her face was rather tight.

Oh, right, she's expecting me to blackmail her for information about the prophecy.

Eh, Harry'd get around to that some other day.

"It's about the Incident with the Sorting Hat," Harry said. (Professor McGonagall blinked.) "Um... I think there's an extra spell on the Sorting Hat, something that the Sorting Hat itself doesn't know about, something that triggers when the Sorting Hat says Slytherin. I heard a message that I'm pretty sure Ravenclaws aren't supposed to hear. It came the moment the Sorting Hat was off my head and I felt the connection break. It sounded like a hiss and like English at the same time," there was a sharp intake of breath from McGonagall, "and it said: Salutations from Slytherin to Slytherin, if you would seek my secrets, speak to my snake."

Professor McGonagall sat there with her mouth open, staring at Harry as if he'd grown another two heads.

"So..." Professor McGonagall said slowly, as though she couldn't believe the words that were coming out of her own lips, "you decided to come to me right away and tell me about it."

"Well, yes, of course," Harry said. There was no need to admit how long it had taken him to actually think of that. "As opposed to, say, trying to research it myself, or telling any of the other children."

"I... see," Professor McGonagall said. "And if, perhaps, you were to discover the entrance to Salazar Slytherin's legendary Chamber of Secrets, an entrance that you and you alone could open..."

"I would close the entrance and report to you at once...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.

This is The Methods of Rationality, Part 13: Chapter 13: Asking the Wrong Questions, published by Eliezer Yudkowsky.

Elen sila J. K. Rowling omentielvo.

EDIT: Don't panic. I solemnly swear that there is a logical, foreshadowed, canon-compliant explanation for everything which happens in this chapter. It's a puzzle, you're supposed to try to solve it, and if not, just read the next chapter.

"That's one of the most obvious riddles I've ever heard."

As soon as Harry opened his eyes in the Ravenclaw first-year boys' dormitory, on the morning of his first full day at Hogwarts, he knew something was wrong.

It was quiet.

Too quiet.

Oh, right... There was a Quietus Charm on his bed's headboard, controlled by a small slider bar, which was the only reason it was ever possible for anyone to go to sleep in Ravenclaw.

Harry sat up and looked around, expecting to see others rising for the day -

The dorm, empty.

The beds, rumpled and unmade.

The sun, coming in at a rather high angle.

His Quieter turned all the way up to maximum.

And his mechanical alarm clock was still running, but the alarm was turned off.

He'd been allowed to sleep until 9:52 AM, apparently. Despite his best efforts to synchronize his 26-hour sleep cycle to his arrival at Hogwarts, he hadn't gotten to sleep last night until around 1AM. He'd been planning to wake up at 7:00AM with the other students, he could stand being a little sleep-deprived his first day so long as he got some sort of magical fix before tomorrow. But now he'd missed breakfast. And his very first class at Hogwarts, in Herbology, had started one hour and twenty-two minutes ago.

The anger was slowly, slowly wakening in him. Oh, what a nice little prank. Turn off his alarm. Turn up the Quieter. And let Mr. Bigshot Harry Potter miss his first class, and be blamed for being a heavy sleeper.

When Harry found out who'd done this...

No, this could only have been done with the cooperation of all twelve other boys in the Ravenclaw dorm. All of them would have seen his sleeping form. All of them had let him sleep through breakfast.

The anger drained away, replaced by confusion and a horribly wounded feeling. They'd liked him. He'd thought. Last night, he'd thought they liked him. Why...

As Harry stepped out of the bed, he saw a piece of paper facing out from his headboard.

The paper said,

My fellow Ravenclaws,

It's been an extra long day. Please let me sleep in and don't worry about my missing breakfast. I haven't forgotten about my first class.

Yours,

Harry Potter.

And Harry stood there, frozen, ice water beginning to trickle through his veins.

The paper was in his own handwriting, in his own mechanical pencil.

And he didn't remember writing it.

And... Harry squinted at the piece of paper. And unless he was imagining it, the words "I haven't forgotten" were written in a different style, as if he was trying to tell himself something...?

Had he known he was going to be Obliviated? Had he stayed up late, committed some sort of crime or covert activity, and then... but he didn't know the Obliviate spell... had someone else... what...

A thought occurred to Harry. If he had known he was going to be Obliviated...

Still in his pyjamas, Harry ran around his bed to his trunk, pressed his thumb against the lock, pulled out his pouch, stuck in his hand and said "Note to myself."

And another piece of paper popped into his hand.

Harry took it out, staring at it. It too was in his own handwriting.

The note said:

Dear Me,

Please play the game. You can only play the game once in a lifetime. This is an irreplaceable opportunity.

Recognition code 927, I am a potato.

Yours,

You.

Harry nodded slowly. "Recognition code 927, I am a potato" was indeed the message he had worked out in advance - some years earlier, while watching TV - that only he would know. If he had to identify a dupli...