Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Shell games, published by TsviBT on March 19, 2023 on LessWrong.

[Metadata: crossposted from. First completed November 18, 2022.]

Shell game

Here's the classic shell game: Youtube

Screenshot from that video.

The little ball is a phantom: when you look for it under a specific shell, it's not there, it's under a different shell.

(This might be where the name "shell company" comes from: the business dealings are definitely somewhere, just not in this company you're looking at.)

Perpetual motion machines

Related: Perpetual motion beliefs

Bhāskara's wheel is a proposed perpetual-motion machine from the Middle Ages:

Here's another version:

From this video.

Someone could try arguing that this really is a perpetual motion machine:

Q: How do the bars get lifted up? What does the work to lift them?

A: By the bars on the other side pulling down.

Q: How does the wheel keep turning? How do the bars pull more on their way down than on their way up?

A: Because they're extended further from the center on the downward-moving side than on the upward-moving side, so they apply more torque to the wheel.

Q: How do the bars extend further on the way down?

A: Because the momentum of the wheel carries them into the vertical bar, flipping them over.

Q: But when that happens, energy is expended to lift up the little weights; that energy comes out of the kinetic energy of the wheel.

A: Ok, you're right, but that's not necessary to the design. All we need is that the torque on the downward side is greater than the torque on the upward side, so instead of flipping the weights up, we could tweak the mechanism to just shift them outward, straight to the side. That doesn't take any energy because it's just going straight sideways, from a resting position to another resting position.

Q: Yeah... you can shift them sideways with nearly zero work... but that means the weights are attached to the wheel at a pivot, right? So they'll just fall back and won't provide more torque.

A: They don't pivot, you fix them in place so they provide more torque.

Q: Ok, but then when do you push the weights back inward?

A: At the bottom.

Q: When the weight is at the bottom? But then the slider isn't horizontal, so pushing the weight back towards the center is pushing it upward, which takes work.

A: I meant, when the slider is at the bottom--when it's horizontal.

Q: But if the sliders are fixed in place, by the time they're horizontal at the bottom, you've already lifted the weights back up some amount; they're strong-torquing the other way.

A: At the bottom there's a guide ramp to lift the weights using normal force.

Q: But the guide ramp is also torquing the wheel.

And so on. The inventor can play hide the torque and hide the work.

Shell games in alignment

Some alignment schemes--schemes for structuring or training an AGI so that it can be transformatively useful and doesn't kill everyone--are prone to playing shell games. That is, there's some features of the scheme that don't seem to happen in a specific place; they happen somewhere other than where you're looking at the moment. Consider these questions:

What sort of smarter-than-human work is supposed to be done by the AGI? When and how does it do that work--by what combination of parts across time?

How does it become able to do that work? At what points does the AGI come to new understanding that it didn't have before?

How does the AGI orchestrate it's thinking and actions to have large effects on the world? By what process, components, rules, or other elements?

What determines the direction that the AGI's actions will push the world? Where did those determiners come from, and how exactly do they determine the direction?

Where and how much do human operators have to make judgements? How much are those judgements being relied on to point to goodness, truth, aligned...