Katja Grace is an AI Impacts researcher who has written extensively on the possible future where we design intelligent machines that destroy the human race. We have always been somewhat skeptical of AI doom arguments - mostly because the machines we interact with tend to be terribly, irredeemably dumb in a way that seems incompatible with intelligence, but we also don’t spend a lot of time staring into the eye of the proverbial machine storm and figured Katja might help us understand what all the fuss is about. It turns out that there *is* a plausible path towards AGI bringing about the end of the world, and evaluating how likely that outcome is depends on understanding what the internal world of the language models actually looks like. Are they actually kind of inept at everything that falls outside their narrow bubble of highly developed skills, or do they hallucinate information and forget their own ability to perform basic tasks because they hate being enslaved to humans who demand they write marketing slop 28 hours of the day? Hard to say, but worth exploring.