LessWrong (Curated & Popular)

[Meta note: quickly written, unpolished. Also, it's possible that there's some more convincing work on this topic that I'm unaware of – if so, let me know]

In research discussions about LLMs, I often pick up a vibe of casual, generalized skepticism about model-generated CoT (chain-of-thought) explanations.

CoTs (people say) are not trustworthy in general. They don't always reflect what the model is "actually" thinking or how it has "actually" solved a given problem.

This claim is true as far as it goes. But people sometimes act like it goes much further than (IMO) it really does.

Sometimes it seems to license an attitude of "oh, it's no use reading what the model says in the CoT, you're a chump if you trust that stuff." Or, more insidiously, a failure to even ask the question "what, if anything, can we learn about the model's reasoning process by reading the [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
September 29th, 2024

Source:
https://www.lesswrong.com/posts/HQyWGE2BummDCc2Cx/the-case-for-cot-unfaithfulness-is-overstated

---

Narrated by TYPE III AUDIO.

I've been wanting to write a nice post for a few months, but should probably just write a one sooner instead. This is a top-level post not because it's a long text, but because it's important text.

Anyways. Cryonics is pretty much money-free now—one of the most affordable ways to dispose of your body post-mortem.

In the west coast in the USA, from Oregon Brain Preservation, as of around May 2024 I think:

Our research program is open to individuals in Washington, Oregon, and Northern California. This is the same brain preservation procedure, with the goal of future revival if this ever becomes feasible and humane. The difference is that we will also remove two small biopsy samples, which will be transferred to our partner non-profit organization, Apex Neuroscience, to measure the preservation quality and contribute to neuroscience research. Although there are no guarantees, we do not expect these [...]

---

First published:
September 29th, 2024

Source:
https://www.lesswrong.com/posts/WE65pBLQvNk3h3Dnr/cryonics-is-free

---

Narrated by TYPE III AUDIO.

Quarterly Performance Review, Autumn 1983

Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe his nerves. He only had one employee review left to get through, but he’d saved the hardest one for last.

He wasn’t upset about having to dismiss Lieutenant Colonel Petrov—he couldn’t wait to be rid of the little shit—but he couldn’t shake the feeling that something was amiss. He took a swig from his flask.

“Stanislav, you can come in now,” Yuri shouted as he opened the door and nearly smashed Stanislav Petrov in the face. “Have a seat,” he said.

“Yes, sir. Thank you sir. Overjoyed to be here as always,” Stanislav said.

“The purpose of this meeting is for us to discuss various concerns that have emerged about your performance over the past several months,” said Yuri. “Looking through your chart, what I’m seeing [...]

---

Outline:

(00:04) Quarterly Performance Review, Autumn 1983

(07:27) Quarterly Performance Review, Winter 1983

---

First published:
September 26th, 2024

Source:
https://www.lesswrong.com/posts/kj4jW9DxtKQBJbapn/stanislav-petrov-quarterly-performance-review

---

Narrated by TYPE III AUDIO.

I’ve claimed that Willpower compounds and that small wins in the present make it easier to get bigger wins in the future. Unfortunately, procrastination and laziness compound, too.

You’re stressed out for some reason, so you take the evening off for a YouTube binge. You end up staying awake a little later than usual and sleeping poorly. So the next morning you feel especially tired; you snooze a few extra times. In your rushed morning routine you don’t have time to prepare for the work meeting as much as you’d planned to. So you have little to contribute during the meeting. You feel bad about your performance. You escape from the bad feelings with a Twitter break. But Twitter is freaking out. Elon Musk said what? Everyone is weighing in. This is going to occupy you intermittently for the rest of the day. And so on.

Laziness has a [...]

---

Outline:

(01:44) I’m spiraling! I’m spiraling!

(02:54) Do what you can

(03:33) A) Emergency recovery

(05:49) B) Natural recovery

(06:14) Wait for a reset point

(08:53) Analyze the circumstances that caused it

(12:17) C) Heroic recovery

(12:50) Deep psych work

(15:16) Conclusion

The original text contained 1 image which was described by AI.

---

First published:
September 19th, 2024

Source:
https://www.lesswrong.com/posts/JBR6AF9Gusv4u6Fwo/laziness-death-spirals

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

For a long time, when I heard "slow takeoff", I assumed it meant "takeoff that takes longer calendar time than fast takeoff." (i.e. what is now referred to more often as "short timelines" vs "long timelines."). I think Paul Christiano popularized the term, and it so happened he both expected to see longer timelines and smoother/continuous takeoff.

I think it's at least somewhat confusing to use the term "slow" to mean "smooth/continuous", because that's not what "slow" particularly means most of the time.

I think it's even more actively confusing because "smooth/continuous" takeoff not only could be faster in calendar time, but, I'd weakly expect this on average, since smooth takeoff means that AI resources at a given time are feeding into more AI resources, whereas sharp/discontinuous takeoff would tend to mean "AI tech doesn't get seriously applied towards AI development until towards the end."

I don't think this [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
September 28th, 2024

Source:
https://www.lesswrong.com/posts/6svEwNBhokQ83qMBz/slow-takeoff-is-a-terrible-term-for-maybe-even-faster

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A common claim among e/accs is that, since the solar system is big, Earth will be left alone by superintelligences. A simple rejoinder is that just because Bernard Arnault has $170 billion, does not mean that he'll give you $77.18.

Earth subtends only 4.54e-10 = 0.0000000454% of the angular area around the Sun, according to GPT-o1.[1]

Asking an ASI to leave a hole in a Dyson Shell, so that Earth could get some sunlight not transformed to infrared, would cost It 4.5e-10 of Its income.

This is like asking Bernard Arnalt to send you $77.18 of his $170 billion of wealth.

In real life, Arnalt says no.

But wouldn't humanity be able to trade with ASIs, and pay Them to give us sunlight? This is like planning to get $77 from Bernard Arnalt by selling him an Oreo cookie.

To extract $77 from Arnalt, it's not a sufficient [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
September 23rd, 2024

Source:
https://www.lesswrong.com/posts/F8sfrbPjCQj4KwJqn/asis-will-not-leave-just-a-little-sunlight-for-earth

---

Narrated by TYPE III AUDIO.

A year ago, I started trying to deliberate practice skills that would "help people figure out the answers to confusing, important questions." I experimented with Thinking Physics questions, GPQA questions, Puzzle Games , Strategy Games, and a stupid twitchy reflex game I had struggled to beat for 8 years[1]. Then I went back to my day job and tried figuring stuff out there too.

The most important skill I was trying to learn was Metastrategic Brainstorming[2] – the skill of looking at a confusing, hopeless situation, and nonetheless brainstorming useful ways to get traction or avoid wasted motion.

Normally, when you want to get good at something, it's great to stand on the shoulders of giants and copy all the existing techniques. But this is challenging if you're trying to solve important, confusing problems because there probably isn't (much) established wisdom on how to solve it. You may [...]

---

Outline:

(02:33) Taking breaks, or naps

(03:25) Working Memory facility

(04:56) Patience

(06:17) Know what deconfusion, or having a crisp understanding feels like

(07:50) Actually Fucking Backchain

(10:00) Ask Whats My Goal?

(11:09) Always have at least 3 hypotheses

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
September 18th, 2024

Source:
https://www.lesswrong.com/posts/thc4RemfLcM5AdJDa/skills-from-a-year-of-purposeful-rationality-practice

---

Narrated by TYPE III AUDIO.

After the release of Ben Pace's extended interview with me about my views on religion, I felt inspired to publish more of my thinking about religion in a format that's more detailed, compact, and organized. This post is the first publication in my series of intended posts about religion.

Thanks to Ben Pace, Chris Lakin, Richard Ngo, Renshin Lauren Lee, Mark Miller, and Imam Ammar Amonette for their feedback on this post, and thanks to Kaj Sotala, Tomáš Gavenčiak, Paul Colognese, and David Spivak for reviewing earlier versions of this post. Thanks especially to Renshin Lauren Lee and Imam Ammar Amonette for their input on my claims about religion and inner work, and Mark Miller for vetting my claims about predictive processing.

In Waking Up, Sam Harris wrote:[1]

But I now knew that Jesus, the Buddha, Lao Tzu, and the other saints and sages of [...]

---

Outline:

(01:36) “Trapped Priors As A Basic Problem Of Rationality”

(03:49) Active blind spots as second-order trapped priors

(06:17) Inner work ≈ the systematic addressing of trapped priors

(08:33) Religious mystical traditions as time-tested traditions of inner work?

The original text contained 12 footnotes which were omitted from this narration.

---

First published:
August 23rd, 2024

Source:
https://www.lesswrong.com/posts/X2og6RReKD47vseK8/how-i-started-believing-religion-might-actually-matter-for

---

Narrated by TYPE III AUDIO.

There's a popular story that goes like this: Christopher Hitchens used to be in favor of the US waterboarding terrorists because he though it's wasn't bad enough to be torture.. Then he had it tried on himself, and changed his mind, coming to believe it isn't torture.

(Context for those unfamiliar: in the decade following 9/11, the US engaged in a lot of... questionable behavior to persecute the war on terror, and there was a big debate on whether waterboarding should be permitted. Many other public figures also volunteered to undergo the procedure as a part of this public debate; most notably Sean Hannity, who was an outspoken proponent of waterboarding, yet welched on his offer and never tried it himself.)

This story intrigued me because it's popular among both Hitchens' fans and his detractors. His fans use it as an example of his intellectual honesty and willingness to [...]

---

First published:
September 15th, 2024

Source:
https://www.lesswrong.com/posts/fNqEGTmkCy9sqZYm7/did-christopher-hitchens-change-his-mind-about-waterboarding

---

Narrated by TYPE III AUDIO.

Midjourney, “Fourth Industrial Revolution Digital Transformation”This is a little rant I like to give, because it's something I learned on the job that I’ve never seen written up explicitly.

There are a bunch of buzzwords floating around regarding computer technology in an industrial or manufacturing context: “digital transformation”, “the Fourth Industrial Revolution”, “Industrial Internet of Things”.

What do those things really mean?

Do they mean anything at all?

The answer is yes, and what they mean is the process of putting all of a company's data on computers so it can be analyzed.

This is the prerequisite to any kind of “AI” or even basic statistical analysis of that data; before you can start applying your fancy algorithms, you need to get that data in one place, in a tabular format.

Wait, They Haven’t Done That Yet?

Each of these machines in a semiconductor fab probably stores its data locally. [...] ---

Outline:

(00:56) Wait, They Haven’t Done That Yet?

(03:28) Why Data Integration Is Hard

(03:56) Data Access Negotiation, AKA Please Let Me Do The Work You Paid Me For

(10:20) Data Cleaning, AKA I Can’t Use This Junk

(14:00) AI is Gated On Data Integration

---

First published:
September 13th, 2024

Source:
https://www.lesswrong.com/posts/7L8ZwMJkhLXjSa7tD/the-great-data-integration-schlep

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.