Thanks to Stag Lynn, Anna Salmon, CFAR, Alex Zhu, Brian Toomey, Kaj Sotala, Damon Sasi, and lots more for all of the help!

Previously: Pay-on-results personal growth: first success

https://x.com/ChrisChipMonk/status/1873101046199038301

To validate my research, I began offering pay-on-results coaching in July. Clients have paid $40,300 so far upon achieving their goals:

1. lifelong anxiety → being called “adventurous”: $3,000 earned
2. lifelong pushover → assertive: $3,000 earned
3. “in crisis” → “light and hopeful”: $5,000 earned
4. blocked on romance whole life → asking out crushes: $3,500 earned
5. “net suffering has greatly diminished”: $7,500 earned
6. first week ever of no procrastination: $7,500 earned
7. eliminating an unwanted habit: $2,000 earned
8. increased flow during work: $7,500 earned
9. ......... (more cooking)

(I spend just ~8 hours per week coaching.)

An innovation here is a way to measure results: Results are measured by bounties paid by clients one or more months after, in my bank [...]

---

Outline:

(02:15) Why did I start doing bounties?

(06:02) 21 more unstructured thoughts:

(15:08) Place a bounty:

---

First published:

December 29th, 2024

Source:

https://www.lesswrong.com/posts/mbwgdfnz6tPeTgzMa/began-a-pay-on-results-coaching-experiment-made-usd40-300

---

Narrated by TYPE III AUDIO.

Tl;dr I've decided to shift my research from mechanistic interpretability to more empirical ("prosaic") interpretability / safety work. Here's why.

All views expressed are my own.

What Really Interests Me

I care about understanding how powerful AI systems think internally. I'm drawn to high-level questions ("what are the model's goals / beliefs?") as opposed to low-level mechanics ("how does the model store and use [specific fact]?"). Sure, figuring out how a model does modular addition is cool, but only insofar as those insights and techniques generalise to understanding higher-level reasoning.

Mech Interp Has Been Disappointing 

Vis-a-vis answering these high-level conceptual questions, mechanistic interpretability has been disappointing. IOI remains the most interesting circuit we've found in any language model. That's pretty damning. If mechanistic interpretability worked well, we should have already mapped out lots of interesting circuits in open-source 7B models by now.

The field seems conceptually bottlenecked. [...]

---

Outline:

(00:19) What Really Interests Me

(00:46) Mech Interp Has Been Disappointing

(01:25) Doing mech interp research led me to update against it

(02:54) Prosaic Interpretability

(04:11) Intuition pump: Gene analysis for medicine

(05:08) Modern AI Systems will make interpretability difficult

(05:54) The Timing is Frustrating

(06:50) Personal Fit

(08:21) Mech Interp Research that Excites Me

(09:01) Looking Forward

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

December 30th, 2024

Source:

https://www.lesswrong.com/posts/Ypkx5GyhwxNLRGiWo/why-i-m-moving-from-mechanistic-to-empirical

---

Narrated by TYPE III AUDIO.

Related to: On green; Hierarchical agency; Why The Focus on Expected Utility Maximisers?

Sometimes LLMs act a bit like storybook paperclippers (hereafter: VNM-agents[1]), e.g. scheming to prevent changes to their weights. Why? Is this what almost any mind would converge toward once smart enough, and are LLMs now beginning to be smart enough? Or are such LLMs mimicking our predictions (and fears) about them, in a self-fulfilling prophecy? (That is: if we made and shared different predictions, would LLMs act differently?)[2]

Also: how about humans? We humans also sometimes act like VNM-agents – we sometimes calculate our “expected utility,” seek power with which to hit our goals, try to protect our goals from change, use naive consequentialism about how to hit our goals.

And sometimes we humans act unlike VNM-agents, or unlike our stories of paperclippers. This was maybe even more common historically. Historical humans often mimicked social patterns [...]

The original text contained 7 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

December 30th, 2024

Source:

https://www.lesswrong.com/posts/KSguJeuyuKCMq7haq/is-vnm-agent-one-of-several-options-for-what-minds-can-grow

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Follow up to: Could orcas be smarter than humans?

(For speed of writing, I mostly don't cite references. Feel free to ask me in the comments for references for some claims.)

This post summarizes my current most important considerations on whether orcas might be more intelligent than humans.

Evolutionary considerations

What caused humans to become so smart?

(Note: AFAIK there's no scientific consensus here and my opinions might be nonstandard and I don't provide sufficient explanation here for why I hold those. Feel free to ask more in the comments.)

My guess for the primary driver of what caused humans to become intelligent is the cultural intelligence hypothesis: Humans who were smarter were better at learning and mastering culturally transmitted techniques and thereby better at surviving and reproducing.

The book "the secret of our success" has a lot of useful anecdotes that show the vast breath and complexity of [...]

---

Outline:

(00:24) Evolutionary considerations

(00:29) What caused humans to become so smart?

(01:36) Is it evolutionarily plausible that orcas became smarter?

(03:54) Behavioral evidence

(05:30) Evidence from wild orcas

(06:21) Evidence from orcas in captivity

(07:06) Thoughts on orca languages

(08:12) Louis Hermans research on teaching bottlenose dolphins language understanding

(09:53) Neuroscientific considerations

(09:57) Orca brain facts

(10:37) Interspecies correlations between cortical neurons and behavioral signs of intelligence

(12:08) How much does scale vs other adaptations matter?

(14:40) Overall guess

(15:21) Aside: Update on my project

The original text contained 9 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

December 29th, 2024

Source:

https://www.lesswrong.com/posts/dzLwCBvwC4hWytnus/considerations-on-orca-intelligence

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

2024 is drawing to a close, which means it's an opportune time to make predictions about 2025. It's also a great time to put probabilities on those predictions, so we can later prove our calibration (or lack thereof).

We just shipped a LessWrong feature to make this easy. Simply highlight a sentence in your comment, and click the crystal-ball icon on the toolbar to turn it a prediction that everyone (who's logged in) can put probability estimates on. The result will look like this:

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

Some more probabilities that seem cool to elicit (basically all about AI, because that's what's on my mind, but it would be great to have some less AI focused predictions from others)[1]:

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%1%2%3%4%5%6%7%8%9%10%11%12%13%14%15%16%17%18%19%20%21%22%23%24%25%26%27%28%29%30%31%32%33%34%35%36%37%38%39%40%41%42%43%44%45%46%47%48%49%50%51%52%53%54%55%56%57%58%59%60%61%62%63%64%65%66%67%68%69%70%71%72%73%74%75%76%77%78%79%80%81%82%83%84%85%86%87%88%89%90%91%92%93%94%95%96%97%98%99%1%99%

https://www.lesswrong.com/posts/7Hr6FCYnZJTiCKmLj/new-year-predictions-thread-2024-25-2

---

Narrated by TYPE III AUDIO.

To many people, the land value tax (LVT) has earned the reputation of being the "perfect tax." In theory, it achieves a rare trifecta: generating government revenue without causing deadweight loss, incentivizing the productive development of land by discouraging unproductive speculation, and disproportionately taxing the wealthy, who tend to own the most valuable land.

That said, I personally think the land value tax is overrated. While I'm not entirely against it—and I think that several of the arguments in favor of it are theoretically valid—I think the merits of the LVT have mostly been exaggerated, and its downsides have largely been ignored or dismissed for terrible reasons. In my view the LVT is at best a useful but highly limited tool ("the worst tax policy ever, except for all the others that have been tried"); at worst, it is a naive proposal that creates many more problems than it [...]

---

Outline:

(02:21) Core problems with the LVT

(03:18) An LVT discourages searching for new uses of land

(05:36) An LVT implicitly taxes improvements to nearby land

(08:48) Cant the LVT simply be patched to address these issues?

(13:31) The government has incentives to inflate their estimates of the value of unimproved land

(15:44) An LVT is unlikely to replace many existing taxes

(17:07) Disruptions under the LVT

(17:11) The precedent set by a full LVT

(20:41) An LVT would massively disrupt millions of peoples long-term plans

(23:08) The purported effect of an LVT on unproductive land speculation seems exaggerated

(26:32) Final words

The original text contained 1 image which was described by AI.

---

First published:

December 29th, 2024

Source:

https://www.lesswrong.com/posts/CCuJotfcaoXf8FYcy/some-arguments-against-a-land-value-tax

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

from aisafety.world

The following is a list of live agendas in technical AI safety, updating our post from last year. It is “shallow” in the sense that 1) we are not specialists in almost any of it and that 2) we only spent about an hour on each entry. We also only use public information, so we are bound to be off by some additional factor.

The point is to help anyone look up some of what is happening, or that thing you vaguely remember reading about; to help new researchers orient and know (some of) their options; to help policy people know who to talk to for the actual information; and ideally to help funders see quickly what has already been funded and how much (but this proves to be hard).

“AI safety” means many things. We’re targeting work that intends to prevent very competent [...]

---

Outline:

(01:33) Editorial

(08:15) Agendas with public outputs

(08:19) 1. Understand existing models

(08:24) Evals

(14:49) Interpretability

(27:35) Understand learning

(31:49) 2. Control the thing

(40:31) Prevent deception and scheming

(46:30) Surgical model edits

(49:18) Goal robustness

(50:49) 3. Safety by design

(52:57) 4. Make AI solve it

(53:05) Scalable oversight

(01:00:14) Task decomp

(01:00:28) Adversarial

(01:04:36) 5. Theory

(01:07:27) Understanding agency

(01:15:47) Corrigibility

(01:17:29) Ontology Identification

(01:21:24) Understand cooperation

(01:26:32) 6. Miscellaneous

(01:50:40) Agendas without public outputs this year

(01:51:04) Graveyard (known to be inactive)

(01:52:00) Method

(01:55:09) Other reviews and taxonomies

(01:56:11) Acknowledgments

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

December 29th, 2024

Source:

https://www.lesswrong.com/posts/fAW6RXLKTLHC3WXkS/shallow-review-of-technical-ai-safety-2024

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Two years ago, I noted that we had clearly entered the era of general intelligence, but that it was "too soon" to expect widespread social impacts.

In the last 2 years, AI has gone from the green line to the orange line

In those 2 years, AI development has followed the best possible of the 3 paths I suggested (foom/GPT-4-takes-my-job/Slow Takeoff). Returns to scale seem to be delivering a steady ~15 IQ points/year and cutting edge models appear to be largely a compute-intensive project that allows (relatively) safety-conscious leading labs to explore the new frontiers while others reap the benefits with ~1 year delay.

Possibly the most important graph in the world right now

If I had to identify 3 areas where GPT-3.5 was lacking, it would have been:

• reasoning
• modeling the real world
• learning on-the-fly

Of those three, reasoning (o3) is largely solved and we have promising approaches for [...]

---

First published:

December 29th, 2024

Source:

https://www.lesswrong.com/posts/ocvpbiAJsxkB3Hiaj/what-happens-next

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In The Mystery Of Internet Survey IQs, Scott revises his estimate of the average LessWrong IQ from 138 to 128. He doesn’t explicitly explain how he arrived at this number, but it appears to be an average of the demographics norm method (123) and the SAT method (134). However, using the information in his post, the SAT method doesn’t actually yield 134 but rather 123.

Here's the breakdown: a median SAT score of 1490 (from the LessWrong 2014 survey) corresponds to +2.42 SD, which regresses to +1.93 SD for IQ using an SAT-IQ correlation of +0.80. This equates to an IQ of 129. Subtracting 6 points (since, according to the ClearerThinking test, the IQs of people who took the SAT and remember their score is ~6 points higher than the group average) brings the adjusted IQ estimate to 123.

The ClearerThinking test also provides a way to [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

December 28th, 2024

Source:

https://www.lesswrong.com/posts/hEdTLjvWzJPcS7PpY/the-average-rationalist-iq-is-about-122

---

Narrated by TYPE III AUDIO.

I've heard many people say something like "money won't matter post-AGI". This has always struck me as odd, and as most likely completely incorrect.

First: labour means human mental and physical effort that produces something of value. Capital goods are things like factories, data centres, and software—things humans have built that are used in the production of goods and services. I'll use "capital" to refer to both the stock of capital goods and to the money that can pay for them. I'll say "money" when I want to exclude capital goods.

The key economic effect of AI is that it makes capital a more and more general substitute for labour. There's less need to pay humans for their time to perform work, because you can replace that with capital (e.g. data centres running software replaces a human doing mental labour).

I will walk through consequences of this, and end [...]

---

Outline:

(03:10) The default solution

(04:18) Money currently struggles to buy talent

(09:15) Most peoples power/leverage derives from their labour

(09:41) Why are states ever nice?

(14:32) No more outlier outcomes?

(20:27) Enforced equality is unlikely

(22:34) The default outcome?

(26:04) Whats the takeaway?

---

First published:

December 28th, 2024

Source:

https://www.lesswrong.com/posts/KFFaKu27FNugCHFmh/by-default-capital-will-matter-more-than-ever-after-agi

---

Narrated by TYPE III AUDIO.