I’ve been fascinated recently by historical linguistics, and in particular reconstructed proto-languages such as Proto-Indo-European.

But what even is a “proto-language”?

Well, if we go for the literal operationalization, Wikipedia defines a proto-language as:

a postulated ancestral language from which a number of attested languages are believed to have descended by evolution, forming a language family.

[…]

It is by definition a linguistic reconstruction formulated by applying the comparative method to a group of languages featuring similar characteristics.

So a proto-language is supposed to be the closest common ancestor of a language family (or subfamily), and it is reconstructed using the comparative method (which as the name suggests, works by comparing the different daughter languages to reconstruct (mostly) sound structure (phonology) and word structure (morphology).

But they’re a weird kind of thing.

First, proto-languages are not attested. This means that we have no example of writing in any proto-language. [...]

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

expectation calibrator: freeform draft, posting partly to practice lowering my own excessive standards.

So, a few months back, I finally got around to reading Nick Bostrom's Superintelligence, a major player in the popularization of AI safety concerns. Lots of the argument was stuff I'd internalized a long time ago, reading the sequences and even living with AI alignment researchers for a few months. But I wanted to see the circa-2015 risk-case laid out as thorougly as possible, partly as an excusion to learn more about my own subculture, and partly to see if I'd missed anything crucial to why all of us consider future advanced AI systems so dangerous.

But for the most part, the main effect of reading that book was actually to finally shatter a certain story I'd been telling myself about the AI doom scenario, the cracks in which had been starting to show for several [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is a guest post by my student Ruiqi Zhong, who has some very exciting work defining new families of statistical models that can take natural language explanations as parameters. The motivation is that existing statistical models are bad at explaining structured data. To address this problem, we agument these models with natural language parameters, which can represent interpretable abstract features and be learned automatically.

Imagine the following scenario: It is the year 3024. We are historians trying to understand what happened between 2016 and 2024, by looking at how Twitter topics changed across that time period. We are given a dataset of user-posted images sorted by time, $x_1$, $x_2$ ... $x_T$, and our goal is to find trends in this dataset to help interpret what happened. If we successfully achieve our goal, we would discover, for instance, (1) a recurring spike of images depicting athletes every four [...]

---

Outline:

(04:21) Warm-up Example: Logistic Regression with Natural Language Parameters

(07:12) Formalizing Regression with Natural Language Parameters.

(10:47) A Modeling Language for Specifying More Complex Models

(14:20) Example Applications:

(14:49) Application 1: monitoring trends in LLM usage via time series modeling

(16:01) Application 2: taxonomizing product reviews via clustering

(18:05) Application 3: Explaining memorable visual features via Classification.

(20:12) Conclusion

The original text contained 5 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a collection of every unidentified GPT2 glitch token listed in the third glitch token archaeology post. I was able to find the source of every single one, except for "?????-" and "?????-?????-"[1]. Please tell me if I missed one, or you've discovered one and don't understand where it came from. This isn't meant to be a well-written analysis, just a quick repository of my glitch-hunting observations.

I plan on writing up and categorizing all of these in greater detail in future posts. The first of which is here.

I used OpenWebText, a recreation of GPT2's training data, for all experiments in this post. I tokenized every .gz file in the archive and made a boolean Numpy array of each tokens that was present at least once. This allowed me to quickly identify infrequent tokens in the dataset and pull up the textual context with regular expressions. If [...]

---

Outline:

(02:23) Glitch Tokens and Where They Came From

(02:28) 48193 @#and 25

(03:34) 35496 ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ 26

(06:09) 31727 cffff 156

(07:11) (7131 \]\[ 20471) and (3693 .\[ 20604) and (42669 ).\[ 19013) and (42924 .\[ 19219)

(08:00) 31708 ーン 635

(08:48) 48396 ÛÛ 3

(09:10) 24440 ュ 1338

(09:28) 39165 catentry 4

(09:44) 39253  UCHIJ 5

(10:08) 47182 :},{ 21       //    23785 \]= 32    //     32047  $:/ 3

(10:24) //    47182 :},{ 21

(10:34) 21807 \\\\\\\\\\\\\\\\ 45

(11:45) 17629  practition 13

(13:30) 41441 \\\\- 645

(45:09) 49781 EngineDebug 3

(45:34) 42470 TextColor 97

(46:13) 43177 EStreamFrame 0 |  39906 EStream 0

(46:37) 41383 assetsadobe 0

(47:15) Non-English Languages

(48:21) Hypotheses

(48:24) Why does this happen?

(53:22) Glitch token classifications

(54:38) Future Research Plans

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 7 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We (Flo & Rudolf) spent a month fleshing out the idea of an insurance-for-AI company. We talked to 15 people in the insurance industry, and did 20 customer interviews. We decided not to continue, but we think it's still a very promising idea and that maybe someone else should do this. This post describes our findings.

The idea

Theory of change

To reduce AI risks, it would be good if we understood risks well, and if some organisation existed that could incentivise the use of safer AI practices. An insurance company that sells insurance policies for AI use cases has a financial incentive to understand concrete AI risks & harms well, because this feeds into its pricing. This company would also be incentivised to encourage companies to adopt safer AI practices, and could incentivise this by offering lower premiums in return. Like many cyber-insurance companies, it could also provide [...]

---

Outline:

(00:28) The idea

(00:32) Theory of change

(01:12) Concrete path

(04:55) How insurance works

(04:59) Why insurance

(07:27) Capital efficiency in insurance comes from scale

(09:19) The insurance industry is complicated and allows for startups

(11:19) General liability exceptions are key for new insurance products

(12:10) Our ideas for pricing risk

(15:43) Notes on professional indemnity insurance for law

(17:38) Relevant Stakeholders

(18:04) Findings

(18:07) Customer demand

(19:45) Findings about insurance

(22:52) Cruxes

(24:22) Alternative approaches

(24:26) Labs / Big Tech as reinsurers

(28:13) Selling a risk model rather than an insurance policy

(29:52) Should AI evaluation / auditing orgs do this?

(30:07) Acknowledgements

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Previously Alex Altair and I published a post on the applications of chaos theory, which found a few successes but mostly overhyped dead ends. Luckily the comments came through, providing me with an entirely different type of application: knowing you can’t, and explaining to your boss that you can’t.

Knowing you can’t

Calling a system chaotic rules out many solutions and tools, which can save you time and money in dead ends not traveled. I knew this, but also knew that you could never be 100% certain a physical system was chaotic, as opposed to misunderstood.

However, you can know the equations behind proposed solutions, and trust that reality is unlikely to be simpler[1] than the idealized math. This means that if the equations necessary for your proposed solution could be used to solve the 3-body problem, you don’t have a solution.

[[1] I’m [...]

---

Outline:

(00:25) Knowing you can’t

(02:16) Explaining you can’t

(04:02) Conclusion

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary: I am an independent researcher in agent foundations, and I've recently received an LTFF grant to fund someone to do research with me. This is a rolling application; I'll close it whenever I'm no longer interested in taking another person.

If you're not familiar with agent foundations, you can read about my views in this post.

What the role might be like

This role is extremely flexible. Depending on who you are, it could end up resembling an internship, a research assistant position, a postdoc or even as a mentor/advisor to me. Below, I've listed out the parameters of the fellowship that I am using as a baseline of what it could be. All of these parameters are negotiable!

• $25 per hour. This is not a lot for people who live in the SF Bay area, or who are used to industry salaries, but it looks to [...]

---

Outline:

(00:28) What the role might be like

(03:33) The research problems

(05:27) Application process

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Previous post: How good are LLMs at doing ML on an unknown dataset?

A while back I ran some evaluation tests on GPT4o, Claude Sonnet 3.5 and Gemini advanced to see how good they were at doing machine learning on a completely novel, and somewhat unusual dataset. The data was basically 512 points in the 2D plane, and some of the points make up a shape, and the goal is to classify the data according to what shape the points make up. None of the models did better than chance on the original (hard) dataset, while they did somewhat better on a much easier version I made afterwards.

With the release of o1-preview, I wanted to quickly run the same test on o1, just to see how well it did. In summary, it basically solved the hard version of my previous challenge, achieving 77% accuracy on the test [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

I'm running more 4-day "Cognitive Bootcamps" over the next couple months (during Lighthaven Eternal September season). DM me if you're potentially interested (either as an individual, or as a team).

The workshop is most valuable to people who:

• control their decisionmaking process (i.e. you decide what projects you or a team work on, rather than working at a day-job on someone else's vision)
• are either a) confused about planmaking / have a vague sense that they aren't as strategically ambitious as they could be.
• and/or, b) are at a place where it's natural to spend a few days thinking big-picture thoughts before deciding on their next project.

There's a secondary[1] focus on "practice solving confusing problems", which IMO is time well spent, but requires more followup practice to pay off.

I wrote about the previous workshop here. Participants said on average they'd have been willing to pay $850 [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I’ve claimed that Willpower compounds and that small wins in the present make it easier to get bigger wins in the future. Unfortunately, procrastination and laziness compound, too.

You’re stressed out for some reason, so you take the evening off for a YouTube binge. You end up staying awake a little later than usual and sleeping poorly. So the next morning you feel especially tired; you snooze a few extra times. In your rushed morning routine you don’t have time to prepare for the work meeting as much as you’d planned to. So you have little to contribute during the meeting. You feel bad about your performance. You escape from the bad feelings with a Twitter break. But Twitter is freaking out. Elon Musk said what? Everyone is weighing in. This is going to occupy you intermittently for the rest of the day. And so on.

Laziness has a [...]

---

Outline:

(01:44) I’m spiraling! I’m spiraling!

(02:54) Do what you can

(03:33) A) Emergency recovery

(05:49) B) Natural recovery

(06:14) Wait for a reset point

(08:53) Analyze the circumstances that caused it

(12:17) C) Heroic recovery

(12:50) Deep psych work

(15:16) Conclusion

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.