Seb Krier on AGI, Scaffolding, and Coasean Bargaining at Scale
In this episode of Justified Posteriors, we welcome Seb Krier — policy lead for AGI at Google DeepMind and excellent Twitter poster. Speaking in his personal capacity, Seb walks us through his understanding of AGI, why AI alignment has gone better than expected, the potential and limitations of a world where agents constantly barter on our behalf, and — of course — electronic music.
We also cover AI in London vs. New York, how Seb went from reading Marginal Revolution for 15 years to becoming a recurring character on it, and Seb’s side-splitting humor on mediocre AI conferences.
Related Links
* Seb Krier on X: @sebkrier
* Seb’s Substack, Technologik
* “Coasean Bargaining at Scale” — Seb’s essay at the Cosmos Institute (also republished here)
* “Musings on Recursive Self-Improvement” — Seb’s essay separating model-side RSI from societal-side
* “The Cyborg Era: What AI Means for Jobs” — Seb’s guest essay on Alex Imas’s Substack, defending the scaffolding view
* Anthropic’s Project Deal — the agent-bargaining experiment among Anthropic employees
* Fradkin & Krishnan, “MarketBench” — Andrey and Rohit experiment of LLMs bidding in procurement auctions as an investigation of the future of AI marketplaces and the companion writeup: Rohit Krishnan, “Agent, Know Thyself! (and bid accordingly)”
* Edge Esmeralda — Devon Zuegel’s pop-up village in Healdsburg, CA
* MATS — for junior economists looking to skill up on AI safety/governance
* Cosmos Institute and FIRE
* bianjie.systems — the art platform Seb is co-organizing a dinner with in NY (Seb’s announcement)
* Drexciya — James Stinson, Gerald Donald, and the Detroit electro-afrofuturism canon
Timestamps
(00:00) Intro (01:16) What is AGI? (07:30) In defense of scaffolding — Hayek, division of labor, and why one giant model won’t do it (13:00) Markets for cognition: will agents bid in procurement auctions? (18:40) Recursive self-improvement — separating the model side from the societal side (24:44) Alignment has gone better than 2017-Seb expected; prefer “intent following” (31:14) What economists should actually work on to inform AI labs(33:32) What does a DeepMind policy lead’s day look like? (38:20) AI Conferences(41:52) Coasean bargaining at scale — the positive vision(55:00) Inequality, property rights, and who gets the initial allocation (01:03:00) The Helldivers 2 “Managed Democracy” dystopia as Coasean bargaining gone wrong (01:09:00) Sponsor: Revelio Labs (01:09:30) Lightning round
Justified Posteriors is a reader-supported publication. To receive new posts and support our work, consider becoming a free or paid subscriber. You’re also invited to our discord community at: https://discord.gg/b8VpPbBUt
Transcript
00:00:00,100 --> 00:00:20,480 [Seth]
[upbeat music] Welcome to the Justified Posterior’s podcast, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, the number two biggest fan, after Tyler Cowen, in the Seb Krier fan club.
00:00:20,480 --> 00:00:20,740 [Andrey]
[laughs]
00:00:20,740 --> 00:00:24,660 [Seth]
Coming to you from Chapman University in sunny southern California.
00:00:24,660 --> 00:00:34,120 [Andrey]
And I’m Andrey Fradkin, coming to you from San Francisco, California. And Justified Posterior’s is sponsored by the fine folks at Revelio Labs.
00:00:35,560 --> 00:00:45,600 [Andrey]
We’re very excited to have Seb Krier here with us today. He is the policy lead for AGI at Google DeepMind, and is,
00:00:46,840 --> 00:00:52,400 [Andrey]
dare I say, a thought leader in this space. Welcome to the show, Seb.
00:00:52,400 --> 00:00:54,200 [Seb Krier]
Thank you very much. It’s great to be here.
00:00:55,380 --> 00:00:58,160 [Seb Krier]
Yeah, I’m Seb, calling in from New York.
00:00:58,160 --> 00:01:00,320 [Andrey]
And we should remind our listeners that
00:01:01,340 --> 00:01:08,410 [Andrey]
Seb is, during this podcast, expressing his personal opinions, and is not speaking on behalf of DeepMind. All right.
00:01:08,410 --> 00:01:09,740 [Seb Krier]
Indeed. [laughs]
00:01:09,740 --> 00:01:11,060 [Andrey]
[laughs]
00:01:12,780 --> 00:01:13,900 [Andrey]
The usual caveat.
00:01:15,260 --> 00:01:16,760 [Andrey]
Seb, what is AGI?
00:01:18,080 --> 00:01:19,450 [Seb Krier]
What is AGI? [laughs]
00:01:19,450 --> 00:01:19,570 [Andrey]
[laughs]
00:01:19,570 --> 00:01:19,580 [Seth]
[laughs]
00:01:19,580 --> 00:01:19,780 [Seb Krier]
Great question.
00:01:19,780 --> 00:01:21,900 [Andrey]
We’re going to start with the big questions.
00:01:21,900 --> 00:01:22,880 [Seb Krier]
Yeah, might as well.
00:01:24,259 --> 00:01:54,840 [Seb Krier]
[sighs] I think there’s so many definitions out there of what AGI is, and I think most of them are kind of unsatisfactory in one way or another. I’ve seen stuff like many definitions are indexed on the societal transformations or economic impacts of the technology, which I don’t really like very much because it makes it very dependent on external factors whether or not we have AGI. If it’s banned, we don’t have AGI, and if it’s not banned, we have AGI. Is it?
00:01:54,840 --> 00:01:55,480 [Andrey]
[laughs]
00:01:55,480 --> 00:02:04,670 [Seb Krier]
And there are other tests, like if an AI makes $1 million or something, which I find is very weird because most humans do not make $1 million in the first place.
00:02:04,670 --> 00:02:05,080 [Andrey]
[laughs]
00:02:05,080 --> 00:02:11,359 [Seb Krier]
So the one I kind of like is actually Shane Legg’s definition-
00:02:11,360 --> 00:02:11,620 [Andrey]
Mm
00:02:11,620 --> 00:02:12,420 [Seb Krier]
... who’s at Deep Mind, who is
00:02:13,640 --> 00:02:16,980 [Seb Krier]
more of a capability-based definition, which is something along the lines of
00:02:18,420 --> 00:02:20,960 [Seb Krier]
an AI or a system that does most
00:02:22,380 --> 00:02:30,360 [Seb Krier]
standard cognitive tasks that people typically do. [lips smack] So it’s kind of the bar isn’t too low, and it’s also not too high either.
00:02:32,220 --> 00:02:35,480 [Seb Krier]
And so I think he’s got this definition of a minimal AGI,
00:02:36,580 --> 00:02:43,020 [Seb Krier]
and I think that we’re not exactly there yet. I would disagree with people saying that we have AGI today because I think
00:02:44,220 --> 00:02:48,900 [Seb Krier]
a lot of the systems we have, there’s many things that a human can do that they don’t really do very well.
00:02:48,900 --> 00:02:50,360 [Seth]
What’s the biggest gap that we’re missing?
00:02:52,020 --> 00:03:47,740 [Seb Krier]
I’d say there’s a few. One of them might be continual learning, or at least the ability to adapt and learn over time, and in different contexts and situations, just kind of update your own world model or whatever. If I think of a new joiner in a company, they’re not super useful the first day, but their value goes up over time because they learn all sorts of things. And so [lips smack] that might be one of them. A lot of the systems we have today, I think, are not very good at software, and you’re using graphical user interfaces and software and whatnot. If I ask an agent right now to go and use a music production software and make a track, I think they’d generally struggle. That doesn’t mean it’s impossible to solve or anything like that, but I think, in many respects, they’re not as general as you’d want them to be. And then the other bit also is, [lips smack] and of course they still make some silly mistakes here and there, but I think that’s getting it fixed. But the creativity point is one that I’m really interested in as well, in that I think they’re really good at kind of
00:03:48,780 --> 00:04:02,700 [Seb Krier]
exploiting maybe an existing paradigm or an existing knowledge and so on, and recombining knowledge and whatnot. But I think really coming up with new concepts and abstractions entirely is something I think humans can do, but I don’t see our current systems really doing either.
00:04:02,700 --> 00:04:10,060 [Andrey]
How do you measure whether humans can do creative tasks? One of the things that
00:04:11,200 --> 00:04:15,940 [Andrey]
strikes me as a bit of an unfair test in that,
00:04:17,060 --> 00:04:23,290 [Andrey]
let’s say you ask an LLM to write a poem or to write a story. It’s very-
00:04:23,290 --> 00:04:23,290 [Seth]
[laughs]
00:04:23,290 --> 00:04:32,050 [Andrey]
... times more entertaining than what a random human would write. So, do you have a benchmark for creativity?
00:04:32,050 --> 00:04:35,390 [Seth]
This is the meme where the robot asks Will Smith if he can compose an opera.
00:04:35,390 --> 00:05:14,700 [Seb Krier]
[laughs] Can you? Yeah, exactly. It depends, and you’re right. Obviously, most people aren’t creating new abstraction and concepts on a day-to-day level. But I imagine there’s still something qualitative about that kind of creativity that I think does get applied in everyone’s day-to-day life in various kind of ways. Maybe they’re not as big or significant as creating a symphony. But I don’t really have a strong test. There’s actually an interesting podcast that had Ben Goertzel and Yoshua, I think a few years ago, where they were saying something like, if you had a model that was trained knowing only classical music and West African drumming, could it come up with jazz in the first place, or recreate jazz?
00:05:16,460 --> 00:05:27,880 [Seb Krier]
And I quite like that test. And in principle, I can imagine it being possible. You could kind of decompose all sorts of different kind of elements and variables here and just get something jazz-like. But it still feels a bit...
00:05:29,580 --> 00:05:40,580 [Seb Krier]
It’s not the same as just coming up with the idea of jazz in the first place and saying, oh, I’m going to try these things out. And for whatever reason, I’m going to stick to that. And I don’t know. It’s-
00:05:40,580 --> 00:05:53,190 [Seth]
Recombination versus paradigm shifting. I’ve also heard one test people would want for AGI is, can you train the model on the 1900s corpus and it comes up with Einsteinian physics?
00:05:53,190 --> 00:05:53,200 [Seb Krier]
Yeah.
00:05:53,200 --> 00:05:54,720 [Seth]
That would be really impressive.
00:05:54,720 --> 00:06:36,151 [Seb Krier]
Yeah, I think actually Demis uses that test sometimes, or I think Pele Gritzer as well mentioned it before. And there are some people, I think David Duvenour and Nick Levine, I think, had this recent kind of language model talky that was trained up in, I think, the 1930s or something. And I tried to play around with it a lot. It was like, let’s try to get it to create something new, and it’s pretty tricky. Although they have apparently recently, some people kind of fine-tuned it on a very few examples of coding and gotten it to be good at coding. But for some reason, that doesn’t impress me maybe as much as other things I would’ve expected. It’s like [laughs] there’s the-I agree that the goalposts also kind of move a little bit over time, and it’s also maybe unfair of me. It’s like, oh, well, can it create a new programming language from scratch or something?
00:06:37,272 --> 00:06:43,052 [Seb Krier]
So it’s a tricky one to kind of square off, but it does still feel like there’s a lack of that kind of true creativity, at least in my
00:06:44,212 --> 00:06:45,072 [Seb Krier]
interactions with them.
00:06:46,392 --> 00:06:57,342 [Andrey]
I am really worried that it is a goalpost moving exercise here. We don’t have a benchmark for creativity and therefore,
00:06:58,432 --> 00:07:03,211 [Andrey]
all these claims are not quantitative in a way that I’d like. And let-
00:07:03,212 --> 00:07:10,612 [Seth]
Right. What about all those IS papers we see where one of the axes is creativity and we instrument for something? [laughs]
00:07:10,612 --> 00:07:11,032 [Andrey]
Yes.
00:07:13,132 --> 00:07:13,592 [Seth]
There’s a lot of bad measures of creativity.
00:07:13,592 --> 00:07:19,762 [Andrey]
Those are not creative, to be clear. I’m sure I’ve offended a ton of people. Sorry.
00:07:19,762 --> 00:07:20,992 [Seth]
It’s okay.
00:07:20,992 --> 00:07:56,432 [Seb Krier]
I think it’s fair. I agree that it’s a bit like... But I still feel like there’s, at least if part of the reason you’re going to create these systems is to come up with kind of also new sorts of theories and so on. And I think you can probably get that through good search and a lot of inference compute and trying out lots of different things. And I think there are many low-hanging fruits there, to be clear. So it’s not like I think, oh, we’ve hit some sort of wall or something. And I think there’s a lot that you can kind of get in terms of new knowledge and new creative knowledge from that. But I feel like there’s maybe something more needed. It’s maybe not that kind of magical or anything, right? Maybe you just need better scaffolding or better multi-agent systems. But
00:07:58,992 --> 00:08:02,072 [Seb Krier]
yeah, at least so far, I would say that I see a bit more creativity, say, in
00:08:03,652 --> 00:08:11,612 [Seb Krier]
humans so far as a collective. And maybe that’s, again, an unfair comparison. You don’t have a culture of AIs and AGIs to compare that against. So-
00:08:11,612 --> 00:08:11,682 [Andrey]
Yeah
00:08:11,682 --> 00:08:15,092 [Seb Krier]
... the right comparison is also a hard one to do.
00:08:15,092 --> 00:08:52,772 [Andrey]
So, you mentioned scaffolding, and I guess a question, you recently wrote about a defense of scaffolding, and I think just to frame things, some people you talk with, especially very AGI-pilled people, are like, “Scaffolding, it’s an epiphenomenon. It doesn’t matter. In the end, we are going to train a smarter model with more parameters and more training data, and it’s just going to do it out of the box. And so all these scaffolding hacks are just very temporary.” And then other people like yourself, I guess, argue the opposite. So what do you think about scaffolding?
00:08:54,832 --> 00:08:55,052 [Seb Krier]
Yeah.
00:08:56,572 --> 00:08:59,372 [Seb Krier]
The first thing is I’m definitely not sure. This is kind of
00:09:00,532 --> 00:09:39,672 [Seb Krier]
one of many hot takes, but I think, I guess there are a few reasons why I see it as, I think it’s going to stay over time. The first is that I think it’s plausible that as, I think scaling laws continue, I think you scale models and they get better over time and so on, but I think the inputs are expensive and grow over time. And I also think that it’s plausible that you might get more and more diminishing returns over time. And if that’s the case, I see the kind of utility of the scaffolding side and the harnesses as going up because you’re going to want to make more, you’ll want more bang for your buck kind of thing. You’re going to want to extract this intelligence and use this resource as efficiently as possible.
00:09:40,772 --> 00:09:51,532 [Seb Krier]
So that’s maybe one reason. The other one is a bit more, I guess, Hayekian in nature or something, in that I see a lot of, I think there’s a lot of local knowledge, a lot of
00:09:53,212 --> 00:10:18,592 [Seb Krier]
stuff that isn’t necessarily kind of codified. And I don’t really see one big giant AGI model now kind of perfectly guessing everything forever at infinite scales. And in a way, I see this as a little bit like a division of labor in that I think it’s actually more efficient to have this kind of integration layer that is closer to the local information or to the ground or to demand side that can better integrate this kind of cognitive resource
00:10:19,812 --> 00:10:23,632 [Seb Krier]
to satisfy and create value and satisfy whatever consumers and businesses want.
00:10:25,552 --> 00:10:31,352 [Seb Krier]
So to help with all the sorts of constraints and the context they’re dealing with, I think it’s very useful to have that.
00:10:33,712 --> 00:10:39,112 [Seb Krier]
Of course, I don’t think this necessarily also implies or means that you’re going to get complete, full decentralization or something.
00:10:40,772 --> 00:10:42,212 [Seb Krier]
Walmart gets huge
00:10:43,872 --> 00:10:48,872 [Seb Krier]
returns from the scale that they have, and you don’t have loads of businesses downstream kind of reselling their stuff.
00:10:51,252 --> 00:10:53,932 [Seb Krier]
But there’s two things. The first is that-
00:10:53,932 --> 00:10:56,812 [Seth]
We have bodegas reselling stuff from Walmart on the corner.
00:10:56,812 --> 00:11:18,992 [Seb Krier]
Actually, that’s a good point, yeah. And also, there are all sorts of other businesses kind of selling different things, right? If the task is generic and the demand is homogenous, then sure, maybe you can do more of that. But also, even Walmart relies on all sorts of kind of suppliers, local labor, compliance system, inventory systems, third parties, and whatnot, that help with this kind of integration and the delivery of these services.
00:11:18,992 --> 00:11:25,862 [Seth]
So if I may summarize your answer, you’re very Hayek-pilled, but maybe not as Bitterlesson-pilled as most.
00:11:25,862 --> 00:11:25,972 [Seb Krier]
Well,
00:11:27,212 --> 00:11:31,052 [Seb Krier]
I think I’m definitely Bitterlesson-pilled in the sense that I don’t think you should
00:11:33,652 --> 00:11:48,992 [Seb Krier]
try to kind of cement some sort of rules-based system you either devise or something and kind of hope that this just takes forever. If anything, I think the scaffold needs to be a lot more adaptive and evolve over time. In the same way as if you have a small startup and they have all sorts of kind of rules and,
00:11:50,332 --> 00:12:02,772 [Seb Krier]
sorry, not rules, different functions. When the startup grows and gets more capabilities, they also kind of change from the inside. So I think that, of course, if you have some sort of light GPT-type wrapper that kind of makes your system a little bit better, whatever, yeah, that was not going to
00:12:03,812 --> 00:12:23,652 [Seb Krier]
work out over time. But I think there are kind of scaffolds that help better integrate the wider environment, private data, deals with permissions or liability regimes or user preferences and whatnot. And also, at a somewhat higher level, kind of more coordination-type scaffolds maybe in terms of market interfaces, like clearing house equivalents or something.
00:12:24,516 --> 00:12:33,536 [Seth]
The third example you gave is maybe it’s not the super frontier model that are going to these scaffolds, but simpler models that are still very useful and cheaper to run with a scaffold.
00:12:33,536 --> 00:12:46,176 [Seb Krier]
Yeah, totally. Because I think you’re not going to need the enormous, super expensive brain for every single random task. And so it’ll make, for most kind of basic queries, people aren’t using Opus’s latent space or something as-
00:12:46,176 --> 00:12:46,186 [Seth]
[laughing]
00:12:46,186 --> 00:12:48,236 [Seb Krier]
... it’s a big waste in some sense.
00:12:48,236 --> 00:12:50,036 [Seth]
What toothbrush should I buy? [chuckles]
00:12:50,036 --> 00:12:51,196 [Seb Krier]
Yeah. Exactly.
00:12:51,196 --> 00:12:53,896 [Andrey]
Wait. That is an important question, Seth.
00:12:53,896 --> 00:12:54,516 [Seb Krier]
I mean-
00:12:54,516 --> 00:12:56,536 [Andrey]
I would definitely use Opus for that.
00:12:56,536 --> 00:12:57,385 [Seb Krier]
It’s funny because I’ve actually-
00:12:57,385 --> 00:12:59,696 [Seth]
Use all the collective intelligence of reality. [chuckles]
00:12:59,696 --> 00:13:02,266 [Seb Krier]
I have actually used Opus for that exact question not long ago-
00:13:02,266 --> 00:13:02,626 [Seth]
[laughing]
00:13:02,626 --> 00:13:06,256 [Seb Krier]
... in trying out this new electric toothbrush that I found out as a result. But,
00:13:07,636 --> 00:13:22,076 [Seb Krier]
so yeah, I agree there’s that and also there’s all sorts of ways in which actually kind of using tools or specialized kind of tools is just more effective and more efficient. Why would you expect a large model or something to kind of calculate things innately or something when you can just access a calculator? It’s a much better use of tokens.
00:13:22,076 --> 00:13:36,856 [Andrey]
But it should kind of know that the calculator is available and then use it when it’s there. So that’s the argument against scaffolding, or you’re giving it a general environment, but you’re not scaffolding it much. I think a curious thing is just,
00:13:38,376 --> 00:13:40,356 [Andrey]
it seems like most people who are using
00:13:41,416 --> 00:13:49,156 [Andrey]
scaffolded agents today are using them with essentially one of two scaffolds, with Cloud Code or Codex. And
00:13:50,236 --> 00:14:00,475 [Andrey]
those seem to be good enough maybe. I guess, do we see a lot of people customizing, a lot of people, whatever, companies customizing their scaffolds?
00:14:00,476 --> 00:14:03,856 [Seth]
CladBot, do the CladBots count as that, I guess?
00:14:03,856 --> 00:14:04,236 [Andrey]
Yeah.
00:14:05,396 --> 00:14:39,676 [Seb Krier]
They are a form of it. I don’t know. I think a lot of power users and people in our immediate communities use a lot of Cloud Code and Codex, and particularly software engineers. But I don’t think most legal departments and most kind of firms out there are necessarily using Cloud Code either. And it’s not clear to me that this is necessarily the optimal interface or, there may be better systems that are Cloud Code-like, or CLI-like perhaps in some way. But, so I don’t know, maybe they’re sufficient, but even these tools end up kind of calling on loads of other external APIs and tools and so on in how they
00:14:40,836 --> 00:14:57,576 [Seb Krier]
function. So if anything, these are actually scaffolds. You’re not kind of calling the model directly. There’s all sorts of different sub-agents behind the scenes. It’s not just a one-shot call. There’s quite a lot going on, which is in fact this more, I don’t know, dynamic scaffolding thing I was mentioning earlier, I guess.
00:14:58,976 --> 00:15:06,736 [Andrey]
Okay. The natural question here is, what is going to be the role of the market in coordinating-
00:15:06,736 --> 00:15:07,375 [Seb Krier]
Mm
00:15:07,375 --> 00:15:11,276 [Andrey]
... AI here? And I’ll just very shamelessly plug-
00:15:11,276 --> 00:15:11,285 [Seb Krier]
[chuckles]
00:15:11,285 --> 00:15:24,796 [Andrey]
... some recent work with Rohit Krishnan, where we’re kind of playing around with the idea of LLMs bidding in a procurement auction and seeing whether that results in more efficient use of AI.
00:15:26,696 --> 00:15:29,655 [Seb Krier]
Well, first of all, I need to properly read that again. But the-
00:15:29,655 --> 00:15:30,476 [Andrey]
[laughing]
00:15:30,476 --> 00:15:31,016 [Seb Krier]
In terms of,
00:15:32,496 --> 00:15:32,916 [Seb Krier]
I guess,
00:15:34,556 --> 00:15:46,396 [Seb Krier]
at a very high level, markets are good at just coordinating in general, including AI. And so, assuming they function as intended in it, you’ve got the pricing mechanism to get...
00:15:47,556 --> 00:15:49,396 [Seb Krier]
I don’t know. I expect that to kind of work as well with
00:15:50,476 --> 00:15:52,616 [Seb Krier]
matching, I guess, supply and demand or something.
00:15:54,016 --> 00:15:55,196 [Seb Krier]
The supply of this
00:15:56,216 --> 00:16:00,036 [Seb Krier]
raw resource of cognition or something, and the demand of all sorts of different businesses and users.
00:16:01,696 --> 00:16:05,516 [Seb Krier]
So maybe, at a very high level, I don’t know. What exactly do you mean by the role of the market or something here?
00:16:09,076 --> 00:16:21,356 [Andrey]
Obviously the market is involved in many parts of the AI vertical supply chain, right? From competition in chips. There’s competition between models. There might be also competition between
00:16:22,516 --> 00:16:28,576 [Andrey]
scaffolds, bundles of environments, scaffolds, and LLMs.
00:16:28,576 --> 00:17:06,496 [Seth]
I guess maybe it would be useful to juxtapose this versus, so what Andrey, one of the things he’s imagining is, I have a job. I post it to some sort of Upwork-like future platform. Different companies that host different AI models bid to do that job. “Oh, I think I can do that job with $1 of electricity and tokens,” versus another model, and then we get efficient allocation of intellectual tasks to models, right? So do we think that that’s going to be important, or is it going to be more like I ask the super model what the best model is, and I just get allocated in a non-market way? Might be one version of this question.
00:17:08,156 --> 00:17:18,836 [Seb Krier]
I guess intuitively, my mind goes to the former question. But, or there’s a little bit of both in some sense, because even in the former one, you’re going to be using the large model for some sort of
00:17:20,436 --> 00:17:26,686 [Seb Krier]
cognitively demanding task or something. It kind of depends what kind of quality of output you also need and want.
00:17:26,686 --> 00:17:26,706 [Seth]
[chuckles]
00:17:26,706 --> 00:17:27,056 [Seb Krier]
But then
00:17:28,376 --> 00:17:49,636 [Seb Krier]
you’re still going to be constrained by your own resources or something, and depending on what you have to spend, if you can get the output for cheaper by kind of relying on this kind of competitive marketplace of smaller models or something, not even smaller models, they might just be all be big and kind of just scaffolding different, you’re offering a slightly different thing. Why wouldn’t you go for that, and why wouldn’t that exist in the first place? Unless the very first-
00:17:49,636 --> 00:17:52,216 [Andrey]
Doesn’t exist yet, just to be clear.
00:17:52,216 --> 00:17:52,716 [Seb Krier]
Um-
00:17:52,716 --> 00:17:58,416 [Seth]
A, it doesn’t exist yet, and as Andrey proves, at least current models are bad at understanding their own capabilities.
00:17:58,416 --> 00:17:58,666 [Andrey]
Oh, yeah.
00:17:58,666 --> 00:18:00,496 [Seth]
Now maybe that’s going to be fixed.
00:18:00,496 --> 00:18:08,096 [Seb Krier]
Yeah. Oh, no, I agree. I think that we’re not there yet, right? I think, again, and that goes back to the earlier AGI question, is there’s all sorts of, then again, what’s the right comparator? But,
00:18:09,476 --> 00:18:21,316 [Seb Krier]
yeah, I don’t think we’re exactly there. Yeah, I think a lot of this will have to be built as well. The kind of an ability for a model to just better kind of operate in a more multi-agent environment, kind of have a better sense of
00:18:22,596 --> 00:18:32,556 [Seb Krier]
delegation. I think the kind of, yeah, industrial intelligence or something seems to be maybe more neglected, as opposed to just single-agent intelligence or something, if that makes sense.
00:18:32,556 --> 00:18:34,776 [Seth]
Do we need to bring the word cybernetics back?
00:18:34,776 --> 00:18:35,496 [Seb Krier]
Yeah.
00:18:35,496 --> 00:18:36,116 [Andrey]
[laughs]
00:18:36,116 --> 00:18:38,816 [Seb Krier]
Somewhat. [laughs]
00:18:40,756 --> 00:18:51,256 [Andrey]
All right. A little change in subject, but I know this has been in the discourse, the topic of recursive self-improvement, RSI.
00:18:51,256 --> 00:18:52,956 [Seth]
Ooh, very scary.
00:18:52,956 --> 00:18:54,896 [Andrey]
Jack Clark recently had an essay about it.
00:18:56,376 --> 00:18:58,876 [Andrey]
Seb, what is your take?
00:18:58,876 --> 00:18:59,206 [Seb Krier]
[chuckles]
00:19:00,316 --> 00:19:07,896 [Seb Krier]
What is my take? I don’t know. I think it depends what exactly we mean by recursive self-improvement.
00:19:09,096 --> 00:19:50,336 [Seb Krier]
I had a blog post not long ago, I guess, when trying to disentangle a little bit what I have in mind when I think about this. On the one hand, there’s the model getting recursively better through the usage of more AI and whatnot. And on the other hand, there’s the more kind of societal side of things, the transformation side, which I think very often, these two worlds are a little bit blurred in the discourse. It’s like, oh, you get RSI, and then X, Y, Z about the world or something. Things go really fast or they don’t go fast. And, I think these should be separated very neatly because on the model side, of course, I expect, already there’s a lot of AI being used everywhere to kind of create models. And I expect that to continue.
00:19:52,536 --> 00:19:55,976 [Seb Krier]
But it’s not clear to me that this necessarily now leads to a dynamic by which
00:19:57,156 --> 00:20:16,596 [Seb Krier]
the model now gets extremely or exponentially intelligent in a very short amount of time. It’s still kind of bottlenecked by all sorts of resources. And as I was saying earlier, I still see them as better at kind of paradigm exploitation than kind of exploration, which I think is the thing you might need to get to the next step. But, first of all, what do I know? But secondly,
00:20:17,616 --> 00:20:19,986 [Seb Krier]
the other thing is, yeah, on the societal side of things,
00:20:20,996 --> 00:20:29,756 [Seb Krier]
people sometimes talk about foom or hard takeoffs and whatnot, and these have very clear kind of real-life implications. It’s not just kind of a model of getting better in a
00:20:31,216 --> 00:20:34,576 [Seb Krier]
data center somewhere. And that side, I think, is where you have to think about
00:20:36,116 --> 00:21:27,056 [Seb Krier]
[lip smack] all the kind of usual bottlenecks, adoption, deployment, diffusion, the kind of productive integration of all these systems at scale, both in terms of manufacturing and so on and so forth. And, I guess it’s not clear to me that the shift from GPT-2 to GPT-3 or coming up with kind of, we’re just very classic kind of software engineering, meat and potatoes type tasks that you can just easily just automate away. It’s maybe one of these things that’s maybe easy to say ex post, but, I’m not sure. And certainly, my expectation is you’re going to get loads of gains in the coming years of kind of automating part of that pipeline. But that seems good. You just get better models, and that’s just overall helpful for all sorts of other things, even if you’re doing safety work and kind of governance work and whatnot, we benefit a lot from that cognitive resource, I guess.
00:21:27,056 --> 00:21:40,696 [Andrey]
What would happen in the world for you to change your mind? Is there any, let’s say that recursive self-improvement is actually kind of this much more profound change than you’re painting.
00:21:41,816 --> 00:21:42,036 [Andrey]
What
00:21:44,136 --> 00:21:45,696 [Andrey]
signs would there be, I guess? Yeah.
00:21:45,696 --> 00:21:51,656 [Seb Krier]
But to be clear, I’m not claiming it’s just business as usual, nothing to see here or whatever, right? I’m
00:21:52,796 --> 00:22:14,936 [Seb Krier]
kind of just claiming that some of the stronger versions of the claim aren’t kind of self-evident. And so I see a lot of this happening in some sense. Certainly, in 10 years, I expect to have larger kind of more, again, acceleration of economic growth and whatnot and kind of faster diffusion across the board. I certainly don’t expect diffusion to take the same amount of time as, say, electricity or these other technologies.
00:22:16,576 --> 00:22:23,236 [Seb Krier]
So it depends what exactly you mean, because what specifically am I looking to change my mind on?
00:22:23,296 --> 00:22:30,656 [Andrey]
Well, let’s say the scenarios of AI 2027, right? Presumably,
00:22:31,996 --> 00:22:45,176 [Andrey]
in 2027, you’ll see something that’s like, “Oh, wow, I was wrong. This is not going to be so gradual. This is going to be this sudden foom,” that you’re criticizing. Yeah.
00:22:45,176 --> 00:22:52,236 [Seb Krier]
The original foom or hard takeoff definition literally talks about this change happening within hours or days.
00:22:52,236 --> 00:22:53,236 [Andrey]
[chuckles]
00:22:53,236 --> 00:22:56,056 [Seb Krier]
Which is not even, it’s not what the 2027 scenario, I think, predicts.
00:22:56,056 --> 00:22:56,296 [Andrey]
Yes.
00:22:57,556 --> 00:23:00,446 [Seb Krier]
But the 2027 scenario, from what I remember, again, it’s been a bit of time now.
00:23:01,796 --> 00:23:08,816 [Seb Krier]
One thing with the scenarios there is that there’s the kind of misalignment assumption, and which I’m kind of uncertain about.
00:23:08,816 --> 00:23:09,255 [Andrey]
Mm.
00:23:09,256 --> 00:23:17,296 [Seb Krier]
And it also talks about a lot of progress in robotics, which I think is a bit further away. I think it’s close. We’re getting there, too.
00:23:19,116 --> 00:23:19,476 [Seb Krier]
But
00:23:21,156 --> 00:23:25,916 [Seb Krier]
I don’t know. Probably kind of AI, if in 2030, we start seeing AI is making all sorts of crazy
00:23:26,956 --> 00:24:06,196 [Seb Krier]
inventions, innovations in fields other than just kind of perhaps math and coding across the boards, and I’m like, okay, this is clearly-- And you get extremely fast adoption, too, right? You have entire businesses doing completely, it’s not business as usual, clearly, in the economy or something and wide adoption. But it’s hard to say because I expect all that to some degree, right? It’s not that I’m saying, “Oh, this is never going to happen.” I just think of it as a little bit more elongated and the implications of that being maybe not as like, we have Dyson spheres in five years or something like that, so. It’s more of a disagreement maybe on the extremes or the margins or something, but not so much at the core of the claim that yes, models are going to make models better and...
00:24:07,276 --> 00:24:27,536 [Seb Krier]
But, again, even having-- In fact, actually, here would be a thing. If Anthropic or DeepMind or something in 2037 have fewer and fewer employees, fewer people kind of just doing AI research, engineers and so on, you’re clearly seeing kind of that profession. Because of course, I can imagine these jobs to change, right? Maybe you’re kind of managing more agents or something. That
00:24:28,616 --> 00:24:35,966 [Seb Krier]
I expect. But the fact that you just need far fewer people to kind of do not only these large training runs, but the kind of
00:24:36,976 --> 00:24:43,476 [Seb Krier]
large training runs that give you just much, much better systems, then I think I’d be like, okay, this is going a little bit faster than maybe expected or something.
00:24:44,656 --> 00:24:51,676 [Andrey]
Okay. One thing you mentioned in that kind of hints at another hot take you have, which is about alignment.
00:24:51,676 --> 00:24:52,026 [Seb Krier]
Uh-huh.
00:24:54,596 --> 00:24:55,926 [Andrey]
What’s the deal with alignment?
00:24:57,196 --> 00:24:58,086 [Andrey]
[laughs]
00:24:58,086 --> 00:24:58,136 [Seb Krier]
[laughs]
00:24:58,136 --> 00:25:02,136 [Seth]
Is it hard? Is it easy? Is it different than we would’ve expected going in?
00:25:02,136 --> 00:25:19,646 [Seb Krier]
Yeah. It’s perhaps that. I think my take about alignment is something-- Well, first of all, I just don’t like the word. I think it’s a bit of an annoying word because it’s being used for all sorts of things. The AI says something that we just kind of don’t like, or you say, “Oh, it’s misaligned.” No one pre-registers what they expect the aligned behavior to be, and then just kind of tests.
00:25:19,646 --> 00:25:20,116 [Andrey]
[laughs]
00:25:20,116 --> 00:25:35,626 [Seb Krier]
But I think my general claim is maybe the fact that it’s been easier than we would’ve predicted a decade ago or so. Then when I first got into AI in 2017, that was partly as a result of reading things like “Superintelligence” by Bostrom.
00:25:35,626 --> 00:25:36,236 [Andrey]
Mm-hmm.
00:25:36,236 --> 00:25:48,496 [Seb Krier]
And you’d read these books, like Stuart Russell’s “Human Compatible” and others, that kind of had all these analogies like King Midas and you ask a system to optimize for goal X, and in pursuit of that goal, it does all sorts of other things that you don’t want it to do.
00:25:48,496 --> 00:25:51,916 [Seth]
Right. The paperclip maximizer, and we seem to not have those.
00:25:51,916 --> 00:25:57,476 [Seb Krier]
Yeah. It’s like one version of it or one variant of it. And certainly at the time you didn’t really have language models. A lot of these intuitions were kind of based off
00:25:58,596 --> 00:26:48,236 [Seb Krier]
reinforcement learning systems in very basic kind of game scenarios where they were actually given a single goal to optimize for. And this is not actually what we do, I think, with models. And you had these kind of examples, even the value loading problem was something discussed at the time where actually specifying these complicated nuanced human values in mathematical terms would be extremely hard. So even if you managed to tell a robot to clean the room, it would then just pick up a baby and put it in the trash or something. And I think it turns out a lot of this stuff is actually much easier. You have problems. You’ve got things like reward hacking. You’ve got AIs behaving in weird ways that we were not always kind of anticipating because of the ways they were post-trained. So my claim is not like, oh, again, it’s all fine, and safety is a scam or whatever. It’s more that it’s certainly much easier than, or at least we’re in a much better track than I would’ve at least guessed perhaps a decade ago. And secondly, I think it
00:26:49,916 --> 00:26:54,816 [Seb Krier]
just seems tractable. There’s a lot of progress in terms of chain-of-thought monitoring and all these other things. And
00:26:56,696 --> 00:26:57,796 [Seb Krier]
I also think that the
00:26:59,016 --> 00:27:05,825 [Seb Krier]
hard part is maybe more the kind of normative question of whose values and when, and what and everything. That’s the kind of thing that we’re looking into more. But
00:27:07,096 --> 00:27:13,696 [Seb Krier]
yeah, I prefer the word actually instruction following or intent following or something instead of alignment. And I think by and large, they’re actually pretty good at that.
00:27:14,796 --> 00:27:31,636 [Seb Krier]
So again, that doesn’t mean you have to dismiss all sorts of theories and all the kind of power optimization stuff. But I guess my immediate outcome is this goes rather well. Or if I am more concerned by other things like misuse, if you’d like, than kind of the AI’s being innately, inherently kind of internally misaligned.
00:27:31,636 --> 00:28:03,676 [Seth]
This really seems related to your take that intelligence is not at odds with being a tool, right? So a lot of people have this intuition where if you had a super-duper intelligent genie or oracle, it would develop even implicitly some sort of value or goal that orthogonality thesis might have nothing to do with what we want. But you’re more optimistic about the idea that the LLM doesn’t want anything. It’s incorrect to take the intentional stance towards an LLM.
00:28:03,676 --> 00:28:09,236 [Seb Krier]
Not incorrect. It’s actually kind of descriptively useful, even functionally sometimes to use that language.
00:28:10,796 --> 00:28:18,836 [Seb Krier]
But that’s the thing, right? I think we kind of lack the language to properly delineate and differentiate when it’s useful to use that or appropriately descriptive and when it’s not.
00:28:20,076 --> 00:28:41,496 [Seb Krier]
And so I agree that, of course, I think the take I had on this was something like, and I can imagine a tool being an agent and an agent being a tool. Or in principle, I can imagine something being hyper-capable and still being broadly instruction following rather than at a certain level of capability, aha, that’s when the goals change and things get... And it kind of depends on the type of system as well. I imagine not all
00:28:42,656 --> 00:28:45,116 [Seb Krier]
paths lead to the same kind of outcome. But,
00:28:46,256 --> 00:29:13,596 [Seb Krier]
so again, I can see plausible versions of the world where homo hundrio drives or something are a more salient feature of the way we kind of train models. Right now, it doesn’t seem to me very likely that this is a core feature that they have. But of course, it’s hard to kind of either prove or disprove, right? Because someone might just say, well, that’s because they’re very good at hiding this or something, or once they’re capable enough or whatever. So there’s always a bit of this kind of gotcha thing. It’s like deception. But
00:29:14,936 --> 00:29:39,896 [Seb Krier]
yeah. So in principle, I guess I can totally conceive of at least a superintelligence that is controllable, that is benign, that is at least subservient to the goals of humanity or a user or principle or whatever. That could still be used to cause enormous harm, but it’s just I don’t necessarily think the analogies of, oh, I think Tegmark was thinking, look at the zoo where the monkey’s going. I think these are just not really
00:29:41,736 --> 00:29:43,136 [Seb Krier]
helpful kind of analogies.
00:29:44,276 --> 00:30:02,396 [Seth]
Monkey at the zoo, but you’ve also got the monkey’s paw, right? Maybe the reason some prefer alignment to instruction following is we all know the story of, be careful what you wish for. You wish for something, and it’s under-specified, and you get the bad version of it because the AI doesn’t understand the context.
00:30:02,396 --> 00:30:08,336 [Seb Krier]
I think that’s why, yeah, I think maybe instruction following is maybe too... Intent following or something gets to it more.
00:30:09,936 --> 00:30:18,316 [Seb Krier]
But of course, that problem doesn’t go, even if it follows intent or something, you could still have all the problems because your intent is nefarious or whatever. So
00:30:19,436 --> 00:30:19,816 [Seb Krier]
I think the
00:30:21,356 --> 00:31:06,756 [Seb Krier]
way you deal with that is all sorts of, I don’t know how to conceptualize it, but in fact scaffolds. It’s a bit more this outside of the model or something. I’m kind of almost indexing on a world that will indeed have agents that are trained to be bad or whatever, or someone going to be instructed to do bad things. But just like with humans, you come up with all sorts of kind of systems, rules, laws, norms, kind of protocols that either discourage the kind of bad behavior, or punishes it, or makes it just not worthwhile or something. But I’m not going to put all my bets on the, oh, it has to be pure-hearted, and that will be sufficient. And then you just scale it forever, and it’s going to be an amazing goal. I just think that the way of seeing or thinking about AI is that I just find kind of a bit
00:31:08,096 --> 00:31:12,656 [Seb Krier]
too narrow, I guess. I think it’s important, it’s just insufficient, and it’s certainly not my main kind of a-- yeah.
00:31:14,946 --> 00:31:15,206 [Andrey]
Okay.
00:31:16,666 --> 00:31:20,086 [Andrey]
Our audience is very much composed of economists.
00:31:22,586 --> 00:31:30,506 [Andrey]
If you’re an economist and you’re very interested in AI, what sort of work would you be trying to do?
00:31:30,506 --> 00:31:32,146 [Seth]
Maybe to be useful to AI people-
00:31:32,146 --> 00:31:32,216 [Andrey]
Yes
00:31:32,216 --> 00:31:37,466 [Seth]
... in particular. What would you want, what did the DeepMind team want to read from economists?
00:31:37,466 --> 00:32:20,766 [Seb Krier]
I think kind of engaging with their assumptions or something, right? If you assume, let’s say, an AG-- and I think some do, to be fair. I actually think there’s a lot more, I think, discourse now going on between economists and AI people, whatever. But assuming that you do have AI systems that are interchangeable or almost quasi-fully substitutable with humans, that come up with good ideas, that are parallelizable and whatnot, what does that change to your kind of growth function and so on? So, maybe that’s useful. Right now, in the short term, at least, there’s all sorts of questions around labor, there’s questions around productivity or adoption. Clearly, there’s useful work to be done there. But I think in terms of AGI specifically, given that a lot of the field just thinks you’re going to get to AGI in the next five to 10 years,
00:32:22,746 --> 00:32:26,806 [Seb Krier]
what are the implications for taxation? What are the implications for
00:32:28,626 --> 00:32:37,786 [Seb Krier]
how that’ll affect different states across the world? I think I’m probably more worried about a call center in Hyderabad than I am about the white-collar worker in North America or something. So,
00:32:39,066 --> 00:32:57,306 [Seb Krier]
yeah. I think all these kind of questions, but just indexing more and making fewer, I guess, assumptions around the limits of capabilities. Because sometimes you see them kind of being implicitly snuck in somewhere or something of like, well, because AIs can’t do XYZ, therefore... And yeah, fine, but maybe they will do XYZ. And then what? How does that change your thinking? Yeah.
00:32:57,306 --> 00:32:59,506 [Seth]
Maybe more scenario planning than,
00:33:00,526 --> 00:33:04,746 [Seth]
here’s my median projection, or here is one projection I think is plausible.
00:33:04,746 --> 00:33:22,846 [Seb Krier]
Yeah. And embedding the kind of thoughtful models and thinking that economists have within these scenarios and making them more salient to the kind of computer scientists, right? Even when I brought up competitive advantage, people will be like, “Oh, but what if the AI is cheaper and better?” It’s like, well, that’s not the point. The opportunity cost point of competitive advantage, there’s a difference.
00:33:22,846 --> 00:33:23,286 [Andrey]
[laughs]
00:33:23,286 --> 00:33:31,786 [Seb Krier]
And again, there are answers to that as well, but I think just kind of better translating, I think, some of these insights to the AI tribe, the thing is useful.
00:33:32,846 --> 00:33:40,526 [Andrey]
So that’s very naturally leading us to this question about yourself. And you do lots of different things.
00:33:41,946 --> 00:33:50,426 [Andrey]
You’re prolific on Twitter, for sure. But also, you’re doing internal work for DeepMind. How do you allocate your time?
00:33:52,066 --> 00:33:52,166 [Seb Krier]
I don’t know.
00:33:52,166 --> 00:33:53,266 [Seth]
What percentage is Twitter?
00:33:53,266 --> 00:33:54,646 [Andrey]
Yeah. [laughs]
00:33:54,646 --> 00:34:04,686 [Seb Krier]
Twitter is actually not that much today. It must be an hour max or something, an hour and a half, two hours, maybe, something. But that is maybe much by others’ standards. But the-
00:34:04,686 --> 00:34:06,476 [Andrey]
[laughs] What is the optimal amount of Twitter? [laughs]
00:34:06,476 --> 00:34:29,866 [Seb Krier]
[laughs] Yeah. It’s the Pareto optimal. I guess, in my day-to-day work, it’s a mixture of proactive and reactive. Proactive in the sense that I think, oh, these questions of agents and cybersecurity and liability and whatnot, and biosecurity are kind of important things to look into, and therefore, there’s a lot of research that I do and colleagues do, and a lot of coordination across the org.
00:34:31,026 --> 00:34:39,486 [Seb Krier]
But there’s also more reactive stuff because we’re a policy team, and so there’s things happening in the external world like CA 53, the preemption debates.
00:34:40,546 --> 00:34:48,386 [Seb Krier]
So it’s a bit of a mix of that. And of course, all sorts of internal dynamics. But, yeah. I guess I’m curious about all sorts of other things, and so when I do have time, and I’ve kind of
00:34:50,006 --> 00:34:58,106 [Seb Krier]
completed the main quests, I try to keep some time for other stuff I’m interested in. I work with some research teams and kind of look into what they’re into. I’ll
00:34:59,266 --> 00:35:09,826 [Seb Krier]
find topics or themes that I think are maybe kind of neglected or underrated or I just don’t see out there as much, and like, “Oh, cool. We’re going to try to find out about this more.” But I think it’s just very kind of curiosity driven, and the allocation of time is
00:35:11,566 --> 00:35:16,705 [Seb Krier]
not super thought out. It’s more like, oh, I think these things are interesting, and I’m going to get into that for a bit. [laughs]
00:35:16,706 --> 00:35:22,306 [Andrey]
So it wasn’t a deliberate strategy of getting Tyler’s attention and adoration. [laughs]
00:35:22,306 --> 00:35:25,126 [Seb Krier]
No, not at all. Not at all. But I’m very-
00:35:25,126 --> 00:35:25,746 [Seth]
The long play
00:35:25,746 --> 00:35:30,565 [Seb Krier]
... very grateful for his... [laughs] For the meme. But-
00:35:30,566 --> 00:35:41,766 [Seth]
What kind of, but I know you can’t be specific, but for your sort of internal work, what does a work product look like? Are you participating in a meeting and giving hot takes? Are you writing internal memos? What is-
00:35:41,766 --> 00:35:42,026 [Seb Krier]
Yeah
00:35:42,026 --> 00:35:42,276 [Seth]
... in-
00:35:42,276 --> 00:35:56,406 [Seb Krier]
It’s a mixture. Obviously, meetings. Any large bureaucracy will have meetings. But I think a lot of analysis, memos to execs sometimes. Just research, managing researchers sometimes, depending on the project.
00:35:57,626 --> 00:36:04,106 [Seb Krier]
We’ll have a lot of coordination. Actually, I’m realizing through a lot of these kind of meetings, a lot of it is just kind of coordination and information transfer, right?
00:36:04,106 --> 00:36:04,146 [Andrey]
[laughs]
00:36:04,146 --> 00:36:07,006 [Seb Krier]
It’s maybe why I’m so obsessed with the Coasean bargaining thing. Just let-
00:36:07,006 --> 00:36:07,326 [Seth]
Ah
00:36:07,326 --> 00:36:08,546 [Seb Krier]
... the agents do it. But,
00:36:09,806 --> 00:36:34,116 [Seb Krier]
yeah. I think the day-to-day work is a lot of reading, a lot of meetings, a lot of writing, and distilling and translating information, I think, across different tribes also. So if I’m talking to legal people, like lawyers, about what’s going on in, say, the more technical side of the org, or if I’m speaking to the researchers about something that’s more... But yeah, there’s a lot of translating of concepts across different stakeholders, I guess.
00:36:34,116 --> 00:36:45,726 [Andrey]
So how does that work in an org like Google? Because I think in a lot of orgs, they’re really obsessed with KPIs and output metrics.
00:36:45,726 --> 00:36:46,156 [Seb Krier]
Mm-hmm.
00:36:46,156 --> 00:36:48,746 [Andrey]
And what you’re describing sounds very-
00:36:48,746 --> 00:36:49,706 [Seth]
Hot takes per meeting. [laughs]
00:36:49,706 --> 00:36:54,926 [Andrey]
Yeah. Very much amorphous, very hard to measure.
00:36:56,066 --> 00:36:56,196 [Seb Krier]
Yeah.
00:36:56,196 --> 00:37:00,606 [Andrey]
Obviously, you have a lot of external visibility, but is that
00:37:02,786 --> 00:37:07,846 [Andrey]
a problem? Or is that just it’s understood that that’s how this goes? Yeah.
00:37:07,846 --> 00:37:13,846 [Seb Krier]
I think the external stuff is kind of almost just very separate from the kind of day-to-day work side of things.
00:37:14,986 --> 00:37:23,366 [Seb Krier]
And yeah, internally, we do have KPIs or equivalents or whatever. I think they may be less numerical in nature. But you might still have some, develop a consistent position on
00:37:24,506 --> 00:37:30,819 [Seb Krier]
X issue or something in the next two, three months.And that requires a lot of research work, coordinating.
00:37:30,819 --> 00:37:32,929 [Seth]
Have 10 opinions. [laughs]
00:37:32,930 --> 00:37:38,100 [Seb Krier]
No, ideally they just want one. I think 10 opinions, that’s the issue. There are a lot of opinions out there. You’ve got to find the good ones.
00:37:38,100 --> 00:37:39,530 [Seth]
That’s the main problem with economists.
00:37:39,530 --> 00:37:42,350 [Seb Krier]
But [laughs] yeah. Exactly. Who was that quote?
00:37:43,830 --> 00:37:44,290 [Seth]
Truman.
00:37:44,290 --> 00:37:44,330 [Seb Krier]
Yeah.
00:37:44,330 --> 00:37:46,210 [Seth]
Truman begged for the one-handed economist.
00:37:46,270 --> 00:38:20,990 [Seb Krier]
Yeah, exactly. But, so I think, yeah, I think internally it’s just a kind of analysis or something. Say you’re thinking about, oh, agents and legal liability. How do these things work? What does the existing legal environment say and prescribe? What happens if something goes wrong? What are relevant factors? There’s a lot of that kind of thing. And I guess particularly within the DeepMind side, because when we’re on the frontier side, we’re thinking about the next five years as opposed to what’s going on right now. But yeah, the other side stuff is really just kind of out of personal interest and just me writing stuff, and they seem fine with it so far. [chuckles]
00:38:20,990 --> 00:38:26,510 [Andrey]
What about... So we’ll be at a conference together, the Post-AGI conference-
00:38:26,510 --> 00:38:26,830 [Seb Krier]
Ooh
00:38:26,830 --> 00:38:28,370 [Andrey]
... at Lighthaven, Berkeley.
00:38:28,370 --> 00:38:30,110 [Seth]
Ooh. Prestigious.
00:38:31,130 --> 00:38:32,990 [Andrey]
I don’t know if it’s prestigious.
00:38:34,550 --> 00:38:34,629 [Seth]
[laughs]
00:38:34,630 --> 00:38:45,730 [Andrey]
But you’ve gone to a few of these conferences, like the Curve is another fairly well-known one. What’s your take on these?
00:38:45,730 --> 00:38:54,750 [Seb Krier]
I think some are useful. The majority of conferences I go to, I don’t exactly find that life-transforming, I guess.
00:38:54,750 --> 00:38:57,610 [Andrey]
[laughs] You’re going to the wrong conference. [laughs]
00:38:57,610 --> 00:39:09,290 [Seb Krier]
I know. Can someone show me the... But I think, yeah, they obviously perform a social function to some degree, right? There’s a lot of meeting people, some networking or something, some kind of finding out new ideas. But
00:39:10,390 --> 00:39:20,310 [Seb Krier]
my issue with conferences, very often they’re just very tame. They’re very risk-averse. They’re very the same ideas you’ve-- Already if you can read it online or something, it depends on the conference. But,
00:39:21,510 --> 00:39:24,190 [Seb Krier]
although I have been to really good ones, too. There was this
00:39:25,570 --> 00:39:43,529 [Seb Krier]
IMF conference with Econ Ty, with I think Anton Korinek and others had organized. And that was great because that was a nice one where you had both the technologists and a lot of economists and loads of presentations, and you got to learn lots of new things. But, in general, I don’t see a huge... Beyond maybe showing, again, some hot takes here and there.
00:39:45,370 --> 00:39:49,990 [Seb Krier]
Yeah, some I assume are good conferences. [chuckles]
00:39:49,990 --> 00:40:00,670 [Seth]
I’m just the exception, but you had a great joke on your Twitter the other day about this, which is, Caveman panelist one, “Fire is bad.” Caveman panelist two, “Fire is good.”
00:40:00,670 --> 00:40:00,770 [Seb Krier]
Yeah.
00:40:00,770 --> 00:40:02,100 [Seth]
Caveman panelist three,
00:40:03,450 --> 00:40:07,120 [Seth]
“We need to balance the upsides and downsides of fire and use it wisely.”
00:40:07,120 --> 00:40:07,320 [Seb Krier]
Absolutely.
00:40:07,320 --> 00:40:09,620 [Seth]
Wild applause. [laughs]
00:40:09,620 --> 00:40:09,650 [Andrey]
[laughs]
00:40:09,650 --> 00:40:14,850 [Seb Krier]
Exactly. There’s a lot of that. That’s the energy that I’m getting very tired of because it’s-
00:40:14,850 --> 00:40:15,050 [Seth]
[laughs]
00:40:15,050 --> 00:40:21,700 [Seb Krier]
And I like playing the role of the wise centrist opinion, whatever. But it does get very-
00:40:21,700 --> 00:40:23,150 [Seth]
You do get wild applause.
00:40:23,150 --> 00:40:24,470 [Seb Krier]
Yeah. All the time. [chuckles]
00:40:26,490 --> 00:40:29,770 [Seb Krier]
But yeah, I think there’s a lot of that. I wish there were more
00:40:30,810 --> 00:40:35,090 [Seb Krier]
almost private Chatham House-y conferences, where you had people who highly disagreed with each other-
00:40:35,090 --> 00:40:35,210 [Andrey]
Mm
00:40:35,210 --> 00:40:36,770 [Seb Krier]
... but were polite and didn’t get at
00:40:37,950 --> 00:40:49,370 [Seb Krier]
each other’s throats. And you had more setups that actually allowed ideas to clash a bit more, in a civilized way, of course. But that would be a bit hard, but also much more interesting, I think, than
00:40:51,490 --> 00:40:55,390 [Seb Krier]
everyone broadly agreeing that it’s good to be good and it’s bad to be bad, and yeah. [chuckles]
00:40:55,390 --> 00:41:03,710 [Andrey]
I do feel like the Lighthaven conferences are quite good for this, in that there’s an enormous amount of free time and-
00:41:03,710 --> 00:41:04,130 [Seb Krier]
Mm-hmm
00:41:04,130 --> 00:41:07,770 [Andrey]
... free space that’s not where the talk is happening.
00:41:07,770 --> 00:41:07,940 [Seb Krier]
Yeah.
00:41:07,940 --> 00:41:10,630 [Andrey]
And so you do get a lot of this.
00:41:10,630 --> 00:41:11,040 [Seb Krier]
Well, yeah, I agree.
00:41:11,040 --> 00:41:21,090 [Andrey]
But I agree that many conferences are not like that, where you’re just packed. You have a conference hall, and you don’t have anywhere else to go, and it’s packed with talks. Yeah.
00:41:21,090 --> 00:41:21,710 [Seb Krier]
Yeah. No, totally.
00:41:21,710 --> 00:41:23,550 [Seth]
NBER Summer Institute. [laughs]
00:41:24,750 --> 00:41:28,330 [Andrey]
Seth, there is disagreement. Say what you will. At NBER-
00:41:28,330 --> 00:41:28,540 [Seth]
There is fire
00:41:28,540 --> 00:41:29,430 [Andrey]
... people throw down.
00:41:30,450 --> 00:41:31,430 [Andrey]
[laughs]
00:41:31,430 --> 00:41:37,720 [Seth]
[laughs] I’ve never seen a meaner comment than I have seen from a discussant at NBER Summer Institute. [laughs]
00:41:37,720 --> 00:41:52,570 [Seb Krier]
[laughs] The Progress Conference, for example, last year, was one that I thought was really good. That was at Lighthaven, in fact. I think the setup and the kind of people and the curation and so just made it something that I found quite engaging. [upbeat music]
00:41:52,570 --> 00:41:56,490 [Seth]
So you brought up this idea, as we were talking, about you
00:41:58,330 --> 00:42:21,049 [Seth]
think there are so many meetings in your organization because it’s so hard, yet so critical to transfer information. And there’s this Coasean idea that so much of why the economy works the way it does is just the idea of transaction costs, right? In addition to kind of this Hayekian idea of local information that’s hard to share.
00:42:21,050 --> 00:42:21,810 [Seb Krier]
Mm-hmm.
00:42:21,810 --> 00:42:23,960 [Seth]
You have a very influential essay
00:42:25,130 --> 00:42:30,230 [Seth]
that kind of maybe stole some of Andrey’s thunder, but is still an excellent essay-
00:42:30,230 --> 00:42:31,040 [Seb Krier]
[laughs]
00:42:31,040 --> 00:42:46,210 [Seth]
... about this idea of, well, what happens when AIs go out there and can micro-bargain costlessly with each other at high frequency over very, what might seem to us, small issues.
00:42:47,570 --> 00:42:57,440 [Seth]
Tell us maybe in a few sentences, what’s that vision and what’s the positive vision for why that would be good for society, for us to have AI agents constantly bargaining for us over stuff?
00:42:59,130 --> 00:43:01,810 [Seb Krier]
Yeah. I guess the idea is, as you mentioned, there’s all sorts of
00:43:03,990 --> 00:43:26,350 [Seb Krier]
transaction costs that mean that we don’t get to bargain on things that we would otherwise bargain for. And instead, you get these blunt rules and these solutions that kind of work, but come with all sorts of externalities or aren’t super efficient. And so the idea is, if you can actually do this kind of negotiation at scale for very little, and that’s a big assumption. That’s not a given either,
00:43:27,850 --> 00:43:35,586 [Seb Krier]
then you could solve all sorts of things thatAnd also just kind of problems that would otherwise not be even conceivable in the first place.
00:43:36,726 --> 00:43:41,186 [Seth]
One example you give, just so we can be a little bit more specific, is noise standards, right?
00:43:41,186 --> 00:43:41,456 [Seb Krier]
Right.
00:43:41,456 --> 00:43:57,226 [Seth]
So you can’t throw a loud party after 10:00 PM in such and such a place. But you think that maybe AI agents could come to a less coarse rule that is, get us more to the grand coalition of allocative efficiency than a coarse rule like that.
00:43:57,226 --> 00:44:01,166 [Seb Krier]
Yeah. To be fair, that’s probably a problem that no one really cares about except me because of like- [chuckles]
00:44:01,166 --> 00:44:02,086 [Seth]
No. Dude.
00:44:02,086 --> 00:44:03,645 [Andrey]
I care about it so much.
00:44:03,645 --> 00:44:04,626 [Seb Krier]
Oh, really? Okay, cool.
00:44:04,626 --> 00:44:04,746 [Andrey]
Yes.
00:44:04,746 --> 00:44:07,816 [Seb Krier]
Maybe that’s a good example then. But yeah, the idea here is,
00:44:09,146 --> 00:44:17,006 [Seb Krier]
my neighbor is throwing a party, and instead of there being some sort of rule that says you’re not allowed to throw parties after 11:00, he could maybe just compensate me for the noise or something.
00:44:18,326 --> 00:44:21,686 [Seb Krier]
Or in fact, that’s one of the key crux of the whole Coasean thing is maybe
00:44:24,186 --> 00:44:36,085 [Seb Krier]
I have to compensate him to stop his parties. And it kind of depends where the initial right is. But broadly, you could have these kind of, my whole neighborhood doesn’t want me to party, and they’re just giving me a small payment or the reverse, depending on where the initial allocation is.
00:44:37,226 --> 00:44:44,446 [Seb Krier]
But I think you could have all sorts of micro ways in which these transaction costs at scale help you get much better beneficial outcomes.
00:44:45,486 --> 00:44:48,486 [Seb Krier]
And so that would be the noise one would be like, okay.
00:44:50,406 --> 00:45:18,666 [Seb Krier]
And it’ll probably just also let people kind of regroup into the party people just going into the neighborhood where that’s just generally more party tolerant or something, and the kind of peace and quiet preferring people just... Because I think one of the points with the piece was that AI also helps you coordinate better. You can use this stuff to find people who have the same interests and preferences as you or something, and just then bargain or negotiate or whatnot in that way as well.
00:45:20,626 --> 00:45:27,386 [Seth]
So it’s not just bargaining over externalities that are negative, it’s maybe coordinating over positive externalities, right?
00:45:27,386 --> 00:45:27,526 [Seb Krier]
Yeah.
00:45:28,766 --> 00:45:51,746 [Seth]
What pieces do we need in the economy to make this a reality, and what time horizon are you thinking about? So obviously this is an idea that you could have a small version of, and then like the sci-fi, this is constantly, I’m allowed to speed in my car today because I really need to get to work because I’m late, and it’s bargaining with all the cars on the highway at ultra-high frequency. So what are the time horizons you have in mind, and what pieces do we need?
00:45:51,746 --> 00:46:21,786 [Seb Krier]
Honestly, I haven’t even thought about the timelines really. [laughing] For me, this was mostly kind of an aspirational thing of like, well, it looks like we could unlock some cool things, and because there’s all these-- It’d be nice to have a positive vision of how things might pan out. It certainly doesn’t mean that everything has to be negotiated and bargained over. But I could see a large proportion of things, certainly in everyday life, like I could just tell my aunt, “You don’t have to worry about your parking issues anymore. It’s just sorted now,” whatever. The agents are taking care of that. And so it kind of depends on what scale you’re talking about. Certainly having democracy at scale and
00:46:23,626 --> 00:46:29,086 [Seb Krier]
half automated and half made more efficient through these systems or something is something that I think is going to take a long time.
00:46:30,426 --> 00:46:47,986 [Seb Krier]
But I can see smaller versions of it happening. The smallest version I can think of is just even basic calendar management. Until recently, that would be impossible to get automated really well. But gradually now I could expect an agent to really know my preferences, the context in which I operate, the hierarchical
00:46:49,006 --> 00:47:12,466 [Seb Krier]
relationship with the people I’m organizing meetings with and whatnot, and the agent coming to tell me something like, “Okay, I’ve moved your meeting here to next week because this one’s not very important, and you don’t really care about it. But I’ve brought this meeting with your director forward because it’s urgent given what I’ve seen in that email. And also, I’ve set the location in this place because you’ll both be around that area in any event around then, given what I’ve gathered from talking to the other agent.”
00:47:13,946 --> 00:47:23,766 [Seb Krier]
Something like that I think I could easily see in the next couple of years. But something at a higher, things like automating part of a local authority and-
00:47:23,766 --> 00:47:26,786 [Seth]
So let’s stick to the noise example because that’s a core one.
00:47:26,786 --> 00:47:30,276 [Seb Krier]
Well, the noise one, I think takes a bit longer because it requires this kind of,
00:47:31,486 --> 00:47:44,496 [Seb Krier]
well, this adoption and integration of this technology at my local city council or whatever. And that’s one of the things that takes a long time. It’s not immediate, and it requires a shift in behaviors. People have to also then use these agents. It has to be legitimized. The
00:47:45,726 --> 00:47:48,386 [Seb Krier]
value proposition has to be made a bit clearer.
00:47:48,386 --> 00:47:48,586 [Seth]
Mm-hmm.
00:47:48,586 --> 00:48:01,666 [Seb Krier]
And there’s all the usual political issues of like, well, you’ve got these vested interests, and it’s not in some people’s interests for that whole thing to work out. In the same way as if you wanted to have some sort of YIMBY paradise, a lot of kind of planning authorities wouldn’t be very delighted. They would probably find ways to stop the
00:48:02,786 --> 00:48:03,166 [Seb Krier]
deployment of-
00:48:03,166 --> 00:48:07,046 [Seth]
But they could be compensated through micro-transactions, dude. [chuckles]
00:48:07,046 --> 00:48:11,866 [Seb Krier]
Exactly. Right? It’s a recursive Coasean improvement. But-
00:48:11,866 --> 00:48:15,566 [Andrey]
So, I guess just like a thought here.
00:48:15,566 --> 00:48:15,885 [Seb Krier]
Mm-hmm.
00:48:15,886 --> 00:48:19,106 [Andrey]
I’m sure you’ve heard of Edge Esmeralda-
00:48:19,106 --> 00:48:19,116 [Seb Krier]
Mm-hmm
00:48:19,116 --> 00:48:31,936 [Andrey]
... which is kind of this experimental new town. It seems just like we think that with new firms that are entering, they’re going to be able to organize in a way that’s going to better be able to take advantage of AI capabilities.
00:48:31,936 --> 00:48:32,726 [Seb Krier]
Mm-hmm.
00:48:32,726 --> 00:48:43,006 [Andrey]
To the extent that we can have newly incorporated cities or other jurisdictions, they might allow for agentic representation at the local council meeting-
00:48:43,006 --> 00:48:43,646 [Seb Krier]
Mm-hmm
00:48:43,646 --> 00:48:50,666 [Andrey]
... and other such things. And if that works really well, then it would actually
00:48:51,786 --> 00:48:56,166 [Andrey]
show other places what they’re missing out on. I’m not saying that the San Francisco-
00:48:56,166 --> 00:48:56,176 [Seb Krier]
Yeah
00:48:56,176 --> 00:48:59,076 [Andrey]
... City Council would do this. [laughs]
00:48:59,076 --> 00:48:59,586 [Seb Krier]
[laughs]
00:48:59,586 --> 00:49:07,265 [Andrey]
But some smaller cities might start doing this, or school boards. That’s a great example. I hear people have a lot of issues with those.
00:49:07,266 --> 00:49:30,866 [Seb Krier]
Totally. I think that’s kind of what I concluded in the essay, is that you’ve got to start in these kind of small proof of concepts and then kind of make the idea actively desirable and for people to kind of actually see the benefits of these things. And so I agree, The Edge Esmeralda, I think they’re trying something out with agents, and that’s going to be pretty exciting to see. And maybe then that kind of takes on, right? In the same way as a lot of conferences now have apps or something to...
00:49:31,886 --> 00:49:36,446 [Seb Krier]
I guess the more pessimistic version is, well, if you look at Estonia, they’ve been really good at
00:49:37,506 --> 00:49:42,666 [Seb Krier]
integrating technology and automating all sorts of parts of their government in really efficient and effective ways.
00:49:43,642 --> 00:49:44,622 [Seth]
Estonia?
00:49:44,622 --> 00:49:46,102 [Seb Krier]
Yeah. [laughs]
00:49:46,102 --> 00:49:46,982 [Seth]
[laughs]
00:49:46,982 --> 00:49:58,162 [Seb Krier]
And yet you don’t necessarily see neighboring countries have a strong incentive to do the same straight away. And I would expect the equivalent to take quite a long time, say, if you try to do the same thing in Brussels. So there are other-
00:49:58,162 --> 00:49:58,902 [Seth]
[laughs]
00:49:58,902 --> 00:49:59,401 [Seb Krier]
... dynamics.
00:49:59,401 --> 00:50:00,802 [Seth]
Anything takes longer in Brussels.
00:50:00,802 --> 00:50:03,082 [Seb Krier]
That’s true. It’s part and parcel of it.
00:50:03,082 --> 00:50:04,262 [Seth]
Except for the french fries.
00:50:04,262 --> 00:50:08,022 [Seb Krier]
I’d say, yeah. Which are indeed very good. But, I think
00:50:09,562 --> 00:50:25,742 [Seb Krier]
you’ll want some of these kind of smaller challenges, I guess, to really push for these things and put more pressure on the incumbent systems, whether it’s in the private sector or in the public sector. The only thing is, I expect it to be a bit slower in the public sector than it will be in the private sector.
00:50:27,382 --> 00:50:41,082 [Seb Krier]
Once a bunch of startups start using agents that, I don’t know, coordinates on meetings or whatever, maybe that’s going to take off as well with all the wider organizations much more quickly. Whereas, you still have parts of the German or UK government using fax machines today.
00:50:41,082 --> 00:50:59,042 [Andrey]
Well, there’s this example that we have in our paper where some agentic environments are designed for agents only, right? And so you might imagine that to work well within a firm where everyone has their calendar scheduling agent, and they all talk to each other. But I think
00:51:00,102 --> 00:51:07,482 [Andrey]
you can also imagine an environment where the agent has to be able to interface also with humans, right?
00:51:08,782 --> 00:51:10,042 [Andrey]
And those are quite different
00:51:11,762 --> 00:51:13,022 [Andrey]
capabilities
00:51:14,782 --> 00:51:18,842 [Andrey]
or tests of the system. And if you want maybe
00:51:20,062 --> 00:51:25,642 [Andrey]
more useful versions of this, they should also be able to interact with humans who are not using the agents.
00:51:25,642 --> 00:51:37,682 [Seb Krier]
True. And that probably acts also as a speed limiter, I guess, as well, right? If you have to then... But I agree. I think that’s going to be an important part, and I think that’s where protocol design and all these kind of things start coming into play.
00:51:39,022 --> 00:51:48,362 [Seb Krier]
And I liked actually in your paper the whole bowling shoe versus bring your own question as well. Because you’d expect some platforms to just say, “Well, just use our agent here” or something.
00:51:49,422 --> 00:51:54,362 [Seb Krier]
And it’s not clear to me that you’ll necessarily have one agent that you’re going to use across all platforms the same as something.
00:51:56,002 --> 00:52:06,842 [Seb Krier]
Maybe you probably want certain characteristics or certain knowledge to be more segmented. Maybe you want your agent doing high-frequency trading to be a bit different than your agent doing, I don’t know, social life coordination or something.
00:52:08,402 --> 00:52:14,622 [Seb Krier]
And it’s not clear whether it’s an agent that’s necessarily owned in some way by me or if it’s in fact...
00:52:15,882 --> 00:52:25,202 [Seb Krier]
So yeah, there’s all sorts of interesting design questions that aren’t fully clear to me just yet. And I think that would be the kind of thing I’d love to see more work on. Actually, I think also Anthropic had this project deal recently, which was interesting, too.
00:52:27,282 --> 00:52:29,662 [Seb Krier]
And it seems to have worked out quite nicely, I think, in their experiment.
00:52:31,242 --> 00:52:32,262 [Seth]
It is, yeah.
00:52:32,262 --> 00:52:34,242 [Andrey]
Yeah. You’re welcome to explain it. Yeah.
00:52:34,242 --> 00:52:41,262 [Seb Krier]
[laughs] Again, I don’t have deep knowledge of what they’ve done or anything. I think Christy glanced at the... But I think broadly, they
00:52:42,822 --> 00:52:51,922 [Seb Krier]
let part of their employees have access to agents to do some sort of negotiating and bargaining on their behalf for all sorts of commercial items or things they would want to trade.
00:52:53,102 --> 00:52:59,302 [Seb Krier]
And they also did some experiment where I think some had access to more capable models, others had access to slightly less capable models. And I think the
00:53:00,842 --> 00:53:01,842 [Seb Krier]
insight was something like
00:53:03,162 --> 00:53:11,622 [Seb Krier]
everyone was happy in the end, but actually people with the more capable models got better surplus or something. And those without the capable models didn’t realize the
00:53:12,642 --> 00:53:20,782 [Seb Krier]
gain they would have made had they had the more capable one. But in any event, everyone was happy. Isn’t it? It apparently worked out and people traded stuff.
00:53:20,782 --> 00:53:28,921 [Andrey]
Well, my understanding is the setting was people literally took photos. They inventoried the stuff they didn’t need in their home.
00:53:28,922 --> 00:53:29,242 [Seb Krier]
Yeah.
00:53:29,242 --> 00:53:29,932 [Andrey]
And then-
00:53:29,932 --> 00:53:31,522 [Seth]
[laughs]
00:53:31,522 --> 00:53:37,942 [Andrey]
... if you can get rid of it, you might already be happy if you can get rid of it, let alone get something in return for it.
00:53:37,942 --> 00:53:47,182 [Seb Krier]
It’s true. Well, yeah. That was the thing we were discussing earlier. I’d be happy to have an agent sell my records, but not buy them or some silly [chuckles] certain things.
00:53:47,182 --> 00:53:47,861 [Andrey]
Mm.
00:53:47,862 --> 00:53:51,862 [Seb Krier]
But, yeah. So I think I didn’t go too deep into that specific
00:53:53,022 --> 00:54:03,162 [Seb Krier]
paper or webpage or whatever they did on that. But there seems to be more experimentation along these lines. I think Rohit, of course, has his own kind of things. I had a MATS
00:54:04,322 --> 00:54:09,922 [Seb Krier]
stream with Yoav Shavit at OpenAI on kind of multi-agent coordination.
00:54:09,922 --> 00:54:13,992 [Andrey]
Can you explain what MATS is? Because I don’t think many in our audience know what that is.
00:54:13,992 --> 00:54:16,622 [Seb Krier]
Oh, it’s basically a fellowship.
00:54:18,662 --> 00:54:28,062 [Seb Krier]
You have mentors and mentees, and they work on all sorts of projects, and there’s close collaboration with labs. But it’s broadly on technical projects for AI safety and governance.
00:54:28,062 --> 00:54:34,122 [Andrey]
Is this a program that our junior economist listeners should try to apply to?
00:54:34,122 --> 00:54:59,482 [Seb Krier]
I think so, yeah. I don’t know if they have an econ stream. Well, they should if they don’t. And there is, I think, a wider kind of governance stream, so I wouldn’t be surprised. If they don’t have one, I would imagine there to be one eventually. But I think a lot of economists who’ve got this expertise in mechanism design or game theory, I think there’s a lot to offer here, even in the more classical AI safety type streams as well. So yeah, they should look into MATS for sure.
00:55:00,882 --> 00:55:13,182 [Seth]
Well, a lot of pickup on those answers. Very interesting stuff. I guess one question I have here that you already introduced is the idea of, okay, so maybe we’re going to get allocative efficiency from all this Coasean bargaining.
00:55:13,182 --> 00:55:13,222 [Seb Krier]
Mm-hmm.
00:55:13,222 --> 00:55:41,202 [Seth]
We’ll make sure that the lawnmower doesn’t get mowed at 2:00 a.m. when it’s really annoying for me. But there’s a distributional question here, too. So one concern that you already kind of aired out was the idea that people with better models who can pay a little bit more might be able to bargain harder and get a bigger share of the pie. Another distributional concern might be just the step zero. Which is for any sort of bargaining over
00:55:44,802 --> 00:55:48,561 [Seth]
an externality to happen, you need to assign that property right first.
00:55:48,562 --> 00:55:48,752 [Seb Krier]
Mm-hmm.
00:55:48,752 --> 00:55:52,382 [Seth]
And that could be a process that is very unequal. So,
00:55:53,822 --> 00:55:59,962 [Seth]
feel free to take up either of those concerns. Is Coasean bargaining going to lead to a dystopic inequality hellhole?
00:56:01,490 --> 00:56:12,270 [Seb Krier]
Right. I don’t think so. Certainly, I would imagine that in the first instance, I think people are still net better off than the counterfactual of not having any bargaining agents at all.
00:56:12,270 --> 00:56:13,590 [Seth]
Well, that’s the Pareto promise, right?
00:56:13,590 --> 00:56:13,610 [Seb Krier]
Yeah.
00:56:13,610 --> 00:56:18,230 [Seth]
First, Fear Theorem tells us we’re going to get a Pareto efficient, allocative efficiency outcome.
00:56:18,230 --> 00:56:18,550 [Seb Krier]
Mm-hmm.
00:56:18,550 --> 00:56:20,370 [Seth]
But, a lot of people-
00:56:20,370 --> 00:56:20,610 [Seb Krier]
Sure
00:56:20,610 --> 00:56:23,110 [Seth]
... trade this efficiency for some equality, right?
00:56:23,110 --> 00:57:04,610 [Seb Krier]
I guess to me, it depends a little bit on the complexity of the problem, and I would guess that over time, you’ll get diminishing returns. I think we’re discussing this, asking Claude Opus for her toothpaste recommendation or whatever. There’s a point to which I would imagine, I guess I don’t know this empirically, and certainly in the project deal and tropics that you had kind of better returns by using the better model, but it’s not clear to me whether over time, actually, this kind of scales linearly in terms of if you’ve got the mega guard model, you necessarily get a better deal. There’s a point to which I think this doesn’t... In the same way as Elon Musk has the same phone as you and I have, and he’s not getting that much more just by virtue of being rich or something. At least, he does in general, just not on the phone part. [chuckles] And
00:57:06,370 --> 00:57:23,230 [Seb Krier]
in any event, I think even then, to the extent that there were to be some sort of inequity, nothing stops, I think, whether it’s the state or philanthropy, whoever, to kind of subsidize access. Maybe you can have vouchers, you have systems that essentially re-equilibrate things. So I’ve always liked the voucher systems, even for private schooling and so on.
00:57:24,250 --> 00:57:26,530 [Seb Krier]
So maybe that’s one way around that.
00:57:27,630 --> 00:57:29,160 [Seb Krier]
And the property-
00:57:29,160 --> 00:57:34,850 [Seth]
Decide whether to use my precious 100,000 tokens to bargain for food or to bargain for noise complaints.
00:57:34,850 --> 00:57:40,590 [Seb Krier]
[laughs] Well, yeah. That’s on you to decide how you want to use it. [laughs] But-
00:57:40,590 --> 00:57:42,100 [Andrey]
Always has been, Seth. Always has been.
00:57:42,100 --> 00:57:42,290 [Seth]
Always has been.
00:57:42,290 --> 00:57:50,370 [Seb Krier]
Yeah. Exactly. That’s not me. And then the other one with the property right assignment is a tricky one, too. And it’s deeply political ultimately as well because-
00:57:50,370 --> 00:57:50,650 [Seth]
Right
00:57:50,650 --> 00:57:51,690 [Seb Krier]
... of course, it’s-
00:57:53,090 --> 00:57:59,050 [Seth]
It reminds me of land redistribution, right? It’s like, “All right, we’re breaking up the plantations. Who gets it?” [laughs]
00:57:59,050 --> 00:58:19,190 [Seb Krier]
Yeah. And I think, I guess what I claim in the essay or something was something like, well, you can just start with what we already have. There’s already a certain kind of distribution and allocation of rights and why do you need to just start anew? Just start with what we have. Then I did agree that there’s a need for them to change. You just then kind of, again, that’s the whole democratic process.
00:58:19,190 --> 00:58:21,110 [Seth]
We’ll start with noise complaints.
00:58:21,110 --> 00:58:21,250 [Seb Krier]
Well-
00:58:21,250 --> 00:58:24,410 [Seth]
Because right now, it’s unclear to me who has the... I have the... Yeah.
00:58:24,410 --> 00:58:43,210 [Seb Krier]
Sorry, I thought of that a little bit. I was thinking that actually, you can just keep the 11:00 PM threshold or curfew. It’s just that this switches the property right allocation. So before 11:00 PM, I have to pay you to be quiet, and after 11:00 PM, you have to pay me to be noisy or something.
00:58:43,210 --> 00:58:43,710 [Seth]
[laughs]
00:58:43,710 --> 00:58:53,540 [Seb Krier]
So I think that still works, but you come up with some sort of agreements as to where these kind of fluids, property rights, or something apply. But for these to be decided, I think you need the
00:58:54,670 --> 00:59:17,890 [Seb Krier]
whole democratic apparatus and this is the other thing I was mentioning earlier, and wanting to make that part of, I don’t know, local democracy, for example, more responsive and effective. So there’s all sorts of kind of decisions being taken every day at the local council or something that I’m completely unaware of and have, unfortunately, very little time to go and engage with. But it probably would be better for me to have-
00:59:17,890 --> 00:59:22,190 [Seth]
Unfortunately, you’re telling me that if I took away an hour of Twitter, that’s what you’d be doing.
00:59:22,190 --> 00:59:23,439 [Seb Krier]
Exactly. I could be online.
00:59:23,439 --> 00:59:26,630 [Andrey]
Wait, you could definitely be on Twitter and go to the council meeting.
00:59:26,630 --> 00:59:26,650 [Seth]
[laughs]
00:59:26,650 --> 00:59:32,150 [Seb Krier]
Yeah, but I have to focus my attention on... [laughs] But I could be consuming slop instead.
00:59:33,430 --> 00:59:34,850 [Seb Krier]
So yeah, I think that
00:59:36,030 --> 00:59:36,690 [Seb Krier]
for that, I wouldn’t
00:59:37,730 --> 00:59:40,070 [Seb Krier]
mind having an agent come back to me and say, “Oh, there’s this
00:59:41,330 --> 00:59:50,850 [Seb Krier]
new road that’s being built, and it would be pretty good for you, given that you take that route a lot and stuff, but it seems to be not going well. Do you want to register some sort of...” But
00:59:52,010 --> 01:00:16,720 [Seb Krier]
yeah. And again, I can see versions of that where this is used for vitocracy, and this is where the whole mechanism design stuff comes into play. But for the property right assignment side of things, I think you just start with what we have today, and then you adjust over time using this mechanism. And again, it’ll probably depend on the setting or where you’re doing that. The curfew one, I guess, would be with your local council, but for all sorts of other things, there’ll be different kind of actors and
01:00:17,970 --> 01:00:20,970 [Seb Krier]
institutions in place for that. You probably need... Yeah.
01:00:20,970 --> 01:00:37,450 [Seth]
Can I go, how about something like stuff that we don’t have property rights around at all right now, or even rules around, but are definitely negative externalities, right? So for example, I have a neighbor in my office who does not shower, who smells really bad, right? So how do we decide-
01:00:37,450 --> 01:00:40,330 [Andrey]
Are you willing to reveal this on a-
01:00:40,330 --> 01:00:40,500 [Seth]
No
01:00:40,500 --> 01:00:42,450 [Andrey]
... public podcast? [laughs]
01:00:42,450 --> 01:00:48,950 [Seth]
So the question is, how do we decide whether he has a right to be smelly, or I have a right to not smell him?
01:00:49,990 --> 01:01:00,549 [Seb Krier]
I guess it, yeah, once again, highly political. [laughs] But I think my bias, of course, is one that’s more, I guess, freedom maximizing a classical liberal or something. So it’s more like-
01:01:00,550 --> 01:01:01,490 [Seth]
The right to be smelly.
01:01:01,490 --> 01:01:01,630 [Seb Krier]
Yeah.
01:01:01,630 --> 01:01:01,990 [Seth]
There we go.
01:01:01,990 --> 01:01:06,630 [Seb Krier]
It’s the harm principle, right? He’s not harming you by being smelly, unfortunately, so you’re going to have to-
01:01:06,630 --> 01:01:08,940 [Andrey]
What do you mean? No, he definitely harms me.
01:01:08,940 --> 01:01:09,910 [Seth]
It really smells bad, dude.
01:01:09,910 --> 01:01:14,540 [Seb Krier]
[laughs] It’s not decreasing your longevity as far as I’m aware. [laughs]
01:01:14,540 --> 01:01:17,550 [Seth]
[laughs] Actually, it makes me want to decrease my longevity.
01:01:17,550 --> 01:01:33,610 [Andrey]
But no, but more seriously, this is actually a great use case for agents because the agent could receive anonymous... A lot of effective altruist rationalist types have this anonymous-
01:01:33,610 --> 01:01:33,910 [Seb Krier]
Oh, yeah
01:01:33,910 --> 01:01:36,710 [Andrey]
... link in their profile, where you can give them anonymous
01:01:38,350 --> 01:01:38,690 [Andrey]
feedback-
01:01:38,690 --> 01:01:39,710 [Seb Krier]
Yeah
01:01:39,710 --> 01:01:43,390 [Andrey]
... about how much they smell. And so with the world of agents then-
01:01:43,390 --> 01:01:43,700 [Seth]
[laughs]
01:01:43,700 --> 01:01:49,150 [Andrey]
... an agent could figure out a way to give anonymous feedback to the person’s agent
01:01:50,350 --> 01:01:52,190 [Andrey]
that the person smells.
01:01:52,190 --> 01:01:52,550 [Seb Krier]
Well-
01:01:52,550 --> 01:01:55,420 [Andrey]
And then that agent could pass it on to the person
01:01:56,206 --> 01:02:14,466 [Seb Krier]
Exactly. To be fair, you can also do that without the agent. You can just put a message on their desk saying, “By the way, please just shower,” or something. [laughing] But I think that in principle, what would be actually more interesting is having the agent solve that problem without the person necessarily being offended by the thing or having the message straight away.
01:02:15,806 --> 01:02:20,186 [Seb Krier]
So having the other agent just... This is getting into nudging territory, right? And this is also controversial in its own right.
01:02:21,426 --> 01:02:43,126 [Seb Krier]
But it actually connects to this other question of do you want your agent to be purely instruction following, or do you want it to help you maybe achieve some higher order goals? And so maybe one of my goals is I want to be not disliked at the office. And so one of the ways is my agent, well, I’m not the smelly guy, but let’s assume [laughing] then the agent will kind of find a subtle way-
01:02:43,126 --> 01:02:43,416 [Seth]
Sure.
01:02:43,416 --> 01:02:46,686 [Seb Krier]
... to be like, “Seb, it’s time to log off Twitter and go and take a shower.”
01:02:47,926 --> 01:02:53,455 [Seb Krier]
[laughing] So yeah, that might be one of them. But yeah, in terms of an initial allocation, I don’t think that he would
01:02:54,986 --> 01:02:58,706 [Seb Krier]
have to pay me to continue being smelly perhaps.
01:02:59,986 --> 01:03:01,185 [Seb Krier]
[chuckles]
01:03:01,186 --> 01:03:02,746 [Seth]
Okay. So
01:03:04,826 --> 01:03:13,406 [Seth]
let me give you some concerns about this vision. Right? So we’ve laid out what could be beautiful about this. We can efficiently allocate externalities, blah, blah, blah.
01:03:14,766 --> 01:03:30,326 [Seth]
One set of concerns I’ve heard from my non-economist friends when I talk about this is this idea that micro bargaining at scale could harm social legitimacy. So if I may give you two exaggerated sci-fi versions of this, and then maybe we can reel it back in into reality.
01:03:30,326 --> 01:03:31,186 [Seb Krier]
Mm-hmm.
01:03:31,186 --> 01:03:37,106 [Seth]
The first version of this comes from the video game Helldivers 2. Are you familiar with that game?
01:03:37,106 --> 01:03:39,606 [Seb Krier]
Yeah, I haven’t played it, but I’m aware broadly of the... Yeah.
01:03:39,606 --> 01:04:29,386 [Seth]
All right. So the premise is kind of like a Starship Troopers sort of universe, where it’s officially a democracy, but really it’s fascism. And their political system is called Managed Democracy. And the way that Managed Democracy works is everyone fills out a preference questionnaire about what they like and they don’t like, and then that goes up into the big algorithm in the sky, and then boop boop boop boop boop, here’s your optimal social policy that we’ve calculated over everyone. Right? So on the one hand, it kind of seems like that is one end limit of what Coasean bargaining at scale would look like. I just fill out a survey, and my AI negotiates with all the other AIs, and I don’t really interact with it beyond that. But you can kind of see how dystopic that looks. It has a lot less social legitimacy than maybe two choice first past the post elections.
01:04:29,386 --> 01:04:39,366 [Seb Krier]
Yeah, I agree. That’s the extreme version of it or something. Seems very undesirable. And so that’s why I was very careful to say, by the way, this is not saying make every single aspect of your life
01:04:41,266 --> 01:04:46,186 [Seb Krier]
a transaction. In fact, I think there’s the whole Michael Sandel stuff, right?
01:04:46,186 --> 01:04:47,666 [Seth]
Oh, Sandel. That’s my next question.
01:04:47,666 --> 01:04:47,906 [Seb Krier]
Right.
01:04:47,906 --> 01:04:49,746 [Seth]
You’re jumping ahead. [chuckles]
01:04:49,746 --> 01:04:51,046 [Seb Krier]
Well, but I think that the
01:04:52,406 --> 01:05:22,676 [Seb Krier]
idea, at least for me, is not for the agent to completely automate your agency away. If anything, ideally, this kind of enhances it in some way. So, I like the idea of the local authority thing because it’s something I wouldn’t be doing in any event. It’s not like I’m having this taken away from me and that, but for the agent, I would be going there or something. So I think there has to be some sort of conscious choice as well from individuals to where do they want to kind of exercise their agency and where do they not. And I think there’s some people who actually do not care as much about participating and
01:05:23,926 --> 01:05:25,386 [Seb Krier]
others who do, and I think that’s fine.
01:05:27,146 --> 01:05:38,386 [Seb Krier]
But I agree that also, there are kind of non-market norms that I think are worth preserving, reciprocity and being a good neighbor and civic participation. And you don’t want to kind of get rid of these entirely either. So,
01:05:39,406 --> 01:06:07,506 [Seb Krier]
in my mind, this will never be the kind of perfect maximalist version of the whole thing. It’ll probably work in some instances. In others, you’re not going to use the system. And in general, I suspect that for many things and trivial things, like actually the smelly coworker, I don’t know, maybe that one is the one where you actually want to use an agent rather than deal with it yourself. But I agree, and it’s a matter of, I guess, developing the right norms around how we would be using there. Where is it acceptable, where is it not?
01:06:09,126 --> 01:06:31,386 [Seb Krier]
But I agree with the bad version of this where no one does anything, and you just have an agent infer everything on your behalf, and now you’re just basically removing the whole participatory element of democracy, which I think is still very important. And so again, here too you will want your agent... In a way it’s not dissimilar to
01:06:33,146 --> 01:06:41,326 [Seb Krier]
these questions of cognitive offloading and so on. Of like, well, if the AI writes on your behalf versus it gives you advice on how you can write the thing better.
01:06:42,606 --> 01:06:46,106 [Seb Krier]
And so now I’m going a bit on a tangent, but I was mentioning this
01:06:47,326 --> 01:06:59,026 [Seb Krier]
positive alignment thing earlier of do you want your agent to be purely instruction and then following, or would you want to specify higher order preferences? I think this is where this comes into play, where I might say, actually, I
01:07:02,126 --> 01:07:06,326 [Seb Krier]
want you to help me achieve this kind of higher order goal or something, which may entail, in fact, asking you to just
01:07:07,346 --> 01:07:46,266 [Seb Krier]
read up on this. And there’s also an element of me actually needing to retain that choice as well and as an individual, right? I will sometimes summarize a paper using an AI, and sometimes I will... And that’s fine because the counterfactual wouldn’t have not been me reading the whole paper because I don’t have... But there are also papers where I would have read the whole paper, and I’m still asking the AI to do it, and that’s kind of also on me to learn and for, I guess, the right norms to rise in terms of, okay, when it’s too much, what is right, what is not. In the same way as, I don’t know, I’ve learned how to do regressions by hand or something one day in the past, and that was helpful in some sense, but then I’ve never done that again, and I’ve started using Stata and R or whatever. So,
01:07:47,406 --> 01:07:47,985 [Seb Krier]
there’s an element of-
01:07:47,986 --> 01:07:48,906 [Seth]
What?
01:07:48,906 --> 01:07:50,786 [Seb Krier]
[chuckles] Actually, yeah.
01:07:50,786 --> 01:07:52,886 [Seth]
Are you telling us you’re not an economist?
01:07:53,966 --> 01:07:55,196 [Seb Krier]
No, I am not an economist.
01:07:55,196 --> 01:07:55,246 [Seth]
You got into Stata and R.
01:07:55,246 --> 01:08:11,046 [Seb Krier]
I was never an economist. [laughing] So yeah, there’s this wider question of where do you want to preserve agency and then also where you want to preserve certain norms on a day-to-day level. And so I think it’s going to be a big messy thing in reality as opposed to the maximalist version of either vision.
01:08:12,142 --> 01:09:27,362 [Seth]
For those of you playing along at home, now is your chance to think about how this conversation has changed your priors. This chance to contemplate your posteriors is sponsored by Revelio Labs. Revelio Labs is a leading provider of labor economics data and data services for companies, academics, and independent researchers. Andrey and I have been working in economics of AI, digitization, and automation for a long time, and we can confirm just how useful Revelio’s data is. Revelio’s team combines comprehensive micro-level data on employee professional profiles, job postings, and employee sentiment with standardizations, mappings, and enrichments available, all to make that data useful without making your modeling decisions for you. The data can be flexibly aggregated to company, market, or industry, and can be used to study questions ranging from career trajectories to occupational transformation, to the returns to skills, and the impact of AI on labor demand for tasks. Can’t imagine anyone who would be interested in that. And Revelio data is available on WRDS. So if you’re an academic with a good library, go see if you have access to their premier data already. And if you don’t, you can reach out to their excellent economics team, and they’ll hook you up. I feel like it’s time for lightning round.
01:09:27,362 --> 01:09:27,682 [Andrey]
All right.
01:09:27,682 --> 01:09:28,342 [Seth]
Let’s do it.
01:09:28,342 --> 01:09:33,702 [Andrey]
Let’s do it. [upbeat music]
01:09:34,802 --> 01:09:35,962 [Andrey]
So we’re-
01:09:35,962 --> 01:09:35,972 [Seth]
Nice
01:09:35,972 --> 01:09:39,372 [Andrey]
... yeah, very excited because you’re a master of the take.
01:09:39,372 --> 01:09:41,252 [Seth]
[laughs]
01:09:41,252 --> 01:09:41,742 [Andrey]
[laughs]
01:09:43,282 --> 01:09:47,702 [Andrey]
So you fairly recently moved from London to New York City.
01:09:47,702 --> 01:09:48,242 [Seth]
Mm-hmm.
01:09:48,242 --> 01:09:49,122 [Andrey]
What are
01:09:50,422 --> 01:09:53,022 [Andrey]
the words that come to mind-
01:09:53,022 --> 01:09:53,312 [Seth]
[laughs]
01:09:53,312 --> 01:09:56,762 [Andrey]
... when you’re thinking about AI in New York City versus AI in London?
01:09:58,042 --> 01:10:05,402 [Seth]
Oh. AI in London feels more academic, theoretic,
01:10:06,562 --> 01:10:53,002 [Seth]
I guess more deep researchy, and more governance-oriented in some way. I think there’s this whole Oxford, Cambridge, UCL connection, the government and UK AI Safety Institute, the fact that government was already dealing with AI quite early. I think that that whole ecosystem is fairly developed here, whereas New York is a bit of a weird, secret third thing of like, well, there’s some interesting people, more like heterodox, random, the edge of culture. Some interesting kind of frontier AI people too, but I wouldn’t really see it as some sort of AI capital or something. It’s just a bit more diverse in terms of the offering. But I kind of like that. That’s the point also. I don’t want to be fully NSF. I kind of like being somewhat distant.
01:10:54,062 --> 01:10:59,551 [Andrey]
What is the reaction of people you meet in Brooklyn or at your-
01:10:59,551 --> 01:11:00,202 [Seth]
[laughs]
01:11:00,202 --> 01:11:04,382 [Andrey]
... or at your EDM shows when you tell them where you work?
01:11:04,382 --> 01:11:07,771 [Seth]
First of all, I do not like EDM at all. But- [laughs]
01:11:07,771 --> 01:11:08,111 [Andrey]
[laughs]
01:11:08,111 --> 01:11:08,122 [Seth]
[laughs]
01:11:08,122 --> 01:11:09,242 [Andrey]
Whoa.
01:11:09,242 --> 01:11:09,322 [Seth]
[laughs]
01:11:09,322 --> 01:11:10,382 [Andrey]
Hottest thing right now.
01:11:10,382 --> 01:11:11,782 [Seth]
Yeah. Going-
01:11:11,782 --> 01:11:13,782 [Andrey]
So how much would you have your DJ-
01:11:13,782 --> 01:11:13,802 [Seth]
Electronic-
01:11:13,802 --> 01:11:14,162 [Andrey]
... mic?
01:11:14,162 --> 01:11:16,912 [Seth]
Fine electronic music. [laughs]
01:11:16,912 --> 01:11:19,672 [Andrey]
[laughs] Oh, yeah, that one’s good. The good-
01:11:19,672 --> 01:11:20,212 [Seth]
[laughs]
01:11:20,212 --> 01:11:20,262 [Andrey]
[laughs]
01:11:20,262 --> 01:11:21,682 [Seth]
Aligned electronic music.
01:11:23,082 --> 01:11:24,232 [Andrey]
Yes. [laughs]
01:11:24,232 --> 01:11:25,732 [Seth]
[laughs] Yeah, I think it
01:11:29,002 --> 01:11:41,262 [Seth]
wouldn’t be the first thing I tell people, perhaps, when I meet someone in Brooklyn. It’s like, “Oh, I work for Vigo AI Lab, by the way.” But I think it depends who and where and what. You get a bit of the stereotypes of, oh, AI bad, evil, blah, blah.
01:11:42,402 --> 01:11:51,922 [Seth]
But you also have some cool communities and interesting creative scenes that actually do engage with technology AI in very interesting ways. And so,
01:11:53,402 --> 01:11:56,412 [Seth]
there’s a bit of everything, but that’s kind of the point also. I’m
01:11:57,522 --> 01:12:01,051 [Seth]
enjoying being here, so I don’t have to talk about AI with every random person I meet. [laughs]
01:12:01,051 --> 01:12:01,122 [Andrey]
[laughs]
01:12:01,122 --> 01:12:11,922 [Seth]
And there’s all these other things that I can kind of explore and discuss and learn about, too. But the attitude really depends on where I end up. There’s a bit of a selection effect going on as well, to be fair. [laughs]
01:12:11,922 --> 01:12:16,022 [Andrey]
The coolest thing in terms of art and AI that you’ve seen out there?
01:12:17,122 --> 01:12:17,402 [Seth]
Hmm.
01:12:18,982 --> 01:12:20,202 [Seth]
In New York specifically, a
01:12:23,142 --> 01:12:27,582 [Seth]
lot of the recent art stuff I’ve been to was not really AI related. Although next week, though, I
01:12:28,862 --> 01:12:31,182 [Seth]
am in fact co-organizing, helping
01:12:32,302 --> 01:12:43,542 [Seth]
some sort of art dinner thing with a bunch of interesting artists coming from London [smacks lips] and showcasing a lot of their work. And so I think they’re called Biangi Systems.
01:12:44,862 --> 01:13:01,742 [Seth]
I’ll try to put a link in the chat or something. But, yeah, shout out Biangi. But so they do interesting stuff, and you get a lot more interesting artifact from their work than, say, a one-shot Suno prompt or something. [laughs]
01:13:01,742 --> 01:13:03,842 [Andrey]
All right. So then natural segue then.
01:13:03,842 --> 01:13:04,902 [Seth]
[laughs]
01:13:04,902 --> 01:13:14,002 [Andrey]
What is your philosophy of electronic music? Or what do you like about electronic music? What do you not like about electronic music? Yeah.
01:13:17,262 --> 01:13:18,002 [Seth]
I don’t think there’s a
01:13:20,022 --> 01:13:24,322 [Seth]
whole philosophy behind it. I think I just kind of like some types of electronic music. I
01:13:26,082 --> 01:13:30,882 [Seth]
like a lot of late ‘80s, early ‘90s electronic music. And they had a particular kind of...
01:13:32,702 --> 01:13:49,742 [Seth]
They’re basically the original futurist in some sense. And a lot of the themes and the music was very much sometimes around, in fact, AI, or sometimes around robots, sometimes about the future, and both utopian and dystopian visions of the future. And I don’t know. I like the kind of
01:13:51,462 --> 01:13:59,982 [Seth]
[smacks lips] audio equivalent of all these stories or something. I read too much, and so [chuckles] it’s nice to have something that’s a bit more like
01:14:01,402 --> 01:14:02,422 [Seth]
that engages in a different way.
01:14:03,822 --> 01:14:07,682 [Seth]
So I don’t know. There’s a lot of early ‘90s techno, acid, trance.
01:14:07,682 --> 01:14:09,822 [Andrey]
What are those for our listeners?
01:14:09,822 --> 01:14:16,582 [Seth]
Well, Drexciya is the kind of canonical one, which is kind of Detroit electro techno from the early ‘90s.
01:14:17,270 --> 01:14:24,430 [Seb Krier]
And they have a whole mythology around the aesthetics, the themes, the kind of
01:14:25,530 --> 01:14:26,430 [Seb Krier]
storyline behind
01:14:27,530 --> 01:14:30,670 [Seb Krier]
the tracks and the EPs and albums they put out.
01:14:31,770 --> 01:14:45,630 [Seb Krier]
So I think Drexciya is a really interesting one. There’s also very cool long articles about them, I think James Stinson and Gerald Donald, and a few others. I think a lot of that Detroit scene at the time was very much into that stuff. So, Ox88, Jeff Mills, Underground Resistance, a lot of the
01:14:46,730 --> 01:14:52,950 [Seb Krier]
classic techno electro stuff from that time. So these would be some examples, but the UK had some really cool stuff too.
01:14:54,770 --> 01:15:02,690 [Seb Krier]
Yeah. And I guess I like the weirder side of music, and when it’s a bit more creative. Which is why I’m going to--
01:15:04,050 --> 01:15:08,110 [Seb Krier]
To the earlier discussion about creativity, I kind of think of slop as
01:15:09,310 --> 01:15:12,330 [Seb Krier]
very much something that’s kind of highly predictable.
01:15:13,910 --> 01:15:18,270 [Seb Krier]
Low creativity, high predictability, low effort.
01:15:21,110 --> 01:15:44,950 [Seb Krier]
One thought I had was that over time, you see this kind of lowering of the barriers to all sorts of art forms. So even this early ‘90s stuff. You had a lot of really good concentrated stuff, but very few people had access to this stuff. And of course, as these synthesizers would be cheaper, as they’re then made kind of digital, as you had tools like Ableton and whatnot, a lot more people were able to create and make music. But as a result of that,
01:15:46,030 --> 01:15:51,810 [Seb Krier]
you had a lot more of the mediocre and average stuff too. But that’s fine. There’s also a lot more of the really good stuff, and if you’re
01:15:53,110 --> 01:15:57,030 [Seb Krier]
good at finding it, or if you can rely on good algorithms, you can filter through the slop.
01:15:58,710 --> 01:16:00,190 [Seb Krier]
Yeah. That’s one thought.
01:16:01,670 --> 01:16:05,730 [Andrey]
I think that the
01:16:07,070 --> 01:16:11,130 [Andrey]
current-- I don’t even know current. Over the past decade, the-
01:16:11,130 --> 01:16:13,170 [Seth]
[background chatter]
01:16:13,170 --> 01:16:13,330 [Andrey]
... the
01:16:14,370 --> 01:16:23,150 [Andrey]
EDM wave. Does it feel very sloppy to you, even somehow before we even had AI-generated music?
01:16:23,150 --> 01:16:27,769 [Seb Krier]
Yeah, definitely. But on the one hand, it’s kind of my taste. I’m sure there’s
01:16:28,990 --> 01:16:34,890 [Seb Krier]
a big level of subjectivity in a lot of this stuff. And then also there’s-- I think you had-
01:16:34,890 --> 01:16:36,650 [Seth]
This podcast is not Yuck Yums.
01:16:36,650 --> 01:16:37,010 [Seb Krier]
[laughs]
01:16:38,790 --> 01:16:40,000 [Seb Krier]
But also, there’s I think all sorts of--
01:16:41,650 --> 01:16:49,370 [Seb Krier]
You had slop equivalent in other fields, in music genres, and domains over time. And I think even in the, I don’t know, was it the ‘80s or something, people would consider,
01:16:51,150 --> 01:17:08,830 [Seb Krier]
I would say smooth jazz as being the equivalent of slop at the time or something, right? [laughing] And I like smooth jazz. I’m really keen on smooth jazz. Even library music or something at the time. So yeah, library music actually was seen as at the time it was just mass-produced, super commercial beast film score music that
01:17:10,030 --> 01:17:28,649 [Seb Krier]
was probably kind of not seen with a lot of, I don’t know, awe and respect or something. Maybe the bye bye. Whereas now it is, of course, because what is slop also changes over time. But yeah, I think just like photography. Once that got democratized, you’ve got trillions of photos every day, and there’s a huge amount of slop in that whole sea of photos that people take.
01:17:28,650 --> 01:17:33,630 [Seth]
And there’s like you say, there’s also a going back and rediscovering. Lovecraft at one point was slop.
01:17:33,630 --> 01:17:33,820 [Seb Krier]
Yeah.
01:17:33,820 --> 01:17:35,130 [Seth]
And now he’s beloved.
01:17:35,130 --> 01:17:36,750 [Seb Krier]
Yeah. Exactly. There’s
01:17:37,870 --> 01:18:21,510 [Seb Krier]
a lot of house music from at some point that was just deemed pretty kind of basic or something, and now it’s coming back a little bit in some communities in some shape or form. So it’s a weird mix of taste and rediscovering old stuff and scarcity because people like being special unique snowflakes and so on. So there’s [chuckles] also the different dynamics at play. But yeah, in my mind, I don’t like EDM or something, but who knows? Maybe in 20 years my tastes will have changed, and maybe I’ll look back and be like, “Oh, actually there’s some pretty good stuff here.” I remember very vividly when I was 18 thinking, “I really hate the sound of acid.” As in the acid techno, these two or three machines. I was like, “I can’t understand how one would like that.” And then literally five years later, six years later, I was just obsessed with acid. And I was like, “How did that happen?”
01:18:22,870 --> 01:18:26,380 [Seth]
It’s because the opposite of love isn’t hate, the opposite of love is indifference.
01:18:26,380 --> 01:18:26,490 [Seb Krier]
[laughs]
01:18:26,490 --> 01:18:30,370 [Seth]
The fact that you hated it so much was a sign.
01:18:30,370 --> 01:18:37,149 [Seb Krier]
Well, I didn’t hate it that much. I just didn’t find it particularly interesting or pleasurable, whatever, at the time. But so yeah.
01:18:38,490 --> 01:18:38,700 [Seb Krier]
Who knows?
01:18:38,700 --> 01:18:41,090 [Seth]
You mentioned reading. You mentioned reading a lot.
01:18:41,090 --> 01:18:41,290 [Seb Krier]
Uh-huh.
01:18:41,290 --> 01:18:43,470 [Seth]
What’s the book that’s been the most influential on you?
01:18:47,170 --> 01:18:53,470 [Seb Krier]
I don’t know. I definitely don’t have an immediate answer to that question because I think particularly in the last decade, my
01:18:54,910 --> 01:18:58,510 [Seb Krier]
consumption habits have been a lot more chaotic.
01:18:58,510 --> 01:18:58,950 [Seth]
[chuckles]
01:18:58,950 --> 01:19:04,630 [Seb Krier]
So there’s been a bit of-- I don’t know. If you asked me the same thing 10 years ago, I would’ve said something like, oh, “On Liberty” by John Stuart Mill or something.
01:19:05,830 --> 01:19:11,150 [Andrey]
See, that’s why-- Isn’t that Tyler’s favorite philosopher?
01:19:11,150 --> 01:19:12,160 [Seb Krier]
I don’t know if it is.
01:19:12,160 --> 01:19:13,030 [Andrey]
Sorry. Economist, excuse me.
01:19:13,030 --> 01:19:13,050 [Seth]
No.
01:19:13,050 --> 01:19:14,270 [Andrey]
His favorite economist. Yeah.
01:19:14,270 --> 01:19:16,750 [Seb Krier]
Right. Well, once again, great minds. What can I say?
01:19:17,890 --> 01:19:21,190 [Seb Krier]
[laughing] But no, I do really like John Stuart Mill. But I
01:19:22,550 --> 01:19:30,570 [Seb Krier]
wouldn’t-- Right now, I think it’s kind of a weird patchwork of loads of different kind of text sources.
01:19:30,570 --> 01:19:32,590 [Seth]
You mentioned Bostrom’s “Superintelligence” during the conversation.
01:19:32,590 --> 01:19:44,470 [Seb Krier]
Yeah. I mean, that one was clearly super influential. Because I think that was-- He published in 2014, I think, and in 2016, ‘17 was when I was starting to read it. And that got me to actually kind of what is part of the motivation to leave my job as a lawyer at the time.
01:19:45,870 --> 01:19:54,570 [Seb Krier]
And see that AI was obviously going to be a big thing, and you had a need for non-technical work as well in that whole thing. So,
01:19:55,690 --> 01:19:59,790 [Seb Krier]
that was fairly influential, I think. Yeah, Bostrom’s “Superintelligence” clearly played a role.
01:20:01,918 --> 01:20:04,138 [Seth]
Mentioned being loved by Tyler
01:20:04,138 --> 01:20:06,547 [Seb Krier]
I didn’t mention that, you did. [laughs]
01:20:06,547 --> 01:20:09,838 [Seth]
Question, why does he love you so much? Do you have kompromat on him?
01:20:09,838 --> 01:20:14,848 [Seb Krier]
[laughs] No, I don’t know. I was really happy that he kind of enjoyed my
01:20:16,458 --> 01:20:38,538 [Seb Krier]
hot takes and rambles on Twitter. And I’ve messaged him to say the same. And funnily enough, I’ve been a kind of a Marginal Revolution reader since the last 15 years or so, or for a very long time. So it’s actually pretty weird for me to be like, “Oh, cool. My name’s popping up on the blog,” and stuff. A lot of my friends, in here at least, knew about Marginal Revolution, so they’re like, “What are you on about? Who cares?” And to me-
01:20:38,538 --> 01:20:39,447 [Andrey]
[laughing]
01:20:39,447 --> 01:20:46,158 [Seb Krier]
... that was pretty cool. Like, yeah, Seb Krier, whatever. And like, all right. But, no, I don’t know. But I think I
01:20:48,038 --> 01:20:52,378 [Seb Krier]
like Tyler and his work a lot. I think Tabarrok as well. I think the whole,
01:20:54,598 --> 01:20:55,298 [Seb Krier]
what’s it called,
01:20:57,618 --> 01:21:14,238 [Seb Krier]
that cluster of blogosphere online economics, GMU, and so on, has been fairly influential, I think as well for me. Even 15 years ago, when I was kind of switching from my Francophone world and upbringing to the kind of more Anglophone where discovering the Adam Smith Institute in London or something.
01:21:15,278 --> 01:21:17,598 [Seb Krier]
All these different influences, I think, definitely had a
01:21:18,698 --> 01:21:20,618 [Seb Krier]
lasting effect in some sense.
01:21:20,618 --> 01:21:24,058 [Andrey]
Pretty good time to wrap up. Is there anything else you want
01:21:25,538 --> 01:21:31,078 [Andrey]
to direct our audience to? Any underappreciated people, ideas,
01:21:32,138 --> 01:21:32,738 [Andrey]
anything else?
01:21:34,077 --> 01:21:34,217 [Seb Krier]
[laughing]
01:21:35,358 --> 01:21:35,798 [Seth]
[laughing]
01:21:35,798 --> 01:21:41,178 [Seb Krier]
That’s a wide question. I don’t know. There’s all sorts of interesting people and ideas. I
01:21:43,518 --> 01:21:46,418 [Seb Krier]
don’t know. Cosmos Institute has been doing some interesting work recently.
01:21:46,418 --> 01:21:47,358 [Andrey]
Mm-hmm.
01:21:47,358 --> 01:22:02,158 [Seb Krier]
I think, as well as Fire, actually. They’ve kind of done some stuff together, which I find very valuable. Yeah, I’d have to think about that one more. I’d prefer preparing a nicely curated list of underrated-
01:22:02,158 --> 01:22:02,168 [Andrey]
Yeah. [laughs]
01:22:02,168 --> 01:22:04,378 [Seb Krier]
... blogs, people, and ideas rather than-
01:22:04,378 --> 01:22:06,718 [Andrey]
Well, you should tweet that. That could be your next-
01:22:06,718 --> 01:22:08,348 [Seth]
There we go. Yeah, I’ll retweet it
01:22:08,348 --> 01:22:09,598 [Andrey]
... super tweet. Yeah.
01:22:09,598 --> 01:22:13,898 [Seb Krier]
Yeah, no, for sure. I’m sure, that’d be a good idea, actually, because I think there
01:22:15,278 --> 01:22:31,458 [Seb Krier]
are a lot of interesting people in the AI world right now doing cool work that isn’t maybe as discussed or something. There’s Calcifer Computing did some interesting work recently. Yeah, there’s a bunch of people here and there, but I’ll have to think about it to give you a more interesting answer.
01:22:32,998 --> 01:22:35,147 [Seth]
Great. We’ll put them in the show notes if you think about it.
01:22:35,147 --> 01:22:35,318 [Seb Krier]
Sounds good.
01:22:36,778 --> 01:22:44,998 [Seth]
Seb, it’s been such an honor having you on. You were an awesome participant and had a lot of patience with our silly questions. [laughs]
01:22:44,998 --> 01:22:52,738 [Seb Krier]
No, it’s been a pleasure. I’ve been a big fan of the show. So again, just like Marginal Revolution, now I can say I’ve got the Justified Posterior’s take. [chuckles]
01:22:52,738 --> 01:22:53,238 [Seth]
There we go.
01:22:53,238 --> 01:22:54,638 [Andrey]
Nice. Thank you.
01:22:54,638 --> 01:22:54,958 [Seb Krier]
Thank you.
01:22:54,958 --> 01:22:56,198 [Andrey]
All right.
01:22:56,278 --> 01:22:56,558 [Seb Krier]
Cool.
01:22:56,558 --> 01:22:58,978 [Andrey]
Keep your posteriors justified. And-
01:22:58,978 --> 01:23:00,338 [Seb Krier]
[laughs]
01:23:00,338 --> 01:23:04,318 [Andrey]
... like, follow, comment, subscribe, et cetera.
01:23:04,318 --> 01:23:04,958 [Seb Krier]
And retweet.
01:23:04,958 --> 01:23:05,598 [Andrey]
Thank you.
01:23:05,598 --> 01:23:09,508 [Seb Krier]
And retweet. [laughs]
01:23:09,508 --> 01:23:17,198 [Andrey]
Of course. [outro music]
Get full access to Justified Posteriors at empiricrafting.substack.com/subscribe
