“The fact is that its input came from billions of humans… When you’re interacting with an LLM, you are interacting with a collective, not a singular intelligence sitting out there in the universe.”

–Michael I. Jordan

Michael I. Jordan is the Pehong Chen Distinguished Professor in Electrical Engineering and Computer Science and professor in Statistics at the University of California, Berkeley, and chair of Markets and Machine Learning at INRIA Institute in Paris. His many awards include the World Laureates Association Prize, IEEE John von Neumann Medal, and the Allen Newell Award. He has been named in the journal Science as the most influential computer scientist in the world.

Website:

arxiv.org

LinkedIn Profile:

Michael I. Jordan

University Profile:

Michael I. Jordan

Redefining the meaning of intelligence

The social and cultural roots of human genius

Why AI is not true superintelligence

Collective genius as the driver of innovation

The missing link between economics and AI

Decision making under uncertainty and asymmetry

Building AI systems for social welfare

Ross Dawson: Michael, it’s wonderful to have you on the show.

Michael I. Jordan: My pleasure to be here.

Ross: Many people seem to be saying that AI is going to beat all human intelligence very soon. And I think you have a different opinion.

Michael: Well, there’s a lot of problems with that framing for technology. First of all, we don’t really understand human intelligence. We think we do because we’re intelligent, but there’s depths we haven’t probed, and there’s the field of psychology just getting going—not to mention neuroscience.

So just saying that something that mimics humans, or took a vast amount of data and brute-forced mimicked humans, seems like a kind of leap to me—that it has human intelligence nailed. Moreover, the idea that it was a sequence of logic doesn’t particularly work for me. We figured out human intelligence, now we can put it in silicon and scale it, and therefore we’ll get superintelligence.

Every step there I mean the scaling part, I guess, is okay, but we have not figured out human intelligence. Even if we had, it’s not really clear to me as a technology that our goal should be to mimic or replace humans. In some jobs, sure, but we should think more about overall social welfare and what’s good for humans. How do we complement humans?

So, no, I don’t think we’ve got human intelligence figured out at all. It’s not that it’s a mystical thing, but we have creativity. We have experience and shared experience, and we plumb the depths of that when we interact and when we create things.

Those machines that are doing brute force gradient descent on large amounts of text and even images or whatever—they’re not getting there. It is brute force. I don’t think sciences have really progressed by just having brute force solutions that no one understands and saying, “That’s it, we’re done.”

So if you want to understand human intelligence It’s going to be a while.

Ross: There’s a lot to dig into there, but perhaps first: just intelligence. You frame that as, among other things, social and cultural, not just cognitive?

Michael: Absolutely. I don’t think if you put me on a desert island, I’d do very well. I need to be able to ask people how to do things. And if you put me not just on a desert island, but in a foreign country, and you don’t give me the education—the 40 years of education I had as well—that imbued me with the culture of our civilization.

Anytime I’m not knowledgeable about something, I can go find it, and I can talk to people. Yes, I can now use technology to find it, but I’m really talking to people through the technology. I don’t think we appreciate how important that cultural background is to our thinking, to our ability to do things, to execute, and then to figure out what we don’t know and what we’re not good at. That’s how we trade with others who are better at it, how we interact, and all that.

That’s a huge part of what it means to be human, and how to be a successful and happy human. This mythological Einstein sitting all by himself in a room, thinking and pondering—I think we’re way too wedded to that. That’s not really how our intelligence is rolled out in the real world.

Generally, we’re very uncertain about things in the real world. Even Einstein was uncertain, had to ask others, learn things, and find a path through the complexity of thought.

Also, I’ve worked on machine learning for many years, and I’m pretty comfortable saying that learning is a thing we can define, or at least start to define: you improve on certain tasks. Intelligence—I’m just much less happy with trying to define it. I think there’s a lot of social intelligence, so I’m using that term loosely. But human, single intelligence—what is that? What does it mean to generalize it?

Talking about thought in the computer is the old dream of AI. I don’t know if we have thought in a computer. Some people sort of say, “Yeah, we have it,” because it’s doing these thinking-like things. But it’s still making all kinds of errors. You can brute force around them for as long as you can and get humans to aid you when you’re making errors.

But at some point you have to say, “Wait a minute, I haven’t really understood thought. I’m not getting it. I’m getting something else. What am I getting? How do I understand that? How does it complement things? How does it work in the real world?”

Then you need to be more of an engineer—try to build it in a way that actually works, that is likely to help out human beings, and think like an engineer and less like a science fiction guru.

Ross: So you’ve used the phrase “human genius” as a sort of what we compare AI with. And the phrase “human collective genius,” I suppose, ties into some of your points here—where that genius, or that ability to do exceptional things, is a collective phenomenon, not an individual one.

Michael: Oh no, without a doubt. I’ve known some very creative people, and every time you talk to them, they make it very clear that the ideas came from the ether—from other people. Often, they just saw the idea develop in their brain, but they don’t know why.

They are very aware of the context that allowed them to see something differently, execute on it, and have the tools to execute. So my favorite humans are smart and humble. Right now in technology, we have a lot of people who are pretty smart but not very humble, and they’re missing something of what I think of as human genius: the ability to be humble, to understand what you don’t know, and to interact with other humans.

Ross: One of the other things you emphasize is when we’re designing these systems. We’ve created some pretty amazing things. But as you suggest, there seems to be this very strange obsession with artificial general intelligence as a focus.

For all of the reasons that’s flawed, one of them is being able to imbue social welfare as a fundamental principle that we should be using to design these.

Michael: I think you’ve just hit on it. To me, that’s the fundamental flaw with it. I mean, you can say the flaw is that you can’t define it, and so on and so forth. But for me, the flaw is really that it’s an overall system.

In fact, if you think about an LLM, whether it’s smart or not, or intelligent or not, it’s almost beside the point. The fact is that its input came from billions of humans, and those humans did a lot of thinking behind that. They worked out problems, they wrote them down, they created things. Sometimes they agreed, sometimes they disagreed, and the computer takes all that in.

To the extent that there’s signal, and there’s a lot of agreement among lots of humans, it’s able to amplify that and create some abstractions that characterize that. But when you’re interacting with an LLM, you are interacting with essentially all those humans. You’re interacting with a collective. You are not interacting with a singular intelligence sitting out there in the universe.

You’re interacting with all of humanity—or at least a lot of humanity—and all of the background that those people brought to it. So if you’re interacting with a collective, then you have to ask: is there a benefit to the collective, and what’s my contribution? What’s my role in that overall assemblage of information?

It’s not just that the whole goal is the Libertarian goal of the individual being everything. Somehow, the system should work such that there are overall good outcomes for everyone. It’s kind of obvious.

It’s obvious like traffic. All of us want to get as fast as possible from point A to point B. But a designer of a good traffic and highway system does not just think about the individual and how fast the car will go. They think about the overall flow of the system, because that may slow down some people, but it’ll make everybody ideally get there as fast as possible.

It’s a sum over all the travel times of all the people. Let’s call that social welfare. The design is usually a huge amount of hard work to achieve such a thing, and then empirically test it out and work out some theory of that.

And that’s going to be true of just about any domain. Think of the medical domain. It’s really not just the doctor and a patient and focusing on one relationship. It’s the overall system. Does it bring all the tools to the right place, at the right time? Has it tested things out in the right way? Things that have been learned about one group of people or one person—does that transfer easily to other people?

Any really working system of humans at scale is someone to sit down and think about the overall flow and flux at a social level. And again, this is not at all novel to say. Economists talk about this.

Yes, what economists do is think about the group and then the overall social welfare. How does the outcome lead to allocations that everyone considers favorable and fair? And then people argue about boundary conditions. Should you make sure there’s a floor or a ceiling, or whatever, and so on? Lots of people talk that language.

Computer scientists, for some reason, seem immune to thinking about economic principles, microeconomic principles, and social welfare. It comes as an afterthought. They build their system, they try it out, it doesn’t work, and they say, “Oh, we screwed up social welfare somehow.”

Then you get people criticizing, other people defending. And it’s like—is this the way to develop a technology? Roll it out, let it mess things up, give life to the critics, and then defend yourself. It’s just a mess right now.

Ross: Yeah, well, particularly given the extraordinary power of these tools. So I think the perspective is useful.

Michael: They’re powerful, and there’s absolutely no denying they’re surprisingly good. I call it brute force and all, but I don’t mean to denigrate it. At that scale, it really is better than one would have thought.

But what’s the business model? They’re powerful—for who? Yes, they sort of empower all of us to do certain things. But in the context of an overall economy, are they actually going to be healthy for everybody?

Are they going to make the rich much, much richer, and put that power in the hands of a few? Definitely those issues are what a lot of people talk about and think about. But Silicon Valley, again, seems immune to worrying about it.

They just say, “This brute force thing is a good machine. Obviously there’ll be some problems, but not big ones. We’ll figure them out as we go.”

That just hasn’t happened in other fields of engineering, to the extent it’s happening now. In chemical engineering, electrical engineering—people thought about the overall systems they were building and whether they’d work or not as they were building them.

Here, there’s just very little thought leaders and a lot of irresponsible people.

Ross: Which takes me to your recent paper—excellent paper—A Collectivist, Economic Perspective on AI. That seems to crystallize a lot of the thinking, a lot of what we’ve been talking about. There’s quite a lot of depth to the paper, and I wonder if you’re able to distill the essence of that in a few words.

Michael: Sure. Thanks for calling out that paper. I hope people will read it. I worried about the title for quite a while. The word “collectivist,” of course, was just a kind of little bit of a thumb.

In the libertarian tradition in Silicon Valley, “collectivist” has been associated historically with socialism, communism, and so on. But really, it’s a technical word that we should own and imbue with our support, with our technology. It is an economics word.

So I made sure the word “economics” is in there, because to me, that is the critical missing ingredient. There has been a lot of talk about networks and data, and then cognition and so on. Rarely do we hear talk about incentives and social welfare.

The paper also aims not to be just negative. There are a lot of people who use the arguments, who are pained in the same way I am about the way technology is being rolled out—but it’s just a critique. I want to turn it into an engineering field.

I want to say: look, what you can do with data and data flows at large scale is make even better markets than we ever had before, and different kinds of markets. Markets arose organically thousands of years ago where people would trade. You had to have some information, but there was always some hidden information.

This is what economics calls information asymmetry. There’s also always a social context to the things you’re doing.

One of the examples I give in the paper is about a duck—or I forget what example I use in the paper, but in my talks I use a duck. A duck is trying to figure out where to get food. There are two choices: one side of the lake or the other side. There’s twice as much grain to be found on one side of the lake than on the other.

The duck has been a statistician over the years and has gotten good estimates of those values. So what should it do the next day?

A Bayesian-optimizing duck would go to the side of the lake where there’s twice as much food. Of course, it’ll give the optimal amount of food. But that’s not what actual ducks do, nor what humans do. They do what’s called probability matching. That means if there’s twice as much food here as there, then twice as much time I will go to that side than the other side.

That’s viewed as a flaw in ducks and in humans. If you’re in a casino and you do that, it’s kind of dumb. But evolutionarily, it makes total sense.

If we’re not just one person but a collective, and all the ducks go to one side, then there’s a resource not being used on the other side. You could say the goal is to build a collectivist system that tells who should go where. But that’s the Soviet Union—that doesn’t work, that’s top-down.

Instead, you ask: are there algorithms that will actually do a better allocation, that aren’t just everybody for themselves? There’s an algorithm—randomized probability matching. With probability two-thirds I go there, with probability one-third I go there. If everyone does that algorithm, they don’t have to coordinate at all. They just go. That will lead to the maximum amount of food being eaten by everybody. That’s called high social welfare.

Now you see that the context of the problem I’m trying to solve—the decision-making problem—involves the collective. If I didn’t have the collective in the context, I would do the wrong thing. In the context of the larger collective, evolution worked that out.

But as engineers, we’re trying to build these new systems, and we don’t have time to wait for evolution. We have to build the system such that the collective is taken into account in the design.

I go through examples like that where uncertainty is shaped by the collective, and then the collective helps reduce uncertainty. Because, again, I can ask people when I don’t know things, and LLMs reduce uncertainty. That’s kind of what they’re doing. It’s part of the you know, their collective property is that they help the collective to reduce its total uncertainty. But then I also get into the so that’s kind of one side of economics is, how do you mitigate uncertainty and how do you think about the social context of your decisions.

And the other probably even more important side is incentives and information asymmetry.

If I come into a market, I don’t know a lot of things. Why am I still incentivized to come in, especially if I know there are adversaries in this market? Well, I’m incentivized because I know enough, and I can probe and I can test, and there are mechanisms I can use to still get value.

We’ve learned how to do that, and our systems should be able to know that kind of way of thinking. And so information asymmetry.

So there are two kinds of uncertainty that, as engineering-oriented people, I think we have to be focusing on—and machine learning has been kind of remiss in thinking about them.

One is just statistics and error bars. We see that in our LLMs: there’s very little concern about error bars around answers, about uncertainty. It’s ad hoc. The LLM might say, “Well, I’m not very sure.” Or, actually, it tends to be oversure: “I’m very sure.” Then it changes its mind in the conversation completely.

Humans are much, much better at saying, hey, when I’m sure and when I’m not, and that’s kind of statistical uncertainty. I haven’t got enough data. I need more. As soon as I get more data, my confidence goes up and so on. So most machine leaders are aware of that, but it’s not very actionable in the machine learning field. Just get more data and the problem will go away. That’s not true in many domains—most domains.

But the other kind of uncertainty is information asymmetry. If you and I are interacting in a market setting, you’re trying to get me to do something, there will probably be a payment involved. You’re going to offer me some price for my labor. What price you offer depends on how good I am.

Well, I know that. So I’m going to pretend to be better than I am—or maybe the opposite way, pretend to be less good than I am, so I can loaf on the job and still make as much money.

All of these things I know that you don’t know—you would love to know them, and then you could design an optimal policy, which in this case would be a price. But you don’t know them.

So what are you reduced to doing? You’re reduced to making some modeling assumptions. Or you can do what economists call Contract Theory. You give me a list of options, and each option has different features associated with it and a price.

If I go to the airline and I want to get on an airplane, there’s not going to be just one price. There’s business class and economy and so on. Everybody gets the same list, but everyone doesn’t make the same choices because they have a different context. The airlines don’t know that context, but the people do.

That’s a different mindset in designing a system: you can’t just dictate everything, you can’t know everything. You have to build in options that are smarter—options that lead to actual good social welfare.

I just don’t think Silicon Valley gets that. I think they think the goal is this superintelligence that somehow knows everything, and we’ll just go to the superintelligence and it’ll tell us the answers.

Just because of information asymmetry, not true. There’ll be lots of lying going on—by the computer, but also by the humans involved in the system. Because lying is not a bad thing. It’s how you interact when there’s uncertainty and information asymmetry.

Ross: One of the things that comes out from what you’re saying is the overlap between decision making—where I’d like to get to in a minute—and that economic structure, which is emergent from decisions.

But just coming back to the paper, you refer to this missing middle kingdom which, crudely, could be described as what’s missing between engineering and the humanities. So how is it that we can fill that? What is that middle kingdom, and how can we fill that so that we do have that bridge between engineering—the main tools we’re creating—and the humanities, in understanding us as a collective group of humans?

Michael: That point in the paper was really somewhat narrowly construed. It was for academics. Anyone who’s been in a university has seen this wave: first it was called data science, or big data, then machine learning, then AI, and so on.

As this wave has hit, there have been initiatives to bring people together on campus. It’s not enough to just have engineers building systems with data. You’ve got to have commentary on that, or critique of that.

There’s a side of campus that loves to comment and critique—and that’s often humanities. Historians will weigh in on previous waves of technology, ethicists will weigh in, sociologists will weigh in.

The language gap is so huge that it just turns into bickering. The computer scientist will say, “Well, our system works. That’s all I care about. You get bits across the stream. I can’t think of anything else.”

The ethicist will say, “We have consequences, and the consequences are this, and blah, blah, blah.” But there are no solutions proposed across that gap. Both are right at some level, but the overall consequence is no progress. There’s no dialog.

I’ve seen many institutes created at many universities—I won’t name them—but it’s basically a computer scientist next to philosophers, and they call it an institute. They talk and “solve” problems. Or you add a few classes in AI and ethics to a computer science curriculum, or a couple of programming classes to a philosophy curriculum.

The naïveté of that is breathtaking.

There are others on campus—and hopefully more of them emerging—that sit more in the middle. Economists, for example, are in the middle. They can talk the technical language, they can think about systems, but they also do it as a social science. Many are behavioral economists, actually studying social systems, so they are really a bridge.

But they’re not the only bridge. Statisticians are also a bridge. They want real data, they want to test things, they want to find causes. Many work with social networks, social systems, and scientific problems.

I could go on. There are large numbers of people in academia, and in the intellectual sphere more generally, who can talk the technical language and the social language. And the social gets into the legal and ethical.

Really, there should be a big collaboration of all these things. If the only “middle” is humanities on one side and engineers on the other, that’s naïve. Unfortunately, that’s what many institutions do. They create institutes where philosophy meets computer science, and think it’s done. Usually it’s physicists creating these things, and it’s just a mess.

Part of the problem is dialog. A journalist will write about some new tech development and explain how exciting and breathtaking it is. Then they bring in an ethicist who says, “Yes, but the consequences will be terrible.”

We’re so awash in this.

Ross: Clearly, you think far more at the systems level than at the granularity of how academic institutions are structured. But I’d like to turn to decision making.

It’s a massive topic. Some of your work has shown that you can actually delegate fully to algorithmic systems, decisions that can be safe within particular domains.

But what I’m most interested in is complex, uncertain decisions—around strategy, life choices, building systems, better frames.

There are a number of aspects that come together here. You’ve already discussed some of them—uncertainty in decision making, information asymmetry. But if we just think from a humans-plus-AI perspective: we’ve got humans with intelligence, perspective, understanding. We have AI with great deals of confidence.

How can we best combine humans and AI in complex, uncertain decisions?

Michael: That’s the million-dollar—or billion-dollar—question. That’s what I think we should all be working on. I don’t have the answer to it. I believe we’re being extremely naïve about how we approach it.

You just gave a good problem statement. When faced with grand problems like that, I typically go into a more concrete vertical. I’ll think about transportation or health care, and I’ll try to write down: who are the participants? What are the players? What are the stakes? What are the resources?

Now, what’s different from just a classical economist or operations research person of the past? Well, again, there’s this huge data source flowing around.

It’s not that now everyone knows everything, and it’s not that you should pull it all into a superintelligence that becomes the source of all knowledge. Rather, you should think about that as you’re thinking about how the system is going to work.

Search engines already did this. They made us capable of knowing things more quickly than we otherwise would have. That changed things.

I think what will probably happen in the first wave—beyond just systems design—is almost an anthropology of this. We already see LLMs in all kinds of environments, like companies, being used in certain ways. There’ll be best practices that emerge.

Meta-systems will arise that don’t just give everybody an LLM. They’ll structure interactions in certain ways. That structure will involve meeting certain human needs that are not being met.

I don’t think it’s going to be academics or mathematics dictating or telling us the story. First, there will be lots of use cases. That’s true of other engineering fields I’ve alluded to, like chemical or electrical engineering.

You had a basic phenomenon—electricity could be moved from here to there, motors could be built, basic chemicals created. Then people would try it out, and they would say, oh, that approach didn’t work. And they would reorganize. There had to be auditors, checkers, specialists in aspects of the problem.

There’ll be brokers emerging. In fact, I don’t see many of us necessarily interacting with LLMs very directly. Take the medical domain: instead, there’ll be brokers whose job is to bring together different kinds of expertise. I bring in a problem, they assemble the appropriate expertise in that moment.

They themselves could be computing-oriented, but probably not purely. It’ll be a blend of human and machine. I’m not going to trust just a computing system—I’ll want a human in the loop for various reasons.

So there’ll be a whole network of brokers emerging. Mathematics won’t tell us how to build that, but it will support us in thinking, “Oh, here’s a missing ingredient. We didn’t take into account information asymmetry, or a certain kind of statistical uncertainty, or causal issues.”

Then people using systems will say, “Oh yeah, let’s do that,” and they’ll try things out. That’s how humans make progress: people become aware of what they could do, and aware of what’s missing. Best practices start to emerge.

I think it’ll be pretty far from where we are right now. The search engine–oriented human-LLM interaction, scaled up to superintelligence—that doesn’t feel right. It’ll be much richer.

Ross: So like you, I think of it in terms of some of the interfaces. What are the interfaces, and how do we present AI inputs in terms of, as you mentioned, degrees of certainty and a whole array of other factors—visualizations to provide effective input to humans?

But just to come back to that phrase of the broker—and whether that aligns with what I’m describing here—what specifically is the nature of that broker in being able to bring together the humans and AIs for complementary decision making?

Michael: Yeah. In my paper, I have another set of examples of different kinds of markets. I try to make them very concrete so that people will resonate with them.

One of them is the music market. You have people who make music, and you have people who listen to music. But you also have brands and other entities that use music in various ways as part of their business model.

For example, the National Basketball Association has music behind its clips. What music? Well, you don’t just randomly pick a song. There’s someone who helps pick the song. Sometimes it’s a recommendation system that uses data from the past to pick it. But it’s also a human making judgments.

You connect all this up. Certain listeners like certain kinds of music—that’s a classical recommendation system. Musicians see that, and they make different kinds of music. But now, especially with brands in the mix, they have money, and they’re willing to pay for things.

So now incentives come into play. Am I incentivized to write a certain kind of song because a brand will be interested in me? Maybe I will. And if a brand notices that a certain demographic listens to a certain artist, they may want to pair with that artist.

All of that is not just made up by sitting down and looking at an Excel spreadsheet. It’s a big system. It has past data, it has to be adaptive, and it has to take into account asymmetries—people gaming the system. It’s a very interesting kind of system.

Plus, you’ll analyze the content itself. The music will be analyzed by the computer, helping to make good decisions.

Ross: So currently, AI is an economic facilitator.

Michael: AI is that economic facilitator. It helps create a market and make that market more effective, more efficient, more desirable.

It doesn’t try to just replace the musician with an AI making music. Rather, it thinks about what kind of overall system we’re trying to build here. What do people really want?

Well, people want to make music. And some people really want to listen to music that is obviously made by humans. That difference, that gap, will continue to be there.

Some brands want to ally themselves with actual humans, not robotic entities—not with Elon Musk. Supporting those kinds of multi-way markets with technology.

You could have talked about that in economics years ago: “I have three kinds of entities, here are my preferences and utilities.” But it wouldn’t have been operable in the real world.

Now, with all the data flowing around, you can have all these connections be viable. You can think about it as a system.

So in some ways, this is not a unique perspective, not all that new. But it really helps. I’m just trying to get people to reorient.

And I keep mentioning Silicon Valley because I can’t believe more of them are not understanding a path that has more of an economic side to it. Instead, they’re just competing on these very, very large-scale things where the business model is unclear. That boggles my mind.

Ross: So to round out, I believe in the potential of humans plus AI. What do you think we should be doing? What are the potentials? What is it right now that can lead us towards humans plus AI as complements—humans first and AI to be able to amplify? 

Michael: I guess I’m more of an optimist. I don’t think humans will tolerate being told what to do by computers, or having the computers take over things that really matter.

They’ll be happier when computers take over things that don’t really matter, or things they don’t want to do. I do think humans will keep in the driver’s seat for quite a while.

I am very concerned, though, about the asymmetry of a few people having not just money, but immense power—and all the data flowing to them. The incentives can get way out of whack, and it would take a long time to undo some of that.

Like with the robber barons 100 years ago—there was some good in it, but then it became bad and had to be unwound. I hope we don’t have to get too much to the unwinding stage, but I think we are headed there.

On the other hand, you do see evidence of entities collecting data and using it in various ways, telling Google, “We will not just give you this data, you have to pay for it.” And Google saying, “Yeah, okay, we’ll pay.”

I do think there are some enlightened people who agree that’s a better model. The words “pay” and “markets”—it’s funny. The engineers and computer scientists I know never use those words. But then the humanities people get outraged when you talk about markets and payments.

That’s not human? Of course it is. It’s deeply human to value things and to make clear what your values are.

So I think there will be some good. Right now we just see kind of a mess. But I think that actual humans, when they start using systems and really start to care about the outcomes, and when payments are being used effectively.

These experiments are being run all around the world. It’s not just one country doing it. I don’t think the idea that China or the US is going to take this technology and dominate the world is right. That’s another dumb way to think.

Rather, these experiments will be done worldwide. Different cultures will come up with different ways of using it. Favorable best practices will emerge. People will say, “Look at how they’re doing it, that’s much better,” and those things will flow.

So overall, I’m more optimistic than not. But it is a very weird time to be living in.

Ross: More or less the right things all had the right way. So—

Michael: I can’t tell you where humans should go. I just know that, for example, when the search engine came in, young in my career, it was great. And I think to most of us, it just made life better.

That was an example of technologies expanding our knowledge base, and then people did what they did with it. The designers of the search engine kind of knew it would help people find stuff, but they couldn’t anticipate all the ways it got used.

Another part of technology—more like 100 years ago—was music. The fact that you could have recorded music and everyone could listen to it by themselves changed a lot of people’s lives for the better.

I don’t think the people who wrote down Maxwell’s equations—Maxwell himself, writing down the equations of electromagnetism—were necessarily aware that this would be a consequence. But humans got the best out of that in some ways. And then there were side effects.

Same thing here. I think humans will aim to get the best out of this technology. The technology won’t dictate.

Humans are damn smart. And I really think this “superintelligence” word just bothers me—especially because I think it’s diminishing of how smart humans really are. We can deal with massive uncertainty. We can deal with social context. Our level of experience and creative depth comes through in our creations in ways these computers don’t.

They’re doing brute force prediction sorts of things. Sure, they can write a book, a screenplay, whatever—but it won’t be that good.

I do think humans will be empowered by the tool and get even more interesting. The computers will try to mimic that, but it’s not going to be a reversal.

Ross: Yeah, absolutely agree. Thank you so much for your time and your insight, and also your very strong and distinctive voice, which I think most people should be listening to.

Michael: I appreciate that. Thank you.

The post Michael I. Jordan on a collectivist perspective on AI, humble genius, design for social welfare, and the missing middle kingdom (AC Ep15) appeared first on Humans + AI.

“The potential is boundless, but it doesn’t come automatically; it comes intentionally.”

–Paula Goldman

Paula Goldman is Salesforce’s first-ever Chief Ethical and Humane Use Officer, where she creates frameworks to build and deploy ethical technology for optimum social benefit. Prior to Salesforce she held leadership roles at global social impact investment firm Omidyar Network. Paula holds a Ph.D. from Harvard University, and is a member of the National AI Advisory Committee of the US Department of Commerce.

Website:

salesforce.com

LinkedIn Profile:

Paula Goldman

X Profile: 

Paula Goldman

Redefining ethics as trust in technology

Designing AI with intentional human oversight

Building justifiable trust through testing and safeguards

Balancing automation with uniquely human tasks

Starting small with minimum viable AI governance

Involving diverse voices in ethical AI decisions

Envisioning AI that enhances human connection and creativity

Ross Dawson: Paula, it is fantastic to have you on the show.

Paula Goldman: Oh, I’m so excited to have this conversation with you, Ross.

Ross: So you have a title which includes your the chief of ethical and humane use. So what is humane use of technology and AI?

Paula: Well, it’s interesting, because Salesforce created this Office of Ethical and Humane Use of Technology around seven years ago, and that was kind of before this current wave of AI. But it was with this—I don’t want to say, premonition—this recognition that as technology advances, we need to be asking ourselves sophisticated questions about how we design it and how we deploy it, and how we make sure it’s having its intended outcome, how we avoid unintended harm, how we bring in the views of different stakeholders, how we’re transparent about that process.

So that’s really the intention behind the office.

Ross: Well, we’ll come back to that, because I just—humane and humanity is important. So ethics is the other part of your role. Most people say ethics are, let’s work out what we shouldn’t do. But of course, ethics is also about having a positive impact, not just avoiding the negative impact.

So how do you frame this—how it is we can build technologies and implement technologies in ways that have a net benefit, as opposed to just not avoiding the negatives?

Paula: Well, I love this question. I love it a lot because one of my secrets is that I don’t love the word ethics to describe our work. Not that—it’s very appropriate—but the word I like much more than that is trust, trustworthy technology.

So what happens when you build—especially given how quickly AI is evolving, how sometimes it’s hard for people to understand what’s going on underneath the hood and so on—how do you design technology that people understand how it works? They know how to get the best from it, they know where it might go wrong and what safeguards they should implement, and so on.

When you frame this exercise like that, it becomes a source of innovation. It becomes a design constraint that breeds all kinds of really cool, what we call trust patterns in our technology—innovations like we have a set of safeguards, customizable safeguards for our customers, that we call our trust layer.

And this is one of our differentiators as we go to market. It’s things that allow people—features that allow people—to protect the privacy of their data, or make sure that the tone of the output from the AI remains on brand, or look out for accuracy and tune the accuracy of the responses, and so on.

So when you think about it like that, it becomes much less of this mental image of a group of people off in the corner asking lofty questions, and much more of an all-of-company exercise where we’re asking deeply with our customers: How do we get this technology to work in a way that really benefits everyone?

Ross: That’s fantastic. Actually, I just created a little framework around trust in AI adoption. So it’s like trust that I can use this effectively, trust that others around me will use it well in teams, trust that my leaders will use it in appropriate ways, trust from customers, trust in the AI. And in many ways, everything’s about trust.

Because a lot of people don’t trust AI, possibly justifiably in some domains. So I’d love to dig a little bit into how it is you frame and architect that ability—this ability to have justifiable trust.

Paula: Do you mean the justifiable trust from the customers, the end users?

Ross: Well, I think at all those layers. I think these are all important, but that’s a critical one.

Paula: Yeah, I think a lot of it is about—I actually think about our work as sort of having two different levels to it. One is the objective function of reviewing a product.

We do something called adversarial testing, where we’ll take, let’s say, an AI agent that’s meant for customer service, and we’ll try all these different variations on it to see if we can get it to say things that it shouldn’t say. We’ll involve our employees in that, and we’ll take people from lots of diverse backgrounds and say, “Hey, try to break this product.”

And we measure: How is it performing, and what are the improvements that we can make to get it to perform? That’s a big part of trust, right?

When we think about AI, is the product doing what it says it should do? Is it doing what we’re asking it to do? And with a non-deterministic technology like this wave of AI, that’s a very important question.

You want to harness the creative potential of AI—its ability to generate and communicate in human-sounding terms—but also marry it to accuracy and outcomes that are more predictable.

So that’s one side of it. But the second side, the second part of the job, is really a culture job. It’s about listening—listening to our employees, our customers, our end users.

It’s about participating in these multi-stakeholder groups. I was a member of the National AI Advisory Committee in the US. In many jurisdictions, we’re part of these multidisciplinary forums where people are bringing up different concerns about AI, whether that’s about how work is changing or particular questions about privacy.

We integrate those questions into the work itself and integrate solutions into the work itself, but really have it be so that everyone owns it—so that the solutions are generated by everyone.

That’s, I think, the cultural part of it. I’m an anthropologist by training, and I always think about it like that. If you want technology to serve people, people have to be involved in determining those goals.

Ross: Which goes to the next point. This is Humans Plus AI podcast, and I’ve heard you use the term “human at the helm.” AI capabilities are pretty damn good, and they’re getting better all the time. So how do we architect that humans remain at the helm?

Paula: We coined the phrase “human at the helm” a couple of years ago, as we realized there were these older frameworks about having a human in the loop for consequential decision making.

Back with machine learning, you have predictions or recommendations on a consequential decision. You want a human to take responsibility for that decision and exercise oversight.

We realized that with agentic AI, and with AI increasingly empowered to take tasks autonomously—not just make a recommendation, but carry out a task from start to finish—we needed a new way of conceptualizing how people work alongside AI but remain in control.

Know how to get the right outcomes from AI. Know what to ask for and what not to ask for, and what tasks should remain uniquely human.

I think it’s an ever-evolving framework. I know you’re deeply looking at those sets of questions. I honestly think, going back to the ethics exchange, that’s one of the most important ethical questions of our time: How do people work alongside AI? How do we implement AI in work in a way that keeps people at the center of that?

So that’s what we’re doing, discipline by discipline. For example, going back to AI in customer service—AI is very good at answering questions that are routine, that have been answered a number of times before, like “Where’s my order?” or “Where’s my return?” or “Have I gotten the money for my return?”

When it comes to unusual circumstances or emotionally challenging circumstances for the customer, human touch can make the world of difference between a terrible interaction, an interaction that does maybe okay, or an interaction that leaves a lasting impression and causes a customer to go talk to 10 other customers about how important your company is to them.

That is the kind of marriage we’re talking about between the capabilities of AI and the capabilities of people. It’s a very simple example, but we see them across every single discipline.

I think the more clear we are about how these combinations work and how people can—we have a function called “command center,” a feature that allows people to see exactly what’s going on across hundreds of agents and millions of interactions and summarize what’s going on and find anomalies—the more that people can stay in control and understand what’s going on, the more trust they’ll have, the more they’ll use AI. And it’s sort of a virtuous loop.

Ross: Yes, absolutely. The phrase “human in the loop” kind of suggests all they do is press “approve, approve.” Whereas a reframing I heard recently, a little while ago, which I think is really lovely, is “AI in the loop”—as in, humans are there, AI is in the loop, as opposed to humans just being an afterthought.

Paula: Yeah, that makes sense, right? And it’s consistently going to be a give and take. We talk a lot about, and we could design a lot for, a key question: How and when should AI escalate to a human?

The truth is, as this matures, it’s going to be AI escalating to people, and people handing it back to AI, and back and forth and back and forth.

We need to be able to observe all of those transactions. We need to be able to know when it was actually AI making a decision versus a person.

I think these are the types of changes that are going to be transformative for businesses, but it requires really understanding the workflows. Really understanding AI is not a substitute for every human task—it’s just not—but it can do a lot of powerful things.

So where is it that you use AI to perform certain tasks? Where is it that you don’t? And what does that mean for how roles evolve in time?

Ross: So whenever an organization says, “All right, we need to get some AI in, let’s build an AI roadmap,” basically the very first thing, pretty much, is governance. So I’d love to know—Salesforce is a very unique organization, so perhaps looking more at your client organizations—what is the on-the-ground reality of saying, “Okay, we’re going to set up our AI governance”? What does that look like?

Paula: It’s an excellent question. One of the things I often say in customer conversations about this is that, I know it can sound like a cliché, but the most important thing is just to get started.

In reality, there are a number of different building blocks that are important, and none of them are necessarily going to be absolutely polished and perfect. They’re iterative.

For example, it’s very important to clean up data—especially to label sensitive data and make sure that privacy controls are in place. Data is obviously the foundation of getting good results from AI, but that is a continuous process. It’s not a once-and-done thing.

We end up talking a lot about the types of features I was mentioning earlier—that is to say, having people that really know and understand how to make sure that there isn’t unintended bias in the results of different AI applications, or ensuring and judging outputs from AI and making sure that they’re accurate and looking at measures of precision and recall.

You start small and look at the results, and as you trust the results more, you grow whatever application it is that you’re building and build from there.

The other thing I would say, and that I often talk to boards about, is: How do you have the right board-level oversight of AI?

I do think we’re unique at Salesforce—we have had a board committee that our Office of Ethical and Humane Use has reported to every quarter for years. And we use that conversation to talk about emergent issues, what’s going on, what’s a snapshot of what’s going on with AI, what are the questions that we’ve got under control, and what are these emerging questions that we really are thinking about.

Given the technology is evolving so quickly, how do we set new standards for new technologies that maybe we hadn’t dreamed of two years ago? I think that board-level conversation, that sort of bottom-up and top-down awareness of what’s going on, is also very important to governance.

Ross: I mean, are there any particular structures, as in an AI Oversight Committee, which requires a board, or anything else which you have seen?

Paula: Yeah, I think so. We have an Ethical Use Advisory Council, and I think many companies have some variant of this kind of structure where we have frontline employees, and that’s intentional. We want people who are very, very close to the work.

We have executives from many different functions of the business. We also have outside civil society groups that are contributing to bringing in concerns that are coming in from various communities around the world.

Then we bring to that council a number of items, including policies. We have an AI acceptable use policy that applies to our customers. It sort of sets a floor for what responsible use looks like, and we will kick the tires on all of these questions with this group.

We’ll kick the tires with other stakeholders, as you can imagine, depending on what question comes up. Then we’ll bring recommendations up our executive chain as we want to make changes. That’s how we came up with our AI acceptable use policy, for example.

So I do think it really is all about who’s at the table, and I think creating these big tables for lots of people to have ideas and to have questions really enriches the quality of the solution.

I’ll add one more thing there too, which is, I mentioned culture as part of the job of AI ethics, one thing that’s really important is making sure that we have channels for people to ask questions, and not just if you happen to be part of a council.

We have anonymous channels where employees can ask questions. We have Slack channels where people can ask questions. We have workflows for people to ask questions, and so on.

All of that also remains very, very important. As we get more and more technologically sophisticated, these very deeply human structures of how we listen to each other, and how we make each other feel heard, and how we acknowledge questions that come up all become even more important.

Ross: That’s fantastic. Yeah, no, it’s—well, going back to the trust point, if people feel that they are listened to, then that’s a pretty good starting point. If they don’t feel listened to, that’s not going to engender lots of trust.

But going back, you said, “You’ve got to get started on the AI journey.” All right, we’re going to plan for six months and then do something. So what is minimum viable governance? As I say, we’ve got to get started. . We do need something. What’s that minimum to be able to say? All right, we’re going to try some stuff, but we have some atleast some guidelines around it.

Paula: Yeah, that is an excellent question, and I’ll answer it off the cuff, but it’s one I want to think more about.

I think you want to know, if you’ve got a significant project going, you want to know about it. You want to know what its goal is. You want to start with some, at least elementary, measures of what’s going on with it—how is it doing, and what are the critical benchmarks, and if anything unexpected has come up from it.

It really is kind of that sort of elementary. And yet, you also want to make sure that there are certain domains of AI that have higher obligations for diligence.

So if you’re using AI, for example, to give tailored financial advice, or you’re using AI to determine who gets a job, those are things that have higher levels of scrutiny. So I would say that’s also a second thing—think about the requirements for those types of forms of AI.

But the reason I started with that first example of—and it sounds fairly elementary—but if you’re measuring success and looking out for anomalies, and being able to fine-tune that, it ends up actually being very important. You can start with some sort of basic measurements, and as you get more sophisticated with it—let’s say you’re using AI for marketing personalization—you can start with looking at marketing segments and looking at how click-through rates on particular campaigns.

Obviously, there’s a whole world of things you can do with AI agents in marketing. As you gain confidence with it, those measures become much more precise, and the measures almost evolve automatically from the work that you’re doing.

I guess I’ll add one more thing, which is that I think the culture of transparency and documentation is very important. So that, I think, is a huge part of ethics and AI—being able to say, for example, when we publish AI models, when we put them out there in the world, we have model cards, we have system documentation for all of our products.

That documentation will have all the things you expect about what the product is and what it does and how to use it, but it will also say, “Here are some of the risks, and here’s how we tested for those risks, and here are ways not to use the product, or here are best practices to try to avoid those risks.”

I think that level of transparency is what helps humans to use AI in ways that get them the results that they expect. And that’s also, I think, a very important piece of it.

Ross: So the future of work is here, yeah, moving pretty fast. And I suppose, of course, Salesforce is a leader in generative AI, and this idea of agents being able to collaborate, form teams, do an extraordinary array of tasks and capabilities.

So how do we design the use of these wonderful tools and agentic AI so that they are augmenting people, developing skills, creating more opportunities for people in this future of work?

Paula: Well, this is the question that animates me every day, and it is a question that I’m personally paying attention to. It’s a question that we’re paying attention to in our product development life cycle. It’s a question that we’re living every day as we ourselves use AI in Salesforce.

And I think it really does come down to having a shared understanding of what tasks we do ask AI to take on, and what tasks we leave to humans, and where those boundaries get made, and how people collaborate with AI.

Just to give you a couple of examples, I was thinking about one of our customers, 1-800 Accountant. This is a firm that does, amongst other things, as you would guess by its name—it helps with tax preparation, helps with financial planning, and so on. We’ve been working with them on their customer service AI.

And, particularly as tax season arises every year in the United States, you can imagine the company gets inbound requests that can exceed capacity to respond to in a quick way. AI is really able to handle some of those routine requests.

But for a number of reasons—regulatory, ethics, and otherwise—personalized financial advice is something that you want now for people to be giving. You want to make sure, to the extent that AI is augmenting people and giving that advice, you also want to make sure that it’s accurate and that it’s looking out for risks.

So 1-800 Accountant was very deliberate in creating that division of labor where the less routine requests are going to their experts to handle. And because there were those less routine requests, they have more capacity. They have less of the “Where am I? Where’s my order?” type requests, and they’re able to respond more effectively and reduce call volume by significant percentages.

There are so many stories like this. The orchestration has to be intentional. These are intentional decisions we have to make about how we prepare people to work with AI, and how we allow them to oversee personal AI agents working on their behalf—whether that’s drafting emails to customers, or orchestrating a whole marketing campaign.

It’s really that intentionality around it, and how people oversee it, and what jobs and roles and tasks within those roles are reserved for people, that I think is one of the most crucial questions of our day.

Ross: So always, always get very practical. So, where’s the intentionality come from? As in, so who’s who in the organization?

So intention, I think, is—I always talk—is a really critical part of how it is we create this positive future. So intentionality, certainly. So where does that intentionality reside? Is there the chief AI officer or chief ethics officer? Or is it people who are delegated to have the capabilities to implement their intention around how they use the agents? How do we cascade?

Paula: It is interesting because you use the word “cascade,” and that’s what I was going to say—yes, right.

So a chief AI officer, head of IT traditionally, would be kind of thinking about those big bets on AI and setting goals around them, and making the technology available to customize for different groups and different departments within an organization.

But increasingly, we’re seeing the sort of matrix approach where people that are closer to how the work is getting done are in the position to say, “Here’s how it can help me, and here’s how it shouldn’t help me.”

It’s interesting. I was on a business trip last year, and I was in the UK, and I met an IT person, an executive from a local town council, and he was telling me the story about how he began trying to implement—for their social workers—a note-taking tool that would basically take notes during a site visit with a person in need and then summarize the conversation so that social workers didn’t have to go back to the office to type up the notes, which was taking hours, and they could just go home.

And he told me that the team rejected it. They wanted to go back to the office. Why? Because they wanted to talk to their colleagues, and they wanted to talk to their colleagues to trade notes, because they didn’t want to burden their spouses with this.

And so, because they missed how humans actually wanted to use the technology, they had to put the whole experiment on hold for a while. But a small tweak—observing what is this flow of work, and which are the tasks we should and shouldn’t delegate, and how do we make it so that it actually makes people better at their jobs—would have made the difference to a successful implementation.

And that is not rocket science, but it can sometimes be overlooked because it’s not just about the technology; it’s about the technology plus the people working together that gets the best results.

Ross: Yeah, so essentially giving people the knowledge and the power then to be able to make their choices or use it the way which they see fit.

Paula: I think so. And maybe there’s a balance, as there’s AI that can help us individually in our roles, and there’s AI that helps us as teams do better work. And I think there’s a place for both of those things, but you want the people that are closer to the ground to be able to impact how those technologies get deployed.

Ross: So to round out—a nice big question—what is the positive potential for humans plus AI?

Paula: Well, in my mind, I think it’s about enhancing human connection and enhancing human creativity and innovation—I’m thinking mostly in a work context. Right?

There’s a number of other societal potential benefits—there’s deep science, which I’m deeply excited about, and medicine, and so on. In a work context, it’s about being able to free people up so that they are able to make these deeper connections with each other.

And I think about that. I think about stories that are sort of way predate AI. But I also think about, for example, in medicine, I think about the doctor that doesn’t have to take notes, but is able to look someone in the eye and understand if maybe there’s something that’s not being said or a question that needs to be asked.

Or I think about—well, you and I have talked a bunch, Ross, about strategy—and I think about how AI would enable us to ask better, deeper, more incisive questions about where organizations should be going and to see unmet needs.

Let’s say we’re running a nonprofit and see unmet needs in our community and find better solutions around it. I mean, it’s really—it’s almost one of the hardest, because it’s so boundless. The potential is boundless, but it doesn’t come automatically; it comes intentionally.

And I think I might round out my answer by saying, when I first joined Salesforce, one of the things that attracted me to the company was its long history of being a values-led company. And Marc Benioff, our CEO, often says, “Technology is not good or bad, it’s what you do with it that matters.”

And here we are at the inflection point of one big technology leap forward with this wave of AI. And yet that adage, I think, remains true. It’s what we do with it, and that is why the intentionality that we talked about is so deeply important.

Ross: That’s fantastic. Thank you. Paula, is there anywhere people can go to find out more about your work or Salesforce’s framing of ethics?

Paula: Yeah, for sure. Well, we have a site—the Office of Ethical and Humane Use site for Salesforce—and you can just Google it, and there’s lots of material there to look for. You could ChatGPT it, you can do whatever you’d like, but it’s all there for the taking.

Ross: Fantastic. Thank you so much for your time and your insights and your work in creating a positive future for humans plus AI.

Paula: Ross, thank you for your leadership in this domain.

The post Paula Goldman on trust patterns, intentional orchestration, enhancing human connection, and humans at the helm (AC Ep14) appeared first on Humans + AI.

“What I need is someone who will have an idea I would never have had. In fact, better yet, an idea no one else in the world would ever have. That’s human space. That’s our job now: the unknown infinite.”

–Vivienne Ming

Vivienne Ming is a theoretical neuroscientist, entrepreneur, and author. Her AI inventions have launched a dozen companies and nonprofits with a focus on human potential, including Socos Labs and Dionysus Health. She is Professor at UCL Global Business School for Health, with her work featured in media including Financial Times, The Atlantic, and New York Times.

Website:

socos.org

dionysushealth.com

optoceutics.com

LinkedIn Profile:

Vivienne Ming

X Profile: 

Vivienne Ming

Unlocking human potential through AI

Building health systems with humans and machines

Why AI should challenge—not replace—us

The danger of cognitive atrophy in education

Fostering metacognition and meta-uncertainty

Diversity as a driver of collective intelligence

Preparing for a future of infinite unknowns

Ross Dawson: Vivian, it is fantastic to have you on the show.

Vivienne Ming: It’s a pleasure to be here.

Ross: So you are being described as obsessed with using technology to maximize human potential. So that’s a big topic, where, how do you see it? What is the potential?

Vivienne: Yeah, I mean, when I was interviewing to go to grad school, I used to tell people that I wanted to build cyborgs, which is an excellent way to get everyone to scoot away from you for fear that your crazy will rub off and they won’t get accepted either.

But one of my claims to notoriety is when my son was diagnosed with type one diabetes, I hacked all those medical equipment. Turns out, I broke all sorts of US federal regulations. And little did I know at the time, I invented one of the first ever AIs for diabetes.

And I mention that here in answer to your lead-in because as much as I’m thrilled that I helped my son—it’s a project I’m more proud of than any other—there is some kid in a favela in Rio, a village outside Kinshasa, down the street from me here in California. This kid has the cure—not some crummy AI, not a treatment, a cure for diabetes—in their potential.

But the overwhelming likelihood is they’re never going to live the life that allows them to bring that into the world.

And there’s tons of research on this. I’m a hard number scientist, so words like human potential can feel very flowery. But to me, it’s grounded and sort of strangely selfish. What could all of these lives be doing transforming the world for the better?

And for some reason, we are so under-motivated to make that potential a reality. So this is—when I come at these sorts of problems, that’s really where I’m coming from.

And I’ll even share, just as a personal motivation, I spent a solid chunk of the 90s miserable and homeless. And since then, I’ve gotten to found—or been involved in founding—12 different companies. I’ve invented six life-saving inventions. I’ve written books. I’ve gotten to do so many things.

And I get it. I have a weird life, a wonderful life. Maybe not everyone’s going to have that same life, but everyone could.

And how many lives never got off the streets, or never got out of the favela? Or, for that matter, how many lives were, exceptional in some sense, but kind of stalled out at a solid job somewhere, doing something anyone else could have done?

But you enjoy things and you led a good life. But again, you could have done something transformative, and the world didn’t call on you. It didn’t give you that opportunity.

That’s what human potential is about for me.

Ross: Fantastic. And so I think just digging into that healthcare piece—so one of the really interesting things about diabetes, or AI and diabetes, is this idea of a closed system, where the human and the AI system—as it is data coming to human to be able to adjust glucose levels and so on…

And I think some of your other work around, for example, bipolar or other domains as well, where it’s looking at humans and AI as a system—where humans are, we are obviously an integral system—but we have data, and we’re using the AI or technology as an external system to be able to build a bigger system which can enhance our health, be that in glucose levels, be that in our ability to respond to ways our neurology is going awry.

So I suppose you can speak to any specifics around how it is we can build those humans-plus-AI health systems.

Vivienne: Yeah. Again, coming from my original world—and it’s still my world. In terms of my academic work, I still have a toe over there—and it’s in what’s called neuroprosthetics. So we don’t call them cyborgs nowadays.

And what I always think of there is: my technologies should only ever make people better. It shouldn’t replace something you can already do for yourself. In fact, I should never build something…

In fact, let’s come up with a line. It’s a line I’ve said before: technology should not only make us better when we’re using it—we should be better than where we started when we turn it off again.

And so this then becomes my rule. If I’m building a system to predict manic episodes in bipolar sufferers… working in diabetes… I got to build a system for Google Glass to help autistic kids read facial expressions.

And let’s be clear: the right version of that system would know, actually, my wearer doesn’t need help right now. That nice, big smile—they got it. I don’t need—because otherwise, it becomes a crutch.

And sometimes that’s not a bad thing. If there truly is a lost capability on some level—and with, for many people with severe autism, facial expression reading, just don’t get it for free like the rest of us—but for others, it’s so much more complicated.

And so I really want to create… there’s different models you can think of. One terminology people use is a digital twin. We want to use AI to build essentially a separate version of you, and we can kind of experiment on that version, find out what works, and then bring it back into you.

But as someone that’s been building and using AI for years, some of my favorite examples of AI is me using it as a tool—as a scientist—to explore a question, in which case I think of the whole thing as me. This is my extended capability.

AI, rather than as a co-pilot or a crutch—AI is a medium in which I can explore the world. And importantly, it’s a medium which has been tuned to challenge me—one of the few things I can’t do for myself—and so that becomes really important.

My latest company is Dionysus Health. We’ve developed the first ever biological test for postpartum depression. And like most modern biotechnology, it just wouldn’t work without AI—analyzing brain activity. In our case, analyzing epigenetic data—impossible without machines.

But what it also gives us is a genuinely nuanced look at that expected mom—in this case, a nuanced look at their epigenetics—but that’s a big part of who we are.

And what’s exciting is we have developed other AI that integrates into that—that you can actually talk to. And not for treatment, but to understand this person, improve the diagnostics—thinking of that a little holistically.

Who is this person? Instead of a separate test that you go and do, what is something that can integrate into an everyday experience?

And then, with real trust—and if we want to, we can always talk about the ethics of all this—with real trust, can then become part of a precision health.

Like, for you: here’s a treatment for postpartum depression that is likely to work. And unfortunately, you are likely to experience postpartum depression.

Whereas for you: you’re much less likely. And so here’s a much more moderate approach that we’re going to take. There’s always a risk—pregnancy is stressful—but we don’t think this sort of biologically driven postpartum is in your immediate future. So we’re going to take a different approach.

Again, we’re building a very integrated sense of AI—adding capability. We can’t predict the future. An AI that can give us, if you will, the odds.

I know it’s not sexy to put it this way, but it’s like an actuarial. It just gives us insights. So now we’re not guessing at the future—we’re planning for it.

And different problems might have different kinds of AI and different approaches. But it’s that very integrated kind of human-centric way that really resonates with my work.

Ross: So that’s fantastic, but let’s turn to the education piece. Because you made the points around this human plus AI system—we can be learning systems—but this critical thing of not being a crutch.

And so, of course, this is very much of the moment, with people worried about cognitive atrophy and where there are risks of that, and this delicate balance of how do we use AI in education.

Perhaps we can take it—any age group you choose to start off. But how do we use AI to truly create a better system, as people who learn better—assisting them, without, where exactly as you say, that they are better, you take the AI away as well as helping them in that process?

Vivienne: There’s a couple of high-profile studies here. There’s the recent one out of MIT, where they talk about cognitive debt. It’s gotten some criticism.

They found that people using AI essentially to help write essays had lower functional connectivity, as evidenced by EEG—sort of brain waves, if you will. And people criticized that interpretation.

Those criticisms are somewhat valid, but also note: people could remember fewer of the words that got used in those essays. They felt dramatically lower senses of ownership.

When I read that study, I took a different perspective—but one that maybe is even more alarming—which is the neural data that they produced, along with that behavioral data.

To me, done day after day, month after month, year after year—especially starting early in someone’s life, but even as an adult—you are talking about not engaging your full cognitive faculties. What we might nerdily call, I’m not seeing a lot of gamma activity—this high-frequency activity, particularly frontal lobe, hippocampus—which is evidence that you are thinking hard.

So, do we need to think hard all the time? Does it matter in their study that for a little while, students weren’t thinking that hard? No. But accumulated over time?

This isn’t—I don’t even feel like this is a prediction. If I saw two and we were doing a twin study, and one of them had an LLM doing all their essay writing for them, and the other was writing essays from scratch—the essays may be worse, but when they hit 50, the one using the LLM, the one using GPT—I would start early dementia testing.

Because I guarantee you, they are at dramatically heightened risk. That would follow all of the other risk factors we see in people’s lives. Cognitive engagement is a fundamental risk factor for dementia.

And if you start this early in someone’s life, you’re not just talking about dementia eventually. You’re talking about lower cognitive ability on day one.

That’s why we want you to read to your kids. Cognitive engagement—or enrichment, as it’s sometimes called.

So this is the real issue for me.

The second announcement—just today—OpenAI has announced a version of ChatGPT, which I think they call study mode. So you put ChatGPT in study mode, and now it’s been fine-tuned.

They train up these giant AI models on all sorts of data. Then a human-in-the-loop fine-tunes the model to do specific things.

If an AI has ever said to you, “Oh my goodness, that is the greatest idea I’ve ever heard,” that’s the product of the fine-tuning. And it means you’re almost certainly using GPT, because it’s the one that does that the most.

So in this case, it’s fine-tuned to kind of be Socrates—to talk to you. And hey, that’s great. That is, in fact, one of the issues.

I would say, if there’s—to me—a golden rule in AI and education, it is: if the AI ever gives the students the answers, the students never learn anything.

And there is no such thing as an average student. Some kids are going to flourish with these tools.

Evidence already shows some kids—adolescent girls—flourish with social media. They don’t show mental health challenges. They don’t show education challenges. Most do.

So kids are different. Adults are different. And understanding that explains why things come out messy in the research.

But the overwhelming story will be: if every kid has an AI tutor—an LLM that just answers the questions for them—that’s that crutch. That it does for them the thing they could have done themselves.

I guarantee you, the world will be worse 30 years from now—like measurably, dramatically so.

Ross:I think you’ve already raised in what you said—I think there’s two things which come out.

One is, well, we have the tools. It’s how we use the tools as an individual. Also, how we design the tools.

And so, to your point, if OpenAI has done a good job, and this truly is Socratic, and it is starting to engage us and give us the thinking pathways—rather than the things as effective of that—then that’s one thing: to design the tools.

And there also is the way in which—well, how do we choose to use these tools? Where do we just do the cognitive sourcing or not?

So I think there’s a few layers in which we can avoid the downfall and maximize the outcome.

Vivienne: One of the important issues here, again, whether we are talking about kids, early learners, or ourselves as continuing learners throughout adulthood, is recognizing that issue: how will people use these tools?

It’s, in part, a learned experience itself.

We don’t teach this stuff really in any formal school system anywhere. I know that there are nonsense stories about AI-first schools being built in Austin and elsewhere. I’ve looked into them. I wish they were better, but I’m not seeing it.

Instead, we look essentially at the luck of the draw. The household you grew up in. Did you grow up in a household where your parental role models, your siblings and neighborhood peers are engaged in this sort of behavior?

If you are, you pick up on it too.

Raj Chetty—the noted, likely someday Nobel Prize-winning economist—has research around this. His Lost Einstein research.

We think so hard about kids learning facts, or us learning facts. Facts don’t predict life outcomes.

Resilience. Perspective taking. Analogical reasoning skills. Metacognition.

These are things that predict life outcomes. The skills—those are tools that you can learn later.

But my metaphor is: we have built an education system—and even a workforce system—entirely on tool belts. People are tools. Hire them because they have the right tools, and deploy them as such.

People are artists. They’re craftsmen. They just happen to wear tool belts. And they are better when they have them.

Let me be clear: to anyone who thinks I’m not a true-blue believer in what AI can do for the world, then you missed the opening to this interview here. I’ve been building AI systems for 30 years.

But it’s not the human or the AI that matters. It is the hybrid collective system. What emerges when you bring a particular person together with a particular technology?

And yes, that gets super messy and sometimes unpredictable.

But I will tell you this: our education system should be about the artist, not about the tool belt.

Ross: So you mentioned the lovely word metacognition, which is something we delve into a lot.

And I suppose this is—think of this as not only looking at your own thinking, your own thinking capabilities, and how to improve those, but also the cognition of both yourself and the AI. So going above that to look at: where does—where should—the cognition reside, and how do I do this the best?

So I suppose, how do we, individually or collectively, enhance our metacognitive capabilities to be better at this?

Vivienne: The funny thing about metacognition—let’s break it up. We can call this—I often call it meta-learning. We can call them foundational skills. People use different language.

Yes, sometimes people call stuff like this soft skills. But I hate that. These are measurable. They make measurable differences in lives—much more so than hard skills.

Should you know how to factorize a polynomial? 100%. It does you no good if you don’t have these foundational skills.

So let’s give broad categories: metacognition, general cognitive ability, social skills, emotional intelligence, and then a kind of catch-all category that I just call creativity-related skills.

The interesting thing about all of them—except metacognition—is they are experiential learning factors, skills. They’re hard to learn.

I can’t give you a pamphlet. There’s no lecture about courage that’s going to make you courageous. There isn’t a book I can give you to read that will make you more resilient.

These require, bluntly, rewiring your brain. And so these are slow, but they can be developed almost throughout your entire life. General cognitive abilities are very formative—do it early, that’s where the real value is—but almost throughout your life.

What’s great about most metacognition is it really is something that, on some level, I can explain to you. It’s a more traditional learning experience: thinking about your own thinking, strategic thinking, or self-assessment.

Sometimes it has funny definitions. Self-assessment sometimes gets defined as: wrong or right, your assessment of yourself. I don’t find that particularly productive. I want to know whether I’m right or not—so how accurate is my self-assessment?

But really, it comes down to: am I engaged in this reflective activity?

If I am an older adult online and the whole world is telling me I’m sharing a lot of misinformation—when I read that next article, do I take three seconds and think to myself, “What do I believe about the source of this article?”

And it turns out, shockingly few people do that. If you add in a little hint, a little thing—layer it on top of any social media—and it just says, “Hey, did you think about the source of this information?” Sharing rates just plummet. People stop sharing misinformation.

But the way I always read that research is: why aren’t they doing this themselves?

So metacognition is huge. It’s powerful. But it’s actually a little bit—it’s not paradoxical—but it’s almost a weird dilemma or a chicken-and-egg.

If I don’t have the emotional intelligence to engage my metacognition, then it’s like a tool I never take out of my tool belt.

But if I don’t have the sort of metacognition to reflect on some of these issues—and let’s get into AI and metacognition—then when do I know, if I’m a student using an AI tutor, when do I know to push back?

All of these AI tutors are LLM-based, so they’re going to get things wrong. Because—let’s be clear—LLMs know everything. They understand nothing. So they will confidently state things as true which aren’t true.

And when is a learner going to push back on that? When is a learner going to say, “That sounds wrong,” because they almost never would do that to a teacher.

And I think that becomes part of how I have to think about metacognition in learning: when have we trained a generation of kids that are willing to say, “Wait a minute, I’m suspicious of what I’m getting, and I’m going to push back and go deeper”?

Ross: Yes, yes. I guess the question is: how do we breed or inculcate that ability to push back? I mean, obviously, to teachers as well as to LLMs, we need to.

And I think there are some people that teach talk-back in class or question teachers.

Vivienne: There are some which just become evidence we could do it right.

Ross: But probably people question the AI less. And so there is this point where experts are the ones who can use their LLMs the best because they can sort of say, “That’s good, that’s not.”

Whereas if you’re not the expert, then that’s the real challenge—because you don’t know when it has given you good stuff and bad stuff.

Vivienne: There was a nice study by a group at Harvard looking at BCG consultants. So these are highly educated adults, ambitious, motivated.

The nature of the study is this work kind of went into their permanent records, so they were also motivated to genuinely engage.

They ended up loosely breaking up the behavioral component—the engagement mode of the consultants with the AI (which was GPT in this particular case)—into three categories.

There were self-automators. They gave a task to the AI, it gave them results, and that was it. They did what the AI told them to without reflection. The results there were terrible.

A lot of people fall into that. If you look at Anthropic sort of usage statistics for university students, I would say there’s a lot of self-automators in that world.

The next group were the centaurs, in which certain things were human tasks, certain things were AI tasks. The human would essentially review, assign out, and then what they got back from the AI, they either accepted it or rejected it—but they didn’t interact with it.

So clearly, this is maybe a step in the right direction. It’s starting to be explicitly metacognitive—thinking about what I’m doing, thinking about where will an AI be successful, and where will I be successful in using this.

So there’s kind of prediction, a kind of self-assessment, if you would. Include the AI as a kind of part of yourself: which part of me is going to do this task?

But the best results came with—well, they called it cyborgs, so clearly, I’m going to like them—and the cyborgs: every task was an integrated task.

Every task, the AI was involved. The person interacted with it. But they pushed back. They didn’t just accept or reject.

“I don’t think that’s right. I bet we could do it better. What if we changed the wording? What if we did a different analysis?”

I know it seems truly absurd to use a superhero movie as my example, but there’s a scene in The Avengers: Endgame in which Tony Stark is sort of running through models of time travel.

It’s not a perfect analogy, but essentially, he’s saying, “We’ll try this,” to—I can’t remember what the name of his AI is—but you know, the superintelligence system. Except you can really imagine he’s talking to Gemini or Claude nowadays.

“Try this. What if you imagined a Möbius? What if it was shaped in this way?”—these different sort of geometric approaches, which is an actual part of a lot of computational science.

And suddenly, one of them clicks and it actually works. So he and the AI are interacting in real time, trying different ideas out, exploring hypothesis space, until something actually worked in a computer-based simulation.

Silly superhero stuff, except it captures a little bit of the best engagement.

So I’m going to add in—we’re talking about metacognition—I’m going to add in a term many people probably haven’t heard before, but I’m going to call it meta-uncertainty.

It turns out some people are better at assessing their own uncertainty—predicting the outcomes of their own understandings and actions. It’s a little bit explicit and a little bit implicit, so it’s a bit different from certain other metacognitive qualities.

Let me tell you: one of the main things in making a metacognitive judgment about AI is—what’s within its ability and what isn’t? And how do I define the fuzzy space in between?

Because if it’s not, it’s a human task. If it’s totally within—we’re going to call it its distributional training set—then it’s, in some sense, totally an AI task.

And it’s that fuzzy space, where it begins to break down—but the AI doesn’t know it—that’s where the human has to be.

Not just—this is the problem with the metaphor—not just, “It’s a copilot.”

I actually use a metaphor, which I get isn’t going to resonate for many people because you haven’t had the chance to be a professor—but it’s like working with my grad students.

My grad students, for years of their life, will be studying one thing to a depth that almost no one else is studying it. They will know more about that one thing than I will.

So why am I involved? Why am I there an hour every week, answering their questions, if they are one of the world experts?

Well, I already said it—but I said it about AI: they know everything, but they understand nothing.

Obviously, the point of being a grad student is you, slowly, you’re getting trained in how to do the understanding. Because the knowing—you could have done on your own. Anyone can know things, unless you need access to like CERN to run a giant physics experiment.

It’s the understanding—that meta-uncertainty. What do I truly understand about this problem, and what don’t I?

And how do I explore problems where, not only are there no answers, we don’t even know what the right questions are?

That is a fundamental human space right now.

So when you’re engaged in this, I either want that strong metaphor of: the AI is a part of you, it’s a medium that you’re interacting in.

Or, if that’s a little too abstract for you—it’s your grad student, and you’re the mentor. And it’s doing a lot of the execution—it’s true—but it’s not doing the busy work.

I think that’s a mis-sell of what AI is. “AI will do all the boring stuff, so you can do all the fun, creative stuff.”

I will just say—I have a book coming out, and I have a whole chapter about why that’s a bad thing.

Let’s just say, if you make it easier to do boring work—guess what? You get a lot more boring work. You have to make it easier to do the creative work if you want more creative work.

So that’s what AI needs to be supporting.

It can also automate some other stuff you don’t care about, but it’s the making you better at exploring—that’s what the world needs. That’s human potential.

So that’s a big part—this meta-uncertainty and metacognition around: how do I mentor my AI? How do I guide it to the right answers that, in some sense, neither of us could have arrived at on our own?

But together, we have this new hybrid collective intelligence that literally never existed before—and it’s as unique to you and the tool you’re using as you are unique in the world.

And so it’s just this amazing potential that I think is really undersold—if you see AI as something that can read and write emails for you.

Ross: Yeah, no, that’s absolutely fantastic. I love the meta-uncertainty frame. I think that’s something that applies to LLMs as well as to humans. But it is a fundamental capability for humans.

And I think, yeah, what you’ve just described is a really nice way of evoking that capability—which we need to develop.

So what I really want to get to the point of, which we’ve sort of touched on in various guises, is that there are people adverse—there are lots of different people who are very different in their cognition and all sorts of other ways.

And there’s, yeah, there’s obviously dangers in homogenization through LLMs. There’s also potential to allow individuals to express themselves more fully with AI.

And so, just like to think about: how can we enable more diversity, more inclusion, more expression of our individuality—of our uniqueness—through AI than we have before?

Vivienne: I said one rule that I have, which is: I don’t want to build technologies that can do for you what you can already do for yourself.

In fact, I think one of the great use cases of AI—which Anthropic’s own research shows is dramatically underused—is actually using LLMs to critique you. We call this productive friction. Wildly underused.

People are not choosing in to self-criticism, unsurprisingly. No one wants to. Our whole lives are sort of trained away from that.

My wife and I published research years ago showing that students chatting online—long before LLMs came along—we could build an NLP that could listen to those students and actually predict at week one the grade they would get in the course.

And what was fascinating wasn’t the difference between the students that passed the course and those that didn’t. It was between the very top students and the next group down.

Interestingly, the next group down were always right. They always gave normative answers. You would have gotten the exact same answer out of every one of them.

The students that our system predicted would do the best—and interestingly, our cost function wasn’t the grade in the course. It was years to matriculation and then their first job after university. So we were looking at this bigger outcome.

And what we found was the students that had the best outcomes were regularly wrong in these discussion forums. They were regularly exploring—taking an idea, and instead of just saying what they learned in the lecture or from the reading, they took it somewhere.

And as a result, they were often wrong, because they were going outside of what they knew and understood. They were moving fundamentally into uncertainty.

But the crazy thing is: one, that behavior was very predictive of long-term student success. And two, there was no cost. These discussion forums—the students were required to participate, but beyond that, all they had to do was show up.

They could have talked about the weather, and they’d get full credit. And trust me, plenty of students just talked about the weather. That did not predict great things for their coursework.

But even when being wrong wasn’t held against you, the vast majority of students would not be wrong publicly—even though the best students evidenced a completely different set of behaviors. And that really worries me.

And so this normative behavior, this desire to fit in—we now have that for free in our pocket. I can get the right answer—the normative answer—on almost any question to expert level, for a million tokens, for a dollar.

Why do I need you in it? Why do I need some fancy university degree?

And what I need—and I mean when I say I, I mean when I’m hiring, when I’m bringing students into my lab—what I need is someone who will have an idea I would never have had. In fact, better yet, an idea no one else in the world would ever have.

And the great thing is, everybody has that capability. But it has been absolutely trained out of the vast majority of people.

But now this is a forcing function. Can you give the normative answer to problems—the correct answer? If that’s all you can give me, I don’t need you anymore.

What’s left for humanity aren’t these well-posed problems that have definitively wrong and right answers—because all of that’s for free in your pocket. And that’s terrifying. And it feels like it’s dissolving away so much of what we built our sort of world around.

But I like to at least partially look on the positive side. I think the positive side is amazing.

What these systems know is vast. We should be proud that we’ve discovered so much about the universe.

What they don’t know is infinite—definitionally so—and will always be infinite. So that’s human space. That’s our job now: the unknown infinite. And there will always be a job there.

It’s just—I’m going to be perfectly blunt, like aggressively so—that’s an exhausting job. And we have not prepared almost anyone in society for that job.

It’s just the dumb luck that some people have gotten good at this for reasons other than intentionality. But these things are trainable. They’re developable.

We could explicitly make this part of educational experiences—and we have to—because that’s what I’m hiring for: you having an idea no one else in the world would have.

So you plus AI is—your job is to start actually… well, put it in advice I’ve given to many teachers and professors:

Let students use AI to answer your questions. Just let them know: don’t be an idiot. I have the same AI.

So if all you do is put the question into GPT, and it gives you an answer, and you turn that in—you know, in a sort of American system, that’s the D answer.

I’m not going to give you an F. You just gave me the right answer. How can I call that a fail?

But you’re going to get—that is the minimum entry for an answer. Now your job is to make that answer yours.

It just gave you the accepted right answer to the problem. Starting from there, now how would you change that answer to be what you—and only you—on the entire planet would say about that problem?

Because now we live in a world where that’s what society needs from you.

Ross: Yeah, no, what you’ve said there is, I think, fundamentally important. I love that—not just what you said, but also the way you answered my question.

Because this is, as you say, the scope of who we can be is infinite. And we are enabled in that more than ever before. So we just need to accept that challenge.

Vivienne: The very word diversity in many countries is sort of under attack. And we don’t need to call on it totemically.

The research on collective intelligence is clear: teams made up of very different people are smarter, all else being equal. And for obvious reasons—very different people bring a lot more together to the table. So they have a lot more to contribute. Then the collective is smarter than if everyone is very, very similar.

And it is true. So I had a study—set of millions of people interacting in real time during lockdown, when our lives were going through Google Docs and Zoom. Collaborating with a variety of organizations, I had a chance to look at that data and see what made the smartest teams.

The smartest teams—whether they were on the surface, kind of looked identical (same accents, same skin tone, same smell) or they were very, very different—the smartest teams produced the most different ideas.

Using AI to assess the novelty during ideation processes, you could see: the smartest teams explored. Explored aggressively. Thought.

Took other people’s—this is another part of metacognition which is powerful—took other people’s thoughts into account. “Those two are probably thinking this, so even though I kind of agree with that, maybe that’s where the solution lies, I’m explicitly going to start thinking over here, because there’s more open space, more unexplored space over there.”

We looked at optimal incentive systems to maximize collective intelligence, and it was all about getting people—whether they were, you know, facultatively diverse or not—getting them to think differently.

It is just true though—if you bring a bunch of people that have wildly different lived experiences, that difference of thought just comes a lot more easily.

And the last thing you’d want to do is force them back towards the center, to think the same again. We get that for free out of our pockets nowadays.

So whether we’re talking about the notion of diversity that’s played out over the last 10 years, or we want to be thoughtful about some even more kind of inclusive idea about: how do we get anyone to think differently?

And then neurotypicality—my son has autism. Bluntly, so do I. Even though it has not affected me in the hard ways that it has affected him.

The last thing—I want a cure for my son’s diabetes. I don’t want to cure his autism.

Yeah, it creates struggles for him. But he gets being different, like for free. He sees the world so much differently than the rest of us.

Literally, we have brain structures—circuits in our frontal lobes connecting into reward centers—that explicitly cause us to see the world more like the people around us. These social cognitive circuits in our cortex.

And that has value in a world where we need to get along and fit in.

But people with autism don’t share those circuits—at least, they don’t seem to be connecting it up quite the same way.

And so someone like my son can learn, “Oh, other people are saying something—I should take that into account,” but it doesn’t actually change his perception of the world.

I mean, when I say actually—a great experiment was done. Like, am I sitting on a grey chair right now or a blue chair?

Depending on what someone else says, I can change the activity in your visual cortex, all the way back there, to literally make it look more blue or more grey based on what literally another person is saying.

And being freed from that—to truly, veridically see the world as it is—is a superpower my son has, though a lot of people don’t.

So yes, I do not want to use AI to make us all boringly the same. AI already does that for us. So we, we as humans, need to do the exact opposite and explode in every different direction.

Ross: Absolutely right. So where can people find out more about your work?

Vivienne: Well, if you are so foolish enough to still be part of the whole book-reading world—early next year, How to Robot-Proof Your Kids will be out.

And because it’s not out yet, go visit my website. It’s called socos.org—S-O-C-O-S—and you can learn about the book, my newsletter (which is free—unless you’re stupid enough to pay for a free newsletter, which then supports some of my philanthropic work, so always appreciated).

And if you want to learn more about my work in postpartum depression, go visit Dionysus Health. I guess you can’t see this on the camera, but you can visit Dionysus Health.

And if you want to buy into another project—my work in Alzheimer’s—my other company is called Optoceutics. They’re doing amazing work, which doesn’t happen to involve AI.

Not everything has to be AI. But it is about long-term cognitive health and amazing work that we’ve done in that space.

So: socos.org, Dionysus Health, and Optoceutics.

Ross: Fantastic. Thanks so much for your work and your wonderful insights today.

Vivienne: It’s been such a pleasure.

The post Vivienne Ming on hybrid collective intelligence, building cyborgs, meta-uncertainty, and the unknown infinite (AC Ep13) appeared first on Humans + AI.

“The primary source of our reliable ability to produce results under pressure—i.e., skill—is attempting to solve complicated problems with an expert nearby.”

–Matt Beane

Matt Beane is Assistant Professor at University of California Santa Barbara, and a Digital Fellow with both Stanford’s Digital Economy Lab and MIT’s Institute for the Digital Economy. He was employee number two at the Internet of Things startup Humatics, where he played a key role in helping to found and fund the company, and is the author of the highly influential book The Skill Code: How to Save Human Ability in an Age of Intelligent Machines.

Website:

mattbeane.com

LinkedIn Profile:

Matt Beane

University Profile:

Matt Beane

Book:

The Skill Code

Ross Dawson: Matt, it is awesome to have you on the show.

Matt Beane: I’m delighted to be here. Really glad that you reached out.

Ross: So you are the author of The Skill Code. This builds on, I think, research for well over a decade. It came out over a year ago, and now this is very much of the moment, as people are saying all over the place that entry-level jobs are disappearing, and we’re talking about inverted pyramids and so on. So, what is The Skill Code?

Matt: Right. The first third of the book is devoted to the working conditions that humans need in order to build skill optimally.

The myth that is supported by billions of dollars of misdirected investment is that skill comes out of training. And that is—we just have a mountain of evidence that that’s not so. It can help, it can also hurt. But the primary source of our reliable ability to produce results under pressure, IE, skill, is attempting to solve complicated problems with an expert nearby.

Basically, we can learn, of course, without these conditions—sort of idealized conditions—but it can be great. And the first third of the book is devoted to what does it take for it to be great?

I got there sort of backwards by studying how people were trying to learn in the midst of trying to deal with new and intelligent technologies at work—and mostly failing. But a few succeeded. And so I just looked at those success cases and saw what they had in common across many industries and so on.

So, I break that out in the beginning of the book into three C’s—thankfully, in English, this broke out that way: Challenge, Complexity, and Connection. And those roughly equate—well, pretty precisely, actually, I should own the value of the book—they equate to four chunks of characteristics of the work that you’re embedded in that need to be in place in order for you to learn.

Challenge basically is: are you working close to, but not at, the edge of your capacity?

And complexity is: in addition to focusing on getting good at a thing that you’re trying to improve at, are you also sort of looking left and looking right in your environment to digest the full system you’re embedded in? That’s complexity.

And connection is building warm bonds of trust and respect between human beings. All three of those things—I could go into each—but basically, in concert, in no particular sequence—each workplace, each situation is different—but these are the base ingredients.

I used a DNA metaphor in the book. These are sort of the basic alphabet of what it takes to build skill, and your particular process or approach or situation is going to vary in terms of how those show up.

Ross: So, for getting to solutions or prescriptions, I mean, it’s probably worth laying out the problem.

AI or various technologies are making those who are entering the workforce—or entering particular careers—be able to readily do what they do. And essentially, a lot of the classic apprenticeship-style model has been that you learn by making mistakes and, as you say, alongside the masters.

And if people, if organizations, are saying, “Well, we no longer need so many entry-level people to do the dirty, dull work,” then we don’t have this pathway for people to develop those skills in the way you described.

Matt: Yes, and it’s even worse than that.

So, for those that remain—because, of course, organizations are going to hire some junior people—the problems that I document in my research, starting in 2012… Robotic surgery was one early example, but I’ve since moved on to investment banking and bomb disposal—I mean, very diverse examples.

When you introduce a new form of intelligent automation into the work, the primary way that you extract gains from that is that the expert in the work takes that tool and uses it to solve more of the problem per unit time, independently.

That word independently—I saw in stark relief in the operating room. When I saw traditional surgery—I watched many of these—there’s basically two people, shoulder to shoulder, four hands inside of a body, working together to get a job done. And that’s very intensive for that junior person, the medical resident in that case, and they’re learning a lot.

By contrast, in robotic surgery, there are two control consoles for this one robot that is attached through keyhole incisions into the patient. One person can control that robot and do the entire procedure themselves. And so, it is strictly optional then for that senior surgeon to decide that it’s time to give the controls to the junior person.

And when’s the right time to do that, given that that junior person will be slower and make more mistakes? This is true in law, in online education, in high finance, professional services—you name it. The answer is: never.

It is never a good time. Your CFO will be happy with you for not turning the controls over to the junior practitioner. And you yourself, as an expert, are going to be delighted.

People these days, using LLMs to solve coding problems, report lots more dopamine because they can finally get rid of all this grunt work and get to the interesting bits. And that’s marvelous for them. It’s marvelous for the organization—even if it’s uncertain there’s a little ROI.

But the primary, the net, nasty effect of that is that the novice—the junior person trying to learn—is no longer involved in the action. Because why would you?

And that breaks the primary ladder to skill for that person. And so, that, I think, is happening at great scale across…

Let’s put it this way: the evidence I have in hand indicates to me that there will be very rare and rare exceptions to the rule that junior people will be cut out of the action. Even when they’re hired and in the organization and are supposed to be involved, they will just be less involved—because they’re less necessary to support the work.

So even if you get a job as a junior person, you’re not necessarily guaranteed to be learning a dang thing. It’ll be harder these days by default.

Some interesting exceptions—and that’s what I focus on in the book. But that is the—in my view—I’ve done some arithmetic around this, and it’s all estimation of course. I published a piece in The Wall Street Journal on this about eight months ago.

This is a trillion-dollar problem for the economy, in my view.

Ross: Obviously, this is not destiny. These are challenges which we can understand, acknowledge, and address.

So, let’s say—obviously, part of it is, of course, the attitudes of the senior people and how it is they’ll be on frame. A lot can be organizational structures and how work is allocated. There’s a whole array of different things that can be done to at the very least mitigate the problem—or, I think, as you lay out in your book, move to an even better state for the ability to learn and grow and develop in conjunction, not just using learning tools.

But why don’t we go straight to Nirvana? Or what an ideal organization might do. What are some of the things they might do to be able to give these pathways where people can contribute and add value immediately, as well as rapidly grow and develop their capabilities?

Matt: Right. So, I’ll give you a few examples, one of which was evident in my book—and a couple examples, one of which was in the book, and one of which is new since the book’s publication.

So, the one that’s in the book—and that has always occurred, I think, and is more intensely available now and is a real cool and valuable opportunity for organizations—is what I called inverted apprenticeships.

This comes out of a study that I did with a colleague at NYU named Callan Anthony, where we contrasted our surgical and high finance data. We both have sort of “who said what to who every five seconds” kind of transcript data on thousands of hours of work in both contexts.

What was very clear, as we looked across our data, is that it’s not common for this to go well—but it can go well—for senior people to learn about new tech from junior people.

The “ha ha” example at a cocktail party is the CEO learning about TikTok from their executive assistant. But in the real world, senior software developers are definitely learning about how to use AI to amplify their productivity from junior people.

Organizations now are talking out of both sides of their mouth. On the one hand, you have people saying, “Well, we’re only going to hire senior people.” At the same time, “You have to be AI-native as a junior person.” That’s what we’re looking for, and that’s a prized skill.

Whether they know that that’s what they’re after or not, what they’re setting up when those people arrive is this relationship where the junior person hangs out and works with—and gets to teach, so to speak, or show by example—the senior person how to use AI.

The senior person, sort of as the price of entry for that working relationship, gives that junior person more access to their work and complex problem solving.

The paper itself is worth reading. The section in the book is worth reading because there are lots of ways to do this that are quite exploitative with respect to that junior person—sort of, they have to pay double. But there are ways of doing it where both people—it’s sort of a win-win.

That mode of simultaneous bi-directional learning is going to be really important if you want to adapt as an organization, just on a hyper-local level. So, that’s example one.

The other example—I’ve been, in the last four months now, in a new study I’ve been doing with five doctoral students here at University of California, Santa Barbara. It’s an interview-based study of the use of generative AI in software development across over 80 organizations.

One of the things that has emerged as a working pattern there, that I think is really intriguing and potentially a great example to think with—a sort of design template for how to set work up in a way that seizes the gains while also involving junior people and building your bench strength—is that:

In some cases, anyway, senior software engineers, rather than writing code, will get, say, four to five junior engineers together and give them all impossible tasks—like hugely complicated work and very limited time.

They will all try their… and by the way, obviously, the only way you could attempt this is to use AI—just cheat as aggressively as possible—and then submit your code. You’re talking three weeks of work in two hours, or eight hours, or something like that.

Under that kind of pressure, junior people—their neuroplasticity and willingness to throw themselves into the breach—is the hugest asset.

Everyone involved knows that what they submit may work, and it will be terrible. But it will be terrible in subtle ways.

Then that senior person spends some time with each of those junior people to do a code review or some pair programming, to say, “Right, here are the three or four areas. I’m not going to tell you what the problems are—where there’s problems—go have a go at figuring out what they are and fixing them.”

Or maybe: “I’ll just tell you what they are, and do you see why those are problems?”

Basically, we’re just focusing on the parts of what you built that are problematic—that you might not quite get yet. But 80% of what you built is fit for duty, and I got it 90% faster than I would have otherwise.

That senior person then is sort of a filter feeder. They deal with code and process and review a lot more than they used to actually just write.

But the unit total factor productivity for that group is an order of magnitude higher than it used to be. So, that’s become the sort of template—or the sort of fractal example—that I think…

Treating this hallucination and inconsistency and output problem as a feature, not a bug, and designing your organization to take advantage of that—I could easily see that kind of example scaling into professional services, into law, into medicine.

I mean, where failure in process is acceptable—it’s the output that needs to be high quality—it just seems like savvy organizations are going to be making design choices like that left and right.

Ross: That’s fantastic. So, where did that come from? Is that something which you created and then shared with these organizations? Or did you see this in the wild?

Matt: This is from this interview study. We have a globally representative sample of firms, and all we’re doing is asking them, “What are you doing with Gen AI in software development?” And then they talk for an hour, basically. We have a bunch of specific questions.

So no, we’re not priming anything, we’re not suggesting anything, we’re not sharing information in between them. And this is showing up independently across a number of organizations.

So anyway, there are lots of other cool things popping up, but the fact that these organizations aren’t in touch with one another—they don’t seem to be—they aren’t saying that they got this trick off of Reddit or from some influencer on Twitter, and that some subset of them have invented it locally, is a pretty strong indicator that it’s at least representative of a new potential direction.

Ross: So, this is work yet to be published?

Matt: Correct.

Ross: When? When will it be out?

Matt: That doesn’t operate on AI time. That’s on academic time. If we get enough findings together that I believe will meet the high A+ academic journal standard that I’m used to—which is not obvious, but I think we have a good shot—we’ll submit it for publication sometime in the fall.

Then it’ll probably be two years before the findings come out. You can post a working paper right away, and so as soon as we can do that, we will.

Ross: Awesome. Yeah, because this is the Humans Plus AI podcast. And really, the core of what I think about is humans plus AI workflow.

What is the flow of work between humans and AI? What are their respective roles? How does it move? What are the parallel and linear structures, and so on?

And what you’ve described is a wonderful, pretty clear humans plus AI workflow which is replicable. It can work in different contexts, as you say, across different domains. And these archetypes—if we can uncover these archetypes at work—then that is extraordinary value.

Matt: I think so, yeah. And what’s important is that, I think for them to be valid, they have to show up independently in very different contexts.

Then you’ve got your hands on—potentially, anyway—something that is suited to the new environment. There are many, many cases in which these best practices get trotted out, and they’ve been started by one organization and then shared across.

You can see a clear lineage, and then you have real questions about what, in academic speak, is endogeneity. In other words, it might be that this new best practice is not actually useful. It’s just that people are persuasive about it, and it travels fast because people are desperate for solutions.

So, we have to be very careful about grabbing best practices and labeling them as such.

Ross: You mentioned investment banking as a domain you’ve been exploring. And I think—I look a lot around professional services—and I think professional services are not just your classic accounting and law, and so on.

I mean, arguably, many industries—healthcare is professional services. I mean, if you look inside a consumer products company, they are professionals. You know, the building… there’s a lot of archetypes of things or structures there.

So I’m very interested to see what of what you have seen work in that context—what has been effective in being able to develop capabilities of junior staff.

Matt: Right, yeah. And I have less data there, but I’m always on the hunt for patterns in work that—when you look at them—you think, “I would need some evidence to conclude that that is not valuable or showing up somewhere else.”

In other words, it seems quite portable and generalizable. It’s not bound to the content of the work or some legal barriers or structures around the occupation or profession.

There are some places where that really is true. But as long as it seems like you could do the same thing in any knowledge work profession, then I agree with you. I think those are really important tactics.

And I don’t think anybody really—aside from what I offer in the book, which was my best offering then and I still feel very good about it now—is that whatever the new workflow is, imagined workflow, I offer a ten-point checklist for each of those three Cs in each of those chapters.

It’s about how you would know—very specifically and measurably—whether work was skill-enhancing or skill-degrading the more you did it over time.

Anyone, anywhere, I think, can take a look at any new way of doing the work that involves AI and interrogate it from that lens. So, in addition to a productivity lens—which is obviously critical—you can also say, “Is this likely to enhance skill development or not, if we do it this new way?”

And you can. It takes work, but I think that’s quite necessary.

Ross: So, looking at your three elements of challenge, complexity, and connection—AI used well could assist on a number of those.

Perhaps for me, most obviously in connection, where we have a lot of great studies in collaborative intelligence, where AI is playing a role in being able to nudge interactions that support collective intelligence. Again, we could have AI involved in interactions and able to say, “Well, here’s an opportunity to connect in a particular way to a particular person in a particular context,” for example.

Or it could be able to say, “You’re working on this particular challenge. Let’s give some context to this,” and so on. So either hypothetically or in practice—where are ways you’ve seen AI being able to amplify the challenge, complexity, or connection of skill development?

Matt: I have a Substack called Wild World of Work. It’s at wildworldofwork.org, and one of the first posts I wrote there—forgotten, it’s over a year ago now—is called Don’t Let AI Dumb You Down.

In that piece, I talk about how default use of GenAI—to ChatGPT—is, just as with all these other forms of intelligent automation I’ve studied, likely to deprive you of skill over time.

I’ll just start with connection. One of the reasons for that is that you don’t leave your screen. You get your answer, and it might even be good, and you might even learn some new information—so it’s not just passive, like “do my homework for me” kind of interaction.

But what you won’t notice is missing—and definitely is—is another human being. And ChatGPT is currently not configured—it’s not post-trained, technically—to do anything about that, to attend to that, or to have your welfare with respect to your skills in its consideration set at all.

You can make it do that, though. This is the amazing thing. Even what I suggested in that article back then is still true today. You can go into the custom settings for ChatGPT—and all these models have this now—and you can tell it how to interact with you, basically.

What I have in my custom settings in ChatGPT are specific sets of instructions around: basically, annoy me to some degree so that I need to do things for myself. Keep me challenged. Expose me to complexity—other things going on related to this work—and, as you just said, push me towards other human beings and building bonds of trust and respect with them.

Because otherwise, I’ll just rely on you. And that is what ChatGPT does to me every single time now.

Do I heed its advice all the time? No, of course not. But I have definitely learned a lot of things and met new people that I wouldn’t have if I hadn’t done that. It’s certainly not perfect. And it’s gotten better, but still.

And by the way, it should not be incumbent on the user, in my opinion, to go fix these things for themselves. That’s like asking cigarette smokers to install their own filters or something. You could, in principle, do that, but…

I think—put it this way, positively—there’s a huge market opportunity for these model providers. For any one of them to hold up their hand and say, “We have configured our system such that just by using it, you’re going to have more skills at the end of next week than this week. And you can have your results too.”

None of them have done that. Isn’t that interesting?

I’m trying to embarrass them into doing it, basically, because I think people have a strong and growing intuition that they’re trading something away in exchange for just getting their answer from this magical tech.

A few people aren’t. A few people are both getting their answer and pushing themselves farther than they ever could have before. That’s magical territory, and we need to understand it.

Anyway, I think once the word gets out that this trade-off is going on, then people are gonna start to insist. And I hope we can get some model company to lead in that regard.

Ross: Fantastic. In your book, you refer to Cabrera’s—essentially bringing the humans and AI together. Obviously, people use different terminology around that.

But where do you see the potential now for these human-AI integrations?

Matt: Yep. I have not yet seen this implemented, but the idea I’m just about to describe could have been implemented a year ago—very clearly. Technically, it was possible then; it’s even better now.

Let’s just say I’m a worker at Procter & Gamble, and I work in the marketing function. My agent could be eating all of my emails, all my calendar appointments, and all the documents I produce. It could be looking at my projects and looking for opportunities for me that might offer useful sort of upramp for a certain skill area that I’m interested in.

That agent could then also be conferring with other agents of project managers throughout the corporation to see if there’s a good match.

We’ve seen this “chain of thought” in models before. Just imagine two models meshing their chains of thought. Lots of back-and-forth, like:

“Hey, Matt Beane’s looking to develop this kind of skill, and it looks like you’ve got a project over there.”

This agent over there is more plugged into that context. They spend some time—they can do this at the speed of light—but there’s a quick burning of tokens to assess the utility of that match from my point of view and from that project’s point of view.

Then you get much finer-grained, higher-quality matches of resources, human resources—to projects. The project wins. I win because I get a skill development opportunity. And those agents do most of the legwork to make that match.

You could likewise imagine that with a performance review. So if you’re my manager and I’m your employee, our agents are conferring regularly, constantly about my work, your availability, and so on.

Your agent might pop back to you and say, “Hey, I’ve been talking to Matt’s agent, and it looks like now’s a pretty good time for you two to have a quick performance-oriented conversation about his project—because he’s done really well on these three things and is struggling on these and could use your guidance.”

Then we get these regular—but AI-driven and scheduled—performance review conversations. Both those agents could help us prep for those conversations.

“Here’s a suggested conversation for you two.”

When it comes time for performance reviews—the formal one—we’ve already had a bunch of those. But they aren’t just some arbitrary every-two-week check-in kind of thing. Each is driven by a real, actual, evident challenge or opportunity or strength in my work.

So anyway, I think those are just two kind of hand-wavy examples that I think are implementable now.

Increasingly autonomous AI systems that can call tools, have access to memory, and confer with one another can solve this sort of talent mobility problem within firms—making matches so that I build my skill and we get results and performance optimization.

Any firm would be… I mean, that’s low-hanging fruit almost. For somebody who has no technical expertise to set up—you can just build an internal GPT that does those things. There’s a little bit more required, but anyway…

There is a universe of new modes of organizing that assume agents will be doing most of the talking, and just set humans up for success whenever possible.

You can always turn it away. It’s like getting a potential match on a dating app. You’d be like, “No, not that one.”

But at least—no human could ever manage an organization that well and make matches at that frequency and level of fidelity.

Ross: Yeah, this goes very much to what I’ve long described as the fluid organization, where people get connected to where they can best apply their capabilities—and also to learn—completely fluidly.

Not depending on where their particular job description lies, but simply where their talents and their talent development can be best applied across the organization.

There have been, for quite some time, talent platforms within organizations for connecting people with opportunities or work, and so on. But obviously, AI-enabled—and particularly with a talent development focus—provides far more opportunity.

Matt: I’ve been trying to track this pretty closely because I have a startup now focused on this joint optimization of work performance measurement with human capability development.

The previous wave of firms—B2B SaaS firms—that are trying to solve this talent mobility problem have really been focused on extracting skills from workers’ data and collecting those as a bag of nouns, and trying to match that bag of nouns against a potential opportunity.

And those nouns are just not sufficiently rich to capture what it is that those people are capable of—or not.

But I think a much richer sort of dialogue-based, dynamic, up-to-date, in-the-moment interaction between two informed agents…

You’re informed about the opportunity on the project. You have all the project docs spun up into you—I mean you as an agent.

And then another agent—that is mine—advocates for me on my behalf and has a giant RAG-based system (or whatever is the state of the art) that knows all about me: my preferences, what motivates me, my background, my capabilities under pressure, my career aspirations—all the rest.

Then they could spend a 100-turn conversation assessing fit in a few seconds. And that will be radically better than, “Does this noun match that noun?”

Ross: Yeah, a lot of potential.

So, to round out—for organizational leaders, whether they be the C-suite, or board C-suite, or HR, or L&D, or organizational development—what are the prescriptions you give? What is the advice you would give on how to evolve into an organization where you can have a talent pipeline and maximize the learning that is going to be relevant for today?

Matt: I mention this in the book—lean on the vendors of these AI systems and demand that they give you a product that will enhance the skills of its users while generating results.

There are plenty of design decisions you could make about how to build the organization. We’ve talked about some of them. I think those are important. They’re necessary.

You can hire for AI-native talent. You can set up inverted apprenticeships. But if the root stock—or the new tool that everyone is supposed to use to optimize whatever they’re trying to optimize—is infected with a virus, and the virus is that it will drive experts and novices apart in search of results, almost unwittingly…

Very few will even notice this, or if they do notice it, they’re just not incented to care.

There’s really—I mean, L&D is maybe the only function in the organization that is explicitly put together to know about and deal with this problem—but it’s now a compliance function. The training that L&D offers is just kind of a box-checking activity too often.

So you can’t count on yourself and your own organization and your own chutzpah—and pulling yourself up, or asking your employees to pull themselves up by their bootstraps—as a primary means of ensuring that you grow your talent bench while improving results from AI.

I think companies—and executives in particular—are in a very powerful position right now to choose between model vendors and ask:

“Give them two extra weeks to come back with something in their proposal that gives you reasonable assurance that just by using their product—versus their competitors’—your employees will build skill more and end up with better career outcomes, while still getting productivity gains.”

How can we use this tech and build employee skill at the same time? — that is the powerful question.

So it’s not… I think these vendors need to start to feel some heat. And if you’re a manager, you should be thinking:

“Fine, I’m getting some uncertain and notional—or nominal—productivity gain out of these new tools now just by buying them, and I don’t want to get left behind.”

So not buying is probably not an option.

But anyway, know also that if you just turn it on and hand out licenses, you will de-skill your workforce faster than you expect, and you will be knee-capping your organization for, say, three years from now or five years from now. And you will lose to your competitors.

I guarantee it.

Well, no—guarantee with a big asterisk. There will be many cases in which having fewer junior employees is the right thing to do. There will be many cases in which you don’t really care about de-skilling relative to the gains that you could get productivity-wise. I’m not naive about any of that.

But if you have areas in your organization where you have highly paid talent that is very mobile and wants to learn and grow, they will figure out which organizations are giving them work that will drive their skill curve upward—and they will vote with their feet.

And then you will stop getting high-quality talent.

That is one problem area I would get ready for.

And the other is: get ready to offer remedial training for those people who should know how to do their jobs—but in fact, have not been upskilling because they’ve been using AI too much. And you’ll be bearing that cost as well.

Organizations that invest now to address this problem—they will not. Might come slower out of the gate right now. Maybe they won’t. Maybe they’ll jump ahead faster.

So I think intervening with the model provider is one unexpected and easy place to go—because they won’t see it coming. They will be surprised.

And if a smart business development person—who wants their commission—will go back to their organization and tell OpenAI or Anthropic or Google, “Hey, what can we do?”

And I’m hearing this from lots of people. So I’m not naive to think that just me saying this to you on this podcast is going to have that effect.

I think really what’s starting to happen is that professionals—especially software professionals, right now—are starting to notice this effect without Matt Beane being in the picture at all.

There are articles out there now by software developers saying: “The death of the junior developer” is one. It’s a great one.

They’re all getting concerned on their own.

So I hope that the pressure just gets turned up, and that one of these companies comes out with something that will make a difference.

Ross: Fantastic. Thanks so much for your time, Matt.

Matt: Pleasure

Ross: Wonderful work. Very, very much on point for these days. Extraordinarily relevant. And I very much look forward to seeing what you continue to uncover and share and publish.

Matt: Perfect. Thank you. Like I said, I really appreciated the invite and happy to talk.

The post Matt Beane on the 3 Cs of skill development, AI augmentation design templates, inverted apprenticeships, and AI for skill enhancement (AC Ep12) appeared first on Humans + AI.

“We’re in this process where we should be discovering what’s possible… That’s what I mean by AI-native — just go figure out what the AI can do that makes something so much easier or so much better.”

– Tim O’Reilly

Tim O’Reilly is the founder, CEO, and Chairman of leading technical publisher O’Reilly Media, and a partner at early stage venture firm O’Reilly AlphaTech Ventures. He has played a central role in shaping the technology landscape, including in open source software, web 2.0, and the Maker movement. He is author of numerous books including WTF? What’s the Future and Why It’s Up to Us.

Website:

www.oreilly.com

LinkedIn Profile:

Tim O’Reilly

X Profile:

Tim O’Reilly

Articles:

AI First Puts Humans First

An Architecture of Participation for AI?

AI and Programming: The Beginning of a New Era

Redefining AI-native beyond automation

Tracing the arc of human-computer communication

Resisting the enshittification of tech platforms

Designing for participation, not control

Embracing group dynamics in AI architecture

Unlocking new learning through experimentation

Prioritizing value creation over financial hype

Ross Dawson: Tim, it is fantastic to have you on the show. You were my very first guest on the show three years ago, and it’s wonderful to have you back.

Tim O’Reilly: Well, thanks for having me again.

Ross: So you have seen technology waves over decades and been right in there forming some of those. And so I’d love to get your perspectives on AI today.

Tim: Well, I think, first off, it’s the real deal. It’s a major transformation, but I like to put it in context. The history of computing is the history of making it easier and easier for people to communicate with machines.

I mean literally in the beginning, they had to actually wire physical circuits into a particular calculation, and then they came up with the stored program computer. And then you could actually input a program one bit at a time, first with switches on the front of the computer. And then, wow, punch cards.

And we got slightly higher level languages. First it was big, advanced assembly programming, and then big, advanced, higher level languages like Fortran, and that whole generation.

Then we had GUIs. I mean, first we had command lines. Literally the CRT was this huge thing. You could literally type and have a screen.

And I guess the point is, each time that we had an advance in the ease of communication, more people used computers. They did more things with them, and the market grew.

And I think I have a lot of disdain for this idea that AI is just going to take away jobs. Yes, it will be disruptive. There’s a lot of disruption in the past of computing. I mean, hey, if you were a programmer, you used to have to know how to use an oscilloscope to debug your program.

And a lot of that old sort of analog hardware that was sort of looking at the waveforms and stuff — not needed anymore, right?

I remember stepping through programs one instruction at a time. There’s all kinds of skills that went away. And so maybe programming in a language like Python or Java goes away, although I don’t think we’re there yet, because of course it is simply the intermediate code that the AIs themselves are generating, and we have to look at it and inspect it.

So we have a long way before we’re at the point that some people are talking about — evanescent programs that just get generated and disappear, that are generated on demand because the AI is so good at it. It just — you ask it to do something, and yeah, it generates code, just like maybe a compiler generates code.

But I think that’s a bit of a wish list, because these machines are not deterministic in the way that previous computers were.

And I love this framework that there’s really — we now have two different kinds of computers. Wonderful post — trying to think who, name’s escaping me at the moment — but it was called “LLMs Are Weird Computers.” And it made the point that you have, effectively, one machine that we’re working with that can write a sonnet but really struggles to do math repeatedly. And you have another type of machine that can come up with the same answer every single time but couldn’t write a sonnet to save its life.

So we have to get the best of both of these things. And I really love that as a framework. It’s a big expansion of capability.

But returning back to this idea of more — the greater ease of use expanding the market — just think back to literacy. There was a time when there was a priesthood. They were the only people who could read and write. And they actually even read and wrote in a dead language — Latin — that nobody else even spoke. So it was this real secret, and it was a source of great power.

And it was subversive when they first, for example, printed the Bible in English. And literally, when they printed the printed book — the printed book was the equivalent of our current, “Oh my God, social media turbocharged with AI, social disruption.”

There was 100 years of war after the dissemination of movable type, because suddenly the Bible and other books were available in English. And it was all this mass communication, and people fought for 100 years.

Now, hopefully we won’t fight for 100 years. But disruption does happen, and it’s not pretty. But it’s not — there’s a way that the millennialist kind of version of where this is somehow terminal is just wrong.

I mean, we will evolve. We will figure out how to coexist with the machines. We’ll figure out new things to do with them. And I think we need to get on with it.

But I guess, back to this post I wrote called “AI First Puts Humans First,” there’s a lot of pressure from various companies. They’re saying you must use AI. And they’ve been talking about AI first as a way of, like, “If you try to do it with AI first because we want to get rid of the people.”

And I think of AI first — or what I prefer, the term AI native — as a way of noticing: no, we want to figure out what the capabilities of this machine are. So try it first, and then build with it.

And in particular, I think of the right way to think about it as a lot like the term “mobile first.” It didn’t mean that you didn’t have other applications anymore. It just meant, when companies started talking about mobile first, it meant we don’t want it to be an afterthought.

And I think we need to think that way about AI. How can we reinvent the things that we’re doing using AI? And anybody who thinks it’s just about replacing people is missing the point.

Ross: Yeah, well, that’s going back to the main point around the ease of communication. So the layers of which we are getting our intent to be able to flow through into what the computers do.

So what struck me with the beginning of LLMs is that what is distinctive about humans is our intention and our intention to achieve something. So now, as you’re saying, the gap between what we intend and what we can achieve is becoming smaller and smaller, or it’s getting narrower and faster.

Also, we can democratize it in the sense of — yeah, there is more available to more people in various guises, to different degrees, where you can then manifest in software and technology your intention.

Yeah, so that democratizes — as you say, this is — there are ways in which this is akin to the printing press, because it democratizes that ability to not just understand, but also to achieve and to do and to connect.

Tim: Yeah, there is an issue that I do think we need to confront as an industry and as a society, and that is what Cory Doctorow calls “enshittification.”

This idea — actually, I had a different version of it, but let’s talk about Cory’s version first. The platforms first are really good to their users. They create these wonderful experiences. Then they use the mass of users that they’ve collected to attract businesses, such as advertisers, and they’re really good to the advertisers but they’re increasingly bad to the users.

Then, as the market reaches a certain saturation point, they go, “Well, we have to be bad to everybody, because we need the money first. We need to keep growing.”

I did a version of this. I wrote a paper called Rising Tide Rents and Robber Baron Rents, where I used the language of economic rents. We have this notion of Schumpeterian rents — or Schumpeterian profits — where a company has innovated, they get ahead of the competition, and they have outsized profits because they are ahead.

But in the theory, those rents are supposed to be competed away as knowledge diffuses. What we’ve seen in practice is companies put up all kinds of moats and try to keep the knowledge from diffusing. They try to lock in their users and so on. Eventually, the market stagnates, and they start preying on their users.

We’re in that stage in many ways as an industry. So, coming to AI, this is what typically happens. Companies stagnate. They become less innovative. They become protective of their profits. They try to keep growing with, effectively, the robber baron rents as opposed to the innovation rents.

New competition comes along, but here we have a problem — the amount of capital that’s had to go into AI means that none of these companies are profitable. So they’re actually enshittified from the beginning, or the enshittification cycle will go much, much more quickly, because the investors need their money.

I worry about that.

This has really been happening since the financial crisis made capital really cheap. We saw this with companies like Lyft and Uber and WeWork — that whole generation of technology companies — where the market didn’t choose the winner. Capital chose the winner.

The guy who actually invented all of that technology for on-demand cars was Sunil Paul with Sidecar. Believe it or not, he raised the same amount of money that Google raised — which was $35 million.

Uber and Lyft copied his innovations. Their venture was doing something completely different. Uber was black cars summoned by SMS. Lyft was a web app for inner-city people trying to find other people to share rides between cities.

They pivoted to do what Sunil Paul had invented, and they threw billions at it, and they bought the market.

Sure enough, the companies go public, unprofitable. Eventually, after the investors have taken out their money — it’s all great — then they have to start raising prices. They have to make the service worse.

Suddenly, you’re not getting a car in a minute. You’re getting a car in 10 minutes. They’re telling you it’s coming in five, and it’s actually coming in 15.

So it’s — and I think that we have some of that with AI. We’re basically having these subsidized services that are really great. At some point, that’s going to shake out.

I think there’s also a way that the current model of AI is fundamentally — it’s kind of colonialism in a certain way. It’s like, we’re going to take all this value because we need it to make our business possible. So we’re going to take all the content that we need. We’re not going to compensate people. We’re going to make these marvelous new services, and therefore we deserve it.

I think they’re not thinking holistically.

Because this capital has bought so much market share, we’re not having that kind of process of discovery that we had in previous generations. I mean, there’s still a lot of competition and a lot of innovation, and it may work out.

Ross: I’m just very interested in that point. There’s been a massive amount of capital. There’s this thesis that there is a winner-takes-most economy — so if you’re in, you have a chance of getting it all.

But overlaid on that — and I think there’s almost nobody better to ask — is open source, where of course you’ve got commercial source, you’ve commercially got open source, and quite a bit in between.

I’d love to hear your views on the degree to which open source will be competitive against the closed models in how it plays out coming up.

Tim: I think that people have always misunderstood open source, because I don’t think that it is necessarily the availability of source code or the license. It’s what I call an architecture of participation.

This is something where I kind of had a falling out with all of the license weenies back in the late ’90s and early 2000s, because — see, my first exposure to what we now call open source was with Berkeley Unix, which grew up in the shadow of the AT&T System V license. That was a proprietary license, and yet all this stuff was happening — this community, this worldwide community of people sharing code.

It was because of the architecture of Unix, which allowed you to add. It was small. It was a small kernel. It was a set of utilities that all spoke the same protocol — i.e., you read and wrote ASCII into a stream, which could go into a file.

There were all these really powerful concepts for network-based computing.

Then, of course, the internet came along, and it also had an architecture of participation. I still remember the old battle — Netscape was the OpenAI of its day. They were going to wrest control from Microsoft, in just the same way that OpenAI now wants to wrest control from Google and be the big kahuna.

The internet’s architecture of participation — it was really Apache that broke it open more than Linux, in some ways. Apache was just like, “Hey, you just download this thing, you build your own website.”

But it wasn’t just that anybody could build a website. It was also that Apache itself didn’t try to Borg everything.

I remember there was this point in time when everybody was saying Apache is not keeping up — Internet Information Server and Netscape Server are adding all these new features — and Apache was like, “Yeah, we’re a web server, but we have this extension layer, and all these people can add things on top of it.”

It had an architecture of participation.

The same thing happened with things like OpenOffice and the GIMP, which were like, “Okay, we’re going to do Microsoft Office, we’re going to do Photoshop.”

They didn’t work, despite having the license, despite making the source code available — because they started with a big hairball of code. It didn’t have an architecture of participation. You couldn’t actually build a community around it.

So I think — my question here with AI is: Where is the architecture of participation?

Ross: I would argue that it’s an arXiv, as in that now basically, the degree of sharing — where you get your Stability and your Googles and everyone else just putting it out on arXiv for your deep seek — really detailed.

Tim: Yeah, I think that’s absolutely right. There is totally an architecture of participation in arXiv.

But I think there’s also a question of models. I guess the thing I would say is yes — the fact that there are many, many models and we can build services — but we have to think about specialized models and how they cooperate. That’s why I’m pretty excited about MCP and other protocols.

Because the initial idea — the winner-takes-all model — is: here we are, we’re OpenAI, you call our APIs, we’re the platform. Just like Windows was. That was literally how Microsoft became so dominant.

You called the Windows API. It abstracted — it hid all the complexity of the underlying hardware. They took on a bunch of hard problems, and developers went, “Oh, it’s much easier to write my applications to the Windows API than to support 30 different devices, or 100 different devices.” It was perfect.

Then Java tried to do a network version of that — remember, “Write once, run anywhere” was their slogan. And in some sense, we’re replaying that with MCP.

But I want to go back to this idea I’ve been playing with — it’s an early Unix idea — and I’ve actually got a piece that I’m writing right now, and it’s about groups. Because part of an architecture of participation is: what’s the unit of participation?

I’ve been thinking a lot about one of the key ideas of the Unix file system, which was that every file had, by default, a set of permissions. And I think we really need to come up with that for AI.

I don’t know why people haven’t picked up on it. If you compare that to things like robots text and so on, there’s a pretty simple way. Let me explain for people who might not remember this. Most people who are developers or whatever will know something about this.

You had a variable called umask, which you set, and it set the default permissions for every file you created. There was also a little command called chmod that would let you change the permissions.

Basically, it was read, write, or execute — and it was for three levels of permission: the user, the group, and the world (everyone) right?

So here we are with AI, saying, “We, OpenAI,” or “We, Grok,” or whoever, “are going to be world,” right? “We’re going to Borg everything, and you’re going to be in our world. Then you’ll depend on us.”

Then some people — like Apple maybe — are saying, or even other companies are saying, “Well, we’ll give you permission to have your own little corner of the world.” That’s user. “We’ll let you own your data.”

But people have forgotten the middle — which is group.

If you look at the history of the last 20 years, it’s people rediscovering — and then forgetting — group. Think about what was the original promise of Twitter, or the Facebook feed. It was: I can curate a group of people that I want to follow, that I want to be part of.

Then they basically went, “No, no, actually that doesn’t really work for us. We’re going to actually override your group with our algorithmic suggestions.”

The algorithmically generated group was a really fabulous idea. Google tried to do a manual version of that when they did — originally Buzz — and then, was it called Circles? Which was from Andy Hertzfeld, and was a great thing.

But what happens? Facebook shuts it off. Twitter shuts it off.

And guess what? Where is it all happening now? WhatsApp groups, Signal groups, Discord groups. People are reinventing group again and again and again.

So my question for the AI community is: Where is group in your thinking?

How do we define it? A group can be a company. It can be a set of people with similar beliefs.

There’s a little bit of this, in the sense that — if you think Grok, the group is — even though it aspires to be the world-level — you could say Anthropic is the, let’s call it, the “woke group,” and Grok is the “right group.”

But where’s the French group? The French have always been famously protective. So I guess Mistral is the French group.

But how do people assert that groupness?

A company is a group.

So the question I have is, for example: how do we have an architecture of participation that says, “My company has valuable data that it can build services on, and your company has valuable data. How do we cooperate?”

That’s again where I’m excited — at least the MCP is the beginning of that. Saying: you can make a set of MCP endpoints anywhere.

It’s a lot like HTTP that way. “Oh, I call you to get the information that I want. Oh, I call you over here for this other information.”

That’s a much more participatory, dynamic world than one where one big company licenses all the valuable data — or just takes all the valuable data and says, “We will have it all.”

Ross: That’s one of the advantages of the agentic world — that if you have the right foundations, the governance, the security, and all of the other layers like team, payments, etc., then you can get entirely an economy of participation of agents.

But I want to look back from what you were saying around groups, coming back to the company’s point around the “AI first” or “AI native,” or whatever it may be. And I think we both believe in augmenting humans.

So what do you see as possible now if we look at an organization that has some great humans in it, and we’ve got AI that changes the nature of the organization? It’s not just tacking on AI to make each person more productive. I think we become creative humans-plus-AI organizations.

So what does that look like at its best? What should we be aspiring to?

Tim: Well, the first thing — and again, I’m just thinking out loud from my own process — the first thing is, there’s all kinds of things that we always wished we could do at O’Reilly, but we just didn’t have the resources for, right?

And so that’s the first layer. The example I always use is, there are people who would like to consume our products in many parts of the world where they don’t speak English. And we always translated a subset of our content into a subset of languages.

Now, with AI, we can make versions that may not be as good, but they’re good enough for many, many more people. So — vast expansion of the market there, just by going, “Okay, here’s this thing we always wished we could do, but could not afford to do.”

Second is: okay, is there a new, AI-native way to do things?

O’Reilly is a learning platform, and I’m looking a lot at — yeah, we have a bunch of corporate customers who are saying, “How do you do assessments? We need to see verified skills assessment.” In other words, test people: do they actually know this thing?

And I go — wow — in an AI-native world, testing is a pretty boneheaded idea, right? Because you could just have the AI watch people.

I was getting a demo from one startup who was showing me something in this territory. They had this great example where the AI was just watching someone do a set of tasks. And it said, “I noticed that you spent a lot more time and you asked a lot more questions in the section that required use of regular expressions. You should spend some time improving your skills there.”

The AI can see things like that.

Then I did kind of a demo for my team. I said, “Okay, let me just show you what I think AI-native assessment looks like.” I basically found some person on GitHub with an open repository.

I said, “Based on this repository, can you give me an assessment of this developer’s skills — not just the technical skills, but also how organized they are, how good they are at documentation, their communication skills?”

It did a great write-up on this person just by observing the code.

Then I pointed to a posted job description for an engineer working on Sora at OpenAI and said, “How good of a match is this person for that job?”

And it kind of went through: “Here are all the skills that they have. Here are all the skills that they need.”

And I go — this is AI-native. It’s something that we do, and we’re doing it in probably a 19th-century way — not even a 20th-century way — and you have completely new ways to do it.

Now, obviously that needs to be worked on. It needs to be made reliable. But it’s what I mean by AI-native — just go figure out what the AI can do that makes something so much easier or so much better.

That’s the point.

And that’s why it drives me nuts when I hear people talk about the “efficiencies” to be gained from AI.

The efficiencies are there. Like, yeah — it was a heck of a lot more efficient to use a steam engine to bring the coal out of the mine than to have a bunch of people do it. Or to drive a train. I mean, yeah, there’s efficiency there.

But it’s more that the capability lets you do more.

So we’re in this process where we should be discovering what’s possible.

In this way, I’m very influenced by a book by a guy named James Bessen. It’s called Learning by Doing, and he studied the Industrial Revolution in Lowell, Massachusetts, when they were bringing cotton mills and textile mills to New England.

He basically found that the narrative — AI had unskilled labor replaced skilled labor — wasn’t quite right. They had these skilled weavers, and then these unskilled factory workers. And he looked at pay records and said it took just as long for the new workers to become fully paid as the old workers.

So they were just differently skilled.

And I think “differently skilled” is a really powerful idea.

And he said okay, why did it take so long for this to show up in productivity statistics — 20, 30 years? And he said, because you need a community.

Again — this is an architectural part. You need people to fix the machines. You need people to figure out how to make them work better. So there’s this whole community of practice that’s discovering, thinking, sharing.

And we’re in that ferment right now.

That’s what we need to be doing — and what we are doing. There’s this huge ferment where people are in fact discovering and sharing.

And back to your question about open source — it’s really less about source code than it is about the open sharing of knowledge. Where people do that.

That goes back to O’Reilly. What we do — we describe our mission as being “changing the world by spreading the knowledge of innovators.”

We used to do it almost entirely through books. Then we did it through books and conferences. Now we have this online learning platform, which still includes books but has a big live training component.

We’re always looking for people who know something and want to teach it to other people.

Then the question is, what do people need to know now that will give them leverage, advantage, and make them — and their company — better?

Ross: So just to round out, I mean, you’ve already — well, more than touched on this idea of learning.

So part of it is, as you say, there are some new skills which you need to learn. There’s new capabilities. We want to go away from the old job description because we want people to evolve into how they can add value in various ways.

And so, what are the ways? What are the architectures of learning?

I suppose, as you say, that is a community. It’s not just about delivering content or interacting. There’s a community aspect.

So what are the architectures of learning that will allow organizations to grow into what they can be as AI-native organizations?

Tim: I think the architecture of learning that’s probably most important is for companies to give people freedom to explore.

There’s so many ideas and so much opportunity to try things in a new way. And I worry too much that companies are looking for — they’re trying to guide the innovation top-down.

I have another story that sort of goes back to — it’s kind of a fun story about open source.

So, yeah, one of the top guys at Microsoft is a guy named Scott Guthrie. So Scott and one of his coworkers, Mark Anders, were engineers at Microsoft, and they had basically this idea back in the early — this is 20-plus years ago — and they basically were trying to figure out how to make Windows better fitted for the web.

And they did a project by themselves over Christmas, just for the hell of it. And it spread within Microsoft. It was eventually what became ASP.NET, which was a very big Microsoft technology — I guess it was in the early 2000s.

It kind of spread like an open source project, just within Microsoft — which, of course, had tens of thousands of employees.

Eventually, Bill Gates heard about it and called them into his office. And they’re like, “Oh shit, we’re gonna get fired.”

And he’s like, “This is great.” He elevated them, and they became a Microsoft product.

But it literally grew like an open source project.

And that’s what you really want to have happen. You want to have people scratching their own itch.

It reminds me of another really great developer story. I was once doing a little bit of — I’d been called into a group at SAP where they wanted to get my advice on things. And they had also reached out to the Head of Developer Relations at Google.

And he asked — and we were kind of trying to — I forget what the name of their technology was. And this guy asked a really perfect question. He said, “Do any of your engineers play with this after hours?”

And they said, “No.”

And he said, “You’re fucked. It’s not going to work.”

So that — that play, 

Ross: Yeah. Right?

Tim: Encourage and allow that play. Let people be curious. Let them find out. Let them invent. And let them reinvent your business.

Ross: That’s fantastic.

Tim: Because that’s — that will, that will — their learning will be your learning, and their reinvention of themselves will be your reinvention.

Ross: So, any final messages to everyone out there who is thick in the AI revolution?

Tim: I think it’s to try to forget the overheated financing environment.

You know, we talked at the very beginning about these various revolutions that I’ve seen. And the most interesting ones have always been when money was off the table.

It was like — everybody had kind of given up on search when Google came along, for example. It was just like, “This is a dead end.” And it wasn’t.

And open source — it was sort of like Microsoft was ruling the world and there was nothing left for developers to do. So they just went and worked on their own fun projects.

Right now, everybody’s going after the main chance. And — I mean, obviously not everybody — there are people who are going out and trying to really create value.

But there are too many companies — too many investors in particular — who are really trying to create financial instruments. Their model is just, “Value go up.” Versus a company that’s saying, “Yeah, we want value for our users to go up. We’re not even worried about that [financial outcome] right now.”

It’s so interesting — there was a story in The Information recently about Surge AI, which didn’t raise any money from investors, actually growing faster than Scale (scale.ai), which Meta just put all this money through — because they were just focused on getting the job done.

So I guess my point is: try to create value for others, and it will come to you if you do that.

Ross: Absolutely agree. That’s a wonderful message to end on.

So thank you so much for all of your work over the years and your leadership in helping us frame this AI as a positive boon for all of us.

Tim: Right. Well, thank you very much.

And it’s an amazing, fun time to be in the industry. We should all rejoice — challenging but fun.

The post Tim O’Reilly on AI native organizations, architectures of participation, creating value for users, and learning by exploring (AC Ep11) appeared first on Humans + AI.

“If we’re faced with problems that are moving fast and require collective solutions, then collective intelligence becomes the toolkit we need to tackle them.”

– Jacob Taylor

Jacob Taylor is a fellow in the Center for Sustainable Development at Brookings Institution, and a leader of its 17 Rooms initiative, which catalyzes global action for the Sustainable Development Goals. He was previously research fellow at the Asian Bureau of Economic Research and consulting scientist on a DARPA research program on team performance. He was a Rhodes scholar and represented Australia in Rugby 7s for a number of years.

Website:

www.brookings.edu

www.brookings.edu

www.brookings.edu

www.brookings.edu

loyalagents.org

LinkedIn Profile:

Jacob Taylor

X Profile:

Jacob Taylor

Reimagining Team Performance Through Collective Intelligence

Using 17 Rooms to Break Down the SDGs Into Action

Building Rituals That Elevate Learning and Challenge Norms

Designing Digital Twins to Represent Communities and Ecosystems

Creating Interspecies Money for Elephants, Trees, and Gorillas

Exploring Vibe Teaming for AI-Augmented Collaboration

Envisioning a Bottom-Up AI Ecosystem for People and Planet

Ross Dawson: Jacob, it is awesome to have you on the show.

Jacob Taylor: Ross, thanks for having me.

Ross: So we met at Human Tech Week in San Francisco, where you were sharing all sorts of interesting thoughts that we’ll come back to. What are your top-of-mind reflections of the event?

Jacob: Look, I had a great week, and largely because of all the great people I met, to be honest. And I think what I picked up there was people really driving towards the same set of shared outcomes.

Really people genuinely building things, talking about ways of working together that were driving at outcomes for, ultimately, for human flourishing, for people and planet. 

And I think that’s such an important conversation to have at the moment, as things are moving so fast in AI and technology, and sometimes it’s hard to figure out where all of this is leading, basically.And so to have humans at the center is a great principle. 

Ross: Yeah, well, where it’s leading is where we take it. So I think having the humans at the center is probably a pretty good starting point.

So one of the central themes of this blog—for this podcast for ages—has been collective intelligence. And so you are diving deep into applying collective intelligence to achieve the Sustainable Development Goals, and I would love to hear more about what you’re doing and how you’re going about it.

Jacob: Yeah, so I mean, very quickly, I’m an anthropologist by training. I have a background in elite team performance as a professional rugby player, and then studying professional team sport for a number of years.

So my original collective is the team, and that’s kind of my intuitive starting point for some of this. But teams are very well built to solve problems that no individual can achieve alone, and really a lot of the SDG problems that we have—issues that communities at every scale have trouble solving on their own—need a whole community to tackle a problem, rather than just one individual or set of individuals within a community.

So the SDGs are these types of—whether it’s climate action or ending extreme poverty or sustainability at the city level—all of these issues require collective solutions. And so if we’re faced with problems that are moving fast and require collective solutions, then collective intelligence becomes the toolkit or the approach that we need to use to tackle those problems.

I’ve been thinking a lot about this idea that in the second half of the 20th century, economics as a discipline went from pretty much on the margins of policymaking and influence to right at the center. By the end of the 20th century, economists were at the heart of informing how decisions were made at the country level, at firms, and so on. That was because an economic framework really helped make those decisions.

I think my sense is that the problems we face now really need the toolkit of the science of collective intelligence. So that’s kind of one of the ideas I’ve been exploring—is it time for collective intelligence as a science to really inform the way we make decisions at scale, particularly for our hardest problems like the SDG.

Ross: One of your initiatives—so at Brookings Institution, one of the initiatives is 17 Rooms. I’m so intrigued by the name and what that is and how that works.

Jacob: Yeah. So, 17 Rooms. We have 17 Sustainable Development Goals, and so on. Five or so years ago now—or more, I think it’s been running for seven or eight years now—17 Rooms thought: what if we found a method to break down that complexity of the SDGs?

A lot of people talk about the SDGs as everything connected to everything, which sometimes is true. There are a lot of interlinkages between these issues, of course. But what would it look like to actually break it down and say, let’s get into a room and tackle a slice of one SDG?

So Room 1: SDG 1 for ending extreme poverty. Let’s take on a challenge that we can handle as a team.

And so 17 Rooms gathers groups of experts into working groups—or short-term SWAT teams of cooperation, basically—and really gets them to think through big ideas and practical next steps for how to bend the curve on that specific SDG issue.

Then there’s an opportunity for these rooms or teams to interact across issues as well. So it provides a kind of “Team of Teams” platform for multi-stakeholder collaboration within SDG issues, but also connecting across the full surface of these problems as well.

Ross: So what from the science of collective intelligence—or anything else—what specific mechanisms or structures have you found useful? Are you trying to enable the collective intelligence within and across these rooms or teams?

Jacob: Yeah, so I think—I mean, they’re all quite basic principles. We do a lot on trying to curate teams and also trying to run them through a process that really facilitates collaboration. But the principles are quite basic, really.

I mean, one of the most fundamental principles is taking an action stance. One of the biggest principles of collective intelligence is that intelligence comes from action. This is a principle we get from biology. In biology, biology acts first and then learns on the run. So you don’t kind of sit there and go, what kind of action could we take together as a multicellular organism—rather, it just unfolds, and then learning comes off the back of that action.

So in that spirit, we really try to gear our teams and rooms into an action stance, and say, rather than just kind of pointing fingers at all the different aspects of the problem, let’s say: what would it look like for us in this room to act together? And then, what could we learn from that?

Trying to get into that stance is really foundational to the 17 Rooms initiative.

And then I think the other part is really bonding or community—so knowing that action and community are two sides of the same coin. When you act together, you connect and you share ideas and information. But likewise, communities of teams that are connected are probably more motivated to act together and to be creative and think beyond just incentives. But like, what can we really achieve together?

And so we try to pair those two principles together in everything that we do.

Ross: So this comes back to this point—there’s many classic frameworks and realities around acting and then learning from that. So your OODA Loop, your observe, orient, decide, act, or your Lean Startup loop, or Kolb’s learning cycle, or whatever it might be, where we act, but we only learn because we have data or insight.

So that’s a really interesting point—where we act, but then, particularly in a collective intelligence perspective, we have all sorts of data we need to filter and make sense of that not just individually, but collectively—in order to be able to understand how it is we change our actions to move more towards our outcomes.

Do you have any structures for being able to facilitate that flow of feedback or data into those action loops?

Jacob: Yeah, I think—and again, I’m very biased as an anthropologist here—so the third principle that we think about a lot, and that answers your question, is this idea of ritual.

We’re acting, we’re connecting around that action, and that’s a back-and-forth process. But then rituals actually are a space where we can elevate the best ideas that are coming out of that process and also challenge the ideas that aren’t serving us.

Famously across time for humans, ritual has been an opportunity both to proliferate the best behaviors of a society, but also to contest the behaviors that aren’t serving performance. Ultimately—you don’t always think about this in performance terms—but ultimately, when you look at it big picture, that’s what’s happening.

So I think rituals of differentiation between the data that are serving us versus not, I think is really important for any team, organization, or community.

Ross: That’s really interesting. Could you give an example of a ritual?

Jacob: Well, so there are rituals that can really—like walking on hot coals. Again, let’s start anthropological, and then maybe we can get back to collective intelligence or AI.

Walking on hot coals promotes behaviors of courageousness and devotion. Whereas in other settings, you have a lot of rituals that invert power structures—so men dressing up as women, women dressing up as men, or the less powerful in society being able to take on the behaviors of the powerful and vice versa.

That actually calls out some of the unhelpful power asymmetries in a society and challenges those.

So in that spirit, I think when we’re thinking about high-performing teams or communities tackling the SDGs, I think there needs to be more than just… I’m trying to think—how could we form a ritual de novo here?

But really, there needs to be, I guess, those behaviors of honesty and vulnerability as much as celebration of what’s working. That maybe is easier to imagine in an organization, for example, and how a leader or leaders may try to really be frank about the full set of behaviors and activities that a team is doing, and how that’s working for the group.

Ross: So you’ve written a very interesting article referring to Team Human and the design principles that support—including the use of AI—and being able to build better team performance. So what are some of the design principles?

Jacob: Well, I think this work came a little bit out of some DARPA work I did on a DARPA program before coming to Brookings around building mechanisms for collective intelligence. And when you boil it down to that fundamental level, it really comes down to having a way to communicate between agents or between individuals, which in psychology is referred to—the jargon in psychology is theory of mind.

So, do I have a theory of Ross—what you want—and do you have a theory of what I want? That’s basically social intelligence. It’s the basic key here.

But it really comes down to some way of communicating across differences. And then with that, the other key ingredient that we surfaced when we built a computational model of this, in a basic way, was an ability to align on shared goals.

So it feels like there’s some combination of social intelligence and shared goals that is foundational to any collective intelligence that emerges in teams or organizations or networks. And so trying to find ways to build those—whether that’s at the community level…

For example, if a city wants to develop its waste recycling program—but if you break that down, it really is a whole bunch of neighborhoods trying to develop recycling purposes. So the question for me is: do all those neighborhoods have a way of communicating to each other about what they’re doing in service of a shared goal of, let’s say, a completely circular recycling economy at the city level?

And if not, then what kind of interaction and conversations need to happen at the city level so that you can share best practices, challenge practices that are hurting everyone, and then find a way to drive collective action towards a shared outcome. But I’d also think about that, like, at the team level, where there are ways to really encourage theory of mind and perspective sharing.

Ross: So, in some of that work, you refer to digital twins—essentially being able to model how people might think or behave. If you are using digital twins, how is that put into practice in being able to build better team performance?

Jacob: Yeah, great. Yeah, that’s probably really where the AI piece comes in.

Because that recycling-at-the-city-level example that I shared—this kind of collective intelligence happens without AI.

But the promise of AI is to say, well, if you could actually store a lot of information in the form of digital twins that represented the interests and activities of, let’s say, neighborhoods in a city trying to do recycling—

Well, then beyond our human cognition, you could be trying to look for patterns and opportunities for collaboration by leveraging the power of AI to recognize patterns and opportunities across diverse data sets.

The idea is you could kind of try to supercharge the potential collective intelligence about problem-solving by positioning AI as a team support—or a digital twin that could say, hey, actually, if we tweak our dials here and use this approach, that could align with our neighbor’s approach, and maybe we should have a chat about it.

So there’s an opportunity to surface patterns, but then also potentially perform time-relevant interventions for human decision-makers to help encourage better outcomes.

Ross: I think you probably should try a different phrase, because “digital twin” sounds like you’ve got a person, then you’ve got a copy of that person.

Whereas you’re describing it here as representing—could be a neighborhood, or it could be a stakeholder group. So it’s essentially a representation, or some kind of representation, of the ways of thinking or values of a group, potentially, or community, as opposed to an individual.

Jacob: Indeed, yeah. I think this is where it all gets a bit technical, but yeah, I agree that “twin”—”digital twin”—evokes this idea of an individual body.

But if you extend that out, when you really take seriously some of the collective intelligence work, it’s like intelligence collectives become intelligent when they become a full thing, like a body—when they really individuate as a collective.

Teams really click and perform when they become one—so that it’s no longer just these individual bodies. It’s like the team is a body.

So I think in that spirit, when I think about this, I actually think about neighborhoods having a collective identity. That could be reflected in their twin, or like, of the community.

But I agree there’s maybe some better way to imagine what that kind of community AI companion looks like at higher scales.

Ross: So at Human Tech Week, you shared this wonderful story about how AI could represent not just human groups, but also animal species.

Love to—I think that sort of really brings it to—it gives it a very real context, because you’re understanding that from another frame.

Jacob: Yeah. And I think it’s true, Ross.

I’ve been struck by how much this example of interspecies money—that I’ll explain a little bit—is not only exciting because it has potential benefit for nature and the beautiful natural environment that we live in, but I think it actually helps humans understand what it could look like to do it for us too.

And so, interspecies money, basically, is this idea developed by a colleague of ours at Brookings, Jonathan Ledger. We had a room devoted to this last year in 17 Rooms to try and understand how to scale it up.

But what would it look like to give non-human species—like gorillas, or elephants, or trees—a digital ID and a bank account, and then use AI to reverse engineer or infer the preferences of those animals based on the way they behave?

And then give them the agency to use the money in their bank account to pay for services.

So if gorillas, for example, most rely on protection of their habitat, then they could pay local community actors to protect that habitat, to extend it, and to protect them from poachers, for example.

That could all be inferred through behavioral trace data and AI, but then also mediated by a trustee of gorillas—a human trustee.

It’s quite a futuristic idea, but it’s actually really hit the ground running. At the moment, there are pilots with gorillas in Rwanda, elephants in India, and ancient trees in Romania.

So it’s kind of—the future is now, a little bit, on this stuff.

I think what it really does is help you understand: if we really tried to position AI in a way that helps support our preferences and gives agency to those from the bottom up, then what?

What world would that look like?

And I think we could imagine the same world for ourselves. A lot of our AI systems at the moment are kind of built top-down, and we’re the users of those systems.

What if we were able to build them bottom-up, so that at every step we were representing individual, collective, community interests—and kind of trading on those interests bottom-up?

Ross: Yeah, well, there’s a lot of talk about AI alignment, but this is, like, a pretty deep level of alignment that we’re talking, right?

Jacob: Right.

And yeah, I think Sandy Pentland, who I shared the panel with—he has this idea of, okay, so there are large language models.

What would it look like to have local language models—small language models that were bounded at the individual.

So Ross, you had a local language model, which was the contents of your universe of interactions, and you could perform inferences using that.

And then you and I could create a one-plus-one-plus-one-equals-three kind of local language model, which was for some use case around collective intelligence.

This kind of bottom-up thinking, I think, is actually technically very feasible now.

We have the algorithms, the understanding of how to train these models. And we also have the compute—in devices like our mobile phones—to perform the inference.

It’s really just a question of imagination, and also getting the right incentives to start building these things bottom-up.

Ross: So one of the things you’ve written about is vibe teaming.

We’ve got vibe coding, we’ve got viable sorts of things. You and your colleagues created vibe teaming.

So what is it? What does it mean? And how do we do it?

Jacob: Good question.

Yeah, so this is some work that a colleague of mine, Kirsch and Krishna, and I at Brookings did this year.

We got to a point where, with our teamwork—you know, Brookings is a knowledge work organization, and we do a lot of that work in teams. A lot of the work we do is to try and build better knowledge products and strategies for the SDGs and these types of big global challenges.

The irony was, when we were thinking about how to build AI tools into our workflow, we were using a very old-school way of teaming to do that work.

We were using this kind of old industrial model of sequential back-and-forth workflows to think about AI—when AI was probably one of the most, potentially the most, disruptive technologies of the 21st century.

It just felt very ironic. To do a PowerPoint deck, Ross, you would give me the instructions. I would go away and draft it. I would take it back to you and say, “Is this right?” And you would say, “Yes, but not quite.”

So instead, we said, “Wait a minute. The internet is blowing up around vibe coding,” which is basically breaking down that sequential cycle.

Instead of individuals talking to a model with line-by-line syntax, they’re giving the model the vibe of what they want.

We’re using AI as this partner in surfacing what it is we’re actually trying to do in the first place.

So Kirsch and I said, “Why don’t we vibe team this?”

Why don’t we get together with some of these challenges and experts that we’re working with and actually get them to tell us the vibe of what they’ve been learning?

Homie Karas is a world expert—40-year expert—on ending extreme poverty. We sat down with him, and in 30 minutes, we really pushed him to give us, like:

“Tell us what you really think about this issue. What’s the really hard stuff that not enough people know about? Why isn’t it working already?”

These kinds of questions.

We used that 30-minute transcript as a first draft input to the model. And in 90 minutes, through interaction with AI—and some human at the end to make sure it all looked right and was accurate—we created a global strategy to end extreme poverty.

That was probably on par with anything that you see—and probably better, in fact, than many global actors whose main business is to end extreme poverty.

So it’s an interesting example of how AI can be a really powerful support to team-based knowledge work.

Ross: Yeah, so just—I mean, obviously, this is you.

You are—the whole nature of the vibe is that there’s no explicit, well, no specific, replicable structure. We’re going with the vibes.

But where can you see this going in terms of getting a group of complementary experts together, and what might that look like as the AI-augmented vibe teaming?

Jacob: Well, I mean, you’re right. There was a lot of vibe involved, and I think that’s part of the excitement for a lot of people using these new tools.

However, we did see a few steps that kept re-emerging. I’ve mentioned a few of them kind of implicitly here, but the big one—step one—was to really start with rich human-to-human input as a first step.

So giving the model a 30-minute transcript of human conversation versus sparse prompts was a real game changer for us working with these models.

It’s almost like, if you really set the bar high and rich, then the model will meet you there—if that makes sense.

Step two was quickly turning around a first draft product with the model.

Step three was then actually being patient and open to a conversation back and forth with the model.

So not thinking that this is just a one-button-done thing, but instead, this is a kind of conversation—interaction with the model.

“Okay, so that’s good there, but we need to change this.”
“Your voice is becoming a little bit too sycophantic. Can you be a bit more critical?”

Or whatever you need to do to engage with the model there.

And then, I think the final piece was really the need to go back and meet again together as a team to sense-check the outputs, and really run a rigorous human filter back over the outputs to make sure that this was not only accurate but analytically on point.

This idea that sometimes AI looks good but smells bad—and with these outputs, sometimes we’d find that it’s like, “Oh, that kind of looks good,” but then when you dig into it, it’s like, “Wait a minute. This wasn’t quite right here and there.”

So just making sure that it not only looks good but smells good too at the end.

Yeah. And so I think these basic principles—we’re seeing them work quite well in a knowledge work context.

And I guess for us now, we’re really interested in a two-barrel investigation with approaches like vibe teaming.

On the one hand, it’s really about the process and the how—like, how are we positioning these tools to support collaboration, creativity, flow in teamwork, and is that possible?

So it’s really a “how” question.

And then the other question for us is a full “what.” So what are we pointing these approaches at?

For example, we’re wondering—if it’s ending extreme poverty, how could we use vibe teaming to actually…

And Scott Page uses this term—how can we use it to expand the physics of collective intelligence?

How can we run multiple vibe teaming sessions all at once to be much more inclusive of the types of people who participate in policy strategy formation?

So that when you think about ending extreme poverty, it’s ending it for whom? What do they want? What does it look like in local communities, for example?

That idea of expanding the physics of collective intelligence through AI and approaches like vibe teaming is very much on our minds at the moment, as we think about next steps and scale-up.

Ross: Obviously, the name of the podcast is Humans Plus AI, and I think what you’re describing there is very much the best of humans—and using AI as a complement to draw out the best of that.

Nice segue—you just sort of referred to “where next steps.”

You’ve described a lot of the wonderful things you’re doing—some fantastic approaches to very, very critically important issues.

So where to from here? What’s the potential? What are the things we need to be doing? What’s the next phase of what you think could be possible and what we should be doing?

Jacob: Yeah, I think I’m really excited about this idea of growing an alternate AI ecosystem that works for people and planet, rather than the other way around.

Part of the work at Brookings is really setting up that agenda—that research agenda—for what that ecosystem could look like.

We discussed it a little bit together at Human Tech Week.

I think of that in three parts.

There’s the technical foundation—so down to the algorithms and the architectures of AI models—and thinking about how to design and build those in a way that works for people.

That includes, for example, social intelligence built into the code.

Another example there is around, in a world of AI agents—are agents working for humans, or are they working for companies?

Sandy Pentland’s work on loyal agents, for example—which maybe we could link to afterward—I think is a great example of how to design agents that are fiduciaries for humans, and actors for humans first, and then others later.

Then, approaches like vibe teaming—ways of bringing communities together using AI as an amplifier.

And then I think the key piece, for me, is how to stitch the community of actors together around these efforts.

So the tech builders, the entrepreneurs, the investors, the policymakers—how to bring them together around a common format.

That’s where I’m thinking about a few ideas.

One way to try to get people excited about it might be this idea of not just talking about it in policy terms or going around to conferences.

But what would it look like to actually bring together a lab or some kind of frontier research and experimentation effort—where people could come together and build the shared assets, protocols, and infrastructures that we need to scale up great things like interspecies money, or vibe teaming, or other approaches?

Where, if we had collective intelligence as a kind of scientific backbone to these efforts, we could build an evidence base and let the evidence base inform new approaches—trying to get that flywheel going in a rigorous way.

Trying to be as inclusive as possible—working on everything from mental health and human flourishing through to population-level collective intelligence and everything in between.

Ross: So can you paint that vision just a little bit more precisely?

What would that look like, or what might it look like?

What’s one possible manifestation of it? What’s the—

Jacob: Yeah, I mean, it’s a good question.

So this idea of a frontier experimental lab—I think maybe I’m a little bit informed by my work at DARPA.

I worked on a DARPA program called ASSIST—AI, I mean, Artificial Social Intelligence for Successful Teams—and that really used this kind of team science approach, where you had 12 different scientific labs all coming together for a moonshot-type effort.

There was that kind of idea of, we don’t really know how to work together exactly, but we’re going to figure it out.

And in the process of shooting for the moon, we’re hopefully going to build all these shared assets and knowledge around how to do this type of work better.

So I guess, in my mind, it’s kind of like: could we create a moonshot for collective intelligence, where collective intelligence is really the engine—and the goal was trying to, for example, end extreme poverty, or reach some scale of ecosystem conservation globally through interspecies money?

Or—pick your SDG issue.

Could we do a collective intelligence moonshot for that issue?

And in that process, what could we build together in terms of shared assets and infrastructure that would last beyond that one moonshot, and equip us with the ingredients we need to do other moonshots?

Ross: Yeah, well, again, going back to the feedback loops—of what you learn from the action in order to be able to inform and improve your actions beyond that.

Jacob: Exactly, yeah.

And I think the key ingredients here are really taking seriously what we’ve built now in terms of collective intelligence. It is a really powerful, transdisciplinary scientific infrastructure.

And I think taking that really seriously, and drawing on the collective intelligence of that community to inform, to create evidence and theories that can inform applications. And then running that around.

I think what I discovered at Human Tech Week with you, Ross, is this idea that there’s a lot of entrepreneurial energy—and also capital as well.

I think a lot of investors really want to put their money where their mouths are on these issues.

So it feels like it’s not just kind of an academic project anymore. It’s really something that could go beyond that.

So that’s kind of time for collective intelligence. We need to get these communities and constituencies working together and build a federation of folks who are all interested in a similar outcome.

Ross: Yeah, yeah. The potential is extraordinary.

And so, you know, there’s a lot going on—not all of it good—these days, but there’s a lot of potential for us to work together.

And again, there’s amplifying positive intent, which is part of what I was sharing at Human Tech Week.

I was saying, what is our intention? How can we amplify that positive intention, which is obviously what you are doing in spades.

So how can people find out more about your work and everything which you’ve been talking about?

Jacob: Well, most of my work is on my expert page on Brookings.

I’m here at the Center for Sustainable Development at Brookings, and I hope I’ll be putting out more ideas on these topics in the coming months.

I’ll be mainly on LinkedIn, sharing those around too.

Ross: Fantastic. Love what you’re doing.

Yeah—and yeah, it’s fun. It’s fantastic. So really, really glad you’re doing that.

Thank you for sharing, and hopefully there’s some inspiration in there for some of our listeners to follow similar paths.

Jacob: Thanks, Ross. I appreciate your time. This has been fun.

The post Jacob Taylor on collective intelligence for SDGs, interspecies money, vibe-teaming, and AI ecosystems for people and planet (AC Ep10) appeared first on Humans + AI.

“Strategy really must focus on those purely human capabilities of synthesis, and judgment, and sense-making.”

– Ross Dawson

Ross Dawson is a futurist, keynote speaker, strategy advisor, author, and host of Amplifying Cognition podcast. He is Chairman of the Advanced Human Technologies group of companies and Founder of Humans + AI startup Informivity. He has delivered keynote speeches and strategy workshops in 33 countries and is the bestselling author of 5 books, most recently Thriving on Overload.

Website:

Ross Dawson

Advanced Human Technologies

LinkedIn Profile:

Ross Dawson

Books

Thriving on Overload

Living Networks 20th Anniversary Edition

Implementing Enterprise 2.0

Developing Knowledge-Based Client Relationships

Ross Dawson: This is a little bit of a different episode. Instead of an interview, I will be sharing a few thoughts in the context of now doubling down on the Humans Plus AI theme. Our community is kicking off the next level. As you may have noticed, the podcast has been rebranded Humans Plus AI, and really just fully focused on this theme of how AI can augment humans—individually, organizations, and society.

So what I want to share today is some of the thoughts which came out of Human Tech Week. I was fortunate to be at Human Tech Week in San Francisco a few weeks ago. I did the opening keynote on Infinite Potential: Humans Plus AI, and I’ll share some more thoughts on that another time.

But what I also did was run a lunch event, a panel with myself, John Hagel, and Charlene Lee, talking about AI and the future of strategy. So it was an amazing conversation, and I can’t do it justice now, but what I want to do is just share some of the high-level themes that came out of that conversation, and I suppose, obviously, bringing my own particular slant on those.

So we started off by thinking around how is change generally, including AI, impacting strategy and the strategy process. So fairly obviously we have accelerating change. That means that decision cycles are getting shorter, and strategy needs to move faster.

It also means that there is the ability for creation of all kinds to be democratized within, across, and beyond organizations, allowing them to innovate, to act without necessarily being centralized. And this idea of this abundance of knowledge, coupled with the scarcity of insight, means that strategy really must focus on those purely human capabilities of synthesis, and judgment, and sense-making.

There’s also a theme where we have institutional trust is eroding. So this means that more and more, strategy shifts to relationships-based models, ecosystem-based models.

And this overlying theme, which John Hagel in particular brought out, is this idea that there is greater fear amongst leaders. There’s greater emotional pressure, and these basically shrink the timeline of our thinking. It forces us to shorter-term thinking. We are based on fear—of a whole variety of pressures from shareholders, stakeholders, politicians, and more.

We need to allow ourselves to move beyond the fear, as John’s latest book The Journey Beyond Fear lays out—highly recommended—which then enables us to enable our strategic imagination and ways of thinking, and how we do that.

So one of the core themes of the conversation was around: what are the relative roles of AI and humans in the strategy process? Humans are strategic thinkers by their very nature, and now we have AI which can support us and complement us in various ways.

Of course, there is a strong way in which AI can use data. It can do a lot of analysis. It is very capable at pattern recognition. It can move faster. It can simulate scenarios and futures, identify signals, and so it can scale what can be done in strategy analysis. It can go deeper into the analysis.

But this brings the human role of the higher levels: of the creativity, of the imagination, of the judgment, the ethical framing, the purpose, the vision, the values.

One of the key things which came out of it was around storytelling, where strategy is a story. It’s not this whole array of KPIs and routes to get them—that’s a little part of it. It is telling a story that engages people, that makes them passionate about what they want to do and how they are going to do it—that’s their heroes and heroines’ journey.

So this insight, this sense-making, is still human.

There’s a wonderful quote from the session, saying, “AI without data is extremely stupid,” but even with the data, it can’t deliver the insight or the wisdom on its own. That is something where the human function resides.

And so we are still responsible for the oversight and for the ethical nature of the decisions. Especially as we have more and more autonomous agents, we have very opaque systems. And accountability is fundamental to all leadership and to the nature of strategy.

So a leader’s role is to be able to bring together those ways in which we bring in AI, deciding when to trust it, deciding when to override, and how to frame its contribution for leaders. So that’s an intrinsic part of strategy: the role of AI in the way the what, how the organization functions, and how the organization establishes and communicates direction.

Well, there was a lot of discussion around the tensions. And again, John shared this wonderful frame he’s been using for a while about “zoom out and zoom in.” Essentially, he says that real leaders—the most successful organizations—they have a compelling 10- or 20-year vision, and they also have plans for the next six to twelve months, and they don’t have much in between.

And so you can zoom out to sort of see this massive scale of: Why do we exist? What are we trying to create? But also looking, shrinking down to saying, All right, well, what is it we’re doing right now—creating momentum and moving towards that.

And so this dual framing is emotionally resonant. It shifts people from fear to hope by being able to see this aspiration and also seeing progress today.

And so there are these polarities that we manage in strategy. We’re balancing focus with flexibility. We need to be clearly guided in where we are going. So we need this coherence. We need to be able to know what we are doing, but we also need to be able to focus our resources.

And so this balance between flexibility—where we can adapt to situations—while maintaining continuity in moving forward, is fundamental.

One of the fundamental themes, which, again, which came out of the conversation, which comes back to some of my core themes from a very long time, is this idea of knowledge and trust.

So AI is widely accessible. Everyone’s got it in various guises. So where does competitive advantage reside? And fundamentally, it is from trust. And it is trust that in the AI. Distrust about how the AI is used is around trust in the intentions. It’s around trust in, ultimately, the people that have shaped the systems and used the systems well.

So this means that as you create long-term, trust-based relationships, you get more and more advantages. And this comes back to my first book on Knowledge-Based Client Relationships, which I’ve extended and applied in quite a variety of domains, including in my recent work on AI-driven business model innovation.

We’re essentially saying that in an AI-driven world, that trust in the systems means you can have access to more data and more insight from people and organizations, which you can apply in building this virtual virtuous circle of differentiation. So you add value, you gain trust, you get insight from that, flowing through into more value.

So ultimately, this is about passion. What John calls the passion of the explorer, where we are committed to learning and questioning and creating value.

So I suppose that, in a way, the key theme that ran through the entire conversation was around learning, and where learning is not something about, how do we do these workshops, and how do we—

Take these bodies of knowledge and get everybody to know them. It is about this continuous exploration of the new. And every successful organization needs to harness and to enable people inside those organizations to be passionate about what they are learning, to explore, to learn from their exploration, to share that, and so building this sustainable learning and scalable learning, which is the nature of a fast-moving world,

Where we can have some consistent strategy based around that learning, which enables us to continue to both have direction and be flexible and adaptable in an accelerating world.

So that just touches on some of the themes which we discussed in the session, and I will continue to share, write some more—what I call mini reports—just to frame some of these ideas.

But the reality is that the nature of strategy is changing. This means the nature of leadership is changing, and we need to understand and to dig into the nature of the changing nature of strategy—where AI plays a role, how that shifts human roles, how leadership changes.

Because these are fundamental to our success, not just as individual organizations, but also as industries and society at large. Because our strategies, of course, must support not just individual entities or organizations, but the entire ecosystems and communities and societies in which they are embedded.

So we’ll come back. We’ve got some amazing guests coming up in our next episode, so make sure to tune in for the next episodes.

Please continue to engage. Get onto Humans Plus AI, sign up for our newsletter, and we’ll see you on the journey.

The post AI & The Future of Strategy (AC Ep9) appeared first on Humans + AI.

“The big picture is that every human on Earth deserves to live a life worth living… free of mental strife, physical strife, and the strife of war.”

– Matt Lewis

Matt is CEO, Founder and Chief Augmented Intelligence Officer of LLMental, a Public Benefit Limited Liability Corporation Venture Studio focused on augmenting brain capital. He was previously Chief AI Officer at Inizio Health, and contributes in many roles including as a member of OpenAI’s Executive Forum, Gartner’s Peer Select AI Community and faculty at the World Economic Forum’ New Champions’ initiative.

Website:

Matt Lewis

LinkedIn Profile:

Matt Lewis

Using AI to support brain health and mental well-being

Redefining mental health with lived experience leadership

The promise and danger of generative AI in loneliness

Bridging neuroscience and precision medicine

Citizen data science and the future of care

Unlocking human potential through brain capital

Shifting from scarcity mindset to abundance thinking

Ross Dawson: Matt, it’s awesome to have you on the show.

Matt Lewis: Thank you so much for having me. Ross, it’s a real pleasure and honor. And thank you to everyone that’s watching, listening, learning. I’m so happy to be here with all of you.

Ross: So you are focusing on using AI amongst other technologies to increase brain capital. So what does that mean?

Matt: Yeah. I mean, it’s a great question, and it’s, I think, the challenge of our time, perhaps our generation, if you will.

I’ve been in artificial intelligence for 18 years, which is like an eon in the current environment, if you will. I built my first machine learning model about 18 years ago for Parkinson’s disease, under a degenerative condition where people lose the ability to control their body as they wish they would.

I was working at Boehringer Ingelheim at the time, and we had a drug, a dopamine agonist, to help people regain function, if you will. But some small number of people developed this weird side effect, this adverse event that didn’t appear in clinical trials, where they became addicted to all sorts of compulsive behaviors that made their actual lives miserable. Like they became shopping addicts, or they became compulsive gamblers. They developed proclivities to sexual behaviors that they didn’t have before they were on our drug, and no one could quite figure out why they had these weird things happening to them.

And even though they were seeing the top academic neurologists in this country, United States, or other countries, no one can say why Ross would get this adverse event and Matt wouldn’t. It didn’t appear in the studies, and there’s no way to kind of figure it out.

The only thing that kind of really sussed out what was an adverse event versus what wasn’t was advanced statistical regression and later machine learning. But back in the days, almost 20 years ago, you needed massive compute, massive servers—like on trucks—to be able to ship these types of considerations to actually improve clinical outcomes.

Now, thankfully, the ability to provide practical innovation in the form of AI to help improve people’s actual lives through brain health is much more accessible, democratisable, almost in a way that wasn’t available then.

And if it first appeared for motor symptoms, for neurodegenerative disease, some time ago, now we can use AI to help not just the neurodegenerative side of the spectrum but also neuropsychiatric illness, mental illness, to help identify people that are at risk for cognition challenges.

Here in Manhattan, it’s like 97 degrees today. People don’t think the way they normally do when it’s 75. They make decisions that they perhaps wish they hadn’t, and a lot of the globe is facing similar challenges.

So if we can kind of partner with AI to make better decisions, everyone’s better off.

That construct—where we think differently, we make better decisions, we are mentally well, and we use our brains the way that was intended—all those things together are brain capital. And by doing that broadly, consistently, we’re better off as a society.

Ross: Fantastic. So that case, you’re looking at machine learning—so essentially being able to pull out patterns. Patterns between environmental factors, drugs used, background, other genetic data, and so on.

So this means that you can—is this, then, alluding, I suppose, to precision medicine and being able to identify for individuals what the right pharmaceutical regimes are, and so on?

Matt: Yeah. I mean, I think the idea of precision medicine, personalized medicine, is very appealing. I think it’s very early, maybe even embryonic, kind of consideration in the neuroscience space.

I worked for a long time for companies like Roche and Genentech, others in that ecosystem, doing personalized medicine with biomarkers for oncology, for cancer care—where you knew a specific target, an enzyme, a protein that was mutated and there was a degradation, and identified which enzyme was a bit remiss.

Then tried to build a companion diagnostic to find the signal, if you will, and then help people that were suffering.

It’s a little bit more—almost at risk of saying—straightforward in that regard, because if someone had the patient, you knew that the drug would work.

Unfortunately, I think there’s a common kind of misconception—I know you know this exceptionally well, but there are people out there, perhaps listening, that don’t know it as well—that the state of cognitive neuroscience, that is what we know of the brain or how the brain works and how it works in the actual world in which we live, on planet Earth and terra firma, is probably about as far advanced as the state of the heart was when Jesus Christ walked the Earth about 2,000 years ago.

That is, we probably have about 100 years of knowledge truly about how the brain truly works in the world, and we’re making decisions about how to engineer personalized medicine for a very, very, very young, nascent science called the brain—with almost no real kind of true, practical, contextual understanding of how it really works in the world.

So I think personalized medicine has tremendous possible promises. The reality of it doesn’t really pan out so well.

There are a couple of recent examples of this from companies like Nomura, Alto Neuroscience, and the rest, where they try to build these kind of ex post facto precision medicine databases of people that have benefited from certain psychiatric medicines.

But they end up not being as beneficial as you’d like them to be, because we just don’t know really a lot about how the brain actually works in the real world.

There even still is the debate for people—but even if you extend past the brain and mind debate—I think it’s hard to find the number of people that are building in the space that will recognize contextual variables beyond the brain and mind.

Including things like the biopsychosocial continuum, the understanding of spirituality and nature, all the rest.

All these things are kind of moving and changing and dynamic at a constant equilibrium.

And to try to find a point solution that says Matt or Ross are going to be beneficial at this one juncture, and they’re going to change it right now—it’s just exceptionally difficult. Important, but exceptionally difficult.

So I think the focus is more about how do we show up in the real world today, using AI to actually help our actual life be meaningful and beneficial, rather than trying to find this holy grail solution that’s going to be personalized to each person in 2026.

I’m not very optimistic about that, but maybe by 2036 we’ll get a little closer.

Ross: Yeah. So, I mean, I guess, as you say, a lot of what people talk about with precision medicine is specific biomarkers and so on, that you can use to understand when particular drugs would be relevant.

But back to the point where you’re starting with this idea of using machine learning to pick up patterns—does this mean you can perhaps be far more comprehensive in seeing the whole person in their context, environment, background, and behaviors, and so on, to be able to understand what interventions will make sense for that individual, and all of the whole array of patterns that the person manifests?

Matt: Yeah, I think it’s a great question. I think the data science and the kind of health science of understanding, again, kind of what might be called the inactive psychiatry of the person—how they make meaning in the world—is just now starting to catch up with reality.

When I did my master’s thesis 21 years ago in health services research, there were people trying to figure out: if you were working in the world, how do we understand when you’re suffering with a particular illness, what it means to you?

It might mean to the policy wonks that your productivity loss is X, or your quality-adjusted life years is minus Y. Or to your employer, that you can’t function as much as you used to function. But to you—does it really matter to you that your symptom burden is A or Z? Or does it really matter to you that you can’t sleep at night?

If you can’t sleep at night, for most people, that’s really annoying. And if you can’t sleep at night six, seven, ten nights in a row, it’s catastrophic because you almost can’t function. Whereas on the quality score, it doesn’t even register—it’s like a rounding error.

So the difference between the patient-reported outcomes for what matters for real people and what it matters to the decision-makers—there’s a lot of daylight between those things, and there has been for a long time.

In the neuropsychiatric, mental health, brain health space, it’s starting to catch up, for I think a couple of reasons.

One, the lived experience movement. I chair the One Mind Community Advisory Network here in the States, which is a group of about 40 lived experience experts with deep subject matter expertise, all of whom suffer from neuropsychiatric illness, neurodivergence, and the rest. These are people that suffer daily but have turned their pain into purpose.

The industry at large has seen that in order to build solutions for people suffering from different conditions, you need to co-create with those people. I mean, this seems intuitive to me, but for many years—for almost all the years, 100 years—most solutions were designed by engineers, designed by scientists, designed by clinicians, without patients at the table.

When you build something for someone without the person there, you get really pretty apps and software and drugs that often don’t work. Now, having the people actually represented at the table, you get much better solutions that hopefully actually have both efficacy in the lab and effectiveness in the real world.

The other big thing I think that’s changing a lot is that people have more of a “citizen data scientist” kind of approach. Because we’re used to things like our Apple Watch, and our iPads, and our iPhones, and we’re just in the world with data being in front of us all the time, there’s more sensitivity, specificity, and demand for visibility around data in our life.

This didn’t exist 20 years ago.

So just to be in an environment where your mental health, your brain health, is being handed to you on a delivery, if you will—and not to get some kind of feedback on how well it’s working—20 years ago, people were like, “Okay, yeah, that makes sense. I’m taking an Excedrin for my migraine. If it doesn’t work, I’m clearing to take a different medicine.”

But now, if you get something and you don’t get feedback on how well it’s working, the person or organization supporting it isn’t doing their job.

There’s more of an imprimatur, if you will, of expectation on juxtaposing that data analytics discipline, so that people understand whether they’re making progress, what good looks like, are they benchmarking against some kind of expectation—and then, what the leaderboard looks like.

How is Ross doing, versus how Matt’s doing, versus what the gold standard looks like, and all the rest. This didn’t exist a generation ago, but now there’s more to it.

Ross: That’s really interesting. This rise of citizen science is not just giving us data, but it’s also the attitude of people—that this is a normal thing to do: to participate, to get data about themselves, to share that back, to have context.

That’s actually a really strong positive feedback loop to be able to develop better things.

So I think, as well as this idea of simply just getting the patients at the table—so we’ve talked quite a bit, I suppose, from this context of machine learning—of course, generative AI has come along.

So, first of all, just a big picture: what are the opportunities from generative AI for assisting mental well-being?

Matt: Yeah. I mean, first of all, I am definitely a technophile. But that notwithstanding, I will say that no technology is either all good or all bad. I think it’s in the eyes of the beholder—or the wielder, if you will.

I’ve seen some horrific use cases of generative AI that really put a fear into my heart. But I’ve also seen some amazing implementations that people have used that give me a tremendous amount of hope about the near and far future in brain health broadly, and in mental health specifically.

Just one practical example: in the United States and a lot of the English-speaking countries—the UK, New Zealand, and Australia—there is a loneliness epidemic.

When I say loneliness, I don’t mean people that are alone, that either choose to be alone or live lives that are alone. I actually mean people that have a lower quality of life and are lonely, and as a result, they die earlier and they have more comorbid illness. It’s a problem that needs to be solved.

In these cases, there are a number of either point solutions that are designed specifically using generative AI or just purpose-built generative AI applications that can act both as a companion and as a thought partner for people who are challenged in their contextual environment.

They act in ways where they don’t have other access or resources, and in those times of need, AI can get them to a place where they either catalyze consideration to get back into an environment that they recall being useful at an earlier point.

For example, they find an interest in something that they found utility in earlier—like playing chess, or playing a card game, a strategy game, or getting back to dancing or some other “silly” thing that to them isn’t silly, but might be silly to a listener.

And because they rekindle this interest, they go and find an in-person way of reigniting with a community in the environment. The generative AI platform or application catalyzes that connection.

There are a number of examples like that, and the AI utility case is nearly free. The use of it is zero cost for the person, but it prevents them from slipping down the slippery slope of an actual DSM-5 psychiatric illness—like depression or anxiety—and becoming much, much worse.

They’re kind of rescued by AI, if you will, and they become closer to healthy and well because they either find a temporary pro-social kind of companion or they actually socialize and interact with other humans.

I have seen some kind of scary use cases recently where people who are also isolated—I won’t use the word lonely—don’t have proper access to clinicians.

In many places around the world, there is a significant shortage of licensed professionals trained in mental health and mental illness. In many of these cases, when people don’t have a diagnosed illness or they have a latent personality disorder, they have other challenges coming to the fore and they rely on generative AI for directional implementation.

They do something as opposed to think something, and it can rapidly spiral out of control—especially when people are using GPTs or purpose-built models that reinforce vicious cycles or feedback loops that are negatively reinforcing.

I’ve seen some examples, due to some of the work I do in the lived experience community, where people have these built-in cognitive biases around certain tendencies, and they’ll build a GPT that reinforces those tendencies.

What starts out as a harmless comment from someone in their network—like a boyfriend, employee, or neighbor—suddenly becomes the millionth example of something that’s terrible. The GPT reinforces that belief.

All of a sudden, this person is isolated from the world because they’ve cut off relationships with everyone in their entire circle—not because they really believe those things, but because their GPT has counseled them that they should do these things.

They don’t have anyone else to talk to, and they believe they should do them, and they actually carry those things out. I’ve seen a couple of examples like this that are truly terrifying.

We do some work in the not-for-profit space trying to provide safe harbors and appropriate places for care—where people have considerations of self-harm, where a platform might indicate that someone is at risk of suicide or other considerations.

We try to provide a place where people can go to say, “Is this really what you’re thinking?” If so, there’s a number to call—988—or someone you can reach out to as a clinician.

But I think, like all technologies: you can use a car to drive to the grocery store. You could also use the same car to run someone over.

We have to really think about: what in the technology is innate to the user, and what it was really meant to do?

Ross: Yeah. Well, it’s a fraught topic now, as in there are, as you say, some really negative cases. The commercial models, with their tendency toward sycophancy and encouraging people to continue using them, start to get into all these negative spirals.

We can and have, of course, some clinically designed tools—generative AI tools to assist—but not everybody uses those. One of the other factors, of course, is that not everybody even has the finances, or the finance isn’t available to provide clinicians for everybody. So it’s a bit fraught.

I go back to 15 years ago, I guess—Paro, the robot seal in Japan—which was a very cute, cuddly robot given to people with neurodegenerative diseases. They came out of their shell, often. They started to interact more with other people just through this little robot.

But as you say, there is the potential then for these not to be substitutes. Many people rail against, “Oh, we can’t substitute real human connection with AI,” and that’s obviously what we want.

But it can actually help re-engage people with human connection—in the best circumstances.

Matt: Yeah. I mean, listen, if I was doing this discussion with almost any other human on planet Earth, Ross, I would probably take that bait and we could progress it.

But I’m not going to pick that up with you, because no one knows this topic—of what humans can, should, and will potentially do in the future—better than you, than any other human. So I’m not going to take that.

But let me comment one little thing on the mental health side. The other thing that I think people often overlook is that, in addition to being a tool, generative AI is also a transformative force.

The best analogy I have comes from a friend of mine, Connor Brennan, who’s one of the top AI experts globally. He’s the Chief AI Architect at NYU here in New York City.

He says that AI is like electricity in this regard: you can electrify things, you can build an electrical grid, but it’s also a catalyst for major advances in the economy and helps power forward the industry at large.

I think generative AI is exactly like that. There are point solutions built off generative AI, but also—especially in scientific research and in the fields of neurotechnology, neuroscience, cognition, and psychology—the advances in the field have progressed more in the last three years post–generative AI, post–ChatGPT, than in the previous 30 years.

And what’s coming—and I’ve seen this in National Academy of Medicine presentations, NIH, UK ARIA, and other forums—what’s coming in the next couple of years will leapfrog even that.

It’s for a couple of reasons. I’m sure you’re familiar with this saying: back in the early 2000s, there was a saying in the data science community, “The best type of machine learning is no machine learning.”

That phrase referred to the fact that it was so expensive to build a machine learning model, and it worked so infrequently, that it was almost never recommended. It was a fool’s errand to build the thing, because it was so expensive and worked so rarely.

When I used to present at conferences on the models we would build, people always asked the same questions: What was the drift? How resilient was the model? How did we productionize it? How was it actually going to work?

And it was—frankly—kind of annoying, because I didn’t know if it was going to work myself. We were just kind of hoping that it would.

Now, over the last couple of years, no one asks those questions. Now people ask questions like: “Are robots going to take my job?” “How am I going to pay my mortgage?” “Are we going to be in the bread lines in three years?” “Are there going to be mass riots?”

That’s what people ask about now. The conversation has shifted over the last five years from “Will it work?” to “It works too well. What does it mean for me—for my human self?”

“How am I going to be relevant in the future?”

I think the reason why that is, is because it went from being kind of a tactical tool to being a transformative force.

In the scientific research community, what’s really accelerating is our ability to make sense of a number of data points that, up until very recently, people saw as unrelated—but that are actually integrated, part of the same pattern.

This is leading to major advances in fields that, up until recently, could not have been achieved.

One of those is in neuroelectronics. I’m very excited by some of the advances in neurotechnology, for example—and we have an equity interest in a firm in this space.

Implantable brain considerations is one major place where mental illness can advance. AI is both helping to decipher the language of communication from a neuroplasticity standpoint, and making it possible for researchers and clinicians to communicate with the implant in your brain when you’re not in the clinic.

So, if you go about your regular life—you go to work, you play baseball, you do anything during your day—you can go about your life, and because of AI, it makes monitoring the implant in your brain no different than having a continuous glucose monitor or taking a pill.

The advances in AI are tremendous—not just for using ChatGPT to write a job description—but for allowing things like bioelectronic medicine to exist and be in the clinic in four or five years from now.

Whereas, 40 years ago, it would have been considered magic to do things like that.

Ross: So, we pull this back, and I’d like to come back to where we started. Before we started recording, we were chatting about the big picture of brain capital.

So I just want to think about this idea of brain capital. What are the dimensions to that? And what are the ways in which we can increase it? What are the potential positive impacts? What is the big picture around this idea of brain capital?

Matt: Yeah. I mean, the big picture is that every human on Earth deserves to live a life worth living. It’s really that simple. Every person on planet Earth deserves to have a life that they enjoy, that they find to be meaningful and happy, and that they can live their purpose—every person, regardless of who they’re born to, their religion, their race, their creed, their region.

And they should be free of strife—mental strife, physical strife, and the strife of war. For some reason, we can’t seem to get out of these cycles over the last 100,000 years.

The thesis of brain capital is that the major reason why that’s been the case is that a sixth of the world’s population currently has mental illness—diagnosed or undiagnosed. About a quarter of the world’s population is living under what the World Health Organization calls a “brain haze” or “brain fog.”

We have a kind of collective sense of cognitive impairment, where we know what we should do, but we don’t do it—either because we don’t think it’s right, or there are cultural norms that limit our ability to actually progress forward.

And then the balance of people are still living with a kind of caveman mindset. We came out of the caves 40,000–60,000 years ago, and now we have iPhones and generative AI, but our emotions are still shaped by this feeling of scarcity—this deficit mindset, where it feels like we’re never going to have the next meal, we’re never going to have enough resources.

It’s like less is more all the time.

But actually, right around the corner is a mindset of abundance. And if you operate with an abundance mindset, and believe—as Einstein said—that everything is a miracle, the world starts responding appropriately.

But if you act like nothing is a miracle, and that it’s never going to be enough, that’s the world through your eyes.

So the brain capital thesis is: everyone is mentally well, everyone is doing what’s in the best collective interest of society, and everyone is able to see the world as a world of abundance—and therefore, a life worth living.

Ross: That is awesome. No, that’s really, really well put. So, how do we do it? What are the steps we need to take to move towards that?

Matt: Yeah. I mean, I think we’re already walking the path. I think there are communities—like the ones that we’ve been together on, Ross—and others that are coming together to try to identify the ways of working, and putting resources and energy and attention to some of these challenges.

Some of these things are kind of old ideas in new titles, if you will. And there are a number of trajectories and considerations that are progressing under new forms as well.

I think one of the biggest things is that we really need both courage to try new ways of working, and also—to use a Napoleon expression—Napoleon said that a leader’s job is to be a dealer in hope.

We really need to give people the courage to see that the future is brighter than the past, and that nothing is impossible.

So our considerations in the brain capital standpoint are that we need to set these moonshot goals that are realistic—achievable if we put resources in the right place.

I’ve heard folks from the World Economic Forum, World Health Organization, and others say things like: by this time next decade—by the mid-2030s—we need to cure global mental illness completely. No mental illness for anyone.

By 2037–2038, we need to prevent brain health disorders like Alzheimer’s, Parkinson’s, dystonia, essential tremor, epilepsy, etc.

And people say things like, “That’s not possible,” but when you think about other major chronic illnesses—like Hepatitis C or breast cancer—when I was a kid, either of those things were death sentences. Now, they’re chronic illnesses or they don’t exist at all.

So we can do them. But we have to choose to do them, and start putting resources against solving these problems, instead of just saying, “It can’t be done.”

Ross: Yeah, absolutely. So, you’ve got a venture in this space. I’d love to round out by hearing about what you are doing—with you and your colleagues.

Matt: So, we’re not building anything—we’re helping others build. And that’s kind of a lesson learned from experience.

To use another quote that I love—it’s a Gandhi quote—which is, “I never lose. I only win or I learn.”

So we tried our hand at digital mental health for a time, and found that we were better advisors and consultants and mentors and coaches than we were direct builders ourselves.

But we have a firm. It’s the first AI-native venture studio for brain capital, and we work with visionary entrepreneurs, CEOs, startups—really those that are building brain capital firms.

So think: mental illness, mental health, brain health, executive function, mindset, corporate learning, corporate training—that type of thing. Where they have breakthrough ideas, they have funding, but they need consideration to kind of help scale to the ecosystem.

We wrap around them like a halo and help support their consideration in the broader marketplace.

We’re really focused on these three things: mental health, mindset, and mental skills.

There are 12 of us in the firm. We also do a fair amount of public speaking—workshops, customer conferences, hackathons. The conference we were just at last week in San Francisco was part of our work.

And then we advise some other groups, like not-for-profits and the government.

Ross: Fantastic. So, what do you hope to see happen in the next five to ten years in this space?

Matt: Yeah, I’m really optimistic, honestly. I know it’s a very tumultuous time externally, and a lot of people are suffering. I try to give back as much as possible.

We, as an organization, we’re a public benefit corporation, so we give 10% of all our revenue to charity. And I volunteer at least a day a month directly in the community. I do know that a lot of people are having a very difficult time at present.

I do feel very optimistic about our mid- and long-term future. I think we’re in a very difficult transition period right now because of AI, the global economic environment, and the rest. But I’m hopeful that come the early 2030s, human potential broadly will be optimized, and many fewer people on this planet will be suffering than are suffering at present.

And hopefully by this time next decade, we’ll be multi-planetary, and we’ll be starting to focus our resources on things that matter.

I remember there was a quote I read maybe six or seven years ago—something like: “The best minds of our generation are trying to get people to click on ads on Facebook.” When you think about what people were doing 60 years ago—we were building the space shuttle to the moon.

The same types of people that would get people to click on ads on Meta are now trying to get people to like things on LinkedIn. It’s just not a good use of resources.

I’ve seen similar commentary from the Israeli Defense Forces. They talk about all the useless lives wasted on wars and terrorism. You could think about not fighting these battles and start thinking about other ways of helping humanity.

There’s so much progress and potential and promise when we start solving problems and start looking outward, if you will.

Ross: Yeah. You’re existing in the world that is pushing things further down that course. So where can people find out more about your work?

Matt: Right now, LinkedIn is probably the best way.

We’re in the midst of a merger of equals between my original firm, Elemental, and my business partner John Nelson’s firm, John Nelson Advisors. By Labor Day (U.S.), we’ll be back out in the world as iLIVD—i, L, I, V, D—with a new website and clout room and all the rest.

But it’s the same focus: AI-native venture studio for brain health—just twice the people, twice the energy, and all the consideration.

So we’re looking forward to continuing to serve the community and progressing forward.

Ross: No, it’s fantastic. Matt, you are a force for positive change, and it’s fantastic to see not just, obviously, the underlying attitude, but what you’re doing. So, fantastic. Thank you so much for your time and everything you’re doing. Thank you again.

Matt: Thank you again Ross, I really appreciate you having me on, and always a pleasure speaking with you.

The post Matt Lewis on augmenting brain capital, AI for mental health, neurotechnology, and dealing in hope (AC Ep8) appeared first on Humans + AI.

“Successful AI ventures are those that truly understand the technology but also place real human impact at the center — it’s about creating solutions that improve lives and drive meaningful change.”

– Amir Barsoum

Amir Barsoum is Founder & CEO of InVitro Capital, a venture studio that builds and funds companies at the intersection of AI and human-intensive industries, with four companies and over 150 professionals. He was previously founder of leading digital health platform Vezeeta and held senior roles at McKinsey and AstraZeneca.

Website:

InVitro Capital

LinkedIn Profile:

Amir Barsoum

X profile:

Amir Barsoum

Ross Dawson: I’m here. It’s wonderful to have you on the show.

Amir Barsoum: Same here, Ross. Thank you for the invite.

Ross: So you are an investor in fast-moving and growing companies. And AI has come along and changed the landscape. So, from a very big picture, what do you see? And how is this changing the opportunity landscape?

Amir: So, actually, we’re InVitro Capital. We actually started because we have seen the opportunity of AI.

We actually started with the sort of the move. And a big part of the reason of what we started is we think that the service industry—think about healthcare and home repair, even some service providers today—they’re going to be hugely disrupted by AI. Whether there will be automation, replacement as a bucket, or augmentation as a bucket, or at least facilitation.

And we’ve seen a huge opportunity that we can build. We can build AI technology that could do the service. Instead of being a software-as-a-service provider, we basically build the service provider itself.

So that’s what excites us about what we’re trying to do and what we’re building.

Ross: So what’s the origin of the word InVitro Capital? Does this mean test tubes?

Amir: So, I think it originates from there. I think the idea is we’re building companies under controlled conditions. And it’s kind of the in vitro—in vitro fertilization, like the IVF.

We keep on building more companies under these controlled conditions. That’s the idea, and because we come from a healthcare background, so it kind of resonated.

Ross: All right, that makes sense. So, there’s a lot of talk going around—SaaS is dead. So this kind of idea, you talk about services and the way services are changing.

And so that’s—yeah, absolutely—service delivery, whether that’s service by humans, whether it’s service by computers, whatever the nature of that, is changing. So does this mean that we are fundamentally restructuring the nature of what a service is and how it is delivered?

Amir: I think, yes. I think between the service industry and the software industry, both of them are seeing a categorical change in how they’re going to be provided to the users. And, I mean, the change is massive. I’m not sure about the word “dead,” but we’re definitely seeing a huge, huge change.

Think about it from a service perspective, from a software perspective. In software, I used to sell software to a company. The company needs people to be smart enough, educated enough, trained enough to use the software and give you value out of it. They used to be called the system of records with some tasks, but really it’s a system of record that has a lot of records, and then somebody—some employee—who sits there and does the job.

In the service, it’s kind of, you think this is going to be very difficult, and they’re going to do somebody as an outsource to do the service for me. Think about, I’m going to go and hire someone who’s going to help us do marketing content, or someone who would do even legal—and I’m going to the extreme.

And I think both are seeing categorical change. The software and the employee, both together, could become one, or at least 80% of the job could be done now by AI technologies. And the service—the same thing. So we’re definitely seeing a massive change in these aspects. And talk legal, talk content marketing—all of them.

Ross: I’d like to dig into that a little bit more. But actually, just one question is around pricing.

Are you looking at or exploring ways in which fee structures or pricing of services change? I mean, that’s classically where services involved humans—there was some kind of correlation to the cost of the human plus the margin.

Now there is AI, which has taken often an increasing proportion of the way the service is delivered. So—and different perceptions where clients, customers think, “Oh, you must be able to do it really cheap now, so give it to me cheaper.” Or maybe there’s more value created.

So are you thinking about how fees and pricing models change for service?

Amir: I have a strong concept when it comes to pricing and innovation.

Think about ride-hailing when it has been introduced in the market. It was introduced at the price-competitive advantage compared to the yellow cab, right? Yes, you can come in with many other better benefits, like it’s coming with security and safety, but the reality is, it’s there. That’s when you hit the mass market—you need to play on pricing. And I think that’s the beauty of innovation.

And I think AI as a technology, and with its very, very wide use cases, it’s going to make every single thing around us significantly cheaper. Let’s take the same ride-hailing example. If you introduce the auto-driving ride-hailing, we are literally taking almost 70% of the cost today off of the table.

So if you’re not going to introduce a significantly cheaper price, I don’t think it’s going to find the mass market. So that’s from a form of the absolute value of pricing—how to think of pricing.

So where I would split this into two categories—we tried going and we basically say, “You know what, if you hire a person, it will cost you X. Hire an AI person, and it will cost you Y.” I find this not working very well.

Where where seeing the pay-as-you-go model is the easier way, the more comprehensible way. So, you think about the SaaS pricing, if you think about the service pricing—and that’s a continuum—I think we’re somewhere in the middle.

And I think the best is closer a bit to the SaaS pricing, but more about—you use more, you pay more—kind of a game, rather than feature-based pricing. So consumption-based pricing, but less related to the FTE.

Because, for example, when you say, “We have a recruiter, AI recruiter,” and you say it’s $800 a month instead of paying for a full-time recruiter, human recruiter, who is $7,000 a month—then what is the capacity of this AI recruiter? Is it equal to the human recruiter? Or it’s an unlimited capacity?

So we find this is not working really, really well. What works is really use-based, not feature-based events.

Ross: Right. So moving away from necessarily time-based subscription to some kind of consumption—

Amir: Consumption-based, yeah. Yeah, you could time it. You can time it a little bit as to timing, but really, it’s a consumption-based.

Ross: Yeah, and there is also the new models like outcome-based, such as Sierra, where they price—if you get an outcome, where you get a customer resolution, you pay. If you don’t, then you don’t.

Amir: Which is—this one is actually—so we have a company that we’re going to put in the market that is related. That is related to AI GTM—so AI go-to-market solution.

We’re going to go with the model of, “You know what? Pay when you get.” Which I think is a very, very interesting model. It’s a super good and easy way to acquire customers.

But you also need them to be a little bit engaged in some input so you can do a great job for them. But if they haven’t paid, then you’re going to find it—the engagement component—I think the funnel drops a little bit there.

We haven’t fixed that yet, but I think somehow it mixes between the consumption-based, but very, very small, and then more of the pay-per-outcome. I think this would be the fascinating solution.

Ross: Yeah, yeah. Well, it’s certainly evolving fast, and we’ll experiment, see what works, and work on that.

Amir: So I’ll tell you about it. I’ll tell you what’s going to happen. 

Ross: So you have your healthcare background, you’re InVitro Capital, you are investing in the healthcare sector.

So I’d like to sort of just pull back to the big picture. So there’s a human-plus-AI perspective. And thinking more—there are humans involved, there’s AI, which is now complementing us.

Of course, there’s been many things in healthcare where AI can do things better and more effectively than people—just doing things which are not inspiring. And there’s also a lot of abilities to complement AI in how we deliver services, the quality of those services, and so on.

So best you can, just sort of take a big picture and say, what are that—where in this flow of healthcare do you see opportunities for humans and AI to do things better, faster, more effectively than they’ve done before?

Amir: So, healthcare—because the technicalities of the technical component of healthcare—is a very sensitive topic. When you start getting into the clinical decisions of it, it’s a very sensitive topic.

But in reality, healthcare is written in books, right? Especially the non-intervention healthcare. You think about the primary care—most of the non-intervention is written in books. And the LLM models know them. And even with many other data models you have—and even the big healthcare systems—they have tons of this data.

So you can actually today go straight away with some of the clinical solutions. You know, if you take a picture now as a consumer of something on your skin and put it on, it can kind of give you a very, very good answer.

But is this something that we think is ready to be commercialized and go to market? The answer is: no, not today.

But we’re seeing, on the other side, every single thing until the clinical decisions is seeing massive, massive augmentation.

And we think about it from a patient journey perspective. And in the patient journey, there are anchors as well. You can see it with the provider side, but see the accessibility component—where can patients access healthcare?

And I mean having the conversation and the scheduling and the difficulty of the scheduling and getting third parties involved. And this is not a typical administrative task—there are some medical people who used to be involved in this.

So, for example, the patient can’t see the diagnostic center unless you get an approval. But when you try to get an approval from the insurance firm, the insurance firm declines. So you need to get one more comment here, one more writing here, to get the insurance firm to approve.

Can you do these kinds of things—which is not the billing part, it’s still accessibility? And we are—we’re seeing AI technology playing a significant role in this.

Take it to the next step: billing, for example, which is really getting the provider to be paid for this visit, and maybe start diverting what is the copay and what is not. A lot of people are involved in this, and we’re seeing massive, massive implementation in that space, and workflow automation in that space as well.

Ross: So just coming back to that. So this idea of workflow, I think, is really critical. Where you mentioned this idea of approvals.

And so, yes, part of it is just flow—okay, this thing is done, it then goes on to the next person, machine, whatever system or organization. But this comes back to these decisions where essentially AI can provide recommendations, AI can basically be automated with some kind of oversight. There may be humans involved with providing input.

So in healthcare—particularly pointed—I mean, I suppose even in that patient experience process. So how are you thinking about the ways in which AI is playing a role at decision points, in terms of how we shift from what might have been humans making decisions to now what are either AI-delegated or AI-with-humans-in-the-loop or any other configuration?

So how are we configuring and setting up the governance and the structures so these can be both efficient and effective?

Amir: In very simple terms, there are the workflows and there are the AI workflows, which are very different—very different from how technology is designed and built, and very, very different in their outcome.

I think every single thing that we tried to do before in healthcare using workflows was, at best, not working. It even could look nice, but it just didn’t work. That’s the fact.

Because you start building algorithms and start putting rules in your code—if this happens, do that; if this happens, do that—you never cover enough rules that would make it really solid. And if you do, then the system collapses.

I think now we are at the stage where there are data models that you keep on indicating—this data model on whether this worked or not, the satisfaction level of the patient, whether this ended up in crazy billing and payment or not, whether this ended up in actually losing money for the provider or not losing money for the provider, the amount of time that has been lost, whether we have utilized the provider’s time or not—which is the most expensive component until today.

We talk AI, but still, we need healthcare providers.

So there, you build these data models that make the AI take decisions on: shall I book Amir tomorrow at 2 p.m., or I’d rather book Amir the day after tomorrow?

There are many, many data points that need to be considered in this intervention—Amir’s timeline, the doctor’s timeline, availability. These are the easy parts.

But the not-easy part is what the data models tell us—that makes the AI think like a human and on its feet, and saying, “You know what, I would book Ross tomorrow, but Amir the day after tomorrow,” because of the tons and tons of things: utilization, expectation, what’s the time that you’re going to take—leveraging on history of data that could work.

And the more you move into the billing component—and by the way, I know most of the people in healthcare think more about the clinical decisions—but in reality, healthcare is decided by payment and billing. These are the two biggest points, right?

Ross: So one of the interesting things here—I guess, pointing to the fact—we’ve got far more data than ever before, and hopefully we’re able to do things like measure emotional responses and so on, which is important if we’re going to build experience.

I mean, just massive things that can feed into models. But one of the points is that healthcare is multiparty. There’s a whole wealth of different players involved.

And there are some analogies to supply chain, except supply chains are far easier than healthcare. You have multiple players, and you have data from many different participants. And there is value to optimizing across the entire system, but you’ve got segregated data and then segregated optimization algorithms.

And in fact, if you optimize within different segments of the entire piece, then the entire thing as a whole may, in fact, end up being worse off than it was before.

So do you have a vision for how we can get to, I suppose, healthcare-wide optimization based on data and effective AI?

Amir: That’s a very, very, very good question, honestly—and quite deep.

So, healthcare has been—there are the payers, the insurance—the guys that pay the money. There’s the healthcare providers. And those—think about it—healthcare providers as entities, the organization. And healthcare providers as individuals—the doctors, the nurses, the pharmacists, right?

And then there’s the patient. And there’s the employer.

So there are all of these components together.

And we have been seeing a trial of creating vertical integration in healthcare in the past—a payer buying hospitals, buying clinics—and thinking that this is going to be cheaper for him, and it is. And to do it—but it has been slow because it’s very difficult to run a complete insurance firm and a healthcare provider—hiring doctors and managing workflows and payroll and the quality of patients, and making sure that the patient liked your hospital so they can come again, or your clinic, and not leaving or walking away.

And then what we are seeing is—there’s a very well-known concept in AI, which is the flattening of the org structure.

I think we’re going to see this in healthcare.

It becomes easier to do this vertical integration—the clinic, the scan centers, the pharmacists, the hospitals. It’s becoming way easier with time, by basically automating and using AI and augmenting what we do today—and shrinking, kind of running it together.

I think we’re going to see this more and more in the future.

Ross: The existing players start to do roll-ups or consolidate. So that becomes quite capital intensive in terms of being able to build this vertical integration. So either you build it out, or you buy it.

Amir: Or you build it out without being super capital intensive, because you’re using tech that is, again, you don’t need to be as capital intensive as you used to be before.

You know, for example, the working capital of people involved is going to be significantly less than what you used to see before. I’m less talking about the hospitals at this stage, but I think the outpatient setup will definitely see this.

I’ll give you an example. In the pharmacy business, we have automated—not fully automated, but augmented. In the pharmacy business, think about it. It’s a small factory. You get the prescription, somebody needs to type the prescription—we call them typists. Then somebody needs to fill it, and then a pharmacist needs to check it.

So we’ve automated many, many of those, using even some machines there until the filling component. Then the pharmacist would show up and do the checking.

So the working capital is shrinking, the time is becoming way and way more leaner, and hence it’s way more efficient.

Ross: So let’s pull back to the big picture of how does AI impact work, society, investments—everything. So what are any macro thoughts you have around how some of the big shifts—and how we can basically make these as positive as possible?

Amir: So I will give you my answer from what I’m seeing in the AI and the service intersection, because we are doing a lot of work in that space.

I think many, many of the jobs gonna vanish and cease to exist. But also, very interesting, we’re seeing a massive uptake in the EdTech, where people are jumping in to kind of elevate their skill sets.

And I think the time is there. It’s not a crazy, gloomy picture. I think there’s some time they can actually get that and fill up the space.

The level of shortage we’re seeing in healthcare is unheard of, and we are aging as a population. And the reality of the matter—we need those. So I need less people working on the reception and the billing department, and I want more people who could provide some level of healthcare component.

And we’re seeing this happening. Think about the home repair. I need less people who do administrative work, and I need more electricians, and I need more plumbers.

And I think we’re seeing more jumping into the EdTech components, getting these preparation to the exams and the test to kind of elevate themselves into this role.

And I think AI is definitely accelerating the elimination of the jobs, but also accelerating access to education so that you can capture the new job. And we’re definitely seeing these two pieces happening, I would say, in a very, very fast pace.

So that’s one thing we’re seeing.

From an investment perspective, we look at investment into three categories. We look at them into:

Category A: Investing into foundational models like the LLMs. And I think foundation models—and I would say the whole foundation thing about AI—there’s the foundation model, then the infrastructure game there. I think it’s a very, very interesting space. It’s the power law game—applies very strongly.

And I think the choice there is, I would say, the biggest factor. And obviously access at the right time. So this is category A.

Category B: The application layer. And I think in the application layer, I personally believe—and I think there’s a lot of belief that’s happening—we’re seeing less of the power law exercise.

We’re not expecting people to exit $10 billion in that space. I would say there’s democratization of the application layer. And I think the best there is how you can build with a very cost-efficient manner, so that the return on capital is as good as people expected.

And that’s what we operate, actually, as a venture studio and a venture builder.

Category C: What’s going to happen in the businesses, and the mom-and-pop shops, and the street businesses in the service industry.

And I think for this category and the second category, we’re going to see a lot of kind of merging together—roll-ups between the second category and the third category.

Either the big ones of the third buy some of the second, or the big ones of the second buy some of the third. We’re seeing this—even Silicon Valley start to talk about the roll-ups in the VC space and the like.

So that’s how we think about it.

Ross: So actually, I want to dig into the venture studio piece. All right, so you’ve got a venture studio, you’re running a bunch of things in parallel. You’re doing that as efficiently as possible. So what? How? Right? Well, what are your practices? How do you amplify your ability to build multiple ventures simultaneously, given all of the extraordinary technologies? What are the insights which are really driving your ability to scale faster than ever before?

Amir: So, usually, when we try to scale, we think about whether there is recyclability of customers. That’s the first thing we think about.

I think we use, you know, if you look at our first deck, there was recyclability of customer, recyclability of technology. Honestly, if we talk about a segment of technology now, it’s a joke. You know, it’s going to take, what, a month to build? So we took this out.

Really, it’s distribution. And this has become, again—think about the application layer—distribution is the most difficult thing to capture, because everybody would come and tell you, “I’m a sales AI, I’m a sales AI.” Okay, I’ve got 20 emails about sales AI, 20 emails about AI recruiters. And distribution is a very big component.

So the recyclability of customers is a very big part. The second part is availability of data, because if you don’t build your own data models, train your own solutions, so you create a very, very unique quality of the product.

The product wouldn’t be good enough for the expectations, because today, the consumer and the business expectations when you say AI have been super high, because they think it’s going to produce the same value as if you are a consumer using ChatGPT, which in most of the cases, that doesn’t happen unless you have a very unique data model that helps fix a very unique solution.

And again, think about the diagnostics that we’re doing in the home repair space. We collected millions and millions of pictures and images, and we spend—we even keep our model trained ourselves.

So we go into the—we provide the—we do the service, and we take it and we make sure that we can start providing feedback, and then we feed back to the system so that we can start creating these data models that would make sense. Otherwise, the solution is not as great.

And if you think about the solution that we launched in the market three months ago, I would say bad at best. Now, I would say it’s significantly better. And I still think that we have a way to go adding more and more data to what we are building to fix that.

So, recyclability of consumers and of customers is a big thing, and availability of data—I would say these are the two big components that, if we find them, say, “That is something to do.”

I’m also not going to say all the clichés, you know, find the pain in the market. I think it’s a standard. 

Ross:Yeah, yeah that’s not new. That’s all. 

Amir: Yeah, yeah.

Ross: Fabulous. That’s, that’s really interesting. So to round out, I mean, what’s most exciting to you about the potential of humans plus AI — around where it is we can go from here?

Amir: Oh, I’m, I’m very excited to see the—so I will say something that is, I’m not sure how contrarian it is.

I think we’re going to see the quality of products and service around us in a totally different level. Totally different level. We’re gonna—I think our generation has been living in prosperity significantly better than the previous generation. And the previous generation is that, you know, if you—a very, very rich man 100 years ago lives way worse of a life than a poor man who lives today, right?

I know if you compare it—just get this one and get this one—and you see the level of comfort, the day-to-day work and so on. But this takes 100 years to see major, major difference.

I think we’re going to see this now in much shorter periods, I would say, 10 years. And that’s the positive and the good part of it. But also that comes with a little bit of, you know, it’s a scary belief, okay, what’s going to happen tomorrow? Am I fast enough as an investor, as a human being? What’s our kids going to do?

So I think these questions also pop up and make us think about it. But I’m quite excited, generally, about how the quality of life is going to move significantly in an upward trajectory. Hopefully, we, as humans, we’re going to mitigate the risks that we’re seeing potentially going to happen—security risks, cyber security risks, and tons of others.

Ross: So where can people go to find out more about your work in ventures?

Amir: InVitroCapital.com. I’m on LinkedIn. Just writing the name “Amir Barsoum,” you’re going to find me there. On LinkedIn, our team is there. So all of these are the best ways to reach out.

Ross: Fantastic. Thank you for your time and your insights, Amir.

Amir: Ross, that was great. Thank you very much for the deep questions.

The post Amir Barsoum on AI transforming services, pricing innovation, improving healthcare workflows, and accelerating prosperity (AC Ep7) appeared first on Humans + AI.

What are the goals I really want to attain professionally and personally? I’m going to really keep my eye on that. And how do I make sure that I use AI in a way that’s going to help me get there—and also not use it in a way that doesn’t help me get there?

– Minyang Jiang (MJ)

Minyang Jiang (MJ) is Chief Strategy Officer at business lending firm Credibly, leading and implementing the company’s growth strategy. Previously she held a range of leadership positions at Ford Motor Company, most recently as founder and CEO of GoRide Health, a mobility startup within Ford.

Website:

Minyang “MJ” Jiang

Minyang “MJ” Jiang

LinkedIn Profile:

Minyang “MJ” Jiang

Using ai to overcome human constraints

Redefining productivity through augmentation

Nurturing curiosity in the modern workplace

Building trust in an ai-first strategy

The role of imagination in future planning

Why leaders must engage with ai hands-on

Separating the product from the person

Ross Dawson: MJ, it’s a delight to have you on the show.

Minyang “MJ” Jiang: I’m so excited to be here, Ross.

Ross: So I gather that you believe that we can be more than we are. So how do we do that? 

MJ: Absolutely I’m an eternal optimist, so I’m always—I’m a big believer in technology’s ability to help enable humans to be more if we’re thoughtful with it.

Ross: So where do we start? 

MJ: Well, we can start maybe by thinking through some of the use cases that I think AI, and in particular, generative AI, can help humans, right?

I come from a business alternative financing perspective, but my background is in business, and I think there’s been a lot of sort of fear and maybe trepidation around what it’s going to do in this space. But my personal understanding is, I don’t know of a single business that is not constrained, right? Employees always have too much to do. There are things they don’t like to do. There’s capacity issues.

So for me, already, there’s three very clear use cases where I think AI and generative AI can help humans augment what they do. So number one is, if you have any capacity constraints, that is a great place to be deploying AI because already we’re not delivering a good experience. And so any ability for you to free up constraints, whether it’s volume or being able to reach more people—especially if you’re already resource-constrained (I argue every business is resource-constrained)—that’s a great use case, right?

The second thing is working on a use case where you are already really good at something, and you’re repeating this task over and over, so there’s no originality. You’re not really learning from it anymore, but you’re expected to do it because it’s an expected part of your work, and it delivers value, but it’s not something that you, as a human, you’re learning or gaining from it.

So if you can use AI to free up that part, then I think it’s wonderful, right? So that you can actually then free up your bandwidth to do more interesting things and to actually problem-solve and deploy critical thinking.

And then I think the third case is just, there are types of work out there that are just incredibly monotonous and also require you to spend a lot of time thinking through things that are of little value, but again, need to be done, right? So that’s also a great place where you can displace some of the drudgery and the monotony associated with certain tasks.

So those are three things already that I’m using in my professional life, and I would encourage others to use in order to augment what they do.

Ross: So that’s fantastic. I think the focus on constraints is particularly important because people don’t actually recognize it, but we’ve got constraints on all sides, and there’s so much which we can free up.

MJ: Yes, I mean, I think everybody knows, right? You’re constrained in terms of energy, you’re constrained in terms of time and budget and bandwidth, and we’re constrained all the time.

So using AI in a way that helps you free up your own constraints so that it allows you to ask bigger and better questions—it doesn’t displace curiosity. And I think a curious mind is one of the best assets that humans have.

So being able to explore bigger things, and think about new problems and more complicated problems. And I see that at work all the time, where people are then creating new use cases, right? And it just sort of compounds.

I think there’s new kinds of growth and opportunities that come with that, as well as freeing up constraints.

Ross: I think that’s critically important. Everyone says when you go to a motivational keynote, they say, “Curiosity, be curious,” and so on. But I think we, in a way, we’ve been sort of shunned.

The way work works is: just do your job. It doesn’t train us to be curious. So if, let’s say, we get to a job or workplace where we can say—we’re in a position of work where you can say—all right, well, all the routine stuff, all the monotony, we’ve done. Your job is to be curious.

How do we help people get to that thing of taking the blinkers off and opening up and exploring?

MJ: I mean, I think that would be an amazing future to live in, right? I mean, I think that if you can live in a world where you are asked to think—where you’re at the entry level, you’re asked to really use critical thinking and to be able to build things faster and come up with creative solutions using these technologies as assistance—wouldn’t that be a better future for us all?

And actually, I personally would argue and believe that curiosity is going to be in high demand, way higher demand than it’s been in the future, because there is this element of spontaneous—like spontaneous thinking—which AI is not capable of right now, that humans are capable of.

And you see that in sort of—even sort of personal interactions, right? A lot of people use these tools as a way to validate and continue to reinforce how they think. But we all know the best friendships and the best conversations come from being called out and being challenged and discovering new things about yourself and the thing.

And that same sentiment works professionally. I think curiosity is going to be in high demand, and it’s going to be a sort of place of entry in terms of critical thinking, because those are the people that can use these tools to their best advantage, to come up with new opportunities and also solve new problems.

Ross: So I think, I mean, there is this—I say—I think those who are curious will, as you say, be highly valued, be able to create a lot of value. But I think there are many other people that have latent curiosity, as in, they would be curious if they got there, but they have been trained through school and university and their job to just get on with the job and study for the exam and things like that.

So how do we nurture curiosity in a workplace, or around us, or within?

MJ: I mean, I think this is where you do have this very powerful tool that is chat-based, for the most part, that you don’t require super technical skills to be able to access. At least today, the accessibility of AI is powerful, and it’s very democratizing.

You can be an artist now if you have these impulses but never got the training. Or you can be a better writer. You can come up with ideas. You can be a better entrepreneur. You can be a better speaker.

It doesn’t mean you don’t have to put in the work—because I still think you have to put in the work—but it allows people to evolve their identity and what they’re good at.

What it’s going to do, in my mind, rather than these big words like displacement or replacement, is it’s going to just increase and enhance competition.

There’s a great Wharton professor, Stefano Plutoni, who talked about photography before the age of digital photography—where people had to really work on making sure that your shutter speed was correct, you had the right aperture, and then you were in the darkroom, developing things.

But once you had digital photography, a lot of people could do those things. So we got more photographers, right? We actually got more people who were enamored with the art and could actually do it.

And so some of that, I think, is going to happen—there’s going to be a layering and proliferation of skills, and it’s going to create additional competition. But it’s also going to create new identities around: what does it mean to be creative? What does it mean to be an artist? What does it mean to be a good writer?

In my mind, those are going to be higher levels of performance. I think everyone having access to these tools now can start experimenting, and companies should be encouraging their employees to explore their new skills.

You may have someone who is a programmer who is actually really creative on the side and would have been a really good graphic artist if they had the training. So allowing that person to experiment and demonstrate their fluidity, and building in time to pursue these additional skill sets to bring them back to the company—I think a lot of people will surprise you.

Ross: I think that’s fantastic. And as you say, we’re all multidimensional. Whatever skills we develop, we always have many other facets to ourselves.

And I think in this world, which is far more complex and interrelated, expressing and developing these multiple skills gives us more—it allows us to be more curious, enabling us to find more things.

Many large firms are actively trying to find people who are poets or artists or things on the side. And as you say, perhaps we can get to workplaces where, using these tools, we can accelerate the expansion of the breadth of who we are to be able to bring that back and apply that to our work.

MJ: I mean, I’ve always been a very big fan of the human brain, right? I think the brain is just a wonderful thing. We don’t really understand it. It truly is infinite. I mean, it’s incredible what the brain is capable of.

We know we can unlock more of its potential. We know that we don’t even come close to fully utilizing it.

So now having these tools that sort of mimic reasoning, they mimic logic, and they can help you unlock other skills and also give you this potential by freeing up these constraints—I think we’re just at the beginning of that.

But a lot of the people I work with, who are working with AI, are very positive on what it’s done for their lives.

In particular, you see the elevated thinking, and you see people challenging themselves, and you see people collaborating and coming up with new ideas in volume—rewriting entire poorly written training manuals, because no one reads those, and they’re terrible. And frankly, they’re very difficult to write.

So being able to do that in a poetic and explicable way, without contradictions—I mean, even that in itself is a great use case, because it serves so many other new people you’re bringing into the company, if you’re using these manuals to train them.

Ross: So you’ve worked on Jedi, Jedi projects in the workplace—put this into practice, sort of. So I’d love to hear just, off the top of your mind, what are some of the lessons learned as you did that?

MJ: Yeah, we’ve been deploying a lot of models and working with our employee base to put them into production. We also encourage innovation at a very distributed level.

The biggest thing I will tell you is—change management. For me, the important part is in the management, right? Change—everybody wants change. Everyone can see the future, and I have a lot to say about what that means. But people want change, and it’s the management of change that’s really difficult. That requires thought leadership.

So when companies are coming out with this AI-first strategy, or organizations are adopting AI and saying “we are AI-first,” for me the most important lesson is strategically clarifying for employees what that means.

That actually isn’t the first thing we did. We actually started doing and working and learning—and then had to backtrack and be like, “Oh, we should have a point of view on this,” right?

Because it’s not the first thing. The first thing is just like, “Let’s just work on this. This is fun. Let’s just do it.” But having a vision around what AI-first means, and acknowledging and having deep respect for the complexities around that vision—because you are touching people, right? You’re touching people’s sense of self-worth. You’re touching their identities. You’re touching how they do work today and how they’re going to do work three to five years from now.

So laying that out and recognizing that we don’t know everything right now—but we have to be able to imagine what different futures look like—that’s important. Because a lot of the things I see people talking about today, in my view, is a failure of the imagination. It’s pinning down one scenario and saying, “This is the future we’re going to march towards. We don’t love that future, but we think it’s inevitable.”

As leaders—it’s not inevitable. So doing the due diligence of saying, “Let me think through and spend some time really understanding how this affects my people, and how I can get them to a place where they are augmented and they feel confident in who they are with these new tools”—that are disruptive—that’s the hard work. But that is the work I expect thought leadership and leaders to be doing.

Ross: Yes, absolutely right. And I think this—as you say—this getting any sense of the inevitable is deeply dangerous at best.

And as you say, any way of thinking about the future, we must create scenarios—partly because there are massive uncertainties, and perhaps even more importantly, because we can create the future. There are no inevitabilities here.

So what does that look like? Imagination comes first if we are building the company of the future. So how do we do that? Do we sit down with whiteboards and Post-it notes? What is that process of imagining?

MJ: There’s so many ways to do it, right? I mean, again—I took a class with a Wharton professor, Scott Snyder. He talked about “future-back” scenario planning, which is basically:

First, I think you talk to many different people. You want to bring in as many diverse perspectives as possible. If you’re an engineer, you talk to artists. If you’re a product person, you talk to finance people. You really want to harness everyone’s different perspectives.

And I think, along with the technology, there’s one thing that people should be doing. They should first of all think about defining—for your own function or your own department—what does it mean to be literate, proficient, and a master at AI? What are the skill sets you’re going to potentially need?

Then it’s really up to every company. I myself created a strategic framework where I can say, “Okay, I think there’s a spectrum of use cases all the way from a lot of automation to AI being simply an assistant.” And I ask different people and functions in the company to start binning together what they’re doing and placing them along this spectrum.

Then I would say: you do this many times. You write stuff down. You say, “Okay, perhaps I’m wrong. Let’s come up with an alternate version of this.”

There are several levers that I think a lot of people could probably identify with respect to their industry. In my industry, one of the most important is going to be trust. Another one is going to be regulation. Another one is going to be customer expectation.

So when I lay out these levers, I start to move them to the right and left. Then I say, “Well, if trust goes down in AI and regulations go up, my world is going to look very different in terms of what things can be automated and where humans come in.”

If trust goes up and regulations go down, then we have some really interesting things that can happen.

Once you lay out multiple of these different kinds of scenarios, the thing you want to look for is: what would you do the same in each one of these scenarios? Would you invest in your employees today with respect to AI?

And the answer is always yes—across every single scenario. You will never have less ROI. You will always be investing in employees to get that ROI.

So now you look at the things and say, “What am I going to do in my AI-first strategy that’s going to position me well in any future—or in a majority of futures?”

Those are the things you should be doing first, right now.

Then you can pick a couple of scenarios and say, “Okay, now I need to understand: if this were to change, my world is going to be really different. If that were to change, my world is going to be really different.”

How do I then think through what are the next layer of things I need to do?

Just starting with that framework—to say, what are the big levers that are going to move my world? Let’s assume these things are true. Let’s assume those things are true. What do my worlds look like?

And then, is there any commonality that cuts across the bottom? The use cases I gave earlier—around training, freeing up capacity—that cuts across every single scenario. So it makes sense to invest in that today.

I’m a big believer in employee training and development, because I always think there’s return on that.

Ross: That’s really, really good. And I can just, I can just imagine a visual framework laid out just as you’ve described. And I think that would be extremely useful for any organization.

So you mentioned trust. There’s obviously multiple layers of trust. There’s trust in institutions. There’s trust in companies—as you mentioned, in financial customer service, very relevant. There’s trust in society. There’s trust in AI. There’s trust in your peers.

And so this is going to be fundamental. Of course, your degree of trust—or appropriate trust—in AI systems is a fundamental enabler or determinant of how you can get value from them. Absolutely.

So how do we nurture appropriate trust, as it were, within workplaces with technology in order to be able to support something which can be as well-functioning as possible?

MJ: Yeah. I mean, I think trust is foundationally going to remain the same, right? Which is: do you know what is the right thing to do, and do people believe that you’re going to consistently execute on that right thing, right?

So companies that have values, that have principles that are well-defined, are going to continue to capitalize on that. There’s no technology that’s going to change that.

Trust becomes more complicated when you bring in things like AI that can create—that’s very, very persuasive—and is mimicking sort of the human side so well that people have difficulties differentiating, right?

So, for example, I run a sales team. And in sales, often people use generative AI to overcome objections. That is a great usage of generative AI. However, where do you draw the line between that—between persuasion and manipulation—and between manipulation and fraud, right?

I don’t think we need technology to help us draw the line. I think internally, you have to know that as a business. And you have to train your employees to know where the line is, right?

Ethics is always going to be something that the law can’t quite contain. The law is always what’s legal, and it’s sort of the bottom of the ethics barrel, in my opinion, right? So ethics is always a higher calling.

So having that view towards what is the use of ethical or accountable or responsible AI in your organization—having guardrails around it, writing up use cases, doing the training, having policies around what does that look like in our industry.

In many industries, transparency is going to be a very big factor, right? Do people know and do they want to know when they’re talking to a human versus talking to generative AI, right?

So there’s customer expectations. There’s a level of consistency that you have to deliver in your use cases. And if the consistency varies too much, then you’re going to create mistrust, right?

There’s also bias in all of the data that every single company is working with. So being able to safeguard against that.

So there are key elements of trust that are foundationally the same, but I think generative AI adds in a layer of complexity. And companies are going to be challenged to really understand: how have they built trust in the past, and can they continue to capitalize and differentiate that?

And those that are rushing to use generative AI use cases that then have the byproduct of eroding trust—including trust from their own employees—that’s where you see a lot of the backlash and problems.

So it pays to really think through some of these things, right? Where are you deploying use cases that’s going to engender credibility and trust? And where are you deploying use cases that may seem like it’s a short-term gain—until a bad actor or a misuse or something happens on the internet?

Which now, with deepfakes, it’s very easy to do. Now your reputation is very brittle because you don’t have a good foundational understanding of: do you have the credibility of your customers, of employees, that they trust, that you know what to do on what’s right, and then you can lead them there.

Ross: Yeah, that’s obviously—in essence—trust can be well-placed or misplaced. And generally, people do have a pretty good idea of whether people, technology, institutions are trustworthy or not.

And especially the trustworthiness is ultimately reflected in people’s attitudes and ultimately that which flows through to business outcomes.

So I think the key here is that you have to come from the right place. So having the ethical framework—that will come through. That will be visible. People will respond to it.

And ultimately, customers will go to those organizations that are truly trustworthy, as opposed to those that pretend to be trustworthy.

MJ: And I think there’s—and I think trust is about—there’s a time dimension here. There’s a time dimension with any technology, which is: you have to do things consistently, right?

Integrity is not a one-day game. It’s a marathon. It’s not a sprint. And so if you continue to be consistent, you can explain yourself when you make mistakes, right?

You know how to own up to it. You know what to say. You know how to explain it to people in a real way that they can understand.

That’s where trust—which is hard—trust is earned over time, and it can be depleted very quickly. And I think many, many companies have been burned through not understanding that.

But overall, it is still about doing the right thing consistently for the majority of the time and owning up to mistakes.

And to the extent that having an ethical AI framework and policy can help you be better at that, then I think those use cases and organizations and companies will be more successful.

And to the extent that you’re using it and it creates this downstream effect of eroding that trust, then it is extremely hard to rebuild that again.

Ross: Which takes us to leadership and leadership development. Of course, one foundation of leadership is integrity. There’s many things about leadership which aren’t changing. There are perhaps some aspects of leadership that are changing in a—what is—a very, very fast-moving world.

So what are your thoughts around how it is we can develop effective leaders, be they young or not so young, into ones that can be effective in this pretty, pretty wild world we live in?

MJ: I think with leadership, as it is, always a journey, right? There’s two things that in my mind leadership sort of comes back to. One is experience, right? And the other is the dimension we already mentioned, which is time.

As a leader, first of all, I encourage all senior leaders of companies—people who are in the highest seats of the companies—to really get in the weeds involved with generative AI. Don’t outsource that to other people. Don’t give it to your youngest employees. Don’t give it to third-party vendors. Really engage with this tool.

Because they actually have the experience and the expertise to understand where it’s working and where it’s not working, right? You actually recognize what a good product looks like, what’s a good outcome, what seems like it’s not going to work.

A great marketing leader lives in the minds of their customers, right? So you’re going to know when it produces something which is like, this is not hitting the voice, this is not speaking with my customers, I’m going to continue to train and work. A new marketing analyst is not going to have any idea, right?

And also as a great leader, once you actually get into the guts of these tools and start to learn with it, then it is, as we mentioned before, your role to think about:

How do I create the strategy around where I’m going to augment my company—the growth, the business, the profit, and the people? What am I going to put in place to help foster that curiosity? Where am I going to allow for use cases to break those constraints, to create this hybrid model where both AI can be used and humans can be more successful?

What does being more successful mean outside of just making more money, right? Because there’s a lot of ways to make more money, especially in the short term. So defining that after having learned about the tool—that’s really the challenge that every leader is going to face.

You have this vastly changing landscape. You have more complexity than you’re dealing with, right? You have people whose identities are very much shaped by technology and who are dealing with their own self-worth with respect to these tools.

Now you have to come in and be a leader and address all of these dimensions. And exactly what you mentioned before, this idea of being a multidimensional leader is starting to become very important, right?

You can’t just say, “I’m going to take the company to this.” Now I have to think about: how do I do it in a way that’s responsible? And how do I do it in a way that guarantees long-term success for all of the stakeholders that are involved?

These questions have never really changed for leadership, but they certainly take on a new challenge when it comes to these tools that are coming in.

So making strategic decisions, envisioning the future, doing scenario planning, using your imagination—and most of all, having a level of humility—is really important here.

Because this idea of being able to predict the future, settle into it, and charge in—really, that looks fun on paper. That’s very flashy. And I understand there’s lots of press releases, that’s a great story.

The better story is someone who journals, takes time, really thinks about what this means, and recognizes that they don’t know everything. And we are all learning. We’re all learning. There’s going to be really interesting things that come up, and there’s going to be new challenges that come up.

But isn’t that what makes leadership so exciting, though, right? If everyone could do it, then that would be easy, right?

This is the hard thing. I want leaders to go and do the hard thing, because that’s what makes it amazing. And that’s what makes AI suitable for you. It’s supposed to free up your constraint and help you do harder, more difficult things—take on more challenges, right?

And that’s where I think we can truly all augment ourselves.

Ross: Yes, it is. Any effective leader is on a path of personal growth. They are becoming more. Otherwise, they would not be fulfilling the potential of the people in the organization they lead—let alone themselves, right?

So to round out, what are a few recommendations or suggestions to listeners around how they can help augment themselves or their organizations—and grow into this world of more possibilities than ever before?

MJ: Yeah. So my best advice is asking people to separate the product from the person, right? You can use AI to create a better product, but in doing so, understand—is that making you a better person, right? Is that making you better at the thing that you actually want to do?

We know about people actually having to understand the product. But even so—if your goal is to be a better writer, for example, and you use Generative AI to create beautiful pieces—is that helping you be a better writer?

Because if it’s not, that may not be the best use case. Maybe you use it for idea generation or for copy editing. So being able to separate that and really understanding that is going to be important.

The other thing is: understand what parts of your identity you really value, that you want to protect, right? And don’t then use these tools that are going to slowly chip away at that identity. Really challenge yourself.

The thing about AI—until we get to AGI—that is interesting is that it is always going to validate you. It is always going to support what you want it to do. You’re going to give it data, and it’s going to do what you tell it to do. So it’s not going to challenge you, right?

It’s not going to make you better by calling you out on stuff that your friends would—unless you prompt it, right? Unless you say, “Critique how I can be better. Help me think through how I can be better.”

And using it in that way is going to help you be a better leader. It’s going to help you be a better writer, right? So making sure that you’re saving room to say, “Hey, yes, I’m talking to this machine,” but using it to make you better—and separating the product you’re going to create and the person you want to become.

Because no one is going to help you be a better person unless you really want to make an effort to do that. And so that, I think, is really key—both in your professional and personal life—to say:

What are the goals I really want to attain professionally and personally? I’m going to really keep my eye on that. And how do I make sure that I use AI in a way that’s going to help me get there—and also not use it in a way that doesn’t help me get there?

Ross: I think that’s really, really important, and not everyone recognizes that. That yes—how do we use this to make me better? Better at what I do? Better person?

And without intent, you can achieve it. So that’s very important.

So where can people follow you and your work?

MJ: Well, I post a lot on LinkedIn, so you should always look me up on LinkedIn.

I do work for Credibly, and we recently launched a credibly.ai webpage where we constantly are telling stories about what we’re doing.

But I’m very passionate about this stuff, and I love to talk to people about it. So if you just look me up on LinkedIn and connect with me and want to get into a dialog, I’m more than happy to just share ideas.

I do think this is one of the most interesting, seismic shifts in our society. But I’m a big believer in its ability—when managed correctly—to unlock more human potential.

Ross: Fantastic. Thank you so much for your time, your insight, and your very positive energy and how we can create the future.

MJ: Thanks, Ross.

The post Minyang Jiang on AI augmentation, transcending constraints, fostering creativity, and the levers of AI strategy (AC Ep6) appeared first on Humans + AI.