“What I’m really interested in and fascinated about is that, as AI penetrates and spreads throughout the workplace and gets placed into or integrated into workflows, the first thing that happens is that people in the mix are going to have to learn how to use AI and learn why to use AI when they do.”

–Jon Husband

Jon Husband is the Founder and Principal of Wirearchy, a creative research and experimentation laboratory exploring the crossroads of AI and networked workplaces and society. He works as a coach, consultant, speaker and writer, and has co-authored three books, including Wirearchy.

Website:

wirearchy.com

LinkedIn Profile:

Jon Husband

Ross Dawson: Jon, it is wonderful to have you on the show.

Jon: Thank you very much, Ross, it’s good to see you again.

Ross Dawson: We’ve known of each other and each other’s work for a very, very long time now from, I suppose, the roots of—yeah, I suppose you can crudely say—the intersection of knowledge and networks. So, as I think many of us who have come from that background, we now are thinking about humans and their relative role to AI.

Some people will know of your wirearchy and a lot of your work of the past; others will not. So I’d love to just start off with: what is the concept of wirearchy? And then, how is that morphing or evolving, or are you building on that in how you’re thinking now? We’ll dig in and explore that.

Jon: Okay, well, I started paying attention to knowledge work and work in organizations and so on as I changed careers in my early 30s, moving from banking, where I was in management, into management consulting. I ended up working for a large global HR consulting firm that, amongst several others—all the major consulting firms that address organizational issues—have services where they do what’s called job evaluation.

What job evaluation does is put a size or a measure or a weight to a job, which then basically places it on the organization chart. I spent quite a few years writing thousands of job descriptions and helping streamline workflows and so on and so forth.

So, when the internet came along, I had always been an avid reader, and I suppose a wannabe futurist—a wannabe Ross Dawson, if you will. I was reading all sorts of books back then. Instead of dating, because I was single in my mid-30s, I was spending Friday nights reading books about organizations, like “The Living Company” by Arie de Geus, the Tofflers’ work, “Powershift,” certainly Peter Drucker’s work.

There was one day—well, I was reading all of these books, and all of the books were about the coming Information Age. The Information Age had not arrived yet; this was roughly late ’80s, early ’90s. All of a sudden, we hit 1994. I’m sitting in London, and I was just told by my team leader in my consulting firm that I was going to be proposed as one of the next global partners.

Three weeks later, I quit my job in the consulting firm because I had begun to feel very uneasy about the work I was doing. If I was made a partner, your job becomes basically selling larger projects to keep the younger consultants employed. I realized that I would be selling methods that I had come to not believe in anymore, and the reason for that is that all of the job evaluation methods sold by all the major consulting companies are all versions of generic Taylorism.

They have semantic statements that you pick to figure out a level of a job on a number of different factors. This is one of the things I’ve talked and written quite a bit about in wirearchy: this generic Taylorism is still deeply at the core of most of the work of most organizations. It’s how the work is designed.

There has been now, what, 15 or 20 years—how far back does Enterprise 2.0 go?—about collaboration and cooperation and better knowledge management and sharing and transfer of knowledge, and so on and so forth. If you know these semantic statements, which are burned into my brain from this method—the Hay method—you realize that no amount of talking about doing things differently is going to make much difference.

It’s not going to change much. And the remuneration—the way people get paid—every single person in every single company, is tied to all of that. It’s tied to your job size, it’s tied to the compensation practice, it’s tied to your performance management, it’s tied to your career plans, if an organization is still doing career planning. Frankly, it has not been touched in 75 years now.

Ross Dawson: Used to describe it as a job as a box.

Jon: Well, sure, and that’s where that term “think outside the box” comes from. I wrote an article about this at one point in time—oh, I can’t remember the title, so it doesn’t matter—but about the semantic statements essentially becoming semantic straightjackets, because they put limits around what you do.

They’re a graded level of permissions, basically, or amounts of influence and authority, and that’s the codified, official organizational chart.

So anyway, I was working with this all the time, and I realized if I was going to be made a big-time partner, I’d have to be selling these tools all the time. The internet had come along, so I quit, and I didn’t know what to do after that. I had to move from the UK because I was on a work permit, had to go back to Canada.

When I went back to Canada, all the companies I tried to approach to work as an independent consultant didn’t want to engage me, because all of the work I’d been doing in the UK was with really large multinationals, and according to them, too sophisticated for what they were doing in Vancouver.

But at the same time, I was still reading all the time—reading Charles Handy’s work, reading Gerard Fairtlough’s work on heterarchy, and so on. I came to believe very strongly that the ongoing sharing of information—which we were starting even 20 years ago to build into constant, incessant flows of information carried via hyperlinks—was going to inevitably begin to affect, I’m going to use the word affect, the traditional top-down power of hierarchy. That comes from the “knowledge is power” by Francis Bacon kind of perspective.

Now, that was 25 years ago. What we’ve seen since is, of course, what you know—one umbrella term I could apply to much of what’s going on outside of organizations is the “enshittification” of the web. The same thing applies in a lot of ways, I think, to people doing work, sitting behind screens in organizations.

Now, a whole host of things have happened in the past 10 or 15 years: there were armies of developers sitting in office spaces, all of them with their headphones on behind screens coding. There were all sorts of people beginning to understand how to use the internet. There were many failed attempts at effective knowledge management because of the idea that it’s still just good search, find documents, retrieval, without really paying any attention to the connections between people and how they work together, and so on.

Ross Dawson: So, the frame there is, I mean, obviously, moving—the wirearchy being an arche of the organization being essentially a network. Obviously, there’s more richness to that as you describe the organization as a network, as opposed to the rigid structures, which are still very much rampant. But fast-forwarding to today, what we’ve overlaid is, whilst the old rigid structure is in place, organizations are effectively a lot more loosened up by Enterprise 2.0 and other types of frames, and essentially more peer communication.

Now AI is changing a fundamental role, now being, in many ways, a participant in those workflows, in the creation of value. So where does that take us today, in this humans-plus—essentially wirearchy—pulled into where AI plays a role within those networks?

Jon: Well, it’s a fascinating question for which I don’t have an answer. I have some responses, I suppose. The notion of wirearchy came, as you pointed out, out of everybody being wired, everybody being networked—the organization as a network.

What I’m really interested in and fascinated about is that, as AI penetrates and spreads throughout the workplace and gets placed into or integrated into workflows, the first thing that happens is that people in the mix are going to have to learn how to use AI and learn why to use AI when they do. Often, it’s very soft at the beginning because it’s reminders, or “did you want to do that,” or “do you want to say that,” and so on. Increasingly, the AI, I think, will have more and more coaching built into it. But what I’m interested in is how, as we learn from the mistakes that are made in integration, and also learn from the successes that are made from integration, is that going to decompose a knowledge worker’s work and eventually capture most of their tacit knowledge and ways of working to reduce the cost of doing that kind of work?

Then, on a larger scale, what is the active decomposition of types of work through the influence and integration of AI? How is that going to change the fundamental assumptions about work design? My belief is that the work of Dave Snowden and others with respect to complex adaptive systems is what is going to become—and this is a poorly connected parallel or analogy—but I think something like the Cynefin framework, or a unified approach to complex adaptive systems, will become the Taylorism of the 21st century.

In other words, there will come to be forms of patterns and models and actions that help you address certain kinds of conditions, because I think, especially with AI, work and outputs are going to become continuous flows. They are the push and the pull, or the dynamic flow of power and authority that is alluded to in the working definition of wirearchy, the working definition of wirearchy includes knowledge, trust, credibility, and a focus on results, each of which you could write a book about. But as general headings, they are what capture what’s in play, I believe.

Ross Dawson: Yeah, no, I think absolutely still relevant today. Now, the point I was going to make was around, in complex adaptive systems, a really central concept is emergence—

Jon: Yes.

Ross Dawson: —where you are not planning or overlaying or dictating a structure; the structure and the value and how that’s created emerges. And to your point, a lot of the key aspect in that world is, how do you create the conditions for emergence of positive outcomes, as opposed to less positive outcomes?

And that’s still, of course, arguably at least as much an art as a science, particularly when you’re looking at complex adaptive systems composed of not just many humans, but also AI, which are stochastic in nature.

Jon: Yes, well, it’s a very, very good point. I think it relates to the paper I shared with you a couple of days ago about what the author is calling “weaving the web.” There is an enormous amount of human input and activity, combined with the AI, that doesn’t get measured and is not seen in our currently technocratic, generic Taylorist worldview. That’s not seen, not captured, and it arguably is the kind of human input, work, and knowledge that is going to make this whole new era operate fairly well. That’s this notion of exchanges of value.

Once that code is cracked, in terms of how to understand it, surface it, see it, measure it, this is going to lead to more and more of what Nvidia’s Jensen Huang is doing with respect to tokenization. There are some people who say tokenization will become the replacement for money in some cases, or even many cases in another, let’s say, 10 years or so. It’s kind of hard to imagine, but if you come back to the paper that you and I first connected on—Alex Imas’s review of the structural changes to the economy—if you can see the logic of his argument, he says there’s going to be a lot more work, but it’s going to be relational economy work, which ties directly into value exchange and surfacing how that exchange of value operates, say, between two people at work, or a group and a person, or two groups, and so on.

This notion of value exchange is going to ground a lot of the conceptual and abstract issues that we talk about when we talk about, you know, why is making effective collaboration so hard? Why is it hard to de-silo an organization? All of those kinds of things are going to, I believe, eventually be washed away in this continuous flow of information. So we have to look for new concepts and new ways to measure what’s being created, the value that’s being created.

Ross Dawson: Well, that’s—I mean, this is really interesting. As long as you do not recall, in “Living Networks,” I was actually laying out a quite similar thesis around value creation and network structures, and I did quite a bit of work with Verna Allee on value networks. We ran some workshops together, and we’re essentially—a lot as laid out in the paper you described, and as you’re saying now—a lot of it is saying, how do you look at the non-financial or intangible exchanges of value, which sometimes are apparent and sometimes less apparent? There are all sorts of these structures where, as you say, there is an exchange of value. Sometimes it involves money, oftentimes it doesn’t. To understand the landscape, you do need to understand all of these non-financial structures.

But are you suggesting that in this tokenization or other structures, there is a way then of being able to, I suppose, capture some of these non-financial values, which does imply there needs to be some kind of measurement, or at least a mutual agreement or assessment on what that value is?

Jon: Yes, the paper that I sent you, and the tool that I’m interested in and think is important, is called VEMapper—Value Exchange Mapper—which has some sophisticated capabilities with respect to AI, mainly by calling the main AI engines into the conversation. There’s a process set out whereby, in a dialogue that’s captured both by recording and by typing, there’s a record of a conversation or a dialogue about value exchange.

I’ve carried out a few of them. I recommend trying it, because it’s quite remarkable. You really just tell your story, but it surfaces the tacit knowledge often that you’ve put to work in the creation and exchange of the value. The tool is also quite sophisticated today in terms of its databases and other components. Please forgive me, I’m not a technologist, but it creates a data commons. You, as a participant in a value exchange using this tool, your data, your output, is yours and yours alone. You own it. There’s a notion of data ownership and privacy, and as you carry out more and more of this value exchange, the way it’s captured—and again, I don’t really know about this, but I do know about the structure of the semantic web—it captures triplets: subject, predicate, object, which then makes them readable, makes them discoverable in knowledge graphs and other ways.

The tool also has a 3D knowledge graph. If you read that paper, it’s really following the logic, the reasoning, and the innovations that were introduced by Vint Cerf long ago in terms of how knowledge would work, whether there would be things like knowbots, which are agents, and so on. So it stores all of this, and then there’s a process whereby you enter into a dialogue. The AI coach helps you clarify, elaborate, and so on, and then you revisit this process. What this does is it builds and scaffolds trust between people and between groups or whomever is working on a problem.

Ross Dawson: Back to a broader frame here. So, what you’re describing—this tool or other tools—has been able to, as you state, capture or make visible value exchange in various guises, with the potential to shift to where we are looking and understanding far beyond the exchanges of financial or overt products and services, and so on. But we’re also relating it to Alex Imas’s thesis that we are moving into a relational economy, where the value—what is scarce—is not AI churning away on reasoning; what is scarce is human relation and judgment.

In a whole variety of exchange contexts, including in simple conversations or other knowledge exchange, they’ll be able to apply human expertise to people in situations and organizations. So perhaps, if we just marry those two, what do you see might happen if we move into both a relational economy with the potential to surface more of the nature of how value is exchanged?

Jon: Wow, that’s quite a question. I think it’s one of those things where there’s likely to be a very large and durable polarity emerge. I think that the polarity is that there will be some people—probably younger, I’m guessing under 45-ish—that will take to the new environment like ducks to water. They’re already living it in many ways. Their work is much more precarious. They operate in networks that are often networks of support and help, and so on.

I think the other end of the polarity is that there will be lots of people who are—I sent you another piece about a week ago called “Artificial Intelligence and Sleeping Humans,” which was about the fact that many of us are, whether we like it or not, not all that much awake when we’re walking around every day, particularly after we’ve been working for 10 or 15 or 20 years, and, you know, kids, busy life, and so on. As AI moves through the workplace, different industries, different natures of work, and brings up issues of relation and so on, I think that relational work will always be AI-aided and supported.

I think there’s a significant possibility of something emerging that currently I’m calling AI psychosis. I think that it will disturb a lot of people. They’ll try to build habits or create habits, and they’ll be trained for this with organizations with respect to using AI, but I think it will feel very foreign to them. I think there’s been something—you probably have talked about this before somewhere; I seem to remember reading something from you—but there’s been about 25, 30, 40 years of what I’d call atomization and augmentation in the social fabric. I don’t think that the introduction of AI on a widespread basis throughout work and everything is going to help with that atomization very much.

So I think that the longer-term, emergent impacts of AI—I don’t think they’re going to be about productivity and efficiency. They’re going to be up a level or two in terms of the discombobulation and ongoing anxiety that are created. That makes sense?

Ross Dawson: Yeah, yes, it does. I think most people can relate to what you’re saying. So, you were just saying before we started the podcast, you’ve, in a way, come back to your work. You’ve been reinvigorated by seeing some interesting shifts in the world. So, what are the next years for you? What do you think we should be thinking about? What should we be focusing on? What should we be creating to enable, as much as possible, all of this to go in a positive direction?

Jon: Again, a tough question. It’s so hard because these conditions are all swirling around us. But for me, 10 years—10 years, I’ll be in my early 80s. I don’t like to play golf. I like to swim, so I’ll probably still be swimming. I think we’ll see more and more evidence of the relational economy, with respect to wirearchy and my implication.

I’m going, in about a week, to Cambridge to start a creative residency there that involves a number of components. I’ll meet people with the Digital Futures Institute at the University of Bristol, some people at Cambridge. What I’m going to be doing with this creative residency is paying attention to and learning about improvisational facilitation. I think what’s going to happen, what I’m seeing happen everywhere, is shifts in what will be brought to work around the integration of AI.

I think the evolution of wirearchy, which implies a different kind of leadership and power, will mean there will just be more and more—how do I want to say it? What I’m noticing is that there’s an enormous amount of talk on LinkedIn and other places where people are wondering about similar things to what we’re talking about. They’re emphasizing the ability to listen, the ability to suspend judgment, the ability to allow the time and the space for emergence—a very, very different mindset than the predict, plan, execute, control, linear types of work.

This will be more circular. Many of the elements are already there. We’ve already seen in the last 10 years: develop fast, push versions out fast, fail faster—sort of recursive feedback loops. We’ll all be operating in recursive feedback loops, probably forever more.

Ross Dawson: That’s actually very central to my own beliefs.

Jon: Yeah, and we just—we have to get used to it. There’s an example I like. It’s not specifically apt for this, but I think you’d probably relate to it. Living in Bondi and in Australia, I presume you’ve gone scuba diving more than once in your life. There’s a kind of dive called a drift dive. Do you know what a drift dive is?

Ross Dawson: No.

Jon: Okay, I participated in one once, and it was really fascinating. At certain places, there are coral reefs where, I guess because of the topography, the current moves past it quite quickly—more quickly than you can swim against or manage yourself in. So if you go on a drift dive, the dive masters take you out, drop you in somewhere. They know how fast the water is moving, they know how much air you have, they know where you’re going to come up, so they meet you when you come up. But while you’re in the drift dive, what you do is essentially drift along the coral reef, watching the reef vertically because you can’t really swim.

I learned about that reading a book a long time ago called “The Horizontal Society” by a Yale Law professor. I can find the title and I’ll email it to you. He described that living in our media-saturated environment—and this was a long time ago—was like living in a drift dive. I think we’re all going to be living in a big drift dive for the next forever—well, certainly for the rest of my life. It’s really interesting to think about things in that way. It relates particularly poignantly to my quitting my job as a management consultant, where I learned all of the method with the generic Taylorism.

Because if you go back 20 years ago, the assumption—I know you’ve done a lot of strategic planning with companies and organizations—the assumption was that the next thing, the next time, and we get the strategy right, this thing is going to be stable. This is how it’s going to operate.

Ross Dawson: Yes, it’s a common fallacy.

Jon: Yeah, exactly. That wasn’t the case 20 years ago, and I started realizing it, and it’s much less the case today than it was 10 years ago. So, you know, I guess it’s like, get used to it.

Ross Dawson: Yeah. So where can people go to find out more about your work and what you’re doing, Jon?

Jon: At the moment, just LinkedIn. I’m going to put up a new site. I keep—another interesting, fascinating little story. I’ll do it quickly. I was over in England about a month ago, and there’s a guy, a friend of mine, whose claim to fame is, I think he built the first website in the UK in 1994. His name is Felix Velarde, and he’s run a number of agencies and is on the board of directors of a number of digital agencies now, as he’s gotten older.

When I visited him a couple days later, I said, “Okay, I want to build a new website. I want to develop a new website, and I have some ideas. But Felix, can you point me to—you know a lot of really talented people—to help me design my next website?” He said—we were on a Zoom like this—he said, “Hang on for a sec.” Started typing into Claude a pretty general statement of, “Give my friend Jon Husband—go scrape his website and blah, blah, blah, and give him an idea of what a good website would look like.” Enter. Wow. Wow, just wow.

I started playing with it, and I can do all sorts of interesting things. I can take the wirearchy graphic, I can embed that as a semi-opaque in the back. Anyway, just astonished. I don’t have it up yet, but I will have a new website called wirearchy.com in, I don’t know, about a month or so. I’ll try to put up a couple of my key pieces, but it’s mainly just going to be a landing page. I’ve decided that I don’t have any answers for anything, but I have, you know, 40 years of knowledge about watching organizations morph and change. So I’m going to really just offer half-day and one-day master classes. I respond to all sorts of different situations with different methods, done a lot of facilitation. I think facilitators and coaches are going to be very happy in this new era.

Coaching is really interesting. From what I’ve used—Claude, you know, a bit as a personal coach, haven’t tried the others—but I’m really impressed with what they’re going to be able to do, or already can do. Where coaching is going to become critical is at the higher levels, the top of the organization, because all of what we’ve been talking about—sensing, listening, allowing for emergence. The phrase I used to replace “command and control” was “champion and channel”: champion ideas, channel resources. See what happens. Does the node light up? Does the node wither? Does the node connect to other nodes, and so on. This is the world where I think we’re going to be living in, and coaches will be operating at the higher levels to help executives—who have typically been hard-charging and with mindsets they learned 20 or 30 or 40 years ago—helping them adapt, which will be critical.

Ross Dawson: Absolutely. There are many people who, for a long time, have been following and applying your insights, Jon, so I’m sure they’ll all be glad to get the update from this podcast and also when your website’s back up. Thank you so much, Jon.

Jon: Thank you, Ross.

The post Jon Husband on wirearchy, web weaving, the relational economy, and drift diving (AC Ep41) appeared first on Humans + AI.

“Freedom no longer exists outside the systems, and it depends on the design. Coming back to the design, it’s about understanding that we need to distinguish between intelligent systems and agency.”

–Dr Michael Gebert

Dr Michael Gebert is Chairman of the European Blockchain Association and co-founder of AI Expert Forum. He works at the intersection of artificial intelligence, digital sovereignty, and institutional responsibility. His book 2079 – Designing Freedom is just out.

Website:

2079.life

LinkedIn Profile:

Dr Michael Gebert

Ross Dawson: Michael. It is awesome to have you on the show.

Michael Gebert: Hey, great to be on the show. Thanks for having me.

Ross Dawson: So we connected first, probably around 15 years ago, and we were both involved in crowds, creating value from many people. And I think, you know, there’s one of the interesting points now is, I guess, you know, we still live in a world of many people. We’re trying to create collective value. AI is laid over that.

So it’s interesting to see that journey from where we’ve come to where we are today.

Michael Gebert: Absolutely, and I really remember visually when we first had contact about this very exciting topic of crowdsourcing and empowerment of the crowd, and really making people believe, not only in themselves, but really in communities. And therefore, not only strengths in terms of crowdfunding, crowd investing, their financial gains, but also being empowered in what they do. And this is a very fundamental, I would say, even a right for humanity to reflect on and do that.

I think the methodology and technology back then helped a lot. And to be honest, I’m still partly involved in some of those efforts. Even the big crowdfunding platforms, also here in Europe and in Germany, are vital and really active. Of course, not in that dramatic media shift hype that we experienced, but they’re still there, and it proves that it’s a concept that should stay.

Ross Dawson: Yep, absolutely. You know, there’s obviously collective intelligence, amongst other facets. But this goes to, I think, the frame of your new book, 2079, Designing Freedom. So freedom is an interesting word, and something which I hope we all aspire to.

Michael Gebert: Yeah, you know, freedom, of course, is one of those very multifaceted words, right? It could be translated in a political context. It could be translated in an economic concept, meaning monetary-wise. It could be translated—and this is my translation—in a very personal, one-to-one reflection about how do I as a human being see myself in that surrounding, bombarded not only by information but by intelligent systems, basically AI as we describe them, and all that is behind those systems.

Ross Dawson: So there’s a few things I want to dig into here. And I guess there’s another word there: designing. Obviously, at a societal infrastructure layer, we want to be able to design the systems whereby we can all individually have that freedom of choice in how we live our lives.

Michael Gebert: Yeah, and not always, I would say, looking at the world geopolitically, of course, there is sometimes no choice. And if you are able to generate those choices, first of all by understanding how to design them, that’s a very good first step. So when I wrote the book, the prior part was basically a research paper I did, a small research paper also on ResearchGate. This is the foundation where I started thinking and reflecting. Basically, the core there is about a question that I think is becoming unavoidable now and for the future.

The question is: if more and more cognition or judgment and action are delegated to intelligent systems, what has to be true for human beings in order to remain genuinely free? So the book is really about freedom, agency, responsibility, and at the end, about belonging in a world of increasingly disruptive intelligence.

Ross Dawson: Yeah, yeah. So the word agency is obviously very much of the moment, in lots of ways. But I think human agency is absolutely critical. One of the central things you lay out in the paper, which I think is really, as you were saying a moment ago, is on everyone’s minds. You’re saying this idea of agency used to be about making decisions, whereas now, as you describe it, agency is shifting to authoring the conditions for decision making. So we’re not necessarily making the decisions ourselves, but we do control and guide the conditions, the context, or the structures for decisions so that we retain responsibility and accountability, and those decisions are the ones we would want. So how do we do that?

Michael Gebert: Yeah, you know, the question before asking how is really to understand under what conditions do human beings remain authors of their lives when more and more of those decisions are shaped by, as you say, agency systems or whatever name they go by, whether fancy, new, or already existent. So the how—and it’s not about lifting a secret—is about going back to cognition and having that cognitive intelligence and cognitive roots, which are in us, but which, over the years—and you reflected on the last 15 years, especially the generation after 2008, meaning after the iPhone—have lost large parts of that ability, which is very human.

So it’s not really a reshaping or something new. It’s also not a book advising how to; it is really a finger going up and saying, people, please remember that the deeper question is under what conditions do human beings remain genuinely free when more and more cognition, judgment, and action is to be owned back and not delegated to the systems. This is, of course, very formal in the need and in the demand, but especially, as you mentioned, when laying it out into organizations or government structures, it is hardcore policy and hardcore principle. You can write a lot of things in your genuine AI policies, but what I see right now is that in reality, first of all, nobody’s really reading them in depth. Secondly, there is really no reflection point on this cognition, judgment, and delegation. Therefore, this is really prior before any interest in how-to in terms of technology and what LLM to choose. This is really prior—it’s day zero—when you think about what’s going on, and when you think about how to position yourself, your company, and your team in there. Then this is the next step of thinking.

Ross Dawson: So I want to come back to that, but I think one of the phrases you use is cognitive sovereignty, and this is in a context where one of the most shared papers recently is around cognitive surrender. Cognitive sovereignty is the opposite of cognitive surrender. But the reality is that in interacting with LLMs, it does change our cognition.

Michael Gebert: As long as we, yeah, as long as we delegate cognition, basically. The auto effect is—

Ross Dawson: Conversation with a human changes our cognition too, and I think we need to recognize that. So it’s not just conversing with LLMs. Conversing with a human changes the way we think, which is a good thing because we’re getting more diverse opinions. But obviously, LLMs are not humans, and while possibly that interaction could enhance our thinking, if we get some great ideas and different perspectives from an LLM, then we’re still retaining cognitive sovereignty. So let’s frame this: how do we as individuals get to cognitive sovereignty? What does that look like?

Michael Gebert: Yeah. So first of all, I think we need to understand that when we delegate cognition to an AI, we redesign responsibility. This is undisputably non-negotiable. This is a fact. When you compare it to a human interaction, there is no default responsibility redesign necessary. It’s a reflection point, it’s a discussion. If it’s a good conversation, it’s uplifting for both ends. You go out of this conversation and you have, yeah, uplifted cognition.

Surrendering cognition, as you said, is a very factual statement that brings a lot of views, but it’s basically raising the white flag and saying, I surrender. What I say is, no, it’s not time to surrender. It’s time to appreciate, and it is time to understand that freedom no longer exists outside the systems, and it depends on the design. Coming back to the design, it’s about understanding that we need to distinguish between intelligent systems and agency. We need to separate the capacity for governance. Therefore, we should distinguish between formal freedom and substantive freedom. The difference there is that there are two parts: assistance and substitution. Understanding that there is a very important difference, and really feeling that difference personally with input, makes it powerful. When we think about AI and all those systems, we often confuse optimization with empowerment, and this is one of those very dangerous paths.

Even, you know, you are very active on LinkedIn, I’m a little bit active on LinkedIn, and we see all those posts. To be honest, I would say since the start of ChatGPT and all the other LLM models, 80–90% of those posts and comments are now AI-driven, and you see it, you read it, once you’ve been longer on those platforms. Therefore, people think they feel empowered, but it is not empowerment. It is maybe optimization, but it’s not a reflection point. Coming back to your core question of cognitive sovereignty, cognitive sovereignty would be really going back and abstracting and saying, all right, AI can absolutely expand human possibility, but it is hopefully about human potential and not about completely outsourcing and empowering the systems.

Ross Dawson: So, so what? Let’s just—what does an individual do when they’re working with an LLM? What are the practices that enable them to retain cognitive sovereignty?

Michael Gebert: Yeah, I think, first of all—and this is, of course, a lot of work—every output of any system is a suggestion. Treat it as a suggestion. Compare it to a conversation: if you have a conversation with a very wise person, very reflective, very well known, normally you don’t instantly believe what’s coming out of their mouth. It depends, of course, on your dependency on that person, but normally, you reflect.

What we see right now is a dramatic shift towards instant reputation and instant recognition of AI output. Even though I’m not a skeptic about augmentation, I’m skeptical about unexamined delegation. That means there is human flourishing everywhere possible, but it does not emerge automatically from capacity. This is the reflection point, and it is, as I said, not easy. It’s a routine. It’s basically a self-delegated routine, saying, all right, this is the output, that’s interesting. Maybe it’s misleading. Maybe it is another opinion. Maybe it really substitutes my argumentation. It feels like empowerment, but at most it’s optimization.

Ross Dawson: So, you know, obviously this requires that metacognition, as in, to be aware of your own thinking processes, individually and with the machines and with others, and at which point you can start to observe and reflect.

Michael Gebert: It’s, you know, Ross, to be honest, it’s hard work. Because in the daily life, for a regular person at work, there’s time pressure, social pressure, work pressure—there’s a lot of pressure. The core motivation for most companies is efficiency: to integrate AI and AI systems to be faster, easier, leaner, to make more profit. So the human factor is not in the center. We learned that also from crowdsourcing and crowd intelligence. My PhD about crowdsourcing integration in companies many years ago was about the same reflection: once people have those pressure points triggered, then the reflection within that, that is needed as we talked about, goes down massively.

So the things that are coming now, historically and consequentially, is that the whole AI should not be a technological footnote. It should be really a core issue, to integrate that cognitive sovereignty, and out of that, basically the designing process—what I call now freedom—is ongoing. Because it’s kind of then on auto-shift at some point. But really, there are a lot of stakes that become reasonable here in the Western civilization and in our civilization. So it’s not about tools. The point is at which a tool becomes an environment. This is really what I think a lot about, and it is mind-blowing on the one hand, and on the other hand, really frightening to see, as you say, also the opposite that is happening.

Ross Dawson: Yep, yep. So we’ll come back to that. We’ve still been talking about, in many ways, these decision structures. So, I guess, in an organization, let’s say a head of transformation or CEO says, “Okay, we need to move to what I call humans plus AI decisions,” where humans are involved and AI is involved, and we get to decisions that may be better, faster, cheaper, but also still retain governance, meeting your ethical and compliance requirements, and that the humans are accountable. Of course, there are many types of decisions, and so that will play out in different ways across different types of decisions. But what is the process for just thinking through and implementing those decision structures or conditions whereby you can have better decisions while still maintaining that control or freedom, as well as accountability?

Michael Gebert: Yeah, first of all, I think the real leadership challenge is not just to deploy, right? It’s about preserving agency while doing so. This is the critical factor. I don’t know if you can recall in history, but from my understanding, it’s the first time that we have this hyper-integration of AI usage in both private and commercial business environments. There is no real cut, meaning that the person, the human, is using AI systems privately—shopping lists, optimization, planning, automation, personal agents—and it’s used in the company. Therefore, two things should happen structurally.

First of all, the reflection on how to integrate cognitive sovereignty has to be ramped up, learned, taught, and really developed within the organization. Optimal would be beforehand, but to be realistic, while deploying AI with that knowledge, this is a training program. So how is it? It is a training program. I know that you are a fan and you have superb pictorials and structural views that you post on LinkedIn, and this would be a perfect example of producing such a roadmap, basically saying, “All right, these are the basic steps. You may not be able to follow them 100%, but just to give you a core idea of step 1, 2, 3,” and then follow the roadmap, a framework. But now, with the difference that as it is so integrated, the person understanding the framework can reflect the framework also for their private lives, meaning with their children, godchildren, partners. This is why it’s so interesting, because it’s core learning.

Right? So basically—and I know you have a couple of those already in existence—so it’s kind of the next step. What should come out, or should be produced, is a combination, saying, “Okay, this is the addition to that framework, in combination with that framework, understanding what myself and others try to explain here.”

Ross Dawson: Fantastic. I interrupted you, and you were at the point of saying, okay, this training or these frameworks are assisting people to have agency in this process. Let’s come back to that. You’re helping people to frame or to have agency themselves, but this is part of a process where you are starting to bring AI into decisions. So where does that take us?

Michael Gebert: It takes us to a very fragile and really hard-to-judge state where we are at the moment. I just can really reflect on my experience right now with training and with conversations within organizations—not just because maybe the book is a foundation, but because I’ve been doing that for the last 30 years. Having that reflection point, I would say it has never been easy to have a disruptive framework implemented in a running ship. The company is moving. There are goals. There are different goals. There may be goals that are totally the opposite to what the framework says. But realism kicks in very easily. My first door opener is saying, if you as a company want in a possible future to integrate human potential into your upcoming company framework, then we have to talk and put a framework about cognitive sovereignty and understanding of systems of agency into your existing and upcoming, mediated, intelligent systems. Otherwise, if that is not understood, then we will have a dependency on decision, which is not only bad for your employees, but in the medium term, maybe even in the short term, depending on where you integrate the AI systems, can be very destructive for the whole company. This understanding is a massive shift from a regular decision, which is mostly still coming out of the technical department—meaning the CTO or the CIO are fascinated by the possibilities, they report it to the board, the board sees efficiency, and out of that, a testing period and pilots are developed, and then the rollouts begin. Which is all fine in the old thinking, because it doesn’t price in what’s happening on the cognitive and human potential side. So it’s an additional card that has to be integrated very early on.

Ross Dawson: So are there any organizations that you have seen who are doing any of this well, or even just a little bit well, in terms of even just taking this framing into how they’re trying to approach it?

Michael Gebert: You know, in general, I would say there are a couple. I have one from a city company who is worldwide active, who is doing, on a department level, a very good job. Generally, overall, the whole company is fragmented, and therefore decision making is fragmented. Therefore I cannot really judge on how they are doing as a whole, as a company.

Ross Dawson: Just on the department. If they were doing it well, what were they doing?

Michael Gebert: In that specific company, they understood—and maybe that is the interesting part—they understood relatively early, due to the fact that they are coming from a very human-side factor of product, meaning pharmaceuticals. Because whatever you take in, a pharmaceutical elevates or alters your human condition, and therefore they have this sensitivity for the topic very early on, which made it very helpful to attract attention and also understanding within the leadership and decision making to integrate, in the development and R&D departments for future potential aids and medicals, that thinking. Which I think is perfect and fascinating and it fits, but the foundation was a preset of basic understanding which is bounded to the product, or bounded to the industry itself. The other one was automotive. You know, I’m in Munich, so there are, and in Germany, there are still a couple of automotive companies left, and they understand that there is a big shift on robotics, FSD, and there is the other shift of human-centric driving. But still, in the car is a human person, so somebody has to be transported from A to B. The department there on AI and future development understands this cognitive sovereignty also very well, because their approach is coming from a very human angle. What I want to say is, it benefits a lot once you have that framework integrated into existing acceptance of the importance of the topic. What I found is that especially in the financial sector, it is, at the moment, not really recognized. It’s very product-focused, very output-focused, very efficiency-focused. It’s not really focused on preservation of human intelligence and reflection and agency, and therefore, you know, designing their cognitive sovereignty—aka freedom. I think that will fall back massively, but we will see. This is just a reflection point now in Europe, or especially in Western Europe, like Germany. But the similarities appear to be there on a global scale, because the systems tend to be very similar that are being used.

Ross Dawson: So which kind of just takes us to round out, the big picture. Your book is for, amongst others, policymakers, and we’ve talked about the individual and organizational level. So now pulling it up to the macro level, as those who are creating the policies for governments and supranational organizations and so on, what are just a few core lessons or insights for how we design policy to enable human freedom, agency, and dignity?

Michael Gebert: Yeah, maybe I’ll give you some really concrete examples, because I presented the book this year in Davos at the World Economic Forum. I had a reading session there. Of course, it’s kind of a competition between giants, so I was humbled to have a couple of people there, but not as many as I wished, to be honest. Still, I was there talking to a couple of those macro-level, high-end policymakers, and what they said is very similar to what I heard back in my crowdsourcing research: they have the data, they know the importance, they sometimes even have a hint of a framework to do it. However, inside the rollout pattern and inside the organizations themselves, there are a lot of—not risks, but—hindering mechanisms that tend to prevent an instant understanding. What they sometimes do—and this was a gentleman, interestingly enough, from a country in Africa—he said, “We need to have, like in the old days, ambassadors of freedom within the organization at all levels.” Basically, they are the spearheads, they’re the flag keepers and the wisdom keepers, in a very front-end way, understanding the core concept and elevating the rest of the crowd, of the team, to a level where they are open to discuss, understand, and integrate. This, I think, was one of the most hands-on approaches I’ve heard, because all the others about training and retraining and certification—it’s all good, but it doesn’t really guarantee integration.

Ross Dawson: Yeah, yeah. So, Michael, where can people go to find more about your work and your book?

Michael Gebert: So, basically, if you have a ResearchGate account, the free prelude—the research there—can be downloaded for free. It’s a PDF. I would be happy to extend or expand it. If there are researchers or organizations out there that want to use that as a foundation or expand it to their special needs, I’m more than happy to assist. The book itself is at 2079.life. It’s a dedicated website for it, and you can buy it, of course, online or from any dealer that you want. Interestingly, with that book, I really have lifted it to a hardcover version—not that I’m old school, but I think there is something about seeing it physically, marking it. I’ve seen it now, when I did the promotion, I gave it to a couple of people who normally don’t really read so much because they have audiobooks or PDFs and a lot of work but no time. But with that book, they came back to me and made photos where they really underlined things, marked it, put their reflection points. I think this is what this book is about, because it’s not a 300-plus page book. It’s quite condensed, but it should bring you, in basically every paragraph, to rethinking about your approach to the topic. When that is reached, the book is 100% where I want it to be. It’s definitely not a how-to book—how to be great, or “in 30 minutes you’re an AI prompt magician,” or anything like that. It’s quite the opposite. It really goes way deeper. A lot of books kind of flag it at some point, but not in that condensed area. As you may have read, there’s no version 4.0. When I started thinking about it, it was COVID times, and the first version I gave to you has nothing to do with the current version. The first version was a blue pill, red pill approach—really, there will be a dystopian version and there will be a freedom version. Over the years, now in the fourth year after COVID, with all that’s happened on the technology side, geopolitical, and human side, this is the output now, a development. So the book itself is not a still space; it is a development space.

Ross Dawson: Fantastic. Well, thank you so much for your time and your insights on the call today and the very important work, because obviously freedom is something which we need to work on. Thank you, Michael.

Michael Gebert: I think that’s the core. Thank you so much, Ross. And have a great day. Thanks for having me.

The post Michael Gebert on designing freedom, human self-determination, cognitive sovereignty, and systems of agency (AC Ep40) appeared first on Humans + AI.

“The technology we’re working with today really makes a lot of those best practices and mental models and the whole toolkit more accessible than ever to more people.”

–Marshall Kirkpatrick

Marshall Kirkpatrick is founder of sustainabilty consultancy Earth Catalyst and AI thinking tool What’s Up With That. His many previous roles include founder of influence network analysis tool Little Bird, which was acquired by Sprinklr, where he was last Vice President Market Research.

Website:

whatsupwiththat.app

LinkedIn Profile:

Marshall Kirkpatrick

Ross Dawson: Marshall, it is awesome to have you back on the show.

Marshall Kirkpatrick: Oh, thank you, Ross. It’s such a pleasure to be reconnecting with you here. Thanks for having me on.

Ross Dawson: So back you were very, very early on in the podcast when it was Thriving on Overload, and it was interviews with the book, and you got incorporated—some of the wonderful things you were doing in Thriving on Overload. So I think today, in this world of generative AI, which has transformed everything, including the way in which we think, the Thriving on Overload themes are still super, super relevant, and in a way, we need to be talking about them more.

That theme at the time was finite cognition, infinite information. How do we work well with it? I don’t know if our cognition has become more finite, but the information has become more infinite, and there’s just more and more. But also, it cuts two ways, as in, what is the source of all the information? AI is also a tool. So anyway, let’s segue from some of your cognitive thinking tools, technology-enabled cognitive thinking tools and so on, which we looked at. So how do you—where are we? 2026, what do you think about human cognition in our current universe?

Marshall Kirkpatrick: Well, especially when you frame it up in Thriving on Overload terms. I mean, those were four, five long years ago that we last spoke, and the book that came out of it was just fantastic. I think it has some timeless qualities, and I think that the technology we’re working with today really makes a lot of those best practices and mental models and the whole toolkit more accessible than ever to more people. That’s what I hope.

I think that, yeah, between individuals and organizations, there’s so much that, historically, someone like you or me or the people closest in our networks were willing and able to do and excited to do, that many other people said, “That sounds like a lot of work.” The bar is lower now, because a lot of just the raw cognitive processing can be outsourced into a technology that serves as a lever.

Ross Dawson: Well, I mean, that idea of levers for these cognitive tools is interesting. I guess, the very crude way of saying it is, we’ve got inputs into our human brain, and then we are processing information. I’m just thinking out loud a bit here, but it’s like, okay, we have tools to be able to filter, to present, to find what is most relevant, to present it to us in the ways which are most useful—very obvious, like summarization, visualization.

Then as we are processing it ourselves, we have dialog, or we can have interlocutors who we can engage with and be able to refine and help our thinking. Does that sort of make sense, or how would you flesh that out?

Marshall Kirkpatrick: Yeah, I mean, when you put it that way, it makes me think about Harold Jarche and his Seek, Sense, Share model, right? I think that AI, especially when connected to things like search and syndication and other traditional technologies, can impact all three of those stages. It can hypercharge our search. I think the archetypal example of that, on some level, feels like the combinatorial drug research being done, where just an otherwise cognitively uncontainable quantity of combinatorial possibilities between molecules can be sought out and experimented with for a desirable reaction.

And then that sensing, or the pattern recognition that AI is so good at, is something that we do as humans—some of us better than others—and it’s a lifelong muscle to build and what have you. But the AI is really, really good at it, and so it’s a ladder to climb up in some of that sensing. And then the sharing component becomes so much easier with the rewriting capabilities—turn A into B, reformat something into a summary or a set of bullet points, or ideas and words into code. AI is just so excellent for that translation that makes new levels of sharing possible.

Ross Dawson: That’s fantastic. Yeah, I had Harold on the show again in the Thriving on Overload days. But you’re right, that’s extremely relevant. Let’s dig into that. I love that you brought up that combinatorial search, which is so important. As opposed to going into Perplexity to do a search, it’s far more interesting to find the uncovered connections between things, which are relevant to what you’re doing. And that’s—

Marshall Kirkpatrick: Absolutely. I remember reading, years ago, Dan Pink’s book “A Whole New Mind,” which preceded the generative AI era. But he said, if your kind of work is something that’s easily reproducible by computers, good luck to you. You really are going to need uniquely human practices in the future, and what exactly those are, I’m not sure, because the one that he identified, I don’t think has proven to be uniquely human.

But I really appreciated learning about it from him, and that was what he called symphonic thinking, or the ability to draw connections between seemingly unconnected phenomena. So for many years, I have been doing a personal exercise with pen and paper that I call triangle thinking, where I’ll take three different phenomena—maybe that’s the owl outside my window, one of the notes that I’ve taken on paper, and something I come upon on the internet, or maybe it’s three very deliberately related things. I label them A, B, and C, and I ask, what might A have to say about B? What might B offer to A, and vice versa? I write out the six unidirectional connections between those things. And without fail, one, two, or three of those end up being real keepers, where I say, “Aha, that’s a really interesting idea. I’m going to take action on that.” And now, by the time I’ve got the letter B written out, an AI has done that ten times over. I like to do it both ways—still both AI and with my naked brain—but that combinatorial ideation, the generative combinatorial ideation, is, yeah. I’m curious what your thoughts and experience and hope for that might be.

Ross Dawson: Well, there’s a prompt I use called “Apply Diverse Thinking,” where it generates extremely diverse perspectives on a topic—who might those very unusual people to think about something be, and then what would they think about this particular situation? Of course, there are a whole array of different thinking tools. There’s Marshall McLuhan’s tetrad, which is a little bit similar to your thing where, again, you can and should do it—well, not manually. What’s the manual equivalent of brain?

Marshall Kirkpatrick: Thoughtfully, perhaps. Yeah, good one—deliberately, manually. I mean, Azeem Azhar over at Exponential View uses a fountain pen and paper and will sometimes have his team come online and they’ll do two-hour thinking sessions with no AI allowed. They just get on, I believe, Zoom, and just think through things with pen and paper, individually and together. And then they’ll kick off OpenAI or what have you, and use all the tools afterwards.

Ross Dawson: Yeah, well, a couple of things. Actually, research has shown that in brainstorming, it is better for everyone to ideate individually before doing it collectively. And of course, that’s unaided.

I think there are analogs there where—actually, one of the frameworks I just released last week was basically to say, think it through for yourself before you ask the AI, because then you have a reference point. If not, you don’t have a reference point to say, “Well, what am I expecting it to do? Let me think it through for myself,” even if it’s just a little bit, as opposed to just going in blank—”All right, give me an answer.” Just that simple thing of thinking through for yourself first is enormous. What it does is, obviously, give you a reference point for that.

And I’m going on a lot about appropriate trust at the moment—as in, trust the AI enough, but not too much, which I think is absolutely critical capability. And part of it is being able to say, “Well, this is what I think it should be giving me.” Now you have a reference point for what it gives you.

Marshall Kirkpatrick: Yeah, that sounds great in many cases. I do think that’s the right tool for the job in a lot of places, but not necessarily all. I’m thinking of the Iron Triangle of product management—fast, cheap, good, pick two. On some level, just handing the AI the keys for certain decisions is uniquely fast and cheap, right? And maybe it’s good enough.

Ross Dawson: Oh yeah. Well, you’ve got to choose your battles, because if you’re now doing ten times what you were doing last week, then maybe for a tenth of those you can do some thinking before you delegate it to the AI.

Marshall Kirkpatrick: Yeah, a strategy for how to do that. I think, well, that sounds important—some checkpoints along the way, some random selection of testing things.

Ross Dawson: Well, that’s interesting. One of the critical things people talk about with AI model oversight is sampling. As they say, “Okay, I’ve got 1,000 outputs—I’m going to take 20 of them and check how good they are.” You’re not checking every output, but you’re doing some kind of ongoing sampling.

Marshall Kirkpatrick: Are you checking with your own deliberate brain, or are you checking with another AI?

Ross Dawson: It could be either, depends on the case—how critical it is. This comes back, of course, to the fact that accountability is only human, and so the human who is accountable has to make that decision: “All right, I’m happy for another AI to check it,” or, “Actually, I want to go in myself to see.” And that’s a judgment call.

Marshall Kirkpatrick: Totally. And it feels like a process design issue and a personal accountability matter. I mean, “The AI made me do it” is not a viable excuse.

Ross Dawson: Let’s hope it remains that way. So, good for those Seek, Sense, Share stages. Sense is one of your superpowers, both in the way you think and also the way you use the tools. It’s probably worth introducing—now you’ve just released this wonderful product called What’s Up With That. So just tell us about the product, but also, I want to go to the bigger context of sense—sensemaking, how we use it generally, how AI can use that, and your role with the tool in that.

Marshall Kirkpatrick: Yeah, you know, I think there are so many different ways that sense can be made of anything, so many different ways that anything you read or think about or do can be put into context. It’s just overwhelming. I think we all have our favorite—not all of us, but those of us who are into this have our favorite tools, our favorite ways to—you know, a lot of people will think about something in terms of its past, its present, and its future, or they will break it down in analysis into parts, or they’ll synthesize it together with other phenomena and see how to understand.

I think sometimes of the famous Donella Meadows quote, the mother of systems thinking, who said, “Systems thinking isn’t any better than analytical linear thinking than a telescope is better than a microscope.” So there’s just a superabundance of fascinating, powerful tools that all provide different views on anything we’re trying to make sense of. One of the things that I’ve always found a lot of joy and usefulness and power in is learning about new lenses and processes and tools. Now that generative AI has put the ability to develop software into my hands—instead of having to go and hire someone else to build that software—I have built a system that takes as many of those different models and lenses and processes for making sense of something as I can.

I mean, it would be trivial to pull up a list of 200 mental models. I might go visit Shane Parrish’s website and The Knowledge Project. I think of ones that would be particularly useful, like, “Tell me who the intellectual predecessors are of this thing I’m reading,” or one of the other capabilities inside of What’s Up With That—my favorite, probably, is a combinatorial one called Fertile Edges. That says, “Take what I’m reading right now, identify the topic that it is a constituent of, and then find other adjacent topics where innovative people have built bridges between those adjacent topics and what I’m reading about, and tell me who those people are.” And that’s really fun. So I have built this sensemaking system, and that’s a part of What’s Up With That.

There are really three parts to it. The first is, it analyzes whatever you’re reading or watching, and it pulls out the net new, truly novel, most notable elements. Yesterday, I was telling you, it was a little bit inspired by the US military intelligence guideline that says, when you’re writing up a report about something, focus on what’s new in that situation—tell us what we don’t already know. That’s the first thing that What’s Up With That does. It says, “All right, here’s what’s new in this document relative to its field,” because we just drew a real-time map of the state of the art, and we say, “Okay, here’s what’s really novel there.” The second thing that it does is that toolbox full of all the different mental models and lenses, and it recommends a sequence. One of my favorite books I ever read was “On Grand Strategy,” about strategic thinkers throughout history, who talks about the significance of thinking in terms of sequences of actions. So now, What’s Up With That will say, “Here’s a sequence of analytical lenses we recommend that you subject this document to,” and with a click, it’ll go and do that for you—it’ll do that cognition for you and then just give you a report.

The third thing that it does is probably—it, the shorthand for it is compound learning. You don’t have to remember all the things that you read anymore, because our system extracts the causal claims from everything you read, archives them, and then compares everything you read in the future that you analyze with our system to your library of causal connections in the past, to say, “Whoa, we just found a chain of claims that could surface a multi-step risk or opportunity that’s relevant to your work.” We do that both for your data exhaust—your history of things you’ve analyzed—and we do persistent monitoring of the web to detect anything that could be relevant to a project or chain by that same kind of symphonic synthesis and connection. So those are the categories that it has.

Ross Dawson: Yeah, I think you’re only scratching the surface of what your tool actually does, and obviously, more generally, these are just pointing in wonderful ways to how you can go beyond saying, “Tell me about this, ChatGPT,” to some far more nuanced ways of getting AI to do it.

Marshall Kirkpatrick: People have had the same challenge with Google, historically. Google has struggled with that, to figure out—”I’m feeling lucky” was probably the first intervention in a novice, beginner’s mind, coming to a hyper-complex opportunity space. Even still, now, 20 years since Google launched, I feel like you can tell people that they can search for “site:domain keyword” to find instances of that keyword not in the web at large, just inside that specific domain, and most people don’t know that. It’s a simple power, and there’s a bunch of things like that.

So figuring out how to unlock—and I don’t know how much they’ve even worried about it, because they’ve got that cash cow of advertising—but people don’t even recognize, sometimes, whether they’re clicking on an ad or a search result. In polls, when people are asked, they say, “No,” even if they put the ads at the top or mark them as ads, or a bunch of stuff they do do, but nobody notices. So that interface of complexity and accessibility and scale—we’re in it again here now, in this generative AI era. There’s so much more that could be done than is immediately obvious. It’s a real challenge.

So I’ve taken the approach that I have, which is to roll up a bunch of that and turn them into buttons and recommend them automatically and try to recommend them just in time, and stuff like that. But I’m sure lots of different people are going to try to respond to that gap of simplicity and complexity in different ways.

Ross Dawson: Yeah, that’s—which comes back, I think, a little bit to, you know, I firmly believe that the heart of the future is interfaces. We have these extraordinary capabilities—against finite cognition and infinite capabilities, let’s call them. That’s very much to the individual. The adaptive interface, I think, is going to be absolutely critical. All right, well, it’s after lunch and I’m not feeling so—the interface adapts to you.

Marshall Kirkpatrick: So I heard you say that.

Ross Dawson: The interface adapts again.

Marshall Kirkpatrick: Right? I heard you say that in a conversation with Ramez Naam some time ago. I was listening to that interview that the two of you did together while I was playing hacky sack out in front of my house. I grabbed my hacky sack and I said, “I’ve got to go inside and do something about this idea of Ross—yes, interface variability.”

In that case, I did a little experiment that I didn’t implement because I decided not to, but the general idea I want to pursue further, and I’ll tell you what that experiment was. One of the capabilities inside of What’s Up With That is that you can get a reading review synthesized, so that instead of just a list of links, you can get a narrative document exploring the themes, weaving together the last ten articles that you’ve read, and it’s easier to remember and to think about. I decided to hit the Nanonets API and have an image put up at the top that illustrated the themes. Now, maybe it’s just because I read a lot of dystopian AI, authoritarian politics type of stuff, but the images were terrifying, and they’re kind of expensive and slow, and they also look kind of repetitive. I was like, “All right, Ross, I haven’t cracked that nut quite yet in the variable interface, but I think you’re really on to something there.”

Ross Dawson: I’ll try to work on that too, a little bit. So coming back to this wonderful thing we laid out, alluding to some of the wonderful ways we can use for really rich investigation of ideas and how to think. It comes back to this frame of mental models. All of us get our mental models from the moment we’re born—we get this understanding of the world, which is hopefully useful. Sometimes, some people’s mental models are not very effective in guiding them in how they work. Our role is to continue evolving, getting better. I call it enriching mental models. Back in my first book, I talked about that, and of course, that’s in the context of the world changing, so mental models can’t be static anyway.

In a way, what you’re pointing to is the many, many ways in which we can, at one point, improve our mental models. All right, I understand this linear lineage of thinking, and I can see the strands between that, and these neurons are connecting in my brain in some form. But how can we pull to that bigger picture of all of this lattice of things to be able to say, “All right, I am actually thinking better through these interactions”?

Marshall Kirkpatrick: You know, I think that there is a visceral sense—a sense of safety that can come sometimes when a new mental model illuminates a risk that you hadn’t considered before, and you breathe a sigh of relief and say, “Oh, thank goodness, I can now account for that.” And there’s an excitement with opportunity. There is something about a collective greater-than-individual opportunity here, because it’s tempting to—I’m not sure what that looks like, but I feel like there’s some social and interpersonal and network-based.

One of the other things I do is build systems for network self-awareness, to build metacognitive network monitoring kinds of systems. I feel like there are mental models on that level as well.

Ross Dawson: So I’ve got to dig into that—metacognitive network monitoring. Explain

Marshall Kirkpatrick: Yeah. So every one of us, and our organizations, exists in a network of customers, suppliers, competitors, regulators, thought leaders, with orbits that extend out. The signals are strongest in the closest ones, and perhaps they are weaker and harder to hear, but really significant coming from outer orbits—even from other industries or other topics. It is overwhelming.

It is cognitively uncontainable for any of us to keep up with all the work being done, all the thoughts being shared, all the new developments and opportunities from all the different entities that we’re interconnected with. One of the other offerings that I build for organizations is a system where I go out and map as many of those as possible with people. Those might be your target accounts you’re wanting to sell to, or your peers in a community of practice. Then I set up systems, basically using RSS, email newsletters, web page change notification—the technical underpinnings—to say, especially when organizations are—there are some forms of communication that organizations do naturally by default, and those tend to be speaking to their own customers.

If you can listen to what organizations are saying to their own customers at scale, you can pull in a large quantity of signal, and then the challenge is to winnow that down into just the filtered signals that are most relevant to your priorities. I’ve got a system that uses AI to do that. Then there are combinatorial possibilities as well. I’ve started merging that in with What’s Up With That now, for example, where when we’re watching your broader network and a signal gets picked up on the back end, we’re generating hundreds of possible scenarios for that signal to intersect with your work and projects and priorities, and then we’re filtering to say, “Yeah, but tell me just the subset of these that are most significant and imminent and actionable and interesting.” If there’s something, then we will alert you and tell you what’s going on. Otherwise, you never hear from us, and you just go about your business.

But a couple times a day, I get alerts. Yesterday I got an alert that said, “Hey, one of the founders of Manus, the AI platform that Meta just acquired for $2 billion, just got detained in China trying to go back to Singapore. Given your interests in AI and anti-authoritarian politics and the infrastructure battles around AI, we thought you might want to know about this.” I said, “Thanks, What’s Up With That, I really appreciate it.” That’s an example of the sort of thing—so that’s how I do it. Other customers will take that and use it to populate a podcast or a newsletter, and do both an intake and an output as a conduit of that kind of network self-awareness.

Ross Dawson: Yeah, well, as you know, my kind of—my metacognition is my mantra. I think one of the key points is this simple question: How can AI assist me in getting to a point of metacognition? I would argue, if we use AI even vaguely well, it’s already doing that, because you’re saying, “Okay, well, let me think about what I can do and what the AI can do,” and you’re starting to think of that system. The only thing that enables this humans plus AI is metacognition, because you can actually see above and see your role and the AI’s role. I think this broader question of saying, many of the things you’ve been talking about are how AI is helping us to get to a point in metacognition.

Marshall Kirkpatrick: Ross, can I ask you a question adjacent to that? I think I am not the only one who wants to know, perhaps—and maybe this is a trade secret, I don’t know—but how you think about your analysis and sharing of scientific research papers online? You’re so good at that, and you do a lot of it, and it’s really valuable. It comes to my mind when you talk about metacognition—what role does that function, what are you doing there, what role do you see that playing in this bigger conversation?

Ross Dawson: Well, I’ll just tell you the mechanics of it, which might partly answer your question. I go into, often, three or four of the AI engines, including Grok, actually, because it’s very good at search. I say, “Tell me the most interesting research papers in the last few weeks,” whatever—on, I might say, human-AI collaboration or AI and strategy, whatever it might be, just different frames. Then I go and look at them. To be frank, I probably should do some more filtering with AI and tell them, “Only from reputable authors,” etc., because I have to just look at a lot of stuff, but that’s useful in its own right. Then I start to see, okay, this is a paper which is not only interesting, but actually would be useful to summarize for other people.

I do a lot of surfacing—a lot. I’m very quick at scanning, so that’s just a mental process. At that point, when I found the paper, I’ve got a Gemini gem and an OpenAI GPT, both of which I call Insight Distiller. Basically, I stick the paper in there, it comes out, and I always rewrite it. I will either prompt the AI to improve it in various ways, and then always just rewrite or choose which of the points I put in, and so on. So there’s actually a fairly manual process, but very, very AI-assisted. To your point, there’s so much extraordinary research going on, and people don’t look at it. The function, I think, is what you’re alluding to—it’s just like saying, “This is the essence of a paper, and you can read it in a few minutes and get some really good insights, and hopefully that will inspire you to go have a proper look at the paper, because there’s a lot more in there.” To myself, of course, going through all that is enormous and valuable to me, but it’s useful to others too.

Marshall Kirkpatrick: Absolutely, wow. That is a high-touch. That’s great. I bet you really have a lot of compounding learning as a result of it.

Ross Dawson: Yeah, it’s kind of this thing where, just the nature of how my brain works and my immersion in stuff, I think it somehow gets me to some decent understanding of what’s going on. So to round out, what’s the next phase? I think this is an extraordinary time, but in the frame of what we’re talking about—AI and cognition—from your perspective, or just the world’s perspective, where do we go from here?

Marshall Kirkpatrick: Well, I think that it comes down, in part, to values. I can’t help but think about this K-shaped future that we risk moving towards, where some people are using all kinds of augmented capabilities and building on top of past experience and education and what have you, and income inequality just gets more and more intense. The gap between people who are excited about this stuff and can use it, and everyone else, just gets all the bigger. That’s not good for anybody. I really hope that isn’t the case. I’d love to get the J of exponential change without too much of the K of increasing inequality. I think that’s the direction we’re pointed in, but I do hope that we can democratize access to a lot of these capabilities and figure out how to use them in partnership with other ways of thinking—like Azeem and his team, writing on paper, like some of the indigenous traditional knowledge practices around the world that are very place-based and around ecosystem balance and recognizing humans as a part of nature, working with AI and technologies. I’d love to see this be an additive experience, more than a destructive experience for humanity and the rest of the planet.

Ross Dawson: Yeah and that’s why you and I both working on is doing whatever we can to nudge things in those directions. So where can people go to find out more about your wonderful work?

Marshall Kirkpatrick: Well, these days, I am pointing people mostly to whatsupwiththat.app. That’s kind of my home these days for all the different work.

Ross Dawson: I’ll recommend it.

Marshall Kirkpatrick: Oh, thank you so much, Ross.

Ross Dawson: Very useful, and I’ve only just begun to use it so—

Marshall Kirkpatrick: Awesome, well, let’s stick some of those papers in there and red team it and hit “Find Science” and get other scientific reviews of the claims in the paper, etc. Thanks—it’s so great to be back in touch with you here and not just watch from a distance, but to get to put our heads together like this is a real pleasure.

Ross Dawson: Thanks so much, Marshall.

The post Marshall Kirkpatrick on cognitive levers, combinatorial possibilities, symphonic thinking, and compound learning (AC Ep39) appeared first on Humans + AI.

“Fiction has this unprecedented power in tech spaces. The more I started talking to engineers about their technical problems, the more I realized there’s so much more that humanities could offer.”

–Nina Begus

Nina Begus is a researcher at the University of California, Berkeley, leading a research group on artificial humanities, and the founder of InterpretAI. She is author of Artificial Humanities: A Fictional Perspective on Language in AI, which received an Artificiality Institute Award, and First Encounters with AI.

Website:

ninabegus.com

LinkedIn Profile:

Nina Begus

 Book:

Artificial Humanities

Ross Dawson: Nina, it is wonderful to have you on the show.

Nina Begus: Thank you for having me.

Ross Dawson: You’ve written this very interesting book, Artificial Humanities, and I think there’s a lot to dig into. But what does that mean? What do you mean by artificial humanities?

Nina Begus: Well, this was really a new framework that I’ve developed while I was working on the relationship between AI and fiction, and I started working on this about 15 years ago when I realized that fiction has this unprecedented power in tech spaces. So this is how it all started, but then the more I started talking to engineers about their technical problems, the more I realized there’s so much more that humanities could offer in this collaborative, generative approach that I’ve developed.

I would say that now, as the field stands, it’s really a way to explore and demonstrate how humanities—as broad as science and technology studies, literary studies, film, philosophy, rhetoric, history of technology—how all of these fields can help us address the most pressing issues in AI development and use. And it’s been important to me that this approach uses traditional humanistic methods, theory, conceptual work, history, ethical approaches, but also that it’s collaborative and exploratory and experimental in this way that you can look back into the past and at the present to make a more informed choice about the future. You can speculate about different possibilities with it.

Ross Dawson: Well, art is an expression of the human psyche, or even more, it is the fullest expression of humanity, and that’s what art tries to do. Also, I’m a deep believer in archetypes, human archetypes, and things which are intrinsic to who we are, and that’s something which you can only really uncover through the arts.

Now we have arguably seen all these archetypes play out in real time, these modern myths being created right now in the stories being told of how AI is being created. So I think it’s extraordinarily relevant to look back at how we have depicted machines through our history and our relationship to them.

Nina Begus: Yes, this is the reason why I started exploring this topic, actually, because there were so many ancient myths, these archetypal narratives that I’ve seen at the same time, both in technological products that were coming to the market and in the way technologists were thinking about it, and also in fictional products and films and novels in the way we imagined AI. I framed my book around the Pygmalion myth, but there are many, many other myths—Prometheus, Narcissus, the Big Brother narrative, and so on—that are very much doing work in the AI space.

The reason why I chose the Pygmalion myth is because it’s so bizarre in many ways: you have this myth where a man creates an artificial woman, and then in the process of creation, falls in love with her. So there’s the creation of the human-like, and there’s also this relationality with the human-like. You would think this would not be a common myth, but quite the opposite—I found it everywhere I looked. It wasn’t called the Pygmalion myth, but the motif was there. I found it on the Silk Road, in ancient folk tales, in Native American folk tales, North Africa, and so on.

So I think this kind of story is actually telling us a lot about how humans are not rational, how we have some very deeply embedded behaviors in us, and one of them is that we anthropomorphize everything, including machines.So I think this was a really important takeaway that we got already from the early days of AI with the first chatbot, Eliza. We’ve learned that that will be a feature of us relating to machines.

Ross Dawson: So Joseph Campbell called the hero’s journey the monomyth, as in, there is a single myth. And I guess what you are doing here is—well, if you agree with that, which I’d be interested in—is that there are facets. The classic hero’s journey is quite simple, but there are facets of that monomyth, or something intrinsic to who we are, that is around this creation. And in this case, as you say, this relation we have with what we have created. Would you relate that at all to Joseph Campbell’s work?

Nina Begus: I haven’t thought about it in this way, because I thought about myth and myths more and less of a storytelling issue, which here is definitely happening—the hero goes on a task, returns back changed, and maybe changes something in the community. The myths that I was looking into and the metaphors that I was exploring, primarily this huge metaphor of AI as a human mind, as an artificial reason—I think it works differently. It’s less of a narrative; it’s more of an imaginary of how or towards what we are building.

I think this is a big problem, actually, because the imaginary around AI is very poor. What you get is mostly imagining machine intelligence on human terms, and a lot of people are bothered by that in the AI discourse—right, when you say the machine thinks, or the machine learns, or it has a mind, and some people go as far as to say it has consciousness. I think this kind of debate is actually not that productive. I think it’s more important to see how all these different AI products that we’ve created—and mostly when we talk about AI, people think of language models now—are very much designed as a sort of character, almost as an artificial human that, in literature, authors have been creating for a long time.

So I think in that case, we can get back to a hero’s journey. But I think what I was looking at was actually more on the surface level of what kind of shortcuts we are using with these metaphors that we’re employing when building and using AI. I think the book makes a really good case showing that, yes, this is actually a very cultural technology. It’s very much informed by our imaginaries. One surprising part of it was really how hard it was to break out of this human mold. It was pretty much impossible to find examples of machines that are not exclusively human-like. I think Stanislaw Lem is one of the rare writers who can consistently deliver this kind of imaginary.

Even looking at more recent works, like popular films such as Hollywood’s Ex Machina or Her, you can see how the technologists themselves would say, “Oh, we were influenced by this film,” in a way that it affirmed their product development trajectory. You can see it now, at this moment, with OpenAI launching companionship. So in many ways, not a lot has changed.

Ross Dawson: Yeah, there’s a lot to dig into there. I just want to go back—in a sense, Pygmalion is a metaphor, but it’s also a myth. It is a story: creates a woman, and then falls in love with her, and then whatever happens from there. There is this, something happens, and then something else happens. That’s what a story is. I think that can impact the implicit metaphor, but coming back to the metaphor—so George Lakoff wrote the beautiful book Metaphors We Live By. I think the way the brain works is in metaphors and analogies to a very large degree. Some of those are enabling metaphors, and some of those are not very useful metaphors.

I think part of your point is that some of the metaphors that we have for thinking about AI and machines are not useful. There may be, or we could create, some metaphors that are more useful. So, what are some of the most disabling metaphors, and what are some of the ones which could be more constructive?

Nina Begus: Yes, So I think this main metaphor that I’ve mentioned—of AI as a human mind—is very limiting. I think it really limits the machinic potential to actually do something good with it. The fact that we’re still using the criteria that were made for humans, like different criteria developed on human language—the Turing test was one of them, right, a while ago. Now we have stricter ones. I think this tells you a lot about how we actually evaluate AI and how even these benchmarks that are supposed to be quantitative are actually often qualitative, often stories, like mini-narratives.

But yeah, when we look at different metaphors in this space, there are other ones that also emerge from fiction. I mentioned the Big Brother, the AI as an Oracle, and we need to be aware that these ideas inform the very interaction we have with AI. If we think of it as a mirror, we’re going to use it differently—it’s almost as a bouncing board. If we think of it as a teacher, or as a coach, or as an assistant, it would again create a different use. So I think there are a lot of these metaphors that the companies themselves are trying to decide which one they will go with, because it completely changes the user and the interaction.

I think they’re also very cultural, even though you might say, “Oh, it’s a categorical mistake to treat a machine as a human.” I think you can see this kind of treatment across, at least in part, and it doesn’t mean that we consider it human. It just means that we’re engaging with it on our own terms, as if it was human.

Now, what could be productive? I do think metaphors, even if they’re not accurate, can be productive. My goal, really, with the book was to break out of this projection of what the machine could be, to find in this exploratory way other directions, other landscapes where we couldn’t go because we’re being limited by our imaginary, by our ideas. So in this way, I think humanistic approaches can be very helpful to designers, to technology builders, to artists, to explore the novelty that so many of these sectors are after.

Ross Dawson: Yeah, and I guess people latch on to what they know. I think that’s part of the thing where with AI, “Oh, it’s like a human. Let’s treat it like a human, and let’s make it like a human.” It is, amongst other things, a lack of imagination. That’s where the humanities, the arts, can offer us—those who have the imagination to be able to envisage different possibilities or relationships.

But I guess part of it is also that humans relate, and so we have learned to relate to other humans and also to other animals and hopefully to nature as well. But these are all established patterns of relating. So do we need to discover in ourselves new ways of relating to new categories—things which are not humans, not animals, and not nature?

Nina Begus: Exactly, this is the exact problem we’re dealing with, and because we’re dealing with a yet unexplored, yet undefined relation, and we’re using old, outdated terms for that relation. This is why we don’t really have a good way of describing it and establishing it. It will take a while for this to develop, which is fine, but we need to realize that there are some concepts that we’re using that we better leave behind and go ahead by building new ones.

This is why I think it’s really important to work in a more interdisciplinary collaboration, so that you can see what you can actually build from the technical perspective, so that you can see what these machines are actually capable of. Because you usually don’t know when you create them right?Machine learning is sort of exploratory by design.

Ross Dawson: So, just to call it out more explicitly, what are the metaphors you think are the most destructive or most inappropriate, and what are some of the ones which you think are the most promising?

Nina Begus: Well, I’m just writing on the Midas myth, which is sort of the opposite of the Pygmalion myth. With Pygmalion, you lean into that human imitation, but with Midas, you lean into the liminality that Midas presents as this sort of hybrid creature. I think leaning into the boundaries that we draw for ourselves—and now AI is not cooperating with them—this is where the productive part will be in actually creating something that has philosophical dignity, but also a kind of productive trajectory for the machines to go.

I feel like we’re still in this first phase of developing AI, because when you look at it historically, we haven’t really moved from the conceptual and philosophical premises that were established in the 1940s, 50s, and 60s for this technology. We have now gotten the technology that caught up to the ideas from the 60s, but we’re still stuck in the same conceptual space.

Ross Dawson: Yeah, very much so. And, you know, of course, what is AGI, which everyone talks about, is basically—the only way in which people seem to be able to frame it is as relative to humans, which is the only reference point we have. I mean, there’s, of course, animal intelligence, but that’s because of that. It is, again, that lack of imagination—saying, “Well, intelligence, oh, intelligence is what humans do, so let’s do something which is the same as that,” whereas there’s so much white space in what intelligence could be.

I think this almost comes back to definition. When people say intelligence, the word, when they use the word intelligence, they are referring to what humans do. It’s not a general term, and so it all becomes a language problem as well, because we are so rooted to relating our language to human capabilities, as opposed to a more general potential.

Nina Begus: Yes, I think you’re really on to something here, because I can see it also—because I work with animal communication researchers, and we’re finding things there that we didn’t find because we limited ourselves to thinking language is just a human production, that it needs a human subject. Now, as soon as we got rid of this presumption, we’re finding new things, things that are basically parallel to what we do in our language.

So language is in a space of tension because it’s being attacked both from the animal side and from the machinic side, which is why I really focused on language in this book. It’s not a coincidence that we centered artificial intelligence in language as the interface, because this is how we relate to the world—this is our interface to talk to each other, to understand each other.

I think the fact that language is coming under such pressure as an interface brings with it a lot of other concepts that are being challenged. Are only humans creative? Is there a natural creativity, machinic creativity? Is there a different kind of intelligence that’s maybe solely biological, embodied? How do we think about cognition? How do we think about culture? In AI and in the natural world, there’s so much that comes with it: agency, autonomy, freedom, community, which I think we will be grappling with for the next few decades, at least.

Ross Dawson: I think you alluded before to the potential for AI to have its own languages. 

Nina Begus: I’ts happening already. The reason why I like Stanislaw Lem so much is because he can actually think about a machine—back in the 1970s, he’s doing that—about a machine that’s not human-like, that’s not limited to human language. It is trained on human language, but then it goes its own way, where the human linguistic ceiling just cannot go anymore.

We’re already seeing that in the models, in Berkeley’s Biological Artificial Intelligence Lab, in the models that are not large language models, but generative adversarial networks that are based on speech. We see that as they are learning the words, they are encoding some information into silences that we don’t know what it is.

I think what’s really exciting to me are two things about language in machines. The first one is, what is this non-human production of language? We did not think that non-humans can produce language, even though we had parrots who had to crawl their way to us to speak in “humanese,” to show that they have some kind of intelligence—even if it’s just parroting, even if it’s just what we call imitation, which some people consider not to be intelligence.

We’ve had these examples before, but now it’s gotten nuclear—on this scale that LLMs are performing, it’s really challenged a lot of our solely human attributes: creativity, storytelling. A lot of journalists come to me because there’s this existential fear of machines taking over their work and so on. So we’ve been thinking about those things, and now it’s actually happening.

Ross Dawson: One of the other key points here, I think, is that humanity is—the arts—there’s so much, as you mentioned, in terms of fiction, in terms of films, in terms of visual arts, and many other artistic domains. We have reference points that we use, and the amount which people refer to the movie Her in the last years is pretty extraordinary, partly because it’s obviously coming very much true. I think the Ex Machina story is very interesting as well, as are many others in the past.

But there is also this act of imagination. There are people who have written these books, who have crafted these films, who have created these things, and they are the ones who have been not just manifesting our human psyche, but also pushing that out and coming up with ideas which others haven’t had, to give us something. So one thing we can certainly do is mine and dig into what has been created. But is there a way to interface through this to this act of imagining, which can give us new artifacts and ways of thinking and ways of relating?

Nina Begus: Yes, I think imagination and humanities in general are going to become more and more important, because AI will do a lot of technical work, but imaginaries—this is what we really excel at. It’s actually interesting to see how you think fiction is this unbounded landscape where you can imagine anything, and yet it’s really hard to find examples of machines that are beyond the human.

Even these writers, like the screenwriters for Her and Ex Machina, create these completely Pygmalion-esque films, where you have an artificial woman leading a relationship with a human man, and so on. For the whole film, you have her act as a human-like entity. But then at the end of each of those films—well, particularly in Her—Spike Jonze really tried to break out of this and show her AI side. Basically, there was no language to describe it, so he resorted to a metaphor—the metaphor of a book, where Samantha, the operations assistant, explains that her world is falling apart, like the way words are floating further and further apart in a book. That’s how she’s able to describe it; that’s the closest she gets.

And then in Ex Machina, Alex Garland really wanted to portray the world from the social robot Ava’s perspective in a visual way. He wrote down a scene, but he said, “I failed to execute it visually. I just couldn’t do it well.” So instead, he gave us a different scene that’s shot from afar, where Ava embarks onto a helicopter and she has to undergo her Turing test—the helicopter pilot cannot recognize her as a robot; he needs to think she’s a human woman.

There have been attempts, I think even in Garland’s next film Annihilation, they’re trying to set the grounds for something that’s entirely new and hard to imagine. I think a big takeaway for us is this is very hard to do.

Ross Dawson: Yes, well, given that context, I do want to—as in the human plus AI framing—given all of this, what is it that we can do or should be doing in order to amplify our humanity, our capabilities, the positive aspects of what it is to be human? How can we relate to or use AI in order to amplify the best of us?

Nina Begus: Yeah, I actually had, while I was writing the book Artificial Humanities, this other dream project to work with writers—professional writers, creatives, people who live in a world of words—to see what they make of AI. I waited a little bit for the public’s polarized reactions to calm down a bit and gathered 16 writers, some of whom already made a space for themselves in the field, like Sheila Heti and Ken Liu and Ted Chiang, and then some of the more junior writers who I knew were thinking about that—a Netflix screenwriter, and so on.

I gathered them to see—I think the creative people are really the answer here—I gathered them to see how they approach this very human part of the new human and AI collaboration zone. What was common across a lot of essays that are coming out in October under the title “First Encounters with AI” is this argument that, well, AI doesn’t have subjectivity, it doesn’t have emotions, it doesn’t have a body, it doesn’t have experience, it doesn’t have meaning—all of these things that really make us human, all of these parts that actually make art compelling and literature compelling.

So Ken Liu’s argument, for example, was, let’s leave machines what they’re good at—they’re good at imitating and copying—and we’re good at interpreting, we’re good at creating and imagining. I think this is really a way to go with this. This catastrophizing that’s very present in the public discourse, I think, is a bit misleading. I wish we had a more nuanced approach to what’s actually happening, particularly in the space of writing.

Obviously, AI is a groundbreaking technology that affects pretty much every one of us and all the sectors, but when it comes to writing, we just don’t think it’s killable. We think that there’s this perennial impulse that humans have to play with language, and that is not going to go away with AI. We’re just going to amplify it through AI, through this new possibility that has now opened in many ways.

I like to think about AI as—you know, we’ve figured out how to fly. As soon as we figured out the physics of flight, we had planes and helicopters and drones and kites, and these are the new possibilities for human activities. In the same way, we figured out the machine learning principles, and now we have large language models and diffusion models, and we have GANs and so on, and there will be more. These are the new spaces of possibility that have opened for our activities, for our spirit to work on, but they do not replace the human in a meaningful way. It’s more about extension than it is about automation.

Ross Dawson: Yeah, that’s a wonderful way of framing it. So where can people go to find out more about your work?

Nina Begus: I have a pretty populated website with my name, ninabegus.com, where I write about my books, I write about my public work. I have videos on there, podcasts, links, and so on. I also have a pretty lively lab with a lot of collaborators and students, where a lot of what I imagined when writing Artificial Humanities—where a lot of collaborative projects happen. We have artists, we have engineers, we have philosophers that work on the same question, but come at it from very different backgrounds and with very different skills. I think this is becoming more and more important in the world of AI.

Ross Dawson: Yes, yes, bringing all of those disciplines and frames and thinking together. That’s wonderful. I love what you’re doing—very important. I hope the messages ripple through, and obviously wonderful to be able to share this with the Humans Plus AI audience. Thank you so much.

Nina Begus: Thank you, Ross, and thank you all for listening.

The post Nina Begus on artificial humanities, AI archetypes, limiting and productive metaphors, and human extension (AC Ep38) appeared first on Humans + AI.

“The center of any change that we’re doing in the fourth industrial revolution is always the human being, because humans have an ability to adopt, adapt to skills, and adjust to an environment.”

–Henrik von Scheel

Henrik von Scheel is Co-Founder of advisory firm Strategic Intelligence, Chairman of the Climate Asset Trust, Vice Chairman of Regulatory Intelligence Committee, and Professor of Strategy, Arthur Lok Jack School of Business, among other roles. He is best known as originator of Industry 4.0, with many awards and extensive global recognition of his work.

Website:

von-scheel.com

LinkedIn Profile:

Henrik von Scheel

Ross Dawson: Henrik, it is wonderful to have you on the show.

Henrik von Scheel: Thank you very much for having me, Ross.

Ross Dawson: So I think we’re pretty aligned in believing that we need to approach AI from a human-centered perspective and how it can bring us prosperity. So I’d just love to start with, how do you think about how we should be thinking about AI?

Henrik von Scheel: Well, I think, like every technology that comes into play, it brings a lot of changes to us. But I think the center of any change that we’re doing in the fourth industrial revolution is always the human being, because humans have an ability to adapt, adapt to skills, and adjust to an environment. So technology is something that we apply, but it’s the strategy on how we adapt with it that makes a difference. It’s never the technology itself.

So I’m excited. It’s one of the most exciting periods for the industry and for us as people.

Ross Dawson: There’s a phrase which I’ve heard you say more than once around AI should make us smarter, healthier, and wealthier. So if that’s the case, how do we frame it? How do we start to get on that journey?

Henrik von Scheel: So I think what people experience today in AI is that they experience a lot of media hype—large language models, ChatGPT, and all of this—and they consume it from the media. So there’s a big hype around it, and I believe that AI is about to crash fundamentally, but crashing in technology is not bad, right? There are a lot of promises and then an inability to deliver, and then it crashes. What you hear in the media today is very much driven by a story of them raising funds because it’s so expensive, and so they are promising the world of everything and nothing, and the reality looks a little bit better.

The world that they are presenting is that you will be replaced, and you will be happy, and you’ll be served by everything else. And somehow it will work out. We don’t know how, but it will work out. And that’s not a future that is really a real future.

The future must include that everybody gets smarter, wealthier, and healthier. And when I say everybody, I mean not only the guys that have money, that they become more rich, or the middle class. It’s like everybody in society should get smarter from AI. That means part of the things that they need to learn or how human evolution works should be better, and it should make us healthier people and wealthier people.

So it should not only be that we sacrifice our convenience with our freedom, with our privacy, with our environment, or any other things that we put on the table to get convenience back. That exchange we have done a couple of times, and it’s not working really well for humans, and it’s not a good trade for us, right?

Ross Dawson: Yeah, I love that. And since it’s quite simple, you know, you can say it, it’s clear, it sounds good, and it is a really clear direction. But you’re actually pointing in a couple of ways there to capital allocation. So obviously, if you’re looking at the AI economic story, this is around this diversion of capital from other places to AI model development, data centers, deployment, and so on. But also, when you’re saying wealth here, this is around the distribution of wealth—where we’re allocating capital to AI development, but also from the way in which AI is developed, there will be creation of wealth. There is the real potential for productivity improvement.

But then it’s about finding, how do we have the mechanisms for allocation of wealth or capital from that which is allocated? Let’s call it equitably.

Henrik von Scheel: I’m a firm believer that this year, 35 to 45% of the money invested in AI will evaporate. Companies that have invested—they’re the early adopters—they have this format, so they’re rushing to it. From a company perspective, you always adapt the best practices. When it goes beyond the hype, and the performance curve and adoption curve is low. For example, for AI, the simple version is there. You heard that Deloitte and McKinsey talked 10 years ago about robotic process automation like God’s gift to mankind in AI. Today, you don’t hear them talking about it, because you can download it for free—for HR, for forecasting, planning, budgeting, and so on, you can save 20 or 30%, and as an organization, you can do it yourself. You download two, three models, you test it, and you run it. Good, okay, so that’s when you apply best practices.

Then you have industry practices, like AI agents. So when you have AI agents for manufacturing, for industrial sectors, for energy sectors, they are nothing else than workflow optimization. You use robotic process optimization, you do a visualization on it, so it’s far more practical at a level, because you use the data they already have in the organizations under a simple line on the process flow, on the safety, security—it’s very much down at the level where they can apply it and use it. So this version of large language models, where you have this magic powder you spread over the organization and it’s totally working—it’s not really there.

And then there’s the third leg that companies are quite aware of. It’s called Shadow AI, right? Shadow AI is because AI is the biggest infringement on intellectual capital within organizations. The reason why normal people are not allowed to look at pornography at their work is because of cybersecurity. It’s not that your boss doesn’t like you to look at pornography; it’s because of cybersecurity. It’s the same reason with AI—you should not be allowed to use Copilot latest version or large language models as a CFO or as a worker, because you’re exporting your own information outside. Copilot takes, every five seconds, a screenshot for the large language models’ learning. So as a corporate point of view, that’s the first thing—you should actually protect your own data so you can monetize your data in the future.

From an economic point of view, if you go two, three steps behind this, you ask, okay, what is it that makes sense in this? There’s something really, really strange in this. Australia was built by building railways—they take 100 years to build, they also last 100 years. The infrastructure that lasts. So there’s a return on investment. You build streets, you build education systems—everything we build as humans, as society, has a lasting element to it. Now, we build data centers that last three months until the chips need to be returned, or six months. So there’s no sense in that we are building data centers around the world where we capture all data. It has a volume of hundreds of trillions of dollars, and we need to exchange them at a rate between three to six months to maintain the data. And then you say, wow. And you do that via license models of large language models—the data can never, in its entire life cycle, be that much worth.

So there’s a very strange element, because most of the entrepreneurs that go to large language models and use their solutions on Gemini and ChatGPT and so on, you say, okay, you are building your solution on large language models, but you don’t own the model. You don’t own the data. You don’t own your own data. So what are you doing?

Ross Dawson: You have architectural choices, to a point, as to—

Henrik von Scheel: That’s Architectural choices, but you are limiting yourself. So the first element you always say, if my value is customizing a solution, your value is actually the data. So you must have a way to keep and maintain the data yourself. We can take another call to say how you apply AI and what the future of AI looks like, because AI today is very much focused on language models, and language models are the most limited version of AI science of all. It has the least data, but it’s the one we’re most excited about, because it resembles something we do—our wording, our formation of words. It’s a recognition. Recognitions are what we do.

I wanted to come back to this about the economy, right? The US economy puts all chips on this. It’s highly energy sensitive, and it’s working all railroads. However, the US dollar is on a really, really bad track record. Three and a half years ago, there was a president in the US—he was sleeping—and meanwhile, he was sleeping, Saudi Arabia’s King MBS went in and he did a divorce, which is called the divorce of the petrodollar. So the gold linked with US dollar linked with oil—that was the solution. The US had that anybody, they could print as much money as they wanted, and the rest of the world was paying the dividend for it. It was the only country that could just print money. That brought the US into a mode, and when the new president came into his office, it’s very rare that in the US, you are writing an accord. An accord is only written when the Federal Reserve goes into the president’s office saying, guys, we’re hitting the wall. We need to do something. And they wrote five plans, what they wanted to do. And here’s the funny thing—when I mention them, you will recognize them very much.

Number one, bring back manufacturing. Number two, implement tariffs so they can pull back US dollars. Number three, then they wanted to implement stable coins to pull back US dollars. I forgot number three, actually. Number four, and number five was actually they want to go to war. Now they go to war, right? So they are going to war, not because of any reasons besides their economy is based on a war machine, and the economy is becoming unstable. So that’s one of the main reasons. The US has put all cards on AI—all their economy cards are on AI. And that’s, from a country perspective, a very dangerous thing to do because you need energy and you need data, and AI from the US perspective has become a defense mechanism.

When you look at the regulatory aspect of AI, Europe is very much put into human and center, and that the human owns the data, protects teenagers up to 16 years old, and that you can work as an entrepreneur with data, but you have to coordinate how you protect and manage the data. You have to be transparent on how you use the data and how much data you use. The US is very different—red tape off, no regulations at all, full-blown power to the market, and you are seen as a consumer, Ross, so all power to the guys who earn money to make more money. So no protections of anything, of your data—that’s the US version and literally, no regulations, no redtape regulations.

Ross Dawson: In a moment, I want to move on to the human-AI collaboration. But just to round this out, you said before about your prediction that 35 to 40% of the investment in AI is gone, which I think is very, very fair. So back when we both were speakers at the Future of Sex Summit in Dubai last year, I was on a panel where I was asked, is it boom or bust? And basically both, in the sense of 35–40%—that’s bust. But at the same time, there are other parts of the market which can prosper. Of course, consolidation of the market means that there’s massive investments and in some cases massive losses, but there still are sectors where high value can be created. But this goes back to your point where still a lot of the center is in the US. We are starting to see sovereign AI initiatives and other initiatives around the world, but those are often open source foundation models. And obviously the regulation, particularly around the EU, provides a still very differentiated AI landscape with US, China, EU, and then some other players as well, where if we see boom and bust, that could be very much focused on the US, with potential for other parts of the world to see more growth in AI.

Henrik von Scheel: So Ross, you’re using large language models, right?

Ross Dawson: Yes.

Henrik von Scheel: Do you have the feeling that they, since last year, are getting stronger or weaker? they’re getting weaker?

Ross Dawson: They’re getting better.

Henrik von Scheel: My feeling is the opposite. My feeling is that they’re getting weaker and weaker, and that’s because part of the data —

Ross Dawson: In which content?

Henrik von Scheel: They’re using old, old content. They’ve already used old content. So now you need to go to specialized, you need to go to public sources, to go for research data, you know. But from a content-wise perspective, it becomes extremely weak. I mean, last year, I’m extremely disappointed by large language models—very, very disappointed in terms of what they can deliver and what they do. Ask it whatever—ask it about futurism prediction, or ask about Industry 5.0, 5.6, whatever answer you give it, you can get an answer. You know, 110%—like CPAM, there are 19 regulations on CPAM, and you ask, how many regulations are there? They will give you sometimes 19, sometimes 17, sometimes 23—they just make up stuff. It just gets worse and worse. So if the valid data is not strong enough, it becomes actually a very, very weak tool after all, right?

Ross Dawson: So are these using the top models from the frontier labs, because they are very good.

Henrik von Scheel: Yeah, but then you have to have the paid model. But it’s not like I’m really, really impressed by it. It’s not kicking my bum where it says, holy smokes. In the beginning, the first two years, you were surprised, right? So I have a little bit of the feeling that AI today is a little bit where emails were in the beginning, and then digitalization came. With emails, we were all excited, but emails just created not less workload, but more workload for us—it decreased our productivity. There are really good signs of this.

Then you look at digitalization, right? We were all excited because we can connect, we can talk to our friends, all of this. But what ended up with WhatsApp Business? WhatsApp Business is no business, right? We are using it, but it decreases our productivity level far more. So today, with digitalization, we are becoming generalists—quick information, we know something, but we don’t know anything, right? It’s not that you would put the finger on it and say, well, it has really increased our innovation level. No. Has it really increased our research level? No. Has it really made us better human beings? No. So I’m not negative against it. I’m just saying we have to be careful, because we have a knife or a hammer—we shouldn’t use the hammer for everything. And you mentioned that really well, right? AI’s hype cycle is, with any technology, there’s a hype, and then it goes down and matures, and then the application of this is different than what you thought in the beginning, of course, but that’s AI—it’s very much relevant. But you know, the big message today in AI is AI physical, right? What is AI physical?

Ross Dawson: Well, just going back to the point—a lot of what I’m working on at the moment is the idea of appropriate trust. So you trust the models enough, but not too much, so that if they are going to give you bad results, you’re not relying on them. But if they are useful, you can use them. So we have to continue to calibrate for any particular model, which is different in every particular context. This is both essentially a skill or a capability, where we need to know when and how to use models at any particular time, because they’re changing in whatever way. So that becomes a foundation of how we can trust them to the right degree—not too much, but enough that we can actually use them if they are useful. Which comes back to this frame of the human-AI collaboration, which you’ve been doing a lot of work on. So if AI can be useful in some contexts, how is it that we can best build effective human-AI collaboration?

Henrik von Scheel: I like this. Let’s play a little bit, right? So if human evolution is evolving with the birth certificate, we go to kindergarten, we go to school, and we learn differently. Everybody’s individual—we learn differently, right? It takes humans a long time to learn, to sense, to do all of this. And then you have AI, which is a supporting learning model for you to store information. But today you learn, and the model learns on you. You log in, and every time you learn, the model learns from you. That means that all your information is captured there, right? So the next evolution of a model should be that the privacy of Ross is throughout your last five years with large language models—you’ve studied Porter’s models, you’ve studied this and this. Well, if I ask you next day about Porter’s model, you still forget it, but the machine should be able to help you to learn, to adopt the skills in your daily life. So it cannot be a machine knowledge learning that is owned somewhere else by a big company—it must be something that is attached to Ross throughout your life, that you go from where you are now, and in five years, you’re somewhere else. So the knowledge that you have searched and gained and adopted, it follows your life, right?

This is, for me, AI—the real AI revolution happens in the bio revolution in 2030, because the biggest amount of data we have is biophysical data. So the interconnection between our body, the modules, the biosystem modules, the biophysical systems, how we eat food, how material, with their level, is coming all in there, and part of this is the knowledge center of you, Ross. So if you learn something, how does it follow your evolution? Do you learn the same way today you learned 10 years ago?

Ross Dawson: And it’s a wonderful thing that we continue to learn and forget and evolve. We are the same person, sort of, but, you know, we are a different person at the same time.

Henrik von Scheel: I was talking yesterday to a psychiatrist who’s studying human evolution, and she’s called Trina Gondo, and I had this interesting discussion with her, because she says humans’ learning capacity changes throughout their life. So if we have learning modules that can support us throughout our life—to go through how conscious, how focused we are on things, how much stress level we can take, because stress levels are also different, how much breadth are you covering in terms of your work, your private life, how are you in terms of setup, in terms of your spiritual life—all of this has something to do with your learning, because it’s your perspective you drive. It’s your values you drive.

I actually developed with her a model in terms of how the six aggregates of the brain work to understand our human evolution. For the last eight months, I’m trying to map human evolution, to map it to what AI—how it affects it, what we should regulate and how we should protect it, and how the human can monetize its own data, right? So just look at—

Ross Dawson: The initiative by Doc Searls. So there’s a couple of really interesting initiatives. This is one where he worked originally on VRM, the vendor relationship management—you own your own data and trade that as effective—and is now building, or being instrumental in setting up, an AI initiative where it is around your personal AI, so you own the data, you own the systems, and you’re able to evolve with it. There are some other interesting initiatives like this, but these are obviously very tiny compared with the ways in which most people are using—essentially giving off their data to other people. But this is certainly part of the potential, to build the structures and architectures where we do own our data and our models and how they are used and what comes from them.

Henrik von Scheel: So let’s go back into one element, right? Originally, Ross, you and everybody else of us who live in a society, we made an agreement with the government—a social agreement. And the social agreement is, I’m using, you’re protecting me, and I’m willing to pay tax somehow, right? So in reality, the government you made an agreement with should have the ability to protect you. However, in an AI model today, it’s not possible, because if they should protect you from the very beginning and keep the store of your data and maintain your data, the amount of money they need just to maintain your data is immense.

So we need to define and find a model with governments where governments and the human being can, in co-ownership, hold the data structure—like in a blockchain, that you have a public and a private key, and both can hold the data, but the data is only unlocked both ways. Why? Because there’s a monetization model on your own data throughout your life. And when you die, your data goes on to your children, because that’s your DNA data, that’s your history life data, that’s all of it. So there should be an ability to monetize it.

The challenge we face with this is the amount it will cost to maintain your data throughout your life, and we need to find—in the fourth industrial revolution, we’re going through the bio revolution, then we’re going to the consumer revolution, and then we go to the fusion revolution. And in the fusion revolution, the objective and the hope is that we are finding mechanisms to have cheap energy, because the amount of energy we use today in terms of data is literally crazy. It’s utterly, utterly crazy. We should be ashamed of ourselves if we see that, and that’s just for the amount of convenience.

So if we find a model for our government to do this, we should actually work on this. This is what I’m trying to look at. I want to alert you to one interesting thing. My key field of study is patternicity with probabilities. So when you look at trends that are coming, you look at probabilities—not ChatGPT stuff, right? When you look at this, there’s one trend that emerged last week that hasn’t been emerging before—the trend of anarchy in Europe. Anarchy is an interesting aspect, because anarchy is your distrust in the government. And when anarchy comes, it’s just an equation of 25%. If 25% in a country like Germany or UK or France will take it, 25% is a flipping chart for everybody, because the petrol prices are too high, expenses for food are too high, they get too many promises they never—and then take the power in their own hand.

When you look at it a little bit, you say, but anarchy—is that something new? No, the US is living in anarchy today. Trump is the true version of anarchy. They distrust the government, and they choose him, and he, from all aspects, says, okay, I’m doing something very different. I give all the power to the market. There’s been no time in history where all the power is residing within the market—Elon Musk and Amazon, Apple, all of them have literally all the power. It’s totally, utterly crazy. This is the highest version of anarchy you can see in a country. And if we’re not careful, it’s spreading.

Why am I discussing this in an AI human element? Because if the human is the centerpiece, what is the core element of human development? It’s that we have safety, security, and trust. If trust is broken, anarchy emerges. So if anarchy emerges, AI can take on very different versions that we don’t want in a scenario thinking, but AI can also take on the version that it can support us in our evolution.

Ross Dawson: Well, just going to that—education. You are a professor. You are an educator. You look at the future of education, and you alluded to that before. So in this world where AI is already and is becoming more significant, how do we reinvent education? How do we educate ourselves as individuals, as educational institutions, or society? How do we shape the education that we need for the exciting coming times?

Henrik von Scheel: I think one of our challenges with education is that we as people, when we go beyond eight years old, the key element we’re learning is reasoning, and our reasoning skills are learned by doing mistakes, unfortunately. We never learn by getting an answer. If you study Porter’s model on ChatGPT, and you get all the answers from Porter’s model, and I ask you the next day, if you haven’t applied it, you haven’t learned it. If I would ask you, you will learn it. You do mistakes, and it’s by doing the mistakes, by putting yourself into the content, working with the content, and doing mistakes, you learn. Unfortunately, most of the stuff we learn today—now, human evolution in reasoning is by doing mistakes. So we need to find a very smart way how AI can support us in this mistake learning phase, because it’s the way that we are built to learn, right?

Ross Dawson: And I think that’s a critical thing—where as individuals, we need to understand that if we delegate our thinking to AI, it’s not going to work; you’re going to be dumber rather than smarter. But if we can have the intent of using it to hone our thinking and helping us to make mistakes or be a Socratic dialog or whatever, we can do that, but that requires the individual intent. So again, we also need to frame as educators and also in organizations—which should be educational institutions in their own right, because they are learning organizations—it’s this framing of the use of AI as a cognitive foil for us, as opposed to something where we delegate our work, which is never going to get us anywhere good.

Henrik von Scheel: And where do you think we can use it in education?

Ross Dawson: The good thing is, you know, personalized education, where I think that there is definitely this ability to address where individuals are and their understanding, the metaphors that will be relevant to them, the frames for that. But it never has to be in a form of giving the answer. So there’s always this complement of human—as in, the educator needs to be inspiring. They need to help the person to find themselves. They have that relationship with them. So it’s this complement with the AI, which can guide to specific lessons or frames or examples that people resonate with, which can assist them. And so again, it needs to be very much—individuals need to understand, they have to shape it for themselves. I think we can present things in the right way. And there’s very much a human plus AI educational frame.

Henrik von Scheel: I think you’re spot on with this. When you look at the five aggregates that we have in human evolution and in education phases, our sensory—our forming of ourselves to the outside world—is shaped quite early on, until we are maybe 12 years old, but quite early, the first two years. That means our sight, our smell, how we hear, how we taste, how we feel, and how our balance works—we learn quite fast. This is what AI is focusing on in AI physical today. They’re trying to come from a language model point of view outside to the physical world.

Then we have this cognitive version of us, which is the intellect version. It’s very different. The intellect version of us is a version of awareness, a version of how we comprehend things, how we understand things, how our knowledge is conceived and given out. So it’s both communications, it’s storytelling, it’s our comprehension, it’s our perspective, it’s our reasoning, it’s our awareness. These four things are never the same for the same person. I can have a room of 200 students, I can talk about the same element on Adam Smith’s first principle, and they will all understand it differently because of their different backgrounds. So this part of cognitive understanding, the intellect, is far more complex.

Then you go to the versions of who we are as a person. Our memories—our memories are a whole element of our emotions, which is a hugely important part of our learning, because memories have nothing to do with truth. Large language models always look for the truth, but in our own memories, we are lying to ourselves to keep our sanity. We are partly, not consciously but unconsciously, lying to ourselves because we view it only from one perspective. So our reflection of our memories or our impulses are related to our memories or our conceptual things. All these elements are our emotional elements, in terms of how strongly we can link to knowledge, how strongly we can see the future, how we can see ourselves in the future—all of this.

When you look at the crisis now, the memory is on how resilient we are as people, how resilient we are in our learning phase, how comfortable we are with the unknown, how comfortable we are to learning. Then you have the next two ones. The other one is our mental formation or our identity. This is the element we’re trying to protect in digitalization—how we form our opinions, our insight, our resolution, our understanding, ourselves, and our retentiveness, who we are. All of these things are being shaped as teenagers. We don’t want this to be in a social aspect. We want this to be a safe, secure element. So this is the identity you form.

Then you have the consciousness. The consciousness is a strange thing. You have two layers running in your education. You have the layers that are running long term and the unconsciousness that actually takes the decision—the analytical versions and the underlying elements. For example, why are you doing something? So you come with purposes, you come with energy, you come with desire, or you come with willpower. Then you say, well, they’re more etheric. No, they’re not. Because, Ross, you wake up every morning with that much amount of energy. You can use this the next eight hours you work. You can use it on emails the first four hours, but then you’re using your most precious willpower and energy right then. You have your willpower to train, for example, if you want to do training. When you want to train in the evening, when your willpower is lower, you want to train early in the morning. So this willpower and the energy is what we as humans in our consciousness—how we are aware of things, what we focus on, we magnify. So these are the five aggregates you’re using from the learning perspective.

If we apply these, you and I, Ross, we would go into an initiative to say, how can we apply this to understand human evolution when we evolve this? Because I’m nearly 60 years old now, and that means, for me, my concept of life, experience of life, is different than when I was 30, than when I was 20. You cannot go to a young person that is 15 years old and say, let me tell you about love—there are four different phases of love. They need to experience them themselves, because it’s not my job to take that away from them. And it’s not my job to tell a young man, now you want to conquer and do, you want to have freedom, Generation X and all of this. And then you realize, easy, easy, easy. I’ll let you know. When you fall in love and you become a father, it changes you. Why does it change you? Because accountability moves into a man’s focus area, as before he was conquering. And then accountability—a man wants to be a caretaker of something, and it fulfills and magnifies a man. And then you say, well, this is not part of the five aggregates—very much so, right? Because it’s part of human evolution. Ross, you have experienced that in your life. So then you say, how do we connect that with our evolution and learning?

Ross Dawson: Yeah, no, I think that’s a really important point around accountability for ourselves, for those around us, directly in the broader community. And I think that’s kind of this big humans plus AI frame. So we’re obviously just touching the surface of what we could dig into now. But how can people find out more about your work Henrik?

Henrik von Scheel: I’m a public figure. I’m doing a lot of research projects with universities. I have a lot of PhD students and coaching and supporting governments on policy initiatives. Currently, I’m focusing a lot in the Gulf regions on strategic briefings, on crisis management, in terms of doing scenarios for strategic, tactical, operational, for short term and long term. But my passion is actually teaching, and this is far more a personal story on teaching.

People see me always as the Industry 4.0 originator on everything I have accomplished. But my true story is actually quite different. When I was young, I was dyslexic. I’m actually double dyslexic, and I was stuttering. I had a very, very difficult time in school. That’s why I am a little bit passive aggressive, because I’m always on the defensive, because many years I went through life just being some sort of an outcast. So within that phase, I had a very strong teacher that actually supported me and used time and effort to see my skills, and he helped me to overcome my dyslexia—which is not really true. You never overcome your dyslexia. You are just getting tools to work with it. So that means I’ve written today nine books, and five of them are bestsellers, but I cannot even read my own books aloud.

So what is the message I’m giving? Everybody of us is made different, and because we’re made different, it’s not that—because society is often built on, if you don’t fit that frame, then you’re not part of that frame. But I think AI opens up something for us—that the breadth of who we are as people is a beautiful thing. And because I cannot speak the same way, like I have a good friend Tarek, who is also your friend—he’s a gifted storyteller. My gift is that I can see patterns. So I believe that every human being should be able to see their superpower. Your gift, Ross, is a very different gift. You can gather communities, you can convey difficult things in a simple thing, you have an ability to put the human in the future, where everybody sits today and they freak the hell out because they don’t see them part of the future. So I think everybody has a future in that.

To answer your question, I’m a quite reachable person. I believe the future looks like a good future for us, Ross. I believe this is the time for our educators to wake up out of their long-term sleep. We need to evolve our teaching material. We need to evolve the way that we learn and teach. We have terrible lessons in terms of how boys and girls evolve in their learnings, and we’re not doing anything about it. This is our chance with AI to change the learning mechanisms for boys and girls, our learning mechanisms if you’re one like me that doesn’t fit these templates, if you have special needs. We have the ability with AI to specialize ourselves far more in detail.

One of the challenges we have with education today—when you go from primary school to higher education, and then go beyond higher education—our challenge with higher education is we have become generalists, and our generalism is actually inhibiting us to innovate, so we’re not meeting some of the core challenges that we have in science today, and we need to push the boundaries on where we go to research to really become innovative. We need to push our boundaries in terms of manufacturing, energy sector, and so on, to specialize in special fields. When you look at engineering schools, engineering schools have become more and more generalist in six fields, and they should become specialists in fields. So I think that’s where we need to really push the boundaries.

Ross Dawson: Yeah, no, I think, to your point, what I see as one of the ultimate possibilities from AI is that it amplifies our individuality. And so that’s an extraordinary possibility. So thank you so much for your time and your insights, Henrik. You’re sharing some great work, and we’ll share in the show notes links to one of your research papers and the work you do. Thank you.

Henrik von Scheel: Okay, thanks a lot. Good. Goodbye.

The post Henrik von Scheel on making people smarter, wealthier and healthier, biophysical data, resilient learning, and human evolution (AC Ep37) appeared first on Humans + AI.

“Determining accountability, the ability to intervene, the time to intervention, the time to stop, pause, change, alter—there are so many different layers that need to be thought through.”

–Joanna Michalska

Dr Joanna Michalska is Founder of Ethica Group Ltd., Co-Founder of The Strategic Centre, and an advisor to boards on AI risk, ethics, and governance. She holds a PhD in Strategic Enterprise Risk Management and has twenty years’ experience leading enterprise risk, strategy and transformation at J.P. Morgan and HSBC.

Website:

ethicagroup.ai

LinkedIn Profile:

Dr Joanna Michalska

Ross Dawson: Joanna, it’s a delight to have you on the show.

Joanna Michalska: Well, thank you for having me, Ross.

Ross Dawson: So, AI is wonderful, but it also brings us into a whole lot of new territory where we have to be careful in various ways. I’d love to just hear, first of all, the big framing around how boards and executive teams need to be thinking about governance and accountability as AI is incorporated more and more into work and organizations.

Joanna Michalska: I think we’re all very excited about the capability that exists today to help us enhance our performance and the way we think about strategic execution for our organizations. It has multidimensional consequences for how we adapt it. What’s very important right now is, as executives and boards think about accelerating their ambitions and growth plans, there needs to be awareness of two components.

First, how do we as leaders, as humans, need to adapt to that new environment? There are new conditions, or perhaps existing conditions that really need to be enhanced. They’re very important to exist in order to be able to adapt and to scale. Second, do we actually have the right systems in place to enable that scale?

I think it’s important to recognize that, yes, governance has always existed, but the way it existed was more as external supporting scaffolding, rather than being built into an organizational ecosystem. We also need to have the right leadership in place to ensure that decisions are made in the right way and the organization is designed in a much more robust, agile way. These two conditions are critical for not only increasing adoption, but also doing so in a safe and responsible way, especially as we expand our ambitions for the future. It’s exciting, but there’s also a lot of caution and a lot of questions being asked by executives at this time.

Ross Dawson: Yes and I guess the more we can address those concerns upfront, the more it enables us to do. I have this idea of minimum viable governance—at least having some governance in place so we don’t go too badly astray. But I always think of governance for transformation as: how do you set governance not as a brake to slow you, but in fact to accelerate you, because you have confidence in how you’re going about it?

Joanna Michalska: Absolutely! I think the mindset shift is very important, because governance, to your point, has always been seen as a compliance-driven thing that we must do because regulators require us to, and we need to demonstrate we have these policies and procedures in place and the right people in the right positions. Now, what the new environment is requiring of us—as executives, even board members—is a different set of responsibilities that really cannot be assumed as pre-existing.

In this accelerated environment—let’s call it that, rather than just “AI,” because it’s so overused and can mean so many different things—where the automation rate is fast and overtaking everything, governance needs to change. It can’t be an afterthought or something we designed at one point in the past and now just try to fit into what’s happening. It really needs to become a well-designed, living organism. It needs to organically evolve. It needs to have the right people with the right accountability that is well understood. Accountability that was designed in the past needs to be looked at, discussed, and understood by all executives and across the organization, cross-functionally, to really work.

Another important thing is to make sure executives have the right level of ownership and responsibility to ensure the conditions exist to enable that system to work. That’s a very difficult thing to do, because now you’re talking about having designed human oversight that doesn’t just become a “human in the loop,” but the right human in the right loop. By “right,” I mean: does this person, or these people, understand exactly what the output of the automated system is? How has this decision been made? Is there the right level of executive oversight when that decision is already made? How confident are we that we can say, with a level of certainty, “I’m comfortable with this, and this is not going to create negative consequences I’m not willing to accept”? That’s not an easy thing to do—to create those conditions of trust and safety.

Ross Dawson: Particularly when there are so many decisions and outputs throughout the organization. Let’s go into decision making. I’ve built a little framework around going from humans-only through to AI-only decisions. Hopefully, there are no purely human decisions anymore; at least you can ask an AI, “Am I crazy or not?” even if it’s a human decision. Some decisions are already fully automated, but they still need oversight.

You can bring in exceptions, conditional things, humans in the loop for approval, humans in the process, or build an explainability layer. There’s a whole array of different things. For every decision, you need to create the right way to implement it. In an organization with that profusion of different decisions and possible approaches, how can you actually make that happen?

Joanna Michalska: Yeah, it’s a great question. Decisions are at the center of everything, and the quality of those decisions—and the whole architecture, how it’s designed for decisions to be made—is really important. It doesn’t stay static; it evolves as the organizational structure evolves. Questions like accountability—what does it look like, and what is the governance around accountability—are critical.

Intervention capability is also very important, because with this level of automation, the whole design of how automated decisions are made raises multiple questions. Are these decisions made by old algorithms that are very simple, where the risk is determined by a set of rules? Is there clarity around who actually has the decision intervention rights in the organization, and how does that roll up to an executive layer?

Determining accountability, the ability to intervene, the time to intervention, the time to stop, pause, change, alter—there are so many different layers that need to be thought through. The quality of human decision-making, and determining when a human is able to review decisions made by complex systems—whether agentic or whatever structure the organization has—is critical at any level, whether it’s middle management, executive management, or board. There are different layers of how the architecture requires design and measurement.

Escalation pathways are another one. People will not naturally escalate if they fear negative consequences, retaliation, or any type of fear created because there isn’t psychological safety or trust within the organization. Even if there is an escalation protocol in place within the decision architecture, how do we know that people will raise the problem?

Ross Dawson: The accountability. Of course, only humans are accountable. Ultimately, the board and their executives are accountable. But what you’re suggesting, it sounds like, is that for every decision, there is somebody where you can say, “That person is accountable.” Obviously, it cascades up to who they’re reporting to, but there is human accountability for every decision made, even if it’s a thousand decisions where somebody has oversight and responsibility that those are the right decisions.

I want to talk about escalation and how that might happen, but perhaps we can ground this with a couple of examples. What are some examples of decisions made in organizations—hopefully well-designed, or perhaps not so well-designed and haven’t worked out?

Joanna Michalska: Yes, I have a couple of good examples where an automated system allows review of multiple false positives, where a human would spend months or weeks looking at exceptions. From an optimization perspective, that’s really valuable. For example, in fraud detection or sanction screening, you can design a process where your algorithm applies rules very quickly with specific risk tiering. You know which decisions need an additional level of checks—let’s say, automated checks. With a confidence of over 90%, your queue for checking and looking at exceptions—what would otherwise have to be done by a human—is not really necessary; it’s done by the algorithm.

In terms of decisioning, from a human oversight perspective, you’re really looking at things that are very high risk and require additional human review, or exceptions to the usual flow that break the rule designed for the algorithm to execute. Then, somebody picks it up and looks at it. These are powerful examples where there’s potentially a high human risk of misinterpreting something, but if the algorithm is designed appropriately and has the right governance in place, it can really speed things up and make space for a human who otherwise would be involved in that process to actually develop and do something different.

In the example I’m talking about, we’ve focused on retraining people to expand their roles and do something else, rather than just being involved in checking decisions or reviewing boring exceptions that were really false positives.

Ross Dawson: What’s another different example?

Joanna Michalska: Fraud detection is another really good one where—

Ross Dawson: Because one of the things about fraud detection is there is an answer—as in, it is fraud or it isn’t fraud. You can get false positives and false negatives, but that’s kind of reductionist. There’s a whole array of decisions where you can’t necessarily say before the fact whether it’s a good decision or not. It’s interesting to look at these very different types of decisions, not just ones that can be very algorithmic because they’re data-based and there is a true or false. Many decisions don’t fit those parameters.

Joanna Michalska: No, that’s very true. Actually, what I’ve seen, especially recently, is that there are a lot of questions being asked by the board or executives when they get to the point where a decision isn’t easy or clear. They look at sets of metrics that do not make a lot of sense, and then the question becomes, “Who can explain to me how this decision was made, that this metric shows me X? Who in the organization can I go to, and how quickly, for them to explain that to me?” In my experience, especially recently, that’s a very difficult and uncomfortable question to ask and answer, because it’s not clear—especially when it comes to things that don’t have a clear accountability pathway, because more than one person is accountable. So the question is, is this a Risk Officer question, or is it a Data Officer, CTO, CSO—who actually is responsible?

In these instances, it’s particularly important to have the right accountability that is understood at that level—who is accountable for what part of the process? It’s not easy, because it is quite complex and creates a lot of challenging discussions. Very often, it depends on the organizational maturity and the level of AI adoption. What systems do we have? Do we understand what is an AI system in the first place? That part is not easy, it’s complicated, and it creates quite a lot of challenging discussions.

Ross Dawson: Well, it is difficult and complicated and challenging, but that’s not very useful. Let’s map the pathway. Give me a roadmap for an organization: we’re going to assess our decisions, rank them in order of priority or risk or uncertainty, apply AI, and put accountability in place for all of these. This might take us sixteen years, but we’re going to start somewhere else.

Joanna Michalska: I think a really good example is a new European EU AI Act, where there is a very clear starting point and clear requirements. For many organizations, that’s step one: what are we actually required by regulators to do? We look at what processes, systems, and outcomes we’ve got. That’s step one. Then, we look at which of our applications are actually what risk, and we tier them. We assign the right executives for the right processes. First, we identify where we are today, then work with our compliance or risk officers to understand where we think we are versus where we are according to the regulation.

That regulation—the high-risk identification of those systems—is going live in August this year, so it’s a very pressured point for people to address. Once that’s done, there’s a clear inventory of the current state, a clear inventory of where we need to be, gap identification for which high-risk systems require transformation and to what extent, and then the right people need to be in the right places so the transformation roadmap is defined. There’s accountability for that transformation to occur, but often a lot of external advisors are invited to help. In that case, my work usually starts with an exposure review, where I speak to everyone accountable and get a view of where the organization is, maturity-wise, versus where it needs to be for the implementation deadline.

Then, there’s a very clear prioritization roadmap: what’s the impact and consequences for these processes or systems not to comply, and what are the next steps for compliance? Who needs to be in the process? What metrics need to exist? What’s the gap to the right level of maturity to demonstrate that we are compliant and can confidently talk to regulators, our organization, stakeholders, whoever, to demonstrate, “Here’s where we were or thought we were, these are the steps we’re taking, these are the people who are accountable, these are the decisions we’re going to make, and we’re going to demonstrate that we’ve taken them this way.” That’s how we design our journey.

All of this sounds very simple, but the initial assessment is always very complicated, because everyone goes through the process and, in what I call a deep dive into documentation and governance structures, very often the outcomes show governance is not mature. It often needs to change—even the level of metrics, the maturity of the metrics thresholds in place is very immature and very legacy. Ultimately, it applies to the old world, but not to the new world.

Ross Dawson: Well, every organization needs to evolve, and potentially rapidly. Do you look at strategic decisions, or the role of AI in strategic decisions, or other very complex, high-level decisions?

Joanna Michalska: Yes, and it’s interesting to see that there is, what I would call, a cognitive dissonance between where people would like to apply AI for strategic decisions versus the reality. AI is actually used much more for operationalization or speeding up optimization—very performative. How can we quickly improve performance? There’s a lot of discussion about it, and I see that people want to think about it, especially as boards are putting a lot of pressure to improve strategic ambition and create a competitive advantage, which is well beyond just regulatory compliance. But it’s not very mature—let’s just put it that way. It’s much more integrated within improving operational performance — let’s just say that.

Ross Dawson: Yeah, well, we might loop back to that. Part of your background is neuroscience, and we’ve been talking a lot about decision making. There’s been a lot of wonderful work over the last seven or eight decades on the role of human cognition in decision making. Tell me about the way you see this understanding of neuroscience being applied to cognition, work, and decision making in a world where we have AI as part of those processes.

Joanna Michalska: I think it’s a fascinating area of science, and we as humans, especially in this fast-paced environment, as leaders, really need to evolve our capability of not only managing organizations, but also leading the people side of things. Historically, the human brain and mindset take time to change; it’s not something that changes overnight, and it typically requires a trigger, which is usually not very pleasant for someone to take responsibility or ownership of that change.

Now, when you think about the acceleration of decisions and what happens around the organization—because when we think about business, evolving organizations, or being accountable to shareholders, regulators, or society—we can’t just remain the same. That means our mindset and our ability to become more human really matter. Emotional intelligence, relationship-building skills, recognizing the importance of trust, building psychological safety so people can take responsibility at every level of the organization, and having the courage to say, “This is not working, I’ve seen this problem, something doesn’t feel right, I’m going to escalate to the right person because I know who the right person is”—all of that really comes from personal qualities and owning those qualities that just cannot be replaced by machines.

There’s a gap between what machines are taking over in terms of processes and things that can be done easily, and, to my earlier example, that almost brings a higher weight and pressure onto us to become better—much more resilient, agile, responsible, and accountable. Those qualities, maybe we weren’t really owning to the same extent in the past because we were focused on performative activities much more. That will be required of us through all the external pressures, but also because we want to achieve better outcomes not just for organizations, but for broader society. That sense of responsibility for an impact that is much deeper and more long-lasting is very important. From a leadership perspective, it’s almost like role modeling becomes even more important for everyone else in the organization, and it creates a higher level of satisfaction, engagement, and level of happiness for everyone.

Ross Dawson: So, this is about personal growth in a way—becoming more, as you say, building these human qualities that we need so much in this world. But at the same time, very few people are not experiencing pressure or stress, not least from the pace of change through AI and many other things. There are countervailing forces: we’re being called to be more human, to bring out more of ourselves, but the context is extraordinary challenge. What are specific things that leaders or organizations can do to help people draw out those capabilities?

Joanna Michalska: I think there are a couple of things to start with. I would always say awareness is the first step, and leadership awareness of where the gap is and what’s actually required is very important. Integrity and honesty follow right after, because people know what’s happening—or even if they don’t know, they’ll fill in the blanks with probably not very positive things. Once fear starts to creep in, it erodes trust and confidence, and it also takes away from participation. No matter what transformation you’re trying to achieve in your organization, you need your people engaged.

Executives need to be honest about what’s happening. I’ve seen a lot of examples where leadership is not honest—they say, “Don’t worry, you’re not going to lose your job, it’s going to be fine,” while everyone knows their job will either be lost or significantly changed. That level of honesty is important: having a uniform communication strategy to communicate honestly to people and say, “Things will change. Things are already changing, but we will take care of you, and this is how it’s going to look.” I’m not saying to lie; you need to be appropriately honest and say, “Yes, there will probably be a reduction in roles, but this is the plan.” How do we communicate honestly to people and make sure they understand that if their job will be eliminated, how will they be supported to develop capabilities and skills to go into another role or do something else somewhere else? That’s a human thing to do—be honest and help people develop that capability.

The second part of that plan is to have, whether it’s AI literacy or AI retraining, whatever the organization decides to do to help people develop the skill set they don’t have. Organizationally, but also, as someone said at an event I attended recently, “What used to be a soft skill now really becomes a hard skill,” because that’s at the heart of everything. As more process-driven tasks are taken away by machines, those human skills will become very, very important and already are.

Ross Dawson: Yeah, and arguably, that’s one of the possible benefits of AI—it helps us to become more human, or develop our intrinsically and distinctly human capabilities. So, Joanna, where can people go to find out more about your work?

Joanna Michalska: They can reach out to me on LinkedIn, And I do have also my website, that’s called ethicagroup.ai, and I’m happy to connect on any topic related to what we’ve just discussed, especially executive authority, how we become more human, and how we can be at the center of what we can actually do within this very fast-moving environment. How do we have as executives and leaders more impact on changing this reality? Because, to your point earlier, if each of us doesn’t take that responsibility, nothing’s really going to change.

Ross Dawson: Indeed. Thank you so much for your time and your insights, Joanna.

Joanna Michalska: Thank you, Ross. Thank you for having me.

The post Joanna Michalska on AI governance, decision architectures, accountability pathways, and neuroscience in organizational transformation (AC Ep36) appeared first on Humans + AI.

“You and I, we’re part of this last analog generation. We had the opportunity to grow up in a time and age where our brains had to evolve against friction.”

–Cornelia C. Walther

Cornelia C. Walther is Senior Fellow at Wharton School, a Visiting Research Fellow at Harvard University, and the Director of POZE, a global alliance for systemic change. She is author of many books, with her latest book, Artificial Intelligence for Inspired Action (AI4IA), due out shortly. She was previously a humanitarian leader working for over 20 years at the United Nations driving social change globally.

Website:

pozebeingchange

LinkedIn Profile:

Cornelia C. Walther

University Profile:

knowledge.wharton

Ross Dawson: Cornelia, it is fantastic to have you on the show

Cornelia Walther: Thank you for having me Ross.

Ross: So your work is very wonderfully humans plus AI, in being able to look at humans and humanity and how we can amplify the best as possible. That’s one really interesting starting point is your idea of the hybrid tipping zone. Could you share with us what that is?

Cornelia: Yes, happy to. I would argue that we’re currently navigating a very dangerous transition where we have four disconnected yet mutually accelerating phenomena happening. At the micro level, we have agency decay, and I’m sure we’ll talk more about that later, but individuals are gradually delegating ever more of their thinking, feeling, and doing to AI.

We’re losing not only control, but also the appetite and ability to take on all of these aspects, which are part of being ourselves. At the meso level, we have AI mainstreaming, where institutions—public, private, academic—are rushing to jump on the AI train, even though there are no medium or long-term evidences about how the consequences will play out.

Then at the macro level, we have the race towards AI supremacy, which, if we’re honest, is not just something that the tech giants are engaged in, but also governments, because this is not just about money, it’s also about power and geopolitical rivalry. And finally, at the meta level, we have the deterioration of the planet, with seven out of nine boundaries now crossed, some with partially irreversible damages.

Now, you have these four phenomena happening in parallel, simultaneously, and mutually accelerating each other. So the time to do something—and I would argue that the human level is the one where we have the most leeway, at least for now, to act—is now. You and I, we’re part of this last analog generation.

We had the opportunity to grow up in a time and age where our brains had to evolve against friction. I don’t know about you, but I didn’t have a cell phone when I was a child, so I still remember my grandmother’s phone number from when I was five years old. Today, I barely remember my own. Same thing with Google Maps—when was the last time you went to a city and explored with a paper map?

Now, these are isolated functions in the brain, but with ChatGPT, there’s this general offloading opportunity, which is very convenient. But being human, I would argue, it’s a very dangerous luxury to have.

Ross: I just want to dig down quite a lot in there, but I want to come back to this. So, just that phrase—the hybrid tipping zone. The hybrid is the humans plus AI, so humans and AI are essentially, whatever words we use, now working in tandem. The tipping zone suggests that it could tip in more than one way.

So I suppose the issue then is, what are those futures? Which way could it tip, and what are the things we can do to push it in one way or another—obviously towards the more desirable outcome?

Cornelia: Thank you. I think you’re pointing towards a very important aspect, which is that tipping points can be positive or negative, but the essential thing is that we can do something to influence which way it goes. Right now, we consider AI like this big phenomenon that is happening to us. It is not—it is happening with, amongst, and because of us.

I think that is the big change that needs to happen in our minds, which is that AI is neutral at the end of the day. It’s a means to an end, not an end in itself.

We have an opportunity to shift from the old saying—which I think still holds true—garbage in, garbage out, towards values in, values out. But for that, we need to start offline and think: what are the values that we stand for? What is the world that we want to live in and leave behind?

As you know, I’m a big defender of pro social AI, which refers to AI systems that are deliberately tailored, trained, tested, and targeted to bring out the best in and for people and planet.

Ross: So again, lots of angles to dig into, but I just want to come back to that agency decay. I created a framework around the cognitive impact of AI, going from, at the bottom, cognitive corruption and cognitive erosion, through to neutral aspects, to the potential for cognitive augmentation. There are some individuals, of course, who are getting their thinking corrupted or eroded, as you’ve suggested; others are using it well and in ways which are potentially enhancing their cognition.

So, there is what individuals can do to be able to do that. There’s also what institutions, including education and employers, can do to provide the conditions where people are more likely to have a positive impact on cognition. But more broadly, the question is, again, how can we tip that more in the positive direction? Because absolutely, not just the potential, but the reality of cognitive erosion—or agency decay, as you describe it, which I think is a great phrase. So are there things we can do to move away from the widespread agency decay, which we are in danger of?

Cornelia: Yeah, I think maybe we could marry our two frameworks, because the scale of agency decay that I have developed looks at experience, experimentation, integration, reliance, and addiction. I would say we have now passed the stage of experimentation, and most of us are very deeply into the field of integration. That means we’re just half a step away from reliance, where all of a sudden it becomes nearly unthinkable to write that email yourself, to do that calendar scheduling yourself, or to write that report from scratch. But that means we’re just one step away from full-blown addiction.

At least now, we still have the possibility to compare the before and after, which comes back to us as an analog generation. Now is the time to invest in what I would call double literacy—a holistic understanding of our NI, our natural intelligence, but also our algorithmic, our AI. That requires a double literacy—not just AI literacy or digital literacy, but the complementarity of these two intelligences and their mutual influence, because none of them happens in a vacuum anymore.

Ross: Absolutely, So what you described—experiment, integration, reliance, addiction—sounds like a slippery slope. So, what are the things we can do to mitigate or push back against that, to use AI without being over-reliant, and where that experiment leads to integration in a positive way? What can we do, either as individuals or as employers or institutions, to stop that negative slide and potentially push back to a more positive use and frame?

Cornelia: A very useful tool that I have found resonates with many people is the A frame, which looks at awareness, appreciation, acceptance, and accountability. I have an alliteration affinity, as you can see. The awareness stage looks at the mindset itself and really disciplines us not to slip down that slope, but to be aware of the steps we’re taking.

The appreciation is about what makes us, in our own NI, unique, and the appreciation of where, in combination with certain external tools, it can be better. We all have gaps, we all have weaknesses, and that’s what we have to accept. The human being, even though now it’s sometimes put in opposition to AI as the better one, is not perfect either. Like probably you and most of the listeners have read Thinking, Fast and Slow by Daniel Kahneman and many others—there are libraries about human heuristics, human fallacies, our inability for actual rational thinking. But the fact that you have read a book does not mean that you are immune to that. We need to accept that this is part of our modus operandi, and in the same way as we are imperfect, AI, in many different ways, is also imperfect.

And finally, the accountability. Because at the end of the day, no matter how powerful our tools are going to be, we as the human decision makers should consider ourselves accountable for the outcomes.

Ross: Absolutely, that’s one of the points I make. We can’t obviously make machines accountable—ultimately, the accountability resides in humans. So we have to design systems, which I think provides a bit of a transition to pro social AI. So what is pro social AI, how do we build it, how do we deploy that, and how do we make that the center of AI development?

Cornelia: Thank you for that. Pro social AI, in a way, is very simple. It’s the intent that matters, but it starts from scratch, so you have the regenerative intent embedded into the algorithmic architecture. It has four key elements that can be measured, tracked, and can also serve to sensitize those who use it and those who design it—tailored, framed, tested, targeted.

The pro social AI index that I’ve been working on over the past months combines that with the quadruple bottom line: purpose, people, profit, planet. Now all of a sudden, rather than talking in an airy-fairy way about ethical AI—which is great and necessary, but I would argue is not enough—we need to systematically think about how we can harness AI as a catalyst of positive transformation that is with environmental dignity and seeks planetary health. How can we measure that?

Ross: And so, what are we measuring? Are we measuring an AI system, or what is the assessment tool? What is it that is being assessed?

Cornelia: It’s the how and the what for. For example, what data has been used? Is the data really representative? We know that the majority of AI tools are biased. And the other question is, is it only used for efficiency and effectiveness, but to what end?

Ross: Yes, as we are seeing in current conversations around the use of models at Anthropic and OpenAI, there are tools, and there are questions around how they are used, not just what the tools are.

Cornelia: Yes, so again, it comes back to the need for awareness and for hybrid intelligence, because at the end of the day, we can’t rely on companies whose purpose is to make money to give systems that serve people and planet first and foremost.

Ross: This goes on to another one of your wonderful framings, which is AI for IA—AI for inspired action—around this idea of how do we amplify humans and humanity. Of course, this goes on to everything we’ve been discussing so far. But I think one of the things which is very useful there is AI, in a way, leading to humans taking action which is inspired around envisaging what is possible. So, how can we inspire positive action by people in the framing we’ve discussed?

Cornelia: AI for IA is the title of the new book that’s coming out next month. But also, as with most of the things I’m saying, it’s not about the technology—it’s about the human being. We can’t expect the technology of tomorrow to be better than the humans of today.

As I said before, garbage in, garbage out, or values in, values out—it’s so simple and it’s so uncomfortable, it’s so cumbersome, right? Because we like quick fixes. But unfortunately, AI or technology in general is not going to save us from ourselves, and as it is right now, we’re straightforward on a trend to repeat the mistakes made during the first, second, and third industrial revolutions, where technology and innovation were driven primarily by commercial intent.

Now, I would argue that this time around, we can’t leave it at that, because this fourth industrial revolution has such a strong impact on the way we think, feel, and interact, that we need to start in our very own little courtyard to think: what kind of me do I want to see amplified?

Ross: Yes, yes. I’ve always thought that if AI amplifies us, or technology generally amplifies us, we will discover who we are, because the more we are amplified, the more we see ourselves writ large. But we have choices around, as you say, what aspects of who we are as individuals and as a society we can amplify. That’s the critical choice. So the question is, how do we bring awareness to your word around what it is about us that we want to amplify, and how do we then selectively amplify that, rather than also amplify the negative aspects of humanity?

Cornelia: The first thing, and that’s a simple one, is the A frame. I would argue that’s something everyone can integrate in their daily routine in a very simple way, to remind us of the four A’s: awareness, appreciation, acceptance, accountability. The other one, at the institutional level, is the integration of double literacy.

Right now, there’s a lot of hype in schools and at the governmental level about AI literacy and digital literacy. I think that’s only half of the equation. This is now an opportunity to take a step back and finally address this gap that has characterized education systems for many decades, where thinking and thinking about thinking—metacognition—is not taught in schools. Systems thinking, understanding cognitive biases, understanding interplays—now is the time to learn about that. If the future will be populated by humans that interact with artificial counterparts configured to address and exploit every single one of our human Achilles heels, then we would be better advised to know those Achilles heels. So, I think these are two relatively simple ways moving forward that could take us to a better place.

Ross: So this goes to one of your other books on human leadership for humane technology. So leadership of course, everyone is a leader in who they touch. We also have more formal leaders of organizations, nations, political parties, NGOs, and so on. But just taking this into a business context, there are many leaders now of organizations trying to transform their organizations because they understand that the world is different, and they need to be a different organization. They still need to make money to pay for their staff and what they are doing to develop the organization, but they have multiple purposes and multiple stakeholders. So, just thinking from an organizational leader perspective, what does human leadership for humane technology mean? What does that look like? What are the behaviors? What are the ways we can see that would show us?

Cornelia: I think first, it’s a reframing away from this very narrow scope of return on investment, which has characterized the business scene for many decades, and looking at return on values. What is the bigger picture that we are actually part of and shaping here? What’s the why at the end of the day? I think that matters for leaders who are in their place to guide others, and guidance is not just telling people what they have to do, but also inspiring them to want to do it.

Inspiration, at the end of the day, is something that comes from the inside out, because you see in the other person something that you would like in yourself. Power and money are not it—it’s vision. I think this is maybe the one thing that is right now missing. We all tend to see the opportunity, but then we go with what everybody else is doing, because we don’t really take the time to step back and think, well, there is the path of everyone, and there’s another one—how should I explore that one?

Especially amidst AI, where just upscaling your company with additional tools is not really going to set you apart, it matters twice as much to not just think about how do I do more of the same with less investment and faster, but what makes me unique, and how can I now use the artificial treasure chests to amplify that?

Ross: Yes, yes. I think purpose is now well recognized beyond the business agenda. One of the critical aspects is that it attracts the most talented people, but also, over the years, we’ve had more and more opportunities to be different as an organization. Back in the late ’90s and so on, organizations looked more and more the same. Now there are more and more opportunities to be different. The way in which AI and other technologies are brought into organizations gives an extraordinary array of possibilities to be unique, as you’ve described, and distinctive, which gives you a competitive position as well as being able to attract people who are aligned with your purpose.

Cornelia: Yes, exactly. But for that, you need to know your purpose first.

Ross: From everything we’ve just been talking about, or anything else, are there any examples of organizations or initiatives that you think are exemplars or support the way in which, or show how, we could be approaching this well?

Cornelia: I think—this will now sound very biased—but I’m currently working with Sunway University, and I think they are the kind of academic institution that is showing a different path, seeking to leverage technology to be more sustainable, bringing in dimensions such as planetary health, like the Sunway Centre for Planetary Health, and thinking about business in a re-envisioned way, with the Institute for Global Strategy and Competitiveness.

I think there are examples at the institutional level, there are examples at the individual level, and sometimes the most inspiring individuals are not those that make the headlines. That’s maybe, sorry, just on that, for me the most important takeaway: no matter which place one is in the social food chain, the essential thing is, who are you and how can you inspire the person next to you to make it a better day, to make it a better future.

Ross: Yes, in fact, that word “inspired,” as you mentioned before. So that’s Sunway University in Malaysia?

Cornelia: I think they are definitely a very, very good illustration of that.

Ross: Just pulling this back to the global frame, and this gets quite macro, but I think it is very important. It pulls together some of the things we’ve pointed to—the difference between the approach of the United States, China, Europe, in how they are, you know, essentially the leaders in AI and how they’re going about it, but where the global south more generally, I think there’s some interesting things. Arguably, there’s a far more positive attitude generally in the populations, a sense of the opportunity to transform themselves, but of course a very different orientation in how they want to use and apply AI and in creating value for individuals, nations, and society. So how would you frame those four—the US, China, Europe, and the global south—and how they are, or could be, approaching the development of AI?

Cornelia: Thank you for that. I think right now there are three mainstream patterns: the US, which is—I’m overly simplifying and aware of that—the US path, which is business overall; the European model, which is regulation overall; and the Chinese model, which is state dominance. I would argue there’s a fourth path, and I think that’s where leaders in the global south can step in.

You might know I’m working, on the one hand, in Malaysia and, on the other hand, in Morocco, on the development of a sort of national blueprint of what pro social AI can look like. I think now is the time—again, coming back to leadership—to think about how countries can walk a different path and be pioneers in a field that, yes, AI has been around for various decades, but the latest trend, the latest wave that is engulfing society since November 2022, is still relatively new. So why not have nations in the global south that are very different from the West chart their own path and make it pro social, pro people, pro planet, and pro potential—and that potential that they have themselves, which sets them apart and makes them unique.

Ross: Absolutely. Again, you mentioned Malaysia, Morocco. Looking around the world, of course, India is prominent. There are some African nations which have done some very interesting things. Just trying to think, where are other examples of these kinds of domestically born pro social initiatives happening? Of course, the Middle East—it’s quite different, because they’re wealthy, though they’re not among the major leaders, but there’s a whole array of different examples. Where would you point to as things which show how we could be using pro social AI at a national or regional level?

Cornelia: Unfortunately, right now, there is not one country where one could say they have taken it from A to Z, but I think there are very inspiring or positive examples. For example, Vietnam was the first country in ASEAN to endorse a law on AI ethics and regulation—I think that’s a very good one. Also, ASEAN has guidelines on ethics. All of these are points of departure. Switzerland did a very nice example of what public AI can look like. So there are a lot of very good examples.

The question is not so much about what to do, I think, but how to do it, and why. At the end of the day, it’s really that simple. What’s the intent behind it? What do we want the post-2030 agenda to look like? We know that the SDG—Sustainable Development Goals—are not going to be fulfilled between now and 2030. So are we learning from these lessons, or are we following the track pattern of doing more of the same and maybe throwing in a couple of additional indicators, or can we really take a step back and look ourselves and the world in the face and think, what have we missed? Now, frame it however you want, but think about hybrid development goals and ways in which means and ends—society and business—come together into a more holistic equation that respects planetary health. Because at the end of the day, our survival still depends on the survival and flourishing of planet Earth, and some might cherish the idea of emigrating to Mars, but I still think that overall the majority of us would prefer to stay here.

Ross: Yes, planet Earth is beautiful, and it’d be nice to keep it that way. How can people find more about your work? Could you just tell people about your new book and any resources where people can find out more?

Cornelia: Thank you so much. They are very welcome to reach out via LinkedIn. Also, I’m writing regularly on Psychology Today, on Knowledge at Wharton, and various other platforms. The new book that you mentioned is coming out next month, and there will be another one, hopefully by the end of the year. Overall, feel free to reach out. I really feel that the more people get into this different trend of thinking, the better. But thank you so much for the opportunity.

Ross: Thanks so much for all of your work, Cornelia. It’s very important.

The post Cornelia C. Walther on AI for Inspired Action, return on values, prosocial AI, and the hybrid tipping zone (AC Ep35) appeared first on Humans + AI.

“Transparency has to be built into the structure so that you know where the decision is made, what authorizations are given, and have an audit trail visible so you can always see what is going on.”

–Ross Dawson

Ross Dawson is a futurist, keynote speaker, strategy advisor, author, and host of Amplifying Cognition podcast. He is Chairman of the Advanced Human Technologies group of companies and Founder of Humans + AI startup Informivity. He has delivered keynote speeches and strategy workshops in 33 countries and is the bestselling author of 5 books, most recently Thriving on Overload.

Website:

Collaborating with AI Agents

Intelligent AI Delegation

Agentic Interactions

LinkedIn Profile:

Ross Dawson

Ross Dawson: This episode is a little bit different. Instead of doing an interview with somebody remarkable, as usual, today I’m going to just share a bit of an update and then share insights from three recent research papers that dig into something which I think is exceptionally important, which is how humans work with AI agentic systems. And we’ll look at a few different layers of that, from how small humans plus agent teams work through to how we can delegate decisions to AI through to some of the broader implications.

But first, a bit of an update.

2026 seems to be moving exceptionally fast. It’s a very interesting time to be alive, and I think it’s pretty even hard to see what the end of this year is going to look like.

So for me, I am doing my client work as usual. So I’ve got keynotes around the world on usually various things related to AI, the future of AI, humans plus AI, and so on. A few industry-specific ones in financial services and so on.

And also doing some work as an advisor on AI transformation programs, so helping organizations and their leaders to frame the pathways, drawing on my AI roadmap framework in how it is you look at the phases, mapping those out, working out the issues, and being able to guide and coach the leaders to do that effectively.

But the rest of my time is focused on three ventures, and I’ll share some more about these later on. But these are fairly evidently tied to my core interests.

Fractious is our AI for strategy app. So this was really building a way in which we can capture the detailed nuance of the strategic thinking of leaders of the organization, to disambiguate it, to clarify it, and enable that to then be built into strategic options, strategic hypotheses, and to be able to evolve effectively.

So that’ll be in beta soon. Please reach out if you’re interested in being part of the beta program, and that’ll go to market. So that’s deeply involved in that.

We also have our Thought Weaver software, rebuilding previous software which had already built on AI-augmented thinking workflows. So again, that’ll be going to beta. That’s more an individual tool that will be going into beta in the next weeks.

So again, go to Thought Weaver. Actually, don’t—the website isn’t updated yet—but I’ll let you know when it’s out, or keep posted for updates on that.

And also building an enterprise course on humans plus AI teaming. It’s my fundamental belief that we’ve kind of been through the phase of augmentation of individuals, and we still need to work hard at doing that better. But the next phase for organizations is to focus on teams.

How do you work with teams where we have both human members and AI Agentic members? And it creates a whole different series of dynamics and new skills and capabilities. It really calls for how to participate in the humans plus AI team and how to lead humans plus AI teams.

And that is again going into the first few test organizations in the next month or so. So again, just let me know.

So today what we’re going to look at is this theme: teams of humans working with AI agents. So not individual AI as in chat, but where we have a lot of agents with various degrees of autonomy, but also agentic systems where these agents are interacting with each other as well as with humans.

So there are three papers which I want to just talk about, just give you a quick overview, and please go and check out the papers in more detail if you’re interested. There’ll be links in the show notes.

First is Collaborating with AI Agents: A Field Experiment on Teamwork, Productivity and Performance, by Harang Ju at Johns Hopkins and Sinan Aral at MIT.

So this, there was an experiment which had over 2,300 participants who were working on creating advertisements. And they had a whole array of humans plus AI, human-human teams, human-AI teams, sort of quite small or just in duos and so on, working on being able to create those which were then assessed in terms of quality and how they worked.

So a few particularly interesting findings from that.

So individually, just having a human-AI team essentially enhanced performance significantly compared to just human-only teams. And so they were able to move faster and to complete more of their tasks, and the quality was strong.

But there’s a phrase which is commonly used around the jagged frontier of capability of AI, and it was quite clear that there were some domains where AI does very well and others where it didn’t.

And so this comes to the part where, in terms of the design of the tasks, the design of the human-AI systems, and also the understanding by the human users of what AI is good at or not, is fundamental in being able to do that.

And so in some cases, if AI was used in some domains such as image quality, they actually decreased quality. So we need to understand where and how both to apply AI in this jagged frontier and design the systems around that.

This changes the role of the humans, of course. Humans then tend to delegate more. And there’s one of the things which they tested for, which is how do you behave differently if you know your teammate is an AI as opposed to not knowing whether a human or AI.

And it changes. So they become more task-oriented. They are less using the social cues to interact, and they are essentially becoming more efficient.

But some of these social cues which are valuable in the human-human collaboration started to disappear.

And this automation process meant that there was not, in the end, as much creative diversity.

Now I’ve often pointed to the role of AI in creativity tasks. It depends fundamentally on the architecture—where does the AI sit in terms of initial ideas which are then sorted by filtered by humans and then are involved, or where it sits in that process.

But in this particular structure, they found that humans plus AI teams started to create more and more similar-type outputs.

So this homogenization of outputs in these human-AI teams was very notable and significant. And so this again creates a design factor for how it is that we build human-AI systems which actually do not lead to homogeneous output.

And we’re making sure that we are ensuring that the human diversity is maintained.

Often that can be done by being able to have human outputs first without AI then blunting or narrowing the breadth of the creative outputs of humans.

Second paper I’d like to point to is called Intelligent AI Delegation, from a team at Google DeepMind.

So this is this point where we now have not just single AI agents to delegate decisions to or problems to, but in fact systems of AI. And so this creates a different challenge.

And the key point is, I’m saying this, is around you are delegating tasks, but when you are delegating tasks it’s more than just saying, okay, which agent gets the task. You have to understand responsibility.

So where does accountability reside? Who is responsible for that? How clarity around the roles of the agents, what are the boundaries of what it is they can do and cannot do, the clarity of the intent, and how that’s communicated and cascaded through the agents, and the critical role of trust and appropriate degrees of trust in the systems.

So this means that we have to define what are the different characteristics of the task. And in the paper it goes through quite a few different characteristics. And a few of the critical ones was the degree of uncertainty around the task.

Obviously, if it is very clear that can be appropriately delegated, but many tasks and problems are uncertain. And so this creates a different dynamic.

Whether verifiable, as you know you have high-quality information, or whether that’s the degree of uncertainty around whether decisions are reversible, the degree of subjectivity, because not everything is data-driven.

And so assessing these task characteristics start to define where human judgment plays a role, how do you create those checks, and how do you build that.

So this creates a system so intelligent delegation is not just how the humans delegate, but in turn the structure of how that cascades down through the agents.

So this requires this idea of dynamic assessment.

So you’re not just setting and forgetting. You are continuously reassessing what is happening with the context, what is changing in the stakes, any uncertainty.

So you’re coming back to be able to ensure there’s not just a single delegation structure, but you’re changing it over time.

And you’ll continue to adapt as you’re executing, and be able to monitor, replan, and set.

So transparency has to be built into the structure so that you have where the decision is made, what authorizations are given, you know where the audit trail is visible so you can always see what is going on in those structures.

And being able to scale how you are coordinating the systems.

And if it’s just small scale that’s fine, but you want to be able to build something which has been able to move across many agents.

And so this requires a way of being able to discover which agents are most appropriate and be able to essentially establish the delegation of a particular task to them again on a dynamic basis.

And essentially this final principle of systemic resilience, where you have to expect that things will go wrong.

So there’s continuing monitoring, being able to understand that these systems can be attacked in various ways and being able to recover.

So, very solid paper, quite deep, but really giving some very good principles for how it is we can delegate to AI systems.

So the final of the three papers goes to a bit of a higher level.

It’s called Agentic Interactions, and it’s from Alex Imas, Sanjog Misra of the University of Chicago, and Kevin Lee at the University of Michigan.

And what they’re looking at is what happens on a macro scale when increasingly decisions are delegated to AI agents.

So this is the agent economy that I’ve been talking about for a very long time, which is now very much coming to the fore.

And so what they do is they look at what happens when we start to delegate more and more economic decisions, such as buying and selling decisions.

So what they found is extraordinarily interesting.

They found that the AI agents in fact do behave very similarly to their human creators.

And in fact what you can observe is that there are differences in the agents where you can infer the gender and the personality of the person who is delegating the agent.

Even though there is no information, the agent doesn’t even know what the gender or the personality is, they are actually flowing through.

So in fact agents represent us in the market as it were, potentially very accurately.

But this goes directly to the second point where this idea of machine fluency.

And so AI fluency is very much a term in vogue at the moment.

So the authors talk about this idea of machine fluency which is how well can a user put their intent and align that with the agent so the agent is aligned with them.

And in fact they found that there’s very significant degrees of difference in those.

And those people who are better at being able to get their agents to express their wishes could in fact amplify the economic outcomes of these people.

And related to that in fact they showed there was a correlation that higher educational levels mean that you were able to better delegate to AI, and your AI agents performed better and gave you better returns.

So again pointing to these ways in which we’re starting to see potentials for aggravation of differences in the agentic economy when our agents who act for us in the economy start to reflect among other things educational differences or capabilities in how it is we express our results and our intentions through AI.

There was one very interesting and I suppose counterintuitive result.

Women get better outcomes in negotiation when using AI agents than they do in human-to-human interactions.

Again this is without the AI agents knowing that they are representing a woman or not.

But in fact this shows that the style and the way on the machine fluency the ways in which women are able to instruct and put their intent into the AI agents is in this study superior to those of males.

And there’s of course in the real world unfortunately a bias towards male performance in negotiation.

And that was inversed in the study.

So exceptionally interesting.

So just pulling back some of the common themes of these three papers.

We increasingly want a world where humans have relationships to agents.

We are starting to work with them in teams and systems.

And we’re starting to build economies where humans are represented by agents.

And essentially our relationship to those agents and our ability to delegate effectively is driving value of course to the individual but also across these agentic systems that are emerging.

So this is early on because the realities of these agentic human-agent systems are pretty early at this point.

But this starts to point to some of the potential, some of the challenges, some of the opportunities, and some of the work that we have to do.

So I will be sharing more on these kinds of topics in my interviews with people and also of course on the Humans Plus AI website.

So just go to humansplus.ai.

Actually to be frank it hasn’t been updated a lot recently but we will be sharing a lot more there.

Or LinkedIn is where I share the most actually, and getting back on Twitter as well if you’re interested.

But I’ll be diving deep and trying to share what I find is useful as well as interesting in helping us to create a world where humans are first.

AI complements us.

The reality is we are moving to humans plus AI systems.

And if we design that well with the right intentions we can make this absolutely one which drives human value first.

So glad to have you on the journey.

Have a wonderful rest of your day.

The post Ross Dawson on Humans + AI Agentic Systems (AC Ep34) appeared first on Humans + AI.

“In this sense, human and AI means a synergy where teams of humans and AI together lead to superior outcomes than either the human or the AI operating in isolation.”

– Davide Dell’Anna

Davide Dell’Anna is Assistant Professor of Responsible AI at Utrecht University, and a member of the Hybrid Intelligence Centre. His research focuses on how AI can cooperate synergistically and proactively with humans. Davide has published a wide range of leading research in the space.

Website:

davidedellanna.com

LinkedIn Profile:

Davide Dell’Anna

University Profile:

Davide Dell’Anna

Ross Dawson: Hi Davide. It’s wonderful to have you on the show.

Davide Dell’Anna: Hi Ross, nice to meet you. Thank you so much for having me.

Ross: So you do a lot of work around what you call hybrid intelligence, and I think that’s pretty well aligned with a lot of the topics we have on the podcast. But I’d love to hear your definition and framing—what is hybrid intelligence?

Davide: Well, thank you so much for the question. Hybrid intelligence is a new paradigm, or a paradigm that tries to move the public narrative away from the common focus on replacement—AI or robots taking over our jobs. While that’s an understandable fear, more scientifically and societally, I think it’s more interesting and relevant to think of humans and AI as collaborators.

In this sense, human and AI means a synergy where teams of humans and AI together lead to superior outcomes than either the human or the AI operating in isolation. In a human-AI team, members can compensate for each other’s weaknesses and amplify each other’s strengths. The goal is not to substitute human capabilities, but to augment them.

This immediately moves the discussion from “what can the AI do to replace me?” to “how can we design the best possible team to work together?” I think that’s the foundation of the concept of hybrid intelligence. So hybrid intelligence, per se, is the ultimate goal. We aim at designing or engineering these human-AI teams so that we can effectively and responsibly collaborate together to achieve this superior type of intelligence, which we then call hybrid intelligence.

Ross: That’s fantastic. And so extremely aligned with the humans plus AI thesis. That’s very similar to what I might have said myself, not using the word hybrid intelligence, but humans plus AI to say the same thing. We want to dive into the humans-AI teaming specifically in a moment.

But in some of your writing, you’ve commented that, while others are thinking about augmentation in various ways, you point out that these are not necessarily as holistic as they could be. So what do you think is missing in some of the other ways people are approaching AI as a tool of augmentation?

Davide: Yeah, so I think when you look at the literature—as a computer scientist myself, I notice how easily I fall into the trap of only discussing AI capabilities. When I talk about AI or even human-AI teams, I end up talking about how I can build the AI to do this, or how I can improve the process in this way. Most of the literature does that as well. There’s a technology-centric perspective to the discussion of even human-AI teams.

We try to understand what we can build from the AI point of view to improve a team. But if you think of human-AI teams in this way, you realize that this significantly limits our vocabulary and our ability to look at the team from a broader, system-level perspective, where each member—including and especially human team members—is treated individually, and their skills and identity are considered and leveraged.

So, if you look at the literature, you often end up talking about how to add one feature to the AI or how to extend its feature set in other ways. But what people often miss is looking at the weaknesses and strengths of the different individuals, so that we can engineer for their compensation and amplification. Machines and people are fundamentally different: humans are good at some things, AI is good at others, and we shouldn’t try to negate or hide or be ashamed of the things we’re worse at than AI, and vice versa. Instead, we should leverage those differences.

For instance, just as an example, consider memory and context awareness. At the moment, at least, AI is much more powerful in having access to memory and retrieving it in a matter of seconds—AI can access basically the whole internet. But often, when you talk nowadays with these language model agents, they are completely decontextualized. They talk in the same way to millions across the world and often have very little clue about who the specific person is in front of them, what that person’s specific situation is—maybe they’re in an airport with noise, or just one minute from giving a lecture and in a rush. The type of things you might say also change based on the specific situation.

While this is a limitation of AI, we shouldn’t forget that there is the human there. The human has that contextual knowledge. The human brings that crucial context. Sometimes we tend to say, “Okay, but then we can build an AI that can understand the context around it,” but we already have the human for that.

Ross: Yes, yes. I don’t think that’s what I call the framing. Framing should come from the human, because that’s what we understand—including the ethical and other human aspects of the context, as well as that broader frame. It’s interesting because, in talking about hybrid intelligence, I think many who come to augmentation or hybrid intelligence think of it on an individual basis: how can an individual be augmented by AI, or, for example, in playing various games or simulations, humans plus AI teaming together, collaborating. But the team means you have multiple humans and quite probably multiple AI agents.

So, in your research, what have you observed if you’re comparing a human-only team and a team which has both human and AI participants? What are some of the things that are the same, and what are some of the things that are different?

Davide: Yes, this is a very interesting question. We’ve recently done work in collaboration with a number of researchers from the Hybrid Intelligence Center, which I am part of. If you’re not familiar with it, the Hybrid Intelligence Center is a collaboration that involves practically all the Dutch universities focused on hybrid intelligence, and it’s a long project—lasting around 10 years.

One of the works we’ve done recently is to try to study to what extent established properties of effective human teams could be used to characterize human-AI teams. We looked at instruments that people use in practice to characterize human teams. One of them is called the Team Diagnostic Survey, which is an instrument people use to diagnose the strengths and weaknesses of human teams. It includes a number of dimensions that are generally considered important for effective human teams.

These include aspects like members demonstrating their commitment to the team by putting in extra time and effort to help it succeed, the presence of coaches available in the team to help the team improve over time, and things related to the satisfaction of the members with the team, with the relationships with other members, and with the work they’re doing.

What we’ve done was to study the extent to which we could use these dimensions to characterize human-AI teams. We looked at different types of configurations of teams—some had one AI agent and one human, others had multiple agents and multiple humans, for example in a warehouse context where you have multiple robots helping out in the warehouse that have to cooperate and collaborate with multiple humans.

We tried to understand whether the properties of—by the way, we also looked at an interesting case, which is human-animal-animal teams, which is another example that’s interesting in the context of hybrid intelligence. You see very often in human-animal interaction—basically two species, two alien species—interacting and collaborating with each other. They often manage to collaborate pretty effectively, and there is an awareness of what both the humans and the animals are doing that is fascinating, at least for me.

So, we tried to analyze whether properties of human teams could be understood when looking at human-AI teams or hybrid teams, and to what extent. One of the things we found is that some concepts are very well understood and easily applicable to different types of hybrid teams. For example, the idea of interdependence—the fact that members in the team, in order to be a team, need to be mutually dependent, at least to some extent. Otherwise, if they’re all doing separate jobs, there’s a lack of common goal.

There are also things related to having a clear mission or a clear objective as a team, and aspects related to the possibility of exhibiting autonomy in the operation of the team and taking initiative. Also, the presence and awareness of team norms, like a shared ethical code or shared knowledge about what is appropriate or not. These were things that we found people could easily understand and apply to different configurations of teams.

Ross: Just actually, one thing—I don’t know if you’re familiar with the work of Mohammad Hussain Johari, who did this wonderful paper called “What Human-Horse Interactions May Teach Us About Effective Human-AI Interactions.” Again, these are the cases where we can have these parallels—learning how to do human-AI interactions from human-human and human-animal interactions. But again, it comes back to that original question: what is the same? I think you described many of those facets of the nature of teams and collaboration, which means they are the same. But there are, of course, some differences. One of the many differences is accountability, essentially, where the AI agents are not accountable, whereas the humans are. That’s one thing. So, this allocation of decision rights across different participants—human and AI—needs to take into account that they’re not equal participants. Humans have accountability, and AI does not. That’s one possible example.

Davide: Yeah, definitely. I totally agree, and I remember the paper you mentioned. I agree that human-animal collaboration is a very interesting source of inspiration. When looking at this paper, we looked at the case of shepherds and shepherd dogs. I didn’t know much about it before, but then I started digging a little bit. Shepherd dogs are trained at the beginning, but over time, they learn a type of communication with the shepherd. Through whistles, the shepherd can give very short commands, and then the shepherd dogs—even in pairs—can quickly understand what they need to do. They go through the mountains, collect all the sheep, and bring them exactly as intended by the shepherd, with very little need for words or other types of communication. They manage to achieve their goals very effectively.

So, I think we have a lot to learn from these cases, even though it’s difficult to study. But just to mention differences, of course—one of the things that emerged from this paper is the inherent human-AI asymmetry. Like you mentioned, accountability is definitely one aspect. I think overall, we should always give the human a different type of role in the team, similar to the shepherd and the shepherd dogs. There is some hierarchy among the members, and this makes it possible for humans to preserve meaningful control in the interactions. This also implies that different rules or expectations apply to different team members.

Beyond these, there is asymmetry in skills and capabilities, as we mentioned earlier, and also in aspects related to the identity of the members. For instance, some AI could be more easily replaceable than humans. Think, for example, of robots in a warehouse. In a human team, you wouldn’t say you “replace” a team member—it’s not the nicest way to say you let someone go and bring someone else in. But with robots, you could say, “I replace this machine because it’s not working anymore,” and that’s fine. We can replace machines with little consequence, though this doesn’t always hold, because there are studies showing that people get attached to machines and AI in general.

There was a recent case of ChatGPT releasing a new version and stopping the previous one, and people complained because they got attached to the previous version. So, in some cases, replacing the AI member would work well, but in others, it needs to be done more carefully.

Ross: So one of the other things looked at is the evaluation of human-AI teams. If we’re looking at human teams and possibly relative performance compared to human-AI teams, what are ways in which we can measure effectiveness? I suppose this includes not just output or speed or outcomes, but potentially risk, uncertainty, explainability, or other factors.

Davide: Yes, this is an interesting question, and I think it’s still an open question to some extent. From the study I mentioned earlier, we looked at how people measure human team effectiveness. There are aspects concerning, of course, the success of the team in doing the task, but these are not the only measures of effectiveness that people consider in human teams.

People often consider things related to the satisfaction of the members—with their teammates, with the process of working together, and with the overall goals of the team. This often leads to reflection from the team itself during operation, at least in human teams, where people reassess and evaluate their output throughout the process to make sure satisfaction with the process and relationships goes well over time.

In general, there are aspects to measure concerning the effectiveness of teams related to the process itself, which are often forgotten. It’s a matter, at least from a research point of view, of resources, because to evaluate a full process over time, you need to run experiments for longer periods. Often people stop at one instant or a few interactions, but if you think of human teams, like the usual forming, storming, norming, and performing, that often goes over a long time. Teams often operate for a long time and improve over time. So, the process itself needs to be monitored and reassessed over time. This is a way to also measure the effectiveness of the team, but over time.

Ross: Interesting point, because as you say, the dynamics of team performance with a human team improve as people get to know each other and find ways of working. They can become cohesive as a team. That’s classically what happens in defense forces and in creating high-performance teams, where you understand and build trust in each other. Trust is a key component of that.

With AI agents, if they are well designed, they can learn themselves or respond to changing situations in order to evolve. But it becomes a different dynamic when you have humans building trust and mutual understanding, where that becomes a system in which the AI is potentially responding or evolving. At its best, there’s the potential for that to create a better performing team, but it does require both the attitudes of the humans and well the agents.

Davide: Related to this—if I can interrupt you—I think this is very important that you mentioned trust. Indeed, this is one of the aspects that needs to be considered very carefully. You shouldn’t over-trust another team member, but also shouldn’t under-trust. Appropriate trust is key.

One of the things that drives, at least in human teams, trust and overall performance is also team ethics. Related to the metrics you mentioned earlier, the ability of a team to gather around a shared ethical code and stick to that, and to continuously and regularly update each other’s norms and ensure that actions are aligned with the shared norms, is crucial. This ethical code significantly affects trust in operation.

You can see it very easily in human teams: considering ethical aspects is essential, and we take them into account all the time. We respect each other’s goals and values. We expect our collaborators to keep their promises and commitments, and if they cannot, they can explain or justify what they are doing. These justifications are also a key element. The ability to provide justifications for behavior is very important for hybrid teams as well. Not only the AI, but also the human should be able to justify their actions when necessary.

This is where the concept of hybrid teams and, in general, hybrid intelligence requires a bit of a philosophical shift from the traditional technology-centric perspective. For example, in AI, we often talk about explainability or explainable AI, which is about looking at model computations and understanding why a decision was made. But here, we’re talking about a different concept: justifiability, which looks at the same problem from a different angle. It considers team actions in the context of shared values, shared goals, and the norms we’ve agreed upon. This requires a shift in the way we implement AI agents—they need to be aware of these norms, able to learn and adapt to team norms, and reason about them in the same way we do in society.

Ross: Let’s say you’ve got an organization and they have teams, as most organizations do, and now we’re moving from classic human teams to humans plus AI teams—collaborative human-AI teams. What are the skills and capabilities that the individual participants and the leaders in the teams need to transition from human-only teams to teams that include both humans and AI members?

Davide: This is a complicated question, and I don’t have a full answer, but I can definitely reflect on different skills that a hybrid team should have. I’m thinking now of recent work—not published yet—where we started moving from the quality model work I mentioned earlier towards more detailed guidelines for human-AI teams. There, we developed a number of guidelines for organizations for putting in place and operating effective teams.

We categorized these guidelines in terms of different phases of team processes. For instance, we developed guidelines related to structuring the teamwork—the envisioning of the operations of the team, which roles the team members would have, which responsibilities the different team members should have. Here, I’m talking about team members, but I’m still referring to hybrid teams, so this applies to both humans and AI.

This also implies different types of skills that we often don’t have yet in AI systems. For example, flexible team composition is a type of skill required to make it possible at the early stage of the team to structure the team in the right way. There are also skills related to developing shared awareness and aspects related to breaking down the task collaboratively or ensuring a continuous evolution of the team over time, with regular reassessment of the output.

If you think of these notions, it’s easy to think about them in terms of traditional organizations, but when you imagine a human-AI team or a small hybrid organization, then this continuous evolution, regular output assessment, and flexible team composition are not so natural anymore. What does it mean for an LLM agent to interact with someone else? Usually, LLM architectures rely on static roles and predefined workflows—you need to define beforehand the prompts they will exchange—whereas humans use much more flexible protocols. We can adjust our protocols over time, monitor what we’re doing, and reassess whether it works or not, and change the protocols.

These are skills required for the assistants, but also for the organization itself to make hybrid teaming possible. One of the things that emerges in this recent work is a new figure that would probably come up in organizations: a team designer or a team facilitator. This is not a team member per se, but an expert in teams and AI teammates, who can perhaps configure the AI teammates based on the needs of the team, and provide human team members with information needed about the skills or capabilities of the specific AI team member. It’s an intermediary between humans and AI, with expertise that other human team members may not have, and could help these teams work together.

Ross: That’s fantastic. It’s wonderful to learn about all this work. Is there anywhere people can go to find out more about your research?

Davide: Yeah, sure. You can look me up at my website, davidedellanna.com. That’s my main website—I try to keep it up to date. Through there, you can see the different projects I’m involved in, the papers we’re working on, both with collaborators and with PhD and master students, who often bring great contributions to our research, even in their short studies.

That’s the main hub, and you can also find many openly available resources linked to the projects that people may find useful.

Ross: Fantastic. Well, it’s wonderful work—very highly aligned with the idea of hybrid intelligence, and it’s fantastic that you are focusing on that, because there’s not enough people yet focusing in the area. So you and your colleagues are ahead, and I’m sure many more will join you. Thank you so much for your time and your insights.

Davide: Thank you so much, Ross. Pleasure to meet you.

The post Davide Dell’Anna on hybrid intelligence, guidelines for human-AI teams, calibrating trust, and team ethics (AC Ep33) appeared first on Humans + AI.

“You can create a virtual board of directors that will have different expertises and that will come up with ideas that a given person may not come up with.”

– Felipe Csaszar

Felipe Csaszar is the Alexander M. Nick Professor and chair of the Strategy Area at the University of Michigan’s Ross School of Business. He has published and held senior editorial roles in top academic journals including Strategy Science, Management Science, and Organization Science, and is co-editor of the upcoming Handbook of AI and Strategy.

Website:

papers.ssrn.com

LinkedIn Profile:

Felipe Csaszar

University Profile:

Felipe Csaszar

Ross Dawson: Felipe, it’s a delight to have you on the show.

Felipe Csaszar: Oh, the pleasure is mine, Ross. Thank you very much for inviting me.

Ross Dawson: So many, many interesting things for us to dive into. But one of the themes that you’ve been doing a lot of research and work on recently is the role of AI in strategic decision making. Of course, humans have been traditionally the ones responsible for strategy, and presumably will continue to be for some time.

However, AI can play a role. Perhaps set the scene a little bit first in how you see this evolving.

Felipe Csaszar: Yeah, yeah. So, as you say, strategic decision making so far has always been a human task. People have been in charge of picking the strategy of a firm, of a startup, of anything, and AI opens a possibility that now you could have humans helped by AI, and maybe at some point, AI is designing the strategies of companies.

One way of thinking about why this may be the case is to think about the cognitive operations that are involved in strategic decision making. Before AI, that was my research—how people came up with strategies. There are three main cognitive operations. One is to search: you try different things, you try different ideas, until you find one which is good enough—that is searching.

The other is representing: you think about the world from a given perspective, and from that perspective, there’s a clear solution, at least for you. That’s another way of coming up with strategies. And then another one is aggregating: you have different opinions of different people, and you have to combine them. This can be done in different ways, but a typical one is to use the majority rule or unanimity rule sometimes. In reality, the way in which you combine ideas is much more complicated than that—you take parts of ideas, you pick and choose, and you combine something.

So there are these three operations: search, representation, and aggregation. And it turns out that AI can change each one of those. Let’s go one by one. So, search: now AIs, the current LLMs, they know much more about any domain than most people.

There’s no one who has read as much as an LLM, and they are quite fast, and you can have multiple LLMs doing things at the same time. So LLMs can search faster than humans and farther away, because you can only search things which you are familiar with, while an LLM is familiar with many, many things that we are not familiar with. So they can search faster and farther than humans—a big effect on search.

Then, representation: a typical example before AI about the value of representations is the story of Merrill Lynch. The big idea of Merrill Lynch was how good a bank would look if it was like a supermarket. That’s a shift in representations. You know how a bank looks like, but now you’re thinking of the bank from the perspective of a supermarket, and that leads to a number of changes in how you organize the bank, and that was the big idea of Mr. Merrill Lynch, and the rest is history.

That’s very difficult for a human—to change representations. People don’t like changing; it’s very difficult for them, while for an AI, it’s automatic, it’s free. You change their prompt, and immediately you will have a problem looked at from a different representation.

And then the last one was aggregating. You can aggregate with AI virtual personas. For example, you can create a virtual board of directors that will have different expertises and that will come up with ideas that a given person may not come up with. And now you can aggregate those. Those are just examples, because there are different ways of changing search, representation, and aggregation, but it’s very clear that AI, at least the current version of AI, has the potential to change these three cognitive operations of strategy.

Ross Dawson: That’s fantastic. It’s a novel framing—search, representation, aggregation. Many ways of framing strategy and the strategy process, and that is, I think, quite distinctive and very, very insightful, because it goes to the cognitive aspect of strategy.

There’s a lot to dig into there, but I’d like to start with the representation. I think of it as the mental models, and you can have implicit mental models and explicit mental models, and also individual mental models and collective mental models, which goes to the aggregation piece. But when you talk about representation, to what degree—I mean, you mentioned a metaphor there, which, of course, is a form of representing a strategic space. There are, of course, classic two by twos. There are also the mental models which were classically used in investment strategy. So what are the ways in which we can think about representation from a human cognitive perspective, before we look at how AI can complement it?

Felipe Csaszar: I think it’s important to distinguish—again, it’s three different things. There are three different types of representations. There are the internal representations: how people think in their minds about a given problem, and that usually people learn through experience, by doing things many times, by working at a given company—you start looking at the world from a given perspective.

Part of the internal representations you can learn at school, also, like the typical frameworks.

Then there are external representations—things that are outside our mind that help us make decisions. In strategy, essentially everything that we teach are external representations. The most famous one is called Porter’s Five Forces, and it’s a way of thinking about what affects the attractiveness of an industry in terms of five different things. This is useful to have as an external representation; it has many benefits, because you can write it down, you can externalize it, and once it’s outside of your mind, you free up space in your mind to think about other things, to consider other dimensions apart from those five.

External representations help you to expand the memory, the working memory that you have to think about strategy. Visuals in general, in strategy, are typical external representations. They play a very important role also because strategy usually involves multiple people, so you want everybody to be on the same page. A great way of doing that is by having a visual so that we all see the same.

So we have internal—what’s in your mind; external—what you can draw, essentially, in strategy. And then there are distributed representations, where multiple people—and now with AI, artifacts and software—among all of them, they share the whole representation, so they have parts of the representation. Then you need to aggregate those parts—partial representations; some of them can be internal, some of them are external, but they are aggregated in a given way. So representations are really core in strategic decision making. All strategic decisions come from a given set of representations.

Ross Dawson: Yeah, that’s fantastic. So looking at—so again, so much to dive into—but thinking about the visual representations, again, this is a core interest of mine. Can you talk a little bit about how AI can assist? There’s an iterative process. Of course, visualization can be quite simple—a simple framework—or visuals can provide metaphors. There are wonderful strategy roadmaps which are laid out visually, and so on.

So what are the ways in which you see AI being able to assist in that, both in the two-way process of the human being able to make their mental model explicit in a visualization, and the visualization being able to inform the internal representation of the strategist? Are there any particular ways you’ve seen AI be useful in that context?

Felipe Csaszar: So I was very intrigued—as soon as LLMs became popular, were launched—yeah, ChatGPT, that was in November 2022—I started thinking, there are so many ways in which this could be used. So myself and two co-authors, Hyunjin Kim and Harsh Ketkar, we wrote a paper, one of the initial papers on how AI can be used in strategy. It’s published in Strategy Science, and in that paper, we explore many ways in which AI could be used in strategy.

Of course, you can ask AI about coming up with answers to questions that you may have. You can also use AI to use any of these frameworks that have been developed in strategy. It was very clear to us that it was usable. Then the question was, how good are those uses? What’s the quality of current AI doing this type of task?

So what we did is an experiment where we compared the performance of AI to the performance of humans. In strategy, there are two types of tasks: one is to generate alternatives, and the other is to select alternatives. You have a problem—the first thing you want to do is have possible solutions, and then you want to be able to pick the best out of those. So we had two experiments: one where we measured the ability of AI to generate alternatives, another to select.

For generation, what we did is we got data from a business plan competition where people were applying with business plans that all had the same format. The important thing is that the first paragraph of that application had the problem—a problem that they thought was important. So we took all of those applications and removed everything except for the problem, and then we gave that problem to an AI and asked the AI, “Hey, complete the rest of the business plan.” So now we have business plans that are real, and the AI twins of those—business plans created by an AI that try to solve the same problem. Then we put both in a kind of business plan competition, where we had people with experience in investments ranking all of these business plans, and they didn’t know which ones were created by humans and which ones were created by AIs.

We looked at their evaluations at the end of the day, and on average, the ones that were generated by the AI were ranked a little bit higher—7% higher—than the ones that were generated by humans. So at least in this very specific context of business plan competitions, there’s potential. We’re saying, hey, AI could generate things at a level that is comparable to the people applying to this type of business plan competition. That has a lot of potential. We could use it in different ways.

The other part of this study was to measure the ability of AI to select strategies among strategies. There, what we did is use data from another business plan competition, where all of the business plans had been evaluated by venture capitalists according to 10 dimensions: how strong is the idea, how strong is the team, how strong is the technology, etc. Then we gave an AI the same rubric that the venture capitalists received and asked the AI to rank or grade each one of these startups according to these 10 dimensions. Then we compared how similar the evaluations of the LLM were to the evaluations of the venture capitalists, and we showed that they are quite similar—there’s a correlation of 52%. This, again, tells us that there is potential here.

An AI could do things that are quite similar to an experienced human evaluating this type of startup. A very interesting result there is that the correlation between two venture capitalists is lower than that 52%. So if you want to predict what a venture capitalist is going to say about your business, you’re better off asking an LLM than asking another venture capitalist.

Ross Dawson: Yes, which perhaps shows the broad distribution of VC opinions. So obviously, LLMs can play valuable roles in many aspects of the strategy process, but this brings us back to the humans plus AI role. There are many—again, a big topic—but rather than looking at them, comparing what humans and AI did, where do you see the primary opportunities for humans and AI to collaborate in the strategy process?

Felipe Csaszar: Yeah, yeah. So I think that’s a fascinating question, and my guess is that the study of the strategy process will completely change in the next 10 to 20 years. So far, all of the strategy process has been to study what happens when you have multiple people making strategy decisions. In the past, we studied things like devil’s advocate, or we have studied the role of changing the size of the group of people making decisions, or the consensus level required. But in the future, there will be AIs in this process that will have completely different bounds or capacities than humans. So we will need to learn what’s the best way of collaborating with them and including them into the strategic decision making process.

Today, we don’t know much about it. We are beginning to learn things, like the study I mentioned—hey, in this task, it seems to be better—but there’s so much that we need to learn. I am working on some things, but it’s still early.

Ross Dawson: Going back to the distributed representation—this is something where, of course, distributed representation can be in multiple people. Arguably, it can include human and AI agents as each having different representations. But this goes, of course, to the aggregation piece, where the aggregation is—you have a board of directors, group of executives, potentially a participative strategy process bringing more people into the organization.

What are the specific roles of AI in assisting or facilitating effective aggregation to form a cohesive strategy?

Felipe Csaszar: Yeah, so the truth is, we yet don’t know. There’s not enough research. We’re starting to think about it. We can see many uses, and I think what people should be doing now is running experiments to see when those add value and when they don’t. It will be different for different companies in different industries, so probably there’s no one solution that’s the same for everybody.

For example, one possible use in strategic decision making is predicting what your competitors would do. If I do this, what would be the most likely reaction of my competitor? That’s one. Another one is predicting consumers: if I launch this product with this set of characteristics, what would be the most likely response of my consumers? In strategy, something that has been very popular for the last 20 years is something called the Blue Ocean Strategy, which is a method to come up with new offerings, with new value propositions, but that requires a lot of creativity. With AI, you can automate part of that. At the end of the day, it’s a search process. You have to think about what would happen if I add this, or if I add this other thing, or if I increase this. Part of that can be automated—that would be another use.

Or if you have different proposals—in this other study, we show, hey, AI is good at evaluating, so if you have the right rubric, this can automate the evaluation, or can automate the first part of that evaluation so that you only have to spend your time among the really complicated, more sophisticated decisions or alternatives. There are many, many things that can be done at this point.

Ross Dawson: Which goes to, I think, one of the interesting points in your work—representational complexity. Some strategies are arguably simple; other strategies, you can call them more sophisticated, but they are more complex. The representation of complexity is greater.

There are two things that are required for that. One is, of course, sophisticated thinking, but also, because strategy in any organization involves multiple people, it requires that there is an ability for a number of people together to hold a hopefully similar or very similar representation of a quite complex topic. What are ways in which AI can be used to enhance that development of more sophisticated or nuanced or complex representations that can support a better strategy?

Felipe Csaszar: So that’s a great point. I have a paper from before this new round of AI called exactly that—representation complexity. There has been a long-standing discussion in strategy of when you want to use a simple representation, whether it’s better to use a complex representation, or something in between. We tried to clarify when each one of these applies.

But then came this new round of AI, and I think it changes things a lot. I talk a little bit about this in a chapter I uploaded recently—it’s called “Unbounding Rationality.” The key thing there is that humans—we have our own computer here, it’s the brain, and the brain has some constraints. One very important for strategy is the capacity of our working memory. There’s this famous paper from the 1950s called “The Magical Number Seven,” that we can hold in our working memory seven plus or minus two items—so between five and nine things we can keep at the same time in our mind. That’s why, for example, I think all strategy frameworks are very simple. There’s the five forces—fits within our working memory—or these typical two by twos, they have four quadrants—fits within our working memory.

But AIs don’t have that bound. They are not constrained by the same working memory constraint that we have. So I would expect that future frameworks will be much more complex, that representational complexity will increase because of AI. Of course, frameworks of the future won’t have a million things, because when you put too many things, you’re overfitting—it works well with things that happened in the past, but not in the future—but they will probably have more than five things.

Also, another reason for not having a million things inside a framework is that at the end of the day, you will still need to communicate frameworks. You will need to convince the other people in the organization, the ones that are implementing the strategy, that this is the right strategy. You will need to convince them, so you don’t want to have something that’s extremely complex. But my guess would be that the complexity of frameworks and of strategies will increase with AI.

Ross Dawson: So looking forward—you talked about 10 or 20 years. If we see the current pace of capability development of LLMs on a similar trajectory, where do you see the remaining role of humans as a complement to AI in shaping strategy? I think you mentioned this possibility of essentially AI forming strategy, but I think for a wide array of reasons, it will be human plus AI—humans will play a role as final decision maker or other things. So where do you see those fundamental human capabilities still being retained for the foreseeable future, as a complement to AI in strategy?

Felipe Csaszar: So I think that for the next 10, 20, maybe 30 years, humans will be really busy coming up with how to use AI—all of these experiments that we mentioned, people will be running all of those things in all different industries, and that takes a while. That will require human ingenuity and trying things and really understanding strategy and understanding the capabilities of AI. So I don’t see AI replacing human strategists in the very short term. On the contrary, because of AI, strategists will be more busy finding what are the best ways of using AI in their businesses.

I think 10, 20, or 30 years is very reasonable. If you think about the previous technological revolution, which I could say was the Internet—the technology for the Internet, we could say, existed since around ’94. The World Wide Web is from ’94, browsers are from ’94, bandwidth enough to send email. Essentially all of the technology that supports internet business today was mostly in place in the mid to late ’90s. But the businesses, or people, ended up using all of those things 10 or 20 years after that, because it takes a long time for people, for strategists, to come up with the idea—for someone to come up with the idea of, let’s say, Netflix or eBay or PayPal or Facebook—all of those things, they take time for people to understand this is doable. Then it takes time to implement. Then it takes time for users to say, “Hey, this is useful.” There’s a lot of adaptation, and then there will be regulation. So the whole process takes a long time. I don’t think that businesses will change from one day to the next. It will be a relatively slow process that will take decades. When we look back in 20 years from now, we will see, “Hey, everything changed,” but every year we will see just a little bit of change, like what happened with the Internet. So I imagine that people designing strategies, implementing strategies, they will be very busy in the next 20 years.

Ross Dawson: So to round out, I won’t ask you to make predictions, but maybe some hypotheses. What do you think are some interesting hypotheses that will inform your research—not just next year, but in the years beyond? Where do you think are the interesting avenues that we should be not just exploring and researching, but where there is a valid and useful hypothesis?

Felipe Csaszar: Yeah, so many things, but one very important—I think that strategy will be more about making the right predictions. The role of foresight. It turns out that when you want to train a machine learning algorithm, you need to have some signal that informs how you train the system. It’s called the gradient, or the objective function. So in strategy, we will need to make that more central, and then think, what are the best ways in which you can use AI to make the right predictions? That requires measuring the quality of predictions. So you change this in the business, and this ends up happening. We want an AI to be able to do that. So coming up with ways in which you can measure the quality of decisions will become more important, so that we can train those AIs.

That’s one. And very related to that is, well, the thing that’s generating the predictions are representations, and then it’s coming up with those more complex representations that are better at making decisions or are better at discovering things that are hard for humans to discover. Those are the two main things. I think the future of strategy will be about finding ways of improving foresight and finding ways of improving the thing that creates that foresight, which are the representations. All of that will change what has been called the strategy process—how we make decisions in strategy.

Ross Dawson: So I just need to pick up on that point around prediction. One of the challenges with external predictions is that, then, as a strategist, you have to say, either I will build my strategy based on that prediction, or I question that prediction. I think there are alternatives or attribute probabilities to it. So even if a prediction machine gets better, it’s still very challenging, particularly cognitively, in terms of accountability for the strategist to incorporate a prediction where you don’t necessarily have all of the logic behind the prediction as a machine learning model to incorporate. So how can a strategist incorporate what may be a relatively black box prediction into an effective strategy?

Felipe Csaszar: Yeah, well, and here we are in the conjecture part of this interview. So my answer is in that spirit. I think there are two ways out of this. One is that we will ask for explainable predictions. There’s a whole area of AI called Explainable AI, which is exactly trying to do this—not just say what’s the best prediction, but why the AI is saying that’s the right prediction. So that could develop, and probably that will develop, because humans will question whatever the AI will predict. That’s one way.

The other is, imagine that the AI becomes very, very, very good at making predictions. Then at some point, it doesn’t matter if it can explain it or not—it’s just making very good predictions. It’s like, imagine you want to win at chess and you have this machine that can play chess very well. This machine wins at chess. You don’t need to exactly understand how that machine is making each one of those decisions. But if the machine is very good at it, and it’s consistently good at it, people will use it. In a sense, the market will decide. If this works better than a machine that provides an explanation for each one of the steps, people will just go with the one that’s making the right prediction.

Ross Dawson: I think there’s all sorts of other places we can go to from there, but that’s fascinating. So where can people go to find out more about your work?

Felipe Csaszar: Well, I upload all of my stuff to SSRN. So if you Google my name and SSRN, you will find all of my papers. In the near future, like in the next three months or so, I’ll have two things coming out. One is a Handbook of AI, written also with my co-editor Nan Jia from USC, that will have 20 chapters that will explore different ways in which AI will be affecting strategies—the Handbook of AI and Strategy, published by Elgar. And then around that same time, there will be a special issue of the Strategy Science journal where I’m one of the co-editors, which will be exactly about the same—about AI and strategic decision making. We already have accepted several of the papers for that special issue. Those papers will be pushing the frontier of what we know about AI and strategic decision making.

Ross Dawson: That’s fantastic. I will certainly be following your work—very highly aligned with the humans plus AI movement. And thank you for all of the wonderful research and work you’re doing.

Felipe Csaszar: Thank you so much, Ross. It’s been a pleasure.

The post Felipe Csaszar on AI in strategy, AI evaluations of startups, improving foresight, and distributed representations of strategy (AC Ep32) appeared first on Humans + AI.