Agents Of Tech

AI companies largely check their own models for safety. The former head of AI ethics at Twitter gives them a C or D.Rumman Chowdhury, founder and CEO of Humane Intelligence and the first US Science Envoy for AI, has spent her career testing AI systems for real-world harm. Autria Godfrey, Laila Rizvi and Stephen Horn put her arguments to the test: whether companies should red-team their own models, whether the public should be asked to help, and why AI gets less independent scrutiny than the chair you're sitting on.The panel doesn't agree. Laila thinks the labs are trying. Stephen calls self-testing "amateur night out."Should AI companies be allowed to regulate themselves?

The people building AI say it poses an existential risk. Amba Kak thinks the bigger danger is not the machines, but the monopolies that own them, and that governments may already be slowly bailing them out.Amba Kak is Co-Executive Director of the AI Now Institute, a former Senior Advisor on AI at the US Federal Trade Commission and a member of New York Mayor Zohran Mamdani's transition team. To make good on the roughly $5 trillion of AI spending committed over the next five years, Bain estimates the industry would need close to $2 trillion in annual revenue, more than the entire global enterprise software market. If that revenue does not arrive, who is on the hook? Amba explains why AI investment is moving from Big Tech's cash reserves into debt markets that reach our 401(k)s and insurance policies, and why she believes a "slow bailout" is already underway.We also ask whether AI is heading for a 2008-style "too big to fail" moment, how the "what about China?" argument is used to shield the industry from regulation, whether rules will entrench Big Tech or rein it in, and whether all of us are responsible for driving AI demand.

Turing Award winner John Hennessy argues that even a few percent more productivity a year adds up in an economy the size of America's, and explains why the return could take time to show up.John Hennessy shared the 2017 Turing Award, often called the Nobel Prize of computing, for work that shaped the modern microprocessor. He served as president of Stanford University for 16 years and is chairman of Alphabet, Google's parent company. In this conversation, he explains what AI has to deliver to justify the money pouring into it, why reasoning is the next frontier, and how he thinks the technology should be regulated.In his words: "It's a new Industrial Revolution. It's one that potentially is much larger size than earlier ones. But it's going to play out over time."

Boris Gamazaychikov measured AI's footprint inside Salesforce and co-created a star rating for AI models. Now he sorts the real climate numbers from the fake ones. Sam Altman calls AI's water numbers "totally fake." Critics say one chat uses a whole bottle. The truth depends on which AI, and where it runs.He now works at Sustainable AI Group, which estimates the energy, carbon and water use of closed AI models and pushes AI companies to be more open about it.

In this Agents of Tech episode, Autria Godfrey, Laila Rizvi and Stephen Horn revisit four of the sharpest voices in AI, and their answers could not be further apart:🔹 Gary Marcus, cognitive scientist and AI author, argues we're probably in a financial bubble and that LLMs are "just not as smart as they look."🔹 Emily Bender, professor and author of The AI Con, says the hype is driven by money, marketing and access to power.🔹 Derya Unutmaz, immunologist and our most optimistic guest ever, believes AI will save billions of lives, but warns the road there will be painful and no one is preparing for it.🔹 Santa Ono of the Ellison Institute of Technology at Oxford makes the historical case for calm: technology eliminates jobs, but it creates new ones too.Everyone is worried the AI bubble will burst. The bigger danger might be what happens if it doesn't.The world has bet trillions of dollars on artificial intelligence. If the bet fails, we lose the money. If it pays off, we could face mass disruption to jobs, education and society, and nobody has a plan for it.Then the hosts deliver their verdict. Are AI valuations overhyped? Do the hyperscalers know something we don't? And if AI truly succeeds, who is getting society ready for what comes next?

AI agents can already connect and communicate. But can they understand one another, coordinate around a shared goal and genuinely think together? Vijoy Pandey, Senior Vice President and General Manager of Outshift by Cisco, joins Autria Godfrey to introduce the Internet of Cognition: a vision for enabling AI agents and humans to share knowledge, coordinate decisions and combine their intelligence. Vijoy explains why connecting agents is only the first step and uses a deceptively simple travel-booking example to show what happens when two AI agents interpret the same instructions differently. He also discusses the three layers he believes are needed to scale collective intelligence: connectivity, coordination and expertise. According to Vijoy, new coordination protocols have increased alignment between agents from less than 20% to more than 90% in Outshift's testing. The conversation also covers the development of AGNTCY, the open-source infrastructure for the Internet of Agents. Vijoy says its code has now recorded more than 1.7 million downloads, with over 100 organisations contributing to and building on it. Could the next major leap in AI come not from building a bigger model, but from enabling intelligence to work collectively? https://outshift.cisco.com/ Mycelium GitHub: https://github.com/mycelium-io/mycelium

Will AI take your job? An MIT labor economist says that is the wrong worry. The real shift has already been in play: firms facing AI uncertainty stop hiring employees and start hiring disposable workers.Paul Osterman is Professor Emeritus at MIT Sloan and author of Disposable Workers: The Transformation of Employment. His survey of more than 6,000 US workers found that 35 percent are marginal employees, contractors, freelancers or gig workers. In this conversation he explains why US employment kept growing through the internet, the mainframe and the China trade shock, why AI will still intensify disposability, and what a social contract for displaced workers would actually require.In his words: "Firms don't want to have real employees. They want to have contractors, or freelancers, or what I call marginal workers."

AI has the potential to change everything, from work, education, healthcare, and even democracy itself. But with that promise comes pressure on those in positions of power to get it right. So if you were advising the next president or prime minister, what part of the AI strategy playbook would you tear up first? Are governments even remotely ready for the age of AI? Are we prioritizing prosperity for all with the implementation of AI technologies across almost all areas of modern life?

This week on Agents of Tech, we’re pivoting from the tech bros to policymakers as we dive into the role lawmakers should play as we weigh where value, risk, and responsibility move next. Hosts Autria Godfrey, Stephen Horn, and Laila Rizvi sit down with Benedict Macon-Cooney, Chief AI and Innovation Officer at the Tony Blair Institute, as he lays out whether the state can act before the AI disruption arrives.

In our pre-interview discussion, our hosts explore the rapid pace of change and the challenges it brings. But while Laila sees governments that don’t understand AI and are not capable of keeping up with the speed of change, Stephen thinks that’s “a bit harsh.”

So what is the first thing that governments should do? According to Benedict Macon-Cooney, it starts with “understanding this technology and adopting it into your day-to-day life… before you can then begin to look at the wider tranche of government where there'll be lots of other points of opportunity for process automation, improving public service delivery, and just the speed and execution of government more broadly.”

Autria brings up the issue of universal basic income, the idea that AI will create so much abundance that people won’t need to work. Even if UBI does become part of the social security net, Macon-Cooney says he thinks, “people will ultimately want to find some meaning in work…I think people will always want to work and will find ways to do so.”

Stephen points out that work looked very different before and after the Industrial Revolution, and that the AI Revolution will also result in a complete transformation of work, and “how human beings get meaning and identity…arguably could change.”

Next, we ask Benedict what he thinks the biggest transformation will be as a result of AI, and he outlines the influence of AI on education. He talks about democratizing access to high-class education and personalized AI tutors for every student.

After the break, we shift the conversation to sovereign AI, and whether AI dependence could lead to national vulnerability. Much like our conversation with Benedict’s colleague at The Tony Blair Institute, Ryan Wain (Watch our episode with Ryan: https://open.spotify.com/episode/0YVaIEtjMo8DLOr2XozVPc?si=1c717f3e52ae4a17), Benedict explains that building frontier models isn’t for every country, but it’s also not the only way countries can take part in the AI revolution. “So then the question of sovereignty becomes, "Where do I have leverage and where can I have comparative advantage…?"

Some of that advantage, Benedict says, comes in the area of first-of-a-kind technologies like geothermal and nuclear, especially fusion, as well as biotech and life sciences. Laila, Stephen and Autria discuss how the UK and the US fit into the evolving power dynamic, while also debating whether Silicon Valley is stealing the UK’s AI talent.

We finish up by asking Benedict the big question: What should governments do now? His answer compares how quickly the AI revolution is playing out in the private sector to how slowly the government moves. “Solving that structural change is the number one priority because if you don't actually improve the speed of how government does regulation, policymaking, et cetera, it will never be able to catch up to the speed of change in the economy and address some of these big structural changes.”

What do you think? Are lawmakers moving quickly enough to keep up with the changing AI landscape? Join the discussion in the comments.

Trillions of dollars are being poured into the AI promise of a global economic transformation. We're being sold a productivity revolution, faster output, greater growth, and unprecedented prosperity. But when? Real, tangible gains are barely showing up in the data. So is the return on investment actually going to be smaller and slower than promised? And could these billion-dollar bets bankrupt the companies making them and take the global economy down with it?

This week, hosts Stephen Horn and Laila Rizvi are talking to globally renowned economist and Director of the Oxford Internet Institute, University of Oxford, Professor Carl Benedikt Frey, co-author of the famous “Future of Employment” study which garnered headlines about robots taking over our jobs and, more recently, his book "The Technology Trap," which showed that revolutionary technologies can take generations to pay off.

Laila kicks off the interview by asking Professor Frey if he thinks that the trillions of dollars being invested in AI will pay off in the next five years. The more interesting question, he says, is whether it’s going to pay off for society and the economy. His answer to that one: probably not! He also questions how many of the firms investing enormous resources in the AI race will come out profitable on the other side.

Next, Stephen and Carl compare the AI Revolution to the Industrial Revolution. Professor Frey points out that the first Industrial Revolution caused disruption, job loss and decline of wages at the lower end of income distribution, but that the second Industrial Revolution brought us broad-based improvements in standard of living and income.

“I think a key question going forward is whether AI will mainly take the form of automation, and more closely mirror the First Industrial Revolution, or whether we will see it develop new products, new industries that create new jobs and activities, that create more broader shared abundance like the Second Industrial Revolution.”

So where will innovation come from? Professor Frey points out the decline in breakthrough innovation and breakthrough patents since the computer revolution, which he attributes to an increase of projects, but at a lower quality. Laila asks whether access to AI is leading to more “AI slop” in academic papers, work and essays, while Stephen wonders how companies are actually going to innovate by putting AI at the heart of process.

Laila asks Professor Frey what happens to society if AI doesn’t deliver productivity growth soon? Has AI done more harm than good? “Well, AI is clearly a tremendously unpopular technology… It's hard to think of any other technology where the developers of that technology are saying, "Well, best case, this will take your job, but worst case, it's going to kill you." Plus, he says, there’s plenty of hype because the stakes are so high.

Stephen turns to the role of governments in the development of AI, and Professor Frey describes approaches that make sense for Europe, China and the USA.

When will AI boost productivity and how significant will it be? Professor Frey thinks the answer “depends to what degree we'll see new firms coming in, building new products and new industries around AI” rather than AI merely being used as a productivity tool by individuals, and that “institutional change is going to be needed for these productivity gains to be realized.”

As always, we end with our Rapid Fire segment.

Laila asks Professor Frey for something that's universally accepted in his field that he strongly disagrees with: “It's seen as almost universally true that the best predictor for the future of work is what machine capabilities are. What's missing in there is that we need to consider more what humans actually want.”

Finally, Stephen asks Carl what he’s optimistic about: “I'm really quite optimistic about the potential for AI in medical discovery. I think we do have an extraordinary opportunity for curing many diseases, including, hopefully, cancer.”

If public money helped build the foundations of AI, but now private companies control much of its future, who should be reaping the benefits? Is there a role for the government? Who should be in charge of shaping what comes next?

In this episode of Agents of Tech, our hosts Autria Godfrey, Stephen Horn, and Laila Rizvi are talking AI, publicly funded science, and the future of innovation with former National Science Foundation Director, Dr. Sethuraman Panchanathan, aka Panch, the Professor of Technology and Innovation at Arizona State University.

Our guest, Dr. Sethuraman Panchanathan ran the National Science Foundation in the United States, one of the most important and powerful public science agencies in the world. The NSF was investing in AI long before names like Anthropic or OpenAI even existed. According to Panch, over the last six decades, the NSF has funded 80% of non-defense compute research – even, as Laila says, during “AI winters.”

We start off by asking, how can anyone make the argument that AI and the current AI structure that we know and use today did not come from the public? And what role should the government play in AI research, or in passing legislation that will put AI safeguards in place?

Dr. Panchanathan tells us about the 27 AI institutes the NSF launched while he was there that permeate all aspects of AI. He talks about the exciting opportunity that the co-evolution of AI and humans represents.

Stephen asks how we can make sure AI is for everybody and it doesn’t lead to greater inequality, given the trillion dollars hyperscalers are pumping into AI? Panch describes the importance of the democratization of AI access, and the role of the federal government in these efforts. He explains how the NSF built the National AI Research Resource (NAIRR), not only as a federal government project, but as a hyper-partnership approach that also leveraged all of the assets and infrastructure of the private sector.

Laila points out that it goes further than just having access – it’s reaping the benefits of those tools. Panch agrees, and likens the situation to the “digital divide” we talked about with broadband access. “We have had these kinds of challenges in the past,” Dr. Panchanathan says, and the answer is “democratized access in the form of education, training, awareness.” He talks about how the NSF’s Regional Innovation Engine program brought stakeholders together in a synchronized manner.

After the break, Autria asks Dr. Panchanathan whether the government has enough in-house technical expertise to try to evaluate frontier AI systems properly and keep up with the speed at which things are progressing. Yes, says Panch, but expertise isn’t enough. “You want to be partnering with the people who are in the frontier...they are the people in the industry. And I can tell you, I've had many conversations with leaders in industry. They are only too happy to partner with the government in helping shape all of this future together in an unselfish way.”

As always, we end our interview with our Rapid Fire segment, where our hosts ask our guests a series of three questions.

Autria asks Panch where people will draw the line when it comes to allowing AI in their personal lives; his answer: compromising their privacy.

Laila asks him for something that's universally accepted in his field that he strongly disagrees with; Dr. Panchanathan talks about the future of learning and why he doesn’t agree “that somehow learning happens in the campuses and four walls by the faculty that are going to be training you.”

Finally, Stephen asks him what he’s optimistic about. “I am very, very optimistic about what technology and the human can do to unleash the latent potential of humans. We have not seen anything yet, my dear friends.”

What about you? Would you like to see private industry continue to captain the ship, or should government and federal research institutions regain a bigger role? Weigh in by dropping your comments below.