November 24, 2021

The Genius Makers: The People that Shaped Neural Networks in A.I. feat. Cade Metz

45 minutes

The same technology that lets your daughter call up her favorite songs in Alexa is also used for government surveillance, racial profiling, and the creation of deep fake YouTube videos from troll farms. While neural networks make our lives easier, they also create ethical tangles and questions. New York Times Silicon Valley reporter, Cade Metz tackles these moral disparities through hundreds of exclusive interviews in his book, Genius Makers.

In this episode, Cade talks about factors that shaped the A.I. technology fueling the biggest tech companies: Google, Microsoft, Facebook, and OpenAI, a new lab founded by Elon Musk. Listen as Greg and Cade discuss the fierce conflict between national interests, shareholder value, the pursuit of scientific knowledge, and the very human concerns about privacy, security, and prejudice.

Episode Quotes:

Why are open-source companies like Geoffrey Hinton's DNN research considered valuable when people already have access to their information?

It's an idea that dated back to the fifties, but by 2012, when Geoff is essentially auctioning himself off, there are few people on Earth who know how that idea works. Because most of the world thought it would never work. And that's the dynamic there. To this day, it's the talent that is valuable. We needed a lot of stuff for this to work. You need the data, and you need the computer processing power needed to analyze that data. But you need the people to make that work. Getting a neural network to work, some people described it as a dark art or black magic. It's about sort of coaxing something out of this data. These systems literally learn by analyzing the data, and it's more data than you and I could ever wrap our heads around. So, it's about coaxing those machines to learn on their own. They do take off in ways that are beyond us, but you need these people to guide them. And that's really what happened

Thoughts on institutional frictions that shaped game-changing ideas and progress on neural networks

You have these battles between academics and people like Marvin Minsky ended up having the upper hand, right? Sometimes, it's about who has the loudest voice and who can convince the Department of Defense to give them the money for their particular project.

And you see the whole industry shift to what you call good old-fashioned AI, that symbolic AI. Where you're basically putting engineers in a room, and they define how the technology's going to work— rule by rule, line of code by line of code. That became what people had the most hope for. That would be the future and not these systems that could learn on their own from data.

Thoughts on pursuing ideas and harnessing curiosity to overcome dead ends

I love that you mentioned this theme of his own lab at the University of Toronto. It was old ideas are new. What that meant was, it didn't matter how old the idea was, what mattered was, had you proven that it wouldn't work? If you had not proven that, then you should keep working on it. No matter how much time went by. If you got to that point where you proved it was wrong, then you could put it aside. But until then, you keep working.

How did persistence help Geoffrey succeed in finding the missing piece in the neural network?

Most of the world at that point had discarded the idea of a neural network. Even his own thesis adviser had abandoned this idea and had recently moved on to that symbolic method you talked about. Yet, Geoff still grabbed hold of that idea and did not let go for decades. So, he had this fundamental belief and that is what drove him.

Show Links:

Guest Profile