Sign up to save your podcasts
Or
Share Flexing Your Brain in XR, with Cognitive Design’s Todd Maddox
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Flexing Your Brain in XR, with Cognitive Design’s Todd Maddox
We often talk about
how XR technologies are great tools for education and training on
this podcast. But why is that? Like, physiologically? Turns out, XR
tickles the thalamus in ways traditional learning strategies never
could, and that’s not us just whistling Dixie.
Today’s guest —
Cognitive Design & Statistical Consulting, LLC CEO Todd Maddox —
has a PhD in Computational and Psychological Science, meaning there’s
no one better to explain why XR and your brain are a match made in
heaven.
Alan: You’re listening to the XR
for Business Podcast with your host, Alan Smithson. Today’s guest is
Todd Maddox. He is a cognitive design specialist. Todd is a PhD, and
the CEO and founder of Cognitive Design and Statistical Consulting
LLC. He’s also a learning scientist and a research fellow at Amalgam
Insights. His passion is to apply his 25 years of psychological and
neuroscientific expertise gained by managing a large human learning,
memory, and performance laboratory to help build better education and
training solutions. Todd has published over 200 peer reviewed
scientific articles, resulting in over 10,000 academic citations and
hundreds of speaking engagements. During his 25 year academic career,
he’s awarded $10-million in federal grants from the National
Institute of Health, National Science Foundation, and the Department
of Defense to support his research. Since entering the private
sector, Todd has embarked on a mission to translate the amazing body
of research conducted in the ivory towers into plain English and help
companies leverage this research to build better products. Todd is
especially interested in applying his expertise in the psychology and
neuroscience of learning, memory, and performance, and to use
immersive technologies in manufacturing, health care, corporate
training, and retail, to name a few. You can follow Todd on LinkedIn.
Just look for “Todd Maddox PhD.”
Todd, welcome to the show.
Todd: Hey, Alan, it is fantastic
to be here. Thank you.
Alan: It’s such an honor. I’ve
been reading your posts and your articles, and trying to get through
some of your scientific papers is a challenge. It’s so much
information there.
Todd: Yeah, I hear you. And to
be honest, my recommendation is to sort of skim the peer-reviewed
stuff, because it does seem like it’s written in a foreign language,
even though it is English. And the LinkedIn post and the more recent
stuff, where I really try to talk in plain English, because if a
scientist can’t present their work in plain English then there’s
something wrong. So that’s what I’m trying to do.
Alan: I love it. And one of the
articles that was recently published was a report on VR as an empathy
builder, through Tech Trends.
Todd: Yeah.
Alan: Here, I’m just going to
read a quote from it:
“Any profession that requires
interpersonal interaction, such as education, retail, food service,
call centers is better served with strong empathy.” Let’s start
with that.
Todd: Totally, yeah. Every one
of those examples is a people example; people interacting with other
people. I know we’ve got amazing technologies; we’ve got robots,
we’ve got all these wonderful things that are making our lives
better. But let’s face it, in the end, it’s about people interacting
with other people and caring for other people, walking a mile in
somebody else’s shoes. That is really just so critical.
These technologies — in particular
virtual reality, I would say — this is an immersive technology. I
could be dropped into any environment. That’s amazing, that’s very
cool. But now imagine: Todd, a middle-aged hetero white guy gets
dropped into an environment where Todd is now a young
African-American lesbian woman. And whatever’s happened to her has
happened to her and hey, it’s happening to me now. Now, granted, is
this the same as a lifetime of “go back where you came from?”
Of course it isn’t. But it’s a start, and it’s visceral. It engages
emotion centers in the brain in a way that is rare. And of course,
you can do this over and over and over again in virtual reality,
because I can be plopped into any environment, into any person’s body
that we like. So when it comes to empathy and understanding, I really
think that’s one of its many sweet spots, I’ll say.
Alan: There’s been some recent
research around using this for human resources, for hiring, and then
also one thing that I saw was really neat is, as a manager, you were
able to sit in virtual reality and reprimand somebody. You had to
give a disciplinary action. You sat there and you talked to the
person and it recorded your eye tracking, your hand motions, your
head pose. And then afterwards, you got to sit in the seat of that
person and watch yourself give yourself the reprimand.
Todd: Yeah. And that’s what it’s
all about, right? Just to start to dive a little bit into the
neuroscience behind all this, because that’s really where I reside
and am most fascinated. It’s one thing to study what it’s like to
give somebody a reprimand. “Here’s a PowerPoint on how you give
somebody a reprimand,” or even, “here’s a video — watch this
person getting a reprimand. What did they do right? What did they do
wrong?” All of that information is processed by what’s called
the prefrontal cortex, which is right behind your forehead. I call it
the “What System.” It’s the part of your brain that learns
what you’re supposed to do. OK, that’s great, and it’s really
important to know what to do. But let’s face it: I know that I
should eat a little more healthy than I do. So, I know what to
do. But do I do it all the time? Knowing what to do, and
knowing how to do something are two completely different
things. They’re mediated by completely different parts of the brain,
and the processing characteristics of those two brain systems are
very different.
So what you’ve got in the VR example
that you’re suggesting is, you’re actually in there doing, instead of
“here’s what I should do — watch this PowerPoint, hopefully
you’ll go do that stuff — you’re in VR and you’re actually doing it.
And I love that it’s measuring things like body language, non-verbal
communication. What’s fascinating about non-verbals is, it’ll
actually provide an enormous amount of information about whether
somebody is being genuine, showing empathy, whether what they’re
saying is not really what they mean. These are absolutely powerful
tools.
And then, yeah, when you can then swap,
you’re now getting the reprimand from you. It’s truly remarkable.
You’re just so broadly engaging so many of these parts of the brain
that are critical for these kinds of tasks. It’s just an amazing
technology.
Alan: And in this podcast,
there’s always the same things that come up: “I want to be able to
empower my field service workers hands-free.” “I want to use
virtual reality for training.” Now, the training aspect of things
is starting to accelerate much faster than anything else, because the
results that we’re seeing across the board in companies like Boeing
and Lockheed Martin and Wal-Mart, these big companies are seeing
real, tangible results using this technology. Why do you think it’s
so powerful? What is it about this technology that’s creating those
lasting synapses in the brain that feels like you’ve done it?
Todd: Yeah, it really comes down
to the brain. And there are a couple of things. Traditional ways of
training – really, what I did for 25 years when I was a college
professor educating undergrads — you have butts in the seat and
you’re droning on at them. Well, what’s happening in your brain?
First — again, we’re focused exclusively on the prefrontal cortex —
if you ask your grandmother, you know, well, what does it mean to
learn? Ask just any old buddy. That’s really what they’re talking
about. They’re talking about the prefrontal cortex. “Okay, I
have to sit down. I have to concentrate. I have to process this
information. I’ll mentally rehearse it so that I can, quote/unquote,
remember it.” Everything is driven through the prefrontal
cortex. It’s part of the brain that sets us apart from,
quote/unquote, lower animals — which is the term actually I don’t
like that much — but in an evolutionary sense, it allows us to have
what we call metacognition. That is, we can think about the fact that
we think.
But let’s take VR — or AR, for that
matter. First thing is, I’m learning through experience. I am in
the environment while I’m learning. I’m not in a classroom. I’m not
at a desk studying. I am in the environment. Why does that matter?
Well, that broadly engages perceptual representation regions in the
brain — the back of the brain, the sides of the brain, the top of
the brain; all these parts of the brain that represent the
environment that you’re in. So, you’ve already got that activation
alone.
Now let’s take, say, a hands-free AR
device. Now you’re actually generating the behaviors of whatever your
job is. And you’re being guided, let’s say, with AR assets in the
Hololens: you’re actually generating these behaviors in a guided
fashion. You’re not just reading a textbook that tells you what the
behaviors are — that hopefully you’ll remember when you’re out on
the field — but rather, you’re being guided through these behaviors,
and you’re doing them in real-time out in the field.
So, I’ve got the behavioral centers of
the brain that are activated. These are the centers deep down in the
brain in a region called the striatum. Simultaneously, if it’s an
engaging and rich environment — or like with Wal-Mart, if it’s Black
Friday is the environment that you’re in — you’re going to have
emotion centers lit up. It is as if you’re plopped down in the middle
of Black Friday. So, emotion centers in your brain are lit up — like
the amygdala and some other limbic structures. Long story short:
traditional training engages one part of your brain — the prefrontal
cortex — which is a part of your brain that cognitive load is a
problem; working memory capacity is a problem; attention span is a
problem. Versus virtual reality or augmented reality, where you’re
engaging experiential learning centers, cognitive centers, behavioral
centers, and emotional centers. So, much more the brain is being
engaged in synchrony.
This is why you learn more quickly, you
make fewer errors, and you retain the information more. This is
really what is driving these amazing return on investment for forward
thinking companies that are using these technologies.
Alan: That was actually…
[chuckles]
you know how to speak my language on this show! The XR for
Business Podcast is all about ROI — what are the best investments we
can make as businesses, to drive our business forward using
virtual/augmented/mixed reality technology? So, my next question was,
of course: what are some of the ROIs being seen by businesses?
Todd: Yeah. So you mentioned
several of them: Lockheed, Walmart. I’ve actually been following PTC
quite a bit. Actually, that’s where you and I met. I’ve been
following some of their technology and actually met several people
who were using one of their products called Expert Capture, and
getting… trying to remember the exact numbers. But, training times
cut in half, standard operating procedures being generated ten times
faster than they were before — you’re seeing numbers like this from
company after company after company. And again, from a neuroscience
of learning and a neuroscience of performance perspective, none of
this surprises me, because we’re talking about engaging one system in
your brain, versus engaging three or four systems in your brain, and
engaging them in this… almost like a ballet. It’s this beautiful
synchrony, all in the interest of achieving whatever it is that your
goal is. Whatever you’re learning task is, your training task,
whatever it is you’re trying to perform. And so you’re seeing these
ROI that’s off the charts.
I will say — and I actually just wrote
a report on this last week, also at Tech Trends — that I’ve been
making this case. I’m going to continue to make this case so people
listen to me. These technologies are fantastic. They broadly engage
all these parts of the brain, which is super. And you’re seeing
amazing ROI. My very, very strong belief is we can double those at a
minimum. Wow, how can we do that? I mean–
Alan: Hold on, hold up! So,
we’re already seeing 50, 60, 70 percent increases right across the
board. Retention rates, memorization techniques, faster training
time. You’re saying we can even double that?
Todd: We can do better on those
metrics, and we’re going to get other value that we’re not even
really measuring. For example — and I think Expert Capture’s a good
example — you’re going to retain this information better. You’re
going to learn more quickly. Okay, that’s great. You’re gonna know
what the standard operating procedures are, and you’ll be able to
regurgitate those back to me. I think you can — and I don’t want to
get too into the weeds here — but there are aspects of, in
particular AR, that are not optimized for training behavior.
Most of these AR assets… let’s say
you have the Hololens on. You’re working on a machine. The Hololens
says “move your left hand over here, and turn this knob.”
Okay, that’s great. That’s called “guided learning.” And
guided learning is solid. There’s nothing wrong with it. But if you
start tweaking and incorporating other ways of doing this, that are
less guided and more discovery-based by the learner, and –
critically — involve real-time reward and punishment. You are now
going to be training the muscle memory correctly. It’s actually
better for you to discover these things. So, be guided partially.
Then you have to generate a response and you get reward or
punishment. That is how you’re going to train the muscle memory.
You’re going to be able to train
expertise. I have 25 years of expertise in manufacturing. I believe
that with these tools, we could create experts much, much faster. And
that’s not really the goal of a lot of these technologies. It’s
really just “translate that expertise into a series of steps
that I present on the Hololens, show it to the workers to get them up
to speed.” So, they’re up to speed. That’s great. But they’re
not experts. They don’t have all of the expertise that that baby
boomer who’s retiring has. I believe we can impart that as well. But
we’re gonna have to make some changes to the way that these
technologies work. We’re going to have to optimize the AR assets to
the way the brain learns best. AR works very well now, because it
broadly engages all these parts of the brain. And that’s great.
That’s a good starting point. Now we need to optimally engage each of
those parts of the brain.
Alan: Yes. How do we do that,
then? You’ve got an AR experience that guides me through fixing a
machine — and I’ve been able to expert capture, I’ve been able to
put on a camera on one of my experts, because here’s the rub: as the
workforce starts to age and retire, the problem isn’t so much that
there aren’t jobs, the problem is that the jobs are changing
slightly. The experts who are the best in the world at that job are
starting to retire. How do we have the skills transfer from one
generation to the next?
Todd: The first thing is we need
to capture that expertise, that boomer expert who’s in manufacturing.
If you tell him, “go sit at that computer and write out your
standard operating procedures,” for one, he’s not going to be
happy. And for two, he’s not going to do a very good job. And you
might ask, “well, why is that the case? He’s got all this
expertise.” He has behavioral expertise. His expertise is in
how he interacts with that machine. That’s muscle memory. That’s
understanding situational awareness. So he actually has a feel
for when that machine isn’t working right. He can’t even describe it,
he just sort of knows it’s off a little bit. And you’re asking him to
sit down and type out standard operating procedures, which is
knowledge. So you’re asking him to use his prefrontal cortex to write
out procedures, when his expertise is behavioral and situational
awareness.
So that’s one thing: we have to do a
better job of capturing it. Expert Capture does a good job on that!
I’m not saying it doesn’t, because it says, “hey, go do your job
and I’m just going to capture it.” But how we curate that, and
how we present that to young workers is absolutely critical. And
we’re not optimized. That has not been the focus to this point.
Alan: I think a lot of the focus
has been just trying to get the technology to work, let’s be honest.
Todd: Absolutely.
Alan: We’re only 12 months into
having headsets that actually turn on when you want them to do it,
and object recognition. It’s a very new field. And I’m really excited
to listen to the next part of what you were going to say, and that
is, how then do we optimize for this? Because we have the very basics
worked out. I can put on a pair of glasses. I can recognize a
machine. I can walk you step-by-step through how to fix that machine.
Now, what you’re saying is, let’s create a more discovery-based, or
maybe even gamified experience, where I learn, but I’m learning while
doing and making mistakes.
Todd: Absolutely. To your
initial point, I want to be clear that I am not making the case that
people [in the industry] are too slow. No, not at all. And you’re
right, we are we’re in the very early stages. But we have these
technologies, and it’s time to say, “okay, I’ve got this great
technology. I can present any AR asset I want, I can present it
anywhere I want, and I can present it anytime I want.” The
question is, “when do I want it? What do I want? And where do I
want it?”
That’s where science comes in. And
first: Neuroscience. I actually had an interesting discussion about
this with Jim Heppelmann months ago at PTC, he was talking about “we
use green for some things, and red for other things.” And I
said, “Yeah. Do you know why that is, Jim?” And he’s like,
“well, green means go and red means stop.” I said, “well,
there’s a reason for that.” The cells in your brain that respond
to green are the same cells in your brain that respond to red. But
they are excitatory to green and inhibitory to red. In other words,
it is impossible not to discriminate green from red. You can
never confuse the two, because it’s built into the neurons.”
We need to leverage more things like
that. We need to go back and look at how the brain processes the
what, the where, and the when. And we need to build that into these
AR assets to optimize these things. Gamification, yeah. Start with
directed learning and then start weaning this worker off of the
directed learning, toward more discovery-based. That is going to
speed the muscle memory. That also requires rewards and punishments.
So there’s gamification right there. That’s going to be the next set
of major upgrades, I think.
Alan: What about artificial
intelligence? I just read an article recently that was talking about
a company that’s using AI for recruitment; they’re able to run these
people through a couple of games, a couple of questions, and a video.
The AI’s analyzing the video, it’s analyzing their results on the
game. And then it’s saying, “out of these hundred thousand” — it
was Unilever, by the way, that it’s doing this.
Todd: OK.
Alan: —
“out of 100,000 applications, we’ve narrowed it down to
3,500, based on just this algorithm for this particular job.” And
then they’ve also started using it to then unlock what does the
potential look like for, “maybe you’re not right for this job,
but hey, you scored in the 98th percentile on this other job that you
didn’t apply for.” It may make for much better recruiting.
One of the things that I think is
necessary that we haven’t even scratched the surface of: we use
Netflix algorithms every day to give us better movies to watch. We’re
not even touching the surface of what’s possible, when we create that
sort of AI algorithm to give us better learning in a way that works
best for us.
Todd: Totally. Definitely
preaching to the choir on that. I mean, big picture, you do an
assessment. If I was involved in something like that, I’d say, “okay,
great. So what is it that we’re trying to train for? What is the task
that the worker will be doing?” And that’s, of course, what
we’re selecting for. What are the aspects of that task? Oh, it’s a
very behavioral task. Or it’s very cognitive-heavy. Or, well, you’ve
got to have situational awareness, you’ve got to deal with any old
thing that might happen to you. Or all three.
I’d look at what are the
characteristics of the perfect worker: you’ve probably got some [in
your company]. We’re going to measure all those aspects of those
workers. And in particular, I would be guided by the neuroscience of
performance and learning. Then what you do is you use AI. And of
course, you’ve got new data coming in, so you’re constantly updating
your algorithm. But then what you do is, you have a potential new
hire or recruit, you get measures from them. It’s not a
template match per say, but it’s a match. You’re basically
correlating that new recruit’s scores with the scores for the ideal
employee, and you get sort of a measure of fit.
And I love the idea that you apply for
a job A; you’re actually not a very good fit for job A. Well, we have
job B over here; you’re a really good fit for. It’s such a more
efficient way of doing things, because let’s face it: people don’t
know what job they want, what job they’re going to fit for. If we
could use AI to help guide that process and put people in the jobs
that are best-suited for them, whether they, quote/unquote, know that
or not, right? We’re not that great at introspecting about ourselves.
We really don’t know that much about ourselves. We think we do, but
we don’t. Whereas these kinds of tools can actually give us a better
insights into what we would be good at. So, AI machine learning,
these kinds of algorithms: awesome, awesome future.
Alan: So we’ve got all of these
new technologies. They’re happening fast. We’re seeing great results
across enterprises. You said we can do more; we can do better. How
does a company even start to evaluate or look at these tools? Because
you’ve got a handful of companies working on manufacturing and
industrial, and those seem to be getting really great traction. But
what about sales training, or HR training, or soft skills training,
or retail, or… there’s so many other aspects of business that maybe
aren’t as obvious at front, but what are some of the other ones that
you see these techniques working on?
Todd: I’m really glad you
brought that up, because we’ve been talking a lot about the
manufacturing and industrial sector, and I think these tools are
taking off there, for one, because the ROI is so clear.
I was actually having a discussion with
a colleague a couple days ago, that the beauty of manufacturing is, I
can have you use one of these AR tools and I can see how quickly you
complete the task. Boom. I mean, there’s my data; it’s right in front
of me. It’s a short-term ROI. I see it immediately, and I can see it
from all of the people that I have use this AR tool.
For HR, for sales, for what people call
soft skills — I’ll use the term people skills – but, interpersonal
skills. These are harder to measure. They’re slower to develop and
evolve. They’re a little mushier to measure. And so it’s been more of
a challenge to measure the ROI for, let’s say, people skill training
with VR, than in the manufacturing sector. But it’s still there. And
I think maybe that’s where the scientists can really come in and try
to identify some short-run ROI, but also really that longer-run ROI.
Let’s face it: it’s a lot easier to interact with a machine than it
is interact with people.
Alan: People are… this
morning, before we jumped on this, I said, “you know, I feel like I
have a weekend hangover, without the alcohol.” I mean, people are
not like a machine. The machine wakes up Monday morning and just
works. It doesn’t have three days of camping, or it didn’t go to
Vegas for the weekend. Machines don’t do that. But humans, we have
all of these complicating factors. And it really complicates
business. And it’s part of the interconnected web of humanity. But at
the same time, if we can better attune ourselves to watch out for
these things using this technology. I think it’s really powerful.
Todd: I mean, let’s take
healthcare. Let’s take law enforcement, firefighters, call center,
retail — anybody who has to deal with, quote/unquote, putting out
fires with adversity. The term I use is situational awareness. It’s
an ability to always know what you need to do, right now. It’s like
you always make the right choice, and this uncanny ability to kind of
predict what’s going to happen in five minutes. There are people that
have that. And guess what? It’s trainable.
Alan: Now you’re getting crazy,
Todd. Come on now.
Todd: Well, I–
Alan: You can train a sixth
sense?
Todd: You can. You absolutely
can.
Alan: OK, let’s unpack this.
What you’re saying is, using situational awareness and situational
training in environments such as VR — where you can recreate that
environment — that situational awareness becomes built up. But how
can you build that sixth sense in somebody to predict what will
happen?
Todd: OK, so yesterday — or I
guess it was Saturday — it was the 50th anniversary of [the Apollo
11 moon landing] — those
guys had a sixth sense. They had these simulators that cost
equivalent of billions of dollars; over and over and over again, they
got thrown at them every possible situation, no matter how likely or
unlikely. They were prepared for anything. There is a use case right
there. It has been trained, and it can be trained.
“Yeah, but gosh, that was like, a
handful of guys, and it cost a ton of money. So that’s not too
realistic, Todd, in the real world.” No, you’re right. Okay, but
now imagine putting somebody in VR. You could throw all the same
situations at them. You can measure their physiological responses. VR
focuses mostly on where you look, and of course we have auditory, but
there’s no reason we can’t — actually, we should — start
looking physiological responses.
Alan: Yeah, I saw something,
somebody had taken the Gear VR — which is Samsung, you slot your
phone in and put it on your head — and it was a meditation app that
used the Samsung Gear Watch to keep your pulse rate. As you’re
meditating, you can watch your own heart rate inside the VR headset.
If we start thinking about… most of the headsets right now, the VR
headsets don’t have eye tracking. I think the only one that really
has eye tracking is the HTC Vive Pro Eye. Something so simple as
being able to know where exactly the person is looking has
dramatic effects on the knowledge base of how we move forward with
this technology. Then, when you increase that with skin response or
heart rate? We haven’t even really touched on that stuff.
Todd: Totally. And I think
that’s… I’ve talked to a number of people recently, some pretty big
companies that are starting to bring VR and/or AR into their
companies. I’ll talk to them about the neuroscience, and they’ll say,
“we’d really like to do some studies where people get a magnetic
resonance imaging machine or put the EEG on them.” And I said,
“I’ll be really honest. Those are really, really great tools.
But why? Why do you want to run one study — that’s going to cost you
an enormous amount of money — to learn one thing?” I mean, you’ll
have some pretty pictures of the brain, but there are strengths and
weaknesses of all of those techniques. And that’s a time for another
discussion.
Alan: Well, I think you can get
really deep with this technology or you can just take it one step at
a time and say, “okay, let’s just measure your heart rate.”
That’s simple with a watch band.
Todd: Totally. And if we’re
talking about situational awareness, and we’re talking about dealing
effectively with stressful situations, knowing that you’ve got EEG
activation, or that your amygdala is lighting up is not really
relevant. What’s relevant is: are you calm? You can measure that with
heart rate, galvanic skin response. We can measure that with things
that are on the market today, that are super effective.
Alan: And inexpensive too, let’s
be honest.
Todd: And inexpensive.
Alan: Skin response and heart
rate is cheap.
Todd: Very. And they’re
everywhere, right?
Alan: The ubiquity of
sensors.
Todd: And you’re seeing more and
more of… certainly, the military spending a lot of money on this,
but understanding the physiological responses: hey, perfect
application of AI. I can determine whether — to use a military
example — whether a war fighter is ready to go to war today. I can
put them through a simulation; I can be measuring their physiological
response. “You’re not up to snuff today.”
Alan: Military, they’re way
ahead. They’re using this technology, they’re studying these types of
things. But let’s just take it to even the most basic aspects: a
K-to-12 learner; some kid in grade 6. We send our kids to go to
school from 8:30 in the morning until 3:30 in the afternoon, and they
learn science from 10 to 11, math from 11 to 12. We just run them
through this gamut. But have we ever really looked at what is their
optimal time for learning? Maybe those kids learn math better at 8:30
in the morning, or maybe phys-ed in the morning. Have we ever really
looked at that? And of course, every individual learner is going to
be different.
Todd: Yeah. OK, so there’s a lot
to say and there’s a — I’m actually really glad you brought up kids,
and I’m going to add middle-aged and older adults to the mix as well,
so–
Alan: All the people.
Todd: Basically, anyway.
[laughs]
Well, okay.
Alan: You know, to be honest,
we’re all lifetime learners now.
Todd: We are.
Alan: Gone are the days where
you go to school, you graduate, you go into a job, and that’s the end
of your learning career. Now we’re entering the exponential age of
humanity. We must maintain our lifetime learning status.
Todd: Totally. There’s no doubt
about it. And we need to use learning tools that are optimal for
where we are in our lives. Kids — actually, the prefrontal cortex is
not fully developed until you’re about 25 years old — yet we have
kids learn math, like you say, from 10 to 11 in the morning. And what
are they learning it with? Their prefrontal cortex. We are training
children to learn information with a system that’s not fully
developed. That’s crazy. We should be using immersive technologies
that more broadly engage more parts of the brain, that are fully
developed. I’m not saying we shouldn’t still work the prefrontal
cortex, but relying exclusively on the prefrontal cortex with
children is incredibly suboptimal.
Move to the other end of the lifespan.
Your prefrontal cortex actually starts declining in your 40s. On that
end of the spectrum, we shouldn’t be relying purely on the prefrontal
cortex. Healthcare examples are great: I go into my doctor, they tell
me I need to have some procedure. They give me some bunches of pieces
of paper to take home to read about the procedure. Incredibly
ineffective. How about if I go into a VR experience, I see what their
procedure’s like? I’m going to be less stressed, I’m going to have
more knowledge, I’m going to be more prepared for it, and even more
satisfied. Guess what? I’m going to be more likely to heal well after
that surgery. There’s actually data on this.
Alan: There’s a sick kids
hospital here in Toronto, that did a partnership with Samsung,
originally. And what they did was, they just put a simple 360 camera
on a gurney and they wheeled it through, as if it was a kid. They
wheeled it right through to surgery so that you know which hallways
you’re going to go down, and which room you’re going to go in, what
the room looks like, and what the sounds are like. They let kids wear
that in VR before the surgery, and it decreased their stress. And
when you’re going in for surgery, stress is actually the opposite of
what you need. When you’re going into surgery, you need to be calm
and relaxed, and let your body heal.
Todd: Yep, you don’t want to be
stressed out. Reading a piece of paper does not provide the
information that you want, so you’re still uncertain. You’re still
stressed. Put on a VR headset: again, you’re broadly engaging more
parts of the brain than just the one. I don’t have to generate a
mental representation of what the hospital’s like, like I have to do
with the piece of paper. No, it’s in front of me. I know exactly what
the hospital is like. I know where the surgery room is, I have the
feel of it. And I’m going to be less stressed because of that. I’m
going to be more likely to heal. No brainer. And that’s cheap.
Alan: Really cheap. So, to put
it in perspective: in all of human history, we’ve never had a device
that can deliver training and education as efficiently and
effectively as virtual and augmented reality. Now, you combine that
with studying our emotions through eye tracking, head pose — that
sort of thing — our movements, and then also through our biometric
responses — our heart rate, our galvanic skin response — when you
combine that all together and then run it through AI to deliver this
in a personalized manner: we really are setting ourselves up for what
I believe to be the most powerful tool we’ve ever created to train
people. And I think it’s the only way that we’re gonna be able to
remain competitive in a world where everything is increasing
exponentially.
Todd: Yeah, I mean, I completely
agree. And the beauty is that each of the parts of that puzzle, we’re
making progress on. We’re making progress on AI. We’re making
progress on wearables that tell us all kinds of stuff. The VR
headsets are getting better, and the cost is going down.
We need to put all that stuff together
— which we’re doing — and we need to use science. We need to look
at how the brain processes things, do some experimental science as we
optimize these tools. And you’re absolutely right: the savings,
people are going to learn better, they’re going to be less stressed
in medical situations. We’re going to have better-trained police
officers and, I mean, you name it: everything. And it’s just around
the corner.
Alan: One thing we didn’t touch
on, which I think is timely right now, is neural link, or
brain-computer interfaces. Elon Musk’s company, Neuralink, announced
that they’re going to be able to embed these little microfibers that
act as almost like a brain stimulator, and also capturing the data.
So we’re gonna be able to capture and read-write onto our brains.
What are your thoughts on that?
Todd: Yeah, I actually have a
colleague who’s really into it, so we’ve been talking a lot about
this. I actually read a really cool article about neural link a
couple days ago. You know, it’s insane. OK, so look. So what we’re
trying to do is we’re trying to link to the brain. OK, that’s great.
Well, what is step one? “Step one: understanding how the brain
works.” Hmm, we really don’t know how the brain works all that
well, do we? “No, we don’t.” So step one is to understand how the
brain works a lot better than we do today.
Now, I think we can do that in concert
with the goals of neural link, but I think we have to always keep our
eye on the ball, which is: how does the brain process? What do these
signals mean? Understanding that there are different parts of the
brain that provide different types of information and signal
different things, and making sure that we combine that information in
the same way that the brain does. So, the engineering side is great.
We need good engineers. But we also have to always pay attention to
the “neuro” side of neural link, and really understand how the
brain works. Then we’re gonna make some real progress. And it’s
incredibly exciting.
Alan: It really, really is. I
read this quote in one of your articles that you wrote, and it really
stuck with me. And I think this kind of will cap off this
conversation:
“Learning is an experience.
Everything else is just information.” -Albert Einstein.
Todd: [laughs] A colleague of
mine, Tim Fitzpatrick, who was actually the CEO of a VR health care
company. He exposed me that quote nine months ago, and I was just
blown away by it. I mean, I’m so enamored with Einstein anyways, and
was like, “yeah, he’s a learning scientist, too, because it’s
brilliant.”
But absolutely. Learning is an
experience. You learn while doing. You learn in your environment.
Why? Because it broadly engages so many parts of your brain at once,
that the information sticks; it’s retained. You learn the behaviors.
Everything else is just information. Information is the prefrontal
cortex. It’s how we focus almost all of our learning tools. We try to
drive everything through the prefrontal cortex to the information
system, when we should be training people through experience.
Alan: Well, thank you, Todd.
We’re at the end of the XR for Business Podcast with your host, Alan
Smith, and this has been an amazing exploration of what’s possible
when we use virtual/augmented/mixed reality for training, artificial
intelligence, and biometric responses.
I have one more question to ask: what
problem in the world do you want to see solved using XR technologies?
Todd: Oh my goodness, there’s a
lot of problems in the world, but I guess the one I’ve been most
focused on — and there’s some personal reasons for this – but:
senior care. We have — some people call it the silver tsunami —
running down the train track. We’ve got boomers, myself included, who
are aging, and we do not have enough people to take care of seniors.
And we need to do a better job of training not only the seniors on
what’s to come; their family members on what to expect; and experts
to help them. And I really believe there’s so much empathy involved
there, and so much detailed understanding and training: these
technologies can absolutely solve that problem. So that’s one of the
ones that I’m most excited about and want to put a lot of my time and
energy on. But many, many more. Many, many more.
“Cha-ching”
sound effect by Muska666
View all episodes
By Alan Smithson from MetaVRse
4.5
1212 ratings
We often talk about
how XR technologies are great tools for education and training on
this podcast. But why is that? Like, physiologically? Turns out, XR
tickles the thalamus in ways traditional learning strategies never
could, and that’s not us just whistling Dixie.
Today’s guest —
Cognitive Design & Statistical Consulting, LLC CEO Todd Maddox —
has a PhD in Computational and Psychological Science, meaning there’s
no one better to explain why XR and your brain are a match made in
heaven.
Alan: You’re listening to the XR
for Business Podcast with your host, Alan Smithson. Today’s guest is
Todd Maddox. He is a cognitive design specialist. Todd is a PhD, and
the CEO and founder of Cognitive Design and Statistical Consulting
LLC. He’s also a learning scientist and a research fellow at Amalgam
Insights. His passion is to apply his 25 years of psychological and
neuroscientific expertise gained by managing a large human learning,
memory, and performance laboratory to help build better education and
training solutions. Todd has published over 200 peer reviewed
scientific articles, resulting in over 10,000 academic citations and
hundreds of speaking engagements. During his 25 year academic career,
he’s awarded $10-million in federal grants from the National
Institute of Health, National Science Foundation, and the Department
of Defense to support his research. Since entering the private
sector, Todd has embarked on a mission to translate the amazing body
of research conducted in the ivory towers into plain English and help
companies leverage this research to build better products. Todd is
especially interested in applying his expertise in the psychology and
neuroscience of learning, memory, and performance, and to use
immersive technologies in manufacturing, health care, corporate
training, and retail, to name a few. You can follow Todd on LinkedIn.
Just look for “Todd Maddox PhD.”
Todd, welcome to the show.
Todd: Hey, Alan, it is fantastic
to be here. Thank you.
Alan: It’s such an honor. I’ve
been reading your posts and your articles, and trying to get through
some of your scientific papers is a challenge. It’s so much
information there.
Todd: Yeah, I hear you. And to
be honest, my recommendation is to sort of skim the peer-reviewed
stuff, because it does seem like it’s written in a foreign language,
even though it is English. And the LinkedIn post and the more recent
stuff, where I really try to talk in plain English, because if a
scientist can’t present their work in plain English then there’s
something wrong. So that’s what I’m trying to do.
Alan: I love it. And one of the
articles that was recently published was a report on VR as an empathy
builder, through Tech Trends.
Todd: Yeah.
Alan: Here, I’m just going to
read a quote from it:
“Any profession that requires
interpersonal interaction, such as education, retail, food service,
call centers is better served with strong empathy.” Let’s start
with that.
Todd: Totally, yeah. Every one
of those examples is a people example; people interacting with other
people. I know we’ve got amazing technologies; we’ve got robots,
we’ve got all these wonderful things that are making our lives
better. But let’s face it, in the end, it’s about people interacting
with other people and caring for other people, walking a mile in
somebody else’s shoes. That is really just so critical.
These technologies — in particular
virtual reality, I would say — this is an immersive technology. I
could be dropped into any environment. That’s amazing, that’s very
cool. But now imagine: Todd, a middle-aged hetero white guy gets
dropped into an environment where Todd is now a young
African-American lesbian woman. And whatever’s happened to her has
happened to her and hey, it’s happening to me now. Now, granted, is
this the same as a lifetime of “go back where you came from?”
Of course it isn’t. But it’s a start, and it’s visceral. It engages
emotion centers in the brain in a way that is rare. And of course,
you can do this over and over and over again in virtual reality,
because I can be plopped into any environment, into any person’s body
that we like. So when it comes to empathy and understanding, I really
think that’s one of its many sweet spots, I’ll say.
Alan: There’s been some recent
research around using this for human resources, for hiring, and then
also one thing that I saw was really neat is, as a manager, you were
able to sit in virtual reality and reprimand somebody. You had to
give a disciplinary action. You sat there and you talked to the
person and it recorded your eye tracking, your hand motions, your
head pose. And then afterwards, you got to sit in the seat of that
person and watch yourself give yourself the reprimand.
Todd: Yeah. And that’s what it’s
all about, right? Just to start to dive a little bit into the
neuroscience behind all this, because that’s really where I reside
and am most fascinated. It’s one thing to study what it’s like to
give somebody a reprimand. “Here’s a PowerPoint on how you give
somebody a reprimand,” or even, “here’s a video — watch this
person getting a reprimand. What did they do right? What did they do
wrong?” All of that information is processed by what’s called
the prefrontal cortex, which is right behind your forehead. I call it
the “What System.” It’s the part of your brain that learns
what you’re supposed to do. OK, that’s great, and it’s really
important to know what to do. But let’s face it: I know that I
should eat a little more healthy than I do. So, I know what to
do. But do I do it all the time? Knowing what to do, and
knowing how to do something are two completely different
things. They’re mediated by completely different parts of the brain,
and the processing characteristics of those two brain systems are
very different.
So what you’ve got in the VR example
that you’re suggesting is, you’re actually in there doing, instead of
“here’s what I should do — watch this PowerPoint, hopefully
you’ll go do that stuff — you’re in VR and you’re actually doing it.
And I love that it’s measuring things like body language, non-verbal
communication. What’s fascinating about non-verbals is, it’ll
actually provide an enormous amount of information about whether
somebody is being genuine, showing empathy, whether what they’re
saying is not really what they mean. These are absolutely powerful
tools.
And then, yeah, when you can then swap,
you’re now getting the reprimand from you. It’s truly remarkable.
You’re just so broadly engaging so many of these parts of the brain
that are critical for these kinds of tasks. It’s just an amazing
technology.
Alan: And in this podcast,
there’s always the same things that come up: “I want to be able to
empower my field service workers hands-free.” “I want to use
virtual reality for training.” Now, the training aspect of things
is starting to accelerate much faster than anything else, because the
results that we’re seeing across the board in companies like Boeing
and Lockheed Martin and Wal-Mart, these big companies are seeing
real, tangible results using this technology. Why do you think it’s
so powerful? What is it about this technology that’s creating those
lasting synapses in the brain that feels like you’ve done it?
Todd: Yeah, it really comes down
to the brain. And there are a couple of things. Traditional ways of
training – really, what I did for 25 years when I was a college
professor educating undergrads — you have butts in the seat and
you’re droning on at them. Well, what’s happening in your brain?
First — again, we’re focused exclusively on the prefrontal cortex —
if you ask your grandmother, you know, well, what does it mean to
learn? Ask just any old buddy. That’s really what they’re talking
about. They’re talking about the prefrontal cortex. “Okay, I
have to sit down. I have to concentrate. I have to process this
information. I’ll mentally rehearse it so that I can, quote/unquote,
remember it.” Everything is driven through the prefrontal
cortex. It’s part of the brain that sets us apart from,
quote/unquote, lower animals — which is the term actually I don’t
like that much — but in an evolutionary sense, it allows us to have
what we call metacognition. That is, we can think about the fact that
we think.
But let’s take VR — or AR, for that
matter. First thing is, I’m learning through experience. I am in
the environment while I’m learning. I’m not in a classroom. I’m not
at a desk studying. I am in the environment. Why does that matter?
Well, that broadly engages perceptual representation regions in the
brain — the back of the brain, the sides of the brain, the top of
the brain; all these parts of the brain that represent the
environment that you’re in. So, you’ve already got that activation
alone.
Now let’s take, say, a hands-free AR
device. Now you’re actually generating the behaviors of whatever your
job is. And you’re being guided, let’s say, with AR assets in the
Hololens: you’re actually generating these behaviors in a guided
fashion. You’re not just reading a textbook that tells you what the
behaviors are — that hopefully you’ll remember when you’re out on
the field — but rather, you’re being guided through these behaviors,
and you’re doing them in real-time out in the field.
So, I’ve got the behavioral centers of
the brain that are activated. These are the centers deep down in the
brain in a region called the striatum. Simultaneously, if it’s an
engaging and rich environment — or like with Wal-Mart, if it’s Black
Friday is the environment that you’re in — you’re going to have
emotion centers lit up. It is as if you’re plopped down in the middle
of Black Friday. So, emotion centers in your brain are lit up — like
the amygdala and some other limbic structures. Long story short:
traditional training engages one part of your brain — the prefrontal
cortex — which is a part of your brain that cognitive load is a
problem; working memory capacity is a problem; attention span is a
problem. Versus virtual reality or augmented reality, where you’re
engaging experiential learning centers, cognitive centers, behavioral
centers, and emotional centers. So, much more the brain is being
engaged in synchrony.
This is why you learn more quickly, you
make fewer errors, and you retain the information more. This is
really what is driving these amazing return on investment for forward
thinking companies that are using these technologies.
Alan: That was actually…
[chuckles]
you know how to speak my language on this show! The XR for
Business Podcast is all about ROI — what are the best investments we
can make as businesses, to drive our business forward using
virtual/augmented/mixed reality technology? So, my next question was,
of course: what are some of the ROIs being seen by businesses?
Todd: Yeah. So you mentioned
several of them: Lockheed, Walmart. I’ve actually been following PTC
quite a bit. Actually, that’s where you and I met. I’ve been
following some of their technology and actually met several people
who were using one of their products called Expert Capture, and
getting… trying to remember the exact numbers. But, training times
cut in half, standard operating procedures being generated ten times
faster than they were before — you’re seeing numbers like this from
company after company after company. And again, from a neuroscience
of learning and a neuroscience of performance perspective, none of
this surprises me, because we’re talking about engaging one system in
your brain, versus engaging three or four systems in your brain, and
engaging them in this… almost like a ballet. It’s this beautiful
synchrony, all in the interest of achieving whatever it is that your
goal is. Whatever you’re learning task is, your training task,
whatever it is you’re trying to perform. And so you’re seeing these
ROI that’s off the charts.
I will say — and I actually just wrote
a report on this last week, also at Tech Trends — that I’ve been
making this case. I’m going to continue to make this case so people
listen to me. These technologies are fantastic. They broadly engage
all these parts of the brain, which is super. And you’re seeing
amazing ROI. My very, very strong belief is we can double those at a
minimum. Wow, how can we do that? I mean–
Alan: Hold on, hold up! So,
we’re already seeing 50, 60, 70 percent increases right across the
board. Retention rates, memorization techniques, faster training
time. You’re saying we can even double that?
Todd: We can do better on those
metrics, and we’re going to get other value that we’re not even
really measuring. For example — and I think Expert Capture’s a good
example — you’re going to retain this information better. You’re
going to learn more quickly. Okay, that’s great. You’re gonna know
what the standard operating procedures are, and you’ll be able to
regurgitate those back to me. I think you can — and I don’t want to
get too into the weeds here — but there are aspects of, in
particular AR, that are not optimized for training behavior.
Most of these AR assets… let’s say
you have the Hololens on. You’re working on a machine. The Hololens
says “move your left hand over here, and turn this knob.”
Okay, that’s great. That’s called “guided learning.” And
guided learning is solid. There’s nothing wrong with it. But if you
start tweaking and incorporating other ways of doing this, that are
less guided and more discovery-based by the learner, and –
critically — involve real-time reward and punishment. You are now
going to be training the muscle memory correctly. It’s actually
better for you to discover these things. So, be guided partially.
Then you have to generate a response and you get reward or
punishment. That is how you’re going to train the muscle memory.
You’re going to be able to train
expertise. I have 25 years of expertise in manufacturing. I believe
that with these tools, we could create experts much, much faster. And
that’s not really the goal of a lot of these technologies. It’s
really just “translate that expertise into a series of steps
that I present on the Hololens, show it to the workers to get them up
to speed.” So, they’re up to speed. That’s great. But they’re
not experts. They don’t have all of the expertise that that baby
boomer who’s retiring has. I believe we can impart that as well. But
we’re gonna have to make some changes to the way that these
technologies work. We’re going to have to optimize the AR assets to
the way the brain learns best. AR works very well now, because it
broadly engages all these parts of the brain. And that’s great.
That’s a good starting point. Now we need to optimally engage each of
those parts of the brain.
Alan: Yes. How do we do that,
then? You’ve got an AR experience that guides me through fixing a
machine — and I’ve been able to expert capture, I’ve been able to
put on a camera on one of my experts, because here’s the rub: as the
workforce starts to age and retire, the problem isn’t so much that
there aren’t jobs, the problem is that the jobs are changing
slightly. The experts who are the best in the world at that job are
starting to retire. How do we have the skills transfer from one
generation to the next?
Todd: The first thing is we need
to capture that expertise, that boomer expert who’s in manufacturing.
If you tell him, “go sit at that computer and write out your
standard operating procedures,” for one, he’s not going to be
happy. And for two, he’s not going to do a very good job. And you
might ask, “well, why is that the case? He’s got all this
expertise.” He has behavioral expertise. His expertise is in
how he interacts with that machine. That’s muscle memory. That’s
understanding situational awareness. So he actually has a feel
for when that machine isn’t working right. He can’t even describe it,
he just sort of knows it’s off a little bit. And you’re asking him to
sit down and type out standard operating procedures, which is
knowledge. So you’re asking him to use his prefrontal cortex to write
out procedures, when his expertise is behavioral and situational
awareness.
So that’s one thing: we have to do a
better job of capturing it. Expert Capture does a good job on that!
I’m not saying it doesn’t, because it says, “hey, go do your job
and I’m just going to capture it.” But how we curate that, and
how we present that to young workers is absolutely critical. And
we’re not optimized. That has not been the focus to this point.
Alan: I think a lot of the focus
has been just trying to get the technology to work, let’s be honest.
Todd: Absolutely.
Alan: We’re only 12 months into
having headsets that actually turn on when you want them to do it,
and object recognition. It’s a very new field. And I’m really excited
to listen to the next part of what you were going to say, and that
is, how then do we optimize for this? Because we have the very basics
worked out. I can put on a pair of glasses. I can recognize a
machine. I can walk you step-by-step through how to fix that machine.
Now, what you’re saying is, let’s create a more discovery-based, or
maybe even gamified experience, where I learn, but I’m learning while
doing and making mistakes.
Todd: Absolutely. To your
initial point, I want to be clear that I am not making the case that
people [in the industry] are too slow. No, not at all. And you’re
right, we are we’re in the very early stages. But we have these
technologies, and it’s time to say, “okay, I’ve got this great
technology. I can present any AR asset I want, I can present it
anywhere I want, and I can present it anytime I want.” The
question is, “when do I want it? What do I want? And where do I
want it?”
That’s where science comes in. And
first: Neuroscience. I actually had an interesting discussion about
this with Jim Heppelmann months ago at PTC, he was talking about “we
use green for some things, and red for other things.” And I
said, “Yeah. Do you know why that is, Jim?” And he’s like,
“well, green means go and red means stop.” I said, “well,
there’s a reason for that.” The cells in your brain that respond
to green are the same cells in your brain that respond to red. But
they are excitatory to green and inhibitory to red. In other words,
it is impossible not to discriminate green from red. You can
never confuse the two, because it’s built into the neurons.”
We need to leverage more things like
that. We need to go back and look at how the brain processes the
what, the where, and the when. And we need to build that into these
AR assets to optimize these things. Gamification, yeah. Start with
directed learning and then start weaning this worker off of the
directed learning, toward more discovery-based. That is going to
speed the muscle memory. That also requires rewards and punishments.
So there’s gamification right there. That’s going to be the next set
of major upgrades, I think.
Alan: What about artificial
intelligence? I just read an article recently that was talking about
a company that’s using AI for recruitment; they’re able to run these
people through a couple of games, a couple of questions, and a video.
The AI’s analyzing the video, it’s analyzing their results on the
game. And then it’s saying, “out of these hundred thousand” — it
was Unilever, by the way, that it’s doing this.
Todd: OK.
Alan: —
“out of 100,000 applications, we’ve narrowed it down to
3,500, based on just this algorithm for this particular job.” And
then they’ve also started using it to then unlock what does the
potential look like for, “maybe you’re not right for this job,
but hey, you scored in the 98th percentile on this other job that you
didn’t apply for.” It may make for much better recruiting.
One of the things that I think is
necessary that we haven’t even scratched the surface of: we use
Netflix algorithms every day to give us better movies to watch. We’re
not even touching the surface of what’s possible, when we create that
sort of AI algorithm to give us better learning in a way that works
best for us.
Todd: Totally. Definitely
preaching to the choir on that. I mean, big picture, you do an
assessment. If I was involved in something like that, I’d say, “okay,
great. So what is it that we’re trying to train for? What is the task
that the worker will be doing?” And that’s, of course, what
we’re selecting for. What are the aspects of that task? Oh, it’s a
very behavioral task. Or it’s very cognitive-heavy. Or, well, you’ve
got to have situational awareness, you’ve got to deal with any old
thing that might happen to you. Or all three.
I’d look at what are the
characteristics of the perfect worker: you’ve probably got some [in
your company]. We’re going to measure all those aspects of those
workers. And in particular, I would be guided by the neuroscience of
performance and learning. Then what you do is you use AI. And of
course, you’ve got new data coming in, so you’re constantly updating
your algorithm. But then what you do is, you have a potential new
hire or recruit, you get measures from them. It’s not a
template match per say, but it’s a match. You’re basically
correlating that new recruit’s scores with the scores for the ideal
employee, and you get sort of a measure of fit.
And I love the idea that you apply for
a job A; you’re actually not a very good fit for job A. Well, we have
job B over here; you’re a really good fit for. It’s such a more
efficient way of doing things, because let’s face it: people don’t
know what job they want, what job they’re going to fit for. If we
could use AI to help guide that process and put people in the jobs
that are best-suited for them, whether they, quote/unquote, know that
or not, right? We’re not that great at introspecting about ourselves.
We really don’t know that much about ourselves. We think we do, but
we don’t. Whereas these kinds of tools can actually give us a better
insights into what we would be good at. So, AI machine learning,
these kinds of algorithms: awesome, awesome future.
Alan: So we’ve got all of these
new technologies. They’re happening fast. We’re seeing great results
across enterprises. You said we can do more; we can do better. How
does a company even start to evaluate or look at these tools? Because
you’ve got a handful of companies working on manufacturing and
industrial, and those seem to be getting really great traction. But
what about sales training, or HR training, or soft skills training,
or retail, or… there’s so many other aspects of business that maybe
aren’t as obvious at front, but what are some of the other ones that
you see these techniques working on?
Todd: I’m really glad you
brought that up, because we’ve been talking a lot about the
manufacturing and industrial sector, and I think these tools are
taking off there, for one, because the ROI is so clear.
I was actually having a discussion with
a colleague a couple days ago, that the beauty of manufacturing is, I
can have you use one of these AR tools and I can see how quickly you
complete the task. Boom. I mean, there’s my data; it’s right in front
of me. It’s a short-term ROI. I see it immediately, and I can see it
from all of the people that I have use this AR tool.
For HR, for sales, for what people call
soft skills — I’ll use the term people skills – but, interpersonal
skills. These are harder to measure. They’re slower to develop and
evolve. They’re a little mushier to measure. And so it’s been more of
a challenge to measure the ROI for, let’s say, people skill training
with VR, than in the manufacturing sector. But it’s still there. And
I think maybe that’s where the scientists can really come in and try
to identify some short-run ROI, but also really that longer-run ROI.
Let’s face it: it’s a lot easier to interact with a machine than it
is interact with people.
Alan: People are… this
morning, before we jumped on this, I said, “you know, I feel like I
have a weekend hangover, without the alcohol.” I mean, people are
not like a machine. The machine wakes up Monday morning and just
works. It doesn’t have three days of camping, or it didn’t go to
Vegas for the weekend. Machines don’t do that. But humans, we have
all of these complicating factors. And it really complicates
business. And it’s part of the interconnected web of humanity. But at
the same time, if we can better attune ourselves to watch out for
these things using this technology. I think it’s really powerful.
Todd: I mean, let’s take
healthcare. Let’s take law enforcement, firefighters, call center,
retail — anybody who has to deal with, quote/unquote, putting out
fires with adversity. The term I use is situational awareness. It’s
an ability to always know what you need to do, right now. It’s like
you always make the right choice, and this uncanny ability to kind of
predict what’s going to happen in five minutes. There are people that
have that. And guess what? It’s trainable.
Alan: Now you’re getting crazy,
Todd. Come on now.
Todd: Well, I–
Alan: You can train a sixth
sense?
Todd: You can. You absolutely
can.
Alan: OK, let’s unpack this.
What you’re saying is, using situational awareness and situational
training in environments such as VR — where you can recreate that
environment — that situational awareness becomes built up. But how
can you build that sixth sense in somebody to predict what will
happen?
Todd: OK, so yesterday — or I
guess it was Saturday — it was the 50th anniversary of [the Apollo
11 moon landing] — those
guys had a sixth sense. They had these simulators that cost
equivalent of billions of dollars; over and over and over again, they
got thrown at them every possible situation, no matter how likely or
unlikely. They were prepared for anything. There is a use case right
there. It has been trained, and it can be trained.
“Yeah, but gosh, that was like, a
handful of guys, and it cost a ton of money. So that’s not too
realistic, Todd, in the real world.” No, you’re right. Okay, but
now imagine putting somebody in VR. You could throw all the same
situations at them. You can measure their physiological responses. VR
focuses mostly on where you look, and of course we have auditory, but
there’s no reason we can’t — actually, we should — start
looking physiological responses.
Alan: Yeah, I saw something,
somebody had taken the Gear VR — which is Samsung, you slot your
phone in and put it on your head — and it was a meditation app that
used the Samsung Gear Watch to keep your pulse rate. As you’re
meditating, you can watch your own heart rate inside the VR headset.
If we start thinking about… most of the headsets right now, the VR
headsets don’t have eye tracking. I think the only one that really
has eye tracking is the HTC Vive Pro Eye. Something so simple as
being able to know where exactly the person is looking has
dramatic effects on the knowledge base of how we move forward with
this technology. Then, when you increase that with skin response or
heart rate? We haven’t even really touched on that stuff.
Todd: Totally. And I think
that’s… I’ve talked to a number of people recently, some pretty big
companies that are starting to bring VR and/or AR into their
companies. I’ll talk to them about the neuroscience, and they’ll say,
“we’d really like to do some studies where people get a magnetic
resonance imaging machine or put the EEG on them.” And I said,
“I’ll be really honest. Those are really, really great tools.
But why? Why do you want to run one study — that’s going to cost you
an enormous amount of money — to learn one thing?” I mean, you’ll
have some pretty pictures of the brain, but there are strengths and
weaknesses of all of those techniques. And that’s a time for another
discussion.
Alan: Well, I think you can get
really deep with this technology or you can just take it one step at
a time and say, “okay, let’s just measure your heart rate.”
That’s simple with a watch band.
Todd: Totally. And if we’re
talking about situational awareness, and we’re talking about dealing
effectively with stressful situations, knowing that you’ve got EEG
activation, or that your amygdala is lighting up is not really
relevant. What’s relevant is: are you calm? You can measure that with
heart rate, galvanic skin response. We can measure that with things
that are on the market today, that are super effective.
Alan: And inexpensive too, let’s
be honest.
Todd: And inexpensive.
Alan: Skin response and heart
rate is cheap.
Todd: Very. And they’re
everywhere, right?
Alan: The ubiquity of
sensors.
Todd: And you’re seeing more and
more of… certainly, the military spending a lot of money on this,
but understanding the physiological responses: hey, perfect
application of AI. I can determine whether — to use a military
example — whether a war fighter is ready to go to war today. I can
put them through a simulation; I can be measuring their physiological
response. “You’re not up to snuff today.”
Alan: Military, they’re way
ahead. They’re using this technology, they’re studying these types of
things. But let’s just take it to even the most basic aspects: a
K-to-12 learner; some kid in grade 6. We send our kids to go to
school from 8:30 in the morning until 3:30 in the afternoon, and they
learn science from 10 to 11, math from 11 to 12. We just run them
through this gamut. But have we ever really looked at what is their
optimal time for learning? Maybe those kids learn math better at 8:30
in the morning, or maybe phys-ed in the morning. Have we ever really
looked at that? And of course, every individual learner is going to
be different.
Todd: Yeah. OK, so there’s a lot
to say and there’s a — I’m actually really glad you brought up kids,
and I’m going to add middle-aged and older adults to the mix as well,
so–
Alan: All the people.
Todd: Basically, anyway.
[laughs]
Well, okay.
Alan: You know, to be honest,
we’re all lifetime learners now.
Todd: We are.
Alan: Gone are the days where
you go to school, you graduate, you go into a job, and that’s the end
of your learning career. Now we’re entering the exponential age of
humanity. We must maintain our lifetime learning status.
Todd: Totally. There’s no doubt
about it. And we need to use learning tools that are optimal for
where we are in our lives. Kids — actually, the prefrontal cortex is
not fully developed until you’re about 25 years old — yet we have
kids learn math, like you say, from 10 to 11 in the morning. And what
are they learning it with? Their prefrontal cortex. We are training
children to learn information with a system that’s not fully
developed. That’s crazy. We should be using immersive technologies
that more broadly engage more parts of the brain, that are fully
developed. I’m not saying we shouldn’t still work the prefrontal
cortex, but relying exclusively on the prefrontal cortex with
children is incredibly suboptimal.
Move to the other end of the lifespan.
Your prefrontal cortex actually starts declining in your 40s. On that
end of the spectrum, we shouldn’t be relying purely on the prefrontal
cortex. Healthcare examples are great: I go into my doctor, they tell
me I need to have some procedure. They give me some bunches of pieces
of paper to take home to read about the procedure. Incredibly
ineffective. How about if I go into a VR experience, I see what their
procedure’s like? I’m going to be less stressed, I’m going to have
more knowledge, I’m going to be more prepared for it, and even more
satisfied. Guess what? I’m going to be more likely to heal well after
that surgery. There’s actually data on this.
Alan: There’s a sick kids
hospital here in Toronto, that did a partnership with Samsung,
originally. And what they did was, they just put a simple 360 camera
on a gurney and they wheeled it through, as if it was a kid. They
wheeled it right through to surgery so that you know which hallways
you’re going to go down, and which room you’re going to go in, what
the room looks like, and what the sounds are like. They let kids wear
that in VR before the surgery, and it decreased their stress. And
when you’re going in for surgery, stress is actually the opposite of
what you need. When you’re going into surgery, you need to be calm
and relaxed, and let your body heal.
Todd: Yep, you don’t want to be
stressed out. Reading a piece of paper does not provide the
information that you want, so you’re still uncertain. You’re still
stressed. Put on a VR headset: again, you’re broadly engaging more
parts of the brain than just the one. I don’t have to generate a
mental representation of what the hospital’s like, like I have to do
with the piece of paper. No, it’s in front of me. I know exactly what
the hospital is like. I know where the surgery room is, I have the
feel of it. And I’m going to be less stressed because of that. I’m
going to be more likely to heal. No brainer. And that’s cheap.
Alan: Really cheap. So, to put
it in perspective: in all of human history, we’ve never had a device
that can deliver training and education as efficiently and
effectively as virtual and augmented reality. Now, you combine that
with studying our emotions through eye tracking, head pose — that
sort of thing — our movements, and then also through our biometric
responses — our heart rate, our galvanic skin response — when you
combine that all together and then run it through AI to deliver this
in a personalized manner: we really are setting ourselves up for what
I believe to be the most powerful tool we’ve ever created to train
people. And I think it’s the only way that we’re gonna be able to
remain competitive in a world where everything is increasing
exponentially.
Todd: Yeah, I mean, I completely
agree. And the beauty is that each of the parts of that puzzle, we’re
making progress on. We’re making progress on AI. We’re making
progress on wearables that tell us all kinds of stuff. The VR
headsets are getting better, and the cost is going down.
We need to put all that stuff together
— which we’re doing — and we need to use science. We need to look
at how the brain processes things, do some experimental science as we
optimize these tools. And you’re absolutely right: the savings,
people are going to learn better, they’re going to be less stressed
in medical situations. We’re going to have better-trained police
officers and, I mean, you name it: everything. And it’s just around
the corner.
Alan: One thing we didn’t touch
on, which I think is timely right now, is neural link, or
brain-computer interfaces. Elon Musk’s company, Neuralink, announced
that they’re going to be able to embed these little microfibers that
act as almost like a brain stimulator, and also capturing the data.
So we’re gonna be able to capture and read-write onto our brains.
What are your thoughts on that?
Todd: Yeah, I actually have a
colleague who’s really into it, so we’ve been talking a lot about
this. I actually read a really cool article about neural link a
couple days ago. You know, it’s insane. OK, so look. So what we’re
trying to do is we’re trying to link to the brain. OK, that’s great.
Well, what is step one? “Step one: understanding how the brain
works.” Hmm, we really don’t know how the brain works all that
well, do we? “No, we don’t.” So step one is to understand how the
brain works a lot better than we do today.
Now, I think we can do that in concert
with the goals of neural link, but I think we have to always keep our
eye on the ball, which is: how does the brain process? What do these
signals mean? Understanding that there are different parts of the
brain that provide different types of information and signal
different things, and making sure that we combine that information in
the same way that the brain does. So, the engineering side is great.
We need good engineers. But we also have to always pay attention to
the “neuro” side of neural link, and really understand how the
brain works. Then we’re gonna make some real progress. And it’s
incredibly exciting.
Alan: It really, really is. I
read this quote in one of your articles that you wrote, and it really
stuck with me. And I think this kind of will cap off this
conversation:
“Learning is an experience.
Everything else is just information.” -Albert Einstein.
Todd: [laughs] A colleague of
mine, Tim Fitzpatrick, who was actually the CEO of a VR health care
company. He exposed me that quote nine months ago, and I was just
blown away by it. I mean, I’m so enamored with Einstein anyways, and
was like, “yeah, he’s a learning scientist, too, because it’s
brilliant.”
But absolutely. Learning is an
experience. You learn while doing. You learn in your environment.
Why? Because it broadly engages so many parts of your brain at once,
that the information sticks; it’s retained. You learn the behaviors.
Everything else is just information. Information is the prefrontal
cortex. It’s how we focus almost all of our learning tools. We try to
drive everything through the prefrontal cortex to the information
system, when we should be training people through experience.
Alan: Well, thank you, Todd.
We’re at the end of the XR for Business Podcast with your host, Alan
Smith, and this has been an amazing exploration of what’s possible
when we use virtual/augmented/mixed reality for training, artificial
intelligence, and biometric responses.
I have one more question to ask: what
problem in the world do you want to see solved using XR technologies?
Todd: Oh my goodness, there’s a
lot of problems in the world, but I guess the one I’ve been most
focused on — and there’s some personal reasons for this – but:
senior care. We have — some people call it the silver tsunami —
running down the train track. We’ve got boomers, myself included, who
are aging, and we do not have enough people to take care of seniors.
And we need to do a better job of training not only the seniors on
what’s to come; their family members on what to expect; and experts
to help them. And I really believe there’s so much empathy involved
there, and so much detailed understanding and training: these
technologies can absolutely solve that problem. So that’s one of the
ones that I’m most excited about and want to put a lot of my time and
energy on. But many, many more. Many, many more.
“Cha-ching”
sound effect by Muska666