Sign up to save your podcasts
Or
Share Massive XR Environments and Transforming Education with Alvin Wang Graylin
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Massive XR Environments and Transforming Education with Alvin Wang Graylin
The latest generation of XR technologies introduces radical new capabilities, from multi-user tracking in massive spaces, to 6DOF standalone headsets, to the ability to track hands, eyes, and lips. HTC’s Alvin Wang Graylin discusses what this means for everything from automotive design to helping children learn about the universe.
Alvin Wang Graylin is an industry leader, evangelist and passionate driver of XR technologies, particularly virtual reality. As the China President at HTC, he leads all aspects of the company’s VR and smartphone business in the region.
Alan: Today’s guest is an industry leader, evangelist and passionate driver of XR technologies, and particular virtual reality, Mr. Alvin Wang-Graylin. Mr. Graylin is the China President at HTC, leading all aspects of the Vive/VR (VIVE.com) and the Smartphone businesses in the region. For those of you not familiar with HTC Vive, VIVE is a first-of-its-kind virtual reality platform, built and optimized for room-scale VR and true-to-life interactions. Delivering on the promise of VR with game-changing technology and best-in-class content, VIVE has created the strongest ecosystem for VR hardware and software, bringing VR to consumers, developers and enterprises alike.
He is also currently Vice-Chairman of the Industry of Virtual Reality Alliance (IVRA.com) with 300+ company members, President of the $18B Virtual Reality Venture Capital Alliance (VRVCA.com) and oversees the Vive X VR Accelerators (VIVEX.co) in Beijing, Shenzhen and Tel Aviv. Mr. Graylin was born in China and educated in the US. He received his MS in computer science from MIT and MBA from MIT’s Sloan School of Management. Mr. Graylin graduated top of his department with a BS in electrical engineering from the University of Washington, where he had specialized in VR and AI over two decades ago under the tutelage of VR pioneer Tom Furness.
Please welcome to the show Mr. Alvin Wang-Graylin.
Alvin:
Hi, how are you doing, Alan?
Alan:
I’m fantastic. There we go, we got you now. Awesome. Where are you
calling in from?
Alvin:
I’m in Beijing, China.
Alan:
Beijing right now. And what time is it? It’s gotta be in the middle
of the night, I think.
Alvin:
About 10:30 PM.
Alan:
Oh, well thank you so much for taking the time to record this with
us. I’m going to jump right into it. For the people listening, you know,
really, really exciting things happening at HTC right now, and you just held
your fourth annual VIVE Ecosystem Conference, or VEC conference in Shenzhen.
Can you maybe speak to some of the announcements and how their impact is going
to really impact business use cases of VR?
Alvin:
Yeah, I’m happy to jump right in. We just had our yearly biggest
conference of the year, and had about a thousand people come in, and about a
hundred press, and essentially all the industry folks that are in China – and
actually, quite a few folks from around Asia and even parts of the US – came.
People, from the developers, from our sales channels, our accessory partners. A
lot of investment companies, as well as Chinese carriers, governmental
organizations that are involved with high tech. Essentially, what we do every
year is gather together all of the leading players in the industry and try to
create a unified direction. And the key direction that we were trying to point
to this year is something called multi-mode VR. That’s when, you know, VR can
be used, not just for one way of connecting, but it could be connected to your
PC. It could be connected to 5G cloud VR. It can be connected to a console, or
a 360 streaming camera, etc. So it’s a very exciting time for us in the
industry, that these new types of innovation’s happening.
Alan:
I got to read some of the amazing announcements. You know, one of
the ones that I think everybody’s really excited for is the VIVE Focus Plus.
And you know, it’s for those of you listening, it’s a standalone headset that
is completely six degrees of freedom, meaning you can look left, right, up, and
down, but you can also move around. How do you think this new piece of hardware
is going to unlock the enterprise use cases of the technology?
Alvin: Absolutely. I think that was the highlight of the show, and you know, this is the first full six degrees of freedom standalone device that has both six degrees of freedom in the head, as well as the hands. And to be able to have that freedom, where you can move around in as large of a space as you want to, and be able to see, you know, and use all the existing content that’s out there – it’s going to be amazing, for both developers and for users out there. You know, for enterprise, you can essentially not worry about having the wires of the devices being connected. If you’re a design firm and you want to look at a car or an airplane, you can actually walk around freely without having any burden. And we actually showed off something that was quite exciting, is a large area multi-user tracking system that allows tracking to spaces up to 88,000 square meters, which is like 900,000 square feet space, with only four sensor units, and up to about 40 devices in that space. So it allows for shared teams to be able to look together and review and work together on a virtual space as large as four football fields. Or, it could be applied to theme parks, where you could have you know people go into a big, flat field and look up and see a giant amusement park in front of them. So that’s, you know, that was one of the other exciting parts that was announced last week as well.
Alvin:
That’s absolutely incredible. You just… you just literally blew my
mind. That’s the one thing I hadn’t read about. So, to just recap: with four
sensors, you can have up to 40 devices – meaning 40 people can be in a shared
experience – in the size of four football fields, 900,000 square feet.
Alan:
Yep that’s right.
Alvin:
So I just have to kind of… I’ve got to unpack that. So, for
enterprise use cases, you can now have a shared design experience, or
experience in general, where people can train together. This would be good for
military, I think; for police. For, you know, virtually any company that has
many people training at one time. I think the training applications are just
going to be massive with this technology.
Alvin:
Yeah. I mean, if you think about it, let’s say you have a
firefighter squad that wants to train together, and, you know, they have a
giant field that they can simulate a real fire in a building. How would they
behave? And it can give you a full 6DOF tracking, and be able to have a unified
positional system, so that everybody can see each other in physical space
that’s fully aligned to the virtual space.
Alan:
I’m speechless. That is a game changer. Speaking of game changers, one
of the other things you guys announced at VEC this year is six degrees of
freedom video. So, normal 360 video allows you to kind of look around from the
point of the camera, but what you guys have developed is a way to give, kind
of, more presence by adding that ability to move up and down, and left and
right. Can you maybe speak to that?
Alvin:
Yeah. So, you know, there’s thousands – probably tens of thousands –
of 360 videos out there, but it’s really designed for just rotational viewing.
So you’re in the middle of this photo/video bubble, and you can rotate your
head. But when you move nothing really happens; that bubble moves with you.
What happens with this 6DOF – what we call 6DOF light video – it’s not full
6DOF, because you can’t really move around in a large space, but allows you to
move one meter in any direction: up, down, side-to-side, and give you that
illusion of being able to have a movable space, and lets you create a higher
level immersion than you can’t have with these existing content. Now, the best
part of this is that you don’t need a new camera. You don’t need new processing
equipment. You don’t need to do anything. You just take your existing 360 video
files or streams, stream it through it as this new video player, turn on the
6DOF light mode, and boom – you’re able to have that movement within the video.
Now, there are, you know, can people use
light VR cameras? Which allow you to make larger space movements – a few
meters, and maybe tens of meters – in a space? But they require very expensive
camera rigs, to be able to create what this allows now. Essentially, you can
have a standard $200-$300 360 camera, and bring that 6DOF experience to anybody
who you stream to. So, I think this is going to open up a new concept called
“life streaming,” where it’s not… you know, people right now, there
are these influencers who will shoot a little video of where they are and talk
about a story. But I think in this case, you can turn anybody into a
“lifestreamer,” where they can share their life with their loved ones
or with their fans.
Alan:
You just encouraged me to break out my 360 camera again, and start
making some videos. This is incredible. So, I’m going to touch on some other
things that really kind of stood out to me. One of the things that you guys
announced was StreamLink, which is the ability to connect your devices – your
VIVE devices, your VR devices – to computers, and consoles, PlayStation, I even
saw something about XBox there. Is this for real? I can just plug in my headset
to any console now, and have a powerful VR experience?
Alvin:
Well so, it’s, what StreamLink does is it allows you to take 2D
content, whether it’s from your computer screen or from your TV set top box, or
from any of the existing consoles, because it takes any HDMI video stream,
turns it into a slightly warped, rounded virtual screen in VR. So that gives
you essentially an IMAX experience with your VR headset. So it’s more of a
monitor replacement than a full VR experience. What it also, because of the
size, it actually does almost feel like you’re there. I was playing FIFA soccer
on the XBox, but then I click it in and I feel like I’m actually on the field,
because I can’t see the edges of that screen. If you get it closer to the
screen, you actually, you know, there’s a spatial element of that screen. So
once you get closer, you can actually have the screen go around you, so you
can’t see, if you back up from it, you can see the screen. And you can actually
size the screen as well. You can also change the shape to be flat or to be
slightly rounded. And one of the really interesting parts is that it also has a
video see-through a window. If you look down, you can actually see your
keyboard, or you could see your hand game controller, and that way you’re able
to manipulate physical objects, even when you’re in this headset as a monitor
replacement.
Alan:
So that leads me to the next thing that I saw. First of all, that’s
incredible. The visual that I’m getting is of, you know, a trader or a
financial trader being able to put up their screen in a massive screen, so your
computer now is a wrap-around IMAX screen. Imagine the amount of work you could
get done by having an IMAX-sized screen while you’re doing your work. That’s
just incredible.
Alvin:
Yes. I think it’s both good for work and for entertainment, that’s the
best part. In fact, our guys are trying to create a little program to get
people to spend a whole day, like 24 hours, inside VR. And just to use this one
single device, because it’s able to connect to all of the current media sources
that we have anyways. And you know, the Focus Plus also allows you to connect
wirelessly to your phone and wirelessly to your PC, to experience PC VR. So all
the 4000+ pieces of PC VR content can be streamed directly to this standalone
device. So it’s kind of like The Lord of the Rings, this is like the Lord of
HMV. You know, they’re the HMV that connects to everything.
Alan:
I can tell you, this is what I personally have been waiting for. I
want to be able to open my – I know it sounds boring to everybody – but I want
to be able to open my emails and work in VR on a giant screen and have, you
know, my podcast link over here, and my e-mails over here, and all these things
kind of spread out. I am really excited for this. And you know, one of the
things you mentioned was looking down and seeing your hands. Something else you
guys addressed at VEC this year was native hand-and-finger tracking.
Alvin:
Yes.
Alan:
This is going to unlock so much.
Alvin:
Yeah, we definitely have a lot of news packed in to–
Alan:
I’ve got a long list here! I’m gonna keep going. This is amazing,
you guys announced so many things.
Alvin:
Yeah, I think we announced too many things, because most people only
heard probably maybe a third of it, or only comprehended a third of what we
what we announced. Yes, so at the VEC – and at GDC – we actually announced our
USDKs for hand tracking. About a year ago actually, at the last VEC, I had
announced that we were going to bring gesture control to VIVE, using the
existing cameras that are on both the VIVE and the VIVE Focus. So at the GDC
this year, we demonstrated the hand-and-finger tracking on the VIVE Pro. But at
VEC, we also demonstrated the finger tracking on the VIVE Focus. So, on a
standalone device, to be able to have 21-point finger tracking, that’s amazing.
We actually didn’t think it was possible, because of processing limitations of
the mobile chipsets. But after almost a year of optimizing, we got it all to
work, and then we also released publicly the SD case for the VIVE and the VIVE
Pro, and the gestures on the VIVE Focus, so that any developer, now, can essentially
incorporate natural hand movement directly into the app without having to buy a
third-party piece of accessory like the Leap Motion.
Alan:
So I keep coming back to training, but let’s really kind of take a
second to unpack the native hand-tracking and finger-tracking; if you’re able
to reach out, see your hands, and interact with things in virtual spaces, this
is going to literally unlock unlimited potential for training. So, if you want
to teach somebody how to do something in real life, they can reach out, grab
it, learn it. And you know, one of the things that you posted recently was on
education and how VR is improving students’ concentration. I think both
students in grade school from K to 12, but also university, but I think right
through to enterprise training scenarios, I think virtual reality has a
potential to really lead the way with how we educate in the future, and how we
train. It’s kind of my personal passion as well, but can you speak to what you
posted the other day about the VR improving students concentration? By a
considerable amount.
Alvin:
This is a brand new study – it has not actually been published yet.
It will be published in about two weeks. And there was a study that we helped,
I guess, fund, but then it was completely done independently from us. We had no
influence over the study processes. But what they did was essentially brought
two different groups of kids that were 12-to-13-year-old kids, and taught them
basic physics, mechanics. And so, two kids had the same courses, the same
questions, the same tests, and one supplemented the experience with
VR-supplemented material, to allow the kids to be able to interact with the
principles that was discussed. And the other one was just using more blackboard
and teacher discussions. So, your normal classroom-type methodology.
What we found was that the concentration
went… and for both sets of students, we, for both at the students we used EEG
sensors on them during the entire class, and was able to capture the movement
and the intensity of the frontal lobe, which was to to identify the
concentration level and focus of the students. And it was very, very clear that
the students that were in the VR session, they had, I think it was a 6x
improvement, in terms of concentration level, which led to grades that were
significantly higher. Essentially, the control group had scores that was about
a C+, right after they did the course, in terms of passing the tests, whereas
the VR group right after was an A score, in terms of their average score. And a
week later, it was still in mid-B range for the VR students.
So even a week later, after not being
reminded the content after they were retested, they outperformed the immediate
test scores of the control group. And a week later, on the control group, was
down to, I think it was a D+. So either way, what it shows is that for complex
topics, this type of a training mechanism really both helps to increase
concentration and retention for the content, and you know, very, very
statistically significant.
Alan:
It’s incredible, and you know, we’re seeing already some major
shifts in how enterprises are using this, and I just want to touch, before we
move on to that, the one other stat that I saw here from one of your
presentations, was VR training is useful also in sports education. You guys did
a study showing improvement using soccer teams, children’s soccer teams, and
using VR training. One of the things that you showed was a 36 per cent increase
in performance after using VR. Can you speak to that?
Alvin:
So actually, before going into that, I want to add one more point to
the last study on the concentration level, because one piece of data that’s not
actually in the slide I published was that there was a distinctive difference,
actually, in the concentration improvements for male versus female. It seemed
like the more distractable boys actually had even bigger improvements than the
girls, who are normally very able to concentrate in class, whereas the boys
usually tend to be harder to manage and are easily distractable. And what we
found is that they actually had a bigger improvement because of VR. So the more
distractable that the age group or the child, the more benefit they actually
will get from this from this technology. So that is a very surprising finding.
But I guess it’s intuitive after the fact, looking at it.
Alan:
It’s interesting you say that, because I’d been reading a lot of
articles around how virtual reality is being used for autism, and to really
train people with different levels …on the spectrum, to interact with other
students, and kind of studying their eye contact and that sort of thing. And
one of the things that you guys announced at VEC – I keep going back to this –
but, you know eye tracking and lip tracking.
Alvin:
Yes. Yes.
Alan:
This is incredible. So, you know, I’m thinking for HR, if you wanted
to do an interview with somebody, you could send them a headset, you could do a
full interview in VR with them and really get a sense of, are they looking at
you? Do they make eye contact? Now that you can see their hands and fingers, do
they use their hands and gestures? And lip tracking allows you to kind of match
what they’re saying to a visual, maybe–
Alvin:
It can probably go further than that, because micro expressions in
your body really gives a sign of whether or not you’re telling the truth, or
whether you’re fibbing, or whether you’re kind of making up things, and what part
of your brain you’re looking at, that you’re trying to withdraw information
from. Well actually, your eyes point to different places. So, by looking at eye
tracking, you can actually use it as a lie detector, and probably in a more
accurate way than the current polygraphs that are out there. So yeah, that’s
also a new technology that we are putting out.
Even though we just announced recently our
VIVE Pro Eye, which has built-in eye-tracking, the SDK that we are releasing
will actually work with any existing eye-tracking system. So in fact, very soon
we’ll be coming out with third party accessories that can add eye-tracking onto
our existing VIVEs, VIVE Pros, and also the VIVE Focus.
Alan:
That’s incredible. I actually… it’s interesting, the whole idea of
the lie detector. I just read a piece by Jeremy Bailenson – and for those of
you who don’t know he’s a researcher at Stanford – and he was on the privacy of
people while collecting physical data. So, things like eye tracking, now lip
tracking, but also their movements. You know, their posture, how they move, and
that sort of thing. And the study or the piece really talked about, by
collecting this data, we’re able to now, down to a very granular level, one
that kind of transcends all of the other studies we’ve ever had before into
collecting data about people that is way more accurate than just asking them on
a survey or that we have…this is next-level data collection. And I think,
while it opens a Pandora’s box for ethical concerns, it also, you know, can
serve businesses very, very dramatically on both their HR needs, their training
needs. There’s so many different aspects of this.
Alvin:
Yeah yeah absolutely. And let’s say if you’re an advertising agency,
and you want to know if your ad works – what are people really looking at?
What’s attracting their attention? You know exactly by looking at, by using eye
tracking and putting them in front of that ad, either in the virtual space or a
physical space. If you’re designing a cockpit for a car or an airplane and you
want to make sure that it’s easy for people to understand, you put them into
that space and you know how long did it take them to find the right buttons.
Where are they spending their time looking when they’re when you’re driving
this car, when they’re using this car? So those kinds of information, it’s very
difficult to get in the real world, or you had to spend a lot of money to to
actually build out versions of that physical cockpit, and then have people try
it and then give you a feeling of what which one felt better. Right? With this,
you know exactly how long it would take them to find that button. Did they look
at the speedometer numbers in the right way, or do they prefer to look up or
down, et cetera. That kind of information can give you so much value in terms
of designing something that’s more human-centric.
Alan:
Well, since you brought it up, I’m going to talk about the
automobile industry and logistics and travel, automotive in general. So you’ve
done work, HTC has done work with BMW, Volkswagen, McLaren, probably a number
of other companies in the periphery as well. Can you speak to some of the ways
that automotive companies are using this? I know Volkswagen’s using it for
collaborative training, for factory logistics. BMW is doing virtual prototypes.
McLaren allows you to race one of their McLarens. What’s going on? Why are
automotive companies jumping on so much, and what are they doing with it? Where
are they seeing the most value?
Alvin:
I think it actually can be used in essentially all steps of their
creation/design/testing/training/manufacturing process. Normally, whether
you’re talking about a car or a plane, you’re talking on the order of years, or
a number of years, to design and build a new product. And what we’re finding is
that, in the design industry, we can get a 2x, up to maybe a 10x improvement in
terms of how long it takes to go from concept to product. Because a lot of
what’s happening today, in terms of creating these things, is that they have to
create physical prototypes that are made of clay, have people come from around
the world to review them, make changes, come back a few weeks later, etc.,
which just doesn’t produce very fast turnaround. In VR, you can do all of that
virtually. And in fact, we had a very interesting case study just released a
few months ago – two to three months ago – where Bell Helicopters, normally it
takes five to seven years to build and design a new helicopter. It took them
six months to go from concept to a working prototype that was flying, and that
that’s just crazy fast. I think that’s faster than that would take us to build
a new headset. So it’s amazing how much value they got from using that. I don’t
think that’s typical. I would I would say typically, probably an improvement of
2-to-3x in terms of design time is probably typical.
Alan:
So… let’s just kind of stop for one second. So Bell Helicopter
designed a new helicopter in six months that would have taken them five years.
Alvin:
Yes, exactly.
Alan:
And that’s an outlier. OK. We’ll take that as an outlier. But
typically, average what you’re seeing is a 2x to 3x expedited process of
design.
Alvin:
Yeah. And that’s not just for vehicles. It can be used for
buildings, for interior design; essentially anything that requires review and
design and collaboration between teams to create. It can be dramatically
faster, improved.
Alan:
So that leads me to the next question. Why isn’t every company doing
this already? If you’re seeing two to three times improvements, why is this not
in every single design office in the world?
Alvin:
Actually, I would say we’re working with almost every major auto
manufacturer and airplane manufacturer out there. In fact, if you look at
Boeing and Airbus, they were working 5-10 years ago with CAVE systems, and CAVE
systems essentially are an early version of VR, where you’re in a room and it’s
projecting the expected outcome around you in these screens that that feels
like you’re in that environment. It’s not as individualized as you would get
with current headsets, but for the technology of the time, it looked and felt
very much like today’s VR. And those were million-dollar systems, but they’ve
been using them for 10, 15 years.
Alan:
Absolutely incredible. So it’s really interesting, I mean, you
studied under kind of the godfather of VR, Tom Furness, and you were looking at
how VR can disrupt education.
Alvin:
Yes.
Alan:
You shared some stats, around six times improvement of
concentration, and one of the other stats was a 36 per cent improvement in
sports education, performance, and training.
Alvin:
Yes, actually, I think I forgot to mention the sports education when
I said – let me let me go back to that case that you asked for. So we had
worked with the national youth team that was managed by the Chinese Ministry of
Sports, and they’re essentially the elite, kind of next generation soccer
athletes in China, and they were brought in for a one month training camp. So,
they had four teams, and two teams were using VR, two teams were not using VR,
and they had the same the same kind of national-level coaches helping all four
of these teams, except some were supplemented with VR strategy education.
What they found is, at the end of the
month, the group that was using VR improved by 30 per cent in terms of their
strategy scores. In the elite athlete level, it’s not the physical capability
of these kids or adult athletes that separates them; it’s all about their
mental acuity, in terms of understanding the deeper level of the strategy of
the game. And so that’s something that’s actually very difficult for people to
train, and you have to create varied scenarios, and it’s hard to simulate those
scenarios. But when you can put these kids into thousands of potential
scenarios, if these defenders are here and the ball is here and the goalie was
there, what should you do? Who should you pass to and how should you kick this,
or how should you defend? And by putting them in those scenarios, and then
being able to score them and be able to measure them, they were able to
accelerate significantly something that would take years to train. And they
essentially got that level of improvement in the month, whereas the average
non-VR student or athlete only improved by about 5 per cent in terms of their
post-month training scores. So it’s, again, like 6, 7, 8x difference in terms
of improvement.
Alan:
That would probably explain why Walmart has decided to roll out VR
across their entire company to train people.
Alvin:
Yes. Yes. I mean, the larger your workforce, the more important
training is to your teams. And it can be used for sales training, like what
Wal-Mart’s doing. But also at Volkswagen, they are using it for assembly line
training, so people don’t make any mistakes when they… or, they make the
mistakes in virtual reality first, so they don’t make it on the real car. And
you have people like Send For Help, who are using it to train nurses so that
they don’t harm the patients. They make all their mistakes and VR, and their
teacher can be alongside them to watch them and see what they’re doing in the
virtual world. Again, training can be applied to essentially every industry.
Alan:
It’s interesting, you know, one of the companies that we’re working
with is called Career VR, and what they’ve done is created simulators to show
youth and people looking to do different jobs and explore different career
opportunities. One of them is you know a crane operator, and one of them is a
forklift operator, one of those is a heavy machinery operator. Plumbing, and
welding. So by creating these virtual experiences, it lets people really get an
understanding of what is this job all about. Do I like it? Is it something I
want to go into? And then from there they can train in it. I know you guys did
some work with Raymond Forklift. Can you maybe speak to that and what they’re
using it for?
Alvin:
Yes. So, you know, these forklifts are used in warehouses to move
very heavy equipment and materials, and if you make a mistake, it could be
thousands of dollars, or tens of thousands of dollars of damage. Right? So what
they did was they essentially created a cockpit that was completed at the same
size, shape, and all the buttons in the right places. You put on this headset
and you can simulate yourself driving this forklift, and they can test and be
able to measure all the things that you’re doing to make sure that you’re doing
it safely and you’re following all the procedures. And then you’re also getting
your muscle memory, because you’re touching all the right buttons, and those
buttons and knobs and levers link exactly to what would happen in the physical
world in the headset space that you’re in. So it is kind of a 1:1 physical,
kind of like a simulator training for airplane pilots; they’re doing it for
forklifts.
Alan:
It’s incredible. I actually got to try, with the Career VR guys, I
got to try an excavator and, I mean, I’ve never been in one of these machines
before, I had no idea. But step by step, they walk me through starting the
machine, turning the pivot, driving it, moving the dump or the bucket. And by
the end of it, I was like, okay, you know, I’m pretty sure I could get into any
of these excavators and work it. And a friend of mine at VR Scout, he did a
crane trainer, where you drive a crane and you go through this thing, and in
one hour and crane training in VR. Then they took him outside put him in a real
crane and said, “here, drive it.” And he was able to drive it. And
that’s wow. I mean, incredible!
Alvin:
Yeah. I mean, those cranes are pretty expensive; you don’t want them
making their mistakes on those things. So that’s, you know, just to spend one
hour or two to prevent a potential damage to the device or the property around
it? That is well worth it. And I’ll give you another example of a very
different type of training, is police training. We actually worked with some of
the Chinese police force, where they’re trying to get their officers to be to
react properly to very high-stress environments. And these high stress
environments are very hard to simulate in the real world. You’re not going to
have bad guys and good guys all in the same place, and be able to use a gun to
shoot at them. But to be able to distinguish, to be a know how to react, to be
able to stay calm, and be able to measure and see it from a trainer’s perspective,
what the natural reaction of these different officers are. That’s something
that they’ve told us is immensely valuable.
Alan:
Yeah, I think police training is going to be huge, and also
emergency services. One of the other ones that… we were speaking with a
client, and they own nuclear facilities, and one of the problems that they have
is that once the nuclear reactor is started, you can’t train for emergencies.
You can’t shut down the nuclear reactor to train for it. So they’re looking at
ways and scenarios of, how do we train for scenarios that we just can’t train
for in real life? And one of the people that I had on my podcast – I’m actually
gonna check my notes here – came up with a really good…it was Steve Grubbs
from Victory VR, and they’re doing a number of different education things. They
did a frog dissection in VR and stuff. And he said, you know, the case for VR
learning really hits home when it’s rare, impossible to train for, dangerous
environments, or expensive. And he came up with the acronym RIDE: Rare,
Impossible to train for, Dangerous environments, and Expensive. And it seems to
be, literally, the perfect case for that.
Alvin:
Yeah. I mean, I think those are definitely probably the most extreme
examples. But we’re working with banks right now to do kind of hostile customer
training, where somebody comes in and they’re complaining, and you know how do
you react to that? And then, in fact, you can then tie it to things like your
heart rate, and do heart rate monitors and breathing monitors. In fact, you can
tie to your EEG brainwave monitors to be able to measure the response of the
customer service agents, so that they can stay calm and be able to give the
proper response. So something like that.
Also, there could be empathy training.
Going back to the officers, they have officers who recently, in the last few
years, there’s been a lot of issues with officers and violence towards some of
the assailants. And now, if they put the officers actually on the other side –
they actually are the ones being accosted by the officers – they then, when
they come out, they can actually have more empathy for the potential assailant
and not try to be as hostile, or have a much calmer demeanor when dealing with
them. So it can be used for both dangerous and hard-to-create situations, or it
can be used to create empathy.
Alan:
Absolutely. I think that’s a really great point. Empathy training,
you know, and I think it was Chris Milk who coined the phrase, “VR is the
ultimate empathy machine,” and he’s really kind of pushed that. And kind
of a fun fact: Chris Milk was actually the first person to ever show me VR.
Alvin:
That’s something to be proud of. Well, I mean, he’s done a lot with
his various documentary films, trying to bring VR to a larger group of people.
In fact, now, I think the films that he had made, if you put those into our
6DOF volumetric player, they’d actually bring even closer to the people that he
was filming.
Alan:
Incredible. I love the fact that it’s just a player. I mean, it’s
all built in and it just works. I think it’s gonna be a huge hit for you guys.
I want to just ask you a personal question: What is the most impressive
business use case of virtual and augmented reality that you’ve seen so far? You
talked about Bell Helicopters. What is the most impressive one that you’ve seen
personally?
Alvin:
I mean, I think… I’ve seen so many, so I honestly feel like every
one I’ve seen is impressive in a different way, and people have utilized it.
You have people who have who have prevented various levels of potential damage,
from mispractice for doctors. Or you have people who have taught kids, you
know, things that would be impossible at their age level. Or you have, like
you’re saying, vehicles that are created in one fraction of the time. Just,
every use case I’ve seen comes out with a level of results that far outweighs
anything that I’ve seen with other practices. Right? So, I just feel that there
is almost limitless possibilities of utilizing this technology, to make the
work process more efficient for almost any industry. And that’s what’s most
exciting, because every time I see a customer, or I talk to a channel partner,
or I go visit an investor, I hear stories that I’m just amazed at.
Alan:
It’s incredible. You know, you mentioned more efficient but I also
think, you know, less mundane. You did a TED Talk last year, where you
described VR and how VR can unlock us from the mundane work and education
systems of today, and give us new challenges that will expand our minds and
create untold wealth and prosperity. And you asked a question at the end of
your TED Talk: how would you spend your time if money and location were not a
factor? So I sometimes wonder, what parts of our current jobs will be replaced
by AI and robotics? But more importantly, what new, amazing jobs are going to
be created that challenge us in ways that we never before imagined? To invent
new things at a pace we’ve never seen in human history? So, how do you see VR
changing the world of work as we know it today?
Alvin:
I actually think…I think about this question quite a bit, because
I do believe that within the next 15-20 years, most of the jobs that are out
there today probably won’t exist, or won’t need to exist, because machines will
do it better than we do, and do it for less money and at higher quality, in
which case, we need to find something else that we’re going to be more suited
to do.
The fact that VR can train us to
become skilled in almost anything, and to do it faster, because all the
research out there clearly shows that VR is probably the most effective means
for us to put information into our brains and to have it stay there. Both
because of the ability to use your full brain to learn – to be actively
participating in the learning – versus using just your ears or your eyes to
bring information in, but also the fact that, by being focused, you’re not
distracted by other things around you, and also the fact that it can put you
into environments that would be impossible for you to understand. Let’s say you
want to understand subatomic particles; if you could put yourself to be the
size of a subatomic particle, and see the interactions and feel the
interactions between these particles, that helps you to understand quantum
physics a lot better than if you were just being told something, you know, and
there’s a little picture on the blackboard.
Alan:
It’s interesting you say that, because I got to try a VR experience
where it was looking at carbon, and I was at the molecular level looking at a
piece of carbon in a diamond format, versus a graphene sheet. And it showed the
way it’s the same molecules, but they’re stacked a different way. I really got
the spatial understanding of what that means, and I could draw it out on paper
now that I’ve seen it. I think this is going to unlock a new era of everything
in education, and my personal mission is to inspire and educate future leaders
to think and act in a socially, economically, and environmentally-sustainable
way. And I think the current education systems do a great job of preparing
people for jobs, but those jobs are not going to be there. So I think virtual
and augmented reality will hyper-accelerate the learning path for children and
everybody to really find their purpose, and their mission and passions. And I
think that can really drive humanity forward.
Alvin:
I couldn’t agree with you more. In fact, just this morning when I posted
that new study about the concentration level in education, I had a person
respond to me and say, “oh, you know, but there’s just not enough budget
to invest in these devices; they’re just too expensive, and we don’t have the
money for it.” But you know, at the end of the day, if you look at what’s
going on in most of the developed countries in the world, the countries are
spending somewhere between 3 to 5 per cent of total GDP of the country on
education, but they’re doing a pretty poor job at it because everybody is
teaching kids how to take tests. They’re not teaching kids how to learn.
If we change the mindset to say, instead of
having bigger buildings and nicer computers and bigger whatever, nicer chairs,
if we give them this equipment that helps them to unleash the genius that’s in
every single child, I think that’s the part that is worth investing in. There’s
a quote from Warren Buffett, that the best investment you can make is in
yourself, is in educating yourself. So I think in terms of a society, that
should be our number one investment. Instead of buying a new car, it probably
makes more sense to buy a headset that can then teach you and your kids how to
be smarter in everything that you’re interested in; to learn a language in a
matter of weeks instead of years. That’s the kind of things that’s possible.
Alan:
It’s interesting that you touched on priorities and budgeting. I
don’t get involved in these political things, but the US military budget is $600
billion. That’s enough to give every person in the US, I don’t know, three or
four VR headsets? So I mean, you know, there’s that.
Alvin:
It’s not that there isn’t the money. It’s about how we prioritize
that money. And that’s something that there needs to be more education out
there to the decision-makers, to the policymakers, so that they can understand
the benefit of this. And it’s not that expensive. I mean, a device like the
VIVE Focus Plus, we’re talking about $800, and you can do everything you can do
a few years ago with a PC-based VR that was several thousand dollars. So the
costs are already coming down. And from a usability perspective it’s even
easier. It takes a few minutes to set up instead of a few hours set up. It
takes a few seconds, you put it on and you’re in VR, instead of several
minutes. So in every aspect, the devices today are superior to the ones just
three years ago.
Alan:
It’s amazing, when I started in VR in 2015, we used to carry around
a giant computer, and the Oculus DK1 I think, or DK2 – we’d carry it around,
and it would take 40 minutes to set it up, and oh my God. It was a disaster.
And that was just to be able to put it on someone’s head and say, hey, check it
out. So we’ve come a long way.
Alvin:
Yes. I think if you put on a Focus device, a Focus Plus, I think
you’ll feel so much more efficient, in terms of telling the VR story.
Alan:
Absolutely.
Alvin:
Because as you know, with VR, if you don’t get people in the
headset, they don’t fully understand.
Alan:
It’s like explaining the color red to somebody who’s blind.
Alvin:
Yes. So that’s why we have to let that process, to put this device
in front of people, so much easier. The easier it is, the more people will try
it; the more people try it, the more people will believe it and understand.
Alan:
Absolutely. So, I really want to thank you so much for taking the
time to come on the show and explain the transformative power of
virtual/augmented/mixed reality, or XR technologies, to the business community.
My final question today: How would you spend your time if money and location
were not a factor?
Alvin:
You know what? I’d probably spend a lot of the ways I would do
today, but probably more on the investment and the mentoring side of it. I
spend a lot of time right now managing my team, and getting our products out,
and doing the messaging and sales and marketing and all of that, whereas
another part of my job is to work with startups, and that’s the part that I get
the most personal fulfillment out from, is that. When I talk to these young
entrepreneurs and are able to help them clarify their stories, help them be
able to get funding, help them realize their dreams. If the other piece of my
work was less of what I did, I would have more time to focus on the more
personal aspect of these one-on-one mentorships for the various potential
portfolio companies, or just startups in general. I just really like spending
time with people who are passionate about what they’re doing, and who have a
shared dream of trying to change the world in some way.
Alan: It’s
amazing. And before we hit record today, I told you about what we’re working
on, and I think our visions and our passions are fully aligned. So, I want to
again thank you so much for taking the time to come on this podcast. This has
been amazing. Do you have any final words?
Alvin: If you
haven’t tried VR, go try it. If you have, go and take it and bring it into your
company, and really start piloting and experimenting. If you’ve already piloted
it, then start deploying, because it is probably going to be the highest ROI
investment you’ll make, at least in the next few years. So good luck out there.
View all episodes
By Alan Smithson from MetaVRse
4.5
1212 ratings
The latest generation of XR technologies introduces radical new capabilities, from multi-user tracking in massive spaces, to 6DOF standalone headsets, to the ability to track hands, eyes, and lips. HTC’s Alvin Wang Graylin discusses what this means for everything from automotive design to helping children learn about the universe.
Alvin Wang Graylin is an industry leader, evangelist and passionate driver of XR technologies, particularly virtual reality. As the China President at HTC, he leads all aspects of the company’s VR and smartphone business in the region.
Alan: Today’s guest is an industry leader, evangelist and passionate driver of XR technologies, and particular virtual reality, Mr. Alvin Wang-Graylin. Mr. Graylin is the China President at HTC, leading all aspects of the Vive/VR (VIVE.com) and the Smartphone businesses in the region. For those of you not familiar with HTC Vive, VIVE is a first-of-its-kind virtual reality platform, built and optimized for room-scale VR and true-to-life interactions. Delivering on the promise of VR with game-changing technology and best-in-class content, VIVE has created the strongest ecosystem for VR hardware and software, bringing VR to consumers, developers and enterprises alike.
He is also currently Vice-Chairman of the Industry of Virtual Reality Alliance (IVRA.com) with 300+ company members, President of the $18B Virtual Reality Venture Capital Alliance (VRVCA.com) and oversees the Vive X VR Accelerators (VIVEX.co) in Beijing, Shenzhen and Tel Aviv. Mr. Graylin was born in China and educated in the US. He received his MS in computer science from MIT and MBA from MIT’s Sloan School of Management. Mr. Graylin graduated top of his department with a BS in electrical engineering from the University of Washington, where he had specialized in VR and AI over two decades ago under the tutelage of VR pioneer Tom Furness.
Please welcome to the show Mr. Alvin Wang-Graylin.
Alvin:
Hi, how are you doing, Alan?
Alan:
I’m fantastic. There we go, we got you now. Awesome. Where are you
calling in from?
Alvin:
I’m in Beijing, China.
Alan:
Beijing right now. And what time is it? It’s gotta be in the middle
of the night, I think.
Alvin:
About 10:30 PM.
Alan:
Oh, well thank you so much for taking the time to record this with
us. I’m going to jump right into it. For the people listening, you know,
really, really exciting things happening at HTC right now, and you just held
your fourth annual VIVE Ecosystem Conference, or VEC conference in Shenzhen.
Can you maybe speak to some of the announcements and how their impact is going
to really impact business use cases of VR?
Alvin:
Yeah, I’m happy to jump right in. We just had our yearly biggest
conference of the year, and had about a thousand people come in, and about a
hundred press, and essentially all the industry folks that are in China – and
actually, quite a few folks from around Asia and even parts of the US – came.
People, from the developers, from our sales channels, our accessory partners. A
lot of investment companies, as well as Chinese carriers, governmental
organizations that are involved with high tech. Essentially, what we do every
year is gather together all of the leading players in the industry and try to
create a unified direction. And the key direction that we were trying to point
to this year is something called multi-mode VR. That’s when, you know, VR can
be used, not just for one way of connecting, but it could be connected to your
PC. It could be connected to 5G cloud VR. It can be connected to a console, or
a 360 streaming camera, etc. So it’s a very exciting time for us in the
industry, that these new types of innovation’s happening.
Alan:
I got to read some of the amazing announcements. You know, one of
the ones that I think everybody’s really excited for is the VIVE Focus Plus.
And you know, it’s for those of you listening, it’s a standalone headset that
is completely six degrees of freedom, meaning you can look left, right, up, and
down, but you can also move around. How do you think this new piece of hardware
is going to unlock the enterprise use cases of the technology?
Alvin: Absolutely. I think that was the highlight of the show, and you know, this is the first full six degrees of freedom standalone device that has both six degrees of freedom in the head, as well as the hands. And to be able to have that freedom, where you can move around in as large of a space as you want to, and be able to see, you know, and use all the existing content that’s out there – it’s going to be amazing, for both developers and for users out there. You know, for enterprise, you can essentially not worry about having the wires of the devices being connected. If you’re a design firm and you want to look at a car or an airplane, you can actually walk around freely without having any burden. And we actually showed off something that was quite exciting, is a large area multi-user tracking system that allows tracking to spaces up to 88,000 square meters, which is like 900,000 square feet space, with only four sensor units, and up to about 40 devices in that space. So it allows for shared teams to be able to look together and review and work together on a virtual space as large as four football fields. Or, it could be applied to theme parks, where you could have you know people go into a big, flat field and look up and see a giant amusement park in front of them. So that’s, you know, that was one of the other exciting parts that was announced last week as well.
Alvin:
That’s absolutely incredible. You just… you just literally blew my
mind. That’s the one thing I hadn’t read about. So, to just recap: with four
sensors, you can have up to 40 devices – meaning 40 people can be in a shared
experience – in the size of four football fields, 900,000 square feet.
Alan:
Yep that’s right.
Alvin:
So I just have to kind of… I’ve got to unpack that. So, for
enterprise use cases, you can now have a shared design experience, or
experience in general, where people can train together. This would be good for
military, I think; for police. For, you know, virtually any company that has
many people training at one time. I think the training applications are just
going to be massive with this technology.
Alvin:
Yeah. I mean, if you think about it, let’s say you have a
firefighter squad that wants to train together, and, you know, they have a
giant field that they can simulate a real fire in a building. How would they
behave? And it can give you a full 6DOF tracking, and be able to have a unified
positional system, so that everybody can see each other in physical space
that’s fully aligned to the virtual space.
Alan:
I’m speechless. That is a game changer. Speaking of game changers, one
of the other things you guys announced at VEC this year is six degrees of
freedom video. So, normal 360 video allows you to kind of look around from the
point of the camera, but what you guys have developed is a way to give, kind
of, more presence by adding that ability to move up and down, and left and
right. Can you maybe speak to that?
Alvin:
Yeah. So, you know, there’s thousands – probably tens of thousands –
of 360 videos out there, but it’s really designed for just rotational viewing.
So you’re in the middle of this photo/video bubble, and you can rotate your
head. But when you move nothing really happens; that bubble moves with you.
What happens with this 6DOF – what we call 6DOF light video – it’s not full
6DOF, because you can’t really move around in a large space, but allows you to
move one meter in any direction: up, down, side-to-side, and give you that
illusion of being able to have a movable space, and lets you create a higher
level immersion than you can’t have with these existing content. Now, the best
part of this is that you don’t need a new camera. You don’t need new processing
equipment. You don’t need to do anything. You just take your existing 360 video
files or streams, stream it through it as this new video player, turn on the
6DOF light mode, and boom – you’re able to have that movement within the video.
Now, there are, you know, can people use
light VR cameras? Which allow you to make larger space movements – a few
meters, and maybe tens of meters – in a space? But they require very expensive
camera rigs, to be able to create what this allows now. Essentially, you can
have a standard $200-$300 360 camera, and bring that 6DOF experience to anybody
who you stream to. So, I think this is going to open up a new concept called
“life streaming,” where it’s not… you know, people right now, there
are these influencers who will shoot a little video of where they are and talk
about a story. But I think in this case, you can turn anybody into a
“lifestreamer,” where they can share their life with their loved ones
or with their fans.
Alan:
You just encouraged me to break out my 360 camera again, and start
making some videos. This is incredible. So, I’m going to touch on some other
things that really kind of stood out to me. One of the things that you guys
announced was StreamLink, which is the ability to connect your devices – your
VIVE devices, your VR devices – to computers, and consoles, PlayStation, I even
saw something about XBox there. Is this for real? I can just plug in my headset
to any console now, and have a powerful VR experience?
Alvin:
Well so, it’s, what StreamLink does is it allows you to take 2D
content, whether it’s from your computer screen or from your TV set top box, or
from any of the existing consoles, because it takes any HDMI video stream,
turns it into a slightly warped, rounded virtual screen in VR. So that gives
you essentially an IMAX experience with your VR headset. So it’s more of a
monitor replacement than a full VR experience. What it also, because of the
size, it actually does almost feel like you’re there. I was playing FIFA soccer
on the XBox, but then I click it in and I feel like I’m actually on the field,
because I can’t see the edges of that screen. If you get it closer to the
screen, you actually, you know, there’s a spatial element of that screen. So
once you get closer, you can actually have the screen go around you, so you
can’t see, if you back up from it, you can see the screen. And you can actually
size the screen as well. You can also change the shape to be flat or to be
slightly rounded. And one of the really interesting parts is that it also has a
video see-through a window. If you look down, you can actually see your
keyboard, or you could see your hand game controller, and that way you’re able
to manipulate physical objects, even when you’re in this headset as a monitor
replacement.
Alan:
So that leads me to the next thing that I saw. First of all, that’s
incredible. The visual that I’m getting is of, you know, a trader or a
financial trader being able to put up their screen in a massive screen, so your
computer now is a wrap-around IMAX screen. Imagine the amount of work you could
get done by having an IMAX-sized screen while you’re doing your work. That’s
just incredible.
Alvin:
Yes. I think it’s both good for work and for entertainment, that’s the
best part. In fact, our guys are trying to create a little program to get
people to spend a whole day, like 24 hours, inside VR. And just to use this one
single device, because it’s able to connect to all of the current media sources
that we have anyways. And you know, the Focus Plus also allows you to connect
wirelessly to your phone and wirelessly to your PC, to experience PC VR. So all
the 4000+ pieces of PC VR content can be streamed directly to this standalone
device. So it’s kind of like The Lord of the Rings, this is like the Lord of
HMV. You know, they’re the HMV that connects to everything.
Alan:
I can tell you, this is what I personally have been waiting for. I
want to be able to open my – I know it sounds boring to everybody – but I want
to be able to open my emails and work in VR on a giant screen and have, you
know, my podcast link over here, and my e-mails over here, and all these things
kind of spread out. I am really excited for this. And you know, one of the
things you mentioned was looking down and seeing your hands. Something else you
guys addressed at VEC this year was native hand-and-finger tracking.
Alvin:
Yes.
Alan:
This is going to unlock so much.
Alvin:
Yeah, we definitely have a lot of news packed in to–
Alan:
I’ve got a long list here! I’m gonna keep going. This is amazing,
you guys announced so many things.
Alvin:
Yeah, I think we announced too many things, because most people only
heard probably maybe a third of it, or only comprehended a third of what we
what we announced. Yes, so at the VEC – and at GDC – we actually announced our
USDKs for hand tracking. About a year ago actually, at the last VEC, I had
announced that we were going to bring gesture control to VIVE, using the
existing cameras that are on both the VIVE and the VIVE Focus. So at the GDC
this year, we demonstrated the hand-and-finger tracking on the VIVE Pro. But at
VEC, we also demonstrated the finger tracking on the VIVE Focus. So, on a
standalone device, to be able to have 21-point finger tracking, that’s amazing.
We actually didn’t think it was possible, because of processing limitations of
the mobile chipsets. But after almost a year of optimizing, we got it all to
work, and then we also released publicly the SD case for the VIVE and the VIVE
Pro, and the gestures on the VIVE Focus, so that any developer, now, can essentially
incorporate natural hand movement directly into the app without having to buy a
third-party piece of accessory like the Leap Motion.
Alan:
So I keep coming back to training, but let’s really kind of take a
second to unpack the native hand-tracking and finger-tracking; if you’re able
to reach out, see your hands, and interact with things in virtual spaces, this
is going to literally unlock unlimited potential for training. So, if you want
to teach somebody how to do something in real life, they can reach out, grab
it, learn it. And you know, one of the things that you posted recently was on
education and how VR is improving students’ concentration. I think both
students in grade school from K to 12, but also university, but I think right
through to enterprise training scenarios, I think virtual reality has a
potential to really lead the way with how we educate in the future, and how we
train. It’s kind of my personal passion as well, but can you speak to what you
posted the other day about the VR improving students concentration? By a
considerable amount.
Alvin:
This is a brand new study – it has not actually been published yet.
It will be published in about two weeks. And there was a study that we helped,
I guess, fund, but then it was completely done independently from us. We had no
influence over the study processes. But what they did was essentially brought
two different groups of kids that were 12-to-13-year-old kids, and taught them
basic physics, mechanics. And so, two kids had the same courses, the same
questions, the same tests, and one supplemented the experience with
VR-supplemented material, to allow the kids to be able to interact with the
principles that was discussed. And the other one was just using more blackboard
and teacher discussions. So, your normal classroom-type methodology.
What we found was that the concentration
went… and for both sets of students, we, for both at the students we used EEG
sensors on them during the entire class, and was able to capture the movement
and the intensity of the frontal lobe, which was to to identify the
concentration level and focus of the students. And it was very, very clear that
the students that were in the VR session, they had, I think it was a 6x
improvement, in terms of concentration level, which led to grades that were
significantly higher. Essentially, the control group had scores that was about
a C+, right after they did the course, in terms of passing the tests, whereas
the VR group right after was an A score, in terms of their average score. And a
week later, it was still in mid-B range for the VR students.
So even a week later, after not being
reminded the content after they were retested, they outperformed the immediate
test scores of the control group. And a week later, on the control group, was
down to, I think it was a D+. So either way, what it shows is that for complex
topics, this type of a training mechanism really both helps to increase
concentration and retention for the content, and you know, very, very
statistically significant.
Alan:
It’s incredible, and you know, we’re seeing already some major
shifts in how enterprises are using this, and I just want to touch, before we
move on to that, the one other stat that I saw here from one of your
presentations, was VR training is useful also in sports education. You guys did
a study showing improvement using soccer teams, children’s soccer teams, and
using VR training. One of the things that you showed was a 36 per cent increase
in performance after using VR. Can you speak to that?
Alvin:
So actually, before going into that, I want to add one more point to
the last study on the concentration level, because one piece of data that’s not
actually in the slide I published was that there was a distinctive difference,
actually, in the concentration improvements for male versus female. It seemed
like the more distractable boys actually had even bigger improvements than the
girls, who are normally very able to concentrate in class, whereas the boys
usually tend to be harder to manage and are easily distractable. And what we
found is that they actually had a bigger improvement because of VR. So the more
distractable that the age group or the child, the more benefit they actually
will get from this from this technology. So that is a very surprising finding.
But I guess it’s intuitive after the fact, looking at it.
Alan:
It’s interesting you say that, because I’d been reading a lot of
articles around how virtual reality is being used for autism, and to really
train people with different levels …on the spectrum, to interact with other
students, and kind of studying their eye contact and that sort of thing. And
one of the things that you guys announced at VEC – I keep going back to this –
but, you know eye tracking and lip tracking.
Alvin:
Yes. Yes.
Alan:
This is incredible. So, you know, I’m thinking for HR, if you wanted
to do an interview with somebody, you could send them a headset, you could do a
full interview in VR with them and really get a sense of, are they looking at
you? Do they make eye contact? Now that you can see their hands and fingers, do
they use their hands and gestures? And lip tracking allows you to kind of match
what they’re saying to a visual, maybe–
Alvin:
It can probably go further than that, because micro expressions in
your body really gives a sign of whether or not you’re telling the truth, or
whether you’re fibbing, or whether you’re kind of making up things, and what part
of your brain you’re looking at, that you’re trying to withdraw information
from. Well actually, your eyes point to different places. So, by looking at eye
tracking, you can actually use it as a lie detector, and probably in a more
accurate way than the current polygraphs that are out there. So yeah, that’s
also a new technology that we are putting out.
Even though we just announced recently our
VIVE Pro Eye, which has built-in eye-tracking, the SDK that we are releasing
will actually work with any existing eye-tracking system. So in fact, very soon
we’ll be coming out with third party accessories that can add eye-tracking onto
our existing VIVEs, VIVE Pros, and also the VIVE Focus.
Alan:
That’s incredible. I actually… it’s interesting, the whole idea of
the lie detector. I just read a piece by Jeremy Bailenson – and for those of
you who don’t know he’s a researcher at Stanford – and he was on the privacy of
people while collecting physical data. So, things like eye tracking, now lip
tracking, but also their movements. You know, their posture, how they move, and
that sort of thing. And the study or the piece really talked about, by
collecting this data, we’re able to now, down to a very granular level, one
that kind of transcends all of the other studies we’ve ever had before into
collecting data about people that is way more accurate than just asking them on
a survey or that we have…this is next-level data collection. And I think,
while it opens a Pandora’s box for ethical concerns, it also, you know, can
serve businesses very, very dramatically on both their HR needs, their training
needs. There’s so many different aspects of this.
Alvin:
Yeah yeah absolutely. And let’s say if you’re an advertising agency,
and you want to know if your ad works – what are people really looking at?
What’s attracting their attention? You know exactly by looking at, by using eye
tracking and putting them in front of that ad, either in the virtual space or a
physical space. If you’re designing a cockpit for a car or an airplane and you
want to make sure that it’s easy for people to understand, you put them into
that space and you know how long did it take them to find the right buttons.
Where are they spending their time looking when they’re when you’re driving
this car, when they’re using this car? So those kinds of information, it’s very
difficult to get in the real world, or you had to spend a lot of money to to
actually build out versions of that physical cockpit, and then have people try
it and then give you a feeling of what which one felt better. Right? With this,
you know exactly how long it would take them to find that button. Did they look
at the speedometer numbers in the right way, or do they prefer to look up or
down, et cetera. That kind of information can give you so much value in terms
of designing something that’s more human-centric.
Alan:
Well, since you brought it up, I’m going to talk about the
automobile industry and logistics and travel, automotive in general. So you’ve
done work, HTC has done work with BMW, Volkswagen, McLaren, probably a number
of other companies in the periphery as well. Can you speak to some of the ways
that automotive companies are using this? I know Volkswagen’s using it for
collaborative training, for factory logistics. BMW is doing virtual prototypes.
McLaren allows you to race one of their McLarens. What’s going on? Why are
automotive companies jumping on so much, and what are they doing with it? Where
are they seeing the most value?
Alvin:
I think it actually can be used in essentially all steps of their
creation/design/testing/training/manufacturing process. Normally, whether
you’re talking about a car or a plane, you’re talking on the order of years, or
a number of years, to design and build a new product. And what we’re finding is
that, in the design industry, we can get a 2x, up to maybe a 10x improvement in
terms of how long it takes to go from concept to product. Because a lot of
what’s happening today, in terms of creating these things, is that they have to
create physical prototypes that are made of clay, have people come from around
the world to review them, make changes, come back a few weeks later, etc.,
which just doesn’t produce very fast turnaround. In VR, you can do all of that
virtually. And in fact, we had a very interesting case study just released a
few months ago – two to three months ago – where Bell Helicopters, normally it
takes five to seven years to build and design a new helicopter. It took them
six months to go from concept to a working prototype that was flying, and that
that’s just crazy fast. I think that’s faster than that would take us to build
a new headset. So it’s amazing how much value they got from using that. I don’t
think that’s typical. I would I would say typically, probably an improvement of
2-to-3x in terms of design time is probably typical.
Alan:
So… let’s just kind of stop for one second. So Bell Helicopter
designed a new helicopter in six months that would have taken them five years.
Alvin:
Yes, exactly.
Alan:
And that’s an outlier. OK. We’ll take that as an outlier. But
typically, average what you’re seeing is a 2x to 3x expedited process of
design.
Alvin:
Yeah. And that’s not just for vehicles. It can be used for
buildings, for interior design; essentially anything that requires review and
design and collaboration between teams to create. It can be dramatically
faster, improved.
Alan:
So that leads me to the next question. Why isn’t every company doing
this already? If you’re seeing two to three times improvements, why is this not
in every single design office in the world?
Alvin:
Actually, I would say we’re working with almost every major auto
manufacturer and airplane manufacturer out there. In fact, if you look at
Boeing and Airbus, they were working 5-10 years ago with CAVE systems, and CAVE
systems essentially are an early version of VR, where you’re in a room and it’s
projecting the expected outcome around you in these screens that that feels
like you’re in that environment. It’s not as individualized as you would get
with current headsets, but for the technology of the time, it looked and felt
very much like today’s VR. And those were million-dollar systems, but they’ve
been using them for 10, 15 years.
Alan:
Absolutely incredible. So it’s really interesting, I mean, you
studied under kind of the godfather of VR, Tom Furness, and you were looking at
how VR can disrupt education.
Alvin:
Yes.
Alan:
You shared some stats, around six times improvement of
concentration, and one of the other stats was a 36 per cent improvement in
sports education, performance, and training.
Alvin:
Yes, actually, I think I forgot to mention the sports education when
I said – let me let me go back to that case that you asked for. So we had
worked with the national youth team that was managed by the Chinese Ministry of
Sports, and they’re essentially the elite, kind of next generation soccer
athletes in China, and they were brought in for a one month training camp. So,
they had four teams, and two teams were using VR, two teams were not using VR,
and they had the same the same kind of national-level coaches helping all four
of these teams, except some were supplemented with VR strategy education.
What they found is, at the end of the
month, the group that was using VR improved by 30 per cent in terms of their
strategy scores. In the elite athlete level, it’s not the physical capability
of these kids or adult athletes that separates them; it’s all about their
mental acuity, in terms of understanding the deeper level of the strategy of
the game. And so that’s something that’s actually very difficult for people to
train, and you have to create varied scenarios, and it’s hard to simulate those
scenarios. But when you can put these kids into thousands of potential
scenarios, if these defenders are here and the ball is here and the goalie was
there, what should you do? Who should you pass to and how should you kick this,
or how should you defend? And by putting them in those scenarios, and then
being able to score them and be able to measure them, they were able to
accelerate significantly something that would take years to train. And they
essentially got that level of improvement in the month, whereas the average
non-VR student or athlete only improved by about 5 per cent in terms of their
post-month training scores. So it’s, again, like 6, 7, 8x difference in terms
of improvement.
Alan:
That would probably explain why Walmart has decided to roll out VR
across their entire company to train people.
Alvin:
Yes. Yes. I mean, the larger your workforce, the more important
training is to your teams. And it can be used for sales training, like what
Wal-Mart’s doing. But also at Volkswagen, they are using it for assembly line
training, so people don’t make any mistakes when they… or, they make the
mistakes in virtual reality first, so they don’t make it on the real car. And
you have people like Send For Help, who are using it to train nurses so that
they don’t harm the patients. They make all their mistakes and VR, and their
teacher can be alongside them to watch them and see what they’re doing in the
virtual world. Again, training can be applied to essentially every industry.
Alan:
It’s interesting, you know, one of the companies that we’re working
with is called Career VR, and what they’ve done is created simulators to show
youth and people looking to do different jobs and explore different career
opportunities. One of them is you know a crane operator, and one of them is a
forklift operator, one of those is a heavy machinery operator. Plumbing, and
welding. So by creating these virtual experiences, it lets people really get an
understanding of what is this job all about. Do I like it? Is it something I
want to go into? And then from there they can train in it. I know you guys did
some work with Raymond Forklift. Can you maybe speak to that and what they’re
using it for?
Alvin:
Yes. So, you know, these forklifts are used in warehouses to move
very heavy equipment and materials, and if you make a mistake, it could be
thousands of dollars, or tens of thousands of dollars of damage. Right? So what
they did was they essentially created a cockpit that was completed at the same
size, shape, and all the buttons in the right places. You put on this headset
and you can simulate yourself driving this forklift, and they can test and be
able to measure all the things that you’re doing to make sure that you’re doing
it safely and you’re following all the procedures. And then you’re also getting
your muscle memory, because you’re touching all the right buttons, and those
buttons and knobs and levers link exactly to what would happen in the physical
world in the headset space that you’re in. So it is kind of a 1:1 physical,
kind of like a simulator training for airplane pilots; they’re doing it for
forklifts.
Alan:
It’s incredible. I actually got to try, with the Career VR guys, I
got to try an excavator and, I mean, I’ve never been in one of these machines
before, I had no idea. But step by step, they walk me through starting the
machine, turning the pivot, driving it, moving the dump or the bucket. And by
the end of it, I was like, okay, you know, I’m pretty sure I could get into any
of these excavators and work it. And a friend of mine at VR Scout, he did a
crane trainer, where you drive a crane and you go through this thing, and in
one hour and crane training in VR. Then they took him outside put him in a real
crane and said, “here, drive it.” And he was able to drive it. And
that’s wow. I mean, incredible!
Alvin:
Yeah. I mean, those cranes are pretty expensive; you don’t want them
making their mistakes on those things. So that’s, you know, just to spend one
hour or two to prevent a potential damage to the device or the property around
it? That is well worth it. And I’ll give you another example of a very
different type of training, is police training. We actually worked with some of
the Chinese police force, where they’re trying to get their officers to be to
react properly to very high-stress environments. And these high stress
environments are very hard to simulate in the real world. You’re not going to
have bad guys and good guys all in the same place, and be able to use a gun to
shoot at them. But to be able to distinguish, to be a know how to react, to be
able to stay calm, and be able to measure and see it from a trainer’s perspective,
what the natural reaction of these different officers are. That’s something
that they’ve told us is immensely valuable.
Alan:
Yeah, I think police training is going to be huge, and also
emergency services. One of the other ones that… we were speaking with a
client, and they own nuclear facilities, and one of the problems that they have
is that once the nuclear reactor is started, you can’t train for emergencies.
You can’t shut down the nuclear reactor to train for it. So they’re looking at
ways and scenarios of, how do we train for scenarios that we just can’t train
for in real life? And one of the people that I had on my podcast – I’m actually
gonna check my notes here – came up with a really good…it was Steve Grubbs
from Victory VR, and they’re doing a number of different education things. They
did a frog dissection in VR and stuff. And he said, you know, the case for VR
learning really hits home when it’s rare, impossible to train for, dangerous
environments, or expensive. And he came up with the acronym RIDE: Rare,
Impossible to train for, Dangerous environments, and Expensive. And it seems to
be, literally, the perfect case for that.
Alvin:
Yeah. I mean, I think those are definitely probably the most extreme
examples. But we’re working with banks right now to do kind of hostile customer
training, where somebody comes in and they’re complaining, and you know how do
you react to that? And then, in fact, you can then tie it to things like your
heart rate, and do heart rate monitors and breathing monitors. In fact, you can
tie to your EEG brainwave monitors to be able to measure the response of the
customer service agents, so that they can stay calm and be able to give the
proper response. So something like that.
Also, there could be empathy training.
Going back to the officers, they have officers who recently, in the last few
years, there’s been a lot of issues with officers and violence towards some of
the assailants. And now, if they put the officers actually on the other side –
they actually are the ones being accosted by the officers – they then, when
they come out, they can actually have more empathy for the potential assailant
and not try to be as hostile, or have a much calmer demeanor when dealing with
them. So it can be used for both dangerous and hard-to-create situations, or it
can be used to create empathy.
Alan:
Absolutely. I think that’s a really great point. Empathy training,
you know, and I think it was Chris Milk who coined the phrase, “VR is the
ultimate empathy machine,” and he’s really kind of pushed that. And kind
of a fun fact: Chris Milk was actually the first person to ever show me VR.
Alvin:
That’s something to be proud of. Well, I mean, he’s done a lot with
his various documentary films, trying to bring VR to a larger group of people.
In fact, now, I think the films that he had made, if you put those into our
6DOF volumetric player, they’d actually bring even closer to the people that he
was filming.
Alan:
Incredible. I love the fact that it’s just a player. I mean, it’s
all built in and it just works. I think it’s gonna be a huge hit for you guys.
I want to just ask you a personal question: What is the most impressive
business use case of virtual and augmented reality that you’ve seen so far? You
talked about Bell Helicopters. What is the most impressive one that you’ve seen
personally?
Alvin:
I mean, I think… I’ve seen so many, so I honestly feel like every
one I’ve seen is impressive in a different way, and people have utilized it.
You have people who have who have prevented various levels of potential damage,
from mispractice for doctors. Or you have people who have taught kids, you
know, things that would be impossible at their age level. Or you have, like
you’re saying, vehicles that are created in one fraction of the time. Just,
every use case I’ve seen comes out with a level of results that far outweighs
anything that I’ve seen with other practices. Right? So, I just feel that there
is almost limitless possibilities of utilizing this technology, to make the
work process more efficient for almost any industry. And that’s what’s most
exciting, because every time I see a customer, or I talk to a channel partner,
or I go visit an investor, I hear stories that I’m just amazed at.
Alan:
It’s incredible. You know, you mentioned more efficient but I also
think, you know, less mundane. You did a TED Talk last year, where you
described VR and how VR can unlock us from the mundane work and education
systems of today, and give us new challenges that will expand our minds and
create untold wealth and prosperity. And you asked a question at the end of
your TED Talk: how would you spend your time if money and location were not a
factor? So I sometimes wonder, what parts of our current jobs will be replaced
by AI and robotics? But more importantly, what new, amazing jobs are going to
be created that challenge us in ways that we never before imagined? To invent
new things at a pace we’ve never seen in human history? So, how do you see VR
changing the world of work as we know it today?
Alvin:
I actually think…I think about this question quite a bit, because
I do believe that within the next 15-20 years, most of the jobs that are out
there today probably won’t exist, or won’t need to exist, because machines will
do it better than we do, and do it for less money and at higher quality, in
which case, we need to find something else that we’re going to be more suited
to do.
The fact that VR can train us to
become skilled in almost anything, and to do it faster, because all the
research out there clearly shows that VR is probably the most effective means
for us to put information into our brains and to have it stay there. Both
because of the ability to use your full brain to learn – to be actively
participating in the learning – versus using just your ears or your eyes to
bring information in, but also the fact that, by being focused, you’re not
distracted by other things around you, and also the fact that it can put you
into environments that would be impossible for you to understand. Let’s say you
want to understand subatomic particles; if you could put yourself to be the
size of a subatomic particle, and see the interactions and feel the
interactions between these particles, that helps you to understand quantum
physics a lot better than if you were just being told something, you know, and
there’s a little picture on the blackboard.
Alan:
It’s interesting you say that, because I got to try a VR experience
where it was looking at carbon, and I was at the molecular level looking at a
piece of carbon in a diamond format, versus a graphene sheet. And it showed the
way it’s the same molecules, but they’re stacked a different way. I really got
the spatial understanding of what that means, and I could draw it out on paper
now that I’ve seen it. I think this is going to unlock a new era of everything
in education, and my personal mission is to inspire and educate future leaders
to think and act in a socially, economically, and environmentally-sustainable
way. And I think the current education systems do a great job of preparing
people for jobs, but those jobs are not going to be there. So I think virtual
and augmented reality will hyper-accelerate the learning path for children and
everybody to really find their purpose, and their mission and passions. And I
think that can really drive humanity forward.
Alvin:
I couldn’t agree with you more. In fact, just this morning when I posted
that new study about the concentration level in education, I had a person
respond to me and say, “oh, you know, but there’s just not enough budget
to invest in these devices; they’re just too expensive, and we don’t have the
money for it.” But you know, at the end of the day, if you look at what’s
going on in most of the developed countries in the world, the countries are
spending somewhere between 3 to 5 per cent of total GDP of the country on
education, but they’re doing a pretty poor job at it because everybody is
teaching kids how to take tests. They’re not teaching kids how to learn.
If we change the mindset to say, instead of
having bigger buildings and nicer computers and bigger whatever, nicer chairs,
if we give them this equipment that helps them to unleash the genius that’s in
every single child, I think that’s the part that is worth investing in. There’s
a quote from Warren Buffett, that the best investment you can make is in
yourself, is in educating yourself. So I think in terms of a society, that
should be our number one investment. Instead of buying a new car, it probably
makes more sense to buy a headset that can then teach you and your kids how to
be smarter in everything that you’re interested in; to learn a language in a
matter of weeks instead of years. That’s the kind of things that’s possible.
Alan:
It’s interesting that you touched on priorities and budgeting. I
don’t get involved in these political things, but the US military budget is $600
billion. That’s enough to give every person in the US, I don’t know, three or
four VR headsets? So I mean, you know, there’s that.
Alvin:
It’s not that there isn’t the money. It’s about how we prioritize
that money. And that’s something that there needs to be more education out
there to the decision-makers, to the policymakers, so that they can understand
the benefit of this. And it’s not that expensive. I mean, a device like the
VIVE Focus Plus, we’re talking about $800, and you can do everything you can do
a few years ago with a PC-based VR that was several thousand dollars. So the
costs are already coming down. And from a usability perspective it’s even
easier. It takes a few minutes to set up instead of a few hours set up. It
takes a few seconds, you put it on and you’re in VR, instead of several
minutes. So in every aspect, the devices today are superior to the ones just
three years ago.
Alan:
It’s amazing, when I started in VR in 2015, we used to carry around
a giant computer, and the Oculus DK1 I think, or DK2 – we’d carry it around,
and it would take 40 minutes to set it up, and oh my God. It was a disaster.
And that was just to be able to put it on someone’s head and say, hey, check it
out. So we’ve come a long way.
Alvin:
Yes. I think if you put on a Focus device, a Focus Plus, I think
you’ll feel so much more efficient, in terms of telling the VR story.
Alan:
Absolutely.
Alvin:
Because as you know, with VR, if you don’t get people in the
headset, they don’t fully understand.
Alan:
It’s like explaining the color red to somebody who’s blind.
Alvin:
Yes. So that’s why we have to let that process, to put this device
in front of people, so much easier. The easier it is, the more people will try
it; the more people try it, the more people will believe it and understand.
Alan:
Absolutely. So, I really want to thank you so much for taking the
time to come on the show and explain the transformative power of
virtual/augmented/mixed reality, or XR technologies, to the business community.
My final question today: How would you spend your time if money and location
were not a factor?
Alvin:
You know what? I’d probably spend a lot of the ways I would do
today, but probably more on the investment and the mentoring side of it. I
spend a lot of time right now managing my team, and getting our products out,
and doing the messaging and sales and marketing and all of that, whereas
another part of my job is to work with startups, and that’s the part that I get
the most personal fulfillment out from, is that. When I talk to these young
entrepreneurs and are able to help them clarify their stories, help them be
able to get funding, help them realize their dreams. If the other piece of my
work was less of what I did, I would have more time to focus on the more
personal aspect of these one-on-one mentorships for the various potential
portfolio companies, or just startups in general. I just really like spending
time with people who are passionate about what they’re doing, and who have a
shared dream of trying to change the world in some way.
Alan: It’s
amazing. And before we hit record today, I told you about what we’re working
on, and I think our visions and our passions are fully aligned. So, I want to
again thank you so much for taking the time to come on this podcast. This has
been amazing. Do you have any final words?
Alvin: If you
haven’t tried VR, go try it. If you have, go and take it and bring it into your
company, and really start piloting and experimenting. If you’ve already piloted
it, then start deploying, because it is probably going to be the highest ROI
investment you’ll make, at least in the next few years. So good luck out there.