XR for Business

By Alan Smithson from MetaVRseBusiness Technology Arts

XR for Business episodes

January 20, 2020 The IKEA of AR: Making Content Effortlessly, with EON Reality’s Dan Lejerskar
It’s been said on this show before; XR doesn’t have a technology problem, it has an adoption problem. In Dan Lejerskar’s experience, everyone from universities to governments see the value of XR — they just lack the content to make it a worthwhile, everyday tool. He and Alan explore how EON Reality is addressing this discrepancy.
Alan: Hi, it’s Alan Smithson
here. Today we’re speaking with Dan Lejeskar, founder and chairman of
EON Reality, a world leader in
virtual/augmented reality based knowledge transfer for industry and
education. They believe that knowledge is a human right and it’s
their goal to make knowledge available, affordable, and accessible
for every human on the planet. We’re going to find out how, in the
next XR for Business Podcast. Dan, welcome to the show, my friend.
Dan: Thank you so much.
Alan: I’m really, really
excited. I know you guys have been working– well, you specifically
have been working in the 3D virtual space for many years now. How did
you get involved in VR and learning?
Dan: In my past, I used to work
with simulators — big aircraft simulators, etc. — and I got really
excited about seeing the effect it has on pilots and soldiers, and I
always thought that it would be useful to do the same, but for normal
people, nurses, etc. But obviously these people couldn’t afford a
$50-million simulator. So I had to be patient and wait until the
computers follow Moore’s Law; become cheaper, faster, better. And by
’99, the hardware was there, so you can start running this on PCs. So
we were very early adopters of virtual reality already in that
period.
Alan: We’re talking 20 years.
Most people know VR and AR as if kind of something in the last five
years. But what was it like kind of going through these growing pains
of going from a million dollar simulator — millions of dollars
simulator — to now we can buy an Oculus Quest for 500 bucks?
Dan: It’s been an interesting
journey, with a lot of ups and downs. And very much VR has been like
AI. I’m sure you’ve read about the “AI Winter”, when things
didn’t go that well. We’ve had quite a few ups and downs in virtual
reality. ’99 was fantastic, because that was the era of dot-coms. And
we started with something called Web3D, so you can do 3D on the web.
It had actually millions of users. Then we had a hard landing 2001.
Remember when dot-com crashed? And we had to move our business from
industry and education to defence because we had September 11th. So
that was kind of what saved our business, doing homeland security
centers and the like. And then slowly and surely, we picked up the
business up to 2007, 2008. And during this period, there were several
iterations. There was something called people avatars and virtual
worlds, that was very popular around 2007. That raised and crashed
also, pretty tough. But we managed to navigate those water until I
would say 2011, 2012, when the hardware became available for mobile
devices. So this was before Oculus. Already then we could see where
the industry was going.
Alan: Oh, you guys, you never
lost your path. You’ve veered a little bit from military, to industry
and education, back to military, and then back to industry and
education. Obviously, the passion is in the industry, knowledge
transfer and education. What are some of the projects that you guys
have done in the last few years that really just made you go, “Wow,
this really is something that, quote unquote, normal people can use?”
Dan: So, you’re right. We
realized quickly that the biggest value has to do with knowledge
transfer. And we started thinking how can this technology be used to
solve big problems? And we identified three areas. One is government.
We have an initiative that I’m happy to talk about. We call Human 2.0
or Enhanced Humans. And we roll this out with more than 50
governments around the globe, national governments. Then we have also
the initiative for academia, which we call Classroom 3.0. It is
transforming education by experiential learning. And then we have a
third group we call Industry 4.0, or our enterprise solution. And in
each of these sectors, we have concrete projects. So in the
government sector we do something called interactive digital centers.
Those are pretty big investment from governments, can go up to 6-7
million dollars per location. And I’m just about to inaugurate the
new one now in Morocco — of all places — that we work with USAD.
And what we’re focusing on this bigger
project is to address the gap of skilled, smart workers. As you know,
technology is a big killer of jobs, but also big creator of jobs. And
in this creation process, we have to teach new people new skills. And
virtual reality/augmented reality proves to be very efficient there.
That’s one type of projects. With education, we do the same. We work
with big universities. We work with regions. And again, that is to
help students learn faster, retain information longer, and make
better decisions. And industry, we work with two areas. We work both
with the area of productivity increase, predominantly using augmented
reality, but also with learned trade; training using virtual reality.
So that’s very broadly what we do. I can go into the specifics of
each.
Alan: Why don’t we break that
down? And so we’ve got kind of these large government centers. You
mentioned they cost between six and seven million dollars. What is in
that? What are they getting for that? What is the value created?
Dan: So concretely, there are
four elements in a center. The biggest one is a dissemination machine
to up to 7-10,000 users. We have a platform we call the
Augmented/Virtual Reality Platform that essentially does both, it’s a
mix of virtual reality, augmented reality, and mobile based
solutions. So it works agnostically on everything from an iPhone to
very sophisticated headset. So that, the first part of the center is
to essentially do a regional deployment. I’ll give you an example.
We’ve just done one in Bologna in Italy. So that deployment is done
both with a university — the University of Bologna, with 6,000
students — and it’s done also with industry, the Ferrari, Maserati,
Ducati, all this industry that they have there. So that’s the first
part.
The second part is we have a showroom,
and the showroom displays all type of technology, from large screen
environments, room size environments [which] we call Icube, to
headsets. So people actually can not only be aware of this
technology, but understand what it does.
The third element of this is an IP and
content creation. So we have something we call the VR Innovation
Academy, and we train local resources how to develop and deploy
virtual reality and generate local IP. And the fourth element is a
marketplace. We have something called the Vault and the Vault
contains lots of assets, that we accumulated over 20 years, developed
by ourselves or partners. Think about it as a Netflix, but containing
virtual reality objects, objects and application focused knowledge
transfer. So those are the four elements of a center.
Alan: When you’re building these
centres, I guess you’ve mentioned the universities in an industry. I
assume government is involved in that. What are the benefits, then,
to the university? Is it more to [bring] their students up to speed?
Because you mentioned also the knowledge transfer or content creation
using local resources. Is this something that the universities are
using to bolster their employment statistics or…?
Dan: Yes. So if you look at
academia in general, there are three big problems today they have.
One is a quantitative problem. They need to teach more with less;
less time, less money. They have a second problem, which is a
qualitative problem. The skills that they used to teach — which is
remembering and learning, understanding — they are not any longer
applicable. You have to now do analyze, evaluate, create, so it’s a
different type of skill set. And the third problem is what I call
“the sage on the stage.” Since 1400, we used to have a guy
standing in front — typically guy — stand in front of a lecture
hall and preaching. And that doesn’t work any longer, because today’s
students are social learners, multi-taskers, short attention span. So
you have to universally start realize that they have to do something
different, and that something different is what I call experiential
learning.
So rather than just teaching you about
logistics, we give you a Heineken brewery and you learn logistics
within that context by solving a problem. Now, that is fantastic, but
it’s pretty expensive to get your hand on a nuclear power plant or a
brewery. And that’s where virtual reality really comes to play. So
Eon has the world’s largest library. We have 870,000 assets. So if
you say, let’s say particle accelerator: sir, we have it. So what we
did is we went to vacuum clean all 3D warehouses and created these
building blocks. And then we developed the platform because today–
this is a big problem. I don’t know if you realize, 82 percent of
universities have tried VR and AR. 82 percent, in some fashion. But
when a majority of them — I would say 90 percent — do not pursue
it. Why? Why do you think that is?
Alan: They don’t see the value
in it. For one, my second guess would be that it’s too expensive for
two. The third one, in my opinion, would be the fact that they just
don’t know what to do.
Dan: You’re pretty much right.
They see the value. The value, most people understand the value.
Expense is a problem, because if you want to roll this out to 8,000,
you’ve got to buy Oculus for everybody, and even that’s too
expensive. But the biggest problem is actually they don’t have the
content. It takes time to develop content, and it takes skills to
develop content. So I would say that’s the biggest one. I just was
recently with a university in Mexico. They spent $4-million on
equipment, but they don’t use it, because they had only two or three
curriculum that they can use, and it took them two years to develop.
So fundamentally, we address these problems with our platform. Number
one, our platform works on any device. So if you have your iPhone or
the 4 billion of smart devices, so you don’t have to have a fancy
headset. It works with a headset also. So that’s how we eliminate the
cost aspect.
Then the other aspect, make it easy.
How do we do that? If you’re familiar with IKEA, right? IKEA, you buy
a furniture, build it yourself. So our mantra is easy. It stands for
effortless. So I can teach you to develop an application in less than
one hour. And the skillset you have is PowerPoint or less. So you
don’t know anything about coding, Unity, Unreal. So that’s the first
one. The second one is affordable. So for $12 a student a month,
you’ll get access to the full platform. The third one is
self-service. So that’s the part of do it yourself. And the third–
fourth one is interconnected. So you can interconnect. I kind of– it
was a segue from your question about academia. But academia wants to
do it, most of them. And they want to do it, but they don’t have the
ways. We help them with a platform to integrate and deploy it, not to
30 guys in a lab, but to thousands and thousands of students.
Affordable.
Alan: So how do you then get
people to learn how to do Unreal and Unity without teaching them? How
does that work?
Dan: Let’s dissect the problem
with content creation and virtual reality. So there’s three elements
to this, if you think about it. There’s one, you need to get data in
somehow, right? So you need to get 3D models created. Then the second
thing is, you need to bring that to life. So let’s say you bring in a
piece of machinery. When you push a button, something needs to
happen. And further, you need to publish this on something, whether
it’s a headset. And then, of course, you need to manage. So the
traditional way to do it is to use Unreal or Unity. And that’s what
we used to do. And that takes a lot of time. So after about 10 years
of doing that, we said, “Wait a second, 80 percent of the time
we do this repetitive thing, that we can make it easier.” So we
don’t have to go in scripting and coding, because 99 percent of our
customers don’t have a clue how to do that.
So what did we do? First of all, we’re
getting data in. We teamed up, so you don’t have to create models.
You can go into a library and search about 870,000. And if it happens
that your specific model that you want is not there, you can also
import under 20 different formats. So CAD, Autodesk, Siemens. We
teamed up with SAP, so you can do that. And it happens behind the
scenes. So you just drag a model in, and it pops up in the
application. So that solves the first problem. Second problem is
bring that model to life. How do you do that? So we created a
non-code interface that allows you to do a number of things, easy. As
easy as PowerPoint. If you want I can go more into detail on that.
And the last one is publishing seamlessly to anything. So that’s the
platform. And we have a lot of lectures in our website how to do it.
And we are coming up with a freemium model actually later this month.
So you will have a free version also of the platform that you can
play.
Alan: So basically what you’ve
done is you’ve templatized the creation of virtual and augmented
training.
Dan: That’s it, yep. You’ve got
it.
Alan: All right. So is there
anything you want to touch base on the showroom?
Dan: Not so much, to be honest.
The showrooms value is diminished, because if you go back 5, 10
years, there were no major producer of hardware, for virtual reality.
So we kind of have to integrate our own systems. We have something
like Icube and obviously it’s like a cave, basically you walk in and
you’re surrounded by walls and you get the same experience that you
do in a headset. But we still have–
Alan: Yes, I’ve been in one of
those giant caves. They’re awesome, except very nauseating.
Dan: Yeah. If you don’t know
what you’re doing, and if you have a bad match between the refresh
rate, especially if you do roller coasters or stuff like that. Yeah.
Alan: Ugh. Why do VR people show
roller coasters as the first experience? Stop doing that! Anybody
listening, stop doing that! [laughs]
Dan: [laughs] That’s true.
Alan: Then you talked about IP
and content creation using local resources. So how does that work?
Dan: So let me give an example.
So we teamed up with Loyola University in Chicago and the best
ophthalmologist, among the best in US. And they were very interesting
to use this for learning how to examine a patient. What’s the
process? Because a lot of students are not comfortable to do it, do
that physically. So they had all the knowledge and we had, of course,
some resources and we had the platform. So we teamed up together.
This IP is developed, validated by the university and the experts
there. And then once when that’s created, first of all, of course,
they use it themselves. And then in a span of a few months after
releasing that, we had everyone from Harvard to Stanford to purchase
this application. And then we do a revenue split between us and the
university. And that IP then propagates, it’s put in the Vault in the
marketplace and it’s sold around the world.
Alan: Pretty awesome. So kind of
like– how did I say it? Maybe I guess the Unity asset store would be
the same. Only instead of for games, you’re doing it for learning.
Dan: Yes, you can say that. It
is important because you still have customer that not even want to
create, even if it’s easy. They just want to buy something off the
shelf. And that’s where we have the Vault. The Vault contains not
only building blocks, but full applications that have been certified
by an expert, because we are not experts in ophthalmology. Or almost
any of the areas we cover.
Alan: Huh, interesting. So I
guess you’re relying on your partners to, I guess, help develop– not
only develop, but certify it.
Dan: Yep.
Alan: To make sure that it is
doing what it says. Now, how do you deal with that? So let’s say, for
example, you work with the university in the US to develop this
content and then somebody in Europe wants it. But maybe the education
is a little bit different. How does that work?
Dan: Basically, it’s– the idea
is, a large amount of our customers don’t want to basically learn how
to create content themselves and they much rather prefer to get the
flying start by selecting existing assets. So we have this– looks
almost like Netflix, with description of each application, and then
we have a needs assessment session. And during that session we walk
through them, what exactly they’re interested. How do you want to
start? And then we together pick up maybe a dozen application for a
pilot and subject to their satisfaction. Then it could get full
access to both the platform and evolve.
Alan: You do a trial with them,
say here’s ten pieces of content, try it out for a month and then pay
us and you have access to everything.
Dan: Yeah, I mean, we are
flexible. Typically we want to make sure that there is a genuine
interest. So we do insist on having this type of session, a workshop
essentially where we fully understand their needs. We make sure that
we under-promise, over-deliver. But once we establish that and
establish that they have a genuine interest in a budget. So assuming
that they are happy, they will purchase. Yes, no problem. We can give
it to them for 30 days at no cost.
Alan: How do you– I guess the
question I have is, it’s a Netflix. So is it like Netflix, 7.99 and
all you can watch? Or is it three dollars for this, and four dollars
for that, or five hundred for that?
Dan: Yes. So we are not yet
there. We have two types of sales. We have what I call top-down,
where it means we are approaching large organization or national
governments or regional governments. And that’s when we do this many
10,000 licenses at once. And then we have what we call bottom-up,
when we start with much smaller things. As little as 50,000. But we
typically are B2B. We are not yet B2C. So if an individual person
wants to use it, we are coming out this month with a freemium so you
can actually get to test everything. But for a low amount of cost,
but you don’t have all the features. For example, there some patients
who can do five lessons in application, but that’s it. So we are
coming up with models, but we are not yet at the Netflix level with
all-you-can-eat buffet for a amount of money. But it’s going that
direction.
Alan: I think everybody is
trying to figure out the Netflix model of everything. And then, of
course, Disney+ comes along out of nowhere. One of the things that
has come up previously on the podcast is 3D models, and the fact that
there’s no standardization. You mentioned that you work with SAP to
kind of import any types of 3D models. Can you talk us through that?
Dan: Yes, absolutely. So this is
a well-known problem that we’ve had for 20 years, even in the
previous platforms. The more advanced ones we used to do, that
require programming skills. Still, you have to be able to import. And
originally when we started, we had to import through neutral file
formats. But if you do that, you lose a lot of data. So you have to
go native. Not only do you have to go native, you have to — if you
want to support a Boeing, or Airbus or, any of the big companies —
you have to support various versions. So, for example, Catia, which
is a Dassault’s CAD system, you have to support version 6.467. And to
do that, it requires a lot of resources.
So what we did is we looked at who’s
the best in the world to do that. So we worked with two or three
companies. One of them was Aries, not long ago acquired by SAP, so
that’s why we have that partnership. And we basically licensed their
importers and embedded in our platform, but make it much more easier
for someone that doesn’t know a lot about CAD and polygons and
polygon reduction. So that’s our approach. And then we also assure
that as new versions of CAD come out, that we are compliant with
that. Because of this important. So currently we can support 120+
formats through these various collaborations we have, which is pretty
much everything except the obscure formats. And it’s not only CAD.
It’s also if you want MRI device or you have GIS data, BIM data. So
it’s– or scan data. Or even 360 videos, we support that, too.
Alan: It’s kind of crazy if you
think about it. The world of 3D, 120 formats. For people listening,
if you don’t understand what this– imagine JPGs, you want to send a
photo to somebody. But there was 120 different photo formats that you
could send to somebody, and not every phone or device would accept
it. This is crazy.
Dan: Yeah, it is. There is a
reason it’s not so easy with content. You have to tackle this
problem, otherwise you don’t get even started without the model.
Alan: It’s true. It seems like
more of a non-starting issues with all of 3D, not just VR and AR, but
even just putting something. Because you mentioned when you first
started out, you had Web3D back in– before ’99 you said, I think.
You had 3D on web.
Dan: Yes. So we would start with
that, which was quite fantastic. I still– I mean, we had a very
large contract — more than $5-million — with office depot to do
furniture configuration of chairs, and office planners where we had
hundreds of thousands of skews of furniture. And it was used and it
worked, very well. We had done configurators for Suzuki’s motorcycle,
because they make most the money in part. But those were kind of
kiosk like settings in a dealership, where you basically picked the
favorite bag you have, and then put all the gear that you want,
saddlebags, everything you want, and then automatically we got you
the price, etc. So that type of things we had to do then, because
nobody was wooed by virtual reality. So how do you make money and
support yourself? We’ve been self-funded since 2001 as a company. We
don’t have any VC money. We just make money based on the value we
provide to our customers. And that puts us in a pretty unique
situation.
Alan: Amazing. It really does.
Then you look at something like Blippar, who raised 110 million–
actually more than that, $110-million and then went bankrupt in a
four year span. What are your thoughts on that?
Dan: You know, it’s nothing
different from a dot-com or any similar cycles that you have this
hype curve. People come in, they overestimate the benefits,
underestimate the issues, the practical things that you have to fix.
Then people realize the problems with– that happened with dot-com
exactly. And then most of the companies then crashing. And then there
are a few that survive. Small companies like Google, and eventually
people like Facebook and others, and create this revolution. And I
don’t think it’s different in anything. It’s the same happened in VR.
The interesting thing with VR is we’ve had these cycles three, four
times during 20 years. You may not remember, but there was a VR
revolution back in ’95 and I was part of that. There were clunky
headsets. You basically have to have something supporting your head.
And then we had crash 2001 and then we had another crash in 2007. A
billion dollar to spend on virtual worlds. I don’t know if you
remember that wave.
Blippar is part of this– and Blippar
is not the only one. I would say 90 percent of Chinese VR companies
went bankrupt last two years, and you have similar in the US, a lot
of hardware companies that demolished in the long run. And you do
take a big swing from an investment perspective and you’re all in and
if it doesn’t work, it doesn’t work. We have a slightly different
approach to it, because we are owning our shares and we care about
our customer. And we don’t take huge risks. And it proved to be a
marathon.
If you told me in ’99 that virtual
reality didn’t take off still– in fact, to be very honest with you,
what happened with me is that by 2007, 2008, I said, “This is
it. I’m quitting VR. I mean, it’s a wonderful business, but it’s more
lifestyle business. Yes, we make money and yes, we grow. But this is
not going to be a billion dollar business anytime soon.” And I
was instrumental to take when I was 37 a company to $500-million. So
I was like, “This is it.” So what I did, I left the
company. Still being a big shareholders in custody of my friend, the
current CEO, and I said, “I’m gonna do something totally
different.” So I went into smart devices. I created a company
called Greenway System with a friend, former CTO for the consumer
department of a large IT company. And we took that company to
$250-million between 2008, 2009 to 2011. And I was planning to never
come back to virtual reality.
So then I get a call in 2011 from my
friend, and he says, “Dan, you have to come back.” I said
“Why?” “Virtual reality’s coming back.” I said,
“No way. It’s not going to happen in our lifetime.” But
then I came back and visited, and I saw that his order picked up and
the volume of business picked up. And I said, “What’s the
secret?” And the secret was very simple. The thing you have in
your pocket. These phones could handle applications that we used to
run for uncle for million dollar systems. You could run it on your
phone. So then I realized, wait a second, this is earth-shattering.
And VR has been the love of my life for long periods. I said, “I
have to come back.”.
So I sold my shares with Greenway and
came back. Then I came back and said, “OK, let’s take a hard
look at the business and see how can we grow it. We have fantastic
customers. We have a good product. We have good reputation. We do
make money.” We actually know how to make money, which is not so
easy in this business, as you know. So how can we expand? So then we
came up with this idea. Why don’t we take our knowledge and share it
with various locations around the world by building centers? Those
are the centers I was just talking about. So we embarked with our
team. We went to 130 different countries, visited predominantly with
governments and universities. And I think we had a problem solution
to the problem. What keeps them up at night? That’s what I typically
take. If you ask a minister in a country in Europe what keeps him or
her at night?
Most likely the answer will be that I’m
worried about the jobs. I’m worried about the disruption that
technology causes, because it kills and creates so many jobs. And I
wonder what in 10 years what’s going to happen with all these
displaced people, and how do we take a truck driver that today will
become unemployed in the next five years? We can’t turn him into a
coder, right? That’s not going to happen. So what we do is we help
them to use virtual reality — and augmented reality — to train them
to become welders or nurses. And that’s the programs we put together
with government. So we came up with the idea to set up this center to
solve big problems, not small problems. Because if you talk with a
minister, virtual reality is not on a top– probably not top 100
list. So anyway, so we started setting up those large centres, and
then we partnered each location with either governments or
institutions. And this way we not only had a presence in all these
places, but could grow the business, but we also had a customer
because oftentimes these governments want to do workforce
development, you have complete projects. And that allowed us to
expand the business and have a presence that today, although we don’t
have investment, I think we have more locations than Magic Leap, that
has a $2.7-billion investment.
Alan: They’ve spent an enormous
amount of money, and we shall see.
Dan: No, no, no! Actually, no,
I’m just going to Vegas this year. And I’m together with Magic Leap,
I think Magic Leap is a wonderful product. We are very proud to
announce that we are releasing the platform now in January together
with them. And the actual senior vice president of Magic Leap, John
Gaeta, joined our advisory board. So we think it’s a great company.
So is, I think, Microsoft with Hololens. So we tried to stay– I’m
from Sweden, we try to stay neutral in this contest. We like
neutrality.
Alan: I love it. All right. Is
there anything else that you think you want people to know?
Dan: I’m very excited about
success stories. And in our company, we have a three word mantra.
Build the platform, sell the platform. It makes customers successful.
And if you sell software as a service platform, you’re not better. If
a customer wakes up tomorrow, says, “OK, I don’t like the
platform any longer. I’m not happy with it.” Then they stop
paying, and we charge per month. So customer success is super
important. And we are just about to release a series of — from 28
countries — customer success stories. We’ve published– I think I
published one of them just recently in my LinkedIn. And we are
rolling those out, because people are curious. And in the first six
hours, I have thousand views, because people really are hungry for
it. So we understand the benefits of VR and AR. But show me someone
that actually benefits. And I don’t want to hear from you. I want to
hear from these customers; why they use it, how they use it, and why
they continue to expand. So that’s something that we’ve put a lot of
effort in and you’ll see a lot more news from us in that context.
Giving the customers a voice.
Alan: That’s wonderful. I think
that’s what’s been missing. A lot of people over the last few years
have been saying, “Oh, VR can do this. AR can do this. We could
do this.” But when you think about it, there’s not been a lot of
people saying “We have done this. And here’s our results.”
Dan: Yep, yep, that’s correct.
And by the way, we don’t really think AR, VR, mobile, it is in our
context and for our platform, it’s mainly the same. But if I was to
think about it a bit deeper, I would put it in three buckets. The
first one — and this is how would we approach a product, also — the
first part of our problem is the learning. You use it to create
content to learn. The second part of our platform is for training to
apply what you learn. And the third part of this is performing. So
once you learned and trained, then you– let’s say you work with
engines, and you’ll get to Alaska and you have to repair that engine.
And you may not remember all the steps. That’s where AR and the
contextual knowledge injection comes to place. So those are– it’s a
learned trained performance.
And that’s how we see that more or less
all the customer success stories, they evolve. It may take baby steps
in the beginning with learning. And we want them to make it easy,
because if you tell a large corporation — even an Exxon — that “Oh,
you want to roll out to 10,000 workers? Oh, you have to be 10,000
Oculus.” They get discouraged. They will never do that. But if
you tell them that, yes, we could roll out 10,000 workers, but you
can use your tablet, existing tablet, even your own devices. And
then, slowly, let’s say you have 10,000 users on these devices, then
you have maybe 200 headsets. And over time, as the headsets become
cheaper, faster, better, you can have more. So I think that has been
a big limitation for a lot of companies that try to sell platforms,
because they are dependent on the high-end devices. And let’s face
it, the larger devices have disappointed, right? Even great guys like
Magic Leap or Hololens. If you look at the figures of units, how many
many they’ve sold, they are not that much, yet. But it will come.
It’s just a matter of time. So our strategy is to make sure that we
satisfy customers today, tens of thousands of users with an
organization at an affordable way so they don’t have to wait for the
cheaper, faster, better devices.
Alan: Now, there’s some really
amazing advice. So I think with that, I think we should wrap it up.
Any final words, Dan?
Dan: No, other than that, I’m
living like a kid in a candy store, you know, I plan my next 7,000
days, which is– 7,000 days is about 20 years. So I hope with good
health and exercise and a few other tricks in my bag, I’ll be part of
this amazing journey. Just fun to be around and see this revolution
in VR and AR.
Alan: Well, what problem in the
world do you want to see solved using XR technologies?
Dan: Ha, that’s quite simple.
I’m sure you’ve seen Matrix, right? Where you’ve got to take the blue
pill or the red pill. I think humanity at the moment is at the
bifurcation point. You can take the left turn, which means that
increasingly machines take over humanity’ role in workforce. And if
you believe scaremongers on AI that eventually they take over
completely, but at least take our jobs. So you have a huge amount of
population that we can render useless. So that’s one vision.
The other vision is that we become
human 2.0, enhanced humans, where we stand on the shoulders of
machines, use our curiosity and blend ourselves with machines. I
think things like [garbled] working to put wires into your head. I
think that that’s going to take too long time. In fact, if we wait
for that by then, the machines have taken over. So I think AR and VR
has a huge importance to create that bridge between man and machine.
So instead of today interacting with machines at speed of thumb —
which is very, very slow — we operate with machine at speed of
sight. And we get the information. The machine knows where you are,
who you are, what you want to do, and it feeds that information to
you. And you have this superpower. And that’s the future that I’d
love to see for us. And that’s what I will try — in whatever
capacity I can — to do for the next 7,000 days.
Alan: Man, thank you so much,
Dan. Here’s to the next 7,000 days.
38 min
January 20, 2020 The IKEA of AR: Making Content Effortlessly, with EON Reality's Dan Lejerskar
It's been said on this show before; XR doesn't have a technology problem, it has an adoption problem. In Dan Lejerskar's experience, everyone from universities to governments see the value of XR -- they just lack the content to make it a worthwhile, everyday tool. He and Alan explore how EON Reality is addressing this discrepancy.
Alan: Hi, it's Alan Smithson
here. Today we're speaking with Dan Lejeskar, founder and chairman of
EON Reality, a world leader in
virtual/augmented reality based knowledge transfer for industry and
education. They believe that knowledge is a human right and it's
their goal to make knowledge available, affordable, and accessible
for every human on the planet. We're going to find out how, in the
next XR for Business Podcast. Dan, welcome to the show, my friend.
Dan: Thank you so much.
Alan: I'm really, really
excited. I know you guys have been working-- well, you specifically
have been working in the 3D virtual space for many years now. How did
you get involved in VR and learning?
Dan: In my past, I used to work
with simulators -- big aircraft simulators, etc. -- and I got really
excited about seeing the effect it has on pilots and soldiers, and I
always thought that it would be useful to do the same, but for normal
people, nurses, etc. But obviously these people couldn't afford a
$50-million simulator. So I had to be patient and wait until the
computers follow Moore's Law; become cheaper, faster, better. And by
'99, the hardware was there, so you can start running this on PCs. So
we were very early adopters of virtual reality already in that
period.
Alan: We're talking 20 years.
Most people know VR and AR as if kind of something in the last five
years. But what was it like kind of going through these growing pains
of going from a million dollar simulator -- millions of dollars
simulator -- to now we can buy an Oculus Quest for 500 bucks?
Dan: It's been an interesting
journey, with a lot of ups and downs. And very much VR has been like
AI. I'm sure you've read about the "AI Winter", when things
didn't go that well. We've had quite a few ups and downs in virtual
reality. '99 was fantastic, because that was the era of dot-coms. And
we started with something called Web3D, so you can do 3D on the web.
It had actually millions of users. Then we had a hard landing 2001.
Remember when dot-com crashed? And we had to move our business from
industry and education to defence because we had September 11th. So
that was kind of what saved our business, doing homeland security
centers and the like. And then slowly and surely, we picked up the
business up to 2007, 2008. And during this period, there were several
iterations. There was something called people avatars and virtual
worlds, that was very popular around 2007. That raised and crashed
also, pretty tough. But we managed to navigate those water until I
would say 2011, 2012, when the hardware became available for mobile
devices. So this was before Oculus. Already then we could see where
the industry was going.
Alan: Oh, you guys, you never
lost your path. You've veered a little bit from military, to industry
and education, back to military, and then back to industry and
education. Obviously, the passion is in the industry, knowledge
transfer and education. What are some of the projects that you guys
have done in the last few years that really just made you go, "Wow,
this really is something that, quote unquote, normal people can use?"
Dan: So, you're right. We
realized quickly that the biggest value has to do with knowledge
transfer. And we started thinking how can this technology be used to
solve big problems? And we identified three areas. One is government.
We have an initiative that I'm happ
38 min
January 17, 2020 Letting Workers Qualify Themselves in AR, with AR Expert’s Dr. Björn Schwerdtfeger
Getting future workers excited for the jobs they might have tomorrow can be challenging, especially when many young workers tend to enjoy challenging themselves with new tasks. Dr. Björn Schwerdtfeger says that AR training can allow those workers to qualify themselves for all sorts of tasks, and have fun doing it, to boot.
[Transcript coming shortly]
Alan: Hey, everybody, and thanks
for joining in on the XR for Business Podcast with your host, Alan
Smithson. I’m really excited today. I have Dr. Björn Schwerdtfeger
from Germany. He has more than 15 years experience in augmented
reality. Together with the German industry, he’s evaluated almost
every idea for AR in applications in the industry. He’s been a
co-inventor of Pick-By-Vision at TU Munich. And during that time —
when computers for AR glasses were still carried in large backpacks
— Björn holds a PhD in industrial augmented reality as a serial
entrepreneur. And among other things, his company, AR Experts, is
advising about a third of Germany’s most important production
companies, and is shaping their augmented reality roadmaps. You can
learn more about them at ar-experts.de. And they have another product
that they’re gonna be talking about today. It’s ar-giri.com. Björn,
welcome to the show, my friend.
Björn: Yeah. Welcome, Alan.
Nice to meet you. Nice to meet you online. I’m looking forward to
this podcast.
Alan: It’s so exciting. The work
that you’ve been doing over the last few years — like a decade and a
half — is really starting to come to fruition now. I mean, all of
the hard work that you and your team have done to evangelize a
technology that — let’s be honest — 15 years ago, the technology
really wasn’t ready for the market. Tell us, how did you get into
this, into AR?
Björn: It was actually quite
funny, while still studying at the university, computer science, and
then somewhere else, augmented reality which popped up. And someone
had a demonstrator, where someone took some glasses and glued a
webcam — we had external webcams earlier — just hot-glued to some
glasses and using some [unclear] stuff and highlighting it. I think
it was just a cube. A virtual cube… And it was so fascinating that
you can bring this computer interface into the real world. Quite a
long time ago. But it was really nice.
Alan: Björn, did you say there
was a webcam hot-glued to a pair of glasses?
Björn: Exactly. That’s how we
did augmented reality 15, 20 years ago.
Alan: Amazing. You are one of
the OG, the originals of this industry. You’ve been building and
advising brands and companies around their strategy for production.
What is the one thing in augmented reality right now that you’ve seen
the most ROI?
Björn: It’s probably… we’ve
seen a lot of companies trying to do everything. Basically every
single one of us have tried to out, in the last three decades, and
failed with it. And we’re figuring out what is actually the core of
augmented reality. And the core of augmented reality is not– it’s
not a measurement tool, it’s not a tool for everything. It looks like
a display, and it is a good display. But where its core is, where
it’s so good in, is in communication. It displays communication and
augmented reality is big. It’s so much more close to your reality,
that perception is getting much better. So what you tried to
communicate with exosheets, nice PowerPoints; it’s getting so closer
to the user with augmented reality. And they figured out that the
communication got so much better using augmented reality — using
*good* implemented augemnted reality, it’s quite important — you can
do a lot of mistakes there. But this is helping so much. And that’s
why you’re seeing currently augmented reality mainly in marketing,
because marketing is a form of communication. You see it in museums,
because museums are also a form of communication. And then work
training, because training is also a lot of communication.
Communicating the knowledge someone has to another user is key. It’s
the same in the museum. A few people have some knowledge of what has
happened the past, of the stories. This is a tool to improve the way
you’re telling stories, basically. That make sense to you?
Alan: Makes total sense. So,
when you say communications, are you meaning, I put on a pair of
glasses and an avatar pops up and starts talking to me, or maybe a
video screen? Run a certain example of what that would look like in
an industrial space.
Björn: It’s– augmented
reality, it doesn’t do a lot, but what it does, it really improves
the way we communicate things. An avatar can pop up, but it’s also
communicating technically-complex things, like someone has
constructed a car, or a lot of people have constructed a car, and a
lot of other people need to produce that car. There’s a lot of
technical challenge you need to talk about. You can make those things
visible. AR is a good tool to make things visible. Also, for training
processes, AR is really good thing, a tool to make things visible.
And also museums, you’ve got some boring things there, but then you
could put the story over it as a new layer, and the story is being
told through augmented reality. And that’s why it’s so nice.
Alan: I recently read a book
called “The Age of Smart Information” by Mike Pell, and he
talks about how information now is just information. I mean, we have
access to the world’s information — I can Google pretty much any
details — but it just gives me the information. It’s not in context
to my world. It’s not in context to me. I can ask Google questions
now; Voice, Amazon, Alexa, and Google Voice. But really, what
they’re– what you’re discussing here, what you’re talking about is
being able to look at a machine and the machine providing you with
the story on how to fix it, or how to deal with it. Is that what I’m
gathering here?
Björn: Well, for part of it,
and also bringing this information, registering to the real world.
That’s what you’re saying. Yeah, you’ve got the information in
Google, but Google has information somewhere in servers, you can pick
it up using your smartphone. But actually, you want to look at a
machine or an exhibition, and you want to get information which is
there exactly for that machine. And that’s where augmented reality
can help, and where it makes so much more fun; finding the
information so much faster and with making it fun, basically.
Alan: I recently read an article
that came out. It’s a study by IBM saying that over 120 million
people are going to need reskilling and upskilling as AI and robotics
replace human workers. 120 million people. In my opinion, AR and VR
are the fastest ways to do this. But you mentioned something that
really struck me. You mentioned that it’s a lot more fun. And I
think, if you look at education as a system, it’s competing with
Hollywood movies and AAA video games. If we don’t start making
education faster and more fun, then I think we’re gonna start falling
behind, as we need to reskill people as fast as humanly possible.
What are your thoughts on this?
Björn: We’re talking about a
lot of topics now. So, one part is that 120 million people need to be
reskilled. Seems that, I guess it’s one billion people working in
production. One billion people producing things every day. I see that
are a lot of more people need to be reskilled every day, because also
the jobs are getting more complex. So when you’re working with
clients, you always try to do an internship first with the customer.
So you’re trying to do the job, which we try them. They get the
support. We then analyze the situations and realize, hey, this guys
doing a job since 10 years, 20 years, but now life is getting more
complex, because they need to handle the more complex machines. The
tasks are getting more complex, and they need to do so much more
work, so much more different work. The world’s moving faster and the
environments that you’re working in is getting so much more complex.
The answer is always “Okay how can you fix it in the future?”
The always say that you need more people. It’s always an answer,
because those people are not thinking about making it easier to work
on the more complex tasks. At least in Germany, we face a problem.
You cannot plant more people for doing your job.
Björn: This is universal, my
friend. They just did a study recently — I’ll have to put it in the
show notes — but they did a study of 3,000 youth in the US and
China. And they asked them what jobs they wanted. They gave them
seven jobs to choose from. And in China, the number one job was
astronaut. In America, the number one job was YouTube influencer.
Think about that for a second: in Western society, our kids would
rather try to be a YouTube influencer traveling around the world on
Snapchat and Instagram, than actually contributing to a business. And
the mindset of that, I think is going to start haunting us in the
future, where AI is being taught to grade five children, and grade
school in China.
Björn: And also, China used to
be the workbench of the world 20 years ago; everybody was proud and
rising up to get a good job. And now, as you’re saying, everybody
wants to be an astronaut. What we figured out is, there’s some old
people who like their job, but they don’t find new people for the
actual boring jobs. What we figured out is, we did a lot of–
obviously a lot of interviews with the workers, and they’re saying,
when you’re doing this with glasses and I can self-qualify myself for
that workplace? That’s really interesting, because on the one hand,
you don’t put production workers on some different things because
then they’re liable to do a lot of mistakes. But on the other hand,
they want to do so many different jobs. But it won’t be so expensive
to qualify them for so many different jobs. But for the business on
the one hand, they need to do it. And on the other hand, they want to
do it. So when you give them the possibility to qualify themselves
for so many more, different jobs, that’s making it interesting.
Alan: If you look at people’s
LinkedIn — especially younger people — they’re changing jobs every
three years. And that from a process standpoint — from a factory, if
you’re running a factory, you need somebody to work on a machine, and
every three years you’re having to change it out — and there are
people that are working there now, but the average age of people
working in industry is escalating above 50. And people are starting
to retire en masse. So you have this huge group of people retiring
from jobs that they’ve been in for 20, 30, 40 years. People coming
into those jobs, they’re looking at them going, “Well, I only
want to do something for three years, most. And I want to try
something new. I always want to be challenged.” And I think you
mentioned something earlier — and I thought it was bang-on — is
that we can make training fun. And by doing that, you’re not only
being able to train people on new jobs fast, so they can feel like
they’re always growing. But I think also you said that these jobs are
becoming more and more complex, which I think is actually a benefit
to everybody, because if it’s more complex, it’s actually more
challenging for the learner or for the person working. And I think
that’s really what gets people excited, is a real challenge in life.
What are your thoughts on that?
Björn: As you’re saying, it’s a
better fit for everybody. You can look at some jobs and say, okay, I
need more people for this. Perhaps that’s not a good fit for
everybody, because more people means more stress, and more cost. And
everybody’s just doing one job, and people want to do different
things. Or you look at the other side — you give people the
possibility that they are getting the power to do more complex jobs,
and different jobs — qualify themselves. Be more proud of what they
can do, being more interested in that they can do different jobs.
From the business side, that’s more flexible workers, they make less
mistakes. They, say, remember 70 percent more. So it’s a better fit
for everybody. If you look at this from a broader perspective, the
benefit is — as with most things — is survival of the fittest.
That’s the English saying for it, right? “Survival of the
fittest,” Right?
Alan: But I think now we have
the ability to– imagine, 20 years ago, if you wanted the facts about
something, you had to look them up. There was no– well, I guess the
Internet’s been around, but it was really difficult to do it. Now, I
don’t even have to pull my phone out; I can just ask Google and every
answer is there. So, it’s not really about what you know anymore. It
used to be what you know, and what you knew gave you an advantage.
But it’s no longer about that. It’s about how you apply that
knowledge. And I think this is where augmented reality becomes a
tool, like nothing we’ve ever had. You did a PhD in industrial
augmented reality. What did that entail? Cause I know that that was a
big part of your life. What does a PhD in industrial augmented
reality mean, or what does it look like?
Björn: You need to look quite
holistically at augmented reality. You cannot simply say, “oh,
this improves a job,” but you also need to find a way how it
improves the job. Industry is using things only if they work. So,
does it work from the technical side — does it really do the job? If
they work from the business side, does it save you money? And if it
works for the people? If you don’t find a solution for all three
areas — technology, people and business — it’s not working.
Alan: That is pretty much– if
nobody takes anything else away from this podcast, that is it. You
got to have the technology that works. It has to serve the people.
And it has to make good business sense.
Björn: Exactly. And then there
are also a lot of other obstacles. What was funny in the last years
was — particularly in the last year — quite often sitting in
management rounds, and the managers always were saying, “oh, we
cannot give it to the workers. They are not accepting it and they
don’t like it. We’ve got old workers. They cannot work with it. They
were rejecting a lot of technology.” But if you work close
together with people, with the workers, and figure out what they
really need — what really helps those workers standing there, also
the old guys — it’s really age-independent things. “It’s cool.
When can we get it?”
Alan: It’s so true. You
mentioned that– I had somebody else in the podcast, and they were
saying, “Look, you really need to engage. You need three people
in a conversation about rolling this out. You need the very high
levels — you need maybe the CEO or somebody in the C level — to
champion to say, yes, we’re gonna do this, you have my full support,
go for it. Then you need an internal champion from the management
level, who’s going to keep the communication between the C levels to
keep the funding going for the project. And then you need somebody
who’s actually going to be using it. Somebody who’s on the factory
floor really doing this. And by having the buy-in of the three
different levels, that’s how you get real change.” Is that what
you’re seeing?
Björn: Yeah. Also, I think the
most important — you need you need he OK from CEOs, from the
innovation manager, but what you really need is commitment from the
users who will really tell you what they need. It needs to be a
solution for the workers. If it doesn’t work with workers, they are
simply destroying your glasses and they’re not going to use them
anymore.
Alan: Yeah, I can see that. So
if you look at augmented reality in the enterprise, are you seeing
more of this moving to glasses, or is it tablet-based for now? What’s
being used most, and what is kind of delivering the most effective
ROI?
Björn: You see some stuff
happening on iPads. And it’s always quite nice. It’s scalable.
Everybody says it’s a more robust technology. But if you’re doing a
deep dive into technology — into the perception of information —
it’s very different with glasses. You have a window to the world;
you’re basically standing in your house. You’re looking through the
window into the nice world. It’s like there’s some distance. What’s
cool about augmented reality is that there’s this real-world overlay
and it’s the distance between the point where you need information.
It gets information presented. It’s going to zero. And this effect,
you only have those glasses. So that’s why we’re seeing more and more
people working on glasses. Also doing a lot of rollouts with glasses,
for sure. Now you see a lot of rollouts for smart glasses, which is
already — we don’t call it augmented reality in Germany. I know
Americans do — but it’s good that it’s happening. So you see a lot
of rollouts with smart glasses. We hope to see also more rollouts
with the augmented reality glasses, and a lot of VR glasses getting
rolled out — AR glass are not currently not getting rolled out
because no one’s currently shipping AR glasses. There’s Magic Leap–
Alan: Still a challenge.
Björn: Yeah, but there are a
lot of things are solved. The Hololens line. This device is like 4
years old. So yeah.
Alan: Yeah.
Björn: Everybody’s got the
HoloLens 2, but they’re also waiting for some competitors [to start]
shipping their glasses. Because from the general point of view, the
technical challenge which is AR, there are some recent companies that
have solved those problems. But it’s a matter of time. It’s a matter
of months now, that Microsoft is shipping a new HoloLens; some other
companies are shipping glasses. NReal is going to be shipping some
glasses. Magic Leap is putting out their next generation of glasses
and also shipping to the European market for sure — we have the
glasses here, but we cannot currently roll out those glasses, because
they don’t even have certifications for the European markets. So
these are the next-level things which need to happen now; the glasses
getting all the certification for the market.
Alan: It’s interesting, Björn,
you mentioned– the first thing you mentioned was technology, then
people, then has to make business sense. I think we’re still
struggling with the technology side. We have glasses. We’ve proven
the business cases; they work. And it’s just a matter of “Oh,
you’re shipping them at scale now.” I think over the next six
months, we’re going to see an absolute tsunami of technology being
introduced to the marketplace. You mentioned three different types of
glasses. You’ve got overlay glasses — or ones that kind of show you
a heads-up display — like RealWear, for example, who just raised $80
million. And those are not really augmented reality glasses, they’re
more heads-up display, similar to Google Glass. And then you’ve got
VR headsets. And then you’ve got AR glasses. And the latter — the AR
glasses — are the ones that are able to look at a machine and
project images on top of it in context to that, in the real world.
And VR has its place, heads-up displays have their place. But really,
the magic is in the these AR glasses. Is that what you’re saying?
Björn: Exactly. And at the end,
has the biggest benefit also, because the way you’re perceiving
information, it’s made better if you get it presented in a
stereoscopic way. That’s what glasses are doing. They benefit this
way. But if you want to achieve this 70 percent of memory effect —
that works really good, that people really are remembering so much
more information because this is information in place — you need to
do it with glasses.
Alan: I don’t know if you can
give any specifics around companies that are deploying this. Are
there any companies that you know that have deployed AR glasses at
scale? Like, real AR glasses? Or are we still in the pilot purgatory
of this?
Björn: Not really in scale yet.
So, scale is like thousands of units. There are some companies using
several thousands of devices, but really count as a problem. No
devices are available, and the last Hololens was shipped one year
ago. Now it’s like, all the glasses are breaking, because they so
old. So you cannot roll out 1,000 glasses, because you cannot ensure
that they will hold for such a long time until you get a replacement.
That’s currently not happening. But I think it’s a matter of time.
And also, the companies, the glass makers are fulfilling all the IT
requirements that you can integrate into those glasses, and large
infrastructures that can have property-wide management
certifications, you’ve got health certifications, and all that stuff.
But we think, from the technology point of view, it’s actually
solved… well, not every glassmaker has solved every piece of
puzzle.
Alan: Yeah, I think that’s
interesting. You look at all the different glass manufacturers and
they all have different parts solved.
Björn: Technology’s not a
problem. There are many possible companies because we are scouting
for all the AR/VR glasses. So in the last three years we’ve been
reviewing, I think, 228 glasses. 228.
Alan: [chuckles] Holy crap.
Björn: That’s really quite a
lot. For most glasses, half of them are shipped. For most of the
other glasses, we’ve figured out, what is missing there? They all
have different focus; what they want to be. You’ve also seen — which
is quite sad, actually — we’ve seen the Meta glass, for example,
quite early. And it was looked over by physicists and optical experts
who said, “there’s fundamental problems here. Why the hell are
they doing it?” And one year later or two years later, they went
bankrupt. It’s the same device. But it’s so complex, augmented
reality — also, virtual reality — and the companies also need to
learn this. But we’ve got the feeling there are a few companies now
who’ve learnt a lot, and will be able to ship a lot of glasses by the
beginning of next year or so, I guess.
Alan: All right. So let’s talk
about specifics. What are, in your opinion, the top five AR glasses?
Out of– you reviewed 228 pairs of them, which is incredible. I would
love to get that information to share with the listeners, if
possible. But yeah. What are your top five, then?
Björn: I mean, the bigger ones
you probably know as well. Yes, sure. Microsoft tries to be the
leader of market, but they’re not shipping. It doesn’t show up in the
top five company because they don’t ship. Magic Leap for sure. I
mean, everybody’s saying Magic Leap is not so good, and they oversold
what they ship. But a few things about Magic Leap are still better
than Hololens 2.
Alan: Like what?
Björn: Wearer comfort is still
better with Magic Leap than HoloLens 2. Because in the end, it’s way
more lightweight.
Alan: Yeah, because they took
the compute power off of the headset. Have you tried the Hololens 2
on?
Björn: Yeah, for sure. It’s
really good. From the control, it’s way better than the HoloLens. The
interaction is really good — they placed the HoloLens 1 on so many
heads, and they always needed to explain a lot of things.
Alan: Yeah, yeah. That little
pinch to click, nobody gets that. Anybody over 30 doesn’t get that
right. We’re kind of in this weird place, where it’s like we know
where the ROI is, companies want to deploy it, and we can’t [laughs]
For whatever reason.
Björn: And then there are
problems with a second row of companies won’t ship until the
beginning of next year. I kind of was like, you know, all those
companies. NReal doing quite a good job.
Alan: Yeah. NReal really
impressed me as well. NReal is a spin off from Magic Leap. They took
all of the best parts of the Hololens and Magic Leap idea. They’re
like, “hey, really, what we need is just a lightweight display
with a bit of tracking, and that’s it. We don’t need 8 cameras, and
we don’t need a computer on the head, and all this.” And so they
run it through USB-C — which now actually this week you can plug it
in to your computer, I think they just announced — and you can also
plug it into your phone, so your phone becomes the compute device,
which makes the glasses much, much cheaper. I think they’re shipping
at $599. Or that’s what they’re taking pre-orders for. I can tell you
right now, we’ve done a lot of VR and AR development, and we used to
haul around a big huge computer to do demos for people. We’d bring
around, set up the computer, wire it up, put up the sensors, all of
this. And I got the Oculus Go, thinking, “Oh, this is gonna be
great for VR.” And it was just underwhelming. It was only 3
degrees of freedom. You could look around. But it overheated. The
battery didn’t last very long. It was glitchy. And so when the Oculus
Quest came out, we kind of said, “We’ll just wait, it’s not
going to be as good.” But I recently tried it about a month ago,
and I was very, very impressed with the Oculus Quest. As a VR headset
that is standalone, the tracking is amazing. And now they just
introduced a plug-in, where you can actually, via Wi-Fi, stream your
computer to the Oculus Quest, so you can have computer-based
rendering and graphics pushed to the Quest, which now makes the Quest
an incredible tool for VR. I mean, it’s not AR, but man, the
technologies behind these things really are getting much, much more
impressive over the next little bit. So we’ve talked about Hololens,
Magic Leap, NReal. What are some other ones that are kind of popping
up in the top of your head that maybe are shipping in the near
future?
Björn: We’ve gotten a lot over
the years, so I’ve got a huge heap of glasses next to me.
Alan: Got a big pile?
Björn: Yeah.
Alan: You’re going to start a
museum, Björn.
Björn: We called it– it’s a
kind of museum. Yesterday, by the way, I was at the Deutches Museum
— Germany’s technology museum — and they had the Oculus DK1. It was
looking so good, because it’s looking so old already — it’s only 6
years old. But it was already looking so old. This technology has
improved so much.
Alan: Yeah. The DK1 was the
first VR headset I ever put on my head, and I remember looking at it.
Somebody put it on, they had these big headphones. I thought, “Man,
I don’t know what this thing is, but it looks ridiculous. [laughs] So
big!”
Björn: [chuckles] Yeah, it’s
like scuba diving.
Alan: Yeah, it was ridiculous.
And I got to try, was it the Pimax? The one that’s 8K and it looks
like a big V. Oh my goodness. It felt like I strapped a fishbowl on
my head.
Björn: Yeah, but now it’s
getting smaller, actually.
Alan: Yeah.
Björn: It’s small. And then
also some other glasses are getting smaller. So technology’s
advancing quite a lot.
Alan: You know what’s
interesting about mixed reality? The AR glasses are going from a
glasses standpoint, where you can see the real world and then you’re
kind of putting holograms on top. But another company — Varjo from
Finland — they’re taking a different approach. They built a really,
really good VR headset and then they use pass-through cameras to
create the mixed reality, or AR effect. And I tried it and I thought
I was pretty skeptical, because usually when you have pass-through
cameras, it’s nauseating. It makes you very sick — or makes me very
sick. And so I put it on. And we had the pass-through cameras. I
looked at my hands. And first thing you do is look at your hands, to
see if it’s real time. And it was really, really accurate. It didn’t
make me sick. It looked proper. And then being able to bridge that
gap between, OK, you’re in VR — or sorry, in AR — you can see the
whole world and then all the sudden they’re adding virtual layers to
it. And I went from being in the real world with a car in front of
me, to being in a completely virtual space. It was incredible. It’s
maybe not practical for certain applications, but for applications
where you need to see the real world and you can wear a big, bulky
headset connected to a computer, I think it’s great.
Björn: It has some benefits.
Varjo’s doing a great job, also has visors with a rich display, which
helps a lot. Varjo is doing a great job, particularly with their
retina center display. It’s amazing, what they are doing. It already
helps a lot because you can see much more clearer situations in the
industry to evaluate things. That’s quite good. What I didn’t get was
a story with the see-through glasses. I saw it. I tried it. They’re
doing a good job but still see-through. We had also the glasses of a
Canadian startup. I forgot to name that it’s the same. They were
acquired by Apple.
Alan: Oh, Vrvana!
Björn: Vrvana, yeah.
Alan: Yeah, out of Montreal.
Björn: Yeah. Yeah. We’re having
basically the second device that they ever shipped.
Alan: Oh, very cool. You know
why Apple bought them?
Björn: My guess was, as they
were quite good in the latency and [unclear]. What was it?
Alan: There was those two
things. But the one thing that they did — that nobody else could
figure out — was occlusion from a single camera source. And if you
look at the new ARKit system, the new ARKit allows you to do
occlusion. And for people that don’t know what that means, if I’m
looking in AR glasses and I look at a hologram that’s 3 feet away
from me and somebody walks in front of that, it should know that
they’re walking in front of it and to block it out as they walk
through. If they walk behind it, it should walk behind it. But in
most AR, it doesn’t recognize the depth. So as somebody walks through
it, the object just becomes really big because it’s now projected on
top of them instead of into the real world. And these guys solved
that. And if you look at the new ARKit release that just came out,
that’s actually one of the things that’s embedded in it now. So I
think that’s one of the reasons they did that, because occlusion from
a single camera is very, very difficult.
Björn: I saw the demo of the
ARKit was showing this occlusion. I was really impressed. I was
really impressed that this was working because I know about all the
technical challenges behind it. And I was questioning whether it
would only work with a stereoscopic iPhone, or a structured light
sensor. But it’s not. It was only done by a single camera. This was
looking so good, they a person walking around, you could see this
person was walking in front of or behind the table. And this is
amazing for augmented reality, because we test the technology to make
the augmented reality look really good, to be more immersed to it.
Alan: Yeah, there’s nothing
worse than somebody walking through your hologram when you’re trying
to do something. [laughs] It’s very distracting.
Björn: Going back to the Vrvana
device and media see-through. That’s the scientific term we’re having
for it, media see-through augmented reality, in comparison to optical
see-through. Media see-through part has a lot of other obstacles. So
you’re having the frame rate issues, the latency issue. So, saying
“is it real-time” when you move your hands, which is very
important.
Alan: That’s what I think Varjo
got really well. They nailed it. I mean, it was imperceivable, the
latency.
Björn: That’s quite important.
Then you having the amount of between black and white.
Alan: The contrast, yeah.
Björn: Contrast. That’s
important. I mean, if you’re in a dark room, that’s fine.
Alan: How many people work in a
dark room, though? Not very many. I think HoloLens 1 was pretty
bright, and had some good colors and had a good contrast. But I think
we need to do better, because most people work in very bright
environments.
Björn: Yeah, but it’s even more
complex when you’re looking at the biggest issue, because the cameras
have low contrast and it’s only just to a certain area of
environment. So what you can see is the monitor of the device, it’s
only part of the world. If you’re standing in front of the window,
you can’t help that it’s getting quite bright, or it’s getting dark
and you can’t see what is outside the window. That’s the challenge.
Alan: I think it’s gonna be a
challenge. For now, we need to be in windowless rooms for that.
[laughs]
Björn, I want to switch gears, because we have a little bit
of time left and I want to switch gears and talk about AR Giri.
Björn: That’s our approach for
training. For worker training, for training processes. So we figure
it out in 2016 — a long time ago — we were asked to make some
training apps. We did a lot of innovations at the assembly line,
along with interviews with the workers, creating a lot of prototypes.
And then yeah, we went live with some system and performance tests
and we get random people coming in and we taught them, using
Hololens, how to assemble a car engine and everybody managed it. In
general, everybody managed it. And afterwards a guy came out of the
one day experiments and say, “Hey, I never, ever assembled a car
engine — I just assembled a car!” I said, “wow.”
We’re doing it since so many years and now it’s working. There’s some
room for improvement. Yes. But after so many years, now, it’s
working. We also could figure out together with our partners or
customers that the a learning benefit is so high — it is so really
high — that people can quantify themselves. Motivation is higher.
They remember more. But also, the old management guys who aren’t used
to work in augmented reality said, “OK, that’s nice, but that’s
not a business case because you can never, ever manage to scale this
because of the content crash.”
Alan: Yeah. It’s a problem.
Björn: I want to get all this
content inside. We also need to get 3D data. Once we get that figured
out, you don’t get so many 3D data. Arrows are much more important
than 3D data.
Alan: It’s so true. Somebody
else came on the show and I can’t remember, but they were saying that
at the beginning — it might even be you, I think, when we recorded
this previously — you were saying that we used to take a machine and
recreate it in three dimensions and overlay it on top of the real
machine. And then what people realize is that it’s kind of a pain in
the butt, because you couldn’t see the real machine because the
digital one was on top. So just take all that away and just put the
arrows of what you need. Very simple. “Do this.” An arrow,
a finger pointing at it. Seriously, as a technology industry, AR/VR
technology, we really overthink things. And sometimes the most simple
things are what end up being the most impactful.
Björn: Exactly. And we need to
figure this out. So complex to figure this out.
Alan: Yep.
Björn: At the end, we’re only
working with arrows and videos. This really helps a lot. And someone
needs to generate the content.
Alan: Yep.
Björn: And if you’re having,
for example, the trainer who knows how the process works, and then
you’re having the guy who knows how the AR works, those two guys play
ping-pong.
Alan: Yep.
Björn: “Oh I want to have
it like that.” OK. You do it like that. Then the programmer
gives it back to trainer to translate. If it’s not what I tried to
say, they do it again; they play ping-pong all the time. So in the
end, what we’re doing is that we the trainer — the guy who knows
what needs to be trained — he can generate everything by himself.
But where we need to generate it? Not somewhere in the office.
Production’s happening in the production line. It’s the assembly
line. So the trainer puts on the glasses, goes into the editing mode,
drags and drops what needs to be done. Records basically what he’s
doing. It’s pictures, live-recording, using the Hololens or using
some other glasses. Let me just say, “Okay, now start the
training,” and the training is generated automatically.
Alan: Isn’t that amazing? That’s
incredible. So if people are interested in learning more about that,
it’s ar-giri.com. Being able to enable trainers to create this
content is going to be the key. And I think one more piece of the
puzzle is being able to allow companies that create training —
because some training, they’re going to recreate environments and 3D
models and all this — but being able to have some way for them to
share that as well, because the content creation right now is very
expensive, it’s time consuming. But I think what we’re going to see
is companies that spend a lot of money on developing content, they’re
going to want to be able to monetize that as well, because there’s
got to be a way to share that content across different entities. And
that’ll decrease the time to training right cross the board.
Björn: Exactly. And it’s also
important to share the content. Someone could be working with a
civilian training company on sharing some 3D model content. Sometimes
it’s quite good to test 3D models. Also, you need to think about
different training stages of car. For example, the car doesn’t exist;
the assembly line doesn’t exist yet. You need to train others in
virtual reality, because you don’t have a car. And then you are
moving over and the latest car exists, the training is also getting
more concrete. What needs to be done? That’s more augmented reality.
If you can reuse some 3D models — all those items which already
exists — makes the whole process much more scalable. There’s another
thing we realized is content creation from what the workers are
saying. “It’s nice if you guys generate us a training, but
problem is, production is not static. It’s not like you make a
product for years and you do it always the same way.” It’s like,
if you’re having errors and you’re having errors every week, then
they change something, it improves the process. This new process
needs, again, to be trained towards the other people. Basically, a
training can change every week because production is changing. It’s
also not that you’re only assembling one car at one place. You ship
different cars; you produce different cars. It’s the same assembly
line. And there’s a lot of variation in the trainers all the time.
And so for each work place, you need to improve or modify the
training all the time, basically.
Alan: Well, Björn, we’re coming
to the end of the conversation. But before I let you go, and it has
been a fascinating conversation about augmented reality, the
different glasses, how companies can roll it out. To package this
whole conversation of that, those are the key points. What is one
thing, one challenge in this world, one problem in this world that
you want to see solved using XR technologies?
Björn: We’re kind of doing the
training for the boring jobs. Which is our business, which makes a
lot of fun for us. But there are bigger problems in this world, like
in education. I feel like my history education was so bad, really so
bad. I learned a lot of things later. But augmented reality is such a
great tool which is ready to tell stories, stories of our past. Also
to tell stories about technology and about complexities about the
basics of technology. For example, yesterday in the Deutches Museum
they reproduced the labs, the office of Galileo.
Alan: Oh, cool.
Björn: Galileo was the first
guy doing structured experiments and filling out a lot of the basic
principles. It was looking so nice. And there were a lot of
experiments and a few of them explained, but it would have been so
nice to have Galileo be inside there. anyway. It would have been
stirring and would have so much impact to entertain people so much.
It’s the possibility.
Alan: And it’s not hard to do.
I’m writing an article on volumetric capture right now. And there’s
55 volumetric capture companies in the world. So far. That we know
of.
Björn: And that’s good.
Learning and education, you having… in Germany, the best-paid
people that the German government pays are teachers. Sorry to say
this, but most of them — I’m feeling — are not doing the most
wonderful job. There are some good teachers, but most of them, no.
Alan: Well, it’s hard to have
the best teachers in a system where a teacher is in one school
teaching a group of students. If that teacher now can teach students
around the world in a one-to-many VR presentation — or AR — you can
now start to bring really contextualized, personalized learning to
the world. And I think the world’s education system — systems,
because there are multiple systems around the world — but they’re
just not going to be adequate for a world where the jobs are changing
every few years now.
Björn: Exactly. That’s a
different story of what skills that you need in five and 10 years,
and 20. But if you’re starting to digitalize, it’s not about
replacing old school teachers, but it’s about them giving… teachers
need to do different jobs later. And and it’s also about, for
example, if you need to teach a certain topic, and there are also
different views in the world on this topic. But if, for example, you
could take the three best people in the world — you ask a Chinese
opinion, a European opinion, and you take the best teacher from the
US — who has a certain opinion. You take the best teacher from
Europe who has a different opinion. And the teacher from China who is
also different. But you would have the chance to record those people
once and then giving every student the possibility to experience all
those different opinions presented in the best way to give them the
best teachers. And then you can get it for the next topic. Also, the
three best teachers.
Alan: Björn, I really thank you
for joining us and thank everybody for listening. This has been the
XR for Business Podcast. You can learn more about Björn and his team
at the ar-experts.de. And you can learn more about their training
platform, ar-giri.com. Björn, thank you so much.
48 min
January 17, 2020 Letting Workers Qualify Themselves in AR, with AR Expert's Dr. Björn Schwerdtfeger
Getting future workers excited for the jobs they might have tomorrow can be challenging, especially when many young workers tend to enjoy challenging themselves with new tasks. Dr. Björn Schwerdtfeger says that AR training can allow those workers to qualify themselves for all sorts of tasks, and have fun doing it, to boot.
[Transcript coming shortly]
Alan: Hey, everybody, and thanks
for joining in on the XR for Business Podcast with your host, Alan
Smithson. I'm really excited today. I have Dr. Björn Schwerdtfeger
from Germany. He has more than 15 years experience in augmented
reality. Together with the German industry, he's evaluated almost
every idea for AR in applications in the industry. He's been a
co-inventor of Pick-By-Vision at TU Munich. And during that time --
when computers for AR glasses were still carried in large backpacks
-- Björn holds a PhD in industrial augmented reality as a serial
entrepreneur. And among other things, his company, AR Experts, is
advising about a third of Germany's most important production
companies, and is shaping their augmented reality roadmaps. You can
learn more about them at ar-experts.de. And they have another product
that they're gonna be talking about today. It's ar-giri.com. Björn,
welcome to the show, my friend.
Björn: Yeah. Welcome, Alan.
Nice to meet you. Nice to meet you online. I'm looking forward to
this podcast.
Alan: It's so exciting. The work
that you've been doing over the last few years -- like a decade and a
half -- is really starting to come to fruition now. I mean, all of
the hard work that you and your team have done to evangelize a
technology that -- let's be honest -- 15 years ago, the technology
really wasn't ready for the market. Tell us, how did you get into
this, into AR?
Björn: It was actually quite
funny, while still studying at the university, computer science, and
then somewhere else, augmented reality which popped up. And someone
had a demonstrator, where someone took some glasses and glued a
webcam -- we had external webcams earlier -- just hot-glued to some
glasses and using some [unclear] stuff and highlighting it. I think
it was just a cube. A virtual cube... And it was so fascinating that
you can bring this computer interface into the real world. Quite a
long time ago. But it was really nice.
Alan: Björn, did you say there
was a webcam hot-glued to a pair of glasses?
Björn: Exactly. That's how we
did augmented reality 15, 20 years ago.
Alan: Amazing. You are one of
the OG, the originals of this industry. You've been building and
advising brands and companies around their strategy for production.
What is the one thing in augmented reality right now that you've seen
the most ROI?
Björn: It's probably... we've
seen a lot of companies trying to do everything. Basically every
single one of us have tried to out, in the last three decades, and
failed with it. And we're figuring out what is actually the core of
augmented reality. And the core of augmented reality is not-- it's
not a measurement tool, it's not a tool for everything. It looks like
a display, and it is a good display. But where its core is, where
it's so good in, is in communication. It displays communication and
augmented reality is big. It's so much more close to your reality,
that perception is getting much better. So what you tried to
communicate with exosheets, nice PowerPoints; it's getting so closer
to the user with augmented reality. And they figured out that the
communication got so much better using augmented reality -- using
*good* implemented augemnted reality, it's quite important -- you can
do a lot of mistakes there. But this is helping so much. And that's
why you're seeing currently augmented reality mainly in marketing,
because marketing
48 min
January 15, 2020 Live from VRX, with XR Ignite’s Alan Smithson
Regular listeners will know that podcast host Alan Smithson is no stranger to the conference circuit, and is often asked to present or speak at the big XR expos. In a special episode of XR for Business, you’ll get to hear Alan in his element, as we present his opening remarks at this year’s VRX Conference.
“Well, thank you guys for joining. Again, welcome to the Blue Room at VRX 2019. My name’s Alan Smithson, and we’re gonna be talking today about the transformation of education using XR. I want to just quickly talk about MetaVRse. We’re building a platform for future-proof learning. And what that means to us is as more spatial computing technologies come on board, what we want to do is make sure that organizations — from training and enterprise, and also schools and organizations in high schools and universities — all have access to not only the content, but the platforms to let them make their own content. So what we’re building is a platform marketplace for technologies and content to grow.
We’re entering into the exponential age
of humanity. We’re hitting the point at which all of these
technologies converge together. So in the next 10 years, more wealth
will be created than all of previous human history. We’re entering
into an inflection point, where education systems are going to be
stretched beyond our wildest imaginations. Over the next 10 years,
more wealth can be created, but right now — currently — we’re
building a city the size of Manhattan around the world globally,
every single month. Yeah.
We’re going to experience massive
changes, from environmental changes, to job force changes, to
educational changes, all of these changes are happening to us at a
pace that we’ve never had before. It’s happening faster and faster.
And somebody said this to me the other day. They said “Today is
the slowest it will ever be.” It’s terrifying, it’s so fast. But
learning is required at every level, whether it’s skilled trades,
unskilled trades, whether it is retirees. We’re working on
technologies that will make people live to 150 years old. What are
they going to do? We need to rethink learning from a ground-up level.
All types of learning, whether it’s at work or at school, all of
these things that need a complete rethink.
Here’s a crazy stat: 75 percent of the
global workforce will be millennials by 2023. Who else is terrified
by that fact? Right? 120 million people need to be reskill,
retrained, and upskilled due to AI and automation in the next three
years. We don’t have the systems in place to deal with this. Two
trillion dollars, that is the global impact that VR and AR will make
over the next 10 years, by 2030. And this is an estimate by PWC.
So why is now the perfect time to get
into virtual and augmented reality for learning? So over the last
three decades we saw the rise of the personal computer and it took 20
years — 30 years, almost — to get everybody onto the personal
computer. And then we saw the rise of mobile, and that took about 20
years. XR is going to take about 10 years to go to global mass. So by
2030, we’re gonna be wearing glasses around and those glasses will be
inexpensive. They’ll be running on the cloud, so the compute power
won’t be on your phone or on your glasses. It’ll be in the cloud,
it’ll be all edge computing.
So we’re gonna see this massive growth.
And right now, we’re past the hype cycle. We’ve already seen proven
business use cases. We’re seeing real ROI being driven. And if you
look at the compounded annual growth rate of this industry, it’s
unprecedented. The only other industry that’s growing as fast is AI.
And it perfectly correlates with the global education market. This is
all of education, this is corporate training, this is K to 12. This
is all education. We’ll hit ten trillion dollars by 2030. It’s six
trillion now.
“Teach me and I will forget. Show
me and I will learn. Involve me and I’ll understand.” You know,
if you look at how we learn, if we read things, we retain about 5
percent. If we do them, we retain up to 75 percent. And VR, it shows
— and AR — hands-on learning is directly correlated one-to-one with
actually doing it. So we can create scenarios where people are
learning full job requirements with never having set foot on a work
site. Think about this for prisoners or schools, where you have
people that are there trying to learn a trade or a skill, they can
learn it before ever stepping foot on the worksite.
Now, XR and AI are kind of the most
efficient, effective learning systems we’ve ever created, and we’re
only starting to see this happen and come online right now. But we
anecdotally came here three years ago, and it was like “We could
use VR for training, we could use it for this!” But guess what?
Now the proof cases are there. We’re actually seeing real use cases
across 360 video with Strivr. We’re seeing people use this right now.
We’re using AR on our phones. There’s 2 billion devices that have AR
capabilities as of right now. This has a global scale to it. And then
your virtual reality and CG, being able to train people and put them
environments.
And VR has this amazing capability of
not adhering to space and time. You can be the size of an ant and the
size of a god in the same second. You can go back in time, you can go
forward in time. So being able to to transcend space and time using
this technology is something we’ve never had the ability to do in any
education learning format. The amount of data we can collect about
learning is obscene. When you start to have eye tracking, head
tracking, gait analysis, gesture analysis, hand tracking, pose
analysis, speech analysis, biometrics. If you take all of these data
points and then start to apply AI algorithms at a scale, you can
start to deliver hyper-personalized, hyper-contextualized learning,
to everybody, real time.
But the proof is in the ROI. It really
doesn’t matter for businesses, unless there’s an ROI. So you look at
Wal-Mart is seeing 90 percent reductions in training times. Sprint
saved 11 million dollars on one training application and decreased
their time to competency by 85 percent. UPS, is training all of their
drivers now in virtual reality, decreasing their training time by two
thirds, and also increasing safety. Delta Airlines reduced their
maintenance training by 90 percent. And I know Shelley [Peterson] —
is Shelley here? — Shelley has decreased their training times and
their time to completion at Lockheed Martin by 93 percent.
Our mission at MetaVRse is to
democratize education globally by 2040. We believe that those silly
boxes on these kids’ heads will turn into a sleek pair of glasses
that everybody wears. There will be super lightweight and it will be
running on the cloud. And at that point, it really comes down to
creating content at scale. So we believe that investing now in the
content platforms, and creating the standards by which the rest of
the world adheres to, I think is the real key to delivering education
at a scale we’ve never done before.
So I want to just say thank you to everybody for joining us today. We have a really amazing panel and I’d like to introduce the panel now. Come on up.”
8 min
January 15, 2020 Live from VRX, with XR Ignite's Alan Smithson
Regular listeners will know that podcast host Alan Smithson is no stranger to the conference circuit, and is often asked to present or speak at the big XR expos. In a special episode of XR for Business, you'll get to hear Alan in his element, as we present his opening remarks at this year's VRX Conference.
"Well, thank you guys for joining. Again, welcome to the Blue Room at VRX 2019. My name's Alan Smithson, and we're gonna be talking today about the transformation of education using XR. I want to just quickly talk about MetaVRse. We're building a platform for future-proof learning. And what that means to us is as more spatial computing technologies come on board, what we want to do is make sure that organizations -- from training and enterprise, and also schools and organizations in high schools and universities -- all have access to not only the content, but the platforms to let them make their own content. So what we're building is a platform marketplace for technologies and content to grow.
We're entering into the exponential age
of humanity. We're hitting the point at which all of these
technologies converge together. So in the next 10 years, more wealth
will be created than all of previous human history. We're entering
into an inflection point, where education systems are going to be
stretched beyond our wildest imaginations. Over the next 10 years,
more wealth can be created, but right now -- currently -- we're
building a city the size of Manhattan around the world globally,
every single month. Yeah.
We're going to experience massive
changes, from environmental changes, to job force changes, to
educational changes, all of these changes are happening to us at a
pace that we've never had before. It's happening faster and faster.
And somebody said this to me the other day. They said "Today is
the slowest it will ever be." It's terrifying, it's so fast. But
learning is required at every level, whether it's skilled trades,
unskilled trades, whether it is retirees. We're working on
technologies that will make people live to 150 years old. What are
they going to do? We need to rethink learning from a ground-up level.
All types of learning, whether it's at work or at school, all of
these things that need a complete rethink.
Here's a crazy stat: 75 percent of the
global workforce will be millennials by 2023. Who else is terrified
by that fact? Right? 120 million people need to be reskill,
retrained, and upskilled due to AI and automation in the next three
years. We don't have the systems in place to deal with this. Two
trillion dollars, that is the global impact that VR and AR will make
over the next 10 years, by 2030. And this is an estimate by PWC.
So why is now the perfect time to get
into virtual and augmented reality for learning? So over the last
three decades we saw the rise of the personal computer and it took 20
years -- 30 years, almost -- to get everybody onto the personal
computer. And then we saw the rise of mobile, and that took about 20
years. XR is going to take about 10 years to go to global mass. So by
2030, we're gonna be wearing glasses around and those glasses will be
inexpensive. They'll be running on the cloud, so the compute power
won't be on your phone or on your glasses. It'll be in the cloud,
it'll be all edge computing.
So we're gonna see this massive growth.
And right now, we're past the hype cycle. We've already seen proven
business use cases. We're seeing real ROI being driven. And if you
look at the compounded annual growth rate of this industry, it's
unprecedented. The only other industry that's growing as fast is AI.
And it perfectly correlates with the global education market. This is
all of education, this is corporate training, this is K to 12. This
is all education. We'll hit ten trillion dollars by 2030. It's six
trillion now.
"Teach me and I will fo
8 min
January 14, 2020 Digging Up Digital Cadavers in XR, with Sector 5 Digital’s Jeff Meisner
Today’s guest — Sector 5 Digital‘s Jeff Meisner — hopes to put grave robbers out of business, among other things. He pops in to talk to Alan about all the experiential learning experiences his company has developed, from digital cadavers to study anatomy, to the VR design process of Bell Helicopters.
[Editor’s Note: due to an uploading error on my part, this episode was previously released last week with the wrong audio. We’re re-releasing today with the correct audio. We appreciate your understanding, and in particular, Jeff Meisner’s understanding in this matter – Chris, Podcast Editor]
Alan: Hi, I’m Alan Smithson. And
today, we’re speaking with Jeff Meisner, CEO of Sector 5 Digital,
about their pioneering work on the Fantastic Journey Anatomy VR Ride,
Fork Lift Training Simulator, and the work they did with Bell
Helicopters, shortening design times from years to months. All of
this and more on the XR for Business Podcast.
Jeff, welcome to the show, my friend.
Jeff: Thanks, Alan.
Alan: I am super excited. So,
Jeff, you are doing some incredible work at Sector 5. Let’s start
with the Fantastic Journey Anatomy VR. Right. This just blows my
mind.
Jeff: Yeah. Yes. Just as a
historical perspective on this, we’ve been working with this
particular healthcare client for a couple of years now. And we
started out initially doing a 3D digital cadaver, basically, that
allowed them to do facial anatomy. And the company is in the business
of doing injections into the face and hand. And so they needed a way
to have safe areas so the injectors would have training. So we
created a basic virtual training tool and that was initially in 3D,
not in VR, but it was driven through our tablets and things like
that. So it had kind of an AR component to it.
Alan: You will learn in 3D
dramatically better than even just on a 2D screen.
Jeff: Yeah, exactly. And we
actually did a conference which had over somewhere between 200-300 of
their folks training with a massive 3D screen in front of them. So it
was used as a training aid, and really now, it’s gone global. So it
started initially in the U.S. and got picked up by this company,
because they are a global company. And what they wanted to do was
take that next step, if you will. And so We’re creating this, what we
call a VR Fantastic Anatomy Journey. We’re going to be taking their
folks through… well, if you know what Fantastic [Voyage] is —- as
most people do — but taking them through the human body. So you’re
going to have a really cool edutainment-type experience, whereby
you’re going to be on somewhat of a of a VR roller coaster,
although it being through the body, we’re going to be adding some
elements of teaching at various points. So it’ll stop and you’ll be
asked questions. It’s really, the major focus is to be very much a
learning experience. But one of the things we’re finding —- and I
know you are too, Alan —- is if you make it fun or people, it
becomes a much more memorable experience and they want to do it again
and again. We’re combining kind of that gaming-type element, if you
will, but with actual data and experience, to make it something that
their injectors are going to be learning from, and not just the
entertainment element.
Alan: When you guys started
rolling out the 3D digital cadaver, how are they measuring against
baseline? So, what was their baseline learning before? Just a
textbook? Or..?
Jeff: No, they were actually
using “live” cadavers, and cadavers — and this may sound a
little gruesome — but they’re somewhat hard to come by, especially
outside of the US. The regulatory issues that you deal with are very,
very high barriers there. When we came along with the virtual cadaver
initially, as I said, it was really only being used in a very small
area. But when they realized that they could take this globally, and
they now didn’t have the same barriers that they had in the past,
that really opened things up for them and opened up their eyes as to
the value that this would bring.
Alan: It’s really incredible.
Medical is the by far and away leveraging virtual reality more than
any other sector. I mean, design is kind of a close second. You’ve
also done some work with design, and we’ll get into that in a bit,
but the medical industry is just ripe for disruption. I mean, buying
a cadaver, like you said, is onerous, is expensive. It’s heavily
regulated. And let’s be honest, people don’t really need that anymore
with VR.
Jeff: Correct. And even if these
sorts of things are used in front of the intensive training —
obviously there’s regulatory bodies and things like that would demand
certain certifications and things like that — and in some of that,
we can do. But we also can be right in front of that to, again, make
these experiences a little more fun, a little more engaging. I will
have to tell you that one comment that we got from the training we
did was this lady actually said that our training was “more
realistic than the cadaver,” which I had to laugh, because it’s
like, how could that possibly be? But the fact that she thought that
this thing was so immersive that it was actually better than the
training on a cadaver, really spoke to me.
Alan: That’s incredible. That’s
nuts. But if you think about it, you only get one shot at a cadaver.
If you’re pulling it apart, you really can’t, like, split the face
apart and look inside the brain if you want. With VR, you can take
the whole skeletal layer right out. You can literally see the layers
as you need to see them, versus cutting into the skin. There’s
definitely, I think, for the physical training of the last part where
you have to learn how to inject or that sort of thing, you still need
that feel. You need to know how far you go down — there’s a feeling
to it that I don’t think will be replaced. But when you’re learning
about the anatomy, I don’t think there’s any better way; you can
literally just remove all you want. “I don’t want the body to
have any skin. Okay. There’s the bones.” This is really
powerful.
Jeff: And a lot of people now,
instead of what, in the past, you know, they had to fly doctors and
nurses, nurse practitioners into training centers. And the cost of
doing that was amazing. Now they can go to these areas, to these
doctors, set these things up, as you know. I mean, all you really
need is a fairly small technology setup.
Alan: Are you guys moving the
stuff to Quest now? Or are you still on Vives and Rifts?
Jeff: Yeah, we’re fairly
hardware agnostic, so we’ve done projects in just about any type of
hardware. I will say we do have the strongest partnership with HTC —
with Vive. We’ll get into that a little bit more when we talk about
the forklift project. But I mean, it really doesn’t matter to us. As
you know, the technology is changing so quickly right now. And so
we’ll look at it from a perspective of what is the best technology
available at the time for the client.
Alan: Let’s move from Fantastic
Journey Anatomy VR and let’s talk about your forklift training,
because you mentioned it a couple of times, and training people on
moving vehicles, forklifts, excavators, anything where they have to
drive around — it becomes really expensive. It’s kind of like
bringing a cadaver in; bringing people into a facility and letting
them drive around on a forklift, when they have zero experience, is a
little bit dangerous, and a little bit expensive. But imagine giving
them the ability to put on a headset, practice driving around a
warehouse, practice some close calls, maybe some things fall off the
shelf. You could practice scenarios that may happen in a warehouse,
but are very rare. But you can give people that real sense of
practice before they even step on the machine. Talk us through that
forklift training, and how that came to be.
Jeff: Sure. Sure. As I
mentioned, we have a really strong partnership with HTC Vive and they
were coming out with their new Vive Eye Pro headset, which has the
eye-tracking in it. And so we’ve been talking to them and we were
ourselves trying to think of the best demo that we could go, because
we had a conference coming up — the EWTS conference that was just in
Dallas fairly recently — and we were talking to them and we said,
when we do something that’s really going to be an enterprise-type
application, that’s going to take advantage of the new capabilities
of the Vive Eye Pro. And so we kind of went through project scope
with the folks at Vive and we came up with the forklift training
demo. What’s unique and different about this one is utilizing the the
eye tracking software of the Vive Pro. We take the user through a
simulation where they’re essentially inside a forklift. They have to
drive a forklift in a warehouse, and we give them some visual cues.
For example, if they look at the row of pallets that are in the
warehouse, one of the pallets, we’ll highlight in green. So then they
know, that’s where they need to go in and pick up that pallet. And
then we also have a loading dock area where there’s different, like,
A through F, I think loading dock locations. So once they pick up the
pallet, it’s highlighted where they need to drop that pallet off on
the loading dock. But in between that process, we are tracking all of
their eye movements. So, for example, if they pick up the pallet and
they don’t turn their head around to see what’s behind them? We’re
tracking all of that. And I think where it really hit home was we put
a replay function into the application. So someone will go through
this experience — and we timed it specifically to be about a three
minute experience because we knew we were taking it to a trade show
— but we did a replay in double time. So we had about a
one-and-a-half minute replay. And through the replay process, it
shows exactly where their eyes were through the entire experience.
Alan: Cool.
Jeff: Yeah.
Alan: What kind of insights are
you able to glean from that?
Jeff: Basically, we also — kind
of on the side — again, we made a little fun, made it into a game
with kind of a leader board. But you got points for doing things
right, and then you got points deducted for doing things wrong. So,
for example, if you didn’t turn your head when you were backing up,
or you didn’t see the cones that were in the warehouse — those sorts
of things — you got points deducted. So we can actually take a
visual which is showing exactly where the eyes were, and here’s where
you got points deducted, or where you got additional points for doing
things right. So it’s that training element that reinforces to them,
“oh, OK. Yeah, my eyes were down instead of looking up because I
had… I should have been looking up higher to the third row of the
pallets,” and things like that. So you have a visual
interpretation of what you’ve done, but you also have, combined with
a scoring system, to reinforce the points or reinforce what what you
did wrong.
Alan: That’s really impressive.
In one of the things that I think this will do for people is really
shorten the training times, because if you’re training on a real
machine and you don’t do these things, we have no way as a trainer,
or as somebody teaching you, knowing whether you did it right or not.
In VR with the eye tracking, now you can say, “hey, you didn’t
do this right. Do it again until you get it right.” And people
can repeat the training as often or as much as they need to perfect
mastery. Are you seeing a decrease in training times with this as
well?
Jeff: We haven’t gotten to that
point, because it is fairly new to now see, once they actually get on
the forklift, is that reducing the training times? But that’s a
definite goal, is to have those metrics. And as I said before, this
kind of can front end that initial training that people get, that’s
fairly boring and people are just, they want to get on the actual
forklift. Well, let’s do things ahead of that so that when they get
on the forklift, they’re not hurting themselves. They have the
concept. “I’ve got to turn my head. I’ve got to look.” You
know, those those sorts of things. So I think that’s yet to come. But
that is definitely the goal of this moving forward. The other thing
that we noticed, at EWTS — because we had over 100 different
corporations take this forklift demo in and try it out at the show —
what we noticed was a lot of them were saying, “yeah, we have
over a thousand forklifts of all different types,” because I
know that there has been some really immersive form of training done
in the past for specific forklift manufacturers. And that’s
fantastic, probably as a next step. The issue, though, is that a lot
of these corporations have four or five different forklift
manufacturers, so they have something a little more generic. That
kind of front ends up process is very valuable for them.
Alan: And I would think that
despite the fact that there’s 20 different types of forklift, safety
protocols are probably very similar regardless of the machine itself.
Jeff: Exactly. “Hey, you
need to follow your eyes. You need to be alert. You need to be
looking all around.” Those sorts of things are absolutely the
same, regardless of the type of forklift, or regardless of the type
of warehouse or materials. So that’s what really makes this exciting.
Alan: One other thing that I
thought would be really cool is making an option where it’s like an
open play version, where you can have fire come out the back of your
forklift and it can go really fast.
Jeff: [laughs]
Alan: Maybe some missiles or
something. No?
Jeff: I’m sure any studio would
love to add that, Alan.
Alan: [laughs] Yes. You’ve got
to shoot the boxes instead of pick them up.
Jeff: Yeah, there you go.
Alan: So you you recently
deployed 200 virtual reality headsets for a large airline. You want
to talk about that?
Jeff: Yeah. The exciting thing
about that, and I think this is part of the value that XR brings, is
that we did a project — and this is going back over five years ago
— we did a project for the world’s largest airline, whereby they
were placing, at the time, the largest commercial aircraft purchase.
And we modelled all of their business class and first class cabins in
3D. So we use these 3D models, we did a a website for them. We did a
kiosk — in-terminal activations — for them. And so these digital
assets that have been been used and been around for over five years,
we then took those digital assets into VR and we showed that off. We
initially did four headsets at their leadership conference and then
their global sales executives just went crazy over it. And they said,
“this is exactly what we need in Brazil and in Japan and in
Europe and all over the world, because when our sales folks are
sitting down with these corporate buyers and these buyers are trying
to decide if they want to travel on on this airline or another
airline, we can actually take these headsets and we put it into the
Oculus Go headset and we could take these headsets, give them to
these corporate-travel buyers and say, put this on. This is what your
executives would be experiencing if they were traveling on the new
777. Three hundred in first class or business class. And here’s the
bar. And they can explore the cabin and they can see how the seats
fold down and they can check out the workspace and all of those
things.” So that was really revolutionary to them, having that
experience and making it much more immersive. And it’s really
fundamentally changed the game for that.
Alan: They’re taking the
headsets and they’re going to the trade shows or customer meetings.
How are they using it?
Jeff: Yeah, they’re actually
taking them to customer meetings them. And now, as you know, the
barriers to entry with the technology has now come down to make this
opportunity available. And also the fidelity of the headsets
themselves have gotten so much better that they’re actually going to
meetings with these corporations, because — these are like global
2,000 corporations — that their executives are travelling millions
of miles on various airlines and international travel. And so they’re
actually taking that into those meetings, and not only telling those
corporate travel buyers about all the benefits of traveling on this
airline, but also they can actually put them in and make it a more
immersive experience.
Alan: So we went from training
people in forklifts to selling people on airlines.
Jeff: Yeah.
Alan: Oh, and digital cadavers.
Holy crap. So you guys are at the forefront of medical, industrial,
and sales. And there’s one more big one that… before we put a pin
on American — guess I can say what the company has since you said it
was the largest — American Airlines.
Jeff: Sure.
Alan: How are they comparing the
sales with and without it? Is there a way to do that, or are they
just anecdotally saying it’s better?
Jeff: Well, they’ve done their
own internal [research]. But unfortunately, for competitive reasons,
we’re not allowed to say, you know, how much. But, yeah, they
definitely had some kind of before-and-after, A-B-type testing as to
how it was before and the response versus how it is now.
Alan: One project that I know
you can talk about — because we talked about it previously, and I
know there was some massive savings being gained here — but that’s
the product you did for Bell Helicopters. Let’s talk about that.
These guys designed a helicopter and took it from years to months.
Jeff: Yes. And really, we had
been working with Bell for a few years. We had done a VR experience
for one of their military aircraft that had not yet… had not
actually been built yet. But it was a project that they were selling
to the military. And so we did it. We did a VR experience for them,
which took them on kind of a mission, if you will. But it was really
an eye-opening experience for the CEO of Bell. He’s a game-changer. I
mean, he is such a forward-thinking leader. And he really wanted to
transform Bell from a really historical, very engineering-centric
military helicopter manufacturer to be a leading-edge technology
provider of urban mobility solutions. And this was part of the
journey, was when he saw the response at this particular show. And
this show is called AUSA. It’s one of the world’s largest army show,
I believe. And when he saw the response there, he said, “we need
to be thinking more like the car companies and we need to be coming
up with concept aircraft and things like that.” And he
challenged his team — he had created an innovation team within Bell
— his team, and this was basically October. On the commercial side,
they had the world’s largest commercial helicopter show coming up in
March. And he said, “I want a concept aircraft on the show floor
in March.” And so basically from October to March, which ended
up being less than six months, we went from the sketches of the
aircraft; over 100 sketches to 3D models, put the 3D models into VR.
Actually had Bell’s test pilots put the headset on and give us
different changes back and forth of the aircraft. And we went through
that iterative process and came out the other end with only a single
1:1 scale mockup of this aircraft. And we were on the show floor at
at Heli-Expo in March. And oh, by the way, we also incorporated two
Microsoft Hololens experiences into that one; one for the pilot and
one for the passenger. But it was really revolutionary, in that it
was really one of the best use cases of enterprise XR — taking a
process that historically had taken them years to go through, and
multiple models, and to get to that point, and we shrunk that down
into less than six months.
Alan: Well, I mean, if that’s an indication, it looks like right across the board, you guys have been in medical, in industrial, in sales and marketing, and design. You guys have touched everything. Sector 5 Digital is kind of like this powerhouse of design in spatial computing. And I’m really excited for what’s next. So, Jeff, what is one problem in the world that you want to see solved using XR Technologies?
Jeff: I think it goes back to
the training. If we can take people out of the environments that are
hazardous — safety environments — and we can provide training to
them such that they can learn to the point that they no longer have
to be exposed into these environments, not only in a training
perspective, but have the knowledge and capabilities that eliminates
any sort of future hazards in those environments. I think that, to
me, is an area that’s absolutely ripe for this technology. And I’m so
excited about the future and how we can help people in those
environments down the road.
25 min
January 13, 2020 Living in a Post-Scarcity World of Technology, with You Are Here Labs’ John Buzzell
We live in a three-dimensional world, and according to today’s guest — You Are Here Labs president John Buzzell — our computers are finally starting to catch up with that. John shoots the proverbial breeze with Alan on how spatial computing is going to fundamentally change our relationship with computers, and thus, our relationship with the world.
Alan: My name is Alan Smithson,
your host for the XR for Business Podcast. Today’s guest is a good
friend, John Buzzell from You Are Here Labs and You Are Here Agency.
John is an award winning 28 year veteran of the digital industry,
creating interactive experiences across augmented reality, virtual
reality, video games, mobile apps and numerous high volume websites.
To learn more about You Are Here Labs and You Are Here Agency, visit
yahagency.com. John, welcome to the show.
John: Thanks, Alan. Good to be
with you.
Alan: And of all the people
we’ve had on the show, you have a lot of experience in this field. I
mean, you built the AR Porsche visualizer where you could drop a
Porsche right in your living room and I actually have a photo of a
Porsche in my living room from your app.
John: [laughs] That’s great. You
know, that was an interesting project, because we started off on the
Hololens and it was a really interesting project. But at some point,
Porsche said this is a little too future for us at the moment and we
need something that the dealers and the salespeople can use without
fear. And so when ARKit popped up from Apple and they said surprise,
now everybody with an iPhone 6 and above and use augmented reality,
it really changed the game. And we very quickly converted that
experience from the Hololens to the humble iPad and it took off from
there. So we were really excited to have one of the first ARKit apps
that was really connected to a major company or brand. And I’m glad
you liked it, too. That’s cool.
Alan: It was really special. Can
people download it now still?
John: Well, no, they can’t. That
was about two years ago that we did it. And for all of us in
technology, who knows how fast it moves. Porsche is a global company
and they were very impressed with the innovation. And I think they
were excited to kind of pull it back to HQ and see what they could do
globally with it. And also our clients left for jobs at other
companies simultaneously. [laughs] So–
Alan: That’s the challenge in
technology, you’re working on a project with somebody, you’re all in
it, and then they leave. [laughs]
John: I mean, I think that’s one
of the neat things about emerging tech is, is it really can help
vault peoples careers into the next dimension, in the sense that
these technologies are so profound and they will affect the work that
we do and the way we live our lives for so long in the future, that
people that have this experience, it’s really great for them
individually.
Alan: You’ve been doing this a
while longer than myself, but I’ve been in early VR since 2014. And
I’ve noticed that a lot of the people that were just building demos
and stuff like that, now are running huge companies. HP and
Microsoft, they’re running huge departments in this, just because
they were early and learned how to do it. And they learned in a time
when there was no YouTube video on how to make AR, you had to just
kind of guess.
John: Yeah. I mean, my career
resembles that, in the sense that I got started doing interactive
marketing on diskettes before CD-ROM. Our friend Cathy Hackl says,
“Don’t talk about that, it makes you sound old!” but I
think the experience is worthy, because you see things change to
CD-ROM. You watch them change again to narrowband Internet. You see
them change a third time with broadband. You watch it change again
completely with mobile, and then of course with social. And now on to
this. The people that do have the experience, I think have more of a
long view, a different perspective, where they don’t see AR, VR, or
XR. They don’t see it as an adversary or a competitor to things like
5G or IOT or artificial intelligence, machine learning, what have
you. They really see AR and VR as the screen, because if you look at
these other technologies, they’re all ingredients. None of them is an
interface, with the slight exception of perhaps AI and voice. If
you’re able to understand AR and VR, how it’s used, it can really
propel your company and your own career.
Alan: So is it safe to say that
XR is the window to emerging technology?
John: I think so, although I
think it’s fun that we’ve been looking at glowing rectangles since
the first movie back in the 1800s, and now computing is kind of
broken through that window, right? No offence to Microsoft, but we’re
not living in Windows anymore. This technology is aerosolized now and
it can show up anywhere, in any way. And that’s one of the really
exciting things about augmented reality.
Alan: So a lot of people are
calling it “spatial computing.” You want to maybe give it
your– explain to people listening, what does that mean when we break
free from this? You talked about Porsche taking the Hololens and
saying “This is really awesome, but it’s a little bit too out
there, too advanced for us.” And taking it back to an iPad,
which is great, because everybody has it and has massive scale. By
the end of this year, there will be over 2 billion AR enabled
smartphones and devices in the world. So now you’ve got scale. What
does it mean for spatial computing and 3D everything?
John: For so long, we have dealt
with scarcity when it comes to technology. In the 40s, 50s, up
through the 70s, you had to go to a university campus or something
like that to have access to computing. Malcolm Gladwell says that
part of the reason that Bill Gates was so successful — he and Paul
Allen — is because they had summertime access to computers at a
local university.
Alan: And their parents
fundraised and put a computer in their school.
John: Yeah, absolutely. I mean,
part of the reason I got into it is we had a computer club in my
elementary school. And so that was a concentration of computers and
people that liked computers. It’s more rare today, because we don’t
really have a scarcity problem anymore. We tell people don’t text and
drive and we put giant iPads in the Teslas. We, say, got computers on
our refrigerators and we can have Alexa powered microwaves, and
computers aren’t scarce anymore. And so the idea of having to sit
down at a desk in a home office and log onto the Internet, I mean,
there was a a joke about that in the latest Avengers movie. Or no, it
was Captain Marvel, I guess. In any case, when we live with
technology and we’re in a post-scarcity world, what does that mean?
That means I don’t have to go looking for a screen. I don’t go have
to go looking for a device. And much in the way that the phone has
been in our pocket, having migrated from the desk to our backpack or
briefcase and into our pocket and now on our wrist, this slow motion
merger of computers and our brains. The next step is for our eyes.
And for a while we’re going to hold a phone or an iPad out in front
of our face. And then when our shoulders get tired, eventually Apple
and others will sell us this integrated with a pair of glasses that
don’t look too nerdy. As Matt Miesnieks — I don’t know how to say
his last name — but he made the good point that for now, people will
get paid at work to look dorky in these devices. Eventually everybody
will wear them, because they’ll become fashionable. So I think from a
business perspective, so much of what we do is repeated process. For
a trainee or for someone who’s working a long shift, or if they’re
working in a critical application like medicine, having that
attachment and that immersion in process can be really helpful, to
know that “Okay, well, if I’m doing an organ transplant, how far
away is that organ?” or if I’m waiting on the curb to catch an
Uber for my next meeting, “How far away is that? Am I going to
be late?” Or if I’m in the guts of a warehouse or manufacturing
facility, and I need to know what machine to go to next, “What’s
the machine, and which direction do I need to head?” So this is
something that we’re all doing right now, and it feels pretty
seamless to pull the phone out of our pocket or look down at a
tablet. But we’ll probably look back in just a few years and and
laugh at how quaint that seems, because now we’ll be getting it right
in our field of view.
Alan: It’s crazy. My friend
showed up at my house with a BlackBerry and I was like, “What is
that?” [laughs] We’re gonna be looking back. And my guess is ten
years, maybe 20. And we’ll say, “Do you remember the time when
we used to hold these little square boxes, and carry them around in
our pockets all the time?”
John: Well, yeah. And I think
people naturally react with a healthy amount of skepticism for this,
because a lot of people have just now gotten adjusted to smartphones.
But it’s funny you bring up BlackBerry. I had a BlackBerry 10 years
ago. It was a smartphone, had a screen and a bunch of keys on it. But
yeah, it feels so antiquated now. All the technologies that we would
need for these glasses exist today. It’s not able to be made cheaply
enough. Or perhaps the battery life wouldn’t be able to be as long.
But if you wanted to have a next-gen experience for a few minutes,
that experience can be had. So it’s really a question now is do you
want to subsidize as an OEM these devices to make it more affordable
and to spur adoption? Or do you want to kind of squeeze a little bit
more cash out of the current category, the way that smartphone
manufacturers are? I think we’re on the cusp of a change there.
Alan: I think you’re seeing it
with product like Hololens 2, where they could have brought the price
down, for sure. I mean, it doesn’t cost them $3,000 to make this
thing. And maybe they said, “No, we’re gonna keep this at an
enterprise price of $3,500.” And I think it’s the right thing to
do, because people think “Oh, it’s too expensive.” Well,
this isn’t for everyday use. This isn’t for somebody playing video
games. This is for industrial applications where you can either have
remote assistance, see-what-I-see training on the job, instruction
manuals, that type of thing, which are driving real business values.
Jonathan Moss from Sprint was talking about how they used just tablet
based AR for training. And they’ve been keeping some different KPI
metrics, and they’ve made millions of dollars in sales and they’ve
saved millions and dollars in travel, simply by using this AR
education. They’ve been tracking it. And I said, “Well, how much
did it cost?” And he said, “Oh, between a 100 and 200
thousand.” It’s astronomical numbers in savings and profit here.
So I want to dig into a little bit more of the industries and
companies that You Are Here Labs is serving and what you guys are
doing. You want to maybe talk about the different industries and
companies that you’ve been serving lately, and what you guys are
working on?
John: Sure. And if I can try to
tie into your last comment, we really have that same practical world
view. You and I were joking before we started recording this podcast,
that we had better not get into all of the technical gobbledygook
that so many people are very precious about with these devices. The
devices are coming out constantly. Some of them are even being
subsidized to spur increased adoption, and they’ll continue to come
out for some time. I mean, people are getting new TV’s all the time
now, whereas previously they held onto TV’s for a decade. So we take
the long view. We focus on the practical side. We see if this
technology is going to be around for more than twenty five years,
what do you do? Because buying a consumption device isn’t going to
get it done. Buying a guitar doesn’t make you a good guitar player,
you’ve got to practice and prepare. So we work across industries
really quite a few, including automotive, commercial real estate,
construction, consumer packaged goods, energy and oil, food and
beverage, heating and cooling, manufacturing tools, transit, a whole
bunch of different industries. But we’re really try to pull one
thread through — and I think your audience will like this — which
is that we focus on delivering results quickly and over time. And
what I mean by that is that we help companies understand what these
technologies are. We help them explore how they fit into their
business, including the critical applications that their workers go
through. And then we help them figure out how to integrate and scale
those solutions over time in responsible ways. Because all of us,
you, me, and everybody else that’s been on this podcast and more,
we’re all stewards of this fledgling medium. And we want to see it
succeed, not just for us as individuals, but as a whole. And so
hopefully that answers your question.
Alan: Absolutely. And you kind
of touched on something that really resonates with me, especially
with this podcast. I do this podcast just out of a labor of love to
try to promote it and give people that are listening the idea of, I
can invest in this and it will give me a return, because I think
there’s been so much hype around VR and AR for gaming and for this.
And oh, we’re in the trough of disillusionment. We’re not in the
trough of disillusionment. If you, three years ago, put a million
dollars into a company expecting they were gonna be a billion dollar
company by now, yeah, you’re disillusioned. However, if you thought
we’re going to invest a million dollars and start to solve real
problems within industry, you’re doing all right right now.
John: Yeah.
Alan: And you have clearly
figured that out. And we did the same thing. We took this view of,
what are the results we can deliver now versus in the future. Caspar
Thykier from Zappar got a really great point. He’s like, “Yes,
we can talk about WebAR, we can talk about when glasses come, we can
talk about all these future things. But why don’t we just make things
that are existing and capable right now?” The technology that
exists right now in AR and VR is so spectacularly amazing, that we
should be focusing on it now, not a year down the road or five years
down the road.
John: Yeah, absolutely. I mean,
if you’re in an industry where you have physical objects, widgets,
car tires, surgical equipment, tractors, anything really at all —
anything that’s not kind of abstract or ephemeral — then you need to
be investing right now on the tools and the skills to translate that
into digital. The web was kind of quaint until digital cameras and
flatbed scanners got inexpensive, and then suddenly you could have an
eBay, because you could show grandma’s old jewelry that you wanted to
sell. Although that’s kind of sad that you’d sell grandma’s jewelry,
but you could. [laughs] And we’re in a similar space right now with
XR, in that the tools for creating a digital version of physical
objects have never been cheaper. They’ve never been easier to use.
And a lot of businesses spend tens, hundreds of millions of dollars
schlepping around big, heavy, dangerous stuff to trade shows, to
customer events, to do demos. And it doesn’t have to be that way
anymore. There is a web conference style transformation going on,
where you can configure and sell a car without the car.
Alan: Yep.
John: You can imagine how dental
equipment would go into a dental hygienist office, without bringing
any of the equipment. You can decide how much concrete you need for a
giant office complex, without any surveyors. So there are so many use
cases now in business where people can start saving or making money
with AR and VR. I would love to see more people embrace that.
Alan: The great thing is there’s
lots of– when we started a few years ago, there was nobody that
could do this stuff. Literally nobody.
John: Right.
Alan: Colleges are starting to
roll out programs, and there’s a lot more information online. So
people are learning it. And even companies. What we’re doing now,
we’re finding companies want to bring a lot of the stuff in-house.
And so what we’re doing is consulting on how can they build the team
without building a huge team, or whether you need a 3D model or maybe
you need a Unity expert. What are some of the people on your team
that built out your round team, that you guys use on a project like
the Porsche one?
John: Yeah, I mean, I think the
work — like you said — has gotten– for those of us who have been
in the industry for a little while, it’s certainly gotten more
strategic. We’ve moved from an era of “Can we do that?” to
more of an opportunity to ask, “Should we do that?” And so
we definitely have a lot of people that speak XR strategy here in our
team, that can consult with various companies in their embrace of
spatial computing or XR or whatever you want to call it, to help them
find the best opportunities to do first. And the ones to save for
later. And so that’s an important part of what we offer. Similarly,
we have some very bright software developers, people that come from
the game development industry that understand the different engines
— Unity and Unreal being the two most popular — and try to work
that across different devices. We do projects on the Go. We do them
on the Hololens. We do them on the Vive and Rift. We do projects on
iOS and Android. We do projects on the Magic Leap. We really– we
don’t specify devices to people. So you need a versatile team of
developers for that. We have technical art directors that — for
those maybe who don’t understand game engines as much — there’s a
real skill into doing special effects — whether that be lighting or
texturing or particles — to make augmented reality graphics fit into
the real world better. You’d be surprised how hard it is. And then
finally, we really rely on a group of engineers to do 3D scanning,
volumetric and photogrammetry capture, project management and QA. So
it’s a lot of the same roles from other types of software
development, but with some specialties as well.
Alan: I get this question all
the time. Who do I need on my team for this? And my first reaction is
“just hire us and we’ll deal with it for you.”
John: [laughs]
Alan: The second thing is you
need a big team. You need somebody who understands Unreal or Unity.
You need a 3D modeller, you need somebody that understands the
textures. You mentioned making things look real in the real world.
And you’ve entered into the spatial computing era where we’re not
creating something on your phone. We’re creating something on your
phone that has to look real in the real world, despite the different
leading changes and that sort of thing. So if you have your car
sitting in a parking lot next to another car in the shadows, pointing
one direction for real and the other direction, because that’s how
you built it. That’s really weird.
John: You’re talking about a
level of polish that’s possible that really makes apps shine. If
you’re using this for business and you’re trying to sell a bulldozer
or you’re trying to teach someone how to repair a bulldozer, if you
can make it look real, your trainee or your customer, your prospect
can forget that they’re looking at a simulation and focus on what
you’re really trying to tell them. We put great care into making sure
that things look as real as possible, so that you keep that
suspension of disbelief, like they talk about in the movies.
Eventually, this stuff will be so easy, it’ll just happen magically.
You don’t have to worry about lighting or animation. But for now,
there’s a bit of skill and that’s where it can help to partner, as
opposed to having somebody on your team.
Alan: I really love what you
just said. People, because it’s so real, they can forget that it’s a
simulation and focus on the key messaging. And that’s so vital for a
number of things, sales and marketing, but also training and
upskilling. You’ve done a number of things here. What are some of the
real life data metrics, analytics, specific KPIs? What have you guys
seen as those things? How are people measuring their success?
John: Yeah, I would put that–
I’ll actually answer that in two parts. There’s what our customers
care about. And then there’s what we’re looking at. I think, for the
customers, they measure success in a variety of ways. For people that
are still struggling a little bit with their digital transformation,
maybe they finally got their mobile app out recently, or they’re
proud of their website, which is great. Everybody needs to get there.
Success for them can be as humble as just executing a proof of
concept, or pulling together innovation budget from other parts of
the company. It may be getting some employee or market validation. I
would say for the intermediate clients, that they’re maybe comparing
XR solutions to other methods, comparing traffic or lead generation
and retention. That’s what they’re worried about. And then our
advanced clients are really beginning to unearth deeper insights,
based on usage data from these experiences. And that kind of more
closely mirrors what we look at, which is we’re looking at numbers of
users, length and depth of engagement, repeat use. Are they sharing
this experience if they can? Does that converge to lead capture or
commerce for enterprise training? The names are different, but it’s
pretty similar that they’re looking for higher enrolment, time on
task, quality, how well they’re scoring, and what do they retain. So
we use, behind the scenes, everything from head position, eye
tracking, looking at the difference between where they’re holding a
VR controller to where they may end up, as a measure of kind of
intuition. It’s not as tight and concise as it is in other media yet,
but it’s moving there fast. And I think as different companies
embrace these technologies, they’re getting more sophisticated ways
to measure success. Does that answer your question?
Alan: Yes. I have literally
nothing else to add.
John: [laughs] We’ve hung out
too much, right?
Alan: We really have. I want to
talk about specifics. Give us an example of a case study that you
want to share.
John: Apologies in advance, I’m
going to have to kind of thread the needle here to not mention
specifics, but I think there’s some lessons to be learned. We’re in
phase two of a project or a major infrastructure organization, and
they deal with almost 10 million citizens per day in the execution of
their service. And a lot of the equipment that they use to serve
these people is antiquated. I mean, some of it is older than 50 years
old. They’re in a real situation where they need to improve the way
they do things, but they also need to continually replenish the staff
that services this infrastructure, because some people are retiring
and it gets expensive to keep them around. It’s nice work if you can
get it, though. So in any case, we were brought in to create training
materials using augmented reality and virtual reality, but there was
no digital objects to start from. And just like we said earlier, that
web got pretty great when you had digital cameras and cheap flatbed
scanners. There’s finally technology to use to digitize real objects.
And so we got in, scanned a lot of these objects, digitized them,
made them ready for mobile devices, delivered over 3G, 4G, 5G, Wi-Fi,
what have you, and started assembling lessons, working with their
subject matter experts. They have a training program right now that
takes a half a year. And we think we can get a really serious
reduction in that. And we’re starting to see really promising results
in early trials. So for them, it was pulling them out of the 1900s
and really preparing them for the next hundred years. Smart training
can be delivered on any device, in any location, and really across a
range of different skills. So we employed a lot of versatility. We
had to be very nimble on this project to react to different changes.
And we learned a lot. I think they did, too.
Alan: What are some of the early
metrics? Because we’re seeing decreases in training times, dramatic
decreases. You mentioned six months training. My guess is you could
probably get that training down to about 45 days using this
technology.
John: It’s really dramatic.
There’s– I’m going to mangle this old adage, but it’s something
like, “I remember a little of what I see, less of what I hear,
but I remember almost everything that I do.” And in that way,
these people can gather around– currently gather around a big, heavy
piece of equipment. It takes two hours to take it apart. Not
everybody can see what’s going on. And maybe they’ll get a chance to
ask a question and it better be a good one. But with this technology,
everyone can be there all at the same time, moving at their own pace,
asking questions, looking at things from any angle on their own. And
no one has to scratch up their knuckles or injure themselves. Nobody
drops a 400 pound piece of cast iron on their toe. We’re seeing a lot
better retention. We’re seeing faster moves through the curriculum,
with people being able to go through it more often. So I agree with
you. We should be getting hard numbers on that soon. And if I can
share them, I will. But at the moment, we’re already starting to see
giant gains from an industry that’s been doing things the same way
for almost 100 years.
Alan: I get excited about this,
because I see that this type of technology as being the thing that
democratizes education across the world. We’ve got smartphones which
are doing a fantastic job providing the information quickly, but
immersive technologies have been able to do something, and also just
see it in three dimensions. You mentioned being able to see a machine
or whatever and pulling it apart. When you’re in virtual reality and
you make a mistake, there’s no consequence that anybody else can see.
You make a mistake and you can make as many mistakes as you want. And
humans learn through error, we learn by making mistakes. Being able
to make mistakes in a completely private and consequence-free
environment, that reinforces learning at a different level.
John: Yeah, and not only that —
which I love your point — but in addition to that, the software can
be watching you and making suggestions. “We’ve noticed you’re
having a little trouble with this. Would you like to go back and
repeat this part of the training?” There was a particular thing
that we did on this last project, where you were supposed to take
apart a complicated system of parts that all went together in
different ways. Some were threaded, some were slipped into place,
some were bolted down and watching people being — to your point —
being able to try to figure this out. They were learning in a way
that a classroom lecture or a video would never get done. Education,
whether that’s educating somebody about your product, or educating
employees about working with central equipment, or educating
practitioners about compliance and safety, it’s all communication.
And one of the reasons that I’m so amazed and in awe of this
technology is it’s really bringing together all of the progress that
I’ve seen over the course of 30 years working in the industry. It’s
really going to be profound for people.
Alan: I know this is a question
that I get from listeners all the time, and it’s a simple one, how
much this stuff cost? What is our initial outlay? And maybe instead
of just saying this particular one costs, let’s talk about how to
budget, like how can a company from the first minute they meet with
you to rolling out some project like this, what are kind of the steps
and what does the process look like from your standpoint, that you’ve
seen work really well?
John: There’s an XR or AR/VR
solution for every budget. And I’m not saying that as a dodge, or to
be slippery in any way. I think that… look, if your budget is
$5,000, pay somebody that’s an expert in the industry to come talk to
you for a little while. Have them explain their perspectives on the
industry, have them maybe do a little bit of light brainstorming with
you for use cases that make sense for your industry or your company,
your category. If you have $50,000, maybe think about doing a proof
of concept, where you ingrain yourself with real requirements. You do
either a lightweight series of experiments on a particular idea, or
maybe you create a horse race, where you take three different
experiments and you try to see which one is most successful and then
learn from why. If you have $250,000, you’re probably further into
it, having already spent money at the lower levels. But that’s really
when you want to start thinking about doing an integration test or
maybe scaling up a team. And of course it goes on from there. People
can– a buddy tells me, “We can make this as complicated as you
want, John.” [chuckles] But yeah, I think there’s ways for
people to get involved. The most important thing is to understand
that we’re visual creatures and we live in a physical,
three-dimensional world. Computers can finally live alongside with
us, and that can bring just-in-time education, or marketing, for so
many different things. And we don’t have to look at a computer and
try to figure out, “Well, where’s the button for this or that?”
You know, there’s this is great scene in The Matrix from 20 years ago
— which is kind of amazing because it still feels futuristic —
where the main character, Neo, kind of laughably plugs something into
his head and he says “I know kung-fu,” but the idea of
just-in-time education is already here. I mean, if you look at how
many lives an AED — an Automated Emergency Defibrillator — how many
heart attack victims have been saved with those devices, because
somebody got just-in-time education? We can all walk around being
just-in-time experts for any number of things. Administering first
aid, or teaching somebody how to work for a particular problem. And
we’re going to be able to deliver that in a way that’s more seamless
and more compelling than ever before. And I think that’s– you need
to think about your business in a way of like, what is a real
business problem or delivering just-in-time training or education
along with 3D objects would be helpful? And there’s probably a whole
lot of them.
Alan: What is the most important
thing that businesses can do right now to leverage this power of XR?
John: Companies need to do more
— and I’ll get specific in a second — but if you’re not already
spending time or money or both on XR, you’re helping your
competitors. The largest companies in the world have decided that
this is what comes after the smartphone. They’ve seen the smartphone
sales start to plateau. People probably aren’t going to pay more than
a thousand bucks for a smartphone. So what are they going to do next,
to keep us all buying new devices? And if you look at IoT, AI, cloud
computing, and big data crypto, if you look at a 5G — all the
technologies out there — they’re mostly ingredients. We’re visual
creatures. We need a screen and the AR, and VR, XR, spatial computing
— whatever you want to call it — this is how we’re going to
interface with the future of computers.
So companies need to do more.
Businesses need to recognize that this is really serious for their
marketing and training, but also kind of their workplace tools,
workforce development. It’s not a competitor to any of these other
technologies. It is what brings them all together. So if it’s the
evolution of computing and the successor to the smartphone, if you
haven’t started experimenting with this tech yet, you’re falling
behind. So I think if you’re a beginner, you need to do more, attend
to conference, hire one person, do a proof of concept with a local
agency. If you’re intermediate, maybe strengthen your teams and your
partnerships. Try to figure out, “well, OK, so we have a lot of
3D. Let’s scan some something and see what we can do with it.”
If you’re an advanced user, people need to do more integrations, need
to polish their skills, build their teams because the future is going
to be 3D. It’s going to be contextual and it’s going to be spatially
aware. So I think just simple answer would be to more than what
you’re doing now, because this is rapidly approaching. And I see
across industries, companies that are probably competitors of your
listeners already investing lots to learn how to make the most of
this tech.
Alan: This technology; once you
try it, you unlock Pandora’s Box. You’re like, “oh, wait a
second, we just saved $100,000 not flying people around [the world].
We could have a meeting and it was more productive because people
can’t be looking at their smartphones while they’re in VR.
John: I think that’s the best
business case for XR, honestly; shortening and improving logistical
challenges for companies. When the web came out, people in the
magazine and trade show industries were pretty fearful — and with
good reason. And you know, there’s still trade shows and there’s
still magazines, but people do a lot of business online. Similarly,
the opportunity — and there’s some companies with products already
in the market for this — the opportunity to work across devices and
across distance and time to allow people to collaborate and not just
have a web conference where you’re looking at slides, but really, to
manipulate. “What if we put this thing over here? What if this
was smaller? Can we make this out of carbon fiber?” We have
those experiences in our lab. Companies like Spatial or Glue. You can
go take a look at the future for that right now. And I think it’s
going to be profound. You won’t have to go to the office to work. You
won’t have to travel to Hong Kong to have the meeting. You won’t have
to go to Palo Alto to have a design session. You’ll be able to just
put on a headset or hold up a device and do it right there. It’s
happening today and it just hasn’t been deployed at scale.
Alan: We’ve only just unlocked
it and being being able to present and bring knowledge around the
world without having to get on a plane to travel, because let’s be
honest, travel’s fun business travel not so much.
John: Yeah. I mean, I think
you’re already living in that future. There’s this great quote from
William Gibson, a fantastic science fiction writer, and he says The
future is already here. It’s just not evenly distributed yet.
Alan: It’s true.
John: You’re already living in
that future. Right.
Alan: And to be honest, let’s be
fair. There are still challenges. It’s still not the perfect solution
yet, but it’s very close.
John: If you can think of how
many web conference tools there are out there, from Blue Jeans to
Hangouts to WebEx. Slack has theirs; Skype has theirs. There are
going to be so many of the companies that can do this. Like I said,
we have it running in our lab. It is going to be transformative for
people when it comes time to renew your expensive office lease. And
you’ve already got maybe 30-40 percent of your workers working
remotely. You begin to think of, “gosh, is it worth all that
money per square foot?”.
Alan: Yeah.
John: I’ve had conversations
with people in the banking industry that said, “you know, what
about virtual branches? We spend a lot of money on branches. Could we
start off in VR chat where you’re interacting with an A.I. and then
you get escalated to a real person on the other end of the line?”
Absolutely. The technology exists. It’s just spreading out.
Alan: Absolutely
John: So that’s exciting.
Alan: So my final question, what
problem in the world do you want to see solved using XR technologies?
John: Well, I said earlier —
and may have spent one of my good answers on The Matrix example —
but having just-in-time education where average human beings like us
could be activated or mobilized to do super-heroic type things on
demand. I would love to see that, because humanity needs a lot of
help right now. But, you know, I have a couple daughters, and maybe
closer to home for me is with all the money that’s spent moving our
carbon bodies around from home to work to an airport to another place
to another office. I would really love to see spatial computing and
XR helping with climate change. I think that logistically we’re
smarter than… we’re still operating with 19th century technology to
get around in a lot of ways, and we can do better. And I think that
XR offers a chance for all of us to be more efficient and more
powerful on what we do.
38 min
January 13, 2020 Living in a Post-Scarcity World of Technology, with You Are Here Labs' John Buzzell
We live in a three-dimensional world, and according to today's guest -- You Are Here Labs president John Buzzell -- our computers are finally starting to catch up with that. John shoots the proverbial breeze with Alan on how spatial computing is going to fundamentally change our relationship with computers, and thus, our relationship with the world.
Alan: My name is Alan Smithson,
your host for the XR for Business Podcast. Today's guest is a good
friend, John Buzzell from You Are Here Labs and You Are Here Agency.
John is an award winning 28 year veteran of the digital industry,
creating interactive experiences across augmented reality, virtual
reality, video games, mobile apps and numerous high volume websites.
To learn more about You Are Here Labs and You Are Here Agency, visit
yahagency.com. John, welcome to the show.
John: Thanks, Alan. Good to be
with you.
Alan: And of all the people
we've had on the show, you have a lot of experience in this field. I
mean, you built the AR Porsche visualizer where you could drop a
Porsche right in your living room and I actually have a photo of a
Porsche in my living room from your app.
John: [laughs] That's great. You
know, that was an interesting project, because we started off on the
Hololens and it was a really interesting project. But at some point,
Porsche said this is a little too future for us at the moment and we
need something that the dealers and the salespeople can use without
fear. And so when ARKit popped up from Apple and they said surprise,
now everybody with an iPhone 6 and above and use augmented reality,
it really changed the game. And we very quickly converted that
experience from the Hololens to the humble iPad and it took off from
there. So we were really excited to have one of the first ARKit apps
that was really connected to a major company or brand. And I'm glad
you liked it, too. That's cool.
Alan: It was really special. Can
people download it now still?
John: Well, no, they can't. That
was about two years ago that we did it. And for all of us in
technology, who knows how fast it moves. Porsche is a global company
and they were very impressed with the innovation. And I think they
were excited to kind of pull it back to HQ and see what they could do
globally with it. And also our clients left for jobs at other
companies simultaneously. [laughs] So--
Alan: That's the challenge in
technology, you're working on a project with somebody, you're all in
it, and then they leave. [laughs]
John: I mean, I think that's one
of the neat things about emerging tech is, is it really can help
vault peoples careers into the next dimension, in the sense that
these technologies are so profound and they will affect the work that
we do and the way we live our lives for so long in the future, that
people that have this experience, it's really great for them
individually.
Alan: You've been doing this a
while longer than myself, but I've been in early VR since 2014. And
I've noticed that a lot of the people that were just building demos
and stuff like that, now are running huge companies. HP and
Microsoft, they're running huge departments in this, just because
they were early and learned how to do it. And they learned in a time
when there was no YouTube video on how to make AR, you had to just
kind of guess.
John: Yeah. I mean, my career
resembles that, in the sense that I got started doing interactive
marketing on diskettes before CD-ROM. Our friend Cathy Hackl says,
"Don't talk about that, it makes you sound old!" but I
think the experience is worthy, because you see things change to
CD-ROM. You watch them change again to narrowband Internet. You see
them change
38 min
January 10, 2020 The Sound of XR, with Bose’s Michael Ludden
Your various realities — virtual, augmented, X, etc — are often talked about in the realm of vision, since we humans lean on vision as our major sense. But the folks at Bose, like today’s guest Michael Ludden, know that there’s room for sound in XR too.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today’s guest is
Michael Ludden, global head of developer advocacy and principal
augmented reality advocate at Bose Technologies. Michael is a
technologist, futurist, strategist, product leader, and developer
platform expert who loves to operate on the bleeding edge of what’s
possible, and is a frequent keynote speaker at events around the
world. Michael was previously director of IBM’s Watson’s Developer
Lab for a AR and VR, among some other career stops. To learn more
about the work he’s doing at Bose, you can visit developer.bose.com.
Michael, welcome to the show.
Michael: Wow, what an intro.
Thanks for having me.
Alan: It’s my absolute pleasure
and honor to have you on the show. I’m super excited. I was talking
to all fine and last week. I was flying from Toronto to San
Francisco, and I just happened to sit beside a guy who we started
talking about AR and I pulled out the North Glasses. He pulled out
the Bose Frames; we swap. And we had this kind of meeting of the
minds. I had the visual, he had the audio and it was really cool that
I got to try the Bose Frames. What an amazing piece of technology.
Michael: Glad you liked it.
Alan: So you’ve had a storied
career here. You’ve done everything from IBM Watson, to Google, to
HTC, Samsung. How did you end up in technology, and why did you get
so fascinated on futurism?
Michael: Well, it’s sort of been
a running theme in my life. I read a lot of science fiction as a kid
and I was always interested in technology and — not to date myself
— but at a certain point in my life when I was a young adult,
technology started to really aggressively eat everything, starting
with mobile. And I just found that was really the point of inflection
in my life where I studied musical theater in college, I went to
UCLA. I thought that’s what I was going to do. I really did. And I
did get a B.A. so I got a little arts education, too. And at the same
time, I was always tinkering with stuff, building my own PCs. I
started my own web development company at one point to make Web sites
in Flash, CS2, and CS3 in the early days; it was brutal.
Alan: There’s a conference in
Toronto called Flash in the Can; FITC.
Michael: Nice.
Alan: That’s old school.
Michael: It is very old school.
And, you know, I never really thought I’d make a career out of it,
but I needed money. I was a starving actor in L.A. and one of my
friends who I just made by being nerdy, worked for a company called
HTC. They were releasing the first-ever Android phone, which was
called The Dream — or the G1 in the US. So I was in contact with
this guy; he got a promotion. He said, “you should take my old
job,” which was L.A.-based, and I was living there. And I said,
“I want to do it.” I was working on a podcasting platform
called This Week In — not This Week in Tech — but This Week In. It
was a Jason Calacanis-led network out of the old Mahalo Studios in
Santa Monica. But it paid me pennies. And when they told me what the
job paid and what I’d be doing, I said, “OK, I guess I’ll do
it.” I needed the money, and it was very flexible. It felt
really easy to me, like that’s really all you need me to do.
And so I ended up starting to go around
door to door. What was it like? T-Mobile shop, Verizons shop, AT&T
— like, carrier stores — and show them about the phones. And I was
like, this is so easy. And then they started sending me to
conferences. And I started giving talks about stuff and doing the
demos at the booth. And then I got wind that they were starting a
developer relations organization. I have no business even applying
for something like that. But I really wanted to have a forcing
function to teach myself a program for Android, learn Eclipse, etc.
It was Eclipse at the time. And so I basically begged the team for
six months. I just let my enthusiasm kind of guide me, and the rest
of the stuff you mentioned in my past kind of followed from that
enthusiasm. It really actually hasn’t stopped. It’s just sort of
morphed towards different things. The latest thing is obviously
immersive technologies, but I found that they valued my time and it
didn’t feel like work. What more can you ask for from a career,
really?
Alan: Isn’t that awesome? So
let’s fast forward you’re now working with Bose, one of the giant
pioneers of audio in the world. What does Bose have to do with
augmented reality?
Michael: Well, that is a great
question. Yeah. So Bose is doing something really fascinating and
innovative, and that is trying to create a new lane for augmented
reality that is focusing on a different sense that’s not your sight;
that’s sound, which I guess is pretty brand-appropriate. We are
building a whole new area of Bose focused on turning the company into
a platform company and building a platform for developers to build
mobile applications that are sound-focused, that operate via a series
of gestures and spatial sound capabilities, and possibly even voice
input, and prioritize those over staring at your phone screen and
touching, so that you can maybe do things by those methods of
interaction while having your phone in your pocket, even though
there’s an app running. In that way, it actually frees up your visual
to just like if you’re listening to a podcast passively, or music, or
you’re having a phone call actively. [You can] still engage in the
world with your eyes, our most powerful sense and let audio give us
some other capabilities that complement that. If that makes sense.
Alan: Yeah. So let’s put this in
perspective. You guys made a pair of sunglasses, that speakers built
into them. Pretty awesome. The sound is amazing, like what you would
expect from Bose. Now, not only do you have the sound just stereo,
but you also have the ability to make it spatial, correct?
Michael: Yes. The frames are
just one of three devices that support Bose AR at the moment. The
QC35-IIs that are the most popular ones. Every time you pass a
business class on an airplane, everybody’s wearing them. Those
support Bose AR as well, provided they were purchased after November
of last year, 2018. Same thing with the new 700 series we announced a
couple of months ago. I actually got the pleasure of announcing this
on stage at AWE — Augmented World Expo. They’re called the Noise
Canceling 700 series. They are a new super-premium tier of
noise-canceling headphones that’s sitting alongside the QC35s. All of
these devices have a little sensor bundle in them that’s on the right
side of the frames, and that consists of an accelerometer, a
gyroscope, and a magnetometer. And in that way, we have an SDK for
Android, an SDK for iOS, and then also an SDK for Unity, which you
can use to deploy cross-platform that will allow you to interpret and
read data from the sensors to do things like recognize gestures. For
all three platforms, we have native gestures, support for of course,
head nod. So if you’re nodding your head, yes. Shake, negative. So
we’ve changed the word from — sorry — nod to affirmative and shake
to negative, just to sort of future-proof it a little bit with an
intense system. But the idea is, if you want to make an application
that does something when somebody shakes their head, or you prompt
them to shake their head or not their head, and something happens,
you can do that.
And then also there’s input, which on
frames you double tap on the right. Same thing with QC35s and on the
700 series, you touch and hold, because there’s a capacitive touch
panel on that device. And then in terms of spatial sound, to answer
your question, there’s a magnetometer on these devices. So you can
understand directionally with some magnetic declination if somebody
is facing north, south, east, west. You can also do arbitrary
directions. So, for example, there’s an app called Bose Radar. And if
you have a Bose AR-enabled device, you can download that for either
the Google Play store or the Apple App Store. And that if you open
that up, there’s a beachscape scene that I’d like to talk about. So
if you press play it and you close your eyes, you will hear a beach
scene in front of you. But if you look left, you will hear the beach
scene in your right here. And if you look right, you’ll hear the
beach scene in your left.
Alan: That’s so cool.
Michael: So it’ll kind of route
the sound around you based on where you were initially looking — not
based on like the direction of the world, where you were initially
looking — and keep it there. So you could hear seagulls out on the
left side, etc.
Alan: That’s positional sound,
right?
Michael: Yeah, that’s like
spatial sound that’s rooted in a position.
Alan: Very cool. And then I
guess it starts when, whatever direction you’re looking, that’s when
it starts?
Michael: Well that’s an option.
That’s an option. You don’t have to do it that way. But that’s an
option. Yeah.
Alan: I don’t know if you’ve
tried to Magic Leap. I assume you have.
Michael: Yeah.
Alan: They can have spatial
audio built into those. And one of the demos they do is this ball of
light is 10 feet away from you on the floor, and the sound is coming
from 10 feet away from you on the floor. Is that something that is
going to be available through these Bose glasses?
Michael: Absolutely. Yeah. So
there’s a number of ways to tackle that. The advantage that something
like a Magic Leap or even an Oculus Quest has is that it’s tracking
you with cameras. We don’t have any cameras on our devices. And so
it’s very easy to present that sound accurately when there’s an
object attached to it, right? That you can look at and see the
distance, and therefore so can the computer, which is generating that
object. It’s kind of native. The way that you can do it with Bose AR,
if you’re a developer, is within Unity, you can actually do that same
thing. You can create a visual scene and attach sound to an object
that’s at a certain distance, and then present that to a user,
because you know the rough direction that a person is looking. The
ways to overcome the 3DoF versus 6DoF thing — where you can move
forward in space and have it recognize that — there’s two main ways
that we’ve discovered so far. And again, this is a third-party
platform, so I’m expecting we’ll discover new ways for people to do
this with people working with pedometers. What if I can track
somebody, step forward and other ways of doing camera lists, six
degrees of freedom tracking? But here are the two that we found so
far. One is you’re outside, you use the phones’ GPS with the
magnetometer from a Bose AR device, and in that way, you can know
where in the world they are, and also in what direction they’re
looking. And using just those two heuristics, I could see that
someone’s looking at a restaurant — and there is actually an app
called Navigator that does this now, you download it for the Apple
App Store — but you can double tap and it will pull in using Yelp’s
API information about the restaurant you’re looking at, and say
“three stars with 500 people,” and you’ve got to make a
decision about whether or not to eat there.
So that’s one way to understand where
something is in space. There’s another app that uses that technique
to do specialized audio tours. As you walk up to a place, you’ll hear
a ping from a specific location, and you can decide if you want to
enter that audio experience. So that’s cool. There’s a lot of stuff
we’re doing to enable developers to build spatial silent discos. You
could walk up to it, have music fade in, that sort of thing. And then
the other way pertains more to indoor position. So we’ve got some of
our developer advocates on my team experimenting with beacons and
things like that. But if you have no infrastructure, you can always
default to something like the Vuforia or a ARKit or ARCore with your
phone out and the camera pointed at the ground for plane detection,
and then you can actually walk forward and around the space and then
it can also understand where you’re looking relative to that. And
there’s actually an app I’d recommend downloading called Traverse,
which is a really wonderful music experience where you can — for
example, with Elvis — arrange the band around you and then walks
through and around a performance. You can hear Elvis’s voice like a
ghost next to you, and then move behind him, and there’s the person
playing the drums. And you can actually go around a virtual space and
listen spatially to it, using ARKit to support the positioning of the
headset.
Alan: Oh, my God. I can see this
for museums, for public tours or walking tours.
Michael: Absolutely.
Alan: I think it’s great that
this is available on these two headphones and one pair of glasses.
But I can imagine that, through software, you’re gonna be able to
enable people to have these types of experiences across any Bose
device eventually, because if you take away the magnetometer and
accelerometer, you can actually just run it off your phone.
Michael: There’s a couple of
things about Bose’s commitment to this platform. So we’ve already
committed to over a million devices in-market that are Bose
AR-enabled by the end of this year. I think we’ve already basically
hit that. We’ve also started putting them in every wearable we made.
The three wearables are the three most recent wearables we came out
with. The third thing is, you could default to the phone. But what’s
interesting is we know a person is wearing this on their head, and
that’s something interesting and unique. If you combine it with the
capabilities of the phone, you can actually understand where somebody
is looking, for example, and you can understand a lot of other things
as well, by the fact that it’s head-mounted.
Another thing I’d just say about the
platform, briefly, is these are devices people are using every day
anyway. So if I were to make a pitch to developers, the difference
between this and something like a Magic Leap is number one, you
didn’t purchase it just for XR. You purchased it and you’re using it
every day anyway. And then when you build an app for it, it’s not
that somebody has to pick up a device like a Magic Leap and put it on
your head — by the way, I love Magic Leap. There are just different
qualities, to it, right? — You’re probably already wearing your Bose
device. And if you download an app, that has some utility for you.
Those are capabilities. It’s not as friction ball as, “oh, let
me go get my device and put it on my head to do this specific thing,”
if that makes sense.
Alan: Nobody wants to buy
something for a very niche thing. And I have a Magic Leap; we’ve got
them in the office here, and they get used very rarely, when
developers are either making something for them or we’re doing demos
for people. But it’s not something you pick up and put on and walk
around the office with. It just doesn’t.
Michael: I think that’s the way
the world’s going. It might be generational. It might be just a few
years of habit. I’m not trying to say that that’s not viable as a
form factor. But there are some convenience aspects to what Bose is
doing that, I think, are relevant for developers, who we want to
interest in building third-party applications, if that makes sense.
Alan: Absolutely. Let’s look at
this from a business standpoint; what can brands start doing to
leverage these technologies?
Michael: That’s a question we
are answering in various ways internally, and trying to answer with
partners. Hopefully you’ll see some of those answers come to market
later this year. But there’s any number of ways to tackle these
things when it comes to brands. We’re working with a number of
existing application providers to build Bose AR-enabled features that
make sense. For example, a partner of ours is Golfshot, and Golfshot
is a popular application for golfers. And there’s a Bose AR-enabled
feature that, let’s say you’re on the course with a pair of framed
sunglasses. You can actually get contextual advice about where on
that green — based on where you’re looking — you should be hitting
the ball and with what. Little quality-of-life improvement features
are one aspect to existing established branded apps. There’s also
obviously marketing and promotional opportunities. Spatial sound
itself can just transport you very quickly. You just close your eyes
and you’re in the middle of some scene of your favourite superhero
movie, for example. Or whatever the case may be.
Another thing that we’ve done is we’ve
put out something called a creator tool. This is something that feeds
into our Bose Radar app — which does the soundscape that I mentioned
on the beach — and we actually do… oh, my gosh. I don’t want to
butcher his name, that would be terrible, but I think it’s BJ the
Chicago Kid — has some experiences in Bose Radar of his music laid
out spatially around you. And we are working with brands on
delivering more experiences, many of which will be musical, and some
of which will just be artistic and immersive-audio-based. But the
Bose Radar app is fed into by a tool which exists on the web. We’re
doing private invites right now and that’s also available at
developer.bose.com. And that’s going to be a kind of Wiziwig tool
that allows brands to create experiences with spatial sound and some
gesture recognizers, and also use GPS to do location-based and
spatial sound experiences. So there’s a number of channels that we’re
working on now to engage with partners. And if you are a brand that’s
interested in this, just reach out to me or head to our web site.
Alan: Awesome. So what can we
expect in the future? And I’m going to throw this out there, because
I saw — I shouldn’t say I saw — I heard a pair of headphones called
NuraLoop at CES this year, and they were personalized headphones
that… they had some spiel… but man, they they sounded amazing.
And there’s not been much development as far as R&D or real
change in headphones and a long, long time. And it sounds like you
guys are really pushing the limit. So what’s next on the on the
radar? That you can talk about, obviously. Is there something coming
that’s going to take it to that next level?
Michael: There’s always things
coming. We have a future wearables that will be coming out that are
Bose AR-enabled. I think what businesses will need to know is that
we’re really committed to creating a viable, large-target platform,
highlighting our partners that work with us and our channels. And we
have an app called Bose Connect that as over 14-million installs
worldwide. We’re shipping our XR headsets everywhere in the world.
That’s something very few — if anyone else — can claim in this
industry. And I would just say, there’s really exciting things on the
horizon — new capabilities, et cetera. But I also want to reassure
people that this is a platform that can be built on that is
future-proof. And what we’re really trying to get to now, the analog
that makes sense, is we’re kind of with either Web 1.0 or the early
app store for iOS — whatever you want to call it — but capabilities
got added to the Web. Certainly, people were able to develop better
applications as processors got better for mobile. But the core
capabilities are the content. So what we really want to do, what
we’re committed to doing, is working with third-party developers,
small and large. You don’t have to be a big brand. You can be a a
single person. We’re trying to make platform self-service and easy
and free, because the value to us is our customers find a use out of
Bose AR-enabled applications, and they come back to them and they use
them. And that is a win/win for us. So there’s a nice synergy in our
business model for companies, small dev shops, large brands, to come
and work with us because the content is what I think folks who own
Bose AR-enabled devices can look forward to. And that’s what we
really need to bootstrap. In addition to, obviously, future
capabilities, better refinement of the platform as we go along.
Alan: Amazing, there’s so much
coming, I don’t even know what to ask! It’s like you guys have
figured it all out.
Michael: Not at all; it’s a
process.
Alan: It is a process! But I
mean, you’ve been down this road before. You’ve been down the process
from, “we have an idea and couple of nerds in a lab,” to
“hey, look, this is a product and it’s in the market now.”
Michael: It’s what I look for.
Yeah. It’d be boring otherwise.
Alan: Building things for the
world is pretty exciting, and I feel like we’re in this renaissance
moment of technology where it’s unleashing untold possibilities for
humanity. Let’s put our educator’s hat on for a second. How do you
think the Bose technology for spatial audio — or Bose AR — how do
you think it can be used for training or education?
Michael: We are actually already
talking to a number of interested parties on workforce training and
repair enablement scenarios. So the same sort of scenarios that
you’ve heard talked about with augmented reality, we think have some
potential uses with frames. Let’s say, large warehouse rollouts with
a maybe a safety glass version or something like that, where you can
get audio cues, and your eyes aren’t distracted by a screen. You’re
doing your job, but something goes wrong in one corner of the
warehouse. And you hear a ping spatially where that might be. Things
like that that can enable quality-of-life improvements for workers,
while leveraging the fact that there’s not a screen to not distract
from what they’re doing. There’s something to putting something in
front of somebody’s eyeballs that’s just always going to take your
attention. Right? So we think that there’s a lot of
workforce-enablement stuff that can be done with Bose AR-enabled
features within existing applications, and then maybe dedicated
applications for things like I just mentioned, that’s new. Or even
ones we haven’t come up with yet. In addition to that, there’s
there’s a whole host of different potential opportunities. We’ve
obviously, like you mentioned at the top, have lots of interest from
audio tour providers. Well, there’s lots of old-school headsets with
cassette tapes that are still out there for various stores. That’s a
low hanging fruit.
Alan: I used to be a DJ. [record
scratch]
Michael: Oh! Did you?
Alan: Yeah, for 20 years.
Michael: Well, that’s exciting.
Alan: Yeah. Have you ever seen
the Emulator, the big see-through touchscreen DJ controller?
Michael: No! That sounds like–
Alan: Type up “Emulator
DJ.”.
Michael: OK. Yeah.
Alan: I invented this giant
see-through glass touchscreen.
Michael: Woah! That was you?
Alan: Yeah.
Michael: I see this, and it’s
all… wow.
Alan: Yeah. It was a MIDI
controller. We actually worked with lighting controllers. We worked
with video, audio. Oh, ton of big brands. We brought this to
Coachella one year. We made a thing called dream experience, where we
took — Heineken branded it — and one of those buttons was synth,
one was bass, one was high hats, whatever. And we made it so that you
could, no matter what button you pressed, it always was in key. But
we made this really, really cool thing. People were making their own
remixes and emailing it to themselves.
Michael: That is awesome.
Alan: I love me some audio.
Michael: We did an experiment
with Bose AR at Coachella this past year, just to see what sorts of
things were viable at, like, a music festival setting. And I think
you’re going to start to see some of the fruits of those experiments
in the future. It was kind of exciting.
Alan: Did you work with Sam
Schoonover and his team there?
Michael: You know, that was
somebody on my team that took on that. I don’t know. But maybe.
Alan: I think we need to come up
with something crazy for next year, and we’ll all go to Coachella.
We’ll make it a thing.
Michael: Oh, man. I’m sold.
Alan: We’re gonna get everybody
at Coachella wearing the frames.
Michael: Yes! I mean, it’s
perfect. Like a silent disco.
Alan: It’s so perfect. Honestly.
All right. We’re gonna make that happen.
Michael: Awesome. Sounds like
fun.
Alan: I love it. And they
already have a silent disco.
Michael: Yes, I know. I’ve seen
some of these. But seamless, just walking up to it and walking away;
that to me is something I’d like to see realized.
Alan: Yeah. It would be really
cool.
Michael: There’s a lot of hard
tech under the hood that would need to work, and Wi-Fi that doesn’t
suck at a festival. So there’s also challenges.
Alan: But we’re gonna have 5G.
It’s gonna be a thing.
Michael: There you go! Problem:
solved.
Alan: We’re already working with
all the telecos; we’re gonna bring 5G, we can drop it in there. We
got this.
Michael: That would be amazing.
Yeah. If there’s no congestion problems, which I–.
Alan: That’s the promise of 5G,
is getting rid of the bandwidth problem.
Michael: Yeah, I thought it was
the ping problem, or the latency problem. If it gets the bandwidth
problem, awesome.
Alan: It’s three things. It’s
bandwidth, latency, and also capacity of the network.
Michael: But can it make me
dinner? That’s really the question.
Alan: The answer is absolutely.
Uber Eats delivers.
Michael: Awesome. What a world,
yes.
Alan: What a world. So listen;d
we have plans now. We’re gonna go to Coachella. We’re gonna make sure
everybody’s got the frames. We’re gonna have a silent disco. I think
there’s a huge potential for augmented reality as it pertains to
audio, and I think Bose is perfectly situated to take advantage of
that and also bring quality audio to the world.
Michael: Yay! Thank you. I
agree.
Alan: What is one problem in the
world that you think can be solved with XR Technologies?
Michael: I believe in XR as the
empathy machine. I think that it can literally put you into the shoes
of someone experiencing the world from a different perspective. And
in that way, it’s almost not even empathy. It’s almost sympathy.
Like, I was wading through a crowd as a 4’3″-tall person and I
actually experienced what that was like. I think that sort of concept
of experientially being able to put yourself into someone else’s
situation is something that all forms of XR can and will continue to
do, even on accident. Like every time I play a game and things are
sized differently; that’s the size thing. But it could also be
scenarios. “Oh, I was in a scenario where a bunch of people were
bullying me. And how did I handle that?” I think for people who
struggle with being able to empathize with others in different
situations, VR and AR have the ability to give us that new
perspective. I think that’s one of them. Very many exciting things
about XR. I could go on a number of different ways. I think
education, therapy, recovery, the way we work, remote meetings,
collaboration, etc.. I guess I wanted to highlight the empathy
machine aspect of it here.
Alan: You know who quoted that
VR is the ultimate empathy machine?
Michael: Who? Was it you?
Alan: It was not. It was Chris
Milk.
Michael: Milk. I’ve got to give
him credit next time.
Alan: Yeah.
Michael: Did he coin that
phrase, though? Or did he just give a talk on it?
Alan: I’m pretty sure he coined
it; look at the date.
Michael: 2017.
Alan: Yeah. This has been really
wonderful. Thank you so much for taking the time. And I am really
looking forward to Coachella next year.
Michael: Yes. Let’s keep the
conversation going.
28 min

About XR for Business

From the publisher's feed

Meet the leaders who are changing the face of virtual and augmented reality

Share XR for Business

Sign up to save your podcasts

XR for Business

XR for Business episodes

About XR for Business