XR for Business

By Alan Smithson from MetaVRseBusiness Technology Arts

XR for Business episodes

September 20, 2019 It’s Okay to be Small, with Virtual Reality Marketing’s Terry Proto
Don’t let his
impressive stature fool you; Virtual Reality Marketing CEO Terry
Proto knows that, in an industry where there’s a ton of use cases and
many roles to fill, it doesn’t hurt to be small. Heck, it usually
pays to be! Terry joins Alan in a chat about how companies can best
find their niche in the XR realm.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today’s guest is the
one and only: Terry Proto. He’s the CEO of Virtual Reality Marketing.
Terry is an award winning digital imaging and digital games producer.
He has over 15 years of production and sales experience in the US,
Europe and Asia. And he’s been creating images since the very first
version of 3D Studio back in the 90s, and has evolved over the years
working on myriad projects, including agency work and other products
and project endeavors. In a previous life, he struggled with getting
clients and visibility consistently for his own creative studio,
despite the quality of his work. And after connecting with a lot of
CEOs in the XR space, he realized that his problem was a widespread
problem. So for the past two years, Terry and his team have been on a
mission to help studios and brands better connect for everyone’s
benefit. To learn more about his company, Virtual Reality
Marketing.com, go to virtualrealitymarketing.com. It is my absolute
pleasure to welcome Terry to the show.
Welcome to the show, Terry.
Terry: Hey, Alan. Well, thank
you very much. I love the intro. It’s really an honor to be on your
podcast today.
Alan: Thank you. It’s such an
honor to have you on the podcast. I know we finally got to meet in
person for the first time at AWE — Augmented World Expo — what,
about three weeks ago now?
Terry: Yeah. We connect with so
many people, and it’s all digital and it’s all remote. So it truly
feels good to shake someone’s hand now. [chuckles]
Alan: I got a hug from you,
which was awesome.
Terry: [laughs] Exactly.
Alan: You are a very strong man.
I don’t know if you’re benchpressing Volkswagens in your spare time,
but those of you who know Terry; he’s a very large, solid dude. Not
just in physical stature, but in mindfulness and everything. And his
passion shows through in the work that he does. I really want to
start digging into that. So tell us about Virtual Reality Marketing,
and talk about how you got into this.
Terry: I think you nailed it in
the intro. It really started with my problem as a producer. And you
know, when you’re a producer, you’re in your own silo and you’re
working on those products and you’ve got your clients, your team,
you’re flying around for business meetings and events. And you
connect with people, but it’s more superficial. And when I stopped
being a producer, I took a step back and I started talking to a lot
of people. And that’s when I realized that my problem was — I
wouldn’t say everyone’s problem, but very common problem — and I
looked around and I couldn’t find a solution for myself for years.
And I figured it would be time to hack all this and solve this for
everyone.
Alan: So what is the solution
that Virtual Reality Marketing is doing? You’re connecting agencies
and big brands with studios. Is that correct?
Terry: Yeah, exactly. Simply
put, Virtual Reality Marketing, we’re the most comprehensive
directory of AR, VR, 360 studios. And we are also focusing on
building the largest XR case studies library. Right now we’re close
to 150 on the site, and we are on track to have 500 by next year. The
problem that we try to tackle is that… I like to analogize, I like
to say that VR is an hydra, as in it as many heads, and you don’t
know what to do with this thing. And VR is still very much a unicorn,
as in everybody talks about it, but few people have actually seen it.
And that’s where we come in. So on one hand, we have brands and
advertisers and anyone who’s interested in XR, to get involved in
immersives. At this point, those guys realized they need to get
involved, but they don’t know where to start. And it’s difficult for
them to find partners they can trust. And when you’re spending, I
don’t know, $50,000 on a budget for XR project, you want to make sure
you’re spending your money at the right place.
Alan: Agreed. That’s one of the
problems that we are trying to solve with this very podcast.
Terry: Exactly.
Alan: Let’s unpack that. If
you’re a brand that wants to start using virtual augmented reality
for marketing, what’s the first thing that you would recommend to
them?
Terry: Well, the first thing is,
get informed. It’s knowing about the studios. And that’s the focus we
have on the case studies right now. It’s all about this. It’s like,
say, I’m a brand or a company because it works for… say I’m a
training company, and I’m working in medical. And I want to build
this project. I don’t know if it’s possible. I had the case literally
last week. Clients are coming, they’re like, “Hey, we’ve got
this idea. We don’t know if it’s possible. Is it realistic? Is it
unrealistic?” So, first step: looking for the case studies of
what are people doing around us. And right now, it’s not like two
years ago. We’re in this world where we have the case studies. We
have the experience. We can demonstrate the ROI. We can demonstrate
the benefits. And we’re collecting all of this information to share
it easily. So short answer to your question: first step, see the
relevant case studies in your industry, it’s going to inspire you,
and answer tons of questions. Start from there.
Alan: I just noticed — I was
scrolling through the site as you were talking — and one of the
companies that we invested in, 3D Food And Drink, is on there.
Terry: Yeah, so… [laughs] It’s
one of the cool things of what we do. It’s like I said, we connect
with so many people and then we are like, I’m talking with you and
you’re like, “Hey, this is our product.” And I’m like, “Oh,
wow, we didn’t even know about it, but great.” That’s the whole
point. I love when I hear that. It means we’re doing a great job.
Alan: So how do you monetize?
What is your business model?
Terry: In terms of business
model, we really do two things. On one hand, we have the content
creators. We have them connect with studios. So we have them through
the websites. We can give them more visibility through the website
with several packages, share more content, be on top of the list
beyond the home page. We have also a consulting offer, where we can
dive deep with a studio. It’s not for everyone. It’s for select
studios who are solving mission-critical problems for their clients.
There, we can build an advanced lead-generation strategy, connect
with a large number of leads. It’s really basically taking all of
your business developments and we’re handling this for you, starting
from the strategy, all the way to the living leads to the studios.
For brands and advertisers, it’s really
all about connecting with relevant companies they can trust. We’ve
got brands, advertisers — again, anyone interested in using AR and
VR — connecting with us and saying, “we’ve got this project. We
don’t even know if it’s realistic or not. But tell us.” And
based on what they share with us, we can make a recommendation of,
those are the right studios that you want to work with, because
they’ve got the track record; because they’ve got the expertise;
because they’ve got the portfolio. And then we work on commission for
the recommendations.
Alan: Ok. That’s pretty awesome.
What about companies that are just starting out? How do the smaller
studios and startups start to build that book of business and case
study library?
Terry: That’s a good question.
And you know what? It’s still a world of small VR companies. So the
first thing I should say is, it’s OK to be small. I know we’ve been
small for a long time and in France — so not in the US — and we’ve
kind of got that small company complex. It’s okay. It’s okay to be
small. There are so many things to do in this industry. Everyone is
starting, and you need to start somewhere. The most important [thing]
is focus and relevance. You don’t want to be everything for everyone.
“The jack of all trades, master of none?” That’s something you
see a lot in VR. You want to focus on one problem and become the
expert at solving this problem for your clients with AR, VR and other
tech, you don’t need to restrict yourself to VR. Actually, the most
successful companies are integrating VR into a larger vision.
Alan: What would an example of
that be?
Terry: Location-based
entertainment. You have your VR experience, and it’s the center and
the core experience. But at the same time, you’re selling t-shirts
and you’re selling drinks, which is completely low-tech and has
nothing to do with VR. But it’s okay for your clients. If it’s Friday
night and I want to go with my friends for some high-tech
entertainment, and I start to do a laser tag and then I’m going to do
VR. And then it’s Friday, so we’re going to have drinks. And the
experience was cool, so I’m gonna get a t-shirt and gift it to my
friends. It’s a holistic experience. It’s not just about VR. It’s
about them, what they get from it.
Alan: I’ve seen a lot of
location based entertainment facilities where they just set up some
small square rooms with a VIVE in it. And that’s great. But what
really blew me away was when I was in Dubai at VR Park, and they
built the whole experience around each activity. So it may be the
same VIVE as you would play in a small 10×10 room. But for example,
the John Wick VR experience, and there’s like a bank vault and you’re
actually going into a physical set. People don’t think about that
when they’re setting up these experiences. And I think it’s really
important to just get everybody really excited about this technology
before they put it on their heads. And it just adds to the whole
allure of it.
Terry: Exactly, exactly. It’s
not just about VR. It’s about the overall experience and the overall
service for your clients. And it works everywhere. You’ve got this LB
example, but you have your training example on the other side of the
spectrum, where you’re training people. And what matters is that they
get the best experience, so some of the information you will want to
have on iPhone and iPad. Some of the reporting you will want to have
on the Web and some of the most expensive, most complicated, most
dangerous experiences you will want to do in VR. But again, it’s a
whole. And it’s not just VR. It’s also everything else.
Alan: Yeah, absolutely. So in
the last month, what’s the best virtual reality or augmented reality
marketing experience that you’ve seen?
Terry: Ok. So, you know, in our
case, our job is to pretty much see all of them, at least as much as
possible. So it’s really a difficult question for me to answer
because I have to pick one.
Alan: That’s why I picked the
question!
Terry: [laughs] But if I have to
pick one, I’ll give you… for instance, I tend to prefer the ones
that are smart, and fun, and really solve a problem. So in the last
few weeks, we got this experience from a Brazilian studio, VZ Lab,
and it’s the VR vaccine experience. And I love it because it’s clever
and funny and it’s solving a real problem. It’s the problem of
vaccination with children. Huge problem for the parents; they’re
fighting with the children. And the children, some of them are
literally traumatized by the things, they’re crying. Problem with the
nurses, because you have to deal with conflict day in and day out.
Long story short, the studio created
this experience, which is synced with what the nurse is doing in real
time. So for the children, they are plunged into this immersive
adventure all in 3D, it’s beautiful. And they’re going to be given a
shield. The children is being warned that he’s gonna be bitten by
something and it’s gonna be okay. And in real time, the nurse is
doing the injection. It works like a charm. It’s beautiful. And you
ask the children, they’re like, “oh, my God, that was amazing. I
loved it.” No cries. No screams. No nothing. And it’s 180
degrees of something that is a huge problem for everyone. And it’s
been turned into a fun, cool experience. It’s incredible.
Alan: My daughter, she’s 11, and
she is literally terrified of needles.
Terry: Like I said; smart, fun,
really solving a problem. It’s great. But just to give you another
one, on the other end of the spectrum. Children again. So this one
not as glamorous, but super useful. This time we’re talking about a
therapeutic training tool and it’s about understanding the changes in
the brain of a child who was suffered childhood trauma. And I think
this one is built by UK studio, Ignition. And basically you’re living
the experience through the eyes of the child, and you see your
parents fighting, and the isolation, and the tears and everything.
And in the meantime, it’s superimposed with the brain activity and
you see how the brain is being restructured in real time, based on
the experiences the child has lived.
Alan: Wow.
Terry: And you get a different
brain. So it’s really understanding and showing you how every little
detail in your family life is impacting your children and how to
change those behaviors, because especially when you’re young, the
brain is plastic. And those are very strong connections that are
really difficult. Or nearly impossible, I should say. So, another
great one.
Alan: They’re on such wide
spectrums of the technology. One of the things that I think is it is
an issue with our industry in general — and I think maybe you can
address this — is these are great experiences, but how are companies
measuring the success of these? Are they doing it through earned
media captures? Are they doing it through number of people that
they’ve put through the experience? What are you seeing as far as the
analytics and metrics around this?
Terry: It’s a good question. And
I think the best answer I can give you is there’s no
one-size-fits-all answer. I think in training, one of the best
metrics you can get is… I’ll give you one of my favorite points. I
think it’s PIXO VR, they are doing some of the best of the safety
training, like first responders training, firefighters training. And
it’s something that’s simply really difficult and dangerous to
experience in real life. So success here is just having the
experience, they’re building this amazing cinematic experience, which
is exciting. It’s really literally like being in a movie. But it’s
useful. It’s saving lives, because you’ve got a better team. They are
better trained. You can’t train those people like this in real life
because you would be putting their lives in danger. And because
they’re better trained, they’re saving more lives. So that’s one of
the best ROI use case I can give you. Quite literally, because you
were able to be trained in a simulated dangerous environment,
realistically, you know how to handle those situations. You get all
of the experience that you would build a lifetime of being in danger.
And that’s allowing you to save lives. That’s all the ROI you can
get.
Alan: When you put it that way.
I mean, what’s the ROI on a life? The last podcast I did today was
with Dr. Walter Greenleaf, and Dr. Greenleaf has been in this
industry for 33 years, talking about the medical use cases. He kind
of broke it down into five key parts: training, assessment,
intervention, health and wellness, and then the democratization of
care. As we move to more precision medicine and proactive medicine,
VR stands to create unlimited potential for people in underserved
areas.
That’s medical; when you take it to
education, it can unlock the full true democratisation of learning,
globally.
Terry: Absolutely. You know
what? We did have a case — I have to pay attention to what I can
tell you about it — but we did have a case two weeks ago of a
company in medical, working on this training for medical, and that
they want to deploy in developing countries. So Latin America,
Eastern Europe, Asia, and basically it’s about training medical
personnel, and they’ve got tons of problems to do so, because right
now it’s a physical training. So it’s costing a fortune, and you need
to physically move people all around the planet, literally. And then
you also need to work with the local people. But they keep changing,
so you don’t know of their standards. And sometimes you need to have
them rise to your own standards in order to deliver good training, as
opposed to VR where you can get it right once and then make sure
everyone gets the right content. It’s infinitely cheaper to send to
experience once again, yeah, of course.
Alan: One of the things that I
keep thinking; my kids are in grade school, and one’s in high school
now. And if you think about the teachers that are there — they’re
wonderful people — but by no means are they the world’s expert in
anything they’re teaching. And being able to harness the best
possible trainer every time, that’s essential. And I think that’s
really what STRIVR’s doing well, as well. They’re a company based in
San Francisco that’s doing virtual reality training, and they’re able
to capture the best trainers and spread them across the entire
enterprise. Whereas before the best trainer, maybe you could train
20, 30 people at a time or maybe a couple of hundred throughout a
year. But when you’re talking thousand employees, it’s just not
scalable. You can’t send somebody on a plane to visit every employee.
But in VR, you’re just setting a headset.
Terry: Absolutely.
Alan: So what industries are you
seeing that are using this the most?
Terry: Right now, I would say
we’re seeing obviously the most traction is coming from training.
It’s coming from medical. It’s coming from marketing as well. But
marketing is a different beast, because whereas in training and in
medical it’s really about – again — solving those mission critical
problems. In marketing, often it’s more into nice-to-have cool
experiences, so you see trends. For instance, VR for trade shows was
a huge trend two years ago. And this winter we had like the big AR
craze; AR everything. But that’s a big one. That’s still a very big
one.
Not so big ones: travel, real estate–
like, real estate, surprisingly, what we see is that the obvious case
study — it’s selling your house; I want to see the house. I want to
be in the house. VR technology of presence being the house. No
brainer, right? Well, actually, no. And people in real estate right
now, they’re focused on interactive, meaning that the interactive
part of the VR is great. And so interactive presentation of condos,
building developments, and on a large number of platforms. So you
want to show them in your big interactive screen, a table, in a sales
center or you want to have it on your iPhone or iPad, obviously, and
you want to do the VR.
But like, for instance, I talk with
some companies as it is. I’ve got the whole range of services, from
the interactive screens to iPad to VR. And what they see in terms of
use is that… for instance, the sales people on the move, on the go,
will use more iPad or will use more Bigtable, because you can connect
with several people at the same time, convenience, conviviality,
being able to have several– like, me and my wife looking at the
screen, at the same thing, at the same time. So VR is cool, but it’s
like this one thing that you’re doing, but then you fall back to iPad
because it’s more practical.
Alan: VR headsets are dropping
in price. But it still comes down to the fact that every single
person has a phone in their pocket.
Terry: Exactly.
Alan: Just from a scale
perspective, being able to use AR technology on a mobile device, and
the stat that I keep reading is there’ll be over two billion
smartphones that are AR enabled by the end of. This year. That’s real
scale.
Terry: [chuckles] Yep.
Alan: If we’re looking at mobile
phone-based things, what are some of the coolest things you’ve seen
on mobile phone-based AR? I just saw one the other day that was, you
could put the Space Shuttle in your backyard in real size.
Terry: You do have a lot of AR
apps at the moment. Let me think. Well, I’m a big fan of the try-ons.
Alan: Yeah, I love it. I wrote a
whole article on this.
Terry: Yeah, I saw it, and I
like, for instance, at AWE, there was this company and the CEO was
basically barefoot for the whole show — not kidding — because he
was doing a demo of their iPhone try-on and for some reason they
needed to not have shoes. [laughs] And so you take your phone and you
see your sneakers. You can select the colors and everything. And I
must say, it’s a really compelling experience, because when you’re
buying shoes online, you never really know — especially the designs,
the colors or whatever — seeing it on you like this, it’s much more
than gimmick; it’s really useful.
Alan: Yeah. No. You’re
absolutely right, and I think Google just rolled out this virtual
try-ons right in the Google Lens and everybody’s starting to work on
it, which is pretty interesting. There’s a whole bunch. There’s
makeup, watches, shoes, glasses, hats, beards — see what a beard
looks like on you.
Terry: I was talking to also —
it’s funny; the same week of AWE, I was talking to a partner, and she
was getting engaged, and I told her about the try-ons. And so she was
super excited about the jewellery try-ons. And she’s obviously a
woman. And she was like, “oh, my God, you can try and you can
see on your iPhone. And, wow, I need your address. Send me your
address like right now.” Once again, it all comes down to the
case study. She doesn’t care it’s AR or VR; she gets what she gets
out of it.
Alan: Exactly. And it’s
interesting you pointed it out, because Snapchat uses AR all the time
and for everything, for face filters and real world filters and stuff
like this. But nowhere do they mention the words “augmented
reality.”
Terry: Yeah. But, you know, I
think it’s one of those things… I really liked an analogy of the
world of AR and VR right now. And it was like, you know what? I think
we’re kind of the Web, circa 2003 or something, when people were
building websites. And back in 2003, when you were asking someone,
it’s like, “what are you doing?” “Well, I’m doing a dot
com.” “What is your website? What is it doing?” “We don’t
care so much. I have a website; it’s amazing!”
Alan: “What do you mean? I
do AR!”
Terry: Exactly! You see where
I’m going. And now it’s kind of the same like “hey, I’m doing
VR!” “Yeah, but what are you doing?” “Yeah, I don’t
care so much, but I’m doing VR,” or “I’m doing AR!” And at
the end of the day now, yeah, of course you have a website. Amazing.
Extraordinary. Everyone has a website. We don’t care about the
website. We care about what’s going on on the website.
Alan: Exactly. By the way, if
anybody is listening to this and wants to learn, we have a website!
Terry: Alan, we have two
websites.
Alan: Oooohh! What are your
websites? You have virtualrealitymarketing.com; what’s the other one?
Terry: Virtualrealitymarketing.com.
And we have uberealagency.com, and that’s the consulting for studios.
So two fully-functional websites.
Alan: Holy moly! You are way
ahead of the game! [laughs] We also have our pitch for XR Ignite,
which is actually in VR and AR as well. Through a platform called
VRAVO.
Terry: And that’s something
that’s very interesting I’d like to pick on, is that I find that one
of the best ways to evangelize about AR and VR is actually using
it.
Alan: You think?
Terry: Yeah. But, you know, it’s
so funny in what we do, it’s all about the simple things. But so many
people overlook the simple. Everyone is like, “AR and VR, it’s so
amazing and everything.” You know what? How about we actually use
it? One of our clients, they are this Finland company called Glue and
they’ve got this amazing remote presence tech. And they really strive
to do their meetings in Glue with their tech.
Alan: Yeah, yeah. Kalle
was on our show.
Terry: Oh, OK. So, yeah, you
know them. Small world. But things are amazing. I think what’s out
there is beautiful. I had a blast playing with the tech at AWE. AWE
was the place to be this year.
Alan: It was amazing. There was,
I think, 6,000 people they said this year?
Terry: I don’t know, but you
felt like everyone was there.
Alan: By far and away, it is
probably the most important VR/AR conference in the world. Mainly
focused on augmented reality, but–
Terry: A lot of VR as well.
Alan: When I was there I said to
somebody, “if this building collapses, the entire VR industry is
gone.”
Terry: Exactly.
Alan: This year I ran the
startup track, and I did a panel on supercharging your marketing,
actually, with the head of XR for Nestle, Richard, and Mohammed from
Macy’s, and Jason
Yim from Trigger Global. And there was one more I can remember
off my head. But yeah, it was really amazing. This year was just a
beautiful experience, getting to meet you and seeing what you think
of all my friends. It was really cool.
Terry: Yeah. Ours was the same.
We’ve got clients everywhere. In the US, in Europe, as far back as
Finland. And you’re on the floor, and you have everyone. And you–
all those people that you connect on the phone away from everything,
you can shake hands again. And just that was– And most importantly,
the density of smart, talented, dedicated, passionate people. The
density of conversations that you have, really interesting
conversations. You turn your back and, “Oh, yeah. So we are
doing this and that.” That was amazing.
Alan: Yeah. Every single person
you met was doing something revolutionary.
Terry: Exactly.
Alan: I got home and it took me
three days to write my post report.
Terry: [laughs] I’m surprised.
Alan: Yeah, I met over 100
people in this short amount of time. So anybody listening, go to
Augmented World Expo. It’s definitely worth going to next year. And I
think there’s another one in Europe, as well.
Terry: There is one in Europe, I
think like Q3, like September or something.
Alan: And then there’s a VR Days
in Amsterdam as well, which is coming up, I think in October, I’ll be
speaking at that. And that one’s another great one.
So let’s get back to use cases for a
second, because one of the things that I’m starting to see — and
maybe you’re seeing it as well — is, like the virtual try-ons, we’re
starting to move over to more utilitarian use cases of this
technology. One of them that I thought was really cool was, I believe
it’s Dulux, the paint company. They figured out how to segment your
walls. You can put your phone on the wall and change the paint colors
and they look real depending even on the lighting that’s in your
room. So you can see it in the daylight and see what it looks like,
and then see it at night. Being able to do that, that’s an impressive
use case.
Terry: You’re talking about
something really important right there. Back when VR really started
in 2015-16, everybody was super focused on entertainment, games, and
all the things you can do with it. And I’m not saying games are
uninteresting. You can do plenty of things with games and everything.
But the business side of things was, “it’s business; it’s
boring.” Turns out, right now the most successful companies in
VR tend to be solving mission-critical problems for their clients and
business, so a lot of those utilitarian cases. And what we like best
— what I personally liked best — is that you can do smart and you
can do sexy, if you want to. Meaning that, you can be utilitarian,
but especially with VR, you can do it in an exciting and engaging and
just cool way. And that’s great, because the cooler or the more
exciting your utilitarian app is going to be, the more people will
want to use it, the more it is going to solve problem for plenty of
people. We want more of that, and there’s money doing that.
Alan: Yeah, absolutely. The
Triple A gaming market is what people expect now. They expect that
level of quality. So we have to — everybody, the whole industry —
has to step up their game to create experiences that not only solve
problems, but wow people.
What is the most important thing
businesses can do right now to leveraging the power of XR? What would
you recommend as their first step?
Terry: Very simple. It’s the
same first step for everything: it’s get started now. It’s
always about the simple next step you can do to get started and for a
lot of people — especially with XR — it’s very intimidating. We’ve
talked about this many times. And no matter where the company is, now
we live in this world where it’s not so much about “eh, XR. Is
it a fad? This is going to go.” It’s here to stay. And no matter
what you do, it’s going to probably help your business.
So now it’s all about, you know, get
started. Get your hands dirty. Build a project, doesn’t have to
be big. And build your own understanding and experience, fully
understand the impact, the ROI, the benefits for your company.
Because it’s not going to be the same as for this other company, this
other [garbled]. So it’s all about you. And become smarter, become
more efficient through VR and then you repeat the process. And with
each new step, you get bigger and more ambitious, but it’s all about
the first step. Get started.
Alan: That’s some great advice.
So, last question: What problem in the world do you want to see
solved using XR technologies?
Terry: We did talk a lot about
training and I’m going to talk again about training and education. I
think VR is great at helping people understand the world better and
understand others better. And I also think that, on the other hand,
ignorance is often the root of fear. And knowledge and understanding
is often helping to connect with people, and I think that right now
in this world we could use more of that.
35 min
September 20, 2019 It's Okay to be Small, with Virtual Reality Marketing's Terry Proto
Don't let his
impressive stature fool you; Virtual Reality Marketing CEO Terry
Proto knows that, in an industry where there's a ton of use cases and
many roles to fill, it doesn't hurt to be small. Heck, it usually
pays to be! Terry joins Alan in a chat about how companies can best
find their niche in the XR realm.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today's guest is the
one and only: Terry Proto. He's the CEO of Virtual Reality Marketing.
Terry is an award winning digital imaging and digital games producer.
He has over 15 years of production and sales experience in the US,
Europe and Asia. And he's been creating images since the very first
version of 3D Studio back in the 90s, and has evolved over the years
working on myriad projects, including agency work and other products
and project endeavors. In a previous life, he struggled with getting
clients and visibility consistently for his own creative studio,
despite the quality of his work. And after connecting with a lot of
CEOs in the XR space, he realized that his problem was a widespread
problem. So for the past two years, Terry and his team have been on a
mission to help studios and brands better connect for everyone's
benefit. To learn more about his company, Virtual Reality
Marketing.com, go to virtualrealitymarketing.com. It is my absolute
pleasure to welcome Terry to the show.
Welcome to the show, Terry.
Terry: Hey, Alan. Well, thank
you very much. I love the intro. It's really an honor to be on your
podcast today.
Alan: Thank you. It's such an
honor to have you on the podcast. I know we finally got to meet in
person for the first time at AWE -- Augmented World Expo -- what,
about three weeks ago now?
Terry: Yeah. We connect with so
many people, and it's all digital and it's all remote. So it truly
feels good to shake someone's hand now. [chuckles]
Alan: I got a hug from you,
which was awesome.
Terry: [laughs] Exactly.
Alan: You are a very strong man.
I don't know if you're benchpressing Volkswagens in your spare time,
but those of you who know Terry; he's a very large, solid dude. Not
just in physical stature, but in mindfulness and everything. And his
passion shows through in the work that he does. I really want to
start digging into that. So tell us about Virtual Reality Marketing,
and talk about how you got into this.
Terry: I think you nailed it in
the intro. It really started with my problem as a producer. And you
know, when you're a producer, you're in your own silo and you're
working on those products and you've got your clients, your team,
you're flying around for business meetings and events. And you
connect with people, but it's more superficial. And when I stopped
being a producer, I took a step back and I started talking to a lot
of people. And that's when I realized that my problem was -- I
wouldn't say everyone's problem, but very common problem -- and I
looked around and I couldn't find a solution for myself for years.
And I figured it would be time to hack all this and solve this for
everyone.
Alan: So what is the solution
that Virtual Reality Marketing is doing? You're connecting agencies
and big brands with studios. Is that correct?
Terry: Yeah, exactly. Simply
put, Virtual Reality Marketing, we're the most comprehensive
directory of AR, VR, 360 studios. And we are also focusing on
building the largest XR case studies library. Right now we're close
to 150 on the site, and we are on track to have 500 by next year. The
problem that we try to tackle is that... I like to analogize, I like
to say that VR is an hydra, as in it as many heads, and you don't
know what to do with this thing. And VR is still very much a unicorn
0 min
September 18, 2019 Combining the Best of AR and VR with Varjo’s XR-1, featuring Niko Eiden
The human eye
is a wonderful and complex thing, and it’s a technological feat just
to even come close to its natural revolution. Well, the folks at
Varjo have created something that is pretty darn good at it. Alan may
have trouble remembering how to say their company’s name, but he can
attest to the clarity of the XR-1’s display. Varjo CEO Niko Eiden
comes by to give us a look behind the curtain of its creation.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today we have Niko
Eiden, CEO of Varjo. He’s formerly held top product leadership
positions at Microsoft and Nokia. At Nokia, Niko led a product
program team in 2006-2007, and together with researchers from Nokia
Research Center, his team developed the basis for the optical
technology that later became the Microsoft Hololens. Niko has a
Master’s in science in aeronautical engineering. Varjo is redefining
reality by jump starting a new era in computing. Their hardware and
software lets people seamlessly mix realities together, moving from
the real world to extended reality into pure virtual reality, all
with human eye resolution. Their new headset, XR-1, is a mixed
reality developer device for engineers, researchers, and designers
who are pioneering a new reality. With photorealistic visual
fidelity, ultra-low latency, and integrated eye tracking, the XR-1
seamlessly merges virtual content with the real world for the first
time ever. If you want to learn more about Varjo, you can visit
varjo.com. And I want to welcome to the show, Niko. Thanks so much
for joining me.
Niko: Thanks, Alan. Nice to be
here.
Alan: It’s my absolute pleasure.
We got a chance to meet at AWE this year and I got to try the XR-1,
which was — oh my God — what an incredible experience. You put on
the headset, the pass-through cameras were as if I didn’t have a
headset on at all, I could just see the whole world. And then all of
a sudden a car appeared in front of me; in the space I was in, there
was a car. Then I got in the car, and the space around me
disappeared, and I was in the car. And the one thing that really
stuck out with me that blew my mind was. I was looking — it was a
Volvo — and I remember looking at the steering wheel. And the little
Volvo symbol in chrome was so crystal clear. It looked like a real
car. Let’s talk about your technology and how you guys ended up at
this place.
Niko: Sure. We really had the
initial vision– the founding team, we had a long background in
different augmented reality and VR devices. And we had always been
talking about that video see-through type of devices could actually
combine the best of both worlds of AR and VR devices. And every time
you started talking about them, somebody kind of commented basically
that don’t bother: the latency, the lag between what you’re seeing
through the cameras, and what’s happening in reality is gonna be too
long. Or the resolution is not going to be good enough, it’s not
going to look great. But we were in the summer of 2016, we were
looking at a demo and the demo was shown with the Hololens first gen
device. And I got really thinking that this would be such a cool
demo, but it’s really missing a big part of the experience, because
it wasn’t able to show the image in a photorealistic fashion. It was
a very high fidelity graphic scene that we were looking at with
Hololens and we were just thinking that this would be so much better
with a VR device. And combining the reality with a video see-through
device could actually really work. We were a bit curious. We had some
extra time at that point together with Urho [Konttori] — one of the
other founders — and we built our first prototype in 24 hours, and
the experience was really magical. So it was a very simple
experience. But the main thing of that first experience was that we
were able to dim an existing room and make it dark, add some fog into
it, and there were some knights. So it was immediately something that
you couldn’t do with any other device and you still can’t do with any
other device, because with an AR device, you can only add light. You
can’t add darkness. That demo was good enough to get us the initial
funding and get us going, so it really proved the point pretty
neatly, and that’s just how we got started.
Alan: It’s interesting, I
actually tried– there was a company based in Toronto — and I can’t
remember the name of them — but they had a VR device with
pass-through cameras, and then they had this demo where a giant
crashed through your ceiling and it picked you up. And so it was this
blend of real world and then mixed reality, and then you were in full
virtual reality. And that was my first foray into it. But to your
point, the resolution just wasn’t there and there was such a lag in
the latency that it kind of made me feel queasy, and I felt none of
that with your headset. It was crystal clear, and not only crystal
clear with the latency, I waved my hands. There’s actually a photo of
me waving my hand in front of my face, because I just couldn’t
believe that I was seeing my hand. Your secret sauce or your magic
that’s behind it is, you’ve kind of created almost like a false
foveated rendering, meaning if you look directly in front of your
eyes, your fovea are 100 percent focused. And then as you move out
from there, it gets less and less focused. So if you’re staring at
your computer and you hold your hand out to the periphery, your hand
is kind of blurry, you can’t really see it in detail. You guys have
leveraged that by putting a higher resolution screen within a screen.
Can you walk us through that?
Niko: Yeah. So that was actually
the first pivot that we had to make. As I mentioned, the big vision
was to do mixed reality with video see-through device. But in the
beginning, we didn’t plan to set up a company that would produce VR
headsets. So we actually were thinking that it would make sense to do
a good accessory and equip the existing VR headsets with a very good
accessory that would solve the latency problem. We had a few ideas
how we could crack that problem all the way from the very beginning.
But when we were testing it, fairly quickly we realized that we can’t
leverage any of the existing VR headsets on the market still today,
just because the resolution was not on par what we could achieve with
the video see-through system. So fairly early on we decided that
okay, even though we want to do the mixed reality with the video
see-through, we have to park it for a while. We we have to work a bit
in a serial fashion on this problem and we have to tackle the
resolution problem on the VR headsets first. The idea that we came up
with, as you mentioned, was exactly that. So we are utilizing four
displays in a headset, so two displays per eye. And we are using more
of a traditional setup that we call “context display” that
provides you the full field of view and you get the periphery and
that doesn’t have to be in such a high resolution because the way how
your eye works, it’s impossible to see and in high detail for the
periphery part. And in the center we have a second display that we’re
overlaying with an optical mirror, and that way we are able to mix it
almost borderlessly in front of the complex display we call “focused
display.” And this display is able to match the human eye
resolution by something that we call “60 pixels per one degree
field of view” in that high resolution area. Very early on we
had a target that we want to be able to read a newspaper in VR. That
headset is able to perfectly allow you to read a newspaper in VR.
Alan: Now, for those who haven’t
spent a lot of time in VR, that is something that is a real challenge
for VR, is just reading text is a problem. And there’s companies like
Monotype, who has created all the type fonts that you see, — Times
New Roman and all these things — but they actually started creating
type fonts that would be visible in AR and VR. Then you guys took a
different approach and just said, well, the problem isn’t the text,
the problem is the resolution.
Niko: Yeah. And then there were
quite a lot of restrictions, and still are. I mean, one is the
compute power available on desktop PC. So if everything would be high
resolution VR, we would talking about not 4K display; we’d need,
what, 16K, 20K displays that would be required to pull it off in a
reasonable field of view. So that was not the track that we could
take. I mean, the displays didn’t exist, the processing power didn’t
exist. But with this way, by combining these four displays that
existed on the market, — basically off-the-shelf components — and
combining them in a smart way allowed us actually to do something
that we can produce today, and that will work with the computers of
today and provide you this region of high resolution. It would be
bombastic if everything would be high resolution and you couldn’t
distinguish anything anymore from resolution perspective, but
unfortunately that we aren’t get there.
Alan: From a user’s perspective,
when I put it on, I didn’t notice that the periphery wasn’t focused.
As I looked at the steering wheel, I looked around the car, I looked
at the leather stitching and actually my brain just totally ignored
it. It wasn’t until the second part of the demo — I was looking at a
whole area where there was a moose crossing, and it was really neat
— but it wasn’t until then that I kind of realized that there was a
little bit of a border around what I was looking at. The way you guys
have done it, with the super high resolution in the middle and it
kind of almost feathers or fades out to the other resolution. It’s
almost imperceivable if you didn’t know what to look for. I think
it’s really magical. You guys have a partnership with Volvo. How did
that come about?
Niko: The Volvo partnership, we
started off developing the VR-1, which is our headset that brings
human eye resolution to the market and brings an eye tracker to the
market, that also differentiates us from pretty much all of the other
companies out there today. But the Volvo case was a bit special, and
Volvo had a vision of stuff that they wanted to do in the mixed
reality space, and our dream of doing mixed reality with a video
see-through actually matched. And we were able to offer them first
very early prototypes, so that they could start testing. And it
resonated really well. So Volvo had a fantastic engineering team. So
they had very good capabilities of actually doing work from an
engineering perspective. And for us, it was a fantastic case to
pressure test whether the vision, whether the product that we were
actually signing made sense from an enterprise perspective, and in
the case of Volvo it definitely did. So it was these kind of early
discussions. It started around the VR-1, doing VR in high resolution.
And we mentioned that we have this mixed reality prototype that we’re
still working on, and we were able to show. It clicked from there on
that, and we collaborated pretty tightly from there on. We were able
to finalize and understand what an enterprise company would need and
Volvo could get their hands on very early on something that simply
wasn’t available anywhere else.
Alan: What are they using it
for?
Niko: Volvo is planning to
accelerate their design process for cars. The team that we’ve been
working on is mainly working on the interior of future cars. And if
they have a new idea, with our device they can actually test it
before they have to build a single physical prototype. So what they
can do, is they can replace the interior of an existing car and
virtually replace it with a future car interior. And they can start
testing the interior of the instruments, switch locations. And with
the eye tracking, they can actually do proper testing so they can
drive on a real road. And with our headset, we are able to segment
the window so that the windows from inside the car show the reality.
And that view is to our camera setup and then the rest of the
interior is then the future car interior, which is completely
synthetic and created graphics. And with the eye tracking, they can
light up, for example, a warning light and they can check in a real
driving condition how quickly a driver could, for example, see that
light, just has a crude example of the stuff that they could do. Or
head-up display system, mixed graphics and show– your virtual moose
you mentioned, they can show it while driving. There doesn’t have to
be a real moose, but they can simulate the moose and they can start
testing driver reactions.
Alan: How are you capturing
the… I guess the window views? Is that through a 360 camera setup
or something like that?
Niko: No, we have the
see-through cameras, obviously, and then we have a full model of the
car. If you want to replace the interior of the car, you have to have
a full model of the car, and then you just have to have good
calibration so that the tracking of the headset and the head pose is
synchronized and calibrated.
Alan: So, wait a second. Hold
on. People are driving a real car wearing the headset?
Niko: Yes.
Alan: Oh, wow. So they’re
getting in a car, putting on this headset — I’m assuming there’s a
computer in the back seat or somewhere — and they’re driving around
a closed track.
Niko: Yep. Officially, it is a
closed track. Absolutely.
Alan: Wow. Oh, my God. That’s
next level product testing.
Niko: And you don’t have to
build a single thing in order to start testing. So this early on
testing, fail fast type of stuff, completely new world from a design
perspective.
Alan: So what are some of the–
it’s almost silly to talk about ROI because they’re probably seeing a
dramatic decrease in the design times. Do you have any data around
how this is benefiting them?
Niko: Yeah, we have a discussion
on this with Volvo. They didn’t want to go into detail about the ROI
on that one, but it’s as you mentioned, it’s extremely obvious. Being
able to create a faster design cycle and especially doing
auto-validation of something that’s still virtual, just on a computer
is pretty bombastic. The interior design is just one example. Car
design in general, I guess you’ve seen pictures of these full-size
wax models as well, where car designers go around a full-size car,
built out of wax and modify the lines.
Alan: It seems so antiquated.
Niko: Yeah.
Alan: [laughs] Now that we have
VR, it seems so antiquated to do because one, you have to have the
physical model. Two, you have to have everybody physically there to
look at it. And three, you can’t make any changes on the fly.
Niko: Yes.
Alan: So dramatic, dramatic
decreases in design times, which as we’re entering into this area
where AI, and VR, and XR, all these technologies are culminating in
creating this exponential growth pattern. Every single efficiency
that we can afford these companies is going to be snapped up and used
really, really well. So this is a great way to do it.
Niko: It’s not just vehicle
design. I mean, architecture, everything. The fact that you’re able
to see photo-realistically a room, and see how the light plays inside
the room is big. Previously you had to imagine that stuff and wait
until the building is built. Now you can experience light inside a
building before it’s been built.
Alan: There’s so much happening
around ray tracing as well, having light shine in from different
angles, reflect off of materials. There’s been so much research done
on that, Unreal Engine and Unity are really pushing towards that.
“How do we get photorealistic renders without blowing up your
computer?” What are some other businesses that are using this,
and how are they using your headset?
Niko: Well, apart from design,
training is another really big focus for us. Easy to imagine is
really high profile training of airplane pilots, for example, if you
are able to reduce the simulator time required to train a pilot, for
example, by allowing them to retrain in a virtual cockpit beforehand.
Again, the business case is absolutely a no-brainer for even aiming
for a high-end device like ours. But it’s not just the pilot
training. It scales down to everything, to all professions where
there’s something unusual might happen during your day of work, but
it’s hard to prepare and train. So emergency rooms, control rooms,
police, firefighters, those type of professions, I think they will
benefit immensely from proper mixed reality and virtual reality
training programs.
Alan: You know, people say,
what’s the killer use case for VR? Well, I think training is the
killer use case and design is a close second to that. Can you talk
about some other specific examples of companies that are using your
device, and how they’re using it?
Niko: We are still learning how
companies plan to use this device, but it varies quite a bit. And for
us, the key learning has been also we don’t know how a business or
enterprise wants to use a mixed reality or VR device. Running a
business and then, for example, designing something or training
something that’s really the core of that company. We need to work in
close collaboration to understand their specific use case. And for
those companies now thinking whether they should deploy or think
about using mixed reality or VR: for some specific cases, yes, there
will be very clear places to read how you could use it for your own
business, but in most cases you need to understand what you can do,
and then you need to start dreaming. And the good news is there are
thousands of companies out there willing to help, to program and
create the software tools required to do stuff, especially in
enterprise and business segment. But it needs to come from the inside
out. And those companies who are kind of really advanced, like Volvo,
like Audi, for example, a lot of the car companies that’re really
advanced, they usually have a big team of specialists, who are able
to support than those people who have the need and were able to dream
the use case. And let’s say in an automotive company, they can use it
for the accelerated design process that we discussed. They can use it
to accelerate the design process of their car factories and they can
use it to help configure a future car model that they selling to an
end customer in a showroom. So there are plenty of different
completely different use cases with completely different needs, even
within just one large corporation. And to me, that’s super exciting.
But it does require that those companies are pretty active themselves
as well.
Alan: There’s really no part of
any business– and I published a post on LinkedIn a couple weeks ago
and I said: “Is there businesses that won’t be impacted by this
technology?” Can you think of any?
Niko: No. I have a strong
feeling that this will change fundamentally how we will be working in
general, and I think everybody will be impacted one way or the other.
This is kind of a pretty bold statement I’m making. But if you think
about the tools with what we are working in general today, it’s a
phone, it’s a PC, maybe in some cases it’s a tablet. That’s it. It’s
very restricted, it’s very 2D. Once you start to have a device that
can match the resolution of reality — from the VR or mixed reality
perspective — you can move all of the existing tools and make them
part of the virtual experience, the mixed reality experience. And
that’s why I have a feeling that this transition, once these products
start to be out there, is going to happen faster than we anticipate.
And when we hit the first professions where using virtual reality or
mixed reality will be part of their daily routine everyday and it’s a
few hours, I think that’s when we start to see a really big
transition happening also, from professional space all over.
Alan: I was at the PTC LiveWorx
conference in Boston just recently. And some of the use cases that
they’re bringing online now are not even really mixed reality so much
as the RealWear headset where you can just see a little screen out in
front of you and give you a heads up display. And what they’re able
to do is capture the standard operating procedures that experts know
intuitively because they’ve been doing the job for 20 years. As those
experts start to retire, they need to capture that knowledge somehow,
and then transfer that knowledge to younger workforces. And I think
when people start wearing this a couple hours a day on a regular
basis to help them with their work, and I think in some very small
instances that’s happening already and the results across the board–
Shelly Peterson is coming on the show and she’s talking about how
Lockheed Martin is using augmented or mixed reality glasses for
assembly. The original trials that they did had 85 percent reduction
in task completion times. 85 percent reduction! So when you have
these crazy reductions in speed-to-product or speed-to-development,
if you can take a process that takes a year of designing a car and
shrink that to six months, that is a massive savings to a company.
Where I’m going with this is that, it offsets the cost. If, for
example, it costs me $10,000 or $20,000 for a VR headset that shaves
millions off of my design process, who cares what it cost at $20,000?
It doesn’t matter. You guys have the premium headset in the market.
Talk to us about the costs with assembling not just the headset, but
the whole package. What would something like that cost to fully
outfit one designer with a setup, the ideal setup?
Niko: It’s around the 10K mark
and that includes already the state of the art workstation. It’s very
similar than a few years ago, just to workstation alone from a
pricing perspective. We haven’t seen the price be really the issue of
adoption. It’s more about the day to day practicalities and how do
you work. But I think one great example has been on the architecture
and civil engineering side. For them, the benefits are huge. I feel
that they are fairly slow in picking up new technologies in general.
But if you think about what happened during the 90s, I was an
engineer at that point. Then I started my university studies by
drawing with a pencil to a big white paper. And when I ended,
everybody had transition to a CAD system. And that happened all
across the world, not just in the university, but all the companies
transitioned as well. So civil and architect engineers — from my
perspective — they are fairly slow to pick up, but once they pick
up, it’s a huge snowball effect that everybody does it at once. And
that’s something that I’m expecting and hoping that would happen as
well, from a VR/mixed reality perspective fairly soon.
Alan: Your timing is probably
very, very perfect because we’re also seeing this as well. You get
these proof points out and it’s only recently in the last kind of six
months where big companies are releasing their original pilot
information. We’re just starting to come out of pilots with some
companies. They’re going, “Hey, Lockheed Martin, we decreased
our time to production by 85 percent.” Macy’s is using VR for
sales in their stores and they– averaged across 110 stores, they’ve
increased their sales by 45 percent and decreased their returns from
7 to 2 percent. So once you kind of start to see these use cases, if
you’re going to remain competitive and your competitors are seeing
these kind of transformational shifts, it’s only a matter of time
before you have to do it. It’s not nice to have, it’s a must have. So
what is your timeline around where most companies start to roll this
out? Is it the 3, 5, 10 year rollout or what are we looking at?
Niko: We are seeing every
Fortune 500 company looking at this stuff. Some are doing trials and
pilots still fairly early on. Some are super advanced. It’s hard to
say a timeline, but what I can say is that the transition has
started, and this is going to happen. There’s been a lot of
discussion, whether VR or mixture reality. Is it real? Will it take
place? I mean, the consumer side didn’t work out as everybody was
expecting a few years ago. But from an enterprise perspective, from
the perspective of using this stuff to accelerate and make your work
more efficient, I think that has already started and there’s no way
to stop it anymore.
Alan: So what is the most
important thing that businesses can do to get started?
Niko: Take it seriously. The
minimum is, as I said, it’s very hard for externals. It needs to come
from the inside because you need to have the understanding of how
stuff works in the specific business in order to make it more
efficient. And obviously, consultants can come in and do the
interviews and then they provide you an answer. But it really has to
come from inside. So put the few creative people, give them a mandate
to study and really understand what mixed reality in the next few
years can do. And I think it will be very easy to come up with three
to five things that yes, could actually make our life a lot easier
from a business perspective. And then just find a partner with whom
to build a pilot. It’s an upfront investment that companies will need
to do within their own context, but it’s going to be worthwhile.
Alan: I agree. You know,
recently we started XR Ignite, which is a community hub based on the
idea that startup studios, developers, they all have a role to play.
And a lot of VC investment funds only look at platforms and products,
and they’re looking for giant billion dollar unicorns. What we’re
seeing is that there’s going to be a lot of acquisitions around the
studios and developers, because as big companies start to realize
that they spin up a small team, they go, “Hey, this is working
for us, we need to start doing this at scale. We can either hire
people, which is hard because not a lot of talent out there, or we
can acquire a small team that’s already working together, that
already has what we’re looking for.” And Deloitte just acquired
a startup studio, rather, and a couple of other companies have
acquired studios. What do you think the investment landscape is going
to look like? Because VCs don’t typically invest in studios and
content. They’re more invested in the platforms, where we took a very
holistic approach and said, “well, as a community hub, we can
all help each other get more clients, but also facilitate these
smaller acquisitions, because there’s going to be a massive need for
content soon.” How do you think we’re going to address that?
Niko: I think in general, the
appetite from a VC perspective and the interest in VR and mixed
reality, I think it’s turning, and it’s turning towards the positive
side. I mean, the past few years have been fairly difficult for many
startups working in this arena. It hasn’t been easy with the
software, whether platforms, whether content to get funding,
especially if it had a title VR in front of it. It’s a bit easier if
you were somehow in the augmented reality domain in the past years.
But I have a good feeling, and based on the discussions that I’ve had
recently, I think the– there starts to be a lot of appetite, and
this appetite is actually being fueled by these real world use cases:
the companies who haven’t played a role in the VR/AR before actually
showcasing. And I think the Volvo case is a fantastic example of one
of those cases. You would definitely not put them into a VR/AR
bracket, but hey, they are showing something that’s completely
advanced and then blows everybody out of the water at the moment. And
it’s coming from a company who you would position into a fairly
traditional automotive sector and not into the latest stuff of
computer graphics. So I think those showcases are fantastic, and it’s
gonna be easier in the next few years to get funding because there is
a clear need. There is– the business case is there, and demand is
strong. And I think the training and simulator market is a fantastic
example. I think there are thousands of companies — if you go to a
trade show focusing on training and simulation — there are thousands
of companies showcasing already a product. But they’re showcasing it
with hardware that actually doesn’t match the needs of the product or
is somehow inferior, but it doesn’t matter. Despite the fact that the
hardware hasn’t been up to the standards that they would need
actually, they have already created the product and somehow they
found the investment to create that product. And timing wise, it’s
from our perspective, it’s fantastic. We can give them now a product
that fulfills many of their needs and they can go and promote it
again with a complete new spin. I think it’s going to happen and I
think we will be seeing more and more investment in the space as
well.
Alan: I think we’re already
starting to see it. I think the trough of disillusionment when people
went, “Hey, we’re gonna– VR going to be huge! Everybody is
going to have it, and every consumer is going to wear AR glasses!”
Yes, this will all happen, but it will take 10 years. And I think
that the consumer market is really fickle. It has to be perfect,
cheap and lots of content, whereas the enterprise just needs to make
an ROI. And we’ve already proven that across every industry — almost
everything from oil and gas, to automotive, to training, to retail —
every aspect of business is being impacted by this.
But, let’s take this in a different
direction for a second. What problem in the world do you want to see
solved using XR Technologies?
Niko: Change the way how we
work. As I mentioned earlier, for me, that’s one of the big visions.
In the beginning, we wanted to change and create the future displays.
The transition from a 2D display — like a monitor or a cinema screen
— into a more immersive display. That’s still something that I see
as a transition perspective. But before that, for me, the big thing
is to change the way how we work.
Alan: One last thought I was
thinking, people buy all different VR headsets for different reasons,
for consumer reasons now. If you buy a Varjo headset and you’re
designing on it, can you still play Beat Saber on it?
Niko: Not yet.
Alan: Oh, you got to fix that.
[laughs]
Niko: Yeah, we’ll fix it. I
mean, that’s the problem with that is, that all of the games, they’ve
been optimized for the consumer grade headsets, which means very
limited resolution and two displays. So in our case, you would have
to drive four displays and you would have to have enough resolution,
so it would make sense.
Alan: Do you guys have an SDK
then, that helps people — developers, for example — create content
specifically for this?
Niko: Absolutely, yes. So we
have an SDK that plugs in into Unity, Unreal, all the game
development engines are fairly straightforward. We’ve also been
working quite extensively with the vertical engines like Lockheed
Martin, for example, with a CAD system so that you can you can use
our headset directly with those systems with which are not
necessarily on a gaming platform or available and then those consumer
type of stores. And then we are focusing quite a bit on the OpenXR
standard. And at CIGREF we’re actually showing our XR device with a
pure OpenXR demo as well. So I think OpenXR is going to unify quite a
bit all the different standards and it’s going to make everybody’s
life a lot easier.
Alan: I really, really hope so.
Having the ability to pull this up on web, being able to do what you
need to do and then switch between programs seamlessly, I think is
gonna be where this takes off. If we can get it right with
enterprise, it’s going to lay the groundwork and foundations for the
consumers to just enjoy the best resolution, the best everything at
the lowest cost possible. So, Niko, is there anything else you want
to share with the listeners?
Niko: It’s been a pretty
fantastic grind. And I mean, you experienced when we came out with XR
at AWE. For us, that was a huge milestone. It was our initial vision
to do a mixed reality device video see-through. We had to do a couple
of other things before that. So we had to develop our human eye
resolution. We had to do bombastic eye tracking technology. We had to
do the tech, we had to productize it. But the fact that we are able
to bring it to the market and create the product, that’s something
I’m super proud of. And it’s always a big milestone, actually,
instead of just dreaming and envisioning technology, but being able
to make the technology available on a commercial market, that’s a
fantastic achievement. And this is gonna be a fantastic year for us,
and I hope everybody else as well.
35 min
September 18, 2019 Combining the Best of AR and VR with Varjo's XR-1, featuring Niko Eiden
The human eye
is a wonderful and complex thing, and it's a technological feat just
to even come close to its natural revolution. Well, the folks at
Varjo have created something that is pretty darn good at it. Alan may
have trouble remembering how to say their company's name, but he can
attest to the clarity of the XR-1's display. Varjo CEO Niko Eiden
comes by to give us a look behind the curtain of its creation.
Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today we have Niko
Eiden, CEO of Varjo. He's formerly held top product leadership
positions at Microsoft and Nokia. At Nokia, Niko led a product
program team in 2006-2007, and together with researchers from Nokia
Research Center, his team developed the basis for the optical
technology that later became the Microsoft Hololens. Niko has a
Master's in science in aeronautical engineering. Varjo is redefining
reality by jump starting a new era in computing. Their hardware and
software lets people seamlessly mix realities together, moving from
the real world to extended reality into pure virtual reality, all
with human eye resolution. Their new headset, XR-1, is a mixed
reality developer device for engineers, researchers, and designers
who are pioneering a new reality. With photorealistic visual
fidelity, ultra-low latency, and integrated eye tracking, the XR-1
seamlessly merges virtual content with the real world for the first
time ever. If you want to learn more about Varjo, you can visit
varjo.com. And I want to welcome to the show, Niko. Thanks so much
for joining me.
Niko: Thanks, Alan. Nice to be
here.
Alan: It's my absolute pleasure.
We got a chance to meet at AWE this year and I got to try the XR-1,
which was -- oh my God -- what an incredible experience. You put on
the headset, the pass-through cameras were as if I didn't have a
headset on at all, I could just see the whole world. And then all of
a sudden a car appeared in front of me; in the space I was in, there
was a car. Then I got in the car, and the space around me
disappeared, and I was in the car. And the one thing that really
stuck out with me that blew my mind was. I was looking -- it was a
Volvo -- and I remember looking at the steering wheel. And the little
Volvo symbol in chrome was so crystal clear. It looked like a real
car. Let's talk about your technology and how you guys ended up at
this place.
Niko: Sure. We really had the
initial vision-- the founding team, we had a long background in
different augmented reality and VR devices. And we had always been
talking about that video see-through type of devices could actually
combine the best of both worlds of AR and VR devices. And every time
you started talking about them, somebody kind of commented basically
that don't bother: the latency, the lag between what you're seeing
through the cameras, and what's happening in reality is gonna be too
long. Or the resolution is not going to be good enough, it's not
going to look great. But we were in the summer of 2016, we were
looking at a demo and the demo was shown with the Hololens first gen
device. And I got really thinking that this would be such a cool
demo, but it's really missing a big part of the experience, because
it wasn't able to show the image in a photorealistic fashion. It was
a very high fidelity graphic scene that we were looking at with
Hololens and we were just thinking that this would be so much better
with a VR device. And combining the reality with a video see-through
device could actually really work. We were a bit curious. We had some
extra time at that point together with Urho [Konttori] -- one of the
other founders -- and we built our first prototype in 24 hours, and
the experience was really magical. So it was a very simple
experience. But the main thing of that first experience was that we
were able to dim an existing room and make it
0 min
September 16, 2019 Helping Firefighters Douse Blazes Around the World, with RiVR’s Alex Harvey
Firefighters
need to train like any other professional, and their training usually
involves setting a mock set ablaze – which, as you might imagine,
would be costly to reset. Enter RiVR, who are using 360 video and
photogrammetry to recreate these practice blazes digitally. CEO Alex
Harvey and Alan have a heated discussion on the topic.
Alan: Hey, everyone, my name is
Alan Smithon, your host for the XR for Business Podcast. Today we
have Alex Harvey, CEO and creative director at RiVR, a virtual
reality training and visualization company based in the UK. RiVR
harnesses the power of VR and photogrammetry technology to create
interactive, immersive training experiences. They’re currently
working with the UK Home Office, UK Fire Service, Police Service and
the Department of Defense in the US. Their ultimate goal is to
enhance the way humans learn (I love that). Alex has a deep
understanding of the games industry, having worked on commissions for
the likes of Codemasters, the BBC, and Ford Motor Company. He’s
obsessed with harnessing the latest A/V technology to make the real
world differences that we all need. He gets to work with incredibly
talented people to make this happen, and to quote him, “I love
the feelings and memories we can evoke in VR when technology,
creativity, and innovation collide.” I love that quote. RiVR’s
exhibited at six different VR shows this year, including CES Vegas,
and their technology has been reported on by the BBC. To learn more
about RiVR, you can visit rivr.uk.
Alex, welcome to the show, my friend.
Alex: Hi, Alan. Nice to meet
you. Nice to speak again.
Alan: Yeah. We’ve been kind of
back and forth on LinkedIn, and emails, and it’s really finally great
to sit down and have a conversation with you.
Alex: It is such a busy world,
and it’s great to chat in person.
Alan: Listen, let’s dive right
into this. Explain to us what RiVR is and how it’s making a
difference.
Alex: RiVR is “Reality in
Virtual Reality.” We’ve been creating VR experiences now for
probably nearer to three years with the production company, starting
back in 2014, but we started obviously with 360 video doing things
for Thomson Holidays — you experience what it’s like to be on a
cruise ship, or be on a plane. That was three years ago. Then we
started moving into the room-scale photogrammetry world, with very
much a significant push at RiVR for training, and using photorealism
to make sure that the users of our experiences are completely
immersed. I often say to people, “I want you to feel like you’re in
the world, and not in a Simpsons cartoon world.” It is very much
pushing photogrammetry and photo realism into VR. You know, there’s a
lot of people doing photogrammetry now, but two, three years ago? It
was only of the likes of–
Alan: That was you and Simon!
Alex: Yeah! [laughs] Me, Simon
and Realities.IO. They were the guys that were pushing it. And it
really felt like when I saw those early experiences of Realities.IO
and Simon’s stuff, it felt like I was inside a video, but not quite?
I want to try and be inside video content. I think that–
Alan: Let me kind of unpack this
fruit for people listening. So, what Alex and his team do is they go
into a space, and they will take hundreds of photographs — if not
thousands of photographs — of the space, and they’ll convert that
into a game engine-based experience, where you can actually walk
around. Now, what I think is really mind-blowing about what you guys
have done at RiVR is, not only do you create the environment, but
then you take specific parts of the environment — for example,
you’re doing fire recreation studies, and one of the things that you
can do is pick up the different items — and I thought that was
really cool because they look photo-real — they look like they were
part of the scene, and you can pick them up, investigate them, look
underneath them.
Alex: Yeah.
Alan: And the way you guys have
done it is incredible. If you want to take a look at this while
you’re listening to this podcast, go to RiVR.uk. Just look at the
video that’s on the main page. It really explains a lot.
Alex: Yeah. I mean, that was…
I should go into… and I do often, as I think, you know, everyone
does in this industry; we dive into terms like “photogrammetry”
and “photorealism,” but we do need to explain what they are a bit
more. So let me just quickly go into that.

When we recreate those fire
investigation scenes, we burn a real world container. And that’s how
the fire service train their fire investigators all around the world,
currently. But then they have to go into that container with a
certain amount of people, and then they have to — in a week’s time
— do it again. So there’s no consistency in that training. We do the
same thing; burn the container, and then we put the fire out, and
then there might be a hundred items inside that burned container. We
take each item out, one by one, and scan each item from every angle,
using a 12-camera photogrammetry rig — you can see stuff on the
website, actually — and it gets a photo from every angle of the
object. And then we put it back into the software, create really
high-quality, photorealistic model of each item, and then we rebuild
the container as it was in the real world and then allow you to pick
up every item, look underneath it and find the cause of the fire.
We also use 360 video to show you at
the end of it — for your learning outcome — where the fire actually
started. And we’re doing that for the crime scenes as well as the
fire scenes.
Alan: I noticed on your LinkedIn
page or Facebook — I can’t remember — but you guys melted a camera,
a GoPro, the other day.
Alex: Yeah, the GoPro Fusion.
Thanks, GoPro! They do supply us with quite a few cameras. We melt
quite a few GoPros. And also, yeah, the Samsungs get a bit melted. If
anyone knows Kirk McKenzie in Consumers Fire Department in the US
over in Sacramento, he is working with GoPro on their next camera, to
make sure it’s a little bit more sturdy.
Alan: [Laughs]Yes! “Can you please make as the camera doesn’t melt in our
fire?”
Alex: Yeah.
Alan: We actually melted one of
the Samsung Gear VRs back in the day. And I don’t remember how it got
melted. I don’t even remember, now.
Alex: On the last burn, we
actually had to put two Samsungs in and one Fusion. And we pulled the
Samsungs out at different times just to make sure we had a cause of
the fire, up until it gets too dark.
Alan: Yeah. Well, you gotta get
it out before it melts.
Alex: Yeah.
Alan: It’s really amazing work
that you guys are doing. How is this translating to real world
benefits? Because you’re taking a thing, you’re burning it, you’re
then putting in VR. What benefit does that give the fire service?
Alex: At the moment, when they
do the fire investigation training — like I said — they put 20
people through. But if you’re the last person, all you see is a load
of footprints in the container. It doesn’t look like a real scene
would look, because it’s been stomped through by loads of firemen. We
can press reset and give consistent and repeatable training to
everyone, all around the world.
I guess I’ll touch quickly on RiVR
Investigate, which we’ve now got our own facility, where we have
containers and we work with fire investigators to recreate the
scenarios of different types of fires. In September, when Investigate
launches, there’ll be six different fire investigation scenarios, and
two crime scenes scenarios. This gives people the ability to be…
you can be in the scenes together, so you can be in different
locations around the world, but all looking in the same scene. You
can record the whole training scenario, so you can see from any angle
how people pick the items up. And you’re teaching people, because of
the photorealism, about the burn patterns and the smoke patterns and
the fire behavior of a single burn.
Eventually, we are going to have a
library of these scenarios, and the fire service around the world
will not have to create these scenes as much. They can just put a
headset on. And at the moment, there’s like a three-day training
course to do a fire investigation, and they have to take fire
fighters off-duty to go on the course. Well, with this product, you
can be in VR, in the fire station. And if the bell rings, you just
take the VR headset off and go out.
Alan: Oh, that’s amazing. So
this is like a massive time savings.
Alex: Yes. Yeah.
Alan: Are you also seeing an
increase in… because I know in most VR training that we’ve been
seeing, there’s also an increase in retention rates.
Alex: Yeah. Yeah. I mean, I’ve
heard the stuff that Alvin Graylin at HTC talked about with the
knowledge retention being increased by 6x in VR, and we’re actually
working with Coventry University, Leicester Fire Service, and
Derbyshire Fire Service, and they’re evaluating it as we speak. So
they’ve got real fire investigators going through our scenarios, and
they’re comparing it to the real-world training. Like, they’re
comparing it with a real fire investigation and courses, just to see
if that knowledge retention is more. And we’re also putting people
that are non-fire-investigators through the courses, to see how they
retain the information.
Alan: Oh, that’s amazing. How’s
that coming along?
Alex:
It’s great. I’ve been down, actually, and seen the guys using
it. And we’ve got it set up next to a real fire investigation burn in
Derbyshire for their courses. And yeah, so far, the feedback is
amazing, and it’s really good seeing it. It makes everyone at RiVR
know that we are very much on the right track.
Alan: Yeah, I think one of the
main things is that this is scalable training. That’s what people
don’t understand, is that the current training methodologies —
especially in fire services — it’s not scalable. You have to have
every employee travel to a place that has this physical space, where
virtual reality, it can be anywhere. You can ship it to anybody,
anywhere.
And the other thing that I think it’s
really amazing about what you’re doing is you can standardize the
training. Whereas, if you have a fire brigade in the south — and
maybe they train differently than the ones in north — and everybody
gets a different training.
Alex: Yeah, totally. And
everyone trains different around the world. So, this gives the
ability for the UK to see how the Americans do it, and vice versa.
What we’re getting at the moment is, we did quite a few shows around
the US this year and last year. But what we’re finding is that the
fire services in the US, they see the UK burns and they love them,
and they can learn from them, but they want to have their own style
burns, because there’s a few things that they do differently. So what
we offer is the ability for the RiVR team to come over to the US –
or, we’re doing some in the Netherlands with Martine at the moment as
well — where we go and burn with those guys, and scan two or three
of their burns, and put those burns into the library. So, it’s more
like a global fire investigation and crime scene training scenarios.
Alan: That’s just amazing,
because now, rather than standardize it across just the UK, you’re
able to take the best of everywhere in the world.
Alex: Yeah. Yeah.
Alan: And
provide it to everyone else. Like, that alone, I think, is
going to be a critical point in how VR and AR start to really make a
difference in learning. Because now you can learn every which way.
Alex: Yeah, you can learn from
everyone, and wherever you are is irrelevant, as long as you’ve got
a machine.
Alan: What are you using for
hardware?
Alex: So at the moment, yeah, we
are on VIVE and Rift, and just a high-end PC, and we offer wireless
or they can have wired; it’s up to them. But we have seen some good
results from the dev guys testing our stuff on Quest — Oculus Quest.
Alan: Oh, wow.
Alex: So, there will be a
version eventually for Quest. And the price difference is obviously
very appealing to some people.
I was just going to say, that a lot of
the guys in the Middle East come over to the UK and the US to learn
from those guys about fire investigation and firefighting. And this
is going to allow them to have much more easier access to that
training.
Alan: That’s really incredible.
Just for the people listening, In case you’re not familiar; HTC Vive
and Oculus Rift are computer-based systems, where you’ve got to plug
them into a computer and have a fairly decent computer. Your total
build cost is around maybe $5,000 — somewhere in around here. But
the Oculus Quest is this completely standalone unit; it costs $399,
and you can walk around and do all the same things, just a little bit
lower-quality.
Alex: Yeah. So we think… we
have RiVR Link which is — I’ll talk about that shortly — but that
allows classroom-in-a-box for 360 video. But I think eventually,
we’ll have room-scale training in a box. You won’t have to have PCs,
especially when we start talking about VR compute in the Cloud,
straight down to headsets. That’s probably [some] out-there bit of
thinking.
Alan: Incredible. Let me ask
you, what are the ways you’re measuring success? Goals, key
performance indicators? How does somebody measure the success of
this, compared to what they currently measure?
Alex: At
the moment, that is a massive hurdle for a lot of companies to get
over, because there is no way, really, of measuring this new training
technology. We — like yourselves; you’re probably speaking to lots
of companies, and you go back and forward quite a lot with them —
but they can’t prove the return on investment, so it’s very hard to
get them to sign things off. The only way we do measure is by doing
studies on the systems and stuff that we’ve put out there already.
Like with the fire service, and like with 360 video for bus driver
training in London. It’s all happening now. So we’ve made the
content, but they’re now evaluating it all. There’s a lot of projects
probably that are out there at the moment that we’ve created, that we
haven’t got all the results back from yet. I think we are still very
early on that.
Alan: When we first started out
in this, the first questions from everybody was, “who else has done
it?” Because nobody wants to be the first. And then, “what’s the
ROI and the KPIs,” and you’re like, “well, no idea.”
Alex: [laughs]
Alan: They’re like, oh, “how
much does it cost?” “A lot!” [laughs] It’s not the best sales
pitch, but there is definitely more studies coming out. Wal-Mart
published some information, showing that they’ve had a 70 percent
increase in retention rates and a massive decrease in the training
times, which is a time-is-money kind of thing. And if you can
decrease training times…
Alex: The Wal-Mart story is an
amazing stand-out, large company story.
Alan: So, what are some of the
challenges you faced when starting out?
Alex: The main challenge that we
have when we’re pitching these experiences to people is they know
they want VR, but they just see VR as VR. They don’t understand the
difference, a lot of people, between 360 video and room-scale VR. So
every single demo — I imagine, like yourself; you’ve done hundreds
of demos — the first thing is to go through some very basic demos
of, “this is 360 video, and this is room-scale VR.” And then I
have to go through the whole, “this is room-scale, photo-realistic
VR, and this is room-scale Simpsons VR,” if you like.
Alan: For
the people listening, let’s just unpack it here, one at a
time. What’s the difference between 360 video versus room-scale VR?
Alex: So, 360 video; you can
film with a 360 camera that normally would stay static in a scene.
You can move it sometimes on a drone or on a rover. But the 360
camera films from the perspective of your head, if you like, and you
view that content by sitting in a chair normally, and only moving
your head around. You’re basically moving your head around inside a
bubble that is a video wrapped around your head. You’ve got audio and
you can look around — up, down, left, right — but you cannot stand
up and move around, and you cannot stand up and pick anything up.
As you move into room-scale VR — if
you were to take that headset off, put another headset on — you can
then stand up, step forward and pick up a chair in front of you, for
example. And those things — the getting up and walking around bit —
has moved on so much in the last year that now, like you mentioned,
the Oculus Quest has come out and that has inside-out tracking. So,
you move around the space with the headset that has no external
trackers looking at the headset. It’s just got cameras looking at the
floors and walls, and it knows where you are in relation to the
floors and walls. That’s how I always explain it, if we’re just
talking about explanations; sit down on a chair, look around
with your head; or if you want, the more… I call it, like, the more
muscle memory-intensive VR, where you might want to teach people to
pick things up, or use things in certain ways. Then you need to be
able to have room-scale and walk around.
Alan: Yeah, I agree. Gives you
the muscle memory, too.
Alex: But the 360 video has a
massive part to play in training, and also entertainment experiences.
That’s probably a good point to just
mention RiVR Link, unless you wanted to do that later.
Alan: Sure, we might as well
talk about it; what’s RiVR Link?
Alex: Yeah. So RiVR Link. We’re
working at the moment with Pico — with their Goblin headsets.
They’re similar to the Oculus Go, and the software can actually run
an Oculus Go as well. But it is simultaneous playback of a 360 video.
But independent viewing; so you can look up and down, and look around
— wherever you want to look — but you can be in a classroom of
people, up to 50 people in a classroom (maybe not that big).
At the moment, we have sets of 15 headsets. We put them all into a
massive Peli case, with all the things that you need to link them
together, and they talk to an iPad.
You hold up the iPad as the teacher,
and all the headsets have the 360 content on them, and everyone puts
the headsets on. The teacher can now press play on any of the 360
content that is on the tablet. And they can talk through the content,
and they can also pause and then draw on the tablet, allowing every
headset in the room to have the annotations appear over their video
in a paused state, or in a playing state.
Alan: That’s really cool.
Alex: If you’re a company — for
example, we’re doing one in London for bus driver training — and if
you want it to look like it’s your product rather than RiVR Link, you
can brand the app in the headset and on a tablet to look like it’s
your app, with your videos on, so it basically looks like you’ve made
your own 360 viewing app.
Alan: Wonderful.
Alex: That’s rolling out in…
well, it is in multiple places, already.
Alan: Do
you know Jeremy Dalton at PWC, in the UK?
Alex: No, no.
Alan: I’ll have to introduce you
to Jeremy. They did an experience the other day for 200 simultaneous
headsets.
Alex: That’s cool. I think I
heard on the Alvin podcast that they are also looking at a similar
type of solution. It is one of the big pain points of showing 360
video; you can’t see what the people are seeing. So we give you the
ability to look, pause, and draw on the screen of all the headsets.
Alan: Wow, that’s so cool.
Technology is amazing.
So, when you guys are building these
things, like… how many people are on your team? How many people in
developing, and then people doing photogrammetry, and then people out
doing the demos?
Alex:
At the moment, there are 15 people at RiVR. That’s grown quite
a lot over the last two years. But the main people that are out there
doing the shows are… well, we get everyone involved really, but:
me, Ben, Brad — we go out doing most of the demos, and then there’s
a lot of guys that are in the office creating, doing the hard work.
Alan: I noticed you guys have
garnered some pretty amazing media. I know you were on the BBC; can
you maybe talk to that, and how that came about?
Alex: The BBC have been very,
very interested. We’ve done three pieces now. One of them was BBC
Click, which is a tech program every week in the UK — which I’ve
always watch for years, so I was super excited to be on that. All of
that stuff’s been with Leicester Fire Brigade and Paul Speights, the
— and Mike Ferguson from the DSTL — because they allow the BBC to
come into their establishments, and have a go on their experiences
that we’d made for them. They did the crime scenes and they did the
fire investigation scenes. And then also, we were also on The Gadget
Show as well, which is–
Alan: I’ve actually been on the
Gadget Show.
Alex: Have you? That’s great.
Alan: I have! With my last
company, Emulator.
Alex: Yes, you definitely know
that one, then.
Alan: Yeah, it’s funny, because
I don’t think I’ve ever watched it, but I know it was out there.
Alex: Yeah.
Alan: I’ll have to dig it out
the archive somehow.
Alex: Definitely; get it on
LinkedIn.
Alan: Yeah. No kidding.
Alex: Yeah. BBC has been very
kind.
Alan: I’ve seen a bunch of stuff
about VR from the BBC, and it seemed to be very, very supportive of
this technology. I think they really see the benefit to it, and we
need more of that.
Alex: The interview was quite
hard. They weren’t super kind; they wanted to have some hard
questions. So I think they dug quite deep on the last one. They
didn’t publish it all. But working with the fire investigators and
the crime scene investigators that were skeptical when they came to
us to work with us is a great thing, because they find out that most
people are skeptical when we say “we can take your training and
make it repeatable and consistent.”
Jason Dean, the fire investigator that
works with us on our scenes, he says when he first came to work with
RiVR, he was very skeptical because he was a sort of
dirty-trowel-and-overalls type of guy, that was very much of the
opinion that you could not replace the training with virtual
training. But he now gets on his knees and — in one of the videos I
shared with you a minute ago — he gets on his knees in one of the
scenes, and he’s reluctant to go down on his knees because doesn’t
want to get his trousers dirty.
Alan: [laughs]
Alex: He’s in his nice clothes.
Alan: It feels so real.
Alex: Yes, yeah. So we do have a
demo that people can request access to, as well.
Alan: Oh, I’d love that,
actually. We’ve got VIVE in the office here. We’d be happy to take a
look at it, I would love it.
Do you have any, I guess, aspirations
to make this available on, like, Steam or something? Because maybe
there’s some kids out there that could be inspired to become
firefighters because of it.
Alex: We’re working quite
closely with FLAIM Systems. Theirs would be potentially better for
Steam. Ours is a bit more for training; I’m not sure if our
experience would translate well to Steam. We’ve thought about doing a
Steam experience, but maybe not for an investigation. The potential
is there to do a photorealistic crime scene on Steam. That might be
quite cool.
Alan: That would be really neat.
“CSI: VR!”
Alex: Yeah, definitely. Like a
very much interactive Cluedo.
Alan: Yeah. Oh, that would be so
cool.
Alex: Yeah. That would be like
you’re actually in the house, I think.
Alan: Yeah.
Alex: Those sort of experiences
do interest me a lot. I do think that in the future, there’s gonna be
a massive need for every film and TV show to have a photorealistic
experience in VR that sits alongside it.
Alan: Yeah.
Alex: I don’t think people will
be watching 360 versions of films. I think they’ll remain 2D for
quite a while, but they’ll have an interactive five-minute experience
that is downloadable, alongside the TV show or film.
Alan: I agree with that,
actually. And we’re already seeing it. Ready Player One had a couple
of experiences, actually, when they launched the movie. You could go
in and experience what it’s like to be inside the Oasis.
Alex: Yeah. I mean, let me just
touch on that a bit, because I did rant on about that on LinkedIn
recently. These guys are making these experiences for the films, but
they’re not giving you experiences from the film. They’re
mainly be giving you experiences that they think you’d like to play.
But if you watch Game of Thrones, the Game of Thrones experience that
came out recently wasn’t actually a scene from the film. I want to be
sat at the Red Wedding. I want to be on the wall with Jon Snow. I
want to be part of the scenes – photorealistic — rather than part
of a White Walker slaying.
Alan: I agree. And it’s
interesting, because we’ve seen that with a couple of things. Now,
one that was very true to it was… have you seen Silicon Valley, the
show?
Alex: Yes, I know what you mean.
There’s lots of objects to pick up. And I think I’ve seen it but
never played it.
Alan: Yeah. You’re in the Hacker
Hostel.
Alex: Yes. Yes.
Alan: They took all the footage,
and they took the exact specifications, and made it. I think that’s
another UK-based company. Solomon Rogers’ company.
Alex: Yeah, REWIND, yeah.
Alan: Sol is going to be on the
podcast coming up soon, as well.
Alex: Oh, great. Yeah. I spoke
to him recently. I remember that one. And you’re right. That is
exactly what I mean. You want to be inside the film. Like in Ready
Player One, I want to float around in that bar with that music on.
Alan: Cool. Yeah. That was
awesome.
Alex: Yeah, definitely. There’s
a market there. We’ll have to see that coming soon, hopefully.
Alan: Let me ask you a question.
When you first started doing the photogrammetry stuff, what are the
best lessons you learned from doing those projects, that you can pass
on to businesses that are trying to wrap their heads around this?
Because there’s so much technology to unpack. People’s heads must be
exploding when they’re thinking about this. And then I think what
happens is they get overloaded and just say, “well, screw it, we’ll
just do our iPad training as usual.”
Alex: “Go back to eLearning.”
Alan: Exactly.
Alex: That is a super hard
question that I don’t really know. [chuckles] I would say when we’re
speaking to companies, don’t always go for the shiny, photorealistic
photogrammetry stuff that might be more expensive. I think there is
normally a Stage One/Stage Two that companies can do. Stage One would
be: let’s do some 360 3D video first, and let’s have that in a
classroom setting, and take some of their training and put it into
360, because it’s very easy to film that sort of stuff. And then I
sometimes say, let’s go for Stage Two. If they think they see a
return on investment for a 360 video, then they would definitely have
some sort of return. If their training involves a process that can be
repeated easily with room-scale VR.
Alan: One of the things that we
give advice to companies is exact same thing. “Start with 360
video.” Maybe make an AR app, test it. I think that is really
practical advice. It’s great to sell people on the best possible of
everything, but it’s not always necessary. And I think 360 video, if
done right, can create a very good return on investment, because the
costs are marginal compared to doing a full photogrammetry of a
scene, and bringing in a new game engine, relighting it, all of the
rest of it.
Alex: It’s been a big learning
curve for me. I work with Brad and Ben quite closely with the pitches
to different clients, and I’ve had to rein myself in because I want
to make VR photorealistic scenes for everyone.
Alan: Yeah.
Alex: But RiVR Link and 360
video content is more than adequate in most cases, and is a lot
cheaper. So yeah, only go for the high-end stuff potentially after
you’ve done Stage One, unless you already have had a bit of Stage One
from someone else and you’ve done some 360 content. Then maybe you
could move on to some room-scale stuff.
I always say to people that they’re —
like with the fire service and the crime scene, they try and really
spec out in super detail what they’d like — and I say to them,
“however you do your training now is our design brief.” We’re
just going to copy it. It’s a digital twin of what you already do.
Sometimes people try too hard to design things in super detail. With
the fire service, you don’t have to design it, because you’re already
doing it in the real world. You’re already burning containers and
creating crime scenes. We just have to come along and scan them after
you’ve made them. So, don’t do anything different. You already know
what your learning outcomes need to be from these scenes; we’re just
going to make it digital for you, and put it in VR.
Alan: So that’s some really
practical advice, because it also saves costs; you’re not recreating
the learning, and that’s what people really need to wrap their heads
around. That VR and AR, and then these technologies, they’re not
going to create a completely new, revolutionary education out of the
world, like, “yeah we’re inventing the thing from scratch.” No,
it’s it’s just making it better.
Alex: It’s going to allow you to
have repeatable and consistent [results]. I always said to the dev
guys — they wouldn’t let me put it in there — but I wanted a big
red button in every scene. The big red button is the reset button.
Mess the scene up. Play around with the world. Do whatever you want.
Do some learning, and record all the things you do. But at the end of
it, the bonus is that when you press this button, you don’t have to
reset the real world scene. It just does it for you in a millisecond.
Alan: You just touched on
something interesting. How are you measuring things? Like, what are
some of the analytics that you’re gleaning from these experiences?
Alex: Yeah. So with RiVR
Investigate, we have a product that attaches to it, or is part of it.
It’s called VRM: Virtual Reality Monitor. And in that software, it
records all the data of everyone’s movements inside the scene —
where they’ve looked, what they’ve touched, heat maps. Obviously,
looking at the… we’ve got the VIVE, the new VIVE eye tracking– I
always forget the name of it. Vive Eye Pro, can’t remember what they
call it, but–
Alan: VIVE Pro Eye.
Alex: Yes. Sorry. It tracks your
eye. Where you’re looking, rather than just where your head is
turning. So we also incorporate that into the VRM. But the beauty
with the data that comes out of that — and the way that the
developers have made it at RiVR — is that you can replay a training
scene after the fact of anyone that’s been through, pause it at any
point. But then instead of just being able to pause and look at the
screen, you can pause it and look around the scene at that point. So
like The Matrix — bullet time.
Alan: Ah, cool.
Alex: Which is amazing. Another
feature on the VRM is the ability to ingest point cloud data. Laser
scan data from different scanners can be ingested into the RiVR
system. And I always think it’s one of the most underrated things
that people don’t really know about yet, is to be able to view point
cloud data in VR. It doesn’t give you the fidelity that you’d
normally have with the photorealistic scenes, but it gives the most
incredible context to a scene than you can ever get from a 2D screen.
And there’s crime scenes and fire investigation scenes from all over
the world — car crashes — and they’ve got laser data of all of
these scenarios, but they all view them on a 2D screen. This gives us
the ability to view it spatially in VR.
Alan: Interesting. Yeah, because
I know the police service in Toronto has been using laser scanning
for years.
Alex: Yeah. And people find it
very hard to translate what they’re seeing on a 2D screen after
they’ve scanned a real-world scene with a laser scanner. We’re
incorporating that into it. And you can be in there collaboratively,
and measuring points inside the scene. Imagine inside a courtroom,
when you could have all of the jury — they don’t go out to the crime
scene anymore; they can just put a headset on a walk around the point
cloud and they can see where, for example, the body was, where the
car was in relation to the gun or the knife. It’s just that spatial
viewing system, really.
Alan: I’ve been wondering why
police services are not using Matterport cameras.
Alex: They’re very good and
quick.
Alan: People who don’t know:
Matterport is a US company that built a camera with a laser scanner
built into it. And you just put it in, it spins around in a circle,
takes 360-degree images with point cloud data, and then allows you to
do that through the scene. And now you can move around the scene in
VR as needed, to any point that you were there.
Alex: And just touch on it a
little bit, Alan, because it isn’t as detailed as a laser scanner.
The Leica System and the FARA systems give you really accurate data.
The Matterport gives you depth data, and also imagery, so that you
can go from hot spot to hot spot. And it gives you a dollshouse
effect. So it’s a very good scanner for quickness, but it doesn’t
give you that millimeter-accuracy of the point cloud.
Alan: But it is, I think, for
juries, something like that, where you just need to be able to move
around the crime scene. It doesn’t need to be millimeter accurate.
They still have the laser scans. I think–
Alex: I met with those guys in
Florida earlier this year — the Matterport guys — and we talked
about the potential of bringing Matterport data into the RiVR system
as well, to be able to view that alongside the point cloud data.
Alan: It just totally makes
sense. So, we’re getting near the end of the podcast here. What is
the most important things that businesses can do right now to start
leveraging the power of XR technologies?
Alex: It depends what business
they are, really, I guess! But you mean, in terms of which they
should go for first? Or should they just… well, they should
definitely listen to your podcast, so they can understand it more,
because you’ve had some great people on.
Alan: Well, thank you. One of
the the answers that comes up a lot is just… just start.
Just go. It doesn’t matter what we do. Do something, because
there’s so many of these technologies. AR, VR, mixed reality,
Hololens, virtual reality headsets, photogrammetry scanning, 3D
models. It gets confusing, and people don’t understand; this is the
future of computing. And if they don’t now, they’re going to be left
behind. It’s like the early days of the web and everybody went, “oh,
why do I need a website? Nobody is on there.” Well, guess what?
There’s a few people on the Web now.
Alex: Yeah. Just a few. I think
the main thing that they need to look at and ask themselves is, how
expensive is their training to reproduce? In terms of training,
anyway. Is it expensive for them to do it? And is it dangerous for
them to do it? If the answer is yes to both of them, they should
definitely start looking at VR. I should imagine there are some
companies potentially where it’s not as needed, but if it’s expensive
and dangerous, then it’s the best thing that they can start doing.
Alan: Yes. There you go: if it’s
expensive and dangerous, start using VR.
Alex: It will save you money and
provide better results.
Alan: So, Alex, here’s my last
question: what problem in the world do you want to see solved using
XR technologies?
Alex: Oh, god, I should have
done some research on that one. Problem in the world… I’ve just
been on holiday, and it was quite a shock to see how many people are
on holiday, glued to their phones. Maybe there’s a chance that, if
Apple or someone come out with a really nice AR device, we might be
able to fix the problem with people being glued to their phones.
They could then just be looking up and
walking around with an AR headset on, and enjoying the world and the
mixed reality stuff as well. I think that is a bit of a problem at
the moment, because it’s not normal to be looking at a screen, but it
is normal to be walking around the world. So I think, yeah, there’s
going to be some sort of merger that removes the fact of you just
looking down at a phone. I want to be seeing the world with data on
top of it. I think that’s going to be how things go, and that would
be a problem solved if we can stop people bumping into things and
crashing into people out there, looking at their phones.
Alan: Well, I’ll put it this
way: I have the North glasses, and the first day I had them, I was
checking my messages and walking down the street, and I’m just
messing around with them, trying to figure out how to use them.
Alex: Yeah.
Alan: I almost walked over some
poor woman. I was not paying attention, and I was looking at the data
in the screen; didn’t even contemplate this woman, and almost pulled
her right over.
Alex: Well, I see; more
problems, then.
Alan: Yeah, I’m not sure. I
think there might be some unintended consequences here.
Alex: Yeah, definitely.
41 min
September 16, 2019 Helping Firefighters Douse Blazes Around the World, with RiVR's Alex Harvey
Firefighters
need to train like any other professional, and their training usually
involves setting a mock set ablaze - which, as you might imagine,
would be costly to reset. Enter RiVR, who are using 360 video and
photogrammetry to recreate these practice blazes digitally. CEO Alex
Harvey and Alan have a heated discussion on the topic.
Alan: Hey, everyone, my name is
Alan Smithon, your host for the XR for Business Podcast. Today we
have Alex Harvey, CEO and creative director at RiVR, a virtual
reality training and visualization company based in the UK. RiVR
harnesses the power of VR and photogrammetry technology to create
interactive, immersive training experiences. They're currently
working with the UK Home Office, UK Fire Service, Police Service and
the Department of Defense in the US. Their ultimate goal is to
enhance the way humans learn (I love that). Alex has a deep
understanding of the games industry, having worked on commissions for
the likes of Codemasters, the BBC, and Ford Motor Company. He's
obsessed with harnessing the latest A/V technology to make the real
world differences that we all need. He gets to work with incredibly
talented people to make this happen, and to quote him, "I love
the feelings and memories we can evoke in VR when technology,
creativity, and innovation collide." I love that quote. RiVR's
exhibited at six different VR shows this year, including CES Vegas,
and their technology has been reported on by the BBC. To learn more
about RiVR, you can visit rivr.uk.
Alex, welcome to the show, my friend.
Alex: Hi, Alan. Nice to meet
you. Nice to speak again.
Alan: Yeah. We've been kind of
back and forth on LinkedIn, and emails, and it's really finally great
to sit down and have a conversation with you.
Alex: It is such a busy world,
and it's great to chat in person.
Alan: Listen, let's dive right
into this. Explain to us what RiVR is and how it's making a
difference.
Alex: RiVR is “Reality in
Virtual Reality.” We've been creating VR experiences now for
probably nearer to three years with the production company, starting
back in 2014, but we started obviously with 360 video doing things
for Thomson Holidays -- you experience what it's like to be on a
cruise ship, or be on a plane. That was three years ago. Then we
started moving into the room-scale photogrammetry world, with very
much a significant push at RiVR for training, and using photorealism
to make sure that the users of our experiences are completely
immersed. I often say to people, “I want you to feel like you're in
the world, and not in a Simpsons cartoon world.” It is very much
pushing photogrammetry and photo realism into VR. You know, there's a
lot of people doing photogrammetry now, but two, three years ago? It
was only of the likes of--
Alan: That was you and Simon!
Alex: Yeah! [laughs] Me, Simon
and Realities.IO. They were the guys that were pushing it. And it
really felt like when I saw those early experiences of Realities.IO
and Simon's stuff, it felt like I was inside a video, but not quite?
I want to try and be inside video content. I think that--
Alan: Let me kind of unpack this
fruit for people listening. So, what Alex and his team do is they go
into a space, and they will take hundreds of photographs -- if not
thousands of photographs -- of the space, and they'll convert that
into a game engine-based experience, where you can actually walk
around. Now, what I think is really mind-blowing about what you guys
have done at RiVR is, not only do you create the environment, but
then you take specific parts of the environment -- for example,
you're doing fire recreation studies, and one of the things that you
can do is pick up the different items -- and I thought
0 min
September 14, 2019 The Bugatti Built in VR – XR For Business Podcast News, September 13, 2019
LEARNING
1. Developing Communication Skills And
Improvement of Literacy
2. Enlarging Real-Life Experience
3. Supporting Specific Needs of the
Disabled Students
4. Developing and Improving Empathy
5. Receiving Workplace Experience
Read
more:
https://programminginsider.com/5-real-uses-of-virtual-reality-in-education/
Mixed
Reality Classroom
The Mixed Reality Classroom
Microsoft has created an ‘Immersive
Classroom’ that teaches students about all sorts of emerging and
exponential technologies such as VR/AR, AI and 3D Printing. it
incorporates Paint 3D, HoloLens and other Mixed Reality headsets, and
3D printing. Other activities also explore coding and robotics using
tools such as Minecraft and Artificial intelligence, where students
get to build their own chatbot.
The immersive classroom is open for
schools to experiment with incorporating digital tools in learning,
and has facilitated the delivery of more than 60 free workshops for
over 1000 students since opening in 2018
A.I.
and virtual reality can determine neurosurgeon expertise with 90 per
cent accuracy
http://www.thesuburban.com/life/health/a-i-and-virtual-reality-can-determine-neurosurgeon-expertise-with/article_08c12c76-d0b9-11e9-a6c2-6f29fcf49393.html
Machine learning-guided virtual reality
simulators can help neurosurgeons develop the skills they need before
they step in the operating room, according to a recent study.
Research from the Neurosurgical Simulation and Artificial
Intelligence Learning Centre at the Montreal Neurological Institute
and Hospital (The Neuro) and McGill University shows that machine
learning algorithms can accurately assess the capabilities of
neurosurgeons during virtual surgery, demonstrating that virtual
reality simulators using artificial intelligence can be powerful
tools in surgeon training.
This research, published in the Journal
of the American Medical Association on Aug. 2, 2019, shows that the
fusion of AI and VR neurosurgical simulators can accurately and
efficiently assess the performance of surgeon trainees. This means
that AI-assisted mentoring systems can be developed that focus on
improving patient safety by guiding trainees through complex surgical
procedures. These systems can determine areas that need improvement
and how the trainee can develop these important skills before
surgeons operate on real patients.
RETAIL
Walmart
& Lego offer a new Web-based AR Experience
https://www.walmart.com/lego
Jon Cheney, CEO of Seek, an eCommerce
company specializing in AR, showed off an ever-so-cool AR application
that Walmart uses in its Lego kiosks—sending clearly intrigued
eTail participants to their phones to view the demo. How it works:
Capture the QR code displayed on a Lego kit using your cellphone’s
camera, then aim your phone down at the floor to see how the
completed project will look and work. Cheney says 1,700 Walmart
stores are using the application.
In addition, you can now use this on
any smart phone directly from the website. Try it yourself at
Walmart.com/lego Click ‘See it in action’
Puma
Uses Zappar to Bring Web-Based AR Experience to Retail Store
https://mobile-ar.reality.news/news/puma-uses-zappar-bring-web-based-ar-experience-retail-store-0205016/
By scanning QR codes at two in-store
displays via the mobile web app, shoppers can interact with Puma’s
mascot and get directions to the basketball section of the store.
The experience continues with the shoes
themselves. The seven styles in the basketball line-up each include
hangtags that trigger a unique AR content. This amazing experience
was created by AR company, Zappar.
DESIGN
Bugatti
Design Director Achim Anscheidt Discusses Virtual Reality Design of
$9 Million Centodieci
https://www.forbes.com/sites/markewing/2019/08/31/bugatti-design-director-achim-anscheidt-discusses-virtual-reality-design-of-9-million-centodieci/#3cd81b67bffa
Achim Anscheidt has served as Bugatti’s
Design Director since the early stages of the VW era, guiding
development of every Bugatti from the original Veyron and its many
iterations, to Chiron, Divo, and the one-of-one La Voiture Noire.
“It doesn’t take me one and a half
years anymore. It takes half a year with Virtual Reality, VR goggles.
This was the only way to answer the needs of our CEO Stephan
Winklemann, to turn the brand where he wanted to go. Same with
Centodieci, with Divo. Same with La Voiture Noire,” says Achim.
CAT®
SAFETY VR: CATERPILLAR’S NEW VIRTUAL REALITY TRAINING SOLUTION FOR
CONTRACTORS
https://www.aem.org/news/ep-21-cat-safety-vr-caterpillars-new-virtual-reality-training-solution-for-contractors-with-justi/
Statistics show that up to 90 percent
of construction job site accidents are caused by unsafe behavior, not
conditions. However, according to Justin Ganschow of Caterpillar
Safety Services, worksite rookies and veterans alike can benefit from
learning critical safety lessons in a controlled, virtual reality
(VR) environment.
Justin
Ganschow: Virtual reality is really just a better way to
train. It does elicit not only an emotional but a biological
response. And those are the things that form memories.
9 min
September 14, 2019 The Bugatti Built in VR - XR For Business Podcast News, September 13, 2019
LEARNING
1. Developing Communication Skills And
Improvement of Literacy
2. Enlarging Real-Life Experience
3. Supporting Specific Needs of the
Disabled Students
4. Developing and Improving Empathy
5. Receiving Workplace Experience
Read
more:
https://programminginsider.com/5-real-uses-of-virtual-reality-in-education/
Mixed
Reality Classroom
https://techtrends.tech/tech-trends/the-mixed-reality-classroom/
Microsoft has created an ‘Immersive
Classroom’ that teaches students about all sorts of emerging and
exponential technologies such as VR/AR, AI and 3D Printing. it
incorporates Paint 3D, HoloLens and other Mixed Reality headsets, and
3D printing. Other activities also explore coding and robotics using
tools such as Minecraft and Artificial intelligence, where students
get to build their own chatbot.
The immersive classroom is open for
schools to experiment with incorporating digital tools in learning,
and has facilitated the delivery of more than 60 free workshops for
over 1000 students since opening in 2018
A.I.
and virtual reality can determine neurosurgeon expertise with 90 per
cent accuracy
http://www.thesuburban.com/life/health/a-i-and-virtual-reality-can-determine-neurosurgeon-expertise-with/article_08c12c76-d0b9-11e9-a6c2-6f29fcf49393.html
Machine learning-guided virtual reality
simulators can help neurosurgeons develop the skills they need before
they step in the operating room, according to a recent study.
Research from the Neurosurgical Simulation and Artificial
Intelligence Learning Centre at the Montreal Neurological Institute
and Hospital (The Neuro) and McGill University shows that machine
learning algorithms can accurately assess the capabilities of
neurosurgeons during virtual surgery, demonstrating that virtual
reality simulators using artificial intelligence can be powerful
tools in surgeon training.
This research, published in the Journal
of the American Medical Association on Aug. 2, 2019, shows that the
fusion of AI and VR neurosurgical simulators can accurately and
efficiently assess the performance of surgeon trainees. This means
that AI-assisted mentoring systems can be developed that focus on
improving patient safety by guiding trainees through complex surgical
procedures. These systems can determine areas that need improvement
and how the trainee can develop these important skills before
surgeons operate on real patients.
RETAIL
Walmart
& Lego offer a new Web-based AR Experience
https://www.walmart.com/lego
Jon Cheney, CEO of Seek, an eCommerce
company specializing in AR, showed off an ever-so-cool AR application
that Walmart uses in its Lego kiosks—sending clearly intrigued
eTail participants to their phones to view the demo. How it works:
Capture the QR code displayed on a Lego kit using your cellphone’s
camera, then aim your phone down at the floor to see how the
completed project will look and work. Cheney says 1,700 Walmart
stores are using the application.
In addition, you can now use this on
any smart phone directly from the website. Try it yourself at
Walmart.com/lego Click ‘See it in action’
Puma
Uses Zappar to Bring Web-Based AR Experience to Retail Store
0 min
September 13, 2019 Bringing Lego Fish and Global AR Gnomes to Life, with Trigger Global’s Jason Yim
Step 1: Plant augmented reality
gnomes across the world. Step 2: …? Step 3: PROFIT!
Just kidding — Trigger Global’s
army of AR gnomes has a more solid business plan than the Underpants
Gnomes, as well as many other ventures across popular brands,
utilizing mixed reality technologies to bring them to life. CEO Jason
Yim emerges from his hidden meadow to talk about a few of them.
Alan: Hi, my name is Alan
Smithson, the host of the XR for Business Podcast, and today’s guest
is Jason Yim. He is the CEO and executive creative director of
Trigger Global, the mixed reality agency. He has creatively led over
150,000 hours of development in mixed reality, including as a Snap
Lens Studio partner, preferred developer for Facebook, and showcase
developer for Euphoria and Google, as well as an early adopter and
early developer for Magic Leap. Yim’s recent high-profile work
incorporates mixed reality in marketing for Star Wars: The Last Jedi,
product development for Hot Wheels, and location-based experiences
such as the fish designer for Lego House, and of course, enterprise
tools for AR evaluation tool for Honda. Yim is a recognized speaker
around the world and he has held the stage at major technology and
industry conferences in Singapore, Shanghai, Berlin, Tokyo,
Copenhagen, London, New York, Los Angeles, San Francisco, and all
over the place. Jason’s returned to his childhood home to speak at
TEDx Hong Kong on computer vision bringing toys to life. Yim has also
been featured in Apple’s first TV show, “Planet of the Apps,”
and won two LA Auto Show award design challenges back to back, with
his partners at Honda Advanced Design. Additionally, Jason has been
assigned four patents in augmented and mixed reality, with several
more pending. To learn more about Trigger Global, you can visit
triggerglobal.com.
Jason, welcome to the show.
Jason: Alan, thanks for that
kind introduction.
Alan: It’s amazing, just that
introduction; you think “Holy crap, you’ve done work with Honda.
You’ve done work with Lego. You’ve done work with Snapchat, and
Facebook, and Google.” It’s crazy, the things that you’ve done.
And you joined us on stage at AWE this year, to talk about
supercharging your marketing. Tell me about some of the things you
guys are working at right now.
Jason: Yeah, I think for us on
the marketing side it’s actually quite an interesting time. We’re
seeing basically the market maturing a little bit and then kind of
dividing into two big chunks of work. On the introductory to AR side
of things, we have the social lenses. So that’s the
Snap/Facebook/Instagram approach, where it’s a small experience for a
smaller budget and it’s going through someone else’s app, but it’s a
much larger user base, which is a good way to start it off. And then
the other group of projects that we work on are kind of larger
development, where the brand can own their own app or they have an
existing app and we’re pushing an AR module into that existing app.
Alan: Let’s break those into
pieces, here. The first one you mentioned is smaller ones with social
lenses. Can you maybe talk about some of the work you’ve done in
that?
Jason: Sure. We were one of
Snap’s first agencies — the Lens Studio partners. We actually were
kind of a guinea pig as they were developing the Lens Studio itself.
I believe we’re probably one of the first people outside of Snap to
actually use the tool. On the client side, we’ve worked with
everywhere from Adidas, Pepsi, the NFL, all from sports and brands on
the lens side. On the Snap side, typically they are coming to us. We
either bring opportunities to Snap where we have clients coming in,
and that they’re interested in doing a lens, and then we will connect
with a Snap team person as well. Or sometimes Snap brings the
opportunities to us, where they may have something creatively or
technically a little bit unique, and then they’ll bring us in to
collaborate.
Alan: So you guys are literally
the guinea pigs here. You’re the ones who like, “Hey, that’s a
great idea. How do we do that? We have no idea; let’s call them.”
Jason: Yeah, it’s actually a
great time, because you get to innovate every single day. Sometimes
they come in with very baked ideas and we just have to figure out how
to execute. And then sometimes it’s a little bit more open-ended and
we get to concept from scratch.
Alan: So how are these brands —
especially on the lens side — how are they measuring success? I
can’t imagine it’s that cheap. You said it’s on the lower end scale,
but what it would be a minimum engagement? $50,000? Or $20,000?
Jason: I would say lenses are,
at the very minimum, it’s probably in the $20k-plus range. We tend
not to do very many of those, but we know other people are doing
those. And then it will range up from there. Our sweet spot’s
probably — for lenses — $40 to $80k, or something like that. More
than that, people are probably pushing it into the app space a little
bit more.
Alan: Ok, so how are they
measuring success with these? Because typical marketers — and for
the people listening, mixed reality and augmented reality, they’re
really pushing the envelope of the technology, but at the same time,
you still need to be able to justify this kind of spend — so how are
they measuring that?
Jason: Yeah, I think part of the
challenge is sometimes the money is coming from the same budget that
might be coming out of a digital media budget, in which case the Snap
lens or the Facebook effect is basically being compared against more
traditional digital media, like some video buy or a social ad of some
kind. I would say right off the bat that it is very difficult to beat
video placement or something in terms of just impressions. But where
I do think AR wins out, is definitely in engagement, either in
session time, the amount of average time being spent with the
content, or the amount of shares being done with that content. Also
interactions. What are they doing in the Lens itself? Are they
clicking through? Are they doing other things? Are they going through
the transaction and the purchase? Like I think you can pull deeper
interaction through the lens. At this early time, I think that
they’re still earned media that you can get, instead of your Snap
lens may get some press, and then that press then gets its own number
of impressions. Or you create a lens, and you create a video out of
that, and then you share that, and then the video of that.
Alan: I’ve been seeing a lot of
that.
Jason: Yeah.
Alan: It’s like the LeBron James
thing, where LeBron comes out of the poster. I think probably only 10
people in the world ever actually did it, but it got a hundred
million impressions from that video. Crazy, the things that are
possible.
Jason: We’ve done AR things for
some of the big brands that have like seven billion media impressions
worldwide. So the numbers can get quite high that way.
Alan: Wow. What one did you do
that had that many impressions?
Jason: It was a giant movie IP,
which is what we can share, but yeah.
Alan: Oh, okay. Awesome. It’s
incredible. So, you talked about the smaller social lenses and that
stuff, using Snapchat, Facebook, Instagram. What about the larger
builds where you’re building it for a company under their own app,
that type of thing? What does that typically look like?
Jason: A lot of it comes from
innovation groups, so it’s a different budget. On the smaller lens
and effects social AR side, we get a lot of requests via the ad
agency, spending some of the media budget. On the app side, it’s
typically more direct to brand for us. The goals are usually a little
bit deeper, so we would recommend an app or module in an existing
app. If you’re trying to go for more of a repeat experience or some
sort of deeper technology in the AR experience, like are we tying to
real time stats? Are we tying to some big database on the back end or
besides just the two minutes of fun sort of experience?
Alan: Interesting. There’s some
definite ways that this technology can be used for utilitarian
purposes, rather than just entertainment. Is that what you’re talking
about there?
Jason: Yes, I think for everyone
in the industry, it has to move in that direction. The marketing
entertainment side of things will always be there, which is great.
But the more basic side of that stuff, I feel, will quickly be a
little bit of a race to the bottom. The higher level type of
entertainment, like let’s say you have you’re starting to build an
AI, you’re starting to push volumetric content, real game engines,
things like that, that there’ll still be some premium development
there. But I think that lower level simple lens stuff will very
quickly become a race to the bottom. For AR to be more widely
adopted, we’ll be — again, like you said — like the usability
stuff. And it doesn’t have to be the main feature in an app, just
like GPS and mapping isn’t necessarily the main feature in a lot of
apps that we use, but it is a very useful component. Like if I turn
on the Starbucks app and I can find a near Starbucks, I’m not
spending my whole time in the map, but it’s a critical piece. AR will
soon be that piece as well.
Alan: I agree, I find it’s
really interesting that you mention that, because this came up — I
think you brought it up, but also some of the other people at the
panel in AWE brought it up — basically, building AR for the sake of
AR is not what we’re going after, building AR is part of something
else. It’s just another tool in the toolbox.
Jason: Yeah, and I think we were
getting into some projects now that it’s quite interesting what else
we have to tie into, what other backend systems or other technologies
that are involved. Even at AWE, when you had the gentleman from
Macy’s up, there’s a lot of stats out there right now that are
talking about mixed reality and the effectiveness of it. So I do feel
that it’s a great place to be, and I think the future is bright, for
sure.
Alan: You have on your website
here, one of the things I think is just awesome: Travelocity’s
roaming gnome. While not utilitarian, probably doesn’t move the
needle in terms of inspiring me to travel, but just fun. And it
really drives the brand message home. Who came up with that idea?
Jason: They actually came up to
me at another conference, it was at a Google conference I was
presenting, and they came up to talk about what we could do in AR in
their app. I think the interesting challenge for Travelocity was —
and here’s another use of AR — with Travelocity, you use the app for
booking, of course. Right? But they wanted more engagement after the
booking had been done. Like, how do you keep the Travelocity app
relevant through your entire trip, until you have to book for your
next trip? So that was what AR we were hoping to provide there, was
have something fun to do while you’re on the trip, and then also
something sharable to help you remember that trip. And then later on
you could pick up the Travelocity app again when you want to book
your next one.
Alan: So how many gnomes
traveled?
Jason: I actually can’t share
that information, but yeah…
Alan: Ah-ha. OK, so there’s
gnomes everywhere in the world. Augmented gnomes.
Jason: Yes. Everywhere.
Alan: I think step 2 for that is
to leave them in physical space. So you leave your gnome, and your
gnome is tagged with a message.
Jason: That would be great. Like
a big map for everything.
Alan: Find The Gnome. Yeah. Why
not? You get a lot in hot spots. Canadians all go to either Cuba or
Mexico for the winter. So you’d have these hotspots with all these
gnomes everywhere. And they should be able to interact with each
other. You find somebody else’s gnome and your gnome starts–
Jason: If you leave them too
long, they starve.
Alan: They turn to ceramic.
Jason: Yeah. The interesting
part about that, though, was that we were embedding this AR module
into the existing Travelocity app. And that’s something that we
recommend to a lot of clients that have an existing app with a user
base, is to instead of building a separate AR app, which is a lot of
lift and you’ll spend a lot of money and a lot of energy trying to
drive traffic to a brand new app, just place it within an already
successful app. But that, of course, brings its own giant series of
complications. We were really excited that Travelocity allowed us to
do that because sometimes a lot of clients, if the app that they have
is a primary revenue generator, they are super protective of that.
Alan: Oh yeah, they don’t want
you to touch that thing at all.
Jason: Yeah.
Alan: We’ve run into that as
well. It’s one of those things that you’re like, “Yeah, if you
break their app, you’re screwed.”
Jason: Right.
Alan: How do you deal with that?
Do you develop it as a separate thing and then give them a plug-in
for it, or…?
Jason: We want to start
discussing integration at the very beginning of the project. So our
tech teams are normally aligned very early, and the other developer
that actually owns the main app, they probably have a launch cycle
that they are on. So we typically have to dovetail our launch with
whatever cadence that they’re in and work backwards from there.
Alan: Interesting. I want to
switch from LA — because you mostly work in LA — but you also have
an office in… is it Billund or Aarhus?
Jason: Aarhus, in Denmark.
Alan: So you have an office in
Denmark, in the middle of Denmark, not in Copenhagen, but in the
middle of the furthest island from Copenhagen. So what’s that all
about?
Jason: We’ve been working with
Lego for six and a half years now, I believe. Lego headquarters is in
Billund, which is even more remote than Aarhus. Aarhus is actually
the second largest city in Denmark. Billund is about an hour away,
Aarhus is like the closest major town. Our lead there was ex-Lego, he
had left Lego. He actually worked with us on quite a few projects. He
left Lego after 10 years, then came and joined Trigger. So for us, it
was part of an effort to grow our Lego business out there. I’ve been
a Lego fan since I was a little kid, like most kids are. So that was
kind of like a dream come true, to be working with them.
Alan: No kidding.
Jason: We went all in.
Alan: [laughs] Of course. You
guys made a fish visualizer, right?
Jason: It’s called the fish
designer. It’s actually at the Lego Museum. So, super cool. I believe
it’s the best performing digital experience there. Basically, what
happens is, as a kid you come into the space and there’s six giant
digital tanks. So imagine like a big video screen walls that
represent massive fish tanks, and all the fish inside and all the
creatures and rocks and everything are built out of Lego, digital
Lego. And then as a kid, there are these big stations where you can
take physical bricks, and build your own fish. You build physically
and then you walk up to these scanners on the edge of giant tanks and
you scan in your fish. You add eyes and a mouth. There’s like a
little magic moment. And then they come to life in the tank. They get
sucked in through the pipe from your scanner and then they get kicked
back out into the tank as this live fish with their own little bit of
AI. Each tank can hold 300 user created fish. And every minute or so,
there’s big animations that happen inside the whole tank, and all the
fish interact together and stuff. And the kids basically get into
this loop where they go back and just build more fish to stick back
into the scanner to bring to life.
Alan: Oh, cool.
Jason: We’re super proud of it,
because with Lego we often work from concept through to prototyping,
and kid testing, and then final product release. And in this case, we
did a lot of earlier prototypes that had a lot more digital
interaction for the kid, like after you’ve created the fish you could
play a game, you could control it, something like that. It was really
nice that the testing actually took us back to keeping it much
simpler. So it’s a much more elegant solution that the kids just–
they build it physically, which is what they want to do. And then the
digital magic conversion moment is very, very short. It’s only like
30 seconds and then they can sit back and enjoy their creation.
Alan: That’s so cool. Yeah, I
think we as an industry, we tend to overcomplicate things.
Jason: Yeah, for sure. Sometimes
we are over our skis a little bit, right? Like we’re trying to do a
lot more than the consumer wants or needs.
Alan: It’s so true. We had a
meeting about this a couple of weeks ago. We’re talking about how we
did a 360 video three years ago and kind of moved away from it,
because it just– it wasn’t hard anymore. We were always looking for
the challenge and we realized that the majority of people still
haven’t even seen that. Like, “Oh, man. We need to go back to
basics.”
Jason: Yeah. I mean, we hear a
lot of stats of when museums are doing their first VR exhibits.
People come in and for 90 percent of the audience, it’s the first
time they put on a headset.
Alan: Crazy, right? It’s so
second nature for us that we take it for granted.
Jason: Yeah.
Alan: If there’s one takeaway
from this entire podcast, it’s “keep it simple”. You can
stretch and push the boundaries of this technology, but keep it
simple.
Jason: That’s actually sometimes
the hardest piece. How do you reduce the friction in these things?
Because consumers are used to just opening up the app and seeing the
content on a screen. How do you make spatial 3D content just as
simple as that?
Alan: That’s a good question.
Let me ask you. You did a TED
talk on computer vision, bringing toys to life. Is that talking
about the Lego one?
Jason: No, that was more a
generality. It was similar. We’ve done 30+ digital/physical play
prototypes with Lego and other companies, for kid testing and stuff
like that. So we understand that there’s a lot of effort in trying to
find the fun and find that elegant ease-of-use balance. That TED talk
was about those kind of learnings, but in the toy industry.
Alan: Can you send me a link and
I’ll put it in the show notes?
Jason: Sure. Yeah, no problem.
Alan: Amazing. Let’s shift gears
away from toys and social lenses. Let’s talk about the things that
you did for enterprise tools. You designed an AR design evaluation
tool for Honda. Can we talk about that?
Jason: Yeah, so we co-created
with Honda. We own the tech IP for the tool. We work with one of
their design teams. And what we’re trying to solve is– basically, in
the design process for cars right now, you start off on paper, they
do 100 drawings or whatever. Then they start working on 3D versions
of that. Let’s say you get to like 40 designs. But a critical step in
car design is, of course, a volumetric review, like actually seeing
the car in 3D space and getting a sense of its presence in a way. And
traditionally, that’s done in clay. So the problem with clay is that
a life sized clay model takes about eight weeks, and these car
companies are spending upwards of $50,000 a month just buying clay.
Alan: Wow.
Jason: Yeah. So, we were tasked
to try to come up with a system in AR, to not necessary replace the
clay model, but to do an interim step before the clay model. Let’s
say we can get the cost down and things like that, so of course, we
did. Our version, it can be in a single day instead of eight weeks.
The costs are much lower. They can then evaluate 10 cars in AR
volumetrically first, before they can then commit to one clay model
car. Not only are we saving on costs in time, but I feel like you can
end up with a better product because a lot of the design ideas that
would have been cut early, because of costs gets to live through
another milestone, and continue on to the design process. The big
challenges, though, were we started with Hololens, and then it was
ported to ARKit, it was ported to Windows Mixed Reality, full
backpack solution. And then the later version is Magic Leap, but all
of those see-through — besides the ARKit, of course — but
see-through headsets on AR, as soon as you take it outside, it’s, you
know–
Alan: Useless.
Jason: It’s not the best
experience.
Alan: Yeah. It’s funny because
I’ve seen some people put shields in front of the Hololenses and
stuff, light shields to dim it. It’s not the greatest experience when
you can’t really see the holograms. That’s going to be a really hard
problem to solve for consumer augmented reality.
Jason: Yeah. And I think this
for the car design world, it’s actually specifically a very hard
problem to solve, because their legacy approval process happens
outdoors that way. So we had to put digital cars next to physical
cars, which is how they normally evaluate their designs. That’s why a
lot of design studios are actually in California because of–
Alan: Nice weather, all the
time.
Jason: Yeah, nice weather all
the time. So that’s the interesting thing. Like, maybe the technology
drives you to an indoor experience. But like–
Alan: I just tried the Varro —
the Varjo or Varro? — headset, the XR one, which is using front
facing cameras to capture the outside world. So with that, it’s
actually blocking out the real world completely, using cameras to
recreate it, and then creating digital content on top.
Jason: That’s cool.
Alan: Yeah. It works really
well. I think it would actually be a perfect solution for what you’re
doing. And I know they’re working with Volvo.
Jason: That’s cool. What is the
resolution of that?
Alan: It’s actually a foveated
headset. So what they did was, they did a fixed foveation. In the
center, it’s human eye resolution. And then it’s about– it’s a
little square, almost an inch of your vision. And then as it goes out
to the edges, it blends into a more traditional headset. So it’s
very, very clear.
Jason: Wow, amazing. We will
definitely figure–
Alan: The headset’s big and
bulky, but it doesn’t feel like it when it’s on your head, because
they’ve weighted it. They actually put weights in the back of it, to
offset the feeling of how heavy it is. They made it heavier to make
it feel lighter.
Jason: But I feel like it’s
being used in the design process now, like everyone who sees it
understands the benefit of it, and everyone understands that the
hardware and technology is always going to be improving. So we’re on
the right track and we just– with every hardware update — we’ll
check out this Varjo for sure — the whole experience will improve
and hopefully the results will improve as well.
Alan: I think it’s getting there
every step. And then I really love the fact that you guys are right
on the edge of consumer applications that are fun and exciting, kids’
applications. But then also you’re building these real world
enterprise tools, that companies are using to evaluate and design
future automobiles.
Jason: Thanks. I think for
brands and developers getting in, I think what we’ve learned through
this stuff is you just have to be early, try to be first, try to get
as much experience across many industries as possible, because
everything cross-pollinates everything else. The stuff our team knows
from all this trial and error, I think it shows. But also there’s no
other way to learn that experience, besides trying it out.
Alan: Yeah, well, it’s not like
you can look it up and say, “Hey, how do I do this?”
Because what you’re doing is, for the most part, never been done. I
mean, we’ve done, I think, three or four world firsts and– well,
four. There was no manual. There was no “Hey, let’s look it up
on YouTube and how to do that.” It didn’t exist. So I commend
you guys, with 150,000+ hours of development.
Jason: Thank you. That’s no
creative hours, too. That’s just 3D in-dev.
Alan: Straight-up dev. Holy
moly. Well, they say mastery’s at 10,000 hours. So you guys are 15
times that.
Jason: I think the need to
experiment and the need to innovate is really great for the team.
Every day you’re doing something different and there’s always
something proud to write home about. The problem as a business is if
you’re always innovating, how do you–
Alan: How do you make any money?
[laughs]
Jason: You’re burning so much
[money]
.
Alan: People don’t understand.
Innovation is expensive. And even though companies are paying you to
do it, it’s like you build it once, and then it never gets used
again. It’s not like building a product where you build it, and then
keep iterating and making it better and better and better. Projects
are a different animal, and it’s definitely something that you guys
have mastered. So, congratulations.
Jason: Thank you. We are owning
our own IP now, so our tech IP. So there are some platforms that we
can get to build on top of and resell to different clients and
improve over time. So that’s helping us a little bit.
Alan: Love it. Well, let me
know. I’m happy to consider some of those products and platforms for
the XR Ignite program. So with that, is there anything else you want
to leave listeners with? There’s so many different things here to
unpack. But is there anything else you wanna leave people with?
Jason: Yeah, I think the sports
stuff that we’re doing is really interesting right now. We’re doing
kind of AR portals into live games for the NBA, and actually for the
PGA Tour as well. So you’re at home watching TV. You can plant a door
in AR and then step through it and then you’re courtside at the
finals. And then the next level up from that is what we’re doing with
the NHL, and soon with another sports league as well, where we’re
bringing live telemetry from the game, live stats.
Alan: That’s so badass.
Jason: Yeah. When you’re talk
about utility, it’s not a wow factor thing. It’s a– with the NHL,
you’re watching the game. And then, let’s say you see a great play.
You can actually drop the rink onto your coffee table and see that
play, recreate it in AR with all the live data from that play, like
how fast the puck was shot and how fast the players were going. All
that data represented in AR. So I think that’s super interesting to
us.
Alan: That’s badass. Come on,
let’s be honest.
Jason: It’s cool to be first.
So, that was fun.
Alan: It is always cool to be
first. Well, thank you so much, Jason. It’s been an incredible
interview. And I’m sure we’re gonna have to do this again, because in
six months time, you’ll have 100 more things to talk about.
28 min
September 13, 2019 Bringing Lego Fish and Global AR Gnomes to Life, with Trigger Global's Jason Yim
Step 1: Plant augmented reality
gnomes across the world. Step 2: …? Step 3: PROFIT!
Just kidding -- Trigger Global's
army of AR gnomes has a more solid business plan than the Underpants
Gnomes, as well as many other ventures across popular brands,
utilizing mixed reality technologies to bring them to life. CEO Jason
Yim emerges from his hidden meadow to talk about a few of them.
Alan: Hi, my name is Alan
Smithson, the host of the XR for Business Podcast, and today's guest
is Jason Yim. He is the CEO and executive creative director of
Trigger Global, the mixed reality agency. He has creatively led over
150,000 hours of development in mixed reality, including as a Snap
Lens Studio partner, preferred developer for Facebook, and showcase
developer for Euphoria and Google, as well as an early adopter and
early developer for Magic Leap. Yim's recent high-profile work
incorporates mixed reality in marketing for Star Wars: The Last Jedi,
product development for Hot Wheels, and location-based experiences
such as the fish designer for Lego House, and of course, enterprise
tools for AR evaluation tool for Honda. Yim is a recognized speaker
around the world and he has held the stage at major technology and
industry conferences in Singapore, Shanghai, Berlin, Tokyo,
Copenhagen, London, New York, Los Angeles, San Francisco, and all
over the place. Jason's returned to his childhood home to speak at
TEDx Hong Kong on computer vision bringing toys to life. Yim has also
been featured in Apple's first TV show, "Planet of the Apps,"
and won two LA Auto Show award design challenges back to back, with
his partners at Honda Advanced Design. Additionally, Jason has been
assigned four patents in augmented and mixed reality, with several
more pending. To learn more about Trigger Global, you can visit
triggerglobal.com.
Jason, welcome to the show.
Jason: Alan, thanks for that
kind introduction.
Alan: It's amazing, just that
introduction; you think "Holy crap, you've done work with Honda.
You've done work with Lego. You've done work with Snapchat, and
Facebook, and Google." It's crazy, the things that you've done.
And you joined us on stage at AWE this year, to talk about
supercharging your marketing. Tell me about some of the things you
guys are working at right now.
Jason: Yeah, I think for us on
the marketing side it's actually quite an interesting time. We're
seeing basically the market maturing a little bit and then kind of
dividing into two big chunks of work. On the introductory to AR side
of things, we have the social lenses. So that's the
Snap/Facebook/Instagram approach, where it's a small experience for a
smaller budget and it's going through someone else's app, but it's a
much larger user base, which is a good way to start it off. And then
the other group of projects that we work on are kind of larger
development, where the brand can own their own app or they have an
existing app and we're pushing an AR module into that existing app.
Alan: Let's break those into
pieces, here. The first one you mentioned is smaller ones with social
lenses. Can you maybe talk about some of the work you've done in
that?
Jason: Sure. We were one of
Snap's first agencies -- the Lens Studio partners. We actually were
kind of a guinea pig as they were developing the Lens Studio itself.
I believe we're probably one of the first people outside of Snap to
actually use the tool. On the client side, we've worked with
everywhere from Adidas, Pepsi, the NFL, all from sports and brands on
the lens side. On the Snap side, typically they are coming to us. We
either bring opportunities to Snap where we have clients coming in,
and that they're interested in doing a lens, and then we will connect
with a Snap team person as well. Or sometimes Snap brings the
opportunities
0 min

About XR for Business

From the publisher's feed

Meet the leaders who are changing the face of virtual and augmented reality

Share XR for Business

Sign up to save your podcasts

XR for Business

XR for Business episodes

About XR for Business