XR for Business

By Alan Smithson from MetaVRseBusiness Technology Arts

XR for Business episodes

July 22, 2019 Dress for Success: Talking Headsets and Haptic Suits with Skarred Ghost Antony Vitillo
From headsets to haptic suits, there is going to be a lot of accessories and apparel in XR to chose from, including some that expand senses you didn’t even know were XR-compatible. Antony Vitillo – AKA Skarred Ghost – drops in to discuss different devices with Alan, their use cases, and what companies should consider when they go shopping for some.
Alan: Today’s guest is Antony
Vitillo, better known as Skarred Ghost. Antony is an XR consultant
and author of an amazing blog called The Ghost Howls. He also runs a
consulting company called New Technology Walkers, where they develop
VR solutions and advise companies about how best to use VR and AR.
Antony recently traveled to the fourth annual VIVE Ecosystem
Conference — VEC — in Shenzhen, China. If you’re not already
following Tony, you can learn a lot by connecting with him on
LinkedIn and subscribing to his newsletter at skarredghost.com. Tony
welcome to the show.
Antony: Hello, Alan! Thanks for
this opportunity.
Alan: It’s so great to have you
on the show. I had a wonderful opportunity to speak with you many
times, and we are both very, very passionate about virtual and
augmented reality. I want to just thank you for taking the time to be
on the show.
Antony: I’m very happy to be
with you. I’m happy to speak to you live, after so many messages
we’ve done on LinkedIn. Super happy to be here.
Alan: Let’s just dive right in,
and we’re going to try to bring as much value as we can to the
listeners today. We have a lot to go through; we’re going to go
through all of the different hardware aspects involved in
Virtual/Mixed/Augmented Reality — XR — and it’s not just the
headsets or headphones. You’ve got things like haptic suits, haptic
gloves. You’ve got touch-sensitive stimulators. You’ve got VR
headsets, AR headsets. You’ve got mobile phone-based AR, eye tracking
set devices, taste experiments, hot and cold devices, thermal
devices, and then tracking systems for motion capture, and of course,
treadmills for omni-directional walking. So there’s a lot to unpack
here. Let’s start at something crazy; haptic suits. Let’s talk about
haptic suits, and why and where these would be used in any
industries.
Antony: I’m very interested to
have these suits, because they offer the promise of letting you use
your full body in VR. So, finally, you can be there with all your
body. You know, my first startup was about full-body in VR, but using
Kinect. So, a different approach, but I’m a big fan of having the
possibility to kick objects, to move your body in every possible way,
and see your full self replicated in VR.
The advantage of using the haptic suits
over other approaches, like the one that they used with Kinect, is
that you don’t only have your full body — your full movement — in
VR, but you can also feel sensations. You can have haptic feedback.
So you can [feel] hot, cold. You can feel pain and whatever. It’s
really full immersion; a bit like we have seen in the Ready Player
One movie. Wade Watts wore that expensive suit, to fully be inside
the Oasis. This is why I think they’re very interesting, because they
can really enhance your visual experience; your sense of presence,
like your ancestors like to say.
Alan: So, let’s unpack this for
a second, Tony. What would some of the practical use cases of this…
I can see one in military training, where you’re in a virtual world,
you’re in a hostile environment, and maybe something explodes behind
you — a piece of shrapnel hits you — and maybe it vibrates. Maybe
explain some other instances, where this could be used in enterprise.
Antony: I think about different
possibilities. Like, for instance, I was talking some time ago with
some psychologists, and this can be interesting for rehabilitation.
How do you perceive your body? You can see yourself as an avatar that
is a bit too fat, too skinny, that lacks some parts of the body or
such. So, rehabilitate yourself psychologically. But it can also be
used for rehabilitation of your body. So if I can track all the
movements, and then medics have to check my patients that have
problems with the back, with their legs, whatever; I can really
observe them when they’re moving. For instance, you talk about the
military, but I think lots of industries and may have interests in
evaluating all of the body forces during training. So, if they’re
training for particular movements — and all the body — I think the
haptic suits are the only real possibility.
Alan: I was actually reading an
article the other day — it was was more of a scientific paper —
talking about privacy, and the fact that, with haptic suits and these
headsets, we’re actually able to collect insane amounts of data. We
can not only collect data around your height — because we know how
high you are from the floor — but your gait, how you walk, your
movements, what you’re looking at, what you’re experiencing, your
heart rates. There’s so many physiological aspects that we’re able to
collect incredible amounts of data, and so, by collecting this data,
all sorts of trainers will have unprecedented levels of data around
the person they’re studying. It’s funny, because for years and years,
we’ve studied people’s movements, but we’ve never had anything this
accurate. It’s really exciting.
Antony: Yeah. It’s all exciting,
for that high input. But what I also would like are certain kinds of
suits — for instance, we can name the Teslasuit. That is a very
complete device that should be available, maybe next year, and it can
also provide feedback to the user. So for instance, I sent this video
out; people wearing the Teslasuit were able to feel hot, to feel
cold, and also to feel pain. You mentioned, for instance, the
training of the military; you can really feel the pain of having been
shot. If you are a firefighter; maybe if you don’t extinguish the
fire fast enough, you can really feel the hot or your body. I think
that can also be managed to–
Alan: It’d be amazing for
firefighters or paramedics or people who are in emergency situations,
where training for these things is almost impossible. You can’t
really train for every scenario in the real world; in virtual reality
— with these suits — you can now train in a really realistic way,
for things that are very rare. I love that.
Antony: Yeah, because the great
thing about VR is that you can perform this training, realistically
simulating the situation the person will be in. The more the
simulation is realistic, the more this person who has to solve the
problem will we be prepared to solve it. For instance, if you really
have to extinguish a fire, and in the individual simulation you’ve
already felt the cold — maybe also the humidity, whatever — of that
moment; you’ll be in front of the fire, and you’ll really feel the
hotness. You will be already prepared. This is what I think will also
be very important of these suits. Of course there will be products
tailored to enterprises; there are no products ready for the
consumers. But since we’re talking about using this stuff for
industries, for enterprises, I think that it will be a very important
application.
Alan: I agree. And just to talk
some brands that are out there now, you mentioned Teslasuit. Are
there any other ones that you know of that are out there right now?
Antony: Well, I know a few
names. Another one that I want to mention, that I tried two times —
one also at the VEC — is bHaptics. That is a Korean company, and the
thing that is interesting is that they are making a modular suit. So,
you can buy the model for the face, for the chest, for the arms, and
for the legs. So, you can also only buy the pieces that you need, and
it can provide mostly depression feedback. But it’s good, because it
can be localized. I tried a paintball game, and I really could feel
the vibration in the exact point I was shot — in the chest, or also
in the face; it was very strange when I got the vibration in my face,
because I’ve been headshot-ed by my opponent — and it also works
with various devices, and now it works also with the VIVE Focus, so
you don’t need a PC, it doesn’t have cables; you just have a headset
on your head, and the suit on your body. This can be very, very–
Alan: It’s interesting that you
mentioned the VIVE Focus, because you were just in China at the VIVE
Ecosystem Conference, and one of the things — I actually interviewed
the president of HTC VIVE, Alvin Wang Graylin, and he’s actually
gonna be on a different episode of the podcast (so if you’re
listening, you can look up that podcast as well) — one of the things
they mentioned is the launch of the VIVE Focus Plus, which I believe
you had a chance to take a look at. And one of the things they
mentioned was this ability to take up to 40 headsets at once,
synchronize them in an up-to-900,000-square-foot space, so that you
could do very large-scale trainings. And I think, now that you
mentioned that bHaptic suit combining with that, this is going to be
a very, very powerful tool for enterprises.
Antony: Yeah, of course, I was
there; I listened to all the presentations by HTC. I saw other
companies working with them — like, for instance, Modal VR. Maybe
let’s talk later about that here on this podcast. I think that the
strength of HTC now is the services that they’re offering for
enterprises. This ability to configure more headsets at once, and
also to create such kinds of situations so you can have multiple
players in those spaces. And yeah, both for entertainment — you can
play large spaces — but [also] for serious applications. You can
have serious games, or directly training with different teams,
attempting to train together. And everything without a cable, because
the Focus Plus is completely standalone. I think that it can be
really important. I have one here on my desk, and I think that’s an
interesting device. Yes, I think that companies may evaluate the use
of this device, especially for when there are multiple people
involved, and the company doesn’t want to buy lots of pieces have
lots of cables, stuff like–
Alan: It really democratizes it,
and makes it easy for businesses to get involved. I think, to be
honest, if you look at the roadmap of VR in general, everything kind
of launched and kicked off in 2016, to the public. But I think
businesses are really going to start embracing these standalone
headsets. While we’re talking about VR headsets, let’s unpack some of
the other ones that are becoming more prevalent in industry. We’ve
got the VIVE Focus Plus. Oculus Quest is coming out. The VIVE Cosmos,
the Pimax 8K, the Varro, the Pico… so, which one of those of you
tried, and where do you see them fitting in to different enterprises?
Antony: I think that it’s great
that this year, we are going to have lots of interest in devices.
It’s great to see the industry growing, but it’s also important for
companies to start understanding what are their needs, and so, what
are the devices that can fit them? The Oculus Quest is a great device
that is coming — very polished by Oculus, it will be quite cheap,
only $400. There is also the Oculus Go, that is already on the
market; it’s just a viewer of 360 content. One other thing that is
important is that they also understand the business license. The VIVE
Focus Plus is quite expensive — it costs $800 — but it has clear
business licensing, does business services, assistance, there is a
kiosk mode, and other things that are fundamental for a company. The
Oculus Glass — currently — when it will be launching, most probably
will be mostly a consumer device. So it will be probably a bit better
than the Focus, for what concerns the comfort and the controllers —
it is more ergonomic. But it’s not clear if it will have a business
license and business services since Day 1. There are rumors about a
business version of the Quest coming next month, but it is important
that companies understand if there is a business licensing or not,
and what it offers. This is the first thing that is important to say.
Alan: I agree with you, Tony; I
think it’s really important to unpack that just a little bit, because
Oculus and HTC both realized that the path to mass consumer adoption
of this technology is actually through enterprise applications. We
saw that very early with mobile phones, with the BlackBerry being a
very powerful business tool, and then becoming a consumer tool after
that. But really, what’s going to happen with these — and I think
Oculus, their official stance is that they want to market towards the
enterprise, but all of their advertising is towards the consumer. So
this is this kind of disconnect, and Oculus being owned by Facebook,
I really don’t know that they have the experience…
Antony: The interest.
Alan: Yeah, and it doesn’t seem
like they have that. But I mean we’ll see. With the Oculus Quest,
we’ll see what kind of services they provide. But definitely, HTC is
really far ahead with the services and, being able to do that. So I
want to just shift focuses for one second and — for those of you who
are listening who don’t really know about the VR headsets and the
difference — there’s kind of two types: there’s Three Degrees of
Freedom, and Six Degrees of Freedom. Three degrees means you can look
up, down, left, right, and you’re in the space, but you can’t move
around. Then, Six Degrees allows you to look up, down, left, right,
but also move in those [dimensions], and the newer headsets will
allow you to move. It makes a huge difference in the amount of
immersion, but also the things you can do; being able to have
controllers, or see your hands in virtual reality, and connect and
move things around. That’s Six Degrees of Freedom.
So, you have something like the Oculus
Go, which is a $200 headset, which is perfect for 360 videos; if you
want to do some basic training for people, and they can interact
using Gaze-type controls. But you can’t move around, and that’s kind
of the difference between the two. Then you have tethered and
untethered, meaning connected to a big computer or a backpack
computer, or standalone, meaning the entire computer device is on the
headset. So just for those of you who are new to this, that’s the
difference between the different headsets. Is there anything else you
want to add to that before we move on?
Antony: No, I think that was a
precise list.
Alan: There’s another headset —
there’s two out there. One’s called the Pimax, which is an 8-K
headset; it’s like wearing a giant scuba mask. It’s massive! And the
view is beautiful. I mean, it looks really gorgeous. But there’s no
way anybody in the public is going to wear this on their head,
because it is like wearing two massive cell phones strapped to your
head on a diagonal. But then, there’s another one called The Varro
out of — I believe, is it’s Sweden? or Finland?
Antony: Varro is from Finland.
Alan: Finland, sorry. And these
guys, they’re selling the most expensive VR headset on the market.
What their claim to fame is is having a really, really wide field of
view and very, very high optics and resolution. That thing is — I
think — it’s $7,000.
Antony: I remember something
around $6,000, maybe–
Alan: $6,000.
Antony: — but it was super,
super expensive.
Alan: It seems super expensive,
but let’s break it down for a second. About five years ago, companies
would spend millions and millions of dollars building a virtual
reality CAVE, so that people could start working in three dimensions.
If you fast forward now, $6,000 for a headset that allows your
companies to design in virtual reality, and have meetings in virtual
reality while designing; one headset saves one flight, and there’s
one flight, paid for the headset. So, in enterprise, this is not a
lot of money. In consumer, I can’t imagine anybody is gonna buy one
of these headsets. But, in the market they’re going after, it’s
really valuable to design companies. One of the other interviews we
did was with Elizabeth Baron, who headed up the VR division of Ford,
and every single car that they make has to be viewed in virtual
reality by all the senior executives, and they have a big meeting
where they’re all around the world meeting about the car and seeing
in different lighting conditions and all this, and what they need is
the best possible quality. I think for that reason, these high-end
headsets are going to work.
Speaking of high-end headsets, let’s
touch on some of the AR headsets — or the augmented reality, or
mixed reality — headsets. You’ve got a ton coming out, now; you’ve
got the Hololens, Hololens 2, Magic Leap, nreal, Realmax, Vuzix,
North glasses, Epson MOVERIO, Google Glass. So, let’s start at the
top; let’s talk about the Hololens. What are your thoughts on that?
Antony: My thought is that
Microsoft is doing a great job, because it has created the AR market
with the first Hololens, and now after some years, it has created the
new Hololens 2, that is a big improvement over the previous one. I
have not had the pleasure of trying it yet, but from what I have
read, the greatest improvement is that it is much smaller and
useable. If you have ever tried to use a Hololens One for a training
experience, it was really a pain, because the only interaction was
through the “air tap” gesture; it was like a click with the
index finger and the thumb.
Alan: Oh my goodness, it was
impossible. Basically, you had to put this headset on people, and
explain this weird clicking, like… “it’s like a mouse, only
you gotta stick your finger up and point it out,” and if anybody
who’s ever tried this, knows what I’m talking about. Anybody over 40
had a real hard time trying to figure this out; anybody under 20
picked it up instantly. But the ability to get people working on this
immediately was difficult. You posted something yesterday with the
Hololens 2 and the interactions, maybe talk about the new
interactions that Hololens 2 brings.
Antony: Well, the first
interaction that went along with the Hololens One was a disaster,
because there was only one interaction, and the system wasn’t able to
adapt it well — I had tremendous experience with that. You said the
Hololens 2 brings some more natural interactions with both hands, and
so basically, you don’t have to teach how things work. You just use
it like in real life. You have to scroll things? Just scroll them
with your hands, moving the hands from down to up. If you had to
click buttons, just to put a finger of yours on the button and press
it. It’s hard for me to explain that, because there’s no explanation
needed. You just do what you think is intuitive to do. The system can
detect all your hands, all your fingers, and so everything just
works. And it’s not only the hands. It’s also the eyes. There is a
demo by Microsoft, that you are reading a text. And when your eyes
are at the end of the text, the system detects that you have read
everything, and closes automatically — the text — so you can
continue reading it.
Alan: Hold on a sec. So
basically, because there’s eye tracking, and because the system knows
exactly where you’re looking at all times, it can know when you’re at
the end of a sentence, and move it up for you? Think about that for a
second; the world is going to move to spatial computing, and… let’s
just talk about the difference between VR and AR for a second,
because I think we skipped past that. Virtual reality puts on a
headset and transports you to another world. All of these headsets
are gonna start to have eye tracking and all of these things. But in
augmented reality, or mixed reality, you’re actually seeing your real
world, with data painted on top of it. The ability to look at
something and instantly have the information in context to that,
immediately in front of you — and now with eye tracking, it knows
exactly what you’re looking at and, can bring up information, and
know when you’re finished reading it and get it out of the way. So
this is really, really important, fundamental; eye tracking is going
to be an every single pair of glasses.
Antony: Yeah. Sorry if I
interrupt you, but something has come to my mind. Some companies that
have asked of us to make a system for the Hololens — so, in AR — so
that the worker — like, a maintenance worker — could do something
to repair machinery with the hands, and at the same time, see the
manual in augmented reality in front of him. The great advantage of
using eye tracking — for instance, the solution provided by
Microsoft — is that the worker can have their hands in the machine,
performing his work, and at the same time with the eyes, look at the
manual that will scroll, automatically, the instructions of how to
repair the machine. So this is something that can be very important
for maintenance, in my opinion.
Alan: I agree, and in fact,
there are some studies by Boeing — that are using this technology
immediately, now — and they’re seeing a 25 to 45 percent decrease in
the amount of time that it takes a worker to complete a task. Now,
think about that; 25 percent faster. That alone is incredible, but
the real kicker comes in the fact that they have near-zero error.
Antony: Wow!
Alan: So, by putting these
instructions up in front of them, they’re seeing near-zero errors.
And there are companies out there like Upskill — they’re going to be
on the podcast as well — there’s some other companies there, that
are really starting to take digital manuals, and put them into a
heads-up display, so you’re completely hands-free, and you have the
information, when you need it, in context to what you need
immediately. And that is a really powerful tool that… well, very
few enterprises are working on now, but I think it’s going to explode
in 2019.
Antony: In the end, it will
disrupt completely the maintenance sector, in my opinion. In maybe
five years, all the maintenance operations that we know now that
completely changed by AR and MR.
Alan: I agree. So, we’ve talked
about the VR headsets — for training, for simulations, for design.
We’ve talked about augmented or mixed reality headsets — the
Hololens, Hololens 2. Magic Leap is kind of a Hololens competitor;
they really went after the consumer market, and they’re actually
gonna be selling through AT&T stores — starting this week, I
think. And I think Microsoft really has a firm grasp on the
enterprise of this, and I think they’ve got a really good head start,
because 1. They have great relationships with all the enterprise
clients already. 2. They’re building services into their Azure cloud,
so that’s really exciting. And now, everything’s gonna just work with
your current BIM systems (if you’re in construction), or your CAD
diagrams. I think one of the things that came up at the Hololens 2
launch — that I think is going to be revolutionary — is a program
called Spatial. What were your thoughts on that?
Antony: Well, Spatial. I think
that it’s one of the best — from what I’ve been able to see —
collaboration tools. It’s not the only one, because there are other
ones for VR; there are some custom solutions. For instance, there’s
one from Nvidia in VR that is also very, very good (also very, very
expensive). It is important, because it lets the workers of a company
that are maybe in offices in different parts of the world — maybe
some are in Beijing, some are in New York, some are at home — and
they can meet in augmented reality. They can discuss ideas, they can
work on 3D models together. For instance, to refine a prototype. They
can meet as if they were in the same space, seeing them together,
interacting, talking. Imagine how this is great, because this can
save lots and lots of money for companies.
I was talking with the CEO of a Chinese
company that is working on another collaboration tool in VR, called
XCOL. She explained to me that for certain kinds of companies that
produce objects — so, not people like me that create software that
can be exchanged easily by just sharing a folder, maybe on Dropbox —
people that have to create concrete objects — real objects — maybe
they have to share prototypes made with chalk, wood, or whatever.
Creating these prototypes, sharing them by sending these packages all
over the world, then having a meeting. It’s all a waste of time and
money — big money, it’s not just a hundred dollars. So, the
possibility to meet in only one virtual space, talk together, and
modifying objects together. So instead of 3D printing the object to
walking on — I don’t know, a remote control for a new TV — we can
see the 3D model of the remote control in front of us and we can
discuss in front of it, and decide to change it together. All with
zero cost, because that 3D model is just a digital object, so it can
be modified on the fly. We can take pictures. We can see multimedia
elements together. At the end of the day, there is a modified version
of this object that is okay for all of us, and without spending
money. The saving for companies is really huge, and that’s why there
are lots of companies working on these kinds of solutions, and
Spatial can be one of the best, also, because it works with the
Hololens.
Alan: Yeah, and one of the
things that has come out of these types of collaboration tools — and
I think it’s important to note — is that they have to be completely
interchangeable with all the different XR technologies. So, if you
break down XR or — extended reality, or whatever you want to call it
— into its individual components: you’ve got the real world, and
then you’ve got the sliding scale of immersion, where you have
augmented reality — or overlays of computer graphics on top of the
real world. Then you’ve got mixed reality, meaning overlays of
computer graphics on top of the real world in context, so it knows
that’s a table or a chair, and it builds the experience around the
objects that are in your real world. And then you have virtual
reality, where it completely hijacks your whole world. If you look at
the scale of these, one of the things that I think is gonna be an
important factor in mass consumer adoption — but also, in businesses
— is leveraging the power of the mobile phone. Mobile phone-based
augmented reality has only scratched the surface; it’s been around
for five or six years, and companies are only starting to scratch the
surface.
One of the things I saw which is really cool is a company called Placenote, where you hold up your phone, and you can leave a note for somebody in 3D space. So, say you’re working on a house, and you want to leave some note saying, “don’t forget to move this thing here,” or you can leave notes for your housekeeper, or your Airbnb host, or whatever. There’s some incredible things that can be done with the mobile phones. If we take away the glasses for a minute, let’s see what we can do with the mobile phones that are in everybody’s pocket. Because by the end of 2019, there will be about 2 billion devices that have powerful AR built right into them, and they’re in everybody’s pocket. So let’s unpack some of the things that we can do with those phones.
Antony: Well, I think that — as
you said — the augmented reality runs in the phone is great, because
every one of us has a phone. The classic example is that we see how
it is done with Pokémon Go. There are lots of people running like
zombies in the cities, hunting for Pokémon. So we see how it can be
powerful, augmented reality on the phone. I think the tablet, with
its wider screen, can be more important for AR, even more than the
phone. I’ve seen some examples — I’ve tried some examples — where I
could see augmented reality through the tablet is great, because you
can see wider space that’s augmented. Some applications come to mind:
I’ve seen interesting things — again — one in maintenance. For
instance, there was this company — I don’t remember the name —
where I could look at my car that is not working. I just open it, I
see the engine. I take my tablet. I find my engine. There is a worker
that sees what my phone sees, and can write on my screen, can send me
instructions to fix what is not working. So what I’m going to say is
that I can see the augmentations from my phone, how can I fix my car
if it is broken. I’m not a mechanic, so it could be important and
interesting for everyone of us, and especially in certain sectors.
Well, maintenance is important. Another experiment that I’ve seen
that was quite original, was using the phone to see MRI scans of the
body. So, the world’s doctors sharing these MRI scans. By moving the
phone, the doctor was able to see the particular size of the scan,
and so; analyze better, in a more natural way, what could be the
problem for the patient.
Alan: That’s incredible. And I
think that’s a really great use case. We don’t touch too much on the
health care use cases, but they’re so vast. I mean, we could build a
podcast just around the health care uses of this; everything from MRI
scans in augmented reality; to mixed reality, aiming surgical tools
to make sure you’re accurate; to virtual reality simulators to treat
PTSD and other things like this. The applications in the medical
industry are literally endless.
Antony: Can I just add one thing
that I think is fundamental for your listeners? It’s that, even if
it’s more dedicated to consumers, what is huge now in 2019 with
mobile phones in the AR is advertisement. Because there are lots of
experiments how to use advertisements with AR filters; we all know a
Snapchat is doing great things with Snap filters and such. You can
try, for instance, makeup on your face directly with an AR filter.
You can try some glasses. There is also a great campaign by Burger
King that makes you burn the ads of their competitors in augmented
reality. And I think that is important for companies to know that ads
in AR are really outperforming the standard advertisement services.
Alan: You’re absolutely right.
That Burger King one: basically, you take your phone, you point it at
a competitor’s ad, and it catches on fire and gives you a free
Whopper, and then allows you to post the video of you burning down
their competitor’s poster or billboard or anything. In the first
week, they give away 50,000 Whoppers. Imagine the earned media that
they got around the world from that. I know when I posted it on my
LinkedIn, it got over 100,000 views, just on my LinkedIn. So, you can
imagine the amount of eyeballs that Burger King got from this, and it
probably only cost them maybe — I don’t know — $50-60 thousand to
build that application.
Antony: Yeah, probably.
Alan: Incredible. I don’t know
how much it cost, but it far less than the revenue that they brought
in from it and the marketing. So, you’re right, absolutely marketing.
If you look at Snapchat alone, Snapchat filters — they have built
over 400,000 filters on Snapchat. This is not them specifically, but
people building them, and brands are starting to jump on board.
Nike’s done a bunch of stuff with LeBron and with Michael Jordan. So,
Snapchat is leading the way in mobile phone-based augmented reality
marketing and advertising. So, you really nailed it on that one. And
there’s companies like Admix. Samuel Huber has been a guest on this
podcast as well; he’s creating programmatic augmented reality
advertising, so that brands can now scale their advertising using his
platform, on Instagram and Facebook, and now Snapchat. So, it’s
really an exciting time for advertising, as well.
Antony: Yeah. It’s a new world,
and I feel that it’s important, also, to jump now, that there is not
much competition, maybe? For companies, it’s a great opportunity to
start to make advertising in the new and more effective way.
Alan: I agree. I look at the
mobile phone-based a AR as the training wheels to where the world’s
going. In the next five years, the devices like Hololens and Magic
Leap and these really industrial glasses that are being used for
enterprise, I think, are going to end up on the faces of everybody,
because you just won’t be able to compete anymore. If I wear a pair
of glasses, and it gives me all the information I need — real-time
and in context of the world around me — and you don’t, good luck
trying to compete with me in my job. So I think we’re going to end up
having these glasses that give us superpowers on a daily basis. Some
of the other things that we didn’t touch on yet: haptic gloves. We
talked about haptic suits and wearing like a full suit for haptics,
but what about just something like a pair of gloves, that allow me to
reach out and grab something, and feel that it’s there. The feeling
of touch and seeing your hands and in virtual spaces is absolutely
incredible.
Antony: Well, it’s fantastic.
It’s a new field. It’s something that is not consumer ready, but
there are some enterprise solutions that are really interesting. One
that got very popular on the social media and such lately as being
HaptX gloves. The device is really good, because — while they are
incredibly cumbersome and expensive — but it has a haptic engine
that is really sophisticated, so you can really feel the sensations
of touch all over your hands, with a lot of positions, with bigger
solutions; so you just don’t feel a vibration on the one finger;
you’ll feel the vibration on the single point on your finger, for
instance. There is a demo with these glasses, where there is rain in
virtual reality. You can really feel the drops of the rain falling on
your hand.
Alan: Oh wow!
Antony: Yeah, it’s amazing.
Alan: The haptics ones also have
force feedback. So, I reach out and grab a can; it feels like a can
in my hand.
Antony: It’s a completely
realistic haptic sensation. You can feel objects on your hand as if
they were there. You can feel how they are heavy and such. It’s
really something that I will really want to have, but–.
Alan: Me too!
Antony: — but I don’t need them
now.
Alan: Tony, I tried the
Ultrahaptics. It’s just a little finger sensor; it looks like a blood
measure — like a pulse oximeter — on your fingers. I tried them at
CES, and I reached out and moved some blocks or whatever, and I could
feel the haptic feedback. But the second part: they told me to stand
by a fire, and then reach my hand into the fire. When I reached my
hand in the fire, the finger haptics buzzed on my fingers, and scared
the living crap out of me. I jumped back about three feet, and I must
have looked like a complete idiot, because there’s nothing there. But
it scared me in a way that would be an incredible training tool. I
mean, here’s here’s something like McDonald’s; “don’t reach your
hand into the deep fryer, because this is what happens.” Better
to train somebody in virtual reality than to train them when they
actually could burn themselves. That, really… it’s stuck in my
head, like, I touched a fire; it burned me. And even though I know it
wasn’t real, it felt so real.
Antony: Yeah it’s fantastic. You
mentioned Ultrahaptics. It’s completely the opposite approach from
HaptX gloves, because HaptX gloves, it’s a really cumbersome device
on your hands, but gives you every kind of possible sensation, while
the Ultrahaptics leaves you with your bare hands. So this is really
like the future. You don’t need any gloves, but there is a device
that showers ultrasound waves throughout your fingers, and can give
you the sense of touch. But you can’t have all the sensation that a
glove can give you. One of these can be great relief for training,
because you can really make the people have the sensation of having
an object in the hand. So, every kind of training that requires
tools; if there is not such a kind of powerful gloves, it’s just
like… I don’t know how to explain that if you play a game in
virtual reality with the standard controllers, like when you have a
sword, or you have a gun. Everything seems fake, because you don’t
have the sense of weight. You don’t have the real sensation of the
things in your hand. When you use these kind of gloves, you can
really feel the weight. You can feel the recoil of the weapon, and
this means that, for every kind of training that requires a tool,
from the most precise one, to the more hardcore ones — like a gun —
I think that gloves can easily improve the training experience.
Alan: I agree 100 percent.
Something else that I want to bring up, because it’s underestimated
on how this is gonna be really important — and the last thing that I
want to touch on — is scent devices. Being able to provide people
with a realistic scent… there’s two; there’s VAQSO, and then
FEELREAL is a Kickstarter on right now. I’ve actually had a chance to
try VAQSO; I put on the headset, and it had this little device that
was underneath my nose that gave me scents, and the scents were
programmed in the experience. So for example, I reached out and
grabbed a cup of coffee. I smelled the cup of coffee — brought it
close to my face — and as I brought it close to my face, the scent
was released, and I could smell coffee. Then I did the same thing
with a chocolate bar. And then the last part of the demo, they told
me, “now smell the girl.” I thought they were Punking me —
I thought it was gonna be on some television show. I looked to my
left, and there was a Japanese anime character, and you have to lean
in and smell her. When I leaned over — I smelled the coffee and the
chocolate — and then when I leaned over, she smelled like perfume. I
will never forget that.
Scent is one of the most incredible
ways for our brain to remember things, and I think in industrial
applications where there are times where scent is really, really
important. Maybe you’re underneath a mine that has sulfur, and you
want to give people that realistic experience of being there before
they get there, because some people maybe can’t take going into a
mine shaft with sulfur smell. Maybe they throw up immediately, and
it’s better to know before they send them down there and spend
thousands of dollars and training them to send them down. So, I think
there’s a really interesting opportunity to use these scent machines.
What are your thoughts?
Antony: Well, I’m a huge fan of
all the research that doesn’t go to the immediate direction of
improving just optics and audio, but also try to improve these other
sensations, like the scent and the taste. I think that…our sense of
smell is like one of the most ancient ones; it is wired in a
particular way in the brain, so it is connected with lots of regions.
Why have we just been ignoring it in including it in our headsets?
It’s actually one of the most important senses that we have. So it is
important that we we implement data in VR, if we want the simulation
to be realistic. What VAQSO is doing is great. Also, FEELREAL. We’ll
also be able to provide the sense of humidity in front of your face.
So not only the sense of smell, but also a bit of simulation of how
the air can be. It’s great, not only for entertainment; it can be
also good for marketing, because you can associate VR marketing
campaign with pleasant scents that can make the user remember the
brand better.
Alan: Oh my goodness — you know
what? This could be used for real estate. You go into a real estate,
and you smell cookies baking in the oven. Maybe you’re selling
cannabis, and you want the cannabis smell. Think about it: it can be
used for literally anything, and to market anything. Creating a
powerful smell interaction, your brain doesn’t easily forget that. I
think it’s really something that nobody has really fully embraced
yet. I’m really excited to do some experiments on that.
Antony: Yeah, we we should do
that together! Imagine that, as you say, you’re selling a house, and
when customers enters the house, maybe there is someone that is
cooking, and they can really smell the taste of food or whatever. You
can maybe dangle it off the balconies, and it could have the smell of
the sea, of the grass. I think that’s something that is sticking with
their head, and will make them continues to think about the house. He
can create a connection.
Alan: I agree.
Antony: That would be great.
Alan: Man, that would be
amazing. I think we found our new business model! [laughs]
Antony: Yeah! [laughs]
Alan: Tony, I really want to
thank you so much for joining us on the podcast today. I’m going to
ask you one final question: what do you see for the future of XR, as
it pertains to business?
Antony: Well, the sure thing that I can see as a trend is that it will be used always more. I assume that also, among my customers, maybe someone, some years ago, just said, “ah, VR. I’ve heard about it, but I’m not interested,” and now they’re coming again because their competitors started using VR. So, it’s something that will be fundamental for business in the next [few] years. It is fundamental because it makes companies spend [less] money, and makes their jobs [work] in more effective ways. I’ve seen lots of articles; I’ve seen lots of videos about the efficiency that companies are gaining. For instance, HTC always talks about Bell, that can prototype a helicopter now in months, and not in years, as before, thanks to VR. I see this great trend. Regarding the technology, what I can envision is that this kind of virtual reality is going to become always more realistic. As we’ve discussed, that would be slowly, with the adoption of new sciences. But visuals are going to become very realistic. We are seeing the Varjo headset, that has the resolution of the human eye; we can go further than that. Audio is already well emulated. With the gloves, we are going to improve; now, the devices are very expensive, but yes in a few years, it will be better. The sense of smell and taste will come later, but I’m sure that they will come, anyway. So, I envision lots of great things coming — lots of great hardware coming — and I’m so happy to be here, now.
49 min
July 22, 2019 Dress for Success: Talking Headsets and Haptic Suits with Skarred Ghost Antony Vitillo
From headsets to haptic suits, there is going to be a lot of accessories and apparel in XR to chose from, including some that expand senses you didn't even know were XR-compatible. Antony Vitillo – AKA Skarred Ghost – drops in to discuss different devices with Alan, their use cases, and what companies should consider when they go shopping for some.
Alan: Today's guest is Antony
Vitillo, better known as Skarred Ghost. Antony is an XR consultant
and author of an amazing blog called The Ghost Howls. He also runs a
consulting company called New Technology Walkers, where they develop
VR solutions and advise companies about how best to use VR and AR.
Antony recently traveled to the fourth annual VIVE Ecosystem
Conference -- VEC -- in Shenzhen, China. If you're not already
following Tony, you can learn a lot by connecting with him on
LinkedIn and subscribing to his newsletter at skarredghost.com. Tony
welcome to the show.
Antony: Hello, Alan! Thanks for
this opportunity.
Alan: It's so great to have you
on the show. I had a wonderful opportunity to speak with you many
times, and we are both very, very passionate about virtual and
augmented reality. I want to just thank you for taking the time to be
on the show.
Antony: I'm very happy to be
with you. I'm happy to speak to you live, after so many messages
we've done on LinkedIn. Super happy to be here.
Alan: Let's just dive right in,
and we're going to try to bring as much value as we can to the
listeners today. We have a lot to go through; we're going to go
through all of the different hardware aspects involved in
Virtual/Mixed/Augmented Reality -- XR -- and it's not just the
headsets or headphones. You've got things like haptic suits, haptic
gloves. You've got touch-sensitive stimulators. You've got VR
headsets, AR headsets. You've got mobile phone-based AR, eye tracking
set devices, taste experiments, hot and cold devices, thermal
devices, and then tracking systems for motion capture, and of course,
treadmills for omni-directional walking. So there's a lot to unpack
here. Let's start at something crazy; haptic suits. Let's talk about
haptic suits, and why and where these would be used in any
industries.
Antony: I'm very interested to
have these suits, because they offer the promise of letting you use
your full body in VR. So, finally, you can be there with all your
body. You know, my first startup was about full-body in VR, but using
Kinect. So, a different approach, but I'm a big fan of having the
possibility to kick objects, to move your body in every possible way,
and see your full self replicated in VR.
The advantage of using the haptic suits
over other approaches, like the one that they used with Kinect, is
that you don't only have your full body -- your full movement -- in
VR, but you can also feel sensations. You can have haptic feedback.
So you can [feel] hot, cold. You can feel pain and whatever. It's
really full immersion; a bit like we have seen in the Ready Player
One movie. Wade Watts wore that expensive suit, to fully be inside
the Oasis. This is why I think they're very interesting, because they
can really enhance your visual experience; your sense of presence,
like your ancestors like to say.
Alan: So, let's unpack this for
a second, Tony. What would some of the practical use cases of this...
I can see one in military training, where you're in a virtual world,
you're in a hostile environment, and maybe something explodes behind
you -- a piece of shrapnel hits you -- and maybe it vibrates. Maybe
explain some other instances, where this could be used in enterprise.
Antony: I think about different
possibilities. Like, for instance, I was talking some time ago with
some psychologists, and this can be interesting for rehabilitation
0 min
July 19, 2019 Axon Park's Taylor Freeman and Jacki Morie Talk Radically Reshaping the Way the World Works with XR
In our first two-guest episode, Alan chats with Axon Park founder & CEO Taylor Freeman, along with Jacki Morie from his board of advisors, about the limitless applications of XR technologies to tectonically reshape whole industries from the ground up. Everything from education, to healthcare, to even the nature of global economy can be improved and democratized within digital space.
Alan: Today's guest is a good
friend of mine, Taylor Freeman, from Axon Park and formerly of
UploadVR. Taylor is a serial founder in the media and immersive
technology space. He's been focused on growing and supporting the
virtual reality industry since early 2014 at UploadVR, where he
established coworking spaces in L.A. and San Francisco that have been
home to over 150 VR and AI startups. He's posted over 500 events all
focused on XR, trained thousands of students on XR development, and
worked with companies like Google and Ideo to train their development
teams. He's reached over 200 million people online through
UploadVR.com, the VR industry's leading news outlet. Taylor was
awarded Forbes Top 30 Under 30 for his work in building the VR
industry. He recently taught the first in-VR class at M.I.T., and
recently started a new company called Axon Park, focusing on using
immersive technology to power the next generation of higher
education. Taylor is dedicated to equalizing access to education
using XR and artificial intelligence. I'm really excited to introduce
Taylor -- and his advisor, Jacki Morie -- to the show. Jacki and
Taylor from Axon Park; if you want to learn more, visit axonpark.com.
Taylor and Jacki, welcome to the show.
Taylor: Thanks so much, Alan.
Thanks for having us.
Jacki: Yeah, thank you.
Alan: It's my absolute pleasure.
I don't even know where to begin. You've done so much in this
industry. Maybe we can just start with a recap of where the
industry's been from your standpoint and where we are right now and
where it's going, kind of in the general gist of things. Then we'll
get more into the work you guys are doing it Axon Park.
Taylor: Sure, thank you. And
Jacki, maybe if you want to speak to sort of the history of VR --
Jacki has been an early pioneer in the VR industry for over 30 years
now, if I'm not mistaken.
Jacki: Somewhere along those
lines, yes.
Alan: Wow, incredible.
Jacki: It's interesting, in
terms of education. Really, virtual reality was started to service
training for all kinds of activities, such as NASA astronauts who
would have to remotely control things on the space station. Or the
military, and looking at how immersive a technology could enhance the
training that the military people were getting. So those two areas
were very early in setting the pace for virtual reality as a learning
mechanism.
Alan: You know, one of the
people that I had the honor of meeting was Evelyn Miralles from NASA.
Jacki: Oh yes.
Alan: It's really incredible to
learn how long they've been working on this technology. This is not
the world where it woke up to virtual/augmented reality when Facebook
bought Oculus for 3-billion dollars. This work has been going on for
many, many years in research labs in universities, and in
corporations where it just wasn't available to the public. It was
just something they were working on, because it does hold such a
tremendous value.
Jacki: Right. It was about 1965
when this first started, and I call that the first wave. The second
wave was in the mid '80s, when we had NASA and more activity by the
military and
0 min
July 17, 2019 The Right Displays for Challenging Tasks: XR on Oil Rigs, with Shell’s Michael Kaldenbach
Many scenarios that might be improved by an augmented reality heads-up display shouldn’t require an overly arduous selection process; most gizmos will do if you’re checking a weather app while jogging. The same can’t be said for picking out a device to help oil rig workers work safely and efficiently in the middle of the Permian Basin. Shell’s VR incubator lead Michael Kaldenbach talks with Alan about the things his team had to consider when selecting the right device for the job.
Alan:
Today’s guest — Michael
Kaldenbach – is an augmented, mixed, and virtual reality incubation
lead at Shell, the global oil company. He is a driven, goal-oriented,
resourceful, and creative person, who really understands the
usefulness of this technology, and bringing how to bring it to the
market. He’s chosen the family motto of Arctic explorer Sir Ernest
Shackleton, as it accurately reflects how he approaches any challenge
or goal: “victory through perseverance,” or “Fortitudine
Vincimus.” He strives
to apply entrepreneurial mindsets and thinking up out-of-the-box
solutions and approaches when working in this technology. If you want
to learn more about his work, you can visit Shell.com.
I
want to welcome to the show, Michael Kaldenbach. Welcome to the show,
Michael.
Michael:
Hi, Alan. Thank you very
much for having me on.
Alan:
It’s my absolute
pleasure. I’m really excited. I want to dig right in here, because I
know you guys at Shell have been doing a ton of work in everything
from kind of marketing and trade shows, right through to oil wells
previsualization. So let’s talk about some of the ways that you and
your team are using virtual/augmented reality right now.
Michael:
So I think One of the
better case studies we have is around augmented reality remote
assistance, and I’m sure you’ve seen examples in the wider industry
for that one. But for us at Shell, that means that we utilize a
head-mounted display — in this case specifically, the Realwear —
and it is used for our operators; for quick resolution, and to get
remote expertise to be brought in.
I
think it always helps if I provide a little story to set the scene;
think of an offshore oil platform out there in the ocean. Typically,
the most senior person is the control room operator, and there are
more junior operators that are assisting the running and maintaining
of these kind of assets. If in the control room, they see a deviation
on one of the many dashboards they have, they send out a more junior
operator to investigate — normally with a radio phone or walkie
talkie — and then they guide them through, they get back to “what
is the situation; what’s the sound the machine is making?” But
where we really revolutionize that process is with a head-mounted
display. It is as if the experienced operator has immediate eyes on
the situation. So think about [it] — you see (or I see) what the
junior operator is seeing, and thereby, I can use my years of
expertise to resolve the issue, and get back to safe operations.
In a
case where my expertise set is also not sufficient, we can quickly be
joined by a remote expert who can be onshore — can be anywhere in
the world — to join that same virtual room, so that a three-way
conversation happens. Not only that: instead of having those
conversations like, “I recognize the problem; you need to switch
off the third button from the left, it’s kind of greenish on the left
side, bottom side of the machine,” instead, we use something called
“telestration,” and that’s the benefit of having a head-mounted
display, whereby I — as the remote expert — can draw on my screen
and the same visual is replicated to the junior operator, so in his
line of sight, he gets an annotation — a circle or an arrow,
whatever is helpful; it could also be a video — to resolve the
situation. Thereby, we quickly resolve issues that might end on
production deferment, or something even more serious. It is really
helping to revolutionize our operations.
Alan:
Just to kind of recap:
for people the junior operator’s got a pair of glasses on. He goes on
site, or she goes on site. The senior operator is sitting at a desk,
being able to monitor multiple junior operators, I would expect.
Michael:
Correct.
Alan:
And that person is now able to remote see exactly what the junior
operator is seeing, and not only that, but annotize on top of their
vision. Is that correct?
Michael:
Correct. And as you can
imagine, the environments that we as an energy business operate in,
they can be far-flung. They they can potentially be dangerous. So,
you really want to have the knowledge with everyone out there in the
field. This world is changing so fast, and it’s really difficult for
everyone to keep up with all these changes. By having the ability to
pool your resources — bring them via virtual reality into the
situation — that is really helping us.
Alan:
So this is real-time. So
This isn’t like virtual reality, where you know you’re training in
something — and then obviously, you can use that as well — But this
is real-time. See-What-I-See interactions. Now, what about some of
the lag issues? Or are you guys experimenting with 5G technologies to
eliminate the lag?
Michael:
As
you can understand, connectivity is a struggle, whether it’s 3G, 4G,
and the signal range when you’re out there in the Permian Basin, or
out there in the North Sea — it’s difficult to get that signal.
Wi-Fi is also very difficult. We tend to have a lot of steel
environments, and line of sight obstructions. And in general, if you
think about telecommunications – 3G, 4G – that’s pretty much
centered around populated environments. So definitely, we struggle,
and we look to all sorts of possible solutions for connectivity —
whether that is low-part/wide-area network, whether that’s 5G,
whether it’s low-orbit satellites — it’s always dependent on the
actual business case, and what makes sense in that situation.
Alan:
I always find it interesting: when we’re talking about connectivity,
and you actually cut out right when you’re talking about 5G.
Michael:
[both
laugh] I
mean that’s why we need 5G.
Alan:
It always happens right
around the connectivity conversation. “How do we get connectivity
stable?” And you’re like, “oh-ah-erk-uhh.” Okay! [If] we can’t
make a podcast recording work, what are we supposed to do in an oil
site in the middle of the Congo?
Michael:
Exactly. Exactly.
Alan:
You guys are really
solving massive problems, using this technology. You mentioned
Realwear; maybe you can walk listeners through what the Realwear
glasses are, and why you chose them over other pairs of glasses?
Michael:
Well, again – it’s
about the environments that we are in. (And let me know when I start
cutting out again).
We
are predominantly in something we call ATEX zones, which means that
it’s specific safety regulations due to the substances we use in our
production environments; they require our devices to be intrinsically
safe. For that reason, we look towards the market. The hardware
market, and identify the pieces of equipment that are licensed to
operate in that environment.
When
we started looking at these head-mounted displays about two years
ago, we did try pretty much everything that’s out there in the
market, ranging from the old-but-true Google Glass, to an ODG or
Realwear. There used to be a fair number of opportunities for us, but
again, we were sort of restricted by that ATEX certification
requirement. We went through quite a rigorous phase of proof of
concept. We tested connectivity – is this even going to work at our
sites? We looked at ruggedization – if we drop it, does it
immediately get damaged? And for us, the Realwear — which markets
itself as a head-mounted tablet, which… uh, fair enough – it’s a
seven inch screen. I don’t know if you want to call it a tablet, or
just a phone-sized screen, but it came out on top. It’s very
ruggedized, it comes with an ATEX certification, and it has all the
essentials we need; a good camera, a good microphone, and a good
screen, for us to look at the information that we need.
Alan:
You mentioned good
camera good microphone. I think these are essentials, and one of the
things that I saw recently at CES was a company called Kopin, and
they make the micro displays – I think they even make the one for
Realwear – they make the actual micro displays. Something that they
introduced me to was their Whisper technology, where you could – in
very loud environments – you could talk to your device, and it
would understand it, and it got rid of all of the external noise
altogether. It was really quite amazing; somebody who is standing
right next to me was trying to tell it the instructions, and it would
only respond to my voice, which I think – as we move into these
headsets that do more than just be able to bring them information,
but they’re actually bringing up information using AI algorithms and
stuff to give us real-time information – I think having a system
that allows you to whisper or talk in really loud environments is
key.
When
you chose to go with Realwear, how do you then start developing for
that? Are there off-the-shelf products? Are you looking to startups
to to help develop these things? Are you building things in-house?
Michael:
Well… if you don’t
mind, let me get back to what you said, because you did trigger me a
little bit. The reason why I’m passionate about this technology, and
I’m passionate about the team within Shell, is because I firmly
believe that we’re on the cusp of a paradigm shift. We went from
using your mouse and keyboard to get information, to using the speed
of your thumb, whereby we use our thumb to operate a mobile phone and
tablet. And now we’re at the cusp of changing that technology to
voice-driven.
So,
you are absolutely right in saying it becomes exceedingly important
for these kinds of voice recognition algorithms to not only work in
noisy environments, but to then also accurately translate what you
need, whether you have an accent or not. This journey started a
couple years ago with the likes of Alexa coming to the market. We
have other players like Cortana and Google Allo. But really, I’m so
excited. I also don’t think that we should separate voice and natural
language processing from the idea of the extended realities, whether
it’s virtual or augmented reality. For me, they represents the
foundational blocks upon which we build everything.
So,
sorry. I wanted to leave that with you, because I feel so passionate
about that.
Alan:
I think that’s a really
good point, and I think we should just dig into this a little. On my
last interview, I interviewed Rori DuBoff from Accenture, and she
mentioned XR – or extended realities – you’ve got virtual and
augmented reality, mixed reality. So, virtual reality; you put on a
headset, and you can change the whole environment you’re in for
training and stuff like that. Augmented reality; overlays data. And
then mixed reality; overlays data in context to the real world –
when you’re looking at a machine, it can give you very specific data
around how to turn the knobs and stuff like that. As an extension of
that, you’ve got voice recognition, or human language processing
(which I guess falls under the category of artificial intelligence),
but then you also have computer vision, and machine learning, and big
data analytics, and no one of these technologies on their own is a
useful solution.
People
need to understand that this is a continuum of a number of
technologies coalescing together, and I think this – as somebody
who studies the future of humanity – we’re reaching this
exponential growth point, where 5G connects with artificial
intelligence/computer vision/machine learning/human language
processing/virtual and augmented reality, and all running on a block
chain. If you take all that technology separately, they’re amazing;
put them together, and now we’re really revolutionizing businesses.
Michael:
I think that’s, for me
also, what’s exciting about working for a larger enterprise. Because
we do have all those individual components. We have dedicated teams
working towards that, which allows the digital realites team that
we’re leading – I’m just gonna put it out there; it’s going to
stretch the mind a little bit – but we believe in this thing called
“Digital First,” whereby sort of the digital twins and the
various digital realities become a fundamental component of
operations.
The
idea there is that any action or operation is initiated through these
kind of digital methods, with the physical reality only being an
outcome of that digital reality. Whether it’s like collaboration in
virtual reality in order to prepare for a complicated task with
colleagues across the globe, or it’s about simulating the outcome of
like a task with a digital twin, where you run through all the
permutations. Or – one of my favorites – step-by-step guidance in
augmented reality to really perfectly execute difficult procedures,
so that we are much safer and much more efficient. That sort of
digital-first mindset builds upon all those capabilities – and some
people call it the “metaverse,” I like to term “the
digitalverse” – but honestly, it’s an exciting time, and I hope
everyone joins us.
Alan:
“Metaverse,” for the
people that are listening, the term was coined in a book called Snow
Crash about, oh, maybe 15 years ago? The term refers to the world
you’re in when you’re in virtual and augmented reality – you’re in
the “metaverse” – and it’s kind of got this worldwide thought
to it.
But I
am a little partial to that word. [laughs]
Michael:
No, no; fair. But, as I
was sort of ranting on, I am reminded that what we’re doing, it’s
more than just technology. It really is about people, and creating
more agile ways of working, and for safety. And I don’t want people
to forget that; all the amazing things that we can do, it is for the
benefit of people. Just wanted to put that out there.
Alan:
Let’s unpack this one by
one, here; you talked about collaboration and VR. Whether that’s
training or previsualization for something, what does that mean to
people? What is the benefit to the people in your organization, being
able to collaborate in VR?
Michael:
Well, I think it comes,
first, from a mindset that we recognize that we don’t know
everything. They always said – I don’t know if I have this saying
correctly – it takes a village to raise a child. In this case, it
takes a large organization to tackle some of these big problems worth
solving. In that sense, we really need to work together. But we’re in
all places in the world. In order to really utilize the diversity
that we have within the company, you want to bring them together, but
at the same time, not be a burden to the earth, in terms of having
everyone fly in from all over the world. Where virtual reality is
definitely changing the game here is: it’s as simple as putting one
of these headsets on, and then you meet your co-workers wherever they
are.
Alan:
What platform are you using for that type of thing in virtual
reality? Did you build your own, or are you experimenting still?
Michael:
For that, we are very much in the experimentation phase. The idea’s
there – and I mean when I say that the idea’s there, the core idea
has been around for ages – and when we first started this journey
two years ago, we saw maybe three players out there in the market. In
the last year, they have popped up like mushrooms.
Alan:
Probably
three coming every month now.
Michael:
[Laughs] Exactly!
So that makes it a
little bit–
Alan:
The ones we’ve tried are Roomi, Alt Space – obviously, is not
really one for professional – but Alt Space, Roomi, VR Chat, High
Fidelity. There’s one called Glu out of Finland. There’s a bunch that
have come up. I tried the Glu one; I was really impressed with that
one, actually. It was really good.
Michael:
Well, it sort of reminds
me – I might be betraying my age, now – but it reminds me of
Second Life, and when they started, how amazed people were, and how
it brought people together – not in the sense of a game, but in the
sense of a community. You saw the community efforts build beautiful
things, and functional things as well. We saw companies go to Second
Life and open up their branch there, for customer support.
So
when I say that, I see this sort of same moving happen out there in
the market right now. But we’ve learned from Second Life, and these
kind of relatively uncontrolled environments are difficult to manage
for an enterprise such as ours – never mind the plethora of legacy
systems that we have in operation that you would like to tie into
these kind of environments. The current market, with the many, many
players out there, is not conducive for us to select one. Right now,
we’re very much testing the waters; finding what works for us, and
what not does not work for us. In honesty, I think we’re going to
have to go with one partner, and then help them achieve what we need
them to achieve.
Alan:
Yeah, I think that’s
what other people are doing, as well. They find the solution that’s
closest to what you need, and everybody rolls up their sleeves to
just make it what you needed to be.
This
is a great example – and it’s a great segue – because something
that we’re about to announce… I’m not going to talk about it on the
show right now, but something we’re going to announce is that
collaboration between enterprises, like Shell, and startups that are
building these technologies. Because you guys could probably build it
in-house, but you’d have to find the people to do it; find people
that are passionate. It’s better just to partner with a startup, give
them some funding, and say, “here’s some funding so you don’t have
to worry about paying your bills, and we’re going to be your client,
and we’re going to tell you what we need, and help you build it for
us. Then you can have it as a product after that.”
Michael:
Correct. And in that
sense, companies like Shell, we’re not – I have to be careful when
they say it is a little bit – we’re not an IT company. We have a
lot of IT components, and we spend a lot of money on IT systems; but
in our nature, we are not a company that’re going to build these kind
of platforms, and then sustain them, make sure that you evolve into
the market. In that sense, we’ve adopted a platform as a service;
software as a service preference. So, market standard, unless we have
a severe gap in what the market is doing. In that way, we prefer
working with startups. We prefer working with the market, so that the
products that we use get also used by other parties, and that then
evolves the ecosystem in ways that we could not have imagined if we
were to come in heavy-handed and build it ourselves.
For
the people out there: don’t be afraid of the larger corporations. I
can only speak for Shell, of course, but we’re definitely open to
these kind of collaborations, and sometimes, it’s only a five-man
team that comes up with something brilliant. We definitely keep our
eyes and ears open to the whole market.
Alan:
Some of the best things
that we’ve seen recently have been… it’s – again – that three-,
four-, five-person team, because here’s the thing: with digital, as
long as you have some talented people, and you have a vision, you can
build something – pretty much anything! If you have a vision, and
you have a strong team that can build it; it won’t take that long. A
lot of these platforms have been built over the course of six months
to a year, maybe a year and a half. And these are really robust
systems.
I
think one of the challenges that you mentioned is integrating with
legacy systems. I think this is going to be the biggest challenge for
any startup, building something; how do you build it to work with all
the different, weird legacy systems that companies are using? That’s
going to be a challenge, I know.
Michael:
Oh definitely,
definitely. But not insurmountable.
Alan:
No, definitely not! And
I think working with partners like yourself, that understand that
working together is beneficial for everyone, is the key. For the
businesses listening: it’s really vital that you… there’s some
companies that I’ve heard, they’ll invite startups in, they sit down
with them, they get as many ideas as possible, and then they build
their own team and try to cut the startup out of it. I’ve seen it
happen over and over again, and it’s not a good way of doing business
in the long run. You’re going to miss out on opportunities by doing
that. It may save some money in the short run, but by embracing these
startups and really working with them, I think it’s really beneficial
for everybody in the entire ecosystem as well. And it gives you the
first foot in the door to acquire them, should you find that this is
something you really want to have.
The
second thing you mentioned was simulating outcomes using digital
twins. Can you maybe unpack that a bit for people? What do you mean
by simulating outcomes using digital twins?
Michael:
Well, let me start by
perhaps explaining what we mean with a “digital twin.” For us,
the definition of a digital twin is “a digital representation of
something physical,” and the reason why I say that “for us,” a
digital twin can be as small as a screw that is critical to a
process, but it can also be the size of a full-blown asset, or
offshore plant. It takes a lot of these digital twins to work
together to have logic behind them.
That then allows us to simulate certain scenarios. If you can imagine a situation where you want to perform a certain type of maintenance – whereby we have to take a compressor offline – a piece of equipment in the whole process. Then, the rest of the process usually continues, but everyone has to take a step up and run at 120-130 percent efficiency. Now, when you have these digital twins, you can perform the operation in – typically, we use virtual reality for these kinds of scenarios – whereby you simulate, “okay, if I take this compressor offline, then the other compressor is going to have to work at 130 percent. However, we also have quite detailed maintenance logs.” So, using very advanced analytics – and honestly, that’s not my field, so I won’t comment further on that – we are able to make probability statements around potential failures, what the probability of those failures are, and then, how do we visualize the impact of that failure? Does that result into a spill? Does that result into something more serious? All these variables allow us to be more prepared, before we do the actual operation. It’s all with the preparedness mindset where we use – again, usually virtual reality – but you can also use augmented reality to simulate, before you do something, “what am I going to expect in the future?”
Alan:
Can you give an example?
Like, a specific example.
Michael:
Well,
I mean, I just gave the
example of the compressors.
Alan:
When you’re talking
about the compressors, though, what you mean? So, you watching a
compressor, and then… what would that look like, when when you guys
were doing it?
Michael:
A good example is we had
to update a piece of equipment in an asset recently, and it needed to
be lifted out of the asset for the new one to be placed back in. What
we did is, we had the digital twin in virtual reality, and by the
power of being in a virtual reality where everything is rendered by
computer, we were able to lift that piece of component up way into
the sky, to detect whether it would hit any cross beams, or any other
pipes that were still in the vicinity. And we lifted it up, no
problem. It cleared everything. So that means that, if we were to use
a crane, we could just lift it straight up and we could go ahead.
However,
the new piece of equipment actually had a protrusion. The base was
still the same, so the technical drawings, they looked to be okay,
but one of the protrusions was about a meter out to the side, which –
when we lowered it back in – we had a collision detection on the
digital twin over our environment, with the new digital twin of the
component. That allowed us to preemptively say, “okay, well, in
this case, we’re gonna have to reroute certain pipelines, in order to
get the new piece of equipment in.”
That
kind of simulation is very powerful, because if you were to
experience this in the real
reality, you would have manpower there; it would constitute a
potentially unsafe situation. And, when lowering the piece of
machine, you would actually detect, “we can’t do this. You have to
lift it back up.” Then you have to take action. So, these kind of
processes get delayed, and we are working on the clock, with very
specific permits there. So it’s a very costly affair. By having these
digital twins available, you save a lot of cost and time.
Alan:
Okay. Let’s just take
that one specific example. What kind of cost savings do you think
that created? Just a ballpark, I mean; like, is it in the tens of
thousands or hundreds of thousands?
Michael:
Yeah, I don’t like to go
into specifics on that one.
Alan:
But it far outweighs the
costs of creating the digital twins, would be my… that’s what
I’m…
Michael:
Definitely, by far.
Alan:
People
think, “it’s expensive to create these scenarios,” or whatever,
but the consequence of not
creating the scenarios is exponentially more expensive.
Michael:
Correct. I’ll subscribe
to that statement, yes.
Alan:
Perfect. And the last
one you talked about – and we’ve kind of dug into this a bit more –
is remote collaboration. Are there examples of how this has
benefited… have you seen a time where remote collaboration has
averted a downtime that would have costs millions of dollars to have
downtime? Because I would assume for every hour of downtime of an
oil-producing facility is going to be in the millions of dollars of
lost revenues – or lost production, anyway. Do you a specific
example of when remote collaboration saved the day?
Michael:
I do not have, at the
top of my mind, an example where we save the day. But we had a
collaborative session last week, which is why it’s sort of at the
front-and-center of my mind, where we were gearing up for a workshop
to tackle a very difficult challenge.
We
got into the virtual space, and we did a little bit of an intro, and
one of the gentlemen who were joining us from… I believe he was
somewhere in the Americas region, he explained his background and
experience, and he mentioned offhand that it felt very similar to
something he experienced before. So – by the power of being in
these kind of virtual realities – we conjured up the documents,
which were stored on a server somewhere, and we found out it’s
actually quite similar and (what I was not looking forward to) what
was going to be a three-day workshop turned into a 30-minute
exercise, whereby we had a clear idea of how we were going to resolve
and tackle the serious challenge.
For
me, that’s also the power of this collaboration. It is utilizing the
knowledge that might not be readily available – might not even be
documented – but because this person was willing to join us in this
virtual space, and we were able to connect – and not not via chat,
but he was talking, we got to know each other – thereby, we linked
on something that’s – well, again – not saving the day, but
certainly saved a lot of time and money.
Alan:
Well, yeah. I mean, if
you take – I don’t know how many people were there – but let’s
say five people, and you took something that would have taken three
days to 30 minutes; that’s an enormous savings for any company, and
just that one simple use case probably paid for all the technology
investment that you made in those VR headsets. It’s crazy. The
exponential savings and profitability from using these technologies
cannot be ignored anymore.
Michael:
Agreed. Yep.
Alan:
What is the most
important thing businesses can do to leverage the power of XR
technologies right now? What is the thing that you would say – for
a company that’s listening now that maybe hasn’t even experimented,
doesn’t know really anything [about XR] – what’s the first step?
What’s the most important thing that they can start to do?
Michael:
This
is a little bit going back to university for me, but honestly, ask
the question “why” first. I’m very passionate about all the
digital realities, but you have to take it into account; is it worth
it for the business? Why are you doing it? And what is the
differentiator?
Just
to reflect back on the example I gave around collaboration; could we
have done the same with a Skype call, rather than a virtual room that
we were in? Make sure that you tell the story of why
something is differentiating from a current capability. It might not
always be down to, “you’re gonna save X
amount,” but it is about the intangible aspects; you’re going to
save time, or this is allows you to operate safer. But in a lot of
cases, you really need to have the “why” clear, then the “how”
sort of follows up on that. And the “how” typically describes
which of the digital realities you would want to use, because the
situation will lean towards one or the other. Again: preparedness is
more virtual reality, when you’re out there in the field; augmented
reality is preferred because… well, if you’re out there, you’re not
going to put up a headset, and thereby lose your complete line of
sight by doing virtual reality when you’re out there.
So,
focus on the “why,” and then the others – the “hows,” and
especially the “how much”
– will tell itself.
Alan:
My last question for
you, Michael – and this is more of a personal one – what problem
in the world do you want to see solved using XR technologies?
Michael:
For me, I’m going to
throw some jargon at this: it’s the idea of instant upskilling. Now,
I’m going to explain that one a little bit, “instant upskilling.”
What that means is – again, it’s a little bit of a jargon – but
when we started this conversation, I told you that right now, the
world is changing so fast, that you can learn everyday. You can learn
every hour,
and still something new will pop up. So there is a competency gap
that is growing.
I see
these kind of extended realities – whatever you want to call it –
fill that void, where we have much more immediate access to expert
knowledge, allowing us to go much broader and – when necessary –
that you have it available to you. I don’t want to go too far into
the future. I don’t want to go Matrix style, where we inject it into
your brain, so you have it immediately. But think about augmented
reality. You’re out there, you have to change some wiring. You may
have done that specific wiring course three years ago – again, I’m
just painting a picture. Then you can conjure up the step-by-step
instructions of how you actually need to change that wire. Then, you
can continue doing what you need to be doing, so that… I want to
say “at your fingertips,” but that’s outdated – by talking to
your voice-driven device, you can have that information available to
you. That’s, for me, the big idea.
Alan:
Amazing. I love the fact
that you [said, “have all the information at your fingertips…
wait a second – that is outdated!” That’s a crazy statement, that
having all the information “at your fingertips” is now an
outdated statement.
Michael:
[Laughs] Exactly.
Alan:
That was the catch
that’s going to get people hooked on this podcast, for sure.
Well
Michael, I want to thank you so much for your insights and your
input; it’s been very valuable. I’m sure listeners – if they want
to reach out to you – how can they find you?
Michael:
The best ways to find me
on LinkedIn. Again, I am very eager to connect to the wider market.
I
really respect, Alan, what you’re trying to do. As a whole, it is
about raising awareness about the potential, and then following
through with it. So everyone, please reach out; follow us on
LinkedIn. We tend to share what we do there, via press releases and
blog posts, et cetera. So, happy to connect.
33 min
July 17, 2019 The Right Displays for Challenging Tasks: XR on Oil Rigs, with Shell's Michael Kaldenbach
Many scenarios that might be improved by an augmented reality heads-up display shouldn't require an overly arduous selection process; most gizmos will do if you're checking a weather app while jogging. The same can't be said for picking out a device to help oil rig workers work safely and efficiently in the middle of the Permian Basin. Shell's VR incubator lead Michael Kaldenbach talks with Alan about the things his team had to consider when selecting the right device for the job.
Alan:
Today's guest -- Michael
Kaldenbach – is an augmented, mixed, and virtual reality incubation
lead at Shell, the global oil company. He is a driven, goal-oriented,
resourceful, and creative person, who really understands the
usefulness of this technology, and bringing how to bring it to the
market. He's chosen the family motto of Arctic explorer Sir Ernest
Shackleton, as it accurately reflects how he approaches any challenge
or goal: “victory through perseverance,” or “Fortitudine
Vincimus.” He strives
to apply entrepreneurial mindsets and thinking up out-of-the-box
solutions and approaches when working in this technology. If you want
to learn more about his work, you can visit Shell.com.
I
want to welcome to the show, Michael Kaldenbach. Welcome to the show,
Michael.
Michael:
Hi, Alan. Thank you very
much for having me on.
Alan:
It's my absolute
pleasure. I'm really excited. I want to dig right in here, because I
know you guys at Shell have been doing a ton of work in everything
from kind of marketing and trade shows, right through to oil wells
previsualization. So let's talk about some of the ways that you and
your team are using virtual/augmented reality right now.
Michael:
So I think One of the
better case studies we have is around augmented reality remote
assistance, and I'm sure you've seen examples in the wider industry
for that one. But for us at Shell, that means that we utilize a
head-mounted display -- in this case specifically, the Realwear --
and it is used for our operators; for quick resolution, and to get
remote expertise to be brought in.
I
think it always helps if I provide a little story to set the scene;
think of an offshore oil platform out there in the ocean. Typically,
the most senior person is the control room operator, and there are
more junior operators that are assisting the running and maintaining
of these kind of assets. If in the control room, they see a deviation
on one of the many dashboards they have, they send out a more junior
operator to investigate -- normally with a radio phone or walkie
talkie -- and then they guide them through, they get back to “what
is the situation; what's the sound the machine is making?” But
where we really revolutionize that process is with a head-mounted
display. It is as if the experienced operator has immediate eyes on
the situation. So think about [it] -- you see (or I see) what the
junior operator is seeing, and thereby, I can use my years of
expertise to resolve the issue, and get back to safe operations.
In a
case where my expertise set is also not sufficient, we can quickly be
joined by a remote expert who can be onshore -- can be anywhere in
the world -- to join that same virtual room, so that a three-way
conversation happens. Not only that: instead of having those
conversations like, “I recognize the problem; you need to switch
off the third button from the left, it's kind of greenish on the left
side, bottom side of the machine,” instead, we use something called
“telestration,” and that's the benefit of having a head-mounted
display, whereby I -- as the remote expert -- can draw on my screen
and the same visual is replicated to the junior operator, so in his
line of sight, he gets an annotation -- a circle or an arrow,
whatever is helpful; it could also be a video -- to resolve the
situation. Thereby, we qui
0 min
July 15, 2019 Get a Glimpse of a New Way to Accelerate Startups with DJ Smith
DJ Smith is the co-founder of The Glimpse Group, an XR technologies accelerator that’s doing startup incubating in a whole new way. Listen as Smith explains to Alan how Glimpse works, and some of the tech companies that are already beginning to grow under its umbrella.
Alan: We have an amazing guest today: DJ Smith is the co-founder and chief creative officer at The Glimpse Group. The Glimpse Group is a holding company for a portfolio of 10 startups, focused on virtual and augmented reality industries. At Glimpse, DJ’s responsibilities include overseeing the production of all the VR and AR content, as well as leading efforts to locate new subsidiary companies. In addition, DJ is the organizer of the New York virtual reality meetup. NYVR hosts monthly events focused on virtual reality technology and the premiere venue for the industry networking collaboration within New York City. NYVR is the second-largest virtual meetup in the world, with over 6,000 members. Prior to entering the VR/AR industry, DJ worked 20 years in the real estate and construction industries.
The Glimpse Group is a company designed with the specific purpose of cultivating entrepreneurs in VR, AR, and of course, XR. The business model simplifies many of the challenges faced by entrepreneurs, while simultaneously providing investors with an opportunity to invest directly into the VR/AR space. The Glimpse Group will fund, cultivate, and manage business operations while providing a strong network of professional relationships. Being part of The Glimpse Group allows entrepreneurs to maximize their time and resources in pursuit of their mission-critical endeavors. They’ve invested in 10 companies, which we’ll get into this show, but the 10 companies are Adept Reality, In It, D6 VR, Kobach, Immersive Health Group, KreatAR, Number9, Early Adopter, MotionZone, Foretell Studios, and I’m sure there’s gonna be a lot more on the show. To learn more about The Glimpse Group, visit TheGlimpseGroup.com.
Welcome to the show, DJ.
DJ: Thank you very much for the amazing introduction.
Alan: It’s my absolute pleasure. I’ve been waiting for this call for so long, I’m really excited to have you on the show. Tell us, how did you get into virtual/ augmented/mixed reality, or XR technologies?
DJ: Sure. It’s a life calling that I was seeking for many, many years. In 2012, I saw the Palmer Luckey Kickstarter video, and I was like, “it has finally arrived!” So from that point on — it’s kind of a pivotal moment in time for me and my life – I started buying all the toys; really, just absorbing as much tech as I possibly could.
The meet up was formed shortly after that. I think the meet up started with 10 people in a dusty old conference room, and it just has steadily grown and grown and grown, and as it grew, my involvement in the industry grew and grew and grew. So we’re now the, I believe, the second-largest VR meetup in the world. It basically put me in direct connection with many of the developers in the city. The big entities; the Googles, Microsofts, where we would hold our event, as well as the whole investment community.
Then, about four years ago, the Oculus Rift and the VIVE was coming out, and I saw that there was an opportunity to actually start making a living out of it. I put in my notice in my construction job, and a week later I met my current business partner, Lyron Bentovim, who saw the same timing and opportunity within the space. He came from the investment side of things; had his own tech startup, ran hedge funds, and was most recently involved in publicly-traded companies. He had the idea of the Glimpse business model, which was taking bits and pieces from his experience of hedging bets and the challenges with a typical VC angel startup scenario, but was looking for somebody like myself that was deep into the industry already, and had connections and knowledge about the tech. So I had recently had some — plenty — of time on my hands, because I just gave my notice, so we teamed up, and that’s when The Glimpse Group was formed.
Alan: So you basically jumped off the cliff and said, “I’m in.”
DJ: [Laughs] Yeah, that was the sell-or-die moment. Actually, I am very blessed to have a wonderful, supportive wife who knew that there was no way I was not going to do it, but also happened to have a good job with health benefits. So I said, “you know what, I got probably a year’s runway to see what we can make happen, and four years later, it’s still going strong.
Alan: Amazing.
DJ: Yeah, it’s been really great.
Alan: We met, I think was it three years ago, at… I want to say SVVR.
DJ: That’s right. That’s right. A very late night at a bar, explaining this crazy Glimpse Group business model. And at the time, we were a fresh start. I always like catching up with the folks that I’ve seen the whole rollercoaster ride, and how it’s panned out.
Alan: It’s been pretty impressive. I mean, we started our companies almost the same time. We took a different approach; we went down the consulting and custom-built [road], as a company. And it served us very well. You guys took a different approach, and bet on 10 companies. Explain the business model, because I think it’s really unique.
DJ: Sure. I’ll just preface it with, nobody really does it like we do it. That has its pros and cons, and trailblazing is a difficult road sometimes. But in comparison to the typical startup path, we thought that it would have some advantages, and hindsight is 20/20, and we think that it really does help — especially in an early-stage industry like AR/VR. So the way that we work is, The Glimpse Group is a holding company. We basically raise money at the top level to seed out to our startups; our startups are acquired in stock in the parent company, and then our entrepreneurs that come on to lead each of the divisions are given a basic salary, health benefits, a ton of front office and back office support by the parent company, and then basically a 10 percent undiluted stake in their entity upon exit. In that way it’s analogous to running through the whole Series A, B, and C rounds, but our entrepreneurs can really focus on their technology. They don’t have to worry about a lot of the things that a typical startup entrepreneur has to worry about.
Alan: I think it’s wonderful. One of the things that we keep seeing over and over again is these amazing startups who have fantastic products, and they’re very passionate. But they don’t have the marketing prowess, or they’re missing the backend systems, or they’re just don’t have the business acumen to take their products to market. It’s actually one of the reasons we started XR Ignite; to help these startups foster their growth and introduce them to big companies. But you guys are doing that in a different way; instead of taking a small percentage, like an accelerator — like Techstars or Ycom — you’re acquiring the companies wholly, which is really interesting.
DJ: Yeah, I think that the first conversation with entrepreneurs is always a little bit unique, because they’re used to the B-C conversation of 25 percent and a bunch of money and six months runway. We just think, especially in the XR space, that’s a really, really challenging road.
Alan: It’s a really hard bet. There’s gonna be… well, there already is an entire graveyard of dead startups in this space.
DJ: That’s right. Here, it’s just different. But also, it may not be right for every entrepreneur. I think the glue that holds everything together here is everyone here sees the value in the entire organization, and it truly is a team environment. That’s one of the most striking things. You get out of the elevator, and come into our midtown office; we have 45 folks right now. And anybody that comes into the organization — into the office — just feels the excitement and the camaraderie. And the fact is, if one person succeeds, the whole organization succeeds. So it’s a really unique and wonderful place to come to work.
Alan: It sounds very exciting. Let’s unpack each of the startups that you’ve invested in, because I think that’s really exciting. Most of these are B2B startups, meaning that they sell their products and services. Some are in stealth. Some are actually selling. So maybe… do you want to start at the top, and walk us through the startups that you’ve invested in?
DJ: Sure. Sure. So, that’s right. We are completely enterprise-focused. No consumer things. We did have one consumer gaming entity out of the gate, but it became clear that consuming gaming was just going to take a while in the VR space. We were basically placing bets in all of the major verticals that we see big opportunities.
So I’ll just run through quickly, and then we can dive deeper into each one of them.
Adept Reality is an enterprise training platform, which I know you’re super excited about. Really, I think in the immediate, that one has a really great ROI conversation, and they’re having great traction. In It is all focused on marketing and branding applications. D6 is all focused on data visualization. They are working with big financial organizations, as well as universities on the education side. Kobach is one of the premiere augmented reality modeling companies, initially focused on the food industry. They’re super well-known, and their models are just, I think, second-to-none in the space. Immersive Health Group is focused on clinical training. So, another training division, but more focused on the health world. KreatAR, it’s basically founded on the principle of the user being able to create their own AR content. They have a new product called Post Reality, where user can upload a JPEG poster and then drag and drop augmented reality content on top.
Alan: What did you call it?
DJ: Post Reality.
Alan: Cool.
DJ: Number9 has several products. It’s focused on broadcasting; so live, photographic capture into AR and VR experiences. They have a couple of products that blend multi-user VR space with 180 captured VR photographic content. That product is called Project Chimera, and that one, we’re focused on scholastic application. So, we’re doing a bunch of work with universities, and you can basically have a professor give a class or lecture to a real classroom, as well as a virtual classroom. I’ll send you the link to show what that looks like. Early Adopter is focused on education. Foretell is building a platform, almost like a white-label solution for Social VR experiences. The MotionZone is a platform for sports data visualization, and they basically have an engine that can be used for fan engagement, as well as the training of athletes.
Alan: So it sounds like you guys are really broadly looking at… the only, I guess, thing that brings these technologies together is the fact that they’re using virtual/augmented/mixed realities, or XR technologies. Other than that, they’re in every different industry: it’s in marketing, training, data visualization, food industry, clinical… it’s crazy. How do you guys manage the fact that you’re in so many different verticals?
DJ: Yeah, it is a little bit of a trick, but the glue for that is that they are led by a general manager, and they’re their own startup, and they’re responsible for what they need to do. The parent organization, we have a couple of C-levels — myself included — and we meet with the general managers every two weeks in a strategic meeting. That keeps us informed and gives us the ability, as the executive team, to provide recommendations. But the general manager ultimately is in control of their entity, and the executive team at the end of day almost works like a company board. We have revenue coming in and we have whatever we’ve raised, and we just have to decide how the cash gets distributed, the resources get distributed.
We meet as a whole entity on a six-month basis. In that town hall, everyone presents what they’ve done and what their goals are for the next six months, so everyone is kind of aware, and the people that are getting a little bit more traction, perhaps, get a little bit more of the resources. And if a company is struggling, then we can step in and help them figure out a direction that maybe is a little bit better-suited. I think that that’s a big advantage of our organization, is the flexibility in the business model.
Alan: I really love it. I think it’s amazing, because one of the things that I keep seeing over and over again is that VCs will write huge checks, and these startups are just kind of like, “here’s a million dollars. Good luck!”
DJ: Yeah
Alan: And a lot of times, there are younger people, and they’ve never run a company before, and they’re just technology people. I love the idea that you guys are kind of wrapping professional management around them, as well. Also, that kind of community — if one of your startups is in, they don’t compete — which is really interesting, because if one of your startups says, “hey, we’re having a problem with this type of technology, does anybody else know how to fix this?” Do you find that they’re sharing technology between the companies? Or… how does that work?
DJ: So… the workplace really is very collaborative, even down to when we’re finding new startups. We have specific verticals that we’re excited about, but we also know that it’s great to have all different skill sets within the organization. So when an entrepreneur comes on, and — little examples; we have one entity that are masters of photogrammetry; we have another entity that’s focusing on WebAR and WebVR; we have another entity that’s focusing on AI; or social. What ends up happening is, when you have 45 people under one roof, with their own expertise, it makes a very, very powerful organization. And many, many, many of our projects are collaborative efforts between organizations.
For example, our social VR white label solution is currently partnered with our data visualization company, our health organization, our training organization, and basically giving their social solution to them, which is a way to bring revenue into multiple units of the organization from any one given client.
Alan: I really love that. It’s amazing. So basically you have one client, you say, “hey! Guess what? We have this amazing solution for you, and by the way, we’ve got six other ones.”
DJ: That’s right. That’s right. And it’s very common that we’ll go into an organization based on an expected interest in one entity, and then through conversation, they bring in entity 2, 3, and 4 for other things. So it’s really been a wonderful, collaborative, and great ability that — frankly — our competitors just don’t have, because they don’t have the scale.
Alan: Yeah, it’s amazing. You’ve done what we’re attempting with XR Ignite, to just bring the community together. Because there’s so much amazing talent, but it’s been, as you know a very small, one-, two-, three-, four-person teams. I love it.
DJ: That’s right.
Alan: So let’s dig into each one of these and maybe we’ll do a one-minute you know intro of each of the companies, because we got… yeah, let’s do it. Adept Reality, training platform. Tell me all about it.
DJ: Sure. So it — and actually, for anybody listening: if you go to TheGlimpseGroup.com, there’ll be a link to each one of our entities, and each one of the entities are in a different stage of their life – so, Adept actually is one of the newer organizations, and they’re building a training platform. They have several existing customers that are prototyping, and we do a lot of engagements. The first one is a first proof of concept, and then it graduates from there. We would try to get a success with a small engagement, and then that is the building block for the round two and round three, which, fortunately, we’ve been successful with, with several of our clients. The training platform is highly customizable, and it is designed to basically do anything from photographic type of content to computer graphic type of content, and it’s hardware-agnostic. So we tend to pick the right hardware for the job.
Alan: Interesting. What type of industries, or… who would use it?
DJ: We’re working with some pharmaceuticals. It’s actually really pretty varied. I think the good thing about training is that it can be done for any organization. So right now, it’s varied.
Alan: Got it. So if people are interested in enterprise training: Adept Reality.
DJ: That’s right.
Alan: So, “Inuit…”
DJ: Actually, it’s “In It.”
Alan: Oh, “In It.” How do you spell it?
DJ: Just “In It.”
Alan: Oh, got it, okay.
DJ: Yeah.
Alan: Perfect. In It:
DJ: So they are basically producing brand activations, and it ranges from AR stuff to VR stuff, to WebAR stuff. Really, any agency or brand needs an activation. We can come in and create it for them.
Alan: So what are some of the activations you’ve done with that company?
DJ: They’ve done some wine labels, AR work, some retail work, some VR activations for trade shows; again, very varied.
Alan: Very cool. I love it. So, let’s move on to the next one: we’ve got D6 VR data visualization.
DJ: Yep. So, D6 has been building a platform for data visualization for some time. It’s really, really cool. Started for tethered solutions, and we’re now moving towards more of the mobile solutions. They have done work with a bunch of the big financials; I would say more on the proof of concept/experimental side of things. I think it’s going to take some time before analysts are in a VR headset for a significant amount of time during the day. But what we found a bunch of success with is using it for storytelling. So, for these organizations, to be able to tell their story in a much more profound way. And then most recently, we’ve been working with universities who love the tool in order to bring data visualization — or immersive data visualization — to their students.
Alan: It’s amazing, I’ve noticed a lot of universities are now jumping on the VR bandwagon. They saw some early successes and wins, with things like YouVisit, where you can see a 360 tour of the campus, and I think that — having those early wins — really unlocked the potential to use virtual and augmented reality for universities; it seems to be their voracious appetite for this right now.
DJ: Yeah, that’s right. We’re in active conversations with a bunch of universities, which has really been great. And their interest is across several of our entities, data visualization just being one of them.
Alan: Incredible… which leads us to Kobach. The first time I saw the Kobach guys, they showed me a hamburger in augmented reality, that looked as real as a real hamburger. I couldn’t tell. I was like, “what the heck?” I had to look behind the phone to see, is it real?” So, it was awesome.
DJ: [laughs] They are masters, and now they’re actually moving into other verticals outside of the food industry, just because they’ve gotten so much traction. They’re amazing artists, and optimization is a key component of that. So, being able to do photogrammetry skin is… it’s pretty straightforward, but getting it to a size that can actually be distributed easily? There’s a little bit of special sauce in that.
Alan: People don’t realize that it’s easy to take a thousand pictures of a product and turn out a beautiful 3D model. It’s another thing to get it into a reasonable form factor or size that will run on a mobile phone, because you end up with this 200-300 megabyte file, which is beautiful, but kind of useless.
DJ: That’s right. That’s right. And to do it quickly.
Alan: Yeah, that’s the key.
DJ: A lot of their focus now is creating that pipeline, to be able to pump them out quickly.
Alan: Amazing. What about…what’re we on now… Immersive Health Group?
DJ: So the Immersive Health Group is really seeing wonderful traction within the healthcare space. They’re focused on clinical training. Right now, there’s a disconnect between how many nurses and clinicians there are available, versus how many patients there are. We have a population that’s getting older. So the problem is going to get worse, and VR is just a clear better [option] – economically, scalability-wise, and effectiveness — in training clinicians. So we’re really excited about that entity. And they’re currently working with several organizations on building that platform.
Alan: It’s incredible. I think that one there is… I mean, I’ve seen some really cool things; Precision OS is virtual reality training for operations, for knee surgeries and stuff. And I got to try it at AWE this year. It was amazing. I’m not a surgeon, so I don’t know how accurate it was, but it felt like… the way they did the haptic feedback in the controllers. And the thing is, with Oculus Quest now, they didn’t have a computer. They just literally pulled a headset out of their backpacks and said, “here, try this.” I got to try it and I was in a surgical suite, drilling into a knee. It was amazing.
DJ: I think the Oculus Quest is such an enabler for our entire organization. We had done many, many proof of concept types of VR activations, but dealing with tethered solutions, and PCs, and lighthouses, and Steam.
Alan: Oh my God.
DJ: Organizations, they don’t want to mess with that.
Alan: I don’t want to mess around with it! The other night, we set up the whole VIVE and we’re like, “yeah we’re gonna get into VR!” And an hour later, I’m still doing Steam updates and messing around with it, trying to get it to work. And it was just ridiculous. So yeah, I agree: we need to be able to pick it up, do what we need to do, put it down, get back to work.
DJ: Yeah, yeah. We’re really excited about the Quest. I mean, the optimization required to do it is a little bit of a trick.
Alan: [scoffs] It sure is.
DJ: And it’s early days of the Quest. We’re working through those things, but it’s clear that that is an amazing device, and really allows any enterprise application to really scale. And I’m a big fan of all of the untethered solutions that have gotten us here, but having true six degree of freedom — head and hand — is a game changer in my mind. We’re really excited about it.
Alan: Incredible. That leads us to KreatAR.
DJ: Yeah. So again, they have a new product called Post Reality, definitely check it out. It’s really easy and effective to create your own augmented reality posters. They have, also, more traction with the universities. One of the use cases is for PhD student research poster events. So, pretty common; you walk down a hallway and you’re surrounded by these flat posters that don’t have any engagement. We have several contracts with universities that have licenses, and their students are able to make their own augmented reality research posters.
Alan: That’s incredible, is it app-based?
DJ: That’s right. It’s a web app. You upload your JPEG. You can bring any of your digital assets – slideshows, videos — just drag and drop them onto it, and then download the app to your mobile device. At that event, just pop it open and you get to see all of this additional engagement. We’ve even built in a link for emailing, so there’s also a level of communication that you didn’t have before.
Alan: Incredible. So, Number9.
DJ: Sure. So, Number9 is still a little bit in stealth mode here, but Number9 is really focused on broadcasting, video capture, and then being able to combine that with virtual spaces. So like I said, I’ll send you a quick video showing their current product that — again — they’re working with some universities on. It’s basically a virtual classroom.
Alan: Very exciting; I’m actually really excited about that. I just so everybody who’s listening knows, we’ll put all of these links into the show notes at XrforBusiness.io.
So, talk to me about Early Adopter.
DJ: Sure. Early Adopter is focused on education. They actually have a great partnership with Montefiore, and they’re working with the kids’ hospitals. They’re building a social world for kids with cancer, for when they’re getting infusion. That’s a partnership with our social entity Foretell, which is amazing. They have an augmented reality timeline that goes into a classroom, and they’re partnered with another entity called VR Quest. That’s a great platform, where kids can basically build virtual worlds — almost like Minecraft — on a standard PC, and then it gets exported to a headset. It’s a way of basically dealing with the fact that schools nowadays wouldn’t have 30 headsets lying around. But they can afford to get one headset, and still have the kids creating the world, and the assets are all aligned with core curriculum. They build their world, and then they can export it, and then walk around in that world. It’s a really, really wonderful concept.
Alan: Incredible, oh man. Anything that can make kids’ stay at a hospital better, or make their lives better in general, I’m all in.
DJ: Then Foretell is our white label social solution. They partnered with a bunch of our internal entities. They’re kicking off a new project, which is basically support groups in VR, which I also think is wonderful.
Alan: Oh my God, what a great idea.
DJ: Yeah. So you have basically – again — either kids with cancer, or perhaps it’s folks that are afraid to leave their home. You can almost cherrypick the use cases, but still get a ton of value of being around other people, and being able to share their emotions. And this one, I really like this concept, because in many of our social VR use case discussions the avatar is a huge problem right. You’re talking to an enterprise business, and they want to do productivity in VR, but they don’t want to look like a character, or…
Alan: [laughs] You want to be doing business with a robot and a Minecraft character; you want to do business with your colleague.
DJ: That’s right. That’s right. And the avatars are improving quickly, but we’re not quite there yet. But for a support group, that level of being a little anonymous, I think it’s actually a really good pro for the use case. Again, now that Quest is here, it makes it much more viable. Right? It’s can fit in a shoe box and get mailed to each person, and they’re gonna be comfortable, because it’s full six degrees of freedom.
Alan: I love it. Things like AA, you have to go to a meeting and this. It’s disruptive in people’s lives, and I think we’re moving more towards being able to have these experiences at home. So, I think it’s wonderful.
DJ: Yeah, I think it is the eventual path of telehealth. We’re just the next level in it. And there’s this sense of presence that you get in VR, and I talk about this a bunch. It’s just way more impactful than Skyping; being able to feel as if you’re with somebody is way more powerful than just seeing their picture on your screen.
Alan: I agree, I can’t wait for when the next generation of Quest comes out, when it has eye tracking, and you can actually really look at somebody. When the avatars are… they don’t need to be photo-real, but they need to represent you and who you are, and then being able to look at somebody, see their body movements… I think we’re still early days with all of this stuff, but social VR is so powerful, and people don’t realize it. Everybody was all, “VR is an isolating experience,” but it really isn’t. When you’re in a space with other people, it’s magical, actually.
DJ: That’s right. The times that I’ve been in VR the longest is always when I’m in a social setting; time goes away and you’re living your life, and you’re just in a virtual world. It truly is the killer app.
Alan: Amazing. So let’s talk about MotionZone.
DJ: Sure. MotionZone, they’re basically building their platform on WebAR and WebVR, which I think will be huge. It’s tough these days, because it is so early. But using machine vision to track players on a court, grabbing that data, and then putting it into an AR and VR experience so that, either for fan engagement or for training purposes, people can understand what’s happening on a court. So, that one is a little bit more in the early stages. But we haven’t actually even decided whether to go with the fan engagement or the training. We’ve gotten a little bit more traction on the training side, but we’re exploring where that’ll go once the full platform is built.
Alan: Very exciting. Well that’s…man. There’s a lot to unpack there. You guys have raised capital, purchased — or, I guess acquired — 10 companies in 10 different verticals, doing 10 different things, put them in a room and said, “we’re all going to work together and be successful.”
DJ: That’s the idea. Our goal eventually is to expand geographically. We think that the model works really great, and there’s a ton of value having all of those folks in New York, but it doesn’t mean that there isn’t five or ten great startups in Boston or wherever. And we’ll probably expand geographically-close so that we can stay involved and keep a close eye on it as it grows. But in the long range, there might be Glimpse hubs all over the space, because we just think it’s a great mechanism for startups to grow and, eventually, succeed.
Alan: I love it. I think it’s a really wonderful business model, and I can’t wait to learn more about it. What’s your long-term plan? Is the plan to acquire as many startups as possible, and then go do an IPO? Or…?
DJ: We always said that we wanted to get to 10 in New York. We recently did that. Right now, we’re not looking to expand geographically; we want to basically stay focused on the 10 that we have in New York, and get to be cashflow-positive. And definitely, an IPO is a potential. But we want to do it when it’s the right time.
Alan: Oh, absolutely right. My guess is it’s going to be 24 months from now. I’ll put my… between 24 to 36 months — there’s my bet.
DJ: I don’t want to bet against you, because I would put it probably at that point, too.
Alan: I think it’s interesting, because people don’t realize that it took 10 years for our industry to get to 10 billion dollars. 10 years! And it’s going to be three to get to 100.
DJ: Yeah.
Alan: So the timing is now, and I think you guys you’ve got a good head start on the industry, so congratulations.
DJ: Yeah. Thank you, thank you. We’re just really excited, and we’re appreciative to the big organizations. AR is just becoming more and more accessible with Quickview and ARKit. It’s just, all of the hurdles from consuming this content are just pulling away. And then on the VR side too, with products like the Quest. It’s really exciting to see, and we think like you: in the next two years, it’s gonna be a big, big jump.
Alan: I agree. Well, again, thank you so much for joining me, DJ. It’s been it’s always a pleasure hanging out with you, and you know getting to learn this. And now, we can share our conversations with the world. I’ve been really looking forward to this podcast. Thank you so much.
DJ: Absolutely! It’s a pleasure being on board for this. I’m sure that we’ll stay in touch.
41 min
July 15, 2019 Get a Glimpse of a New Way to Accelerate Startups with DJ Smith
DJ Smith is the co-founder of The Glimpse Group, an XR technologies accelerator that's doing startup incubating in a whole new way. Listen as Smith explains to Alan how Glimpse works, and some of the tech companies that are already beginning to grow under its umbrella.
Alan: We have an amazing guest today: DJ Smith is the co-founder and chief creative officer at The Glimpse Group. The Glimpse Group is a holding company for a portfolio of 10 startups, focused on virtual and augmented reality industries. At Glimpse, DJ's responsibilities include overseeing the production of all the VR and AR content, as well as leading efforts to locate new subsidiary companies. In addition, DJ is the organizer of the New York virtual reality meetup. NYVR hosts monthly events focused on virtual reality technology and the premiere venue for the industry networking collaboration within New York City. NYVR is the second-largest virtual meetup in the world, with over 6,000 members. Prior to entering the VR/AR industry, DJ worked 20 years in the real estate and construction industries.
The Glimpse Group is a company designed with the specific purpose of cultivating entrepreneurs in VR, AR, and of course, XR. The business model simplifies many of the challenges faced by entrepreneurs, while simultaneously providing investors with an opportunity to invest directly into the VR/AR space. The Glimpse Group will fund, cultivate, and manage business operations while providing a strong network of professional relationships. Being part of The Glimpse Group allows entrepreneurs to maximize their time and resources in pursuit of their mission-critical endeavors. They've invested in 10 companies, which we'll get into this show, but the 10 companies are Adept Reality, In It, D6 VR, Kobach, Immersive Health Group, KreatAR, Number9, Early Adopter, MotionZone, Foretell Studios, and I'm sure there's gonna be a lot more on the show. To learn more about The Glimpse Group, visit TheGlimpseGroup.com.
Welcome to the show, DJ.
DJ: Thank you very much for the amazing introduction.
Alan: It's my absolute pleasure. I've been waiting for this call for so long, I'm really excited to have you on the show. Tell us, how did you get into virtual/ augmented/mixed reality, or XR technologies?
DJ: Sure. It's a life calling that I was seeking for many, many years. In 2012, I saw the Palmer Luckey Kickstarter video, and I was like, “it has finally arrived!” So from that point on -- it's kind of a pivotal moment in time for me and my life – I started buying all the toys; really, just absorbing as much tech as I possibly could.
The meet up was formed shortly after that. I think the meet up started with 10 people in a dusty old conference room, and it just has steadily grown and grown and grown, and as it grew, my involvement in the industry grew and grew and grew. So we're now the, I believe, the second-largest VR meetup in the world. It basically put me in direct connection with many of the developers in the city. The big entities; the Googles, Microsofts, where we would hold our event, as well as the whole investment community.
Then, about four years ago, the Oculus Rift and the VIVE was coming out, and I saw that there was an opportunity to actually start making a living out of it. I put in my notice in my construction job, and a week later I met my current business partner, Lyron Bentovim, who saw the same timing and opportunity within the space. He came from the investment side of things; had his own tech startup, ran hedge funds, and was most recently involved in publicly-traded companies. He had the idea of the Glimpse business model, which was taking bits and pieces from his experience of hedging bets and the challenges with a typical VC angel startup scenario, but was looking for somebody like myself that was dee
0 min
July 12, 2019 Put the Car of Your Dreams in Your Living Room, with ZeroLight’s Barry Hoffman
If picking out a car off a lot is like picking out a handful of eggs from a basket, then assembling your perfect vehicle – fine-tuned to your specifications, BY you – is like picking out a few hundred grains of sand from the Sahara desert. At least, that’s how ZeroLight’s Barry Hoffman sees it. Hoffman shares this with Alan, and other philosophies about XR as a great asset to the automotive industry.
Alan: Today’s guest is Barry
Hoffman from ZeroLight. Barry is the chief strategy officer of
ZeroLight, a leading real-time visualization company in the
automotive industry. He has a background with telco, gaming,
automotive, and data science, and interactions with CRMs being the
major thread in his career. At ZeroLight, Barry is responsible for
their US operations, and also leads strategic partnerships and
ZeroLight. This company, just so you know, is really incredible.
They’ve taken virtual and augmented reality for the automobile
industry to the next level; from AR apps where you can see cars in
your living room, to full VR simulators where you can drive the cars
and see what they interact like. It’s an incredible company, I
suggest you check it out at zerolight.com.
Barry, welcome to the show.
Barry: Nice to be here, Alan,
thank you for inviting me.
Alan: It’s my absolute pleasure.
I’ve been a huge fan of your work for a long time. You guys have done
one car after another — I think one of the ones that I saw on there
was a Pagani. It’s just, I’m a carhead as well, so being able to see
the work that you guys are doing and making things look photo-real is
just incredible.
Barry: Yeah, that’s true. You
mentioned . It’s funny because it’s quite a known car, of course —
they only make 100 of series, like how the Pagani Huayra Roadster, I
believe there were only 100 made. But the funny thing is that all
those 100 were sold digitally first. So, there is no real car
available. If you think about the starting price of $2.3-million,
it’s sort of like a digital-reality sales case of $230-million. It’s
something, if you do the math, how incredible that is.
Alan: So what you’re saying is,
ZeroLight contributed to probably the largest use of VR for an
economic benefit ever.
Barry: Yeah, that’s true. This
one is, of course, flipped to VR and especially screens, because a
lot of the clientele will want to use it on screens. I would say
Pagani is definitely a case like that. Audi, we definitely
contributed to that part. Most recently, we released a Cadillac in
their showrooms as well, with VR.
All these different car manufacturers,
they tell their story differently. They have different brands, and
they use the technology differently. That’s the coolest pitch, and I
like instead of just saying, “okay, there’s one type of showcase,
and this is how you do VR.” That would be the same as, “there’s
one type of app in the App Store, and that is all you can do.” It’s
cool to see this diversity, these ideas coming out of all these
different clients, and then working together with them and turning
that into their story. And not just their story — because that would
make it only a brand experience — but also a buying experience,
because a lot of things of what we do is on the high-end
personalization side, like what you just said. Pagani has 18,000
parts, and all those 18,000 parts can be changed into something
unique. That’s the ultimate buying experience, I almost say.
Audi, for instance, if you take their
custom build program, I believe there’s more Audis in variants
available than there are grains of sand in the world or in the
Sahara. That’s the small thing that I have to say. All these things,
if you only do traditional photo shoots, or even traditional CGI, you
can only show a limited set of those variants. What we did is, at one
point, we took their… let’s call it their engineering bill of
materials — in simple words, that’s basically the models coming out
of their engineering cycle — those were turned into a marketing bill
of materials. So, more marketing-ready CG models. We turned this into
a library that can be picked up by any channel, in real-time, and
turned into any variant that is available in that region (because of
course not every region has the same kind of colors available, the
same kind of trim levels, lights that kind of stuff).
In the end, it’s not just that one
showroom. It’s being able to do that worldwide; being able to deploy
it, being able to update it. It’s really cool.
Alan: Tell
me, how did ZeroLight get started? I always wanted to know: how do
you go from being a computer graphics company to now, you’re
designing the new Pagani, and you can choose any color in the rainbow
– plus, changing the spark plugs if you want?
Barry: It started out of a
project which we did in collaboration with IBM and Jaguar Land Rover
in 2012 when Jaguar Land Rover was rebranding their whole setup. They
were just bought by Tata and they came out with the new F-Type and
the new Evoque, and they wanted to have something that the world had
never seen. So they said, “okay, can you bring the car on the
bigger screen possible? Then we want to do stuff with Kinect and all
that stuff.” Which was, at that time, the coolest stuff. We come
from the gaming industry, originally; the company beforehand was like
26 years old, in various names, involved in probably 70-80 percent of
all the car racing games ever built. We had a lot of legacy in the
car world, and of working with interactive technology, with all the
different console and PC cycles, always having to take the bleeding
edge out of what was available.
So, when IBM came, they went to some of
the larger games companies, who could not do that, because they were
tied up to their mother companies. But those companies pointed to us,
because we had worked with them in the past, and they said, “can
you work with us? We thought it was a cool project. Plus, when we
were doing racing games, we always thought, “can we actually
sell cars from racing games?” That was the original thought —
can you sell cars from there? But in the end, it turned out it’s the
other way. It’s actually, “can you sell cars by using racing
game technology?” That’s almost how it turned out to be. In the
end, what you see is that people go into a racing game to be
entertained; people go into a showroom to buy a car. Combining that
technology-wise is possible, but narrative-wise is difficult. Also, a
lot of people always ask me, “can you have a driving car in VR?”
And I say, yeah, you can have a driving car in VR, because we do
those things. We have drive-in experiences, we have drive-out
experiences; where we show the movement of the car, but then they
say, “can have the steering wheel? Here’s a Pagani Roadster or a
Lamborghini — I want to drive that car.” I said, “have you
ever played a real simulation race game?” And they say yeah, and
I said, “do you remember the first corner where you have to go
through on the track? What happens with you?” “I end up in
the gravel.” I said, “well, that’s what’s happening with
you in the shop as well. And do you know what happens in the shop if
a buyer ends up in the gravel?” “No.” I said, “they
walk out of the shop — they’re ashamed of their behavior, and they
never come back. If you know that there are less than two people per
day coming into a retail shop to actually buy a car, you have to take
it a little bit more step-by-step approach.”
Slowly, we’ve been — on top of the
graphical quality — we’ve been bringing in those moving and driving
experiences, but always by keeping in mind, “does it contribute to
the buying experience of the customer? Does it contribute to the
selling experience of the client? Is there a good engagement? Is
there a good conversion? Is there a high book of value coming out of
that?” With every part of the channel, whether it’s with the media
creation we do — which is not extended reality, AR/VR — whether
it’s advertising, whether it’s web, or whether it’s in the
dealership; in the end, we’ll get those components.
Sorry, I’m going a bit left and right.
Alan: No,
it’s wonderful! It’s really interesting; in the automotive
world, you have very discerning clients. They have big
budgets, they know what they want, and it has to be right. The first
or second interview I did for this podcast was actually Elizabeth
Baron from Ford, and Elizabeth is a pioneer in this industry. She was
at Ford for 30 years, and ran their immersive technology lab for 20
of them, I think. She was explaining that they’re using it for
design. Their designers are putting the 3D models in AR, walking
around them, and changing the lighting. And now, Ford is using those
VR experiences to show every executive the car before they’re built.
So, every executive has to approve the car in VR before it goes to
production. You’ve got people using it for design. You’ve got people
using it for sales. Then, you’ve got people at home racing them,
driving them into the cabbage.
Barry: Heh, yeah — exactly.
Alan: Let me ask you a question.
What is your favorite racing simulator?
Barry: I think, still, Forza.
I’m playing that at the moment, on my XBox. That’s probably my most
favorite, because it’s not just a simulator with tracks; it’s much
more a narrative, as well. I kind of like that. I like stories;
that’s the red thread. Next to data, I like storytelling. I like
listening for stories, I like reading stories, I like viewing
stories.
Alan: Iagree, and I think that’s one of the things that’s missing from
current VR. I’m not saying all of it, but a lot of people — up until
this point; up until maybe a year ago — were just trying to make it
work. And it works; okay, great. Can we make it not make people sick?
Okay, we got there. Can we make it so that it doesn’t cost us an arm
and a leg? Okay, we’re good. Now you’re finally starting to see that
point where the technology exists; we know it works, we know how to
make it — let’s start telling proper stories again.
Barry: Like you say, if making
it work takes up like 70-80 percent of your time and your budget,
there’s no more time for a good narrative. Even though, if they had a
good plan with a good narrative, in the end, the execution does not
lead to it. At the moment, there’s a couple of blockers there. You
still see it on the hardware, software, and interaction side.
On the hardware side, if you set up a
high-end VR experience, it would have a freaking bloody big PC. It
has those cables attached to it and all that stuff. So, cutting the
machine; cutting the cord? That’s one of my missions in life, and
that’s why we started working with Qualcomm together, as well. A sort
of boundless XR thing with Hugo Swart.
Alan: I just posted about Hugo’s
talk at AWE. Actually, I’ve been bugging him to come on the podcast.
Hugo, if you’re listening. You’ve got to come on my podcast!
Barry: Send
him an e-mail. That’s cool.
Alan: I have! That’s the problem
now. It’s probably buried in a thousand others.
Barry: Yeah, that’s true. I had
the pleasure of meeting him and working with his team. We worked
towards the next Dallas event that… I’m trying to think, what is
it? BrainXchange?
Alan: Oh yeah! Okay.
Barry: We
will come there, and we will show some really interesting stuff
there. Cannot tell too much about it, but if you look at the last
press release that Qualcomm did, where our name was mentioned, it
gives you a little bit of a hint.
On the software side, that’s another
thing. Up until even maybe a half a year ago to a year ago, a lot of
companies could not even get up to the level of 90 frames per second.
So we had all those sick there, that kind of stuff in VR. Thankfully,
a lot of that stuff starts to get done well. Some of the guys from
the engine companies, they’ve been sorting it out, those layers. So
that’s good. But in the end, it’s also about the quality. If you do
VR or AR, the quality needs to match the world around you. It cannot
just be uncanny valley, like digital people in there, or simple
holograms — that kind of stuff. That is great to see those
experimentations going on, but for the wider audience, they don’t see
it. They think, “what am I looking at?” That’s where it really
needs to push.
I was, this weekend, at Shape from
AT&T.
Alan: Yeah? How was it?
Barry: It was great. They had
some really interesting examples there. HTC did something with
Batman; that Game of Thrones thing; some of the demos that Magic Leap
had really. But if you take on an overall business, it was more still
a technology show. Then it would have been “okay, this is the
future of Hollywood” for me.
I think we’re still in the phase of
artistry. When you learn how to appreciate art — especially modern
art — you look through your eyelashes and you try to imagine what
you see. That’s sometimes still a little bit of the work of VR and AR
I have to look at, versus it’s immediately clear what you’re trying
to do with it; the utility, the immersion. What you just said — when
we started out a couple of years ago, you had incredible productions
(and you still have, like the productions from Chris Milk and his
team).
Alan: Can
I tell you a quick story?
Barry: Oh, yeah. Please!
Alan: The first time I ever
tried VR was that Curiosity Camp; that takes place in the Santa Cruz
Mountains, and it’s put on by Innovation Endeavors, which is Eric
Schmidt’s investment fund. I got invited to this DJ at it with my
emulator board, and it was Robert Scoble and I; we both went into the
little tent, and there was this guy showing these VR demos. The big
huge was the DK1 with big headphones. I put it on, and it was Chris
Milk who put the VR headset on my head. It was an experience where I
was onstage, standing next to Beck in a concert, and I just had this
“holy crap” moment. It was just amazing. It was from that moment
on that I knew that virtual and augmented reality is the future of
human communications, and I wanted to be part of it.
Five years ago, I said I’m going to be
one of the experts in the world in VR. I tried to aim for that,
anyway.
Barry: No that’s true. But you
felt the presence. You felt presence, you know? You were there,
you’ve got the goosebumps on your arm; that’s exactly the same thing.
I just wonder, how does the industry as a whole — and maybe that’s
idealistically talking — how does the industry as a whole make
certain that every first entrance into…well, let’s call it in this
case VR, gets that presence moment? Otherwise, it’s just a “wow”
thing.
Alan: Here’s
the issue: 360 cameras cost $200, and somebody can go film there
their holiday vacation, make it all unstable, throw in a VR headset,
and start showing people. And people will go, “that’s VR? Oh, that
sucks.”
Barry: Yeah. It’s maybe elitist
to think this way, but maybe this is the push. I was just thinking
about it this morning, and when you talked about this DK1, I had the
same thing with DK1; we had a racing game running at that time, and
we got a DK1 for ZeroLight. I said, “I want one for that racing
game, as well.” We went to China, we had ChinaJoy, and I said,
“let’s do one side of the thing with the DK1.” That was the
first time — at least on that show – that anybody had shown VR. I
thought, “that’s a nice marketing tool.” And these guys go
in, and everybody comes out of it, and their hair was standing
straight up. They’d say, “it’s amazing!” You saw them
almost dropping down because they were so wobbly because of the
experience. Everything was amazing, and because it was a racing game,
you were tied into this chair, you see your car around you, so you
never had that feeling you were walking. It sort of works from the
get-go. It was funny to see that.
But from that moment, up until today,
in the streets? I would have expected it to go faster. And do you
know what happened? The one thing that created that image now, in the
streets, that I would have expected like XR to do; it’s those air
pods from Apple. You see them now everywhere in the street. That was
a revolution happening in a couple of years — even one-and-a-half
years — and that’s sort of what was missing, you know? A big company
like Apple, Google, really pushing the thing out and saying, “this
is what it’s going to be,” and then making it part of everyday
life.
Alan: You nailed it there,
because you mentioned two big companies. I’ll mention a bunch more:
Apple, Google, Amazon, Facebook, Qualcomm, Intel, Unity — you’re
looking at these massive companies now, and they’re all betting big
on VR and AR. Billions of dollars have gone into this market, and I
knew it was going to take 10 years. I kind of looked at the market —
I got into it in 2015, I said: “okay, it’s going to take 10
years.” I built my schedule around that, and I said, “it’s
going to start with enterprise. It’s not going to start with
consumers. Consumers will buy it, but it’s going to start an
enterprise.” And it’s exactly what we’re seeing.
Barry: It’s true, it’s true.
Just to come back to ZeroLight for a second: when we started doing
this, of course, we came from the racing game world. But I had so
many companies talking to me to say, “oh yeah, and what’s your
next industry?” I said, “well, to be honest, at the moment,
we want to focus on automotive, due to the fact I only have a limited
amount of people — we’re a 125-man company. We want to make certain
what we do is the best that you can find in that industry. And if
we’re going to all these other industries, there needs to be a case
for it. It needs to be able to… because, like you say, enterprise
was the first thing; B2C was far too early. And within enterprise,
you need to find enterprises that are willing to spend the budget on
it, and seeing the results coming out of it. Even though everybody
sees it for architects and other things, because that market — and
especially, the data for those markets — are so dispersed amongst
architects, engineers, building owners, it’s a completely different
market than when you have an OEM who controls, basically, the PLM
cycle — the product lifecycle management cycle — and the CRM cycle
— the customer relationship cycle.
Them actually owning both that part,
and having high-end data available, was for us the sweet spot to be
able to show that thing. That’s where you’re right, making that plan
for 2025, pushing for that thing. We keep razor-sharp on what we’re
doing. It’s the same with hardware partners. We’ve done a lot of
support for new hardware makers; some of them are unfortunately no
longer here. I think about Meta, I think about Star VR — incredible
products, but also–
Alan: ROFvr,
StarVR — yeah. There is a graveyard, for sure. ODG.
Barry: We
all supported them; Last year at SIGGRAPH, we did the first foveated
rendering with StarVR. But on the other hand, part of my strategic
relationships, of course, is sometimes making a calculated bet that
some of these partners will survive, and we are first together with
them. Because in the end, we’re a startup as well — we not only want
to go with big companies; we want to grow these markets together. So
we did that. On the other hand, sometimes you see it working. We did
stuff with HTC Vive at GTC — our first foveated rendering in the
world with them. We did human eye resolution with Varjo at GTC this
year.
Alan: I got to try the Varjo…
or is it “Varro?” Or “Var-JO?” Anyway…
Barry: I always say “Var-Jo,”
but probably, if you’re Finnish, you’re saying, “hey, Barry: shut
your Dutch mouth! Let’s talk in Finnish.” [both laugh]
Alan: Varjo is a is a new VR
headset; it just introduced a new one. They claim to have
near-human-eye resolution graphics. They’ve kind of created a fake
foveated rendering, where the center of your vision is really
super-high resolution, and the rest of the screen is regular. It
works, and it’s funny; unless somebody had told me that, I would not
really notice that.
Their new headset is called the XR-1,
and it’s got cameras on the outside facing, so you have a full
augmented reality experience. Meaning, you can see the world around
you. It felt amazing; it didn’t make me sick, it didn’t lag. I’ve
done a lot of these things — it was a company out of Toronto here
called… well, I can remember now. But they did a Passthrough camera
type of thing, and it was nauseating, to be honest.
Barry: We’ve been doing some
interior, internal stuff with the Varjo XR-1. The Passthrough cameras
especially, because if you look at the whole AR thing, and especially
the wearables, the headsets — think about Hololens and Magic Leap —
I see the utility factor of where they are. The whole training
aspect, doing it with Hololens or with Magic Leap — great, I love
it. Unfortunately, it’s not directly my market to be in utility,
although we’re working on some training utilities for automotive,
where we use high-end quality.
But if you look at what Passthrough can
do — what holographic, until now, cannot do — it cannot show black.
It cannot show real black. If you try to sell a car — a black car —
in that kind of stuff, it’s sort of muddy gray, or vague gray. That’s
the stuff we’ve always said, “until someone comes up with the
quality there, we believe — up until a certain moment — more a
Passthrough, where you see the real world, and then the digital
object is rendered on that. For me at the moment, that is still my
bet: Passthrough. Seeing that Apple, of course, at one point
bought… what was the company again? Vrvana? Which, for me, was the
first time I saw that Passthrough thing, when they showed it. Of
course, the model was not that high-end, but still; the experience
they built with that helicopter flying around, and then all of a
sudden, it was there? I thought that was amazing.
Alan: Somebody asked me why
Apple bought them. The simple answer is, they figured out occlusion
with single cameras. That was a big problem then, and they had
figured it out. There was a company… I can’t remember. ~tsk~
I’ll have to look it up. It was a company that did inside-out
tracking, before that was a thing. They raised, I think, $5-million,
and I said, “look, if you guys are going to make a go of this, you
should either sell your company now to somebody bigger, or you’re
gonna need raise $50-million and make a go of it. But I think they
went out of business.
Barry: It’s one of the things
you talk about, of course. The moment you start a hardware company,
raising those double-digit numbers is a necessity.
Alan: At
Meta,I said, “if you’re gonna go at this, you need
half a billion dollars; you need $500-million.
Barry: Yeah that’s true.
Alan: Hardware is hard.
Barry: Yeah, that’s true. I
don’t want to go too deep there, Alan, because I know too many people
there, and I think everybody made a great effort, but…
Alan: Oh yeah. Meta was a
wonderful product. I had one of the very first demos. It was
incredible.
Barry: But also a little bit,
the problem is, the moment you get investor money in, you need to
start showing results. Because they want to see results. For
instance, with Magic Leap; Hololens comes out with Hololens 2, they
need to show out some stuff as well. Even though you know — if
you’re truly honest, Alan — it’s still a little bit away from what
we truly want to see from it. On the other hand, they need to get
their developer community there. They need to get ideas in, of
course. So I get that.
Alan: It’s a different world we
live in. Can we just bring it back? Because I think if there are
people listening from the automotive sector, how can ZeroLight help
people in the automotive sector? Let’s maybe talk about some of the
use cases; you’ve built buying simulators, but more importantly,
configurations and stuff in full 3D. On web, in-store, in VR, in AR.
You’ve done all these creative things. What are the foundations of
how car companies can reach out and connect with ZeroLight, to really
bring this to fruition?
Barry: That’s a good one. Of
course, we can talk about this for days. If you look at their
traditional way of how — not even the traditional way of
buying cars, because we’re already beyond that; more than 90 percent
of people start their search online – still, most of the people go
to either a car lot or to a dealership and buy a car from there.
That’s current-days, how people buy cars.
Think about how a car company wants to
sell cars at the moment, especially in the US; they have a lot of
cars on the car lot, and they want to sell that from those lots. In
the end, you want to lead them to those inventories. The start of
that is the awareness cycle; basically, where people look, see
certain media coming by — 50 percent of the digital ad market is
probably owned by Facebook and Google at the moment; a couple other
platforms and now you’ve got the rest — the first part is being able
to produce high-end media assets that wow people, and get them to
actually go to the next stage, which is the explorer pages, and the
build-your-own pages on the OEM websites, or on some of the owned and
earned media – like, the social media pages and that kind of stuff.
If you think about media pages, what we
currently do is build a library for the car companies, which have all
their cars, all their variants, all their things which they can
produce for worldwide, and deliver that on the spot. In the case of
media, we deliver on-the-spot, personalized views, videos and
everything of those things that can be done through an automated
process that is called RenderChain. Or, they can do it themselves
with a tool called Spotlight. They can make images. They can make
videos. But also, VR components, like a 360 panoramic shot of a car
in a scene. That’s step one.
Once people are aware of that they
click on things, they go to a certain piece of online real estate,
which is either the OEM website, the dealer network websites, and
there, they make decisions off, “I’ve seen this. I want to make
choices,” and the choice could be, I only show what’s on the lot,
or I show every grain of sand in the Sahara available, like with Audi
in Europe, for instance. Then, they make choices there, and out of
those choices — all those choices need to be shown on the screen.
For instance, in our case with Audi, we’re rolling out to 31
countries — we’re beyond halfway there — where they do use a
real-time 3D technology to show the cars on the screen. So, instead
of the traditional images that car companies have of a car on a white
background, you can order your car looking at an immersive
environment. You can spin it 360 around, in real-time, open ooors,
open boot space — you can view that car. And by doing that — and
this is public knowledge — we’ve significantly increased conversion,
engagement, and booking value of the car already online, before going
into the dealership. That’s a great thing.
Alan: What kind of numbers are
we talking here?
Barry: I cannot go into direct
numbers, because that’s all Audi. But–
Alan: Not just with Audi, but in
general. Are we talking about a 5 percent lift? A 2 percent lift?
Barry: Well, let’s say one of
the things we can say; the engagement in one of the cases went up by
66 percent.
Alan: Holy crap!
Barry: Yeah. And a lot of times,
they are they’re not just 1 or 2 percent — they’re significant
numbers, very high-end numbers. Higher-end accessories being used.
And that’s only online. When you go online, this is the results we
had. And if you think about the fact, what what kind of XR components
do we see there? We started building some showcase pages, which we
released at CES this year, where if you go from the web page and you
sign up for things – basically, we send you an email, and out of
that email you can download your AR app, and actually see your car in
front of your door, in front of your house, but also looking at the
interior at a quality level that is the same as the level on the
website. And it’s all automated; it’s not handmade. The cars are all
coming out of the library, and we’ve created some technology where we
take that car model — which is normally very high end, I believe
it’s like 5 to 6 to 7-million polygons — and then we shrink them to
fit the headsets or the handsets. In the case of an ARKit setup — a
couple of hundred thousand polygons — you can look at the car, walk
through the car with your tablet or your phone, and actually feel
that you look at the real car. So that’s another accomplishment.
Alan: Barry,
can yousend me the link? I’m going to put it in the
show notes on XRforBusiness.io in the show notes, there’ll be the
link to the Android and iOS apps.
Barry: Yeah that’s cool, and
I’ll also send you a link to the white paper where all those results
were mentioned; we did a white paper with intel about the XR results,
so you can look that up as well.
But at the moment, what you see with VR
and AR, in the case of like the online components, it’s more
more-or-less like a breakout component. You know, “I want to see a
bit more of the car, so I now go to the 360. I want to see my car in
front of my house, so now I go into AR.” So that’s why I call the
breakout technology; it is not yet the core technology. It’s a
sideshow. It’s an interesting sideshow, and there, we are trying to
determine: is this sideshow adding to the conversion? But that’s
something we haven’t found yet.
Then the third part is, once you go to
the dealership and you want to make a decision: on the one hand, you
can say — and that’s a couple of years ago when I first landed in
the US, because I’ve been here now for three years — the first
discussions with dealers networks was, “we don’t use digital
technology; we sell from the lot.” The funny thing is, that is
true… except, a lot of cars are being sold already before they are
on the lot — they’re still on the truck. That’s a cool thing,
especially new models. That’s one use case. Another use case is, they
might be on the lot but you’re in the… well, I believe you’re
living in Toronto, isn’t it?
Alan: Yep.
Barry: So, are you going — in
the middle of winter — to a lot, and then look across the lot of 500
cars to think, “which car do I want to buy?” No, you don’t want
to go on the lot. You want to only go into that car, which is the end
thing. It’s the same in Arizona in summer. Those kind of things. So,
there’s a lot of use case. And the funny thing is, one of the most
traditional US brands, Cadillac. For me, that was amazing that they
started doing it, because that shows that even though the car
companies know, “we want to sell cars from the lot,” but they
recognize it’s not just to sell cars but it’s to upsell cars.
It’s to sell those accessories; it’s to sell those special wheels;
it’s to sell that bike rack, those boats racks. It is to create those
extra things where the salesperson doesn’t have time for. They’re not
gonna put a bike rack on top of the car to show you how it looks
like, and that’s a very good use case.
So I still think, at the moment, XR can
be part of the core process of the dealership. XR, at the moment, is
a breakout technology for BYO. XR, at the moment, is a breakout
technology for media, in awareness. But although, at the moment, in
Google — if you type in the words — you start to get those WebAR
things. I think that’s where it shows. It slowly turns into a core
technology, being used for media. Give it a couple more years and
it’s mainstream.
Alan: Absolutely.
Barry: Hopefully that helps to
bring it together. We’ve got like a whole range of tools to deliver
things in real-time, with as little use of manpower as possible. It
gives our customers everything in time to do things. And because
we’ve been able to do that all, of a sudden, tools or channels that
were traditionally not available to do configurators — for instance,
we just did a Facebook Kansas campaign with Audi, in the UK, and I
believe it’s going to run in Germany as well, and a range of other
countries — they could create 900 different content assets on the
fly for a very economic value: for, I believe, a factor of 50
[percent]
cheaper than normally, because it was all automated by our
tools. They had that library available. We didn’t need to import it
again. Because of that, they created those assets, and they created
— on the fly — a new app that was then used in a campaign.
Because you organize the world for them
— you organize their products for them in a way they can expose it
in new formats — allows for them to work in that. Because otherwise,
it’s always a new budget discussion. If you come up with an AR or VR
tool and say, “okay, now I need another proof of concept,” you
have to go through this whole budget range with every OEM. If you’re
right in the budget cycle, are you going to get that money, yes or
no? Then it still needs to prove a point, and they might be through
the budget, and you’re a year further.
If you already have that technology,
you have that library; every time there’s a new technology, we can
just show it to our customers, because we have their library
available. When ARKit came out, within a week, we showed them an AR
viewerr of their car, and we said, “ this is how it looks like —
how do you want to interact with it? What kind of story do you want
to tell?” Because of that, there was one or two or three that said,
“oh yeah, if it can do that, then it can actually solve this
problem,” or, “it can solve that problem.” Otherwise, it’s just
a PowerPoint show, and somebody has to visualize that problem and
come up with a solution, then say, “because I see the solution in
my mind, I’m now going to give you your budget.” So it helps to
shorten sales cycles on our side. It shortens missed projects on the
client side. So yeah, win-win.
Sorry — long answer.
Alan: No!It’s wonderful. I bet you people are gonna be making notes, and
then going back and re-listening to this, because the power of what
you guys have created — to be able to take even engineering assets
and convert them into marketing assets, and then scale those across a
whole buying decision tree for customers — you’re really making the
job for automotive sales much easier and streamlining it. I think
it’s a wonderful example of how XR is being used.
Barry: Yeah. I think the funny
thing is, like at CES, we worked with Audi and with AWS on their
booth, to show the buying journey of a car across those channels. You
can go to our website — or I’ll send you the link to that one —
where we used part of the thing, like using machine learning to help
the buying decisions. But the cool thing was, a lot of my clients
came by there — or potential clients, or new clients — and
they said, “so, you’re trying to show here a CRM cycle?” I said,
“yeah, that’s exactly what I’m doing.” “How would that work
with salesforce,” they said, and I thought, “oh yeah, of course
that’s what they want to do.” So what we focused on a couple of
months ago, we started — instead of showing a generic CRM cycle —
we started building it in leading tools; in the CMS, CRM, ERP-cycle,
in SAP, CRM and Salesforce. Where can all these visualizations live?
And then push at the right moment, the right kind of assets, out to
the right kind of person.
We’re in the middle of that you; will
see a lot more coming out this year. But having the visualization,
and then taking the data off all those visualizations, and working
together with the customers and saying, “if you have these special
kinds of data that you didn’t have before – angles, for instance —
how does a customer look at a certain angle with a certain timeframe?
That’s data nobody ever had before, and all of a sudden, because you
use a screen or an angle or a VR headset – or even if you go
deeper, with foveated render — you use where the pupil is looking.
What can you do that? How can we shorten cycles? How could we nudge
the user further? (Nudge is a thing I can do another session on that
whole thing.)
I think where we’re not we’re in a
really good position here, too.
Alan: I think you are. I just
clicked on the projects section of your website, and it goes: “Audi!
BMW! Lamborghini! Nissan! Pagani! Porche! Toyota! Volkswagen!”
Okay, so yeah, that leaves… uhhh… Ford!
Barry: I just signed up with a
very big Canadian company. Hopefully in the next three four weeks, I
can talk about that one — which is not cars! That’s the first time
I’m going to do something mass market without cars. We’ve done Air
Force/aerospace as well, but that was more on the POC side, and some
of the of like Executive Decision-ing tools. But this is the first
time going out there, and there’s a couple of more brands coming,
which we can talk about in the near future.
Alan: Amazing. We’ll have to do
another episode, not about cars.
Barry: AboutCanadian companies! [laughs]
Alan: [Laughs]
Pagani could give us a car for the day, and we’ll do an
episode from inside the Pagani.
Barry: The funny thing is,
Pagani doesn’t own the cars; they don’t own any car. It’s the
owner. So you need to find an owner of a Pagani, and then say, “hey,
can I own your Pagani, to show a Pagani?”
Alan: Honestly?
It would betoo loud anyway.
Barry: No worries. It’s a
beautiful car. You can close the doors, and it’s as luxurious as can
be.
42 min
July 12, 2019 Put the Car of Your Dreams in Your Living Room, with ZeroLight's Barry Hoffman
If picking out a car off a lot is like picking out a handful of eggs from a basket, then assembling your perfect vehicle – fine-tuned to your specifications, BY you – is like picking out a few hundred grains of sand from the Sahara desert. At least, that's how ZeroLight's Barry Hoffman sees it. Hoffman shares this with Alan, and other philosophies about XR as a great asset to the automotive industry.
Alan: Today's guest is Barry
Hoffman from ZeroLight. Barry is the chief strategy officer of
ZeroLight, a leading real-time visualization company in the
automotive industry. He has a background with telco, gaming,
automotive, and data science, and interactions with CRMs being the
major thread in his career. At ZeroLight, Barry is responsible for
their US operations, and also leads strategic partnerships and
ZeroLight. This company, just so you know, is really incredible.
They've taken virtual and augmented reality for the automobile
industry to the next level; from AR apps where you can see cars in
your living room, to full VR simulators where you can drive the cars
and see what they interact like. It's an incredible company, I
suggest you check it out at zerolight.com.
Barry, welcome to the show.
Barry: Nice to be here, Alan,
thank you for inviting me.
Alan: It's my absolute pleasure.
I've been a huge fan of your work for a long time. You guys have done
one car after another -- I think one of the ones that I saw on there
was a Pagani. It's just, I'm a carhead as well, so being able to see
the work that you guys are doing and making things look photo-real is
just incredible.
Barry: Yeah, that's true. You
mentioned . It's funny because it's quite a known car, of course --
they only make 100 of series, like how the Pagani Huayra Roadster, I
believe there were only 100 made. But the funny thing is that all
those 100 were sold digitally first. So, there is no real car
available. If you think about the starting price of $2.3-million,
it's sort of like a digital-reality sales case of $230-million. It's
something, if you do the math, how incredible that is.
Alan: So what you're saying is,
ZeroLight contributed to probably the largest use of VR for an
economic benefit ever.
Barry: Yeah, that's true. This
one is, of course, flipped to VR and especially screens, because a
lot of the clientele will want to use it on screens. I would say
Pagani is definitely a case like that. Audi, we definitely
contributed to that part. Most recently, we released a Cadillac in
their showrooms as well, with VR.
All these different car manufacturers,
they tell their story differently. They have different brands, and
they use the technology differently. That's the coolest pitch, and I
like instead of just saying, “okay, there's one type of showcase,
and this is how you do VR.” That would be the same as, “there's
one type of app in the App Store, and that is all you can do.” It's
cool to see this diversity, these ideas coming out of all these
different clients, and then working together with them and turning
that into their story. And not just their story -- because that would
make it only a brand experience -- but also a buying experience,
because a lot of things of what we do is on the high-end
personalization side, like what you just said. Pagani has 18,000
parts, and all those 18,000 parts can be changed into something
unique. That's the ultimate buying experience, I almost say.
Audi, for instance, if you take their
custom build program, I believe there's more Audis in variants
available than there are grains of sand in the world or in the
Sahara. That's the small thing that I have to say. All these thing
0 min
July 10, 2019 The Jetset with Headsets: How XR Will Revolutionize Air Travel, with Neutral Digital’s Greg Caterer
Aviation itself is one of humankind’s great technological marvels – something that can be easy to forget when we’re wedged between passengers in coach on some redeye flight. Neutral Digital’s Greg Caterer is using another one of our technological revelations – XR – to reinvent the airline industry, everything from designing aircraft, marketing at trade shows, and making the flight more comfortable for the passenger.
Alan: Today’s guest is Greg
Caterer. Greg is the chief operating officer at Neutral Digital, an
end-to-end immersive content creator, focusing on the luxury travel
sector. Neutral Digital is an aviation-focused content creation house
at the cutting edge of immersive interaction solutions; they deliver
augmented reality, virtual reality, digital design, architectural
visualization, in-flight entertainment solutions, and apps and
websites for the aviation industry. In a nutshell, they deliver
technology for clients’ campaigns. The Neutral Digital team consists
of professionals with a wide-ranging expertise in VR digital
experience, design, software development, and CGI production. Neutral
Digital can be found at neutral.digital. Welcome to the show, Greg.
Greg: Thank you very much, Alan.
Thanks for having me on the show.
Alan: My absolute pleasure. I’m
really excited to learn about the stuff you’re doing. I’ve seen some
of the videos, I’ve seen what you guys are doing; holy crap. It’s
really, really awesome, the stuff you’re doing.
Greg: Thank you. Yeah, it is.
We’re certainly very proud of all the work that we’ve been doing,
particularly since specializing in the aviation sector, and then
obviously more broadly, the travel sector as well. We feel as though
we get the chance to educate an industry, as well as creating and
selling a product, and really helping to define exactly how this
niche can use extended reality, and in particular, virtual reality
technology. So, yeah, we’re very proud of what we do.
Alan: So, okay — let’s get
right into this. One of the things that blew me away was the
photorealism that you guys have created of 3D models and virtual
environments, of being in an airplane. Maybe explain — if you can,
speak to brands that are using this; if you can’t, that’s fine — but
speak to what it is you’re building, and why that’s important.
And let’s unpack this, because if
you’re somebody who’s in the aviation world, this is a technology
that can be used right across your enterprise; from previsualization,
marketing, sales, training, remote assistance, remote collaboration
— it can be used everywhere. So, what is the focus of what you guys
have been doing, and what are the results that people are seeing?
Greg: You’re absolutely right
with your observation, Alan, about it. Especially the breadth of use
cases that this technology has.
So, there’s a lot in this. I’m going to
try and condense this down to a relatively concise answer. But
broadly speaking, because of the replicability and the repeatability
of everything that’s created in CG, we would build experiences that
focus on marketing, training, and design for airlines, the component
manufacturers, the training bodies, et cetera. Anybody who’s got
anything to do with aviation, whether it’s the high cost of
acquisition for the product itself, or whether something intensely
physical that needs to be shown off.
So, a really good use case — and one
of the main workstreams and strands that we tend to focus on — is
marketing. There’s a very, very big cost benefit to this, and also,
it calls out, perhaps, the benchmark technology that existed before
this, which may be — from the aviation sector’s point of view — a
little bit fell into the realms of “tech for tech’s sake,”
in a way. So we’ve taken this from a marketing point-of-view, and
I’ll talk about the Air Canada project — that was our very first and
biggest in this domain that we did a couple of years ago, in a minute
— the solves two really key problems.
First of all, by creating experiences
that focus on the airline experience itself – so, all of the
branding piece, what it’s actually like to be on board an aircraft
with a specific carrier — you can, for trade shows or for sale
centers, for example, we found that our technology really has its
home in solving a cost problem. Traditional methods of going about
the trade shows particularly, as the focus, would be to send a cabin
cross-section cutout — or indeed, physical seats — to trade shows
at great expense. That can cost anything up to $100,000-$120,000 per
trade show, depending on the complexity of the physical setup.
Alan: Wow!Hold on a sec. So, these
plane companies would bring, like, a section of the cabin? This is
insane. And now with virtual reality, what would the cost be to
deploy… to make everything, top to bottom, would be far less
than one tradeshow. Am I wrong?
Greg: No, you’re not wrong at
all. A little bit varies on the complexity of what it is that wants
to be shown off. But typically, these kinds of experiences in VR pay
for themselves against logistics costs and physical setup costs
within 1-2 trade shows. So not only do you get that benefit to the
costs of shipping — and this still happens a lot of the time, by the
way. This is still very much a method that a lot of airlines are
still preferring. But also, beyond that, it allows a user — as we
know, the full VR technology is extremely effective at doing — it
allows the user to feel completely immersed in the experience, and it
allows them to drive it as well, in the way that it’s built is the
same as the way that you build VR video games. We prefer the Unreal
Engine to build all of our expenses of this kind.
Alan: Let’s
just pause for a second, and just touch on on that. You
mentioned Unreal Engine. Can you maybe explain to listeners who have
never heard of Unreal — maybe they’ve heard of Epic Games and
Fortnight, and they think “video games” or something like that —
can you explain how the game engines, or the development engines, are
being used to develop this type of content? Just just a quick
overview.
Greg: Yeah, sure. There are,
broadly speaking, two game engines that are used for this. There’s
Unreal, and then there’s Unity. Unity, we find, is extremely
effective for slightly more screen-based experiences. We have been
working in Unreal for a long time, and we’ve certainly built a team
around that. We’ve got a number of Unreal developers who work for us
here in-house, and they’re extremely experienced at what they do.
We find that the combination of the
ability to produce experiences that are both extremely
visually-complex and as photo-real as possible, along with the
interactive properties of using that engine, are extremely powerful
for these kinds of experiences. In the same way, like I said before,
as if you would build a video game VR using the Unreal Engine, that
really immerses the user. We find that that’s an extremely powerful
sensory tool, in the same way as fully interactive, fully immersive
gaming is to make the user feel like they’re somewhere else, and
that’s why we use this methodology.
Alan: Wonderful. Is there a cost
difference between using Unity versus Unreal? Or even finding
developers that are good for each of them? Is that a challenge for
different ones? I know the CEO of the Unity had mentioned, at one
point, that about 70 percent of all virtual and augmented reality was
built on Unity. And then somewhere else, I read a stat that 70
percent of all the money is made on Unreal. So, is there…?
Greg: I didn’t realize that. I
think industry, generally, is still getting used to this a little
bit, because perhaps using this means of building these experiences
is still relatively new from the B2B point of view. So, access to
these skills, potentially, is still not quite as open and plentiful
and vast as it could yet be.
We haven’t found any difficulties in
accessing the right talent for this kind of thing. Like I say, all of
our guys in-house are extremely skilled. The blend of Unreal
development and Unity developers that we have is obviously something
that favors the Unreal Engine at the moment, because that’s the
engine that we typically build most of our experiences in. But I
think, as VR cements itself as a medium of communication for a number
of different sectors — and indeed, is a necessary business process,
which we at Neutral Digital firmly believe is going to be the case
within not so many years from now – obviously, as any industry goes
and builds itself up, I think access to those skills is going to
become a lot wider and a lot more freely available. But for now,
there’s good access to the right talent, and we at Neutral certainly
feel like we’ve picked pretty much the best of the bunch.
Alan: I’ll be honest. I’ve seen
a lot of VR and AR, and the stuff that you guys are doing… that’s
why I asked you to be on the show, because 1. It’s beautiful. It’s
really well done. But 2. You’re working with some really big brands.
Let’s look at how the brands are using this. You talked about
bringing it to trade shows. Can you describe specific projects,
goals, and KPIs — or key performance indicators — of what they’re
using to measure success with this?
Greg: Absolutely. So let’s use
the Air Canada example as the main one. They were our original client
in this domain: they approached us with a desire to go beyond the
physical infrastructure of trade shows, and create something a lot
more immersive that saves a lot of cost from logistics and having
these physical infrastructures in place, on the one hand. Secondly,
it went a little bit beyond 360 degree video, for example, in that,
for this kind of use case, 360 definitely has a really solid place,
but being able to build these experiences digitally means that you
can create something that’s first of all user-led, and then obviously
with that, it is extremely interactive.
So, they wanted to create something
that first of all could harness those cost savings, from being able
to just create an experience that was very moldable, very
multisensory. And then they also wanted to create something that went
a little bit beyond what the benchmark was at the time. The fact that
all of this technology and all these experiences — every asset that
you build in the Unreal Engine and use in this way — can be
replicated for other disciplines is also extremely powerful for
airlines, component manufacturers, aircraft manufacturers alike.
For example, we’re now starting to see
a very, very strong set of use cases in the training sphere; training
for cabin crew, and also training for ground operations, and then
design. The ability to collaboratively design a cabin without using
physical mockups until any designs — for example, of the entire
cabin, or even just an individual seat-by-seat design — has been
completely signed off by all parties involved in VR. We’re finding
that, from the marketing use case, it’s grown legs, in the way that
this can be used is really starting to define itself. And we’re
certainly seeing a snowball effect, in terms of the way that our
content is digested, and the kinds of expenses that we’re building.
Alan: You
mentioned 360, and for the people who are listening who don’t
understand the difference, virtual reality – basically, you’re
taking a scene and putting it into a headset and immersing somebody
in this complete environment. There are two types right now; there’s
what’s called three degrees of freedom, meaning you can look left,
look right, look up and down, but you can’t move in the space.
Then you have what’s called six degrees
of freedom, meaning [you can] look left, right, up and down, and then
move left, right, up and down. You have this real ability to move
around. Just that simple change really makes the immersion amplified
by factors of exponentials, and then adding controllers or the
ability to reach out and interact with things — maybe turn on the TV
or touch something — that just adds a whole new layer.
When people are in virtual reality and
they’re able to interact with the world around them, it’s as much a
memory as a real memory of doing something. I remember one of my
first experiences: I went into a human heart — actually walked
around inside a human heart. I will, for the rest of my
life, never forget that — I feel like I walked into a human heart;
very much the way that you guys are making people feel like they’re
sitting in a beautiful, first-class lounge of an airline.
Greg: It’s amazing you mentioned
that as well, Alan, because there’s a piece of research that’s been
done by the National Training Laboratory, which has found that
retention rates — speaking of learning — for lecture-style
learning, for example, are roughly at 5 percent across the board; and
in reading, rates at roughly about 10 percent; whereas VR, as a
medium of communication and learning, scored a retention rate of 75
percent, which is only just below the idea of teaching others as a
means of learning something or retaining information. So in that
sense, it’s no accident that VR is already being rolled out as a
really cool part of the syllabus in a lot of schools here in the UK,
as well.
Alan: It’s no surprise to me,
because the first time I tried virtual reality — the very first time
— I put it on my head, and there’s a guy called Chris Milk who
showed me. I put it on, and I was standing on stage next to Beck in a
concert hall, looking around from a first-person view. I was on
stage! It was just this kind of “aha!” moment, and it was
in that moment that I had what most people in this industry have —
that epiphany moment, and they get into the industry right away —
because I realized that this is more than just entertainment. It’s
more than just videos, or being on stage. This is the future of human
communications.
If you take it one step further, it’s
obviously the future of education. Because if I can put you in a
lecture, and you remember 5 percent; give you a book, you read, 10.
Okay. That’s 15 percent retention. But if I put you in VR, and give
you an experience that you actually do?
At 75 percent retention rates, that is off the charts. And if you
take that into enterprise for training, you cannot fight that, and
you can’t argue against it. There’s no way, shape or form… there’s
no cost that will come close to offsetting that type of engagement,
that type of retention.
I personally see the future of all
education and training as virtual and augmented reality — mixed
reality — as we move to the glasses in the next five years. Boeing
is seeing a 25 percent decrease in the time it takes workers to do
complex tasks like wiring, harnesses, stuff like that, using heads-up
displays. But more importantly, they’re seeing near-zero error rates.
When you combine the increase in retention rates, decrease in error
rates, this is something that the whole world must get on. And they
will. And it’s happening, as you know: it’s 2019, it’s starting to
blow up like crazy.
So let me ask you: what are some of the
major challenges that you guys faced when you were starting? What are
the challenges that people just starting out now are going to face?
What can they expect?
Greg: One of the biggest
challenges that we face, I think, is in terms of educating these
sectors to a set of use cases and benefits, at the same time as
building and selling a product. Because this way of doing things —
especially for a sector like aviation — potentially is still
relatively new, we find that, in order to convince of the
compellingness or the coherence of the product, we need first to
educate as to what it’s actually doing.
I think that challenge comes from a
preconception — a totally understandable and natural preconception
— this is, in various forms, an industry has been around for a
little while. But 360 technology’s been around for quite some time.
The perceptions of it are that, perhaps, people don’t necessarily
understand exactly which pieces of their business process it can
benefit just yet. Although, like you say, this is very, very quickly
going to become cemented as something absolutely necessary within the
business process, across the board, across all spectra.
So, that’s one challenge: helping
people to see exactly what the use cases are, what the financial
benefit can be. Because when you think of marketing experiences, for
example — this being one of the biggest chunks of the types of work
that we tend to take on — aside from the soft benefit of an airline
working with us on the projects and starting to use it at a major
trade show they’re there to sponsor, and seeing thousands of people
go through the experience in a weekend. They’re compelled by, for
example, seeing somebody within a pod have, effectively, a game-style
— very fun, very memorable — experience within an Oculus Rift
headset, the entire experience being reflected on the screen. That’s
got a lot of pull. But at the moment, other than recording the number
of people who go through it, and then using that as a KPI, it’s quite
hard to see the exact tangible benefits of a case.
By implementing VR, I’m going to see
purchase consideration, number of flights booked — if I’m an airline
— increased by X, or the popularity of this route increased by Y
percent, for example. That’s still something that we’re a little bit
far away from.
Alan: It’s interesting you say
that, because when we first started selling this, the first question
out of customers’ minds was, “who else is doing it, and what is the
ROI?” You’re like, “nobody, and we have no idea; still want to
spend $10,000?” Man, it is really expensive.
Greg: That’s so true. And
therefore — just going back to the educational piece of what we’re
trying to help a sector to understand is — from a cost/benefit point
of view, and from an immersion point of view, the weight of this
changes again. The way that you can, at the same time, save a whole
heap of costs on things. Like grounding planes, for example. If we
look at this from a training perspective, depending on the aircraft,
it can easily cost up to $150 K to ground a plane for enough time to
enact training on the cabin crew. Again, that’s one of the use cases
whereby having a VR experience would pay for itself really, really
quickly.
Not only that, but in grounding a plane
to give training – typically, that’s for educating, let’s say,
cabin crew on a piece of the experience that might be relatively
administrative — something that they could definitely do in a more
fun way more easily in VR; it doesn’t need to be a particularly
complex scenario, necessarily. By having training so scalable, you
can deploy training to wherever you can put an Oculus Rift and a
laptop. Saves on costs, means that people can digest top-up training
all the time.
One of our biggest things at Neutral is
that — in focusing on the aviation sector — one of our real primary
goals, with all of our clients across the board, is to improve the
passenger experience, through various different means. With training,
for example, you’re delivering top-up training, or complementary
training, or on-the-side training from the physical aircraft piece.
That allows cabin crew as part of their six-week journey, at the
beginning of their training course, to get even more familiar than
they would otherwise be able to with various different processes that
they need to go through in flight. That would help to improve the
passenger experience from a design point of view, you’re designing
better spaces for them. From a marketing point of view, in helping to
understand exactly what experience they’re going to be having when
they’re on board with you guys in a really, really fun way.
It’s largely about educating them. I
personally give quite a lot of talks around this subject, to
aviation-focused trade shows and that kind of thing. Beyond that,
obviously, it’s selling the product itself, but it’s been described
to me recently – and I don’t know if you’d agree with this, Alan,
as an XR expert — as being a bit like skiing. It’s something that’s
actually quite hard to describe to people in words what it feels
like, and what it does. And people’s minds typically start to whir
and buzz and really think of all the possibilities, once they’ve
experienced something in VR properly. I think we find that in pretty
much 100 percent of cases, whereby we’re giving demos or we’re at
trade shows, for example. It’s really about experiencing and feeling
how powerful it is.
Alan: No, I couldn’t agree more.
One of the things that you just touched on a second ago that really
made me think, “wow, this is amazing,” is that you’re using
these… you’re partnered with the airlines, and you say, “here,
we’re going to create this experience for your marketing and your
trade shows. That same experience, we’re going to alter slightly, and
use it for your training. That same experience can be used for
training for new employees, can be training for specific parts of the
airline.” They don’t have to recreate everything from scratch. They
just can add onto these modules. That’s a really powerful thing,
especially in technology. It’s very rare when you can take one asset
and start reusing it across the enterprise.
Greg: Oh, 100 percent. That’s
also one of the benefits that we find a building these experiences in
full CG over 360, is that you’ve got the ability to stack on top,
take away, change, chop-and-change as you go. Whereas with the 360
experience, for example — not wanting to, by any means, dumb down
the benefits that they have and how fun they can be, of course — one
thing you do miss out on is the fact that, if you want to change
anything, it’s video-based, so you need to start again. You’re right
back to the beginning.
We find that working with a lot of
clients, as soon as they need to do anything from… there’s been an
amenity kit update in a certain class on board, for example. That’s
an individual asset, that in itself can be changed enormously easy,
and then just re-aggregated into the experience. The ease of being
able to keep current with what exactly it is that anybody deploying
these kinds of experiences wants to show is incredibly powerful.
And yes, like you say, the ability to
replicate the use cases, once you’ve got a base asset in your library
— in the form of an aircraft, for example — you can just as easily
create a marketing experience onboard a virtualized A350-1000 as you
can a training one. It’s very, very powerful indeed.
Alan: It really is. So, I know
what’s going to happen. People are going to ask questions like, “how
many people?” “What does it take to build up something like
this?” When somebody calls you and they say, “we have a new
airplane, we want to make a marketing experience,” and you say to
them, “OK, well, we’re going to make this for you.” How long is
it going to take for the guys to build it? What are the costs
surrounding this? What are some of the things that you need from a
customer to get building on this?
Greg: Good question. The
timelines for — let’s say, for example — an experience that
incorporates an exterior element, where there’s a specific aircraft
involved, and is an interactive outside piece, maybe celebrating
various deliveries, and then maybe two or three classes on board,
with a very, very coherent storyboard — I’ll touch on that again in
a second as well, but we firmly believe that storyboard holds
paramount importance to the relevance of the experience, and its
ability to achieve a business goal. I’d love to talk about that more
in a second — to integrate and experience that does that and really
shows off the brand in the best possible way.
That takes about 12 to 16 weeks,
typically. In terms of cost, it’s very variable. It’s hard to put a
specific cost on on any kind of work.
Alan: What are some of the
variables? Give us a range: is this a quarter million to a million?
Or $100,000 to half a million? What is the range, and what are some
of the variables that people need to think about? Photo realism
versus AI-driven avatars. What are some of the things that drive the
costs up?
Greg: Complexity of storyboard
is probably one of the biggest things; the way you want the
experience to pan out. So, how gameified it is, for example. So we
created something very, very cool with Cathay Pacific last year, for
example, to celebrate the arrival of the A350-1000 into their fleet,
and the opening of the new Hong Kong-Washington route which, again,
was launched to a really, really cool trade show called Wine &
Dine in Hong Kong last October. That has a gameified element in it
where, for example, the user gets to role play as a member of cabin
crew, which makes them feel a whole lot closer to the brand, really
conveys the warmth of the Cathay brand, that kind of thing.
That’s a relatively complex experience.
It could just be, for example, that we’re working with somebody to
celebrate the arrival of a new cabin layout in a certain class, or a
new seat. Those are the kinds of things that add variables in the
number of assets as well, and is obviously a pretty
heavily-influencing factor. In terms of the cost ranges…
unhelpfully, Alan, it’s pretty much any of the above that you just
mentioned, in terms of cost brackets, depending on the complexity of
the expense itself.
Alan: What would be the minimum
entry point? So my guess is — and I don’t know your business — my
guess is between $100,000-$200,000, would be the entry to do
something like this.
Greg: Yeah, I think an entry
piece where you’re looking at celebrating something pretty specific
would be towards the lower end of those two figures. It would be more
like around the $100K-mark than it would the $200K-mark.
On average we tend to say that, for
example, if you creating a marketing experience with us that
celebrates the same kinds of things that you would normally want to
bring out at a trade show, and you want to do it in such a way as
you’re not making your visitors sit in it just in a seat, still with
the trade show environment surrounding them, such they can’t really
tell that they’re meant to be on board an aircraft necessarily: these
kinds of things typically pay for themselves within one to two trade
shows.
Alan: Yeah, absolutely. And then
the ability to reuse assets, that’s vital.
Greg: Yeah,
exactly.
Alan: Here’s
something that I’ve not touched on with any other guest on the show;
what about the earned media that these companies are getting, by
being forward leading and forward-thinking on this? Some of the
earned media that some of these brands are getting far outweighs the
cost of even developing this. So, they spend a quarter-million
dollars, and then they get $10-million in earned media. Is that in
line with what you guys are seeing?
Greg: Totally, 100 percent. So a
couple of examples against that, Air Canada being the most current
one right now. There is shortly to be an Epic Games case study on the
experience that they’ve built with us — or maybe all of the
experiences that they’ve built with us — across the entire spectrum
of aircraft with which we worked with them, and we worked with them
on three aircraft. That’ll generate enormously good PR, perhaps in
circles that, without doing VR experience, wouldn’t have been perhaps
quite so forthcoming. It’s definitely something that is going to help
the Air Canada brand, to really cement itself as being one that
invests in innovation, and beyond that — and indeed as a consequence
of that — has the passenger experience and the comfort of the
passenger rights at the front of its thinking all the time.
That’s definitely a very big PR benefit
to this. It shows the brand as being an investor in tech, and
investor in innovation, and importantly, a brand that knows how to
use tech for good, for the benefit of the passenger. With Cathay, for
example, at the Wine & Dine show, where they released this
experience the first time last year — whereby they were prize
giveaways around this as well, being able to win a place on that
flight from Hong Kong to Washington — we really found that the
softer benefits of people being able to walk past a really cool
stand, which was meant to be the the front of an A350-1000 cut off,
with a glass panel at the back of it, with the experiences going on
inside, and with the TV screen pointing outwards to attract more
people into the queue, it really got people’s creative juices
flowing. It really got people feeling excited, in the same way as
they would if they were going into a traditionally B2C-focused VR
experience. Perhaps you can compare it to something along the lines
of Secrets of the Empire, the Star Wars experience — that I believe
his company in Toronto, actually — it’s that kind of emotion that
it’s calling on, and that kind of PR value as showing off the brand,
as being an investor in tech, and the brand that’s seeing this with
the communication medium that it’s going to be across the board very
soon. They’re showing themselves as early adopters, real innovators
in this space, and that brings with it enormous PR value, of course.
Alan: It’s interesting that you
say early adopters – and I think brands that are getting it now are
still early adopters. But by the end of 2019, this is just going to
be the way you do business. We’re going to go away from, “you’re
technologically advanced because you’re using VR/AR,” to, “you’re
not using VR/AR? What’s wrong with you?”
Greg: I
really think we’re on the cusp of that. I think you’re
absolutely right. You could well be right on the money and saying
it’s going to be within 2019. We’re certainly starting to see a
snowball effect, and we’re really now starting to see the more
projects that we work on really call to a specific use case, and the
more brands are starting to approach us really understanding what the
benefits are. I think once that has reached its tipping point, then
this medium, this business process, will show itself as being as
vital as we know it’s going to be very, very quickly.
Alan: I want to talk about one
other thing that I saw that you guys are doing. It’s not in VR. It’s
not in AR. It’s actually in 3D on Web. What you’ve done is taken the
same asset that you put people in VR on, and brought it onto Web in a
3D player environment, so people can spin the plane around and look
at the plane from different aspects. Talk to us about that type of
idea — of using this asset for a web-based experience, or a mobile
experience as well, because it’s not all about just having it on your
face as a headset at a trade show. This is something that can scale
to their website, and to millions and millions of people.
By the end of 2019, there will be over
two billion smartphones that will be AR-enabled. That’s a lot of
people that will have powerful AR in the pocket of their jeans, being
able to pull out their phone, and maybe drop a 747 in their driveway.
Greg: You’re absolutely right.
I’ve seen some really cool experiences in the automotive sector that
do that. I know BMW got one for the i8 as well, where you can drop
one in front of your face in the driveway. That’s obviously something
that is extremely useful for tech that they know that consumers have
got everyday access to.
Alan: I’m going to put —
because we have a Lamborghini one — I’m going to drop a Lamborghini
in my living room, take a picture, and I’ll put it in the show notes
below.
Greg: That’s a great idea. I
know McLaren have got a really good experience in that regard as
well. They built something really cool in VR as well, that’s more of
a configurator setup. But there’s a lot going on in the automotive
space as well, which I think is an entire other podcast in itself,
potentially, material-wise.
But from an aviation point of view and
these complementary assets — these extra assets that you can get
from the experience, you’re absolutely right. Whilst we’re in a day
and age where hardware, such as the Oculus Rift, the upcoming Oculus
Quest, for example, the HTC Vive, are much more applicable piece of
hardware for businesses to buy and use for trade show use cases, for
example, where you put a lot of for people going through the same
device. And it’s really important to be able to scale these
experiences out to what consumers can currently tap into from the
comfort of their own homes.
One of the beauties of having both
incredibly interactive and demonstrative experiences, as well as
incredible visuals, one of the best things that that does, it means
that we can boil down assets — boil down slices of the experience
from the VR experience itself — that can be used on marketing
channels, websites, YouTube channels, Vimeo channels, for example, to
allow the consumer at home to access this through different devices,
and to still have much of the same exploratory benefits of the
experience we create without necessarily having the access to the
hardware that the experience itself was built for in the first place.
Whilst the hardware market is where it is, and whilst it’s obviously
not something that everybody’s got access to all the time for price
or the ability to put it up in a home, all those kinds of things are
there. Obviously, it’s not reached 100 percent mass adoption just
yet. It’s really powerful for brands to be able to have that.
One of the most recent projects that we
completed, we had the pleasure of working with British Airways last
few months, around the release of their new Club Suite offering for
the A350 aircraft. A large chunk of the benefit beyond the
familiarization piece, of course, was the fact that those assets can
be used in marketing channels, can be used to spread awareness about
and generate appetite for a really, really exciting development for
them. Also, as part of their 100-year anniversary, and very much as a
traditional marketing piece that had these extra 360 video pieces
built into it from the assets we created, and the ability to, again,
show themselves as being an investor in tech and demonstrate this
class that obviously doesn’t physically yet exist, but will do very
soon. Having built it in VR, it’s now important for us to to work
with them to be able to get it out into the public domain. And that’s
exactly what these assets are used for. It’s definitely something
that has great effect.
Alan: So, moving a little bit
along, what I would ask you is: what’s one of the best XR experiences
that you’ve ever had personally? It can be within your company or
outside. But what is a thing that you did that was like, wow? what
was your “wow” moment?
Greg: Our
head of VR, Sergio, who’s got an incredible amount of industry
knowledge and been through an incredible number of experiences like
this would a hundred percent agree with me on this: The Secrets of
the Empire Star Wars experiences is pretty much streets ahead, in
terms of everything that I’ve experienced. And this is very much my
B2C point of view, obviously. It spent a little time here in London,
in Westfield. The ability to do exactly what XR is meant to do, in
the sense that it transports your mind to a completely different
reality and makes you genuinely convinced that you’re somewhere else,
and taking part in a different set of activities, and opens up a
whole number of possibilities of different kinds of interactions. You
can have different worlds you inhabit, combined with the way that it
interacts with physical assets. For example, there’s a piece at the
beginning, where you can pick up a gun as the start of your mission,
and there’s a physical gun in front of you, as well as being in the
VR experience — the way that the physical world and the virtual
world have been meshed and embedded together is mind-blowingly
powerful. As far as anything I’ve experienced, that’s definitely
right at the top for now.
Alan: That’s the Void, right?
Greg: Sorry.
Alan: That’s the Void.
Greg: Yes. Yes, it is the Void.
Absolutely.
Alan: That’s the Void, and they’re in a number of different cities. There’s one in New York. It’s a Utah-based company called The Void. And I think it’s thevoid.com. I’m pretty sure it is. There’s one in Toronto. There’s actually two in Toronto, believe it or not; we’re the only city in the world to have two. They have a bunch of different experiences: Ghostbusters, Star Wars, Wreck-It Ralph, where you’re actually going in in the Wreck-It Ralph world. So that’s an incredible experience, and you’re not the first person to say that as well. So I think the guys at the Void have really done a great job at bringing the magic into VR and put haptic floors, scent machines — they’ve hijacked all your senses. And I think that’s really important.
Do you guys use anything other than visuals and audio? Have you used haptics or scent machines or anything like that?
Greg: Yeah. We have. To a
relatively simplified degree, we do use haptics to demonstrate touch,
and pick up, and various different actions within a lot of our
airline-focused experiences. One of the reasons why we haven’t gone
too much further with that just yet, and rather used as a mechanism
to demonstrate what it would feel like to pick something up in the
physical world, is because of the wide variety of different people
who are going through these kinds of experiences – and I want to
talk about The Void a bit more in that sense in a second, because
creating content like that and experiences like that is enormously
helpful and just generally increasing the awareness of what VR can
do, and it’s something that’s going to help all content creators
across the board to be able to communicate their message more
clearly.
One of the things that we really focus
on with abilities experiences is making it is as intuitive and as
simple as it possibly can be for the user; not adding too many
buttons and bells and whistles, making the instructions really clear,
really highlighting the interactions that there are, but not
including too much so as to confuse or overwhelm the user. But
certainly to the extent that we can without over complicating things
and really helping the user to intuitively use what we build. And we
have integrated set pieces of haptic response within the Oculus Rift
controllers for pieces of the experiences that we build, yes. I think
that’s definitely a whole heap more potential for that, as the
industry grows and consciousness towards this means of doing things.
Alan: I’m just going to throw
this out, because you talked about the best experience you had. But
what is the most impressive business use case that you’ve seen so
far? What is the one thing that you go, wow, I never thought of that,
but wow, that is a really good business use case?
Greg: That’s a really good
question. To be honest, we’re at the stage in the industry where
there’s a whole lot of impressive experiences out there. And having a
tour of the Epic Games studio last year was really something that —
I’d only recently joined the digital by this stage — that was really
an educational day for me.
There’s not really an individual one.
I’m disappointing the going to have to offer that in my answer. The
space, however, where I feel like the most impressive
across-the-board B2B technology is currently existing is in
automotive. A very obvious use case and very obvious extension to
what previously existed where VR would find its home would be
configurators; would be in building a luxury car from the ground up.
I know McLaren got something like this. Toyota have got something
that’s really cool along these lines as well. It’ss the ability to
really feel as though you’re in the room with that car.
You mentioned the idea of using AR to
place a Lamborghini on your driveway. I’ve seen the BMW i8 experience
as well, which is extremely powerful. I really feel as though, in VR,
you can go even a step beyond that, and within whatever space you
want to place it in — whatever environment you want to place it in
— to be able to be really close up to a super luxury vehicle, to
take it apart using interactive VR, to examine different elements of
it, to see how it’s built from the ground up. I’ve seen experiences
where you can explode the car, if you will, out, so you can examine
every single tiny detailed component of what makes up the beautiful
thing that you see in front of you, and then zoom it all back
together, and just see how it meshes together and works as a system,
is incredibly powerful. I think a lot of the most powerful B2B
visuals, the most powerful means of really getting a market excited
about a product, a lot of that exists within automotive. It’s a
really, really impressive space we’re moving up to right now.
Alan: It’s interesting you
mentioned that, because one of our previous guests on the show was
Elizabeth Baron, who was the head of VR for Ford Motor Company in
Detroit for the last 20 years. She has seen everything, from CAVE
systems, to early VR headsets, to multi-million dollar experimental
headsets. They even built one with magnetic tracking. But if you can
imagine magnetic tracking, you can’t have any metal. So they built an
entire cockpit out of wood.
Greg: Oh,
wow.
Alan: The great thing about the
way VR is being used at Ford is they’re actually using it for design
first. They’ll bring the car in, they’ll have design meetings.
They’ll look at different aspects of the car real-time, and then
management will come in — in virtual reality from around the world
— and look at the vehicle from all angles, different lighting.
They’ve got real-time ray tracing, meaning the lighting bounces off
the car the right way. They’ve got emulators where you can drive the
cars, and then, that same asset that they’re using for design — once
that car’s designed and approved by everybody and they know they’re
going to go to build with it — now, you can take that and use it as
a marketing asset. They’re doing the same thing you guys are doing
with airlines, only with cars, and reusing those assets for marketing
and sales distribution. One of the coolest things I saw was Jaguar
using VR to sell cars that wouldn’t be ready for three years.
Greg: That’s insane. That’s
absolutely the power of it. The ability to generate an appetite for
something that, otherwise, you would only have sketches or words to
be able to describe these things in. Bravo to Ford; that’s a
fantastic use case and especially the ability to bring, like you say,
upper management in from remote parts of the world and co-design and
co-approve this thing before it’s even been physically created, and
then reuse that asset for various different purposes. How powerful is
that, in terms of being able to save on costs? In terms of being able
to save on logistics, and people having to be in the same space in
order to be able to save on physical prototyping of components? Or
indeed, the entire vehicle? And in order to make the experience fun,
hyper-visual, hyper-interactive, and make you feel like you’re right
next to the actual vehicle, when it doesn’t exist yet!
Alan: Yeah, I know it’s crazy.
It doesn’t even exist!
Greg: Mindblowing.
Alan: It’s here’s another crazy
one. HTC has been promoting this; Bell Helicopters just designed a
new, future-age helicopter. It normally takes them 2-3 years to
develop a helicopter. They built the whole thing in virtual reality
in six months.
Greg: Geez.
Alan: So they saved a 10 times
increase in productivity.
Greg: Exactly. And that’s an
asset to an experience that they’ve built that’s never going to
become defunct. If there are any changes required to that base
helicopter model, for example, they can be made. They can be made in
real-time, and they’re going to be able to use that for a variety of
different purposes going forward, which is just not something they’d
have access to with a physical mockup.
When I first joined Neutral, I hadn’t
had a massive amount of exposure to VR that just yet, so there was a
lot of learning to be done, a lot of upscaling in the first month or
so. I really saw, from having seen some unusual experiences before I
actually joined the company, I really saw this as being about as
close to teleportation as you can get. And I know that’s a very
childlike, somewhat basic way of describing what VR does. But it’s
got the same kind of multi-dimensional transportative benefits to it,
and abilities to it, that can genuinely make you feel like you’re
completely somewhere else, and can genuinely make you feel like
you’re next to something that doesn’t actually exist in reality, but
really tricks your brain, convinces your brain into thinking that it
does. It’s so powerful, and it really is. Have you read the book, The
Fourth Transformation, by Robert Scoble?
Alan: I have. Robert actually is
going to be a guest on the show, as well.
Greg: Noway. Well, that’s one of the things that I read as part of my
upscaling before I joined this company.
Alan: You‘ve got to get Charlie
Fink’s Convergence as well. And Charlie Fink’s Metaverse, even though
he’s spelled MetaVRse wrong. Oh, no.
Greg: Oh
no! He put an E between the V and the R, I take it.
Alan: What the heck was he
thinking?
Greg: I’m sorry to hear that.
Alan: I was actually one of the
contributing authors to Convergence. I know Robert, and I know
Charlie, very well. Robert and I have geeked out many a times, and as
a matter of fact, the first VR experience I ever did with Chris Milk
with it, that concert was with Robert Scoble.
Greg: Oh, no way. Amazing.
Alan: We both tried it together
at the Curiosity Camp, was Eric Schmidt’s camp for tech people?
Greg: Geez. No way. That’s so
cool.
Alan: That was my introduction
to VR. So I feel very blessed to have been brought into this world by
the fathers of the industry.
Greg: Yeah. People of that sort
of caliber. That’s really amazing. What an intro.
Alan: Yeah, no kidding. I dove
in headfirst and it’s been an incredible run.
So let’s let’s shift gears to ask one
final question, then we’ll recap. This has been an amazing interview
so far, and I really want to get your insights on this next part.
What do you see for the future of VR/AR and XR as it pertains to
business? What do you see the future is?
Greg: I 100 percent agree with
an observation that you made earlier, that potentially, by the end of
2019 — or indeed, whenever this is going to happen; it’s a matter of
when, not if — that VR is going to be something that businesses
simply need as a core business process. Whether that be for
communication purposes, or whether it be for familiarization with a
product internally, or for training purposes, or for using as part of
a consultative engagement with a client. If you’re a large
consultancy, for example, I think clients are going to release and
start seeing the need for this. And those who create projects and
business, and work collaboratively with their own clients, will need
to start integrating this into their own business processes in time.
It’s going to be an incredibly core part of innovation centers at
large companies, and business processes, and the product development
lifecycle — for all sizes of business — within whatever timeframe
that may be. I think, certainly, within five years — a little bit
contingent on the hardware market and developments therein, a little
bit on the number of content creators out there really focusing on
specialism and really honing in on having very specific skill sets. I
think it’s really largely about that.
I think it maybe is looking like it’s
going to be a relatively fragmented marketplace from the content
creator’s point of view, which I think is why it’s so important to
specialize, to retain a sense of serious definition, potentially.
Although, there are obviously lots of lots of agencies out there
doing a more generalist approach really, really well. So perhaps I’m
completely wrong with that. But I think it’s going to be something
that within the not-so-distant future is going to be an opportunity
cost if you don’t have it; it’s going to be a much more rare
state of affairs that the company doesn’t use VR, AR, or any of the
extended realities, the mixed realities, in some capacity for
something. I know that that’s a very sweeping answer, but I very much
passionately believe that this is indeed exactly as a Scoble
describes it in his book; that it’s going to be the industry
revolution that the smartphone was in so many ways. It’s going to
become 100 percent necessary for business.
49 min

About XR for Business

From the publisher's feed

Meet the leaders who are changing the face of virtual and augmented reality

Share XR for Business

Sign up to save your podcasts

XR for Business

XR for Business episodes

About XR for Business