XR for Business

By Alan Smithson from MetaVRseBusiness Technology Arts

XR for Business episodes

September 11, 2019 Making Everyone an Expert, with Scope AR’s Scott Montgomerie
The old-school way to train someone
for a task involves memorization, repetition, and practice, in order
to make it like second nature. Not only is that time-consuming, but
also, people aren’t very good at it. So why train, when AR makes it
obsolete? Scope AR aims to help companies get out of old habits, and
CEO Scott Montgomerie drops by to explain how.
Alan: Today’s guest is Scott
Montgomerie from Scope AR. Scott is the CEO and co-founder of Scope
AR, a global leader in developing augmented reality solutions and
products for industrial clients focused around field maintenance,
manufacturing, and training. As the pioneer of utilizing AR for
industry support and training, Scope AR are partnered with technology
leaders such as Google and Microsoft. Since founding the company in
2011, Scott as one of the first executives to get augmented reality
tools in use by multi-billion dollar corporations. Having launched
many AR firsts, Scott has become one of the industry’s thought
leaders and visionaries. He’s shared his knowledge and spoken about
some of the most innovative uses of AR at several leading
conferences, including South by Southwest, Augmented World Expo,
Unity Vision Summit, and XRDC. Some of the clients include Unilever,
Prince Castle and Lockheed Martin. To learn more about Scope AR,
visit scopear.com. Scott, welcome to the show.
Scott: Thanks a lot, Alan.
Alan: Yeah man, I’m really super
excited. We’ve been kind of chatting offline and it’s amazing, the
work you guys are doing and you’re starting to really see this uptake
of augmented reality being used in enterprise. Can you maybe give
people a 10,000 foot view of Scope AR, what you’re doing, and who
your clients are, and what they’re using it for?
Scott: Yeah, sure. So we really
view that augmented reality is a way of interacting with the world in
a way that’s much more intuitive, the way that we evolved with our
hands and our eyes. And so we really view that there’s a huge market
central there. I think it was a stat out there that said that, 90
percent of Silicon Valley is focused on the worker that’s at their
desk, using computers and screens. And there’s a vast market out
there that is untapped, in these field workers that are using their
hands and their eyes. And so if we can use augmented reality to get
them the information they need, at the time of need, and really help
them become an expert when they need to know that information. And
like I said, we think that’s a huge market. So we really approach the
problem in two different ways with our products. The first is a
remote assistance capability. So we were the first to market with a
product called Remote AR, we launched in 2015. So it was far before
any of the other 30 companies that are out there today. The idea is
that it allows you to communicate over video between a technician and
an expert. So it’s almost like FaceTime. If you’re looking at a piece
of equipment — maybe a car engine — you take your phone or a pair
of smart glasses like a Hololens, and you can look at this piece of
equipment and transfer this video back to somebody with expertise.
And this expert can now draw on their side of the screen, and get a
really good remote guide instructions. So the problem with something
like FaceTime is that the communication channel is not wide enough to
provide really good instructions. When was the last time you actually
communicated with a mechanic over the phone or over FaceTime? There’s
no chance.
Alan: Never.
Scott: Yeah, exactly. It would
probably be very painful for him to guide you how to replace
something simple like a spark plug. “It’s that one right there.
No, to the left. No, no, the other left!” But with augmented
reality, it’s a lot easier. You can just point, drop an arrow, or
some other annotation, communication channel’s much more rich. The
second capability is all about work instructions. We define work
instructions as any type of linear instruction that can show you
step-by-step how to do things. Going back to the mechanic example,
now this mechanic could maybe load up some instructions for you, and
it would overlay a 3D model on top of that car engine. Nice and rich
animations showing that you got to take a screwdriver, undo the
screw, and… It’s a really intuitive instruction. So that can be
applied to anything from training, to meetings instructions, to
manufacturing instructions. The whole purpose of the company is all
around making everyone an expert. So either through real time
guidance, allowing an expert — who understands what he’s doing — to
transmit his knowledge in real time in an efficient way to a remote
technician, or by loading up these instructions to provide that
technician really intuitive instructions such that you might actually
not even need training.
Alan: Well, I got to try
something similar at the LiveWorx Conference. I pulled apart a brake.
There was a brake caliper, I held up an iPad, it said “Pull the
pin. Unscrew this.” And in four or five steps, I had
disassembled the brake calipers and reassembled it. I have never done
that before in my life. And I was able to do it. And something else
that I think is amazing is, I had this RealWear device on, where I
could see a screen in front of me — it was a tiny little screen —
and I was able to repair an air filter on a giant John Deere tractor.
I don’t know where the air filter is. I’ve no idea. But it walked me
through step by step by step how to do it. And I think as we move
into exponential growth, these types of technologies are not only
going to be nice to have, but they’re going to be a must-have in all
enterprises.
Scott: I totally agree. As
businesses need to always improve the bottom line, this is the way to
do it, by making your workers more efficient, getting them with the
knowledge that they need. That’s a great way to make them a much
better worker. Do their jobs better, and safer, and faster. There’s
also some pretty massive macroeconomic benefits to this as well. We
keep hearing about the aging workforce and how businesses literally
can’t find good workers. And a lot of those original workers that
have been in their careers for 35 years are leaving their jobs.
Knowledge transfer between those older workers before they leave, and
younger workers, to get them trained up as fast as possible is really
important. AR really has a huge benefit there as well.
Alan: Let’s talk numbers,
because it’s one thing to say “AR provides better value.”
But, let’s put it this way: If any enterprise in manufacturing, for
example, were to see a 5 to 7 percent increase in efficiency, that
would be reason to celebrate. And you guys are seeing numbers that
are 10x that. Maybe let’s talk about Unilever, one of the case
studies you have listed on your Web site.
Scott: We went to the factory in
Gloucester, UK, and we introduced our remote assistance application
into the factory. We saw some pretty great results in reduction of
downtime. The use case around this factory was an ice cream factory.
The line was going down more often they’d like. The problem is, this
is a clean factory; to enter the actual facility, you have to go
through a clean room process that takes a couple hours. It wasn’t
necessarily that fixing the line was difficult. It was that the guy
that knew how to fix the line wasn’t actually there. So whenever
something went down, they had to call this guy, he had to drop what
he was doing, he had walk across the campus to enter the clean room,
and then go and fix the problem. This was multi hours before they
could even start fixing the problem. What we did is we introduced our
remote assistance application. So now when they call the guy for
help, the guy can start guiding these frontline technicians on how to
solve the problem. And quite often the problem is fairly simple, it’s
“go replace this thing” or “it’s just a fault in this
switch.” We were able to reduce their downtime by about 50
percent.
Alan: You reduced their downtime
by 50 percent. Now, for every minute of downtime, there’s a specific
cost for that: €80,000 a month in productivity.
Scott: Yeah. And that was on
that one factory line at one facility. So scaling that across is
pretty ridiculous return on investment.
Alan: So 50 percent, you’re
cutting their downtime in half, which saves them 80 grand a month.
And your solution costs a fraction of that.
Scott: That’s correct.
Alan: Why isn’t every company
doing this?
Scott: Part of challenge is that
change is hard, especially the executives. They’ve been doing the
same thing that, quote unquote “works,” for the past few
decades. It’s pretty tough for them to buy into. “Oh, there’s
this new technology that’s putting up these types numbers, are they
repeatable?” Another customer similar to Unilever, I was
overhearing, one of my partners would call with one of the C level
execs at this company. We’d already gone through a pilot with similar
numbers to Unilever. And this exec was like, “That’s great. But
have you actually replicated this?” We kind of responded to them
— it was the innovation team we’re talking to — “Listen, we’re
seeing 50% reduction in downtime. Even if these are ridiculously far
off — by an order of magnitude — this is still a very good
investment and we should probably implement this across the board.”
It’s just that change, it’s almost too good to be true, which is a
real big problem. With one of our other customers, Lockheed Martin —
I can talk about those numbers in a second — but initially when the
results came back, they were far too good to be true. So much so,
that they told them to go back and do it again a second time, to
prove it.
Alan: You’re the second person
to say this. I spoke to Mohamed Rajani from Macy’s, and they
conducted an experiment with one location, using VR for sales and
marketing. And they saw 65 percent increase in sales conversions. 65
percent!
Scott: Wow.
Alan: And they’re like,
“something’s wrong.” So they did it with six locations,
still 65 percent. So rather than roll it out slowly, they rolled it
out across their entire enterprise, so now they have over 100
locations. And now their average — across the 100 locations — is
still 45 percent increase.
Scott: Wow.
Alan: So the numbers are real.
They may sound too good to be true, but they are true. That is the
transformative power of virtual/augmented/mixed reality. This
technology is the most powerful technology we’ve ever invented. It’s
crazy.
Scott: It’s merging the power of
computers and their vast capacity to form infinitely fast
calculations and have infinite memory, with the problem solving and
mechanical ability of the human race. We as humans, we’re really good
at individual problem solving, but repetitive calculations, we’re not
super accurate. Whereas computers do it correctly every single time.
The whole reason for training is repetition, so that you can hammer
something into our brain so that when you actually need that
procedure to be there, it’s there. But if you can rely on perfect
infinite memory of computers and then transfer that information into
your brain in an efficient way when you need it, that’s the whole
benefit. So it’s really– we’re becoming cyborgs. It’s essentially
what this is. But it’s for the benefit of the human race.
Alan: Yeah, it’s crazy because
this has the potential to disrupt the entire education system. Our
entire education system is predicated on forcing people to memorize
things. We don’t need to do that anymore.
Scott: Absolutely.
Alan: We can get the answer to
anything, as needed, in real time, that we need them. And with the
introduction of cloud computing, edge computing and 5G, we’ll be able
to get answers to literally anything in context to the world around
it. So I’ll be able to look at a machine with my smart glasses on,
and it’ll automatically walk me through step-by-step how to fix it.
That’s what you’re doing, right?
Scott: That’s exactly what we’re
doing, yep.
Alan: So if that’s the case,
then you’ve got Lockheed Martin, Unilever, you’ve got a bunch of
other clients. They’re all doing this. Is anybody starting to roll
this out at scale now? Is that the next step? They run this through
into their whole system and into their organization?
Scott: Yeah. We’re definitely
starting to see scale across organizations. What are the challenges
with this technology? Is it so new, that you kind of have to be
careful with it? If you have a pilot that goes sideways, that can
derail the whole thing. With one customer we actually implemented it
in, I think three factories, and one of the factories had a really
bad experience. They chose a really bad use case. They weren’t really
careful about what they were doing. We actually told them that what
they’re trying to do was not possible. And lo and behold, yep, it
didn’t work. We had really great results in the two other factories.
The third one kind of derailed the whole thing. So these days, we’re
really making sure that we handhold our customers to make sure that
they are using it in a way that’s appropriate, and are going to have
a good experience. There’s just a few fundamental things in the
technology that cause problems. For example–
Alan: Yeah, I was going to say,
can we unpack that, because people listening: Listen up! This is the
moment! This is the education!
Scott: It’s not a
one-size-fits-all technology. And I think that’s where executives are
getting confused. There’s so much FUD out there. And we’re putting up
these results that are too good to be true, and in some instances,
they are. So, for example, if you’re using the Microsoft Hololens, if
you look at a shiny surface — the side of a big beer tank — it’s
not going to track, because the lasers on the Hololens get reflected
and confuse it, and so you get a lot of drift. The Hololens doesn’t
work particularly well outside, because the lasers get drowned out by
sunlight. There’s these little gotchas of things that, unless you
really have a deep understanding of the technology, you wouldn’t
expect, right? When we do implementations, first of all, we work with
our customers very closely. I think one of the reason why we win
deals, versus our competitors, is we’ve been told that our customer
support is by far the best. And that’s why we win deals. It’s because
we handhold these guys. We’re not trying to grow too fast. We’re
trying to make them successful. Choosing the right pilot, get them
the right numbers so that they can have success, and then we can
teach them how to scale this into production, so they have the best
possible outcome. The last thing we want to do is have a poor
experience for anyone. Through that education, we can make those
people our champions in those organizations, make them successful.
And really, these people can really improve their careers by becoming
experts in this, and grow it throughout the organizations.
Alan: Are you seeing people in
these organizations starting to put together teams specifically for
XR technologies?
Scott: Yeah, we are, yeah. And
it’s kind of funny, because a few years ago these teams were
basically Unity developers. They hired a bunch of Unity guys to
create one-off proof-of-concepts, without really realizing that a
scalable solution existed. The whole reason we built our software was
so that it didn’t require anybody to have to code, because coding is
not scalable, it’s not maintainable. It takes months to develop a
single application for one time use. Then you basically throw it
away.
Alan: Yeah, I know.
Scott: What we want to do is we
want to enable guys like documentation specialists and mechanical
engineers to be able to create content very quickly. Something that a
team of Unity developers would take two months to develop, you can do
pretty much in about a day or two with our platform. So it’s much,
much faster. The iterations are much faster. And you can really get
into your pilot and your production a lot faster. And then obviously
there’s a whole lot of production level stuff or encryption data
management and all the enterprise readiness.
Alan: What are some of the costs
associated, because I know that’s a question that comes up a lot when
we’re speaking with customers. How much does it cost to get started?
What does that look like?
Scott: You can get into a pilot
pretty cheaply, five figure range. Like I said, we try to handhold
our customers, with really low numbers.
Alan: Give me a number, what’s
the minimum?
Scott: I don’t like disclosing
private pricing publicly. [laughs] These are enterprises and–
Alan: Yeah, but like, is it a
100 grand? A 100 grand for an enterprise is nothing. So like, is it
50 grand, is it 100 grand? What would be a starting number that
people have to have in mind? Because for a lot of companies we could
do a 360 video for $5,000. The guy came over and told me “It’s
not the same.” And they need to understand that there is a
difference between this.
Scott: We can definitely get
started for less than hundred grand, substantially less than hundred
grand. And then at scale, yeah, it’s in the six figures.
Alan: So that’s reasonable. An
hour of downtime on a machine is potentially millions of dollars.
Scott: Absolutely. One of my
slides, I think, said something like $50,000 per minute of downtime.
So get started with a pilot for 50 grand is nothing.
Alan: I think people need to
understand that this is not just a regular investment. A lot of times
companies will invest in technology that gets them marginal results,
these little incremental improvements. But this is a exponential
improvement on what they’re doing.
Scott: Absolutely. This is
really a generational shift in technology. I think this is going to
be as big as the Internet and tablets were in terms of
revolutionizing how people interact with data. This is gonna be the
same thing, but on a much broader scale on the manufacturing side.
You go into any given factory, most people are still doing things
with paper, binders with instructions. I remember I was on the
assembly line at Boeing a few years ago, and they told us they have
one binder, a singular binder that’s outside of the assembly area.
Because it has to be one binder, because if the instructions change,
you can’t have a duplicate or an old copy sitting around. That’s
incredibly 1990s, so we were pretty shocked at the lack of IT in that
process. So just having the ability to have an electronic version of
these instructions that’s up to date is pretty revolutionary. And
then being able to give your workers the information they need,
contextual information is just a sea change in how these companies
operate.
Alan: Yeah, I think it’s pretty
revolutionary, and I think one of the questions that’s come up a lot
is, people don’t want to wear glasses. But what I think people don’t
understand is, in manufacturing and field service, they’re already
wearing safety glasses. That’s not anything new.
Scott: Yeah. Absolutely. I mean,
I do think we have a ways to go with the hardware. The RealWear
device is pretty good. It’s got certainly some limitations. But for
certain use cases, it’s great. The Hololens too, I think is really
interesting. Can’t wait to see what comes down in the future. I’m
sure there’s lots of really cool innovations coming up in the next
couple of years.
Alan: I think that’s what people
really need to understand, is that this technology, if you go back
five years, didn’t even exist. We had none of them. Zero. There was a
couple of Google Glass type things. But in the last five years we
have come absolute leaps and bounds. I remember going to SBVR and
Augmented World Expo three years ago, and trying some of these
things. You know, there was see-what-I-see, pick-and-pack for
warehousing. And it was so crap that in my mind I was like, “This
is just terrible. It’s going to be 10 years before this is
something.” I went back this year, and that same demo was
absolutely precise and perfect and it just worked perfectly and
flawlessly. I think the time is now for brands and companies to
really start investing in this technology.
Scott: I completely agree with
you. And if your company is not investing in this technology now,
your competitors are. And so when they start rolling out later this
year, next year, they’re going to start seeing these ridiculous
return on investment gains. And if you’re not even building a team
that’s familiar with this technology and certainly thinking about the
change management aspect of it, then you’re gonna be left behind.
Alan: And we’re already starting
to see all the venture capital companies, they all invested in these
platforms and stuff. Now, you guys are venture backed, right?
Scott: Yes, we are.
Alan: So venture backed
companies are like Scope AR. But what they are failing to realize is
that content companies are actually getting scooped up as well. PDC
just bought a content studio — and I believe it was Accenture or…
I think it was Accenture, was one of the two, anyway — they bought a
content studio recently and New York Times bought a content studio.
So there’s kind of this– you need the platforms, but you also need
the content, which is why we started the XR Ignite program, to get
these companies ready for that.
Scott: Absolutely.
Alan: How are you finding the
content creation, for what you guys are doing? Are you building
custom content, or is it just– the platform serves as as its own
standalone content creation system?
Scott: We can build custom
content. We have a team called Creative Services Team that will
generally do a quick Google concepts and consultant pilot projects
for companies. As I said, part of our value proposition is really
that customer support. So we really want an organization to be
successful. So when they’re consulting on a business cases and use
cases, the creative services team can go in there and help and
creating content is part of that. But generally we like to only
create a very simple project for an organization, and then we like to
hand it over with our products and allow their team to start
creating. So that’s been very successful. We need to handhold those
creators. But that’s the only way we’re going to get scale, is by
teaching people how to use this revolutionary tool. Where I kind of
see this is, that this is like introducing PowerPoint in like 1985.
Unless you’ve seen what a really good PowerPoint deck looks like, you
you don’t even know how to use it.
Alan: You’re creating the
standards.
Scott: Yeah, I think it’s more
showing what’s possible in kind of best practices. We like to call
our tool “PowerPoint for augmented reality,” because it’s
drag-and-drop, you don’t need to code, you don’t need to hire an army
of Unity developers to create your proof of concepts. Mostly it’s
people that don’t know how to code that use it and it’s all
drag-and-drop. Those people typically train other people in their
organization. That’s how we grow.
Alan: That makes sense. So let
me ask a question. You’ve got these kind of numbers on the website,
reductions in downtime, and that sort of thing. What are some of the
other KPIs that you guys are using to measure success?
Scott: There’s actually quite a
few KPIs that we track, depending on the use case. This is part of
discovery with our sales team and our customer support team, looking
at those use cases to make sure the customer has success. So in a
manufacturing example, we’ll look at overall efficiency. So, how long
it takes them to manufacture something. So, for example, the by now
famous Lockheed Martin numbers are pretty astounding. They track
things in terms of what they describe as the OODA loop: it stands for
“Observe, Orient, Decide and Act.” For any given procedure,
about 50 percent of the time it’s that first OOD part, Observe,
Orient, Decide. What that means — I’ll give you a concrete example
— they’re building a space shuttle with technology, they’ve done a
whole lot of case studies around a whole bunch of different
procedures and torque fastening. So on the space shuttle, there’s
something like 3,000 fasteners on the space shuttle. And so in the
old world, they would go into a binder. They would flip to the page
that had a table of each of these fasteners, and they’d go find
Fastener 1. They’d memorize the torque setting from this table. They
would go find Fastener 1 in the real world, and they would set their
torque setting, and set it. And then they would crawl out of the
space shuttle, go back to the binder, find torque setting 2, set
their torque wrench, go back in the space shuttle. So this overhead
of reading the manual and then going back in, crawling in, was
accounting for about 50 percent. And again, this was not an isolated
case study, this is replicated across dozens of case studies now.
And so what they were able to do,
simply by putting the information in the Hololens. So now in the new
world, the technician goes in the space shuttle. It shows the
location in 3D space of the fastener number 1. And right above it is
the torque setting. So now the guy sets his torque wrench, does it.
Then it flips the number 2. In 3D space, it shows him where this is.
So they’re seeing a reduction in that overhead that they call
time-to-information. About 99 percent. So that’s resulting in about
42 to 46 percent productivity improvements. That’s one of the really
key metrics we bought is reducing that time-to-information, making it
so intuitive that somebody doesn’t need to be trained and doesn’t
need to go back and consult a manual. It’s just right in your
heads-up-display and being shown to you in a 3D context. You don’t
even have to do that mental mapping of finding something in the real
world. In other cases, if we’re talking about a field service
example, it can be a reduction in downtime or first time fixed rates
or a first time diagnosis rates. Meantime to resolution is another
metric. So it really depends on the use case. We’ve got a pretty
robust return-on-investment calculator that we work with our
customers on for any given pilot they do. And it’s got about a dozen
different metrics in there, just depending on their use case. It
could also be a reduction in travel time. That’s a huge expense. If
you no longer have to fly somebody out to a remote field in Alaska to
fix something, that can be a huge cost savings.
Alan: It’s interesting you say
that, because we had Jonathan Moss, the head of learning for Sprint
on, and they implemented augmented reality training on the phone. So
they– it’s for retail workers. They pull out their phone, they point
it at the thing, and they learn all about the new features. They were
measuring all sorts of different KPIs and the one they didn’t think
to measure — which became the important one — was travel. That
saves so much time and money, not having to fly people around, that
it saved them millions and millions of dollars because you’re talking
30,000 people. It’s crazy, it scales really quickly when you push it
out. Then they actually had one more unintended consequence of that.
The people that we’re learning from on their tablets, they actually
started using their learning modules to teach their customers,
because they were just so good. “Let’s use it.” There’s
definitely all sorts of benefits to this technology as well. And one
thing you’ve touched on right at the very beginning, the aging
workforce is starting to retire. And being able to capture experts’
knowledge is vital.
Scott: Yeah, absolutely. As a
matter of fact, we just rolled out a feature right before the
Augmented World Expo called Session Recording. So the idea is that
while you’re on a call between this technician and expert, we’re
recording the call, but we’re doing it another way, recording it in
three channels instead of two. So recording the audio, the video and
then the 3D annotations, and either the point cloud or the mesh of
what you’re looking at. So that we can then replay those annotations
back on the original piece of equipment. So in this way while you’re
on a call between this technician expert, you’re literally in the
process of transferring knowledge from a person with knowledge to a
person without knowledge. And if you can record that and then use it
for future workers and potentially brain into a training type
scenario, that’s incredibly powerful.
I think as time goes on, we’re gonna
start seeing pretty monumental shifts. I mean, there’s so many
macroeconomic factors. I could talk for hours about this, but one of
the big ones is that previously — let’s call it 10 years ago —
customers used to buy an engine and they own the engine. And any
maintenance on it, they paid for all that they had to do. So they
would probably call up the engine manufacturer and get support and
they would pay a pretty penny for that support, 250 bucks an hour or
something like that. And so for the manufacturer of that engine, if
it broke down, it wasn’t their problem. In fact, it was actually a
profit center for them to send out their technicians to go fix their
faulty engine. But these days, the whole business model has shifted.
They’re buying horsepower. They’re not buying furnaces, they’re
buying BTUs. So it’s almost like an SLA type model. And so now when
something goes wrong, it’s up to the manufacturer to go out and
service that. Now it’s a cost center. What this means for the
workforce is that previously, because of the profit center, if you
had an older worker and a younger worker, it would make total sense
for them to go out to the fields and work with each other for six
months. This young apprentice could learn tons of stuff on the job.
But now it’s not really economical. You want to shorten those
training times because it’s a cost center. That means that these
younger people in the workforce are actually getting less trained. We
can’t bring in this technology soon enough, just because of so many
different factors.
Alan: It’s pretty impressive. Of
all of the interviews we’ve done, you’re episode 40 of the XR for
Business Podcast, and it’s such a varied group of people, and
training and real time collaboration comes up in almost every call.
So it’s interesting that you guys have been doing this– you’ve been
doing this since 2011, right?
Scott: Loosely. Really, we
started taking it full time in about 2015.
Alan: How did you end up saying,
“Oh, I’m going to make smart glasses for the future of
manufacturing?”
Scott: It was a bit of a windy,
twisty tale. I developed some computer vision technology in 2010 for
a previous endeavor that didn’t pan out. I thought this might be cool
to apply to augmented reality. So my first ambition was in marketing
and advertising. And of course, even today we’re not seeing a whole
lot of penetration in working in advertising in AR. I was trying to
recognize billboards and magazine covers and stuff. At the time I was
in Edmonton and I was going to Toronto trying to work with
advertising executives now like, “Yeah, I don’t see the use for
this.” We landed a couple small contracts, but really the big
break was a big industrial company came to us and said, “Hey, we
use this for training.” We thought, “Oh, that’s a cool
idea.” So we did a quick proof-of-concept for them. And this was
on an iPad 2 back in 2011. They thought it was amazing, but we really
wanted a pair of AR glasses. They said, “Here is a big pile of
money. Go buy every pair of AR glasses on the market. And if
nothing’s suitable, then build some..” So we ended up randomly
meeting with Epson at the time they had the Epson BT-100s, which
didn’t have a camera on them. We went to the project manager and I
said, “hey, can we hot-glue a camera on there?” And they
said, “Yeah, I don’t see why not.” And so we ended up doing
was build these glasses, where we hot-glued this webcam on it, the
webcam ran to a laptop where we did the computer vision calculations,
and then we used the external monitor plug and hacked to get this
cable and hacked into the operating system of the glasses to accepts
this video. And this was kind of one of the proof-of-concepts of the
first AR glasses.
And so we showed this at a trade show
in 2012 in Las Vegas. It’s this giant mining trade show that only
happens every four years. And it was a bit of a novelty for this
organization. And we had center stage. It was an amazing location.
But we were only supposed to show the demo like three times a day
over three days. And we ended up showing it over a hundred times. And
every time we showed it, it was crowded by about a hundred people
around me like, “Oh my god, it’s the coolest thing ever seen.
This is going to revolutionize training.” And this one guy’s
story kind of sticks out: he says, “I’ve been maintaining this
exact piece of equipment for my entire career — it was like a
quarter million dollar rock drill — maintained this for my entire
career, 35 years. I’ve probably shortened the lifespan of this
equipment by half by doing it the wrong way. I’ve trained hundreds of
other guys to do it the wrong way, and so I probably cost the company
tens of millions of dollars. So now I’m on my own. I really need this
technology. How do I get it?” And we kept hearing this type of
story from a ton of customers. So the lightbulb kind of went off,
like “Oh, wow, I think we probably found something here.”
Early on, we really didn’t know– I mean, just like yourself, you
didn’t really know what to build. But we started getting contracts
from guys like Boeing, and Toyota, and NASA to build out
proof-of-concepts. And through these initial proof-of-concepts we
realized that, “Oh wow, yeah, this is gonna be the future. A
scalable platform is the way to take advantage of this technology. We
could potentially be the PowerPoint of augmented reality.” In
2015 we all decided to go full-time. The tea leaves were changing.
Google Glass had launched, ODG had a pretty good pair of glasses at
the time. We thought this time the market would be ready to go, by
the time it gets going. Here we are.
Alan: It’s funny. I thought I
was OG. I got in in 2015. I was like, “Yeah, I’m the OG.”
But you are like, you’ve been in it. Before we get going. If you
could say to any customer “here’s the first step you need to
do,” what would that first thing to do, so that they can start
leveraging this power of this technology?
Scott: I think the first step is
choose somebody who has been in the industry for a while and really
understands what they’re doing, and can identify that use case to
make you successful. You know, if you’re out of innovation teams,
which is where we typically like to work these days, is, you know, a
VP or somebody that really wants to implement this at scale, get in
touch with someone and get started. Try really quick POC as fast as
possible, just get some initial metrics, see if it works, and then
start thinking about rolling this out. The sooner you get started,
the better. It’s going to take time to get people comfortable with
technology, with how to create it and where it’s best applied. A
company like ours can really help get you started in rolling.
33 min
September 11, 2019 Making Everyone an Expert, with Scope AR's Scott Montgomerie
The old-school way to train someone
for a task involves memorization, repetition, and practice, in order
to make it like second nature. Not only is that time-consuming, but
also, people aren't very good at it. So why train, when AR makes it
obsolete? Scope AR aims to help companies get out of old habits, and
CEO Scott Montgomerie drops by to explain how.
Alan: Today's guest is Scott
Montgomerie from Scope AR. Scott is the CEO and co-founder of Scope
AR, a global leader in developing augmented reality solutions and
products for industrial clients focused around field maintenance,
manufacturing, and training. As the pioneer of utilizing AR for
industry support and training, Scope AR are partnered with technology
leaders such as Google and Microsoft. Since founding the company in
2011, Scott as one of the first executives to get augmented reality
tools in use by multi-billion dollar corporations. Having launched
many AR firsts, Scott has become one of the industry's thought
leaders and visionaries. He's shared his knowledge and spoken about
some of the most innovative uses of AR at several leading
conferences, including South by Southwest, Augmented World Expo,
Unity Vision Summit, and XRDC. Some of the clients include Unilever,
Prince Castle and Lockheed Martin. To learn more about Scope AR,
visit scopear.com. Scott, welcome to the show.
Scott: Thanks a lot, Alan.
Alan: Yeah man, I'm really super
excited. We've been kind of chatting offline and it's amazing, the
work you guys are doing and you're starting to really see this uptake
of augmented reality being used in enterprise. Can you maybe give
people a 10,000 foot view of Scope AR, what you're doing, and who
your clients are, and what they're using it for?
Scott: Yeah, sure. So we really
view that augmented reality is a way of interacting with the world in
a way that's much more intuitive, the way that we evolved with our
hands and our eyes. And so we really view that there's a huge market
central there. I think it was a stat out there that said that, 90
percent of Silicon Valley is focused on the worker that's at their
desk, using computers and screens. And there's a vast market out
there that is untapped, in these field workers that are using their
hands and their eyes. And so if we can use augmented reality to get
them the information they need, at the time of need, and really help
them become an expert when they need to know that information. And
like I said, we think that's a huge market. So we really approach the
problem in two different ways with our products. The first is a
remote assistance capability. So we were the first to market with a
product called Remote AR, we launched in 2015. So it was far before
any of the other 30 companies that are out there today. The idea is
that it allows you to communicate over video between a technician and
an expert. So it's almost like FaceTime. If you're looking at a piece
of equipment -- maybe a car engine -- you take your phone or a pair
of smart glasses like a Hololens, and you can look at this piece of
equipment and transfer this video back to somebody with expertise.
And this expert can now draw on their side of the screen, and get a
really good remote guide instructions. So the problem with something
like FaceTime is that the communication channel is not wide enough to
provide really good instructions. When was the last time you actually
communicated with a mechanic over the phone or over FaceTime? There's
no chance.
Alan: Never.
Scott: Yeah, exactly. It would
probably be very painful for him to guide you how to replace
something simple like a spark plug. "It's that one right there.
No, to the left. No, no, the other left!" But with augmented
reality, it's a lot easier. You can just point, drop an arrow, or
some other annotation, communication cha
0 min
September 09, 2019 Getting 110% Out of Training in 360° Video, with VR Vision Inc’s Lorne Fade
The top of a wind turbine a hundred
stories up from the ground is not the best place to be making
mistakes, but making mistakes and learning from them is the whole
point of on-the-job training. That’s why VR Vision Inc helps
companies produce XR training modules, so trainees can make mistakes
in a safe, controlled environment. COO Lorne Fade drops by to talk
about it.
Alan: Today’s guest is Lorne
Fade, co-founder of VR Vision. Lorne is a serial entrepreneur that
has built several businesses over the last 15 years. He’s had the
pleasure of working with some of the world’s largest Fortune 500
brands and award winning marketing agencies all across North America
and Europe. His previous agency, Academic Ads, was acquired, and he
went on to found VR Vision Inc. As the co-founder and COO of VR
Vision, they’re a virtual and augmented reality startup that’s
enhancing immersive training outcomes for some of the world’s largest
brands using VR, AR, and AI technologies. He’s also the founder of
Reality Well, a healthcare technology platform to improve the quality
of life for those living in long-term care facilities. You can learn
more about VR Vision by visiting vrvisiongroup.com. Lorne, welcome to
the show.
Lorne: Thanks for having me,
Alan. Thanks.
Alan: My absolute pleasure, man.
We’ve known each other for quite some time through the VR/AR
Association in Toronto, and we shared some booth space together, and
it’s always great to see what you guys are working on. I know the
last time we saw each other, you were showing me an automotive
manufacturing facility in virtual reality and how you were using
that. So let’s dive in there. Let’s talk about how you guys are using
VR and 360 video to make better training.
Lorne: Yeah, that’s that’s one
of our bigger use cases with Toyota, where we’re training about
10,000 employees currently using 360 video, in immersive training
scenarios in VR. And it works really well for eliminating risk and
providing a safe environment with zero harm. And it’s totally
immersive. So the employees that are getting trained in VR, no
distractions, they can’t be on their phone or anything. It was really
simple the way we did it. We just storyboarded various scenarios with
Toyota on various processes, on safety concerns, on their assembly
lines or processes that were mundane and replicable. And then we went
out and filmed with a stereoscopic 3D camera, so when they put on the
headset they feel like they’re there, fully 3D. And we mapped out, I
guess about two to three minute scenarios, various parts of their
assembly lines and filmed it all in full 3D and then ported it over
to VR, added some overlays, some voice overs, some touch points and
interactivity so that the employees could be trained in a completely
immersive environment. Nothing like this is, from my knowledge, has
ever been done before. So it’s really cool to have this type
opportunity to work on a project like that.
Alan: So how are they measuring
success? For example, STRIVR is doing 360 video with Wal-Mart and
their key performance indicators. They’re measuring training times,
how long it takes to train. They’re also testing retention rates.
What are the KPIs that you and Toyota decided on, how to measure
that?
Lorne: Yes. Great question. We
developed a in-house analytics engine for tracking where the user is
looking, the various touch points of the training scenarios. And
every user that uses the platform gets their own log-in, so we track
each user, their effectiveness, and how well they’re being trained
with the scenarios. And then within the scenarios, there’ll be, let’s
say, about 20 interactive touch points for various risks, or hazards,
or processes that the employee needs to learn. And then at the end of
this scenario, they’ll get a breakdown or a test results screen that
will get pushed to Toyota’s LMS on the backend so they can see how
the employee performs. But also within the headset, the user will get
to see where they performed and get to learn again on the various
things that they might have missed throughout the course of the
module.
Alan: So it’s really giving the
employees the opportunity to learn through making mistakes, which is
funny because our whole lives in school, we learn not to make
mistakes: you get an F, and that means fail, and you’re screwed, and
you can’t go into university, and it’s beaten into us never to make
mistakes. But in the real world, we make mistakes every day, and we
learn from them, and we move on. But this is even better, because
it’s not the real world. You’re able to make mistakes in the privacy
of your own headset, you’re not feeling embarrassed.
Lorne: And it saved you a ton of
money for Toyota overall. Basically, instead of having an employee on
a live assembly line making those mistakes, where they would have to
shut down production, then that could be super costly over time for
the plant itself. This way they’re able to train in a risk-free
environment without shutting down of production, so that when they’re
ready to hit the assembly line – for whatever the processes that
they’re tasked with – they’ll be way ahead of the game, it’ll cause
less mistakes and save a ton of money for Toyota overall.
Alan: So how are you measuring
that specifically, are you measuring training times?
Lorne: Yeah, we’re measuring
training times. We’re measuring efficacy for the employees. And then
when we put them on the live line, we get to compare and contrast
based on their test results, how many mistakes they’re making on the
live line. Now, we’re not fully testing just our training scenarios
as the end-all, because Toyota has a number of other training LMSs
and dojos that they’re using for training the employees, but they
weren’t seeing an improvement overall with the employees that had
done the VR training.
Alan: That’s really interesting.
In your analytics, you mentioned that you’re pushing it to their LMS
system. How difficult was that, to go from one company to another? I
would assume there are different ways of working.
Lorne: The biggest challenge
there was working with their IT, because they had a pretty strict
regimen for their firewall. And then accessing it is a very tight
network. A lot of restrictions, a lot of loopholes we have to go
through. So it took a couple of months of working with their IT team
to be able to pass through data from the headsets, and have the
headsets themselves connect seamlessly to their network, and make
sure they were all on the same MAC address. It’s actually outside of
my technical scope. I’d have to ask our IT guy internally here. But
basically, once we figured out how to pass through their network, it
was seamless.
Alan: What about things like
device management? Because if you’re going to train 10,000 employees,
how many devices does[sic] that?
Lorne: That’s definitely a great
concern that enterprise groups need to be aware of. We’re seeing the
brands like HTC and Oculus start to catch up with their business
solutions that are going to start to offer enterprise management. We
kind of hacked it for the get-go because it wasn’t available as of
yet. There’s a great company you can look up called 42 Gears that
basically provide a mobile management solution, that can be ported to
Android for any devices that are being programmed with Android
backends. And that allows us to see all the devices on the network,
push updates through them, and manage them remotely. And then we went
a step further and we developed a mobile management application for
tablets and cell phones, so that a practitioner or a trainer that’s
managing the training serials for the users can manage which modules
they’re placing the user into, and see where they’re at within the
training program.
Alan: Now, is that done from a
tablet or a phone or something?
Lorne: Yeah, yeah, it can be
done from either a tablet or a phone. Anything Android or iOS based.
Alan: When you’re making the
scenario– so, for example, take us back to the beginning. You meet
with Toyota. They say, “Hey, this is great. We want to do a
trial.” What is the lead time from this first meeting you had,
to deployment to 10,000 employees. Is that like a year or two years?
What’s that look like?
Lorne: I think the development
timeline was about six months, back and forth to storyboard out all
the various modules. We started with a proof of concept with one
simple module to see how effective it would be. They loved the 3D,
they love the immersiveness of it. So we move forward with five
modules, and then those films and the whole processing,
post-production took about a year overall for all five modules. And
now we’re in talks to scale that through more facilities throughout
North America. Per module, it really doesn’t take that long. It’s
just that we have a 360 development production crew, goes on site,
films, takes about one or two days, and then we take it back and
post-produce it with various touch points and voiceovers. And that
whole process for one module takes anywhere between three to four
weeks, overall. I guess the back and forth that took the longest was
working with IT and figuring out some of the other complexities, like
pushing updates to their LMS, things like that.
Alan: I would think also just
the simple procurement process. [laughs]
Lorne: Yeah. Oh, that too.
They’re very–.
Alan: Take longer than
everything.
Lorne: Yeah. Yeah.
Alan: Standard across all
enterprises, yeah. There’s a note to people listening: if you’re
working in the C suite of a large enterprise, perhaps consider
figuring out a way to work with startups more efficiently, through
streamlined procurement processes, because it really is onerous for a
startup trying to innovate on technology, while trying to run the
gauntlet that is procurement.
Lorne: [laughs] And then keep
your overhead going, and runway.
Alan: Exactly. Part of the
reason we started XR Ignite was to really be that – for those of you
who don’t know, XR Ignite is our community hub and connector – so our
goal with XR Ignite is to be the connector between startup studios
and developers and corporate clients, and be that conduit for
conversations back and forth. What our corporate is looking for – and
you mentioned some of them, safety, security, networking, device
management, LMS, integrations – and then bringing that knowledge over
to startups and saying, “OK, what do startups need to do
business with corporate?” and that’s streamlined procurement
processes, faster payments and more streamlined communications. So I
think it’s a time in a place where we need to really bring everybody
together. So that’s what we decided to do with XR Ignite.
Let’s talk about the actual
experiences, because I’ve tried one, it was really interesting. You
put on the headset and it was really cool because I’ve never been to
a car factory, where they build car parts and doors and things, and I
was in there and there’s this woman stamping giant pieces of aluminum
and she’s doing her job. And then you have to look for anomalies. You
have to look for things on the ground, or is she not wearing a
hardhat, or whatever it is? Did they provide you those things or did
you look at the space and go, what if we put a banana peel over here
or…?
Lorne: We basically work with
them on the storyboard to provide the highest risk items that would
be the biggest safety concerns for the employees. Like not wearing
proper PPEs, walking in the laneways where they shouldn’t be walking.
Just not using proper safety gear or leaving things in the wrong
places. And then we went a step further and added our own flair, if
you will.
Alan: I love it. Now, were they
accepting of adding your own flair to that? Because sometimes this
stuff can be really dry and boring.
Lorne: The basic secret sauce,
though, that we provided: we developed this for standalone VR
headsets and a lot of the standalone VR headsets really max out at 4K
resolution, whereas we’re filming in 8K resolution. So we wanted to
push the best quality that we could for the experience, so it was
completely immersive, was exciting. It had replicability and it was
scalable. So on our backend for the post-processing side of things,
kind of did some optimizations with the 360 video to make it appear
around 6K instead of 4K in the headsets. Reduce some of the
screen-door effect, really just to optimize the visual aesthetic of
it so that when they’re playing it in the headset, it just appears as
best as possible for the experience.
Alan: I can attest to that. It
really was a clear situation. It was–
Lorne: It’s like watching a 3D
movie. [laughs]
Alan: It wasn’t even like a 3D
movie. It was like I was in the factory. But by the time I put the
headphones on and the headset, couple minutes in and I was right
there on the factory floor watching this process of stamping these
things out. I’ll never forget it, because I feel like I was right
there, watching it. And I got a few of the things wrong, but…
Lorne: I think that’s the true
value of VR. It’s being able to replicate any type of scenario that’s
in the real world but in a safe, controlled environment. And I think
this works really well for enterprises that have a lot of potentially
harmful, or carry a high risk-versus-reward type of training that may
be expensive for onsite, or dangerous for the people that are
training. There’s another scenario we’re working on right now with a
wind turbine manufacturer, and they’re developing maintenance
technician training and it carries a high risk to go up to the top of
those wind turbines and work on them with a tether. And they’d rather
have these employees trained in a dojo in a safe, VR controlled
environment before sending them up hundred stories high to the top of
a wind turbine.
Alan: You know, that seems to
make sense. I went to a talk the other night and they were talking
about– there was a gentleman who’s making nuclear reactor training,
for the nuclear reactors here in Ontario. And one of the scenarios is
the CANDU reactor, which is a huge reactor. It’s maybe 30 feet high
and it’s got all these little tubes. And in real life, you can’t walk
in front of the tubes, because they emit radiation and there’s just
like invisible beam of radiation. So if you walk in front of the
beam, well, you’re–
Lorne: Chernobyl.
Alan: Well, no, you’re just
going to have a paid vacation. But one of the things that they showed
is, how it’s managed today is, they literally have a piece of tape on
the floor. They have duct tape on the floor saying, “Don’t walk
within these duct tape lines.”
Lorne: Oh, jeez.
Alan: That’s the safety
protocols in a nuclear reactor. So being able to recreate that with a
Hololens – is what they used – and be able to recreate visibly what
that beam of radiation looks like. Then you can get a visual
representation so that when you’re in that facility and you have to
go because it is not something that people do every day, it’s very,
very rare that they have to go in there. But when they have to go in
there, they have this visual representation of these beams of
radiation coming out. And I think that’s a little bit better than
some duct tape on the floor.
Lorne: Yeah, I think nuclear
reactor training is one of the better use cases for creating a safe
controlled environment versus a live test bed.
Alan: You would think, yeah. You
know, we don’t really want to go down that road. You talked about
wind turbines. That’s another big, big area because I mean, clean
power is becoming huge and wind turbines, they’re– I don’t know if
you’ve ever been in one.
Lorne: No.
Alan: But I have, in VR. I’ve
been in a wind turbine. I climbed up the ladder on the inside. I got
inside. I looked at the motor. I stood on top of one, all in VR. And
I’m good with that. I don’t necessarily need to do that in real life.
Lorne: I’ve definitely been in
one in VR. I haven’t been in a real one. [laughs]
Alan: It’s pretty awesome. And
there’s so many things that can be done with this. And let’s talk
about the cost to deploy something like this. For example, company
comes, XYZ company. They say, “hey, we saw what you’re doing or
we heard the podcast. This company is doing this. We make widgets and
here’s our machine factory. We want to start doing safety training in
VR.” What does that typically look like, for as roll-out, your
measurements of success, and the costs as well?
Lorne: The costs of actually
come down with the standalone headsets, because there’s less
graphical work that needs to be done. It’s really linear overall.
Basically, there’s two ways that we develop up here at VR Vision
internally for these training applications. There’s 360 video that’s
ported into VR scenarios, that’s going to be filming or any type of
real world environment. Typically, the 360 video form factor is going
to be cheaper and more cost effective than creating a CGI based
environment, which is basically the other way that we developed
training applications. For the 360 side of things, per module, we
charge anywhere from 15 to 20,000 dollars, but you also need a
platform to interact with those 360 videos. So we start with like a
base layer for anywhere from five to seven thousand dollars for a
platform that’s built out. It’s kind of like the menu selection
screen of Netflix, if you will. And then once you’re in that
platform, you can select the various modules or training outcomes
that business may want to use. And basically, each training outcome
is anywhere from 10 to 20,000 dollars, with interactivity and voice
overs and fully optimized. It really depends on the length of the
training outcome. These are averaging about three minutes long. But
if you have a longer one, it will take more post-production, which
would be more costly.
For a CGI based environment, those
costs can be far reaching. It really depends on the scope and brevity
of the application. The ones that we’ve developed, they fall into
like the 40 to 50,000 dollar range, for basically a three to five
minute CGI based training scenario. We did one for a fire safety
drill for a company down in Texas called Alchemy Systems, and it was
basically replicated version of their factory, one-to-one in a CGI
based environment. And it trained the users that worked in the
factory how to find the fire exit, and what to do in case of an
emergency.
Alan: So how did you get the
factory one-to-one scale? I mean, obviously, they have the
measurements of the factory. You just import that into CAD modeling
program or, how did that work?
Lorne: Yeah, they had FBX files
of a lot of their factory. And then there was another way that we did
it was using LiDAR, which basically went on the floor, scanned the
whole factory. It was pretty boxy, rectangular shaped factory, so
it’s pretty easy to do. Just scanned the length and then the size of
it, and then ported it over into a virtual environment.
Alan: Well, that’s easy.
Lorne: It sounds easy, but
there’s a lot of technical expertise, but…
Alan: If I had asked you the
same question three years ago, it probably wouldn’t have been that
easy.
Lorne: Yeah. Yeah.
Alan: One of the things that
we’ve been seeing as a repetition on this show, is that these
technologies are getting better, faster, cheaper every day. There’s
more talent coming out that know how to use these technologies. But I
think one of the key takeaways is that, this isn’t something that you
should be looking at five years down the road. This is something that
people are utilizing now and getting dramatic results. So let’s talk
about some of the results that your clients are getting.
Lorne: They’re having
resolutions of conflicts that can arise in a workplace scenario.
That’s one of the biggest ones, just avoiding those risks and
avoiding downtime for various training scenarios. They’re getting a
lot of assessments, post training. So with our analytics engine,
we’re tracking where the users are looking, we’re seeing where the
problems may arise, or where things are being missed. And then let’s
say they’re missing an easily overlooked area of just handling a box
or flipping a switch properly. And we see after training 10,000
employees, that maybe half of them are missing this one simple thing.
So now we know that this training outcome needs to be pushed a little
bit heavier for those employee, so they can reduce the problems with
whatever that specific process is.
Alan: Or maybe the process
itself is flawed.
Lorne: Or maybe that as well.
Yes.
Alan: We never want to talk
about that. But let’s be honest, sometimes things were done just
because they were always done that way. And now this can shed a light
on certain processes that are maybe antiquated or out-of-date.
Lorne: Something that helped us
optimize our training programs was to learn from the employee
feedback, and then getting multiple iterations of our training
programs in place, so that the frontline employees can help optimize
training elements to maximize effectiveness.
Alan: So maybe unpack that a
little bit.
Lorne: So basically with the
post-training assessments, we did a lot of surveys on the employees
to see how effective they were finding it. We had some training
modules that were rated much higher than others. So we can go back to
the ones that were lowly rated and find out “Well, maybe this
was too hard for the employee to learn various elements of the
training protocols.” so we can make it a little bit easier for
them to find whatever the risks were or the safety concerns were for
the training scenario.
Alan: So now in that case, you
have to go and refilm this, if it’s 360 video, for example.
Lorne: Yes, it would be to
re-storyboard it from the ground up for 360 video. For CG, it’s just
a matter of tweaking things in-house.
Alan: I think therein lies the
exact cost-benefit analysis of 360 versus CG, because if you’re
filming in 360 video, it’s 15 to 20k to film each one of these
modules. And in CGI you’re looking at 40 to 50k. The difference being
if something needs to change, you have to go re-record that, that’s
another 20k. In CG, if you need to change something, you can change
it on the fly. And one of the things that I love about computer
graphics is that you can reconfigure the warehouse. You can add
elements real time. You can add things in. So there is that benefit
of–
Lorne: Future proofing.
Alan: Yeah, future proofing
that. But it’s not always necessary and it’s not always warranted. So
when do you decide which one to use over another?
Lorne: There’s also factors to
consider, like multiplatform support, having VR/AR functionality, but
also being able to push those exact scenarios to the web. In case
there’s not a VR headset available, being able to have a 360 video on
the web for the user to learn in a dojo or LMS environment, that
doubles the effectiveness and accessibility of the training programs
as well.
Alan: What devices are you
pushing up to now and how does that look like? Let’s take 360 and
then we’ll move into CG, for example, because the headsets are
changing daily. We’ve taken a complete device agnostic approach,
because who knows what the next big thing is gonna be. So how do you
then future proof the content to be available in such a broad range?
How does that look like and what devices does that go to?
Lorne: We’ve kind of
transitioned away from PC powered VR. We think that a lot of the
future is going to be based around standalone devices. And as the
computers get smaller and faster and more portable, people are just
going to want to get away from the cumbersome setups of sensors and
just move toward easily portable and scalable device. Things like the
Oculus Quest, Oculus Go make it really easy for adoption. Then you
see Vive Focus and the Focus Plus, work equally as well. They’re much
more portable and scalable for businesses to adopt, whereas two,
three years ago these devices didn’t exist. So it’s hard to predict
where things are going to be in another two years based on how fast
the industry is moving.
Lorne: From the backend side of
things, for programming, something to be aware of when developing
these – CG based, especially – is there’s a lot of downsizing of
sampling for various graphics, because the standalone devices simply
can’t push the same amounts of power and graphic quality that the PC
powered devices can. So a lot of the times we have to really dumb
down or filter down the polygon counts, just to make sure that the
standalone devices can still push a decent looking scenario but not
overload them, so not to cause frame rate issues and nausea.
Alan: Very interesting.
Lorne: It’s definitely something
that developers should be aware of, or businesses looking to adopt
the technology.
Alan: What’s the biggest
challenge that you’ve found in the adoption of this technology?
Lorne: Tracking issues has been
one of the biggest hiccups for us. Before the Focus Plus came out, we
were really stoked that finally stand alone VR is here and we ported
over a lot of our platforms to the Focus and then we ran into a wall
with tracking issues, the controllers would lose focus when you put
the controller behind your head, for example, simply because the
headset only had cameras front facing. The Oculus Quest has helped a
little bit with that because they have four cameras on the front and
they’re kind of like a fish eye lens. So they track a little bit
better for fronts to the sides and above you and below you. But
still, you’re going to lose tracking if you have to put your hand
behind your back for whatever reason. So that’s something that’s been
a challenge for us, for developing some training scenarios.
Alan: I think the hardware
itself is growing by leaps and bounds. They’ve made really, really
big strides in bringing that one unit without having to have a
computer. And I think that’s one of the biggest challenges with VR
has always been the challenge of just getting it to work. You set it
up, and then all of a sudden you’ve got 30 Windows updates, and then
another Steam update. And then by the time you’re ready to go,
there’s an hour gone. Your training time is missed.
Lorne: Yeah. Definitely
something to be aware of. I think we’re going to see a lot of
advancements in technology in both consumer markets, as well as
industrial and commercial applications. Something that we’ve been
really excited about is, we’ve just been testing the RealWear AR
headset.
Alan: They raised 80 million.
Lorne: Yeah. They raised a ton
of money, but they’re really competing with the Hololens. It’s not
really competing in a sense because Hololens is more for a static
environment, where the RealWear is more for on the job task based,
ruggedized training. And I think there’s gonna be a lot of potential
for hardware – mixed reality based hardware – in the future. I think
they’re going to combine a lot of AR and VR for ruggedized use in the
field. I think that’s where the immersive training side of things
will move towards, although it is hard to predict.
Alan: I got to go to PTC’s
LiveWorx in Boston and I tried the RealWWear headset, and basically
what it is, is a little articulating arm that mounts to your
construction hat, and it’s like pulling down a screen in front of
you. Like imagine pulling up your phone, right? But you pull up a
little screen and it’s like having a 9 inch, 10 inch tablet that’s
about maybe a foot away from your face, in one eye.
Lorne: Interesting.
Alan: But it’s ruggedized. So
it’s waterproof, it’s bombproof. It’s like this big rubber arm. Now
the issue with it – and they’re going to address this, I’m sure, on
subsequent ones – is that finding that little sweet spot of getting
it right in front of your eye in the right spot is kind of finicky,
you kind of wiggle it. And then once you get it, it’s usually fine.
But I put it on and they have this thing called Expert Capture. And
what that means is, you can use the camera on this thing to capture–
let’s say, for example, I’m an expert, I go up to a machine – in this
case that I went on, it was a tractor – I look at the tractor and I
say, OK. And I hit record and I record how to replace the air filter.
And then I hit stop. Now, that’s recorded forever and it can be
pushed out of every headset. Now, what I do is I put on the glass. It
walked me through step by step. A little video said, “here, go
here, pull off this cover, replace the thing, put the cover back,
make sure the switch is turned.” And that was it. And I replaced
an air filter on a tractor. And I’ve never touched that before. I’d
ever been on a tractor before. But that little heads-up display gave
me all the information I needed, real time.
Lorne: So do you think you could
do that on a real world tractor now that you’ve learned it in the
headset?
Alan: Oh my God, yes. I’ve done
it. So it’s in my head. Obviously, I don’t know the model of tractor.
So it would vary by model. But if you put me in front of that model
tractor and said change the air filter, I go to the back of the
tractor, I climb up, I pull the air filter out. I know exactly where
it is. Yeah, I did it.
Lorne: It’s amazing.
Alan: It’s not something that
you told me about or I learned on YouTube. I did it. I did it in real
life with my hands. And I think this is something that being able to
train people on in VR is one thing, where you need a completely
virtual and safe environment, but also taking elements of that 360
video elements or those elements of just the information you need at
the time you need it, into the real world is really important. That’s
why I think RealWear it is a really excellent, elegant solution,
although it is very low tech, if you think about it.
Lorne: Yeah, I think being able
to use your hands in the real world. I think just a hands-on element,
it creates much better retention for learning overall, versus the
scenario where you’re using controllers. You’re still learning, but
being able to get your hands dirty, if you will. And I think that
more than even VR may help learning retention. So it’s interesting to
see where the space goes in the next couple of years.
Alan: Yeah. There’s a trial
we’re going to test. There’s an excavator, a VR experience made by a
Toronto company called Career Labs. And the first thing you do, you
learn how to start it, what all the controls do, and then you drive
it. You go grab some rocks and put it in a dump truck. So we’re going
to put my daughters, who are 11 and 15 in the scenario for an hour
each, and then we’re gonna take them out onto an excavator and see if
it translates from an hour in VR to being able to operate a real
excavator.
Lorne: That’s great.
Alan: Well, we’ll see.
Lorne: See how the results are.
Alan: It’ll either be awesome or
they’ll destroy a couple hundred thousand dollar excavator.
Lorne: [laughs] Let’s hope not.
Alan: I hope not. I have
confidence in the VR training.
Lorne: [laughs]
Alan: So what’s next for you
guys? You’re expanding, you’re growing, you have a new office in
Toronto. What’s next?
Lorne: I guess I’d like to touch
on Reality Well, because that’s a subsidiary brand that we’re
launching. We actually just launched the website and we’re doing a
bunch of PR right now for it. It’s basically a platform built for
standalone VR – for the Vive Focus or Oculus Quest – with a health
care focus, for measuring improvement of quality of life. So we’re
really focused on retirement homes, hospice centers, places like that
for the elderly. We want to help with cognitive thinking, memory
retention, improving mobility, as well as just adding entertainment
and increased mood for people that are otherwise bedridden or just
bored out of their minds.
The platform itself is fully contained with three sections. The first section is CG based environments that are playful and fun, with animals and interactivity, and they’re just meant to be light and fun for the users to explore. There are things like winter scenes, beaches, forests, very vibrant colors, all CG based. The second part of the platform is real world 360 videos and photos, that we’re slowly procuring in 8K stereoscopic 3D. The highest quality that we can really develop for, it’s all our own content. And it’s just places like landmarks all around the world, bucket list items. I’m actually going to Italy in two weeks to film more content there as well. And that’s a great way for the users to visit places that they may not get a chance to visit in their lifetime. The last part of the platform is minigames, but they’re called exer-games, or serious games in the healthcare community. And we’re working with the University of Waterloo to validate these games, to help with things like mobility, to help with memory retention. Some of the games are like rock balancing games. There’s like a music game. It’s kind of like Beat Saber, but you’re on a beach and there’s just some beach balls coming at you instead of the Beat Saber blocks. It’s a lot of fun. They really enjoy it so far. We’re developing more games for that as well. There’s a fishing game that we’re almost finished and there’s gonna be a farming game as well.
Alan: So let me get this straight. You’re hitting beach balls on the beach. Is it things like, [hums jitterbug tune]? Is it like big band swing music? Clearly not techno music like Beat Saber.
Lorne: No, no, it’s not techno.
It’s more classical chill, laid back, relaxing type of music. This is
definitely aimed at a different crowd than the Beat Saber crowd..
Alan: Not going to have the
Skrillex remix?
Lorne: No, no dubstep here. It’s
to help increase their mood and just overall entertainment. So it’s–
Alan: Are you collecting data
about these people as well?
Lorne: Yes. Yes.
Alan: The health care providers
so that they can help with, because I can imagine there’s some
depression and there’s some loneliness, so…
Lorne: Yeah, there’s analytics
for all of our platform and there’s a rating system, as well for a
lot of the experiences. So after they’ve tested out each one, they
can rate it on a scale of 1 to 10. So we can try and drill down what
they like the most. And right now we have pilots in about six
different health care facilities. And we’re gauging and measuring to
see which type of scenarios and environments that they like the best.
And so far, they seem to love animals. We filmed at Toronto zoo, and
that’s one of the favorite 360 video experiences that we’ve shown
them so far. Because you know what it is? It’s the 3D. When you’re
filming in stereoscopic 3D, – let’s say you’re looking like a horse
range – you almost want to reach out and touch the horse’s head,
because it feels like it’s right there in front of you. So it’s
really amazing what we’re able to do with the technology nowadays.
Alan: It’s really fantastic,
being able to provide such a wonderful service to seniors who may or
may not be able to get out, or maybe their memory is failing. And
it’s just, it’s wonderful.
Lorne: Yeah, it’s definitely
heartwarming. And I really hope that it helps. And we can grow this
to provide it to as many facilities as possible, because I think this
could be super beneficial for a lot of people. You know what it is?
It’s like bucket list items. If I’m 80 years old and I can’t travel
anymore and I never got to go to Machu Picchu, bring me a headset and
give me a 3D video or tour of Machu Picchu, so I can feel like I’m
there. To me, that is truly amazing. And that’s what we’re trying to
provide.
Alan: That’s wonderful. So that
leads me to my last question. What is one problem in the world you
want to see solved using XR technologies?
Lorne: I think the most
impactful thing that XR technology can do is train people that save
lives, people that are in roles like firefighters or policemen, in
high risk scenarios – army’s definitely a huge one as well – any type
of role that carries a really high element of risk for real world
scenarios, and has the impact to potentially save lives. I think that
is where I’d like to see the technology used the most. If we could
leverage the technology to mitigate risk in those risky environments,
and at the end of the day, this technology is used to save lives, I
think that would be a beautiful thing to use the technology for.
36 min
September 09, 2019 Getting 110% Out of Training in 360° Video, with VR Vision Inc's Lorne Fade
The top of a wind turbine a hundred
stories up from the ground is not the best place to be making
mistakes, but making mistakes and learning from them is the whole
point of on-the-job training. That's why VR Vision Inc helps
companies produce XR training modules, so trainees can make mistakes
in a safe, controlled environment. COO Lorne Fade drops by to talk
about it.
Alan: Today's guest is Lorne
Fade, co-founder of VR Vision. Lorne is a serial entrepreneur that
has built several businesses over the last 15 years. He's had the
pleasure of working with some of the world's largest Fortune 500
brands and award winning marketing agencies all across North America
and Europe. His previous agency, Academic Ads, was acquired, and he
went on to found VR Vision Inc. As the co-founder and COO of VR
Vision, they're a virtual and augmented reality startup that's
enhancing immersive training outcomes for some of the world's largest
brands using VR, AR, and AI technologies. He's also the founder of
Reality Well, a healthcare technology platform to improve the quality
of life for those living in long-term care facilities. You can learn
more about VR Vision by visiting vrvisiongroup.com. Lorne, welcome to
the show.
Lorne: Thanks for having me,
Alan. Thanks.
Alan: My absolute pleasure, man.
We've known each other for quite some time through the VR/AR
Association in Toronto, and we shared some booth space together, and
it's always great to see what you guys are working on. I know the
last time we saw each other, you were showing me an automotive
manufacturing facility in virtual reality and how you were using
that. So let's dive in there. Let's talk about how you guys are using
VR and 360 video to make better training.
Lorne: Yeah, that's that's one
of our bigger use cases with Toyota, where we're training about
10,000 employees currently using 360 video, in immersive training
scenarios in VR. And it works really well for eliminating risk and
providing a safe environment with zero harm. And it's totally
immersive. So the employees that are getting trained in VR, no
distractions, they can't be on their phone or anything. It was really
simple the way we did it. We just storyboarded various scenarios with
Toyota on various processes, on safety concerns, on their assembly
lines or processes that were mundane and replicable. And then we went
out and filmed with a stereoscopic 3D camera, so when they put on the
headset they feel like they're there, fully 3D. And we mapped out, I
guess about two to three minute scenarios, various parts of their
assembly lines and filmed it all in full 3D and then ported it over
to VR, added some overlays, some voice overs, some touch points and
interactivity so that the employees could be trained in a completely
immersive environment. Nothing like this is, from my knowledge, has
ever been done before. So it's really cool to have this type
opportunity to work on a project like that.
Alan: So how are they measuring
success? For example, STRIVR is doing 360 video with Wal-Mart and
their key performance indicators. They're measuring training times,
how long it takes to train. They're also testing retention rates.
What are the KPIs that you and Toyota decided on, how to measure
that?
Lorne: Yes. Great question. We
developed a in-house analytics engine for tracking where the user is
looking, the various touch points of the training scenarios. And
every user that uses the platform gets their own log-in, so we track
each user, their effectiveness, and how well they're being trained
with the scenarios. And then within the scenarios, there'll be, let's
say, about 20 interactive touch points for various risks, or hazards,
or processes that the employee needs to learn. And then at the end of
this scenario, they'll get a breakdown or a test results scree
0 min
September 07, 2019Digital Vandalism, in the Name of Feminism – XR News for September 4, 2019
Tired of waiting for the US treasury
to put a female face on the $20? (Because you’ll probably be waiting
a while yet!) Google’s got you covered, or rather, your cash: they’ve
developed an AR app to put Harriet Tubman on the $20 bill. That’s
just one of the interesting bits of XR news Alan has in store for
this week’s news update.
A white paper published by Microsoft found that test scores among students using immersive technologies improved by as much as 22 percent, with the effect maintained over a period of weeks, including increased performance on skills-based tasks and gains in knowledge, abstract reasoning, and critical thinking.
In a new survey of 900 developers working on VR/AR, 33 percent of them were focusing on education, and 27 percent were focusing on training
Google puts women on US currency with an AR workaround
New 8-minute galactic primer “Clio’s Cosmic Quest” is the future of AR education
Fortum nuclear power plant uses VR for control room training
Fitz Frames allows kids to virtually try on and customize glasses with AR
US military using Hololens for planning ‘sandbox’
LA Dodgers embrace virtual reality batting practice
7 min
September 07, 2019 Digital Vandalism, in the Name of Feminism - XR News for September 4, 2019
Tired of waiting for the US treasury
to put a female face on the $20? (Because you'll probably be waiting
a while yet!) Google's got you covered, or rather, your cash: they've
developed an AR app to put Harriet Tubman on the $20 bill. That's
just one of the interesting bits of XR news Alan has in store for
this week's news update.
A white paper published by Microsoft found that test scores among students using immersive technologies improved by as much as 22 percent, with the effect maintained over a period of weeks, including increased performance on skills-based tasks and gains in knowledge, abstract reasoning, and critical thinking.In a new survey of 900 developers working on VR/AR, 33 percent of them were focusing on education, and 27 percent were focusing on training Google puts women on US currency with an AR workaroundNew 8-minute galactic primer "Clio’s Cosmic Quest" is the future of AR educationFortum nuclear power plant uses VR for control room trainingFitz Frames allows kids to virtually try on and customize glasses with ARUS military using Hololens for planning ‘sandbox’ LA Dodgers embrace virtual reality batting practice
0 min
September 06, 2019 Volumetrically Capturing Authentic Digital Actors, with Metastage’s Christina Heller
Even with all
the advancements in CG animation, it can’t capture that distinctly
lifelike essence that a real human exudes. But XR can capture that
essence — volumetrically. Metastage CEO Christina Heller drops by to
discuss the process of transcribing the aura of a person into XR
space.
Alan: I have a really special
guest today; Christina Heller, the CEO of Metastage. Metastage is an
XR studio that puts real performances into AR and VR through
volumetric capture. Metastage is the first US partner for the
Microsoft Mixed Reality Capture Software, and their soundstage is
located in Culver City, California. Prior to Metastage, Christina
co-founded and led VR Playhouse. So, between Metastage and VR
Playhouse, she’s helped produce over 80 immersive experiences. To
learn more about Christina Heller and Metastage, you can visit
metastage.com.
Welcome to the show, Christina.
Christina: Thank you so much for
having me.
Alan: It’s my absolute pleasure.
We met, maybe three years ago? At VRTO?
Christina: Yes, that’s correct.
Alan: Yeah, we got to try your
incredible experiences, mostly in the field of 360 video. And you’ve
kind of taken the leap to the next level of this stuff. So, talk to
us about Metastage.
Christina: Sure. As you said,
it’s a company that specializes in volumetric capture. I think, in
the future, you’ll see other things, but at the moment, we specialize
in volumetric capture. Specifically, using the Microsoft Mixed
Reality Capture system, which is an incredibly sophisticated way of
taking real people and authentic performances, and then bringing them
into full AR and VR experiences, where you can move around these
characters, and it’s as if they are doing that action right in front
of you.
Alan: Let’s just go back a
little bit. What is volumetric capture, for those who have no idea
what volumetric capture is?
Christina: Sure. For a long
time, if you wanted to put real people into AR/VR experiences, you
had basically two ways of doing it. You could either animate it; so,
you would try to create — using mo-cap and animation — the most
lifelike creation of a human character possible. Think, like, video
games; when you go play a video game and they’ve got a character
playing a scene out with you. If you wanted to put real people into
these XR experiences, that was the most common way to do it.
Then there was also volumetric capture,
which, for a long time, just wasn’t quite — I would say – at the
technological sophistication that people wanted, to integrate it into
projects. Volumetric capture — thanks to the Microsoft system, I
think — is finally really ready to be used in a major way in all
these projects. And basically what it does is, we use 106 video
cameras, and we film a performance from every possible angle. So,
we’re getting a ton of data. We use 53 RGB cameras and 53 infrared
cameras. The infrared is what we use to calculate the depth and the
volume of the person that’s performing at the center of the stage.
The RGB cameras are what’s capturing all the texture and visual data.
Then, we put that through the Microsoft
software, and on the other end of it you get a fully-3D asset that
really maintains the integrity and fidelity of the performance that
was captured on the stage. Unlike some of the animated assets —
because this was kind of the challenge — the animated assets, they
might get kind of there, but they had that uncanny valley thing
going.
Alan: Yeah, those are creepy.
Christina: Yeah. And so if
you’re not familiar with the term “uncanny valley,”
basically with people and animals – or like, dynamic, organic,
moving objects — if you get it kind of close, but not fully there in
terms of it looking lifelike, you have this inherent rejection of it.
You just… there’s this distaste, like “ew.” It’s called
“the uncanny valley.” It’s close, but it’s not really
there.
So, volumetric capture — and
specifically the captures we’re doing in Metastage — don’t have that
uncanny valley going on. When you look at them, they look like real
people. They maintain all of the nuances and the micro-expressions
and the subtleties of the person that performed on the stage. So you
can bring these fully authentic, fully real captures into your AR and
VR experiences. It just kind of brings the whole thing to life. So,
that’s how I would describe it. It’s volumetric video. It’s a video
asset, but it’s fully-3D, and you can easily integrate it with six
degrees of freedom into AR and VR experiences.
Alan: I’m going to break it down
even simpler: this means you can now step into the movie, and
participate in the movie as it’s going on around you.
Christina: Yeah, correct.
Alan: So exciting.
Christina: One of the things
that I’ve been saying is — part of the reason I’m really passionate
about volumetric capture as a tool inside of the greater medium is —
it’s the real person’s seat at the table. As we move more into these
virtual worlds, it’s important that real people are represented in
them. And when you watch something that was captured volumetrically,
you know that it was captured live. It wasn’t reanimated. It wasn’t
puppeted. This is something that you can watch with the same awe that
you would a live performance happening right in front of you. And I
think that that’s really important, that authenticity.
Alan: I really think we’ve come
a long way with computer and CGI and being able to animate things,
but there really is no substitute for a real actor or actress.
Christina: Absolutely. There is
something about humanity — and it’s part of the reason that
animators are struggling to get it as lifelike as possible — there’s
just something about the way people move, the way that they speak.
There’s just these little nuances that are impossible to fake. It’s
part of what makes watching something that was captured
volumetrically — or something in a film or a TV show — part of what
makes it so satisfying, is to capture all those little quirks, and
those little things that make people, people.
Alan: I think in a world where
AIs and robotics are going to replace a lot of our jobs, it’s nice to
know that we’ll still have some. [laughs]
Christina: Yeah. And I like
connecting with real people, and I like seeing real performers. And
when we’re talking about celebrities or public figures, or the CEO at
your company; I don’t want to watch an animated CEO give a board
presentation. I want to see the real guy. When it comes to anybody in
our society that’s “the real deal,” volumetric really is
the only real way to capture them for these experiences.
Alan: You touched on something
that is really, I think, interesting. You mentioned CEO
presentations, or investor presentations. We could talk about the
entertainment aspects of this — and we met up at the New York
Volumetric Filmmakers event and you spoke at that event, and I was
blown away by the stuff you guys are doing — but a lot of it is
creative and arts and entertainment, which are businesses as well.
But are there companies that are leveraging this technology now, to
broadcast their CEO or whatever? How are companies using this
technology now?
Christina: We’ve done a number
of B2B captures in Metastage, and I’m excited to do more of them.
It’s exciting to see other industries outside of the entertainment
field getting involved with XR, and starting to see how they can use
this tool to not only improve workflow and make money, but also just
to dazzle. That is an exciting opportunity right now, to just be
really ahead of the curve and do something that nobody’s done. You
still have that opportunity right now with augmented and virtual
reality.
Alan: You just mentioned
something: how to “dazzle” using this technology. I think
we always hammer down on “what’s the ROI? What’s the ROI?”
There is an intangible ROI, in the dazzle.
Christina: Absolutely, yeah. And
you got to make sure you partner with the right people, because
you’re not going to dazzle unless you’ve got the right production
team to do it. But if done right, I mean, that’s one of the most fun
parts of my job, is getting to watch people’s eyes light up when they
first see a person appear right in front of them — almost like a
human hologram — using volumetric capture. So, yes, we have done a
few different business applications, and one of them was specifically
for that use case that I just described.
We had an executive come in and do a
board presentation for his CEO on the Hololens. Basically, he came in
and he captured the beginning and end of his presentation,
volumetrically. He went out on the stage, we used 106 cameras, we had
a teleprompter. And basically, the presentation was about the
company’s plans to use technology to build their future, and how
technology was going to affect the future of their company.
By the way, I’m going to use the term
hologram to describe it, because I think that’s an easy way to wrap
your head around what you’re looking at. There’s questions of whether
that’s the correct term or not, but we’ll just call it a human
hologram, when it’s integrated into an augmented reality experience.
So he was using the Microsoft Hololens, which is some glasses that
you wear on your head, and it allows you to place digital objects
into the real world.
So we captured him holographically, and
using the teleprompter, he gave this intro to the CEO. Then it went
into a data visualization sequence. Hologram disappears. Now he’s
showing — using data in a three dimensional space — some of the
ways the technology is going to transform and affect their business.
And this, by the way, is a huge, huge, huge company. I’m not
sure whether I’m allowed to use it as a case study publicly, so I’m
being a little discreet, but a huge company. Anyway, he gives the
intro. There’s awesome data visualization showing how technology’s
going to transform their business. And then it ends with him coming
back and kind of wrapping it up. He said that, in the 35 years he’s
been working at the company, that this was the first time he’s ever
seen the CEO smile. So that was kind of a nice thing.
Alan: Wow. That’s incredible.
Christina: And it’s also now
preserved forever. He only had to do the presentation once. He can
now show it to anybody, anywhere. It’s this evergreen piece of
content that will live on. Incidentally, the executive that we
captured — who had been at the company for 35 years — is leaving
this fall. And so, in some ways, this presentation was his legacy;
talking about his dreams for how he wants the company to progress
when he’s gone. That was really cool.
And then also one thing we’ll do at
Metastage — which I think is always cool — is while he’s in there
— you know, this is a guy that has a family and some kids — I said,
“you know, well, while you’re here, why don’t we record
something for your family, too?”
Alan: Awww, I love that.
Christina: Yeah. You always feel
the energy shift when that moment happens. He got up there and said,
“it’s February 12th, 2019. And I want to tell my family this,
this, this, and this, and tell them how much I love them and how
proud I am.” And it was just this moment where he realized —
now, he’s not going anywhere; he’s not like a super old guy — but he
realized that this little piece of content might actually live on,
and be something that his family could cherish later. And so that was
also a nice moment. And that’s just one example. I’ve got more.
Alan: The preservation of
people, places, and things using volumetric is beautiful. I know a
mutual friend of ours — Simon
Che de Boer — he’s running around the world capturing places; he
does photogrammetry of places. If you take the photogrammetry that
he’s creating of these real places around the world, and you take the
videogrammetry that you guys are doing and put people into those
places? The possibilities are literally endless.
Christina: Definitely. And as a
former documentary filmmaker and journalist, making sure that real
life is preserved and a part of this new virtual landscape — I think
— is an important mission.
Alan: I agree. There’s places in
the world where we still have unrest, and cultural landmarks are
being destroyed. We have to — have to, have to — at least
get them as a digital [simulacrum] — you know, obviously, if we
can’t protect them physically — if we can get a digital version of
them… the fire in Paris is a great example of that. Notre Dame
Cathedral. They have rudimentary LiDAR scans of the building; they’re
not perfect, but they can recreate the building digitally, and then
use those three-dimensional drawings to recreate the actual building.
Christina: Yeah.
Alan: It will never be the same,
but at least they can get close.
Christina: Yeah. And with
volumetric capture, you don’t have to be as concerned that… first
of all, it’s really, really easy process. You don’t have to put on a
mo-cap suit or points, and go out and make a bunch of different
facial expressions. Super high-res face scanning, with the purpose of
being animated later, is a really, really intense process.
Alan: Oh, my goodness. People
don’t realize. It’s a full day, just to be able to say “hello.”
Christina: Yeah, it’s a full
day, and it’s really intense on the performers. Volumetric capture is
super easy on the talent. You just go out on the stage, action, cut,
and you’re done. Off you go.
On our end, we like to take a little
more care than that. We’ll do some tests to make sure your hair looks
right, or your clothes look right, and all of that. We’ve done
celebrities at Metastage that we’ve had a very, very limited amount
of time with. But as long as we can give them a once-over to make
sure they’re volumetrically friendly, they can go out and be off in
no time at all. And you have the added comfort of knowing that this
isn’t going to be something that is puppeted and rigged to say things
that you didn’t want to say. This is really you. It’s capturing you
and making sure that you are coming across in the way that you
actually want to in your real life. It’s a preservation technique.
It’s a performance tool, but it’s nothing to be nervous about. That’s
one of the things that I want to make sure gets across, because I can
imagine an actor, for instance, getting nervous about the increased
digitization of actors.
Alan: “What are you going
to do with my avatar?”
Christina: Exactly.
Alan: It’s interesting, because
some friends of mine, they own a company and they do photorealistic
avatar creation. And they have a side business producing adult
content avatars.
Christina: Right. You don’t even
have to go much further than that to understand how that could cause
some pause for an actor that takes pride in the work they do and how
they do it. Volumetric capture really is video, but it’s a fully
three-dimensional video. That’s a key differentiator.
Alan: So what are some other use
cases that you’re seeing pop up for this type of technology?
Christina: Well, one of the
great projects we did this past fall — and I can talk about it — is
we did something with the CEO and president of the Royal Caribbean
Cruise Line. They’re giving you a virtual tour of the ship. And so,
this has some great and broad applications for a lot of businesses
that, maybe, do a lot of onsite tours. One of the great things that
VR and AR can give you is access. Access to places you can’t normally
go. Access to people you couldn’t normally engage with.
So, there’s two ways of doing the
virtual tours. But the way that the Royal Caribbean did it was the
CEO and president give you a tour of the ship using the Royal
Caribbean Celebrity Cruise app. You can basically make their
holograms appear in different rooms of the ship, and they tell you
about the design, the features, how the ship was built, and why it
was built the way it was built. That is an app that anybody can
access. If you type in “Celebrity Cruise app,” you can
access the CEO’s intro when you’re not on the ship, and the rest of
the holograms can only be accessed on the ship. So, it’s kind of this
cool site-specific augmented reality experience.
Alan: That’s really cool. So,
you have to be on the ship to actually experience the full thing?
Christina: Exactly. You can see
the intro — which is really cool, and you can get an idea for it.
Richard pops up, and he’s holding a model of the Celebrity Cruise
ship in his hands. And he says, “we built this ship using 3D
technology — the most advanced 3D technology. And so we thought it
was only appropriate to use the most advanced 3D capture technology
to explain to you why we built the ship and some of the features.”
Alan: Amazing.
Christina: At that point, he
puts the ship down, and you can explore a little bit of it on the
app. But the rest of it, you have to be on the ship to experience.
Alan: Is this an AR app for your
phone?
Christina: Yes, it’s an AR app
for your phone. Another fun selling point of the Microsoft system and
Volucap system that we use at Metastage is that, the assets are
really, really beautiful, and also super small file sizes, so you can
actually activate them using a mobile device. So yes, using your
phone, you open the app and like magic, he appears right in front of
you, and you can walk around him as if he’s standing right there. He
integrates fully into the scene.
Alan: Incredible. Can you just
send
me the link
to the app and I’ll put it in the show notes, for anybody who
wants to give it a try.
Christina: Yeah, totally. So,
that’s great because it gives people access to Richard and Lisa, who
would never get access to them normally. And from the CEO side of
things, it allows them to reach their customers in this really
intimate and friendly way, without actually having to go out and
shake everybody’s hand. For businesses that maybe do a lot of tours
on site, but would like to give access to more people without
actually having to take the time, energy, and resources to give them
the physical tour, you can do a capture of the facility or the
warehouse or wherever it is, and then integrate your CEO or star
employee or whatever into that environment. And you can give somebody
a realistic and authentic virtual tour of that place without having
to — like I said — dedicate the time and resources of actually
showing them in person.
Alan: You know, this comes back
to something that pops up on every single episode of the show;
training. Immediately, when you said that you could give people a
tour. Imagine: for new employees, working on a cruise ship must be a
daunting experience. The training just to train people where things
are on the ship, it’s got to be incredible to do that. And one thing
that I think Metastage and volumetric capture will really drive home
is the fact that some people are really, really great at training,
and some people are not so great. Maybe they’re great at creating the
content, but not presenting it. Now you can have the best person
train every single employee.
Christina: Exactly. You can get
your star employee to walk them through the process, show them
physically how to do it – which, depending on what field you’re in,
being able to show somebody with one’s body how to do something can
be crucially important. And you’ve immortalized them. That employee
may move on to another position, but you’ll always have that spirit
and that knowledge captured for years to come.
Alan: Employee on-boarding,
training; but I also love this idea of being able to download the
app, see it, it says, “well, the rest of it is on the ship.”
I think there’s so much that can be done with this type of capture.
We’re only really just scratching the surface.
Now, Metastage is not the only
volumetric capture system out there, correct?
Christina: Correct, yeah.
Alan: There’s 8i. There’s what,
Intel Studios? Maybe some other ones that we don’t know about. But I
think there’s more companies realizing the potential of this. What
sets Metastage apart — in my opinion, it’s obvious, because I’ve
seen the results — your partnership with Microsoft really sets it
apart. I’ll let you speak to that.
Christina: Sure. A lot of
volumetric capture stages, from just the outside appearance, will
look similar. You’ll go in and you’ll see a bunch of cameras, all
facing inward at a stage, so it can look on the surface like they’re
all the same. But the truth is, the real magic is in the software.
What the Microsoft Mixed Reality software does better than any other
volumetric system on the market is not only create really clean,
high-fidelity captures that look great in the body, in the face, and
don’t have a lot of artifacting, and look good from every angle that
you happen to be viewing the captured asset from. We are also able to
compress those captures to super tiny file sizes, which — if you’ve
ever tried to make a project for virtual or augmented reality — you
know how important that is.
If you’re making a training app for
your employees, or you’re doing something like the Royal Caribbean
cruise line, you can’t have a 10-gigabyte application when this whole
thing is done. It needs to be something small that isn’t going to
take up a ton of room on their phone and that’s easy to distribute.
There’s other things, but that alone is the key. Ask any other
volumetric caption stage how big their final file size is. Microsoft
has just really got it down to some really workable file sizes. We
know that you’re doing it — let’s say for the HTC Vive or the Oculus
Rift, or you’re trying to do it for a mobile phone device — we can
export at different settings to optimize for mobile AR. Or if we can
push it a little more for VR, then we can add a little bit more
quality. But regardless, like the Royal Caribbean cruise app, when
you download it and you see Richard, he looks really fantastic. So we
can get our files sizes down to… for a minute of volumetric
capture? At 50 megabytes. That’s 5-0 megabytes.
Alan: Holy crap.
Christina: Yeah.
Alan: 50 megabytes.
Christina: For a full minute of
capture. And then it scales up, depending on our export settings and
what your final platform is. That’s pretty incredible.
Alan: It is really amazing. I
think what people don’t realize is that when you’re pushing out this
type of 3D content for a phone, for example, it doesn’t actually have
to be as high-res as you would think, because you’re already looking
through a high-res screen at another image in 3D space. It has to
look clear, but people are like, “oh yeah, I need 4K video in
AR.” Whoa, wait a second. You’re looking at a screen within a
screen. The maximum of it is really only going to be 720p. So there’s
some little tricks that people don’t realize.
Christina: Right. And I show
people the Royal Caribbean app all the time, and they think that
looks better than any volumetric capture they’ve ever seen. And
that’s at our smallest export setting.
Long story short: quality at low file
sizes is the key difference between Metastage and other capture
facilities. But there’s also some added benefits. For instance,
there’s the Microsoft toolset that we deliver along with our
captures, which includes gaze retargeting. One of the things is, when
you’re watching biometric capture — because it’s an authentic
performance of what happened — if I’m watching it, and I step to the
right or the left, it might look like the person is looking past me.
So gaze retargeting will allow the head to subtly follow the viewer.
If you’re trying to make it look like the capture is looking at the
person watching it, there can be a subtle gaze retargeting where the
head just sort of follows the viewer. That’s a standard tool that we
deliver with our captures to the client, along with some relighting
tools for game engine. Almost like… I want to call them “Instagram
filters” for the capture, that allow you to give a little bit of
a dramatic look, or sunset lighting. Those all go along with it as
well.
Beyond that, we offer full end-to-end
project integration. When we first opened, we were offering just holo
capture. But it’s become clear that for some of our clients, they
don’t necessarily have access to game engine developers or
environment creation. So if you come to Metastage and you’re
interested in doing this kind of project, we can produce the project
from conception to completion for you. Or, if you’re a production
company or agency that already has access to those professionals,
then we can simply just do the holo capture and deliver that to you.
Alan: That’s incredible. So, I
know there’s a lot of people listening that are probably thinking,
“oh man, I want to use this. I want to jump in.” Let’s talk
about price. I don’t know if this is something that’s really
expensive. What does a minute of footage cost to develop? What does
the process involve? If somebody wants to dive right in and say,
“yeah, I want to host a two-minute video in AR for my
shareholders,” what would something like that [cost]?
Christina: Well, the prices
start around $15,000, and then it goes up from there, obviously. It
is — I would say — very comparable to what commercial video
production rates are, if that’s something you’re familiar with. And
it’s much, much cheaper than mo-cap and animation.
Alan: To put things in
perspective, to create a photorealistic digital avatar — rig and
everything — you’re looking at, what, $10,000 a second?
Christina: Yeah. I mean, it
depends on the team you work with. It depends on how detailed you
want the face and head to look. But I would say that that’s the price
it costs to get through the door and get something going. And then
obviously, the more content you’re trying to capture, the prices go
up from there.
Alan: I’m really excited. I know
you invited me for a tour of the stage. I haven’t been to LA yet
since we talked about it, but I definitely want to come down and
check it out.
The other thing that I noticed about
Metastage — and I watched the video that you showed — is that it’s
designed like a studio. It’s designed to be in congruence with what
actors and actresses and people doing professional video are used to.
Christina: Absolutely. That is
important, and our production-savvy client-facing staff want you to
have a seamless client experience, and a fun production day; that at
the end of it, you say, “oh my gosh, that was really fun and
easy. I’d like to do more of that.” And so, “easy and fun”
has been kind of a mantra at Metastage since the beginning, and I
think we’ve been successful with that. All of our clients have had a
really good experience, and then on top of it, were surprised when
integrating the assets that it was as plug-and-play as it is.
So yeah, if you’re interested at all,
please don’t hesitate to shoot us a line. You can contact us on our
website and we’ll do consultation with you. We’ll hold your hand.
We’ll work to get your goals achieved using this, and hopefully
something that will be great for your business for years to come.
Alan: Well, is there anything
else that you want to share before we wrap this up? It’s been a great
interview and I can’t wait to record a message. You know, you
mentioned about the family, and I’ve been thinking of it ever since.
I just want to record not only myself, but my children at their age
now. Because as they grow up, you’re never going to get them at this
age again. And it’s like capturing them in a time capsule.
Christina: Totally. And that’s
one thing we have been talking about, having a different pricing
model for something like that. I think that’s really important. I
totally understand the desire to want to capture your family; almost
like a family portrait in this really interesting, three-dimensional
way, that you could then stand next to later and marvel at the
changes. Long story short, I do think that at some point we will have
a way for average people to come and capture their families, as well.
We’re just figuring out exactly how that works with our professional
soundstage.
30 min
September 06, 2019 Volumetrically Capturing Authentic Digital Actors, with Metastage's Christina Heller
Even with all
the advancements in CG animation, it can't capture that distinctly
lifelike essence that a real human exudes. But XR can capture that
essence -- volumetrically. Metastage CEO Christina Heller drops by to
discuss the process of transcribing the aura of a person into XR
space.
Alan: I have a really special
guest today; Christina Heller, the CEO of Metastage. Metastage is an
XR studio that puts real performances into AR and VR through
volumetric capture. Metastage is the first US partner for the
Microsoft Mixed Reality Capture Software, and their soundstage is
located in Culver City, California. Prior to Metastage, Christina
co-founded and led VR Playhouse. So, between Metastage and VR
Playhouse, she's helped produce over 80 immersive experiences. To
learn more about Christina Heller and Metastage, you can visit
metastage.com.
Welcome to the show, Christina.
Christina: Thank you so much for
having me.
Alan: It's my absolute pleasure.
We met, maybe three years ago? At VRTO?
Christina: Yes, that's correct.
Alan: Yeah, we got to try your
incredible experiences, mostly in the field of 360 video. And you've
kind of taken the leap to the next level of this stuff. So, talk to
us about Metastage.
Christina: Sure. As you said,
it's a company that specializes in volumetric capture. I think, in
the future, you'll see other things, but at the moment, we specialize
in volumetric capture. Specifically, using the Microsoft Mixed
Reality Capture system, which is an incredibly sophisticated way of
taking real people and authentic performances, and then bringing them
into full AR and VR experiences, where you can move around these
characters, and it's as if they are doing that action right in front
of you.
Alan: Let's just go back a
little bit. What is volumetric capture, for those who have no idea
what volumetric capture is?
Christina: Sure. For a long
time, if you wanted to put real people into AR/VR experiences, you
had basically two ways of doing it. You could either animate it; so,
you would try to create -- using mo-cap and animation -- the most
lifelike creation of a human character possible. Think, like, video
games; when you go play a video game and they've got a character
playing a scene out with you. If you wanted to put real people into
these XR experiences, that was the most common way to do it.
Then there was also volumetric capture,
which, for a long time, just wasn't quite -- I would say – at the
technological sophistication that people wanted, to integrate it into
projects. Volumetric capture -- thanks to the Microsoft system, I
think -- is finally really ready to be used in a major way in all
these projects. And basically what it does is, we use 106 video
cameras, and we film a performance from every possible angle. So,
we're getting a ton of data. We use 53 RGB cameras and 53 infrared
cameras. The infrared is what we use to calculate the depth and the
volume of the person that's performing at the center of the stage.
The RGB cameras are what's capturing all the texture and visual data.
Then, we put that through the Microsoft
software, and on the other end of it you get a fully-3D asset that
really maintains the integrity and fidelity of the performance that
was captured on the stage. Unlike some of the animated assets --
because this was kind of the challenge -- the animated assets, they
might get kind of there, but they had that uncanny valley thing
going.
Alan: Yeah, those are creepy.
Christina: Yeah. And so if
you're not familiar with the term "uncanny valley,"
basically with people and animals – or like, dynamic, organic,
moving objects -- if you get it kind of close, but not fully there in
te
0 min
September 04, 2019 Lighting the Torch for In-App AR Development, with TORCH’s Paul Reynolds
Game engines
like the versatile Unity have long been the go-to for AR development,
and for good reason. But its reputation as a video game engine can
also be intimidating — especially to those who want to create AR
software for enterprise. That’s why Paul Reynolds lit his TORCH; an
app he co-founded that lets you design your own AR platform, right in
the palm of your hand. He chats with Alan about his claim to flame.
Alan: Hey, everyone, my name’s
Alan Smithson, the host of the XR for Business Podcast. Today’s guest
is Paul Reynolds, the CEO of Torch, a really exciting augmented
reality platform. It’s a mobile augmented reality development and
deployment platform for enterprise. Paul has been a software
developer and technology consultant since 1997 – since before the
interwebs! In 2013, after 10 years of creating video games, he joined
Magic Leap where he was promoted to senior director, overseeing
content and SDK teams. At Magic Leap, Paul recognized the lack of
accessible tools for non-game developers that was hindering
widespread adoption of immersive and spatial computing technologies.
In 2016, Paul moved to Portland, Oregon, where he founded Torch to
address this very problem. To learn more about Torch, you can visit
torch.app. Paul, welcome to the show.
Paul: Thanks for having me.
Alan: It’s such a pleasure. I’ve
been looking forward to this episode. Torch is such a cool platform
and I keep seeing your posts on LinkedIn of putting stuff around your
office and stuff. So tell us, what is Torch, and how did you come up
with this crazy idea?
Paul: The easiest way to think
about it is, it’s a mobile application — currently for iOS — that
lets anyone build interactive spatial scenes. So, you create a
project and you’re building it in the camera of your device, which
means you’re also walking around the space, or moving around the
space and you’re building up interactive experiences visually,
without writing any code. We call that the design environment, and
that’s the freely available [option] — anyone can jump into it and
just start building. What makes it a platform is the capability of
taking what you’ve created in Torch, and exporting it and publishing
it and integrating it into your existing app, or pushing it out to
another platform or tool. What we really wanted to focus on was
allowing people to iterate in augmented reality — directly within
augmented reality — as opposed to sitting on a desktop computer and
trying to figure out how to work a game editor, and get more people
able to work productively in 3D. That’s really the heart of it.
Alan: That’s so cool, because if
you’re sitting at your office, you’re like, “wow, this AR stuff
is hot. It’s amazing.” You know what, go learn Unity and coding
and figure out how to actually make it. Six months later, you’re
like, “oh, look, I made a portal.”
Paul: [laughs] Right.
Alan: What you guys have built
is a simple way to just do it visually.
Paul: Yeah. So, my background
was in video games, back in the day where everyone was building their
own engine. You really didn’t even have time to build a really nice
editor on top of that. So when Unity came out — we’ll pick on Unity
in particular, because it’s just such a well-known product — when it
came out, it was the game engine that most of these game studios I’ve
worked for wanted to build. And that was really unique; they had
basically taken what would normally mean millions and millions of
internal R&D dollars, and turned it into this tool that pretty
much anyone can download for free. But what happened over the past
few years is, it’s become kind of the de facto interactive 3D tool.
And it was for me as well; I’ve been a Unity user forever. What we
learned when we started building a platform at Magic Leap for third
party developers that are not necessarily all looking to build video
games — so, enterprises and brands — the conversation there was
first, somebody in your team needs to learn Unity to even get
started. Just like you said.
Alan: And this like a red flag
for these companies because they’re like, “what the hell is
Unity?”
Paul: Right, yeah.
Alan: And then they ask their
tech teams and they’re like, “does anybody know Unity?” And
nobody knows what it is. Then they look it up, they’re like, “oh.
Yeah, well, that’s for video games.”
Paul: Right! That’s exactly
right. And the disconnect that has been — coming as a veteran game
developer — a young kid comes up to me and says, “I want to get
into making video games.” I’m gonna say “download Unity —
for free — and take the tutorials, and you’ll learn just as much
about real professional video game development as anyone else that’s
actually out there as professional video game developers.” So,
in our world, Unity — and let’s not just pick on Unity all the time,
you know, there’s Unreal. There’s these other engines–
Alan: Improbable.
Paul: Yeah. For us, they’re the
easiest way to get in. But then you have to remember — what I saw
firsthand — was most of our target market for Torch, and most of the
people I was dealing with at Magic Leap, they’d never heard of Unity.
You’ve got to imagine there’s a much bigger world out there besides
ours, that people don’t even know where to get started. I said,
“look, the world is going to move towards spatial computing,
where computing is going to be in our world, and there’s 3D input,
there’s 3D display, and there’s cameras involved. And we’ve got to
come up with more accessible ways to build software and experiences
for these platforms, that doesn’t just come from video game
technology.” (All that said, video game technology certainly
plays an important role in all of this.) We said, “well, how are
people building mobile apps today? How are web apps getting built?”
Businesses are run — and new businesses are created — on top of
these platforms. How do people build software — in 2018 at the time
— what’s their workflow? The workflow is very design-oriented and
very visual, at first. Instead of us building a piece of functional
software, and then handing it to a designer and say, “hey, make
this look nice and usable.” The mobile world in particular has
put a lot of investment and flipped it on its head and said, “let’s
make a functional prototype first — that everyone can all agree
around — and say, yeah, this is the thing we want to build,”
and then you engineer around it.
That was ultimately how we arrived at
the current version of Torch, where we wanted to fit in with that
enterprise workflow. And we say, “well, you’ve already used this
idea in mobile apps in particular: prototype first, in-design first.
So that’s what we built for AR; anyone can just jump into this and
start putting 2D assets in video. We help you find 3D models. We have
the requisite Sketchfab and Poly integrations. You can find 3D
models. We’ve got Dropbox integration. Even though anyone can
download it and start playing with it and working in it, we’ve tuned
the integrations and the workflow around your standard UX designer,
or a creative person at an agency, or someone that’s already building
digital products that are enterprise, and we’re saying “here’s
how you can start to test out, and share, and show some of these
early ideas around augmented reality.” And as you get more
comfortable with what’s good and bad in mobile augmented reality in
particular, we’ll be there for you to help you get that deployed, and
help you get that integrated, and turn it into a real product.
Alan: So it’s really a
prototyping tool then.
Paul: So… it’s funny; it
started strictly as a prototyping tool. Anyone can jump in and build
something. The only thing you can really get out of it, was you could
record a video — which is still very important in the screen-based
world we’re in still, to be able to share an AR experience through
video. But to add a collaborator, or have someone actually view what
you’ve built in real time, you had to add them as an editor. We have
a Google Doc-style, real-time collaborative editing, where you can
have as many people as you want in a project, but you’re also giving
them permission to edit the project.
Alan: That’s the last thing you
want. [laughs]
Paul: Yeah, yeah. We’ve had
people ask, “how many people can I add? Technically, how many
collaborators can you handle?” And the answer is always,
technically way more than you actually want collaborating, anyway.
Alan: No kidding.
Paul: We’ve had upwards of, I
think, 50 people on one project before. The way we do the
collaborative stuff, it’s not very taxing computationally. So, you
can have tons and tons of people in there. After being out for… I
would say, two or three months, we started seeing this trend, where
people were coming to us and saying, “Torch is great. It’s the
first time I’ve been able to actually feel like I’m creating in AR; I
can pull in my assets” — and these are all professionals and
they’re very limited on time — and they’ve said, “but how do we
get this out of Torch? Where does it go?” Keeping in mind, we
were patterning ourselves after — back to the mobile app world —
where people may prototype in something like Sketch and InVision and
Figma — these functional prototyping tools — and Framer. And then,
once they decide to build the production version, they usually go to
code; they’ll build it in React or whatever. And so originally, our
thought was, well, we’ll get the creative people iterating visually
in our prototyping tool, and that will inform the production team.
So, these people will probably wind up building this stuff in Unity,
or Spark, or Lens Studio — whatever the tools are at the time. But
there’s this moment of handoff. The interesting thing in hindsight is
kind of obvious, which was what I said earlier: a lot of the people
that found us, don’t even know what Unity is.
Alan: So you’re like, “oh,
wait a second, we’ve built this tool for you. And yeah, you can use
it. But then you got learn… yeah. Oh, oops!”
Paul: Yeah. And on top of that,
the things people are building today are fairly simple. Even the
people that knew what Unity was, they’re like, “wait, you’re
telling me I’m going to rebuild this thing, but it already works in
Torch? So why can’t I just get it out?” And that was always on
our longer-term roadmap. But we’re like, okay, this is obviously the
direction we need to support, because now we have this very
accessible, extremely fast workflow, that you can use as just a
prototyping, pre-visualization environment. But we started adding
more and more functionality to it, that would allow you to build
full-blown experiences. It’s just a totally alternative augmented
reality workflow, that people could use depending upon their use
case.
And the other part of it was we were
like, today — and still, this is true in the moment — right now,
today, where is all the engagement, and — quite frankly — revenue,
and where’s the rubber really meeting the road? In particular, with
mobile augmented reality? And the obvious answer to me was social:
Snapchat, and Facebook, and Instagram. They are pumping out a lot of
augmented reality content. They’re making real revenue from it.
They’re getting huge engagement. People are building lenses and
filters that are getting hundreds of millions and billions of views.
And brands are trying to figure out how to get into that. So we said,
“well, those are pretty simple experiences.” We don’t do
anything with the face in Torch; we’re fully concentrated on the
world-facing camera, and the world-based experiences on those
platforms are fairly simple. And we said, “well, if we gave
people a few more authoring capabilities beyond just prototyping —
let them hook up some more interactivity and call APIs and things
like that — and then we allow them to publish out to these
production platforms. We’re kind of filling a need in a lot of ways;
filling a very fragmented ecosystem. Because the other part of this
was,we kind of addressed the initial friction — for any enterprise
that’s probably listening to this podcast, and they’re trying to
figure out, “how do we even get started?” — we are trying
to be there to say, “we are literally the fastest and easiest
way to get started.” And we feel like we’ve fulfilled that
promise pretty well.
But then, the other part of it is on
the outside of the creation; it is the distribution and deployment.
And if you look at that, it’s very fragmented. If you want to build
for Snapchat, you’ve got to use Lens Studio. If you want to build for
any of the Facebook properties, you’ve got to use Spark. If you want
to build a fully-dedicated AR app, you’ll probably use Unity. If you
want to embed an AR experience in an existing mobile app — which is
a very popular request we hear — you’ve either got to write to
Google Sceneform or Apple’s ARKit, or kind of hack Unity into your
mobile app. It’s all very fragmented by the tool. So, we’ve
eliminated the fragmentation at the early prototyping part, but
there’s still all these crazy problems on the distribution side. In
the New Year, in 2019 — and most recently — we’ve put a lot of
effort in letting you publish and export out. The most exciting thing
we announced around that was around the time of AWE – Augmented World
Expo – we announced Torch 3, which lets you create a public link to
your project.
Alan: Oh, cool.
Paul: Yeah, it opens in the
Torch app, as a viewer-only mode. So, you’re not adding people as
collaborators. We don’t require the viewer to log-in or register.
They do have to download the Torch app. But it’s a very quick, fast
way to say, “hey, I built something in AR. I want to tweet it
out. I want to put it on LinkedIn,” or, “I want to send it
to my client, or my internal team.” And people can very quickly
iterate and view it, in real-time interactive 3D.
The other thing we announced is our
ability to export a Torch project into another project. Our
demonstration of that was, we were able to generate a Spark AR
project from Torch. So — for Mother’s Day — we built a little
Mother’s Day experience in Torch, but we actually published it
through Facebook–
Alan: Cool.
Paul: –through the Spark
export. So, that’s where we evolved beyond just prototyping, and
[became]
kind of a creative tool.
Alan: It’s awesome that you
didn’t set out with that intention, but you ended up there.
We always knew that we would start with
designers and grow a platform around that. I think what happened was
we mapped to the mobile ecosystem — which is very mature — and the
AR ecosystem is still growing and maturing. And if this were a much
more established market, we probably could have built a pretty tidy
business, just being considered the AR design tool. But we saw these
bigger opportunities for filling in these gaps in the ecosystem. So,
in some ways, it’s where we expected to go. But in other ways, we got
there a little quicker than we had originally thought we should or
would. And it’s been very well-received.
Alan: You’ve been working on
this for a bit. How are businesses using this right now?
Paul: It’s kind of across the
board. We’ve got people building Torch projects as internal tests or
prototypes. But we have had people — and especially now that we just
turned on this ability to publish and export; it’s technically under
early access right now — we’ve been giving it out to people, so
people are still just getting their heads wrapped around what they
can deploy and what they can do with this capability. We’ve seen
people use it for wayfinding — a super popular use case for us —
because if you build content in the environment using a mobile
device, building in a wayfinding experience — where you’re actually
setting the checkpoints on the wayfinding experience physically in
the environment — is just so much more intuitive and fast. For
example, we built a couple wayfinding demos for Torch that have
always gotten huge response online when we post videos. And… it’s
really funny, because the first time I posted a wayfinding demo, of
how to get from the front door of our office building to the front
door of our office–.
Alan: I saw that it was cool.
Paul: It’s funny, because I’ve
had people get to our office by saying, “I watched the video,”
or “I remember the video and I just remembered how to get here,”
which was kind of fun. But the funny thing was the divide in my
super-savvy AR friends who’ve been in the business as long as I have.
They thought we had scanned the building, or done complex
measurements, because the wayfinding experienced in that case
actually goes across two floors — two levels of the building — and
they’re like, “what did you do? Did you scan it in, and then
bring it into Unity?” I was like, “I literally stood at the
front door and I placed the welcome checkpoint, and then I walked to
the bottom of the stairs and place that.” We actually priced
that out, and if you were to build it with the traditional
desktop-based workflow, you’d have a team of people working on it,
and we estimated it would take roughly $100,000 or more, a team of
four people, and probably at least a few weeks to get it something
workable and viewable. And I built it in about 45 minutes on a lunch
break, and probably total cost — including recording the video and
buying a couple 3D models — we were into it for just a couple
thousand bucks. We’re talking about radically transforming the cost
of building these experiences. So, wayfinding’s a great example of
the cost efficiencies brought in when you actually build AR
experiences inside of AR, like we do.
As far as vertical markets, we’re
seeing a lot of interest around eCommerce, obviously — in shopping.
But also physical retail. People are really interested in bringing a
layer of digital experience into the retail environment. Travel and
hospitality has been very engaged. Media companies. One of my
favorite examples — I will qualify it with they are not using Torch
yet — but ABC News Australia. I don’t if you follow Nathan Bazley on
Twitter, but he posts these great little infographics in AR that
they’ve built. We actually reached out to him and talked to him about
his process. And ABC Australia is kind of like the BBC; they’re a
government-funded media company, and two or three years ago, they saw
AR as an interesting way to present information and to engage. And
they actually built an Unreal engine-based app to publish news
content to go on with their news. It was so expensive and difficult
to update the app, every time they had a news story! Still, kudos for
even getting it out, because that’s a huge leap, and no telling how
much they spent.
Alan: Oh my God. In the hundreds
of thousands.
Paul: Yeah. At least. Right? But
they did see engagement. So, when Facebook came out with Spark, they
said, “most of our audience is on Facebook and Instagram, so
this is a great distribution for us. We could try this again.”
And they taught themselves Spark. So now they have a little team of
— I think — four people, two or four people, that build these
little AR experiences that go along with that news story. And Nathan
was telling us — and I think I’m quoting this correctly — when they
put out a news video on those social channels, it gets around
30,000-50,000 views. And when they would put out a world filter, or
AR-based experience, through the same channels, they were getting
hundreds of thousands of views, highly engaged. What’s exciting for
us is that they’re getting this value through this one workflow. So,
what we can offer them — as an example — is, well, how about
everyone in your newsroom can start creating AR experiences to go
along with their stories, instead of running through this really
small team that’s taught themselves Spark? That’s what we think Torch
can provide, is that accessible workflow. And by the way, if you’re
already putting together the experience and these assets, and you’re
building this AR thing, why not deploy it everywhere AR can be
viewed? Why not push it to not just Facebook, but possibly Snapchat?
But also your own mobile app that people have installed? And what
about wearables? Giving people this flexibility and freedom of
publishing is something that we’re seeing resonate with, like, media
companies; people in the book industry, and film and television.
Alan: You don’t want to make
things twice, that’s for sure.
Paul: Not when it’s this hard
and expensive — and experimental — for a lot of people.
Alan: No, you’re right. So
you’ve been working on this for a couple years now. You were Magic
Leap before. What are some of the experiences that you’ve seen —
either made on Torch or otherwise — that have just kind of blown
your mind?
Paul: I mean, obviously, my
early days and Magic Leap, it was really where I saw incredible
things I’d never seen before. That was when I became convinced that
spatial computing was coming. Some of that stuff is now public. Now
that they’ve released the device and some of the content, I was a
part of the org of that company that built the Dr. G, game where
they’re shooting robots.
Alan: Yeah. Cool to see that for
the first time.
Paul: And we’re talking 2014?
That was pretty cool, but it’s only gotten better. Obviously, most of
my most crazy experiences have been around that tech in particular,
just because we used a bunch of different hardware and stuff. The
Magic Leap one that just shipped last year is certainly the most
consumer-friendly version of the tech that we’d worked with. But
there were some demos and things that I saw that were just very
unusual, where people are claiming that they feel temperature changes
on their hands when a little firefly-type robot flies up to their
finger, or they actually have a sense of weight of an object because
of the way the optics were kind of showing stuff in true 3-D. So some
of that stuff was pretty mind-blowing.
I’m trying to think of my most
recent… I really think, because I’m so in the weeds and I always
look at the technical execution of stuff, both Apple and their quick
look stuff that they’re showing, where they’re actually real-time
generating shadows, light estimation and reflections. And I don’t
know if you’ve seen it, but you can bring in, like, a shiny toaster,
and you can wave your hand past the virtual toaster, and you can
actually see your hand reflecting in it.
Alan: How are they doing that?
Paul: They’re building on an
environment map in real time. And so as you’re moving around–.
Alan: What, are they using the
camera to capture the environment?
Paul: Yep.
Alan: Oh, my God. That’s
amazing.
Paul: Yeah, it’s really cool. To
be fair to everyone else, Apple’s very tightly coupled to the
hardware. Everything is super optimized, and they can actually do
these things because they’re in control of the hardware. It’s a
little more difficult to do in a very cross-platform type of context.
I really liked the… I want to say “wonderscape,” but I
think it’s “wonderSCOPE…” the books–
Alan: Oh yeah. Those are cool.
Paul: Yeah, I like to show
those. The other part of it is, is the experimental side. I follow a
lot of the Snapchat lens and the Instagram creator community. There’s
some folks like Zach Lieberman and Max Weisel — they’re all doing
really interesting stuff, and they don’t have a commercial
motivation. They’re just seeing what weird things they can do with
this technology. And I really think that’s where we’re seeing the
most creative stuff come out. Zach Lieberman in particular has an app
called… Weird? Is it Weird Type for ARKit? It’s like a toy type of
thing, where you can walk around and place words, and have them react
to your movements. And that’s always pretty fun to show people is
really cool.
Alan: Have you tried Babble
Rabbit?
Paul: Yeah. That’s Patrick
[O’Shaughnessey]’s. His baby, right? That’s running on top of 6D?.
Alan: Yeah, exactly.
Paul: Yeah. We’re buddies of the
6Ds guys.
Alan: Actually, Matt
[Miesnieks]’s been a guest on the show.
Paul: Oh, nice. Yeah. I’ve known
him for a while. We actually announced that we integrated 6D into
Torch as a proof of concept.
I’m pretty excited about both occlusion
and persistence. I think persistence really changes the game for a
lot of people. Once people start to get their head around — again, I
go back to our target market; they’re so new to this world — I’ve
shown people in real time; I’ve built a project in Torch on an iPad
in front of them, and I’ve built a little scene on top of a table.
And even then, they still don’t quite get that they can walk around
that thing that I’ve put out into the world. They think it’s like a
static [image]; like, a 2D thing on top of video. They just… people
still have not unlocked the spatial part of their mind with this
stuff. Which is crazy, because we live in a 3D world, and 3D people.
When you do see people get past that initial understanding of what’s
going on, then their assumption is, “oh, well, when I build
something, or place that in the world, it’ll just stay there, and
somebody else walks in the room, they’ll see it. And if I leave the
room and come back tomorrow, it’ll be here tomorrow.”
Alan: It should, really.
Paul: Yeah, “should!”
Yeah. They’re absolutely right.
Alan: Let’s be honest. That’s
what we expect. And I even expect that! The only technology that’s
lived up to its promise of, like, rock-solid persistence has been the
Hololens.
Paul: Yeah. It is a hard
technical problem to solve, but there’s so many people working on it,
and we’re getting closer now. Microsoft’s got a great product around
it. Google kind of shocked me a couple of years ago, when they
announced their spatial anchors were cross-platform.
Alan: Apple seems to the only
one that is playing in their own sandbox, and they don’t play well
with others.
Paul: Yeah.
Alan: Like, what is USDZ?
Paul: Yeah.
Alan: Come on. Like, everybody
in the world is moving to GLTF — for the people listening, GLTF and
USDZ and FBX, they’re all 3D model formats, and the world hasn’t come
to a standard. But we were getting close. Everybody was moving
towards GLTF, and then Apple decided to invent their own.
Paul: Yeah, yeah. We used GLTF
at Torch. We have a pretty sophisticated asset processor, so we
actually take 70 different file formats. But on the back end, we
always turn them in the GLTF, and on our export, we always turn them
into GLTF.
Actually, I got a funny little side
story around that; as a part of this export and publish thing —
having the industry agree upon a 3D model format, like you said, it’s
still not fully agreed upon, but it’s getting pretty close; if you
can handle GLTF, FBX, or OBJ, those are pretty well-supported,
well-known file formats, but they’re not interactive. There’s no
interaction in that. And the Torch is all about building interactive
scenes. I put an object in a space, and I want to respond once
somebody walks up to it, or when they look at it, or when they tap
it. To be able to do this whole publishing and export thing, we had
to come up with our own GLTF-equivalent for interactive scenes; this
portable file format for saying, here’s a construction of a scene,
and here’s all the interactions that are connected to it. Apple just
announced — at WWDC a few weeks ago — their reality kit effort, and
included in that is… I forget the name. It might just be Reality
Files. They call them Reality Files, and they’re actually have
interactivity in them, and they can be generated from their tool
chain and be shared across tools and all [sorts of] stuff.
So, I was talking to someone on the
market team about it, and I said, “oh, that’s really cool. You
know, we’ve had to do our own thing, and we want to learn more about
your file format, and maybe it’s gonna become a standard.” And I
was like, “have you guys thought about cross-platform?” And
he said, “we’re already cross-platform.” I’m like, “oh,
wow, that’s great! “And he said, “yeah, you can use it
across iOS; iPad OS or Mac OS.” That was his definition of
“cross-platform.” We think a little bit more in terms of,
we want you to be able to build and view content on Hololens, Magic
Leap, phones, tablets, looking glass display; anything that is a
reasonable place to view AR. We think your interactions should be
able to be distributed on those platforms.
Alan: Let’s put on our business
hat, here: what are some of the business applications that you’ve
envisioned for this? How will people use this?
Paul: So for us, what we’re
seeing is people wanting to add AR capabilities to their existing
systems. And so — as an example, a company that sells CRM platform
for the heavy equipment industry. Well, this is a platform that, when
sales reps go out and they’ve got all this literature around the
different products — and heavy equipment in particular — has all
these crazy configuration options. This company built a platform with
a mobile component that lets them organize all this information, but
it’s all based on, like, PDFs and images and spreadsheets. And so
they said, “we see where AR is super helpful, because we could
— first of all — show something at scale on location to a customer
and say, ‘oh, if you want this backhoe, it’s not going to fit in this
particular area,’ or to show different configurations in the sales
process at scale.” Like, people can actually walk around and
look at this stuff in detail.
So for us, what we’re offering is their
capability to inject an AR capability into their existing platform,
and say, “we’ll just use Torch to build the different pieces of
interactivity in AR, and then use our SDK to be able to surface this
stuff inside of our own app.” We’re seeing a lot of people
thinking like that, which makes a lot of sense, right? As enthusiasts
of the industry, you hope people just go whole hog in, and just say,
“AR is going to disrupt everything, and you should be thinking
about it for not only your marketing, but your internal processes and
your retail side and your sales side.” But rationally, these are
companies that are placing bets, and they’re dipping their toes into
the waters. They’re looking at how they can… oh, man. I just now
thought of this: they’re looking at how they can augment their
current product line, or business, or whatever it is they do. So
we’re seeing a lot of that.
People that are — like it’s said, the
retail side — people that have invested a lot in mobile
applications, and they have these really interesting AR ideas for
their physical locations. Like, when a customer comes in and you have
a personalized experience; you can help guide them to the appropriate
things that they’re looking for. You can make recommendations. All
this great, engaging stuff, but they don’t want to put out the AR app
that that somebody has to download and install, and it’s totally
separate from their primary app that already has millions of
installs. And that way, that works very similar to the CRM example,
where they just say, “we want an AR capability in our app that
we can publish content into, and we want more and more of our team to
be able to create the content and publish it.” So, retail has
been a big one. And eCommerce, you know, obviously; pretty
visualization of a product before you buy it, and making that not
just a model that you stick in your room, and can’t only scale and
rotate it, but it actually tells you about itself and you can
actually buy it in the moment.
This has always been the difficult
question for me to answer, as far as what is the addressable market
for AR. It’s everything. It really is the easier thing for me to say
— as you mentioned in my bio — you know, I kind of got into
software in the very early days of the web. In those early days,
people were saying, “what do we even need a Web site for.
Alan: [laughs]
Paul: And then there’s the
progression of, “well, we should have a Web site, but we’ll just
stick our company’s logo on it. And then an About Us page.” And
then over time it became, “wait a minute: real value is
happening. A real audience exists on this platform. Not only do we
have to have a website, but we’ve got to have some form of
transaction happening there. We could run our support through it. We
could run our business through it.” And then eventually,
everything matures to the point where people say, “well, we’re
going to build a business entirely on top of this. We aren’t this
existing business trying to figure out how to incorporate the web
into it–“
Alan: “–the web IS our
business.”.
Paul: Right. And then the same
exact thing happened with mobile! You have people said, “why do
we need a mobile app?”.
Alan: That is… like, honestly?
The people that are asking these questions: if you’re listening, and
you’re asking, “why do we need an AR app?” Think about what
Paul’s saying here. “Why do you need a website?” “Why
do you need a mobile app?” These are questions that seem
absolutely ridiculous to ask now, because the world is on web, and a
larger portion of the world is now on mobile. If Google is all-in on
AR, Apple’s all-in, Amazon, Facebook, Walmart — like, every major
company in the world gets it. So, it’s coming.
Paul: Yeah. Yeah! Without a
doubt. When? Now that’s the trillion-dollar question.
Alan: You know what? You just
said the magic number: a trillion dollars. And I’m going to unpack
this quick, because I actually predicted that XR will create a
trillion dollars in value, in five years.
Paul: Yeah.
Alan: The industry itself —
just hardware and software — is going to create about between $400-
and $500-billion, right?
Paul: Yeah.
Alan: That’s based on all the
different studies that are coming out. So, just the sales. And if you
add it up… so this year we’ll do probably $20-billion. Last year
was $10-billion. Next year to be $40-billion. It compounds. So by
2021, they’re anticipating $100-billion a year. So, forecast out to
2025: we’re looking at… call it half a billion [dollars] created.
Right? And that’s just half a billion dollars. That’s not factoring
in one dollar of value created for engineers, doctors, hospitals,
designers, retailers, training, education. If you factor in the value
created with this technology, it’s in the multi-trillions.
Paul: Yeah. Yeah! We’re standing
on the shoulders of the web and mobile, and we’re introducing totally
new forms of interaction that we’ve only begun to understand.
Alan: I know! It feels like I’ve
been doing it forever, but it’s only the beginning.
Paul: Yeah, no, it is. It’s
still early days–
Alan: Can I retire yet, Paul?
[laughs]
Paul: No. Not yet. Just like me,
we’ve gotta hang in there a little bit longer.
Alan: I think we’ve got to grunt
it out. So, what is your timeline prediction on this, then? When are
we going to see AR, in its different forms, take off?
Paul: I hesitate to do, like, an
actual year, but what I have observed is that last year, when we were
out in the market, most of the enterprises we were talking to had
experimental and proof of concept-type budgets — things that were in
the emerging tech group, or some kind of R&D fund.
Alan: “We hired a kid out
of high school to work on Unity.”
Paul: Yeah. This year, it’s very
much, very serious conversations around… it’s still early. Most
still haven’t figured it out, how it fits in with their business. We
spend a lot of our time educating — as you do as well — but we are
seeing serious budget considerations around, “this is going to
become part of our business.” And so I do see that progression.
I talked about where it’s, “ehh, this is kind of weird and
experimental. Maybe we’ll do it, just to kind of stand out from the
crowd,” to, “oh, it feels like we really should be on top
of this early, because we’ve got the extra money for it, and time.”
It seems like it’s coming. And then next year, I think there’ll be a
whole lot of people jumping on that bandwagon. I still think we’re
probably — I would say — two to three years out from it being as
vibrant as those Web 1.0 days.
Alan: It’s interesting, because
my prediction — and I don’t say this in public, and I guess this is
going to be the first time I’m saying it — my prediction is: 24 to
36 months, we’re going to see massive growth. Like, exponential
growth. We’re going from $10-billion last year, $20-billion this
year, $40-billion, $60-billion, $100-billion. So, we’ll be in
$100-billion market, which is huge. But we’re also going to be seeing
a roll up of the entire ecosystem. I think there’s going to be big
companies that realize, “oh my God, we need an in-house team for
this.” And rather than try to scrounge it together, they’re just
going to start acquiring studios.
Paul: Yeah.
Paul: And it’s kind of
interesting, because this technology is not just about the platforms
like Torch, but it’s also about the content creators. A lot of
investment has gone into platforms and products, but they’ve
neglected the fact that these studios are really, really vital.
That’s why we actually started XR Ignite; to bring the industry
together, and create a community hub and investment arm and
accelerator. To take these smaller companies that have great promise,
and combine them with corporate clients, and bring them to that point
where maybe they are acquired, maybe they are just selling it. It
doesn’t matter. But we need to bring them together. And I think over
the next three years, we’re going to see an absolute explosion of
growth in this industry.
Paul: Yeah, exactly right. I
think you’re going to see people that already have a kind of
intuitive understanding about how to execute ideas in this new
medium, they’re going to be very valuable people, and there’s gonna
be people that will — like me; I was a graphic designer at a daily
newspaper building ads, and I heard our executive staff of the
newspaper start to think about, “hey, maybe we need a website.”
I’ve been online for a few years at this point, and I knew I could do
enough HTML to help them out. And not only did they let me help plan
the… I was, like a 23-year-old kid helping these executives plan
their online department. But I also got the job as manager once it
got set up.
Alan: Yeah, we’re seeing that
all over the place. Put it this way: one of the kids that we
sponsored when he was 13? He’s 16 now. I think it works for Google
now. He worked for Microsoft last year.
Paul: The other thing that
excites me about it is: I think we should really rethink how software
gets built. AI plays a role in this as well, but that’s one of the
reasons why I was pretty proud that we introduced a totally different
workflow into AR, because we’re not reusing tools from other
industries. We’ve built something from scratch. I think it’ll be
interesting. I don’t know if coding… it may not be coding anymore,
right? It might be application creation. Experience creation.
Alan: Yeah, you’re absolutely
right. I do a talk called The World in 2039, and part of it is, what
are the jobs of the future? What happens when, all of a sudden, our
education system catches onto coding and says, “we’ve got to
teach everybody coding” — which they’re doing now; they’re
starting to teach code, which is great. But what happens when code
starts to code itself, right?
Paul: Yeah.
Alan: We could go down that
rabbit hole for days.
Paul: Yeah. Totally.
Alan: Paul, I want to thank you
so much for taking the time to join me on this podcast. I’m really
looking forward to digging into Torch and seeing what we can build.
Paul: Thank you for having me. I
also wanted to say, you’re a prolific poster on LinkedIn. You’re a
huge advocate for our industry. And I know that’s not easy to do.
Thank you for all the time you put into that.
Alan: I appreciate it. It’s a
labor of love for sure. And my mission in life is to inspire and
educate future leaders to think and act in a socially, economically,
and environmentally-sustainable way. I believe that this technology
is the way we are going to educate in the future. I’m doubling down
on our future, and the kids who will create it.
Paul: Absolutely.
42 min
September 04, 2019 Lighting the Torch for In-App AR Development, with TORCH's Paul Reynolds
Game engines
like the versatile Unity have long been the go-to for AR development,
and for good reason. But its reputation as a video game engine can
also be intimidating -- especially to those who want to create AR
software for enterprise. That's why Paul Reynolds lit his TORCH; an
app he co-founded that lets you design your own AR platform, right in
the palm of your hand. He chats with Alan about his claim to flame.
Alan: Hey, everyone, my name's
Alan Smithson, the host of the XR for Business Podcast. Today's guest
is Paul Reynolds, the CEO of Torch, a really exciting augmented
reality platform. It's a mobile augmented reality development and
deployment platform for enterprise. Paul has been a software
developer and technology consultant since 1997 - since before the
interwebs! In 2013, after 10 years of creating video games, he joined
Magic Leap where he was promoted to senior director, overseeing
content and SDK teams. At Magic Leap, Paul recognized the lack of
accessible tools for non-game developers that was hindering
widespread adoption of immersive and spatial computing technologies.
In 2016, Paul moved to Portland, Oregon, where he founded Torch to
address this very problem. To learn more about Torch, you can visit
torch.app. Paul, welcome to the show.
Paul: Thanks for having me.
Alan: It's such a pleasure. I've
been looking forward to this episode. Torch is such a cool platform
and I keep seeing your posts on LinkedIn of putting stuff around your
office and stuff. So tell us, what is Torch, and how did you come up
with this crazy idea?
Paul: The easiest way to think
about it is, it's a mobile application -- currently for iOS -- that
lets anyone build interactive spatial scenes. So, you create a
project and you're building it in the camera of your device, which
means you're also walking around the space, or moving around the
space and you're building up interactive experiences visually,
without writing any code. We call that the design environment, and
that's the freely available [option] -- anyone can jump into it and
just start building. What makes it a platform is the capability of
taking what you've created in Torch, and exporting it and publishing
it and integrating it into your existing app, or pushing it out to
another platform or tool. What we really wanted to focus on was
allowing people to iterate in augmented reality -- directly within
augmented reality -- as opposed to sitting on a desktop computer and
trying to figure out how to work a game editor, and get more people
able to work productively in 3D. That's really the heart of it.
Alan: That's so cool, because if
you're sitting at your office, you're like, "wow, this AR stuff
is hot. It's amazing." You know what, go learn Unity and coding
and figure out how to actually make it. Six months later, you're
like, "oh, look, I made a portal."
Paul: [laughs] Right.
Alan: What you guys have built
is a simple way to just do it visually.
Paul: Yeah. So, my background
was in video games, back in the day where everyone was building their
own engine. You really didn't even have time to build a really nice
editor on top of that. So when Unity came out -- we'll pick on Unity
in particular, because it's just such a well-known product -- when it
came out, it was the game engine that most of these game studios I've
worked for wanted to build. And that was really unique; they had
basically taken what would normally mean millions and millions of
internal R&D dollars, and turned it into this tool that pretty
much anyone can download for free. But what happened over the past
few years is, it's become kind of the de facto interactive 3D tool.
And it was for me as well; I've been a Unity user forever. What we
learned when we started building a p
0 min

About XR for Business

From the publisher's feed

Meet the leaders who are changing the face of virtual and augmented reality

Share XR for Business

Sign up to save your podcasts

XR for Business

XR for Business episodes

About XR for Business