XR for Business

Businesses that were late adopters of the World Wide Web and the mobile realm are the butt of obsolescence-themed jokes today. Extended Reality evangelist Rori DuBoff from Accenture joins us to advise the businesses of today how not to miss the same boat, and shares strategies on staying ahead of the curve.

Alan: Today’s guest is Rori DuBoff, Managing Director and Head of Content Innovation and Strategy for Extended Reality — that’s VR, AR and MR — at Accenture Interactive. Rori is a strategic and innovations leader with over 20 years experience working in digital marketing, integrated media, creative brand advertising, and emerging technologies.

As a virtual, augmented, and mixed reality evangelist, Rori advises companies on strategy and marketing opportunities for brand and business transformation. Prior to Accenture, Rori was Global Head of Digital Strategy and executive vice president at Havas Media Group, where she led and managed strategic planning worldwide, with a focus on digitally-integrated marketing communications and innovation.

Previous to that, Rori was a partner and director of strategy at Ogilvy, where she focused on developing digital marketing strategies for health, retail, and media industries. Rori holds an MBA from NYU Stern School of Business and a B.A. from the University of Pennsylvania.

She is a regular public speaker, contributing writer for Ad Age, Guardian, Campaign US, Mediapost, and a jury member for the Cannes Lions. With that, I mean, what more can I say? Rori, welcome to the show, and Rori, I just want to let people know that they can find you on Accenture.com and they can follow you on Twitter @RoriDuBoff. Rori, welcome to the show.

Rori: Thank you. Nice to nice to be joining.

Alan: It’s such a pleasure to have you on the show, and I’m really excited. I want to dive right in and get an understanding of what Accenture Interactive does, and what your role there is, and how you’re helping businesses use these virtual/augmented/mixed reality technologies across their enterprise.

Rori: Sure. So, Accenture Interactive is part of the larger Accenture consulting company. So Accenture is a very large technology and strategy company with over 400,000 employees worldwide. Accenture Interactive was developed, I think about five… a little bit more than five years ago, to focus more on the brand experiences that a lot of our clients were looking to develop, in terms of engaging customers. So within Accenture Interactive, we focus on user journeys, on strategy, on experiences, on marketing, and we’ve more recently — in the last two years — been looking at this new space of extended reality. And extended reality is the term that we are using at Accenture, and I think across the industry others are also using this term “XR” to include virtual reality, augmented reality, 3D experiences; all different types of experiences that blend the digital and physical worlds, and work on extending your reality.

Alan: So, would the things like computer vision and machine learning, would that fit? Would you bundle those under XR as well?

Rori: Yeah, absolutely. Those are critical technologies that we are looking at, in terms of making VR and AR experiences smarter, more personalized. Accenture has different groups, like a group headed dedicated to artificial intelligence, things like cloud computing, and machine learning is a part of artificial intelligence. And we work together to sort of use those technologies within the immersive experience area.

Alan: Incredible. So let’s dive in: what are some of the best examples that you or your team has worked on in this type of role? So, you’re meeting with a customer, you’re saying hey, we’ve got a solution for you? Or is it a joint thing? How does the process work when you start to engage with a customer and then work with them on that?

Rori: I’d say, about two years ago, I think a lot of the time with with our clients was spent on educating people on what VR is, on what AR is. And it’s a year later; I think a lot of our clients sort of understand that now. They get that there are these VR headsets out there. Not all of them have tried it, but they understand what the technology is. And with augmented reality — especially, with the growth in mobile augmented reality — I think people are more aware now, of what these technologies are.

So now, what we focus on when we speak with clients is the use case development. We figure out these technologies, how can they be used? For our retail clients, we’ve been talking around different areas, in terms of how do you develop a mobile augmented reality strategy to drive growth through retail? So that could be creating filters or lenses for product visualization. It can also be around planning the future retail store, and using plan-o-grams to set up spaces for future retail. We also speak with automotive manufacturers, around how they can start marketing and merchandising future car models that might not even be done yet, but through the use of these new technology experiences that let us create 3D models of the car, and sort of market that to potential customers before they might even arrive in the showroom.

Alan: Can we unpack that a bit? Because that, I think, is an incredible use case; being able to start selling something before it even exists. And I mean, you can always sell things with 2D diagrams and pitch decks and stuff. But really, what you’re talking about is a completely new thing.

Rori: Yes. So, we were at South by Southwest this year at the end of March, and a lot of the stuff we’re showing is exactly that. We had, for DuPont Corian — which is a client of ours — we had an experience that showed bathroom vanities that you’re able to look at and place inside of your own space, similar to what IKEA is doing, and Amazon. There’s a whole growth around that area, in terms of product visualization, especially for larger, more expensive items that people want to be able to see in their own homes. We had an advance opportunity to start showing some of those potential items before they might be available in-store.

We also had for Kendra Scott, which is a jewelry maker, the ability to try on earrings. Now, some of those models you can buy now, but to be honest, I’m not sure if all of them were you could purchase them now, or maybe some of them were exclusively advance to be able to see there. So you’re seeing companies be able to test out the demand and interest in products, and sell them, and then adapt their inventory based upon how people, you know, what their experience is like trying them on virtually before they might even be physically built.

Alan: So let me ask you a quick question from an ROI standpoint: How much… you can give ballpark numbers in this case, but how much would something like this… let’s say for example I’m a jewelry maker, and I want to be able to try on necklaces and earrings, and I want to have — I don’t know — a hundred of my earrings on this. What would something like that cost, and what are the ROIs that you’re seeing? Like, what is the return on that? So for example, if it cost $100,000 to be able to put [up] my hundred earrings, what is the uptick that you’re seeing in sales? Is there a specific rise in revenues that you’re seeing because of this technology?

Rori: So I can answer that question in terms of the different variables that go into what the ROI would look like, but I unfortunately can’t answer the actual numbers right now because a lot of this stuff is completely new.

The models haven’t been fully developed and we haven’t actually gone out and evaluated their returns. What you have to look at when you’re creating these sort of experiences, though, is the quality of the models, and then the AR based on that. So we as an industry are still trying to figure out. We know that for, like, the automotive space for instance, the quality of a model that you’re creating, or the actual car that you’re experiencing: the higher the quality, the more realistic the car looks, and the end investment in that car, you can talk between 20, 30, 40,000 dollars. So, the process justifies the means at the end.

With things like jewelry, you have to look at how much the quality of the model — and when I say model, I mean the actual physical replica, or we call a digital model that you’re looking at in the virtual space. So the quality can vary; some virtual objects could be lower quality, like a Coke bottle. How much are you going to spend in terms of creating the replica of a Coke bottle, versus an earring? Is it a diamond earring? Do you need to make it uber-exact? When someone tries it on the virtual space, do they need to have intense detail around it?

Those are the kind of levers that we’re trying to look at, in terms of figuring out the costs into going into creating these experiences based upon the value of the object, and how much it can be sold for. And then that dictates how much effort goes into the creation. But we’re not at the point right now where we can give ROI figures, unfortunately.

Alan: It’s interesting, because a lot of clients — and I’m sure you get this every day — they say, we love this thing; we want it, but who else is doing it, and how much does it cost? And what’s my return on investment? Like… nobody else is doing it. We’re not really sure how much it’s going to cost. And we have no idea on the return. Still want to pay? Still want to buy it?

Rori: Well, I understand that. I think that this is where I go into the analogy with the digital space. It’s not just about replicating exactly what you’re doing. It’s about rethinking how you’re doing things. Companies, when the Web came along — or, sorry, “the mobile space,” let’s be more specific — companies like Uber or Lyft didn’t exist. Right? And if a taxi company said, can I save money by creating a mobile app, you would never have had an Uber or Lyft that were created. So I think part of it is figuring out a new channel.

And I 100 per cent believe that the immersive space is just like the web space or the mobile space, in the sense it will overtake everything, and there’ll be no opportunity to say “no” if you want to stay competitive. But I also think you need to say, well, how am I doing business now? Maybe there’s an opportunity to create even more custom offerings, or more personalized offerings that never would have existed if you didn’t have the opportunity to give people a [way to] virtually try things on.

Alan: Well I think you you nailed it there. And you know, people that are listening, if you’re looking to use these technologies and you’re thinking, should I wait? What is your recommendation to people that say, why don’t we just wait and see? Because with mobile, and with Web, there was always this “wait-and-see.” And I think the problem that people don’t understand is that Web came, and it took you know 10-15 years to kind of mature. Mobile took another 10 years to mature. Virtual, augmented, and mixed realities — or extended realities — are happening much faster. So, what do you say to… or, how do you overcome that objection where people say, well we’re just going to wait and see?

Rori: Yeah. It’s a common thing. I think the way to look at it is to think incrementally. You don’t have jump to go buying a three- to five-thousand dollar headset and creating some really complicated experience that only five people can use. Right now with the mobile phone, there are so many cool, interesting experiences that brands are doing, just with augmented reality, through filters on SNAP or Facebook or Instagram that are getting a whole new generation of what we call, I guess, Gen-Z excited and really, really engaged. And the results on that, in terms of engagement, drive to purchase, we did some data research recently with SuperData, which is part of Nielsen now, on the increase of likelihood to purchase through mobile augmented reality — even virtual reality. Mobile virtual reality on a headset like an Oculus Go, which is $200.

So I say, first, don’t think uber-complicated; start off more simple. Not simple in idea and not simple in strategy, but maybe more simple in technology. Start understanding those incremental changes. You have an email; you could potentially add augmented reality to it. You have video that you’re doing; maybe think about how 360 video could enhance the experience, or bring it a step further. So that’s what I would say, is that you have to start moving into that space. Because we are shifting from a world of everything being flat and 2D content-oriented, to a world in which people will be expecting more and more things to be 3D, rotatable, modular, and that’s where everyone needs to be headed, or you will be left out completely in the future.

Alan: I agree. You said that very eloquently. Let’s go a little bit personal, on what you think is some of the greatest examples. There’s tons and tons of examples out there. Last week, Game of Thrones had a Snap filter where you could take the Flatiron building and drop a dragon on top, and Nike had a Snap filter where LeBron James comes out of a poster and slam dunks in your space. There’s been all sorts of amazing marketing — Burger King allowed you to point your phone at a competitor’s ad; it would catch on fire and they give you a free Whopper. All in augmented reality. These are some really cool filters and experiences that are being done from the phone, from the mobile device in everybody’s hand, and this has instant scale. I mean, by the end of this year, we’ll have approximately 2 billion smart devices that will be AR-enabled. So, what are some of the best examples that you’ve seen, or maybe done?

Rori: Well, you named all three, and I knew all of them. And I’m thinking, is it because I’m just in the industry that I uber-focus on that? Probably. But I guess I remember all three of those, especially since I live right near the Flatiron, and I thought, oh my God I should…I need to go replicate that same experience that I saw posted on Twitter with that dragon sort of soaring over. It was really, really cool.

If you asked me if I can recall, that I remember any commercials, or more of the traditional media, and I’d be lost. I guess in many respects, those are just kind of cool experiences, and I stopped even thinking about them as advertising. To your point, we’re seeing a lot of really interesting, smart uses of augmented reality on the mobile phone, and getting people to explore and try and pay attention to brands that maybe they might not normally — I mean, all of those brands also have a high sort of engagement rate — but I think that half of using more and more of the opportunity to engage people through AR through the phone is going to be a continued path of opportunity for a lot of different brands. We are in discussions around that. A few interesting projects.

I think that some of the other spaces beyond mobile augmented reality that we’re also seeing a lot of potential interesting use cases are around virtual reality as well. What I’ve been impressed with is a lot of brands that we’ve had taking another look at VR. About two or three years ago, a lot of people were using virtual reality for branded content, and I think we had a sort of saturation point where a lot of people were questioning, what am I doing with this medium? Why am I spending all this money? What’s the return I’m getting on it? So they pivoted to mobile AR, which is a little bit more accessible — not a little bit; a lot more accessible — and you’re seeing the results, and the reach is pretty pretty awesome. So I’m excited to see that continue, and the connection to commerce we’re going to be seeing evolve as well.

So, you’re gonna start seeing a higher return on investment, because through the mobile phone and a lot of those experiences, the connection to commerce, and the ability to purchase and buy through those lenses — so you see a product and immediately be able to buy it — I’m excited for that path moving forward.

Alan: Yeah I think… well, pretty much everybody… So I would say, Google is leading the way with this, with Google Lens being able to take your phone out, open the camera, point it at a pair of shoes, and it will tell you exactly where to buy those shoes instantly from your phone. I think the camera as a lens to do commerce is really becoming powerful.

Rori: Yes. Yes, exactly. And Instagram as well. So you’re going to be hearing a lot of that.

Alan: So, I think Google’s leading. Snapchat is also doing it. I think one of the things that brands have to understand is all these big platforms — Facebook, Snapchat, Instagram — these platforms have users already there, and they’ve made it fairly simple and fairly easy for brands to create these new experiences using the Snap Lens Studio or Facebook Spark, which is their development platform. Are you finding that brands are starting to try to do things in-house, or are they still looking for advice and help with pulling this together?

Rori: Yeah. I think that there’s still a need to understand these technologies. And there’s also the creative teams that go about creating/producing it. And so we don’t see a massive amount of resource yet in this space. I think that the new generation of talent — say if you were studying right now — creative developer, understanding… I mean, these don’t necessarily all require Unity or Unreal, which are the software programs — the game engines — that often power a lot of this interactivity and experience. But there is a need to to have more talent to help with it, and to strategically think through why and how can I use augmented reality.

I think we’re still stuck in that frame, where it’s like, how do I move the same content I have into a 3D experience? And what we need to do is kind of rethink through what’s the best use of the medium. I do think that there’s a lot of excitement and interest amongst our clients, and some of them have in-house teams. But at the end of the day, there’s also a lot of external specialty companies and groups that are helping, and we’re looking to do that as well. And to also connect it into the broader ecosystem, right? So, if you want to do commerce with AR, how does it tie to your backend inventory? Has it tied to your personalization efforts? How does it tied to the broader system in-store so it’s not just fragmented? And I think, when we look to immersive technology, what we try to remember is what we’ve learned from the past, which is you don’t want to go out and just create a standalone experience. You want it to be connected to the rest of the ecosystem.

You asked earlier on about challenges, in terms of this moving forward. I’d say one of the biggest challenges is making sure that we’re thinking about immersive as part of a larger system of engagement; as part of a larger system of experience, versus a one-off. Because if it’s just a one-off, it’s not going to thrive long-term.

Alan: I agree, and some of the other interviews I’ve done on the show have been around the enterprise applications. So, you know, factories, training — that type of thing. And it really comes down to a lot of companies have done a ton of proof-of-concepts (or POCs), but when it really comes down to it, you have to make sure that whatever it is you’re doing with VR AR, or MR, whatever it is you’re doing has to synchronize with your current systems at scale. And I think that’s really where we’re at in this world, is everybody’s done the POCs, they’ve realized the value. They’re like, OK this is great. It increases our sales by 20 percent. It increases conversions. We’re all in. Now, how do we make sure that this is a seamless integration to our content management systems, or our retail systems?

One of the companies that is really leading the way right now is Shopify. They’ve been pushing towards VR and AR applications for several years, and they’ve just recently introduced their 3D view platform, where you can take your 3D model of your product and host it on their website. And now customers, instead of clicking through six photographs to see the product, they can see one 3D object; spin it around, open it up. And eventually, they’ll be able to hit a button and see that product in their space, whether it be on their dining room table, or on the floor, or whatever it is. So, I think the integration is key.

One of the things that you mentioned was marketing versus experiences. And the last one I would add to that is utilitarian use cases. We’re starting to see some really interesting use cases of AR, especially with the mobile phone, that allow people to maybe measure a table, or a room. What are some of the utilitarian use cases that you’ve seen that made you go, wow, that’s a really good use case?

Rori: So I mean, I’ve seen that measure the phone app. Utilitarian, I think of a kind of connection to mobile AR. So, the Bose AR headset – er, frames, excuse me — which I bought a pair of. I don’t know if you’ve tried them. The audio’s–

Alan: I haven’t. Are they any good?

Rori: They’re amazing.

Alan: I keep hearing about them. I’ve got to get a pair Bose! (If you’re listening, send me a pair of glasses).

Rori: They’re amazing. I listen to the news and music on them every day as I’m walking around the city. If anybody has any idea — which is just great — for people who don’t want headphones in your ears, or don’t like the feeling of headphones (and I’m one of those), it’s awesome. As soon as it’s synchronized with my Google Maps, which… actually, it could be right now. I could play the Google Maps with my ears, I haven’t tried that yet. In terms of maps, instead of having to hold up your phone, you’re able to walk around and your glasses say, you know, “turn left,” “turn right.” That, to me, is a very smart use case.

The struggle with some of the utilitarian uses on mobile AR right now is that you’re still holding up your phone, and that’s why I think AR on the enterprise side has succeeded for utility, because it’s glasses — which are more expensive, but you’re hands-free. Utility with mobile AR, where you have to hold your phone: yes, you can do some things, but they’re still having to hold the phone, which kind of, you know, holds back a little bit, in terms of being totally focused on utility and hands-free.

You already mentioned some of the use cases of that with Google Lens and being able to identify objects. We talked about placing furniture in your space, or other large items for sizing. Those are really smart use cases. For creativity, there are some really cool stuff you can do, in terms of sketching or drawing or designing overlays onto the space around, you which is also very smart. But surprisingly, I think for utility, just to jump back to the virtual reality space; we’ve seen in the immersive learning area, VR is huge growth around utility (immersive learning, I’m kind of putting in the category of utility, because it’s productivity, right?).

We’re seeing that people were questioning in VR, is this ever going to take off? Is it just gaming? In the consumer world it might not be taking off as much. But in the enterprise space, there is a massive, massive amount of interest. So, you know, retail–

Alan: It’s hard to argue when companies like Boeing and Wal-Mart start releasing their statistics, and they’re saying yeah, we have a 45 per cent increase in retention rates and near-zero error rates when using AR glasses and in-VR training. And then they’re going, okay, as an enterprise, even if you got a 10 per cent increase in anything, you’re going to adopt that technology. But when you’re seeing 25 to 50 per cent increases right across the board with this technology, it’s impossible to ignore it.

Rori: Yeah. So, one of the areas that we’ve been investing and exploring and developing a lot around is immersive learning and training, and we actually launched a project in the fall of last year called Avenues, which is a Accenture virtual reality experience solution. It’s spelled avenues, and it’s for social care. It’s to train social care workers using virtual reality. And the situation that we launched was a 360, live-action voice-based experience. Basically, you’re in a scene with parents, and you’re trying to identify whether the child should be placed in foster care. You have a real interview, where you interview and you speak and engage with the family. And it won an award at the Barcelona Mobile World Congress. It was up for an award at South By. And we’re actually building out that platform. It’s been so successful — we’re getting results in now, so we’ll be able to share those soon — in terms of knowledge retention, preparing social care workers for real-life experiences.

Alan: What a great use case. Let’s just stop for a second and take away all the technology; you’re enabling social care workers to make better decisions in high-stress areas, for children. That is just amazing. I have read that the case study, and I will put it in the show notes for anybody who wants to read it.

Rori: And I think one of the most important things that the team — the health and public services team that we worked with on this — pointed out, and I actually owe it to them because now I take it away with me and I think about it and it came up earlier when we are speaking, is this idea that there’s the “headset on” experience, and the “headset off” experience. You can think about it as, here’s just a VR experience to solve a solution. It’s the VR experience wrapped into a bigger strategy, and other materials; a larger experience that’s going to have impact and success. We’re succeeding with this because we’ve created this phenomenal, innovative experience using VR, but there was also a lot of thought and planning around when somebody puts the headset on, what’s that whole experience when they take the headset off, what materials are available for them and how do you support that.

We’re looking at other industries; we’re looking at talking to a large retailer in the technology space, we’re talking with other organizations in the public service sector. We’ve also spoke with a financial organization recently, and with all of them I say, immersive learning, the power we can bring through these simulated experiences of VR, but let’s also think about the broader transformation and broader engagement we need to do in your organization to socialize this work, to educate people on this work. And that’s been very successful.

Alan: It’s amazing, because there is a bit of a stigma around putting a headset on. I’ve done personally, I’ve done probably 500 events showing VR at different trade shows and things, and some people just don’t want to put something on their face and be completely isolated from the real world. I think there is a bit of a stigma there still. But it’s when you see the value that comes out of it, it really shatters that that stigma fairly quickly. It’s great, the work that you guys are doing. I love that use case. I’m going to talk about, you know, what are the most important things that businesses can do right now to start leveraging the power of extended reality or XR technologies?

Rori: Well one is I think just educating themselves, familiarizing themselves with the experience. I think a lot of organizations, they probably just need to go out or have people come in and help them understand what’s there. We were talking earlier that, you know, we’re in the industry. So we are immensely involved and engaged. But the truth is, the rest of the world, people day-to-day, this isn’t a top priority for them. So one is just education/awareness/engagement.

Then I’d say the second thing is, before jumping then right into the technology, sitting down and thinking through: what are my current challenges today? What are the current things I’m doing well? And what are the use cases in which my consumers — or my employees — might benefit from this type of technology? And so instead of thinking how do I create an AR thing or a VR thing, sort of holistically looking and saying, where does it make sense for me to start thinking about changing the way I communicate, train, sell? And that’s the right mindset, I think, going into this. As much as I’d love to see more companies do work and VR/AR/3D, there are still areas that they need to figure out — that’s just in digital media, or digital content — before they need to jump here. So, figure out what what are the right areas in the organization and then educating themselves on the technology and its potential, is where I’d be focusing.

Alan: What do you think is the fastest way for somebody that’s listening to then educate themselves? Do you guys offer strategic workshops? Do you have white papers or case studies? Where would be the best place for people to get this knowledge and educate themselves?

Rori: So we, like I guess all consulting companies, we do our workshops. We also have a bunch of white papers. I post a lot of the work on my SlideShare under Rori DuBoff as well, about different topics. Everything, ranging from marketing to the ethics of designing an immersive world. Accenture has a lot of thought leadership online. When people come to me to ask me how to get involved in the space, in terms of they’re looking to work in the space, I recommend, there’s a lot of meet-ups and a lot of groups that you can go to throughout wherever you live. I mean, I am in New York, but there’s other cities; Boston, on the West Coast, as well as Chicago, L.A. There’s a lot of groups.

This kind of space right now is so organic but so much energy and excitement. It reminds me of the early days of the web. People are out there talking and sharing, and it’s still developing. There’s no such thing as experts. I’d say get in now and learn. For people interested in working in the space, or just clients that are interested, reach out to me personally. We can certainly conduct a workshop or session, and then there’s there’s a ton of knowledge and literature online, through Twitter, through if you search for these topics.

Alan: Yeah and I think another resource that we don’t really touch, talked about, or you haven’t talked about, is the VR/AR Association. Are you members of that?

Rori: Yes, we are members of that. That’s another organization that has events and organizations on a citywide level. So that’s something to consider as well. Yes.

Alan: My wife is the president of the VR/AR Association, Toronto Chapter. If you happen to be in Toronto, we have events all the time.

Rori: Oh yes. Yes.

Alan: Shameless plug!

Rori: No, no; it’s funny. That’s a great organization, but there are a lot of people in the space looking to share, to develop knowledge, and to educate. So I’d say getting online is one step, and then obviously we want to still meet in person. We’re not quite there yet for all virtual communications.

Alan: One of the things that I always tell people is, talking about VR or AR is [tantamount] to try to teach somebody about what the color red looks like to a blind person. It’s impossible to describe it.

Rori: 100 per cent agree.

Alan: You kind of have to try it. And so, we run workshops where we show everybody, you know, here’s VR. Here’s AR. Here’s MagicLeap an HoloLens, and here’s all the different glasses. Here’s a pair of North glasses, here’s all the hardware. Here’s the different problems that can be solved with each one of them, and then go from there, as to, how do you think this can be used for your enterprise. So I think being able to be person-to-person and show people… unfortunately, there’s no advertising campaign for VR yet. In enterprise, anyway.

Rori: Yeah.

Alan: It’s still hand-to-hand combat.

Rori: I mean, just listening to you say all those devices, I’m thinking, it is quite intimidating. I think to most people, it is overwhelming and intimidating. I’m not surprised that the clients and consumers in general are kind of like, oh my god, what do I do with all of this? And with all the jargon — the VR and the AR the XR — I’m really looking forward to our space figuring out how to have less fragmentation, more fluidity. Simplification. I think, as we start to make the technology less technical, and it becomes a bit easier and more accessible for people to try some of these experiences, there’ll be a tipping point at some point, and I think that’s when we’ll start to really see the space skyrocket.

Alan: All right. So hold on, I’m gonna stop you right there. I don’t ask this — I’m going to start asking this is a regular question — when do you think that tipping point is going to be?

Rori: I’d say three-to-five years, maybe. Five years.

Alan: When you say “tipping point,” what does that actually mean? Like, is this when we go to a billion people using VR/AR on a regular basis? Or, what is your measurement around that?

Rori: So, my measurement is like, I’d say… where maybe the closest, five years out, would be where the mobile industry is now. Maybe ten years out, where social is, where you can’t not partake. So five years out… I mean, the mobile space is still somewhat optional. Not every brand or company has figured out how to have even their website mobile-accessible.

Alan: Yes, but they’re becoming more and more necessary.

Rori: Yes, exactly. I remember when being social was optional. You know, like, should I have a Facebook page? If you don’t have some sort of social page, somebody else will have created it for you. Right?

Alan: Yeah, true.

Rori: So, you’re not on Facebook yourself? Somebody else will have created it. And that’s not always a good thing. So most companies have some engagement in the social space, whether it be Twitter or Facebook. I’d say that 10 years out, where this will be a space that you have to be engaged. Now whether it will be called immersive VR/AR, or whether it’ll just have another name and it will be experiential.

Alan: Facial computing, maybe?

Rori: Who knows what it will be called. But the core of it, you said it earlier. Think about a 3D model, and think about when the web first launched, it was text-based. Then suddenly, people started using photography. And now video is very commonplace. And at some point — once again, especially for retail — if you do not have a 3D model of your product or real estate, if you don’t let me explore your property in 3D, you’re going to be like, you know, “inferior.” You’re not going to be part of it; you’re going to look outdated and gone. So, that shift… like right now, if you go to a website and it’s all text, you’re thinking, what, are they stuck in 1990? We’re going to be seeing that. And whether that’s, you know, probably 5-to-10 years is where I would be guessing.

Alan: So let’s talk about the outlier here, and the one that is kind of on everybody’s mind: Apple Glasses.

Rori: Yeah.

Alan: What do you think? Because I heard, I interviewed somebody from one of the big telcos, and they give me a timeline that was way sooner than I had ever anticipated. I mean, they don’t know either specifically, but what are your thoughts around Apple Glasses? Because Apple hired over a thousand AR developers in the last five years, and they’re all in on AR. Tim Cook has been on record saying AR is the future of computing. So, if Apple is going to release a pair of glasses — we don’t know what they’re gonna look like and what they’re going to do — but what timeline do you think? Is that within a year, two years, five years, 10 years?

Rori: So my answer to that one is, I don’t know the answer. I don’t know the timeline around Apple. I know they’re releasing glasses. I think everybody is excited, and some people are concerned, depending if they’re competitors. If you look at the history of Apple they obviously have massive success in terms of consumer adoption. But that being said, even if it’s an amazing product, we’re still at the “nice to have” phase. So what we don’t want is you know the glasses to be just considered a wearable, like the Apple Watch. Which, you know, is nice that people have that. But it didn’t transform the industry. So what I’d say is, I’m cautiously optimistic. But I don’t think at this point, I don’t really know what they entail. I’d be blown over if just these glasses in the next year or two sort of massively change the space. I think a lot more has to happen beyond just one device.

Alan: I agree. I think developers also have to start to learn how to develop in 3D. If you look back to the smartphone — the iPhone 1 was about 11 years ago now — and “app developer” wasn’t a job. That wasn’t something, there was no app developers. Now there’s millions of app developers. So they took a decade to build this ecosystem.

And now with ARCore and ARKit — ARCore being Google’s foundational framework for augmented reality, and ARKit being Apple’s — they’re really giving people that first ability to program in three dimensions. But that’s, I mean, there’s only maybe, I don’t know, a couple hundred AR programs in the world on iOS right now. So that needs to be in the tens of thousands before it would even be useful.

Rori: Yeah, yeah. And this idea, that there weren’t app developers; I believe that, for this space to be successful, there is a new breed of talent that’s required, which is thinking, how do you orchestrate the physical/digital experience. It’s not quite an architect. It’s not experiential marketing as we know, it in terms of retail store-based marketing. It’s the ability to seamlessly blend digital immersive experiences into the physical world and space. And if you talk to clients about that, or if you look at companies, there is very limited talent right now in that area. In that ability to think in that sort of digital integrated physical way. And that is where we’re going to succeed, I think, when we figure that out.

Alan: That’s some pretty exciting stuff.

Rori: Yeah.

Alan: I’m interviewing Matt from 6D.ai today, actually, on the show.

Rori: That’s great, that’s great.

Alan: Yeah. There’s a company… and for people listening, they’ve created a backend system that allows you, with your phone, to start programming cloud meshes of the real world. So it’s using the regular camera on your phone to put a point cloud map around the world, and create real augmented reality that uses the world in context. So imagine if your Pokémon Go could hide behind cars and people. That’s the easiest way to think about it, and that’s what they’re building. It’s really exciting.

Rori: Lots of terms for that, right? The AR cloud. Mirror World. There’s all these ideas around how do we map the physical to the digital world in a way that supports that connectivity, that seamless integration. Synchronicity. Those type of companies are going to be changing the world. So that’s exciting, that’s awesome that you’re connecting.

Alan: Quick question before we end: what problem in the world do you want to see solved with XR technologies?

Rori: A lot of discussion we’re having right now is on the ethics, in terms of sensible, smart thinking on how we handle data and privacy. I don’t think that has been solved for yet, in a way that is transparent, and that people understand, “this is the data providing, this is how it’s being used.” I’m hoping… space that we’re moving into is so personal and so sensitive, that we have to get it right. If we don’t get it right, the potential negative…

Alan: The consequences are very, very real.

Rori: Yes. So I think that figuring out things like accessibility and privacy and ethics, in a way in the space, to me, I’m really interested in it and trying to figure that out, and not take a passive stands to that. Because I think we have seen the consequences of not being more proactive, in terms of how we handle technology and experiences connected to it. So that’s what I’m really interested in solving for.

Alan: Amazing. Well, I’m going to just recap our conversation quickly for listeners. We talked about creating a powerful user journey extended reality, which is virtual/augmented/mixed reality, 3D, computer vision, and machine learning as well. And then really developing use cases around all of this in a strategic way, rather than just making something because it’s cool. We spoke about the DuPont Corian and being able to pre-visualize what some sinks and cabinetry would look like in your real space, so AR product visualization. We also spoke about jewelry try-on virtual trials, which if you go back to the cost of these things, it’s really based on the quality of the 3D models and the quality of the interactions. We spoke about Snapchat, Facebook, and Instagram addressing that Generation Z with these new face filters and things that can be used for retail and marketing. And your advice was, don’t overcomplicate things. Keep it simple and then start right away, but use incremental value gains as you go. And we talked about different types of this technology; AR, mobile-based AR, 360 video, 3D visualizers on websites, stuff like that. Marketing versus experiences, and utility use cases. We talked about VR for branded content. You spoke about your amazing Bose AR glasses; they really give spatial audio, so you can have sound — we’re talking about navigation and having your Google Maps — the sound can come from 50 feet away and tell you to go over there, and really guide you through that AR experience using audio. We talked about VR and immersive learning and training. You talked about the Avenues project that you were working on with social care VR training. We’ll put that in the show notes. And really, the keys to getting into this right now are educating yourself, figuring what problems you can solve using this technology, and really the question comes down to; how will my consumers and employees benefit from this? Taking that lens. You talked about educating yourself through workshops; you can book Rori and their team at Accenture for workshops. You also have your SlideShare which I’ll put in the show notes. Then you mentioned about the tipping point being five years to 10 years out, where it will be a must-have rather than a nice-to-have. And the last part was about physical and digital interface interactions. How do we incorporate our computers into the whole world of the physical world that we live in. And then you mentioned, keep getting the ethics behind this right are the key.

Elizabeth Baron drove innovation forward at the Ford Motor Company since the ’90s, advancing XR technologies in the automotive industry. Now, Elizabeth joins our host Alan as she discusses her new venture, Immersionary Enterprises, as well as her pioneering work at Ford.

Alan: Today’s guest is Elizabeth Baron. Elizabeth has been a true pioneer of virtual and augmented reality as the global lead for immersive realities, bringing together multiple disciplines throughout Ford Motor Company. developing multiple immersive realities using VR, AR, and MR to provide information in context to the design studio, multiple engineering teams, UX developers, and computer-aided engineering analysis and many more. Elizabeth has seen dramatic changes, from huge room-sized, multi-million dollar CAVE systems, to haptic seats, to car cockpits made out of wood. From the promise of virtual reality to it becoming real, Elizabeth has been an industry leader always pushing the limits of technology.

She has just started a new venture called Immersionary Enterprises, to provide probability spaces where an enterprise can study any potential reality, or the art of the impossible or possible, with a host of relevant data. These realities can be shared across a global connected work team for more collaborative decision making. Immersionary Enterprises aims to establish a holistic immersive reviews as a near-perfect communication and collaboration paradigm throughout industrial design and engineering.

It is with great honor that I welcome VR pioneer Elizabeth Baron to the show. Welcome, Elizabeth.

Elizabeth: Oh thank you for having me, Alan. That’s quite an introduction. I really appreciate it.

Alan: Well, you certainly deserve it. You have been in this industry since the very beginning; you have seen some incredible changes, and maybe you can speak to what you’ve seen in the last 30 years of being involved in virtual and augmented reality, from where you started to where you are today.

Elizabeth: Yeah sure. It’s actually quite a transformation I’ve witnessed. It really blows my mind in some regards. So, way back in the day when I started my career at Ford Motor Company, virtual reality was out there, but it wasn’t really a thing in enterprise, per se. And I really became interested in it and started working with it, I would say, before its time. So around the late 90s, I started working in that space and putting together, like, a life-sized human model that could scale to different proportions. We tracked the human through magnetic motion tracking. And since cars are made out of metal, that poses a little bit of a problem, so we created a wood — like, oak and mahogany — adjustable vehicle that could be a small car or a big truck, and then put you in it, and then changed you to be either like a super tall man or a very small woman, and let you do ergonomic assessments. So at that time, we were limited to 60 thousand polys in our entire scene.

Alan: Wow!

Elizabeth: I know! So we were culling data and massaging things and we’d say, you know, two weeks and we’ll have something for you. And we were really working hard because you have to try to represent a vehicle and a person and an environment in 60 thousand polys. So you can imagine what it looked like; it wasn’t very pretty. But we actually were able to get some value out of it. So we progressed from those days to working more with better tools and optical motion tracking became a thing. So that was a big advancement. So we can now work within prototypes of vehicles and it really opened up another whole set of possibilities for us. And so we worked in that regard, and at that time, I really realized the benefit of doing passive haptics. So, putting some of the physical world together with a lot of the virtual world is a really wise idea. And I still think that holds true today.

So we worked on that, and then really the tech evolved, got better visual quality. And then the next big advancement was really the amount of data that we could put in the model and study. So instead of these 60 thousand polygons that were so limiting in what we could do, we were able to get engineering data — real vehicle data — that represented the CAD geometry that was being produced and had a lot of engineering integrity behind it, and then start to get other aspects of engineering, like some computer-aided engineering, some analytics, in the models, and kind of take it from there. Then from there, went to real-time ray tracing, looking at an environment that had real-time reflections, and it measured materials and all of that goodness. So in the end, by the time I left Ford Motor Company, it was quite a nice suite of tools that had a very deep, foundational use in product development, and for manufacturing assessments, too.

Alan: I wonder if it would be, you know, something that would even be possible would be to recreate the timeline of VR through the lens of what you guys were working on, from the 60 thousand polys, to real-time ray tracing — and for those of you who are listening who don’t know, ray tracing is the ability to create reflections of light from different angles, and that’s really important when you’re looking at a vehicle, especially. You want to see, what does it look like in the daylight? In the evening? When the moon shining off of it?

And you guys, you showed on one of your presentations, probably the most photorealistic-looking vehicle. I mean, if you looked at it, you would think it was just a photograph of a car. But it’s in VR. And so you’re looking at this car and it looks exactly like a car standing in front of you. And, I mean, that is a far cry from where you guys started off, and it would be interesting to see in a virtual reality timeline that sort of progression.

Elizabeth: Indeed. Right. And so the interesting thing about that is, the realism actually comes from physics, and I love that. I think that’s how it should be. So, the way the light is being propagated in the scene is based on calculations of the way light behaves physically, and the materials that are in the scene are defined according to how they appear physically. And I think that is a differentiator in how you do virtual reality and actually the visual part of VR for enterprise, because the direction that you’re heading is foundationally correct. It’s not, “Oh this looks really cool it looks really real.” It actually has science behind it, and I think that’s a really important distinction.

Alan: So we’ve kind of come a long way since the beginning days, but let’s talk more about the the actual ROI use case of what you guys were doing at Ford, and what you’re kind of propagating with your new company Immersionary Enterprises. I think the biggest thing that you mentioned is the collaborative tools; being able to collaborate. And I know there was a study recently, or a kind of a case study of Bell helicopter. Normally takes five to six years to design a helicopter, and using virtual reality, they did it in six months. Were you guys seeing similar or… you know, were you seeing better/faster times to market, and what does that look like?

Elizabeth: Absolutely. Better, faster time to market; more things that could be studied in a shorter period of time. So in other words, getting answers sooner, being directionally correct sooner. And also, the way in which the teams worked changed so that, not only did the answers happen better and you get, you know, faster results for those answers, but you’re able to propagate the information sooner, as well. So, more teams can benefit. So there’s a ripple effect in the way product development is done and the way that the information is obtained and disseminated.

Alan: And it is real time, too, so real-time collaboration really is key here. I think one of the things that you are really pushing towards is real-time collaboration and communication. And I think being able to stand in… one of the first experiences I ever had in social VR was in Altspace, and I just remember somebody talking to me, and I turn around, and I couldn’t… I was like, “what?” They said, “oh, you’re new here obviously, because, you know, you don’t know what’s going on,” and just that ability to see somebody else, talk to them. But I mean, you guys took it to the next level. I think in one of the talks, you mentioned that your executives at Ford all go into VR now, before a car is even made.

Elizabeth: Absolutely, and I thought long and hard about what VR would be good for. What AR would be good for. What unique advantages it may or may not have. And one of the things that holds true that I think you’re getting at, is immersion is social. It allows us to amplify meaningful communications, and we can create these infinitely-scalable, connected spaces where we can all relate to the thing we’re producing. Or whatever your enterprise is doing. Everybody knows that, and they’re masters in their own discipline in that thing that you’re making. And then, when you come together as a globally-connected team, you can really create this experience where everyone has a voice. Everyone’s function gets to be properly represented in context, and it allows these really complex stories to be told amongst these multidisciplinary teams in a way that’s, I believe, like no other form of communication.

Alan: It’s pretty incredible. My interview earlier today was with the president of HTC VIVE, Alvin Wang Graylin, and one of the things that they announced last week at the VIVE conference is that they’re now doing eye tracking, and hand tracking, and lip tracking, and I think these are new — fairly new — technologies. I mean, I’m sure you guys were doing some sort of hand tracking with either controllers or gloves, but to be able to have native hand tracking — just put on a headset, you can see what other people are doing, you can engage with them — that’s a game changer, and being able to look somebody in the eyes and understand their body language. I think in a design standpoint when you’re having conversations, I think that’s very important. Can you maybe speak to how these types of new technologies are going to really enhance the experience for users?

Elizabeth: Yeah, absolutely. So a couple points stand out to me on that. One is that immersion is a holistic paradigm. So, you can represent; you can be represented; you can see others who are represented, and then you can look at a world that really doesn’t exist. And then, the more we move toward experiences that have zero physical interface, where we have to put on something or do something different in order to get in that world, the better off we are. And even though, with a VIVE, of course, you’re wearing a device on your face, but it’s doing so much.

I mean, that is just really exciting technology because now you can represent some form of communication regarding how you’re feeling about things, and it kind of breaks down some of those barriers that I think are there when — especially an executive, a C-Suite person — puts a headset on and then they can’t see anyone anymore, and they’re wondering what other people are reacting to. I think it’s important to have the dynamic of the person in that environment also represented. So the more we add in to the environment that is like the physical world, and the easier it is, so we don’t feel like we’re a cyborg when we get in that world, the better this technology will take off and be adapted.

Alan: I agree, and I think there’s some interesting kind of overlap between, you know, virtual reality and mixed reality, or being able to see these types of design communications in augmented reality, and I know Hololens is really leading the way with enterprise augmented reality/mixed reality. Is that a technology that you guys used before, or that you’re using now with clients?

Elizabeth: Oh yeah absolutely. Mixed reality is extremely beneficial. There is a lot of goodness in the physical world as the main part of your experience, and then augmenting that with virtual data that represents the art of the possible. What I love about the immersive paradigm is, you can go full virtual or really full physical because you can be immersed in something physical, and really get benefit out of learning about what’s new.

Alan: Absolutely. So let me ask you more on a personal note; what is one of the best VR experiences you’ve ever had?

Elizabeth: Oh wow, that’s such a great question. Let me think. I don’t know if I could say just one. I will tell you… how about if I tell you one of the most meaningful VR experiences I’ve ever had? How does that sound?

Alan: Sounds great. Sure. And you can list many of you want. I mean, we’re here to learn.

Elizabeth: Okay. So, all right. So, I think the most meaningful experience I had was around putting together the physical world and the virtual world about the time when I would say, I had my new eyes, and I could see in stereo. That is the most meaningful experience I’ve ever had, because I…

Alan: Alright, alright — we’ve got to just go back a little bit, because I know this story and I think it’s important. So, you’ve spent your entire career working in virtual worlds. And up until fairly recently, you couldn’t see in three dimensions, in stereoscopic view. Is that correct?

Elizabeth: That is true.

Alan: So here you are, leveraging the power of a technology that just screams “three dimensions,” and you couldn’t experience it.

Elizabeth: That’s right.

Alan: So you got a surgery. They fix that. And what was it like to go in the first time after your surgery, and experience full three dimensions in virtual space?

Elizabeth: It was… it was amazing. So after I had my surgeries, they basically… I describe it in automotive terms: I had bad camber, castor, and tow. So, my eyes were misaligned in three directions, and they basically realigned them, and it allowed me to triangulate and form a stereoscopic image, and see in 3D. After a period of adjustment — because you can imagine, my world was all-new, and everything was really great and really horrible at the same time — but the first time I got immersed, and I looked at the data and I could see, I stood behind a vehicle and it had, it was like a hatchback. And they opened up the hatch so I could see in, and the feeling of, like, the vanishing point in the seats in front of the seats, and just… and I was moving my head, and I watched the data, you know, move dimensionally with me. I had never seen that before.

And so, what it taught me — and this is the reason why I’m bringing it up — is I think the immersive paradigm has a cognitive/emotional aspect to it that you can’t get by looking at a flat screen. So the meaning behind it, and what I got out of it was, that the emotional connection you get when you are in your world, or you are in a virtual world, and you are learning and sharing and discovering with other people, that is really profound. And when you share something with somebody, and you’re together and excited as a team about this product that you’re putting together? I never really fully understood the power of the connection of immersion before I had my surgeries. I just thought it was useful because it was virtual data, and you could say Option A and Option B and kind of look at things. That’s really where I was at. And then when I had my surgeries, I was just blown away by all of the information that I could get out of this environment, and how I could talk to somebody else about it and they related to it.

Alan: It’s interesting that you talk about the amount of data, and you’re a very analytical person. I’m sure over the course of your career, you’ve managed to collect millions of data points of data from each of these things. What is the main driver? Ford’s a very large company, and they can afford to have kind of skunkworks departments and things like this. But your department wasn’t the skunkworks. This isn’t some VR lab in the corner that is used once a week. This is something that is used by designers and C-levels right across the enterprise all the time now as a tool. This is like, you’re having Adobe and having computers on your desk; this is not a kitschy toy. This is a real tool. When did it go from being a skunkworks project to being a real, validated tool that drives ROI?

Elizabeth: Yeah, so, that’s funny, because while you’re talking, I’m thinking about all the times I was, like, in a garage with a space heater over my head. It was really interesting because I was always grateful that Ford let me try and let me experiment until the time was right to deploy. I really always appreciated that I had that capability, to be like a startup in a multinational company. But I didn’t necessarily have a lot of resources to get it done for a long time, because the timing just wasn’t right for the tech. But the answer to your question is, the collaborative paradigm was the one single thing that sold the tech. So in, I think it was in 2012, there was a need to do a series of collaborative reviews with Australia, and there was a large contingent of people that — including an executive team — that wanted to go to Australia, but had a hard time with their schedules. And then I asked if they could try doing a global collaborative immersive review instead. So I literally had a, I don’t know, maybe 10 or 15 C-level people in my lab, which was really a hoist area. It was a garage at the time. And one guy is holding a dowel rod with you know motion markers on them — mo-cap markers — and a headset, and they’re looking at a 46-inch-screen TV, but they’re seeing what somebody from Australia is seeing, and then somebody from Dearborn, Michigan was looking at, what conversely somebody in Australia is doing.

And that collaboration really sealed it, and they could immediately start to discover things about the product that were good, things that needed changing, and they canceled the trip to Australia and they just did a series of immersive reviews. They could see the power in it, and at that point, some investment came and the technology grew from the garage-band-type approach, to a very well-structured enterprise deployment, where battle prep was handled easily. So, we worked out pipelines and platform issues, and worked to make it global so that we had countries from around the world participating in these immersive reviews. And so, when I left, we looked back and there were… I looked at the amount of attendees going to reviews in a year, and there was over 10,000 people who somehow or other witnessed an immersive review in a year, globally. That’s just phenomenal.

Alan: That’s incredible. Wow. It’s incredible. So you literally kicked off collaborative immersion tools for one of the world’s largest companies.

Elizabeth: Yeah, I know. Wow, cool, when you say it that way! One of the things I probably should point out is… well, a couple of things. One is, nobody does this alone. I worked with a lot of really smart people to make things happen. As they say when you’re raising children, you can only take part of the credit and part of the blame for whatever happens. And another thing that I think is really important, and another part of the reason why it took off (besides the obvious benefits of collaboration), it was the simplicity of how we work with immersive technology. I think it’s really, really important to provide an incredibly simple, very useful method to get immersed in a world. So I came up with these things called, like, “The Tenets of Immersion,” and they were–

Alan: Ooh!

Elizabeth: I know, sounds so formal, but it’s basically what we would do, and we would try to never violate these tenets, so that people could come in and understand their data with little or no training. And so, by the end, I think it’s basically no training. Like, literally put this thing on your head and start moving around in the virtual world and it’s going to make sense. We got it down to about 30 seconds by the time I left. But I think that’s important to know; know your audience for enterprise, and know that they’re extremely busy people, and they really don’t want to or shouldn’t have to take the time to learn a whole new way of interacting with data.

Alan: What are the Tenets of Immersion that you’ve come up with? Can you outline those, or some of them?

Elizabeth: Yeah. So, the Tenets of Immersion are really about how quick and easy it is to get immersed. The other thing I called them at one time was “the prime directive.” Basically, it will be quick and easy to get immersed. It will allow… we can simulate any potential reality. (I’m going from memory here.) The hardware that we use will be simple. It will be unobtrusive, and it will allow a natural navigation and interaction with the virtual world. And then, regarding what you see and how you experience it, it will be realistic when that’s… especially for enterprise and what it has to do with engineering, realism is key. Sometimes with art, it’s not. If you’re trying to do something more artsy, you really don’t want realism. But anyway — realistic, real-time, so as far as keeping up frame rates and making sure the experience is a smooth and steady one. Collaborative, so that you can share between the people in the team, and for automotive, I think we also had that it should be full-scale, for when we were looking at vehicles. So, looking at a model and not knowing how you relate to that data could be death for understanding how to assess the goodness or badness of the features. So if you’re looking at tolerance between body panels, and you have no idea what your frame of reference is with the vehicle that you’re looking at, you can’t really assess accurately if that’s a good or a bad margin.

Alan: It’s a really good point. So to recap, the Tenets of Immersion are: how quick and easy it can be get it right into it and get immersed; the fact that you can simulate any potential environment or application; the hardware has to be simple and unobtrusive and work in a natural way; it has to be realistic, both in graphics and in user interface; it has to be real-time — the frame rates must be quick and fast; it must be collaborative; and also, be able to be in full scale so you understand what it really does. Is there anything else that we’ve missed?

Elizabeth: No. That’s it.

Alan: Wonderful. That is a great framework. I’m going to put that in the show notes for people because I think that’s really important. One of the interviews I did earlier today was with Alvin from HTC VIVE, and what they’ve just introduced last week is a multi-user experience, up to 40 users at once using only four trackers, and they can do up to 900,000 square foot space.

Elizabeth: That’s awesome.

Alan: Right? And these are tetherless, so these are the VIVE Focus, which are the standalone headsets. Now, knowing that, how is that going to change how enterprises use this? You don’t need a powerful computer anymore; you don’t need a backpack. You just put on his headset, you got hand controllers, six degrees of freedom, and you can walk around in a 900,000 square foot collaborative space. How would that have changed the work you were doing there, and now do you think that’s going to impact the work that people are doing around the world?

Elizabeth: That is amazing, and I think it will have a game-changing effect. For what I was doing at Ford, and what I would recommend for a lot of enterprise, I think it should be used with caution, and be used judiciously. So, what I could see this being really good for is… there are times when a product gets to a certain phase, where you build one. So let’s say you build a prototype of an airplane, and you have this whole plane in its entirety. Maybe you’re looking at inside the cabin, and you’re looking at the issues associated with the passengers in-cabin.

I can see, for aerospace, doing interactions with a group of passengers and doing roleplaying and, like, storytelling with multiple people, and really going through a whole scenario about the usability and the ergo and UX concerns for in-cabin experiences. I can see it for different people that have different functions, being able to all say things, be together in the environment and then point out their concern: kind of sharing the ball so-to-speak, and working through issues, just like you would if you were standing at a physical model, or working together in the physical world. I think the cautionary note is that, just like meetings when everybody’s talking, there will be some rule — Robert’s Rules of Order — for immersion. I think, now, that these possibilities exist for us.

Alan: I think that’s a really good point. I think some of the social VR things, like Altspace and Facebook Spaces and these things, they’ve actually built in certain protocols that you can basically silence everybody, or silence people that are kind of outside of your purview. The other thing they’ve done, and this is something that nobody really would have thought of, is creating like a personal space bubble. So when you’re in VR, you can walk right up to somebody and you can actually walk through, them because they’re not real. I think the personal space issue is real, and I think it’s only when you’re in a virtual space and somebody walks up really close to you, it’s this freaky, “hey! I’m actually here! What are you doing?”.

Elizabeth: Right. Exactly. Exactly! And we’ll need to… I think there will be many awkward immersions while we figure out how these things should be deployed, and how we work together and share. And it’s a net gain. We just need to do it properly so that VR doesn’t get a bad name again, for another whole set of reasons.

Alan: Absolutely. And I think that leads me to a question, and this is interesting for the listeners, I think, to understand where we are in kind of… let’s say, zero being 20 years ago where it didn’t really work — even five years ago, when we didn’t really have VR — to where it’s like, completely ubiquitous and everybody uses it, it’s used in every company for everything. Where do you think we are on that zero-to-10 timeline, let’s say?

Elizabeth: For VR use in deployment and enterprise?

Alan: VR and AR – all of the XR technologies.

Elizabeth: XR tech in enterprise? I think it’s more… we’re probably at a 6 for VR, and 2 for AR. That’s what I would say. We have a lot of capabilities to yet conquer; a lot of data that still can be embedded. So I still think we’re more at the beginning of this journey than solid implementation. Years ago — not very many, maybe two years ago — I brought a gentleman into Ford, and he was a supplier. He said, “you know Elizabeth, usually when I come to a big enterprise like this, I meet the VR person and then they take me into the basement, to a room with a bunch of cables and headsets, and that’s where they hold these evaluations.” And that really struck me, because I think he was nicely complimenting the VIVE technology that we had at Ford. But the thing about it is, is I still think, in some regards, we’re there. And we need to get to the point where the immersive paradigms… all XR is part of the platform by which companies do product development, and then there is a pipeline so that you can get data easy and you can see it when it’s relevant, and look at it in context.

Alan: It’s interesting. I think Microsoft is really especially pioneering this, with their mixed reality platforms. They’re really pushing the limits of this technology, and they realize the same thing that you just said: if the systems, the new XR systems — you know, Hololens and these — if they don’t work with traditional systems, if you can’t import your CAD models or your BIM plans, or if you can’t instantly use the tools and toolsets that you’re already working with, this technology is not really going to take off. So I think you nailed it on that one. One of the questions I have for everybody is, what is the most impressive business use case of this technology that you have seen?

Elizabeth: Oh. I would say the most impressive use case is looking at a vehicle in the context of all of the possibilities… or, that represent a large amount of possibilities of how it could be built, and then being able to bring that data in — so, representing any potential build condition — prior to the meeting. So in other words, literally walking in and saying, “I just emailed you a spreadsheet, a big CSV file that has all of the tolerance conditions that happen on this car,” apply it to my car, and then look at it. To me, that was the first time–

Alan: That’s awesome.

Elizabeth: –we did that. I was just, like, blown away, because first of all, there was no pre-work. That gets back to my earlier comment about platform and pipeline. It’s built on how the company works. And then second of all, that you can study all of those things, and decide which things are relevant, and then look at the vehicle in that regard. We kind of do a detective work going back, so that you can apply these conditions to that runtime module. That’s just amazing to me.

Alan: It’s fantastic. I think there’s just… I mean, every time I do one of these interviews there’s completely new ways of using this technology. It blows my mind every single time. I tried VR five years ago, and I put it on my head and it was a concert, and I was standing on stage — that was kind of my “aha” moment when I really realized that this is the future of human communications. And for the last five years, I’ve been studying this industry, looking at all the business use cases. Medical, there’s a huge push in medical to use VR for training and for pre-visualizations of surgeries, and for prepping patients to know what’s going to happen, and showing doctors visualizations of surgeries, and pharmaceuticals how they work. And that’s medical.

Then you’ve got engineering, design, you’ve got HR training, now. It’s being used for empathy training, it’s being used for retail… there’s literally no end to this. And I mean, if you look at what you guys were doing in the design side of Ford, you also were doing it on the sales and promotion side, at car shows and showing potential customers what the new cars that aren’t even built yet are going to look like.

Elizabeth: Exactly. I know! And then, you think that the same data that you’re using for development, you can use for marketing. As time marches on and the data becomes mature and then it gets ready for prime time so to speak. That’s just a whole progression, and a certainty in the way that you’re working that is just… it’s really beneficial and it’s highly productive.

Alan: I love the idea of being able to use the data for engineering and that sort of thing, and design, but also then send that over to your developers, and this is something that I’ve been trying to articulate with the workshops that we’re doing as well, is that it costs a lot of money to build a 3D object. Let’s say you want to build a new car: you’ve got to build the seats and it’s quite costly, but once you have that 3D model, you can use it for design. You can use it for sales and marketing. You can use it in AR for, let’s say, Snapchat: you can export it as a different file, you can use it for Snapchat or Facebook. There’s a million ways you can use it. On web, as a web 3D visualizer. There’s a million ways to use these 3D assets. One of the things that I see as being one of the biggest potential possibilities of this technology is, once the world moves to spatial computing, every single object — whether it be a car or a bottle of water — will need to be converted to 3D. And so, there’s a huge push now for 3D artists and graphic renderers to create this digital twinning of the world. And I think it’s a really awesome space to be in, and I’m really super excited for it.

So, my question is, what do you see for the future of XR technologies as it relates and pertains to business?

Elizabeth: For that answer, and related to what you just said, I see a really great integration with AI. A deep-learning approach where what we’re seeing, we’re actually getting data imparted to us, and imparting data to others. So, if you just think about the power, how much are we throwing away right now in enterprise by looking at problems and then not realizing, like, through the pipelines what those problems are? So if somebody — a human — inherently sees that there’s a problem, they can flag it is an issue somehow. That’s a way. But just think if you look at data, and either you flag it as a problem, or the system realizes that maybe the tolerances are off, or the material is wrong, or that just whatever the issue is, and then provides you the relevant information about that so that you can start to solve the problem. Just think how cool that would be.

Alan: It’s incredible. You know, I kind of study futurism a little bit, and I dabble in what’s to come, what’s going to happen when AI and robotics replace a lot of the jobs. But I really think they’re, while they’ll replace jobs, they’re going to create so many more opportunities than… we can’t even think of what they’re going to create. And you nailed it when you said bringing AI into the mix is really part of it. Machine learning, computer vision. These are deep, deep neural networks. These are all parts and parcels, that you can’t really have virtual and augmented reality — true performance — without 5G, without IOT sensors. Being able to walk through a factory floor, and even though the robots are all kind of working away, being able to put on a headset and easily look at the machines and see a green, red, or yellow light above them, and as you walk closer, that red light opens up and tells you the full information about that system. That’s already happening now. But I mean, we’re only scratching the surface of what’s possible.

Elizabeth: Exactly. And that’s why, on that continuum of where are we with XR technologies, I really think we’re just starting, because there’s this whole component. I mean, just think about training a model as you’re immersed.

Alan: That’s going to be crazy. It’s crazy. I mean, here’s the other thing: I just read a paper on collecting personal data, but it’s not from your surfing history; it’s literally your eye tracking, because all the new headsets are going to have eye tracking. They also have positional tracking, so they know your height. They know your gait. They know how you walk and how you move, and they’re collecting body language data in a way that we’ve never ever in human history been able to collect. I mean you can do body pose estimations and stuff, but nothing as accurate as hand tracking and facial tracking. HTC VIVE announced last week they’re even doing lip tracking. So you’re able to capture the true essence of somebody’s intent without them even knowing it. I mean, it opens up some crazy privacy issues. But for enterprise, this unlocks a new level of data set. And of course, it’s it’s unlocking crazy amounts of data that we’re gonna need AI to analyze, but this unlocks so much potential in the enterprise.

Elizabeth: It’s incredible. It’s it really is an exciting time to be in this space. And I agree with you, that although some jobs might be obsolete, others will come. And I think it will provide even more opportunities, and more exciting ways to make a living, I suppose, in the future.

Alan: Absolutely. I think…I actually need to go back in my notes here for a second, because something that I read this morning… it’s going to give us better ways to work. Being able to, you know… most people sit at a cubicle. But being able to put on a headset, and instead of looking at a 20-inch screen, I can now have 20 20-inch screens around me or one 100-inch screen. And instead of sitting in a cubicle, I’m now sitting on a beach. Just being able to give people a better environment in which to work, I think, is really going to decrease stress.

As things move faster and faster, these tools are gonna give us the ability to do work faster, which is more efficient. And I think everybody is kind of running on a treadmill, trying to catch up. But I think these tools will really give us the opportunity to catch up, and get ahead, because we’re entering in — as you know — we’re entering the exponential age of humanity and we don’t really know what’s going to come out of this.

Elizabeth: Exactly. It’s so true.

Alan: 10 years ago, app developer wasn’t a job. Now it’s everybody’s job. And in 10 years from now, is coder going to be a job? Or is code going to code itself? We don’t know.

Elizabeth: Right. Right! Exactly! It’s fun times. But you’re right about better ways to work, and better environments. I think, if you look at the progression of the change of work, for a lot of people, their office is at home. And I think the immersive paradigm will allow people to have their office be their home, but then also be able to be connected virtually with somebody else who’s also, maybe their office is their home, and maybe now they’re in a collaborative session, or maybe they’re looking at the product that they’re analyzing in some regard. I mean, it’s so much better.

Alan: I can’t wait. So, I’m going to ask you one more question, and then we’ll wrap up. For the listeners that are just learning about VR and AR and MR for the first time, what is your practical advice for somebody who’s looking at this for the very first time, to get started in this technology? What is your recommendation for them to just start working in this technology?

Elizabeth: So I would say, tackle a problem that you know needs a solution. Find something in your organization that is a persistent issue, that is really tough to solve with the standard practices that are used to tackle that problem currently. And then apply some form of XR to that problem and show it for the benefits that you’re getting out of it, and show how the XR platform can be used to understand deeply a problem, and communicate effectively between the different disciplines.

And then tackle things on a case-by-case basis for a period of time, and then build up a library of related cases that you can start getting people in your enterprise to see that these things are… they’re not just disconnected things. They actually relate to how we’re making our product, and we have these valuable ways of gaining insight that we did not in the past.

Alan: Well, that is some sage advice from VR pioneer Elizabeth Baron. Thank you so much. I’m just so honored to have you on the show, and I hope our listeners learned a lot on this. We learned about the Tenets of Immersion; the rules of order. It’s been a fantastic conversation. Thank you very much.

Elizabeth: You’re welcome Alan. Thank you very much for having me. It was a pleasure to have this conversation.

Training employees can be like pulling teeth – both for the employees and management. But thanks to XR technologies, Jonathan Moss has had success in finding new, innovative ways to get the team engaged in training at Sprint.

Moss chats with Alan, discussing how XR can be used to invigorate a corporate team, save a company millions, and how it can all be done in-house with a moderate investment.

Alan: Our guest today is Jonathan Moss head of learning, technology, and sales enablement and XR strategy at Sprint. Jonathan and his team proposed to unleash everyone’s potential by evolving the experience and growing Sprint’s consumer sales organization, through learning and sales enablement. Concurrently, they are taking on the industry through utilization of technology to disrupt and design learning that is different from what we’ve ever experienced to date.

They are on a mission to eliminate dull and ineffective training. Jonathan is a lifetime learner that continues to challenge today’s norms by thinking in terms of possibilities and realities. His team is working with startups and experts to develop virtual training, mixed reality, real gaming for learning — not just points and badges — and has already launched the ability to serve up content at the point of need, using augmented reality.

Jonathan has had the pleasure of leading teams up to 250 people that spanned the entire country, with operating revenues of 14 billion dollars. They have implemented strategies that have changed the growth trajectory of people and results through leadership and employee development programs, redesigning sales processes, integrating technology for improved customer journeys, and cost saving efficiencies, creating operational models that optimize profitability and executing on the fundamentals of business using virtual, augmented, and mixed reality technologies. For more information about Sprint you can visit Sprint.com, and you can follow Jonathan on LinkedIn or on Twitter, and it’s @Jonathan Moss.

Jonathan Moss, welcome to the show.

Jonathan: Hey Alan, great to be here. Thanks for letting me join.

Alan: It’s my absolute pleasure. You and I have connected so many times, and you know I’ve been really looking forward to this interview. You are a pioneer, a leader, and an industry pundit. The work you’re doing — both at Sprint, but also on collaborating with everybody through the virtual and augmented reality association — is fantastic. So thank you for all the work you do. We can’t wait to learn more about what you’re doing, and really drive this message that virtual and augmented reality are not only here, but they’re transforming businesses.

Jonathan: Absolutely. Absolutely.

Alan: So tell me, let’s start with what you are doing on your day-to-day basis that Sprint, and what are some of the things you’re doing right now?

Jonathan: Yeah. Awesome. So a few things that we’re really doing now is trying to understand all the different technology. So, from mobile AR, virtual reality, mixed reality, and really seeing how we can utilize it for our learning curriculum. What we’ve understood is that this is not only a better scalable way for folks to learn, but also with the immersion it allows for elimination of digital distraction, as well as activating some of the brain regions that we need, and we understand from science, that will allow our learners to retain and apply the things that we’re trying to teach them better. So we’re super excited about the technology and all of its use cases that we’re using today.

Alan: Wow that’s incredible. So, what are some of the use cases that are kind of working right now?

Jonathan: Last July, we deployed a mobile AR application called Sprint ARx. It was our first use case, and what we found was, obviously, there’s a plethora of information out there on the Internet; phones, technology are ever-changing, with the features, the benefits, all of the new new things every single time one launches. And also, with the competitive nature of our industry, the pricing continues to change. Services continue to adapt, continue to change depending upon the plan that you get for consumers. It’s really hard to keep up with these days, for anyone to remember.

So what we did was, we really wanted to activate an experience inside of every one of our locations for our employees, is where it started, and then we eventually want to get to our customers. With our mobile AR application, what we do is we have over 20 experiences in the store that are triggered from in-store merchandising. So, the messaging on the walls, the POP — those types of things. What it does is it allows not only a training to happen — so, it delivers information about a specific plan, a specific product, a specific service that we offer — but also is a really good side-by-side selling aid for them to use with customers. So what we’ve seen is, we’ve seen our employees actually use this, not only during a customer transaction, but also during their downtime, and also when a new product launches.

So we’re really finding that they are utilizing it in multiple ways to not only learn about the new products and services, but also retain the information and continue to come back again and again. So that’s one of our first use cases that we delivered. Another use case–

Alan: It seems like a pretty great first use case! “Our first one, it increased sales, it made employee retention rates higher, training was through the roof.” Like… was there any downside to it?

Jonathan: No, absolutely not. It was very cost effective, easy to scale. We actually rewrote our entire new hire curriculum and, mobile AR plays a major component. So, not only in our store, but also in our classroom. And to your point, we’ve seen retention rates go up. We measure that through mystery shops. So, how knowledgeable are our employees, and how confident are they when they deliver the benefits about a product and service? And we’ve also seen decreased speed to competency coming out of new hire training; in fact, our learners coming out of our new hire training or actually beating the enterprise and all of our KPIs in their first full month of selling. And we have a lot to say around the AR technology, and how we’ve developed it and designed it and integrated it inside of our learning curriculum to thank for that.

Alan: Wow. So you’re, like, building this right in.

Jonathan: Absolutely. It’s part of, I think, six or seven days of our 15-day curriculum we actually have mobile AR components built in.

Alan: Wow. So, a lot of companies are just starting to create POCs, they’re just trying to starting to test this out. You guys, you’re beyond that; you’re rolling this out at scale.

Jonathan: Yes, for mobile AR we are. For some proof of concept, we are exploring mixed reality. So currently what we’re looking to do is, we’re actually looking to replace our instructor-led training and all of our product and service training inside of mixed reality. This will give us an ability to not only train more mercifully, we can…our goal is to replace our LMS. We would love to replace our LMS; I’ve never met an organization that says that they’re in love with theirs. So that’s obviously one of our goals.

What we can also do is we can decrease some of our operating expenses by bringing people in and we can increase our time to train as well. So, we can decrease them coming out of the store, the amount of time they’re out of the store; we can increase the amount of time that we train them; and then we can deliver it in such a way with the technology to where they retain it better and come out of the experience and are able to be confident and sell the products and services and be knowledgeable about them as well. So we’re really excited about this proof of concept that we’ll be launching here soon.

Alan: This is really impressive. Almost every single interview on this podcast ends up with somebody talking about virtual and augmented reality’s use in training. Whether we’re talking about airlines and putting headsets on people while they’re while they’re flying, it always ends up, too, “hey, we can also use it to train our flight attendants!” And then retail. “Oh, we’re using this for marketing,” and then, “oh, by the way, we also use it for training.” Are you finding that some of the other parts of Sprint are starting to come to you and say, “hey, we saw that thing you’re doing for training; can we use that for marketing?”

Jonathan: Absolutely. Marketing is the one of the first ones to knock on our door, to see, hey, how can we not only integrate ourselves into your existing AR in-store experience; we’ve actually ran some sweepstakes and some fun things inside of the environment from a marketing standpoint, but now they’re actually looking to see, how can we expand that even broader? Our customer experience team has come to us. We’re working with digital, as well. So, more and more people are coming to us to kind of understand the technology, its abilities, and how they can actually use it in their own business units.

Alan: Very, very cool. It’s really interesting. You and I have talked about web-based augmented reality and stuff like that. Are you guys… is this stuff you’re doing mainly on mobile apps especially, the mobile AR. Is it all app based, or have you been experimenting with web based stuff?

Jonathan: It’s mainly been mobile app-based. We are looking at some of the Web AR stuff for more of the digital and maybe some other things for e-commerce. But right now, for our training applications, we have our Sprint ARx app that we’re typically utilizing the most.

Alan: So people that are listening, they get it. They’re like, “okay, I understand, it increases the amount of time of training we can do, increases retention rates, it decreases the time out of the stores, all these features of it. What are some of the challenges that you have gone through that you could share, that people don’t have to make those mistakes?

Jonathan: Yeah, a couple of challenges. I think one is, initially, we were going down this route without IT. If you’re a larger corporation, one of the things I can tell you is that IT can be your best friend or your worst enemy, so engage them early on in the process. I think another one is having some, in some of the experiences — and not necessarily mobile AR, mobile AR was probably our easy one — but once we started getting into some of the virtual reality/mixed reality components, we were kind of doing it off the side of our desk. And what I mean by that is, we had this proof of concept, we were really passionate about doing this, we understood the benefits, but we didn’t really go hire anyone that had Unity experience, or didn’t hire anyone that had 3D modeling experience. That put us in some snags later on as far as timelines, as far as being able to deploy on our timeline, we are now deployed upon other folks’ timelines. We had to go source out some of this work in order for us to achieve what our goals were.

And then I think the third thing that we ran into was we got executive buy in, but it was one of those things where they were really happy, about it almost like a new shiny toy. But then that started to fade over time. Really, what we had to do is go back and understand how we integrate it as part of our business processes. Thinking about this as just another option or another way to do business, instead of just kind of a cool thing or just a techie thing that someone’s working on — how do you integrate it in every workflow, or every part of the business that you can, that it would be able to be beneficial for your employees and/or consumers? So those would be the three top challenges that we ran into that we had to overcome.

Alan: Amazing. It’s interesting that you talk about those, because often, companies will come to us, the head of marketing will come to us and say, “hey, we want to do something in VR,” and you’re like, “okay? Why?” “Oh, well, our CEO went to a trade show and he saw VR and he said this the future, we’ve got to do it.” It’s like, “okay, well, what is the problem you’re solving here? Is there a reason for this? Or do we just want to make something cool?”

Jonathan: And that’s exactly what we saw, even when we started showing off the original use case for mobile AR, which was our first use case technology that we went after. To your point, everyone was like, “oh my gosh, it’s cool! What can it do? How can it do it?” Those types of things. But then again, the conversation started to dwindle. So we had to keep it coming back as top of mind, and as we beat off these experiences in the store and we got over 20, we were able to showcase all the different ways. And even our network team is another team that came knocking on our door to say, “hey, how do we unleash, inside of the stores, the ability for customers and employees to see our network improvements? We’ve had a lot more root metric wins and other network wins this year than we’ve ever had before; how can we share that with our employees and customers outside of e-mail or putting it on the Internet?” This was an experience that we actually launched in our stores, and we’ll be doing a similar thing once we navigate and launch 5G here in the upcoming next 30-45 days.

Alan: Sorry, say that again? You launching 5G when?

Jonathan: Hopefully in the next week, we’ll put it out there the next 30-45 days. So we should be in a good spot.

Alan: Holy crap! So, today is April 29th. You’re looking at mid-June… July 1? by July 4th.

Jonathan: Yeah. So we’ve stated, by the end of May to mid-June, we’ll be launching in a few markets. So we’re excited to do that.

Alan: That is really, really exciting stuff. You are probably the best person to ask about this. What will 5G enable for the consumer?

Jonathan: Yeah, so for the consumer, I think the initial enablement is really going to be around entertainment and gaming. You kind of see this with some of the big things out there. And I don’t think it stops there. And I think for some of our use cases, specifically around mixed reality or virtual reality, I think it’s going to open the door for the technology to just accelerate. And I think that the abilities that’s going to have with the low latency, the additional speed, the coverage, the additional capacity that you’ll have on the network, some of these things that are not only transmitting a ton of data, but also that the amount of speed at which that data transmits, and being able to have that low latency, I think is going to be beneficial in many ways. But I think it’ll take some time to adopt at the consumer level, as most of the use cases are in enterprise. I would see those types of things coming out the gate.

Alan: So let’s take that to a practical use case: if I buy a 5G phone, and in 45 days I have access to a 5G network, why would I need that?

Jonathan: I think the main reason that you need is just depending upon how you use it. So you may or may not need it, depending upon who you are and how you utilize the device, where you live, work, and play. What I would say, though, is if you stream movies, if you want high resolution — so 1080p, 4k or even some of the devices that we’ve seen will potentially have these 4k streams, and obviously, they’ll keep getting better and better. If you’re gaming, obviously that’s a huge market, eSports. You’ll actually be able to do a lot of the things that you do now and you see via Wi-Fi, with these data-intensive multiplayer games, you’ll be able to do those on your phone.

Alan: So I could play Fortnite with my phone?

Jonathan: Exactly.

Alan: Hell yeah!

Jonathan: If you’re a Fortnite fan, you will be in good shape, my friend.

Alan: Amazing. So I’ll be able to actually keep up now! Getting killed because of the lag. Awesome. So you talked about one of the challenges being bringing the teams together, building everything in-house. And then you mentioned something about getting stuck where you needed to bring people in and you were kind of on their timeline. What are some of the team members that you’ve built because of your work in VR and AR? Like, if I was a business and I wanted to build an internal team to do it? Or, where would it be appropriate to build an internal team, where would be appropriate to go off-shore, and who would be on that team?

Jonathan: So, I think it really depends on your specific needs and how in-depth you want to go. So like I said for mobile AR, my team was able to handle it with the software that we purchased, the player that we got. It was really something that was self-learned, so it wasn’t that difficult. When we got into some more robust… like, with the HoloLens and some other platforms, where we actually needed unity and 3D assets, that’s where we started getting into that snag with not having experience. So in my mind I would say, depending upon the cost model that you’re looking at, how much work that you want done, and how you know how robust it is, you could probably set up a very small team.

We have one Unity developer; we’re looking at 3D design, a designer to be able to put some of these things together. Really, it’s just gonna be a couple of people that will be working on this. So, you don’t have to go out and have a very robust team, even with some of the concepts that we’re looking at. It is good that, when we did outsource some of this work, we learned a ton from it, which was really helpful for us in the future and the way we’re going to work. So I would just say, it really depends upon how fast you need to deploy, and then how robust you need the experience to be. You could do it in-house or you could outsource.

Alan: That’s great advice for people. What are some of the metrics and specific key performance indicators that you guys measure for success? You mentioned time out of the store time to training. You mentioned how you test retention rates using mystery shoppers and stuff like that. What are some of the golden rules? How did you measure your success?

Jonathan: Yeah. So really, what we looked at was a couple of big buckets, and then we broke those down into sub-buckets. The big ones were sales, customer experience, turnover, operating expense, and so those are the four big buckets that we were looking at and say, “here’s our baseline.” Right? Before this technology existed, we knew exactly where we were at as an organization with our learning, and with the tools that we had. So let’s look at those things, and then let’s see how they improve or don’t improve. In each one of those categories — like in sales — there were five KPIs that we measured. In turnover, we looked at zero-to-90-day turnover, and that was specifically for a new hire curriculum. Also, for customer experience, obviously we looked at our CSAT scores when we hear back from our customers. And then for operating expenses, we looked at travel. That was the big one. And so those were really the main KPIs that we looked at to really model out an ROI, and what I can tell you with the improvements that we’ve had, we’ve actually not only added millions of dollars of incremental revenue to the business, but we’ve also saved millions of dollars in operating expenses. So being able to model out that ROI has been extremely important for us to continue on and get investment, get budget, in order for us to do this. So those are really our ROI metrics.

And then, as I mentioned, we looked at some other things, like you know time to train. So, how quickly can I train and then how quickly after that are you able to you know to feel confident to sell those types of things? We looked at how much non-selling time do we take folks out of the store today to train, versus being able to train them in the stores and eliminate that kind of non-productive time. And then also, for me, and being in learning, I think a big one that you can’t really put an ROI on, but I think it is of huge benefit for this technology, is now I’m able to match up the trainer’s strengths to the curriculum. So, I have some trainers that are really good at teaching sales. I have some trainers that are really good at teaching how to coach, or how to lead. I have some trainers that are really more operationally sound. And so what this technology will allow me to do that I can’t do now, because I’m bound by geography, right? So whatever trainer is closest to the training location and closest to where those stores are that’s the trainer that you get. What I’m now able to do is match these trainer strengths up to the exact curriculum that they’re teaching to improve their effectiveness. And I think that’s a huge win for us, and we’re looking at some ways now to be able to measure that, but we haven’t figured that one quite out yet. But I feel like that’s a big one for us.

Alan: I think that… you know what, you nailed that one. Because, you know, and I keep saying this; my daughter is in high school right now, and my two daughters are in school, and they go to math class, and they are not learning math from the top math teacher in the world. Period. They’re not learning anything from the top person in the world, at any point. So, here you are, a trainer that know great at sales training — really gets it, really understands it — but is then forced to teach six other things, just because of geography. The ability to use VR and AR to overcome that, and really, like you said, let them be the expert in what they do. That is… wow. That’s the first time I’ve actually heard that. That’s something you know, that… if nothing else, that’s amazing.

Jonathan: Absolutely. And you’re spot-on; you’re as passionate about the education sector as I am, specifically where there’s technology and kind of its current state. I think that, even in the learning industry, in corporate learning, this is huge for us. To your point, you think about some of the other use cases, like for field technicians, or the remote SMI, for doing work. You can apply that same concept in learning, and I can put the right person with the right group of people to teach them the right things, and then have that experience translate over to them. So we’re super excited about that.

Alan: I can’t wait for you guys to start training the AI algorithms.

Jonathan: (laughs) yeah.

Alan: Is that something on your radar? Are you guys looking at AI in training at all?

Jonathan: We are. So, we’re looking at a few things for AI specifically. We haven’t gotten deep into it yet, but we’re in talks with some external folks about it. But then also, internally, we use AI in our digital and also our customer care. And so, we’re looking at how can we translate some of that, some of what’s been built over there, and some of the learning over into training. So, currently exploring it — not in-depth with it yet, but yes, it’s on our radar for sure.

Alan: You mentioned how, by introducing this training, you’ve literally made billions and saved millions as an organization. But how much did you spend on developing these first protocols, and these first experiments? If we were to put a budget, did you spend a million dollars to develop this? Or was it five million, or fifty thousand? What kind of budget did you start with to get this off the ground?

Jonathan: Yeah, so, for mobile AR, so far we’ve spent less than fifteen thousand on it. So, not a big budget at all, to deliver what we’ve done. For virtual reality, which I know we haven’t spent much on for the 360 degree videos that we’re doing, we spent less than twenty thousand on that. We’re doing some things with leaders and soft skills and those types of things. Some operational training. And then for mixed reality, with the HoloLens, that’s obviously been the most expensive one. And right now, we are we’re into a pilot proof-of-concept, and so far, we spent less than eighty thousand dollars on that one. So, total, with all of these things that I’ve discussed, really less than a hundred thousand dollars year-to-date so for.

Alan: So, OK..(laughs) for the people listening, when we talk about exponential growth of technologies and of these types of things, this is a a prime example of this. You’ve spent, let’s call it a hundred thousand dollars, and you’ve saved the organization millions and made the organization millions of dollars. So, is there any reason that you can think of — at all — why an organization wouldn’t start working on this technology?

Jonathan: Not one. I’ve racked my brain around it, why more organizations haven’t, and haven’t at least looked at specific high-ROI use cases. Or maybe it’s low effort high return, right? So maybe the ROI isn’t high, but at least to prove out a proof of concept or a pilot, it wouldn’t take a lot of expense and a lot of resources. And I just wrap my brain around, really, why more organizations haven’t done these things, because again, we’ve seen so much great success from it. And it’s scalable, especially the mobile AR, it’s easily scalable. And so, I honestly can’t tell you one. I haven’t figured out one yet, to be honest.

Alan: Well, that answers that question! (both laugh) Have you exposed this this information out there? Have you been featured in any learning articles, or media? Has there been any kind of media generated around this?

Jonathan: Not to the degree that I like to, outside of just social and internally. We did do a PR release internally, a couple of them, to showcase it, and we’re looking to you know obviously broad in that scope externally. Obviously, with the VRARA — which is the Virtual Reality Augmented Reality Association — has been great, and we’ve connected with a lot of folks and other members, other folks that have seen things on social, and just had discussions with them. But one opportunity I am excited about that I’ll just give you a little snippet — I can’t divulge many details, but I’ll give you some exclusive — is, in the next couple of weeks, I’ll be meeting with Microsoft, and we’re looking to partner on a possible joint PR campaign with some of the work that we’re doing in the MR space. So we’ve definitely caught their attention, and I’m pretty excited about potentially what can come from that.

Alan: That’ll be amazing. That’s gonna be awesome. I can’t wait to see that. Let me ask you, do the HoloLens2?

Jonathan: I do not, unfortunately. They’ve been really stingy on those. I know that some of the developers that we work with have been working in them, so this experience over the next couple months that I mentioned — or, the next couple weeks — that we’ll be up there, we’re supposed to be able to get our hands on them. So I’m pretty excited about getting in there, and ultimately being able to see how we can even transform our learning even more.  With some of the additional functionality that the HoloLens2 has, we’re pretty excited about potentially even expanding what we were even thinking about doing with some of our experiences.

Alan: That’s incredible. I can’t wait. I mean, I have a HoloLens and a Magic Leap, and I got to try the HoloLens2, but they wouldn’t let me turn it on.

Jonathan: (laughs) They’ve been very secretive about it, but you know, for good reasons. But yeah, I think there will be a lot of folks like you and I who’re really excited to get our hands on it and test it out to see what it can do.

Alan: Indeed. So what, of these kind of developments in mobile AR/VR/360/mixed reality, what are the best lessons you’ve learned from these?

Jonathan: Honestly? How easy the technology is from a user experience standpoint. So, one of the things, when we got in this, we were concerned about — obviously, we’re concerned about the back end process, developing and stuff, and I know we’ve talked about that a little bit — but really, user experience. For folks that haven’t used AR before, or for folks that have not use mixed reality or VR, what would their… would they gravitate to it? Would they be more distracted, because they’re like, “oh this is great, this is cool,” and it was more of a cool factor than actually something that they utilized and then translate it into skill or behavior change? Education/information retention, those type of things. So we were really concerned about the folks adapting to it, and and really utilizing it. That was really something to me. What we did was we really designed it to be easy. Designed it to be intuitive, to where anyone, even if it’s the first time you’ve ever opened the technology — whether it’s an AR application, or whether was getting in the HoloLens, or really getting in virtual reality with the 360 video — we made it simple enough that people gravitated to it, and even the most un-techie person can get in there and actually know what they’re doing without instructions. That’s really what we had to think about with our design capability. So, I think that was probably the biggest for us.

Alan: Amazing. I’m just trying to think… what are some of the most important things that a business can do to start leveraging this now? If you were to give advice to the listeners to say, “here’s one thing you need to do,” what would that be?

Jonathan: I would really say, figure out what problem that you’re trying to solve. I think you mentioned it earlier, but what are some of the top problems that you have in your business that you’re either having trouble solving, or that take a lot of manpower or hours of work to do, or that are manual, and say, “hey, is there a technology out there that I can help solve, or make it easier for my employees, or reduce the time that they have to research something or find something?” Again, that was really our main use case for this, is that we wanted to think about organizing information for them at the point-of-need, versus them having to go search the Internet site for the latest How-To document or something like that. We really wanted to bring that experience to them, in kind of a push-versus-pull method. And we’ve seen great success with that.

And we’ve even done things to where you… think of this use case: So, if you have to fill out paperwork, how to properly fill it out — we still have paperwork that we fill out — now, we can open the augmented reality app, we can hold it over the paperwork, and we actually have an experience that will show you just like, it fills out the paperwork for you and shows you exactly what you’re supposed to do. Just little things like that translate into efficiencies, into less hours worked on certain tasks, less mistakes that we make that cost the business money. I think the biggest advice is list out the main problems that you have, and then have someone, either internally or externally, look at those and see how they could possibly be solved with mobile AR/VR/MR. I think that’s just taking that leap, and you’ll be glad that you did.

Alan: That’s some sage advice. Moving from advice to the listeners: I want to know, from your personal standpoint, what problem in the world do you want to see solved using XR technologies?

Jonathan: Me, personally? I would love to see education solved. I think that, just with the current state that education is in and looking at how we teach our youth — but then also we teach workers, right, so that translates into the classrooms that we have in corporate learning too — I would really love to see kind of a learning ecosystem developed for XR. Where teachers could easily design curriculum that would be unique to the student and the student’s interest, and give them that immersive learning to really help them not only learn the technology, but learn more about exactly what their interests are, and be able to translate that into the future. So if I had to say one industry or one thing I would love to be solved with XR technology, it would be education.

Alan: Well it sounds like you’re already paving the path for others to follow in. So, thank you for that

Jonathan: Absolutely.

Alan: Is there anything else you want to leave listeners with before we go?

Jonathan: I don’t think so. I think the one thing I would say is to everyone, again, I didn’t have a background in XR. I didn’t have a background in 3D modeling. I didn’t have a background in Unity. I didn’t have any… I didn’t work at a tech firm, right? That develop these technologies. And so, the reason I say that is because I think there’s a lot of people that are scared to take the leap, or do research, or talk to the folks that have been in the industry for a long time and learn from them. I would just say, you’ve got to take that step forward or you’re going to miss out or be left behind. That would be the one major piece of advice that I’d give any listener that has not experienced the technology or tried to solve any problems in their business with it.

Alan: Incredible. Well, I personally want to thank you for taking the time out of your busy schedule, and really, thank you again for being an industry leader, and the work that you’re doing and sharing it. That’s the other thing: there’s a lot of companies working on stuff like this, and they’re just keeping it really, really under wraps. And while I understand it from a competitive analysis, this is literally a huge competitive advantage for brands and companies. And so, I can understand them wanting to keep it under wraps, but this technology and the power of this technology is too great to be kept under wraps. So I appreciate and acknowledge the fact that you are willing to share that with the industry, and it means a lot. So thank you.

Jonathan: Absolutely. More we can get it out in awareness, I think the better everyone, you know, people and companies will be.

Alan: Indeed.

You’ve heard of “Comedians in Cars Getting Coffee”; get ready for “XR Moguls in an Uber at AWE.”

Alan holds a quick, guerilla-style backseat interview with Kris Kolo, Global Executive Director of the VR/AR Association. They discuss the awesome things they saw at the expo – like the VR pool party – and the VRARA’s upcoming Enterprise Summit at LiveWorx Boston on June 10.

The future is here, and it looks like…a hipster Colonel Sanders? Strange as it sounds, turning the antebellum-esque brand icon into a Millennial-friendly digital influencer is just one way that brands around the world are turning to XR to reach their audiences in inventive new ways.

Here to delve deeper into the ins-and-outs of marketing in XR is futurist Cathy Hackl, co-author of the brand-new book, Marketing New Realities. From Coachella, to virtual try-ons, and yes, even KFC’s “playboy chicken impresario,” Cathy explains how XR technologies will change the way we interact with our customers.

Alan: Today’s guest is Cathy Hackl. Cathy is a futurist, speaker, and amazing author. Cathy is an Emmy-nominated communicator-turned-augmented-and-virtual-reality-global-speaker, producer and, author. Cathy has worked with You Are Here Labs, HTC VIVE, as a virtual reality evangelist during the launch of their latest headset, the VIVE Pro, and during the company’s partnership with Warner Brothers blockbuster, Ready Player One. Cathy is also the co-author of Marketing New Realities: An Introduction to VR and AR Marketing, Branding and Communications. Cathy has been featured in media outlets like CNN, Silicon Beat, Entrepreneurs, CMO.com, Forbes Venture Beat, and so many more! She is a global adviser for the Virtual and Augmented Reality Association, and a leading voice in the VR space. For more information you can visit cathyhackl.com. It’s with great honor that I welcome developer/marketing specialist Cathy Hackl.

Welcome to the show, Cathy.

Cathy: Thank you Alan appreciate it. Happy to

Alan: It’s such a great pleasure to have you.

Cathy: I know! I’m really excited for your

Alan: Thank you so much. The idea with this

Cathy: For sure. So when we started the book

Alan: Absolutely incredible. So I actually

Cathy: 2017, was it? I think it was 2017.

Alan: I remember reading it on a plane, and I

Cathy: Yeah, I was… I thought you were gonna

Alan: Oh My God. Virtual and augmented

Cathy: I don’t know! I was really excited to

Alan: Snapchat released a lens this weekend,

Cathy: Yeah. I mean, it just… it lets a see

Alan: Harland Sanders. He’s like, their

Cathy: Yeah, I love it. I think he’s like a

Alan: And for those of you who don’t know, if

Cathy: It’s KFC’s handle!

Alan: Oh my goodness.

Cathy: Yeah it’s KFC.

Alan: They literally took this digital avatar

Cathy: “Secret recipe.” That’s what

Alan: Absolutely. I want to go back just for

Cathy: We’ll both be there.

Alan: We will be there. What are you doing

Cathy: I’m speaking on AI and XR.

Alan: Guess what I’m doing soon. I just got

Cathy: Oh, I love it! That’s fantastic.

Alan: It’s so nerdy!

Cathy: I love it.

Alan: I still don’t have a title for it yet,

Cathy: Yeah.

Alan: But Coachella. One of the things — to

So I know you worked with You Are Here

Cathy: You know, AR navigation I don’t know…

I know the DFW airport, Dallas Airport,

Alan: Yeah, A lot of companies and especially

Cathy: I think a WebXR, WebAR, whatever we

Alan: It makes sense. I mean, the thing is

Cathy: As long as the client’s happy and the numbers are there, then that works. I’ll give you an example: one of the clients at You Are Here that we worked on was Oldcastle. They’re a multibillion-dollar construction products company, and they’re already using VR and they wanted to find a way to use AR in a useful way. So we created an app with ARKit, an app for contractors. The whole point of the app is for the contractors to go into a client’s house — a potential client — measure the space with AR, and then immediately measuring the space through augmented reality and the camera, they’re able to pull up how many bags of concrete that person needs and where to buy it. And then also suggests different types of products that could be used, depending on how deep the slab of concrete has to be. So it’s been very useful, because before, they would go in with a measuring tape, they’d have to go back to a chart — like, it was just all these steps that, all of a sudden, they just come together through augmented reality, and it’s pretty precise.

We’re still waiting to see the metrics,

Alan: It’s interesting that, you know, I

Cathy: A contractor would use it every day. We

Alan: So you know what? Let’s talk about

Cathy: Exactly. It’s the visualization; it’s

And that’s what I hear a lot from

Alan: Absolutely. Absolutely. And you know,

Cathy: I mean, I’ve been through so many, and I always usually get asked this. I always go back to an experience I had at Tribeca Film Festival. That’s really my favorite film festival; I mean, Sundance is great, but for some reason Tribeca and I have this kind of relationship, let’s say (ha! I have a relationship with Tribeca). I did an experience there about two years ago called Treehugger, where you would put on a VR headset and you would have this trippy experience, and it had smell — you could smell the redwoods. It was just very powerful. I was really tired when I tried it on. I did the experience, which was like 10 minutes, and then I completely felt re energized after that. And even though things might have advanced technologically a lot more since that experience, it still stays in my mind because it was extremely powerful. I saw the power of being able to use these technologies for meditation, for something beyond just business, let’s say. And that to me was very powerful.

Alan: Was that the experience called Tree?

Cathy: Treehugger. Wawona Treehugger.

Alan: There’s another one tree in VR, where

Cathy: You *are* a tree. Yeah, no, this is

Alan: Incredible.

Cathy: That was pretty exciting. And yeah,

Alan: Incredible. It’s interesting that you

Cathy: That was my pivotal moment. I call that

And for me, it wasn’t until I did that

Alan: We all have that kind of “a-ha”

You mentioned Treehugger, and how you

Cathy: Not really. I haven’t really tried

Alan: There’s only been two for me. One of

Cathy: Oh, that’s funny.

Alan: The second one was Ghostbusters in VR!

Cathy: Oh, that’s right, yeah! That’s right.

Alan: You shoot Marshmallow Man, and all of a

Cathy: Yeah I forgot about that. I guess I was

Alan: We don’t even think about it.

So let’s kind of shift focus to some

Cathy: I mean, definitely training. On the

Raymond Corporation up in upstate New

Alan: It’s interesting, we’re about to make

So what I’m gonna do is, my daughters

Cathy: But no real people around, I hope.

Alan: Well, we will keep clear of them!

Cathy: But that’s awesome! That’s really,

Alan: No, what’s that one?

Cathy: Accenture has really made a lot of

Alan: Yeah there’s been a bunch of

Cathy: Droga5, yeah.

Alan: And I know Deloitte Digital has started

Cathy: Are they going to acquire you? (laughs)

Alan: I, uh…I can’t say anything (laughs).

Cathy: Offline. There you go. You always have

Alan: We’ll be announcing it at AWE this year

Cathy: No I mean–

Alan: It’s called Avenue. Sorry, what did you

Cathy: What did you say about Avenues?

Alan: It’s called Avenues. Virtual reality

Cathy: Ovation, which is for public speaking.

Alan: This is really perfect, because my

Cathy: Ovation, yeah. I mean, it’s not that

Alan: Interesting. You know, we’ve gone the

Cathy: Yeah, I mean, all those things are

You were in China. I was in China as

And obviously, it has an element of

Alan: I actually only saw it in action once,

Cathy: “You can’t come in!”

Alan: One of the workers came along and stuck

Cathy: And it looks like paper! Like, the

Alan: It’s crazy — you can have graphics on

Cathy: Virtual try-ons, yeah.

Alan: That seems to be something that is

Cathy: You know what? I’m a big proponent of

Alan: What did you buy?

Cathy: I bought a pair of glasses from

Alan: No way. And you tried them on first?

Cathy: Yeah, yeah! So, it was fashion week,

Alan: So, let me ask you a question. Do you

Cathy: Yes, I do.

Alan: Can you do a side-by-side of it with

Cathy: I should do that, yeah!

Alan: And then add it into your Real or AR?

Cathy: Yeah. Well yeah, I should do that,

Alan: If you send it over, we will put it in

Cathy: I don’t think I’m doing a lot of Real &

Alan: Oh, whatever.

Cathy: So you know, it’s all good.

Alan: I think, definitely keep using it on

Cathy: Yeah, I was like, “okay, whatever.” I don’t know, I don’t like conflict. I’m actually working on something really interesting for one of my presentations, that’s a little bit, you know, completely different from that. But I think it’ll be exciting. So we’ll see, we’ll see how it goes.

Alan: Amazing. I’m actually starting to

Cathy: I think it’s very valuable because, you

Alan: So, let me ask you something: What do

Cathy: Right now it’s enterprise, to be

Alan: And explain that to people that are

Cathy: You know, it’s more on the verticals.

First of all, obviously have the

Alan: Yeah, we had HTC’s president Alvin Wang

Cathy: Oh, I love him, yeah!

Alan: –And he was talking about Bell

Cathy: And that’s not a consumer project;

Alan: Wonderful people there; they’re very

Cathy: Going rogue.

Alan: –getting drunk at a party. But with

Cathy: You’ve got to pay the people that

Alan: But we also talked about Doha Airport

So, one of the things that I learned

We talked about virtual try-ons; how

Cathy: Definitely, we live in a very exciting time. Those of us like you and I, that have been in the industry for a couple of years, it’s just a really exciting time. It really is a moment that we need to just celebrate.