XR for Business

XR for Business

By Alan Smithson from MetaVRseBusinessTechnologyArts
Download on the App Store

XR for Business episodes

  • Keeping VR Clean with UV Beams, featuring Cleanbox’s Amy Hedrick

    Even in a virtual world, you still sweat. That’s why it’s important to keep your HMDs clean, especially if there’s multiple users. Cleanbox devised UV light technology for just such a purpose, then found a higher calling during the Coronavirus.

    Alan: Hey, everyone. Alan Smithson here, with the XR for Business podcast. Today, we're speaking with Amy Hedrick, co-founder and CEO of Cleanbox, a smart technology hygiene company that's providing hospital grade decontamination of shared hardware, which I guess we can all agree right now, is very necessary in the current situation. In this podcast, we'll also learn how the team at Cleanbox is helping hospitals maintain mask hygiene. All that and more, coming up next on the XR for Business podcast.

    Amy, welcome.

    Amy: Thank you, Alan. It's great to be here.

    Thank you so much for taking time. I know you are crazy busy, working probably 20 hour days right now. You are on the front lines, bringing hygiene in a time where it could not be more important. So thank you for joining us and taking the time.

    Amy: It's my pleasure. I'm happy to do it. And you're right. It is very timely. Everybody's focus right now -- globally -- is on hygiene and public safety and public responsibility, and how we can better stay safe when we're able to go back out in the world.

    Alan: Absolutely. So I'm going to get right into it. I know you were working with the VR community very heavily, but I think right now this is just kind of top of mind with everybody. So maybe in your own words, you can explain what Cleanbox Technologies does.

    Amy: Sure. Well, OK, so Cleanbox technology provides hospital grade decontamination of shared hardware. We designed our core technology with-- actually with XR in mind, with the idea and the belief that virtual and augmented reality had great potential in terms of business and enterprise purposes, as well as in healthcare, and of course, entertainment. Seeing the value there and thinking ahead of what would be the pain points that would prevent a successful global deployment, or a successful corporate deployment, and adequate actual consumer adoption. Few years ago, thinking back on how we would be able to meet some of those pain points and eliminate them and lower the barrier of entry, hygiene seemed to be one of those things that was easy to forget. So with XR technology, it's so sexy and there's a lot of very interesting and cool and unique things you can do with it. So the logistics sometimes gets gets a little bit lost. We came at the XR community from the point of risk mitigation. How could we reduce the risk of contagion transfer and thus some detrimental event happening within the industry? And we've grown since then.

    Alan: So right now it's an interesting time, because we're all stuck at home. So things like location based entertainment facilities are empty. Where is your business coming from now? I know you-- from speaking with one of your advisors, Terry Schussler, that you're booming right now. You can't fulfill the orders fast enough. Where's this business coming from, if it's not coming from location based entertainment?

    Amy: Well, we've always been addressing multiple markets at once. Location based entertainment is, of course, first and foremost in people's minds because that's -- I guess -- the best education for the average consumer into what immersive technology can do. So it's-- and it's very dear to my origins and we definitely take care of the LB community. On that note, I would say that there are plenty of organizations looking forward to the day that they can reopen, and realizing that not only just the idea of hygiene, but actual hygiene is really critically important to reopening those businesses and looking at bringing people back in. But in terms of our other types of clients, we work in enterprise, we work in health care. So anything from automotive to aerospace or airline industry, from retail to automation, mine safety training, there's a lot going on in healthcare as well. We've been very, very active in multiple enterprise business verticals. So Terry's right, there is a huge demand. And that's because people are both still using XR right now, either in -- for instance -- frontline triage training for COVID-19 response teams. They're also still using it -- some of the frontline workers and companies that you would not necessarily think of using XR still -- they're using it for everything from keeping their frontline workers engaged mentally or emotionally -- giving them a sort of a Zen moment, a Zen break -- if they're frontline or first responders, to exercise programs to training. So really, Cleanbox is in a lot of business verticals, because anybody that uses hardware on their face and head needs to have a way to protect the next person.

    Alan: The one thing that you just mentioned was the VR being used for kind of just a moment of peace when you're on the front lines of battling this disease. I think as a world, we've been pretty lucky with this particular virus. Yes, it's spread around the world and it's shuttered economies, but the mortality rate is low enough that we can at least keep hold of it. But our frontline hospital workers are going through a really, really tough time right now, working ridiculous hours and multi-shifts and stuff. So giving them the peace of mind to be able to just sit in a tranquil place and just zone out for a minute, I think is very necessary as well. Terry had mentioned once that you guys can also sterilize masks and things.

    Amy: Well, let me-- I guess I realize I haven't really explained how our technology works.

    Alan: Yeah, let's get into it. What does that do?

    Amy: How does it work? We use UVC light in an LED to decontaminate any kind of shared hardware. It's a 60 second process, and in that 60 seconds, we reach 99.99 percent eradication of any kind of contagion, from viruses, to bacteria, to fungi. We use UVC light. Traditionally, it's been used in healthcare systems for decades. Why is it effective? It's effective because it deconstructs the DNA or RNA strands of a contagion. UVC light's been used in hospitals for the last 40 years-ish. So it's a proven technology. What we do is in an LED, as opposed to a mercury tube. So we've eliminated the heat and the toxin components that would normally come with UVC decontamination. And that's important because, of course, you're wearing this hardware on your face, on your eyes, on your mouth, on your nose. So you definitely want to have a decontamination process that doesn't include chemicals or anything that could be not safe for the end-user.

    Alan: We've been sterilizing, because we bought masks for our VR headsets -- the leatherette masks -- and we've been sterilizing using alcohol swabs.

    Amy: Well, I can tell you that alcohol is an effective form of decontamination on hard surfaces, but you have to leave it wet for two to four minutes for it to actually do the decontamination you're expecting. You'll actually see that if you read the back of your Lysol bottle or whatever other kind of decontaminate you have. If you want to reach that 99 percent, there is a certain way you have to go about doing that, which alcohol on electronics is-- well, any kind of liquid on electronics is not really an ideal situation. So we've been looking at what are the best ways to decontaminate a product? Thinking about alcohol, you've got to have a user wipe it on a certain surface, which brings in human error components that would affect whether or not you're actually decontaminating that surface. Alcohol's also not effective on soft surfaces. So there has to be something that can continue to clean the entirety of a complicated headset. VR and AR headsets are a bit more complicated to clean than your average electronics, for instance. I mean, a cellphone is pretty easy to clean. A headset, you have shadows, and you have multiple types of materials on the surface, and multiple touch points. And they all have to be decontaminated evenly and equally, to make sure you don't pass a contagion from one person to the next.

    Alan: I actually have a question about that. What happens in parts that the light can't see?

    Amy: So it's a-- you're right, UVC's a line of sight technology, but all of Cleanbox's products are engineered for maximum UVC efficacy. And what that means is that we've eliminated those shadows. We have an overlap in radius of our lights that's been tested and gone through the R&D process and gone through independent lab testing, to make sure that there are no shadows and anything that touches your face or head. Another thing about UVC light is that if you want to decontaminate a surface -- for instance -- you need to have the right mixture of the distance from the surface, the time that light is evenly over that surface, and the intensity of the nanometre wavelength. So it's a little bit more complicated than it sounds. [chuckles]

    Alan: Absolutely. Sounds like you guys have some smart people working on this.

    Amy: Yes. I have a really great team. I'm very proud of them.

    Alan: I can imagine.

    Amy: Yeah, well, you had alluded to masks, so I would-- I'll just mention that UVC light -- as I mentioned -- has been used in hospitals for decades, which means that it's a trusted technology. So we know that it's-- our lights have been tested against H1N1. They've been tested against-- they're effective against SARS and MERS, something like bacterial, which is like MRSA, an antibiotic resistant staph infection. So we have a lot of-- there's a lot of data to support the use case. And because of that, we do have clients that have been saying, "Hey, we're using this for PPEs, because UVC light is a known entity." So we've been working with some of our clients, actually, to provide retrofits for our existing product that they've donated, and in parallel working on a product that is specific for masks. So that's been keeping us very, very, very busy as a small team, but a pivot that we really felt was mission critical right now. I mean, I'm sure, Alan, you probably know people, frontline workers, first responders, people that are exposed to this. Well, fortunately, it doesn't have a-- as high of a death rate as it could. It still is impacting people's lives. So this is a way that we knew that we could help, because we know that our technology is effective at eradicating contagions.

    Alan: That's incredible. We're in probably the most unique time in, well, in our lives, definitely. And you have a technology. So how do you get this into the hands of governments? How do you get this in the hands of hospitals, so that they can-- because mask shortages seems to be a problem. But if you could sterilize things quickly and reuse things, it seems to be a win-win.

    Amy: Yes. Well, we have clients already in healthcare, in various departments, in different hospitals, and healthcare centers and universities. And so they've been using our product right now, because they already have it. We've provided a retrofit so that it equally decontaminates both sides of the masks. But getting a mask-specific product out to hospitals, we have a lot of corporate partners that are helping support the work that we're doing. They say, "How can we be involved? How how can we help you guys get product to healthcare centers?" So I'm working on a number of fronts to push that forward, and make sure we can get this out. The product that we build, that we can provide is something that can be used on an ongoing basis, so throughout the day. A healthcare worker can use it in between patients, or a first responder between emergency responses. And this way it's something that is a very light footprint. It's a small product. It's a 60-120 second cycle. And it can be stationed throughout lots of different stations in a healthcare center. That's something that we can do. This way, they don't have to export the mask first, then have it repurposed and then reintroduce back into the healthcare system. Hopefully that makes sense, and I explained that well enough.

    Alan: Yeah, absolutely. I guess for companies that are selling these solutions, let's say, for example, they've got a VR collaboration. So you recently sponsored the XR Collaboration Platform Resource Guide, at XRcollaboration.com. And it's a global resource guide that helps people make decisions on using virtual and augmented reality solutions for collaboration, now that we're basically creating more social interactions in a time of social distancing. So with that, what would your recommendation be to these companies that are selling these solutions to customers, and how can they get a hold of you and then kind of bundle the Cleanbox solution with that?

    Amy: Well, we do have a lot of people in pre-pandemic that have already been kind of going down that path. So we've been anticipating the growth of XR already in enterprise and healthcare. So I think that some companies are further along in that process. But obviously, hygiene is a really critical-- we've always believed this is why Cleanbox exists, is that hygiene is a critical piece of the puzzle, that sometimes is the last piece that you need, but you'll always need it. We work with all types of clients in various types of packagings, bundling other kinds of agreements that we're happy to make and we have set up. So you can always reach out to me by email. My email is [email protected]. Or you can always send an [email protected] email, and someone from the sales team will reach back out to you. But we are also global. So we're entirely made in the US right now, which has been actually good for us, because we've had fewer manufacturing challenges, I suppose. We're still running operating, manufacturing, shipping product. We have had no interruptions for that. We do ship everywhere in the world.

    Alan: So website again, cleanboxtech.com, correct?

    Amy: Yes, correct. On the site there is all of the product that we have right now for HMDs. And then we have other product in the pipeline, too, that I think will be interesting to a lot of companies that are looking for other solutions, in addition to their headsets.

    Alan: So, Amy, what are the different models that you have?

    Amy: Well, we have three products on the market right now: CX-1, CX-2 and the CX-4. The CX-1 cleans one headset at once. The 2 cleans two, and the 4 cleans four. Very, very simple.

    Alan: That seems pretty obvious. [laughs]

    Amy: [laughs] It seems obvious when you say it that way. It's a 60 second cycle, regardless of which product. Whether you're cleaning one at once or four at once, it's still gonna be that one minute cycle time. And then on that note, we do actually have a new product coming out this summer, which is our ad hoc box, and that's intended for other types of small electronics, shared electronics, whether it be your cellphone, or your earbuds, or your tablet, or your handheld haptics. So I think people will be excited to see that one as well.

    Alan: And do you want to speak about the pricing?

    Amy: Well, I will tell you that we do offer tiered pricing. If you want to reach out to us at [email protected], we can send you a pricing sheet for all of our products. And then that way you'll know where any kinds of discounts or other kinds of offerings we might be able to help you with.

    Alan: Excellent. Before we go, what is one problem in the world that you'd like to see solved using XR technologies?

    Amy: Hmm. There are so many. Well, I think-- let's take it back to healthcare quickly, and I'm going to use an example that's not my own example, but a surgeon that has been using XR for-- ahead of a brain surgery, knowing that that taking-- that that technology can impact the level of efficacy on that actual surgery, and increase the chance for a patient's recovery and better that process. I would say that that's a major flag to wave for a successful use cases, that I would hope can drive the message that this is a new technology that can completely impact not only the time that it takes to complete a process, but it can impact the efficiency and the success rate of that particular job. I don't see it as a threat in any way to existing structure. I see it as an enhancement, as an opportunity for change and an opportunity to do a better job, whether you're a physician, a surgeon, an educator, even an entertainer. It's a tool. And I'm hoping that more and more people will start to see all of the options in that toolbox.

    Alan: Well, that is very well said. Thank you so much, Amy, for taking the time out of your busy schedule to join us today. And I want to say thank you to everybody for listening. And also, if you haven't already subscribed, please hit the subscribe button on your podcast. Make sure you subscribe, so you're always alerted of the new stuff. You can also find us at XRforbusiness.io. And one last message, please download the free guide, Resource Guide to XR Collaborations. It's available now at XRcollaboration.com.

    18 min
  • Leveraging 5G Networks to bring VR to South Korea, with Korea Telecom’s Joonmo Park

    For most of the world, a cross-country, gapless 5G network is still the realm of futurists. One exception is South Korea, where a 5G network is quickly proliferating. Korea Telecom’s Joonmo Park discusses how his company is using the network to evangelize VR.

    Alan: Hey, everyone. Alan Smithson here. Today, we're speaking with Joonmo Park, new media business development manager at Korea Telecom, the largest telecommunications company in South Korea. We're going to be talking about 5G, virtual reality and transportation. All that coming up next, on the XR for Business podcast.

    Joonmo, welcome to the show, my friend.

    Joonmo: Thank you. Thanks for having me, Alan. And I'm really happy to be here.

    Alan: It's an absolute honor to have you on the show. I was really impressed with some of the work that I've been seeing come out of Korea. And I was very impressed to find out that it was the work of you and your team, that have put these things together. So maybe just talk about the projects that you've been working on at KT Tel.

    Joonmo: Okay. Our company -- KT -- is the largest telecommunications company in South Korea, with more than 30 years of experience in mobile telephone service, and broadcasting, and Internet. And I've been with the company for over three years across a variety of different functions, including business strategy, B2C business, and B2B business. But about one and half years ago, as a company, we decided to establish a dedicated team that's purely focused on what's new and what's next. So we focused on the emerging consumer landscape and the emerging technology landscape, and make sure that the KT brand continues to be relevant not only today, but five or ten years from now. So we are focusing on looking at new business models and emerging technologies, then tying those to our strategy businesses. So we wanted to make sure that any new innovation that we bring into the society has a lasting impact. But more importantly, a monetary impact that is actually moving the needle. So to introduce a little bit about South Korean XR market, South Korea's telecom operators are really striving to create the right range of immersive mixed reality content to leverage their 5G network and attract more customers. And there are three local carriers, including SK Telecom, and Korea Telecom, and LG Uplus. And they rolled out their 5G network in the nation in April 2019, and together are drawing more than three million subscribers in the first seven months of the launch.

    Alan: Sorry, three million subscribers to the new 5G network?

    Joonmo: Yes, right.

    Alan: So do these people have 5G devices? Is there a lot of 5G devices available?

    Joonmo: Yes, right. Because as the launch of Galaxy S10 with 5G, and that many people bought that device and using 5G network right now.

    Alan: That's incredible. So how many would you estimate are using the 5G network with 5G devices?

    Joonmo: I think until the end of the 2020, more than the five million people will use it.

    Alan: Wow, that's way far ahead of North America. In Canada, we have zero 5G deployment. We only have it in research parks and research areas.

    Joonmo: Well, you need to think about that. South Korea is a very small country with very many numbers of populations. So it is really easy for companies to make the decision to build new 5G infrastructures, because even though they build small 5G networks, many people would use that. So many tech companies feel that building a 5G networks is attracting more money than just using 4G network.

    Alan: So how is 5G -- or just the ability to have this kind of massive data pipe -- affecting your ability to do virtual reality, or augmented reality, or mixed reality?

    Joonmo: Well, as you know, the VR devices right now do not support 5G network, and all of the devices are using Wi-Fi network. However, we use 5G-- we use VR as the marketing attraction for 5G. And to talk about KT, KT launched the 4K High Definitions personal media service with the brand name "Super VR." And we attract 5G users by selling the Super VR. And we are using Super VR as a marketing point for the 5G service.

    Alan: Oh, that's interesting. So Super VR-- and for anybody wants to check it out, it's actually pretty cool: it's ktsupervr.co.kr. Did you guys do everything? So did you make the headsets as well or just the kind of the back end? Or did you partner with companies to do this?

    Joonmo: Actually we partnered with a company called Pico in China and we use Pico's G2 device. And we provide more than 10,000 immersive videos and features, over more than 500 4K high resolution videos, and more than 30 games in various genres, such as sports, and shooting, horror, and rhythm action.

    Alan: That's pretty awesome. So you basically control the content. You're like the Netflix of VR in South Korea.

    Joonmo: Yes, right. Exactly.

    Alan: Can people buy the headset and subscribe to this service, if they're not a KT customer?

    Joonmo: When we first launched the Super VR device, it was only for KT users. But from the last November, we opened the device to all the platform users, even though they are not using KT.

    Alan: That's awesome. That's really great. And how is the response so far?

    Joonmo: Oh, well, many people are really satisfied about that. Where we are -- according to some kind of customer report -- more than 60 percent of customers are really satisfied with it, and more than 10,000s of devices are sold in nationwide. And the interesting thing is that many Korean customers are more satisfied with videos, rather than games. And I think it's a really interesting point, because many VR users tend to buy VR to play games, including Steam or Oculus or HTC Vive. However, in Korea, many people consider VR as a second TV or personal TV, so they tend to use VR to watch video in cinematic situations.

    Alan: So basically what people are using is something like big screen, where they sit on the couch with their friends and watch a large movie in a screen. Which is actually, if you think about it now: now that a lot of people in the world are are stuck in their houses, not everybody has a IMAX sized screen in their house. I know I don't. But being able to put a VR headset on and watch a movie on an IMAX sized screen is actually pretty wonderful. And you guys are streaming these in 4K, which is really beautiful.

    Joonmo: Yeah, right. And actually, as we are the IPTV-- and we serve our TV service, which is the IPTV Broadcasting System. And we first launched IPTV for a full year on Pico headset, which means that people can see the exact same TV, while they are putting on the HMD.

    Alan: So you're basically broadcasting the regular television into the headsets.

    Joonmo: Yeah, right.

    Alan: That makes sense. I think the next thing is you're going to have a big screen, or however many screens you want for maybe your computer, or your laptop, or whatever. And I also think you can have a little-- like a second screen where you see your cell phone as a screen in VR. So you can check your messages quick with your phone.

    Joonmo: Yeah. That is our goal, to merge mobile service, VR, and TV all in one by using our network.

    Alan: Very, very cool. And you said the adoption is picking up. Are you seeing an acceleration now that less people are out? How is the economy downturn affected your business?

    Joonmo: You mean because of the coronavirus?

    Alan: Correct.

    Joonmo: Yeah, but actually -- as you said -- there is a big change for our marketing strategy, as until the last year we've tended to focus on retail stores in order to sell our devices. However, after the outbreak of coronavirus then most of the people don't visit the physical retail store. So we tend to focus on B2B business, including using ENGAGE VR for online conferences. So we are selling our VR devices to schools, or transportation services, or academies for that said, those devices for those enterprise customers.

    Alan: Let's break this down by, I guess, enterprise or business, and then education. How are your business customers using these technologies right now?

    Joonmo: After we commercialized the VR service for enterprise called Super VR Biz -- and Biz refers to business, which is often used in Korea -- we composed this Super VR Biz with Pico VR headsets, and the contents and network services. And many different kinds of industries wanted to apply our service. We started the first service in transportation and we launched a pilot VR service in express bus in November 2019 with a company called Kumho Express Bus, which is similar to Greyhound in America. And really the goal was to understand how our VR technology and content can apply to other industries. We have two ultimate objectives. And one was to drive employee adoption. So our enterprise clients make sure they're adopting the technology. And the second one is passengers adoption on the other end of it, making sure that passengers are adopting on innovative but emerging technology while traveling. And if we can do this first, everything else will come together.

    Alan: So what was the finding of that? Did people use it? Did they like it? What was the outcome?

    Joonmo: After we are running the VR service for transportation and we got the survey answers from hundreds of passengers, there are several interesting results. The first one is people thought about three to five dollars was enough per use, which means that -- as it takes about three to four hours to move from city to city in Korea -- so it refers to the fact that a dollar per hour is considered as not that expensive for the passengers. And secondly, people preferred 3DOF devices than 6DOF devices for travelling. As I mentioned before, many Korean people like to use VR for watching videos, rather than playing games. So I think this is also seen in transportation businesses.

    Alan: It's really interesting you should say that, because I think over here, there's a misconception that everybody that uses VR wants to have a six degree of freedom experience, or being able to move around. But think about it, when you're on a train, let's say -- for example -- you set up a stationary space. As soon as the train starts moving and turning -- or in a car or vehicle, anything that's moving -- you have to recalibrate that. Whereas if it's just a heads-up display where you're watching a video and you're just -- not even a 360 video, just a video -- you can then relax. But the headsets themselves, I guess because you don't need all of the sensors to have a full six degree of freedom experience with these headsets, can they make them into a lightweight pair of glasses, and is that something Pico's looking at? I've seen some of the-- Huawei come up with a pair of glasses and Samsung, they're all kind of teasing at these. And even HTC is teasing it, these lightweight pair of glasses. But for now, we start to wear these big VR headsets. Have you seen any prototypes that have come out, that look really good for just sitting down in a train, watching video?

    Joonmo: You mean something like the Nreal AR glasses, or something like that?

    Alan: No, not AR, but more VR, but in a small form factor.

    Joonmo: We tried very different kinds of devices, including Oculus Quest, or Pico device for 3DOF, and Samsung Gear. And the interesting thing was that as many passengers are not that familiar to VR devices, they had trouble with using 6DOF devices on the bus. However, the 3DOF devices are rather more easy to use than 6DOF. So people felt more comfortable to use that while they are traveling. And while they are riding on the bus, then they feel a little bit nausea while they are traveling, so they tend not to play games. Rather, they tend to see relaxing videos.

    Alan: And the videos wouldn't give you that, because it's fixed to a main plane. You know, it's something that I think a lot of people over here have almost missed that opportunity, or forgot about this opportunity. And when I got into virtual reality in the beginning, the first thing we did was we filmed 360 videos, because we thought that was what VR was. And then we got into 6DOF experiences. It's interesting that something as simple as like a company like Bigscreen, where they just have a big screen, or what you guys are doing is just providing regular 2D content in a much more immersive way for travelers who have to spend time travelling. And I think this is a use case for the technology that we in North America -- and maybe other parts of the world -- have just neglected. And have you seen anybody take the same concept and allow business workers to conduct business -- or at least to work while they're doing that -- replicate their computer screen rather than video? Have you seen that?

    Joonmo: Not really in Korea, but I've seen similar services on the airplane. I saw the device called Skylight or Aerospace, which--

    Alan: Oh yeah, how was that?

    Joonmo: Yeah, yeah. Running the HMD device for first class passengers. And I think the concept is very much similar to that one. And actually, we are preparing for the same business for the planes, too. But as more people are using express bus and trains in Korea, so we are preparing the VR devices, in order to make more people to experience VR more easily and more comfortably, without much knowledge about the VR device.

    Alan: Yeah, I think that's the key, is just making sure that everybody understands by putting this thing on, you have a nicer experience. I think we try to-- a little bit go crazy and say, "Hey, you have to swing your arms around like an idiot when you're in VR." But just sitting, relaxing, watching a nice video on a huge screen with nice sound. It's funny we take it for granted how amazing this technology is in a very simple way. You talked about driving employee adoption of this technology. How is KT using this technology within their own business units to drive value?

    Joonmo: The employee engagement was the main challenge for us. And it has really important things, because if people don't use then it's kind of useless. So for us, it was critical that we had a comprehensive training program for VR adoption. So we worked with our technical partners and marketers to really build a robust end-to-end training program that clearly communicating the benefits of the technology, the practical application of it, and what it means for our clients. So when I was training employees for the transportation businesses, my ultimate goal was to provide unique travel experience to passengers. And I also talked to the bus drivers that you don't need to be intrigued by the passengers who are driving, because we are supporting the technical issues. So I always told to the crew that here is a technology that can not only help you provide unique travel experiences, but also increase revenues in the long term with increased number of passengers. And I also said that we must support customers' complaints about VR with our engineers on phone while you are driving. So it was really clearly outlining the benefits of the VR technology, and let them know that they don't have to worry about technical issues. So the fact that we piloted in two bus lines and sold the reserves for our service, and that allowed us to take it to the balance of the organization and tell them "I showed the benefits, and we have done this before, and this is what it looked like." And that allowed us to drive employee adoption. And it was really critical, because the consumer adoption can accommodate our colleagues who don't adopt the technology. So we were able to cross others as the other one becomes a lot easier to.

    Alan: That's incredible. As part of your training program, what are some of the business use cases that you're using this for, or you're promoting this for?

    Joonmo: We started our pilot service with a Club Med Resort in China, and I trained the reception desk workers on the Club Med about how to handle this device, and how to explain about the contents or etc. And at the first time, the reception workers were not that eager to learn about that, because they thought that they were too busy to just handling with the hotel works, and they thought that the VR device was just so burdensome to them. However, as I taught them in person and I also practiced so many hours for them to use it, and then they get-- once acclimated to use them, then they got easy to serve the device to the tourists.

    Alan: And what is the response been? Have you been able to document an increase in revenues or a decrease in times? What are the results?

    Joonmo: I cannot say that there were more increased customers because of VR, because of the outbreak of the coronavirus at the hotel.

    Alan: Oh, no! So you were launching it right as--?

    Joonmo: Yes. So I cannot say that the numbers are increased because of the VR. But however, as there was the survey -- when people were checked out from the hotel -- and many people were really satisfied about that additional entertainment surveys for the hotel. Many tourists thought VR is very good entertaining service for the kids. So many moms and fathers who were really satisfied about serving VR at the hotel.

    Alan: A great toy and experience for kids that have never tried it, as well. I love it. So I actually had a thought. Are you guys doing so when people are watching-- let's say they're watching videos on a train, or a bus, or wherever they're consuming your content. Do you have inside of their special advertisements in 360?

    Joonmo: Oh, we are. To talk about our service, we have advertisement sections in our device, which is composed of images and 360 videos. And for the Club Med, we received Club Med advertisement videos from them, and we put that on the device in the internal memory. And whenever people click that advertisement, then they can experience the resort in 360 degree video. So the client was really satisfied about that, because the VR is considered as really immersive media and immersive environment can give the customers with more deeply understand about the resort. So many people experienced that video, that advertisement.

    Alan: That's fantastic. What are some of the solutions that you're offering in Super VR Biz? And are you guys experimenting at all with augmented reality, as it pertains to maybe Hololens or Magic Leap or these types of things, where-- for factory worker in manufacturing, and these types of things? Or is this mainly consumer facing application?

    Joonmo: Right now, we only serve VR for Super VR Biz. However, we are planning to serve AR and MR service in our business services, and we are actually preparing for AR engineering service, or AR touring service, or MR education services. And you can see that in the close future.

    Alan: So when you're making -- for example -- augmented reality, is that taking off mobile phone based AR experiences? Is that something that's happening in Korea at all?

    Joonmo: Yeah, about one or two years ago we tried to use AR glasses, such as Magic Leap or Nreal. However, because the high price of the device, it is not that easy to sell to the ordinary customers right now. So we are focusing on mobile AR right now in the market.

    Alan: What are you seeing people using mobile AR for? Because I agree with you, mobile AR is the way to go right now, because it's accessible to everybody and everybody already has a phone. So if everybody has a nice phone in their pocket, what can we do with that? Or what are some of the business use cases you've seen that use mobile based AR?

    Joonmo: There are some differences between tech companies, and my company is focusing on AR travelling, which means that you can put on your mobile phone to the building, and then you can see the explanation of the history of that building. However, some other companies tend to use AR for games, like Harry Potter or AR animal zoos.

    Alan: You mentioned two things. One, Harry Potter, which was made by Niantic, which made Pokémon Go, the same game mechanics. But just this week they bought a company called 6D.ai, which is a cloud mapping company -- who was actually on this podcast previously as well -- but it's interesting, you said that and then you said adding a zoo. And this week -- again -- Google released the ability to see animals just from the browser, using their 3D Model Viewer, I think it's called. So being able to see animals in your space using AR.

    Joonmo: But right now, not many people are using AR right now, because I think it is because of the coronavirus. You mean people cannot go outside right now. So many companies are having some hard time with using AR. But I think it'll be a popular and in here soon.

    Alan: What about using this opportunity that everybody's at home, to maybe show them what products and services look like in three dimensions in their own space? So maybe bringing the shoe store to them, or bringing 3D experiences to their phone? Maybe it's not even AR, maybe just 3D experiences. Have you seen anything like that?

    Joonmo: Well, I saw a AR funny tourist, like Ikea. And many online e-commerce stores are serving AR as tours and also KT also serves AR shopping, which is based on the TV. You can see augmented commerce on the screen, so that people can see the product in more augmented way.

    Alan: Okay, that's really cool. Would they be using image recognition, then?

    Joonmo: Yeah. Yeah. Right.

    Alan: Man, it sounds like you guys are doing so lot. And I think the ability to have so many devices in prevalent 5G network has really unlocked the ability to drive these some of these experiences.

    Joonmo: Oh yeah. Yeah, I think so too. The ability to support 5G network, as well as bringing content for the Korean users was the basis of the VR and AR service in Korean market.

    Alan: Something that I'm really excited about personally is the new Qualcomm Snapdragon XR2 chip that's 5G enabled. So the glasses coming out next year will all be 5G enabled right out of the box.

    Joonmo: Yeah, I'm really looking forward to that, too.

    Alan: Yeah, it's gonna be exciting times in the next few years.

    Joonmo: Yeah, I believe so.

    Alan: Very cool. Well, Joonmo, I just wanted to give the website before I forget: the website for your company is corp.kt.com/eng or kt.com, which is in Korean. And then if people want to visit, they can also visit ktsupervr.co.kr, and that's the Super VR product that you guys have there. Any last words that you want to share with everybody?

    Joonmo: I shared many things with you, and what I learned from the VR project is that I need to really focus on what I'm trying to solve for the client. So identifying a problem that you're trying to serve and figuring out practical applications is really important when you're having a VR or AR business.

    Alan: That is some true words right there, my friend. What is one problem in the world that you want to see solved using XR technologies?

    Joonmo: I really love to see that VR or AR can be used in a very diverse context, that every people can use VR and AR in the real physical world. So I think VR, MR and AR technologies will be merged into physical experiences, such as traveling or engineering. And those technologies will have enterprises to differentiate them from other companies, by creating all these experiences that mimics the physical experience. So obviously I'm at the center of it for some parts of it, but I'm really interested in seeing how that comes to life. And not only does the sort of consumer pain points, but also in assets and augments that experience.

    Alan: Well said, my friend. Thank you very much, Joonmo. And thank you, everybody, for listening. This has been the XR for Business podcast with your host, Alan Smithson. Please don't forget to subscribe to the podcast. Click the button so that you don't miss any of these amazing episodes. Again, my guest, Joonmo Park, new media business development manager at KT Tel or Korea Tel. And thank you again, Joonmo. It's been a wonderful experience.

    Joonmo: Thank you, Alan. Thanks for having me.

    31 min
  • Anchoring AR Content, with Simply Augmented’s Boaz Ashkenazy

    Seeing is believing, but in the age of 3D product views through AR technology, seeing is also conceptualizing. Simply Augmented CEO Boaz Ashkenazy comes on the show to explain how AR-enabled 3D viewing will revolutionize everything from how we shop, to how we work.

    Alan: Hey everyone, Alan Smithson here. Today we're speaking with Boaz Ashkenazy from SimplyAugmented. We're going to be talking about how 3D is revolutionizing furniture views and products on the web. All that coming up on the XR for Business podcast.

    Boaz, welcome to the show, my friend.

    Boaz: Hey, thank you very much. It's good to be here.

    Alan: Oh, it's my absolute pleasure. I love the work you guys are doing. Let's get right into it. What is it that SimplyAugmented does, and how is it benefiting your customers now?

    Boaz: Well, we have a 3D platform that benefits customers both in the sales and marketing teams, and one of our products is called Simply3D. It allows for sales and marketing teams to easily share augmented reality. We found a lot of challenges around sharing augmented reality online and Simply3D.io is a vehicle for helping folks share. And we also create product and room configurators that exist online that allow people to experience environments and products really easily with augmented reality integration.

    Alan: So I got a chance to try one of the samples. I got to configure a-- it was like a quiet room or almost like a study room for open office furniture. So you go in, you close the door, and make yourself a phone call or whatever, but I got to configure it on the web. I hit a button, and it was in my living room here, this giant pod. And I was like, "Oh wow. This pod is huge in real life." So it was a great understanding of how big these things are, and I think that's truly the power of this technology. Maybe you can speak to some of the specific clients that you've worked with. What are they seeing, results wise?

    Boaz: You know what's interesting about what you said is a lot of folks have trouble visualizing products at scale, especially with the bigger the product, the harder it is to visualize. I am an architect by training. And I spent years designing spaces, and a lot of times people are surprised about what those designs end up looking like, because it's hard to understand objects at scale. And the pods that you mentioned, they're called Nook pods. And they're large rooms within a room, basically. You can have a quiet room inside an open office, which is pretty typical these days. For two people for four people, for one person. And to see it online and configure it is one thing, but to see it in the environment and walk into is another.

    One of the things that we recognized was there was two things that people really wanted to do. They wanted to quickly customize any of their products -- We're used to doing it with cars and vehicles, customize your products with colors, with options -- but then immediately be able to drop that in your environment and walk around it, look underneath it, walk inside it. And that's really what our configurator does.

    One of the challenges that a lot of folks face in the XR space is how do you spin up augmented reality with so many possible variations? That's the hard part about building these configurators. There's so many options, that you would never be able to create the augmented reality content beforehand. You wouldn't be able to predict it. And so what we've done is we've allowed for on-demand, real time augmented reality, depending on the choices that you make.

    Alan: So once I've configured it -- and I don't know, I didn't actually think about this until now -- but once I've configured -- let's say -- the Nook and I drop it into my space, can I continue to configure it from that space?

    Boaz: Yeah. Right now the configuration happens on the web. It doesn't happen in augmented reality, but it's something that we're working on. A lot of people, they want to be able to do two things. They want to be able to configure in augmented reality on the web, not just in an app, and then also put multiple items in a space and have them stay in that environment. And so spacial anchors of different varieties are also something that we're working on. I think that's going to be a trend that we see, is people using spacial anchors to establish products in the environment that don't go away -- unless you want them to -- and then be able to configure them in real time.

    Alan: Give me an example of why somebody would want to leave something in fixed space.

    Boaz: I mean, there's so many use cases, but one example would be being able to drop a-- let's keep talking about the Nook. So let's say that you'd want to drop the Nook in an office space in a certain location, but you'd want to tag it so it stays in place. And then when somebody else comes into that environment, they can see it in the orientation that you've placed it in, and then check it out as well. So that's one option. The other option that we're seeing a lot of is in virtual meetings. So with spacial anchors, if you can -- as a team leader and as a presenter -- drop an object onto the table in front of you, everybody else can pick up their phones and look at that object as well. And you've established it as a point in space. And so as we see virtual meetings really taking over, I think that augmented reality is going to be a part of being able to visualize what otherwise would be hard to show if you weren't present.

    Alan: With the coronavirus and all these events being canceled, there's an opportunity for this industry to really springboard itself. We've identified, I think, 60 different VR and AR collaboration platforms so far. And so I think the ability to bring an object in and collaborate with it from anywhere is really going to be something special. What are you mainly targeting now, from the device standpoint? Like obviously virtual reality goggles -- for example -- are great, but there's not that many-- the scale isn't there yet. With augmented reality, you've got maybe phone-based versus HoloLens or something like that. What type of devices are you targeting now, or what is the major consumption of this technology?

    Boaz: For a long time, I worked with Oculus, Vive, HoloLens, and other headsets. And recently I'm seeing a lot of people switching over to mobile devices, there's a lot of headsets that I see in offices that are going unused. I'm still very interested in those use cases, especially in hands-free situations. But most of my clients want to experience augmented reality with their phone or tablet. And then the other thing, too, is when we first started SimplyAugmented, we built apps for the app store and we build Android apps, and built a platform solely based on those apps. And some of the feedback that we were getting was just around friction of having too many apps in their phone, about wanting to get to the augmented reality faster.

    And so in the last six months, we built Simply3D.io, which is a web-based platform where you have text messaging and QR codes and you can share augmented reality directly with your customers and they can just happen in their own browser. And that's a trend that we're seeing. 8th Wall's doing some interesting things with web-based augmented reality. And I just see that exploding. And I think that's going to be a huge trend.

    Alan: I tend to disagree. I think the internet is a passing fad. I don't think it's going to stick around.

    [ba-dum-tsssh]

    [laughs] To be honest, the way the Web is going, with 5G and distributed computing and being able to render things on the cloud-- not only render on the cloud, but with edge computing, you should be able to take some of the rendering power off of the device and put it in the edge, and create those multi-user experiences with photorealistic precision, and allow people from around the world to work together on photorealistic 3D assets, all running from your phone. Because if you can take even 50% of the rendering power off the device, then you're able to push way more pixels. Or voxels, I guess; three dimensional pixels.

    Boaz: Yeah, absolutely. I also do 3D consulting and I work with a lot of design construction and developers in the real estate space, and we've developed some 3D configurators that are room configurators, spacial configurators, where you can go into environments. And we're using WebGL. We're doing a lot of technical work to keep these file sizes really small, so that on the web you can get inside spaces and walk around. And I know you know this well, but with 5G -- and with other new technology -- to be able to do all of this on a website and really explore environments in that way -- even though that's a lot of content -- is going to be really, really powerful. And I think the stuff that we're seeing now with coronavirus, I have a lot of clients that are in the commercial real estate space that do a lot of tours, physical tours, face to face. And being able to do a tour in a photorealistic environment with your clients and not having to be there in person, I think is going to be huge.

    Alan: I think so too. You mentioned something, and I think this is something that I would like to kind of stick a pin in and explore with you a little bit more. One of the things that comes up a lot is that dealing with 3D assets is a pain in the asset. How do you manage these 3D files? How do you manage the back and forth with your customers to say, here's the 3D file, here's the different colors, and have them approve them and back and forth with that, because I know just simply with photographs and logos and stuff with when you're doing traditional marketing, the back and forth content management systems are there, but none of the current content management systems support 3D yet.

    Boaz: Yeah, it's a really good point. What we found is that the configurators that we build are a really good way to communicate a lot of different options. It's one thing if you are building one model with one color and several materials, and you can share that with your client via Simply3D.io or other mechanisms to share. But being able to mark up those 3D objects is really difficult. People currently do is they'll take a snapshot on their machine, and then they'll go through traditional methods of marking that up and red-marking that file.

    Alan: And there's no tracking in that way.

    Boaz: No, not at all. And so one of the things that we're going to be building into Simply3D.io is a collaboration tool that allows you to spin models around and tag it with notes that have replies and have a string of comments, that are typical to the kinds of things we see in Slack and other messaging, because I think that's the way; to be able to have it live on top of that 3D model is the best way to be able to track those changes.

    And the same thing is true for environments as well. Being able to walk through an architectural environment, tag it, and have those comments stay so that when someone comes in later, they can see them and they can respond to them. It's the same thing, whether it's a product or a space.

    Alan: Absolutely. And one of the things that we're just looking at now is how can we then leverage the power of the blockchain to lock the comments and the permissions to the 3D object. Because a 3D object is not a 2D object. If I go into TurboSquid and buy a car, that's fine. I can download the car, I own the 3D object. But if it's a brand who wants to protect their 3D object and also protect the layers of communication back and forth, maybe blockchain is something that can manage that.

    Boaz: That's really interesting. I think that being able to establish who made that model and then who owns the rights to that model, and then track changes across that model to a final change and establish that authenticity is huge. I think that's really interesting.

    Alan: It's funny, a friend of mine, I reached out to him yesterday. I said, "Hey man, how can we use our-- what we're doing through the blockchain and really secure these assets, and then lock in the changes?" Exactly what you said. If somebody needs to go in and make a change, it locks in who they are, where they were, how long they interacted with the model, what their role is and gives them a dashboard based on their hierarchy in the company. I think it's an interesting business model.

    One of the things that keeps coming up, again, being in the 3D space is file sizes. And you mentioned that you've done a lot of work in kind of making the file sizes smaller to fit into these web-based experiences. What are you finding is the maximum kind of file size that you are comfortable with, number of polygons or that sort of-- what is the kind of maximum that you recommend to your customers for a seamless web experience?

    Boaz: For the best web experience, we try to keep the file sizes between two megabytes and five megabytes. And one of the things that we've been working on -- and I've been part of -- is the Khronos Group. There's a working group inside the Khronos Group called the 3D Commerce Group. One of the things we're doing is establishing some guidelines for the world so that there's consistency when folks build models, and send them to online retailers and to manufacturers and others. And one of the challenges we face now is to have it perform for customers on their mobile devices and not lag and not take too long. Really need to get into those numbers, two megabytes to five megabytes. And to do that requires you to keep your models pretty efficient, both in terms of polygons and draw calls, but also in terms of materials. I think that's one of the biggest challenges we face.

    Some of the groups that sit in this working group, folks like Amazon and Wayfair and Shopify, and they're receiving a lot of models from the 3D community every single day. And it's frustrating, because there's a lot of inconsistency in terms of materiality, how those models are built. One of the things we try to do is use PBR materials for all these models -- Physically Based Rendering -- to really make things look real, make colors accurate, make the lighting accurate. And so that's the real challenge. It's a dance between trying to make something look really real and keep it really small so it's performs.

    Alan: Yeah. Interesting. I was researching some file optimization this morning and I found two: one's called Simplygon, which is a polygon reduction tool from-- I believe Microsoft bought them.

    Boaz: Yep, that's right.

    Alan: The other one is InstaLOD. It's mainly used for games, but it looks like something that will be really useful. It's instalod.com.

    Boaz: Interesting. I haven't heard of that before.

    Alan: Again, there's-- one, there's no standardization. So Apple -- I believe -- left the Khronos Group, leaving kind of the worldwide standards. Almost ready to be like, "We're going to have these standards." And then Apple's like "Ehhh, no. We'll create USDZ." I think there's going to be, over the next kind of 12 months, we're going to have this kind of up in the air, what is the format? Because really what it needs to be is a format that'd be like a JPEG. I send you a JPEG, you get a JPEG, you open it up, and it works. Right now, if you send me an OBJ, you've got to send me the OBJ, you've got to send me the texture files, the bitmaps, the normals, the specular files, and then I spend a half an hour just trying to reassemble the file that you sent to me. This is dumb.

    Boaz: Absolutely.

    Alan: And there's no animations built in.

    Boaz: Absolutely.

    Alan: The animations, maybe some are, some aren't, glTFs support it, FBXs don't.

    Boaz: Yeah. I mean, Khronos is really pushing on glTF and GLB to be that standard, but in general, I think that's the big problem and challenge we face. Right now, we're just at the brink of having 3D explode in the eCommerce space, and when it does, the scale by which all these models are going to be created, we really need consistency in a standard.

    Alan: I couldn't agree more. I hope to join the Khronos Group very shortly and help. We've created our own model format, which will release soon. And you and I have spoke offline about it. But I also-- I read a research paper from Microsoft and they're working on using 2D images to create 3D models. So they're using adversarial networks or AI to fill in the blanks of, here's a 2D image, what is the geometry of it in 3D? And it's working. I mean, it's very, very basic now, but give it another -- let's say -- year, maybe two years, and we should be able to take Amazon's catalog of products and take the five photographs from the products and estimate what the 3D object looks like.

    Boaz: Yeah, it's incredible. We've been talking to a couple of companies that are exploring that as well, and it's real and it's going to be something that really revolutionizes the content creation space, because no longer is everything going to have to be created from scratch.

    Alan: It's getting really, really exciting. What are the first steps of a brand [that] wants to get into this and get in touch with you? It's simplyaugmented.com

    Boaz: Yep.

    Alan: What are the first steps that you do with a customer though?

    Boaz: I think the big thing is understanding what we're starting with. Folks are unclear about the assets that they have and if they're going to work for 3D. So you have some customers that are starting with nothing but photographs and specifications. So we just launched a project for one of the larger outdoor furniture companies in the United States called O.W. Lee. And they started-- they had a really good catalog of photographs, good shop drawings, good specifications, but they weren't sitting on a lot of 3D content. Whereas we have other customers at Herman Miller --the office furniture company -- and their dealers are clients of ours, but they have a vault of 3D models that are really good. We start from those 3D models. There needs to be some cleanup and some adjustment to materials, but they're starting from a very different place.

    So it's just kind of understanding and doing an audit of what the catalog looks like right now. That's the first step. And then oftentimes aligning the budgets with the catalog size. Most people want to build out everything all at once, but oftentimes it's good to start slow and build out a line or a collections.

    Alan: So what do you suggest then? Do you typically say, "Okay, let's just take one product and show you how it's going to work?" What is the cost for a customer to have a product? Let's say for example, we use the Nook as an example. You 3D model the Nook, and you change all the colors, the bench could be green, blue, red, five different configurations, maybe the shape of it as five different configurations. Is it based on the number of configurations? Is it based on just a time, like we think this is going to be 40 hours or whatever?

    Boaz: I mean, usually it's based on catalog size and the number of materials as well. And everyone's a little bit different. So some furniture companies will have 20 collections, but they'll have 250 materials and they're going to want to swap those materials. And so in the case of O.W. Lee, we used Sunbrella materials for that product, and there was a lot of materials that needed to get swapped out.

    In the case of Nook, they had, I think, 10 different laminates for one section, some different colors for the other, but it was a much more limited material of variety. The thing that was different about Nook was, unlike O.W. Lee or some furniture companies, where the color of the base is changing -- or the color of the fabric -- the Nook had lots of different options and lots of different pieces and parts, that were turning on and off. So there was a lot more different combinations.

    So you kind of look at the entire size of the catalog and then you look at the number of combinations. The costs can be as little as $10,000 up to a $100,000, so it's a wide range and it's really dependent on how deep you want to go in terms of the number of products, combinations,

    Alan: What is the turnaround time? This is another thing that customers keep asking, is how long is it going to take for me to have my catalog? Is it an hour per model? Is it a day? And how do people budget time-wise for that?

    Boaz: Yeah, I would say that for most projects, we're falling in the 30 to 60 day turnaround time, for a smaller configurator. If it gets really complicated, it can be up to six months to really go through an entire processes and do an entire catalog. But if you start small and you want to do a collection and test it out on your website, usually within 30 days you can get a configurator built.

    Alan: Well, Boaz, thank you so much for bringing your knowledge. Is there anything else you want to tell people about SimplyAugmented or Simply3D?

    Boaz: Yeah, I would love to have people log in to Simply3D.io. You can set up a free account and play around with objects, and see what it's like to share and to do augmented reality in your own environment. And we would love the feedback and get to hear from people about their use cases and their workflows. Because fundamentally, that's the thing that's most interesting to us is can this technology solve problems? And so the more problems that we can hear about and understand the better for us.

    Alan: I'm logging in right now.

    Boaz: Well, thanks a lot. It's just wonderful talking to you today, Alan.

    Alan: Yeah, man. I have one more question for you.

    Boaz: Sure.

    Alan: What problem in the world do you want to see solved using XR technologies?

    Boaz: Oh, that's a good question. You know, I think in the last few weeks I've been thinking about remote meetings and about ways that designers can share complicated information to a group of people, that aren't in the room with them. And as an architect, we thought a lot about how to explain very, very complicated, three-dimensional experiences to people and help them visualize that. And I think now more than ever, being able to do that quickly and get decisions made quickly are really key. And so I'd like to see a growth in the virtual meeting space, and I'd like to see augmented reality integrated into that right alongside.

    Alan: Well, thank you so much, Boaz, for sharing this. If people want more information, you can visit simplyaugmented.com or you can sign up for simply3d.io. I'm signed in there. I'm looking for stuff. I'm going to load my models in and get viewing online. This is awesome.

    Thank you again, everyone, for listening to the XR for Business podcast. Make sure you hit the subscribe button so that you don't miss any episodes. Thanks so much, Boaz.

    Boaz: Hey, thanks, Alan. I appreciate it.

    Alan: Cheers.

    24 min
  • Getting Miners Used to Gigantic Tires in VR, with Sheridan College’s Nick Ullrich

    Surface miners, like the ones safety instructor Nick Ullrich teaches, get to play with some pretty big toys, like loading trucks with tires three times taller than the average person. But tires that big come with some pretty big blind spots, and Nick is using VR to get them used to those blind spots before putting them behind those gigantic wheels.

    Alan: Welcome to the XR for Business Podcast, with your host, Alan Smithson. We all know safety comes first, and today, we're speaking with Nick Ullrich, a safety instructor from Gillette College, who's focused on using VR to train mining employees to become safer and better operators. Coming up next on the XR for Business podcast.

    Nick, welcome to the show.

    Nick: Yes, Alan, thank you very much for having me. I appreciate it.

    Alan: Oh, it's my absolute pleasure. I read an article about how you're using VR to train mining employees. How did you get into this? Tell us a little bit about your background.

    Nick: Ok. Yeah, so the first project that we've done is a blind spot recognition, using virtual reality 360 photography as well. And it's to help miners -- when they're on the mine side -- understand the blind spots of heavy equipment. So we started with that because MSHA -- the Mine Safety and Health Administration -- has an initiative out there about powered haulage. And that's kind of where the idea came from. We see fatalities every year in the mining world. So I wanted to give everybody an understanding of what the blind spots for the heavy equipment that they're working around are on, and give them an idea of that. And virtual reality gave us an opportunity to do that without actually having to have all the equipment here at the college. So it works out great to give them a vision of what they would see if they were in that equipment.

    Alan: That's pretty cool. So you're talking about those big, huge dump trucks with the giant wheels and loaders and all these type of things?

    Nick: Yeah, absolutely. We have-- in the program that we have now, we have 10 pieces of equipment, it includes the 400 ton haul trucks, which is the largest haul trucks in the world. Those tires are approximately 15 feet high.

    Alan: Whoa.

    Nick: Yeah, next to them, you will go about to the middle of the tire.

    Alan: That's incredible. So you've got these trucks. Now, did you create them as 3D models and then climb inside of it? Or is this taken from like a 360 video type of thing?

    Nick: So we do a couple different things with it. We do have 3D models of all the pieces of equipment. So like I said, we have about ten pieces of equipment right now, and we just have 3D models of those, where people can walk around them virtually and see how big they are, put them to actual size or as close as I could get to actual size, by my recollection of them. They can get into them -- for the most part -- and just kind of see it in a virtual spot. And then we did go out to all the different mine sites and take 360 photography of each of those pieces of equipment. And we did it a really cool way. We set up a scene, so we had a whole lot of different things around the piece of equipment. So let's just say a haul trip, we have several different people and smaller vehicles, like light duty vehicles, such as just your normal pickup or van. We had those all set up in a special way, where you couldn't see them from the cab, so they were *in* the blind spots of that equipment. We took that 360 photo from the cab of that piece of equipment, so we can show everybody what the cab looks like and what they could see outside of it, knowing that they couldn't see any of the things in the blind spots. They were able to look around the cab, they were able to see out the windows, they could even use the mirrors. Then what we did is moved outside and we took several different 360 photos from the piece of equipment. And every 360 photo that we took, there's more of the things that were around you at the time. We would jump them from one spot to the next. And with every spot that they jumped, then more of the full scene became visible. We're talking about 10 to 15 pieces of either other equipment or people that were all around them the entire time. So they were able to see that 360 photo eyes and then get back in the cab and then really realize that, "Man, we really could not see any of that stuff that was around us at the time."

    Alan: That's pretty awesome.

    Nick: The virtual reality part was great, because it gives them an opportunity to look around. We're mainly focused on the new miners that are just going through the initial class. MSHA requires that people do a 24 hour training before they go out to a mine site. That's what we teach here at the college. We focus on those people that have never been around that equipment. So having the virtual model, and then being able to see how big it is and actually stand next to the tire -- that I was telling you about -- that got their attention. And then the 360 photo allowed us to show them what the actual piece of equipment looked like, that they'll be working around. It worked out really good to do both things.

    Alan: Almost like a hands-on approach. You get to check it out, be there, experience it before you even get on. I can imagine this would also be good for recruiting people as well, because it's pretty exciting to stand next to a 15 foot tire.

    Nick: We should use it as a recruiting model. That would be great. [laughs] But yeah, it's mainly focused on giving them an opportunity to see that stuff before they get out there, because people don't-- if you've never seen it and you've never been around it, you don't really get the full capacity of it, if I'm telling you about it. But if I can put you next to that tire -- even in a virtual space -- you get the understanding. Everybody asks, "Is this really how big it is?" And you get to say, "Yeah, this is-- you're standing next to the actual truck. It's just not in front of you at this point in time." So it really does get their attention. So that part, just to capture the sheer size of it and what they're getting into.

    Alan: That's pretty awesome. I actually want to try this now. I want to stand next to one of these. I want to drive one of these monster truck things.

    Nick: I'll get into that a little bit later, probably. But our next project is going to be a mine tour, where we'll have them fully immersed in a whole mine situation, so lots of stuff to come with that.

    Alan: How are you combining the kind of 3D models with the photos?

    Nick: The 3D models, when you first start the virtual reality part of it, you're in a warehouse. And then we have all of the 10 pieces of equipment at full size, so you can imagine how big the warehouse looks. And they can choose any of them that they want to go to first or whatever. The full program, they have to visit all 10 of them and go through the whole thing. But they start off in that warehouse. Video plays from MSHA on a big screen that drops down from the ceiling. It's pretty awesome, it gets their attention right away. And it just talks about blind spots. It was put out-- it's a video that's just put out by MSHA, to show that. But then they simply teleport to that piece of equipment and then they can walk around it. They can teleport, we have spots where they can teleport to specifically, like the cab, and some different areas around it, next to the tyre. And they get to take that in. And then we have a spot where they go to the-- we'll take them into the 360 photos. And then they look at a certain spot every time. And then they can move to the next photo, and move all the way around. There's only a specific direction that they can go. They have to see it a certain way. And then there's probably for each one between six and ten different photos of the piece of equipment as they're moving around it. So then they get to move around that. Then when they go back to the cab, once they exit out, they go back to that warehouse where all the equipment is at, and then they can choose their next one and go through that. Once they get done with that blind spot, then they virtually go to a classroom where we have a lot of review information, a lot of photos, a lot of industry specific safety posters hanging on the wall that they can watch, there's another video. And then they can play with the models, we have it so they can pick them up and hold them and see them in a smaller size, so they can actually see what that looks like. And that always gets them, because they're picking up the piece of equipment, moving it around in a virtual world. So that's pretty awesome. Then finally -- especially MSHA-related stuff -- you got to have a test. You start off in a-- it's in another room, and you have a haul truck in front of you, a full sized haul truck. And you have a lot of pickups parked around you in the blind spots of that haul truck. And here's the great thing about using virtual reality is, I told the programmer that I wanted to become a giant, and be able to move those pickups out of the blind spots of the haul truck. And they said, "Yeah, I think we can do that." So you go above the whole scene and you're looking down through a glass ceiling at the whole scene below you. And you have a model of it sitting on a podium next to you and you pick up and move those little pick up models, and it moves the big pick up models down below you. So you move those outside of the blind spots. If you have them in the blind spots, we have it where it will register, where you don't pass the test. So you have to put them outside the blind spots and then you can go back down and see your scene, get in the haul truck and make sure you can see all the pickups before you submit. Make sure they understand -- at least in the haul truck -- where those blind spots are added, and where they can place those pickups to get out of them.

    Alan: So people, when they're operating even smaller equipment, they know not to park or even drive in those blind spots.

    Nick: Alan, that's exactly what we use it for. We train a lot of contractors, so a lot of the people that we train are not necessarily going to be driving those haul trucks, but they're going to be interacting with them. And it's important that they know what they can't see. They might not ever get in one, but at least now when they go out there, they understand if they were in one, what they can and can't see.

    Alan: I would assume this is going to save lives as well.

    Nick: That's my intention, yes. And I have no doubt that it will. You can talk about KPI and what you can actually gain from it. That's a tough one to know for sure, if what you did saved them. But I have no doubt in my mind that it is going to make the mine site a safer place and prevent injuries, yes.

    Alan: It's saving lives, which in turn makes everybody work a little bit smarter. And I think the God view of being able to stand back and look at it and move things around as if you were in that huge giant view, I think that's a really smart idea of doing it. And the fact you've combined CG graphics with 360 photos of the real site is really an impressive part of that. How long did it take to put this together?

    Nick: So the project itself, I'll tell you, I first saw virtual reality about a year and a half ago, it was October of 2018. And that was the first time I ever tried goggles on. NIOSH -- the National Institute of [Occupational] Safety and Health -- they had a setup at one of their conventions that I went to, and they were using it in an underground mining situation. We're surface mining out here, so once I saw that, I said, "Man, we could use this for surface mining." The initiative from MSHA about powered haulage, and we see fatalities from blindspots every year, people getting into them and getting ran over. So that was the start of the project, coming up with how we were going to do that. Did some research then came up with an idea for it. We got a programmer fresh out of school, he just graduated. So we got him for one summer. So we literally had June, July, and August. And me and him worked together every day, pretty much, for that entire time, coming up with it, and putting it all together, and taking the photos. And we crammed a lot into that three months. [chuckles] It took us about three months to get it done. And we can always add to it. At this point time, we have the 10 main things that we use around this area. So it's a done project. Yes, you could always build on it more, but we're moving on now a little bit, getting into the next project. From beginning to end, it took me about eight months.

    Alan: Yeah, and a lot of it was just trying to organize site visits, and making sure that you had all the equipment ready.

    Nick: Yes, that was a huge part. We were lucky that we had mines that would work with us really well.

    Alan: The paperwork usually takes longer than the actual production of the thing. So did you guys build this in Unreal or Unity or...?

    Nick: Yes, we used Unity.

    Alan: And what type of headsets? Are you using the Vives, or are you using...?

    Nick: We're using the Vive headsets in our department. We're currently working on getting a full classroom setup, so we can utilize it for everyone. We have classes that get up to 24 people. So we're working right now to get that setup, where we can have 24 of them going at one time and be able to utilize it in our class that way. Right now it's a little clunky getting them through it all and only having a couple different headsets. We do hook it up to the projector as well, so the whole class can see what one person is looking at, but it's not the same. You need to experience it in the goggles. So we're working hard to get that. Hopefully that's going to be here by the summer, and we'll have that full classroom immersion and everybody will be able to put them on at one time and go through them.

    Alan: You built this for your college. Is this something that you have a mandate to also provide to other mining companies, or other schools, or anything like that?

    Nick: So that's a great question. I'm glad you brought it up. I do work at Gillette College. We are kind of a little bit different in how we do it. We have a grant -- an MSHA-provided grant -- that we operate under and it's just through the college that we do it. It's a weird partnership and I don't know exactly all the rules of it. But with that being said, yes, one hundred percent, we made this with grant money. So if anyone wants this product, I will give it to you. You can use it in your training. I will send you a jump drive. It's only about three megabytes once you put it on the jump drive. So I can send you the entire program and people can use it, view it, whatever they want to do with it. It's provided for free.

    Alan: Did you say three megabytes or three gigabytes?

    Nick: It fits on a small jump drive, so...

    Alan: Yeah, it's got to be gigabytes. Well, I want to try it. I want to go and sit in one of these haulers. You've got to send me one.

    Nick: Absolutely.

    Alan: [laughs]

    Nick: That's truly our goal, is that we save lives.

    Alan: I'll tell you what, if you send me a drive, I will actually go and share it with some friends of ours at a mining company here in Toronto. Because Toronto's a kind of a hub for the mining head offices.

    Nick: Absolutely. I'll send you one.

    Alan: They'll be great. Well, Nick, this is really incredible. I love how you guys pulled this really holistic system, and I can't wait to give it a try. And is there anything else you want to share with everybody before we wrap it up?

    Nick: Yeah, just virtual reality. And I know you guys -- and most of your audience -- is going to know, but I see it as truly the future. Now, I'm focused on safety and safety training, but this is the future of safety training. When you can put people in a certain environment and show them that, without putting them in harm's way, I know it's revolutionary and I know that-- there isn't a word that I could use to explain how I feel about it. But I've only been doing it for about a year and a half. And I know that this next project -- which is a mine tour -- I'm going to put people in situations that they could come across in mining, and I might be able to have a high wall falling on them, and some kind of incident or accident that could happen at a mine site or that we see all the time. You can always talk about those, you can tell them in their class, you can show them pictures of the times when it has happened. But I'm a big believer in the virtual reality side of it, and I can make it happen to you. When you go out to the mine site and that picture brings up a feeling, that's a training that people will not forget. If they have a high wall fall on them in a virtual world, yes, they get to walk away from it. They have no lasting injuries. You can have them fall, not wearing their fall protection. You can do all kinds of things for this safety side of it. And when they get out to the actual job site and they see it, and that triggers the training that you gave them, they'll look at it differently. So I know your audiences, it knows all about virtual reality and they know it can be done to do amazing things. But the safety side of it is where I'm focused. And I can see so much benefit from giving them an idea of what they are seeing before they get out there. That's the type of training that I want to give, and that's what virtual reality has allowed.

    Alan: So basically, what you're saying is you can study it all you want, but virtual reality gives you the feeling of being there.

    Nick: Absolutely. Mine safety, it's a huge deal. And mining is going to be here, it has been here forever, it's going to be here forever. And people don't understand what they're getting into when they go out there. If I can show them what they are getting into and and what they're doing -- without putting them there -- before they even get there, it will literally save lives. And I don't think there's anything more important than that.

    Alan: Thank you so much for sharing this. And if anybody wants to get in touch with you, how can they they reach you?

    Nick: Email is [email protected]. I wish I had a website or something fancy that I could give out, but I don't.

    Alan: Well, I've got the sheridan.edu website, so they can follow up there, and take a look at the courses that are offered through Sheridan and Gillette College. So Nick, thank you again so much for taking the time to share this, and also the generosity of sharing this experience with our listeners and everybody to keep the world safer.

    Nick: Absolutely. Thank you. I appreciate it.

    19 min
  • Using AR to Keep the Temporary Lights On, with Aggreko’s Walter Davis

    Aggreko’s in the business of temporary power - you need an outdoor installation set up for electricity? They’re who to call. But that means that the less time to train, the better. Walter Davis explains how AR is helping trim some time off a new employee’s intake.

    Alan: Hey, everybody, welcome to

    the XR for Business podcast with your host, Alan Smithson. Today,
    we're speaking with Walter Davis, head of talent and learning
    technology at Aggreko. We'll be learning about how they're using XR
    to train technicians and speed up their time to competency, for new
    and existing employees. All that and more on the XR for Business
    podcast.

    Walter, welcome to the show.

    Walter: Thanks, Alan. Glad to be

    here.

    Alan: Really excited. Why don't

    you start by telling us your role at Aggreko and how you started
    working in XR in this role?

    I think first thing to do might help to

    paint a picture of maybe what Aggreko does, which will help in then
    showing how we're applying augmented reality to our services. So,
    Aggreko is the largest provider of temporary power and cooling
    solutions. We work in over 100 countries and we power things like
    Super Bowl here in the US, which many of you might know, or the
    Olympics that are coming up in Japan. That's another project we're
    doing. We also work everywhere, from remote parts of Africa, to
    powering up a data center that needs some backup power. So we're sort
    of everywhere in the background, if you would. Just ensuring that
    major events and operations run smoothly. And with that, obviously,
    we need to ensure that we are providing an expert service and
    minimize any potential downtime. We need to have expert staff to
    support that.

    Alan: So basically, if the

    lights stay on, you guys are there.

    Walter: Exactly. That's a great

    way of putting it. We make it a point to ensure that we have a highly
    reliable service.

    Alan: So I was reading, Aggreko

    specializes in seven key areas, oil and gas, manufacturing, mining,
    petrochemicals, construction. And then events, the live events thing
    almost seems like an add-on to this, but it seems to fit perfectly.

    Walter: It does, yeah. So events

    are really what we're known for, right? I mean, that's where we will
    pop up on occasion. Actually, I could say that we are actually in
    one-- we're in a recent country music video in the background,
    because we are powering the stage that they recorded the music video
    at. But yeah, like I said, keeping the lights on, making sure that
    entertainment can happen, events can happen, and everybody has a
    comfortable experience.

    Alan: So how does VR/AR/XR

    relate to temporary power?

    Walter: With the nature of our

    business -- as we operate in over 100 countries -- we operate very
    broadly but very agilely. And one of the challenges that we tend to
    sometimes have is ensuring that we can bring new hires and products
    together really in that introductory period of time, when someone is
    just joining the organization. We don't always necessarily have every
    type of fleet available for them to familiarize themselves with and
    educate themselves on, to start developing those competencies. And
    where we're seeing AR in particular, is in allowing those individuals
    to gain that knowledge really right when they join the organization.
    They're able to go through an immersive experience in full scale, so
    as if the product was right in front of them. Going-- both
    familiarizing themselves with the key components and aspects of the
    product, but also going through a key service procedures that they
    might be performing in the field, whether it's locking out the
    equipment or going through an A service. This opens up that avenue
    for us to ensure that they can get those experiences and get that
    knowledge really right when they come into the organization.

    Alan: A training module, then.

    Walter: It is, yeah. That's

    where we started. So hence my background, as looking after the talent
    learning technologies. So our focus is on upskilling the folks in the
    first instance. With that, though, we are exploring other avenues all
    throughout there. I was in Glasgow, in the UK here for the last two
    weeks. And I've met with our product design team, who are also really
    starting to leverage some of this technology in the product design
    aspects, our operations team as well, out in the fields. So we're
    seeing it grow. I think learning we've started a little earlier than
    others. So we're a little further ahead. But the application of the
    technology is starting to spread throughout the organization.

    Alan: So can you walk us through

    what the AR experience looks like and feels like for a new employee?

    Walter: Sure. So we've developed

    the experience primarily for use on mobile devices, which is one of
    the reasons why we lean towards augmented reality versus some of the
    others, was the sheer scale that you can deploy AR out within your
    existing infrastructure. So because we could leverage our mobile
    phones, that allows us to deploy this app, to -- you could say -- to
    the employees' mobile devices. And then once they're in the app,
    they'll open it up. They'll choose an experience that they'd like to
    go through. So at the moment, we have it by product, so they'll pick
    that product. Then once they've opened it, it's going to ask them to
    place the product. So that's going to be the first activity. They're
    going to look for a space to place this full scale product. And some
    of our products can be fairly large. So they'll first place it in an
    open area that they have. And then the first thing that is going to
    happen, is they're gonna get a screen that's going to ask them to do
    a walk-around. So a walk-around is something that we've added to the
    introduction of the experiences as we're starting to gain more
    familiarity with augmented reality. And the walk-around helps us
    ensure that they're placing this experience in a safe working area.
    So we show them a video of how to do a walk-around, where it shows
    you walking around the equipment, checking for any objects on the
    ground, making sure that it's not protruding halfway through a wall
    that might be nearby or something, that you have the space you need
    to fully walk around the product safely.

    Alan: They're big, though,

    right? Like, they're the size of a container ship kind of thing.

    Walter: Yeah. So it could be 20

    feet long -- maybe longer -- and 10 feet or so high. So this is very
    large--

    Alan: That's not something

    you're going to be dropping in your living room.

    Walter: No, not something you'd

    be dropping in your living room. And we lean to the full scale option
    simply to give that immersiveness, and give that replication of that
    real life experience. While we did try out a few other scales, we
    found that just having that full scale is just that additional added
    value of "it's really what you're going to see when you do come
    across that real product." And it's going to be in the same way,
    it's gonna be the same positions. There's no compromise. So then
    after you do your walk-around, you've ensured that you've placed this
    object in a safe area to work. You'll get a couple of options to go
    through and perform different activities. So one of the first
    activities that people will learn to perform is LOTO, so learning how
    to Lock Out and Tag Out a piece of equipment. So they'll go through
    the equipment, go through LOTO -- isolating the battery and these
    other sort of technical elements -- to ensure that the equipment is
    safe to work on. And then after that, they can go through other
    activities, whether it's product introduction, where you'll learn all
    the key features and components of the product, or you'll have an
    option to jump in and do any of the service procedures like oil
    changes, fuel filter changes, sort of maintenance/repair procedures,
    if you would. Throughout those procedures what we're doing as well
    is-- so as this is a learning experience, they'll go through a couple
    of sections and then after each section there'll be a Q&A element
    to sort of check their knowledge and ensure that they've-- as they're
    going through it, that they're understanding some of the elements,
    some of the things that are happening. Some of the other things that
    we call out along the way are key safety elements. If there's
    something that they need to be aware of in that particular
    environment, such as recycling, used oil, or if there's a key
    electrical hazard or whatever, we call those elements out as well
    along the process. And then after they finish it up -- it's almost
    literally as quick as I've talked you through it, I mean, obviously
    takes a little bit longer when you have it placed out in reality --
    but then after they've finished up the activity and they've -- and
    again, this is a full scale model -- so when they are going through,
    for example, an A service, some of those activities are on the other
    side. So they will literally have to physically walk around that
    product and go to the other side of the equipment, and do the other
    activities that are on the other side. One of the things we've
    implemented to direct people to different areas is our guide, so a
    guide will pop up, it'll sort of highlight a path for them to walk
    around and they'll walk around the other side to perform that task.
    After they finish now these learning components -- you could say for
    that product -- they've basically finished their AR experience, and
    gained that familiarization that they wouldn't have normally been
    able to have without having the product physically in front of them.

    Alan: How are you measuring

    success of this? Is it just completion, or is it time to completion,
    or is it time to mastery? What are your reference points for KPIs or
    key performance indicators, and what kind of data you're collecting
    about learners?

    Walter: Yeah. So in the first

    instance, one of the first things that we'd focused on was equipment
    downtime. It can be costly to move the product, have these products
    sitting in a learning center or sitting at a at a depot, rather than
    being with a customer. So that was one financial aspect that we
    looked at, was the cost of having these products sit and be
    unavailable for use on a customer site. And depending on the product,
    they'll have a slightly different financial metric. On the other
    side, we know that there's-- I think everyone's aware of obviously
    there's the travel, and other elements as you're using this
    technology. These introductory elements currently before AR, as they
    were being delivered, were being delivered in a classroom. So having
    these components now delivered on as a self-paced module in their own
    time is freeing up up to half a day, a day of training that would
    have happened in the classroom. And what that do-- we're not
    portraying that -- you could say -- as a reduction in classroom time.
    But what we are saying is we can take now this half a day to day of
    training and focus it on a higher level skill of training. So when
    they're coming to the classroom, we're not focusing so much on
    familiarizing them with the product, giving them an introduction on
    how that product works, how you can perform some of these basic
    services. We're focusing more on the expert pieces -- if you were --
    more complex activities. And just that is allowing us to have a
    higher level of knowledge by the end of this process. We're focusing
    on activities we would have normally not been able to do, or required
    to fly or bring someone in through a separate training instance or a
    separate session.

    Alan: Have you experimented with

    head-worn devices? And is there a market difference between using a
    mobile phone and a head-worn device that you've seen?

    Walter: We are experimenting at

    the moment with the head-worn devices. We have a Hololens 2, which
    are not easy to come by these days. So we have received one of those.
    And we're looking at some of its application in the classroom. What I
    would say with the head-worn devices is, yeah, you definitely have a
    slightly more immersive experience than you would have had with the
    phone. I guess the downside of that, obviously, with the head-worn
    devices at the moment is the scalability of it, requiring everyone to
    have a head-worn device. Where we see the head-worn devices applying
    -- as far as into those learning scenarios -- are really once we get
    to more of the multiplayer activities. So what we're looking at right
    now is how can we-- say we were to bring a couple of Hololenses into
    the classroom. One of the pieces we're looking at is, well, how can
    we ensure that everybody who's wearing one of those Hololenses is
    looking at the exact same thing, at the at the exact same time?
    Currently, obviously, we've deployed this as a self-paced element. So
    each person is looking at that experience from their own point of
    view. One person may be on task five or the other one might be on
    seven or eight. Where we're seeing the Hololens -- or even those
    head-worn devices -- adding value are for some of these group
    activities, where we can bring someone through and bring everyone
    through an experience at the same time, and in that same flow. So
    you're really starting to augment almost an instructor based
    learning, if you would. Add in the remote aspect to that, then I
    think you'd have the future of virtual conference training, if you
    would, that web-based virtual training.

    Alan: Have you guys started to

    look at collaboration platforms, things like Meet In VR, or Spatial,
    or things like that?

    Walter: We have started to look

    at a couple. I would say we haven't taken them necessarily as serious
    as of yet. We're really more in the exploratory, if you would,
    looking at a couple of different providers. From that conferencing
    aspect, from our point of view, we're always looking at, "Well,
    how can we still leverage that as well for that knowledge transfer
    piece, whether it's Spatial or Future Visual or any of these others?"
    But now where we're-- I'd say that's an area we aren't as far along,
    especially when I compare other avenues, like I said, where we're
    looking at leveraging augment reality for use in field operations or
    product design or learning. I think collaboration is the avenue that
    we haven't explored as far as others. But I think that's definitely
    the direction of travel. I think everyone's-- we're all at the moment
    in this, you could say single person experience, more or less. And I
    think once you get into that multi-person experiences, where you're
    really going to start to see that ROI jump even higher than it
    already is.

    Alan: What other types of this

    technology, or what other aspects of this technology are you
    investigating? What have you seen that excites you?

    Walter: Yeah. I mean, we've

    we've touched on it already. The multiplayer is really the piece that
    I think is going to get really exciting. It's when you can bring
    people together from all around the world and into these sort of
    shared experiences. I mean, I know you can already do that really
    well with a lot of different platforms in VR. What I'd love to see is
    how we can bring that across those technologies -- where it doesn't
    matter if you're in VR, AR, MR, however you want to-- whatever R you
    want to use -- that you can be together and go through an experience
    together. Have those complex conversations, solving problems, sharing
    knowledge, and just really taking full advantage of this technology
    that allows us to immerse ourselves in conversations -- if you would
    -- and immerse ourselves in creativity, regardless of borders and
    distance, you could say. So I think that the multiplayer is really
    the one that I'm really anxious to see grow and mature really across
    the platforms.

    Alan: It seems like this

    community is really opening up to help each other. What are some of
    the tips that you would give to a new company that wants to start
    deploying this technology for use case?

    Walter: I know a lot of

    companies I think of have looked at this technology, and I know quite
    a few folks who struggled to get out of the pilot purgatory. But one
    of the things that I would recommend is, once you do get that pilot
    and get that initial company centric experience-- because what I have
    found is, although you could have an amazing example of how this can
    apply, unless it's really ticking all the boxes for your organization
    or even has a flavor of what goes on in your own organization, the
    conversation still can be challenging. And I'll even give you an
    example. So if even-- if I've have an example of how to go through a
    service on X product, if it's not necessarily related to our
    industry, it can sometimes be challenging to portray it, how that
    would impact our flow, and the training on actually our products, and
    how it can be used in that way. What I have found is that the things
    I think have helped us get through this is focusing quickly on
    getting an example of one of our products up and going, and then I
    think it's a matter of-- I like the phrase "working out loud".
    If you have any opportunities in your organization to work out loud
    on tools, like we use some of the Microsoft tools here internally,
    like Microsoft Teams and Yammer. I'd recommend sharing those
    successes as early on, getting people excited, trying to build up the
    crowd, if you would. And I'd say one of the things -- which I
    probably myself even could've done a little quicker -- is then once
    you've built up that initial case, you've got a solid use case, ROI,
    and an example is getting in front of the right stakeholders and
    start to build in your partners and getting them committed in
    supporting you. What I found is there has not been any shortcoming of
    support for the technology. It's just a matter of getting it in front
    of the right people, getting them familiar with it, getting them
    introduced, and having the right kind of example that resonates with
    your business. So get a company centric example, work out loud, and
    then get it in front those stakeholders as soon as you can so you
    have a partner to go with you along that journey.

    Alan: Pretty sage advice, my

    friend. What problem in the world do you want to see solved using XR
    technologies?

    Walter: Ooh, yeah. So I think

    that if I had to pick a problem that I'd like to see solved, it would
    be around reducing the time to maturity to leverage new products to
    solve some of the environmental energy challenges that we have in the
    world. I mean, it's an area that obviously here we take quite
    seriously. We have battery storage and other elements that we're
    bringing out. And I'd really love to see this technology leveraged to
    bring those solutions out even quicker, reduce their time to market
    and get that design going even quicker. I think that just in the
    environmental elements, I think there's a lot of opportunities in
    thinking differently, using this kind of technology to visualize
    things we might not have been able to visualize before, look at
    things that we've not been able to see before.

    Alan: Interesting that the value

    chain that XR brings crosses the entire organization. So from a
    collaboration of design, being able to bring in a model of a new
    product, collaborate with people around the world -- so you're saving
    flights, not having to fly people around, that they can collaborate
    together in a virtual space -- then that same asset can be used to
    train new employees without having to have them travel around. And
    you can bring the training to them instead of them to the training.
    And then the last part, you can actually now use the exact same
    assets that you did for design and training, and provide that to your
    marketing teams. And they can then bring these equipment pieces to
    your customers wherever they are, without having to travel or fly the
    giant pieces of machinery around. So you have this use case that is
    the same, yet different stakeholders right across the value chain of
    the organization.

    Walter: That's exactly right.

    And I think that's definitely something that we're seeing. I
    mentioned that I'd been in the UK here a couple of weeks, for the
    last two weeks, and we've been working together on using this
    technology to solve challenges within our particular areas. But one
    of the things I think that we realized even more was how we can
    leverage this technology both across our streams, and save time and
    money across our streams by putting in the right processes upstream,
    and seeing some of those values trickle downstream. I'll give me an
    example: by having higher quality CAD and higher quality designs done
    over in the product design area, that's going to trickle down to
    learning, it's going to trickle down to field ops, and add even more
    value in the experiences we're creating. The experiences we're
    creating in learning can be leveraged as knowledge support and stuff
    in the field. So when you need it, you can reuse some of those
    experiences in the moment you need it. So, as you mentioned, I think
    we're really just replacing the 2D experience and text based
    knowledge that we have, and we're just starting to just bring that
    knowledge into the world. And and with that, we're seeing that it
    applies in a lot of the same ways that it did when it was on the
    screen, and on a website, and in these other places. It's really just
    fascinating and amazing to see this technology evolve and move and
    just change the way we work.

    Alan: With that, I want to say

    thank you, Walter Davis, head of talent and learning technology from
    Aggreko. Thank you again so much for taking the time out of your busy
    schedule to be on this podcast and share your experiences with the
    world.

    Walter: Thank you. And I

    appreciate all the work you and the team are doing. So thank you very
    much for having me.

    Alan: It's our pleasure.

    22 min
  • Shaping the Digital World with Our Hands, with Clay AIR’s Varag Gharibjanian

    We’re used to navigating our computing with keyboards, mice, and maybe track pads — analog input. But those inputs work for desktop computers; they’re clunky for XR interfaces. That’s why we need gesture controls ASAP, according to today’s guest, Clay AIR’s Varag Gharibjanian.

    Alan: Hey, everyone, Alan

    Smithson here. Today we're speaking with Varag Gharibjanian, the
    chief revenue officer at Clay AIR, a software company shaping the
    future of how we interact with the digital world, using natural
    gesture recognition. We're going to find out how Clay will bring our
    real hands into the virtual world. Coming up next, on the XR for
    Business podcast.

    Varag, welcome to the show, my friend.

    Varag: Hey, Alan. Glad to be

    here.

    Alan: It's my absolute pleasure

    to have you on the show. I know you guys are working on some cutting
    edge stuff, so why don't I not ruin it, and just let you tell us what
    is Clay AIR?

    Varag: So Clay is a software

    company, we're specializing in hand tracking and gesture recognition,
    mostly in the AR and VR space. And we're also tackling a couple other
    industries, automotive. And our third product category we call Clay
    Control, which is kind of all the devices that can use gesture
    interaction at a distance.

    Alan: Are you doing this from

    single cameras, or is this from infrared cameras, or a combination of
    everything?

    Varag: Yes, so Clay's-- we're

    hardware agnostic. So it'll work across all those types you just
    said. It could be one camera, two cameras, or more. And all different
    types, so we'll work on RGB cameras that you'll find on everyday
    smartphones, to what you might find embedded in AR and VR devices, to
    monochrome ones, time-of-flight ones, and so we're pure software and
    we've worked across a lot of those different types and have
    compatibility with most of them now, which gives us a lot of
    flexibility and it's really useful.

    Alan: So I'm going to be able to

    look at watches on my wrist in AR, right? Like I'm going to be able
    hold my hand up and see what the newest, latest, greatest watch is?

    Varag: It's actually pretty cool

    that you say that, because that is one of the use cases that comes in
    often inbound to us, as companies -- it hasn't happened yet -- but
    those companies definitely brainstorming around how you track the
    hands even with just a smartphone, like overlaying something.

    Alan: We actually did it. We did

    a project just using Google's hand tracking library. We managed to
    make the watch sit on the wrist, but it was kind of glitchy. It would
    sit weird. And yeah, it was-- it was not great, but we made it work,
    it just wasn't sellable.

    Varag: Yeah.

    Alan: So this is really a

    foundational software. And I know you guys are working with some of
    the larger manufacturers. You want to talk about that -- or can you
    talk about that -- and then what that might look like?

    Varag: Yeah, I can speak a

    little bit about that. So we feel -- like you said -- this is
    software that really needs to be optimized for the hardware that it's
    working on. The deeper it is in the stack, the better performance
    you'll get, and the better synergies you'll get with all the other
    technologies that are working on these devices. So that's why when I
    joined the company, really, I made the focus to get as deep into the
    stack as possible. We looked at the market that time a couple of
    years ago to look at who is really central to defining the reference
    stack. What's going to most AR and VR devices? And to me, Qualcomm
    made the most sense. So we spent a lot of time working with them. As
    you know -- and some of our listeners might know -- they really do
    define a lot of what goes into the guts of most AR and VR devices
    today. And likely in the future, too. So we work closely with them.
    What that means that from a software architecture standpoint and a
    hardware standpoint, we try to make our software as optimized as
    possible for their reference designs. And as a result of that, any
    OEMs that want to pick up a Qualcomm chip and all the technologies
    around it, we're really well suited to fit along the side of all
    those other technologies.

    With Qualcomm's 835 -- or 845, whatever

    -- their new chips are kind of really powering the future devices. I
    know pretty much all of the standalone VR headsets right now and most
    of the AR glasses are running on the Qualcomm chips. So this kind of
    opens up the world of spatial computing. And hand tracking needs to
    be there. It's kind of part and parcel. And now when Qualcomm's
    announced their new XR2 chip, I think this is going to really, really
    unlock it. And it's basically 10x performance right across the board.
    And so you guys are well-positioned for that.

    Varag: Absolutely, yeah. And

    it's pretty cool. We have every chip that comes out. The best part of
    that partnership, in a sense, not just the amazing people we get to
    work with, but also the chips that we get some of those first
    references sites to our office,and to our labs. And it's pretty cool
    that we get to experiment with the latest and greatest machine
    learning models, and try to get the most out of those chips. Which in
    every single chip that comes out -- which makes sense -- like we get
    just a little bit more accuracy, a little more points of interest at
    a lower consumption rate on those pieces of hardware. So it's pretty
    cool, to see that evolve. It's pretty quick. Even though it's day to
    day, it moves more quickly than people think.

    Alan: I think this whole

    technology stack of XR is moving. It's actually moving way, way
    faster than I had ever anticipated. I was looking at 2025 for
    ubiquitous AR glasses. But after seeing what came out of CES this
    year, and learning about this Qualcomm XR2 chip, you've now got AR
    glasses coming out en masse. CES, I think it was 11 AR glasses that
    came out this year, and all of them in the form factor of a pair of
    sunglasses.

    Varag: What did you think of

    that, by the way? Because for me, when I saw that and I saw Nreal's
    glasses, which to me are the closest thing you've got to consumer AR.
    I saw a lot of companies that were coming out kind of mirroring what
    they were doing. Slightly different form factors, seem like fast
    followers in a way. To me that seemed like a good signal.

    Alan: I think it's a great

    signal. I think what will happen is -- like with the VR market --
    there was thousands of Chinese knockoffs of VR headsets and all of
    them have gone out of business now. I think you're going to see this
    kind of flooding of the market of these cheap AR glasses. And it will
    come down to things like embedding technologies like Clay into it,
    and having proper Qualcomm chips in there and in proper chipsets. And
    it really comes down to having the full tech stack to deliver on the
    quality that people really want. So I think you're going to see lots
    of incumbents, you're going to see a lot of people come in and try to
    take over the market. But these are not easy problems to solve, as
    you know. And then--

    Varag: Especially because every

    maker has got to print hardware make-up, in a sense. And so it's
    really about bringing these technologies together in that specific
    device. I think what Qualcomm is doing and companies like us working
    together is making it more-- look, I'm really looking forward to--
    and this is why I joined the company in a sense. I want that iPhone
    moment within this industry, where the right technologies come
    together and a quote unquote, a kind of dominant design is developed.

    Alan: With Facebook now working

    on AR, and you've got Magic Leap, and Microsoft's Hololens.

    Varag: Yeah.

    Alan: So they've kind of set the

    bar pretty high. The Hololens was an amazing piece of tech and now
    Hololens 2. And, you know, Magic Leap-- people are going to expect
    when they walk around holograms, they stay put. They stay rock solid,
    steady, attached to the real world, which is really gives you that
    kind of mixed reality experience. But it's not easy to do that. And I
    think a lot of these companies are like here, "Here's a heads-up
    display, you can see things in mid-air." But they're not
    thinking about the tracking. They're not thinking about object
    recognition. They're not thinking of the full tech stack that's
    required for real pervasive augmented reality in the world. Then
    you've got glasses like the North glasses, which just give you a
    little heads up display, like your Apple Watch display. And I think
    those are going to have a great place, too. But let's get back to
    hand tracking, because this is a vital part. And I know Oculus Quest
    just introduced hand tracking. And I think the Quest -- I just read
    an article today -- the Quest has sold over half of all the VR
    headsets in the world last year.

    Varag: Wow.

    Alan: Yeah, it is-- it's a game

    changer. And they introduced hand tracking. And as soon as we start
    to see apps that has that built in, that will be the new standard.
    That'll be the new normal. Just like standalone, no wires,
    non-tethered VR is now the new normal. We're not going backwards.
    We're not going, "Hey, let me connect this to a big computer and
    cables and stuff." Nobody wants that anymore.

    Varag: Right, exactly, exactly.

    And you know what? By the way, Alan, I think it's already becoming--
    you're right, it's not the standard yet amongst users and consumers,
    but amongst the OEMs. And that's who I speak with a lot. It is
    becoming like, hey, they want the same thing, ASAP.

    Alan: What's the next step for

    hand tracking, then? You've got-- you're able to track hands very
    precisely. Are things like midair haptics-- I know-- I guess Leap
    Motion would be a competitor to you guys, even though they're using a
    hardware solution to do that. But things like midair haptics, with
    the Ultra Haptics or the Ultra Leap now. These things are really
    foundational and interesting.

    Varag: Yeah, I think there's

    still some things to be solved even on basic, marker-less,
    hardware-less sort of computer vision based hand tracking. So even
    things like occlusion or field of view and compute, those all need to
    be even better, so to say. So what I mean by that is, if you take
    your hands and you go out of the field of view and you quickly bring
    them back into the field of view, what's that activation time look
    like? Like, how quickly do you go from not recognizing that hand to
    recognizing it. And in particular, now a lot of people are using
    machine learning models, optimizing those models, making sure that
    there are fewer false positives. There's still some things to be
    solved. If you have other hands in the field of view or-- and then
    from a compute standpoint, just making sure it all works and it's not
    burning the battery. There's still some things to be done there. And
    that's an ongoing challenge, especially if you're not like a huge
    multi-billion dollar company like Facebook, and you don't have all
    the resources they have to stitch that all together in the best way.
    So companies like ours are getting better and better at doing that
    and offering that. So there's still some things to be done there, in
    terms of once that's solved, or in tandem, There's-- you're right,
    there's some cool things you might be able to do with haptics,
    because one thing you notice is that as our technology gets better
    and better, and you really feel like your hands are there, it becomes
    more confusing that when you actually go to touch something in the
    virtual world, especially if we're talking about VR, you don't have
    any of that that sensation back. It's really confusing to your brain.
    So having something like that -- even if it's a small form of
    feedback -- could be really, really helpful. Although I'm -- at least
    for consumer VR -- I'm against adding any extra hardware. I think
    it's hard enough for consumers to adopt -- for the regular consumer,
    even the early majority of consumers -- to adopt VR. We don't want to
    be making them wear anything more on their hands or controllers,
    even, to me sometimes can be excessive or too much of a requirement.

    Alan: It's interesting you say

    that, because I actually got to try something at CES last year, and
    it was these little sensors that went over my fingers, almost like a
    pulse oximeter.

    Varag: Okay.

    Alan: And they just gave slight

    haptic feedback on the tips of your fingers. And it turned out that
    that was actually quite convincing. We didn't need the whole haptic
    glove or anything. It was just a very small sensor that clipped on
    your finger. And I'm assuming -- I was wired -- but I'm assuming it
    could maybe be Bluetooth or whatever. But it was enough sensation,
    that I actually felt it was real. I reached out and touched a fire,
    and it buzzed. Then I jumped back. [chuckles] I felt like an idiot.
    Freaked me out.

    Varag: It's pretty cool. Yeah.

    And I even met this company not too long ago. Unfortunately blanking
    on the name right now, but it was at VRX last time I was there. And
    they were working on something really interesting about haptics. They
    were changing what that haptic feedback felt like, both in frequency
    and intensity, based on what you were touching within VR. Which I
    thought was really cool. So I'm looking at a desk right now, touching
    a desk in real life feels very different from touching this cup,
    right here. So how do you bring those differences in VR too, along
    with hand tracking? That's super, super exciting. And I think it
    makes sense definitely now -- as you know -- to do this all of the
    enterprise space, but rolling that into consumer where prices need to
    lower and things need to be even easier to use, that's yet to be
    seen. Maybe there needs to be something like of what's embedded in
    the headset, where there's ultrasounds around or something like that.
    That's pretty far out there, though, I think.

    Alan: Yeah, I don't know. This

    is going to be so many use cases for this. And I'm just thinking of
    the different ways hand tracking can be deployed, and in just the use
    cases from e-commerce and being able to see products in your hands,
    but also just interacting in the real world in a natural way. One of
    the things that always stuck with me is that when Hololens 1 came
    out, when you show a kid something, they just pick it up and they do
    it. But when you showed anybody over a kind of 30, that gesture
    motion of the clicking that Hololens-- you had to reach out and kind
    of thumb-point-click thing. And nobody could figure it out. Everybody
    just reached out and tried to push things, they just reached out
    naturally to touch it. And I think the Hololens 2, I believe they've
    addressed that. But there's gonna be so many different devices out
    there. And I think you guys have made a really good idea in putting
    it at the chipset level and really making it available as that
    reference design for everybody.

    Varag: Yeah, it's paid off so

    far, in the sense that when we go to OEMs or companies that really
    are building this hardware -- or even those that are buying it --
    they love the fact that we understand the architecture, the software
    itself understands the rest of the architecture. And we've got a
    version of something we could put in there that most importantly, it
    interoperates with everything else. Because there's just so many
    other functions you need to-- and sometimes we're using literally the
    same cameras that are being used for other functions, too. So we need
    to respect those as well and make sure they work harmoniously
    together, to enable those use cases as well.

    Alan: Indeed. So speaking of use

    cases, let's dig into some of the use cases. I've got a pair of AR
    glasses or VR goggles on. What are some of the use cases where I'd
    want to use some of the gesture recognition? I'm just looking at the
    page here on your site about gesture recognition. How you have flip,
    grab, up, down, swipe, pinch, victory point, all these things. But
    what are some of the use cases, practical use cases that people can
    wrap their heads around?

    Varag: First thing I'd like to

    say is, I think of hand tracking and gestures and some of the other
    input methods as kind of what is like the keyboard and mouse of back
    when in the PC era, more like just basic navigating around that
    device, I think is going to not just only be hand tracking, but it
    will be all those, in some multi-modal form together. So I think
    that's number one, is just getting around the device most easily. So
    when you first put that headset on, you don't necessarily want to
    have to reach for the other controllers or something else every time
    you want to just navigate the device. So I think that's first and
    foremost. And they're in app control as well, too. So obviously for
    gaming, just interacting, you can be holding like -- it's more on the
    consumer side -- you can be shooting out of your hand, we've done
    applications like that before. But especially on the enterprise side
    -- where things are being used a lot today -- I see various things
    like just communicating to the device, or doing grab gestures for
    short cuts in the device makes a lot of sense. Sometimes it's just
    like capturing a screenshot of what's going on. You might do one of
    those gestures to take a screenshot or take an image if that's what
    you're looking at in AR. But what comes up a lot definitely -- for
    gestures, at least -- is shortcuts. To the extent that there is
    something you want to do very quickly in-device that you do
    repeatedly very often, using a gesture to do that can make a lot of
    sense. Especially when some of the other inputs are just not
    convenient. So like voice, for example, if there is a noisy
    background, you might want to just use your hands instead. So yeah,
    that's some of the ones we've seen, especially if you've got gloves
    on or something, in the enterprise use case and you don't-- you can't
    necessarily do touch on the device itself and you might want to use a
    gesture. And those functions can include things like-- those
    shortcuts can be like muting/unmuting, changing volume, changing apps
    you're in, waking up the device or making it sleep. So and so forth.

    Alan: The gestures in shortcuts

    are going to be amazing, because I think it's kind of like you said,
    multi-modal is going to be really important, being able to use voice,
    being able to use gaze, understanding what you're looking at. I think
    once we have eye tracking and this gesture recognition and voice,
    it's just going to be "Show me that thing." and it knows
    you're pointing, it knows what you're looking at, and it knows what
    you said, and gives you what you want.

    Varag: And Alan, one thing I

    think about a lot is--t I like to think several years ahead, it's
    fun. When you think about all those things coming together, just
    imagine what you can observe about-- some people might say this is
    creepy, but what you can observe about what a user wants and what
    they're doing. Today we're looking at what's someone's behavior on an
    app, on a phone, or on their laptop. And you can track a lot through
    those sensors that are onboard. But just imagine what you could track
    about where someone's eyes are looking in a given seat, how long
    you're looking at a certain ad, or specifically with their hands. So
    we have at least 22 points of interest tracked in some cases, and we
    could see how they're-- where they are in 3D space, relative to all
    the other things that are in 3D space. And that, I think, is really
    interesting for capturing intent of what a user is doing. Super
    exciting stuff. And how you can maybe monetize that data at some
    point is interesting too. But just the fact that you can understand
    what a user wants and how they're moving their hands around. I think
    hands are a really natural, expressive way of what a user, a person
    is doing.

    Alan: Indeed. I wonder if you're

    going to have people flipping off the bird, and can you edit that
    out, so it doesn't do it? [laughs]

    Varag: [laughs] That could be

    added. Like every time that's read--

    Alan: "This user needs

    help." And support comes up. [laughs] "I see you're having
    troubles. Can I help you?"

    Varag: Exactly.

    Alan: Awesome. So let's get

    practical here. If you're going to be building this into all sorts of
    things, not just VR and AR headsets, put automobiles and that sort of
    thing, why would I want gesturing or hand recognition in a car?

    Varag: That's a good question.

    So we're seeing that in the automotive space, for example, that--
    I'll make it more broad than automotive. Anywhere where the rules of
    how you interact with a device are either being written for the first
    time or being rewritten, I think that's where gestures are
    interesting and hand tracking as well. In the automotive space, I
    think there is a lot of automotive players trying to figure out what
    does the future look like, especially with autonomous driving coming,
    I think that's part of what's driving that. And so that means that
    passengers and drivers have an opportunity to have a different
    experience with how they interact with the car. And that's where
    something like gestures comes in. Even when things are not autonomous
    yet, actually, we were-- one of our clients in that space is Renault.
    And I think one of the key drivers there is safety. And the use case
    there is drivers making gestures to control the infotainment system.
    So any function of the infotainment system that either takes too many
    button clicks to get to that specific function you want, again, have
    a shortcut to the feature. The button you want to press is just too
    far away, and there's too much reach. That seems like it's a small
    thing, but really, when you're driving, any milliseconds matter. It's
    to make it a little more convenient for the driver so they can mute,
    unmute, answer a call, end a call, zoom in, zoom out, so on and so
    forth. Then we can map gestures to different kinds of functions
    there, that make the most sense for that specific car. And it's not
    just for the convenience, but also for safety. The more we can reduce
    the cognitive load of drivers, the more safeness. That's the idea.

    Alan: Yeah, I guess that makes

    sense. So what's next for you guys then? You've partnered with
    Qualcomm. You're gonna be kind of at the root level of all of these
    new devices. What's next then?

    Varag: What's next is going to

    be continuing that work, what we're doing with Qualcomm, and really
    trying to onboard more and more of their customers. This year we've
    been focusing on bringing on Tier 1 companies that are building AR/VR
    devices. And there is still in almost in every case these device
    makers, they make little tweaks and changes to that reference design
    that they get. And so they might take different kinds of cameras. And
    there is some work with Clay to onboard them in the right way, make
    sure that the hand tracking is working well -- I guess -- with all
    the other functions, and we'd like to own that process a little bit
    too, and get involved. So it'll be the year for us, I think, where
    our tech has never been as ready as it's been today. What I mean by
    that is we're interoperable now with cameras that are the most
    available in the market, including monochrome cameras that are used
    for six-degrees-of-freedom. So we can work with those. And so the
    next steps will just be bringing on those big customers. I think the
    demand is going to be there this year more than ever. That's what I'm
    excited for, mainly because we have the software that runs in the
    right hardware more than ever. And I think what Oculus did and
    launched that as a feature proves that it's important, and that
    everyone else now wants it too. So that's what I'm excited for for AR
    and VR in 2020, for sure. And from a technology standpoint, we're
    just always-- that's never going to stop. I think the key thing we're
    going to be driving there is getting more and more detailed tracking
    and more more accurate tracking. And at the same time, monitoring how
    much we're really consuming the CPU, the GPU, or DSPs onboard, and
    optimizing like crazy, because we don't want to be--

    Alan: You guys don't want to be

    the one that sucks up your battery.

    Varag: Totally. Yeah. We want to

    be the one that's like "That works better than anyone else."
    and no one says you're too expensive.

    Alan: I have to ask, because I'm on your new website -- it's not published yet, but it'll be out soon -- you have here, "scalability: one to four hands."

    Varag: [laughs]

    Alan: I'm not sure what what's

    all about, but alright!

    Varag: [laughs] So maybe that

    applies to, like-- [laughs] maybe that applies to the other use
    cases. So we have three product categories, we've got plain reality
    for all AR and VR stuff. But for one to four hands, four hands could
    make sense maybe in-- actually in situations where you've got a
    really, really big screen at big shows. Things like that, you might
    want to have multiple people, multiple hands.

    Alan: I was also thinking being

    able to -- in virtual space -- being able to see other people's hands
    come into the space, and hand them things.

    Varag: Haven't seen that yet

    come up. But I could see that coming for sure. So, yeah, absolutely.

    Alan: Very interesting. What is

    one problem in the world that you want to see solved using XR
    technologies?

    Varag: It's a good, good

    question. I think that there's a lot that AR and VR can solve. I
    guess what I'm like personally most excited about -- this is less
    about Clay and more about me -- but I think the problem of distance
    is just-- and that's that's very general, but what got me super
    excited into XR in the first place was this idea that two people can
    be far away from one another. And I know we can connect now through a
    lot of different mediums, but I think that those mediums really only
    get us halfway there. And strangely, being halfway there makes us
    less often to be there, like in person face-to-face today. And I
    think that-- I really actually-- like I actually despise that about
    social media, and all the other technologies that help us connect
    today. It's like I feel like people are now lazier than ever with
    really connecting. What I'm excited about in AR and VR, I think it's
    the best candidate for a technology to really, actually bring us to
    90 percent there, maybe even close to a 100. There is a lot of
    problems that get solved, obviously, by being able to do that. One is
    just connecting better with people. But all the things that come
    around, all the problems that we get around today by not doing that
    well enough, that will be solved. So that's seeing a loved one more
    often and feeling like you're present with them, that's great. If
    it's trying to do business with someone who's really far away, I
    think like feeling present with them in a room -- like you can do in
    VR -- is pretty amazing. So that's what I'd say.

    Alan: It's a really good answer,

    and I think it's a great way to position XR as a medium by which we
    can create a more global unified world.

    Varag: Absolutely. I mean, I

    remember the closest thing I did to that-- I haven't done enough
    multiplayer or multi-user type of VR experience lately, but I
    remember a while back I tried -- I guess maybe like a year and a half
    ago -- I tried a company's software called -- you may or may not have
    heard them -- called VRChat.

    Alan: Yeah, absolutely. I

    actually did a -- ooh, back maybe three years ago -- I was in
    Gunther's Universe, which was one of the VRChat rooms, and I was
    interviewed in Gunther's Universe. It was super fun.

    Varag: Yeah. And I just remember

    sitting there thinking-- I felt like -- I'm sure it's much better
    today than it was when I tried it -- and I was like, "Oh my
    God." I just remember thinking, "Wow, I really feel like
    I'm here with someone who's a stranger." And it was kind of
    strange, because that kind of creeped me out a little bit, in the
    sense that I was like "Wait, this person, I'm in a room with
    them. I don't know them, though." And strangely enough, I didn't
    have all the information that you get when you're -- because I love
    networking, I love meeting strangers all the time -- but when you're
    in person, you get 100 percent of who they are. You see them,
    interact with them, you shake their hand. But in VR, at that time,
    three years ago, you're only halfway there.

    Alan: The first time I had that

    was with Altspace. I was lying in bed, and I put on Altspace. It was
    with the Gear VR.

    Varag: Yeah.

    Alan: And I was just walking

    around. I was kind of talking myself, like, "What is this silly
    thing?" And somebody stood beside me and started saying, "Hey,
    it must be your first time in here." And I couldn't figure out
    that they were talking to me. I was like, "What?!"

    Varag: That was your first time.

    That's an interesting note, for sure.

    Alan: Yeah, and it's funny

    because up until recently, Altspace really didn't improve. But I was
    in there the other day, and it actually looks like they've made some
    real improvements, which is fantastic to see. Everything's getting
    better.

    Varag: Absolutely. I've got to

    check that out.

    Alan: And then the new one that

    we really love is Meet In VR. It's a Danish company. And they've
    really nailed the interactions for corporate clients, for enterprise,
    because it allows you to be in a VR chatroom. But you have access to
    photos, you have access to 3D models. You can write on the walls, you
    can have conversations. It's-- I don't know. To me, it's the most
    comprehensive business tool for communication, it's really great.

    Varag: That's a big one, because

    I'm on the business side of Clay. And so that means I travel around a
    lot and I actually like traveling lots. I don't mind it too much. But
    being able to do that pretty quickly in virtual reality would be
    amazing. I've got to try that out. But it hasn't happened for me yet.

    Alan: Indeed. Well, I want to,

    Varag, thank you again for taking the time out of your busy schedule
    to join us on the show.

    Varag: Thanks for having me,

    Alan. This is fun.

    Alan: Absolutely. Where can

    people find out more information about Clay and the work you guys are
    doing?

    Varag: So they can go to

    www.clayair.io. They could reach me out on an email, too. It's my
    first name, [email protected], as well. We have a new website coming
    soon, so we'll be announcing that somewhere on LinkedIn, social
    media, and all that good stuff. Alan, I love what you do in the
    industry. Just please keep it up and keep in touch.

    Alan: Thank you very much. And

    you know what? I'm going to take this opportunity then to tell people
    to subscribe to the podcast. I always forget this part and people
    keep telling me, "You've got to tell them to subscribe."
    So, subscribe to the podcast and you'll get more XR for Business
    deliciousness, all the time.

    Varag: [chuckles]

    Alan: [chuckles]

    28 min
  • Pivoting Game Dev Skills into XR Utility for Enterprise, with Extality’s Arash Keshmirian

    Today’s guest got his start in the world of game development. But soon, Arash Keshmirian saw the writing on the wall that XR’s current usefulness was better-suited to the worlds of industry, retail, and journalism. Arash and Alan discuss how he made that transition.

    Alan: Hey, everyone, it's Alan Smithson here, the host of the XR for Business podcast. Today we have Arash Keshmirian, co-founder of Extality. His personal goal is to create powerful content that delivers results. We're going to dig into using Magic Leap and Hololens and mixed reality headsets as a tool for business. So all that and more, coming up next on the XR for Business podcast.

    Arash, welcome to the show, my friend.

    Arash: Thank you for having me.

    Great to be here.

    Alan: It's my absolute pleasure.

    You guys have done some pretty cool stuff. I was on your website
    playing with a shoe. What is Extality?

    Arash: Were a lot of things to

    many people. So we built Extality out of a long, 10 year experience
    in the games industry, building mobile games. Did a lot of games,
    including Zombie Gunship, which ended up being this kind of worldwide
    sensation of shooting zombies from an airplane. That company --
    Limbic -- we ended up doing a lot of XR stuff. And kind of around
    2016, 17, 18 we built a game for ARKit, called Zombie Gunship
    Revenant. And that ended up being a huge hit across the app store.
    Apple featured it a whole bunch of times. It was one of the 2017
    games of the year. And it spread ARKit to a lot of people, trying new
    things on their new iPhones. And we later did a project called Zombie
    Guns Raptor with Oculus and Oculus Go, Gear VR. But really kind of
    around that time -- 2017 -- we started to feel like it was getting
    way too crowded in the games business, and we were starting to look
    around and try to figure out what we could do with our experience in
    high-performance graphics and making cool experiences, immersive
    experiences. I started talking to a guy named Ryan Peterson, who's
    the founder of a CEO called Finger Food out in Vancouver. And he was
    telling me about all these exciting opportunities in AR and VR for
    enterprises. He was talking about how they'd saved millions and
    millions of dollars for a truck company that was looking to move
    their design to virtual reality from using clay models. And this got
    our head scratching, we were like, "You know, maybe there's an
    opportunity to use all of our games experience, to help big companies
    and do more than just give people an entertaining hour on their
    phones." So we founded a new company called Extality. And we set
    out to essentially discover companies that really wanted to explore
    XR, be it on their phones, on headsets, iPods -- every type of XR --
    and leverage our background in doing just really hard graphics
    problems, building scalable global servers and connectivity, all
    those hard things that you learn how to do making games, we quickly
    realized that we're super, super applicable to building enterprise
    solutions as well.

    Alan: Actually, I know Ryan very

    well from Finger Food, really great guy. And they've done some
    amazing work in the space. What are some of the highlights that
    you've done for enterprise? And first of all, I just want to say that
    having a flying zombie shooter game? Pretty awesome.

    Arash: [chuckles] Thank you.

    Alan: The fact you guys had a

    hit with ARKit is pretty amazing, because there's not too much out
    there leveraging the power of ARKit yet.

    Arash: Yeah. I mean, if you want

    to talk about games for just a second, it's an interesting thing. I
    mean, it gives people a totally different experience using their
    phone as the controller and running around the room. We have all
    these videos during our user tests of people crawling on the ground,
    shooting zombies and jumping up and down. And it became a very
    athletic experience, playing games this way. But what we quickly
    found -- and part of why we started to look at where can we use AR
    outside of games -- was that I think session times really suffer
    because if you want to play a game, you want to be lazy, right? You
    want to sit on your couch. You want have your beer. You want to take
    your phone out and just play in the least movement, sort of laziest
    way possible. And asking people to get up and run around for 30
    minutes -- not surprisingly -- led to really short session times. And
    so we gained a lot of people wanting to download the game to try out
    AR, but we lost a lot of retention, because people weren't interested
    in spending the kind of time they would normally spend on a mobile
    game, running around their house shooting zombies. And so that was
    kind of the impetus for us to say, "Well, we've got a lot of
    downloads on this, but it doesn't perform in the way games were
    supposed to perform." But we still just love this technology and
    we want to keep using it in a place where it really will make a
    difference for people.

    Alan: This is why PlayStation VR

    has done better than the other full room scale experiences, because
    most people when playing a game are just sitting there.

    Arash: Exactly.

    Alan: And they want to play the

    game from the comfort of their desk, chair, or couch, or whatever.
    And PlayStation VR lets you do that. Most of the games are on rails
    and you very rarely have to give it in 360 degrees.

    Arash: Yeah, exactly. Yeah. And

    I think that just sort of underscores the importance of knowing how
    things are going to be used and building experience for it. I think a
    lot of people get carried away with, "We have this great
    technology. We can have a 10 meter square play space. Let's use every
    corner of it." And the reality is, that doesn't always make
    sense for what your user is after, for that particular game.

    Alan: Rarely makes sense to use

    it 10 meters by 10 meter. I mean, when is the last time you had a 10
    foot clear space anywhere?

    Arash: Exactly. Yeah. It's

    really just outdoor experiences that can really compete.

    Alan: Enterprise. What have you

    guys done in the enterprise side, what solutions have you created?

    Arash: Yeah, like I said at the

    beginning, we've been kind of all over many different industries. And
    so we got-- our main first break as Extality was leveraging some of
    our game connections and things like that, to get close to Magic
    Leap. And Magic Leap, of course, did an amazing job of evangelizing
    mixed reality across the world. And one of the companies that they'd
    been talking to for a while was CNN, Turner Broadcasting Cable News
    Network. And what they wanted to do was essentially explore with CNN.
    What does it look like to do news in mixed reality and how do we
    augment the experience of essentially what you might do in the
    morning -- which is watching headline news -- how do we augment that?
    How do we use these new display technologies? How do we make that a
    richer, more interesting experience? And so what we ended up looking
    at with them was at first, really, how do we show news and what's the
    benefit of having, for example, Magic Leap device on your head when
    you're having breakfast and maybe getting ready for your day? And we
    started to play around with the first thing that comes to mind, which
    is, well, we can have virtual televisions on multiple different
    channels, in different sizes all around the room. And so we started
    to explore with building user interfaces to create just like a really
    great channel selector. And you can have maybe a little TV with the
    stock market over at the end of the breakfast table, then a really
    big TV that's got a particular show or maybe a Anthony Bourdain
    telling you something about food. Have that at a larger scale and
    then imagine even potentially multiple users seeing the same TVs or
    some shared TVs and some individual TVs. How do we make that a cool
    experience, so that you can have a TV wherever you want it floating
    in space? And that was definitely a new way to to consume media. And
    that's where the conversation started, I think. But as we got deeper
    into it, you realized that, of course, there's a lot of stories that
    really benefit from being told with 3D content. And this isn't
    something that's new for news, right? I mean, we've seen a lot of 3D
    rendered imagery. I think it started with weather. But a lot of the
    time, a lot of even local news stations do this now where there's a
    rendered fly through of the crime scene. Looking around the world for
    something or showing different objects in 3D, so they can pull apart
    parts of it and show the pieces of the latest SpaceX rocket, or
    something like that. And we started there, because a lot of companies
    -- including CNN and Turner -- they have teams that are dedicated
    toward building the sort of content for broadcast news, where they
    prepare a rendering and send it along with the video feed. But we
    thought, "Well, what if we can take that content and somehow
    make it more interactive, and make it so that you can place it on
    your breakfast table, or on your coffee table?" And so we looked
    around for a good story that might have been a good sort of starting
    point for maybe exploring these types of interactions in this kind of
    presentation. One story that was really coming out in the news at the
    time we were working on this project was the Thai cave rescue, where
    these soccer players got stuck in a cave in Thailand and they were
    stuck in there for quite some time. It was a situation where the
    whole world was watching and they had to find a way to rescue these
    kids. And there are a lot of technical challenges. There were a lot
    of weather challenges. It was just a really interesting story. And I
    think where we were really excited to show that with mixed reality
    was actually showing the cave itself and giving people a sense of how
    deep were these kids, and how far below the ground, and how quickly
    did the water rise. And so we started to work with CNN's team that
    builds this kind of contact, to create 3D models of the actual
    geography and the terrain around where this cave was. And we even got
    down to doing things like where did the water rise to, and where were
    they at the time? And building these timeline experiences, where you
    can drag a timeline back and forth and see what kinds of things
    happened during the course of the rescue. But what was really
    exciting for me was actually being able to start to make it in your
    own home. And what I mean by that is we took a cross-section of the
    cave. So there's some parts where it gets as narrow as 16, 15 inches.
    That's really, really small. But you hear that number and you don't
    know what would it be like to crawl through a space that large. And
    so one of the interactive experiences we built for that story was an
    actual cross section that floats in your room and you can actually
    put your hands through it and you get a feel for like, "Wow,
    that's going to be really tight to squeeze through. I can't imagine
    crawling through two or three meters of that." These types of
    things bring the news in in a different way. But what got interesting
    with this sort of presentation of factual journalistic material in 3D
    was -- this actually came up from the CNN creative director -- was
    how do we know what we can show and what we can't? What I mean by
    that is if you were to show in 3D in a fully explorable by users kind
    of way a crime scene, there's a distinction between what we know--
    what we know for sure. And what we may have added to kind of fill out
    the scene. So, for example, let's take the raid on Osama bin Laden,
    for example. If we were to create that on your coffee table, we'd
    have to show the whole building. But it's possible that we don't know
    what's in the other rooms. It's possible that we don't know what the
    content is of one of the back rooms. And we have to make this
    judgment call, because as game designers, we would have just filled
    that up with all kinds of set dressing and filled that with stuff to
    make it look good. But for the news, it's-- they're held to a higher
    standard. They have to know that everything they put into that
    building is actually based on facts, or based off a photograph, or
    based off satellite data. So this is a huge concern for CNN. And
    they're like, "We can't say that this is there, if we don't know
    that it is." And so there was a lot of conversation around in
    the future as this develops as a key part of the way media is
    presented. Do we show it in some way? Do we draw cross hatches
    through those areas? Do we say, "Hey, this area was added for
    visual effect, but we don't know that it conforms to the news."
    And so it creates a lot of actual questions around this new type of
    media. How do we tell the news and how can we verify that everything
    in it is true, because the user can literally poke their head in
    anywhere and we don't want someone saying, "Well, I poked my
    head in the back room of the Abbottabad compound. And I found that
    there was -- whatever -- a table and chairs in the back. And so there
    couldn't have been more than six people, because there were only four
    chairs." And drawing conclusions that are completely unjustified
    by reality.

    Alan: What an ethics dilemma,

    right?

    Arash: XR touches everything.

    Alan: It's funny how the average

    user of this technology would never even consider these types of
    things, and the fact that the CNN team considers these things, it
    really does kind of speak to the real power that they hold as
    journalists in holding themselves to a higher standard.

    Arash: Yeah, exactly right.

    Alan: I wish the whole XR

    industry was held to a higher standard.

    Arash: Yeah, absolutely. I think

    they set a good standard for what is information, and how is it
    presented, and what what sorts of assumptions do we project with
    that, what's implied in the things that we show. So yeah, it was
    cool. And that was really exciting for us as one of our early big
    projects, because it did the exact thing we wanted as founders of
    this company, which was to get a glimpse into how do big companies
    run. What are the exciting challenges that they have under the
    surface? What are the conversations that they're having? We wanted to
    learn -- kind of like Alan Peterson had -- how are trucks designed?
    How is news made? What are those conversations inside the conference
    room and what does all that look like? So for us, just kind of our
    personal curiosity as founders, this was super cool, to go into CNN
    and talk to all the people that are involved in making the news, and
    then present to them this incredible new way of showing their work.

    Alan: I guess one of the things

    the major a-ha moments for you and your team started to deal with
    these big companies, going from a gaming background, sitting in these
    board rooms with these big companies. What was something that was
    kind of totally foreign to you that revealed itself?

    Arash: Well, first of all -- in

    terms of the sales process and decisions -- it's a real challenge,
    because you've got so many different stakeholders at so many
    different levels. And not everybody is gonna be in the conference
    room with you. Not everybody's gonna be in the same conversation to
    give you the chance to convince them that that XR is the way they
    should go. Sometimes that happens outside and and people-- someone
    you've never met and never knew will torpedo the deal because they
    think, "Oh, my son had an Oculus Go and had a terrible
    experience. And this entire technology has no future." One of
    the things for us was just learning who the stakeholders are, and
    dealing with just large scale of an organization that's trying to
    avoid risk. And for us, learning how to -- in the same way that that
    Magic Leap does so well -- learning how to really evangelize this
    within an organization and deal with a lot of the objections was a
    big challenge. And the other piece of it is that a lot of companies
    at this point now have tried some form of XR. And many of the folks
    that we talked to are like, "Well, we tried that two or three
    years ago and it didn't work at all" or "We tried that with
    this other vendor and it was too expensive." They have some
    preconceptions about what this technology is capable of or how much
    it costs or how much work it is.

    Alan: And to be honest and frank

    with everybody, it was expensive and it did not work all that well
    three years ago.

    Arash: Absolutely. And now in

    this industry, we have to come back and say, well, it's different
    now. Will you please try us again? [laughs] And then every year it
    gets better.

    Alan: We all have better

    quality, lower prices right across the board. Just take 360 video,
    for example. Four years ago when we started doing this -- or five
    years ago, whatever it was -- was $10,000 a minute. Because you had
    to hand-stitch-- you had to, first of all, 3D print the camera.

    Arash: Yeah, exactly.

    Alan: And then you had to use

    software and manually stitch every single scene together. And this
    was a crazy tedious process. And then all of a sudden, these
    all-in-one 360 cameras popped out and you're like, "Oh, OK.
    Well, great. I don't have to spend a month stitching this."

    Arash: Yeah, handheld for $200.

    Yeah, exactly. These kinds of things are huge, huge game changers.

    Alan: I went from $10,000 a

    minute, to a $200 camera.

    Arash: So, yeah, I think a lot

    of it's "Come back and take a look at what it is now. It's a lot
    different from three years, we promise." And what helps a lot is
    also that I think overall in this industry, we've done a good job as
    a whole of recording a lot of really good case studies. There have
    been a lot of case studies where I would have expected this to be
    something that either the company or the developer would have kept
    super secret. And this does happen from time to time where they're
    like, "No, no. AR is our secret sauce. We don't want anybody
    hearing about what we did. We don't want anybody to copy us, we don't
    want our competitors to take any of this stuff."

    Alan: And that actually, from a

    business standpoint, makes a lot of sense.

    Arash: It does. But then by and

    large, a lot of companies are looking at this as "Well, let's
    look at the other side of the coin, which is that, hey, we can
    protect ourselves as a very innovative company." This is just
    now and we're already working on the future. So even if you did what
    we're doing now, you're still behind, which I think is great, because
    it gives the industry overall a look at what's working, what's not
    working, and what kinds of gains are people seeing. And then when we
    go to talk to a company that's, for example, a construction company,
    we can pull a lot of different use cases for where other companies
    have been super successful. And that helps to turn that tide, where
    companies that are super risk averse and really slow industries, they
    can say, 2Hey, wait, actually, a lot of companies like us are doing
    this already, and they're really happy with that. So maybe it is
    worth it for us to give it a try." And Alan, I've got to say,
    thanks for all the stuff you're doing to promote the industry. I
    think it's important to have a lot of conversations around this
    stuff. I think the more information that's out there, it keeps
    developers like us doing our best work, knowing about all the
    solutions that are out there to be leveraged and not reinventing the
    wheel ourselves when we need to do something, and then helps
    companies to see how much exciting stuff is happening in space.

    Alan: Wholeheartedly. And

    actually AR new business model with MetaVRse, it's a combination of
    us realizing that we can't possibly build everything for everybody,
    even though we're really focused on training and education and
    learning. We can't possibly build everything for everybody. So what
    we're building is a platform marketplace for learning technologies
    and content. So, for example, you guys have built this player on
    Magic Leap that you built with CNN. Well, maybe they want to resell
    that player as a generic player for other companies to leverage that
    technology. Well, there we go. We can now put that on the platform
    marketplace and make it available to every company, without having to
    pay the million dollars of development costs upfront. And this can be
    done for all sorts of content. We're really focusing on training and
    education and learning content, but having a platform marketplace or
    managed marketplace of this technology will lend itself nicely to
    scaling this technology. As you know, it's not easy to make, and
    companies like Extality are few and far between right now.

    Arash: Absolutely. I think

    standing on each other's shoulders is really the way to go to the
    next step.

    Alan: I think there's room for

    everybody. And if we all work together, then good luck if you're not
    part of the fold. [chuckles]

    Arash: Exactly, yeah. And I

    think we're going to see a lot of technologies raise up as the gold
    standard of doing certain things the right way. And I think the
    benefit of having people just take things that have been established
    as, "Hey, this is the good one, use this." and then build
    all the other stuff yourself. It's kind of like web technologies,
    right? Like there was a point where everybody was implementing their
    own authentication software. And now you don't do that anymore. You
    just use one that's established to be really good and you can make
    sure that it's already secure, because there's people that are
    dedicated to solving that problem. Versus you want to build an XR
    solution and you have to build every piece of it yourself. You may
    not be an expert in the security part of it, or the network layer
    part of it, or the graphics part of it. And we can lean on people
    that are, to really get to what's really driving value for the
    client.

    Alan: Yeah. I think the the

    billion dollar question is which technologies serve which industries
    best, and which products are best to deliver these results.

    Arash: Yeah, absolutely. And

    we've been all over in different industries and we're seeing pretty
    much every single industry demands something different of us. I think
    of ASICS. That was another client of ours where we 'were talking with
    them about building prototypes around some challenges that they had
    with essentially with their distribution model, where essentially
    you've got folks that are buying shoes to sell them. So a company
    like a Macy's or a large retailer in Europe. And one of the
    challenges for ASICS -- and pretty much every other shoe company --
    is that they have to come to these meetings with all the shoes for a
    particular season. And if you consider all the different color ways
    and all the different models, they've got a pile of shoes that are
    being shipped all around, everywhere around the world. And it starts
    to kind of give you this a-ha moment, if you're deep in the XR world
    of "Well, why can't we just digital twin everything? Why can't
    we build really, really beautiful, very realistic looking shoe
    models, and then put these in the hands of the sales and distribution
    reps, instead of having to actually send around giant crates of shoes
    or whatever."

    Alan: It's crazy. And the same

    with apparel. Apparel's a little bit more difficult, but shoes lends
    itself perfectly. We spent a lot of time modeling shoes. Shoes,
    watches, handbags, glasses, and apparel was kind of the--

    Arash: Exactly. The rigid goods.

    Alan: Yeah, rigid goods. And

    then things that are fashion forward, they're always changing, right.
    So when ASICS shoe -- although they kind all at the same eventually
    -- but the shoes of 2019 are going to be different than the shoes of
    2020.

    Arash: Exactly.

    Alan: And 'having to create

    physical models is kind of-- it's almost useless now. You don't need
    that. For how much money just in shipping that stuff.

    Arash: Right. It's crazy. And

    it's just like, you can take down all kinds of-- it's not just money,
    right? There's carbon that you're saving, and there's-- it does a lot
    of good for everybody. They can do the meetings more quickly, it's
    easier to show things, pulling it up is much faster. Like, oh, you
    want the blue, whatever GT-2000? We can show you the model really
    quickly, versus "Oh hold on, let me go dig this out of the crate
    of 300 shoes." And the thing that's interesting is we have
    companies like Audi, where they've they've taken the digital twin
    throughout the entire chain. It's used both in the design side, to
    the production side, to the retail -- or in their case, dealer --
    side. I see that for a lot of different industries. I mean, with shoe
    companies, it's the same thing, the same way that these shoes are
    being sold to retailers and they can figure out for their buyers and
    that kind of thing. You could imagine these same models showing up on
    websites to be used in the retail side, on the e-commerce side. Use
    it in a lot of different places. You can get a lot of value.

    Alan: A lot of times big

    companies, their marketing departments don't talk to their training
    departments. And one of the things that we realizing is that it's
    expensive to make a 3D model, but you can use it across multiple
    parts of the business. You can use the same 3D model to share with
    designers. You can share with buyers. You can use it on your website
    for marketing. You can use it for training. All of these things. Yet
    it's kind of take some time to re-educate businesses on the fact that
    this content can be reused and really creating this digital catalog
    of 3D content.

    Arash: Exactly. And it goes back

    to what you were saying earlier is where in some ways now helping
    businesses talk to their individual parts. And it's almost like a
    transformation challenge and we end up being agents in that way as
    well. It's more than just delivering XR solutions. It's "Hey,
    have you talked to your marketing department? Has the sales talked to
    marketing? Does -- like you said -- design talk to marketing? You
    guys are all using the same file here. You should have a system that
    shares everything." And then you go back to the question of,
    well, the decision maker for marketing is different from the decision
    maker for design. And you start to get into some of these interesting
    corporate challenges. And so, yeah, you asked originally how is this
    different from games? Yeah, that's a huge area where we just learn a
    lot about. "Wow, this is inefficient. And we're not just giving
    them efficiency by giving them XR tools, we're giving them efficiency
    by -- in some ways -- just changing communication inside the
    organization."

    Alan: Arash, you've done your

    work with ASICS. You've done work with CNN and Magic Leap. Is there
    any other products you want to discuss?

    Arash: Well, there's a project

    we did with a small refrigerator company -- well, I don't know if
    they're small, they're pretty big -- it's a refrigerator company
    that's doing refrigerators with solid-state electronics. And so
    normally refrigerators got compressors, and it's got all these like
    fans and stuff like that. These guys found a way to make a
    refrigerator essentially with no moving parts. So it's electronically
    cooled. And this was an interesting marketing project for us, because
    we had this refrigerator. And it's unique, because there's no space
    taken up by a compressor. You can fit it in a lot of different form
    factors. And this was interesting, because their marketing team just
    had this vision and they said, "Can we put this in people's
    homes? Can we put this in stores where they want to use this? Can
    they customize it on-site? Can we make it interactive?" And this
    was an exploration in just, "Let's say yes to everything."
    Let's make a really cool ARKit tool that you can download off of the
    app store, so that anybody -- no matter whether they're a sales
    person for the company, whether they're a customer, whether there's
    somebody who just likes refrigerators -- they can try this and then
    experiment with how do you work with this? Where can I put it? What
    does it look like when it opens? Can I change the skin to say "ice
    cream"? Can I change the skin to say "yogurt"? Can I
    change it to something completely different? And exploring taking
    models from CAD, rendering them realistically in ARKit in this case.
    Can be headsets, can be anything. And that started to become a model
    for us. Are there other companies that can really quickly give us
    their CAD models or give us something that we can photograph and
    create 3D models through photogrammetry? And so that's kind of been
    our focus in recent months, is looking at companies where they're
    just ready to go. There was a bike company, same thing. They have a
    really innovative bike platform, it's an electric pedal combo bike.
    And they just wanted to show like, "Hey, this is a really cool
    new carbon fiber bike, and you can get it in a lot of different
    colors." And creating these just really easy -- kind of goes
    back to the thing we said with shoes, versus in handbags -- can we
    create a really easy model, where people can navigate it to their
    heart's content, where with photos you're like, oh, I don't know what
    that part looks like. I can't see that angle. But with a 3D model,
    and especially if it's on the web or in an app, you can just rotate
    to whatever angle you want to and peek at the smallest, most minute
    detail and get whatever answer you're looking for. Kind of see it
    however you want to see it. So that's that's been interesting. And
    looking at how do we get that data? Does it come from CAD? How do we
    get materials? That's been an area we've been really focusing. But
    then the other really, really big area for us has been construction.
    We've been working with Unity's Reflect team. And for people that
    haven't heard of Reflect, it's basically a tool which allows
    construction companies and architects who are working in a tool
    called Revit -- which is kind of one of the industry standard .BIM
    building information modeling tools -- to export those .BIM data
    files, which are usually huge and contain millions and millions of
    triangles, transfer those onto a mobile device so that you can
    actually look at a model while you're on the job site or somewhere
    else in an office in a meeting. And it creates a two way
    relationship. So if you were to change the file in Revit, it
    instantly updates in mobile. And we're looking at a lot of challenges
    in construction and other large manufacturing organizations, where
    there's just a lot of data, a lot of 3D data and getting that on to
    mobile devices, onto mobile VR/AR devices. It's just a horsepower
    challenge. And by being able to do these operations off-line, over
    the Internet, we're talking a lot about remote rendering. We saw some
    cool stuff coming out of Microsoft with that, the Apollo demo and
    things like that. I think that sort of starts to paint the future of
    how we can use this in larger scale industrial operations. And so
    what we're working on, building tools around Reflect, that's been a
    really big area of internal development for us with some of our large
    construction clients. Visualizing things like build sites, build
    process, how does the 13th floor come in, and what are the crane
    swing operations that we have to do. And helping them really just
    communicate better, being able to put the building right in the
    middle of the conference room table and have everybody, all the
    stakeholders looking at it in one place. With a model that's that
    heavy, normally that's just not possibly, you've got to look at it on
    a high-performance PC and you can't do it on an iPad or a Magic Leap
    for something like that. So we're seeing a lot of big technologies
    that are starting to push us in that direction, which is really cool,
    making that possible.

    Alan: I'm enamored by these

    programs that are taking CAD files and BIM models and transferring
    them into 3D for AR and stuff really simply, because that was a big
    problem. How do you take these CAD models that you have that you've
    spent millions of dollars on and convert them to 3D? It's in 3D, just
    not the right format. So there's like a-- and then having this mess
    of 3D versions and 3D model formats, you've got FBX, and glTF, and
    OBJ, and all these different-- and then you've got Apple with USDZ,
    and there's really a lack of standardization across industries for
    this. And I think we're starting to see these things work themselves
    out of it, which is great.

    Arash: Absolutely. It reminds me

    of the early days of images, too. We got BMPs and PCXs and every kind
    of-- there were so many different formats, if you remember early days
    of graphic design on the PC. And those all sort of consolidated. And
    really, I think it ends up being, what are the major formats that
    people support? What are the major formats that are becoming
    interchange formats for different operations? And we saw it kind of
    in the games industry, where things all kind of centered around FBX
    and COLLADA. And I think we're gonna see that sort of with USDZ and
    glTF. I think there's just a lot of tools like SketchFab and things
    like that, where your experience with certain formats is lot easier
    and we see people gravitating toward those formats.

    Alan: It's interesting you say

    that, because everybody in the world was moving towards glTF, until
    Apple said, "Oh yeah, we got our own one!"

    Arash: "Oh, by the way."

    [laughs]

    Alan: [laughs] "By the way,

    we're Apple. So you can deal." But there's a pretty cool program
    called Meshmorph -- meshmorph.io -- that will convert file formats
    into USDZ.

    Arash: Perfect. Yeah. I'm

    speaking with Apple, and they're investing a lot in making that
    process a lot easier. And again, those types of investments are what
    will make people gravitate toward USDZ over something like glTF.
    Yeah, let the best man win.

    Alan: The biggest problem I see

    is-- I've got a 3D graphic designer, who built just an AR business
    card for us. Sent it over to another group that were posting the same
    graphic on our website in 3D. And back and forth, back and forth. And
    like oh, the textures weren't right and all of the bumpmap wasn't
    right. It's like, God, this is-- there needs to be a standard, how
    hard is it to send a bloody file? But it's apparently hard.

    Arash: Yeah. I mean, the

    challenge that I think we see is as we work more and more with this
    is, that there's so much more data to 3D than there was with images.
    In an image it's like we got pixels and what color's the pixel. But
    here it's what are your materials, what is the shininess, what is
    this thing, which map is being allocated to which channel, and where
    you're dealing with these photorealistic physically based rendering
    models? And it's important to get it all right, because different
    renderers are going to use the bumpmap a different way. They're gonna
    expect a normal map. I mean, there's all these kind of technical
    gotchas that you can get wrong if you're importing from one format to
    another. And again, I think that just underscores the importance of
    this stuff starting to get more standardized. But at the moment, I
    think I agree with you. It's a big mess and different exporters are
    exporting differently. And you'll see two different ways the same
    file formats are being interpreted. I mean, we had challenges with
    FBX where it's just like, "Hey, you can store animation
    different ways, or you can store the materials different ways."
    And people aren't getting what they expect and leads to clients
    getting on the phone and being angry.

    Alan: Yeah. And when you're

    dealing especially with retail, when you drop something in AR, it
    needs to look like -- or even on 3D -- it needs to look like the
    product that I'm going to buy. People are used to seeing a photo of
    the product I'm going to buy. If it's skewed in some way or a
    different colour or different shape or whatever, then you've failed
    in doing that. And then, of course, the challenge becomes, "OK,
    we made it look photo-real. It looks great. How do we make it
    smaller, so that it works on all devices fast?"

    Arash: Right. Exactly, yeah.

    Still lot of model data is quite large. We're getting better with
    compression. But yeah, you're right.

    Alan: Oh, yeah. It's getting way

    better. And then you've got 5G coming. People are like "Oh, 5G
    will unlock this!" I'm like "Yes, but if you can get
    compression great now, when 5G comes along, then you can just do
    more." It still doesn't shortcut the fact you need to get
    compression down.

    Arash: Yeah, things need to be

    efficient.

    Alan: They do. Well, Arash, what

    problem in the world do you want to see solved using XR technologies?

    Arash: Oh, that's a really good

    one. So me personally, I'm a pilot and what I'm looking forward to is
    being able to wear a headset while I'm flying an airplane, and have
    everything just laid out in front of me out in the open, kind of like
    I think F-22 pilots have this with a $20 million headset. I want to
    see every pilot being able to see other planes, all their runways,
    all these kinds of things. And that being extended later to drivers
    and that kind of thing. I think there's so much opportunity to give
    lots of information to people operating vehicles.

    Alan: Interesting you said that,

    I was just at the I/ITSEC conference last week--

    Arash: Oh, yeah. How was it?

    Alan: --the

    Interdisciplinary/Industry Simulation and Training Conference[sic].
    It was amazing. I got to fly a helicopter. I don't know which one,
    but it had the articulating propellers. So it actually can fly like a
    plane as well.

    Arash: Oh, right, yeah.

    Alan: But I was in a giant

    multi-million dollar simulator flying helicopter at one point. I
    was-- I shot things. There was lots of things to shoot. I also got to
    drive a tank, that was pretty cool. Out of about 400 and-- I'd say it
    was about 450 booths there, over 100 of them had VR at them.

    Arash: Yeah.

    Alan: The fact that there's that

    percentage, at least a little bit over a third of every of these
    booths had VR in them. Or AR. I mean, I tried a gun range using the
    Magic Leap where you're shooting a gun. But when you're looking
    through the Magic Leap you see a full range in front of you, with
    grass and trees. It was like, whoa. And it was a one to one exactly
    to the gun. So when I looked down the barrel of the gun and shot it,
    it was very accurate.

    Arash: Yeah. That's no accident

    that even early days, like in the 90s, 80s, we had heads-up displays
    on airplanes, like it's just-- it's huge, huge game changer to have
    information exactly where you're looking.

    Alan: You've got kind of

    pioneers in the space. People that have been working this since the
    60s, 70s in these heads-up displays. And you don't really know what
    you're missing until you have that data in front of your face.

    Arash: Yeah.

    Alan: Do you know Tom Furness?

    Arash: I don't think I do.

    Alan: Tom Furness was one of

    those-- he's known as the grandfather of AR, because he was building
    heads-up displays -- or HUDs -- for the US Navy and Air Force back in
    the 60s, 70s.

    Arash: That's amazing.

    Alan: He's never left the

    industry. So he's actually one of our mentors with XR Ignite. Yeah,
    it's pretty cool.

    Arash: Good stuff.

    32 min
  • Exploring XR Collaboration Statistics in Real-Time, with National Research Group’s Lauren Xandra

    There’s not a lot of good coming out of the current situation affecting the globe, but if there is an upside, it’s the rare opportunity to learn from something this unprecedented. Through her work with National Research Group, today’s guest Lauren Xandra has been able to study newly-emerging work-from-home behaviours, and how that applies to XR adoption.

    Alan: Hey everyone, I'm Alan

    Smithson from the XR for Business Podcast, and today we're speaking
    with Lauren Xandra, vice president of strategy and innovation at
    National Research Group, a leading global insights and strategy firm,
    about the original research on XR, AR, and collaborations in a time
    of Covid-19. All that more coming up next on the XR for Business
    Podcast.

    Lauren, welcome to the show.

    Lauren: Thank you so much for

    having me, Alan.

    Alan: It's my absolute pleasure.

    I'm in day 22 of my quarantine. Nothing really has changed in my
    life, because I work from home anyway. So how are you doing?

    Lauren: I'm doing well. It's a

    healthy adjustment for me, but it's rather timely that we're here
    today, because I'm excited to share new research, looking at how
    different demographics are adapting to our work-from-home reality,
    and to share some exciting findings that we see in the space with
    new, broader, addressable audiences for virtual solutions in light of
    all of this.

    Alan: Well, the timing on this couldn't be any better. Some dedicated, hard-working people in this industry have banded together this week to pull together an XR Collaboration guide, talking about everything from security to device management to vendor selection to feature lists, and really put a lot of effort into creating a tool online that will give people the opportunity to figure out how these tools can be used for their business or school or education, and which one is the right one for them at the time, for the need they have. And so we're really excited about that, and the information on that will come out on XRCollaboration.com. So, Lauren, please tell me, what is the basis of this study?

    Lauren: Sure. So, when suddenly

    millions of people -- seemingly overnight -- became remote workers,
    we're faced with these huge questions about productivity, state of
    mind, social and cultural impact of the situation, and the lasting
    impact, too. We really set out to understand how people are adapting,
    what pain points are felt across work-from-home, and in doing that,
    we can better understand and address whitespace to solve for these
    pain points, to ease our adjustment to this new -- and bizarre --
    reality.

    I'd love to walk you through some of

    the key findings, perhaps starting with generational differences in
    adapting, and then perhaps looking at more industry-specific
    challenges and opportunities.

    Alan: Sounds wonderful. Let's

    let's dig in.

    Lauren: So we see that the

    pre-Covid-19 context really impacts different generations' concerns
    and expectations for the future. Probably one of the most
    counterintuitive insights is that the youngest in the workforce is
    actually the least well-equipped for work-from-home. The digital
    reliance of this generation already makes them victims of social
    distance pre-Covid-19, and for them, the effects of isolation are
    amplified. The impact of Covid-19 on culture is really their
    front-of-mind concern. And here we're thinking about culture in terms
    of how we connect, how we collaborate. Gen Z has really struggled to
    disconnect from technology. They're citing irritation from too much
    screentime, bad work/life balance, well ahead of other groups. And
    this impacts mental health, with already about half of Gen Z
    professionals saying that staying home all day makes them depressed.

    And further still, they're lacking the

    necessary technical and environmental setup to be as productive. We
    see this as a really key audience to benefit from potential that AR
    and VR presents. Firstly, AR and VR has a big base of Gen Z users,
    with about 40 percent identifying as users of both AR and VR. They're
    just waiting for the experiences. I would love to learn more about
    what's on that collaboration list, because there's so many
    possibilities for how AR and VR can viably engage employees, maintain
    focus, enhance creativity and connection -- particularly for this
    young group that's suffering from missed face-time and interaction
    mentorship.

    Alan: Being able to regain some

    normalcy in places that are familiar; I think there's going to be a
    big need for companies to recreate their traditional workspaces
    digitally. Like, when you go to office, you feel like you're in the
    office. When you're at your home, it's not the same. Everybody has
    become their own I.T. department, which is terrifying.

    Lauren: Yes, I'm sure we've all

    seen amusing examples of colleagues' children running towards the
    screen and being blurred out into the Zoom background and the rest.
    Some really interesting virtual solutions have emerged, and will
    continue to. A personal favorite is Spaces. I love the concept of
    using virtual cameras as a webcam, so you can make use of equipment
    that's not actually available in your real-world environment, like a
    VR whiteboard, and communicate progress and integrate into existing
    apps like Zoom and Google Hangouts. I think we'll really continue to
    see exciting solutions like that present themselves.

    Alan: It's an amazing story

    actually. I know the guys at Spaces. They went from a company 100
    percent focused on location-based entertainment -- and of course, all
    location-based entertainment is shut down -- and they pivoted in a
    very short amount of time to create a tool that allows you to be in
    VR and see your Zoom calls, and the Zoom call people can see you in
    VR. And they pulled that off in weeks, from pivoting from
    location-based arcade games to a business tool that we can use. It's
    really incredible. That team is very talented.

    Lauren: Yeah, that's fascinating

    to hear about that fast pivot. And there's certainly -- beyond this
    younger generation's collaboration tools -- so many other
    demographics and industries [that] stand to benefit from other
    solutions. When we look at more mature professionals like X-ers
    who've already lived through layoffs and significant hardship with
    the recession, they're much more concerned about the economy by a
    great margin, while ahead of concerns about culture, social, or their
    personal life, their own professional development. But there's some
    silver lining too; they're more pro-work-from-home, because they're
    more self-sufficient. They generally don't mind staying inside for
    long periods of time, and might enjoy the benefits of freedom and
    convenience.

    Alan: Everybody, as they get

    older and have kids and stuff, they move out to the suburbs, so they
    got a little bit of extra space. Living in a 500 square foot condo is
    pretty awesome. The whole city is your living room. But when your
    living room is your living room, people are going to go nuts.

    Lauren: Yes. And you're right. I

    mean, this group has more space, generally-speaking, and is more
    likely to be parents, of course. But parents generally, we see that
    the top paying point is distraction. And that makes sense.

    Alan: I keep muting myself

    because my kids are banging around upstairs, I'm not sure what's
    happening. But I can tell you from a distraction standpoint -- my
    kids are older, so I don't have to deal with them all the time.
    They've got things that we've given them to do -- but I can tell you,
    if you had a two-year-old running around? There's no way you could be
    productive at home. So the new normal is a culture shock to people.

    Lauren: Absolutely. Yeah. And

    the need for e-learning to improve is sudden and significant.
    Families adjusting to school closures, through to medical
    professionals who are continuing to practice important procedures
    from remote locations.

    Alan: The amount of use cases

    for this technology are endless, and they're not difficult to create
    these scenarios, and the tools for creating rich, 3D scenarios are
    starting to become easier and easier. So I think we're going to see
    both the adoption of the headsets much quicker than we would have
    anticipated, but also the ability to create experiences and create
    from within experiences. I think this is going to be a huge
    opportunity for the whole industry to shine.

    Why don't we talk about some real

    numbers? Can you maybe walk us through the numbers of this?

    Lauren: Sure. So, this was

    fielded March 14th to 15th, so we're a few weeks deep. And we're
    planning a second wave as well in a couple of weeks' time.

    We surveyed over 1,000 people here in

    the US, with criteria being that they must be either currently work
    from home or have worked from home in the past, and it's across 18 to
    54 years old. So it's a professional audience here in the US, a
    reasonably-sized base, 1,010 -- a nationally-representative sample.
    Any other questions around size? Around the audience for this?

    Alan: Well, audience size, you

    nailed. What did you ask them, and what were the results?

    In addition to better understanding

    what areas were front-of-mind concerns -- which we touched on a
    little bit here [on the podcast] -- how certain audiences are more
    concerned about the economy or the market, particularly more mature
    audiences.

    Alan: That's interesting that

    the age groups have different concerns at the same time. Where does
    health concerns fall? If you to compare health concerns vs. economic
    concerns?

    Lauren: We actually didn't

    include health concerns, thinking that health concerns would be
    really front-of-mind for all audiences. It would be interesting to
    better understand that, but we kind of thought about it across the
    framework of more kind of "Macro vs Micro."

    Alan: Maybe you could include

    that in Wave 2: rank these concerns. Because I think the health care
    one. My prediction... I mean, I don't know. But my hypothesis would
    be that younger people would rank health less, because they're not
    fearing of it. As you get older, the fear goes up.

    Lauren: Absolutely.

    Alan: The ranking gets

    reprioritized.

    Lauren: Absolutely. Yeah. I

    mean, what we did look at was in terms of more how health care
    professionals are adapting to this remote work norm, and how their
    experience is similar and different to other industries going through
    this. And we actually see that their concerns are pretty closely
    aligned with those of education professionals. They're struggling
    both with distraction, but also missed access to the devices that
    they have at work, which makes a ton of sense. And of course, part of
    that is e-learning, which we are just speaking about. And another is,
    of course, turning to VR in 3D formats to better approximate
    real-life interactions. Because crossing the chasm to virtual
    requires a major practical shift in how business is done for this
    group.

    Alan: How are the opinions

    different in healthcare and education, as opposed to the business
    community? Was there a breakdown of that at all?

    Lauren: Yes. So we see that the

    types of experiences that they are gravitating towards are much
    higher for learning experiences, and then a range that makes sense in
    approximating real-life interactions. So, things like traveling
    anywhere in the world in VR, visiting remote sites as if you were
    there; 3D object identification and training to, of course, improve
    workflow processes and engage with 3D objects virtually in real time;
    then thirdly, holographic communication or communicating in
    real-time, to get close to real social exchange.

    Alan: So one could almost say

    "XR Collaboration."

    Lauren: Yes, XR collaboration is

    certainly broad brush strokes to capture their needs. These
    professionals too are much more secure about their own career and
    professional development. Of course, this is a real time of need for
    them, but that's not true at all across the board. When we look at
    retail, hospitality professionals, other areas who rely upon
    in-person business but who are suffering more financially, already we
    see two and three have heightened concerns about their own career and
    professional development. And of course, I think everyone's come
    across research by Harvard Business School that the average
    small/medium businesses have 27 days of cash. Right? And Main Street
    has an average of 16 days, making the speed of adjustment to new
    norms really urgent. And another finding through this is that, when
    we look globally -- and this is based on some other research we've
    done tracking Covid-19 -- the impact globally, we see that the
    financial concerns of employees across geographies sort of correlate
    with where they sit in the Covid-19 curve cycle. So here in the US,
    57 percent of employees believe the current situation is getting
    worse economically, whereas in France and Spain, they've potentially
    reached the bottom and may foreshadow the trajectory for the US and
    others. They're starting to see and feel a financial return -- it
    maybe early to say -- but we kind of see these early economic
    indicators that are quite interesting, too.

    Alan: Incredible. The whole

    world is adjusting to a new normal. Now, how we adjust and how we
    adapt will dictate the future of the next exponential age of
    humanity. And I really kind of think this -- philosophically -- at
    least we're giving the earth a break before the real pounding begins.
    And what usually comes out of recessions and the hard times and every
    time we've gone through something like this global challenge -- never
    before have we, first of all, had a combined challenge against
    humanity -- so hopefully this brings us together rather than pushes
    apart. I think there's an opportunity here for the whole world to
    band together, not only to get out of this and be successful; Peter
    Diamandis predicts more wealth will be created in the next 10 years
    than all of previous human history. And I think this is more so than
    ever, because what this will do is give us an opportunity to realize
    what's important: what is important to our families? What do we
    really need? What is an essential service? And also, we're going to
    find efficiencies where there were none. Every time you go through in
    a recession or depression, businesses find new opportunities for
    efficiency. And I think A.I. automation, robotics, virtual/augmented
    reality -- all of these things coming together at one time, and you
    have everybody pushing on these levers, and the whole thing grinds to
    a halt. But people are still working on these technologies and it's
    gonna get amplified. So when the economy comes back online, whenever
    it is -- maybe it's a month, maybe three. Who knows? -- but when it
    starts to come online again, I believe this will be the beginning of
    exponential growth as we know it.

    Lauren: Absolutely. And I love

    how you emphasize how this growth really starts from a place of
    individual growth, from a place of reprioritization around what
    matters. Across broad research that I've done in the AR and VR space,
    one of the really unique benefits of this technology is around
    self-improvement. People see it as a medium for learning, for
    discovery, for becoming better versions of themselves. And in a small
    sense, we see, of course, a trend towards fitness in these times.
    Even in recent weeks, usage of Beat Saber for fitness is seriously
    ramped up. But in a bigger, broader sense, how we communicate -- how
    we collaborate -- is quickly changing. And I hope that this medium,
    and this time period for reflection, will enable more meaningful,
    more thoughtful interactions.

    Alan: If we can't see each other

    in person, the next best thing is being in a virtual space. And I
    thank you, Lauren. Is there anything else you want to share about the
    studies you're working on?

    Lauren: Sure. So the I'm

    thinking about how develop these solutions that can bring us closer
    together, now that we're further apart. I'm also investigating, in
    this next wave of research, looking at the changing services market
    in light of Covid-19. So how our share of time and consumption of
    content and services are changing, and even our tastes; the tone of
    content that we're interested in in these moments. What habits are
    likely to stick? These are all areas that will be coming out in a
    couple of weeks' time. And also later in the year -- this one is
    relevant for the XR community -- we'll be producing a study on the
    future of devices; what the new state of normal is, so to speak. Is
    it dual screen? Post-screen? One device for all? So we're looking at
    that across entertainment, productivity, all kinds of contexts, for
    how we live and work.

    Alan: It's wonderful, being in

    this XR industry at this time, seeing how everybody's pulling
    together. And most people that I talk to -- even on this podcast, and
    most people that I talked to in my daily business -- everybody's
    concern is our health care first, and then our children's education,
    and then the economy. It seems to be in that order in this industry.
    Anyway, it'll be interesting to take a survey of just people working
    in this industry, and see how the mindset is different, because I
    think everybody I've spoken to in the last two weeks sees this as the
    beginning of a new way of working that XR are fundamentally drives.
    So I'm very excited about it.

    Lauren: Absolutely happy to

    collaborate with you on that.

    Alan: And we're going to start

    to be doing LinkedIn Live sessions. So, conversations like this, and
    then we'll be posting them as part of the podcast, so that we can
    just bring more information across a broader range of people. Some
    people don't listen to podcasts. Maybe they're sitting looking at
    LinkedIn. Maybe they're on Twitter. So we're going to try to spread
    this out as many places as possible.

    Lauren, I always ask one last question;

    I think it's more prudent than ever. What problems in the world do
    you want to see solved using XR Technologies?

    Alan: I think I would love to

    see XR solve for a more connected community globally. I think it's
    really powerful to see how accessible XR can make certain experiences
    for people who wouldn't otherwise be able to enjoy those experiences.
    Whether that's travelling anywhere, speaking to anyone without
    language barriers, the possibilities to connect with people in
    unexpected ways we wouldn't otherwise. That's a future I'm excited
    for. And I think it is very timely right now, in that creators have
    to think about developing for the long haul, in terms of virtual
    communication. And the longer this situation lasts, the real benefit
    is that the more pervasive these effective solutions for connecting
    will be.

    Alan: Well said, Lauren, thank

    you so much. Thank you, everybody, for listening. This has been the
    XR for Business Podcast with your host Alan Smithson. And don't
    forget to subscribe, so that you don't miss any future episodes.
    Thank you again, Lauren. It's been wonderful.

    Lauren: Thank you so much.

    21 min
  • Augmenting Enterprise Productivity, with Altoura’s Jamie Fleming and Bharat Ahluwalia

    One of the best use cases for XR technologies is training and enterprise solutions, as long-time listeners know. Today’s guests, Jamie Fleming and Bharat Ahluwalia from Altoura, pop by to explain how they’re optimizing workplace productivity.

    Alan: Hey, everyone. Alan

    Smithson here, with the XR for Business podcast. Today, we're
    speaking with Bharat Ahluwalia and Jamie Fleming from Altoura about
    augmenting enterprise productivity for frontline workers using
    spatial technologies. All that and more coming up next on the XR for
    Business podcast. Jamie and Bharat, welcome to the show, my friends.

    Jamie: Hey, thanks, Alan. Super

    happy to be here.

    Alan: I'm really excited. So,

    Bharat, I know offline you were just-- you were mentioning that you
    were part of the original Hololens team. So I know how you got
    started in this. Jamie, you've been building experiences for 15 years
    in technology. So maybe just give us a bit of your background and
    we'll go from there.

    Jamie: Sure, yeah. For starters,

    this is Jamie. So I actually started out in architecture. I have a
    masters of architecture, and worked as an architect for a number of
    years. And that's really where I was given an introduction to 3D
    modeling and building out immersive experiences in the day-to-day
    practice of design and got the spark of an idea of, hey, I could
    create a company where we really just focused on creating
    experiences, and making them more and more interactive that helped --
    in the early years, particularly -- help designers understand what
    their designs were. And over the years, that has just become more and
    more sophisticated and we've gotten deeper, deeper into the software
    side of things. Now we really have a lot of interesting ways that we
    can leverage these digital twins to help augment enterprise
    productivity.

    Bharat: And this is Bharat here.

    I got started in this space when I joined the Hololens team. It was
    an interesting experience. The project was top secret, as you know,
    and I wasn't even told what I would be working on until I agreed to
    sign the offer letter. But I knew a couple of key people then said,
    "OK, these guys get involved and it will be an awesome product."
    And it was. The first time I saw the vision video and the vision of
    the device, it was, wow. That's what I want to work on. So I worked
    on it for a few years, shipped the V1 of the device and was so in
    love with this technology, that I decided to leave the company and
    build experiences on top of the device and the platform that I was
    part of. I did a small startup that led to being acquired by Studio
    216, which is now named Altoura. And here I am. And I have been
    building Hololens and VR experiences since then.

    Alan: It really comes down to

    the experiences, because the devices themselves-- the Hololens 2 is a
    magical piece of kit, but if nothing's on it, it's kind of useless.
    So you decided to go into the experiences. And what are some of the
    things that you've been building? Because I've seen some of them, but
    I'll let you guys speak to them. What are some of the experiences
    that you've been building?

    Jamie: Yeah, we're really-- the

    one that we're super excited about right now is the work that we've
    been doing with Qantas Airlines. So what we've done with Qantas is
    we've taken the 737-800 cockpit and we've re-created it as a digital
    twin. And then using Hololens 2, we allow you to interact with that
    cockpit as if you were there, so you could be sitting in your living
    room or a classroom or anywhere really in the world, networked
    together and interacting with functions inside the cockpit, the same
    way that you would do in a physical simulator. So as we know with
    Hololens 2, it's tracking all 25+ points on each of your hands. You
    just-- there's very little training in terms of understanding a new
    way to interact with a new UI. You can really naturally just grab
    levers and buttons as if you were really there.

    Alan: I was watching the video as you were talking about it now. You're in a cockpit of this plane where you're actually interacting with it. You're with another person. But in the real world, there's just two people sitting on chairs in a room. It's crazy.

    Jamie: Yeah. You know, it's

    important as cool as Hololens 2, is -- and it does definitely open up
    opportunities in terms of training, because it is so natural -- but
    we understand that people need to have different devices to support
    them at different times, for what they're trying to achieve. So we
    see a lot of people also using tablets or phones or PCs to do the
    same training and be networked across devices. So it's super
    important to match the right kind of device application device type
    to the use case.

    Alan: Also, you can't expect

    everybody in a corporation to use Hololens. It's just not the perfect
    device for everybody until-- it will be, maybe, in a couple of years.
    But I think you're absolutely right being able to do that, and maybe
    the future of that is in Web.

    Bharat: It definitely feels like

    it. What we have seen is there are two types of usage scenarios.
    There's a usage scenario which is more like a console or gaming
    console, in the sense you decide that I'm going to be in this
    experience for an hour or 30 minutes. And in that case, for you to be
    inside a Hololens actually makes a lot more sense, like if you're
    going through that flight simulator training. And then the other use
    case which we have seeing -- say, in real estate vertical -- is much
    more casual. When people come in, they pick up a device, they explore
    the space, they walk around, they change some options, and that gives
    them more confidence in their purchase decision. But that's like a
    five minute experience. Having to wear a Hololens for that experience
    may not be worth it. So that's where we've kind of seen how the
    devices have worked out. If you know you're going to be in that
    experience for a large amount of time, then that initial investment
    of wearing the device and getting acquainted by it gives you back the
    ROI you need. Whereas if you're going to come there casually move
    around for five minutes and change a few options -- very basic
    interactions -- then the tablet device works out much better.

    Alan: And that's something that

    you guys have worked on with one of the telco companies.

    Jamie: Right. With Sprint, I'll

    just give you kind of a quick introduction to how that project came
    into being, which was was really interesting. We do a lot of work
    with Microsoft in helping serve their customers across a variety of
    use cases. And we were in Atlanta talking to a group of Sprint-- the
    different leaders within Sprint. They were spread between the
    training, the retail layout team, and the network team. And we went
    in quickly with our solution. We were able to build a digital twin of
    a Sprint store. They didn't have the CAD drawings to support that
    exact store. So we just-- through a process called photogrammetry, we
    quickly were able to recreate the digital twin of that retail store,
    and bring it and show the team.

    Alan: And I know you-- one of the previous podcast guests, Jonathan Moss, he was spearheading that project. Is that correct?

    Jamie: Yes, exactly. And

    Jonathan was very instrumental in helping us understand the
    connections of really how to leverage that digital twin to fit
    seamlessly within an L&D team's typical process of how they would
    go through and engage their employees on doing things like how to
    interact with different customer personas. Jonathan was really
    visionary in leading and helping us understand that better.

    Bharat: We are not training

    experts, right? We build the tech, but having Jonathan and his team
    on the other side really helped us complete that picture, so we could
    see what they were wanting to achieve, and the process that they
    followed. And that helped us mature our platform in giving the
    toolsets to L&D teams to use. And we've seen that very
    successfully adopted by some of our later clients, where they look at
    that and said, wow, that's super interesting. In fact, what we want
    to do is change our content, just a little bit, to use the same
    tools. So that's been great learning as well.

    Alan: You guys re-created the

    store using photogrammetry. How did that translate to other stores?
    Like if it's a different layout type of thing, or was it just a
    generic one-store-fits-all kind of thing?

    Bharat: So it was just one

    store. And I believe they have roughly 10 different store layouts
    that each of their store follows. So we had to cover one of them. So
    it was a great learning ground for them to say, "Okay, how does
    this play for different products in this particular layout?" The
    important thing over there is to create the content in such a way
    that their team can change it as needed. So they typically will not
    change the geometry of the store, but they do want to change the
    posters inside it, of what's the latest--

    Jamie: Product.

    Bharat: --offering or product

    that they're offering through it. So that, their team can do very
    simply using existing 2D workflows of Photoshop, etc. So that was
    important for them, to be able to make those changes without having
    to come back to us, once they bought the solution.

    Alan: So that they could easily

    just transfer out some of the things. Did you have to build something
    for that, or how did that work? How were they able to edit it so
    easily?

    Bharat: We built an auditing

    system that helped them edit it easily.

    Alan: Now is this multi-device,

    I guess -- or do I say agnostic -- can you pull this up on any
    smartphone, or can you bring it up in VR? What are the limitations of
    it?

    Bharat: It's completely device

    agnostic. As long as you have a recently-modern device -- iPhone,
    Android phones, tablets, laptops -- and when it's AR enabled device
    it works in an AR mode, so you can place it around your physical
    environment. If you're not on an AR device -- like a laptop -- then
    it behaves more like a virtual environment, so you can just move
    using keyboard and arrow keys or mouse.

    Alan: I guess what was the

    feedback from the real people that matter, the employees? Did people
    like this? Was this something that-- in both, in the Quantas example
    of XR training and the retail Sprint training. What was the feedback
    with people? What do people think?

    Jamie: That's part of the joy,

    and why it's fun coming to work every day, because you are creating
    and giving people experiences that really do spark a lot of joy. And
    it takes maybe a session that you weren't as quite excited about
    watching some tutorial videos or working through a PDF. Suddenly
    you're in that environment, you are interacting with that
    environment, and there are definitely a lot of oohs and ahhs. And
    from the trainer's perspective, having a captive audience that is in
    this virtual environment, you know they're not spaced out checking
    their cell phones or not paying attention. You can see exactly what
    they're doing. They're engaging with one another. And it really is
    transforming. We're seeing in feedback that we're getting more
    customers, it really is transforming the way that people are engaging
    with this learning content.

    Alan: Two users as well? Could

    you have multiple people using the same experience at once?

    Jamie: Yes, absolutely. So you

    can have multiple people in that same environment. Those people can
    be either physically located in the same area, or they could be
    totally separated across geographies. They can be on different device
    types. So some people could be in a headset, others could be on a
    phone. Maybe the instructor's on a tablet or a computer. So, yes, we
    really accommodate people knowing that the value-- a lot of this,
    Alan, comes back to what are-- what problems are you solving with
    immersive training? Why is immersive training interesting, or should
    enterprises be thinking about it? And it comes down to, there are
    several different ways to measure the ROI of these applications. And
    one is, does this save me any cost as an enterprise? And so we know
    that a lot of cost and time and energy is wrapped up in people
    traveling to venues, to specific physical locations, to receive the
    training. And so the importance of having-- giving people the ability
    to stay where they are, but still be networked together in a single
    training session, that is critical, I think, for immersive training,
    too; to show out and prove out that ROI. I think there are a handful
    of different data points of ROI, in different problems that you're
    trying to solve. But reduction in travel is certainly a huge one.

    Alan: I don't know the exact

    number -- because the podcast was recorded a year ago -- but I know
    that that Sprint one from Jonathan, it saved them millions in travel
    costs.

    Jamie: Right. The team-- that

    travel team, I believe the stat was that they were on the road about
    80 percent of their time. The trainers, they're just constantly on
    the road. And being able to stay centralized but network in and train
    a session of people anywhere at any time, without even leaving your
    home. That was transformational.

    Alan: So you guys have built

    some pretty amazing experiences. You've got experience in kind of
    building the hardware platform. You've built experiences. What's
    next? What do you guys see as the ultimate potential of this? Where
    does this go from here?

    Bharat: So just the technology

    as a whole is still evolving, as I'm sure multiple guests have said
    this. We do expect augmented reality to be as pervasive as mobile is.
    It may take years. At Oculus, they said 10 years. I personally
    believe that should be less. I don't know who said the same, I think
    Bill Gates said this, "You will always overestimate what you do
    in one year, and underestimate what you can do in ten." So this
    definitely feels like-- at some point the glasses will be thin enough
    and small enough that you can wear them, and when the battery life
    will last the whole day. When that happens, you would see a switch to
    being a consumer phenomena, just like mobile devices was. But
    currently it is definitely an enterprise-first technology. It is
    similar to how PCs came about. They first got introduced into the
    workspace, and then over time got into everybody's home. Unlike
    mobile devices, which kind of did the other way around, people bought
    them for their own use and then kind of started putting your own
    devices into the workspace stream. So we do think this is a wave
    that's coming, and it's going to be used across all kinds of
    industries. But in the next two to three years, enterprises is where
    it is. And we've chosen this very deliberately that it's what the
    frontline workers where your environment influences your
    productivity. That's where this technology is so applicable. And
    that's what we are seeing. We are seeing it in training, whether it's
    flight simulator training, because, of course, your productivity
    depends on the flight deck you are in. Or in retail or training
    because you're selling in the store. So the more training you can
    have in the store, the better your attention is. We're also seeing
    you can real estate sales and marketing, it is so much easier to sell
    a house or lease out commercial office spaces if the person buying or
    leasing can actually see the space, can design in it. So definitely
    enterprises is where the focus will be. You've seen Magic Leap pivot
    to enterprises. Hololens pivoted between 1 and 2, if you see the
    experiences that shipped with Hololens 1 was much more gaming and
    entertainment, whereas Hololens 2 was much more enterprises. So
    that's where our focus is as well.

    Alan: I saw this thing when I

    first got into this, I said, "I'm not interested in games. I
    want to figure out productivity." So you've done training stuff,
    you guys been working-- you used to be called Studio 216. Why the
    change in name? Or was it just because you merged and they said, "You
    know what, we're gonna be altouristic."

    Jamie: [laughs]

    Bharat: I'm going to use that

    line from now on.

    Jamie: Yeah, I know, right?

    That's great. You better quickly trademark that, Alan.

    Bharat: [laughs]

    Jamie: The history and the

    legacy of Studio 216 was really about content creation. We've done
    literally, I would say, tens of thousands of virtual objects and
    environments that we've built out of the last 15 years. A lot of that
    was focused on real estate, of building out projects that didn't yet
    exist and helping pre-sell or pre-lease them. And it was really
    through-- kind of a couple of events happened for our company, one
    which we just mentioned that we acquired Bharat's startup and to
    help. We were very excited about the cloud component of what he had
    built for extended reality and saw it highly applicable to the work
    that we were doing with Studio 216. The other kind of major milestone
    for us is that we raised venture capital, and as part of that we
    raised money to focus on the software applications. And as part of
    that we rebranded as our flagship software product, that we call
    Altoura. And so we just transitioned the name from Studio 216 to
    Altoura.

    Alan: What does the flagship

    Altoura software actually do?

    Jamie: So I'll give you kind of

    a high level view of it and Bharat can take you into the weeds, if
    you're interested. But at a high level, our software allows the
    creation or-- we take a digital twin of an object or an environment,
    and we bring it into our software, which immediately allows it to
    work across devices. You're connected with voice-over-IP. You have
    avatar representations of the different people in the sessions. And
    you can easily make that digital twin interactive, depending on what
    application you're trying to solve for. If that is sales and
    marketing, maybe you want to explore different finishes or different
    layouts of an environment. In training, maybe you're following
    different steps of how you operate a piece of machinery or how you
    interact with customers. And then the other piece that's really
    critical for us with Altoura is that we allow our customers to create
    their own-- for example, in training just using a web portal, you can
    easily integrate your existing L&D content into our software and
    expose it in that 3D environment and make it interactive.

    Alan: That's super cool. What

    does that mean?

    Bharat: We basically have a

    cloud software. Our app runs in the cloud. It is-- we have clients
    with each of our devices that we support from mobile tablets, PCs to
    headsets. But the app that's running the state of the app is in the
    cloud. So if-- let's say if you are on your iPhone, looking at a
    space that we generated, and I'm here in my Hololens looking at it.
    If I was to move a table around, you would see that table being moved
    as well, because the app's kind of running agnostic of the device,
    it's running in the cloud. And so that allows us to do, for example,
    what in for one of our clients, there was a slide deck that they had
    to go to learn about the product. And yes, you can go to a slide deck
    on your laptop, it's a 2D space. But to be able to pull that slide
    deck inside the experience and see it next to the product within
    having the product OK'd and give you a visual to associate that
    training was very important for the client. So that was something we
    could easily do. Similarly, for that product, there was a companion
    app that the sales representative had to learn as to how to operate.
    So we could build a walk-through of that app within that experience.
    So you wear the headset or you're on your phone, on your PC. You're
    navigating the store, you bring up the device. You can say, I want to
    learn about it, I want to see how to use it, or I want to see how to
    sell it. And the "sell" would be a roleplay. There would be
    a virtual customer come up and have-- you could build up a whole
    decision tree where the customer would be saying, "I want to to
    do this." And your response as the trainee would be, should you
    start selling now, or do you want to understand the requirements
    better, or do you still just want to kind of empathize with the
    customer? And you could choose your path and go down and complete the
    roleplay and get feedback back, like was this the right thing to do
    or was that the wrong thing to do? All this while you're feeling like
    you're inside the store that you would be actually working in.

    Alan: Super cool. So people

    could do the training from home before they even stepped foot in the
    store?

    Bharat: Exactly. And this

    company used to have people come in post store closure to do this
    training at odd hours, because they couldn't do it while they were
    customers in the store. So this allowed them to scale that up pretty
    well.

    Alan: So does your system allow

    them to make it themselves?

    Bharat: Yes. In some cases,

    there are companies that have dealt with 3D content like the
    construction companies. They already know how to deal with 3D. And
    for them, it's a very simple training to understand how to transform
    assets from standard authoring tools like Autodesk, 3D Max, or Maya
    into something that our system can understand. And that's absolutely
    about forward. And in some cases retail clients or airlines that
    right now don't have the 3D skillsets, they just have us do it. But
    what we have starting to see, interestingly, is people are thinking
    about investing and hiring people with 3D technology experiences, so
    that they can take more control over the content and how this
    progresses through their enterprises.

    Alan: But I think as we move

    forward, 3D is just going to become like video and like audio.
    Everybody can do it. One of the stumbling blocks you mentioned is
    bringing in 3D models. It seems to be kind of the stumbling block
    everybody runs into. Because let's be honest, there is no
    standardization yet. So do you guys have your own file format that
    you're working with? Or do you use certain specific file formats?

    Bharat: That's a great question.

    And yes, you're absolutely right. When we were pitching to investors,
    for them to even understand that, oh, 3D doesn't have a well-defined
    standard format. It's taken for granted in 2D. So that was
    interesting. We have our own format, not a file format. So to be
    clear, we went after the most optimized format for Unity, which are
    Unity AssetBundles. That's the format we support. What we've done is,
    we have added a lot of decorations to that AssetBundle. So in the
    content itself, you can say, "I want this chair to be
    interactive, but I don't want this table to be interactive. I want
    this table to be--" Or you could say "This is the product
    that I want to associate this decision tree that I created using a
    web portal." So you can decorate the content, and our system
    understands that and brings up the right experiences. So this allows
    us to do a 3D model once and give it to the client, and then the
    client can just link 2D content or the decision tree roleplays to
    different objects in the 3D space, thus minimizing the need for 3D
    changes.

    Alan: So is it a specific model,

    then? Are you using glTF or...?

    Bharat: No, we are using Unity

    AssetBundles, so that's the format.

    Alan: This is all built on

    Unity, then?

    Bharat: And Unity AssetBundles

    is the most optimized file format that Unity understands and that's
    very important, especially for Hololens or any other -- like Magic
    Leap -- because their processing powers are so, so limited on what
    kind of content they can process.

    Alan: So exciting. I'm just--

    the questions are so many. I guess the question would be why wouldn't
    they just build it on Unity, then?

    Bharat: Unity is a big platform

    for developers. To reach where we have reached, we've invested
    decades of-- personal decades of our team into a place where we are,
    where an AssetBundle can be consumed on multiple device types and
    work together in a coordinated manner. Yes, you can open up Unity and
    create a simple experience and sure, if that's how you want to do it,
    that's not a bad format at all. Unity is growing exactly for that
    reason. But if you want a scalable enterprise ready system, then that
    part is a really hard part. Then having our solution ready, where you
    can still take that AssetBundle learning, but use it on our platform,
    which does the rest of the things, like distribution, making sure it
    is running on all devices, that it's networked. All of those benefits
    you get by putting it on our systems.

    Alan: Do you guys manage device

    management as well?

    Bharat: No. We tend to integrate

    with an MDM kind of systems, our Hololens MDM is a good system for us
    to use. Mobile-- anyway, they already have mobile MDM systems.
    Actually, our apps are in the app store. So that's another way, easy
    way for them to get, like they don't have to go and install it from
    someplace. They can just go download it from the app store, log in
    with their credentials and they will get the experiences that they
    signed up for.

    Alan: I guess it's all running

    on cloud, you said. But I know some companies, they want to run it on
    premise. Is that possible as well?

    Bharat: Yes, we have-- our

    system is built on containers. So it's a containerized system, so we
    can move it to their cloud. So some of our clients wanted to run it
    on their tenant, rather than in ours. So, absolutely. That's easy
    enough. And we can run it on local machines, because it's all running
    within the container that we can take anywhere. That has been on us,
    that sometimes they don't have cloud connectivity in locations that
    they want to do training in, for example.

    Jamie: Sometimes there are

    security issues--

    Bharat: Or security issues.

    Jamie: --they don't allow them

    to do that.

    Alan: Yeah, it's-- there's all

    sorts of reasons why. Well, you mentioned doing photogrammetry of the
    store, for example. Maybe you can elaborate on ways people can
    acquire these models?

    Jamie: We use a variety of

    methods to capture and create a digital twin of an environment. So
    photogrammetry is one. Doesn't-- it's not -- I would say -- the best
    for every type of application that you're trying to create a 3D
    environment. But I think right now it seems like you've got a few
    choices. You can buy some generic 3D scenes from digital asset
    stores. But more typically, I think you-- if you don't have the
    experience in-house, you need to find someone that can create the
    digital twin of whatever environment that you're looking for. And
    doing that in a cost effective manner, that's going to allow you to
    interact with it in the ways that you want to interact with it. So
    maybe that's a long-winded way of saying that I think the advantage
    right now -- where we've seen a lot of advantages with our customers
    that don't have the 3D capability -- is they can engage with a group
    like ours to create these environments and then quickly optimize them
    and distribute them across devices using our software. The other
    piece that we've seen is there's just such a wild price, an effort
    range for creating a, say, a training experience, or a sales and
    marketing experience. And we all know the industry is maturing. It's
    evolving every single day. There are all kinds of fantastic news out
    there. But one thing that where we have really benefited is the
    ability to quickly stand up for a very low cost, to stand up the
    digital twin of an enterprise's environment, and allow them to start
    using it in a very short period of time. So it doesn't require a lot
    of-- really, it requires no coding in terms of getting these training
    set up. It's just the effort of building out an environment usually
    takes anywhere from a couple of days. It may take a couple of weeks
    depending on the size and scale of this environment. But for an
    enterprise to be able to begin training in something within a few
    weeks after signing a deal is really transformational.

    Alan: That's another issue that

    comes up with people, is that they don't want to be locking into a
    year long production schedule, to have something made where by the
    time it's made, it's obsolete. So I am going to ask you guys one last
    question, because we're at 30 minutes here. And what I want to know
    from you personally and you can each answer this is, what problem in
    the world do you want to see solved using XR technologies?

    Jamie: That's the million dollar

    question. Coming back to what we have had the opportunity to build
    out lots of different-- to solve lots of different problems already
    with extended reality. And I think the ones that we're really
    interested in are around training, just to be able to give people the
    ability to train anywhere in the world, to network with experts
    anywhere in the world, to preserve the legacy of what other people
    have figured out, and interesting content in the past, to be able to
    preserve that and extend it to people in the future. So I would say
    it would come down to, there'll be more and more interesting and
    exciting ways to propagate training.

    Bharat: And in the long term, if

    you look at this technology as it was and kind of predict out maybe
    even a decade out, it's strongly, strongly believed that this will be
    as pervasive as mobile. So maybe the question that I would like to
    answer is not that which problem it will solve, but the capabilities
    that it will give to us will be so different from what we have now
    from mobile phones -- which were so different from PCs -- that I
    would be surprised if there is any problem that is not impacted by
    this technology. This will solve or this will create solutions or
    create opportunities that you can't even think of, just like before
    the mobile phone arrived. Like if you were to go maybe in the 1990s,
    who would have thought an Uber would be possible? It is only possible
    because you have this device with you that you carry at all times. Or
    things like Instagram, where you can communicate so effectively,
    because you now have a device that can take cam-- that can take
    pictures and apply filters on that. So I feel similarly as this
    technology grows and especially as it gets into consumer adoption, it
    will impact every aspect of our lives, and it will create
    opportunities and experiences that we really can't think of right
    now.

    Alan: If you look back 10 years

    -- well, I guess 12 years -- iPhones didn't exist. iPads didn't
    exist. And now everybody in the world's got one in their pocket, or
    some sort of device like it. What will be the things we do with these
    glasses?

    Bharat: They will impact every

    aspect of communications. I think they will just enhance your quality
    of life, in my point of view.

    Alan: I couldn't agree more.

    Where can people find you guys?

    Jamie: They can find us in

    Seattle, Washington. Or they can find us at altoura.com.

    Alan: Amazing. Well, thank you

    guys so much for joining me on the show.

    Jamie: All right. Thank you,

    Alan.

    Bharat: Thanks, Alan. Great to

    talk.

    35 min
  • Making AR Focals Functional and Fashionable, with North’s Stefan Alexander

    Making any sort of head-mounted AR display has been a challenge, both on the technology front, and from an adaptation standpoint. But Stefan Alexander from North challenged himself even further - by making them look chic, to boot.

     

     

     

     

    Alan: Hey, everyone. Alan Smithson here, with the XR for Business podcast. Today, we're speaking with Stefan Alexander, vice president of Advanced R&D for North, the company this created Focals, the world's first consumer AR glasses. And of course, they're also a Canadian company. And we're really excited to talk about their new product, North Focals 2. All that and more, coming up next on the XR for Business podcast. Stefan, welcome to the show.

     

    Stefan: Thanks, it's great to be here.

     

    Alan: It's my absolute pleasure. As you know, I have had a pair of North glasses for almost-- actually over a year now. I was one of the first 100 people to be lucky enough to get these. I went in for my fitting in Toronto, got these wonderful glasses, I got my little ring. And I proceeded to try all sorts of different things. And super excited to have you on the call and really learn more about what's coming up next for North.

     

    Stefan: Yeah, great.

     

    Alan: Maybe you can just describe the North glasses to the listeners, and how they came about.

     

    Stefan: So, I can give you a kind of brief history of how this whole thing started. So originally when North was founded, it was actually called Thalmic Labs, and the product was a gesture control armband. So this kind of went on your upper forearm. You could make motions with your hand and it would detect your muscle movements and you could control computers, music, do presentation control. But one of the things that they were really passionate about was controlling heads-up displays like Google Glass, which had just come out at the time. I was actually the first person hired to not work on the Myo, that was this armband. And about a year before it came out, they hired me and they said, "Stefan, we think that the control of smart glasses and the control of head mounted displays is really important. But we're not sure if anybody's going to make exactly what we have in mind, what we think is gonna be so big, which is glasses that look exactly like regular glasses. And we don't know the tech to do this, and I don't think it exists yet. But can we work on a way to do these type of smart glasses?" So I had a display background. I was working at OLED displays. And so I started this research program, that turned into the first generation of Focals. And eventually it got so good that it really just took over the company, and we stopped doing the gesture control and we've just kind of went all in on smart glasses and changed our name to North. And that's kind of how we ended up where we are.

     

    Alan: That's fantastic. I wonder-- you started off life as a gesture armband. And it's funny, because I remember this. I was part of the Ryerson Digital Media Zone at the time. And I went to Communitech, which is where North was founded, or I guess Thalmic Labs at the time.

     

    Stefan: Yeah, yeah.

     

    Alan: And I remember going into this tiny little lab with I think there was probably 10 people at the time, and they said, "Hey, try this thing on your forearm." And it was this kind of stretchy, almost like a bracelet with a bunch of black sensors on it. And after that, I went on to create The Emulator, which was the see-through touchscreen DJ controller. And we worked-- we ended up working with Armin Van Buren, who was also working with the Myo wristband. And the way they were using it -- which was really cool -- is they had one on each arm, and he was able to control the visuals onstage, by just simply reaching up and kind of extending his hand, and that motion of extending his hand created fireworks that came off or triggered the fireworks.

     

    Stefan: Right.

     

    Alan: It was kind of that moment went, wow, that's amazing. And then I got into VR and AR. And then, of course, this Canadian company comes along with this pair of glasses that look literally like just a normal pair or glasses, the arms-- everything just looks like a normal pair of glasses. And when I first got my pair, my daughter's 15 -- she was 14 then -- she put them on. She said, "These glasses are amazing. They're really lightweight. They look like normal glasses." She said, "I am going to give them a 9 out of 10 for the fashionable part of it." But functionally, they left a lot to be desired. There was a very small viewing angle and I think for a first pass of a product it was pretty damned impressive, being able to have an ability to see something at, let's say a meter or two meter distance and have like a full heads-up display in front of you. So for those of you who don't know, these glasses have a little tiny-- they're monocular, so it's only one eye, but it's a little projector inside the arm projecting on a little -- I guess -- projection film, I guess. Would it be-- is it a film or is it a--?

     

    Stefan: It's a special film called a holographic film. And the light kind of bounces out of the projector. And it doesn't just reflect towards your eye. The hologram actually kind of focuses it towards your eye, too.

     

    Alan: Yeah, it's super cool. I mean, the amount of tech that's gone into this is just mind-boggling So, walk me through some of the use cases, because I know when I got them, I don't think-- you couldn't do very much, actually. You could see your messages, you could see your calendar, and that was about it, at the very least--

     

    Stefan: There's some maps and navigation, you could call an Uber.

     

    Alan: Yeah. Oh, yeah. Oh, the Uber thing was amazing. Oh, my goodness. Imagine you're calling Uber and you see the heads-up display and it's-- rather than checking your phone every five seconds to see if your Uber's there, it would just alert you in a little pop-up thing, and your Uber's here. That was a really cool-- I actually use that feature. And then, the one thing that I tried to use on stage last year -- but it just wasn't quite ready -- was the ability to see your presentation notes. And I actually did a presentation last year with the Focals on, trying to make this work. It was a talk I did at the Miami VR Expo. And I gave the whole talk with my show notes in the glasses, but I just couldn't synchronize it. But that's now a feature, right?

     

    Stefan: Yeah, it's totally working. I actually did it just on stage about six weeks ago at Photonics West. So I gave a talk there at a kind of optical conference, and I was able to control the presentation and look at my notes using Focals, which was pretty-- it was pretty cool.

     

    Alan: That is very cool, and then the great thing about is, nobody knows you're wearing AR glasses, because they just look like normal glasses. And I even got the clip-on sunglass adapter and everything. The one thing that I just was amazed by was the little ring. The ring that controls your interface. And how did that come about?

     

    Stefan: So we had a pretty high bar, starting off as an interaction company. So if we were going to do smart glasses, the control had to be amazing. And the whole philosophy going into this -- and this evolved a lot, but this core tenet never changed -- where it has to look exactly like regular glasses. And the bar for what that was kept getting higher and higher in our minds. It's not just enough to look like regular glasses. There are so many aspects that are so important with this that we ended up learning and getting mostly for generation one. And I feel absolutely, completely for generation 2. But one of those was, if you're going to interact with it, it can't look strange, it can't be unnatural. There's no point in having something on your face that looks just like regular glasses, and then you're around a bunch of other people and nobody knows you're wearing smart glasses And then you have to reach out and touch your face and touch some buttons, or you have to wave your hands in front of your face, or you have to talk to your glasses and give them voice control. All those things are fine if you're on your own. But then if you're on your own, it doesn't really need to look like regular glasses anyway, does it? Like, you don't care what it looks like. It's when you're around other people and kind of wearing it all day. So the interaction had to be completely natural. And the ring, the thing I love about it is it just seems obvious. Like you use it, you're like, "What, I have to wear a ring, really?" And then you put it on and you control it, and you think, "Oh, this is actually really cool. This is a really great experience." And really, really great interactions should be like that. But we have a human-computer interaction group and we tested so many different aspects of control. We tested wristbands. We tested scrolling on smartwatches. Obviously, we tested kind of moving around with head movements. We tested things with eye tracking for a variety of reasons. All of those didn't work all that well. There's actually something that tested surprisingly well -- we didn't ship it -- but you kind of moved your head around as the cursor and then you clicked your teeth in order to activate something.

     

    Alan: No way!

     

    Stefan: Out of all of the things that I listed, it was actually pretty high. It was surprisingly good experience. People were like-- because it actually worked. It was pretty subtle. I mean, at the end of the day, the ring was better than the teeth clicking interface. But it was just-- We did so much cool stuff. And the ring, it really stood out over anything else, because it was just so intuitive and natural.

     

    Alan: It is very intuitive. I have done literally thousands of XR demos from Hololens to VR, you name it. The one thing I did find it with the Focals was, the first thing is nobody could see the visual spot. They had to kind of aim it. The fact that the field of view is small, that was one challenge. But once they got it and then the ring, I didn't-- you'd say, "OK, press the ring." And then nothing else. You don't have to say go left, go right. It's a joystick ring. And once they realize they can click it and move it, you don't have to say anything else. They just-- the navigation was super simple and real time. It felt right. I'm just going down the North site. And for those of you want to learn more about it bynorth.com. And one thing I totally forgot about is you guys have microphones built into it as well. So you can actually use Amazon, Alexa to help you with stuff.

     

    Stefan: I'd say the integration is pretty cool. And--

     

    Alan: It really is.

     

    Stefan: There's a whole trend in smart speakers now -- which I think is great -- to actually have a screen on them. And you would think, "Why do you need a screen if you have a smart speaker?" But there's actually a lot of times when the result that you want back is best displayed as something visual, instead of something audio. So we have that integration in the glasses. If you ask Alexa something, you ask for the weather, for example, you will get a visual notification of the weather on the screen, which is awesome, which is better -- sometimes like for a five day forecast -- than having Alexa read it out for you.

     

    Alan: Absolutely. And one thing that stood out again for a V1 glass with a tiny micro-projector. And you look at things like the Intel Vaunt, for example, which I believe North ended up buying the patent portfolio on.

     

    Stefan: Yeah.

     

    Alan: But it was monochromatic. So it was monocular -- meaning in one eye -- but also monochromatic, it was just red. But the North glasses, you guys created a full color spectrum display, which-- how did you make a tiny projector small enough to give a full color display bright enough to be used outside? It's just mind-boggling, the sheer science that has gone into these glasses.

     

    Stefan: So this is where I have to give her a lot of credit to the CEO and the founders of the company, and kind of everybody else around us then. Because I-- we really, really wanted glasses that looked like a regular pair of glasses. And that was the most important thing. And there was just no display technology that did it. This whole kind of laser projector and hologram thing, we had to build that ourselves. It didn't exist in something we could just buy and stick in the glasses like with normal micro-displays. So when we're going through it and trying to figure out how it would work, there was a lot of things with it that just weren't working. And one of them was-- so first thing I said was "It's going to have to be lower resolution." It's text. It's not super HD resolution. It's not going to be 1080p, it's not going to be 720p. It's gonna be lower resolution. But I think we can make the display look amazing. I think it's much better to have it really bright and a high quality display, but we can't have a lot of pixels. So that eventually was accepted. And the other thing I said was "Do you know how much easier it is to just do red?" Red lasers are awesome. They're the best types of lasers, they're the cheapest. And we can just save so much space and cost and complexity in the whole system. I don't even know how to make the hologram work with multiple colors. That's really hard. And from working in displays, a monochrome display is hard. Color display that has three different colors -- red, green and blue -- you would think it was three times harder, but it's not. It's usually 10 times harder. It's so much harder, because of the patterning, because of the fact that you have to overlap these colors, it's just-- everything gets so much harder.

     

    Alan: Well, you're also starting to see things like the Hololens 2, with their color banding issue. You've got Microsoft -- who's invested billions of dollars in Hololens 2 -- and they're even struggling with some color banding issues.

     

    Stefan: Color is just-- it's so hard. Just the perceptual science behind it, and the calibration-- the display calibration is going to be ten times more complicated. It's already going to be hard. And so that was something, though, that they just didn't give up on. Like it's fine, let's prototype some monochrome ones. But it needs to be color. It's a consumer product. People need to love these. They need to feel like the display is responsive and alive. And something that's monochrome, even though technically you can list all the use cases, and you can't show me a use case that actually needs color. But the feeling people get when they use it, it's so important with color. So I lost that argument, but at the same time we kept working on it, and we came up with a way to actually do color in there too. And I think it was-- it was after I lost the argument, actually. So I'm like, "Okay, I still don't know how to do color, but I'm going to keep trying on it."

     

    Alan: "We'll figure it out."

     

    Stefan: Yeah. And we'll figure it out. And we *did*. And I love the way that it looks, the color.

     

    Alan: It really does look amazing.

     

    Stefan: And it did make the temple arm slightly bigger. However, for the second generation, we figured out a way to reshrink it again, and all of the space that we allocated to color is actually now even smaller. So I'm so glad we put the color on there.

     

    Alan: That's incredible. I want to talk more about the user feedback. I've had my pair since the very beginning and I have to be honest, I don't wear them all the time. It's mainly because when I got fitted for them-- and I want to talk about the fitting system has also changed dramatically for you guys, because when you introduced these, they needed to be fitted to your face. And you guys created a quite elaborate 3D laser scanning system, where you'd go into a room, this laser scanner would go around your head, create a 3D model of your head, and then fit the glasses to that. Since then, the iPhone 11 came out with IR tracking on the front face or the back-facing camera -- front-facing camera -- and allowed you to scan people's faces with just an iPhone.

     

    Stefan: Yep.

     

    Alan: So is that something you're moving forward with, so that you can now expand these fitting locations, because you were tied to this elaborate fitting station before?

     

    Stefan: I think the way we started off with this is, they need to look like a regular pair of glasses and looking at it, the adjustability of a product -- like a one-size-fits-all -- you make a lot of sacrifices in form factor, it just drives a lot of space in there. And also the eye box, so this is the property of the glasses that show how much can your eye move and still see the display. And the trick with an eye box is that -- especially with this not a very wide screen -- the amount that your pupil actually moves as you're scanning around the display, it's actually not that high. But the amount that you have to allocate to -- let's say -- misalignment or slipping or everything with the glasses. That's where the majority of the eye box comes from. Which is why you can see it reasonably well when it's on you. But if you put it on somebody else, they can't see it at all. So we said, well, eye box drives a lot of space. So what if we had a small eye box? And what if glasses weren't adjustable? But instead, we pack this eye box and adjustability technology. Instead of putting them in the glasses, let's just put them in a measurement machine. Let's put them in our process flow. Let's build it into the rest of the business. If you want to make the glasses as small as possible, I think we just need to become a glasses and a measurement company. And that way, people also get a beautifully fit pair of glasses, perfectly adjusted for their head. So that was kind of the original philosophy on how to do that tradeoff, and how we ended up in the sizing in Gen 1. And you know, it worked. It works well. I think there's two big areas where we still didn't like that. And one is you have to go into a store. It was awesome if you could go into a store, because it was a really cool experience.

     

    Alan: And the stores were nice. Come on, they were really beautiful.

     

    Stefan: Yeah, I loved the stores, the whole retail experience. It was so good. But if you weren't there, and there wasn't a mobile sizing truck that we kind of brought around to, then you were stuck, you couldn't buy the glasses. So that wasn't great. The sizing app certainly helps with that. So it totally resolved that. If you didn't have an iPhone 10, somebody else could have probably also-- like you could have bought it from somebody else or something like that. So that increased the availability. That wasn't that bad. But the real thing that actually held us back there is, what if you wanted to give them to somebody else to try on?

     

    Alan: [laughs] This is my exact problem.

     

    Stefan: Yeah, you're so excited about them. We hear this from so many people.

     

    Alan: Oh, my God. I can't even-- I'm going to explain it to you.

     

    Stefan: Yeah!

     

    Alan: I've got to explain it as a user and somebody in the XR space. I get new technology. We've got a Magic Leap, a Hololens, we have all the tech. And we bring it to events and we show people, "Hey, this is the future." And I love my North glasses, until I brought them to an event and let everybody try them. And then they didn't work anymore. They didn't fit me. I had to go get refitted again. So the ability to show other people was hindered in that way. And that really-- it was a stumbling point for me as a user, because you want to share this technology. You know, we're all a bunch of nerds.

     

    Stefan: Yeah. And it's yet-- it's hard when you can't do that, because you have this cool experience and then nobody else can. They have to go into a store and get a special custom demo, just in order for you to show them something cool. So then what we wanted for Gen 2, and it was also I think-- if people do wear them all day, we said, "OK, they have to wear them all day. They have to be really light." The glasses were 67 grams. I think it was like almost one of the lightest ones that was out there. And it's still after maybe six hours or so, some people were fine and some people felt it was still a little bit heavy. Then we had this goal going into generation 2 where we said, OK, how about this as a goal: if you can fit the glasses on your face physically -- because they still have to be a properly fitting pair of glasses. Not every pair of frames should go in every head, because heads are different sizes and glasses have to look good -- but let's just say if these glasses can fit on your head, and they look okay and they're not squeezing or pinching you or falling off, then you can see the display. So if you should be wearing the frames, if they look half decent on you, then you have to be able to see the display. So that was one of the things. The other one is temple arms have to get smaller and it has to get lighter. It has to get to 50 grams. And the only way to do that is really to shave off space and shave off actual components and cut the battery, cut the power consumption. So we had to make it smaller and we had to cut power. And then we also said, I mean, wouldn't it be nice if we could show a little more text on there? Sometimes you don't want a lot of text, which is great. Like with a message or something like that, you don't want to be distracted. But let's see when you're watching your presentation notes, you don't want to necessarily have like six or seven lines of text. It's nice to be able to see all your text in your speaker's notes, so you don't have to scroll as you're trying to talk at the same time. So what if we could fit a little bit more on the display, too, at the same time? And as an engineer, you do this work and there's no new fundamental physics that we're doing in this. It's just-- it's technology. It's combining things, it's understanding tradeoffs, understanding what you can do. But it doesn't matter what you can do, it only matters what you can mass produce. So if you come up with some incredible new RGB laser and you can embed it in the glasses itself, if nobody can build it, it doesn't matter. So you're pretty constrained. So everything is just managing tradeoffs. And so going into Gen 2 where you said, OK, well, the whole reason why we did the sizing thing is to keep a small eye box to make the glasses small. So now you want to make the eye box so much bigger -- like 20 times bigger -- and then you want 10 times the resolution, *and* you want it smaller at the same time. And it should cost the same price, and we should cut power consumption. I'm like, what are you giving up? You can't--

     

    Alan: I was going to say, what the hell, man?

     

    Stefan: Yeah, technology gets a little bit better, but you can't-- you don't just get to do this from one generation to another one. Things are incremental. We can use a slightly faster processor and save 20 percent power, but you don't just get to do this from one generation to the next. But--

     

    Alan: *But...*

     

    Stefan: The team really came together. But we actually managed to--

     

    Alan: Patent portfolio.

     

    Stefan: Yeah.

     

    Alan: And the company has raised in excess of $100-million.

     

    Stefan: Yeah, yeah. In terms of US dollars, it's over 150 at this point.

     

    Alan: Holy moly.

     

    Stefan: So it cost a lot.

     

    Alan: That's like a 100 million Canadian dollars.

     

    Stefan: [laughs]

     

    Alan: [laughs]

     

    Stefan: Yeah, yeah. It's getting-- it's not quite, but close to like 200 million Canadian dollars.

     

    Alan: Yeah, sorry, two hundred million Canadian. Yeah, that's a lot.

     

    Stefan: It just-- it costs a lot to do this development. So we managed to have an entirely new architecture. A lot of the stuff we were working on for Gen 1 that we kind of-- wasn't quite ready yet, and then some other breakthroughs. And we managed to have a generation 2 that is everything: smaller, high resolution, much lighter. And the greatest thing is, if you can put the glasses on your face, you can see them. So you can have one pair of glasses and 20 people can see it. And that part of being able to show people in the demo. The portion that people saw the Gen 1 and who wanted to see the Gen 2, the most surprising thing to them, the first thing is, "Oh, I can see the display. This is so cool. Like, it's just here." There's never an issue there.

     

    Alan: That's gonna be a game changer, because doing demos and having people go, "I can't see it." And that's just like "Eh." [chuckles] So, yeah. This is fantastic.

     

    Stefan: Definitely credit to the team and also just the whole-- even once Gen 1 was in the engineering phase, I had a whole team and we had 15 or 20 people way before Gen 1 launched, and we were working on Gen 2 technology. The beginnings of this was almost three years ago in terms of the Gen 2. So we've been working on it for a long time. So even when the Gen 1 came out, we said tech takes a long time. So let's keep going with this technology pipeline now that we really, really know what we're doing. And the team really came together and very deeply understands this. And another awesome thing is the suppliers. They believed so much in what we're doing and this product line, they really loved the Gen 1 and participating in there. And so all of them make a lot of custom components for us. We do a lot of custom co-development with our suppliers, because they believe the space -- like us -- is going to be huge. And so we couldn't have done that without those close collaboration, too. Because, again, you can come up with something cool, but if nobody can manufacture it, then it doesn't matter.

     

    Alan: Yeah. And if you look at just the engineering side of the hardware, I mean, that's spectacular on its own. And really, when it comes down to it, though, people once they put it on their face, once they buy into it, the engineering side, the hardware side, the ring, the glasses, the eye box. None of that shit matters at all. What people want to know is what can I do with it, and how can this help my life, and how can this help my day to day? And one of the things that I found as a user experience that was incredible -- and you guys thought this through -- was as I was walking down the street one day, I'm talking to somebody and my alerts kept coming up. And when you're looking at the alerts, you're looking straight through the glasses. And if you're looking at somebody on the other side, it's really strange when you're looking at a message while looking at somebody. It's kind of like you're staring at them, but you're off in space, you're not really there. And so one of the interactions that I realized you guys put in after is, if you're having a conversation with somebody, it doesn't show you alerts. And I don't know if it's through the voice, through the microphone or whatever. But that one simple user interaction -- or taking that away, really -- was amazing, because if I'm in a face-to-face conversation with somebody, the last thing I want is my messages popping up. And you know how annoying it is when people look at their Apple Watch when they're in middle of a conversation, you're talking and then all of a sudden they're looking at their Apple Watch, and they just derailed your whole conversation.

     

    Stefan: Yeah.

     

    Alan: That's what you don't want with these glasses. And you guys have solved for that.

     

    Stefan: Yeah. And this is-- it comes down to that core tenant, too, of looking like regular glasses. And if you can have these glasses that look just like regular glasses, and the display is invisible and you can't tell that somebody is interacting with it, because their arm is down by their side and they're controlling the loop. But if they have this stare, this look where they're clearly looking at you but not paying attention to you, then it just ruins the whole thing, too.

     

    Alan: It is such a weird phenomenon, I'm not going to lie. I've done it on purpose to people, just standing there looking at them and they're looking right at you, but they're not, they're looking right through you or like at the display that's kind of above your head.

     

    Stefan: Yeah. What you're talking about is definitely one of those magical moments that our users really liked and they wanted more of. And the other thing that people love. So, you know, in terms of how people are using this, it's about half of all of the uses of it are when the glasses prompt you and say, hey, you have a notification, a message from another person, a message from an app, notification for an app that you have an appointment coming up, or something like that. So half of all the times when the users use the glasses, they don't even initiate it. So we have to be so good at knowing when they want to use it, and when they don't want to use it. And when you get that right, it's amazing because they get information and they're not even thinking that they might need it. So it's just the right time. But if you do too much, then they're gonna shut them off. Then they're not going to use them and that's going to be frustrating. So getting that right and understanding-- and what you talk about, we call "context". So what is the user doing over the environment? Who are they with? We have done a huge amount of extra work on context, both in sensors and the hardware, but also in very intelligent artificial intelligence algorithms, to detect more and more of your environment. And the other thing we use context for, so this is again, when people do click the ring and they bring up the glasses, which is about half of all interactions. Half the times they click the ring, they see exactly what they need on the home screen and they never go anywhere else, because the home screen is contextual, too, because the home screen knows you have a calendar appointment coming up. Or you're playing Spotify music, you might want to see your music controls and the song. Or this is kind of cool, I don't know if you used them recently, but if you have a flight booked and you're at an airport, then it shows you, here's your gate, here's where you need to go.

     

    Alan: Oh, man. That's like just that one feature alone.

     

    Stefan: Yeah.

     

    Alan: "Where the hell am I going in this airport?"

     

    Stefan: It has a whole contextual airport experience when you're there, too. And so being able to get it right. When people click the glasses and they bring up the home screen. They don't even have to go anywhere else. We've shown them exactly what they wanted to, just with that alone. And then they close it down when they click the glasses. And another half of the time they're going to scroll through and they're going to choose the right application. So that's still pretty quick. But the bar, the thing we want to hit is that magical experience, where the glasses either tell you what you want to know, or when you ask them something, they know exactly what you want. And being able to do that with a whole new variety of sensors that we're using. And a lot more intelligence and just guessing what people want and giving it to them. This is-- it's totally doable, from a technology perspective. And I think the people are going-- I think everyone's going to love it with Gen 2, that's one of the things that we've done the most with Gen 2 in terms of the user experience. So I'm glad you loved that conversation detection, because that's basically the direction we want to take the product.

     

    Alan: I think this is more than a hardware product. This is more than a pair of glasses that you can put some data on. This is a new way to interact with data around you. And I love the simplicity of it. It doesn't have front facing cameras -- I don't know, maybe Gen 2 does -- but it's not trying to world map your whole view, it's just-- it's like having an Apple Watch in a more convenient place. It's like having all the data that you have in your watch, where you don't have to look at your watch or pull out your phone. It's really wonderful. I have to ask, on behalf of everybody listening, when can we expect V2 to drop?

     

    Stefan: So it is going to be still on track for sometime in 2020. There's going to be definitely a gradual reveal here. So it's not just-- we're going to be quiet and you're not going to see anything. And then they'll just appear one day. There will be opportunities for people to see more. Maybe some previews. We have a lot of units that we're using right now in testing. The engineers are testing units for functionality. The production people are testing the mass produce ability and reliability. The product people use them to develop software. Everyone's wearing them regularly, in terms of our employees and some initial beta testers. And more and more will be revealed even before they actually go on sale. People will definitely be able to find out a lot more of them and hopefully even be able to get some demos.

     

    Alan: I would love to be part of that beta test. Hint, hint, nudge, nudge.

     

    Stefan: I'll pass that along.

     

    Alan: [chuckles] Awesome. Is there anything else you want to share with everybody before we wrap this up? Because this has been fantastic interview here. And I think there's so much to unpack. And the North Focals 2 are coming this year, in the year of 2020, which obviously makes sense. So I think people are going to get excited. And you said the similar price point around the-- I think-- what are the other ones, $700?

     

    Stefan: Yeah, I think it was $599, the base price. And then $699 with prescription glasses. I'm not sure of what the price is going to be for--

     

    Alan: So under a thousand dollars, for sure.

     

    Stefan: Exactly. Definitely not substantially different.

     

    Alan: Incredible success story coming out of Canada, which is cool. And I have to ask you, living about 40 minutes away from the office, can I come for a tour?

     

    Stefan: We are doing stuff like that, once in a while. So I think we should get in touch with production and marketing people, and see if we can arrange something like that. If not, not a great time the next two or three weeks.

     

    Alan: [laughs] Obviously not now.

     

    Stefan: But I actually actually think soon. I think that be a lot of fun.

     

    Alan: That'd be great. I would love to do it and share the experience with everybody and show people what does a company developing the future of digital eyewear look like from the inside? What is the research lab, and what are the people doing there? I had an opportunity to go to the Magic Leap head office and tour that, and unfortunately I can't say anything or show any photos. But it was an amazing experience to see just the sheer numbers of people working on different parts of-- things you wouldn't even think of. You mentioned earlier, just the voice detection, the AI. There's so much going into this, that's beyond just a pair of glasses with a display in them. And I think that's what's really important is that we're driving-- you guys are driving the future of of human interaction in augmented reality. Thank you so much, Stefan, for being an industry leader and a pioneer in the space.

     

    Stefan: Oh, you're welcome. It's so much fun.

     

    Alan: With that, I can ask you one last question and then we'll wrap it up. What problem in the world do you want to see solved using XR technologies?

     

    Stefan: That's a great question. I have definitely thought about that from a lot of different perspectives, but in terms of a problem that wants to be solved... I think we have this issue right now, where-- it's so true now, especially with the current health situation, where people are having to be distant socially from other people. And we always have this conflict with our devices right now. Do I look at what's in front of me. Do I pay attention to this person that's in front of me? And do I kind of have my awesome experience of the real world? Or do I go into my digital world, because I have a lot of information to manage there. I have a lot of people who I can only stay in touch with digitally at any given time. I have to choose right now to look at my phone, to look at my watch, or to look at the real world. And every time I'm doing one, I'm not doing the other. And I think there is this potential to not have that conflict anymore. Somebody that doesn't have to feel like they have to put their phone on vibrate or lock it in a safe or something like that in order to be present. What if you could be present and connected and only get interrupted with what's essential, and not get sucked into the phone vortex that you have when you go to check a notification, you went up an Instagram for an hour? But what if you could have all the benefits of the connection of glasses, but without being distracted from what's in front of you right now, and bring the best, simplest parts of your digital world into your reality, in a way that wasn't in conflict? And I think that people could be more present and more connected with everybody, but also their experience of the real world wouldn't really be effective. And I think that would just be so cool if it could move in that direction. And that's-- I think that's what a lot of people at North are very passionate about, is not giving us another tech device, but essentially giving us a way to not have tech be so distracting.

     

    Alan: I love it. And we have to be very careful not to let tech completely envelop us. So that's wonderful. Thank you, Stefan.

     

    Stefan: It's great. Well, thanks a lot for having me on.

     

    Alan: Oh, it's my absolute pleasure. Thank you so much. And thanks, everyone, for listening. Please hit the subscribe button so you don't miss any future episodes. If you want to learn more about North Glasses, you can visit bynorth.com. This has been the XR for Business podcast.

    38 min

About XR for Business

From the publisher's feed

Meet the leaders who are changing the face of virtual and augmented reality