XR for Business

The current generation of 4G devices are great, if you want to chat faster, share photos, or stream a movie on the go. But for real-time spatial computing technologies — like XR, for example — that just won’t cut it, especially when it could mean life or death. Terry Schussler is the director of Immersive Technology at Deutche Telekom, and he’s working to bring 5G into the XR domain, and expand the capabilities of mixed reality technologies.

Alan: Today’s guest is Terry Schussler, “entreprenerd,” technology architect, passionate software designer, writer, speaker, trainer and all-around awesome guy. As a software innovator, Terry’s focus has been making software smarter for users, while leveraging technology to enable new forms of communication. During the development of over 200 commercial software products, reaching over 50 million users on desktop, mobile and tablet devices, Terry has delivered numerous technology innovations; artificial intelligence and consumer products, multimedia, hybrid online/offline CDRoms (what’s a CDRom?), interactive multimedia on the internet, real-time character animations, factory-to-consumer personalized plush toy design, just to name a few. A number of his products have been category creators, opening up new markets with long-tail monetization opportunities. If you want to learn more about the company that Terry works for, Deutsche Telekom is at www.telekom.com

It is with great honor that I welcome Director of Immersive Technology at Deutsche Telekom, and founding member of the Open AR Cloud, Mr. Terry Schuster. Welcome to the show, Terry.

Terry: Thanks, Alan. Nice to have the opportunity.

Alan: Thanks so much. It’s really a pleasure and honor to have you on the show. And I’m just going to dive right in here because I think the people listening really want to get an understanding of how this technology can be used for them. So to start it off, what is one of the best XR experiences that you’ve ever had?

Terry: One of the best experiences I ever had was actually realizing that the technology can be used not just to make people money, or to provide education, but to actually save lives — that it can really be transformative. So a company which unfortunately is no longer in business, ODG, created an oxygen mask for pilots, which allowed the pilots to operate a plane using augmented reality when the cockpit was full of smoke. Seeing that product developed and come to fruition really got me thinking differently about the importance of utilizing these types of technologies to increase human safety and save lives, as well as provide all of the obvious benefits that we’re used to.

Alan: Wow. That is… how do you even… that’s a show-stopper. I had Mark Sage on the show, and he was talking about how firefighters are using this technology for heads up displays, and military are using it for being able to see in the dark and creating that visibility layer. Can you maybe talk a bit more about this, this mask that can help pilots in a distressed situation like that? Because there’s so many ways this technology can be used to save lives. I think we should dig into that.

Terry: So, ODG co-develop this with… I think with FedEx. FedEx, I think, had two flights which had crashed due to a cockpit filled with smoke conditions that prevented the pilots from being able to properly control the plane. They made a decision to look at how they can utilize technology, a heads-up display technology, using AR to give pilots the visual controls that they need to continue flying the plane, even if such a situation happened. And they actually had a live demonstration unit at the Augmented World Exposition last year in Santa Clara, where you could try the mask on and actually see what the experience would be like. 

It really got me thinking differently about some of the goals that I want to achieve personally with XR devices, and how I’d like to utilize them. And the importance of the general technical work I do in terms of creating low-latency experiences, and how they can be used to combine together to create better human safety conditions.

Alan: So, you talked about low-latency experiences in creating this human connection, or human tools that we’re going to be able [to use] to save lives. You work for a telco, and all of the telecommunications companies now are really pushing this 5G wave. So maybe you can speak to how 5G is going to benefit augmented reality, mixed reality, and what are some of these low-latency experiences that businesses will start to tap into?

Terry: Sure. Most people see 5G as, if you were to ask the average person, what does it mean to them? It’s really about bandwidth; the speed at which things could be downloaded. And that certainly is a huge value proposition. But one of the most important things that 5G also does is, it provides a higher reliability of service, a more consistent availability of service. Because of the technology, it’s able to support 100 times more people in the same area, having consistent access to the Internet in a mobile context, or in a fixed context. And that’s really important.

Alan: It’s interesting, we should punctuate that right now, when you cut out a couple of times during this podcast. 

Terry: [laughs]

Alan: Technology, we can’t wait! 5G, come faster!

Terry: That can happen. Of course, when you go to environments where there is a lot of people at one place — a coliseum sporting event, or a shopping mall, or–

Alan: Coachella. 

Terry: Yeah, exactly. Coachella. Then connectivity becomes even more problematic. And certainly, if everybody’s trying to uplink a live video stream of something that’s going on, then it becomes even more overwhelming. Those are a context in which 5G can provide value with certain technologies that we’re utilizing for mixed reality, where we’re taking the camera feed and feeding it back, and processing that camera feed to make decisions that provide spatial mapping information, things like that. And this consistent higher bandwidth connection is important.

Alan: For people who are listening, let’s take it back to basics. Why would a company need some sort of augmented reality, where it’s capturing the world’s data? A lot of people think about AR as, “I can hold out my phone and see a Pokémon,” or “I can see a piece of information overlaid as a digital layer.” But most people don’t realize that it’s the camera that’s capturing as much or more data around the world to create this conceptualized data. And so, capturing data and uploading it to the cloud is probably as important — or more important — than the data being driven back to the device.

Terry: Yeah. And there’s technical limitations that we have today with the on-device sensors that can be enhanced or addressed through cloud-based technologies. So, for example, one company that is in our incubator, hubraum, is a company called 1000 Realities, and they’ve developed a purely cloud-based slam approach, where they take video stream directly off of a device, run it to the cloud, to our edge compute infrastructure, and then process that video to create a feature point cloud, or to localize a user against one. That allows devices which don’t have the compute capabilities today — lightweight AR headset devices — to have the kind of capability that a higher-end device like a Hololens 2 or Magic Leap might have, or even better in some cases.

Alan: The 1000 Realities team is doing some amazing stuff. I think they’re using the VUZIX blade or something… they’re using different hardware devices. But let’s dig into what an actual example, of why you would need to map out the world in real-time like that. Can you think of any real business use cases for that?

Terry: Sure. I mean, positionally, things changed within an environment. You need to be able to track objects in real-time. So, having the ability to perform a non-preprogrammed or dynamically-applied enhancement of information overlay on top of real world objects is super important, in tons and tons of environments: factories, outdoor logistics, and so on. Think of retail, for example, where everything’s moving around. If you’re trying to look at a planogram section of a retail space, you need to dynamically be able to compute what’s there, what’s not there, whether things are in the right place. That’s going to be a changing context over and over again. To be able to map that information in real-time, process that, and then get dynamic overlays on top of that is very valuable for those kind of contexts.

Alan: It’s interesting you mentioned retail, because that’s something that we focus very heavily on — retail and e-commerce, and general marketing as well. You know, something that happened on the weekend — we mentioned Coachella quickly there —  Coachella had AR navigation at the festival this past weekend, and next weekend as well. They also had an AR stage, where you can hold up your phone and see an AR activation. You can see the NASA space shuttle flying through the Sahara tent, which is pretty awesome. 

But they always have these bandwidth issues where, if you get a few hundred people on the system running it, it starts to slow down. But if you get ten thousand people, then it grinds to a halt. I think people don’t realize, fully, the limitations of 4G. They think, “oh, I can watch a movie on my phone. It’s fine.” But when you start to get into these real-time computing scenarios, like firefighters, or police, paramedics, where they need real-time data, and it can’t crash or lag because there happens to be 10,000 people in the place.

Terry: That’s right. Reliable quality of service is super important in those kind of contexts, and that’s a big value proposition that 5G puts on the table. It not only provides the higher bandwidth, but it ensures more spectrum density. So, more people in one place aren’t going to create this cascading failure problem. 

Alan: So, spectrum density is a thing.

Terry: Another really key area that ties to both of those is the idea of precise positioning. Today we have mostly the use of GPS systems, which are highly imprecise in certain contexts, and useless at others. GPS is supposed to be accurate to roughly 4.9 meters, but it’s not. For example, if you’re in downtown San Francisco, and you’re calling an Uber, you’re going to have to manually place on the Google map where you are almost every time, because it’s going to be off by half a block or more as to where your real location is. This kind of problem is going to be exacerbated when we try to get highly-accurate augmented reality content overlaid on the real world. 

We need to more accurately know where we are outdoors and indoors, and we can’t afford the inconsistency. The quality is in the consistency of positioning. This is an area that’s very actively being researched by us, as to how we can layer precise positioning on top of 5G infrastructure, so that we can get the positioning accurate to at least a meter indoors and out, and ideally a lot less than that — a lot more accurate.

Alan: I’ve seen some startups that are working on exactly this, trying to get that down to centimeter accuracy. It can be done with Bluetooth beacons and that sort of thing, but it’s not really practical in large facilities and large public spaces. But what I’ve seen is working fairly well now, is using landmarks. Using the visual camera to lock into so within five meters, you know where you are. Based on that, plus the visual marker, you can really narrow that down. Is that something you’ve seen executed well?

Terry: Yeah. There’s a company, for example, called Sturfee, which has developed an approach using satellite imagery. So, you would use your head-worn AR glasses to take a quick picture from the camera. Combine that image, along with your lat/long information, and send that off to their server. Then they’re able to process on looking in a radius around where you say you are, based on the GPS information. I think they do a search for around 30 meters, and then they’re able to figure out where you are using the image. 

They’re also able to figure out your elevation on your azimuth. They can then calculate on a three-dimensional mesh along the surface of the objects that you’re looking at in the real world, the sides of buildings. The street, of course, is the easy one, but we’re doing that against the geometry of the buildings is pretty interesting. Then you can very quickly do real world, outdoor world scale AR kind of stuff; putting signage and maps and things like that onto the surface of the infrastructure around you.

Alan: That’s interesting. Google, about a month ago, announced their AR navigation system. It does not have that kind of accuracy. So is this something that maybe would be a Google acquisition, to build into their Google Maps?

Terry: Yeah, I think everything is a potential Google acquisition these days [both laugh]. I mean, you see a lot of roll up in the industry right now. Technology roll up, between all the players. Niantic, Facebook, you name it. So, absolutely, I think it’s an innovative technical approach to use the visual positioning approach to things, but it only works in the outdoor context. So, technical problems, different technical solutions. What is very interesting to me for the real world, for business use cases, I don’t see a very good, cohesive outdoor/indoor solution quite yet. I’m a believer that cloud-based solutions like that of 1000 Realities will get us closer to that, where we can have one consistent technological approach that we can use to navigate you in world-scale AR to a business, bring you into the business, and then give you internal navigation and spatial mapping at the same time. Right now, that requires two separate solutions.

Alan: We actually built an AR navigation tool, and it works great outdoors and indoors. It works well, but the way we did it was, it was specific for location. So theme parks and malls and stuff like that. What we were using — and what we are using — is beacons. When you’re outside, it uses GPS, and when you’re inside, it knows where you are in a GPS, and then uses the beacons to triangulate that milimeter-accurate precision.

Terry: Right.

Alan: But I haven’t seen anything that has really been able to say, “hey, this is the penultimate of this.”

Terry: Yeah. Right now there’s vendors like 6D.ai which have really tremendously awesome spatial mapping tools, that require a lot of on-device horsepower to perform their task. You see the envelope of what could be in the future. As equipment and devices get more performant, that kind of technical requirement will become more commonplace available. But you also see devices like the Magic Leap, which are technically really awesome, but don’t work in an outdoor context, not just because of the display components — the optics package — but because of the sensors. You can’t scan at scale outdoors with those devices.

Alan: I wonder if you could do a combination of putting ARCore and ARKit capabilities, mixed with GPS, mixed with the Magic Leap, to give it all in one. But now you’re throwing in a ton of junk. So really, what it comes down to is 5G and cloud computing.

Terry: Right.

Alan: None of these things can really run without it.

Terry: Yeah. A lot of the work that I’m doing at Deutsche Telekom is looking at, how can we enable the Holy Grail device to exist, which is this really light, consumer-fashion-friendly, all-day device. The key to that is that we have to reduce battery drain, and move all the compute off the device that we possibly can. And to do that, we can either tether it to a phone, or we can put the compute on the Internet. Or we can do both. 

I really think that we’re moving to a mesh computing world, where we use all the compute cores, if you will, that are around us in our personal area network of devices. You’ve got your watch with some compute. You’ve got your phone with compute. Maybe you have compute on the headset, and then you’ve got compute on the network, both on the edge and in the backhaul. If you mesh all that together, we can start to shift the burden off the top of your head, and onto the network more and more. That allows these devices to get smaller and lighter, and still be very functional. You know, not to have all the tradeoffs. 

You mentioned, for example, the VUSIX Blade, which looks like it’s a stereographic device, but it’s really just for one eye. The right eye. It’s a great device, and it’s lightweight, and it’s relatively inexpensive. But because it takes a lot of tradeoffs in terms of its compute capabilities, certain business use cases may not be as viable on it as it is with other devices. When you start to integrate the use of cloud compute and technologies that run on the cloud — especially on edge — then you can start to offset some of the compute reduction that you put on the device with the network, and it starts to enable the device to be more and more capable. Like I said, in some cases, more capable than devices with the built-in sensor arrays and the inside track.

Alan: One of the showstoppers that I saw at CES this year was the NReal glasses, from a developer from Magic Leap who left to start his own glasses. He basically took the basics behind delivering three-dimensional AR with one camera, using ARCore, I guess, and then run the computer down to the equivalent of a cell phone pack running Android. I thought that was a really unique way to get some of the weight off the headset. 

But what you’re saying is that having the compute power on the glasses, and then maybe the phone, and then cloud, and then basically edge computing — all of it together — that’s going to need some sort of open frameworks and collaboration. One of the things that you’re involved in is Open AR Cloud. Do you want to talk about that, and what that means to businesses?

Terry: Yeah. So, it’s hard to understate how — or overstate, I should say — how important having a digital representation of the real world, spatially, is going to be. What we call “the cloud,” it’s going to be the foundation under which we… I’m sorry, on top of which we build tremendous amounts of spatial computing applications. We need to know the details of the geometry of the real world, so that we can position things. But we also need to understand it semantically, as well — What it is, not just where it is in house, what size it is. 

The Open AR Cloud Foundation is focused on looking at different categories of use cases, and creating open standards that all the industry players can engage with, so that we have a consistent way that we can utilize these different technical approaches to solving some of these problems. 

I mentioned, for example, the fact that currently, you really have to use different hybridizations of technologies to work indoor and outdoors with spatial mapping. What we need, though, is a consistent — as we refer to it as — a single index method to be able to, say I need a map, based on this: where I am now, or where I need the map for. Different vendors use different approaches today, to provide that indexing into the maps that they generate. And it creates a lot of havoc for people to design applications, to not have a consistent approach, a consistent method for indexing. Kind of like the Dewey decimal system for libraries, to be able to find a book. It’s the same value. If every library had its own different indexing methodology, it would make it really hard for people to go from one library to another and be able to locate books. 

That’s the goal of the foundation, I think, in many ways: create these sort of standards. And there’s a nice integration of other standardization groups which are also, in their own right, trying to create some uniformity with development. For example, if we look at the AR headset market, there’s already silos, right? We’ve got the Hololens silo, we’ve got the Magic Leap silo. We’ve got the Android-based AR headset silos. Apple, when it comes out with its product, will create another silo. But maybe as developers, we want to be able to build applications that run across these devices, and not have to develop them over and over and over again. 

Multi-platform deployment of business logic and code and graphics has been something that’s been a passion of mine since the 80s. Today, without tools that can do that — like Unity — we wouldn’t see as much proliferation of solutions for business or consumer. It’s very, very important, as we move forward, to have that.

Alan: I couldn’t agree more. You mentioned Unity, and Tony Parisi has been working in the WebAR space forever, and really pushing forward. He’s actually going to be a guest on the show as well. So, this is not an overnight fix, and it’s not something that is going to happen overnight. But I think, from a business standpoint, if I’m a business owner, and we’re talking about open AR cloud, and we’re talking about edge computing — what does this mean to a typical business? Because, as we’re doing this interview, it keeps cutting out a little bit. Let’s just unpack this: if we can’t figure out how to make a podcast record smoothly, why are we even trying to make glasses that compute in three dimensions? Because if we can’t figure out the simplicity of a podcast stream — and this is what people are asking me — “why would I get into AR, when I’m just starting to embrace mobile apps?” Maybe you can speak to the transformative power of these technologies as a business owner, as a business person.

Terry: Well, I mean, there’s two parts to this, right? We’ve been in the early days with AR devices, and the challenge has been that, if you are tied to a particular vendor and their vendor’s ecosystem, then it becomes really, really hard for you to take all of the investment you’ve made in building business use cases, and moving them to devices which might be better suited to the use case. 

For example, the Hololens. When people started building applications for the Hololens to do — things like remote maintenance, or remote support — the same applications become very difficult to move to a lighter-weight device that’s perhaps more durable, and better-used in an industrial environment – like, say, a Realwear HMT-1, or something that’s lighter weight that fits, and is more comfortable to wear on the head for an extended period of time. It’s super important that we have some standards that allow us to take the core business logic and the content, and move those projects across to different tangible devices, so that the businesses can adapt and utilize that. Not only for the use case that they’re building, so it’s better for that use case — but also, to look at it from a CapEx perspective. The Hololens is a great product, but it’s $3,500 USD, and companies can’t afford to necessarily give each and every employee [one] — even with a great ROI that might be provided. They may fail to capitalize that much expenditure. So, it becomes very valuable to be able to move your software ecosystem to a device that might be slightly less functional, but also, a lot more affordable and deployable across a wider range of people.

Alan: It’s interesting that you touched on that, because about a year ago now, I think, Microsoft moved the Hololens out of their devices division and into their Azure — or cloud-based — computing division. And I thought that was a really smart move, because once they realized the power of this technology, being able to synchronize that with the business systems that are already in place is vital. So, being able to say, “okay, you’re using a Hololens, and that’s great for these really high-end jobs. But, that same information that you’re using can be used with a smartphone now, and with another pair of glasses.” I think creating that standard, where it can be used across anything, is absolutely essential. 

Our company, Metavrse, we’ve always taken a completely agnostic approach to everything, where we said, “okay, it doesn’t matter what headset or what technology it is; what is the problem solving? How do we take this technology that is right for you, and deploy it, but also in a way that future proofs what you’re working on?” Because let’s be honest, the stuff is changing weekly. 

Our company — and that’s another question that keeps coming up from companies — how do I even get started? What would your recommendation to a company that’s looking at, maybe it’s an enterprise company and they’re going, “we have a factory, and I see these case studies from Boeing how they’re seeing 36 percent increase in efficiency.” What’s the first step for people, in your opinion, for these companies to get into it?

Terry: Today, we see a lot of companies that are doing pilots, and improving the ROI on enterprise business-to-business solutions, specifically looking at the situations. Remote assistance is a really common one. It’s almost the common denominator use case now, because the return investment’s very clearly measurable; you know that you can save money not having to fly an expert from one physical location to another. Same reason that we use Skype or WebEx or Zoom or whatever, but we extend that communication with augmented content, presentation, annotation, information displays. That the over-the-shoulder support, it feels like they are literally over-the-shoulder. Those kind of use cases that are very straightforward ways for businesses, which are in the service industry, or have technically-complex products that need to be serviced, to invest in XR. 

I think that most of the devices that are in the market today are capable of delivering really good value in that. But there’ll always be a need for use case-specific implementations on the device, depending on your context. If you’re in a hazardous environment, if you’re sitting at a desk with large, complex machinery right next to you… different needs. Sometimes you need something that can work in different lighting conditions; very bright light versus very low light conditions. Maybe you need a headset that can have a flashlight turned on, so that you can illuminate the area as you’re looking at objects in a basement or underground location.

Alan: Or night vision for the U.S. military.

Terry: Exactly. Look at the Hololens 2, in the military version/adaptation. And there’s a number of other companies that have made a good living building XR head-worn devices specifically for the military. I mentioned the flight mask for FedEx that ODG had built. I don’t think, for business, there’s going to be a single device that is going to serve all the business use cases. You have to look at the ones that are… what I tell a lot of businesses to do is start to evaluate the environments in which the devices need to be used before you start building the software, because the software is going to be a lot easier to design than trying to pick different hardware. 

For example, do you need to build or use the device indoors and outdoors? Do you need to be able to use it in different lighting conditions? Does a device need to be shared by multiple people, because you can’t afford to put one device into the hand of each individual user? What kind of mobile device management requirements do you have? What kind of security requirements do you have? All of these have to be thought about before you start picking hardware. 

Unfortunately, a lot of times, I see people pick the device first without really thinking all those out, and then they find that they’re kind of in a rut now, because they know now they’re stuck with a specific software development platform, building for a specific device’s capabilities, and they can’t really build the solution they need. It’s not practical. I mean, all these devices have great value. I work with all of them universally. But I wouldn’t take the Magic Leap and build a solution for it that requires being used in a large, open factory space, because it’s just not designed for that.

Alan: Why is that?

Terry: Because it’s both a combination of the way it displays things — the lighting conditions under which it can work — and also the sensor arrays that are on board. It can only scan a certain distance in front of you. So, if you’re trying to do things and you’re in a giant room — a cavernous room — it won’t build the final walls that are in front of you, because they’re too far away. You’ll have to walk around the area and map it manually, and it’ll take a really long time to do that. And also the environment… for example, if you have a lot of reflective white surfaces, or you have glass windows, or things that are less feature-oriented to differentiate content of the real world around you, it’s really hard to map. Magic Leap isn’t going to be that great at some of those environments, at least not currently.

Alan: All of these devices have their pros and cons. The Hololens one, for me, after about five minutes of wearing it, I got a headache, just because of the weight distribution of it. The second one, they fix that and they’ve addressed it, but they all have their limitations. 

I think one of the things that’s across all the conversations that I’ve been having is security.

Terry: Yeah.

Alan: You mentioned device management security, shared by multiple people. One of the things that came up in one of the conversations is eye tracking is going to be more and more prevalent in these headsets. And once we have really accurate eye tracking and head motion tracking — because the device on our head — you’ll be able to use things like gait, retinal scanning, heart rate. These are different biometric markers to enable security at a different level.

Terry: Yeah, absolutely. Magic Leap has some great advances in that, as does the Hololens 2. And you’re going to see more and more use of the value of that, as we move forward to enable on the contextualized and personalized information display for the user, automatically recognized by a mixture of biometrics, like you said. It’s really about reducing the friction on the user experience, so that they can have an easier way. Just pop the headset on, and maybe even have a general profile that’s stored in the cloud someplace that can be brought down to that particular device, so that it’s not just this exact device that they have to use, but it’s a device of that type.

Alan: Yeah, you’re absolutely right. One of the things that we’re working on, it’s a project — I can’t talk about the details — it’s being able to use it as a medical diagnostic device. So, being able to send this device out to remote areas that physicians — it’s either very expensive, or not possible to get physicians to these remote areas — being able to send this device out, capture the medical data from it, and then either transmit it through the cloud, or send the device back. But, how do you secure that data? How does it transfer? How do you make sure that the individual using it, like you said… the onboarding is a hard thing. If I put on a Hololens and it doesn’t have Wi-Fi, I got 10 minutes of trying to mess around just to get the Wi-Fi working. Being able to take out those onboarding challenges is really key. 

I think we’ve only scratched the surface, as an industry, on what eye tracking can even do, with obviated rendering and being able to identify and approve people. It’s incredible, what this technology is going to be able to do.

Terry: For businesses — maybe even as much or more so than consumers — it’s going to be important that the learning curve and the friction points that are involved in getting somebody [to] be able to put the glasses on, and start making practical use out of them needs to get less and less. What we’re learning when we do this for businesses will be readily applicable to consumer-grade devices that will come to market over the next 12-18 months.

Today, the most powerful devices are these all-in-one devices, like the Hololens 2 and Magic Leap-created one. But as we move over the next 12-18 months, you’re going to see more and more of the tethered devices, where they’re leveraging a smartphone and it’s compute capabilities and it’s sensors with lightweight devices. And the flow with which those kinds of experiences happens needs to be super, super easy. Ideally, I just want to be able to put the glasses on and have it start to do things for me without my having to be trained in a special class. 

Right now, it’s not there yet. We’re still finding that we have to educate people on the use of the device in a generic way. Then we have to educate them on the applications. And every application is a completely different approach to the way it spatially displays stuff. So, every time, it’s a massive learning curve. That’s going to go away in the future, but it’s going to take us a few years. It could take us longer to solve the inversion of the ecosystem issue, where we go from an app-first mentality to a people-first mentality, in the way that we design software. Then it’s going to take us time to actually get to consumer-grade device.

Alan: So, in your opinion… I mean, you’re right in the thick of this thing; you’ve created 200 different apps around the world, you work for one of the world’s largest telcos. In your opinion — you mentioned 12-18 months — when do you think consumer-based AR is going to start? When I say start, I mean hit the market where people are actually buying, and not Magic Leap being sold at AT&T stores because, you know, that’s great, but I bet you they’ve sold about 10 of them. But when do you think this is going to start to kick off? The big question mark is Apple, and what they’re going to do. But can you speak to what you think your timeline is around consumer adoption, and where we’re looking for enterprise versus consumer in the next 10 years, or five?

Terry: I think, in terms of the device, the use cases, as I said, the transition now is going to go from the all-in-one devices to the tethered devices. So there’s a few companies like Dream World, NReal, ROKiT, which are coming to the market with really high-quality tethered solutions, where you plug your phone in and leverage that compute, and that will allow the price point for the headsets to go down significantly from where they are today, to under thousand dollars — maybe well under a thousand dollars — and still have a lot of the functionality needed for business use cases. 

That will be the adoption boom. It won’t be for consumer use cases. It will still be for business — or, as I like to say, time-durative B to B to C type use cases. Like, I’m going to go to a sporting event; I want to put the glasses on, I want to wear them for two hours. I’m going to go walk around a city center on vacation; I’m gonna wear the glasses for a few hours. I’m going to go to a museum; I’m going to wear the glasses for a couple hours. So these time-durative use cases will become very valuable in driving adoption rates on the devices themselves. 

In the end, I see that happening in the next 6-12 months. I project that by the end of next year, Apple will make its entry into the ecosystem, and that will be much more of a straight-up consumer play. I don’t think they’re going to look at it as a solution for businesses at all. I think they’re going to go at the opposite end, which is what Microsoft has done. Microsoft said, “we’re going to own the enterprise space.” On Apple, I think, “we’re going to own the consumer space, and we’ll let everybody else play in the middle.”

Alan: Well, if it’s anything like the iPad — they came out with a consumer device that had far-reaching capabilities in enterprise and business.

Terry: I absolutely think there’s that. What you can use it for, and what they position it for initially, right? So, absolutely, there’ll be a lot of envelope-pushing, in terms of the categories of use cases that will be built for the device when it comes out. I just think that right now, we’re in this sort of transition from the all-in-one to the tethered, and then Apple will be playing in the tethered. From my perspective, it’s an ecosystem of devices that work together. You’ve got your Apple Watch. You’ve got your next-generation Air Pods. You’ve got your 5G-capable Apple iPhone, and then you’ve got their glasses. I think that’s going to be the ecosystem that we see when it comes to the glasses and bring it to market. My gut tells me, as a developer who’s been doing work with Apple since ’84, that they’re going to launch at the WWDC, to get the developer community ramped up and building use cases and applications on top of it. I just have a hard time imagining them waiting until 2021 to do it.

Alan: Really? Because my original prediction was mid-2021 when they announced, and then 2022 when they launch. So you’ve kind of hyper-accelerated my…AH! Oh, crap! Anybody who’s listening: you better hustle now, because once that hits, the world gets crazy.

Terry: I’m an optimist, and I have to admit that, within my group of colleagues, I’m as optimistic as it comes when it comes to devices. And I know I’m pushing the envelope a little bit. There’s also some convergence going on, with the rollout of 5G, starting now. Granted, it’s early days. If you stand in the right corner of the park in Chicago, you’ll get 5G radio on Verizon. So it’s still very limited. You have to be in the Mall of America in Minneapolis to get 5G. I think it will be in the Verizon store itself, actually. So, there’s limited access right now. But as we move forward, and all the telcos get to play, you’ll see more prevalence in there. And I think Apple’s waiting, because they’re not in a hurry. They’ve got to wait till there’s a market.

Alan: They did a really good job with introducing ARKit with the acquisition of Metaio, and then turning that into ARKit. And I often say this in my talks, is that ARKit is like the training wheels of spatial computing. “Here’s the device in everybody’s pocket, it’s fully AR-enabled: go out and build something cool. Maybe it’s a game, maybe it’s an experience, maybe it’s a marketing thing. But you have the power of the future technology in your hands. Start programming for it, so when the glasses come, you’re already able to program for these.”

Terry: That’s right. That’s exactly correct. And where we move beyond the preponderance of measuring apps built on top of ARKit, to really seeing a breadth of use cases and a lot more integration of AI on camera vision processing. We’re starting to get not just spatial mapping, but semantical mapping of the real world done. And that’s going to be really exciting, because that really changes the game. That’s why the work that we’re doing at the Open Air Cloud Foundation is so important, because when you have some standardized resources that you can tap into. No matter how big the companies are , no one company is going to be able to own the creation of all maps. So, there’s a need to share and collaborate and work together to deploy as a layer of services with these maps have been built to contain. And absolutely, on top of that, we need to understand what it is that we’ve mapped. We understand the details, the nuances of things, like the type of material that things are made out of. This is concrete. This is plastic. This is wood. This is rubber. That’s going to start to become more tangibly important to the types of applications that we see built.

Alan: It’s crazy, the stuff that’s going to come through the camera. I want to ask you one final question: what do you see as the future of VR/AR/MR/XR, as it pertains to business?

Terry: I think that, transformatively, all the software that we’re used to operating on a desktop computer will have to get redesigned to be spatial computing, where we’re going to think differently. The vision that Meta had about getting rid of the desktop display, and turning to virtualized space, is going to be part of the world that we move to. And so, software will have to be rethought and redesigned and reengineer for that kind of paradigm.

Your company has access to AR technology. Great! …now what? Have you thought about how to effectively implement that technology to solve a problem? Or are you just planning to develop a neat digital gizmo, release it to the world, and hope folks will use it? Zappar CEO Caspar Thykier joins Alan to discuss his personal Three Cs for Success in AR, and how to think critically when developing AR solutions.

Alan: Today’s guest is Caspar Thykier, CEO of Zappar. Caspar has an extensive background in advertising and marketing, and has been lucky enough to work with some of the most successful companies with the biggest blue chip brands, surrounded by the best people. Now he’s spending all of his time in the new world of spatial storytelling — or augmented reality, as some call it. He’s helped so many companies get to market, but the one that he’s working on now that is at the forefront of all this technology is Zappar. Zappar is an augmented reality platform, and a creative studio, all rolled into one, and they’ve been doing augmented reality since 2011. Zappar are really specialists in augmented, virtual, and mixed reality and they’ve been leading the way for over seven years. They work with some of the biggest brands in the world, to meet their marketing and commercial objectives by adding a layer of interactive digital content to their products, packaging, point-of-sale places, and physical marketing collateral. Basically, they turn all passive print into always-on media channels that brand owners can control, which is pretty incredible and amazing stuff. You can learn more about Zappar by visiting them at Zappar.com. “Zed”-A-P-P-A-R dot com. Or, if you’re American, “Zee”-A-P-P-A-R dot com. 

I want to welcome Caspar to the show. Welcome to the show, Caspar!

Caspar: Thank you so much, and thanks for that introduction. Especially making a difference between the US and the UK pronunciation of “Zee” and “Zed”.

Alan: Well, I’m Canadian, so we share the “Zed.”

Caspar: There you go.

Alan: Well, thank you so much for joining me, I’ve been really excited about this podcast interview. You are literally a pioneer — not only a pioneer but a leading pioneer —  in this industry. Zappar, you guys have done work with Wal-Mart, you’ve done work with 7-Eleven, with tons of alcohol brands, with packaging brands, with consumer packaged goods. Let’s just list off a few of the ones — the highlights — that you guys have done recently, and then we’ll dig into what this technology does for a brand, how they can use it, and how they can get really involved.

Caspar: Absolutely. Yeah, that’ll be great. Maybe to preface that, I think how we’ve come to be working with such a diverse range of clients and partners really stems back to where we started. 

As you said, it seems a long time ago now, back in 2011, because we were a company that was self-funded then. When you think back then, you had to explain pretty hard to people what AR was, and we had to make sure that any of the projects we worked on were clearly revenue-driving, revenue-generating for the business. So at that time, we looked pretty horizontal and shallow across lots of different industries. I guess we looked into other black folks of contacts to see where we could fish first. That meant that we ended up exploring the opportunities for AR across entertainment – we did a lot of work with the Hollywood studios at that time, with Warner Brothers and others, both for theatrical marketing and consumer products — but then into retail. Working with retailers across the world, be it here in the UK with Asda, Wal-Mart in the states and 7-Eleven, or Woolworth’s in Australia, et cetera. 

Then into connected packaging, and that’s something that we certainly see coming to have to do more of in the future. But with brands like Nestlé or Unilever, etc. Learning, training, and development became another area of real interest for us. The amazing power of augmented reality to help people with active learning through doing, we saw that to great effect in the financial services industries, and indeed, healthcare as well. Then into work, we’ve done in education as well, and into conferences and events. And you’re right, that we really have gone across an awfully broad spectrum of different services and clients. And I think that’s because of where we came from.

Alan: “We weren’t just some VC-funded company that raised hundreds of millions of dollars and then built an entire tech stack that went sideways.” But we’ll leave it at that. 

I think one of the things that you guys have always done is looked at profitable ways to use this technology — and not just profitable for yourselves, but for the businesses that you consult with and make things for — and I think that is a really big difference. A lot of startups today are getting funding based on the demo, and they’re not really thinking about, “how do we actually make money with this?” And so there’s this kind of weird venture capital-backed scheme going on where it’s really not in the best interests of the business user as well because they’re trying to get users or whatever, but they’re not really trying to bring real business value. I think that’s what you do best.

Caspar: Well, again, thank you. I think that’s because we’ve always had this mentality of trying to live within what AR is capable of now, rather than what it’s capable of in the future. And I think the thing is that, again, even back in 2011-12, there were some amazing things that we could do with AR. In fact, there is still some of those demos that we show now, from back then, that still wow people. It’s living in that mindset that still, for the majority of people out there, they haven’t really experienced AR. Beyond, I guess now, I’d say Snapchat, etc. And it’s the small experiences that still surprise and delight, or can be used to inform and instruct. 

I think that process of living within what the technology can do, and really thinking about the moments of assistance for AR, is the important thing, rather than, “what is the future thing that it will be able to do tomorrow?” And the more that you–

Alan: That is a really, really good piece of advice. We all get caught up — especially the people in the industry — we get caught up in trying to push the limits of the technology. We actually build a platform for people to make AR quickly, and the thing that they did the most, which was shocking to us: They didn’t import 3D models. They didn’t import 3D buttons. They didn’t import anything. They literally just put videos overtop of their images. 

Caspar: Yeah, that’s it 

Alan: It was amazing. I mean, 99 percent of it was just videos on top, and that was mind-blowing for most people.

Caspar: Yeah, that’s it. That’s it. It’s understanding where people are in that expectation curve of those things. 

We have this really simple mantra that we talk about a lot, which is called the “Three Cs for Success,” but for AR. And funny enough, none of those Cs are really about the technology. They stand for: understanding the Context; the Call to action, and, of course, the Content. And actually, that context is really thinking deeply about that moment of assistance. That end user, what are they doing at that point in time, when you’re asking them to take out their phone and point it at whatever objects that they’re about to augment? And in fact, what’s happening at that time? What’s going on with the lighting and the audio? Do they have Wi-Fi network connectivity? Are they likely to have the right amount of time to really want to engage with this piece of content? And those things become incredibly important because they actually frame that moment of assistance — how people are going to interact with this piece.

Alan: I want to put a pin in there just for one second, because I think what you’re talking about, putting it into context:, if you’re going to do an AR thing in a newspaper or whatever, a printed piece, where people may have their phones in their pockets or they’re walking somewhere, I think you really hit it there. Because a lot of brands don’t think about that. That is the one thing that they’re like, “we’re gonna make a poster [in] AR,” and they don’t realize people are not going to pull their phone out to point it out a poster if they don’t even know it’s there.

Caspar: No, you’re absolutely right. I think that’s because it’s very easy to have the idea of AR come into a business, and someone be told to go and do this discovery piece to find out what it’s for. And I guess if you staff that perspective, you’re almost seeing AR as a strategy in and of itself, as opposed to a facilitating technology to solve a particular problem. That context basis is absolutely critical to get right from the very start. 

But then there’s the call to action, as well. It’s incredible how important that is. Unless you tell people really clearly what it is they’re meant to do, and what the value proposition is that they get at the end of this experience, they’re not very likely to do it, frankly. Again, when we first started out, this notion that you can scan anything, anywhere in the world with your device, and have things come to life, itself is a wonderful prospect. But unless you’re at the point where everything does that — does it accurately — then how do you know what things to augment and what things you shouldn’t? You have to use some real estate in order to prepare people for that action, and make sure that they do it right.

Alan: It’s interesting that you say that because… do you know the 19 Crimes wine bottle?

Caspar: Yes.

Alan: It’s this famous wine bottle, where you can download an app and point it at the thing… but the app, nowhere in the app does it say “19 Crimes,” first of all. You can’t google “19 crimes app.” Second of all, on the bottle, it doesn’t say anything anywhere, either. So there’s no call to action. Unless you are specifically looking for it, you’ll never find it. That’s a very good point.

Caspar: Yeah, it’s fascinating. That’s a case study that comes up a lot. And you know what? They’ve obviously had great success with it, and the numbers speak for itself. It’s a beautiful execution. I guess what I do take my hat off to them is, making sure that AR was in the thought process of the design of that entire product experience from the very start, and I think that really shows.

There are other nuances that, I think, are interesting to look at. As I understand it, the size of that app download is in the hundreds of megabytes, which is actually quite a chunk of data to ask people [to download], and then the load times are quite long. This is the thing with AR, is that there are many things to think about when designing it. And you need to think really deeply about not just these three Cs we’re talking about, but the entire AR ecosystem, and how you’re deploying it, and distributing it, and getting people to use it. We definitely see that call to action as – fundamentally — that important thing to get right. It’s not enough to just say, “scan here.” You do need to tell people what that value proposition is. What is the point for them? What is the value exchange?

Alan: So let me ask you a quick question. Typically, you guys have done everything from training and consumer packaged goods. You’ve done poster, you’ve done in-store activations. What is that value proposition, that really tends to resonate with consumers? Is it a free thing? Is it gamification? Is it taking a photo? What is the one — or two — things that really resonate with consumers?

Caspar: It’s hard to give you a single answer for that, because it’s totally dependent on the context; the audience, the sector, the engagement. But you’re right in that… I don’t know, let’s take something like 7-Eleven, for instance, is a good example. There’s a brand, actually, who has been incredibly innovative in their sector, and has embraced AR as this always-on camera function within the 7Rewards app and very much seen as a way to not only surprise and delight users, but also, to give them tangible value through valuable 7Rewards points that they can earn. So not only do they get these digital experiences that they can enjoy and share, but they also have the opportunity to redeem rewards points that they can then use on their next shot. So there’s a very interesting dual benefit there, with both a digital and physical value add, if you like. 

I think that understanding what the offer is, if you like, really does depend on what the execution is, and what you’re trying to achieve. I tend to find that the photo share element to something, that is sort of a ubiquitous feature, as opposed to an endpoint in itself because it’s just become something that is so prevalent and almost expected. That is just one aspect of it that’s there if people want to do it.

Alan: Unless you’re turning the Flatiron Building into pizza and throwing up rainbows. I mean, that’s just pretty cool. [both laugh] The stuff that Snapchat is doing is really incredible.

Caspar: Oh, yeah, yeah. You know, I think their tools are fantastic. And again, a great focus there on how that can work within a specific social network environment. The social play. They’re doing a great job.

Alan: So you mentioned three Cs; We spoke about context. What is the user doing at the time of activation? What are their mindsets? Do they have their phone with them? That sort of thing. Call to Action; what is the value proposition? How do you tell them about it? How do you educate them? What is the third C?

Caspar: Well, that content. The content piece is really just making sure that whatever you’re developing couldn’t have been easier for someone to get, had they literally just gone onto YouTube or something they could have got more easily going through a normal app interface or onto a website. Because if that’s the case, that is obviously something that people are very familiar with. And so to ask them to go through the process of either downloading an app or opening an app to get this experience, it needs to be something that warrants that level of input and attention. So I think that’s just about, then, thinking about, how do we use spatial computing, and how do we design the UX and UI in order to deliver this short-form, “snackable,” bite-size experience that people want to enjoy, and may want to share — and certainly, provide some sort of information, utility, or reward? One of the things we’ve done as a business is try to design the tools, in order to make sure those content experiences can be as expressive as possible. 

We have this platform called ZapWorks that allows people to do that. The way we’ve designed it is to make sure that it can cater for all the different types of context, and all the different types of content, that any brand, or business, or indeed, individual hobbyist might want to use.

Alan: So, let’s unpack your tools, ZapWorks, because I think this is something that… you guys are a content studio. You consult with brands, you figure out what they want, and you design and build it for them. But ZapWorks is a little bit different, in the fact that you’re giving the power to design these incredible experiences to the agency, or to the marketing people, or designers, or developers. You’ve created a platform for them to make the magic themselves.

Caspar: Yeah, that’s correct. Very early on, our driving mantra and passion as a business has always been about, how do you democratize AR? We all know that it’s been around for decades, but the inflection point was when it was available on these hand-held devices, so we could create that connection between the physical world and digital devices. And we always knew that there was only so much of that content that we could — or wanted to — deliver as a studio, and as a business ourselves. Much more exciting was to be able to put those tools in the hands of everyone, and understand what they might come up with and what that AR experienced might look like within their sector, in their area of expertise, and in their business. 

It’s always been important for us that we ensure that the exact tools that we use here as a studio are available to everyone else. Now, I guess we have a nice first step proving-ground of those tools, to make sure they’re solid before we put them out onto ZapWorks. But yes, they’re there for everyone to use. We tried to make it so that it can cater for people of all sorts of abilities. We have this very simple notion — very, very basic level that we call widgets — where you can literally just drag and drop your media files, whether that’s video, or photos, or links, or whatever it might be. That’s actually arranged within the system, and you can publish and preview that instantly. 

There’s a second level, which is called Designer, which, I guess, is sort of similar to PowerPoint, in that you can have a bit more flexibility around controlling the design of the target image that you’re looking at, as we would describe it. You could arrange the content around that, almost create slides, in order to move between them. 

The, at the most powerful end, we have ZapWorks Studio. ZapWorks Studio allows any designers and developers with experience to create some truly incredible AR experiences. It brings into being things like timelines, importing 3D models, and having actions for those. At this point, you’re making many activities and interactions. It really is a very broad range of capabilities. They’re designed for the fact that, within education, sometimes we go into schools and we do work with kids as young as eight, nine or 10, showing them and getting them interested in both computing and spatial computing at that stage, all the way through to these really gifted designers and developers who want to make these complex and deep and interactive experiences, but making sure that all of these things can be downloaded over the air, that they are small package sizes, in order that they can work on the majority of devices, wherever that might be in the world.

Alan: I think we were reaching a tipping point. You mentioned that the packages of data are being set pretty small, but also, the phones are becoming more powerful, and even older phones are starting to be AR-enabled. I read a stat that, by the end of this year, they’ll be close to 2-billion AR-enabled smartphones in the world. And in two years or something, like, 3.5-billion smartphones that will have AR capabilities. As this market literally expands exponentially, how do you see the market for AR across the board? Where do you see the biggest upticks in this? Is is going to be consumer packaged goods? Is is going to be education? Where do you see the biggest…?

Caspar: Well, I think there’s a couple of things happening. There’s always been a pretty enormous install base, to be honest, even over this last period in terms of the devices available on the market. It comes back to that thing that there’s an awful lot you can do with AR now that doesn’t require the absolute top-spec devices. Great that they’re adding these capabilities, and the camera quality is improving, and uses of CPU, GPU,  battery life, and network, and all these things are all great. But I don’t think we’re coming out of a period of darkness, if you like. I think more important we find is, there are a lot of campaigns that we have to deploy in many, many markets around the world. And really, what you need to be more aware of is just data plans that your audience are going to be using, and Wi-Fi connectivity. Making sure that whatever you’re doing isn’t data hungry and can be downloaded pretty quickly. A lot of the thinking and planning is more around that area. 

Having said that, I think one of the really big inflection points that are going to open up the market is the advent of mobile WebAR. There’s obviously a lot of talk about that at the moment. I think that is absolutely fascinating, and something that we’ll be making some announcements around very soon, actually. And that–

Alan: Is that a “hint-hint, nudge-nudge?”

Caspar: [laughs] Yes, it probably was.

Alan: “Zappar acquires 8i! Dominates market in WebAR.” I don’t know — take that with a grain of salt. But this is exciting. You guys are going to be working in WebAR.

Caspar: Yeah.

Alan: We’ve presented AR to hundreds of clients, and I would say out of all of them, about 60 percent of them say, “do we still need an app?”

Caspar: Correct. Correct. And look, I don’t think it’s the death knell of apps. There are clearly some that where AR very mature in a native app infrastructure, and it works incredibly well. And there are certain brands and businesses who can command having that real estate on someone’s phone. There are many that can’t. I think when you get into the area of specific connected packaging — which I think is an absolutely enormous market — I guess the way that we tend to think about that is thinking of the trillions of products that are both on shelves and in people’s homes and the fact that they are all, at the moment, passive. 

What you can do with augmented reality is turn that passive print into this always-on media channel that, as a brand owner, you begin to control. And indeed, you can have a one-to-one conversation and relationship with your end user at a point where it’s quite a black spot, in terms of data collection, from the point of purchase all the way through to the point of consumption and beyond. That is absolutely fascinating if you think of that holy trinity of owned and paid media. Now your own media channel, this passive print, can not only deliver both reach, but engagement; an incredible resource of data. WebAR enables that because, for a lot of household brands, people won’t necessarily want to have their app. But actually — and we’ve seen it through research that we’ve conducted, and with partners as well — people want to know more about products. We now live in a time when understanding a product’s provenance, and its authenticity, and actually knowing more about it, how-to pieces, instruction information — all these things are stuff that we expect. That’s something where we can really bring that to life, through AR, and WebAR will play a big part in that, I believe.

Alan: How do we, then… because you mentioned something earlier about the fact that only a small amount of things right now are AR-enabled. You guys have created what’s called the Zapcode; it’s like a QR code specifically for Zappar. What do you think about potentially creating a universal standard for the AR logo? So then, you can have a QR code-type thing that, anytime you see that QR code, you realize that something is AR-enabled. Whether it’s web or app, it doesn’t matter; open your camera, point it at it, and it automatically will take you to the website, or the app download, or whatever it is you need to enable that. What are your thoughts on that?

Caspar: Might be worth going back to why we came up with a Zapcode in the first place. In 2011 when we did it, I think everyone thought was such a backward step. “No! It’s all about image look-up! Why would I possibly want to put a code on this thing? What are you doing?” There’s some very–

Alan: All they have to do is go to China to answer that. Go to China for a week and try to not use a QR code.

Caspar: Well, this is it. Image look-up is great, but it’s more computationally expensive. You do have an issue if you want to look at something that ostensibly looks the same, which you come across a lot if you’re in the licensing world. 

Alan: If you’re a marketer, you’ve got 10 advertisements, they all look the same and have some different copy on them. 

Caspar: This is it. So, Zapcode was something that came out of Simon’s — our research director and co-founder — work about being able to do incredibly fast detection when you have things that are small in the camera image. Zapcodes really solved an awful lot of problems and meant that we could make custom codes around people’s brand identity. We did some work with Shazam around that, and when we supported all their visual recognition in the Shazam app. [We did some] work for Hasbro, and others around that. It really serves a purpose. 

But we’re pretty agnostic nowadays, about what it is that we’re going to scan. You mentioned this, the QR codes. They’re having their Joe Frazier moment, aren’t they? We’re happy to lean into all those things. Going back when we started, it was always this thought of, “which is the AR app that’s going to rule them all. And analysts and journalists would always ask you, “what’s the killer application?” I’m not sure that’s really asking the right question. AR is this facilitating technology. It’s a camera function. What we really need to start asking ourselves, as brands and businesses, is “what are the stories that we’re telling as brand owners, when seen through the camera? And how can AR enhance that?” 

Alan: So, what is the value to the end user, whether it’s entertainment or delight or functionality? What is the end user’s journey, and why? Why would they do this? I see so many things. I get emails from different startups all over the world creating AR. Great, but why would anybody pull a phone out to do this? What is the actual point of it? That’s what it comes down to your three Cs again — context, call to action, content. Without those three checkboxes, you just have something cool.

Caspar: Yeah, that’s that’s exactly right. But I don’t think it matters what people are pointing their phone at, at that point. I guess we’re not anti-QR codes, if that’s something that people are familiar with. I think the problem we’ve got with them — more in the west — is that the experience up till now has been pretty lousy. So how do we re-engage people with that behavior? I guess we were very flattered that, having done Zapcodes, you know, it wasn’t long before we then saw SNAP codes and Amazon codes and Spotify codes. At that point you go, it’s nice to see that we were in the right direction.

Alan: You and I will talk offline about this, but I have a plan on how we can standardize this, because I think there’s definitely confusion in the marketplace. If I have to open Zappar to point it at  Zapcode, that may be confusing. But if it’s just, I open my camera, and it automatically recognizes universal code? You and I will talk about it. We’ll figure out how to get The Cronos Group to make it standard.

Caspar: Well, that would be a good thing. Because you can now connectively, I guess, access and scan QR codes on iOS devices, and more so with Samsung through Bixby and Glens, et cetera. I think that’s the current de facto standard. But happy to have that conversation offline.

Alan: I’m going to move to something that I know is kind of a question that business consumers and business customers ask me all the time: what are the types of data that we can glean from these activities? You mentioned basically bringing print to life, and you’re filling in this black hole of data that, when you print something, you send it out there — you don’t know. Are people reading it? Are they’re looking at it? Are they going on the web? You have no idea, unless you put a coupon code and then, okay, well, we had 17 visitors from this or whatever. But with AR, you know where they are, how long they dwell. What are some of the data points that you’re able to collect and give to brands, and how are they using those data points?

Caspar: Well, I guess, as you know, we have to do this all under the auspices of the GDPL, which is now in full effect over here in Europe, and I’m sure we’ll be coming to many more markets soon. We do have to ensure, from a Zappar perspective, that any of the data we collect is not personally identifiable. That still means that we can see quite a lot of stuff from the anonymized data. 

We can obviously see a number of unique scans. What time of day, what region, what events occurred. So, if someone scanned our experience, did they play the mini-game? Do they take a photo? Did they receive the coupon? All those sorts of things. We can understand what we’ve done, and we can see their average dwell time. So we’ve got quite a lot of analysis that means that we can understand how a campaign is performing. Now, clearly, if that is then also integrated into a third party app, and they have got those permissions from end users? Well, then all of that can be tied to individuals as well. And then you can get very rich and personal data. But clearly, that has to happen in an infrastructure where someone has opted in. And I think those things are fascinating on both sides. Where we’re doing work with retailers, and they might have their own reward schemes, being able to understand that at an individual level is incredibly powerful. 

But even where we’ve done it for, say, some some other FMCG brands, we’ve had some fascinating times where we can pinpoint at what time of day, what day parts, and indeed, what hours people have scanned over the course of a week, and some fascinating patterns occurred. That information has also been taken to then inform the rest of the media strategy. If people can understand when it’s actually most likely that people will be using their product, well, that’s really interesting information. 

It’s all data, right? It’s what you do with it, that’s the most important thing. So making sure that, once you’ve got it, you’ve got the right analysts, and you’ve been asking the right questions. Indeed, before you even starts the activity, you’ve set your objectives clearly.

Alan: That was going to be one of my next questions. Let’s say a brand comes to you and say, “hey, we want to do AR.” That’s great that you want to do AR; what are you going to measure success on? What are some of the measurements of success that you guys promote, or talk to brands about? What are they looking for? Are they looking for, “we just need a number of clicks?” There’s also earned media. There’s tons of different ways to measure this. What are the typical ones?

Caspar: Well, first thing I’d say is if anyone comes to you and says, “we want to do AR,” that should ring an alarm bell to begin with. That means they not really thinking about what their objectives are. They’re just trying to think of AR as this thing that exists in this vacuum. 

Alan: So what do what you do in that case? Do you educatie people?

Caspar: Yeah, we do, because it’s so important — both for us as a business, but I think more broadly, for everyone — to realize that AR is not like the dog you get for Christmas. It is something that you’ve got to think about for life, because the true value of AR is about this always-on channel, as opposed to this just one-off, “tick the innovation box for your marketing campaign” piece. In fact, you won’t get the value out of doing it that way. The way you can extract the value is understanding how to best deploy it throughout the business, and use it on an ongoing basis. 

There are a number of different ways and objectives that, I guess, we’re asked to achieve, depending on the sector and the project. That could be anything, from people who do have their own app and they’re trying to increase the number of people who’ve installed it and increase the dwell time and the reason for being in the app, in order to explore other features within it. That’s a perfectly legitimate use of AR, as sort of a gateway, if you like, to bring people into an app and to discover other services within it. Clearly, it can also be used, if there are people trying to, in a retail setting, bring people into a store environment. It does some great work with a company called Tilly’s in the States. They’ve been fantastic as well, we’ve worked with them for a few years. And true pioneers as well, in terms of properly integrating into their app, trying to make this connection between the bricks and mortar retail and their digital strategy, giving value add coupons as rewards within the AR experiences, making sure that their internal staff are well informed about it. We also know it’s a great tool for people on the shop floor to excite people about and start up a conversation. But there again, you know, that is about–

Alan: Hold on, hold on. Let’s just stop there for one second. As you know, this is something that I don’t think many brands have considered; everybody is thinking about, “how do we get this app in the consumer’s hands?” But more importantly, how do I enable our retail salespeople to have something engaging to show a consumer? 

Caspar: You’re so right, yeah.

Alan: That is a beautiful use case. And not only could it be used to train the sales reps, but then they can use it to train the customers.

Caspar: Exactly right. It conforms to that wonderful fear of missing out that we all have. I always liken it to, if you’re sitting on the subway, reading the paper (or, back in the day when that used to happen [laughs]), people couldn’t help themselves but look over at your paper and read it. And it’s the same if someone’s on the phone; one eye is slightly looking at the screen. People innately are interested in what others are doing. 

So, if you’re in a shop floor — and we’ve done this at a lot of shows: you hold your device, scan whatever the thing is you’re scanning — people are like a moth to a flame, they’ll come over.

Alan: People love it!

Caspar: Yeah. And it strikes up a conversation, and then suddenly you’re into a different type of sales framework. I think that’s an incredibly powerful use of technology. Think about workforces, where not everyone has access to the computer. Let’s face it; for hospitality or food service, most of the staff will have access to a phone, but they might not have a terminal they can get to. How do you get to that disconnected workforce and tell them about new offers, or new promotions, or new things they can sell? Actually, AR is a very interactive and engaging way for them to do that.

Alan: It’s interesting you say that; we just invested in a company called 3D Food and Drink, and they do super-high-resolution photogrammetry scans of meals, and then they do wine and beer and alcohol pairings with those foods. It’s a great way for the restaurants, the servers, to show the meal in a completely different way. And it’s increasing sales dramatically, by 40-50 percent, because that expensive meal item is now at the forefront in AR. And then the wine upgrade, or the beer upgrade, is increasing the sales as well. So it’s interesting you said that.

Caspar: It’s great to see that. And I think there is now… well, another thing maybe for offline that we need to get to, is a point where all these incredible case studies are all being put together. Because they all exist, and the stats do not lie, and they do speak for themselves. And the incremental value that is being generated by the use of AR for these different experiences is evident. Now, clearly, not everything works brilliantly, and you learn as many things from your failures as your successes. But it is extraordinary that, time and again, we’re seeing that, in terms of the level of engagement, the level of click through, the level of increase in sales volume and footfall, it does have this facilitating effect on different experiences.

Alan: You mentioned stats. What are some of the stats that you can share publicly? I know that there’s probably a lot that you can’t share, but what are some of the statistics around successful campaigns? What constitutes a successful campaign?

Caspar: Well, you’re right. There’s quite a lot that is governed by NDAs that we can’t talk about [laughs]. So that is a harder one to get into. To be honest.

Alan: Because we’re talking about case studies, and we want to be able to write these case studies and scream from the rooftops how great it is. And then we’re all under NDAs, we can’t share it. So it’s like–

Caspar: Yeah, it’s a bit of a Gordian knot, that that is. That is a fair point. I’m sure the right permissions could be sought, in order to get all those things together.

Alan: Are you guys members of the VR/AR Association?

Caspar: We are now. Yes.

Alan: The VR Association is a global organization that has chapters in pretty much every city in the world, and they’ve been really pioneering pulling together these case studies, these white papers. I wrote the enterprise white paper about six months ago. There’s a marketing and retail one. So there is some work being done around collecting all these – and it’ll be interesting, we’ll talk offline — you’re absolutely right, the rising tide raises all boats. And in this case, we’re in an industry that, over the next five or six years, is going to create a trillion dollars in value in the marketplace. So we need to just scream it from the rooftops.

Caspar: Absolutely. I’d definitely love to pick up on that.

Alan: Well, I want to thank you so much for your time. Where we’re coming to the end here, is there anything else? The last thing I wanted to ask you, is there anything else you wanted to mention?

Caspar: Oh, gosh. We have covered a lot of stuff. I think the only thing I’d say is that, this is a space that will continue to evolve. But I’ll just reiterate that point of, I don’t think we should get too caught up in the future. I think we do need to celebrate the now. And augmented reality is a “now” technology. It’s not just a future technology. I think most of the applications that are readily available to pursue now have not been explored to their fullest extent by large swathes of businesses in many, many categories. I would say it’s people really try and embrace what’s there now. Get ready for what’s coming in the future. But take advantage of what you can do today.

The old ways of advertising in legacy media just won’t work in an XR environment, so Admix CEO Samuel Huber has tried to figure out more effective ways to peddle products in a virtual realm. Today he explores his XR marketing ethos, which focuses on the user first.

Alan: Today’s guest is Samuel Huber, founder and CEO of Admix, the first ad tech platform for mixed reality, giving XR developers the best tools to monetize their content.  Admix is a platform that allows advertisers to place non-intrusive ads into VR and AR and gaming content. Their platform allows companies to filter hundreds of advertisers via their programmatic platform, and start generating revenue within minutes. Previous to Admixed, Sam was part of the e-commerce platform Kout.io, social gambling app Betify and the first binary trading game on the app store, Rogue Trader. Admix is a venture-backed startup, having raised $2.4-million. You can learn more about Samuel and Admix by visiting: https://admix.in/

Sam, welcome to the show.

Sam: Hey Alan, how are you? Good to be here.

Alan: I’m fantastic; thank you so much for being on the show.

Sam: Thanks for having me, man.

Alan: It’s my absolute pleasure. For the people who don’t know you, I know you’re very active on LinkedIn. For those who don’t know, do you want to just introduce yourself and introduce Admix in your own words? Describe what your company does and what your service and platform does to serve businesses.

Sam: Yeah, yeah, for sure. I started Admix about two years ago now, initially based in London. Now we have also an office in San Francisco. The whole idea that we realize is that there’s so many great, immersive content being created, whether it’s VR, AR, and this is really going to be the way that content is going to be consumed in the future. Right? Whether it’s for entertainment, games, education, training — all of this stuff is going to be spatial content. And what we figured out is, there’s already great content being created, but very little incentive for the creators — or ways for them — to actually monetize this content. On the web, if you create a website, you can go to Google and you get a tag, and you can put ads and you can make a living out of it. But that’s actually not what happens right now in VR and AR. I think over the next few years, we’re going to see a lot of paid apps migrating to free content. That’s generally the way things go. And at that point, these developers are going to look for a new business model and advertising is going to play a massive part of it. 

However, from the very beginning, our idea was to build a better future for advertising. Not banners and pop-ups and annoying, that are already terrible on the web, and would be even worse in VR and AR. But instead, creating an advertising model that really works for this immersive content. So, we’re talking about product placement. It’s how brands can be part of the experience, part of the creative process without intruding, but while still generating great return for the developers. So that’s really what we’ve built. It’s a simple SDK that integrates with Unity as a content creator, or you can drag and drop product placement within your experience. We then connect that to a massive network of advertisers. 

So basically, you create your app at Admix and you can get revenue generating in about three or four minutes. And it’s continuous. It’s instant revenue. And so far, the developers that we work with really, really like us. We have about 22 people now in the company, raised over 2 million dollars last year, and we are expanding this year internationally as well.

Alan: That’s incredible. So, here you are: you start a company two years ago, you figure out that ads are going to be a thing in immersive content. There’s tons of content being made and developers and content providers are just not getting paid for the work they’re doing. And so you created this platform, to help them monetize that. Give us an example of one of the best ways that this has been used to date.

Sam: There’s a lot of really interesting ways. A very simple way is game developers, game studios; a few people that somehow got an app that is very popular on the Google Daydream — or mobile VR, basically — and they were really struggling to make money with it, because it’s not an app that people would be willing to pay for. It’s very entertaining, but it’s not something that you would happily put your credit card to pay for.

So they tried, actually, to use “normal” — I would call them “desktop” or “mobile” — ads, like a banner at the bottom of your screen. But of course, it’s not adapted to VR. So when you wear the headset, you see this banner that is kind of blurry at the bottom. It kind of counts as an impression, but it’s terrible. It’s a terrible impression. And then you have a video interstitial that interrupts your experience. It’s obviously not VR. You have to take the headset off.

Anyway, so they were making some revenue like this, but the users were really pissed off. You could read the comments. It was like, “cool app, but too many ads,” blah, blah. And so we came in. We basically told them, “look, you guys are doing great, but we can actually get you ads that work for you! Get ads that work for VR and AR. The user plugin… one of their apps was kind of a roller coaster, a typical VR app. And with our plugin integrated with Unity, they were able to place ads alongside the track, by basically planting billboards that are part of the scenery. So you go along this rollercoaster, and sometimes you pass very fast next to a billboard. If you don’t want to look at it, you don’t look at it. It’s just like the real world. It doesn’t take over the experience. It doesn’t look out of place. 

It’s been going fantastically. They have a lot more users now. Users are a lot happier. And just to give you a bit of a figure… I mean, you’re talking about a couple of guys. They’re making around $40,000 a month just using our solution. And it’s been completely changing their business; they’ve been releasing more apps, reinvesting. And that’s really what we all about. It’s about, how do you give the creators a reward for their great work, so that they can reinvest and create more content? That’s what is fueling the whole ecosystem. Because more and more content, then they get rewarded even more, and they can invest more, and they can build better stuff. That’s really what we’re all about, and seeing that from a small team is just super exciting, to be able to empower people like that.

Alan: I think that’s really incredible. Forty thousand a month is a phenomenal amount of money for a small studio.

Sam: And people say, “oh, VR, I don’t make money in VR.” Well, there are ways to do it, and we want to be the ones that create a great way to do it. It’s really about creating a great experience for the developers, of course —  they make money — but at the end of the day, who we are prioritizing is the user. We want users to be comfortable with what they see, because if they hate it, they obviously won’t use it. So it’s all about the user. It’s all about creating a great experience. And so this is why the type of ads that we are creating, we think, are really putting the users first, because they are not intrusive, and they’re really in-line and relevant with the content.

Alan: It’s amazing. I think one of the things that immediately strikes me is that you’re aiming this towards the developers, but… let’s say I’m a marketer and I want to sell my products. Do you take my products and then put me in front of the right developers to match that content? Or is it more of a hands-off thing?

Sam: Our solution is programmatic, and just to explain very quickly what it means; programmatic basically is a way to automate buying and selling of ads. That powers 80 percent of the web nowadays. Before that, if you wanted to get ads from a certain brand, you kind of had to ring the brand and then to reach a deal. It was very manual. Now, you plug in platforms that I call demand-side platforms or supply-side platforms, and basically, you make your inventory available there, and then thousands of advertisers have access to it. That’s how it works at a very, very high level. Obviously, it’s super technical and there’s thousands of these platforms, which we try to connect to as much as possible. But that’s basically the idea.

What we are doing is really focusing on the developer side. We’re giving an SDK – this Unity plugin — to our developers. They can create this inventory. They can say, “I want to put a banner on this wall of my game. I want to place a video on the screen, and I want to place a 3D product here.” And you would trigger when this happens during my game. Then these spaces get sold to a network of advertisers. 

So, from the other side, the advertisers do not use our platform but are connected to our partners. They do have access to all this targeting stuff. Just like on the web, you can say, “I want to reach women between 30 and 40 who like these kinds of things.” You can do exactly the same because we connect to the same platform. The only difference is that we’re giving them access to a new media: VR and AR, which happens to deliver better performance because the ads are less intrusive and people are engaged with them a lot more. So it’s kind of a win-win-win situation for the users, the developers, but also the advertisers. And that’s really important because if the ads don’t work, advertisers are going to spend less. So you really want the three parties to benefit from it.

Alan: Absolutely. Now, we’ve talked a lot about virtual reality content. What other types of content are you seeing an uptake? Are you seeing this type of thing with augmented reality as well? Or…?

Sam: Yeah. So, I mean, our solution is compatible with all of them, because it’s based on Unity and Unreal and basically on the game engine. But we’re kind of following the users. Advertising is all about numbers, right? You won’t be successful as an advertising company by targeting big-name apps that don’t have that many users. So we found a lot more success in Mobile VR at the moment. That’s where the biggest VR apps actually exist; apps that can be consumed on the Daydream, on the Gear VR, even on the Cardboard. You know, some of them have literally millions of users passing through, and this is where we find a lot of our market feet at the moment. 

We’re targeting also, of course, premium VR. But again, we believe that premium content will always exist. People will always be happy to pay for a triple-A game. That’s not really our target market. We want to target the long tail; the free stuff, which actually represents over 90 percent of the content that people are happy to use. But they want to use it for free. So that’s really what our target market is. 

We have a couple of other interesting use cases; a few AR apps. There’s not that many apps that I used by the consumers day-to-day. It’s a lot of demos, a lot of cool stuff. But you don’t have the same kind of retention, the same kind of community, I would say, in terms of AR that they’re using in VR at the moment. We also start to work with a location-based – it’s actually a really interesting project — where everyone is trying to find additional revenue stream, new way to make money without charging the end user. And that’s where advertising so great, because you don’t actually get to charge your user; someone else is paying for it. And I think that’s why it’s very attractive.

Alan: Absolutely. So at the end of the day, users aren’t they aren’t paying for it, and that’s fantastic. So users get that are content; content developers get better revenue streams, and advertisers get more targeted ads. It sounds like win-win all the way around.

Sam: Yeah. And the last spot is – I cannot insist enough on it — it’s very important that advertisers hate spam as much as you. Because, for them, it’s wasting money if you don’t actually interact with the ad. So the better the ad performs, the less ads there will be overall, because advertisers won’t need to do these kinds of spray-and-pray type things. They would focus on people who are actually interested about their product. So overall, if you want — it’s kind of a paradox — but if you want to reduce the total number of ads, which everyone wants to do — I mean, I hate ads as much as the next guy — you need to create ads that work better for the advertisers. That’s the way to get less ads, because they will only be focused on reaching their core audience and therefore, everyone else won’t be bothered with ads that are irrelevant.

Alan: It’s interesting. You say you don’t like ads. It’s funny that… the Super Bowl, for example. This year, I didn’t really watch the Super Bowl, but I found a website that showed all the commercials that we’re going to air on Super Bowl, and I spent maybe an hour just watching all the commercials that were going to be on the Super Bowl. It’s amazing; the amount of production and talent that goes into these things is absolutely incredible. And it’s no different than the amount of work that goes into making a game or an experience. Being able to add a can of Coke, or a Coke ad… did you see the recent Burger King ad?

Sam: The AR ad blocker, right?

Alan: Yeah!

Sam: Yeah. I thought that was actually awesome because it’s an ad itself. We actually thought about, internally, doing something along those lines, and we were thinking what is going to be the future of ad blockers? Thinking that people would walk with their AR glasses and everything would be blocked around them. It was just a fun idea, but it was really fun to see what Burger King actually did.

Alan: For those of you listening who don’t know, Burger King made an augmented reality app that, if you pointed it at any of their competitor’s posters, billboards, or signage, it would burst into flames and then you could get a free flame-broiled Whopper by submitting that photo of the McDonald’s ad in flames to their Instagram or whatever it was. And I thought it was a really genius way… they’ve got millions of… they’ve given away, I think, something like 50,000 burgers or something. It’s crazy.

Sam: Yeah. For me, this Burger King stunt is actually an ad. Because at the end, when the competitor ad was being burnt, their ad was appearing instead of it. So for me, it really shows the power of ads in immersive media. I think that these kind of stunts that used to work super well in the real world, like experiential marketing, that was limited to only the audience that was on-site. Then after that, it was filmed. Then maybe you would become a viral video somewhere else. But only a really small core of people got to experience this really awesome brand experience. But now with VR/AR, this is actually expansion marketing at scale, because everyone can actually create this experience just by using this device like that; like the Burger King experience. So it really helps propagate these really awesome ideas that brands had, but couldn’t push to a larger audience before. But now, with VR and AR, they get the ability to enable people to actually do this stuff at scale, and interact with the brand in a completely new way.

Alan: It’s pretty amazing. I think one of the other brands that’s done a really, really good job at harnessing AR is Snapchat. Their camera-first platform is really kind of… well, I’ll just put it out there: Snapchat is by far and away, the biggest user of augmented reality in the world. And if you kind of look at it through the lens of what they’re doing, using the cameras and being able to put on Snapchat filters in your face;  if you turn the camera around, you can put things into the space. Really amazing what they’ve done. And, you know, I don’t know if you saw the LeBron James Nike ad they made. They took a poster in a store, and when you open the Snapchat filter and point it at the poster, LeBron James came out of the poster and slam dunked a basketball. Here, real world. Amazing use cases. Does your platform serve for things on Snapchat and Facebook and these other platforms as well?

Sam: These platforms are pretty close platforms, so at the moment at least, if you want to advertise on Facebook, you have to go through their buying platform, and they only care about Facebook. It’s a very closed ecosystem and is the same with Snapchat. At least at the moment; we don’t know if eventually, they would open the gates to other tools, other platforms. So we’re not targeting those. 

What we are predominantly targeting, again, is the indie developer that is making a game, that is making an app, and that is looking for ways to monetize. Our target is really not about pushing cool types of ads to big platforms, but it’s really helping small developers to actually generate revenue. So it’s kind of the opposite idea, where we really prioritize the small developers that just want to make a living out of their awesome content.

Alan: Well, I think there is definitely a need for that. One of the things that I saw about a year ago; Unity was starting to look into this programmatic ad model. Do you see that as a competitive thing?

Sam: Well, eventually. Probably, there will be a lot of big companies — Google, Facebook — obviously are going to try to monetize this new media. They’ve invested so much into hardware and everything. But I think that we have a few years of lead ahead of them. Just because, I guess, the market might not be mature enough for them. They’re still trying to sell hardware. They don’t want to sell ads in this new space because advertising does have a negative reputation, especially for those guys. So I think, on that way, we have a strong angle that we can play with.

As far as Unity goes, we are very close to them. That’s what’s exciting about these spaces. It’s so early that everyone kind of wants to try working with others as well. So, yeah, at the moment, it’s more of a friendly relationship.

Alan: So you said that we’re early in this technology, and it’s interesting because one of the other interviews that we did was with Steve Grubbs from Victory VR, and they’ve created 240 different education VR experiences, for everything from science to dissecting a frog. It seems like the appetite for this technology is really starting to open up. I would have said a year ago that we weren’t even started. It was the beginning. So, where do you see us in a timeline? If the beginning of this technology was in, let’s say 2015-2016 and mass adoption is, you know, X; where are we now, and where do you think this is going?

Sam: Well, I have really high hopes for the future. I really believe that at a high level, spatial computing is really the next interface, the next computing platform. Until now, all our information — everything we are doing on this screen — is in two dimensions, and the real world is in 3D. That’s a big limitation. 

Spatial computing is going to take us from 2D to 3D. That’s for me, the same as the way that the Internet took us from offline to online. That’s the same level of revolution, in a way. I have complete faith that this is going to happen over the next — I mean, completely happen, for everyone — over the next decade, for sure. This is inevitable as far as I’m concerned; it’s a natural evolution of media. We went from text to pictures to videos. The next step is obviously becoming part of the content itself. There are a lot of signs that point in that direction. 

So to answer your question, I think that we are at the very, very beginning; like, probably less than 1 percent of the potential of VR could be. Which is very exciting, because we already see a lot of activity right now in the space. Just to give you an idea, we were measuring for us… what’s really interesting about the adoption, is how many people develop built content for it. These are the early adopters, and they come before people actually get to consume this content. And we’re seeing an over 400 percent increase in content being created every year. That’s four times more every year. And the number of developers that are getting into the space is just crazy. You can see that everyone loves these experiences, and just need maybe a bit more clarity as to why they need VR; what the value of VR/AR is for them; and also to make sure that the value that they get out of it outweighs the cost. That’s the basic ratio that defines if you want to buy something or not. It’s like the perceived value is higher than its cost. And I think for a lot of people, it’s not right the case yet. But as more and more applications go out there, the hardware becomes cheaper; it’s just a matter of time before this actually kicks on. 

So, yeah, I mean, it’s hard to put a clear timeline on it, but I think over the next few years, we’re going to start to see wearables as well, like AR glasses, and eventually is just going to really take over our existing displays.

Alan: So with that horizon of, let’s say, 10 years for ubiquity. Ubiquitous spatial computing within the next 10 years. So, 2028, let’s say. With that, what advice would you give somebody in any business, whether it be a Fortune 100 company, or your local store? What advice would you give these companies? Because one of the things that stood out to me at the Microsoft Hololens presentation at WMC this year, Mobile World Congress, one of the things that stood out to me was the fact that they mentioned that every company is now a technology company. So if that’s the case, what advice would you give a company looking to learn and get started in virtual and augmented reality, in spatial computing? What advice would you give them right now?

Sam: I think it’s about paying attention to it early. We’ve seen that happen many times; with people not paying attention to the Internet; people not paying attention to mobile; in the ‘70s, ‘80s, people not thinking that there would be a computer on every desk. It’s the same kind of thing we are again approaching one of these revolutions. I think it’s really about starting to think. 

To come back to this idea, the world is going to become 3D. Our information, our knowledge, the way we share data, is going to be overlaid environments. We won’t need that boundary, which is the screen between content and reality. Knowing that, it really can change the way you have to think about how your business could be transformed by this. Whether it’s about finding a better way to train your employees (that is obvious), or different ways — better ways — to communicate by using spatial computing. It’s really trying to think, if we didn’t have screens, how can my workplace be transformed? And that’s a big thing. 

And of course, he won’t be, from one day to another, we won’t use screens anymore. That’s not what I’m saying. You know, we still listen to the radio and we still watch TV. We’ve been saying that these media are dead for years, but they’re not. Screens will still exist, but they won’t be the ultimate way to consume content. Spatial computing will be. And I think, just trying to think about this transition from 2D to 3D is the best way to start imagining, what can you do with this? How would you implement it in your business?

Alan: So with that, what are some of the challenges that you’ve seen businesses struggle with that you can lend some advice?

Sam: We mostly target consumer apps. We do have a couple of B2B clients as well. I think businesses in the space are sometimes building really cool case studies or experiments that don’t really have a proper use case. And again, it comes back to my idea that the problem — one of the limitations of VR right now, and is totally normal because it’s a new industry — is the fact that most people are not clear about the value proposition, and why they would need, it beyond it being a gimmick. I think businesses that are early on the market and starting to build cool stuff, they really need to think, how high is that actually going to help the business? It’s not enough just to build an app because it looks cool if it has no practical use case. People are not going to use it. It’s not going to catch on. 

I think that’s really one of the main problems. I can see a lot of really cool stuff happening, but not much things that actually improve sales, for example. And I know that with MetaVRse, for example, you’re doing all this great virtual commerce. And that’s amazing because you can clearly measure that someone who tried a watch on their wrist instead of seeing a picture of it, they’re more likely to buy it. So that’s a clear use case. That’s a clear KPI to show that actually, AR can drive sales. I think a lot more businesses should take that as an example, and start to see how it could actually help them, by actually implementing stuff that works and that moves the needle for them.

Alan: It’s really interesting you say that because we’ve done a lot of different things. We’ve made AR just for the sake of AR; we did a marketing thing for the launch of HBO’s West World; we did a number of things like that. But really, when it comes down to it, what kind of drives us is looking for those use cases that drive the needle in a defined way that solves a problem. 

For example, one of our products that we’re bringing to market is called Floorcast, and it allows people to previsualize what their new floors are going to look like. I’m going through this right now; my basement floor is being done, and they brought carpet samples that were literally 4×4 inches. “What color do you want?” I’m like, “I have no idea what it’s going to look like in gray in my whole basement.” Like, I just ordered gray carpet without knowing. And we have this app that does it. But I didn’t have time to import gray carpet in there. So, you know, that’s something I would have used just to be able to see, what does this look like in my basement? Or what is the new hardwood going to look like in my floor? 

From a business standpoint, that is not just solving a problem from a consumer standpoint, but from a global distribution standpoint. Flooring companies, they sell through distributors, and then retailers, and then a guy coming over with a sample and measuring; there’s a lot of layers of value being wasted through something that could be done using an app. And AR is a prime example of that. 

Another idea is IKEA’s Place: being able to see what furniture looks like beforehand. I think these are great use cases. You nailed it.

Sam: Yeah, and it’s funny, what you mentioned about the flooring thing, because right now, they give you a sample — which is kind of what I was talking about this experiential marketing — give you a sample for you to experience it. But it’s not scalable. It’s not something you can distribute at scale. And right now with AR, they just give you an app. They don’t even have to come and meet you and give you the sample. You can just do that digitally. And then you can not only try this sample, but I’m sure you’ve tried different colors and different textures and everything. It’s just the most scalable way to do it. And the person at these flooring companies should start thinking, what can I do? How can I solve the problems that I had every day, which is travelling to clients and giving them samples and they’re not quite sure? Well, there’s a pretty obvious use case here. I think every company can find use cases like that related to VR/AR.

Alan: I agree. Speaking of that, what is the best business use case of XR technologies that you’ve seen so far?

Sam: We are mostly focused on other B2C side, and I’ve seen some really interesting social VR use cases, really aimed at the enterprise, mostly around communication. So, running virtual meetings; you can be in this zoom room with other people, but it’s not quite the same as being able to actually share data together. There’s a couple of apps, some spaces. 

One of them, Glu, is another one where you can literally all be in the same room with your avatar. That’s normal, basic — nothing crazy about that. But you can actually pull out tools, and you can share screens, and everyone can see your screen in everyone else’s screen. And if you just think, in the real world, if I share my screen right now over Skype, I won’t see your face, or vice versa, and someone else’s screen would take over mine. Only in a spatial environment can you see all these screens together. And you can move them closer, and you can actually draw, and everyone sees it together. So I think communication, social; that is really going to be exciting, and companies that understood that will be able to build a massive business out of it.

Alan: Yeah. It reminds me of the company Spatial. Right. They basically created virtual avatars in mixed reality space, and allowing you to bring in 3D objects and–

Sam: Virtual post-it notes and all of that. Their interface was looking very good, actually.

Alan: Really beautiful. I love that. This is great. So, I want to say thank you so much for joining me on this podcast. I’m going to ask you one more question and then we’ll wrap up: what problem in the world do you want to see solved using XR Technologies?

Sam: Well… that’s a big, big question to finish. I like it! 

I think it’s obviously education. I mean, how can we use the power of this technology, this immersion, to have kids learn in a better way, and give them access to education that they probably don’t have? I think that would be top of my list. Very, very important.

Then, on maybe a less serious note is, you know, enabling creators creating. I mean, that’s what we’re doing. That’s what we’ve been Admix for; definitely something that I’m very passionate about. I was a game publisher – I ran my own studio before — and I know how hard it is to actually generate revenue from the content. And I think if we can help young people getting started, generate more revenue so that they can reinvest and build better companies and become successful that way, that’s a really interesting and exciting mission to have.

Alan: I love it. It’s amazing. And I thought of a cool tagline for Admix: “Admix — Ads That Work in XR.”

Sam: That’s pretty good. One of our previous taglines — we tested different things — was “Advertising That Doesn’t Suck.” So, it’s kind of along the same lines.

Alan: I love it. So, you know, I really want to say thank to you for joining. One of the stats that kind of stood out of my head is that there’s a 400 percent increase in content being developed every single year. That is a staggering: four times as much content, every year. And this is not going to slow down. This is only going to get bigger and bigger and bigger. We’re in the industry that’s going from 10 billion last year, to 200 billion in the next few years. And over the next five, six years, there’s going to be a trillion dollars in value created in this industry. And I think it’s interesting, you’ve kind of position yourself perfectly for this.

AR dragons, psychedelic displays at Coachella, and other digital gizmos made possible with XR technologies are fun and all, but Mark Sage, founder of AREA, is on the more pragmatic side of the table; he loves it when XR technologies can solve real-world problems for businesses. Mark and Alan sit down to discuss how to do that, and how that creates a better ecosystem for enterprise XR to thrive.

Alan: Today’s guest is Mark Sage. Mark is a product owner, creator, marketer, innovator, business development professional, evangelist, spokesperson, strategist, program and project manager, and mentor across a range of AR, mobile, B2B and B2C technologies and products in an international context. Mark is currently the executive director of AREA: Augmented Reality in Enterprise Alliance; the only global, membership-funded, non-profit alliance dedicated to helping accelerate the adoption of enterprise augmented reality, by supporting the growth of a comprehensive ecosystem. AREA members include Exxon Mobile, Boeing, Lockheed Martin, NVIDIA, PTC, and so many more. You can learn about The AREA at theAREA.org. It is with great honor that I welcome AREA executive director Mr. Mark Sage; welcome to the show, Mark.

Mark: Thanks so much, Alan. It’s great to be here to speak to you, and to those who listen out there, as well. I’m really excited. Thank you.

Alan: Thank you so much for joining me. We’re really excited; let’s get right into this. I’m going to start — just, dive right in here — what is one of the best XR experiences that you’ve ever had?

Mark: Oh, wow. Gosh.

Alan: I know, I’m going right in there.

Mark: You are, aren’t you? And in the kind of role I’ve got, I have a huge opportunity to go around the world, experiencing all sorts of different experiences. I guess, when I first started, one of the first things I was amazed about was the DAQRI Helmet, back in the day. I remember first wearing that, probably about three years ago, thinking this would be amazing. It didn’t quite end up as it would be. So, they’re still working on some of the areas there. What I’m really thrilled about is the experiences that really solve problems. Being focused on the enterprise space, I love to see things that are solving real-life problems, here and now. So anything from the simple-yet-effective remote assistance services and applications, I love seeing those; the way that you can engage with an expert, and get real detailed information of how to fix things.

I always love trying those things out. I love some of the simple things; I remember being at a shipyard in Finland, and just using a tablet, they were showing me how they look into a new container that had been built, and how they could check what was going on, and using in an eight hour experience to make sure it was all correct. They were cutting down — literally, by hours — the amount of time it took to review things, and make sure it was all set up and stuff like that. Right into the step-by-step innstruction, I always remembered RealWear, when they did their first step-by-step instruction. Doing it in a brewery, and showing how they were moving taps and pipes, and doing work there. So to be honest, anything–

Alan: Do you think they did it in exchange for beer?

Mark: Well, I hope so! I absolutely hope so. So you know, Alan, anything that shows some real benefit… I love some of the kind of cool stuff, but certainly, my experience in the enterprise AR stuff that actually solves a problem, and creates real benefit for enterprises, is really cool for me.

Alan: It’s interesting you mentioned that DAQRI smart helmet, and for the people listening: the DAQRI helmet was this incredible, futuristic helmet — it was white with his blue lens in the front — and I had the opportunity to try it a couple years ago. When you put it on, you had a beautiful heads-up display. It wasn’t quite like the demand Magic Leap or HoloLens, where the holograms were in a positional space; it was more this heads-up display to give you almost, like, superpowers. You can have all this information, just right in front of you when you need it. One of the things that I also noticed was it got really hot.

Mark: Yeah.

Alan: You were pushing a lot of computing power through that thing, and so I think they’ve moved to, now, a pair of glasses, instead of the full helmet thing.

Mark: Yeah. Correct. And I think it’s probably an example of just being a little bit too early with the technology. But I’m sure in the future, we’ll see those types of devices. I know in the first responder, firefighting bit, they’re looking at having that heads-up display, and showing maybe simpler information; infrared or heat sensing, and guidance, and things like that. We’re on the start, as you well know, of this industry, this technology, and I think I’m looking forward to seeing what will happen in the future. I think everybody is, and I’m sure we’re going to see some amazing, cool things in the not-too-distant future, as well.

Alan: You mentioned that we’re going to start to see these different iterations. And we’re already seeing that with the launch of HoloLens 2, and Microsoft really listening to the customers and saying, “hey, we understand that everyone can use this a little differently,” so they had a huge contract — a half a billion dollar contract — with the U.S. military, and the U.S. military just released a little sneak peek of how they’re using it. They’ve taken the regular HoloLens and they’ve added an infrared camera on the top, for giving soldiers a view in the dark. I think that’s really interesting, and I think it’s intriguing to see how other people are using that. And then, if you look at Trimble is another company — I believe those guys are members as well of AREA — they made a custom helmet out of the HoloLens 2, called the “X1.” I think that’s really interesting.

Mark: Yeah, absolutely. And again, focused on the enterprise space, HoloLens 2 is great, but it needs to be safe and being used in those kind of more difficult environments; I don’t know… dirty, damp, and potentially dark, and all those things. I think having [worked] with companies like Trimble, and being able to support those requirements, is really important. And I think you’re right; Microsoft are listening, and they’ve got lots of requirements coming in. But I’m kind of excited where they’re going, and looking forward to them — and the other wearable device providers — driving forward on that. You know, there are other companies out there. Vuzix and Magic Leap. And then also the assistive technology guys — the RealWear and stuff like that. But I guess I’m…

Alan: So for the people listening, let’s break it down a little bit, from a hardware standpoint, and then we’ll dig into software. Because I think this is where people get caught up. They say, “oh well, VR is just for gaming,” or “AR is Pokémon Go.” But let’s break down the different types of glasses. On the mixed reality side, you have HoloLens and Magic Leap, and those can give you full spatial computing. But a lot of times, you don’t need that; maybe you’re in a warehouse, or maybe… maybe break it down as to what the different glasses are doing.

Mark: Yeah, absolutely, and I’ll start with the other end. I remember listening to the guys at Upskill presenting, and they kind of coined the term “assisted reality,” and I was like, “wow, actually; what does that mean?” But it’s very simply just taking basic content and delivering it to the individual. My basic content, I mean it can be as simple as a PDF or a video; something which isn’t augmented, but really helps that worker in their environment, and gives them contextual and relevant information when they need it. RealWear is a good example; the Google Glass, that was, and will probably come back again. But these kind of solutions, quite often I see them is almost AR 0.1 — your first movement or foray into the AR space, and simpler (potentially) to deploy. They bring a huge amount of benefits, and just allows — certainly, bigger enterprises who’ve been looking for solutions — rather than having to go to look at a desk, or look at a PC and come away from their work station, or come away from the work that they’re doing, to be able to give them that information there and then. And especially, keeping their hands free, which is the crucial part.

Alan: It’s interesting, I had a chance to try the RealWear helmet; it’s like a little arm that attaches to your hard hat or your own glasses, and the arm comes up, and it’s like having a 70 inch TV available in one eye when you need it. It just folds up when you need it; you can pull up instructions, it’s all voice-driven. I had the opportunity to also try Kopin’s new Whisper technology, where it can be jackhammers all around you, and you can talk to the headset, and it will understand you and only you, and bring up that information. So what they were showing with the RealWear was something very simple, but very very essential. It’s that just bringing up PDFs of work plans, or instructional information; that, in a hands-free environment, can save hours of time not having to go back and look something up on a computer. You just pull it up as you need it.

Mark: Yeah. I couldn’t agree more, Alan, and I think it works well in the kind of cases where you have infrequent and complex tasks, as well. It’s a real sweet spot there for enterprise, is to say if you’re working on that type of work — it doesn’t happen that often, or it’s more complicated than, I don’t know, just putting a tire on, or something simple — these kind of solutions can really help, and I’m a big fan of getting companies started on AR journey by using something like this. The price point could be slightly cheaper as well. And it gets that worker working hands-free and cuts out a lot of wasted time of going to search things, and looking up computers, logging in, or looking at diagrams which come with that slight risk that they’re out of date and stuff like that as well. So yes, I think it’s really exciting and interesting space. It’s part of a continuum, if you like, of going into the full augmented area as well, where there’s still a number of different use cases that enterprises can benefit from. The important message, I guess, is be very clear on what problem you’re trying to solve, and make sure you’ve got the correct solution to do that. I’ve had a few phone calls where people have called me and say, “hey Mark, what do I do with these HoloLenses, or Vuzix Blades,” or whatever the device is. You know what, guys: go back to really understanding your business, and understanding what problems you’re trying to solve. That’s your first crucial step in all this.

Alan: Yeah, I think companies — like everybody — they get excited. They’re like, “hey, we just bought 25 HoloLenses — now what?” And one of the things that, I think, we need to just get back to basics as well, is that everybody’s got a phone in their hand. And a lot of power can be delivered directly from just a mobile device. It doesn’t need to be a wearable. Wearables are obviously preferable when you need hands-free, but AR in context can be used just from your mobile device. And I think a lot a lot of people have mobile devices — almost everybody. That’s the easy way in for companies, to say, “hey let’s start using some of this technology.”

One of the things I want to bring up is, you mentioned Upskill. Upskill is a company that’s raised an enormous amount of money — I think in the tens of millions — and one of the demonstrations that they’ve done was with Boeing, and they showed a complex wiring harness. A worker with a printed manual beside them — like, right beside them — working on this wiring harness, versus a heads-up display of their Upskill platform showing the same information as the printed book, but in a heads-up display. And they showed, I think it was something like 27 percent faster task completion, using the heads-up display versus a paper.

Mark: Absolutely. It’s actually 36 percent, and the reason why I know is because I use it a lot when I’m explaining the benefits of AR to, particularly, to enterprises. It’s real simple, and often the use case is that you’re trying to maybe fix something for the first time, or set something up, and those wiring boxes are quite complex. And just being able to… you see one guy, he’s kind of moving from side to side. He’s doing a piece of work and going to check the diagram, going back up again.

Alan: Yeah I think one of the companies that’s really pushing the boundaries of this technology; Boeing. Boeing is a great example, because they’ve been working in this space for quite some time now, and they’ve got a lot of examples. Maybe you can talk to some of the specifics of how Boeing is using this technology?

Mark: Yes, certainly, and I’m lucky enough to have the president of The AREA, is a gentleman called Paul Davis, from Boeing. One of the many people we have working with The AREA. Also, they chair the Safety Committee as well. So we have a few other gentlemen there: Greg Ehret and Brian Laughlin. So they have multiple different use cases. In fact, one of the pieces of research that we’ve done, they’re actually using to evaluate new AR projects. And it’s great to see a company that have gone out to their workforce and say that, “here’s what AR is, what you can do with it. Please propose projects and ways that we could use AR to improve your work and become more effective.” And they’ve really taken that on board. The wiring diagram and the harness is one big one they’ve been working on a while for. They’ve actually created a kind of a platform. I think it’s called BARK [Boeing Augmented Reality Kit], which allows them to be quite flexible in setting up new AR projects. They’ve done a whole bunch around training and things like that, and I’m sure to be honest Alan, there is a number of projects going on that they’re not keeping secret, but they’re working on internally and stuff like that.

So I think anything they can do to improve their performance, they’re focusing a lot on trying to keep some of the skills, or understand some of the skills that their key workers, or longtime workers, have had, and use that from a training and guidance bit, and then solving some of those real complex problems. You know, the wiring within an aircraft is very complex, so rather than having to mock that physically up, they’re using it to be able to show all the wiring, take different elements out, work out how to put new wires in, or reroute the wires, and stuff like that. So yes. Paul talks a lot about, they were one of the — well, the company — that come up with the term “augmented reality” back in 1989. They are one of the leading companies in this space, and it’s always great to engage with them to see what they’re doing.

Alan: That’s incredible. So they’re really the pioneers of this. There’s other main companies you know we’ve… NASA has been using this technology as well. Boeing. I think Ford has been using VR/AR for a long time; we had Elizabeth Baron on the show earlier, talking about how they’ve been using it for design, but also for sales and marketing as well. So I think it’s very interesting, and it’s funny, because we had somebody on the show recently, talking about the aviation industry, and how they’re using it. And it wasn’t from the wiring harness, and enterprise side; it was more from the marketing side. How they’re taking airplanes that won’t be built for another three years, creating virtual models of them, and using them as sales tools to show people what’s coming. And I think that’s really cool.

Mark: Yeah, I think on that, Alan, and it’s a question I get asked a lot; which kind of industry is leading, certainly the AR space — and to a certain extent the VR? And I always say, “actually, in my experience, it’s not really by industry. It’s about the use case, or the problems that’re being solved.” I mentioned some of them, like remote assistance, or step-by-step guidance; they’re relevant across a whole bunch of different industries. So if you have that kind of problem, or there’s something you need to solve, then AR is a solution regardless of what industry you’re in.

Alan: I agree, and I think one of the things that sticks with me is that, it’s not… people see AR, and they’re just, “oh, we’ve gonna make AR, or we’re gonna make VR, or something!” It’s not about that. We’re past that kind of kitschy point of this technology, where it’s like, “hey it’s cool; I can make a Pokémon jump out of my desk!” I think the real world applications are starting to really become clear, and something that we discussed offline was how companies, up until now, have been doing a lot of POCs (or Proof of Concepts), and trials, and prototypes. But we’re kind of past that now because the technology is there. We know what it does. We know how to make it. It’s showing real benefits.

One of the things that comes to mind is, and I can’t remember the company offhand, but they started using augmented reality for heads-up, kind of… not remote assistance, but training, and they had a 25 percent increase in retention rates, but a near zero error rate. How do you… as a company, how do you not do that? How do you go to your CEO and go, “by the way, we did this trial, and it increased our productivity by 25 percent and decreased our error rates to almost zero. Can we have a budget for it?”

Mark: You are absolutely right, in that the industry is still… I’d say “littered” is probably the wrong word, but there’s still many trials and pilots and proof of concepts going on, and it’s something within The AREA we’ve been focusing quite a bit on. To help those organizations move to that next stage. So, there’s a few themes we see through that. The first one — and we’ve actually delivered this through The AREA research capability — is a kind of an ROI calculator. So is there a way that you can show, justify — especially from a neutral organization like The AREA — what the ROI is on your project? It’s, again, for members only, but it’s a really great way of… we’ve spent a lot of time speaking to different companies about the sort of benefits, mainly tangible ones — but again, we do want to talk about some of the intangible ones. They can use this calculator put information together, include it within the costs, in some other currency, and other factors, to pop out an ROI. So that’s really important. One of the things I always say to companies thinking about doing an ROI — sorry, an AR project — is think about and measure how you do things to start with, before you start on your prototype. And then measure it when you’re doing your prototype.

Quite often, what people do is crash straight into the prototype and haven’t always captured that previous information. So I think there’s a bit about being ready within your project to show the financial benefits, is one thing. The other issue, or the theme we see, about going from prototype into full deployment is some of the business issues that are often not sorted out. I mentioned before that Boeing chairs The AREA safety committee, working with the safety managers and being able to deploy these solutions into the organization. It just need some kind of pre-thinking; it needs to understand, and you need to work with the safety managers so when they’re deployed on a wider scale, they’re aware of it. They’ve been involved in the business decision-making process, and they’re happy to deploy it. You don’t come along and say, “hey look we’ve got these new devices, wearables, and tablets; we need to start working on them.” Safety is one factor. Security is another one — obviously, we have some members that have real security issues and want to keep things safe. So again, involving in those security people in there.

So I guess in summary, Alan, one of the things that we advocate a lot, from moving from pilot to full deployment, is treat it as a change management process, as well as a technology one. Locate the part of the business that need to be involved, and work with them at those early stages. Hopefully, that should allow to move from this pilot into a full project, overcoming some those business issues at the earliest stages.

Alan: I think you touched on something that’s you know I want to explore a little more. Because we’ve talked about how the use cases of remote assistance and safety/security/training — all of these things we’ve talked about — proof of concepts, and how we’re kind of moving into the real ROI that’s measured by real, defined key performance indicators (or KPIs). One of the things that people always ask me as well is, “what are the benefits of AR?” And it’s one of those things where you’re like, “oh. Well… okay? When you break it down that… so in your opinion, what are some of the main benefits of this technology?

Mark: I will break this down into two high-level segments. The first one is about improving the performance. So that could be about having the most relevant, up-to-date, contextual information, when you need it. Okay? So we’re going back to that bit where people are wandering off, trying to find information; if you can have it when you need it, at the right context at the right time, it’s perfect. The second one in the performance areas is managing your resources. We mentioned one of the key use cases — and probably the biggest use case at the moment — is remote systems. It’s a good example of being able to manage your resources much more effectively, not having them out on the road, or travelling everywhere in the world and stuff like that. So that’s another key performance improvement.

And the third one is real-time compliance. So, being able to capture, record, certify processes — if you’ve got the kind of policies to do that. I know, certainly in the aircraft industry, everything that is done needs to be registered and stuff like that. So all of those are about by improving performances, which actually kind of means that you’re increasing your efficiency. You’re improving the efficiency in frequent and complex tasks. You’re minimizing errors (and preventing, to a certain extent, human errors is miscalculation), and lowering the impacts of task interruption and errors, and stuff like that. So, from an increase in efficiency, it’s about reducing time, minimizing error, and lowering costs.

Alan: Absolutely, and I think you nailed it there. You said improving performance by providing relevant, contextualized information. I think this is going to become more and more prevalent when headsets like the HoloLens and Magic Leap become real enterprise tools. And that’s happening now. But I mean, it’s just going to get better and better and better. But one of the things that people do understand is that even a mobile phone now can look at a machine, and it can understand in context where your phone is in relation to that screen. And if it’s a pair of glasses like HoloLens or Magic Leap, you can look at this machine, overlay an entire digital version of that machine on top, and then step-by-step, walk somebody through this.

And why that’s important is, you know, it’s becoming more important, because the aging workforce in manufacturing and oil and gas and electrical… the aging workforce is starting to retire. And the young people coming into these fields, they don’t have time to catch up on a decade’s worth of experience. Maybe you’ve got somebody who’s got 30 years experience on an oil rig, and maybe they’re ready to retire, but you could say, “hey listen, rather than retire and just throw away the 30 years of experience you have, why don’t you just stay home, work a couple hours a week, and be the remote assistance for those young guys out on the rigs, or traveling around the world, or whatever, and you can be their eyes and ears?” So when they put these headsets on, they can look at the machine, have somebody back in home base see what they’re seeing, and annotate instructions on top of it. So that remote assistance being, able– imagine having Skype with the world’s expert of anything you want, at anytime you want. That’s incredible.

Mark: It is incredible, Alan, I think. And just taking on your theme there, actually — and I’ve been thinking about this a lot — we use the term “training,” and it has a particular emphasis to it. You wonder if a worker in the future will be guided to do their work, so it doesn’t have to be a connection with a remote assistance, but someone virtually or augmented, shown on what to do….

Alan: AI is coming for us!

Mark: It enriches the job role to a certain extent, in that the workers in the future, by using AR technology, can do a whole raft of different tasks, and use not only the remote expert, but use the guidance of the content and the augmented content to do whole sorts of different things. So I can see where somebody — I take a simple example — they can be fixing in a washing machine one day, but the next day, they can be fixing the TV, and the day after, a computer, and a day after, a car. Because they’re using the content that the organization able to provide in a contextual and relevant way. I think it’s kind of an exciting [time]; we still need people to do it. But it allows them to do wider kinds of stuff.

It just plays on one other point, Alan, that I’d like to add, because this goes back a little bit to setting your projects up for success. Quite often, at the moment some of the projects — especially when you’re talking about step-by-step guidance – are a little bit standalone from some of the enterprise systems. So one of the things we talk about is actually connecting into the core enterprise data. Whatever systems it’s captured in, and using that. And if you should see a better way of doing it, you can update it, and then everybody benefits from it. So there’s there’s certainly a theme, I think, in the future where we need to plug in and connect to the content, or the assets, of the enterprise, and use AR not only just to read it, but updates as well.

Alan: Yeah, it’s interesting. We’re working with a company right now who’s taking BIM models and CAD models, and overlaying them real time… basically, the blueprints of a building, overlaying them real-time using AR in context, and then they have a two-way feedback mechanism. So, when they point their device at a building that they’re working on, it’ll show the HVAC system or the electrical system — whatever they happen to be working on — and it shows it exactly where it should be. You can walk around and see it from every angle. But when there’s an error, they can annotate it and it automatically will update the head office and say, “there’s an error here.” They can add a little note and it’ll stay positionally fixed, and it will stay on the blueprints, on the live blueprints. I think this is something that we’re only scratching the surface with this, and it’s going to be a huge thing. The industry that they’re going after is the construction industry, and it’s a 30-billion dollar error problem. Rebuilds and rework in construction is a massive problem! 30-billion dollars lost every year, rebuilding things because somebody put the HVAC system in six inches to the left instead of to the right.

Mark: I didn’t really realize it was such a big number. But I can… yeah I can see it’s huge.

Alan: That’s not including residential; that’s only in commercial buildings. So it’s a big [loss].

Mark: Yeah. Well I think the use of AR to visualize that BIM data will be huge. I always look at the people digging up the roads (which, unfortunately, they’ve been doing a lot of recently from where I live), and it seems to be a trial and error. Or they build it so big, that everything has to close down anyway. So there’s a whole bunch of efficiencies and greater performance we’ll see in the years to come that we probably don’t even think about now. Or even thought about.

Alan: Yeah, there was a company called Esri – E-S-R-I — and they’re using GIS data to be able to overlay… you can actually download the app and play with it. You can sit there, and in your street, look at all the pipes that under your street.

Mark: It’s amazing.

Alan: It’s so cool. I mean, the data is there, whether it’s right or not. I mean, the city plans are as accurate as you can get, and you kind of have to go with it. But being able to overlay that data for people just going to replace a pipe in the street., exactly what you said; don’t dig out the whole damn street. Just put these glasses on, figure out where to dig, and dig there!

Mark: I understand, and I’m a big fan of some of the marketing side of that. But lots of the writers talk about enterprise AR as being certainly the leading kind of space in the immersive technology, and it’s because the ROI, or the benefit, is a lot more tangible at this stage. You can measure the kind of savings, whereas the marketing part, it’s part of a number of different factors which influence the purchasing decision. All of it together is awesome.

Alan: One of the things that we’ve noticed on the marketing side — and that’s kind of where we play — is that the ROI is hard to measure on typical marketing. Like this weekend, for example, Coachella launched their AR app at Coachella, and you could see spaceships flying through the concert venue. And then HBO launched a Game of Thrones AR experience. And they can measure the number of downloads, but there’s really no way to measure, “hey, 16,000 people watched the show because of that.” It was just more of a marketing thing.

One of the things that marketing people are starting to realize is that 3D product views on a website, and being able to try on a pair of glasses or shoes or whatever — virtual try-ons — does directly contribute to sales. So that is an industry use case that I would say is more on the enterprise side. It’s marketing, but it’s direct benefit and ROI.

Mark: Yeah, that’s cool.

Alan: Speaking of benefits and ROI, I wanted to talk to you about some of the direct benefits of member companies joining The AREA. Because you guys are first in class with regards to thought leadership, use cases; you have your ROI calculator, tons of research articles. So maybe speak to each one of those points, and explain to them – to potential members — why they should join AREA. Because I think it is a very valuable organization.

Mark: Yeah. Thank you, Alan. That’s very kind. So I think, simply put, we were focusing on four things.

The first one is about helping to develop and create, curate, and deliver thought leadership content. So, the idea behind this is that business decision makers at the moment just do not have enough information to make informed decisions on investing in AR. There’s obviously lots of competing technologies out there, and they’ve also got to run their business and stuff like that. So anything we can do to focus on the use cases – so, what problems can be solved, providing them with case studies, examples of those use cases being solved in real life, what technologies are needed, and what the return on investment is — to me, is really important. So we’re very keen to listen to case studies, and deliver that content, and focus on those business decision makers. So a lot around thought leadership.

The second benefit of an AREA membership is around networking. It always amazes me that, when companies get together, how similar some of their problems are. And I don’t just mean the technology problems, but their business problems of trying to deploy AR, or even the problems they’re trying to solve. We ran the annual AREA workshop, which for the first time, we did it in the UK a few weeks ago, and someone came up to me afterwards and said, “you know, Mark; that was like group therapy.” It’s an opportunity to meet with like-minded people, to understand some of their challenges, what they’ve been doing to overcome them, and enable them to think about and learn from other companies, as well. So it’s really important to be able to network with like-minded people, and also to build those partnerships. If you’re an enterprise looking to deliver AR solutions, or are providers looking to work together with the startups that we have, networking is really important.

The third element is around what we call educate, but we’re already beginning to get a bit concerned about the availability of skilled workers in this space. Now it’s a huge and growing industry, and companies — even The AREA members now — are struggling to get workers that come out of university with the correct kind of skills. I don’t only mean technical skills; I mean some of the business skills as well. We’re working with educational organizations and universities, to help them define courses, to work with them on, I think, from guest lecturing to outplacements, to setting challenges, and anything we can do to help educate and connect universities with organizations that are developing or delivering AR solutions. That’s been a really interesting journey as well; a lot more universities are beginning to use The AREA, work with The AREA to do that.

And then finally, and probably the biggest strategic pillar we have at the moment, is overcoming the barriers to adoption. I mentioned before, things like safety and security; we have a monthly committee meeting that talks about those kind of areas. I’m really delving into the detail and looking at delivering deliverables that really benefit the members. We also have a requirements committee; we’ve been developing a set of requirements, and actually extended that to capture the use cases in the different scenarios and the different types of workers. The aim being that any enterprise can come along and say, “hey, I’m in the automotive industry — I’m interested in remote assistance.” By the press of a couple of buttons, they can get a set of requirements downloaded (of course, which they can add to and supplement and things like that), but it allows them to fast forward their AR projects and potentially go for RFPs [request for proposal] RFIs [request for information] and stuff like that. And we’re connecting the providers of AR technology to the companies looking to deliver or understand a set of requirements. So that’s requirements committee. We’ve set a committee up on human factors. So, looking at some of the UX and UI issues, and design issues, because that again, is a slightly “Wild West” area, and we want to help bring companies together and define some business best practice. We have a marketing committee, which is about promoting the ecosystem as well, and see that a lot of our social media stuff is driven through that marketing committee. And then finally, the Research Committee. We’ve touched on a few things; the ROI calculator was delivered through the Research Committee. And it’s real simple, we say to all of The AREA members, “what do you need to research? What kind of applied research would help you in your business?” Every AREA member gets an opportunity to make a proposal. We have a little pitching session, then every member gets an opportunity to vote, and whatever research gets the most votes, The AREA then funds and delivers that research. We’re actually just kicking off our fifth research project. The first one was around security and wearables, which we’ve actually made available to everybody now; they can go to the website and download that research. We felt, after a year, that probably it was something that we should make available, because we’re moving on our thinking and work in that space. And the second was the ROI calculator. We’ve done a piece on human factors and safety; a kind of reusable framework. We’re just finishing off a deep dive into the manufacturing industry, and some of the barriers to adoption, and a framework to help companies overcome them. And we’re just kicking off a piece now; it’s basically looking at IoT [internet of things] AI and AR, and how those technologies work together — helping again, business decision leaders — to understand and bring all that stuff together, so they can make informed decisions.

So there’s a huge amount of stuff going. I’m really excited to be able to lead this, and work with all The AREA members. It is very much driven by The AREA members. And you can go to The AREA dot org, or drop me an email at mark at The AREA dot org, to find out more, as well. Thank you, Alan, for giving me the opportunity to talk a little bit about The AREA’s work.

Alan: No. It’s vital. The work you guys are doing is absolutely essential to the success of our industry, and I want to just say, thank you for joining me on this show and sharing your information.

So, Mark, is there anything else? What do you see as the future of XR, as it pertains to business, in your opinion?

Mark: Thank you, Alan. And before I say that, I just want to thank you as well, for the great work you’re doing evangelizing our industry. I follow all the stuff that you’re talking about, too. Thank you, as well.

As for the future, I think we’ve there’s improvements on all fronts, really. I’m sure the technology will improve; we need to deliver technology that can be used in the environments that we’re talking about, oil and gas being one. It’s very difficult to have some of the technology used when you’re out on a rig somewhere in the North Sea. So I can see improvements on all fronts. I think a better understanding of deployment, better business understanding, and at the moment, I always think of the enterprise AR ecosystem as a little bit like an iceberg. We have a few companies and a few enterprises – say, at the peak — and you can see them at the top. But there’s a huge amount underneath the waterline. Not that I want the water to drain away in our environment, but to get more people understanding what benefits they can get, and being able to really master it and become more efficient. I think it’s a steady movement.

6D.ai CEO Matt Miesnieks has been in the AR game since the beginning, and he says the best in the industry have always known the best, native use cases for the technology; the problem was for the technology to catch up to the use cases. Listen as Matt and Alan discuss how the tech and the vision are starting to line up today, with the help of groundbreaking cloud mapping technology.

Alan: Today’s guest is Matt Miesnieks, the CEO at 6D.ai. Matt is renowned as one of the world’s AR industry leaders, and through his influential blog posts and persona around the world. He’s also the co-founder and CEO of 6D.ai, the leading AR cloud platform, which is his third AR startup. He also helped form Super Ventures, which is a platform and V.C. firm investing in AR solutions. He’s built AR system prototypes for Samsung, and had long executive and technical careers in mobile software infrastructure before jumping into AR back in 2009. In his career, he’s been the director of product development of Samsung for VR and AR research and development, co-founder and CEO of Dekko, that created the first mobile holographic and mixed reality platform for iOS, 3D computer vision, and slam tracking. And Layar, he was the worldwide head of customer development; Layar sold to Blippar back in the day. And I want to invite Matt to the show. Thank you so much for joining me on the show.

Matt: Thanks for having me.

Alan: Thanks Matt. This is really an honor to meet you, and you have so much experience in this industry that we can all learn from. You’ve been doing this, it seems like, since the beginning. So why don’t we start with what you’ve seen as the progression of augmented reality over the last decade that you’ve been involved?

Matt: Using words like “decade” brings it home. I mean, I got into AR from working for the company called Openwave that invented the mobile phone browser, and seeing that phones were being connected to the Internet, and started to think about what was next. And realized that interfaces were getting more natural, and we were going to end up connecting our senses to the Internet. You can connect a sense of sight — our dominant sense — that was going to be a really big deal. And I learned that was called augmented reality; that ability to sort of blend digital information and the real world into what you see. I kind of jumped in expecting it to be happening pretty soon. And at the time, there was nobody. At the first AWE conference back then, I think there was 300 people in total. And that was the entire professional AR industry. That included a bunch of researchers, a bunch of, like, science fiction authors, just a bunch of weirdos and a handful of people with some sort of commercial expertise. I think the interesting thing is that, even back then, the use cases and the kind of interactions and those sorts ideas around, these are the things that AR is going to be good for in the earliest days is still the same ones. It wasn’t like anything’s changed; they’ve stayed the same. What’s gotten better is the user experience around those use cases. The technology is improved. There’s like 100x or 50x more processing capacity in our hands. The algorithms have gotten better. And the same use cases that we knew were good ideas back then are now like, oh, these are starting to work now. Enterprises and consumers are starting to get some value out of it.

Alan: So let me interject quickly, for the people listening who are not familiar with this industry: what are these use cases? What are the use cases that have stood the test of time? I know one of them that keeps coming up on almost every podcast that we do is remote assistance. The ability to have other people see what you’re seeing and collaborate with you. So, what are they…?

Matt: Well, probably even to go up a level from that, when I was in mobile and helping to sort of shepherd in that transition from web to mobile Internet, all the first mobile experiences would just take a website, and sort of squish it down to a small screen, and you had, like, eBay on mobile. It was pretty lame. It took a little while to figure out what are the native capabilities of a phone, and what are the use cases that leverage those native capabilities? And it turned [out] to be things like, your phone is with you, and you’ve got a GPS. So things like Uber or Google Maps and directions were native to the phone, that were kind of useless on the PC. We saw things like, your phone was with you, so you could do real-time but short messages always work. So things like Twitter really took off. Not to mention, the camera was there, and Instagram and Snapchat and camera-driven experiences; they’re all native to mobile. And when you think about AR, the best use cases are native to AR.

So we’re finding that, when you think about what AR really is, it really is that ability to have digital content in real time placed in context in the world. What are the scenarios that enable use cases where that’s a negative experience? And so, remote fieldwork is exactly one of those; you want to have someone — basically, an expert — standing at your shoulder, pointing at something or touching things, going, “now press that button; and now, draw a line here; and now, turn this knob,” and AR lets you do that. The expert can be remote, but they can graphically annotate the real world scene and give you that type of engagement. That means that companies can drastically lower their costs of only having all the experts in a centralized location, and they can put lower cost or less-trained employees out in the field, and they can still do as good a job as if the senior guy was there. So that’s one. Another early one that was really obvious is pre-visualization of purchases. Particularly expensive or complicated purchases.

So, you know, the the stereotypical Ikea, “preview my couch in the living room.” That concept is native to AR. And although that people only buy a couch every few years is not a great, repeatable, highly-engaging use case. The idea of just being able to say, “I’m thinking of buying this physical thing but I just want to know what it’s going to look like and how it’s going to work in my world” is very compelling. When you get the user experience right. Companies like Sephora, where they letting you try on makeup. And now, with the latest neural networks and graphics programming, you don’t look like a Ronald McDonald anymore. You look very natural and it looks great, and it looks like what the product would look like as it was applied by a professional makeup artist.

Alan: It’s interesting, I was on a panel with Miriam from Modiface, which was acquired by L’Oreal for their makeup try on. Now they’re venturing out: they’re doing hair try-ons, and jewelry as well. And like you said, it had to look realistic, and it took them a long time to figure that out. When you smile, your lips need to stay red. If you move, the lip gloss can’t be on your cheeks.

Matt: Definitely, most of the problems to be solved — and still being solved — are the technical problems right now; we’ve known these use cases for 10 years. Clothing is another one; I want to try on this outfit, you know, virtually. And that’s one where they use cases are a no-brainer, but the technology isn’t quite good enough yet to get that.

Alan: We’re not quite there.

Matt: The fall of the clothes, and the natural movement of the cloth, and getting the sizing and everything right.

Alan: It’s coming soon, though. I have seen some people working on it; it’s getting there.

Matt: They’ve been working on it for 10 years. Definitely, the advances with neural networks and AI on the graphics side are just making things possible that were never possible before.

Alan: It’s interesting; I wrote a whole article called “The First Killer App for Augmented Reality,” and it was all about virtual try-ons. Whether it’s makeup, shoes, watches, glasses, clothing. Clothing the only one on the list that really wasn’t being used all that well.

Matt: But it also works like enterprise workflows. You know, like construction and engineering companies that want to previsualize what it’s going to look like when we finish building this room and install all the equipment in the pipes and the air conditioning, and is there going to be enough room for some other bit of equipment to be installed? It’s exactly the same use case as the IKEA couch example, but it’s a different industry, and the ROI is much, much faster.

Alan: We just invested in a company — and they’re going to be in stealth for a bit — but they’re solving the problem of taking BIM models or CAD models into AR, overlaying them in the context of the real world, but also looking for errors. Because with products like 6D.ai, you’re now being able to — and we’ll unpack this in a second — but you’re able to create a point cloud map of the real world, overlay the data, and make annotations on that in real time. So for example, in a construction site, if you have an HVAC system that’s off by six inches, sometimes they don’t notice that for a month, and then by the time they realize it, it’s too late and they gotta rip the whole thing down and start over again. And rework in the world of construction is about a 60 billion dollar problem.

Matt: Yeah that’s a big one. In Dekko, our computer vision lead, he literally did his PhD in built-to-plan, for using AR to support built-to-plan for the construction industry. It’s a fantastic use case, and it’s mostly been just limited by the quality of the technology. Even the latest SDKs, like ARKit and ARCore, kind of struggle when you get to a space that’s bigger than, like, a small apartment or a big room. If you want to do a construction site, you need better technology than that just to enable the use case.

Alan: Okay, so you guys have started developing a new foundational framework for capturing point cloud maps — and for those people who don’t know what that is or maybe don’t understand that — maybe you can kind of unpack what are you guys doing at 6D.ai right now, and why is it important to businesses.

Matt: Yeah. Well, we’re a bunch of computer vision experts that spun out of one of the top AR computer vision labs in the world at Oxford University, the Active Vision Lab. That’s kind of what we do, but it’s not why we’re doing it. The reason the company was started was because, like I said, all these use cases were so obvious, especially after being in the space for a long time, that I saw all this amazing commercial potential that was only being unrealized because the technology wasn’t good enough. So we chose to focus on just solving some of these hardest technology problems, and making the solution available for the developers. One of the biggest problems is, how do you make content feel like it’s really part of the world? And the only way you can do that is if that virtual content can interact physically with the world. It means if something goes around the corner, it should disappear and it goes around the corner. Things should bounce off solid objects. The content should really understand that world. And it meant being able to capture a model — a virtual model of the world that perfectly mirrors reality — and then the virtual content can interact with the virtual model, and it matches the real world.

So that was a problem that, in the past, could only be solved with expensive depth cameras or Google Street View-style cars, or take a thousand photos and wait a day for it all to be processed. And so my co-founder at his lab in Oxford really just invented a way to do that in real time on a regular phone, all in-software. No special hardware needed; just wave your phone around, and it builds that 3D model. And that was kind of the germ; the start of the company. And since then, we’ve built it up to be able to support very large spaces. Like, city-scale sort of areas. And we’re adding more and more real-time neural networks, so that we can start to identify and track things that move in the scene. If a person or a dog walks in front of the content, it’s occluded and bumps into things properly.

Alan: If people are listening and they want to try this out, I got to try this program that was built on your backend called Babble Rabbit by Patched Reality. Basically, you scan the area that you’re in, and this rabbit jumps around. And exactly what you said, it can hide behind chairs and couches, and jump off your counter onto the floor. It’s really incredible to watch AR when it’s delivered in perfect synergy with the world around you. It really does make a difference. It’s mind-blowing, because if you look at some of the AR out there — Pokémon Go, for example — the Pokémon look like they’re in front of you, but they’re not moving around anything; they’re not really in context to the world around them, so they do have this kind of look of fake about them. But what you’re talking about is global-scale capture of cloud maps, so that AR interacts seamlessly around you. That’s incredible.

Matt: Yeah, I mean technically, it’s a really big deal. And unfortunately — I mean, I don’t know if “unfortunate” is the word — but it’s one of those problems that, the better we do our job, the less people notice what we’ve done. Because people go, “of course, that’s the way it’s supposed to work. Of course, my rabbit should hide behind the couch.” We’re constantly working just to be invisible, and to let the developer’s content — the experience they create — is what gets the wow effect. But until you get things working properly, and getting the shadows and the lighting pointing in the consistent direction of the real world, and getting the physics and the structure and all that stuff that you don’t want… you want to eliminate everything that’s going to break that sense of illusion. And if you can maintain that sense of illusion and make it really compelling, people just get incredibly absorbed because it is… it becomes magic.

Alan: One of the things that one of our developers was working on was taking the geolocation — so, the weather API; just pulling the weather API from the cloud — and knowing, OK, how cloudy is it? Based on the weather API, how dark should the shadow be? And depending on which direction your phone is pointing, which direction should the shadow point, based on your geolocation. How cool is that?

Matt: We’re barely scratching the surface of when you start mashing up different APIs. The public transport timetables, and can you have a giant bat signal on top of your bus as it heads down the streets towards you, and you can look for it. Who knows? I struggled with…

Alan: All I really want is, at a big festival, to be able to find my friends. Is that too hard?

Matt: Well hopefully, we’ll have that up-and-running before the end of the year.

Alan: The guys at Coachella, Sam Schoonover, he’s gonna be on the show as well. I don’t know if you saw that Coachella did a massive thing this year, where one of the tents was completely AR-enabled. So it used proximity, like geolocation, but also image recognition to be able to place these digital objects in your world on one of the stages. And then they also created an AR scavenger hunt, where if you went around and found different things, you could collect coins to buy t-shirts and stuff like that. Coachella is really pushing the limits with this technology, as well. It’s pretty cool to watch a consumer brand–

Matt: I was down there last weekend talking to Sam about exactly this.

Alan: Oh, awesome!

Matt: And showing him how it could be much, much better than anything he could imagine. He got quite excited.

Alan: I bet; he’s so enthusiastic about the stuff.

Matt: Yeah, we’re excited about the potential there. I was down there because we had… I mean, we’re all computer vision engineers; our customers are all big companies solving the hardest challenges in AR. But one of the things we do is we work with a lot of really high-profile artists to push the limits of what the tech can do. At Coachella, we work with Aphex Twin. They took some of our neural network computer vision technology, and they used it in their live show. They’d point one of their cameras at the crowd and run that camera feed through our software, and then project it back up onto the big screen live. You’d have all these psychedelic effects on individuals in the crowd, and it was just really, really amazing.

Alan: All I can say is, that’s sick!

Matt: It was that thing that went so far beyond what we could imagine. It was so much better. It’s like, wow, you sort of realize that what you’ve built, you can do more with it than even you’d imagined.

Alan: It’s incredible. My last company… I don’t know if you’ve ever seen the big see-through touchscreen DJ controller emulator, but we made basically a touchscreen midi controller before there were touch screens in 2010. And it was see-through, so the audience could see what the artists were doing as well. And we had the opportunity to work with Infected Mushroom, and Morgan Paige, and Linkin Park. And seeing what these guys did… the guys at Linkin Park, Mike Shinoda did something really amazing. He took our midi controller and made a keyboard out of it, but it looked nothing like a keyboard at all. It was just this kind of series of buttons everywhere. And he played it as if it was a customized keyboard for him. It’s like, we had never even considered that.

Matt: Yeah, Weirdcore, who does all of Aphex’s visuals, he did the same thing. He drives all the displays off of a sample controller. So one of those little square pads, with all the different colored light-up buttons. You normally use it to fire off samples, but he drives all the video through that and plays it like an instrument to get all the screen effects, and dropping different visual patches on top. Yeah. It’s impressive.

Alan: Yeah it’s pretty cool. You know, when you go to festivals like Coachella or EDC, you realize pretty quickly that those guys are pushing the absolute limits of this technology. I mean, you go to some show, and it’s got a thousand different laser beams coming out at you, and the video is all synchronized to lasers, is synchronized to the lights, are synchronized to the music; and you’re just like, how is this even possible? Then you’ve got 3D projection mapping into the crowd. I think the electronic music scene has really taken to technology more than any other.

Matt: Yeah. Yeah, I know. And we’re just excited to learn from it. I mean, it’s definitely not our customers or a target market in a commercial sense, but from product learnings and just expanding the realm of what we thought was possible? It’s just been fantastic to work with these guys.

Alan: So with that, what is one of the best use cases or case studies of virtual, augmented, or mixed reality that you’ve seen to date? That kind of made you go, wow?

Matt: Besides our own? It’s…

Alan: Your own as well, or whatever, yeah.

Matt: I think Snap are doing probably the most interesting work in AR right now. They did some stuff with landmarks recently, where they made Big Ben vomit rainbows and stuff. I think most people really underestimate how good Snap is that AR. They’re obviously the number one AR company in the world in terms of usage.

Alan: By FAR. People don’t realize it, but by far.

Matt: Yeah, by far. Their research team, and the sort of quality of the organization that built to this stuff is as good as anybodys.

Alan: So, would Snap be, then, a potential customer? Or use your 6D.ai platform for creating even more immersive–?

Matt: Actually, I mean… right now, their concern is that if they were our customer, that they would just turn the tap on and we’d drown instantly. We’re friends with a whole bunch of people; again, being around for 10 years, most of the guys that run the Snap AR team are folks I’ve known in their previous company and my previous companies. So potentially, who knows? It’s a bit early to say. We’d love, obviously, to have a customer that that big. But right now, we’re still 15 people, and I don’t think we could honestly support them as a direct customer relationship. But there’s lots of ways to potentially partner.

Alan: Amazing. So you mentioned Snap. What are some other, you know…?

Matt: The other one — this sort of thing is the stuff that isn’t obvious to people — but the other big one that I think is a big deal is the Microsoft Dynamics 365 apps. When they launched the HoloLens 2, they also announced this suite of templates called the dynamic 365. They’re kind of like an app skeleton; Microsoft’s customers could build… and I can’t remember which types of use cases they took, but they took things like, for example, a field service type use case. They basically got a skeleton application for that. One of the big challenges in AR right now is building these apps is still complex. The tools are pretty [new], and Unity and things, you need to invest a fair bit of time and effort to build something. The fact that Microsoft kind of recognized that and put this work into saying, “look, here’s kind of a semi-turnkey [template], ready to go… you just need to configure and customize and you now have an enterprise AR app, ready to go. That was impressive. And I don’t think it’s being recognized enough.

Alan: Yeah, I think their whole idea with the HoloLens 2 was, you know… the learnings with HoloLens 1, they took the feedback from the users and actually did something, which is kind of unique in technology; they actually listen to people, imagine that! But one of the things that I think was a problem for everybody, is that they bought these devices and then put them on and went, okay, now what? No out-of-the-box use cases. There was no easy way to make anything. You had some great companies — Finger Food Studios in Vancouver, and Look — you had some great companies making great content, but they had to start from scratch every single time.

Matt: Yeah.

Alan: If you’re mining company, and your AR development company says, “hey, we’re gonna make this thing for you, and it’s going to be a quarter of a million dollars,” that’s great the first time. But when they come back to you and say, “we’re going to make the next thing for you, it’s going to be a quarter of a million dollars,” that’s like, something’s not right. So Microsoft said, “hey, let’s make it out-of-the-box useful. I think that was the best thing that they did with HoloLens 2. Aside from the improvements of the actual hardware itself.

Matt: Totally. Yeah, totally. There’s so many similarities between the way the smartphone ecosystem emerged and the way the AR ecosystem is emerging, and all of those peripherals. And you said it, they’re peripheral, but really they’re core to the end-to-end user experience. It’s gonna be it’s probably gonna take a bit longer than anyone realizes before everything’s in place, but it’s happening. I’m just sitting here, watching history repeat in many ways.

Alan: Okay, so you’ve seen a lot. You’ve been you’ve been doing this; you are…I want to say “OG.” But you’ve invested in companies through Super Ventures; I guess, what are some of the companies that you’ve either invested in, or that you see, that are really bringing value now? In enterprise, or in retail, or marketing, or sales? What are the companies that are driving value, now that businesses can look up research and go, “hey, that will work for me?” Because really, when it comes down to it, it comes down to those specific use cases. And until we have ubiquitous systems that work across the enterprise, it’s going to be these one-off solutions for now, I think.

Matt: I think that one of the mistakes that every… not everyone made this mistake, but a lot of people fell into this trap of believing that gaming is where new technology gets adopted first, and gaming and entertainment are the right places to focus for success in VR and AR. I disagreed with that from the beginning. I thought gaming is where GPUs took off, and 3D graphics took off. But really, PCs took off in the enterprise, and mobile phones took off for the enterprise, and Smartphones took off from the enterprise. And to me, AR just seems like it’s more like that. So, the enterprise has always been the right place. And then looking at, well, what’s stopping enterprises? And largely it was technology problems; both hardware and software. When I look at, definitely, starting my company 6D, and where to where to put my energy, it was “solve the hardest technical problems and enable use cases that are going to work for enterprises.”

When I look at the market, the companies that are taking that same sort of strategy — everything from Microsoft down to startups — are the ones that are doing relatively well. If you’re going to go with something that’s consumer-oriented, you’ve got to either hope for a massive hit, consumer hit (generally based on some high profile IP like Pokémon), or you’ve got to have some huge distribution available to you like Snap or Facebook or Instagram have. So yeah, I kind of advise startups, “whatever you’re building, you need to be able to sell it for $100,000. And if you can’t tell me the name and phone number of a person today who would pay $100,000 for this, then you’re probably gonna struggle for a few years. You’re going to struggle for longer than you have runway in your bank.”

Alan: That’s some sage advice.

Matt: Well, only because I’ve made all those mistakes myself (laughs). Don’t be stupid like I was.

Alan: That’s the best advice ever, from somebody who’s literally done it. “Don’t do what I did.”

Matt: It was worse; I was right. Like my Dekko, and what we built at Samsung were exactly the same technology that ARKit and ARCore and Magic Leap and others built. Like, verbatim, the same technology. I just love that having the correct vision, and building the correct technology, and getting everything right, you can still fail. Just because other aspects of the market that you can’t control aren’t ready. So build something that you can sell today, for enough money today to keep you going.

Alan: You know, it’s interesting. We’ve taken this exact theory, and we’re… you know, I can’t announce anything on the show right now, but I’ll tell you afterwards, but one of the things that we’re working on is being able to help startups do exactly this. And our theory is exactly what you said. Once they sell to — ours is a different level — but once they sell over $250,000 worth of whatever it is they’re selling, that’s when we step in and invest in them, and will match that. So I agree with you wholeheartedly. When I got into VR, I didn’t… I knew nothing about this industry five years ago. I tried VR for the first time. Blew my mind. Went, “that’s it. I’m in it.” Since then, I took a really broad approach; we did everything from 360 videos, AR apps, VR apps, VR training, Photogrammetry, 3D modelling — you name it; we did everything. And the idea was to study the industry, figure out what works, what customers want, what they’re willing to pay for, what they’re not willing to pay for. And through doing all of that, we’ve taken this really amazing lens of the infrastructure, and what works and what doesn’t work. And I think you have that from a decade of experience, and I think your advice is very spot-on, especially for startups listening to this. (to audience) Startups! Pay attention!

Matt: (to audience) Don’t do what I did. Yes, yes. Exactly.

Alan: Let me ask you — in the interest of time, I want to get the most value for people. How would a company start to evaluate or get started in using these technologies? What would your advice be to somebody that’s listening to this podcast, and they’re saying, “wow, I have a HoloLens,” or, “our company bought one; it’s sitting on the shelf, nobody’s using it.” What would your advice be to companies to get started?

Matt: Educate yourself. That’s the big one. Like, there’s nothing… there’s no turnkey, off-the-shelf, kind of like… if you call us, we’ll say we probably couldn’t help straight away, unless you have that education, and you know what problem we need to solve, and you know what you want to do. There’s no real agencies out there that are “hey, talk to them and they’ll definitely figure it out for you.”

Alan: Well that’s us. I’m going to put a plug in for Metavrse.

Matt: Kay, so that’s you guys.

Alan: I’ve never put a plug in, but that’s literally what we do for a company.

Matt: OK. But yeah, we find that people have the same bad ideas over and over again, or they have the same misconceptions about what AR is good for, or how things actually work. By far the best thing you can do if you want to get in early is educate yourself, on what some of these capabilities are, what the constraints are in the technology, where they use cases make sense. It may make sense for your company right now, or maybe a couple of years. I’ve actually written a bunch of blogs where I’ve tried to capture as much as I know and have learned about use cases and design problems and you know how the tech works and all those sorts of things.

Alan: Where can people find that?

Matt: Just on Medium. If you google my name from the podcast — I’m the only [Matt Miesnieks] on earth — and find either the 6D.ai/blog or google my name on my Medium posts.

Alan: I’ll put a link to your Medium in the show notes.

Matt: Thanks. And then, do experiments. Start building on ARKit if you can. Poke around and download apps, and you’ll see all the problems that are there. We’ve always found like, all our customers right now — or, the only ones we can cope with — have usually tried to do something themselves. Like at Coachella, you talk to Sam and he says, “look, this is the best that could be done today, but we’ve got all these problems that we wish we could get around.” And then we can say, “hey, we’ve got solutions for all of those,” and it’s a very easy conversation. But if people don’t really understand those problems, it’s hard, you know, this is going to take a while to get you to this point.

Alan: Yeah I found that people like Sam, he’s actually rolled up his sleeves and built stuff. I think until you’ve run into these problems… we built a Web AR application, oh man, must be two and a half years ago now. And Web AR back then… Web AR now doesn’t work very well; Back then, it sucked! The product that we put out for the client worked perfectly. It did exactly what they wanted. But my development team threatened to kill me if I ever sold it again.

Matt: Yeah.

Alan: But having gone through that problem, the first question out of people’s mouths when they ask about AR is, “can we do this web-based?” and I’m like, “yes you can, but it’s going to be 10 times the time and 10 times the price.” It’s just not there yet. And having realistic expectations where a professional can outline that 1, 3, and 5-year roadmap, I think, is key to that.

Matt: Yes. Yeah. That is the key. Think sort of medium-term for this stuff, and view the first couple of iterations as, really, learning experiments.

Alan: Yeah. And it doesn’t have to be expensive. My last guest on the show was Paul Boris, and he was saying that what they’re doing is, they’re doing enterprise training and equipment and maintenance, using AR. So you know when they’re doing it, this is an enterprise, and for them to build a module it’s about 100 to 200 thousand dollars. But if you’re just experimenting with AR, I mean, you can go get ARKit and ARCore and start messing around it. You can mess around with Amazon Sumerian. You can go on Snapchat and build some filters and try that on their lens studio. You can go on Facebook and start messing around with their, um, I think it’s called Spark AR. So there are a number of different platforms; you can start experimenting, and for zero cost, really.

I always encourage people to do that, especially if they’re in the research and education mode. Try it and fiddle around. One of the things that came up was what are some of the job… or, not job, but I guess roles that would be required to do this? Things like Unity developer or 3D modeller and stuff like that. So I think by slowly introducing it, you’ll realize what’s necessary.

Matt: Yeah, I know; there’s a lot to it. You don’t want to be building big, in-house teams to do this stuff, when you can… you get sucked into trying to boil the ocean. Everything from small in-house projects becomes something with a continuously rolling scope through to… companies like Magic Leap or others — Microsoft — that are basically trying to build entire ecosystems by themselves. And it’s one of the big temptations for AR; the more you learn about it, the more attractive it is, and exciting it is, and the more you want to go after it.

Alan: It’s the shiny penny problem. “AR can solve every problem; let’s do it all!” (both laugh) Amazing. So speaking of that — speaking of problems — and this will be our last question because I know you’re very busy, but what problem in the world would you want to see solved using XR technologies?

Matt: It’s hard to say. People say, “what’s the killer app for smartphones,” when we’re doing the Internet on phones. And really that there was no killer app; the “killer app” is it just lets us connect to everything the Internet offers in a more natural, engaging, convenient way. Phones have given us superpowers in a way; our brains are connected to the sum of human knowledge. We can communicate with anyone, anywhere, anytime. AR is going to do more of that. It’s gonna give us more superpowers. It’s going to expand our sense of sight, expand our ability to know things in real time. It’s that potential that excites me. You know, I feel a responsibility that as this type of power is developed, how is it going to be used responsibly? How can we at least try and imagine some of the things that we try and prevent? But yeah, if I have to think what is that attraction to me, to the space, it really is just that. Those new types of superpowers that were going to enable for people.